docs: replicating the system database with a constrained topology (5.2) - #583
docs: replicating the system database with a constrained topology (5.2)#583kriszyp wants to merge 2 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the replication documentation and release notes for version 5.2.0, introducing the ability to replicate the system database under controlled flow with a constrained topology and scope replication flow per database. A review comment correctly identifies that the YAML configuration example uses the invalid property 'host' instead of 'hostname'.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-583 This preview will update automatically when you push new commits. |
Controlled-flow replication can now include the `system` database while keeping a constrained topology (directional self-record). Update the replication overview (Controlling Replication Flow) — the previous "avoid replicating system" note no longer applies for directional routes — and add a 5.2 release note. Documents per-database directional routes, the add_node database-scoping behavior change, and the central-visibility caveat. Companion to the harper-pro core change (branch kris/systemdb-routing-repro). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4ecc02d to
a290435
Compare
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-583 This preview will update automatically when you push new commits. |
- Fix heading-hierarchy skip (h4 -> h3) and hostname/host key inconsistency in the controlled-flow example. - Extend the per-database routing example to actually route `system`, matching the section it introduces. - Soften "discovered non-neighbor nodes therefore do not open direct connections" to the actual guarantee (no subscription/replication connection), and call out that on-demand residency/retrieval connections are a separate, unaffected mechanism. - Document add_node/set_node's sendsTo/receivesFrom fields in the Clustering reference (previously undocumented) and note the config-authority limitation: dynamic Operations API scoping doesn't make a node advertise a directional self-record — that's derived only from harper-config.yaml routes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-583 This preview will update automatically when you push new commits. |
Ethan-Arrowood
left a comment
Ethan-Arrowood
left a comment
There was a problem hiding this comment.
The core 5.2 constrained-topology content is accurate — I traced every major claim to merged harper-pro#572 and its source (computeSelfReplicates, shouldReplicateFromNode, setNode.ts) and it holds up. The host:→hostname: fix the Gemini bot asked for is in at this head too.
The problem is concentrated in the new add_node parameter bullet, which has a factual error serious enough to break a user's setup silently: a bare-string sendsTo entry is matched as a peer name, not a database name, so the documented "sendsTo": ["cardata"] authorizes nothing and replication just doesn't happen. Details inline, along with a direction-perspective problem and an overstated containment claim.
One of these may actually be an implementation-intent question rather than a docs bug — see the sendsTo direction comment. Since no test pins add_node's direction, it'd be worth confirming with you directly which behavior is intended; the doc needs to state a direction explicitly either way, and a pinning integration test would settle it permanently.
sent with Claude Opus 5
| - `table` — table name | ||
| - `subscribe` — if `true`, transactions on the remote table are replicated locally | ||
| - `publish` — if `true`, transactions on the local table are replicated to the remote node | ||
| - `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node, mirroring the config-route `replicates.sendsTo` / `replicates.receivesFrom` syntax (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is a database name or an object: `{ database, excludeTables? }`. When provided, replication with this node is restricted to the listed databases; to replicate all databases except specific tables, use a wildcard entry (`excludeTables` with no `database`). Mutually exclusive with `subscriptions`. |
There was a problem hiding this comment.
"Each entry is a database name or an object" is wrong, and it fails silently. In the implementation a bare-string entry is matched as a peer/node name, never a database — see routeEntriesIncludePeer (if (entry === peerName) return true) and the gate fallback in shouldReplicateFromNode (sendsTo === getThisNodeName()). So a user writing "sendsTo": ["cardata"] authorizes nothing at all and gets no error — replication simply doesn't happen.
Fix: "Each entry is an object { database, excludeTables? }" — or document strings as node names, matching computeSelfReplicates's { target: e } normalization.
Separately, the sendsTo direction is ambiguous — and the implementation's perspective is the opposite of config routes. In config routes, replicates.sendsTo means "the local node sends these to the route's peer" (computeSelfReplicates puts them in the self record). In add_node, req.sendsTo is written verbatim into the added peer's record (setNode.ts:201), where the receive gate reads a record's sendsTo as "this peer sends to me" — i.e. add_node's sendsTo means "databases the added node sends to the local node".
"Mirroring the config-route syntax" invites the local-perspective reading, which is inverted. Symmetric configs (same list in both keys) mask it; one-way flows silently go the wrong direction. Is the added-node perspective intended, or should setNode match the config-route perspective? No test pins it today.
Minor: "mutually exclusive with subscriptions" is guidance, not enforcement — the Joi schema accepts both and setNode silently prefers subscriptions (if (req.subscriptions) … else if (req.sendsTo || req.receivesFrom)). Either say "if both are provided, subscriptions takes precedence", or file a harper-pro validation follow-up.
| - `publish` — if `true`, transactions on the local table are replicated to the remote node | ||
| - `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node, mirroring the config-route `replicates.sendsTo` / `replicates.receivesFrom` syntax (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is a database name or an object: `{ database, excludeTables? }`. When provided, replication with this node is restricted to the listed databases; to replicate all databases except specific tables, use a wildcard entry (`excludeTables` with no `database`). Mutually exclusive with `subscriptions`. | ||
|
|
||
| > **Note**: `sendsTo` / `receivesFrom` here scope only the connection to _this_ node — they don't change how the local node advertises itself to the rest of the cluster. A node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). |
There was a problem hiding this comment.
"Scope only the connection to this node" overstates the containment, and it breaks in exactly the scenario this PR documents (replicated system). Entries written by add_node carry no target/source, and the peer record replicates cluster-wide; routeEntriesIncludePeer treats an absent target/source as "any peer", so a third node holding the same database can subscribe to the added peer under those entries. Note that computeSelfReplicates deliberately stamps target/source ("fully qualified… for the discovered-peer gates") while setNode does not.
The note's main point — that it doesn't make your node directional — is correct; it's the "restricted to this connection only" guarantee that doesn't hold once the record propagates. (Same phrase appears in the 5.2 release notes.) Arguably the real fix is upstream: should add_node-written entries stamp target/source like computeSelfReplicates does, making the claim true?
2 participants
Summary
Documents the 5.2 replication change that lets the
systemdatabase replicate while keeping a constrained (non-mesh) topology.reference/replication/overview.md(Controlling Replication Flow): the previous "avoid replicating thesystemdatabase" note no longer applies when a node uses directional routes. Adds a per-databasesendsTo/receivesFromexample and a<VersionBadge type="changed" version="v5.2.0" />subsection explaining constrainedsystemreplication, its opt-in nature, and the central-visibility caveat.release-notes/v5-lincoln/5.2.md: adds a "Replicating the System Database with a Constrained Topology" section (also documents theadd_nodedatabase-scoping behavior change).Companion PR
Documents the harper-pro core change on branch
kris/systemdb-routing-repro(feature PR to be opened; this docs PR will be cross-linked once it is).Where to look
The behavior-change framing in the reference doc and the
add_nodedatabase-scoping note in the release note — please sanity-check the wording against the intended product behavior.Drafted by an LLM (Claude Opus 4.8).