From a29043595b2c37ddb1acfb95fa6fe89a9e7a4c1b Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Mon, 13 Jul 2026 17:17:18 -0600 Subject: [PATCH 1/8] docs: replicating the system database with a constrained topology (5.2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Controlled-flow replication can now include the `system` database while keeping a constrained topology (directional self-record). Update the replication overview (Controlling Replication Flow) — the previous "avoid replicating system" note no longer applies for directional routes — and add a 5.2 release note. Documents per-database directional routes, the add_node database-scoping behavior change, and the central-visibility caveat. Companion to the harper-pro core change (branch kris/systemdb-routing-repro). Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/replication/overview.md | 31 ++++++++++++++++++++++++++++++- release-notes/v5-lincoln/5.2.md | 20 ++++++++++++++++++++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/reference/replication/overview.md b/reference/replication/overview.md index a9526c82..57ea163f 100644 --- a/reference/replication/overview.md +++ b/reference/replication/overview.md @@ -198,7 +198,36 @@ replication: In this example, the local node only receives from `node-two` (one-way inbound) and only sends to `node-three` (one-way outbound). -> **Note**: When using controlled flow replication, avoid replicating the `system` database. The `system` database contains node configurations, so replicating it would cause all nodes to have identical (and incorrect) route configurations. +You can also scope flow per database, so different databases flow in different directions between the same two nodes. Use `sendsTo` / `receivesFrom` entries with a `database`: + +```yaml +replication: + databases: + - cardata + - config + - system + routes: + - host: node-two + replicates: + sendsTo: + - database: config # push central config downstream + receivesFrom: + - database: cardata # aggregate telemetry upstream +``` + +#### Replicating the `system` database with controlled flow + + + +Before v5.2, replicating the `system` database under controlled flow was discouraged: because `hdb_nodes` (the node registry) lives in `system` and each node advertised itself as a full-mesh participant, replicating `system` caused every node to discover and directly connect to every other node — collapsing a constrained topology into a full mesh. + +As of v5.2 you can replicate `system` while keeping a constrained topology. When a node has directional routes, it advertises a **directional** registry record derived from those routes (which neighbors it sends to / receives from) instead of a blanket "connect to everyone." Discovered non-neighbor nodes therefore do not open direct connections. This lets central configuration — users, roles, and schemas — propagate transitively across the whole cluster while user-database connections stay on the routes you configured. For example, in a `roadside → middle → core` aggregation tree, a role created on a roadside node reaches the core through the middle tier, yet the core never opens a direct socket to a roadside node. + +Notes and current limitations: + +- This applies only when a node has **directional** routes (`replicates` with `sends`/`receives` or `sendsTo`/`receivesFrom`). A node with no directional routes keeps the legacy full-mesh advertisement. +- Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry (the registry relay differs from data relay). This does not open a connection either way. +- Route changes to a node's own directionality take effect on restart. ### Explicit Subscriptions diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index 992f6fd9..5933eefa 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -31,3 +31,23 @@ The `set_configuration` operation now accepts `"replicated": true` to apply a co ### Middleware routing and ordering Components can now declare `host` and `urlPath` in `config.yaml`, or pass them to `server.http()`, `server.ws()`, and `server.upgrade()`, to create middleware chains routed by virtual hostname, URL prefix, or both. The new `name`, `before`, and `after` options provide explicit middleware ordering. See [HTTP middleware routing](/reference/v5/http/overview#middleware-routing) and [`HttpOptions`](/reference/v5/http/api#httpoptions). + +## Replication + +### Replicating the System Database with a Constrained Topology + +Controlled-flow replication can now include the `system` database while keeping a constrained (non-mesh) topology. + +Previously, replicating `system` was discouraged when using [controlled replication flow](/reference/v5/replication/overview#controlling-replication-flow): because the node registry (`hdb_nodes`) lives in `system` and each node advertised itself as a full-mesh participant, replicating `system` caused every node to discover and directly connect to every other node — defeating the point of a constrained topology. + +As of 5.2, a node with directional routes advertises a **directional** registry record derived from those routes (the neighbors it sends to / receives from) instead of a blanket "connect to everyone." Discovered non-neighbor nodes therefore never open direct connections. This lets central configuration — users, roles, and schemas — propagate transitively across the entire cluster while user-database connections stay on the routes you configured. + +For example, in a `roadside → middle → core` aggregation tree, replicating `system` now lets a role created on a roadside node reach the core (through the middle tier) without the core ever opening a direct socket to a roadside node. + +Behavior notes: + +- This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. +- `add_node` / `set_node` with database-scoped `sendsTo` / `receivesFrom` entries now restrict replication to those databases (consistent with the config-route behavior). To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). +- Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. + +See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details. From 3ac6feef7ab6bb852f49f22a852b541d2dad995b Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Mon, 27 Jul 2026 13:27:07 -0600 Subject: [PATCH 2/8] docs: address review findings on systemdb routing docs (PR #583) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Fix heading-hierarchy skip (h4 -> h3) and hostname/host key inconsistency in the controlled-flow example. - Extend the per-database routing example to actually route `system`, matching the section it introduces. - Soften "discovered non-neighbor nodes therefore do not open direct connections" to the actual guarantee (no subscription/replication connection), and call out that on-demand residency/retrieval connections are a separate, unaffected mechanism. - Document add_node/set_node's sendsTo/receivesFrom fields in the Clustering reference (previously undocumented) and note the config-authority limitation: dynamic Operations API scoping doesn't make a node advertise a directional self-record — that's derived only from harper-config.yaml routes. Co-Authored-By: Claude Sonnet 5 --- reference/replication/clustering.md | 3 +++ reference/replication/overview.md | 8 +++++--- release-notes/v5-lincoln/5.2.md | 7 ++++--- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/reference/replication/clustering.md b/reference/replication/clustering.md index 05bc4c1a..566eee1d 100644 --- a/reference/replication/clustering.md +++ b/reference/replication/clustering.md @@ -32,6 +32,9 @@ Adds a new Harper instance to the cluster. If `subscriptions` are provided, it c - `table` — table name - `subscribe` — if `true`, transactions on the remote table are replicated locally - `publish` — if `true`, transactions on the local table are replicated to the remote node +- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node, mirroring the config-route `replicates.sendsTo` / `replicates.receivesFrom` syntax (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is a database name or an object: `{ database, excludeTables? }`. When provided, replication with this node is restricted to the listed databases; to replicate all databases except specific tables, use a wildcard entry (`excludeTables` with no `database`). Mutually exclusive with `subscriptions`. + + > **Note**: `sendsTo` / `receivesFrom` here scope only the connection to _this_ node — they don't change how the local node advertises itself to the rest of the cluster. A node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). **Request**: diff --git a/reference/replication/overview.md b/reference/replication/overview.md index 57ea163f..550cc264 100644 --- a/reference/replication/overview.md +++ b/reference/replication/overview.md @@ -207,25 +207,27 @@ replication: - config - system routes: - - host: node-two + - hostname: node-two replicates: sendsTo: - database: config # push central config downstream + - database: system # push central config (users, roles, schemas) downstream receivesFrom: - database: cardata # aggregate telemetry upstream ``` -#### Replicating the `system` database with controlled flow +### Replicating the `system` database with controlled flow Before v5.2, replicating the `system` database under controlled flow was discouraged: because `hdb_nodes` (the node registry) lives in `system` and each node advertised itself as a full-mesh participant, replicating `system` caused every node to discover and directly connect to every other node — collapsing a constrained topology into a full mesh. -As of v5.2 you can replicate `system` while keeping a constrained topology. When a node has directional routes, it advertises a **directional** registry record derived from those routes (which neighbors it sends to / receives from) instead of a blanket "connect to everyone." Discovered non-neighbor nodes therefore do not open direct connections. This lets central configuration — users, roles, and schemas — propagate transitively across the whole cluster while user-database connections stay on the routes you configured. For example, in a `roadside → middle → core` aggregation tree, a role created on a roadside node reaches the core through the middle tier, yet the core never opens a direct socket to a roadside node. +As of v5.2 you can replicate `system` while keeping a constrained topology. When a node has directional routes, it advertises a **directional** registry record derived from those routes (which neighbors it sends to / receives from) instead of a blanket "connect to everyone." A discovered non-neighbor node therefore is not subscribed to and does not receive a replication connection. This lets central configuration — users, roles, and schemas — propagate transitively across the whole cluster while user-database connections stay on the routes you configured. For example, in a `roadside → middle → core` aggregation tree, a role created on a roadside node reaches the core through the middle tier, yet the core never opens a direct replication subscription to a roadside node. Notes and current limitations: - This applies only when a node has **directional** routes (`replicates` with `sends`/`receives` or `sendsTo`/`receivesFrom`). A node with no directional routes keeps the legacy full-mesh advertisement. +- This constrains replication subscriptions only. On-demand residency/retrieval connections (for example, sharded or invalidated-cache reads) use a separate mechanism governed by data residency, not by this registry record, and can still open a direct socket to a non-neighbor node. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry (the registry relay differs from data relay). This does not open a connection either way. - Route changes to a node's own directionality take effect on restart. diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index 5933eefa..a9df5b8e 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -40,14 +40,15 @@ Controlled-flow replication can now include the `system` database while keeping Previously, replicating `system` was discouraged when using [controlled replication flow](/reference/v5/replication/overview#controlling-replication-flow): because the node registry (`hdb_nodes`) lives in `system` and each node advertised itself as a full-mesh participant, replicating `system` caused every node to discover and directly connect to every other node — defeating the point of a constrained topology. -As of 5.2, a node with directional routes advertises a **directional** registry record derived from those routes (the neighbors it sends to / receives from) instead of a blanket "connect to everyone." Discovered non-neighbor nodes therefore never open direct connections. This lets central configuration — users, roles, and schemas — propagate transitively across the entire cluster while user-database connections stay on the routes you configured. +As of 5.2, a node with directional routes advertises a **directional** registry record derived from those routes (the neighbors it sends to / receives from) instead of a blanket "connect to everyone." A discovered non-neighbor node therefore is not subscribed to and does not receive a replication connection. This lets central configuration — users, roles, and schemas — propagate transitively across the entire cluster while user-database connections stay on the routes you configured. -For example, in a `roadside → middle → core` aggregation tree, replicating `system` now lets a role created on a roadside node reach the core (through the middle tier) without the core ever opening a direct socket to a roadside node. +For example, in a `roadside → middle → core` aggregation tree, replicating `system` now lets a role created on a roadside node reach the core (through the middle tier) without the core ever opening a direct replication subscription to a roadside node. Behavior notes: - This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. -- `add_node` / `set_node` with database-scoped `sendsTo` / `receivesFrom` entries now restrict replication to those databases (consistent with the config-route behavior). To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). +- This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record. +- `add_node` / `set_node` with database-scoped `sendsTo` / `receivesFrom` entries now restrict replication to those databases (consistent with the config-route behavior; see [Add Node](/reference/v5/replication/clustering#add-node)). To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). This scopes the connection to that one peer only — it does not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details. From 73c4f11e9fd3af3a2f2eb69fe51f5da1a2eaf43e Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 12:46:42 -0600 Subject: [PATCH 3/8] docs: correct add_node sendsTo/receivesFrom semantics (PR #583) Ethan-Arrowood's review traced the actual behavior in harper-pro's knownNodes.ts/setNode.ts: a bare-string sendsTo/receivesFrom entry is matched as a peer name, not a database, and silently authorizes nothing; and add_node's sendsTo/receivesFrom describe the *added* node's perspective (opposite of a config route's local-node perspective) with no peer/target scoping, so the resulting hdb_nodes record is visible cluster-wide rather than confined to one connection. Corrects both the clustering.md reference and the matching 5.2 release note bullet to describe this accurately. Co-Authored-By: Claude Sonnet 5 --- reference/replication/clustering.md | 4 ++-- release-notes/v5-lincoln/5.2.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/reference/replication/clustering.md b/reference/replication/clustering.md index 566eee1d..8cd8db74 100644 --- a/reference/replication/clustering.md +++ b/reference/replication/clustering.md @@ -32,9 +32,9 @@ Adds a new Harper instance to the cluster. If `subscriptions` are provided, it c - `table` — table name - `subscribe` — if `true`, transactions on the remote table are replicated locally - `publish` — if `true`, transactions on the local table are replicated to the remote node -- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node, mirroring the config-route `replicates.sendsTo` / `replicates.receivesFrom` syntax (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is a database name or an object: `{ database, excludeTables? }`. When provided, replication with this node is restricted to the listed databases; to replicate all databases except specific tables, use a wildcard entry (`excludeTables` with no `database`). Mutually exclusive with `subscriptions`. +- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry must be an object: `{ database, excludeTables? }` — a bare string is matched as a peer/node name, not a database, so a string entry naming a database silently authorizes nothing. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). - > **Note**: `sendsTo` / `receivesFrom` here scope only the connection to _this_ node — they don't change how the local node advertises itself to the rest of the cluster. A node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). + > **Note**: These entries carry no explicit peer scoping, so they are not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, and any other node that also holds the listed database(s) can match against it too. They also don't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). **Request**: diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index a9df5b8e..2627477a 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -48,7 +48,7 @@ Behavior notes: - This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. - This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record. -- `add_node` / `set_node` with database-scoped `sendsTo` / `receivesFrom` entries now restrict replication to those databases (consistent with the config-route behavior; see [Add Node](/reference/v5/replication/clustering#add-node)). To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). This scopes the connection to that one peer only — it does not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. +- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). Each entry must be an object (`{ database, excludeTables? }`); a bare string is matched as a peer name, not a database, and silently authorizes nothing. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`, and carry no peer scoping — the resulting record is visible cluster-wide rather than confined to this one connection. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details. From b91f90d4f2c0a6f8a28c0caa4cabcf3423b3f424 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 13:12:57 -0600 Subject: [PATCH 4/8] docs: document target/source peer-scoping for add_node sendsTo/receivesFrom MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Independent review (codex/grok) caught that the previous wording said entries "carry no peer scoping" and must be `{ database, excludeTables? }` objects — both incomplete. harper-pro's RouteEntry type and matcher (knownNodes.ts routeEntriesIncludePeer) support target/source fields that do scope an entry to one peer; the docs just never mentioned them. Also clarifies that the per-database YAML example pushes system/config downstream, whereas the roadside-to-core narrative right after it describes the opposite (upstream) direction, and how to flip it. Co-Authored-By: Claude Sonnet 5 --- reference/replication/clustering.md | 4 ++-- reference/replication/overview.md | 2 ++ release-notes/v5-lincoln/5.2.md | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/reference/replication/clustering.md b/reference/replication/clustering.md index 7fa1ad1e..0a86da95 100644 --- a/reference/replication/clustering.md +++ b/reference/replication/clustering.md @@ -32,9 +32,9 @@ Adds a new Harper instance to the cluster. If `subscriptions` or `sendsTo`/`rece - `table` — table name - `subscribe` — if `true`, transactions on the remote table are replicated locally - `publish` — if `true`, transactions on the local table are replicated to the remote node -- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry must be an object: `{ database, excludeTables? }` — a bare string is matched as a peer/node name, not a database, so a string entry naming a database silently authorizes nothing. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). +- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry must be an object: `{ target?, source?, database?, excludeTables? }` — a bare string is matched as a peer/node name, not a database, so a string entry naming a database silently authorizes nothing. `target` (on a `sendsTo` entry) or `source` (on a `receivesFrom` entry) restricts that entry to one specific peer; omitted, the entry matches any peer on the listed `database`. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). - > **Note**: These entries carry no explicit peer scoping, so they are not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, and any other node that also holds the listed database(s) can match against it too. They also don't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). + > **Note**: An entry without `target`/`source` is not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, so any other node that also holds the listed database(s) can match against it too; set `target`/`source` to scope an entry to one peer. This also doesn't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). **Request**: diff --git a/reference/replication/overview.md b/reference/replication/overview.md index 567c4b4b..3b7bb746 100644 --- a/reference/replication/overview.md +++ b/reference/replication/overview.md @@ -218,6 +218,8 @@ replication: - database: cardata # aggregate telemetry upstream ``` +Direction is per hop and per database: to aggregate a database upstream instead of pushing it downstream — for example, so `system` changes made on a leaf node (like a new role) reach a central node — put that `database` under `receivesFrom` on the upstream-facing route instead of `sendsTo`. + ### Replicating the `system` database with controlled flow diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index 770cbb02..fa1aba32 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -54,7 +54,7 @@ Behavior notes: - This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. - This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record. -- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). Each entry must be an object (`{ database, excludeTables? }`); a bare string is matched as a peer name, not a database, and silently authorizes nothing. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`, and carry no peer scoping — the resulting record is visible cluster-wide rather than confined to this one connection. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. +- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). Each entry must be an object (`{ target?, source?, database?, excludeTables? }`); a bare string is matched as a peer name, not a database, and silently authorizes nothing. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`. Without `target`/`source`, an entry is not confined to this one connection — the resulting record is visible cluster-wide — so set `target`/`source` to scope it to one peer. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details. From 9c07b2b912c3e52e43e1570e9c46db0d0e2a8e31 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 13:23:38 -0600 Subject: [PATCH 5/8] docs: version-gate add_node sendsTo/receivesFrom, clarify upstream-aggregation example MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Independent review (codex/gemini/grok) flagged two more issues: - add_node's sendsTo/receivesFrom are new in v5.2 (harper-pro#572, commit 0b49587f) but the parameter list had no VersionBadge, and the "must be an object" wording incorrectly implied a bare string is never valid (it is — it just matches a peer name across all databases, not a database name). - The "aggregate upstream" guidance didn't say which node's config route needs the receivesFrom entry, so it could be misread as "configure receivesFrom on the leaf's route to the core" (which would do the opposite of what's intended). Spelled out the roadside/middle example explicitly. Co-Authored-By: Claude Sonnet 5 --- reference/replication/clustering.md | 2 +- reference/replication/overview.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/reference/replication/clustering.md b/reference/replication/clustering.md index 0a86da95..94ba1c1b 100644 --- a/reference/replication/clustering.md +++ b/reference/replication/clustering.md @@ -32,7 +32,7 @@ Adds a new Harper instance to the cluster. If `subscriptions` or `sendsTo`/`rece - `table` — table name - `subscribe` — if `true`, transactions on the remote table are replicated locally - `publish` — if `true`, transactions on the local table are replicated to the remote node -- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry must be an object: `{ target?, source?, database?, excludeTables? }` — a bare string is matched as a peer/node name, not a database, so a string entry naming a database silently authorizes nothing. `target` (on a `sendsTo` entry) or `source` (on a `receivesFrom` entry) restricts that entry to one specific peer; omitted, the entry matches any peer on the listed `database`. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). +- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is either an object `{ target?, source?, database?, excludeTables? }` or a bare string naming a peer directly (authorizing **all** databases for that peer, not a single database — a string does not match against `database`). `target` (on a `sendsTo` entry) or `source` (on a `receivesFrom` entry) restricts an object entry to one specific peer; omitted, it matches any peer on the listed `database`. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). > **Note**: An entry without `target`/`source` is not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, so any other node that also holds the listed database(s) can match against it too; set `target`/`source` to scope an entry to one peer. This also doesn't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). diff --git a/reference/replication/overview.md b/reference/replication/overview.md index 3b7bb746..b8d2d796 100644 --- a/reference/replication/overview.md +++ b/reference/replication/overview.md @@ -218,7 +218,7 @@ replication: - database: cardata # aggregate telemetry upstream ``` -Direction is per hop and per database: to aggregate a database upstream instead of pushing it downstream — for example, so `system` changes made on a leaf node (like a new role) reach a central node — put that `database` under `receivesFrom` on the upstream-facing route instead of `sendsTo`. +`sendsTo` / `receivesFrom` are declared from the perspective of the node whose `harper-config.yaml` they're in, for its route to that one peer. To aggregate a database upstream instead of pushing it downstream — for example, so a role created on a roadside node reaches a middle-tier node — the change goes on the **middle-tier node's** route to the roadside node, with `receivesFrom: [{ database: system }]` (middle receives `system` from roadside), not on the roadside node's route to middle. ### Replicating the `system` database with controlled flow From 8b8328e404b4c3edcacb08cfb971dac40de98d1f Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 13:32:03 -0600 Subject: [PATCH 6/8] docs: fix upstream-aggregation example and bare-string wording found by review - The roadside-to-middle upstream aggregation example only described the middle tier's receivesFrom entry. replicationConnection.ts's send-side authority gate (harper-pro#498) independently checks the SENDER's own directional config route, so roadside also needs a matching sendsTo entry or the subscription is rejected as unauthorized. Documented both sides. - The 5.2 release note still said a bare-string entry "authorizes nothing," contradicting the corrected clustering.md wording (a string names a peer and authorizes all databases for it). Aligned. - Added a caveat that routing `system` upstream from an edge node propagates hdb_user/hdb_role along with everything else, so a compromised or careless edge node's changes reach every node downstream of it. Co-Authored-By: Claude Sonnet 5 --- reference/replication/overview.md | 3 ++- release-notes/v5-lincoln/5.2.md | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/reference/replication/overview.md b/reference/replication/overview.md index b8d2d796..b4fe6a04 100644 --- a/reference/replication/overview.md +++ b/reference/replication/overview.md @@ -218,7 +218,7 @@ replication: - database: cardata # aggregate telemetry upstream ``` -`sendsTo` / `receivesFrom` are declared from the perspective of the node whose `harper-config.yaml` they're in, for its route to that one peer. To aggregate a database upstream instead of pushing it downstream — for example, so a role created on a roadside node reaches a middle-tier node — the change goes on the **middle-tier node's** route to the roadside node, with `receivesFrom: [{ database: system }]` (middle receives `system` from roadside), not on the roadside node's route to middle. +`sendsTo` / `receivesFrom` are declared from the perspective of the node whose `harper-config.yaml` they're in, for its route to that one peer, and — because a directional route also gates what it's willing to send — both sides normally need a matching entry. To aggregate a database upstream instead of pushing it downstream — for example, so a role created on a roadside node reaches a middle-tier node — the **roadside** node's route to middle needs `sendsTo: [{ database: system }]`, and the **middle-tier** node's route to roadside needs the matching `receivesFrom: [{ database: system }]`. If either side is missing its half, the middle-tier node's subscription request is rejected as unauthorized. ### Replicating the `system` database with controlled flow @@ -234,6 +234,7 @@ Notes and current limitations: - This constrains replication subscriptions only. On-demand residency/retrieval connections (for example, sharded or invalidated-cache reads) use a separate mechanism governed by data residency, not by this registry record, and can still open a direct socket to a non-neighbor node. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry (the registry relay differs from data relay). This does not open a connection either way. - Route changes to a node's own directionality take effect on restart. +- Replicating `system` upstream (edge → core) propagates `hdb_user`/`hdb_role` along with everything else in the database: a role or user created — or a compromised edge node's route table altered — anywhere on the upstream path reaches every node it flows to. Weigh this against your trust boundary for edge nodes before routing `system` upstream from them. ### Explicit Subscriptions diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index fa1aba32..f70f2a8f 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -54,7 +54,7 @@ Behavior notes: - This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. - This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record. -- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). Each entry must be an object (`{ target?, source?, database?, excludeTables? }`); a bare string is matched as a peer name, not a database, and silently authorizes nothing. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`. Without `target`/`source`, an entry is not confined to this one connection — the resulting record is visible cluster-wide — so set `target`/`source` to scope it to one peer. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. +- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). An entry is either an object (`{ target?, source?, database?, excludeTables? }`) or a bare string naming a peer directly, which authorizes **all** databases for that peer rather than matching a database name. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`. Without `target`/`source`, an entry is not confined to this one connection — the resulting record is visible cluster-wide — so set `target`/`source` to scope it to one peer. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details. From e642dd3ab9463048c58a4f1c9cf1a67cfefef2a1 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 13:44:27 -0600 Subject: [PATCH 7/8] docs: stop recommending add_node target/source peer-scoping (it's broken) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Traced setNode.ts's reciprocal add_node_back registration: it swaps sendsTo/receivesFrom wholesale onto the peer (targetAddNodeObj.sendsTo = req.receivesFrom, etc.) without rewriting each entry's target/source for the new direction. A target/source value that's correct for the calling side ends up wrong on the peer's copy, so the send-authority gate (replicationConnection.ts's shouldCloseSendAuthWatch) rejects the subscription — replication silently doesn't happen. The docs previously told users to set target/source to scope an add_node entry to one peer; that's not a reliable fix given this behavior. Removed the recommendation and pointed to config routes instead, which don't have this problem (computeSelfReplicates stamps target/source correctly on both sides). Also fixed an overstated "rejected as unauthorized" claim for the roadside/middle example: omitting the receiver's half means no subscription is attempted at all, not a rejection. This is a real harper-pro implementation gap, not just a docs issue — noting it in the dispatch Findings for a follow-up. Co-Authored-By: Claude Sonnet 5 --- reference/replication/clustering.md | 4 ++-- reference/replication/overview.md | 2 +- release-notes/v5-lincoln/5.2.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/reference/replication/clustering.md b/reference/replication/clustering.md index 94ba1c1b..ca5b895d 100644 --- a/reference/replication/clustering.md +++ b/reference/replication/clustering.md @@ -32,9 +32,9 @@ Adds a new Harper instance to the cluster. If `subscriptions` or `sendsTo`/`rece - `table` — table name - `subscribe` — if `true`, transactions on the remote table are replicated locally - `publish` — if `true`, transactions on the local table are replicated to the remote node -- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is either an object `{ target?, source?, database?, excludeTables? }` or a bare string naming a peer directly (authorizing **all** databases for that peer, not a single database — a string does not match against `database`). `target` (on a `sendsTo` entry) or `source` (on a `receivesFrom` entry) restricts an object entry to one specific peer; omitted, it matches any peer on the listed `database`. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). +- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is either an object `{ database?, excludeTables? }` or a bare string naming a peer directly (authorizing **all** databases for that peer, not a single database — a string does not match against `database`). The object form also accepts `target` / `source`, but don't rely on them to scope an entry to one peer: the reciprocal registration Harper sends to the added node carries the array over without rewriting these fields for the new direction, so a `target`/`source` value that's correct for your side of the connection is usually wrong for the other side, and replication can silently fail end to end. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). - > **Note**: An entry without `target`/`source` is not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, so any other node that also holds the listed database(s) can match against it too; set `target`/`source` to scope an entry to one peer. This also doesn't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). + > **Note**: These entries are not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, so any other node that also holds the listed database(s) can match against it too. For a restriction that reliably holds to one peer, use [config routes](./overview.md#controlling-replication-flow) instead. This also doesn't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). **Request**: diff --git a/reference/replication/overview.md b/reference/replication/overview.md index b4fe6a04..a1b4f656 100644 --- a/reference/replication/overview.md +++ b/reference/replication/overview.md @@ -218,7 +218,7 @@ replication: - database: cardata # aggregate telemetry upstream ``` -`sendsTo` / `receivesFrom` are declared from the perspective of the node whose `harper-config.yaml` they're in, for its route to that one peer, and — because a directional route also gates what it's willing to send — both sides normally need a matching entry. To aggregate a database upstream instead of pushing it downstream — for example, so a role created on a roadside node reaches a middle-tier node — the **roadside** node's route to middle needs `sendsTo: [{ database: system }]`, and the **middle-tier** node's route to roadside needs the matching `receivesFrom: [{ database: system }]`. If either side is missing its half, the middle-tier node's subscription request is rejected as unauthorized. +`sendsTo` / `receivesFrom` are declared from the perspective of the node whose `harper-config.yaml` they're in, for its route to that one peer, and — because a directional route also gates what it's willing to send — both sides normally need a matching entry. To aggregate a database upstream instead of pushing it downstream — for example, so a role created on a roadside node reaches a middle-tier node — the **roadside** node's route to middle needs `sendsTo: [{ database: system }]`, and the **middle-tier** node's route to roadside needs the matching `receivesFrom: [{ database: system }]`. If the middle tier is missing its `receivesFrom` half, it never attempts the subscription; if roadside is missing its `sendsTo` half, middle's subscription attempt is rejected as unauthorized. ### Replicating the `system` database with controlled flow diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index f70f2a8f..6b5aa9f9 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -54,7 +54,7 @@ Behavior notes: - This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. - This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record. -- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). An entry is either an object (`{ target?, source?, database?, excludeTables? }`) or a bare string naming a peer directly, which authorizes **all** databases for that peer rather than matching a database name. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`. Without `target`/`source`, an entry is not confined to this one connection — the resulting record is visible cluster-wide — so set `target`/`source` to scope it to one peer. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. +- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). An entry is either an object (`{ database?, excludeTables? }`) or a bare string naming a peer directly, which authorizes **all** databases for that peer rather than matching a database name. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`, and are not confined to this one connection — the resulting record is visible cluster-wide, and (unlike config routes) there's currently no reliable way to scope one to a single peer. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details. From 9c018470d4bf4436ed76ac855d91102609aaa2ca Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 13:55:25 -0600 Subject: [PATCH 8/8] docs: bare-string add_node entries have the same reciprocal-registration bug as target/source MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Traced the failure Grok/codex flagged: add_node's bare-string form (entry === peerName) has the identical swap-without-rewrite problem as target/source — a string that's correct for the caller's own send authorization becomes wrong once add_node_back copies it unmodified onto the peer's record, so the peer's send-authority gate rejects the subscription. Consolidated the guidance: only the unscoped object form { database?, excludeTables? } is reliable; steered both the string form and target/source into the same "don't rely on this" warning instead of presenting the string as a working (if blunt) alternative. Co-Authored-By: Claude Sonnet 5 --- reference/replication/clustering.md | 4 ++-- release-notes/v5-lincoln/5.2.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/reference/replication/clustering.md b/reference/replication/clustering.md index ca5b895d..234e9a90 100644 --- a/reference/replication/clustering.md +++ b/reference/replication/clustering.md @@ -32,9 +32,9 @@ Adds a new Harper instance to the cluster. If `subscriptions` or `sendsTo`/`rece - `table` — table name - `subscribe` — if `true`, transactions on the remote table are replicated locally - `publish` — if `true`, transactions on the local table are replicated to the remote node -- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is either an object `{ database?, excludeTables? }` or a bare string naming a peer directly (authorizing **all** databases for that peer, not a single database — a string does not match against `database`). The object form also accepts `target` / `source`, but don't rely on them to scope an entry to one peer: the reciprocal registration Harper sends to the added node carries the array over without rewriting these fields for the new direction, so a `target`/`source` value that's correct for your side of the connection is usually wrong for the other side, and replication can silently fail end to end. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). +- `sendsTo` / `receivesFrom` _(optional)_ — database-scoped controlled-flow entries for this node (see [Controlling Replication Flow](./overview.md#controlling-replication-flow)). Each entry is an object `{ database?, excludeTables? }`, or (less usefully) a bare string naming a peer, intended to authorize all databases for that peer rather than match a database name. **Only the unscoped object form — `{ database }`, with no `target`/`source` — is reliable today.** Harper's reciprocal `add_node_back` registration to the added peer carries the array over without rewriting each entry's peer reference for the new direction, so a bare string or a `target`/`source` value that's correct for your side of the connection ends up wrong on the peer's copy, and the peer's send-authority check silently rejects the subscription — replication doesn't happen. Use [config routes](./overview.md#controlling-replication-flow) for anything that needs to be scoped to one peer. Unlike a config route's `replicates.sendsTo` / `replicates.receivesFrom`, which describes the _local_ node's own direction, these describe the **added node's** perspective: `sendsTo` lists what the added node sends (what this node receives from it), and `receivesFrom` lists what the added node receives (what this node sends to it). If both `subscriptions` and `sendsTo`/`receivesFrom` are provided, `subscriptions` takes precedence and `sendsTo`/`receivesFrom` are ignored. To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`). - > **Note**: These entries are not restricted to the connection with this one node — the resulting `hdb_nodes` record for the added node is visible cluster-wide, so any other node that also holds the listed database(s) can match against it too. For a restriction that reliably holds to one peer, use [config routes](./overview.md#controlling-replication-flow) instead. This also doesn't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). + > **Note**: Because these entries aren't scoped to one peer, the resulting `hdb_nodes` record for the added node isn't restricted to the connection that created it — any other node that also holds the listed database(s) can match against it too. This also doesn't change how the local node advertises itself to the rest of the cluster: a node's own directional (non-mesh) `hdb_nodes` self-record is derived solely from its `harper-config.yaml` routes, so replicating `system` without collapsing to a full mesh requires directional routes in config, not just `add_node`/`set_node` scoping (see [Replicating the `system` database with controlled flow](./overview.md#replicating-the-system-database-with-controlled-flow)). **Request**: diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index 6b5aa9f9..af163346 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -54,7 +54,7 @@ Behavior notes: - This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. - This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record. -- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). An entry is either an object (`{ database?, excludeTables? }`) or a bare string naming a peer directly, which authorizes **all** databases for that peer rather than matching a database name. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`, and are not confined to this one connection — the resulting record is visible cluster-wide, and (unlike config routes) there's currently no reliable way to scope one to a single peer. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. +- `add_node` / `set_node` accept database-scoped `sendsTo` / `receivesFrom` entries to restrict replication with the added node to specific databases (see [Add Node](/reference/v5/replication/clustering#add-node)). Only the unscoped object form (`{ database?, excludeTables? }`) is reliable today: Harper's reciprocal `add_node_back` registration to the added peer doesn't rewrite bare peer-name strings or `target`/`source` fields for the new direction, so attempts to scope an entry to one peer can silently fail to authorize replication. These describe the **added node's** perspective, the opposite direction from a config route's `replicates.sendsTo` / `replicates.receivesFrom`, and (unlike config routes) aren't confined to the connection that created them. They also do not make the local node advertise a directional (non-mesh) `hdb_nodes` self-record, which is derived solely from the node's own config routes. Constraining `system` replication cluster-wide still requires directional routes in `harper-config.yaml`. - Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way. See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details.