From ab07ad40c1c1f654b9513d4bfe60116771da55fe Mon Sep 17 00:00:00 2001 From: acud <12988138+acud@users.noreply.github.com> Date: Mon, 14 Sep 2026 18:16:20 -0600 Subject: [PATCH 01/13] =?UTF-8?q?Add=20SWIP-draft:=20BPS-lite=20=E2=80=94?= =?UTF-8?q?=20single-publisher=20brokered=20broadcast?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A standalone single-publisher profile of the design space SWIP-60 (PR #104) covers: one broker, one publisher, N subscribers, one hop. A cohort is a topic and a topic is a SOC address, so authorship needs no credential, no publisher list and no authenticated handshake — the ANCHOR address check re-establishes it from the signature on every message. The publisher role is bound to the opening stream and transfers on re-Open; the cohort's lifetime is that stream, which removes orphan cohorts and bounds Open as an allocation primitive. Specified standalone rather than as a profile of SWIP-60, with the protobuf inline and no compatibility claimed in either direction. It answers three questions SWIP-60 leaves open (dedup horizon, cohort lifetime, Open as an unbounded allocation primitive) and drops `closed` rather than restating a confidentiality claim an unauthenticated handshake cannot support. The two specs share the pubsub/1.0.0 protocol id, so the handshake carries the compatibility story: SWIP-60's field numbers are reserved, not reused, which is what lets a BPS-lite broker refuse a full-spec Open outright instead of silently decoding it with fields dropped. Note the one divergence that can fail silently for implementers: BPS-lite's ANCHOR applies its address constraint unconditionally, where SWIP-60 relaxes it under an explicit publisher regime. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01E99KGsCW7dyKudy2WbpRco --- SWIPs/swip-draft_bps-lite.md | 469 +++++++++++++++++++++++++++++++++++ 1 file changed, 469 insertions(+) create mode 100644 SWIPs/swip-draft_bps-lite.md diff --git a/SWIPs/swip-draft_bps-lite.md b/SWIPs/swip-draft_bps-lite.md new file mode 100644 index 00000000..775b7252 --- /dev/null +++ b/SWIPs/swip-draft_bps-lite.md @@ -0,0 +1,469 @@ +--- +SWIP: +title: BPS-lite — single-publisher brokered broadcast +author: acud (@acud) +discussions-to: https://discord.gg/Q6BvSkCv +status: Draft +type: Standards Track +category: Networking +created: 2026-09-14 +--- + + + +## Simple Summary + +One broker, one publisher, many subscribers, one hop. A publisher opens a topic on a +broker and pushes single-owner chunks; everyone else connects and reads. Nothing else. + +## Abstract + +BPS-lite is a real-time broadcast protocol in which a cohort is a topic and a topic is +a single-owner chunk address. The peer that opens a cohort is its publisher for as long +as that stream lives; every other peer is a read-only subscriber. Authorship needs no +credential, no publisher list and no handshake authentication: under the `ANCHOR` +binding the topic already names the only key that can produce a chunk hashing to it, so +authorship is re-established from the signature on every message. The cohort's lifetime +is its publisher's stream, which removes orphan cohorts and bounds `Open` as an +allocation primitive. + +It is a deliberate subset of the design space [SWIP-60][swip60] covers, specified as +its own document rather than as a profile: the wire format below is complete, and where +BPS-lite differs from SWIP-60 — most importantly in making `ANCHOR`'s address +constraint unconditional — BPS-lite's rule governs. No compatibility is claimed in +either direction. + +## Motivation + +SWIP-60 spans five cohort shapes, four topic bindings, four publisher regimes, an admin +control plane and a history flag. That parameter space is the right target for the +reference implementation and the wrong one for a second, independent implementation: +most of it is untested surface for anyone whose application is one author broadcasting +to an audience — a live feed, a price ticker, a game's server-authoritative state, a +log tail. + +BPS-lite names the single point in that space which needs none of the machinery, and +shows that pinning it collapses the spec rather than merely constraining it. Three +questions SWIP-60 leaves open — the dedup horizon, cohort lifetime, and `Open` as an +unbounded allocation primitive — are answered here, and one property SWIP-60 claims +(`closed` as confidentiality over an unauthenticated handshake) is dropped rather than +restated. The result is a target roughly a tenth the size, which a second team can +implement and conformance-test without tracking the full spec's revisions. + +## Specification + +The key words MUST, MUST NOT, REQUIRED, SHOULD, RECOMMENDED and MAY are to be +interpreted as described in RFC 2119. + +### Roles and topology + +One broker node. One publisher peer. N subscriber peers. Every peer dials the broker +directly over a single libp2p stream per (peer, topic), protocol id +`pubsub/1.0.0` — the same id SWIP-60 uses, shared deliberately: see *Backwards +compatibility*. No relaying, no referral, no broker discovery: depth is 1 by +construction. + +The publisher and the broker are distinct peers. Brokering for oneself is out of scope. + +**The peer that opened a cohort is its publisher for as long as that stream lives.** +The role is a property of the stream, not of a field in the spec: subscriber streams are +read-only and the broker MUST NOT read `Publish` frames from them. + +### Wire format + +This section is normative and complete: BPS-lite is defined by the messages below and +by nothing else. No import, no companion `.proto`, no field inherited by reference. +Reserved field numbers are SWIP-60's, kept reserved so that a frame from a full-spec +implementation is refused rather than silently misread as a valid BPS-lite frame. + +```proto +// BPS-lite — single-publisher brokered broadcast. Complete wire format. +// +// Enum zero values (*_UNSPECIFIED): proto3 requires a zero value; it is +// deliberately NOT a legitimate wire value. It exists so that an unset field is +// detectable and no implementation can silently rely on a default. Receivers +// MUST reject any message carrying one. + +syntax = "proto3"; + +package bps; + +// ---------------------------------------------------------------- cohort + +// What the topic binds to. BPS-lite has exactly one binding: the topic is the +// full SOC address. Unlike SWIP-60 the constraint is unconditional — there is +// no publisher regime that can relax it. +enum TopicBinding { + TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire + ANCHOR = 1; // topic = SOC address; dedup on the wrapped CAC +} + +// Fixed by the cohort's opener; immutable for the cohort's lifetime. +// A cohort is a topic: there is no publisher field, because under ANCHOR the +// topic already names the only key that can author for it. +message CohortSpec { + bytes topic = 1; // 32B: the SOC address, keccak256(id || owner) + TopicBinding binding = 2; // must be ANCHOR + reserved 3, 4, 5, 6, 7, 8; // SWIP-60: publishers, history, admin, + // publisher_list, po_min, closed +} + +// ------------------------------------------------------------- handshake + +// Opener -> broker: the one peer that fixes the cohort. The stream carrying +// this frame is the cohort's publisher for as long as it lives. +message Open { + CohortSpec cohort = 1; + reserved 2; // SWIP-60: auth. BPS-lite authenticates nothing at + // handshake time; authorship is checked per message. + bool loopback = 3; // deliver the cohort's own Broadcast frames back +} + +// Joiner -> broker: names the topic — nothing more. Subscriber streams are +// read-only; there is no role to claim here. +message Subscribe { + bytes topic = 1; // 32 bytes + reserved 2; // SWIP-60: auth. A frame carrying it is REJECTED. +} + +// Peer -> broker: the first frame on a fresh stream. Open and Subscribe are +// otherwise indistinguishable on the wire — both encode as a single +// length-delimited field 1, and proto3's permissive unmarshalling makes a +// wrong guess succeed silently. +message Hello { + oneof handshake { + Open open = 1; + Subscribe subscribe = 2; + } +} + +enum Status { + STATUS_UNSPECIFIED = 0; // invalid on the wire + OK = 1; + FULL = 2; // broker at a capacity limit: subscribers, or cohorts + UNKNOWN_TOPIC = 3; // Subscribe for a topic with no live publisher + REJECTED = 4; // unacceptable spec, or a reserved field set +} + +// Broker -> peer, answering Open or Subscribe. One per stream; a non-OK Ack +// ends the stream. +message Ack { + Status status = 1; + CohortSpec cohort = 2; // set iff status == OK +} + +// --------------------------------------------------------------- traffic + +// A full single-owner chunk in transit. Every frame is self-contained. +message Soc { + bytes id = 1; // 32 bytes + bytes owner = 2; // 20 bytes + bytes signature = 3; // 65 bytes + bytes span = 4; // 8 bytes LE + bytes payload = 5; // wrapped-CAC data, <= 4096 bytes +} + +// Publisher -> broker. Accepted only on the stream that opened the cohort. +message Publish { + Soc soc = 1; +} + +// Broker -> peer. +message Broadcast { + oneof frame { + Soc soc = 1; + // 2-15 reserved in SWIP-60 for the multihop control plane; BPS-lite is + // singlehop and defines none of them. + } +} +``` + +Everything after the handshake is direction-typed and needs no envelope: peer to +broker is always `Publish`, broker to peer is always `Broadcast`. + +A cohort spec carries two live fields. There is no publisher field, no publisher +list, no publisher regime, no `history`, no `closed`, no `PO_MIN`. Per-topic capacity +is broker policy and is not on the wire at all. + +`Open` carries no credential: nothing at handshake time can prove control of a key, +so BPS-lite does not ask. `Subscribe` carrying the reserved field 2 is `REJECTED` — +there is no second publisher to authenticate, and no role to claim at handshake time. + +### Handshake + +Publisher-first. The publisher fixes the cohort; subscribers join it. + +``` +publisher broker + | Hello{Open{spec, loopback}} | + |------------------------------------->| validate spec; create cohort; + | | this stream is the publisher + | Ack{OK, spec} | + |<-------------------------------------| + | Publish{soc} ... | + | Broadcast{soc} ... | iff loopback was requested + |<-------------------------------------| + +subscriber broker + | Hello{Subscribe{topic}} | + |------------------------------------->| UNKNOWN_TOPIC if no publisher has opened + | Ack{OK, spec} | + |<-------------------------------------| + | Broadcast{soc} ... | +``` + +Rules: + +- A non-`OK` `Ack` ends the stream. On `OK` the stream is retained. +- A `Subscribe` carrying reserved field 2 (SWIP-60's `auth`) is `REJECTED`. +- **`Open` on a live topic with an identical spec transfers the publisher role to the + new stream**: the broker answers `OK`, retains the cohort and its dedup window, and + resets the previous publisher stream. A publisher recovering from a dropped + connection therefore needs no distinct reattach frame, and a stale half-open stream + closing later is not the current publisher stream and has no effect. An `Open` with + a different spec for a live topic is `REJECTED`. +- A subscriber arriving before the publisher gets `UNKNOWN_TOPIC` and retries. The + broker holds no pre-registration state. +- A peer receiving `UNKNOWN_TOPIC` or `FULL` MUST wait before retrying: randomized + exponential backoff from 1s to 30s, full jitter. A broker MAY reset a stream from a + peer that retries faster. Without this, every publisher restart resets N subscriber + streams at once and they all return immediately. +- The publisher's own stream receives the cohort's `Broadcast` frames only if it asked + for them (`Open.loopback`, default false). See *Capacity and slow peers* for why it + is opt-in. + +### Message validation + +A `Publish` is accepted iff, in order: + +1. it arrived on the stream that opened the cohort; +2. the `Soc` decodes and its signature recovers an owner; +3. the SOC address, `keccak256(id || owner)`, equals `spec.topic` — the `ANCHOR` rule; +4. the wrapped CAC's BMT root matches its payload; +5. the CAC address is not in the cohort's dedup window. + +Failures 1–4 are protocol violations: the message is dropped, counted per peer, and +exposed through the implementation's invalid-message hook. Failure 1 additionally +resets the offending stream. No blocklisting is mandated. Failure 5 is ordinary — a +retransmitting publisher — and is counted separately. + +Note what is *not* here: no comparison of the recovered owner against a declared +publisher. Under `ANCHOR` the topic is a SOC address, so step 3 already constrains the +owner — producing a SOC that hashes to `spec.topic` under a different key is a keccak +preimage. An owner check would be a second test that can only fail where step 3 has +already failed. + +On acceptance the broker enqueues a `Broadcast` on every subscriber stream in the +cohort, and on the publisher's own stream if it requested loopback. + +### Cohort lifetime + +**A cohort exists exactly as long as its current publisher's stream.** When that +stream closes or resets, the broker destroys the cohort and resets every subscriber +stream. Subscribers reconnect and receive `UNKNOWN_TOPIC` until the publisher returns. +A stream that has been superseded by a role transfer is not the current publisher +stream: its later close is a no-op. + +This is the largest simplification the single-publisher restriction buys. It removes +orphan cohorts and any reclamation policy, and it bounds `Open` as an allocation +primitive: a peer can hold only as many live cohorts as it holds open streams. + +### Dedup + +A bounded LRU of accepted CAC addresses, per cohort. Implementations MUST bound it; +1024 entries is RECOMMENDED. A role transfer retains the window. + +Replay is prevented by the stream-role rule, not by the window: a captured `Broadcast` +re-`Publish`ed by an observer carries the genuine publisher's signature and passes +steps 2–4, but is refused at step 1 because it arrives on a subscriber stream — +whatever its age. The window bounds duplicate suppression for a retransmitting +publisher only, and 1024 is chosen against that benign case. + +### Capacity and slow peers + +Per-topic subscriber capacity is broker policy, not a cohort parameter. At the limit +the broker answers `FULL` and nothing else; referral is bps-multihop's business. +A broker MUST also bound the number of live cohorts it will create; the lifetime rule +above makes that bound enforceable, and an `Open` at that bound is answered `FULL`. + +Each retained subscriber stream has a bounded outbound queue (64 frames RECOMMENDED) +drained by a single writer. A full queue resets that stream. The broker has no +delivery obligation: withholding is a recoverable liveness fault, not a correctness +failure, and blocking fan-out on one slow subscriber would punish the cohort. + +The publisher's stream is exempt from that reset. It is the cohort's lifetime, so a +policy that is merely lossy for a subscriber is fatal for everyone: a publisher that +does not drain its own loopback echo — a fast publisher, a single-threaded client, a +bridge applying backpressure — would otherwise destroy its own cohort. A full queue on +the publisher's stream drops the frame and increments a counter. Loopback is opt-in for +the same reason. + +Loopback is an echo of *accepted* messages, not delivery confirmation: rejected +`Publish` frames are dropped silently and produce no frame at all, so a publisher +cannot distinguish rejection from delivery by watching its echo. An acknowledged write +path would be an `Ack`-per-`Publish` design, which BPS-lite does not have. + +### Status codes + +| condition | status | +|---|---| +| handshake accepted | `OK` | +| broker at a capacity limit — per-topic subscribers for `Subscribe`, live cohorts for `Open` | `FULL` | +| `Subscribe` for a topic with no live publisher | `UNKNOWN_TOPIC` | +| unacceptable spec, differing spec on re-`Open`, a reserved field set | `REJECTED` | + +A `STATUS_UNSPECIFIED` or any other zero enum value received on the wire is rejected: +the proto's zero values are deliberately not legitimate wire values. + +## Rationale + +### Authorship without a credential + +BPS-lite has no publisher field and no handshake credential. A cohort's topic is a SOC +address; the only key that can produce a SOC hashing to it is the one that owns it. +Authorship is a property of the topic, established on the write path by step 3 and +re-checked on every message, and the handshake asserts nothing. + +Two consequences worth stating. Anyone may `Open` any topic, including a topic whose +key they do not hold — but such a cohort can never accept a message, and the genuine +publisher's `Open` takes the role from them under the transfer rule above, so squatting +costs the squatter a stream and buys nothing. And BPS-lite makes no confidentiality +claim: broadcasts are readable by any subscriber, audience control is the application's +business, and SWIP-60's `closed` flag — which does make such a claim on an +unauthenticated handshake — is deliberately absent. + +### Differences from SWIP-60 + +Non-normative, for readers arriving from the full spec. Three things differ: + +- **No `PublisherRegime`, and no publisher set.** SWIP-60's `publishers`, `admin`, + `publisher_list`, `history` and `closed` have no counterpart. `CohortSpec` is + `{topic, binding}`. +- **`ANCHOR`'s address constraint is unconditional.** SWIP-60 relaxes it under an + explicit publisher regime, where the topic becomes a rendezvous and legitimacy comes + from list membership instead. BPS-lite has no list, so the constraint always applies. + This is the trap for anyone reusing SWIP-60's `anchorBinding` unmodified: its + explicit-regime early return would leave a BPS-lite cohort with no address check at + all. +- **The publisher role is bound to the opening stream**, and transfers on re-`Open`. + SWIP-60 leaves cohort lifetime and reattachment unspecified. + +The first two are what make the spec small; the third is what makes it operable. Each +is a divergence, not a restriction: a SWIP-60 implementation pointed at a BPS-lite +cohort would accept messages BPS-lite refuses, and a SWIP-60 client's `Open` carries +fields BPS-lite reserves. Hence a separate document — but not a separate protocol id. +BPS-lite and SWIP-60 are two specifications of one protocol, `pubsub/1.0.0`, and a peer +dials it without knowing which it will meet. The first frame settles that: a BPS-lite +broker answers a SWIP-60 `Open` with `REJECTED`, because every field the fuller spec +adds is a field this one reserves. Refusal at the handshake is the compatibility story, +and it is why those numbers are reserved rather than reused. + +### Why the publisher field is absent + +Under `ANCHOR` the topic is `keccak256(id || owner)`. Comparing a message's recovered +owner against a declared publisher is then a test that can only fail where the address +check has already failed, short of a keccak preimage. Carrying the field would cost a +20-byte field, a validation step, an owner comparison on re-`Open`, and — because the +field is asserted and never proved — a paragraph explaining what it does not mean. The +topic does the work; the field is deleted. + +### Why loopback is opt-in + +The publisher's stream is the cohort's lifetime, so the queue-overflow policy that is +merely lossy for a subscriber is fatal for the whole cohort. A publisher that does not +drain its own echo would destroy what it is publishing to. Making the echo opt-in and +exempting that stream from the reset removes a failure mode that has no analogue on the +subscriber side. + +## Backwards compatibility + +BPS-lite introduces no incompatibility with any deployed protocol: Swarm has no +broadcast pub/sub in production. + +It shares the `pubsub/1.0.0` protocol id with [SWIP-60][swip60] while **not** being +wire-compatible with it. That is a deliberate choice, and it puts the whole weight of +the compatibility story on the first frame of a stream: + +- A SWIP-60 `Open` sets `CohortSpec` fields 3–8 and `Open` field 2, all of which + BPS-lite reserves, so it is `REJECTED` rather than silently decoded with those fields + dropped — which is what proto3's permissive unmarshalling would otherwise do, leaving + a peer that asked for a five-author closed cohort holding an open single-publisher + one. The reserved numbers are what make the refusal reliable, and they are the reason + a BPS-lite broker MUST reject a frame carrying them rather than ignore it. +- A BPS-lite `Open` is a valid SWIP-60 `Open` — `{topic, binding}` with everything else + unset. A SWIP-60 broker would accept it, and then not enforce the address constraint + (see the next bullet). A BPS-lite publisher therefore gets weaker guarantees from a + full-spec broker than from a conformant one, silently. Applications that depend on + the `ANCHOR` rule should treat an unexpected `Ack` shape as a mismatch. +- BPS-lite's `ANCHOR` applies its address constraint unconditionally; SWIP-60 relaxes + it under an explicit publisher regime. An implementation reusing SWIP-60 `ANCHOR` + code unmodified would perform no address check at all under BPS-lite, which is the + one silent failure this divergence can cause. It is called out again in *Differences + from SWIP-60* above. + +Should BPS-lite and SWIP-60 later converge, the reserved field numbers leave the +migration path open: a future revision can un-reserve them with their SWIP-60 meanings +intact. + +## Test cases + +An implementation is BPS-lite conformant if it exhibits all of: + +1. `Open` with `binding != ANCHOR` → `REJECTED`. +2. `Open` carrying a reserved field, or a `CohortSpec` carrying one → `REJECTED`. +3. `Subscribe` carrying reserved field 2 → `REJECTED`. +4. `Subscribe` before any `Open` → `UNKNOWN_TOPIC`. +5. Re-`Open` on a live topic with an identical spec → `OK`; the prior publisher stream + is reset; the cohort and its dedup window survive. A differing spec → `REJECTED`. +6. A superseded publisher stream closing after a role transfer → cohort unaffected. +7. Current publisher stream closes → every subscriber stream reset, topic subsequently + `UNKNOWN_TOPIC`. +8. `Publish` whose SOC address is not `spec.topic` → dropped and counted. +9. `Publish` on a subscriber stream → dropped, counted, and that stream reset. +10. A duplicate CAC within the dedup window → dropped, counted separately. +11. Subscriber arriving at per-topic capacity → `FULL`, with no referral frame. `Open` + at the live-cohort bound → `FULL`. +12. Publisher stream outbound queue full → frame dropped, stream retained, cohort alive. +13. `Open` without `loopback` → the publisher receives no `Broadcast` frames. + +A broker MUST expose per-cohort counters for the silent-drop outcomes — +`invalid_address`, `invalid_signature`, `invalid_cac`, `wrong_stream`, `duplicate`, +`queue_dropped`. Points 8–10 and 12 are unobservable from the wire without them, so the +counters are part of the conformance surface rather than an implementation detail. + +## Implementation + +Groundwork exists in the bee prototype `pkg/bps` ([bee #5435][bee5435]), written against +SWIP-60. It is a superset of this spec in most places and diverges in one: + +`pkg/bps` is a superset of this spec in some places and diverges in one: it implements +`ANCHOR` and `FEED_TOPIC` bindings, `EXPLICIT_SINGLE` and `EXPLICIT_LIST` regimes, the +`Hello` envelope and a WS bridge. Most of the work is subtraction: + +| file | under BPS-lite | +|---|---| +| `publisher.go` | `authorizePublisher` goes entirely — there is no publisher to authorize against, only the topic | +| `binding.go` | `anchorBinding` only; the registry stays as the extension seam with one entry. `qualifies` **loses its explicit-regime early return**: the address check becomes unconditional | +| `cohort.go` | `Publishers`, `sortedCopy`, the regime switch in `ValidateSpec` and the publisher-list branch of `SpecEqual` go; `ValidateSpec` checks two fields | +| `broker.go` | `admit` sets the publisher role from the `Open` path rather than from `auth != nil`; the `PublisherAuth`-is-not-a-credential commentary goes with it | +| `broker.go` | gains a `publisher *peerStream` on the cohort: teardown fires only when the *current* publisher stream closes, and re-`Open` transfers it | +| `broker.go` | gains the queue-reset exemption for the publisher's stream | + +Two behaviors are new — the lifetime rule and role transfer. Everything else is removal. + +[swip60]: https://github.com/ethersphere/SWIPs/pull/104 +[bee5435]: https://github.com/ethersphere/bee/pull/5435 + +## References + +Parent design space: [SWIP-60 "BPS singlehop — brokered broadcast pub/sub, base +protocol", PR #104][swip60] · origin: +[PR #93](https://github.com/ethersphere/SWIPs/pull/93) "Add: pubsub" · implementation: +bee [#5435][bee5435] + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From 52444a3f9610679614368e824a3706b5ae3f66d9 Mon Sep 17 00:00:00 2001 From: zelig Date: Tue, 22 Sep 2026 16:12:37 +0200 Subject: [PATCH 02/13] =?UTF-8?q?SWIP-074:=20BPS-lite=20=E2=80=94=20rewrit?= =?UTF-8?q?e=20as=20the=20base=20of=20the=20BPS=20family,=20single=20publi?= =?UTF-8?q?sher=20over=20a=20feed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrite of the first draft after review. BPS-lite is now a strict subset/base of SWIP-60 rather than a divergent profile: the live-stream configuration with its wire reduced to what it needs. - cohort spec = {topic, FEED_TOPIC, admin}, and the spec is the cohort's identity - one handshake frame, Join{CohortSpec, Auth?}: create-or-attach, anyone may do either; no Hello/Open/Subscribe, no UNKNOWN_TOPIC - Auth binds the identity to the stream, not the connection; a stream whose Auth recovers to admin is a publisher stream, every other stream is read-only - Ack{status} only: OK / FULL / REJECTED - one Message{soc} frame both directions; the chunk as opaque chunk data with the bare index in the id slot (SWIP-65 carriage) - validation: publisher stream, index > cursor, valid SOC under keccak256(topic||index), owner == admin; no dedup window, gaps allowed, retransmits counted not punished - no loopback: deliveries to subscriber streams only - lifetime by inactivity only — no service messages, no end-of-stream - broker bounds: subscribers/cohort, cohorts/broker, cohorts/peer connection, inactivity deadline, per-stream queue - no history, no bandwidth incentive, no Bee API, one hop, one mode; implementation section removed - "Relation to SWIP-60" lists the amendments SWIP-60 needs to extend this wire (Join, cohorts keyed by spec, no GENESIS, regime reduced to ALL, spectators polarity, opaque chunk) and the one-line SWIP-65 exception - number assigned: 74; file renamed to swip-74.md One open point remains, marked (?): a second stream authenticating as the admin while one is live. Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 474 +++++++++++++++++++++++++++++++++++ SWIPs/swip-draft_bps-lite.md | 469 ---------------------------------- 2 files changed, 474 insertions(+), 469 deletions(-) create mode 100644 SWIPs/swip-74.md delete mode 100644 SWIPs/swip-draft_bps-lite.md diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md new file mode 100644 index 00000000..83ac8eee --- /dev/null +++ b/SWIPs/swip-74.md @@ -0,0 +1,474 @@ +--- +SWIP: 74 +title: BPS-lite — single-publisher live stream over a feed, one broker, one hop +author: Elad Nachmias (@acud), Viktor Trón (@zelig) +discussions-to: https://discord.gg/Q6BvSkCv +status: Draft +type: Standards Track +category: Networking +created: 2026-09-14 +--- + + + +- **Business line**: a live stream on Swarm — one author, an audience, real time, no + storage round trip and no polling: video/audio streaming, a price ticker, a game's + server-authoritative state, a log tail. The stream is a feed, so the same updates can + later be persisted and replayed from storage by anyone who missed the live run. +- **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the three frames + below — one broker, direct streams, one authenticated publisher, read-only + subscribers — and nothing else: **no history, no bandwidth incentive, no service + messages, no Bee API, one hop, one mode.** Done when a broker, a publisher and + subscribers from independent implementations interoperate per the conformance + section. Everything the family adds — service messages and the Bee API, multiple + publishers, self-indexed feeds, multihop — extends this wire without changing it. +- **DISC change**: NO. A new p2p protocol surface; no storage, retrieval or incentive + change. + +## Simple Summary + +BPS-lite is a real-time broadcast protocol: a kind of push notification on a *channel* +that anyone can subscribe to and one participant can publish on. Each channel defines +its own cohort — a set of nodes connected through a unique central broadcaster node, the +**broker**; every node in the cohort is directly connected to the broker, hence +*singlehop*. A channel is identified by its spec — a feed topic and the address of its +**admin**. Anyone joins by sending that spec; the admin also sends a signature that binds +its identity to the stream, and has the exclusive right to publish. Its messages are +single-owner chunks that are the higher-index updates of the feed the topic names, +carried with their bare index; the broker accepts each one iff its index exceeds the +channel's cursor, it validates as a single-owner chunk under the feed's id and its owner +is the admin, and delivers it unchanged to every subscriber stream — never to a +publisher's. All other joiners are subscribers and verify the same way, end to end: the +broker can withhold, never forge. A channel ends when it goes idle. + +## Motivation + +SWIP-60 spans several cohort configurations, five topic bindings, a roster control plane +on a service feed, a history flag and the Bee API. That is the right target for the +reference implementation and too much for a second, independent one whose application +is one author broadcasting to an audience. BPS-lite names that one configuration and +specifies only what it needs, so that a second team can implement and conformance-test +it from three frames — and so that it stays the **base** of the family: nothing here is +a divergence to reconcile later, because the fuller protocol extends these frames. + +One publisher over a feed also removes what the fuller protocol has to carry: a dedup +window (the feed cursor replaces it), a roster and the service feed that carries it (there +is nothing to announce), a publisher regime (there is one publisher), and any question of +who may write (the admin, proved by its signature on every message). + +## Specification + +The key words MUST, MUST NOT, REQUIRED, SHOULD, RECOMMENDED and MAY are to be interpreted +as described in RFC 2119. Terms — cohort (a channel's set of nodes), broker, admin, subscriber — are SWIP-60's. + +### The contract + +Per cohort: messages come from **the admin, and the admin only**; they arrive at **all +subscribers**; they are **feed updates, in increasing index order**. + +### Topology and roles + +- **Broker**: one full node per cohort. Every peer holds one direct libp2p stream per + (peer, cohort) to it, protocol id `pubsub/1.0.0`. Depth is 1 by construction: no + relaying, no referral. Broker discovery is out of scope; deployments configure the + broker. +- **Admin = the publisher**: the address named in the cohort spec. It is the only + identity whose messages are accepted. It need not be first to join, and the cohort + does not end when its stream goes away. +- **Subscriber**: joins by naming the cohort, receives, never sends. A message from a + subscriber stream is a protocol violation. +- **Publisher and broker are distinct nodes**; brokering for oneself is out of scope. + +### The cohort spec + +Three fields, no others: + +| field | value | +|---|---| +| `topic` | 32 bytes: the feed topic | +| `binding` | `FEED_TOPIC`: the signed id of every message is `keccak256(topic ‖ index)` | +| `admin` | 20-byte address: the publisher | + +**The spec is the cohort's identity.** Two peers naming byte-identical specs at a broker +are in the same cohort; a spec that differs in any field names a different one. The +invite for a stream is therefore the spec plus the broker: `topic`, `admin`, broker +address. No publisher regime, no roster, no spectator flag, no history flag, no +proximity constant: a BPS-lite broker answers any spec outside this table, and any +reserved field set, with `REJECTED`. Capacity is not in the spec: a cohort cannot +dictate a remote node's connection count. + +### Wire format + +Normative and complete. Field and enum numbers are SWIP-60's where a counterpart exists, +so that the fuller protocol extends these messages without renumbering; numbers it uses +and this SWIP does not are reserved, never reused. An unset enum (`*_UNSPECIFIED`, the +proto3 zero) is not a legitimate wire value: receivers MUST reject a message carrying one. + +```proto +syntax = "proto3"; +package bps; + +// What the topic binds to. One binding. +enum TopicBinding { + TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire + FEED_TOPIC = 4; // signed id = keccak256(topic || index). SWIP-60's + // number; 1, 2, 3, 5 are SWIP-60's other bindings +} + +// The cohort's identity. Fixed by whoever joins first; immutable. +message CohortSpec { + bytes topic = 1; // 32 bytes + TopicBinding binding = 2; // FEED_TOPIC + bytes admin = 5; // 20 bytes, required + reserved 3, 4, 6, 7, 8, 9; // SWIP-60: publishers, history, (unused), (unused), + // (unused), spectators — none apply here +} + +// Proof of an identity, bound to the stream it arrives on: +// owner = ecrecover( keccak256("bps-join:v1" || topic || admin), signature ) +// The preimage is static and carries no node identity (SWIP-60): the same key works +// from any node, and a join never links an address to a peer id. It is therefore +// replayable, and a replayed identity is worthless: authorship rests on the message +// signature, never on the handshake. +message Auth { + bytes signature = 1; // 65 bytes + reserved 2; // SWIP-60: id, for bindings that do not fix it +} + +// Peer -> broker: the first and only handshake frame on a fresh stream. Creates the +// cohort if no live cohort has this spec, attaches to it otherwise. `auth` binds an +// identity to THIS stream; absent, the stream has none and is read-only. +message Join { + CohortSpec cohort = 1; + Auth auth = 2; +} + +enum Status { + STATUS_UNSPECIFIED = 0; // invalid on the wire + OK = 1; + FULL = 2; // a capacity bound (see Resource bounds) + REJECTED = 4; // spec outside this SWIP, or a reserved field set + reserved 3; // SWIP-60: UNKNOWN_TOPIC — cannot occur: Join creates +} + +// Broker -> peer, answering Join. Status only. A non-OK Ack ends the stream. +message Ack { + Status status = 1; + reserved 2, 3, 4, 5; // SWIP-60: spec echo and service SOCs — nothing to echo +} + +// Both directions after the handshake: admin -> broker is a publication, broker -> +// subscriber a delivery of the same bytes. The single-owner chunk travels as its +// stored chunk data, opaque to the protocol and validated by the ordinary SOC code +// once the `id` slot has been rewritten as the Frames section says: +// id (32) || signature (65) || span (8, LE) || payload (<= 4096) +message Message { + bytes soc = 1; +} +``` + +Three frames — `Join`, `Ack`, `Message` — and the two types they carry, `CohortSpec` and +`Auth`. There is no envelope around `Join` — it is the only first frame there is — and no +separate publish and broadcast types: a broker pushes to subscribers, a non-broker does +not. Fields this SWIP does not define are ignored on receipt, as proto3 does; a `Message` +whose `soc` is empty is invalid. + +### Handshake: `Join` + +Every peer sends `Join` as the first frame on a fresh stream, carrying the full +`CohortSpec` and, if it claims an identity, an `Auth`. + +The broker compares the spec against its live cohorts: **no match → the cohort is +created** with the joiner attached; **match → the joiner is attached** to it. Anyone may +create, including a subscriber arriving before the admin: a publisher-less cohort costs +the broker one map entry until the inactivity deadline reclaims it, and can accept no +message. There is nothing to pre-register and no "unknown topic". + +Then the stream's role, from `Auth`: + +| `Auth` | the stream is | +|---|---| +| absent | a **subscriber stream**, no identity | +| recovers to `spec.admin` | a **publisher stream** | +| recovers to any other address | a **subscriber stream** carrying that identity — no effect in this SWIP | + +**The identity is bound to the stream, not to the peer connection.** One stream per +(peer, cohort); a node may present different identities on different streams, and a +stream without `Auth` has none. **(?)** What if a second stream authenticates as the +admin while a first one is live — the admin from two nodes, or reconnecting before its +old stream is torn down? This draft admits both as publisher streams and lets the +per-cohort cursor arbitrate (neither can publish an index the other already has); the +alternatives are that the new stream supersedes the old (reset), or that the second is +`REJECTED`. + +The broker answers `Ack{status}`: `OK`, `FULL` at a capacity bound, `REJECTED` for a spec +outside this SWIP or a reserved field set. A non-`OK` `Ack` ends the stream; on `OK` the +stream is retained. A peer whose stream was refused or reset MUST back off before +rejoining (randomised exponential, 1 s to 30 s, full jitter); a broker MAY reset a stream +that retries faster. Without this every reclaim or reset of a cohort brings its whole +audience back at once. + +### Frames + +After the handshake there is one frame, `Message`, and it is direction-typed: on a +publisher stream towards the broker it is a publication; from the broker on a subscriber +stream it is a delivery. A delivery is the accepted publication's bytes, unchanged. + +**The `id` field carries the bare index.** A feed update's signed id is +`keccak256(topic ‖ index)`, `index` a uint64 big-endian in 8 bytes. On the wire the +32-byte `id` slot of the chunk data holds that index left-padded with 24 zero bytes, and +the receiver reconstructs the signed id from the cohort's topic (the carriage of +[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). A `Message` whose `id` slot +does not have 24 leading zero bytes is not a feed update: a broker drops it as invalid; a +subscriber drops it without counting it as a violation. + +### Validation: the feed cursor + +The broker keeps one **cursor** per cohort — the highest index accepted so far, initially +none. A `Message` arriving at the broker is accepted iff, in order: + +1. it arrived on a **publisher stream** — on any other stream it is a protocol + violation: the frame is dropped, the stream reset and the peer blocklisted per the + node's policy; +2. its `id` slot is a bare index `n` and **`n > cursor`**; +3. with the id substituted by `keccak256(topic ‖ n)` the chunk **validates as a + single-owner chunk**: the wrapped chunk's BMT address matches `span ‖ payload`, and + the signature over `id ‖ wrappedAddress` recovers an owner; +4. the recovered owner is **`spec.admin`**. + +On acceptance the broker sets `cursor := n` and enqueues the frame on **every subscriber +stream** in the cohort. Publisher streams receive nothing: a publisher never gets its own +messages back, on whichever of its streams they were sent. + +Failures of 3 and 4 are invalid frames: dropped and counted; repeated invalid frames end +the connection (blocklisting policy). A frame failing 2 with `n ≤ cursor` is the one +benign failure — a retransmit, which an admin reconnecting after a reset legitimately +sends when it does not know what the broker last accepted — and is counted separately; a +broker MAY reset a publisher stream whose retransmit rate exceeds its policy, since the +cursor check is cheap and precedes the signature check. A frame failing 2 because the +`id` slot is not a bare index is invalid. + +The cursor is initially **absent, not zero**: index 0 is the first update of every feed +and MUST be accepted on a fresh cohort. + +**There is no dedup window.** The cursor is total: a message is either beyond the last +accepted index or it is not, and there is no eviction and no replay hole. **Gaps are +allowed** at the broker (`n > cursor + 1`): they are the publisher's business, and +SWIP-65 makes them detectable and recoverable at the subscriber. This is stricter than +SWIP-65's general carriage rule, under which a broker does not enforce monotonicity; the +restriction is sound here because one publisher on one hop admits no legitimate +reordering. + +Subscribers re-verify every delivery exactly as the broker does (steps 2–4, against the +spec they joined with and their own cursor), end to end, whatever the broker did. + +### Lifetime: inactivity + +**A cohort ends by inactivity, and by nothing else.** A cohort on which no publisher +stream has had a message accepted for the **inactivity deadline** is reclaimed: the +broker resets every stream in it and forgets it. There is no end-of-stream signal in +BPS-lite — that is a service message, and this SWIP has none; an application that needs +"over" to be distinguishable from "paused" sends it in its last update, or waits for the +service feed the family adds. + +A publisher stream going away — closed or reset — is therefore not an end: the cohort +and its cursor persist, subscribers stay attached, and the admin rejoins with `Auth` and +resumes at `n > cursor`. Likewise a subscriber-only cohort waits for its admin until the +deadline reclaims it. A cohort with **no attached streams** MAY be reclaimed at once — +nothing observes the difference beyond a fresh cursor on the next `Join`. + +After a cohort is forgotten a later `Join` creates a fresh one with an empty cursor. +Subscribers keep their own cursor across that, so a stale republication is caught at the +edge, not at the broker. + +### Resource bounds + +All broker policy, none on the wire. The first four bounds are REQUIRED, with the values +given RECOMMENDED where a value is given; the last two are MAY: + +| bound | answer | recommended | +|---|---|---| +| subscriber streams per cohort | `FULL` to the next `Join` without an admin `Auth` — the audience cannot lock the admin out of its own cohort | implementation-defined | +| live cohorts per broker | `FULL` to a cohort-creating `Join` | implementation-defined | +| **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | +| **inactivity deadline** — reclaims a cohort, see *Lifetime* | reset | 10 min | +| outbound queue per subscriber stream, one writer; a full queue resets that stream | reset | 64 frames | +| a peer connection with no live stream | MAY be disconnected at the transport | — | +| a cohort with no attached streams | MAY be reclaimed at once | — | + +Any peer can make a broker allocate a cohort simply by joining, which is why the second, +third and fourth bounds exist together: a cohort costs a map entry, a peer can hold only +a bounded number of them, and none survives idleness. The broker has no delivery +obligation: a reset for a full queue is a recoverable liveness fault, and blocking +fan-out on one slow subscriber would punish the cohort. + +### Relation to SWIP-60 + +BPS-lite is the **base** of the BPS family and SWIP-60 is its full singlehop protocol. +The relation is subset, and it is kept in one direction only — **later revisions extend +this wire, they never change it**: + +- a BPS-lite peer sends only frames the full protocol accepts, and a BPS-lite broker + accepts only what this SWIP defines; everything the full protocol adds to the + handshake arrives in fields this SWIP reserves, so a full-protocol `Join` at a + BPS-lite broker is `REJECTED` at the handshake rather than silently misread, and + everything it adds to `Message` arrives in fields this SWIP does not define and + ignores; +- a BPS-lite publisher and subscriber at a full broker are conformant peers of a + single-publisher live-stream cohort; the full broker's additions (service messages, + further `Ack` fields) reach them as frames they drop and fields they ignore; +- validation here is stricter, never looser: the cursor refuses out-of-order + retransmits a full broker may pass through; no frame a BPS-lite broker accepts is one + a full broker refuses. + +SWIP-60 is to be amended to match — its author's to-do, listed here so that the subset +claim is checkable: + +- `Hello`/`Open`/`Subscribe` collapse into `Join` carrying the spec, and **cohorts are + keyed by the whole spec, not by the topic**: a `Join` whose topic is live under a + different spec creates a second cohort rather than being `REJECTED`, so a squatter + who pre-creates `(topic, wrong admin)` obtains nothing; +- `GENESIS` goes (with the spec in `Join` it has nothing left to prove) and `Ack` echoes + nothing; +- the publisher regime shrinks to a single value, `ALL` — anyone attached may publish; + absent, the admin publishes and whoever its roster ever names, so a cohort is + multi-publisher iff its admin publishes a roster, and nobody needs to know in advance; +- **`spectators` (field 9) goes, or reverts to a `closed` flag whose unset value means an + open audience** — as a proto3 bool whose unset value is *false*, it reads a lite + spec, which never sets it, as a closed cohort, and a full broker would refuse every + lite subscriber; +- the chunk travels as opaque chunk data. + +And one line in SWIP-65: its carriage rule, under which a broker does not enforce index +monotonicity, gets the exception this SWIP's cursor is. + +## Rationale + +**Why a feed, not an anchor.** The first draft bound the topic to a single SOC address +(`ANCHOR`), so that authorship followed from the topic and no publisher field was +needed. That collapses the wrong thing: one address for every message means messages +are told apart only by their wrapped payload, dedup needs a window, order is invisible, +and nothing connects the live stream to storage. Binding to a feed keeps the publisher +explicit — one address in the spec, one owner check per message — and buys order, gap +detection, replay-freedom and persistence for the same money: the stream *is* a feed, +live. + +**Why `Join` carries the spec, and there is only `Join`.** A joiner that arrives with a +topic and learns the spec from the broker has to be told the spec by someone it then +has to verify. For a live stream the spec is a topic and an address; the invite that +names the broker names them too. Once every joiner carries the spec, the spec is the +cohort's identity, the broker is never trusted about it, `Ack` is a status, the +first-joiner-creates rule is the natural one — and a second handshake type has nothing +to do. + +**Why the identity is on the stream.** `Auth` is in `Join`, and what it proves is bound +to the stream it arrives on. A peer connection is a node; a stream is a (node, cohort) +pair; the identity that matters is the key that signs the messages on that stream. Binding +it there is what lets one node carry different identities on different cohorts, and what +later lets a subscriber stream be promoted to a publisher stream in place, without a +rejoin, when the admin grants it. + +**Why no loopback.** A publisher knows what it published. Echoing it costs a frame per +message on the stream whose back-pressure matters most, and buys no confirmation: a +rejected frame produces no echo either. + +**Why the cursor, not a window.** A bounded seen-set is a memory bound with a replay +hole at its edge, and it exists because the general protocol admits many publishers and +duplicate paths. One publisher on one hop has neither. The feed index is a total order, +and "greater than the last" is both the dedup rule and the memory bound. + +**Why inactivity, and only inactivity.** An end that is *signed* by the admin is a service +message, which this SWIP does not have; an end that is *inferred* from the admin's stream +turns every transport hiccup into a stream-ending event for the whole audience. So there +is no end: a cohort is a map entry that lives while it is used and is reclaimed when it is +not. What that gives up — "over" versus "paused" — is the application's to carry until the +service feed arrives. + +## Security considerations + +Those of SWIP-60, restricted to its live-stream configuration (admin set, the admin +alone publishes, open audience). + +**The publisher role is proved, not asserted**: `Auth` recovers to `admin`, and it grants +nothing the admin's key has not already granted. Its preimage is static, so it is +replayable: a peer that captured the admin's `Join` can obtain a publisher stream, but +can send on it only what the admin already signed. Within one cohort's life the cursor +refuses that (every captured index is at or below it); **across a reclaim or at another +broker the cursor starts empty, and the base protocol does not distinguish a replay of +the admin's history from the admin's history**. Freshness is therefore a subscriber-side +property: a subscriber SHOULD keep its cursor per `(topic, admin)` across rejoins, and an +application that needs freshness for a first-time viewer carries it in the payload +(SWIP-65's timestamp key does). Fan-out of a replayed history is bounded by what was +captured and by the inactivity deadline. + +**No confidentiality**: the broker and every subscriber see plaintext; applications +encrypt payloads. **The broker withholds, never forges**, and a withheld update is +visible as a gap in the index. **No end signal**: a broker can end a cohort for its +audience by resetting their streams, which is withholding, nothing more. **Resource +bounds are policy and the four capacity bounds are required** (above); the cursor +removes the dedup-window bound and its edge. A subscriber that publishes is a protocol +violation and is blocklisted; a squatted cohort is one map entry for one inactivity +deadline, and a peer can hold only a bounded number of them. + +## Conformance (definition of done) + +An implementation is BPS-lite conformant when: + +1. a broker accepts a `Join` whose spec is `{topic, FEED_TOPIC, admin}` and answers any + other binding, an absent `admin`, or any reserved field set with `REJECTED`; +2. a `Join` for a spec with no live cohort creates it, whoever sends it; a `Join` with a + byte-identical spec attaches to it; the broker never sends status 3; +3. `Auth` is verified by recovery over `keccak256("bps-join:v1" ‖ topic ‖ admin)`; a + stream whose `Auth` recovers to `admin` is a publisher stream; any other stream is a + subscriber stream; the identity is a property of the stream; +4. `Ack` carries a status and nothing else; a non-`OK` `Ack` ends the stream; +5. a `Message` on a subscriber stream is dropped, the stream reset, the peer blocklisted; +6. a `Message` on a publisher stream is accepted iff its bare index exceeds the cursor, + it validates as a single-owner chunk under `keccak256(topic ‖ index)`, and its owner + is `admin`; accepted frames advance the cursor and are delivered unchanged to every + subscriber stream in the cohort, and to no publisher stream; +7. `n ≤ cursor` is counted as a retransmit, not a violation, and there is no other + dedup state; gaps are accepted; index 0 is accepted on a fresh cohort; +8. a cohort with no accepted message for the inactivity deadline is reclaimed, every + stream in it reset; a publisher stream going away does not end the cohort; +9. the capacity bounds are enforced, `FULL` is issued at capacity and nothing else is; +10. a subscriber re-verifies every delivery against the spec it joined with and its own + cursor; +11. a BPS-lite publisher and subscriber interoperate with a full SWIP-60 broker on a + single-publisher cohort once SWIP-60 is amended per *Relation to SWIP-60*. + +A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, +`invalid_soc`, `wrong_owner`, `wrong_stream`, `retransmit`, `queue_reset` — since +items 6, 7 and 9 are unobservable from the wire without them. + +## Out of scope (deliberately) + +Service messages of any kind (roster, end of stream) and the Bee API bridge; multiple +publishers and the promotion of a subscriber stream to a publisher stream; the +self-indexed payload construction, gap recovery and persistence of +[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106); multihop +([SWIP-61](https://github.com/ethersphere/SWIPs/pull/105)); history; bandwidth +incentives; broker discovery ([SWIP-59](https://github.com/ethersphere/SWIPs/pull/103)); +confidentiality of any kind. Each extends this SWIP's wire without changing it. + +## Backwards compatibility + +New protocol; no existing behaviour changes. Every message and field number here is +kept by the fuller protocol, which extends by adding fields and messages and never by +changing these; a BPS-lite peer ignores fields it does not define. + +## References + +Full singlehop protocol: [SWIP-60, PR #104](https://github.com/ethersphere/SWIPs/pull/104) +· carriage: [SWIP-65 self-indexed feeds, PR #106](https://github.com/ethersphere/SWIPs/pull/106) +· first draft of this SWIP: [PR #111](https://github.com/ethersphere/SWIPs/pull/111) +· origin: [PR #93](https://github.com/ethersphere/SWIPs/pull/93) "Add: pubsub" +· implementation groundwork: bee [#5435](https://github.com/ethersphere/bee/pull/5435) + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). diff --git a/SWIPs/swip-draft_bps-lite.md b/SWIPs/swip-draft_bps-lite.md deleted file mode 100644 index 775b7252..00000000 --- a/SWIPs/swip-draft_bps-lite.md +++ /dev/null @@ -1,469 +0,0 @@ ---- -SWIP: -title: BPS-lite — single-publisher brokered broadcast -author: acud (@acud) -discussions-to: https://discord.gg/Q6BvSkCv -status: Draft -type: Standards Track -category: Networking -created: 2026-09-14 ---- - - - -## Simple Summary - -One broker, one publisher, many subscribers, one hop. A publisher opens a topic on a -broker and pushes single-owner chunks; everyone else connects and reads. Nothing else. - -## Abstract - -BPS-lite is a real-time broadcast protocol in which a cohort is a topic and a topic is -a single-owner chunk address. The peer that opens a cohort is its publisher for as long -as that stream lives; every other peer is a read-only subscriber. Authorship needs no -credential, no publisher list and no handshake authentication: under the `ANCHOR` -binding the topic already names the only key that can produce a chunk hashing to it, so -authorship is re-established from the signature on every message. The cohort's lifetime -is its publisher's stream, which removes orphan cohorts and bounds `Open` as an -allocation primitive. - -It is a deliberate subset of the design space [SWIP-60][swip60] covers, specified as -its own document rather than as a profile: the wire format below is complete, and where -BPS-lite differs from SWIP-60 — most importantly in making `ANCHOR`'s address -constraint unconditional — BPS-lite's rule governs. No compatibility is claimed in -either direction. - -## Motivation - -SWIP-60 spans five cohort shapes, four topic bindings, four publisher regimes, an admin -control plane and a history flag. That parameter space is the right target for the -reference implementation and the wrong one for a second, independent implementation: -most of it is untested surface for anyone whose application is one author broadcasting -to an audience — a live feed, a price ticker, a game's server-authoritative state, a -log tail. - -BPS-lite names the single point in that space which needs none of the machinery, and -shows that pinning it collapses the spec rather than merely constraining it. Three -questions SWIP-60 leaves open — the dedup horizon, cohort lifetime, and `Open` as an -unbounded allocation primitive — are answered here, and one property SWIP-60 claims -(`closed` as confidentiality over an unauthenticated handshake) is dropped rather than -restated. The result is a target roughly a tenth the size, which a second team can -implement and conformance-test without tracking the full spec's revisions. - -## Specification - -The key words MUST, MUST NOT, REQUIRED, SHOULD, RECOMMENDED and MAY are to be -interpreted as described in RFC 2119. - -### Roles and topology - -One broker node. One publisher peer. N subscriber peers. Every peer dials the broker -directly over a single libp2p stream per (peer, topic), protocol id -`pubsub/1.0.0` — the same id SWIP-60 uses, shared deliberately: see *Backwards -compatibility*. No relaying, no referral, no broker discovery: depth is 1 by -construction. - -The publisher and the broker are distinct peers. Brokering for oneself is out of scope. - -**The peer that opened a cohort is its publisher for as long as that stream lives.** -The role is a property of the stream, not of a field in the spec: subscriber streams are -read-only and the broker MUST NOT read `Publish` frames from them. - -### Wire format - -This section is normative and complete: BPS-lite is defined by the messages below and -by nothing else. No import, no companion `.proto`, no field inherited by reference. -Reserved field numbers are SWIP-60's, kept reserved so that a frame from a full-spec -implementation is refused rather than silently misread as a valid BPS-lite frame. - -```proto -// BPS-lite — single-publisher brokered broadcast. Complete wire format. -// -// Enum zero values (*_UNSPECIFIED): proto3 requires a zero value; it is -// deliberately NOT a legitimate wire value. It exists so that an unset field is -// detectable and no implementation can silently rely on a default. Receivers -// MUST reject any message carrying one. - -syntax = "proto3"; - -package bps; - -// ---------------------------------------------------------------- cohort - -// What the topic binds to. BPS-lite has exactly one binding: the topic is the -// full SOC address. Unlike SWIP-60 the constraint is unconditional — there is -// no publisher regime that can relax it. -enum TopicBinding { - TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire - ANCHOR = 1; // topic = SOC address; dedup on the wrapped CAC -} - -// Fixed by the cohort's opener; immutable for the cohort's lifetime. -// A cohort is a topic: there is no publisher field, because under ANCHOR the -// topic already names the only key that can author for it. -message CohortSpec { - bytes topic = 1; // 32B: the SOC address, keccak256(id || owner) - TopicBinding binding = 2; // must be ANCHOR - reserved 3, 4, 5, 6, 7, 8; // SWIP-60: publishers, history, admin, - // publisher_list, po_min, closed -} - -// ------------------------------------------------------------- handshake - -// Opener -> broker: the one peer that fixes the cohort. The stream carrying -// this frame is the cohort's publisher for as long as it lives. -message Open { - CohortSpec cohort = 1; - reserved 2; // SWIP-60: auth. BPS-lite authenticates nothing at - // handshake time; authorship is checked per message. - bool loopback = 3; // deliver the cohort's own Broadcast frames back -} - -// Joiner -> broker: names the topic — nothing more. Subscriber streams are -// read-only; there is no role to claim here. -message Subscribe { - bytes topic = 1; // 32 bytes - reserved 2; // SWIP-60: auth. A frame carrying it is REJECTED. -} - -// Peer -> broker: the first frame on a fresh stream. Open and Subscribe are -// otherwise indistinguishable on the wire — both encode as a single -// length-delimited field 1, and proto3's permissive unmarshalling makes a -// wrong guess succeed silently. -message Hello { - oneof handshake { - Open open = 1; - Subscribe subscribe = 2; - } -} - -enum Status { - STATUS_UNSPECIFIED = 0; // invalid on the wire - OK = 1; - FULL = 2; // broker at a capacity limit: subscribers, or cohorts - UNKNOWN_TOPIC = 3; // Subscribe for a topic with no live publisher - REJECTED = 4; // unacceptable spec, or a reserved field set -} - -// Broker -> peer, answering Open or Subscribe. One per stream; a non-OK Ack -// ends the stream. -message Ack { - Status status = 1; - CohortSpec cohort = 2; // set iff status == OK -} - -// --------------------------------------------------------------- traffic - -// A full single-owner chunk in transit. Every frame is self-contained. -message Soc { - bytes id = 1; // 32 bytes - bytes owner = 2; // 20 bytes - bytes signature = 3; // 65 bytes - bytes span = 4; // 8 bytes LE - bytes payload = 5; // wrapped-CAC data, <= 4096 bytes -} - -// Publisher -> broker. Accepted only on the stream that opened the cohort. -message Publish { - Soc soc = 1; -} - -// Broker -> peer. -message Broadcast { - oneof frame { - Soc soc = 1; - // 2-15 reserved in SWIP-60 for the multihop control plane; BPS-lite is - // singlehop and defines none of them. - } -} -``` - -Everything after the handshake is direction-typed and needs no envelope: peer to -broker is always `Publish`, broker to peer is always `Broadcast`. - -A cohort spec carries two live fields. There is no publisher field, no publisher -list, no publisher regime, no `history`, no `closed`, no `PO_MIN`. Per-topic capacity -is broker policy and is not on the wire at all. - -`Open` carries no credential: nothing at handshake time can prove control of a key, -so BPS-lite does not ask. `Subscribe` carrying the reserved field 2 is `REJECTED` — -there is no second publisher to authenticate, and no role to claim at handshake time. - -### Handshake - -Publisher-first. The publisher fixes the cohort; subscribers join it. - -``` -publisher broker - | Hello{Open{spec, loopback}} | - |------------------------------------->| validate spec; create cohort; - | | this stream is the publisher - | Ack{OK, spec} | - |<-------------------------------------| - | Publish{soc} ... | - | Broadcast{soc} ... | iff loopback was requested - |<-------------------------------------| - -subscriber broker - | Hello{Subscribe{topic}} | - |------------------------------------->| UNKNOWN_TOPIC if no publisher has opened - | Ack{OK, spec} | - |<-------------------------------------| - | Broadcast{soc} ... | -``` - -Rules: - -- A non-`OK` `Ack` ends the stream. On `OK` the stream is retained. -- A `Subscribe` carrying reserved field 2 (SWIP-60's `auth`) is `REJECTED`. -- **`Open` on a live topic with an identical spec transfers the publisher role to the - new stream**: the broker answers `OK`, retains the cohort and its dedup window, and - resets the previous publisher stream. A publisher recovering from a dropped - connection therefore needs no distinct reattach frame, and a stale half-open stream - closing later is not the current publisher stream and has no effect. An `Open` with - a different spec for a live topic is `REJECTED`. -- A subscriber arriving before the publisher gets `UNKNOWN_TOPIC` and retries. The - broker holds no pre-registration state. -- A peer receiving `UNKNOWN_TOPIC` or `FULL` MUST wait before retrying: randomized - exponential backoff from 1s to 30s, full jitter. A broker MAY reset a stream from a - peer that retries faster. Without this, every publisher restart resets N subscriber - streams at once and they all return immediately. -- The publisher's own stream receives the cohort's `Broadcast` frames only if it asked - for them (`Open.loopback`, default false). See *Capacity and slow peers* for why it - is opt-in. - -### Message validation - -A `Publish` is accepted iff, in order: - -1. it arrived on the stream that opened the cohort; -2. the `Soc` decodes and its signature recovers an owner; -3. the SOC address, `keccak256(id || owner)`, equals `spec.topic` — the `ANCHOR` rule; -4. the wrapped CAC's BMT root matches its payload; -5. the CAC address is not in the cohort's dedup window. - -Failures 1–4 are protocol violations: the message is dropped, counted per peer, and -exposed through the implementation's invalid-message hook. Failure 1 additionally -resets the offending stream. No blocklisting is mandated. Failure 5 is ordinary — a -retransmitting publisher — and is counted separately. - -Note what is *not* here: no comparison of the recovered owner against a declared -publisher. Under `ANCHOR` the topic is a SOC address, so step 3 already constrains the -owner — producing a SOC that hashes to `spec.topic` under a different key is a keccak -preimage. An owner check would be a second test that can only fail where step 3 has -already failed. - -On acceptance the broker enqueues a `Broadcast` on every subscriber stream in the -cohort, and on the publisher's own stream if it requested loopback. - -### Cohort lifetime - -**A cohort exists exactly as long as its current publisher's stream.** When that -stream closes or resets, the broker destroys the cohort and resets every subscriber -stream. Subscribers reconnect and receive `UNKNOWN_TOPIC` until the publisher returns. -A stream that has been superseded by a role transfer is not the current publisher -stream: its later close is a no-op. - -This is the largest simplification the single-publisher restriction buys. It removes -orphan cohorts and any reclamation policy, and it bounds `Open` as an allocation -primitive: a peer can hold only as many live cohorts as it holds open streams. - -### Dedup - -A bounded LRU of accepted CAC addresses, per cohort. Implementations MUST bound it; -1024 entries is RECOMMENDED. A role transfer retains the window. - -Replay is prevented by the stream-role rule, not by the window: a captured `Broadcast` -re-`Publish`ed by an observer carries the genuine publisher's signature and passes -steps 2–4, but is refused at step 1 because it arrives on a subscriber stream — -whatever its age. The window bounds duplicate suppression for a retransmitting -publisher only, and 1024 is chosen against that benign case. - -### Capacity and slow peers - -Per-topic subscriber capacity is broker policy, not a cohort parameter. At the limit -the broker answers `FULL` and nothing else; referral is bps-multihop's business. -A broker MUST also bound the number of live cohorts it will create; the lifetime rule -above makes that bound enforceable, and an `Open` at that bound is answered `FULL`. - -Each retained subscriber stream has a bounded outbound queue (64 frames RECOMMENDED) -drained by a single writer. A full queue resets that stream. The broker has no -delivery obligation: withholding is a recoverable liveness fault, not a correctness -failure, and blocking fan-out on one slow subscriber would punish the cohort. - -The publisher's stream is exempt from that reset. It is the cohort's lifetime, so a -policy that is merely lossy for a subscriber is fatal for everyone: a publisher that -does not drain its own loopback echo — a fast publisher, a single-threaded client, a -bridge applying backpressure — would otherwise destroy its own cohort. A full queue on -the publisher's stream drops the frame and increments a counter. Loopback is opt-in for -the same reason. - -Loopback is an echo of *accepted* messages, not delivery confirmation: rejected -`Publish` frames are dropped silently and produce no frame at all, so a publisher -cannot distinguish rejection from delivery by watching its echo. An acknowledged write -path would be an `Ack`-per-`Publish` design, which BPS-lite does not have. - -### Status codes - -| condition | status | -|---|---| -| handshake accepted | `OK` | -| broker at a capacity limit — per-topic subscribers for `Subscribe`, live cohorts for `Open` | `FULL` | -| `Subscribe` for a topic with no live publisher | `UNKNOWN_TOPIC` | -| unacceptable spec, differing spec on re-`Open`, a reserved field set | `REJECTED` | - -A `STATUS_UNSPECIFIED` or any other zero enum value received on the wire is rejected: -the proto's zero values are deliberately not legitimate wire values. - -## Rationale - -### Authorship without a credential - -BPS-lite has no publisher field and no handshake credential. A cohort's topic is a SOC -address; the only key that can produce a SOC hashing to it is the one that owns it. -Authorship is a property of the topic, established on the write path by step 3 and -re-checked on every message, and the handshake asserts nothing. - -Two consequences worth stating. Anyone may `Open` any topic, including a topic whose -key they do not hold — but such a cohort can never accept a message, and the genuine -publisher's `Open` takes the role from them under the transfer rule above, so squatting -costs the squatter a stream and buys nothing. And BPS-lite makes no confidentiality -claim: broadcasts are readable by any subscriber, audience control is the application's -business, and SWIP-60's `closed` flag — which does make such a claim on an -unauthenticated handshake — is deliberately absent. - -### Differences from SWIP-60 - -Non-normative, for readers arriving from the full spec. Three things differ: - -- **No `PublisherRegime`, and no publisher set.** SWIP-60's `publishers`, `admin`, - `publisher_list`, `history` and `closed` have no counterpart. `CohortSpec` is - `{topic, binding}`. -- **`ANCHOR`'s address constraint is unconditional.** SWIP-60 relaxes it under an - explicit publisher regime, where the topic becomes a rendezvous and legitimacy comes - from list membership instead. BPS-lite has no list, so the constraint always applies. - This is the trap for anyone reusing SWIP-60's `anchorBinding` unmodified: its - explicit-regime early return would leave a BPS-lite cohort with no address check at - all. -- **The publisher role is bound to the opening stream**, and transfers on re-`Open`. - SWIP-60 leaves cohort lifetime and reattachment unspecified. - -The first two are what make the spec small; the third is what makes it operable. Each -is a divergence, not a restriction: a SWIP-60 implementation pointed at a BPS-lite -cohort would accept messages BPS-lite refuses, and a SWIP-60 client's `Open` carries -fields BPS-lite reserves. Hence a separate document — but not a separate protocol id. -BPS-lite and SWIP-60 are two specifications of one protocol, `pubsub/1.0.0`, and a peer -dials it without knowing which it will meet. The first frame settles that: a BPS-lite -broker answers a SWIP-60 `Open` with `REJECTED`, because every field the fuller spec -adds is a field this one reserves. Refusal at the handshake is the compatibility story, -and it is why those numbers are reserved rather than reused. - -### Why the publisher field is absent - -Under `ANCHOR` the topic is `keccak256(id || owner)`. Comparing a message's recovered -owner against a declared publisher is then a test that can only fail where the address -check has already failed, short of a keccak preimage. Carrying the field would cost a -20-byte field, a validation step, an owner comparison on re-`Open`, and — because the -field is asserted and never proved — a paragraph explaining what it does not mean. The -topic does the work; the field is deleted. - -### Why loopback is opt-in - -The publisher's stream is the cohort's lifetime, so the queue-overflow policy that is -merely lossy for a subscriber is fatal for the whole cohort. A publisher that does not -drain its own echo would destroy what it is publishing to. Making the echo opt-in and -exempting that stream from the reset removes a failure mode that has no analogue on the -subscriber side. - -## Backwards compatibility - -BPS-lite introduces no incompatibility with any deployed protocol: Swarm has no -broadcast pub/sub in production. - -It shares the `pubsub/1.0.0` protocol id with [SWIP-60][swip60] while **not** being -wire-compatible with it. That is a deliberate choice, and it puts the whole weight of -the compatibility story on the first frame of a stream: - -- A SWIP-60 `Open` sets `CohortSpec` fields 3–8 and `Open` field 2, all of which - BPS-lite reserves, so it is `REJECTED` rather than silently decoded with those fields - dropped — which is what proto3's permissive unmarshalling would otherwise do, leaving - a peer that asked for a five-author closed cohort holding an open single-publisher - one. The reserved numbers are what make the refusal reliable, and they are the reason - a BPS-lite broker MUST reject a frame carrying them rather than ignore it. -- A BPS-lite `Open` is a valid SWIP-60 `Open` — `{topic, binding}` with everything else - unset. A SWIP-60 broker would accept it, and then not enforce the address constraint - (see the next bullet). A BPS-lite publisher therefore gets weaker guarantees from a - full-spec broker than from a conformant one, silently. Applications that depend on - the `ANCHOR` rule should treat an unexpected `Ack` shape as a mismatch. -- BPS-lite's `ANCHOR` applies its address constraint unconditionally; SWIP-60 relaxes - it under an explicit publisher regime. An implementation reusing SWIP-60 `ANCHOR` - code unmodified would perform no address check at all under BPS-lite, which is the - one silent failure this divergence can cause. It is called out again in *Differences - from SWIP-60* above. - -Should BPS-lite and SWIP-60 later converge, the reserved field numbers leave the -migration path open: a future revision can un-reserve them with their SWIP-60 meanings -intact. - -## Test cases - -An implementation is BPS-lite conformant if it exhibits all of: - -1. `Open` with `binding != ANCHOR` → `REJECTED`. -2. `Open` carrying a reserved field, or a `CohortSpec` carrying one → `REJECTED`. -3. `Subscribe` carrying reserved field 2 → `REJECTED`. -4. `Subscribe` before any `Open` → `UNKNOWN_TOPIC`. -5. Re-`Open` on a live topic with an identical spec → `OK`; the prior publisher stream - is reset; the cohort and its dedup window survive. A differing spec → `REJECTED`. -6. A superseded publisher stream closing after a role transfer → cohort unaffected. -7. Current publisher stream closes → every subscriber stream reset, topic subsequently - `UNKNOWN_TOPIC`. -8. `Publish` whose SOC address is not `spec.topic` → dropped and counted. -9. `Publish` on a subscriber stream → dropped, counted, and that stream reset. -10. A duplicate CAC within the dedup window → dropped, counted separately. -11. Subscriber arriving at per-topic capacity → `FULL`, with no referral frame. `Open` - at the live-cohort bound → `FULL`. -12. Publisher stream outbound queue full → frame dropped, stream retained, cohort alive. -13. `Open` without `loopback` → the publisher receives no `Broadcast` frames. - -A broker MUST expose per-cohort counters for the silent-drop outcomes — -`invalid_address`, `invalid_signature`, `invalid_cac`, `wrong_stream`, `duplicate`, -`queue_dropped`. Points 8–10 and 12 are unobservable from the wire without them, so the -counters are part of the conformance surface rather than an implementation detail. - -## Implementation - -Groundwork exists in the bee prototype `pkg/bps` ([bee #5435][bee5435]), written against -SWIP-60. It is a superset of this spec in most places and diverges in one: - -`pkg/bps` is a superset of this spec in some places and diverges in one: it implements -`ANCHOR` and `FEED_TOPIC` bindings, `EXPLICIT_SINGLE` and `EXPLICIT_LIST` regimes, the -`Hello` envelope and a WS bridge. Most of the work is subtraction: - -| file | under BPS-lite | -|---|---| -| `publisher.go` | `authorizePublisher` goes entirely — there is no publisher to authorize against, only the topic | -| `binding.go` | `anchorBinding` only; the registry stays as the extension seam with one entry. `qualifies` **loses its explicit-regime early return**: the address check becomes unconditional | -| `cohort.go` | `Publishers`, `sortedCopy`, the regime switch in `ValidateSpec` and the publisher-list branch of `SpecEqual` go; `ValidateSpec` checks two fields | -| `broker.go` | `admit` sets the publisher role from the `Open` path rather than from `auth != nil`; the `PublisherAuth`-is-not-a-credential commentary goes with it | -| `broker.go` | gains a `publisher *peerStream` on the cohort: teardown fires only when the *current* publisher stream closes, and re-`Open` transfers it | -| `broker.go` | gains the queue-reset exemption for the publisher's stream | - -Two behaviors are new — the lifetime rule and role transfer. Everything else is removal. - -[swip60]: https://github.com/ethersphere/SWIPs/pull/104 -[bee5435]: https://github.com/ethersphere/bee/pull/5435 - -## References - -Parent design space: [SWIP-60 "BPS singlehop — brokered broadcast pub/sub, base -protocol", PR #104][swip60] · origin: -[PR #93](https://github.com/ethersphere/SWIPs/pull/93) "Add: pubsub" · implementation: -bee [#5435][bee5435] - -## Copyright - -Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From f42b99a904b812bd0129fbc1eb6d55c3fa6b6a44 Mon Sep 17 00:00:00 2001 From: zelig Date: Tue, 22 Sep 2026 16:18:39 +0200 Subject: [PATCH 03/13] =?UTF-8?q?swip-74:=20resolve=20the=20last=20open=20?= =?UTF-8?q?point=20=E2=80=94=20concurrent=20admin=20streams=20are=20both?= =?UTF-8?q?=20publisher=20streams?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A second stream authenticating as the admin while one is live is admitted as a publisher stream too; the per-cohort cursor arbitrates. No supersede rule, no refusal. No (?) marks remain. Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 83ac8eee..9e2cb8f2 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -13,7 +13,7 @@ created: 2026-09-14 PR #111 after review: no history, no bandwidth incentive, no service messages, singlehop only, one mode only — an explicit single publisher (live video streaming). SWIP-60 (PR #104) is the full singlehop protocol and is to be amended to extend this wire rather -than the other way round; see "Relation to SWIP-60". Open points are marked (?). --> +than the other way round; see "Relation to SWIP-60". --> - **Business line**: a live stream on Swarm — one author, an audience, real time, no storage round trip and no polling: video/audio streaming, a price ticker, a game's @@ -198,12 +198,10 @@ Then the stream's role, from `Auth`: **The identity is bound to the stream, not to the peer connection.** One stream per (peer, cohort); a node may present different identities on different streams, and a -stream without `Auth` has none. **(?)** What if a second stream authenticates as the -admin while a first one is live — the admin from two nodes, or reconnecting before its -old stream is torn down? This draft admits both as publisher streams and lets the -per-cohort cursor arbitrate (neither can publish an index the other already has); the -alternatives are that the new stream supersedes the old (reset), or that the second is -`REJECTED`. +stream without `Auth` has none. A second stream MAY authenticate as the admin while a +first one is live — the admin from two nodes, or reconnecting before its old stream is +torn down: both are publisher streams, and the per-cohort cursor arbitrates, since +neither can publish an index the other already has. No supersede rule, no refusal. The broker answers `Ack{status}`: `OK`, `FULL` at a capacity bound, `REJECTED` for a spec outside this SWIP or a reserved field set. A non-`OK` `Ack` ends the stream; on `OK` the From b68092fdaedd11ae3f1c43afb697319a4a014f4c Mon Sep 17 00:00:00 2001 From: zelig Date: Thu, 24 Sep 2026 16:24:16 +0200 Subject: [PATCH 04/13] swip-74 rev 3: the publisher role is claimed by signing a broker-derived challenge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Join{CohortSpec, addr, claim?}: a peer that means to publish declares the address it will publish as; a returning publisher claims in the Join - Ack{status, challenge}: S = H(S_C || S_c || addr), S_C a boot secret never persisted, S_c = H(S_C || H(Marshal(spec))) — nothing stored, recomputed at claim time, the same S for an address on a cohort from any node - Claim{addr, index, auth}: auth = {r, s, v} over H("bps-claim:v1" || S || O_B || index); O_B the broker's overlay (against challenge forwarding), index the publisher's cursor, signed; no reply - Message{address, data}: the whole chunk, so the ordinary SOC validation is non-vacuous; no envelope — what a frame is follows from the stream's role - cursor restated as the lowest index accepted next, initially 0; a claim sets it to max(cursor, index) - one extra stream over the fan-out bound while the admin is absent, with a claim deadline - security: what the claim protects (identity, not history), the case-by-case replay table, the transport precondition made normative - no reserved numbers; unknown fields ignored; conformance and counters updated Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 515 ++++++++++++++++++++++++++++------------------- 1 file changed, 306 insertions(+), 209 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 9e2cb8f2..7f2937f4 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -11,21 +11,24 @@ created: 2026-09-14 +only, one mode only — an explicit single publisher (live video streaming). Rev 3: the +publisher role is claimed by signing a broker-issued challenge (the first draft had a +challenge; rev 2 wrongly replaced it with a static signature). SWIP-60 (PR #104) is the +full singlehop protocol and extends this wire without changing it; see "Relation to +SWIP-60". Open points are marked (?). --> - **Business line**: a live stream on Swarm — one author, an audience, real time, no storage round trip and no polling: video/audio streaming, a price ticker, a game's server-authoritative state, a log tail. The stream is a feed, so the same updates can later be persisted and replayed from storage by anyone who missed the live run. -- **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the three frames - below — one broker, direct streams, one authenticated publisher, read-only - subscribers — and nothing else: **no history, no bandwidth incentive, no service - messages, no Bee API, one hop, one mode.** Done when a broker, a publisher and - subscribers from independent implementations interoperate per the conformance - section. Everything the family adds — service messages and the Bee API, multiple - publishers, self-indexed feeds, multihop — extends this wire without changing it. +- **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the four frames + below — one broker, direct streams, one publisher that proves its key by signing a + challenge, read-only subscribers — and nothing else: **no history, no bandwidth + incentive, no service messages, no Bee API, one hop, one mode.** Done when a broker, a + publisher and subscribers from independent implementations interoperate per the + conformance section. Everything the family adds — service messages and the Bee API, + multiple publishers, self-indexed feeds, multihop — extends this wire without changing + it. - **DISC change**: NO. A new p2p protocol surface; no storage, retrieval or incentive change. @@ -36,12 +39,13 @@ that anyone can subscribe to and one participant can publish on. Each channel de its own cohort — a set of nodes connected through a unique central broadcaster node, the **broker**; every node in the cohort is directly connected to the broker, hence *singlehop*. A channel is identified by its spec — a feed topic and the address of its -**admin**. Anyone joins by sending that spec; the admin also sends a signature that binds -its identity to the stream, and has the exclusive right to publish. Its messages are -single-owner chunks that are the higher-index updates of the feed the topic names, -carried with their bare index; the broker accepts each one iff its index exceeds the -channel's cursor, it validates as a single-owner chunk under the feed's id and its owner -is the admin, and delivers it unchanged to every subscriber stream — never to a +**admin**. Anyone joins by sending that spec, and a peer that means to publish also names +the address it will publish as; the broker answers with a **challenge** it derives from a +secret it never stores, and the publisher claims its stream by signing it. Its messages +are single-owner chunks that are the higher-index updates of the feed the topic names, +carried with their bare index; the broker accepts each one iff its index is at least the +channel's cursor, the chunk validates as a single-owner chunk under the feed's id, and its +owner is the admin, and delivers it unchanged to every subscriber stream — never to a publisher's. All other joiners are subscribers and verify the same way, end to end: the broker can withhold, never forge. A channel ends when it goes idle. @@ -52,7 +56,7 @@ on a service feed, a history flag and the Bee API. That is the right target for reference implementation and too much for a second, independent one whose application is one author broadcasting to an audience. BPS-lite names that one configuration and specifies only what it needs, so that a second team can implement and conformance-test -it from three frames — and so that it stays the **base** of the family: nothing here is +it from four frames — and so that it stays the **base** of the family: nothing here is a divergence to reconcile later, because the fuller protocol extends these frames. One publisher over a feed also removes what the fuller protocol has to carry: a dedup @@ -63,7 +67,8 @@ who may write (the admin, proved by its signature on every message). ## Specification The key words MUST, MUST NOT, REQUIRED, SHOULD, RECOMMENDED and MAY are to be interpreted -as described in RFC 2119. Terms — cohort (a channel's set of nodes), broker, admin, subscriber — are SWIP-60's. +as described in RFC 2119. Terms — cohort (a channel's set of nodes), broker, admin, +subscriber — are SWIP-60's. ### The contract @@ -73,14 +78,14 @@ subscribers**; they are **feed updates, in increasing index order**. ### Topology and roles - **Broker**: one full node per cohort. Every peer holds one direct libp2p stream per - (peer, cohort) to it, protocol id `pubsub/1.0.0`. Depth is 1 by construction: no - relaying, no referral. Broker discovery is out of scope; deployments configure the + (peer, cohort, identity) to it, protocol id `pubsub/1.0.0`. Depth is 1 by construction: + no relaying, no referral. Broker discovery is out of scope; deployments configure the broker. - **Admin = the publisher**: the address named in the cohort spec. It is the only - identity whose messages are accepted. It need not be first to join, and the cohort - does not end when its stream goes away. -- **Subscriber**: joins by naming the cohort, receives, never sends. A message from a - subscriber stream is a protocol violation. + identity whose messages are accepted. It need not be first to join; the cohort does + not end when its stream goes away; and it may publish from any node, or from two. +- **Subscriber**: joins by naming the cohort, receives, never publishes. A publication + from a subscriber stream is a protocol violation. - **Publisher and broker are distinct nodes**; brokering for oneself is out of scope. ### The cohort spec @@ -93,20 +98,22 @@ Three fields, no others: | `binding` | `FEED_TOPIC`: the signed id of every message is `keccak256(topic ‖ index)` | | `admin` | 20-byte address: the publisher | -**The spec is the cohort's identity.** Two peers naming byte-identical specs at a broker -are in the same cohort; a spec that differs in any field names a different one. The -invite for a stream is therefore the spec plus the broker: `topic`, `admin`, broker -address. No publisher regime, no roster, no spectator flag, no history flag, no -proximity constant: a BPS-lite broker answers any spec outside this table, and any -reserved field set, with `REJECTED`. Capacity is not in the spec: a cohort cannot -dictate a remote node's connection count. +**The spec is the cohort's identity.** It has a **canonical serialisation** — +`Marshal(spec)`: fields in number order, unset fields not emitted — and cohorts are keyed +by it: two peers naming byte-identical specs at a broker are in the same cohort; a spec +that differs in any field names a different one. The invite for a stream is therefore the +spec plus the broker: `topic`, `admin`, broker address. No publisher regime, no roster, no +audience flag, no history flag, no proximity constant: a BPS-lite broker answers any +value outside this table with `REJECTED`, and ignores fields it does not define, as +proto3 does. Capacity is not in the spec: a cohort cannot dictate a remote node's +connection count. ### Wire format -Normative and complete. Field and enum numbers are SWIP-60's where a counterpart exists, -so that the fuller protocol extends these messages without renumbering; numbers it uses -and this SWIP does not are reserved, never reused. An unset enum (`*_UNSPECIFIED`, the -proto3 zero) is not a legitimate wire value: receivers MUST reject a message carrying one. +Normative and complete. Field and enum numbers are shared with SWIP-60, which extends +these messages by adding fields, values and nothing else. An unset enum +(`*_UNSPECIFIED`, the proto3 zero) is not a legitimate wire value: receivers MUST reject +a message carrying one. ```proto syntax = "proto3"; @@ -115,72 +122,81 @@ package bps; // What the topic binds to. One binding. enum TopicBinding { TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire - FEED_TOPIC = 4; // signed id = keccak256(topic || index). SWIP-60's - // number; 1, 2, 3, 5 are SWIP-60's other bindings + FEED_TOPIC = 4; // signed id = keccak256(topic || index) } -// The cohort's identity. Fixed by whoever joins first; immutable. +// The cohort's identity. Fixed by whoever joins first; immutable; keyed by its +// canonical serialisation. message CohortSpec { bytes topic = 1; // 32 bytes TopicBinding binding = 2; // FEED_TOPIC - bytes admin = 5; // 20 bytes, required - reserved 3, 4, 6, 7, 8, 9; // SWIP-60: publishers, history, (unused), (unused), - // (unused), spectators — none apply here + bytes admin = 3; // 20 bytes, required } -// Proof of an identity, bound to the stream it arrives on: -// owner = ecrecover( keccak256("bps-join:v1" || topic || admin), signature ) -// The preimage is static and carries no node identity (SWIP-60): the same key works -// from any node, and a join never links an address to a peer id. It is therefore -// replayable, and a replayed identity is worthless: authorship rests on the message -// signature, never on the handshake. +// A secp256k1 signature, as a SOC's: r || s || v. message Auth { - bytes signature = 1; // 65 bytes - reserved 2; // SWIP-60: id, for bindings that do not fix it + bytes r = 1; // 32 bytes + bytes s = 2; // 32 bytes + uint32 v = 3; // 27 or 28 } -// Peer -> broker: the first and only handshake frame on a fresh stream. Creates the -// cohort if no live cohort has this spec, attaches to it otherwise. `auth` binds an -// identity to THIS stream; absent, the stream has none and is read-only. +// A publisher's claim on the stream it is sent on. `auth` signs +// keccak256("bps-claim:v1" || S || O_B || index) +// with the key of `addr`: S the challenge the broker issued for `addr` on this +// cohort, O_B the overlay of the broker the claiming node is connected to, `index` +// eight bytes big-endian. Sent inside Join by a peer that already holds S, or as the +// next frame after Ack by one that has just received it. +message Claim { + bytes addr = 1; // 20 bytes: the address claimed; equals Join.addr + uint64 index = 2; // the publisher's cursor: its next message has an index >= this (?) + Auth auth = 3; +} + +// Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live +// cohort has this spec, attaches to it otherwise. message Join { CohortSpec cohort = 1; - Auth auth = 2; + bytes addr = 2; // 20 bytes: the address this stream will publish as; absent: + // a subscriber, and no challenge is issued + Claim claim = 3; // a returning publisher's claim, verified before any bound } enum Status { STATUS_UNSPECIFIED = 0; // invalid on the wire OK = 1; FULL = 2; // a capacity bound (see Resource bounds) - REJECTED = 4; // spec outside this SWIP, or a reserved field set - reserved 3; // SWIP-60: UNKNOWN_TOPIC — cannot occur: Join creates + REJECTED = 3; // a spec value outside this SWIP } -// Broker -> peer, answering Join. Status only. A non-OK Ack ends the stream. +// Broker -> peer, answering Join. A non-OK Ack ends the stream. message Ack { - Status status = 1; - reserved 2, 3, 4, 5; // SWIP-60: spec echo and service SOCs — nothing to echo + Status status = 1; + bytes challenge = 2; // S, iff status == OK and addr was declared } -// Both directions after the handshake: admin -> broker is a publication, broker -> -// subscriber a delivery of the same bytes. The single-owner chunk travels as its -// stored chunk data, opaque to the protocol and validated by the ordinary SOC code -// once the `id` slot has been rewritten as the Frames section says: -// id (32) || signature (65) || span (8, LE) || payload (<= 4096) +// Both directions after the handshake: publisher -> broker is a publication, broker +// -> subscriber a delivery of the same bytes. The single-owner chunk travels whole, +// address and data, opaque to the protocol and validated by the ordinary SOC code +// once the id slot has been rewritten as the Frames section says: +// data = id (32) || signature (65) || span (8, LE) || payload (<= 4096) message Message { - bytes soc = 1; + bytes address = 1; // 32 bytes: the SOC address, keccak256(id || owner) + bytes data = 2; } ``` -Three frames — `Join`, `Ack`, `Message` — and the two types they carry, `CohortSpec` and -`Auth`. There is no envelope around `Join` — it is the only first frame there is — and no -separate publish and broadcast types: a broker pushes to subscribers, a non-broker does -not. Fields this SWIP does not define are ignored on receipt, as proto3 does; a `Message` -whose `soc` is empty is invalid. +Four frames — `Join`, `Ack`, `Claim`, `Message` — and the two types they carry, +`CohortSpec` and `Auth`. There is no envelope: **what a frame is follows from the +stream's direction and role.** The first peer-to-broker frame is a `Join` and the first +broker-to-peer frame an `Ack`; after that a broker sends only `Message`, a subscriber +stream sends only a `Claim`, and a publisher stream sends only `Message`. A frame that +does not parse as what its stream may send is invalid. -### Handshake: `Join` +### Handshake: `Join`, the challenge, the claim Every peer sends `Join` as the first frame on a fresh stream, carrying the full -`CohortSpec` and, if it claims an identity, an `Auth`. +`CohortSpec`; a peer that means to publish also declares the address it will publish as, +in `addr` — an assertion, proved only by the signature that follows. The broker compares the spec against its live cohorts: **no match → the cohort is created** with the joiner attached; **match → the joiner is attached** to it. Anyone may @@ -188,78 +204,119 @@ create, including a subscriber arriving before the admin: a publisher-less cohor the broker one map entry until the inactivity deadline reclaims it, and can accept no message. There is nothing to pre-register and no "unknown topic". -Then the stream's role, from `Auth`: +**The challenge.** For a `Join` that declares `addr`, the broker derives -| `Auth` | the stream is | -|---|---| -| absent | a **subscriber stream**, no identity | -| recovers to `spec.admin` | a **publisher stream** | -| recovers to any other address | a **subscriber stream** carrying that identity — no effect in this SWIP | - -**The identity is bound to the stream, not to the peer connection.** One stream per -(peer, cohort); a node may present different identities on different streams, and a -stream without `Auth` has none. A second stream MAY authenticate as the admin while a -first one is live — the admin from two nodes, or reconnecting before its old stream is -torn down: both are publisher streams, and the per-cohort cursor arbitrates, since -neither can publish an index the other already has. No supersede rule, no refusal. - -The broker answers `Ack{status}`: `OK`, `FULL` at a capacity bound, `REJECTED` for a spec -outside this SWIP or a reserved field set. A non-`OK` `Ack` ends the stream; on `OK` the -stream is retained. A peer whose stream was refused or reset MUST back off before -rejoining (randomised exponential, 1 s to 30 s, full jitter); a broker MAY reset a stream -that retries faster. Without this every reclaim or reset of a cohort brings its whole -audience back at once. +``` +S_C = a secret drawn once at broker boot, never persisted +S_s = keccak256(Marshal(spec)) the cohort's key +S_c = keccak256(S_C ‖ S_s) the cohort's secret +S = keccak256(S_C ‖ S_c ‖ addr) the challenge for addr on this cohort +``` + +and answers `Ack{OK, S}`. The broker stores nothing: it recomputes `S` from its boot +secret, the spec and the address whenever a claim arrives. `S` is therefore the same for +an address on a cohort for as long as the broker runs, from whichever node the address +joins, and differs at every other broker and after every restart. It goes to the joiner +over the encrypted stream and is useful only to the key of `addr`; anyone may obtain it +by declaring the address, and gains nothing by it. + +**The claim.** A publisher claims its stream by signing, with the key of `addr`, + +``` +keccak256("bps-claim:v1" ‖ S ‖ O_B ‖ index) +``` + +where `O_B` is the overlay of the broker the claiming node is connected to and `index` +is the publisher's cursor: the claim that its next message will have a feed index of at +least `index`. The domain separator keeps the signature disjoint from SOC signatures and +any other protocol's; `S` binds it to this broker, this cohort and this address; `O_B` +binds it to the verifier, which `S` cannot, because `S` is opaque to the signer (see +Security considerations); `index` is signed so that a replayed claim moves no cursor. +The claim is sent in one of two places: + +- **in `Join`**, by a peer that already holds `S` for this address at this broker — a + reconnecting admin, from the same node or another: the broker verifies it before any + bound is applied, and a valid claim makes the stream a publisher stream from its first + frame. A stale claim — the broker has restarted, `S` has changed — is treated as + absent: `Ack{OK, S}` with the new `S`, no penalty; +- **as the next frame after `Ack`**, a `Claim`, by a peer that has just received `S`. + +The broker recovers the signer and checks that it equals `addr` (ecrecover never fails, +it returns *some* address, which is why the address is declared and compared) and that +`addr` is the cohort's admin — the one address that may publish here. Then the stream +**upgrades** to a **publisher stream**: it leaves the fan-out set, and the cohort's cursor +becomes `max(cursor, index)`. There is **no reply**: a publisher sends its claim and its +first publication back to back, and stream ordering guarantees the broker handles the +claim first; a claim that did not upgrade makes the publication that follows a violation, +and the reset is the answer. Anything else — a signature that does not recover to `addr`, +an `addr` that is not the admin, a second claim on a stream — is a protocol violation: +dropped, counted (`invalid_claim`), the stream reset, the peer blocklisted per the node's +policy. Several streams MAY be claimed for the admin at once — the admin from two nodes, +or reconnecting before its old stream is torn down: each is a publisher stream, and the +cursor arbitrates. + +**The rest of the handshake.** `Ack{OK}` without a challenge answers a `Join` that +declared no address: the stream is a **subscriber stream**. `FULL` answers a `Join` at a +capacity bound, `REJECTED` one whose spec has a value outside this SWIP. A non-`OK` `Ack` +ends the stream; on `OK` it is retained. A peer whose stream was refused or reset MUST +back off before rejoining (randomised exponential, 1 s to 30 s, full jitter); a broker +MAY reset a stream that retries faster. Without this every reclaim or reset of a cohort +brings its whole audience back at once. ### Frames -After the handshake there is one frame, `Message`, and it is direction-typed: on a -publisher stream towards the broker it is a publication; from the broker on a subscriber -stream it is a delivery. A delivery is the accepted publication's bytes, unchanged. +After the handshake a broker sends `Message` and nothing else, to subscriber streams; a +publisher stream sends `Message` and nothing else; a subscriber stream sends at most one +`Claim`. A delivery is the accepted publication's bytes, unchanged. + +**The frame is a whole chunk.** `address` is the SOC address and `data` the chunk data, +so that the ordinary SOC validation applies: without the address it would be vacuous, +since recovering a signer always yields *an* address, and with it a bad signature fails +because the recovered owner no longer hashes to the address. -**The `id` field carries the bare index.** A feed update's signed id is +**The `id` slot carries the bare index.** A feed update's signed id is `keccak256(topic ‖ index)`, `index` a uint64 big-endian in 8 bytes. On the wire the -32-byte `id` slot of the chunk data holds that index left-padded with 24 zero bytes, and -the receiver reconstructs the signed id from the cohort's topic (the carriage of +32-byte `id` slot of `data` holds that index left-padded with 24 zero bytes, and the +receiver reconstructs the signed id from the cohort's topic (the carriage of [SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). A `Message` whose `id` slot does not have 24 leading zero bytes is not a feed update: a broker drops it as invalid; a subscriber drops it without counting it as a violation. ### Validation: the feed cursor -The broker keeps one **cursor** per cohort — the highest index accepted so far, initially -none. A `Message` arriving at the broker is accepted iff, in order: +The broker keeps one **cursor** per cohort: **the lowest index it will accept next**, +initially 0 — index 0 is the first update of every feed. A `Message` arriving at the +broker is accepted iff, in order: -1. it arrived on a **publisher stream** — on any other stream it is a protocol - violation: the frame is dropped, the stream reset and the peer blocklisted per the - node's policy; -2. its `id` slot is a bare index `n` and **`n > cursor`**; -3. with the id substituted by `keccak256(topic ‖ n)` the chunk **validates as a - single-owner chunk**: the wrapped chunk's BMT address matches `span ‖ payload`, and - the signature over `id ‖ wrappedAddress` recovers an owner; -4. the recovered owner is **`spec.admin`**. +1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as a + `Claim`, and if it is not a valid one it is a protocol violation: dropped, the stream + reset, the peer blocklisted per the node's policy; +2. its `id` slot is a bare index `n` and **`n ≥ cursor`**; +3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as a + single-owner chunk**: the wrapped chunk's BMT address matches `span ‖ payload`, the + signature over `id ‖ wrappedAddress` recovers an owner, and `keccak256(id ‖ owner)` + equals `address`; +4. `address` equals `keccak256(id ‖ admin)` — the owner is the admin. -On acceptance the broker sets `cursor := n` and enqueues the frame on **every subscriber -stream** in the cohort. Publisher streams receive nothing: a publisher never gets its own -messages back, on whichever of its streams they were sent. +On acceptance the broker sets `cursor := n + 1` and enqueues the frame on **every +subscriber stream** in the cohort. Publisher streams receive nothing: a publisher never +gets its own messages back, on whichever of its streams they were sent. Failures of 3 and 4 are invalid frames: dropped and counted; repeated invalid frames end -the connection (blocklisting policy). A frame failing 2 with `n ≤ cursor` is the one +the connection (blocklisting policy). A frame failing 2 with `n < cursor` is the one benign failure — a retransmit, which an admin reconnecting after a reset legitimately sends when it does not know what the broker last accepted — and is counted separately; a broker MAY reset a publisher stream whose retransmit rate exceeds its policy, since the cursor check is cheap and precedes the signature check. A frame failing 2 because the `id` slot is not a bare index is invalid. -The cursor is initially **absent, not zero**: index 0 is the first update of every feed -and MUST be accepted on a fresh cohort. - -**There is no dedup window.** The cursor is total: a message is either beyond the last -accepted index or it is not, and there is no eviction and no replay hole. **Gaps are -allowed** at the broker (`n > cursor + 1`): they are the publisher's business, and -SWIP-65 makes them detectable and recoverable at the subscriber. This is stricter than -SWIP-65's general carriage rule, under which a broker does not enforce monotonicity; the -restriction is sound here because one publisher on one hop admits no legitimate -reordering. +**There is no dedup window.** The cursor is total: a message is either at or beyond the +next expected index or it is not, and there is no eviction and no edge. **Gaps are +allowed** (`n > cursor`): they are the publisher's business, and SWIP-65 makes them +detectable and recoverable at the subscriber. A claim's `index` moves the cursor forward +to what the admin has published elsewhere, never back. This is stricter than SWIP-65's +general carriage rule, under which a broker does not enforce monotonicity; the restriction +is sound here because one publisher on one hop admits no legitimate reordering. Subscribers re-verify every delivery exactly as the broker does (steps 2–4, against the spec they joined with and their own cursor), end to end, whatever the broker did. @@ -274,12 +331,14 @@ BPS-lite — that is a service message, and this SWIP has none; an application t service feed the family adds. A publisher stream going away — closed or reset — is therefore not an end: the cohort -and its cursor persist, subscribers stay attached, and the admin rejoins with `Auth` and -resumes at `n > cursor`. Likewise a subscriber-only cohort waits for its admin until the -deadline reclaims it. A cohort with **no attached streams** MAY be reclaimed at once — -nothing observes the difference beyond a fresh cursor on the next `Join`. - -After a cohort is forgotten a later `Join` creates a fresh one with an empty cursor. +and its cursor persist, subscribers stay attached, and the admin rejoins, claiming in its +`Join` with the `S` it holds and the cursor it knows. Likewise a subscriber-only cohort +waits for its admin until the deadline reclaims it. A cohort with **no attached streams** +MAY be reclaimed at once — nothing observes the difference beyond a fresh cursor on the +next `Join`. + +After a cohort is forgotten a later `Join` creates a fresh one with the cursor at 0 — +and the same `S`, since the broker derives it, so the admin's claim still holds. Subscribers keep their own cursor across that, so a stale republication is caught at the edge, not at the broker. @@ -290,7 +349,7 @@ given RECOMMENDED where a value is given; the last two are MAY: | bound | answer | recommended | |---|---|---| -| subscriber streams per cohort | `FULL` to the next `Join` without an admin `Auth` — the audience cannot lock the admin out of its own cohort | implementation-defined | +| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except **one extra stream while the admin is absent**: a `Join` declaring the admin's `addr` when no publisher stream exists is admitted over the bound, and disconnected, with a short blocklist, if it has not claimed within the **claim deadline** | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | | live cohorts per broker | `FULL` to a cohort-creating `Join` | implementation-defined | | **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | | **inactivity deadline** — reclaims a cohort, see *Lifetime* | reset | 10 min | @@ -300,9 +359,12 @@ given RECOMMENDED where a value is given; the last two are MAY: Any peer can make a broker allocate a cohort simply by joining, which is why the second, third and fourth bounds exist together: a cohort costs a map entry, a peer can hold only -a bounded number of them, and none survives idleness. The broker has no delivery -obligation: a reset for a full queue is a recoverable liveness fault, and blocking -fan-out on one slow subscriber would punish the cohort. +a bounded number of them, and none survives idleness. The extra stream is what keeps an +audience that fills the cohort while the admin is away from locking the admin out: an +unclaimed admin is a fan-out channel like any other, and its slot exists exactly while it +is absent; a returning admin that claims in its `Join` never needs it. The broker has no +delivery obligation: a reset for a full queue is a recoverable liveness fault, and +blocking fan-out on one slow subscriber would punish the cohort. ### Relation to SWIP-60 @@ -311,38 +373,22 @@ The relation is subset, and it is kept in one direction only — **later revisio this wire, they never change it**: - a BPS-lite peer sends only frames the full protocol accepts, and a BPS-lite broker - accepts only what this SWIP defines; everything the full protocol adds to the - handshake arrives in fields this SWIP reserves, so a full-protocol `Join` at a - BPS-lite broker is `REJECTED` at the handshake rather than silently misread, and - everything it adds to `Message` arrives in fields this SWIP does not define and - ignores; + accepts only what this SWIP defines: a spec value outside this SWIP is `REJECTED`, a + field this SWIP does not define is ignored, so a fuller spec is served as the lite spec + its known fields spell — a feed-topic cohort whose admin later publishes a roster is, + at a BPS-lite broker, a live stream whose roster and grantees' updates are dropped as + invalid; an admin that wants a roster needs a full broker; - a BPS-lite publisher and subscriber at a full broker are conformant peers of a - single-publisher live-stream cohort; the full broker's additions (service messages, - further `Ack` fields) reach them as frames they drop and fields they ignore; + single-publisher live-stream cohort; the full broker's additions (service messages) + reach them as frames they drop; - validation here is stricter, never looser: the cursor refuses out-of-order - retransmits a full broker may pass through; no frame a BPS-lite broker accepts is one - a full broker refuses. - -SWIP-60 is to be amended to match — its author's to-do, listed here so that the subset -claim is checkable: - -- `Hello`/`Open`/`Subscribe` collapse into `Join` carrying the spec, and **cohorts are - keyed by the whole spec, not by the topic**: a `Join` whose topic is live under a - different spec creates a second cohort rather than being `REJECTED`, so a squatter - who pre-creates `(topic, wrong admin)` obtains nothing; -- `GENESIS` goes (with the spec in `Join` it has nothing left to prove) and `Ack` echoes - nothing; -- the publisher regime shrinks to a single value, `ALL` — anyone attached may publish; - absent, the admin publishes and whoever its roster ever names, so a cohort is - multi-publisher iff its admin publishes a roster, and nobody needs to know in advance; -- **`spectators` (field 9) goes, or reverts to a `closed` flag whose unset value means an - open audience** — as a proto3 bool whose unset value is *false*, it reads a lite - spec, which never sets it, as a closed cohort, and a full broker would refuse every - lite subscriber; -- the chunk travels as opaque chunk data. - -And one line in SWIP-65: its carriage rule, under which a broker does not enforce index -monotonicity, gets the exception this SWIP's cursor is. + retransmits; no frame a BPS-lite broker accepts is one a full broker refuses. + +What SWIP-60 adds on this wire: cohort parameters (`publishers`, `history`, `closed`), +the admin's service feed (roster, end of stream) as `Message` frames, a claim by any +rostered address and none at all under `ALL`, one extra stream per absent rostered +publisher, and the Bee API. What it presumes of the transport, this SWIP states below as +a precondition. ## Rationale @@ -359,25 +405,49 @@ live. topic and learns the spec from the broker has to be told the spec by someone it then has to verify. For a live stream the spec is a topic and an address; the invite that names the broker names them too. Once every joiner carries the spec, the spec is the -cohort's identity, the broker is never trusted about it, `Ack` is a status, the -first-joiner-creates rule is the natural one — and a second handshake type has nothing -to do. - -**Why the identity is on the stream.** `Auth` is in `Join`, and what it proves is bound -to the stream it arrives on. A peer connection is a node; a stream is a (node, cohort) -pair; the identity that matters is the key that signs the messages on that stream. Binding -it there is what lets one node carry different identities on different cohorts, and what -later lets a subscriber stream be promoted to a publisher stream in place, without a -rejoin, when the admin grants it. +cohort's identity, the broker is never trusted about it, `Ack` is a status and a +challenge, and the first-joiner-creates rule is the natural one. + +**Why a challenge, and what it protects.** A static signature — over the topic and the +admin, say — would be replayable by design, and the argument that a replayed role can +publish only what its owner already signed is true and beside the point: replaying the +admin's signed updates is *history*, and a first-time viewer receiving them from the +beginning is caught up, not deceived. What a replayed signature buys is an **identity**: +a stream the broker treats as the admin's — exempt from the fan-out bound and its queue +policy, entitled to the extra slot, and in SWIP-60 admitted to a `closed` cohort and +promoted by a roster. A challenge only this broker could have issued, for this address, +signed together with the verifier's overlay, is what makes the identity worth exactly +the key. Deriving it from a boot secret and the spec is what lets the broker issue it +without storing it, and lets an address keep its claim from any node and across +reclaims. + +**Why the identity is on the stream, and the address is declared.** A peer connection is +a node; a stream is a (node, cohort) pair; the identity that matters is the key that +signs the messages on that stream. Declaring the address in `Join` lets the broker derive +the challenge for it, and gives the extra slot only to a stream that says it is the +admin's. Binding the identity to the stream is what lets one node carry different +identities on different streams — and, in SWIP-60, what lets a subscriber stream be +upgraded by a claim when the admin grants it. + +**Why the frame is a whole chunk.** The ordinary SOC validation checks that the signer +hashes to the address. Without the address on the frame there is nothing to hash to, and +a broker that only recovered a signer would accept any signature at all. + +**Why no reply to a claim.** After the handshake each direction carries one frame type, +and the wire has no envelope. A reply would be a second broker-to-peer type on a stream +that carries deliveries, with nothing to tell them apart. It is also unnecessary: streams +are ordered, so a publisher sends its claim and its first publication together and learns +the outcome from whether the stream survives. **Why no loopback.** A publisher knows what it published. Echoing it costs a frame per message on the stream whose back-pressure matters most, and buys no confirmation: a rejected frame produces no echo either. -**Why the cursor, not a window.** A bounded seen-set is a memory bound with a replay -hole at its edge, and it exists because the general protocol admits many publishers and +**Why the cursor, not a window.** A bounded seen-set is a memory bound with a replay hole +at its edge, and it exists because the general protocol admits many publishers and duplicate paths. One publisher on one hop has neither. The feed index is a total order, -and "greater than the last" is both the dedup rule and the memory bound. +and "at least the next" is both the dedup rule and the memory bound — and the claim can +declare it, because a signed lower bound is exactly what a reconnecting publisher knows. **Why inactivity, and only inactivity.** An end that is *signed* by the admin is a service message, which this SWIP does not have; an end that is *inferred* from the admin's stream @@ -391,17 +461,33 @@ service feed arrives. Those of SWIP-60, restricted to its live-stream configuration (admin set, the admin alone publishes, open audience). -**The publisher role is proved, not asserted**: `Auth` recovers to `admin`, and it grants -nothing the admin's key has not already granted. Its preimage is static, so it is -replayable: a peer that captured the admin's `Join` can obtain a publisher stream, but -can send on it only what the admin already signed. Within one cohort's life the cursor -refuses that (every captured index is at or below it); **across a reclaim or at another -broker the cursor starts empty, and the base protocol does not distinguish a replay of -the admin's history from the admin's history**. Freshness is therefore a subscriber-side -property: a subscriber SHOULD keep its cursor per `(topic, admin)` across rejoins, and an -application that needs freshness for a first-time viewer carries it in the payload -(SWIP-65's timestamp key does). Fan-out of a replayed history is bounded by what was -captured and by the inactivity deadline. +**The publisher role takes the key, every time.** A claim is a signature over a challenge +that only this broker could have issued for this address, together with the verifier's +overlay and the publisher's cursor. What can be captured and replayed, by whom, and what +stops it: + +| replayed | by whom | where | result | stopped by | +|---|---|---|---|---| +| the claim signature | a third party or another subscriber | anywhere | cannot obtain it | it travels on the encrypted stream to the broker and nowhere else | +| the claim signature | the node that bridged the admin (holds it, not the key) | this broker, while it runs, from any node | accepted — it upgrades | nothing, by design: the identity continues from another node; that node held the admin's stream anyway and can publish only what the key signed | +| the claim signature | the same node | this broker after a restart, or another broker, or another cohort here | refused: recovers to some other address | `S_C` drawn at boot and never persisted; `O_B` names the verifier; the spec is in `S` | +| the claim signature | anyone | for another address | refused | the address is in `S`, and the signer must equal the declared address | +| the claim with a changed `index` | anyone holding one | anywhere | refused | `index` is inside the signed preimage | +| the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it fetches `S` from the honest broker, hands it over, forwards the signature | the honest broker | refused | the publisher signs the overlay it is talking to, the relay's, and the honest broker checks its own | +| the claim signature as a SOC signature, or the reverse | anyone | anywhere | refused | the domain separator against the SOC's `id ‖ wrappedAddress` | +| the challenge | anyone, by declaring the admin's address | this broker | harmless: `S` proves nothing without the key; the declaration buys the extra slot until the claim deadline, then a blocklist | the claim, not `S`, is the credential | +| the admin's publications | a subscriber or third party | this broker, cohort alive | violation: reset, blocklist | a publication is accepted on a publisher stream only, and that takes the key | +| the admin's publications | the admin's own node, or the admin | this broker, cohort alive | dropped as retransmits, counted | the cursor | +| the admin's publications | the admin's node, or a broker that carried the cohort | after a reclaim, or at another broker, to first-time viewers | delivered — **history**, genuine updates in order, a late viewer caught up | nothing needed; returning viewers keep their cursor per `(topic, admin)`, first-time freshness is the payload's (SWIP-65) | +| a `Join` | anyone | anywhere | attaches or creates, as any join does — no privilege | nothing needed; the bounds and the inactivity deadline | + +**The transport precondition.** The forwarding row above rests on `O_B` being the overlay +the publisher's node is actually connected to, and the claim rests on the broker knowing +the peer it is talking to. A BPS node MUST verify, in the p2p handshake, that a peer's +signed address record names the connection's authenticated peer ID; a record that is +merely self-consistent — signed by the overlay's key but not tied to the connection — can +be presented by anyone who has seen it. (bee's handshake verifies the record and not the +binding as of this writing; the check is one comparison.) **No confidentiality**: the broker and every subscriber see plaintext; applications encrypt payloads. **The broker withholds, never forges**, and a withheld update is @@ -417,47 +503,57 @@ deadline, and a peer can hold only a bounded number of them. An implementation is BPS-lite conformant when: 1. a broker accepts a `Join` whose spec is `{topic, FEED_TOPIC, admin}` and answers any - other binding, an absent `admin`, or any reserved field set with `REJECTED`; + other binding or an absent `admin` with `REJECTED`, ignoring fields it does not define; 2. a `Join` for a spec with no live cohort creates it, whoever sends it; a `Join` with a - byte-identical spec attaches to it; the broker never sends status 3; -3. `Auth` is verified by recovery over `keccak256("bps-join:v1" ‖ topic ‖ admin)`; a - stream whose `Auth` recovers to `admin` is a publisher stream; any other stream is a - subscriber stream; the identity is a property of the stream; -4. `Ack` carries a status and nothing else; a non-`OK` `Ack` ends the stream; -5. a `Message` on a subscriber stream is dropped, the stream reset, the peer blocklisted; -6. a `Message` on a publisher stream is accepted iff its bare index exceeds the cursor, - it validates as a single-owner chunk under `keccak256(topic ‖ index)`, and its owner - is `admin`; accepted frames advance the cursor and are delivered unchanged to every - subscriber stream in the cohort, and to no publisher stream; -7. `n ≤ cursor` is counted as a retransmit, not a violation, and there is no other - dedup state; gaps are accepted; index 0 is accepted on a fresh cohort; + byte-identical canonical serialisation attaches to it; no status other than `OK`, + `FULL` and `REJECTED` is ever sent; +3. the challenge is derived as specified from a boot secret, the canonical spec and the + declared address, issued iff an address was declared, and nothing is stored for it; +4. a claim — in `Join`, or as a subscriber stream's next frame — is verified over + `keccak256("bps-claim:v1" ‖ S ‖ O_B ‖ index)`: the signer equals `addr` and `addr` is + the admin; the stream then becomes a publisher stream and the cursor becomes + `max(cursor, index)`; no reply is sent; a claim in `Join` is verified before any bound; + a stale claim in `Join` is treated as absent; any other claim is a violation; +5. a frame on a subscriber stream that is not a valid claim is dropped, the stream reset, + the peer blocklisted; +6. a `Message` on a publisher stream is accepted iff its bare index is at least the + cursor, the chunk validates as a single-owner chunk under `keccak256(topic ‖ index)` + with the owner hashing to `address`, and `address` is `keccak256(id ‖ admin)`; accepted + frames set the cursor past their index and are delivered unchanged to every subscriber + stream in the cohort, and to no publisher stream; +7. `n < cursor` is counted as a retransmit, not a violation, and there is no other dedup + state; gaps are accepted; index 0 is accepted on a fresh cohort; 8. a cohort with no accepted message for the inactivity deadline is reclaimed, every stream in it reset; a publisher stream going away does not end the cohort; 9. the capacity bounds are enforced, `FULL` is issued at capacity and nothing else is; + one extra stream is admitted for a `Join` declaring the admin's address while no + publisher stream exists, and disconnected if it has not claimed within the claim + deadline; 10. a subscriber re-verifies every delivery against the spec it joined with and its own cursor; -11. a BPS-lite publisher and subscriber interoperate with a full SWIP-60 broker on a - single-publisher cohort once SWIP-60 is amended per *Relation to SWIP-60*. +11. the node verifies in the p2p handshake that a peer's signed address record names the + connection's authenticated peer ID; +12. a BPS-lite publisher and subscriber interoperate with a full SWIP-60 broker on a + single-publisher cohort. A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, -`invalid_soc`, `wrong_owner`, `wrong_stream`, `retransmit`, `queue_reset` — since -items 6, 7 and 9 are unobservable from the wire without them. +`invalid_soc`, `wrong_owner`, `wrong_stream`, `invalid_claim`, `retransmit`, +`queue_reset` — since items 5–7 are unobservable from the wire without them. ## Out of scope (deliberately) Service messages of any kind (roster, end of stream) and the Bee API bridge; multiple -publishers and the promotion of a subscriber stream to a publisher stream; the -self-indexed payload construction, gap recovery and persistence of -[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106); multihop -([SWIP-61](https://github.com/ethersphere/SWIPs/pull/105)); history; bandwidth +publishers and the claim by a rostered address; the self-indexed payload construction, +gap recovery and persistence of [SWIP-65](https://github.com/ethersphere/SWIPs/pull/106); +multihop ([SWIP-61](https://github.com/ethersphere/SWIPs/pull/105)); history; bandwidth incentives; broker discovery ([SWIP-59](https://github.com/ethersphere/SWIPs/pull/103)); confidentiality of any kind. Each extends this SWIP's wire without changing it. ## Backwards compatibility New protocol; no existing behaviour changes. Every message and field number here is -kept by the fuller protocol, which extends by adding fields and messages and never by -changing these; a BPS-lite peer ignores fields it does not define. +kept by the fuller protocol, which extends by adding fields, values and messages and +never by changing these; a BPS-lite peer ignores fields it does not define. ## References @@ -465,7 +561,8 @@ Full singlehop protocol: [SWIP-60, PR #104](https://github.com/ethersphere/SWIPs · carriage: [SWIP-65 self-indexed feeds, PR #106](https://github.com/ethersphere/SWIPs/pull/106) · first draft of this SWIP: [PR #111](https://github.com/ethersphere/SWIPs/pull/111) · origin: [PR #93](https://github.com/ethersphere/SWIPs/pull/93) "Add: pubsub" -· implementation groundwork: bee [#5435](https://github.com/ethersphere/bee/pull/5435) +· implementation: bee [#5626](https://github.com/ethersphere/bee/pull/5626) (groundwork: +bee [#5597](https://github.com/ethersphere/bee/pull/5597), [#5435](https://github.com/ethersphere/bee/pull/5435)) ## Copyright From 1436edbf1bdc4b521bfe719975b275dd1e20c058 Mon Sep 17 00:00:00 2001 From: zelig Date: Thu, 24 Sep 2026 16:41:19 +0200 Subject: [PATCH 05/13] swip-74 rev 3 fixes: signing convention marked, stale-vs-wrong in-Join claims, counters mapped, bounds counted - the claim's signing convention (SOC's EIP-191 prefixed digest) stated and marked (?) - a claim in the Join that does not verify is treated as absent (the broker cannot tell stale from wrong); the peer notices by the S in the Ack; only a post-Ack Claim that fails is a violation; a Claim on a publisher stream is read as a Message - counters: wrong_stream vs invalid_claim defined; claim_timeout added - "first five bounds REQUIRED"; Security counts the bounds the same way - the transport precondition names only what it rests on (O_B) Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 63 ++++++++++++++++++++++++++++-------------------- 1 file changed, 37 insertions(+), 26 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 7f2937f4..4dbebef5 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -140,9 +140,10 @@ message Auth { uint32 v = 3; // 27 or 28 } -// A publisher's claim on the stream it is sent on. `auth` signs -// keccak256("bps-claim:v1" || S || O_B || index) -// with the key of `addr`: S the challenge the broker issued for `addr` on this +// A publisher's claim on the stream it is sent on. `auth` signs the bytes +// "bps-claim:v1" || S || O_B || index +// with the key of `addr`, in the same convention as a SOC signature (the EIP-191 +// prefixed digest bee and bee-js use) (?): S the challenge the broker issued for `addr` on this // cohort, O_B the overlay of the broker the claiming node is connected to, `index` // eight bytes big-endian. Sent inside Join by a peer that already holds S, or as the // next frame after Ack by one that has just received it. @@ -220,10 +221,12 @@ joins, and differs at every other broker and after every restart. It goes to the over the encrypted stream and is useful only to the key of `addr`; anyone may obtain it by declaring the address, and gains nothing by it. -**The claim.** A publisher claims its stream by signing, with the key of `addr`, +**The claim.** A publisher claims its stream by signing, with the key of `addr` and in +the same convention as a SOC signature — the EIP-191 prefixed digest bee and bee-js use +**(?)** — the bytes ``` -keccak256("bps-claim:v1" ‖ S ‖ O_B ‖ index) +"bps-claim:v1" ‖ S ‖ O_B ‖ index ``` where `O_B` is the overlay of the broker the claiming node is connected to and `index` @@ -237,8 +240,11 @@ The claim is sent in one of two places: - **in `Join`**, by a peer that already holds `S` for this address at this broker — a reconnecting admin, from the same node or another: the broker verifies it before any bound is applied, and a valid claim makes the stream a publisher stream from its first - frame. A stale claim — the broker has restarted, `S` has changed — is treated as - absent: `Ack{OK, S}` with the new `S`, no penalty; + frame. A claim in the `Join` that does not verify — stale because the broker has + restarted and `S` has changed, or simply wrong, or naming an `addr` other than the + `Join`'s: the broker cannot tell these apart — is treated as absent: `Ack{OK, S}` with + the current `S`, no penalty. The peer sees that its claim was not taken when the `S` in + the `Ack` differs from the one it signed, and claims again after the `Ack`; - **as the next frame after `Ack`**, a `Claim`, by a peer that has just received `S`. The broker recovers the signer and checks that it equals `addr` (ecrecover never fails, @@ -248,10 +254,11 @@ it returns *some* address, which is why the address is declared and compared) an becomes `max(cursor, index)`. There is **no reply**: a publisher sends its claim and its first publication back to back, and stream ordering guarantees the broker handles the claim first; a claim that did not upgrade makes the publication that follows a violation, -and the reset is the answer. Anything else — a signature that does not recover to `addr`, -an `addr` that is not the admin, a second claim on a stream — is a protocol violation: -dropped, counted (`invalid_claim`), the stream reset, the peer blocklisted per the node's -policy. Several streams MAY be claimed for the admin at once — the admin from two nodes, +and the reset is the answer. A `Claim` after the `Ack` that does not verify — a signature +that does not recover to `addr`, or an `addr` that is not the admin — is a protocol +violation: dropped, counted (`invalid_claim`), the stream reset, the peer blocklisted per +the node's policy; a `Claim` sent on a publisher stream is read as a `Message` and fails as +one. Several streams MAY be claimed for the admin at once — the admin from two nodes, or reconnecting before its old stream is torn down: each is a publisher stream, and the cursor arbitrates. @@ -289,8 +296,9 @@ initially 0 — index 0 is the first update of every feed. A `Message` arriving broker is accepted iff, in order: 1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as a - `Claim`, and if it is not a valid one it is a protocol violation: dropped, the stream - reset, the peer blocklisted per the node's policy; + `Claim`, and if it is not a valid one it is a protocol violation: dropped, counted + (`wrong_stream` if it does not even parse as a `Claim`, `invalid_claim` if it does and + fails), the stream reset, the peer blocklisted per the node's policy; 2. its `id` slot is a bare index `n` and **`n ≥ cursor`**; 3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as a single-owner chunk**: the wrapped chunk's BMT address matches `span ‖ payload`, the @@ -344,12 +352,12 @@ edge, not at the broker. ### Resource bounds -All broker policy, none on the wire. The first four bounds are REQUIRED, with the values +All broker policy, none on the wire. The first five bounds are REQUIRED, with the values given RECOMMENDED where a value is given; the last two are MAY: | bound | answer | recommended | |---|---|---| -| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except **one extra stream while the admin is absent**: a `Join` declaring the admin's `addr` when no publisher stream exists is admitted over the bound, and disconnected, with a short blocklist, if it has not claimed within the **claim deadline** | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | +| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except **one extra stream while the admin is absent**: a `Join` declaring the admin's `addr` when no publisher stream exists is admitted over the bound, and disconnected, with a short blocklist and counted (`claim_timeout`), if it has not claimed within the **claim deadline** | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | | live cohorts per broker | `FULL` to a cohort-creating `Join` | implementation-defined | | **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | | **inactivity deadline** — reclaims a cohort, see *Lifetime* | reset | 10 min | @@ -470,7 +478,7 @@ stops it: |---|---|---|---|---| | the claim signature | a third party or another subscriber | anywhere | cannot obtain it | it travels on the encrypted stream to the broker and nowhere else | | the claim signature | the node that bridged the admin (holds it, not the key) | this broker, while it runs, from any node | accepted — it upgrades | nothing, by design: the identity continues from another node; that node held the admin's stream anyway and can publish only what the key signed | -| the claim signature | the same node | this broker after a restart, or another broker, or another cohort here | refused: recovers to some other address | `S_C` drawn at boot and never persisted; `O_B` names the verifier; the spec is in `S` | +| the claim signature | the same node | this broker after a restart, or another broker, or another cohort here | not accepted: it recovers to some other address — in a `Join` it is treated as absent and a fresh `S` issued, after an `Ack` it is a violation | `S_C` drawn at boot and never persisted; `O_B` names the verifier; the spec is in `S` | | the claim signature | anyone | for another address | refused | the address is in `S`, and the signer must equal the declared address | | the claim with a changed `index` | anyone holding one | anywhere | refused | `index` is inside the signed preimage | | the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it fetches `S` from the honest broker, hands it over, forwards the signature | the honest broker | refused | the publisher signs the overlay it is talking to, the relay's, and the honest broker checks its own | @@ -482,8 +490,7 @@ stops it: | a `Join` | anyone | anywhere | attaches or creates, as any join does — no privilege | nothing needed; the bounds and the inactivity deadline | **The transport precondition.** The forwarding row above rests on `O_B` being the overlay -the publisher's node is actually connected to, and the claim rests on the broker knowing -the peer it is talking to. A BPS node MUST verify, in the p2p handshake, that a peer's +the publisher's node is actually connected to. A BPS node MUST verify, in the p2p handshake, that a peer's signed address record names the connection's authenticated peer ID; a record that is merely self-consistent — signed by the overlay's key but not tied to the connection — can be presented by anyone who has seen it. (bee's handshake verifies the record and not the @@ -493,7 +500,8 @@ binding as of this writing; the check is one comparison.) encrypt payloads. **The broker withholds, never forges**, and a withheld update is visible as a gap in the index. **No end signal**: a broker can end a cohort for its audience by resetting their streams, which is withholding, nothing more. **Resource -bounds are policy and the four capacity bounds are required** (above); the cursor +bounds are policy; the three capacity bounds, the inactivity deadline and the queue bound +are required** (above); the cursor removes the dedup-window bound and its edge. A subscriber that publishes is a protocol violation and is blocklisted; a squatted cohort is one map entry for one inactivity deadline, and a peer can hold only a bounded number of them. @@ -509,11 +517,12 @@ An implementation is BPS-lite conformant when: `FULL` and `REJECTED` is ever sent; 3. the challenge is derived as specified from a boot secret, the canonical spec and the declared address, issued iff an address was declared, and nothing is stored for it; -4. a claim — in `Join`, or as a subscriber stream's next frame — is verified over - `keccak256("bps-claim:v1" ‖ S ‖ O_B ‖ index)`: the signer equals `addr` and `addr` is - the admin; the stream then becomes a publisher stream and the cursor becomes - `max(cursor, index)`; no reply is sent; a claim in `Join` is verified before any bound; - a stale claim in `Join` is treated as absent; any other claim is a violation; +4. a claim — in `Join`, or as a subscriber stream's next frame — is verified over the + bytes `"bps-claim:v1" ‖ S ‖ O_B ‖ index` in the SOC signing convention: the signer + equals `addr` and `addr` is the admin; the stream then becomes a publisher stream and + the cursor becomes `max(cursor, index)`; no reply is sent; a claim in `Join` is verified + before any bound, and treated as absent if it does not verify; a `Claim` after the + `Ack` that does not verify is a violation; 5. a frame on a subscriber stream that is not a valid claim is dropped, the stream reset, the peer blocklisted; 6. a `Message` on a publisher stream is accepted iff its bare index is at least the @@ -537,8 +546,10 @@ An implementation is BPS-lite conformant when: single-publisher cohort. A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, -`invalid_soc`, `wrong_owner`, `wrong_stream`, `invalid_claim`, `retransmit`, -`queue_reset` — since items 5–7 are unobservable from the wire without them. +`invalid_soc`, `wrong_owner`, `wrong_stream` (a subscriber-stream frame that is not a +`Claim`), `invalid_claim` (a `Claim` that fails), `claim_timeout` (an extra stream +disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 +are unobservable from the wire without them. ## Out of scope (deliberately) From 1aacbd25538d6f49bb185d01faff4703ae31f5a5 Mon Sep 17 00:00:00 2001 From: zelig Date: Sun, 27 Sep 2026 13:33:20 +0200 Subject: [PATCH 06/13] swip-74 rev 4: the claim is a chunk (Auth); Message is Broadcast MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Auth{soc}: the claim carried as a single-owner chunk — id = keccak256("bps-claim:v1" || topic), owner = addr, payload = S || O_B || index (span 72) — signed as any SOC is and verified by the ordinary SOC validation against keccak256(id || addr), then the payload against the broker's own S and overlay; settles the signing convention - Join{cohort, addr, auth}; Claim and the r/s/v type are gone - Broadcast{address, data} replaces Message, both directions - four frames: Join, Ack, Auth, Broadcast; one carried type: CohortSpec - wrong_stream defined structurally (not the shape of an Auth); replay table and the verification prose reworded for the chunk (a stale claim fails the payload check, not the address); rationale gains "Why the claim is a chunk" Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 167 ++++++++++++++++++++++++++--------------------- 1 file changed, 93 insertions(+), 74 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 4dbebef5..98ae8ab7 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -13,7 +13,9 @@ created: 2026-09-14 PR #111 after review: no history, no bandwidth incentive, no service messages, singlehop only, one mode only — an explicit single publisher (live video streaming). Rev 3: the publisher role is claimed by signing a broker-issued challenge (the first draft had a -challenge; rev 2 wrongly replaced it with a static signature). SWIP-60 (PR #104) is the +challenge; rev 2 wrongly replaced it with a static signature). Rev 4: the claim is carried +as a single-owner chunk (`Auth`) and verified by the ordinary SOC code; the data frame is +`Broadcast`. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -133,24 +135,22 @@ message CohortSpec { bytes admin = 3; // 20 bytes, required } -// A secp256k1 signature, as a SOC's: r || s || v. +// A publisher's claim on the stream it is sent on, carried as a single-owner chunk so +// that the ordinary SOC validation verifies it in one call: +// id = keccak256("bps-claim:v1" || topic) never a feed id: a 44-byte preimage +// against a feed id's 40 +// owner = addr so address = keccak256(id || addr) +// payload = S || O_B || index 32 + 32 + 8 bytes +// signed as any SOC is, with the key of `addr`: S the challenge the broker issued for +// `addr` on this cohort, O_B the overlay of the broker the claiming node is connected +// to, `index` eight bytes big-endian — the publisher's cursor (?). The receiver derives +// the id from the topic and the expected address from `addr`, validates the chunk +// against it, and checks the payload against its own S and overlay. Sent inside Join by +// a peer that already holds S, or as the next frame after Ack by one that has just +// received it. message Auth { - bytes r = 1; // 32 bytes - bytes s = 2; // 32 bytes - uint32 v = 3; // 27 or 28 -} - -// A publisher's claim on the stream it is sent on. `auth` signs the bytes -// "bps-claim:v1" || S || O_B || index -// with the key of `addr`, in the same convention as a SOC signature (the EIP-191 -// prefixed digest bee and bee-js use) (?): S the challenge the broker issued for `addr` on this -// cohort, O_B the overlay of the broker the claiming node is connected to, `index` -// eight bytes big-endian. Sent inside Join by a peer that already holds S, or as the -// next frame after Ack by one that has just received it. -message Claim { - bytes addr = 1; // 20 bytes: the address claimed; equals Join.addr - uint64 index = 2; // the publisher's cursor: its next message has an index >= this (?) - Auth auth = 3; + bytes soc = 1; // chunk data: id (32) || signature (65) || span (8, LE) || payload (72); + // the id slot carries the full claim id, span is 72; 177 bytes in all } // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live @@ -159,7 +159,7 @@ message Join { CohortSpec cohort = 1; bytes addr = 2; // 20 bytes: the address this stream will publish as; absent: // a subscriber, and no challenge is issued - Claim claim = 3; // a returning publisher's claim, verified before any bound + Auth auth = 3; // a returning publisher's claim, verified before any bound } enum Status { @@ -180,17 +180,17 @@ message Ack { // address and data, opaque to the protocol and validated by the ordinary SOC code // once the id slot has been rewritten as the Frames section says: // data = id (32) || signature (65) || span (8, LE) || payload (<= 4096) -message Message { +message Broadcast { bytes address = 1; // 32 bytes: the SOC address, keccak256(id || owner) bytes data = 2; } ``` -Four frames — `Join`, `Ack`, `Claim`, `Message` — and the two types they carry, -`CohortSpec` and `Auth`. There is no envelope: **what a frame is follows from the -stream's direction and role.** The first peer-to-broker frame is a `Join` and the first -broker-to-peer frame an `Ack`; after that a broker sends only `Message`, a subscriber -stream sends only a `Claim`, and a publisher stream sends only `Message`. A frame that +Four frames — `Join`, `Ack`, `Auth`, `Broadcast` — and the one type they carry, +`CohortSpec`. There is no envelope: **what a frame is follows from the stream's +direction and role.** The first peer-to-broker frame is a `Join` and the first +broker-to-peer frame an `Ack`; after that a broker sends only `Broadcast`, a subscriber +stream sends only an `Auth`, and a publisher stream sends only `Broadcast`. A frame that does not parse as what its stream may send is invalid. ### Handshake: `Join`, the challenge, the claim @@ -221,44 +221,54 @@ joins, and differs at every other broker and after every restart. It goes to the over the encrypted stream and is useful only to the key of `addr`; anyone may obtain it by declaring the address, and gains nothing by it. -**The claim.** A publisher claims its stream by signing, with the key of `addr` and in -the same convention as a SOC signature — the EIP-191 prefixed digest bee and bee-js use -**(?)** — the bytes +**The claim.** A publisher claims its stream with an `Auth`: a single-owner chunk it +signs with the key of `addr`, exactly as it signs any SOC, whose ``` -"bps-claim:v1" ‖ S ‖ O_B ‖ index +id = keccak256("bps-claim:v1" ‖ topic) +owner = addr address = keccak256(id ‖ addr) +payload = S ‖ O_B ‖ index 32 + 32 + 8 bytes ``` where `O_B` is the overlay of the broker the claiming node is connected to and `index` is the publisher's cursor: the claim that its next message will have a feed index of at -least `index`. The domain separator keeps the signature disjoint from SOC signatures and -any other protocol's; `S` binds it to this broker, this cohort and this address; `O_B` -binds it to the verifier, which `S` cannot, because `S` is opaque to the signer (see -Security considerations); `index` is signed so that a replayed claim moves no cursor. -The claim is sent in one of two places: +least `index`. The receiver verifies it with the ordinary SOC validation plus one payload +comparison: it derives the id from the topic, forms the expected address +`keccak256(id ‖ addr)` from the declared address, and validates the chunk against it — +signature, digest and signer in one existing call, with the id slot carrying the full +claim id and the span 72 — then checks that the payload is the `S` it derives for `addr`, +its own overlay, and an index. The separator in the id keeps a claim from ever +being a feed update: a feed id's preimage is `topic ‖ index`, 40 bytes, a claim id's is +44, so neither ever hashes to the other's address. `S` binds the claim to this broker, +this cohort and this address; `O_B` binds it to the verifier, which `S` cannot, because +`S` is opaque to the signer (see Security considerations); `index` is signed so that a +replayed claim moves no cursor. A claim is a storable chunk, at an address nobody reads +as content. It is sent in one of two places: - **in `Join`**, by a peer that already holds `S` for this address at this broker — a reconnecting admin, from the same node or another: the broker verifies it before any bound is applied, and a valid claim makes the stream a publisher stream from its first frame. A claim in the `Join` that does not verify — stale because the broker has - restarted and `S` has changed, or simply wrong, or naming an `addr` other than the - `Join`'s: the broker cannot tell these apart — is treated as absent: `Ack{OK, S}` with + restarted and `S` has changed, or simply wrong, or signed by a key other than the + `Join`'s `addr`: the broker treats them alike — is treated as absent: `Ack{OK, S}` with the current `S`, no penalty. The peer sees that its claim was not taken when the `S` in the `Ack` differs from the one it signed, and claims again after the `Ack`; -- **as the next frame after `Ack`**, a `Claim`, by a peer that has just received `S`. +- **as the next frame after `Ack`**, an `Auth`, by a peer that has just received `S`. -The broker recovers the signer and checks that it equals `addr` (ecrecover never fails, -it returns *some* address, which is why the address is declared and compared) and that -`addr` is the cohort's admin — the one address that may publish here. Then the stream +Having validated the chunk at `keccak256(id ‖ addr)` — which is what makes the signer's +identity a check rather than a recovery, since recovering a signer always yields *some* +address — and matched the payload, the broker checks that `addr` is the cohort's admin, +the one address that may publish here. Then the stream **upgrades** to a **publisher stream**: it leaves the fan-out set, and the cohort's cursor becomes `max(cursor, index)`. There is **no reply**: a publisher sends its claim and its first publication back to back, and stream ordering guarantees the broker handles the claim first; a claim that did not upgrade makes the publication that follows a violation, -and the reset is the answer. A `Claim` after the `Ack` that does not verify — a signature -that does not recover to `addr`, or an `addr` that is not the admin — is a protocol -violation: dropped, counted (`invalid_claim`), the stream reset, the peer blocklisted per -the node's policy; a `Claim` sent on a publisher stream is read as a `Message` and fails as -one. Several streams MAY be claimed for the admin at once — the admin from two nodes, +and the reset is the answer. An `Auth` after the `Ack` that does not verify — a chunk that +is not valid at the expected address, a payload that is not the broker's `S` and overlay, +or an `addr` that is not the admin — is a protocol violation: dropped, counted +(`invalid_claim`), the stream reset, the peer blocklisted per the node's policy; an `Auth` +sent on a publisher stream is read as a `Broadcast` and fails as one. Several streams MAY +be claimed for the admin at once — the admin from two nodes, or reconnecting before its old stream is torn down: each is a publisher stream, and the cursor arbitrates. @@ -272,9 +282,9 @@ brings its whole audience back at once. ### Frames -After the handshake a broker sends `Message` and nothing else, to subscriber streams; a -publisher stream sends `Message` and nothing else; a subscriber stream sends at most one -`Claim`. A delivery is the accepted publication's bytes, unchanged. +After the handshake a broker sends `Broadcast` and nothing else, to subscriber streams; a +publisher stream sends `Broadcast` and nothing else; a subscriber stream sends at most one +`Auth`. A delivery is the accepted publication's bytes, unchanged. **The frame is a whole chunk.** `address` is the SOC address and `data` the chunk data, so that the ordinary SOC validation applies: without the address it would be vacuous, @@ -285,20 +295,21 @@ because the recovered owner no longer hashes to the address. `keccak256(topic ‖ index)`, `index` a uint64 big-endian in 8 bytes. On the wire the 32-byte `id` slot of `data` holds that index left-padded with 24 zero bytes, and the receiver reconstructs the signed id from the cohort's topic (the carriage of -[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). A `Message` whose `id` slot +[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). A `Broadcast` whose `id` slot does not have 24 leading zero bytes is not a feed update: a broker drops it as invalid; a subscriber drops it without counting it as a violation. ### Validation: the feed cursor The broker keeps one **cursor** per cohort: **the lowest index it will accept next**, -initially 0 — index 0 is the first update of every feed. A `Message` arriving at the +initially 0 — index 0 is the first update of every feed. A `Broadcast` arriving at the broker is accepted iff, in order: -1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as a - `Claim`, and if it is not a valid one it is a protocol violation: dropped, counted - (`wrong_stream` if it does not even parse as a `Claim`, `invalid_claim` if it does and - fails), the stream reset, the peer blocklisted per the node's policy; +1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as an + `Auth`, and if it is not a valid one it is a protocol violation: dropped, counted + (`wrong_stream` if it is not even the shape of an `Auth` — a `soc` of 177 bytes and no + other field; a `Broadcast` parses as an `Auth` with a 32-byte `soc` — `invalid_claim` + if it is and fails), the stream reset, the peer blocklisted per the node's policy; 2. its `id` slot is a bare index `n` and **`n ≥ cursor`**; 3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as a single-owner chunk**: the wrapped chunk's BMT address matches `span ‖ payload`, the @@ -393,7 +404,7 @@ this wire, they never change it**: retransmits; no frame a BPS-lite broker accepts is one a full broker refuses. What SWIP-60 adds on this wire: cohort parameters (`publishers`, `history`, `closed`), -the admin's service feed (roster, end of stream) as `Message` frames, a claim by any +the admin's service feed (roster, end of stream) as `Broadcast` frames, a claim by any rostered address and none at all under `ALL`, one extra stream per absent rostered publisher, and the Bee API. What it presumes of the transport, this SWIP states below as a precondition. @@ -441,6 +452,13 @@ upgraded by a claim when the admin grants it. hashes to the address. Without the address on the frame there is nothing to hash to, and a broker that only recovered a signer would accept any signature at all. +**Why the claim is a chunk.** A claim has to be signed by the publisher's key and +verified by the broker; a single-owner chunk is what this protocol already signs and +verifies, with code every node has. Carrying the claim as a chunk with a domain-separated +id makes its verification one existing call, settles how it is signed (as any SOC is), +and keeps it out of the feed's id space by construction. What the chunk carries — the +challenge, the verifier's overlay, the cursor — is what the claim has to say. + **Why no reply to a claim.** After the handshake each direction carries one frame type, and the wire has no envelope. A reply would be a second broker-to-peer type on a stream that carries deliveries, with nothing to tell them apart. It is also unnecessary: streams @@ -469,20 +487,20 @@ service feed arrives. Those of SWIP-60, restricted to its live-stream configuration (admin set, the admin alone publishes, open audience). -**The publisher role takes the key, every time.** A claim is a signature over a challenge -that only this broker could have issued for this address, together with the verifier's -overlay and the publisher's cursor. What can be captured and replayed, by whom, and what -stops it: +**The publisher role takes the key, every time.** A claim is a chunk signed by the +publisher's key over a challenge that only this broker could have issued for this +address, together with the verifier's overlay and the publisher's cursor. What can be +captured and replayed, by whom, and what stops it: | replayed | by whom | where | result | stopped by | |---|---|---|---|---| -| the claim signature | a third party or another subscriber | anywhere | cannot obtain it | it travels on the encrypted stream to the broker and nowhere else | -| the claim signature | the node that bridged the admin (holds it, not the key) | this broker, while it runs, from any node | accepted — it upgrades | nothing, by design: the identity continues from another node; that node held the admin's stream anyway and can publish only what the key signed | -| the claim signature | the same node | this broker after a restart, or another broker, or another cohort here | not accepted: it recovers to some other address — in a `Join` it is treated as absent and a fresh `S` issued, after an `Ack` it is a violation | `S_C` drawn at boot and never persisted; `O_B` names the verifier; the spec is in `S` | -| the claim signature | anyone | for another address | refused | the address is in `S`, and the signer must equal the declared address | -| the claim with a changed `index` | anyone holding one | anywhere | refused | `index` is inside the signed preimage | +| the claim chunk | a third party or another subscriber | anywhere | cannot obtain it | it travels on the encrypted stream to the broker and nowhere else | +| the claim chunk | the node that bridged the admin (holds it, not the key) | this broker, while it runs, from any node | accepted — it upgrades | nothing, by design: the identity continues from another node; that node held the admin's stream anyway and can publish only what the key signed | +| the claim chunk | the same node | this broker after a restart, or another broker, or another cohort here | refused: the chunk is valid from the right key, but its payload is not this broker's `S` and overlay — on another topic the id, hence the expected address, differs as well; in a `Join` it is treated as absent and a fresh `S` issued, after an `Ack` it is a violation | `S_C` drawn at boot and never persisted; `O_B` names the verifier; the spec is in `S` and the topic in the id | +| the claim chunk | anyone | for another address | refused | the chunk does not validate at `keccak256(id ‖ addr)` for any other `addr`, and the address is in `S` | +| the claim with a changed `index` | anyone holding one | anywhere | refused | `index` is in the signed payload | | the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it fetches `S` from the honest broker, hands it over, forwards the signature | the honest broker | refused | the publisher signs the overlay it is talking to, the relay's, and the honest broker checks its own | -| the claim signature as a SOC signature, or the reverse | anyone | anywhere | refused | the domain separator against the SOC's `id ‖ wrappedAddress` | +| the claim chunk as a feed update, or a feed update as a claim | anyone | anywhere | refused | a claim's id is `keccak256("bps-claim:v1" ‖ topic)`, a feed update's `keccak256(topic ‖ index)`: different preimage lengths, never the same address | | the challenge | anyone, by declaring the admin's address | this broker | harmless: `S` proves nothing without the key; the declaration buys the extra slot until the claim deadline, then a blocklist | the claim, not `S`, is the credential | | the admin's publications | a subscriber or third party | this broker, cohort alive | violation: reset, blocklist | a publication is accepted on a publisher stream only, and that takes the key | | the admin's publications | the admin's own node, or the admin | this broker, cohort alive | dropped as retransmits, counted | the cursor | @@ -517,15 +535,16 @@ An implementation is BPS-lite conformant when: `FULL` and `REJECTED` is ever sent; 3. the challenge is derived as specified from a boot secret, the canonical spec and the declared address, issued iff an address was declared, and nothing is stored for it; -4. a claim — in `Join`, or as a subscriber stream's next frame — is verified over the - bytes `"bps-claim:v1" ‖ S ‖ O_B ‖ index` in the SOC signing convention: the signer - equals `addr` and `addr` is the admin; the stream then becomes a publisher stream and - the cursor becomes `max(cursor, index)`; no reply is sent; a claim in `Join` is verified - before any bound, and treated as absent if it does not verify; a `Claim` after the - `Ack` that does not verify is a violation; +4. a claim — an `Auth` in `Join`, or as a subscriber stream's next frame — is a + single-owner chunk verified by the ordinary SOC validation against + `keccak256(keccak256("bps-claim:v1" ‖ topic) ‖ addr)`, with payload `S ‖ O_B ‖ index` + matching the broker's own `S` and overlay, and `addr` the admin; the stream then + becomes a publisher stream and the cursor becomes `max(cursor, index)`; no reply is + sent; a claim in `Join` is verified before any bound, and treated as absent if it does + not verify; an `Auth` after the `Ack` that does not verify is a violation; 5. a frame on a subscriber stream that is not a valid claim is dropped, the stream reset, the peer blocklisted; -6. a `Message` on a publisher stream is accepted iff its bare index is at least the +6. a `Broadcast` on a publisher stream is accepted iff its bare index is at least the cursor, the chunk validates as a single-owner chunk under `keccak256(topic ‖ index)` with the owner hashing to `address`, and `address` is `keccak256(id ‖ admin)`; accepted frames set the cursor past their index and are delivered unchanged to every subscriber @@ -546,8 +565,8 @@ An implementation is BPS-lite conformant when: single-publisher cohort. A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, -`invalid_soc`, `wrong_owner`, `wrong_stream` (a subscriber-stream frame that is not a -`Claim`), `invalid_claim` (a `Claim` that fails), `claim_timeout` (an extra stream +`invalid_soc`, `wrong_owner`, `wrong_stream` (a subscriber-stream frame that is not the +shape of an `Auth`), `invalid_claim` (an `Auth` that fails), `claim_timeout` (an extra stream disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. From f72b309c251b1ae55d6fa37ba92c5ff6228223f2 Mon Sep 17 00:00:00 2001 From: zelig Date: Mon, 28 Sep 2026 10:43:02 +0200 Subject: [PATCH 07/13] swip-74 rev 5: Broadcast carries the chunk data alone; the receiver forms the address MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Broadcast{soc}: no address on the frame. Publishing is restricted to a known owner, so every receiver forms keccak256(keccak256(topic || index) || admin) itself and validates the chunk against it with the ordinary SOC code — signature, digest and owner in one call, the owner forced rather than compared afterwards - validation steps 3 and 4 merge; wrong_owner folds into invalid_soc - Auth and Broadcast are one shape on the wire, so the subscriber-stream shape test is length: wrong_stream iff soc is not 177 bytes - rationale: "Why the frame carries no address" Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 79 +++++++++++++++++++++++++++--------------------- 1 file changed, 45 insertions(+), 34 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 98ae8ab7..fea91503 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -15,7 +15,9 @@ only, one mode only — an explicit single publisher (live video streaming). Rev publisher role is claimed by signing a broker-issued challenge (the first draft had a challenge; rev 2 wrongly replaced it with a static signature). Rev 4: the claim is carried as a single-owner chunk (`Auth`) and verified by the ordinary SOC code; the data frame is -`Broadcast`. SWIP-60 (PR #104) is the +`Broadcast`. Rev 5: `Broadcast` carries the chunk data alone — the receiver forms the +address it validates against from the owner it already knows — and `wrong_owner` folds +into `invalid_soc`. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -46,8 +48,8 @@ the address it will publish as; the broker answers with a **challenge** it deriv secret it never stores, and the publisher claims its stream by signing it. Its messages are single-owner chunks that are the higher-index updates of the feed the topic names, carried with their bare index; the broker accepts each one iff its index is at least the -channel's cursor, the chunk validates as a single-owner chunk under the feed's id, and its -owner is the admin, and delivers it unchanged to every subscriber stream — never to a +channel's cursor and the chunk validates as the admin's single-owner chunk under the +feed's id, and delivers it unchanged to every subscriber stream — never to a publisher's. All other joiners are subscribers and verify the same way, end to end: the broker can withhold, never forge. A channel ends when it goes idle. @@ -176,13 +178,14 @@ message Ack { } // Both directions after the handshake: publisher -> broker is a publication, broker -// -> subscriber a delivery of the same bytes. The single-owner chunk travels whole, -// address and data, opaque to the protocol and validated by the ordinary SOC code -// once the id slot has been rewritten as the Frames section says: -// data = id (32) || signature (65) || span (8, LE) || payload (<= 4096) +// -> subscriber a delivery of the same bytes. The single-owner chunk travels as its +// chunk data, opaque to the protocol; the receiver forms the address it must have — +// keccak256(id || owner), with the owner it already knows — and validates the chunk +// against it with the ordinary SOC code once the id slot has been rewritten as the +// Frames section says: +// soc = id (32) || signature (65) || span (8, LE) || payload (<= 4096) message Broadcast { - bytes address = 1; // 32 bytes: the SOC address, keccak256(id || owner) - bytes data = 2; + bytes soc = 1; } ``` @@ -286,14 +289,16 @@ After the handshake a broker sends `Broadcast` and nothing else, to subscriber s publisher stream sends `Broadcast` and nothing else; a subscriber stream sends at most one `Auth`. A delivery is the accepted publication's bytes, unchanged. -**The frame is a whole chunk.** `address` is the SOC address and `data` the chunk data, -so that the ordinary SOC validation applies: without the address it would be vacuous, -since recovering a signer always yields *an* address, and with it a bad signature fails -because the recovered owner no longer hashes to the address. +**The frame carries no address.** Recovering a signer always yields *an* address, so the +ordinary SOC validation needs an address to hold the chunk against — and every receiver +has one: in BPS-lite the publisher is the admin, so the expected address of an update is +`keccak256(keccak256(topic ‖ index) ‖ admin)`, and validating the chunk against it checks +signature, digest and owner in one call, with the owner forced rather than compared +afterwards. Carrying the address would say nothing the receiver does not already know. **The `id` slot carries the bare index.** A feed update's signed id is `keccak256(topic ‖ index)`, `index` a uint64 big-endian in 8 bytes. On the wire the -32-byte `id` slot of `data` holds that index left-padded with 24 zero bytes, and the +32-byte `id` slot of `soc` holds that index left-padded with 24 zero bytes, and the receiver reconstructs the signed id from the cohort's topic (the carriage of [SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). A `Broadcast` whose `id` slot does not have 24 leading zero bytes is not a feed update: a broker drops it as invalid; a @@ -307,21 +312,22 @@ broker is accepted iff, in order: 1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as an `Auth`, and if it is not a valid one it is a protocol violation: dropped, counted - (`wrong_stream` if it is not even the shape of an `Auth` — a `soc` of 177 bytes and no - other field; a `Broadcast` parses as an `Auth` with a 32-byte `soc` — `invalid_claim` - if it is and fails), the stream reset, the peer blocklisted per the node's policy; + (`Auth` and `Broadcast` are one shape on the wire, so the only shape test is length: + `wrong_stream` if `soc` is not 177 bytes, `invalid_claim` if it is and the claim + validation fails), the stream reset, the peer blocklisted per the node's policy; 2. its `id` slot is a bare index `n` and **`n ≥ cursor`**; -3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as a - single-owner chunk**: the wrapped chunk's BMT address matches `span ‖ payload`, the - signature over `id ‖ wrappedAddress` recovers an owner, and `keccak256(id ‖ owner)` - equals `address`; -4. `address` equals `keccak256(id ‖ admin)` — the owner is the admin. +3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as the + admin's single-owner chunk**: the receiver forms the expected address + `keccak256(id ‖ admin)` and runs the ordinary SOC validation against it — the wrapped + chunk's BMT address matches `span ‖ payload`, and the signature over + `id ‖ wrappedAddress` recovers an owner that hashes with the id to that address, which + only the admin's does. On acceptance the broker sets `cursor := n + 1` and enqueues the frame on **every subscriber stream** in the cohort. Publisher streams receive nothing: a publisher never gets its own messages back, on whichever of its streams they were sent. -Failures of 3 and 4 are invalid frames: dropped and counted; repeated invalid frames end +A failure of 3 is an invalid frame: dropped and counted; repeated invalid frames end the connection (blocklisting policy). A frame failing 2 with `n < cursor` is the one benign failure — a retransmit, which an admin reconnecting after a reset legitimately sends when it does not know what the broker last accepted — and is counted separately; a @@ -337,8 +343,8 @@ to what the admin has published elsewhere, never back. This is stricter than SWI general carriage rule, under which a broker does not enforce monotonicity; the restriction is sound here because one publisher on one hop admits no legitimate reordering. -Subscribers re-verify every delivery exactly as the broker does (steps 2–4, against the -spec they joined with and their own cursor), end to end, whatever the broker did. +Subscribers re-verify every delivery exactly as the broker does (steps 2 and 3, against +the spec they joined with and their own cursor), end to end, whatever the broker did. ### Lifetime: inactivity @@ -448,9 +454,13 @@ admin's. Binding the identity to the stream is what lets one node carry differen identities on different streams — and, in SWIP-60, what lets a subscriber stream be upgraded by a claim when the admin grants it. -**Why the frame is a whole chunk.** The ordinary SOC validation checks that the signer -hashes to the address. Without the address on the frame there is nothing to hash to, and -a broker that only recovered a signer would accept any signature at all. +**Why the frame carries no address.** The ordinary SOC validation checks that the signer +hashes, with the id, to the chunk's address; a receiver that only recovered a signer +would accept any signature at all. But the address is not information the sender has and +the receiver lacks: publishing is restricted to known owners — the admin here, a claimed +or declared address in SWIP-60 — so every receiver forms the expected address itself and +holds the chunk to it. That is the same check with the owner forced, and 32 bytes fewer +on every frame. **Why the claim is a chunk.** A claim has to be signed by the publisher's key and verified by the broker; a single-owner chunk is what this protocol already signs and @@ -545,10 +555,10 @@ An implementation is BPS-lite conformant when: 5. a frame on a subscriber stream that is not a valid claim is dropped, the stream reset, the peer blocklisted; 6. a `Broadcast` on a publisher stream is accepted iff its bare index is at least the - cursor, the chunk validates as a single-owner chunk under `keccak256(topic ‖ index)` - with the owner hashing to `address`, and `address` is `keccak256(id ‖ admin)`; accepted - frames set the cursor past their index and are delivered unchanged to every subscriber - stream in the cohort, and to no publisher stream; + cursor and the chunk validates as a single-owner chunk at the address the broker forms + itself, `keccak256(keccak256(topic ‖ index) ‖ admin)`; accepted frames set the cursor + past their index and are delivered unchanged to every subscriber stream in the cohort, + and to no publisher stream; 7. `n < cursor` is counted as a retransmit, not a violation, and there is no other dedup state; gaps are accepted; index 0 is accepted on a fresh cohort; 8. a cohort with no accepted message for the inactivity deadline is reclaimed, every @@ -565,8 +575,9 @@ An implementation is BPS-lite conformant when: single-publisher cohort. A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, -`invalid_soc`, `wrong_owner`, `wrong_stream` (a subscriber-stream frame that is not the -shape of an `Auth`), `invalid_claim` (an `Auth` that fails), `claim_timeout` (an extra stream +`invalid_soc` (a chunk that does not validate at the expected address: bad signature, +wrong owner or wrong digest alike), `wrong_stream` (a subscriber-stream frame whose `soc` +is not 177 bytes), `invalid_claim` (one that is, and fails), `claim_timeout` (an extra stream disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. From d31e46456fce41aa2f5f8f1e07ece9e9ad4dc28a Mon Sep 17 00:00:00 2001 From: zelig Date: Mon, 28 Sep 2026 14:45:54 +0200 Subject: [PATCH 08/13] =?UTF-8?q?swip-74=20rev=206:=20no=20Auth=20type=20?= =?UTF-8?q?=E2=80=94=20the=20claim=20is=20a=20Broadcast=20carrying=20a=20C?= =?UTF-8?q?LAIM=20service=20message?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - rev 5's 177-byte length test was wrong (a feed update can be 177 bytes too); nothing on the wire is told apart by shape - the claim is a Broadcast like every other frame; its chunk keeps the id keccak256("bps-claim:v1" || topic) and owner addr, its payload is ServiceMessage{kind: CLAIM, index, challenge: S, overlay: O_B}; recognised by id and kind, as the family's service messages are; the id slot carries the full claim id - Join.auth is the claim chunk's bytes; three frames Join, Ack, Broadcast; two carried types CohortSpec and ServiceMessage - wrong_stream covers every invalid subscriber-stream frame; invalid_claim is gone - a non-feed-update id on a publisher stream is dropped and counted, not a violation (a full-protocol admin's service SOC at a lite broker) Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 210 +++++++++++++++++++++++++++-------------------- 1 file changed, 120 insertions(+), 90 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index fea91503..2e2c17f4 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -17,7 +17,9 @@ challenge; rev 2 wrongly replaced it with a static signature). Rev 4: the claim as a single-owner chunk (`Auth`) and verified by the ordinary SOC code; the data frame is `Broadcast`. Rev 5: `Broadcast` carries the chunk data alone — the receiver forms the address it validates against from the owner it already knows — and `wrong_owner` folds -into `invalid_soc`. SWIP-60 (PR #104) is the +into `invalid_soc`. Rev 6: the `Auth` type goes; the claim is a `Broadcast` whose chunk +payload is a service message of kind `CLAIM`, and nothing on the wire is told apart by +shape. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -25,14 +27,15 @@ SWIP-60". Open points are marked (?). --> storage round trip and no polling: video/audio streaming, a price ticker, a game's server-authoritative state, a log tail. The stream is a feed, so the same updates can later be persisted and replayed from storage by anyone who missed the live run. -- **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the four frames +- **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the three frames below — one broker, direct streams, one publisher that proves its key by signing a challenge, read-only subscribers — and nothing else: **no history, no bandwidth - incentive, no service messages, no Bee API, one hop, one mode.** Done when a broker, a + incentive, no roster or end-of-stream messages, no Bee API, one hop, one mode.** Done + when a broker, a publisher and subscribers from independent implementations interoperate per the - conformance section. Everything the family adds — service messages and the Bee API, - multiple publishers, self-indexed feeds, multihop — extends this wire without changing - it. + conformance section. Everything the family adds — the admin's service messages and the + Bee API, multiple publishers, self-indexed feeds, multihop — extends this wire without + changing it. - **DISC change**: NO. A new p2p protocol surface; no storage, retrieval or incentive change. @@ -60,7 +63,7 @@ on a service feed, a history flag and the Bee API. That is the right target for reference implementation and too much for a second, independent one whose application is one author broadcasting to an audience. BPS-lite names that one configuration and specifies only what it needs, so that a second team can implement and conformance-test -it from four frames — and so that it stays the **base** of the family: nothing here is +it from three frames — and so that it stays the **base** of the family: nothing here is a divergence to reconcile later, because the fuller protocol extends these frames. One publisher over a feed also removes what the fuller protocol has to carry: a dedup @@ -137,23 +140,37 @@ message CohortSpec { bytes admin = 3; // 20 bytes, required } -// A publisher's claim on the stream it is sent on, carried as a single-owner chunk so -// that the ordinary SOC validation verifies it in one call: +// What a chunk on this wire says when it is not a feed update. BPS-lite has one kind, +// the claim; SWIP-60 adds the admin's roster and end of stream. Every such chunk is +// recognised by its id and by this kind in its payload — never by its shape. +enum ServiceKind { + SERVICE_KIND_UNSPECIFIED = 0; // invalid on the wire + CLAIM = 1; +} + +// The payload of a service chunk. +message ServiceMessage { + ServiceKind kind = 1; + uint64 index = 2; // CLAIM: the publisher's cursor — its next message has an + // index >= this (?) + bytes challenge = 3; // CLAIM: S, as received in the Ack + bytes overlay = 4; // CLAIM: O_B, the overlay of the broker the claiming node + // is connected to +} + +// A publisher's claim on the stream it is sent on is a Broadcast whose chunk is a +// single-owner chunk it signs as any SOC, with // id = keccak256("bps-claim:v1" || topic) never a feed id: a 44-byte preimage // against a feed id's 40 // owner = addr so address = keccak256(id || addr) -// payload = S || O_B || index 32 + 32 + 8 bytes -// signed as any SOC is, with the key of `addr`: S the challenge the broker issued for -// `addr` on this cohort, O_B the overlay of the broker the claiming node is connected -// to, `index` eight bytes big-endian — the publisher's cursor (?). The receiver derives -// the id from the topic and the expected address from `addr`, validates the chunk -// against it, and checks the payload against its own S and overlay. Sent inside Join by -// a peer that already holds S, or as the next frame after Ack by one that has just -// received it. -message Auth { - bytes soc = 1; // chunk data: id (32) || signature (65) || span (8, LE) || payload (72); - // the id slot carries the full claim id, span is 72; 177 bytes in all -} +// payload = ServiceMessage{CLAIM, index, S, O_B} +// The id slot carries the full claim id (so it never has 24 leading zero bytes and is +// never read as a feed update); span is the payload's length, as for any SOC. The +// receiver derives the id from the topic, requires the slot to equal it, forms the +// expected address from `addr`, validates the chunk against it in one existing call, +// decodes the payload, and checks the challenge and overlay against its own. Sent +// inside Join by a peer that already holds S, or as the next Broadcast after Ack by one +// that has just received it. // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live // cohort has this spec, attaches to it otherwise. @@ -161,7 +178,8 @@ message Join { CohortSpec cohort = 1; bytes addr = 2; // 20 bytes: the address this stream will publish as; absent: // a subscriber, and no challenge is issued - Auth auth = 3; // a returning publisher's claim, verified before any bound + bytes auth = 3; // a returning publisher's claim: the claim chunk's data, + // verified before any bound } enum Status { @@ -177,24 +195,25 @@ message Ack { bytes challenge = 2; // S, iff status == OK and addr was declared } -// Both directions after the handshake: publisher -> broker is a publication, broker -// -> subscriber a delivery of the same bytes. The single-owner chunk travels as its -// chunk data, opaque to the protocol; the receiver forms the address it must have — +// Every frame after the handshake: publisher -> broker a publication or a claim, +// broker -> subscriber a delivery of the same bytes. The single-owner chunk travels as +// its chunk data, opaque to the protocol; the receiver forms the address it must have — // keccak256(id || owner), with the owner it already knows — and validates the chunk -// against it with the ordinary SOC code once the id slot has been rewritten as the -// Frames section says: +// against it with the ordinary SOC code, once the id slot has been rewritten as the +// Frames section says for a feed update: // soc = id (32) || signature (65) || span (8, LE) || payload (<= 4096) message Broadcast { bytes soc = 1; } ``` -Four frames — `Join`, `Ack`, `Auth`, `Broadcast` — and the one type they carry, -`CohortSpec`. There is no envelope: **what a frame is follows from the stream's -direction and role.** The first peer-to-broker frame is a `Join` and the first -broker-to-peer frame an `Ack`; after that a broker sends only `Broadcast`, a subscriber -stream sends only an `Auth`, and a publisher stream sends only `Broadcast`. A frame that -does not parse as what its stream may send is invalid. +Three frames — `Join`, `Ack`, `Broadcast` — and the two types they carry, `CohortSpec` +and, as a chunk payload, `ServiceMessage`. There is no envelope: **what a frame is +follows from the stream's direction and role, and what a chunk is from its id and its +payload's kind.** The first peer-to-broker frame is a `Join` and the first broker-to-peer +frame an `Ack`; after that every frame is a `Broadcast`: a subscriber stream may send +only a claim, a publisher stream only publications, the broker only deliveries. A frame +that is not what its stream may send is invalid. ### Handshake: `Join`, the challenge, the claim @@ -224,29 +243,33 @@ joins, and differs at every other broker and after every restart. It goes to the over the encrypted stream and is useful only to the key of `addr`; anyone may obtain it by declaring the address, and gains nothing by it. -**The claim.** A publisher claims its stream with an `Auth`: a single-owner chunk it -signs with the key of `addr`, exactly as it signs any SOC, whose +**The claim.** A publisher claims its stream with a `Broadcast` whose chunk is a +single-owner chunk it signs with the key of `addr`, exactly as it signs any SOC, whose ``` id = keccak256("bps-claim:v1" ‖ topic) owner = addr address = keccak256(id ‖ addr) -payload = S ‖ O_B ‖ index 32 + 32 + 8 bytes +payload = ServiceMessage{kind: CLAIM, index, challenge: S, overlay: O_B} ``` where `O_B` is the overlay of the broker the claiming node is connected to and `index` is the publisher's cursor: the claim that its next message will have a feed index of at -least `index`. The receiver verifies it with the ordinary SOC validation plus one payload -comparison: it derives the id from the topic, forms the expected address -`keccak256(id ‖ addr)` from the declared address, and validates the chunk against it — -signature, digest and signer in one existing call, with the id slot carrying the full -claim id and the span 72 — then checks that the payload is the `S` it derives for `addr`, -its own overlay, and an index. The separator in the id keeps a claim from ever -being a feed update: a feed id's preimage is `topic ‖ index`, 40 bytes, a claim id's is -44, so neither ever hashes to the other's address. `S` binds the claim to this broker, -this cohort and this address; `O_B` binds it to the verifier, which `S` cannot, because -`S` is opaque to the signer (see Security considerations); `index` is signed so that a -replayed claim moves no cursor. A claim is a storable chunk, at an address nobody reads -as content. It is sent in one of two places: +least `index`. The chunk's `id` slot carries the full claim id — so it never has 24 +leading zero bytes and is never read as a feed update — and its span is the payload's +length, as for any SOC. The receiver verifies it with the ordinary SOC validation plus one +payload check: it derives the id from the topic, requires the slot to equal it, forms the +expected address `keccak256(id ‖ addr)` from the declared address, and validates the chunk +against it — signature, digest and signer in one existing call — then decodes the payload +and checks that its kind is +`CLAIM`, its challenge the `S` it derives for `addr`, and its overlay its own. Nothing is +told apart by shape: the id says what the chunk is, and the kind in the payload says it +again, as it does for every service message of the family. The separator in the id keeps +a claim from ever being a feed update: a feed id's preimage is `topic ‖ index`, 40 bytes, +a claim id's is 44, so neither ever hashes to the other's address. `S` binds the claim to +this broker, this cohort and this address; `O_B` binds it to the verifier, which `S` +cannot, because `S` is opaque to the signer (see Security considerations); `index` is +signed so that a replayed claim moves no cursor. A claim is a storable chunk, at an +address nobody reads as content. It is sent in one of two places: - **in `Join`**, by a peer that already holds `S` for this address at this broker — a reconnecting admin, from the same node or another: the broker verifies it before any @@ -256,7 +279,7 @@ as content. It is sent in one of two places: `Join`'s `addr`: the broker treats them alike — is treated as absent: `Ack{OK, S}` with the current `S`, no penalty. The peer sees that its claim was not taken when the `S` in the `Ack` differs from the one it signed, and claims again after the `Ack`; -- **as the next frame after `Ack`**, an `Auth`, by a peer that has just received `S`. +- **as the next frame after `Ack`**, a `Broadcast`, by a peer that has just received `S`. Having validated the chunk at `keccak256(id ‖ addr)` — which is what makes the signer's identity a check rather than a recovery, since recovering a signer always yields *some* @@ -266,12 +289,13 @@ the one address that may publish here. Then the stream becomes `max(cursor, index)`. There is **no reply**: a publisher sends its claim and its first publication back to back, and stream ordering guarantees the broker handles the claim first; a claim that did not upgrade makes the publication that follows a violation, -and the reset is the answer. An `Auth` after the `Ack` that does not verify — a chunk that -is not valid at the expected address, a payload that is not the broker's `S` and overlay, -or an `addr` that is not the admin — is a protocol violation: dropped, counted -(`invalid_claim`), the stream reset, the peer blocklisted per the node's policy; an `Auth` -sent on a publisher stream is read as a `Broadcast` and fails as one. Several streams MAY -be claimed for the admin at once — the admin from two nodes, +and the reset is the answer. A frame on a subscriber stream that is not a valid claim — an +id other than the claim id, a chunk that is not valid at the expected address, a payload +that is not a `CLAIM` with the broker's `S` and overlay, or an `addr` that is not the admin +— is a protocol violation: dropped, counted (`wrong_stream`), the stream reset, the peer +blocklisted per the node's policy; a claim sent on a publisher stream is a publication +with the wrong id and fails as one. Several streams MAY be claimed for the admin at once — +the admin from two nodes, or reconnecting before its old stream is torn down: each is a publisher stream, and the cursor arbitrates. @@ -285,9 +309,9 @@ brings its whole audience back at once. ### Frames -After the handshake a broker sends `Broadcast` and nothing else, to subscriber streams; a -publisher stream sends `Broadcast` and nothing else; a subscriber stream sends at most one -`Auth`. A delivery is the accepted publication's bytes, unchanged. +After the handshake every frame is a `Broadcast`: a broker sends deliveries, to +subscriber streams; a publisher stream sends publications; a subscriber stream sends at +most one claim. A delivery is the accepted publication's bytes, unchanged. **The frame carries no address.** Recovering a signer always yields *an* address, so the ordinary SOC validation needs an address to hold the chunk against — and every receiver @@ -300,9 +324,11 @@ afterwards. Carrying the address would say nothing the receiver does not already `keccak256(topic ‖ index)`, `index` a uint64 big-endian in 8 bytes. On the wire the 32-byte `id` slot of `soc` holds that index left-padded with 24 zero bytes, and the receiver reconstructs the signed id from the cohort's topic (the carriage of -[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). A `Broadcast` whose `id` slot -does not have 24 leading zero bytes is not a feed update: a broker drops it as invalid; a -subscriber drops it without counting it as a violation. +[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). On a publisher stream, a +`Broadcast` whose `id` slot does not have 24 leading zero bytes is not a feed update: it is +dropped and counted (`invalid_index`), but it is not a violation — it is what a +full-protocol admin's service message looks like at a BPS-lite broker — and a subscriber +drops it likewise. On a subscriber stream such a frame is read as a claim. ### Validation: the feed cursor @@ -310,11 +336,9 @@ The broker keeps one **cursor** per cohort: **the lowest index it will accept ne initially 0 — index 0 is the first update of every feed. A `Broadcast` arriving at the broker is accepted iff, in order: -1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as an - `Auth`, and if it is not a valid one it is a protocol violation: dropped, counted - (`Auth` and `Broadcast` are one shape on the wire, so the only shape test is length: - `wrong_stream` if `soc` is not 177 bytes, `invalid_claim` if it is and the claim - validation fails), the stream reset, the peer blocklisted per the node's policy; +1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as a + claim, and if it is not a valid one it is a protocol violation: dropped, counted + (`wrong_stream`), the stream reset, the peer blocklisted per the node's policy; 2. its `id` slot is a bare index `n` and **`n ≥ cursor`**; 3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as the admin's single-owner chunk**: the receiver forms the expected address @@ -333,7 +357,8 @@ benign failure — a retransmit, which an admin reconnecting after a reset legit sends when it does not know what the broker last accepted — and is counted separately; a broker MAY reset a publisher stream whose retransmit rate exceeds its policy, since the cursor check is cheap and precedes the signature check. A frame failing 2 because the -`id` slot is not a bare index is invalid. +`id` slot is not a bare index is dropped and counted (`invalid_index`) without counting +towards blocklisting, as the Frames section says. **There is no dedup window.** The cursor is total: a message is either at or beyond the next expected index or it is not, and there is no eviction and no edge. **Gaps are @@ -351,7 +376,8 @@ the spec they joined with and their own cursor), end to end, whatever the broker **A cohort ends by inactivity, and by nothing else.** A cohort on which no publisher stream has had a message accepted for the **inactivity deadline** is reclaimed: the broker resets every stream in it and forgets it. There is no end-of-stream signal in -BPS-lite — that is a service message, and this SWIP has none; an application that needs +BPS-lite — that is the admin's service message, and this SWIP has only the claim; an +application that needs "over" to be distinguishable from "paused" sends it in its last update, or waits for the service feed the family adds. @@ -404,8 +430,8 @@ this wire, they never change it**: at a BPS-lite broker, a live stream whose roster and grantees' updates are dropped as invalid; an admin that wants a roster needs a full broker; - a BPS-lite publisher and subscriber at a full broker are conformant peers of a - single-publisher live-stream cohort; the full broker's additions (service messages) - reach them as frames they drop; + single-publisher live-stream cohort; the full broker's additions (the admin's service + messages) reach them as frames they drop; - validation here is stricter, never looser: the cursor refuses out-of-order retransmits; no frame a BPS-lite broker accepts is one a full broker refuses. @@ -462,12 +488,15 @@ or declared address in SWIP-60 — so every receiver forms the expected address holds the chunk to it. That is the same check with the owner forced, and 32 bytes fewer on every frame. -**Why the claim is a chunk.** A claim has to be signed by the publisher's key and -verified by the broker; a single-owner chunk is what this protocol already signs and -verifies, with code every node has. Carrying the claim as a chunk with a domain-separated -id makes its verification one existing call, settles how it is signed (as any SOC is), -and keeps it out of the feed's id space by construction. What the chunk carries — the -challenge, the verifier's overlay, the cursor — is what the claim has to say. +**Why the claim is a chunk, and a service message.** A claim has to be signed by the +publisher's key and verified by the broker; a single-owner chunk is what this protocol +already signs and verifies, with code every node has. Carrying the claim as a chunk with a +domain-separated id makes its verification one existing call, settles how it is signed +(as any SOC is), and keeps it out of the feed's id space by construction. Giving its +payload a kind, as every service message of the family has, means no frame type of its +own and nothing told apart by shape: the id says what the chunk is, the payload says it +again. What the payload carries — the challenge, the verifier's overlay, the cursor — is +what the claim has to say. **Why no reply to a claim.** After the handshake each direction carries one frame type, and the wire has no envelope. A reply would be a second broker-to-peer type on a stream @@ -485,8 +514,9 @@ duplicate paths. One publisher on one hop has neither. The feed index is a total and "at least the next" is both the dedup rule and the memory bound — and the claim can declare it, because a signed lower bound is exactly what a reconnecting publisher knows. -**Why inactivity, and only inactivity.** An end that is *signed* by the admin is a service -message, which this SWIP does not have; an end that is *inferred* from the admin's stream +**Why inactivity, and only inactivity.** An end that is *signed* by the admin is one of the +admin's service messages, of which this SWIP has none — its one service message is the +publisher's claim; an end that is *inferred* from the admin's stream turns every transport hiccup into a stream-ending event for the whole audience. So there is no end: a cohort is a map entry that lives while it is used and is reclaimed when it is not. What that gives up — "over" versus "paused" — is the application's to carry until the @@ -545,15 +575,15 @@ An implementation is BPS-lite conformant when: `FULL` and `REJECTED` is ever sent; 3. the challenge is derived as specified from a boot secret, the canonical spec and the declared address, issued iff an address was declared, and nothing is stored for it; -4. a claim — an `Auth` in `Join`, or as a subscriber stream's next frame — is a - single-owner chunk verified by the ordinary SOC validation against - `keccak256(keccak256("bps-claim:v1" ‖ topic) ‖ addr)`, with payload `S ‖ O_B ‖ index` - matching the broker's own `S` and overlay, and `addr` the admin; the stream then - becomes a publisher stream and the cursor becomes `max(cursor, index)`; no reply is - sent; a claim in `Join` is verified before any bound, and treated as absent if it does - not verify; an `Auth` after the `Ack` that does not verify is a violation; -5. a frame on a subscriber stream that is not a valid claim is dropped, the stream reset, - the peer blocklisted; +4. a claim — in `Join`, or as a subscriber stream's next `Broadcast` — is a single-owner + chunk verified by the ordinary SOC validation against + `keccak256(keccak256("bps-claim:v1" ‖ topic) ‖ addr)`, with a payload decoding to a + `CLAIM` service message whose challenge and overlay are the broker's own, and `addr` + the admin; the stream then becomes a publisher stream and the cursor becomes + `max(cursor, index)`; no reply is sent; a claim in `Join` is verified before any bound, + and treated as absent if it does not verify; +5. a frame on a subscriber stream that is not a valid claim is dropped, counted, the + stream reset, the peer blocklisted; 6. a `Broadcast` on a publisher stream is accepted iff its bare index is at least the cursor and the chunk validates as a single-owner chunk at the address the broker forms itself, `keccak256(keccak256(topic ‖ index) ‖ admin)`; accepted frames set the cursor @@ -576,14 +606,14 @@ An implementation is BPS-lite conformant when: A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, `invalid_soc` (a chunk that does not validate at the expected address: bad signature, -wrong owner or wrong digest alike), `wrong_stream` (a subscriber-stream frame whose `soc` -is not 177 bytes), `invalid_claim` (one that is, and fails), `claim_timeout` (an extra stream +wrong owner or wrong digest alike), `wrong_stream` (a subscriber-stream frame that is not +a valid claim), `claim_timeout` (an extra stream disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. ## Out of scope (deliberately) -Service messages of any kind (roster, end of stream) and the Bee API bridge; multiple +The admin's service messages (roster, end of stream) and the Bee API bridge; multiple publishers and the claim by a rostered address; the self-indexed payload construction, gap recovery and persistence of [SWIP-65](https://github.com/ethersphere/SWIPs/pull/106); multihop ([SWIP-61](https://github.com/ethersphere/SWIPs/pull/105)); history; bandwidth From 6d1e983090c842796268313fb7f35809cba223d8 Mon Sep 17 00:00:00 2001 From: zelig Date: Thu, 1 Oct 2026 02:30:42 +0200 Subject: [PATCH 09/13] swip-74 rev 7: random per-stream challenge salts the feed topic; the first publication is the claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Join carries the spec and the peer's identity only; the Ack challenge is 24 random bytes per stream and salts the session feed, keccak256(keccak256(topic || challenge) || index), carried as challenge || index in the id slot; no Auth type, no service message, no credential in Join, no broker overlay, no transport precondition. A stream declaring the admin's address is pending — outside the fan-out bound, silent until it claims. New bound: streams per peer connection per cohort. Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 566 ++++++++++++++++++++++++----------------------- 1 file changed, 290 insertions(+), 276 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 2e2c17f4..1b97a5cd 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -19,17 +19,24 @@ as a single-owner chunk (`Auth`) and verified by the ordinary SOC code; the data address it validates against from the owner it already knows — and `wrong_owner` folds into `invalid_soc`. Rev 6: the `Auth` type goes; the claim is a `Broadcast` whose chunk payload is a service message of kind `CLAIM`, and nothing on the wire is told apart by -shape. SWIP-60 (PR #104) is the +shape. Rev 7: the challenge is drawn at random per stream, not derived — a derived one +let whoever had captured the admin's claim replay it once the admin dropped — and it +salts the feed topic for the session, so every chunk of the session is signed under it: +the first publication is the claim, there is no claim chunk, no service message, no +credential in `Join`, and `Join` carries the spec and the peer's identity only; a +stream that says it is the admin's is admitted outside the fan-out bound, silent, until +it claims. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> - **Business line**: a live stream on Swarm — one author, an audience, real time, no storage round trip and no polling: video/audio streaming, a price ticker, a game's - server-authoritative state, a log tail. The stream is a feed, so the same updates can - later be persisted and replayed from storage by anyone who missed the live run. + server-authoritative state, a log tail. The stream is a feed, so the author can + persist the same updates for anyone who missed the live run. - **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the three frames - below — one broker, direct streams, one publisher that proves its key by signing a - challenge, read-only subscribers — and nothing else: **no history, no bandwidth + below — one broker, direct streams, one publisher whose every message is signed under + a challenge the broker issued for its stream, read-only subscribers — and nothing + else: **no history, no bandwidth incentive, no roster or end-of-stream messages, no Bee API, one hop, one mode.** Done when a broker, a publisher and subscribers from independent implementations interoperate per the @@ -46,13 +53,15 @@ that anyone can subscribe to and one participant can publish on. Each channel de its own cohort — a set of nodes connected through a unique central broadcaster node, the **broker**; every node in the cohort is directly connected to the broker, hence *singlehop*. A channel is identified by its spec — a feed topic and the address of its -**admin**. Anyone joins by sending that spec, and a peer that means to publish also names -the address it will publish as; the broker answers with a **challenge** it derives from a -secret it never stores, and the publisher claims its stream by signing it. Its messages -are single-owner chunks that are the higher-index updates of the feed the topic names, -carried with their bare index; the broker accepts each one iff its index is at least the -channel's cursor and the chunk validates as the admin's single-owner chunk under the -feed's id, and delivers it unchanged to every subscriber stream — never to a +**admin**. Anyone joins by sending that spec and its own address; the broker answers +with a **challenge** drawn at random for that stream, and for the life of the stream the +channel's feed topic is salted with it: a publisher's messages are single-owner chunks +that are the higher-index updates of that session feed — the feed on +`keccak256(topic ‖ challenge)` — carried with the challenge and their bare index, and the +first of them is the publisher's claim on the stream. The +broker accepts each one iff it carries the stream's challenge, its index is at least the +channel's cursor, and the chunk validates as the admin's single-owner chunk under the +session feed's id, and delivers it unchanged to every subscriber stream — never to a publisher's. All other joiners are subscribers and verify the same way, end to end: the broker can withhold, never forge. A channel ends when it goes idle. @@ -91,8 +100,8 @@ subscribers**; they are **feed updates, in increasing index order**. - **Admin = the publisher**: the address named in the cohort spec. It is the only identity whose messages are accepted. It need not be first to join; the cohort does not end when its stream goes away; and it may publish from any node, or from two. -- **Subscriber**: joins by naming the cohort, receives, never publishes. A publication - from a subscriber stream is a protocol violation. +- **Subscriber**: joins by naming the cohort and its own address, receives, never + publishes. A publication from a subscriber stream is a protocol violation. - **Publisher and broker are distinct nodes**; brokering for oneself is out of scope. ### The cohort spec @@ -102,7 +111,7 @@ Three fields, no others: | field | value | |---|---| | `topic` | 32 bytes: the feed topic | -| `binding` | `FEED_TOPIC`: the signed id of every message is `keccak256(topic ‖ index)` | +| `binding` | `FEED_TOPIC`: the stream is a feed on the topic — on the wire, the session feed on `topic_s = keccak256(topic ‖ challenge)`, whose signed id is `keccak256(topic_s ‖ index)` | | `admin` | 20-byte address: the publisher | **The spec is the cohort's identity.** It has a **canonical serialisation** — @@ -129,7 +138,9 @@ package bps; // What the topic binds to. One binding. enum TopicBinding { TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire - FEED_TOPIC = 4; // signed id = keccak256(topic || index) + FEED_TOPIC = 4; // the stream is a feed on the topic; on the wire the + // session feed on keccak256(topic || challenge): + // id = keccak256(keccak256(topic || challenge) || index) } // The cohort's identity. Fixed by whoever joins first; immutable; keyed by its @@ -140,46 +151,15 @@ message CohortSpec { bytes admin = 3; // 20 bytes, required } -// What a chunk on this wire says when it is not a feed update. BPS-lite has one kind, -// the claim; SWIP-60 adds the admin's roster and end of stream. Every such chunk is -// recognised by its id and by this kind in its payload — never by its shape. -enum ServiceKind { - SERVICE_KIND_UNSPECIFIED = 0; // invalid on the wire - CLAIM = 1; -} - -// The payload of a service chunk. -message ServiceMessage { - ServiceKind kind = 1; - uint64 index = 2; // CLAIM: the publisher's cursor — its next message has an - // index >= this (?) - bytes challenge = 3; // CLAIM: S, as received in the Ack - bytes overlay = 4; // CLAIM: O_B, the overlay of the broker the claiming node - // is connected to -} - -// A publisher's claim on the stream it is sent on is a Broadcast whose chunk is a -// single-owner chunk it signs as any SOC, with -// id = keccak256("bps-claim:v1" || topic) never a feed id: a 44-byte preimage -// against a feed id's 40 -// owner = addr so address = keccak256(id || addr) -// payload = ServiceMessage{CLAIM, index, S, O_B} -// The id slot carries the full claim id (so it never has 24 leading zero bytes and is -// never read as a feed update); span is the payload's length, as for any SOC. The -// receiver derives the id from the topic, requires the slot to equal it, forms the -// expected address from `addr`, validates the chunk against it in one existing call, -// decodes the payload, and checks the challenge and overlay against its own. Sent -// inside Join by a peer that already holds S, or as the next Broadcast after Ack by one -// that has just received it. - // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live -// cohort has this spec, attaches to it otherwise. +// cohort has this spec, attaches to it otherwise. Nothing else is ever in it — no +// cursor, no credential: the stream's role follows from what it sends after the Ack. message Join { CohortSpec cohort = 1; - bytes addr = 2; // 20 bytes: the address this stream will publish as; absent: - // a subscriber, and no challenge is issued - bytes auth = 3; // a returning publisher's claim: the claim chunk's data, - // verified before any bound + bytes addr = 2; // 20 bytes, required: the stream's identity — the address it + // publishes as if it publishes, which only the admin's may; + // a stream declaring the admin's is admitted outside the + // fan-out bound and receives nothing until it claims } enum Status { @@ -192,34 +172,41 @@ enum Status { // Broker -> peer, answering Join. A non-OK Ack ends the stream. message Ack { Status status = 1; - bytes challenge = 2; // S, iff status == OK and addr was declared + bytes challenge = 2; // iff OK: exactly 24 bytes drawn at random for this stream, + // never all zero — the salt of the session feed; held for the + // stream's life, never persisted, never reused } -// Every frame after the handshake: publisher -> broker a publication or a claim, -// broker -> subscriber a delivery of the same bytes. The single-owner chunk travels as -// its chunk data, opaque to the protocol; the receiver forms the address it must have — -// keccak256(id || owner), with the owner it already knows — and validates the chunk -// against it with the ordinary SOC code, once the id slot has been rewritten as the -// Frames section says for a feed update: +// Every frame after the handshake: publisher -> broker a publication, broker -> +// subscriber a delivery of the same bytes. The single-owner chunk travels as its chunk +// data, opaque to the protocol: // soc = id (32) || signature (65) || span (8, LE) || payload (<= 4096) +// with the id slot carrying challenge (24) || index (8, BE). The receiver reconstructs +// the signed id as keccak256(keccak256(topic || challenge) || index), forms the address +// the chunk must have — keccak256(id || admin) — and validates the chunk against it +// with the ordinary SOC code. A stream's first publication under its challenge is its +// claim: the stream is a publisher stream from the moment a frame of its own validates +// under its challenge as the admin's. message Broadcast { bytes soc = 1; } ``` -Three frames — `Join`, `Ack`, `Broadcast` — and the two types they carry, `CohortSpec` -and, as a chunk payload, `ServiceMessage`. There is no envelope: **what a frame is -follows from the stream's direction and role, and what a chunk is from its id and its -payload's kind.** The first peer-to-broker frame is a `Join` and the first broker-to-peer -frame an `Ack`; after that every frame is a `Broadcast`: a subscriber stream may send -only a claim, a publisher stream only publications, the broker only deliveries. A frame +Three frames — `Join`, `Ack`, `Broadcast` — and the one type they carry, `CohortSpec`. +There is no envelope: **what a frame is follows from the stream's direction and role.** +The first peer-to-broker frame is a `Join` and the first broker-to-peer frame an `Ack`; +after that every frame is a `Broadcast`: a publisher stream sends publications, the +broker sends deliveries, and a stream that has not yet published sends its claim — its +first publication under its challenge — and is a publisher stream or is reset. A frame that is not what its stream may send is invalid. -### Handshake: `Join`, the challenge, the claim +### Handshake: `Join`, the challenge, the session feed, the claim Every peer sends `Join` as the first frame on a fresh stream, carrying the full -`CohortSpec`; a peer that means to publish also declares the address it will publish as, -in `addr` — an assertion, proved only by the signature that follows. +`CohortSpec` and its identity, `addr`: the address the stream publishes as, if it +publishes — an assertion, proved only by the signatures that follow. Nothing else is in +it: no cursor, and no credential, because a credential that could be presented in a +`Join` could be presented by whoever captured it. The broker compares the spec against its live cohorts: **no match → the cohort is created** with the joiner attached; **match → the joiner is attached** to it. Anyone may @@ -227,80 +214,74 @@ create, including a subscriber arriving before the admin: a publisher-less cohor the broker one map entry until the inactivity deadline reclaims it, and can accept no message. There is nothing to pre-register and no "unknown topic". -**The challenge.** For a `Join` that declares `addr`, the broker derives - -``` -S_C = a secret drawn once at broker boot, never persisted -S_s = keccak256(Marshal(spec)) the cohort's key -S_c = keccak256(S_C ‖ S_s) the cohort's secret -S = keccak256(S_C ‖ S_c ‖ addr) the challenge for addr on this cohort -``` - -and answers `Ack{OK, S}`. The broker stores nothing: it recomputes `S` from its boot -secret, the spec and the address whenever a claim arrives. `S` is therefore the same for -an address on a cohort for as long as the broker runs, from whichever node the address -joins, and differs at every other broker and after every restart. It goes to the joiner -over the encrypted stream and is useful only to the key of `addr`; anyone may obtain it -by declaring the address, and gains nothing by it. +**The challenge.** With `Ack{OK}` the broker sends a **challenge**: 24 bytes it draws at +random for this stream, holds for the stream's life, and never persists or reuses — on +every stream, whatever its `addr`. It is not a secret: it will be in the clear in every +chunk of the session. It is a **freshness mark**: it exists on this stream only, and +nothing signed for another stream, another cohort, another broker or an earlier session +of the same peer carries it. Anyone may obtain one by joining, and gains nothing by it. +It is exactly 24 bytes and never all zero; a peer treats any other length as a protocol +error. A subscriber's challenge is unused in BPS-lite — it is issued so that the family +can upgrade the same stream later — and is never compared with anything. -**The claim.** A publisher claims its stream with a `Broadcast` whose chunk is a -single-owner chunk it signs with the key of `addr`, exactly as it signs any SOC, whose +**The session feed.** For the life of the stream the publisher does not write the +channel's feed but the **session's**: the feed on the channel's topic salted with the +challenge, ``` -id = keccak256("bps-claim:v1" ‖ topic) -owner = addr address = keccak256(id ‖ addr) -payload = ServiceMessage{kind: CLAIM, index, challenge: S, overlay: O_B} +topic_s = keccak256(topic ‖ challenge) the session topic +id = keccak256(topic_s ‖ index) topic 32, challenge 24, index 8 bytes ``` -where `O_B` is the overlay of the broker the claiming node is connected to and `index` -is the publisher's cursor: the claim that its next message will have a feed index of at -least `index`. The chunk's `id` slot carries the full claim id — so it never has 24 -leading zero bytes and is never read as a feed update — and its span is the payload's -length, as for any SOC. The receiver verifies it with the ordinary SOC validation plus one -payload check: it derives the id from the topic, requires the slot to equal it, forms the -expected address `keccak256(id ‖ addr)` from the declared address, and validates the chunk -against it — signature, digest and signer in one existing call — then decodes the payload -and checks that its kind is -`CLAIM`, its challenge the `S` it derives for `addr`, and its overlay its own. Nothing is -told apart by shape: the id says what the chunk is, and the kind in the payload says it -again, as it does for every service message of the family. The separator in the id keeps -a claim from ever being a feed update: a feed id's preimage is `topic ‖ index`, 40 bytes, -a claim id's is 44, so neither ever hashes to the other's address. `S` binds the claim to -this broker, this cohort and this address; `O_B` binds it to the verifier, which `S` -cannot, because `S` is opaque to the signer (see Security considerations); `index` is -signed so that a replayed claim moves no cursor. A claim is a storable chunk, at an -address nobody reads as content. It is sent in one of two places: - -- **in `Join`**, by a peer that already holds `S` for this address at this broker — a - reconnecting admin, from the same node or another: the broker verifies it before any - bound is applied, and a valid claim makes the stream a publisher stream from its first - frame. A claim in the `Join` that does not verify — stale because the broker has - restarted and `S` has changed, or simply wrong, or signed by a key other than the - `Join`'s `addr`: the broker treats them alike — is treated as absent: `Ack{OK, S}` with - the current `S`, no penalty. The peer sees that its claim was not taken when the `S` in - the `Ack` differs from the one it signed, and claims again after the `Ack`; -- **as the next frame after `Ack`**, a `Broadcast`, by a peer that has just received `S`. - -Having validated the chunk at `keccak256(id ‖ addr)` — which is what makes the signer's -identity a check rather than a recovery, since recovering a signer always yields *some* -address — and matched the payload, the broker checks that `addr` is the cohort's admin, -the one address that may publish here. Then the stream -**upgrades** to a **publisher stream**: it leaves the fan-out set, and the cohort's cursor -becomes `max(cursor, index)`. There is **no reply**: a publisher sends its claim and its -first publication back to back, and stream ordering guarantees the broker handles the -claim first; a claim that did not upgrade makes the publication that follows a violation, -and the reset is the answer. A frame on a subscriber stream that is not a valid claim — an -id other than the claim id, a chunk that is not valid at the expected address, a payload -that is not a `CLAIM` with the broker's `S` and overlay, or an `addr` that is not the admin -— is a protocol violation: dropped, counted (`wrong_stream`), the stream reset, the peer -blocklisted per the node's policy; a claim sent on a publisher stream is a publication -with the wrong id and fails as one. Several streams MAY be claimed for the admin at once — -the admin from two nodes, -or reconnecting before its old stream is torn down: each is a publisher stream, and the +— an ordinary feed, addressable as `(admin, topic_s)` by ordinary feed tooling. The +chunk's 32-byte `id` slot carries `challenge ‖ index` — the challenge followed by the +index as a uint64 big-endian — so that every receiver that knows the topic, subscriber +or broker, reconstructs `topic_s` and the signed id from the slot without being told +anything else, and the broker additionally requires the slot to begin with the challenge +it issued on that stream. The salt is what makes a session chunk worthless anywhere but +on the stream it was signed for; the index in the clear is what the cursor and SWIP-65's +gap detection read. The publisher signs each update once per stream it holds, and MUST +NOT reuse an index across sessions: a subscriber cannot tell sessions apart, and its +cursor per `(topic, admin)` is what keeps an earlier session's update from standing in +for a later one (see Security considerations). The same wrapped payload signed again +under the channel's own id, `keccak256(topic ‖ index)`, is the update for storage (see +Rationale). + +**The pending stream.** A `Join` that declares the admin's `addr` is admitted outside +the fan-out bound, as a **pending stream**: attached, in no fan-out set, receiving +nothing, until it claims or the **claim deadline** disconnects it (*Resource bounds*). +That is what keeps a full audience, or an old stream of the admin's that the transport +has not yet reaped, from locking the admin out, and it is worth nothing to anyone but +the admin: a pending stream costs the broker a map entry and a challenge for one +deadline and delivers nothing. Every other `Join` is a **subscriber stream**, within the +bound. + +**The claim.** A pending stream claims the publisher role with its **first `Broadcast` +under its challenge** — a publication like every one that follows. A frame on it whose +slot does not begin with the stream's challenge is not of this session: it is dropped +and counted (`wrong_challenge`), not a violation — it is what an update the admin's node +signed for its previous stream and still had queued looks like, and what a +full-protocol admin's roster looks like here — and a publisher SHOULD discard or re-sign +what it had queued for a stream that was reset, since nothing signed for the old +challenge will be accepted. For a frame that does carry the challenge the broker checks +that with `id = keccak256(topic_s ‖ index)` the chunk validates as the admin's +single-owner chunk at `keccak256(id ‖ admin)`: signature, digest and signer in one +existing call, the owner forced rather than recovered, since recovering a signer always +yields *some* address. If it does, the stream **upgrades** to a **publisher stream** and +the frame is delivered, or counted as a retransmit, exactly as any publication +(*Validation* below). A signature under this challenge is possible only for the key of +`addr` and only after the `Ack`, so the first valid publication proves the key and the +session at once, and nothing else has to. There is **no reply**: the publisher sends its +first publication and the next back to back, and learns the outcome from whether the +stream survives. If it does not validate, it is a protocol violation: dropped, counted +(`wrong_stream`), the stream reset, the peer blocklisted per the node's policy — a chunk +under this challenge from a key that is not the admin's is nothing a conforming peer +sends; and so is any frame at all from a subscriber stream, whatever its slot. Several +streams MAY be claimed for the admin at once — the admin from two nodes, or +reconnecting before its old stream is torn down: each has its own challenge, and the cursor arbitrates. -**The rest of the handshake.** `Ack{OK}` without a challenge answers a `Join` that -declared no address: the stream is a **subscriber stream**. `FULL` answers a `Join` at a +**The rest of the handshake.** `FULL` answers a `Join` at a capacity bound, `REJECTED` one whose spec has a value outside this SWIP. A non-`OK` `Ack` ends the stream; on `OK` it is retained. A peer whose stream was refused or reset MUST back off before rejoining (randomised exponential, 1 s to 30 s, full jitter); a broker @@ -310,25 +291,35 @@ brings its whole audience back at once. ### Frames After the handshake every frame is a `Broadcast`: a broker sends deliveries, to -subscriber streams; a publisher stream sends publications; a subscriber stream sends at -most one claim. A delivery is the accepted publication's bytes, unchanged. +subscriber streams; a publisher stream sends publications; a pending stream sends its +claim. A delivery is the accepted publication's bytes, unchanged. **The frame carries no address.** Recovering a signer always yields *an* address, so the ordinary SOC validation needs an address to hold the chunk against — and every receiver has one: in BPS-lite the publisher is the admin, so the expected address of an update is -`keccak256(keccak256(topic ‖ index) ‖ admin)`, and validating the chunk against it checks +`keccak256(keccak256(topic_s ‖ index) ‖ admin)`, and validating the chunk against it checks signature, digest and owner in one call, with the owner forced rather than compared afterwards. Carrying the address would say nothing the receiver does not already know. -**The `id` slot carries the bare index.** A feed update's signed id is -`keccak256(topic ‖ index)`, `index` a uint64 big-endian in 8 bytes. On the wire the -32-byte `id` slot of `soc` holds that index left-padded with 24 zero bytes, and the -receiver reconstructs the signed id from the cohort's topic (the carriage of -[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106)). On a publisher stream, a -`Broadcast` whose `id` slot does not have 24 leading zero bytes is not a feed update: it is -dropped and counted (`invalid_index`), but it is not a violation — it is what a -full-protocol admin's service message looks like at a BPS-lite broker — and a subscriber -drops it likewise. On a subscriber stream such a frame is read as a claim. +**The `id` slot carries the challenge and the bare index.** A publication's signed id is +`keccak256(topic_s ‖ index)`, `topic_s = keccak256(topic ‖ challenge)`, `index` a uint64 +big-endian in 8 bytes. On the wire the 32-byte `id` slot of `soc` holds +`challenge ‖ index`, and the receiver reconstructs the session topic and the signed id +from the cohort's topic and the slot. This is the carriage of +[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106), salted: under SWIP-65 the slot +holds the index left-padded with 24 zero bytes and the receiver prepends the topic — +that is the channel's own feed; here the 24 bytes are the challenge — that is the +session's. A slot beginning with 24 zero bytes therefore names the channel's own feed: +a broker never issues an all-zero challenge, never accepts such a frame on any stream, +and never sends one in BPS-lite — stored updates reach a subscriber through the +family's history delivery (SWIP-60), verified under the channel's own id. On a publisher +or pending stream, a `Broadcast` whose `id` slot does not begin with the stream's +challenge is not a chunk of this session: it is dropped and counted (`wrong_challenge`), +but it is not a violation — it is what an update the admin signed for an earlier session +and resent after reconnecting looks like, and what a full-protocol admin's service +message looks like at a BPS-lite broker. A subscriber does not check the salt — that was +the broker's check, and a subscriber does not know which stream a delivery came from — +and reads the index from the slot. ### Validation: the feed cursor @@ -336,84 +327,94 @@ The broker keeps one **cursor** per cohort: **the lowest index it will accept ne initially 0 — index 0 is the first update of every feed. A `Broadcast` arriving at the broker is accepted iff, in order: -1. it arrived on a **publisher stream** — on a subscriber stream the frame is read as a - claim, and if it is not a valid one it is a protocol violation: dropped, counted - (`wrong_stream`), the stream reset, the peer blocklisted per the node's policy; -2. its `id` slot is a bare index `n` and **`n ≥ cursor`**; -3. with the `id` slot rewritten to `keccak256(topic ‖ n)` the chunk **validates as the - admin's single-owner chunk**: the receiver forms the expected address - `keccak256(id ‖ admin)` and runs the ordinary SOC validation against it — the wrapped - chunk's BMT address matches `span ‖ payload`, and the signature over - `id ‖ wrappedAddress` recovers an owner that hashes with the id to that address, which - only the admin's does. +1. it arrived on a **publisher stream** — or on a **pending stream**, as its claim: a + frame there that passes 2 and 4 upgrades the stream, one that fails 2 is dropped and + counted (`wrong_challenge`), one that passes 2 and fails 4 is a protocol violation: + dropped, counted (`wrong_stream`), the stream reset, the peer blocklisted per the + node's policy — as is any frame from a subscriber stream; +2. its `id` slot begins with **the stream's challenge**; +3. the index `n` in the slot's last 8 bytes is **`n ≥ cursor`**; +4. with `id = keccak256(topic_s ‖ n)`, `topic_s = keccak256(topic ‖ challenge)`, the chunk + **validates as the admin's single-owner chunk**: the receiver forms the expected + address `keccak256(id ‖ admin)` and runs the ordinary SOC validation against it — the + wrapped chunk's BMT address matches `span ‖ payload`, and the signature over + `id ‖ wrappedAddress` recovers an owner that hashes with the id to that address, + which only the admin's does. On acceptance the broker sets `cursor := n + 1` and enqueues the frame on **every subscriber stream** in the cohort. Publisher streams receive nothing: a publisher never gets its own messages back, on whichever of its streams they were sent. -A failure of 3 is an invalid frame: dropped and counted; repeated invalid frames end -the connection (blocklisting policy). A frame failing 2 with `n < cursor` is the one -benign failure — a retransmit, which an admin reconnecting after a reset legitimately -sends when it does not know what the broker last accepted — and is counted separately; a -broker MAY reset a publisher stream whose retransmit rate exceeds its policy, since the -cursor check is cheap and precedes the signature check. A frame failing 2 because the -`id` slot is not a bare index is dropped and counted (`invalid_index`) without counting -towards blocklisting, as the Frames section says. +A failure of 4 is an invalid frame: dropped and counted (`invalid_soc`); repeated invalid +frames end the connection (blocklisting policy). A frame failing 3 is a benign +failure — a retransmit, which an admin reconnecting after a reset legitimately sends when +it does not know what the broker last accepted — and is counted separately; a broker MAY +reset a publisher stream whose retransmit rate exceeds its policy. A frame failing 2 on a +publisher or pending stream is dropped and counted (`wrong_challenge`) without counting +towards blocklisting, as the Frames section says. The two cheap checks precede the +signature check. **There is no dedup window.** The cursor is total: a message is either at or beyond the next expected index or it is not, and there is no eviction and no edge. **Gaps are allowed** (`n > cursor`): they are the publisher's business, and SWIP-65 makes them -detectable and recoverable at the subscriber. A claim's `index` moves the cursor forward -to what the admin has published elsewhere, never back. This is stricter than SWIP-65's -general carriage rule, under which a broker does not enforce monotonicity; the restriction -is sound here because one publisher on one hop admits no legitimate reordering. - -Subscribers re-verify every delivery exactly as the broker does (steps 2 and 3, against -the spec they joined with and their own cursor), end to end, whatever the broker did. +detectable and recoverable at the subscriber. A returning admin that continues where it +left off sets the cursor with its first accepted update, never back. This is stricter +than SWIP-65's general carriage rule, under which a broker does not enforce +monotonicity; the restriction is sound here because one publisher on one hop admits no +legitimate reordering. + +Subscribers re-verify every delivery exactly as the broker does (steps 3 and 4, against +the spec they joined with and their own cursor, the index and the session topic read +from the slot — never step 2: the challenge in the slot was the broker's check, and the +subscriber's own challenge is compared with nothing), end to end, whatever the broker +did. ### Lifetime: inactivity **A cohort ends by inactivity, and by nothing else.** A cohort on which no publisher stream has had a message accepted for the **inactivity deadline** is reclaimed: the broker resets every stream in it and forgets it. There is no end-of-stream signal in -BPS-lite — that is the admin's service message, and this SWIP has only the claim; an +BPS-lite — that is one of the admin's service messages, of which this SWIP has none; an application that needs "over" to be distinguishable from "paused" sends it in its last update, or waits for the service feed the family adds. A publisher stream going away — closed or reset — is therefore not an end: the cohort -and its cursor persist, subscribers stay attached, and the admin rejoins, claiming in its -`Join` with the `S` it holds and the cursor it knows. Likewise a subscriber-only cohort -waits for its admin until the deadline reclaims it. A cohort with **no attached streams** -MAY be reclaimed at once — nothing observes the difference beyond a fresh cursor on the -next `Join`. +and its cursor persist, subscribers stay attached, and the admin rejoins — admitted as a +pending stream whether or not its old one is still held — receives a fresh challenge, +and claims with its first update under it, continuing at whatever index it is at. Likewise a subscriber-only cohort waits for its admin until the deadline +reclaims it. A cohort with **no attached streams** MAY be reclaimed at once — nothing +observes the difference beyond a fresh cursor on the next `Join`. -After a cohort is forgotten a later `Join` creates a fresh one with the cursor at 0 — -and the same `S`, since the broker derives it, so the admin's claim still holds. -Subscribers keep their own cursor across that, so a stale republication is caught at the -edge, not at the broker. +After a cohort is forgotten a later `Join` creates a fresh one with the cursor at 0, and +every stream on it a fresh challenge: nothing the admin signed for the old session is +accepted on the new, so a stale republication is caught at the broker as well as at the +edge, where subscribers keep their own cursor per `(topic, admin)`. ### Resource bounds -All broker policy, none on the wire. The first five bounds are REQUIRED, with the values +All broker policy, none on the wire. The first six bounds are REQUIRED, with the values given RECOMMENDED where a value is given; the last two are MAY: | bound | answer | recommended | |---|---|---| -| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except **one extra stream while the admin is absent**: a `Join` declaring the admin's `addr` when no publisher stream exists is admitted over the bound, and disconnected, with a short blocklist and counted (`claim_timeout`), if it has not claimed within the **claim deadline** | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | +| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the admin's `addr`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its first publication under its challenge upgrades it, or the **claim deadline** disconnects it, with a short blocklist, counted (`claim_timeout`) | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | | live cohorts per broker | `FULL` to a cohort-creating `Join` | implementation-defined | | **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | +| **streams per peer connection per cohort** — one connection cannot fill a cohort's fan-out set with declared identities | `FULL` | 2 (?) | | **inactivity deadline** — reclaims a cohort, see *Lifetime* | reset | 10 min | | outbound queue per subscriber stream, one writer; a full queue resets that stream | reset | 64 frames | | a peer connection with no live stream | MAY be disconnected at the transport | — | | a cohort with no attached streams | MAY be reclaimed at once | — | Any peer can make a broker allocate a cohort simply by joining, which is why the second, -third and fourth bounds exist together: a cohort costs a map entry, a peer can hold only -a bounded number of them, and none survives idleness. The extra stream is what keeps an -audience that fills the cohort while the admin is away from locking the admin out: an -unclaimed admin is a fan-out channel like any other, and its slot exists exactly while it -is absent; a returning admin that claims in its `Join` never needs it. The broker has no +third and fifth bounds exist together: a cohort costs a map entry, a peer can hold only +a bounded number of them, and none survives idleness. The pending stream is what keeps +an audience that fills the cohort while the admin is away — or an old stream of the +admin's that the transport has not yet reaped — from locking the admin out, and it is +not a slot anyone can squat: it receives nothing, costs a map entry and a challenge for +one claim deadline, and one connection holds a bounded number of streams per cohort. +Declaring the admin's address buys exactly that. The broker has no delivery obligation: a reset for a full queue is a recoverable liveness fault, and blocking fan-out on one slow subscriber would punish the cohort. @@ -427,8 +428,9 @@ this wire, they never change it**: accepts only what this SWIP defines: a spec value outside this SWIP is `REJECTED`, a field this SWIP does not define is ignored, so a fuller spec is served as the lite spec its known fields spell — a feed-topic cohort whose admin later publishes a roster is, - at a BPS-lite broker, a live stream whose roster and grantees' updates are dropped as - invalid; an admin that wants a roster needs a full broker; + at a BPS-lite broker, a live stream whose roster is dropped (`wrong_challenge`) and + whose grantees' first updates are violations, their streams reset; an admin that wants + a roster needs a full broker; - a BPS-lite publisher and subscriber at a full broker are conformant peers of a single-publisher live-stream cohort; the full broker's additions (the admin's service messages) reach them as frames they drop; @@ -436,10 +438,9 @@ this wire, they never change it**: retransmits; no frame a BPS-lite broker accepts is one a full broker refuses. What SWIP-60 adds on this wire: cohort parameters (`publishers`, `history`, `closed`), -the admin's service feed (roster, end of stream) as `Broadcast` frames, a claim by any -rostered address and none at all under `ALL`, one extra stream per absent rostered -publisher, and the Bee API. What it presumes of the transport, this SWIP states below as -a precondition. +the admin's service feed (roster, end of stream) as `Broadcast` frames whose payload is a +service message, the claim by any rostered address and none at all under `ALL`, pending +streams for rostered addresses, and the Bee API. ## Rationale @@ -465,20 +466,44 @@ publish only what its owner already signed is true and beside the point: replayi admin's signed updates is *history*, and a first-time viewer receiving them from the beginning is caught up, not deceived. What a replayed signature buys is an **identity**: a stream the broker treats as the admin's — exempt from the fan-out bound and its queue -policy, entitled to the extra slot, and in SWIP-60 admitted to a `closed` cohort and -promoted by a roster. A challenge only this broker could have issued, for this address, -signed together with the verifier's overlay, is what makes the identity worth exactly -the key. Deriving it from a boot secret and the spec is what lets the broker issue it -without storing it, and lets an address keep its claim from any node and across -reclaims. +policy, and in SWIP-60 admitted to a `closed` cohort and promoted by a roster. A challenge issued for this stream and answered by a signature +under it is what makes the identity worth exactly the key. + +**Why the challenge is random, per stream.** A challenge derived from a broker secret +and the address is issued without being stored, and it is the same on every stream the +address opens while the broker runs — which is the flaw: whoever once captured the +admin's answer to it (the node that bridged the admin, say) can present it again after +the admin drops, and holds the admin's identity at this broker until it restarts. A +challenge drawn at random for the stream is answered on that stream or nowhere; the +broker keeps it as long as the stream, which it keeps anyway, and nothing survives the +stream to be replayed. + +**Why the challenge salts the feed, and the first publication is the claim.** A +challenge answered once, by a claim, protects the claim; the updates that follow are +signed for the channel, and whoever captured them from one session can present them in +another — to a broker whose cursor has restarted after a reclaim, they are fresh. Salting +the topic with the challenge signs every update for the session it was published in, so +there is nothing to replay: an update carries the challenge in the clear in its `id` +slot, the broker requires its own, and a chunk of another session, another broker or the +channel's own feed fails that check before any signature is looked at. Once every update +answers the challenge, the first one is the claim, and a claim that is anything else — a +chunk with a kind, a frame of its own, a credential in `Join` — is a second thing to +specify, sign, verify and replay. Salting the *topic* — `keccak256(topic ‖ challenge)` +is a topic like any other — keeps the session feed an ordinary feed, so that a +subscriber that saves the run has a feed it can replay with ordinary tooling, under a +topic every chunk of it names. What it costs: the publisher signs once per stream it +holds, and the live chunks are the session feed's, not the channel's — a publisher that +wants the run persisted under the channel's own topic signs the same wrapped payloads +again under `keccak256(topic ‖ index)`, which is one signature each over a chunk it +already has (?). **Why the identity is on the stream, and the address is declared.** A peer connection is a node; a stream is a (node, cohort) pair; the identity that matters is the key that -signs the messages on that stream. Declaring the address in `Join` lets the broker derive -the challenge for it, and gives the extra slot only to a stream that says it is the -admin's. Binding the identity to the stream is what lets one node carry different -identities on different streams — and, in SWIP-60, what lets a subscriber stream be -upgraded by a claim when the admin grants it. +signs the messages on that stream. Declaring the address in `Join` admits as pending +only a stream that says it is the admin's, and tells the broker which address to hold +the stream's first publication to. Binding the identity to the stream is what lets one +node carry different identities on different streams — and, in SWIP-60, what lets a +subscriber stream be upgraded by its first publication when the admin grants it. **Why the frame carries no address.** The ordinary SOC validation checks that the signer hashes, with the id, to the chunk's address; a receiver that only recovered a signer @@ -488,21 +513,11 @@ or declared address in SWIP-60 — so every receiver forms the expected address holds the chunk to it. That is the same check with the owner forced, and 32 bytes fewer on every frame. -**Why the claim is a chunk, and a service message.** A claim has to be signed by the -publisher's key and verified by the broker; a single-owner chunk is what this protocol -already signs and verifies, with code every node has. Carrying the claim as a chunk with a -domain-separated id makes its verification one existing call, settles how it is signed -(as any SOC is), and keeps it out of the feed's id space by construction. Giving its -payload a kind, as every service message of the family has, means no frame type of its -own and nothing told apart by shape: the id says what the chunk is, the payload says it -again. What the payload carries — the challenge, the verifier's overlay, the cursor — is -what the claim has to say. - **Why no reply to a claim.** After the handshake each direction carries one frame type, and the wire has no envelope. A reply would be a second broker-to-peer type on a stream that carries deliveries, with nothing to tell them apart. It is also unnecessary: streams -are ordered, so a publisher sends its claim and its first publication together and learns -the outcome from whether the stream survives. +are ordered, so a publisher sends its first publication and its second together and +learns the outcome from whether the stream survives. **Why no loopback.** A publisher knows what it published. Echoing it costs a frame per message on the stream whose back-pressure matters most, and buys no confirmation: a @@ -511,12 +526,11 @@ rejected frame produces no echo either. **Why the cursor, not a window.** A bounded seen-set is a memory bound with a replay hole at its edge, and it exists because the general protocol admits many publishers and duplicate paths. One publisher on one hop has neither. The feed index is a total order, -and "at least the next" is both the dedup rule and the memory bound — and the claim can -declare it, because a signed lower bound is exactly what a reconnecting publisher knows. +and "at least the next" is both the dedup rule and the memory bound. **Why inactivity, and only inactivity.** An end that is *signed* by the admin is one of the -admin's service messages, of which this SWIP has none — its one service message is the -publisher's claim; an end that is *inferred* from the admin's stream +admin's service messages, of which this SWIP has none; an end that is *inferred* from the +admin's stream turns every transport hiccup into a stream-ending event for the whole audience. So there is no end: a cohort is a map entry that lives while it is used and is reclaimed when it is not. What that gives up — "over" versus "paused" — is the application's to carry until the @@ -527,42 +541,34 @@ service feed arrives. Those of SWIP-60, restricted to its live-stream configuration (admin set, the admin alone publishes, open audience). -**The publisher role takes the key, every time.** A claim is a chunk signed by the -publisher's key over a challenge that only this broker could have issued for this -address, together with the verifier's overlay and the publisher's cursor. What can be -captured and replayed, by whom, and what stops it: +**The publisher role takes the key, every time, and the session takes it again.** Every +update, the first included, is a chunk signed by the publisher's key under a challenge +that exists on one stream. What can be captured and replayed, by whom, and what stops +it: | replayed | by whom | where | result | stopped by | |---|---|---|---|---| -| the claim chunk | a third party or another subscriber | anywhere | cannot obtain it | it travels on the encrypted stream to the broker and nowhere else | -| the claim chunk | the node that bridged the admin (holds it, not the key) | this broker, while it runs, from any node | accepted — it upgrades | nothing, by design: the identity continues from another node; that node held the admin's stream anyway and can publish only what the key signed | -| the claim chunk | the same node | this broker after a restart, or another broker, or another cohort here | refused: the chunk is valid from the right key, but its payload is not this broker's `S` and overlay — on another topic the id, hence the expected address, differs as well; in a `Join` it is treated as absent and a fresh `S` issued, after an `Ack` it is a violation | `S_C` drawn at boot and never persisted; `O_B` names the verifier; the spec is in `S` and the topic in the id | -| the claim chunk | anyone | for another address | refused | the chunk does not validate at `keccak256(id ‖ addr)` for any other `addr`, and the address is in `S` | -| the claim with a changed `index` | anyone holding one | anywhere | refused | `index` is in the signed payload | -| the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it fetches `S` from the honest broker, hands it over, forwards the signature | the honest broker | refused | the publisher signs the overlay it is talking to, the relay's, and the honest broker checks its own | -| the claim chunk as a feed update, or a feed update as a claim | anyone | anywhere | refused | a claim's id is `keccak256("bps-claim:v1" ‖ topic)`, a feed update's `keccak256(topic ‖ index)`: different preimage lengths, never the same address | -| the challenge | anyone, by declaring the admin's address | this broker | harmless: `S` proves nothing without the key; the declaration buys the extra slot until the claim deadline, then a blocklist | the claim, not `S`, is the credential | -| the admin's publications | a subscriber or third party | this broker, cohort alive | violation: reset, blocklist | a publication is accepted on a publisher stream only, and that takes the key | -| the admin's publications | the admin's own node, or the admin | this broker, cohort alive | dropped as retransmits, counted | the cursor | -| the admin's publications | the admin's node, or a broker that carried the cohort | after a reclaim, or at another broker, to first-time viewers | delivered — **history**, genuine updates in order, a late viewer caught up | nothing needed; returning viewers keep their cursor per `(topic, admin)`, first-time freshness is the payload's (SWIP-65) | +| a session's updates, the first included | anyone who received them — every subscriber, the node that bridged the admin, a relay, the broker itself | any other stream: another node, the same node reconnecting, this broker after a restart or a reclaim, another broker, another cohort | refused before any signature is checked — dropped and counted on a stream declaring the admin's address, a violation on any other | the challenge in the `id` slot: the receiving stream's differs, and an update is signed for its session's feed | +| a session's updates, at an index the publisher reused in a later session | a broker that carried the earlier session | a subscriber, in place of the later session's update | delivered: the subscriber cannot tell sessions apart | the publisher never reusing an index; then the earlier update is below every returning subscriber's cursor, and history to a first-time one | +| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the claim deadline, then a blocklist | the first publication, not the challenge, is the credential | +| the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it joins the honest broker as the admin, hands the challenge it receives to the publisher as its own, and forwards what the publisher signs under it | the honest broker | the publisher's own updates reach the honest broker, as the publisher's — the relay is a transparent hop that can withhold, not author; the publisher published to the audience it meant to and one more | nothing needed: a relay of a public broadcast is what SWIP-61 makes of every node, and there is no identity to steal, only updates to carry | +| an update of the channel's own feed, `keccak256(topic ‖ index)` | anyone holding it | any stream | refused | the `id` slot: 24 zero bytes are no stream's challenge, and the channel's topic is no session's | +| the admin's updates, on the admin's own publisher stream | the admin's node, or the admin | this session | dropped as retransmits, counted | the cursor | +| the admin's updates, on another node's publisher stream | a second node the admin publishes from | this broker | accepted: each stream has its own challenge, and the admin signed for each | nothing needed: that is the admin publishing twice | +| the admin's updates, in the channel's own feed | the admin, a broker or a subscriber that saved them | storage, to first-time viewers | delivered — **history**, genuine updates in order, a late viewer caught up | nothing needed; returning viewers keep their cursor per `(topic, admin)`, first-time freshness is the payload's (SWIP-65) | | a `Join` | anyone | anywhere | attaches or creates, as any join does — no privilege | nothing needed; the bounds and the inactivity deadline | -**The transport precondition.** The forwarding row above rests on `O_B` being the overlay -the publisher's node is actually connected to. A BPS node MUST verify, in the p2p handshake, that a peer's -signed address record names the connection's authenticated peer ID; a record that is -merely self-consistent — signed by the overlay's key but not tied to the connection — can -be presented by anyone who has seen it. (bee's handshake verifies the record and not the -binding as of this writing; the check is one comparison.) - **No confidentiality**: the broker and every subscriber see plaintext; applications encrypt payloads. **The broker withholds, never forges**, and a withheld update is visible as a gap in the index. **No end signal**: a broker can end a cohort for its audience by resetting their streams, which is withholding, nothing more. **Resource -bounds are policy; the three capacity bounds, the inactivity deadline and the queue bound +bounds are policy; the four capacity bounds, the inactivity deadline and the queue bound are required** (above); the cursor -removes the dedup-window bound and its edge. A subscriber that publishes is a protocol +removes the dedup-window bound and its edge, and the challenge is 24 bytes per stream +the broker holds anyway; a pending stream is a map entry and a challenge for one claim +deadline, and delivers nothing. A subscriber that publishes is a protocol violation and is blocklisted; a squatted cohort is one map entry for one inactivity -deadline, and a peer can hold only a bounded number of them. +deadline, and a peer can hold only a bounded number of them and of streams per cohort. ## Conformance (definition of done) @@ -573,41 +579,49 @@ An implementation is BPS-lite conformant when: 2. a `Join` for a spec with no live cohort creates it, whoever sends it; a `Join` with a byte-identical canonical serialisation attaches to it; no status other than `OK`, `FULL` and `REJECTED` is ever sent; -3. the challenge is derived as specified from a boot secret, the canonical spec and the - declared address, issued iff an address was declared, and nothing is stored for it; -4. a claim — in `Join`, or as a subscriber stream's next `Broadcast` — is a single-owner - chunk verified by the ordinary SOC validation against - `keccak256(keccak256("bps-claim:v1" ‖ topic) ‖ addr)`, with a payload decoding to a - `CLAIM` service message whose challenge and overlay are the broker's own, and `addr` - the admin; the stream then becomes a publisher stream and the cursor becomes - `max(cursor, index)`; no reply is sent; a claim in `Join` is verified before any bound, - and treated as absent if it does not verify; -5. a frame on a subscriber stream that is not a valid claim is dropped, counted, the - stream reset, the peer blocklisted; -6. a `Broadcast` on a publisher stream is accepted iff its bare index is at least the - cursor and the chunk validates as a single-owner chunk at the address the broker forms - itself, `keccak256(keccak256(topic ‖ index) ‖ admin)`; accepted frames set the cursor - past their index and are delivered unchanged to every subscriber stream in the cohort, - and to no publisher stream; -7. `n < cursor` is counted as a retransmit, not a violation, and there is no other dedup - state; gaps are accepted; index 0 is accepted on a fresh cohort; +3. every `Ack{OK}` carries a challenge of exactly 24 bytes, never all zero, drawn at + random for that stream, held for the stream's life and never persisted or reused, + and no `Join` carries or is answered on any credential; +4. a `Join` declaring the admin's `addr` is admitted outside the fan-out bound as a + pending stream that receives nothing; its first `Broadcast` under its challenge is + its claim: it upgrades the stream to a publisher stream iff, with + `id = keccak256(keccak256(topic ‖ challenge) ‖ index)`, the chunk validates as a + single-owner chunk at `keccak256(id ‖ admin)`; it is then delivered or counted as a + retransmit like any publication; no reply is sent; +5. a frame on a subscriber stream, and a frame under the challenge on a pending stream + that does not validate, is dropped, counted (`wrong_stream`), the stream reset, the + peer blocklisted; a frame not under the stream's challenge, on a pending or a + publisher stream, is dropped and counted (`wrong_challenge`) and nothing else; +6. a `Broadcast` on a publisher stream is accepted iff its slot begins with the stream's + challenge, the index in the slot is at least the cursor, and the chunk validates as a + single-owner chunk at the address the broker forms itself, + `keccak256(keccak256(keccak256(topic ‖ challenge) ‖ index) ‖ admin)`; accepted frames + set the cursor past their index and are delivered unchanged to every subscriber + stream in the cohort, and to no publisher stream; +7. `n < cursor` is counted as a retransmit, not a violation; a slot not beginning with + the stream's challenge is dropped and counted, not a violation; there is no other + dedup state; gaps are accepted; index 0 is accepted on a fresh cohort; 8. a cohort with no accepted message for the inactivity deadline is reclaimed, every stream in it reset; a publisher stream going away does not end the cohort; -9. the capacity bounds are enforced, `FULL` is issued at capacity and nothing else is; - one extra stream is admitted for a `Join` declaring the admin's address while no - publisher stream exists, and disconnected if it has not claimed within the claim - deadline; +9. the capacity bounds are enforced — streams per cohort, cohorts per broker, cohorts + per peer connection, streams per peer connection per cohort — `FULL` is issued at + capacity and nothing else is; a pending stream is disconnected if it has not claimed + within the claim deadline; 10. a subscriber re-verifies every delivery against the spec it joined with and its own - cursor; -11. the node verifies in the p2p handshake that a peer's signed address record names the - connection's authenticated peer ID; + cursor, reconstructing the session topic and the id from the topic and the slot, and + compares the slot's challenge with nothing; +11. a publisher signs every update under the challenge of the stream it sends it on, + discards or re-signs what it had queued for a stream that was reset, and never + reuses an index across sessions; 12. a BPS-lite publisher and subscriber interoperate with a full SWIP-60 broker on a single-publisher cohort. -A broker MUST expose per-cohort counters for the silent outcomes — `invalid_index`, +A broker MUST expose per-cohort counters for the silent outcomes — `wrong_challenge` (a +frame on a publisher or pending stream whose slot does not begin with the stream's +challenge), `invalid_soc` (a chunk that does not validate at the expected address: bad signature, -wrong owner or wrong digest alike), `wrong_stream` (a subscriber-stream frame that is not -a valid claim), `claim_timeout` (an extra stream +wrong owner or wrong digest alike), `wrong_stream` (a frame on a subscriber stream, or a +claim that did not validate), `claim_timeout` (a pending stream disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. From f8aee790725a4436de8847f3b409098f9e25c35c Mon Sep 17 00:00:00 2001 From: zelig Date: Thu, 1 Oct 2026 14:07:44 +0200 Subject: [PATCH 10/13] swip-74 rev 8: an ordinary chunk; the frame carries the kind, challenge and index its id was derived from MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Broadcast{soc, kind, challenge, index}; the challenge is 32 random bytes per stream and still salts the topic: id = keccak256(keccak256(prefix || topic || challenge) || index), the prefix inferred from the kind. Kinds DATA and AUTH — an empty chunk that claims a stream; the first valid frame is the claim. A chunk that does not validate is a violation on any stream. No challenge packed into the id slot. Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 489 ++++++++++++++++++++++++++--------------------- 1 file changed, 271 insertions(+), 218 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index 1b97a5cd..a004e2b8 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -25,7 +25,11 @@ salts the feed topic for the session, so every chunk of the session is signed un the first publication is the claim, there is no claim chunk, no service message, no credential in `Join`, and `Join` carries the spec and the peer's identity only; a stream that says it is the admin's is admitted outside the fan-out bound, silent, until -it claims. SWIP-60 (PR #104) is the +it claims. Rev 8: the chunk is an ordinary single-owner chunk with its full id, and the +frame carries what the id was derived from — the kind, the challenge and the index — +next to it; the challenge is 32 bytes; an empty chunk of kind `AUTH` claims a stream +that has nothing to publish yet; a chunk that does not validate disconnects. +SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -55,15 +59,16 @@ its own cohort — a set of nodes connected through a unique central broadcaster *singlehop*. A channel is identified by its spec — a feed topic and the address of its **admin**. Anyone joins by sending that spec and its own address; the broker answers with a **challenge** drawn at random for that stream, and for the life of the stream the -channel's feed topic is salted with it: a publisher's messages are single-owner chunks -that are the higher-index updates of that session feed — the feed on -`keccak256(topic ‖ challenge)` — carried with the challenge and their bare index, and the -first of them is the publisher's claim on the stream. The -broker accepts each one iff it carries the stream's challenge, its index is at least the -channel's cursor, and the chunk validates as the admin's single-owner chunk under the -session feed's id, and delivers it unchanged to every subscriber stream — never to a -publisher's. All other joiners are subscribers and verify the same way, end to end: the -broker can withhold, never forge. A channel ends when it goes idle. +channel's feed topic is salted with it: a publisher's messages are ordinary single-owner +chunks that are the higher-index updates of that session feed, each sent in a frame that +names its kind, the challenge and its index, so that any receiver derives the id the +chunk must have. The first valid frame on the stream is the publisher's claim — an +update, or an empty chunk sent for the purpose. The broker accepts an update iff the +frame carries the stream's challenge, its index is at least the channel's cursor, and +the chunk validates as the admin's single-owner chunk at the derived id, and delivers +the frame unchanged to every subscriber stream — never to a publisher's. All other +joiners are subscribers and verify the same way, end to end: the broker can withhold, +never forge. A channel ends when it goes idle. ## Motivation @@ -139,8 +144,7 @@ package bps; enum TopicBinding { TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire FEED_TOPIC = 4; // the stream is a feed on the topic; on the wire the - // session feed on keccak256(topic || challenge): - // id = keccak256(keccak256(topic || challenge) || index) + // session feed (see Broadcast) } // The cohort's identity. Fixed by whoever joins first; immutable; keyed by its @@ -153,7 +157,8 @@ message CohortSpec { // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live // cohort has this spec, attaches to it otherwise. Nothing else is ever in it — no -// cursor, no credential: the stream's role follows from what it sends after the Ack. +// cursor, no credential: the declared addr makes the stream pending or a subscriber, +// and a pending stream becomes a publisher stream by its first valid frame. message Join { CohortSpec cohort = 1; bytes addr = 2; // 20 bytes, required: the stream's identity — the address it @@ -166,39 +171,54 @@ enum Status { STATUS_UNSPECIFIED = 0; // invalid on the wire OK = 1; FULL = 2; // a capacity bound (see Resource bounds) - REJECTED = 3; // a spec value outside this SWIP + REJECTED = 3; // a spec value outside this SWIP, or a malformed Join } // Broker -> peer, answering Join. A non-OK Ack ends the stream. message Ack { Status status = 1; - bytes challenge = 2; // iff OK: exactly 24 bytes drawn at random for this stream, - // never all zero — the salt of the session feed; held for the - // stream's life, never persisted, never reused + bytes challenge = 2; // iff OK: 32 bytes drawn at random for this stream — the salt of + // its session feeds; held for the stream's life, never + // persisted, never reused } -// Every frame after the handshake: publisher -> broker a publication, broker -> -// subscriber a delivery of the same bytes. The single-owner chunk travels as its chunk -// data, opaque to the protocol: +// What a chunk on this wire is. BPS-lite has two kinds; SWIP-60 adds the admin's. +enum Kind { + KIND_UNSPECIFIED = 0; // invalid on the wire + DATA = 1; // an update of the stream's feed + AUTH = 2; // an empty chunk: claims the stream, says nothing, is never + // delivered +} + +// Every frame after the handshake: publisher -> broker a publication or a claim, +// broker -> subscriber a delivery of the same frame. The chunk is an ordinary +// single-owner chunk, travelling as its chunk data with its full id: // soc = id (32) || signature (65) || span (8, LE) || payload (<= 4096) -// with the id slot carrying challenge (24) || index (8, BE). The receiver reconstructs -// the signed id as keccak256(keccak256(topic || challenge) || index), forms the address -// the chunk must have — keccak256(id || admin) — and validates the chunk against it -// with the ordinary SOC code. A stream's first publication under its challenge is its -// claim: the stream is a publisher stream from the moment a frame of its own validates -// under its challenge as the admin's. +// and the frame carries what that id was derived from: +// prefix = (empty) kind == DATA +// = "bps-service:v1" || kind any other kind, kind as one byte +// topic_s = keccak256(prefix || topic || challenge) +// id = keccak256(topic_s || index) index as a uint64 big-endian +// The receiver derives the id, requires the chunk's to equal it, forms the address the +// chunk must have — keccak256(id || admin) — and validates the chunk against it with +// the ordinary SOC code. message Broadcast { - bytes soc = 1; + bytes soc = 1; + Kind kind = 2; + bytes challenge = 3; // 32 bytes: the challenge of the stream the chunk was signed for + uint64 index = 4; // the chunk's index on its feed } ``` -Three frames — `Join`, `Ack`, `Broadcast` — and the one type they carry, `CohortSpec`. -There is no envelope: **what a frame is follows from the stream's direction and role.** -The first peer-to-broker frame is a `Join` and the first broker-to-peer frame an `Ack`; -after that every frame is a `Broadcast`: a publisher stream sends publications, the -broker sends deliveries, and a stream that has not yet published sends its claim — its -first publication under its challenge — and is a publisher stream or is reset. A frame -that is not what its stream may send is invalid. +Three frames — `Join`, `Ack`, `Broadcast` — the one message they carry, `CohortSpec`, +and three enums: `TopicBinding`, `Status`, `Kind`. There is no envelope: **what a frame is follows from the stream's direction +and role, and what a chunk is from the frame's `kind`** — which, like the challenge and +the index beside it, is not taken on trust: all three are in the preimage of the id the +chunk is signed under. The first peer-to-broker frame is a `Join` and the first +broker-to-peer frame an `Ack`; after that every frame is a `Broadcast`: a publisher +stream sends publications, the broker sends deliveries, and a stream that has not yet +published sends its claim — its first valid frame — and is a publisher stream or is +reset. A frame that is not what its stream may send is invalid. ### Handshake: `Join`, the challenge, the session feed, the claim @@ -214,38 +234,41 @@ create, including a subscriber arriving before the admin: a publisher-less cohor the broker one map entry until the inactivity deadline reclaims it, and can accept no message. There is nothing to pre-register and no "unknown topic". -**The challenge.** With `Ack{OK}` the broker sends a **challenge**: 24 bytes it draws at +**The challenge.** With `Ack{OK}` the broker sends a **challenge**: 32 bytes it draws at random for this stream, holds for the stream's life, and never persists or reuses — on -every stream, whatever its `addr`. It is not a secret: it will be in the clear in every -chunk of the session. It is a **freshness mark**: it exists on this stream only, and +every stream, whatever its `addr`. It is not a secret: it travels in the clear in every +frame of the session. It is a **freshness mark**: it exists on this stream only, and nothing signed for another stream, another cohort, another broker or an earlier session of the same peer carries it. Anyone may obtain one by joining, and gains nothing by it. -It is exactly 24 bytes and never all zero; a peer treats any other length as a protocol -error. A subscriber's challenge is unused in BPS-lite — it is issued so that the family -can upgrade the same stream later — and is never compared with anything. +A peer treats a challenge of any other length as a protocol error. A subscriber's +challenge is unused in BPS-lite — it is issued so that the family can upgrade the same +stream later — and is never compared with anything. **The session feed.** For the life of the stream the publisher does not write the channel's feed but the **session's**: the feed on the channel's topic salted with the challenge, ``` -topic_s = keccak256(topic ‖ challenge) the session topic -id = keccak256(topic_s ‖ index) topic 32, challenge 24, index 8 bytes +prefix = (empty) for DATA + = "bps-service:v1" ‖ kind for every other kind, kind as one byte (?) +topic_s = keccak256(prefix ‖ topic ‖ challenge) +id = keccak256(topic_s ‖ index) topic 32, challenge 32, index 8 bytes big-endian ``` -— an ordinary feed, addressable as `(admin, topic_s)` by ordinary feed tooling. The -chunk's 32-byte `id` slot carries `challenge ‖ index` — the challenge followed by the -index as a uint64 big-endian — so that every receiver that knows the topic, subscriber -or broker, reconstructs `topic_s` and the signed id from the slot without being told -anything else, and the broker additionally requires the slot to begin with the challenge -it issued on that stream. The salt is what makes a session chunk worthless anywhere but -on the stream it was signed for; the index in the clear is what the cursor and SWIP-65's -gap detection read. The publisher signs each update once per stream it holds, and MUST -NOT reuse an index across sessions: a subscriber cannot tell sessions apart, and its -cursor per `(topic, admin)` is what keeps an earlier session's update from standing in -for a later one (see Security considerations). The same wrapped payload signed again -under the channel's own id, `keccak256(topic ‖ index)`, is the update for storage (see -Rationale). +— an ordinary feed, addressable as `(admin, topic_s)` by ordinary feed tooling, and one +of its own per kind, each with its own index sequence. The chunk is an ordinary +single-owner chunk with that id in its `id` slot, and the frame carries beside it the +three things the id was derived from — `kind`, `challenge`, `index` — because a +subscriber does not know which stream a delivery came from and has no other way to learn +a publisher's challenge. None of the three is taken on trust: change any of them and the +derived id is no longer the one in the chunk the publisher signed. The salt is what makes +a session chunk worthless anywhere but on the stream it was signed for; the index is +what the cursor and SWIP-65's gap detection read. The publisher signs each update once +per stream it holds, and MUST NOT reuse a `DATA` index across sessions: a subscriber +cannot tell sessions apart, and its cursor per `(topic, admin)` is what keeps an earlier +session's update from standing in for a later one (see Security considerations). The +same wrapped payload signed again under the channel's own id, `keccak256(topic ‖ index)`, +is the update for storage (see Rationale). **The pending stream.** A `Join` that declares the admin's `addr` is admitted outside the fan-out bound, as a **pending stream**: attached, in no fan-out set, receiving @@ -256,33 +279,37 @@ the admin: a pending stream costs the broker a map entry and a challenge for one deadline and delivers nothing. Every other `Join` is a **subscriber stream**, within the bound. -**The claim.** A pending stream claims the publisher role with its **first `Broadcast` -under its challenge** — a publication like every one that follows. A frame on it whose -slot does not begin with the stream's challenge is not of this session: it is dropped -and counted (`wrong_challenge`), not a violation — it is what an update the admin's node -signed for its previous stream and still had queued looks like, and what a -full-protocol admin's roster looks like here — and a publisher SHOULD discard or re-sign -what it had queued for a stream that was reset, since nothing signed for the old -challenge will be accepted. For a frame that does carry the challenge the broker checks -that with `id = keccak256(topic_s ‖ index)` the chunk validates as the admin's -single-owner chunk at `keccak256(id ‖ admin)`: signature, digest and signer in one +**The claim.** A pending stream claims the publisher role with its **first valid +frame**: an update — a publication like every one that follows — or, for a publisher +with nothing to say yet, an **`AUTH`**: an empty chunk, no payload, on the session's +`AUTH` feed at any index, which claims the stream, counts as activity for the cohort's +lifetime, and does nothing else. It is never delivered and moves no cursor; a valid +update claims as well. The broker checks the +frame as it checks every one (*Validation* below): that its `challenge` is the stream's, +and that the chunk has the id derived from the frame and validates as the admin's +single-owner chunk at `keccak256(id ‖ admin)` — signature, digest and signer in one existing call, the owner forced rather than recovered, since recovering a signer always -yields *some* address. If it does, the stream **upgrades** to a **publisher stream** and -the frame is delivered, or counted as a retransmit, exactly as any publication -(*Validation* below). A signature under this challenge is possible only for the key of -`addr` and only after the `Ack`, so the first valid publication proves the key and the -session at once, and nothing else has to. There is **no reply**: the publisher sends its -first publication and the next back to back, and learns the outcome from whether the -stream survives. If it does not validate, it is a protocol violation: dropped, counted -(`wrong_stream`), the stream reset, the peer blocklisted per the node's policy — a chunk -under this challenge from a key that is not the admin's is nothing a conforming peer -sends; and so is any frame at all from a subscriber stream, whatever its slot. Several -streams MAY be claimed for the admin at once — the admin from two nodes, or -reconnecting before its old stream is torn down: each has its own challenge, and the -cursor arbitrates. +yields *some* address. If it does, the stream **upgrades** to a **publisher stream**, and +an update is delivered, or counted as a retransmit, exactly as any publication. A +signature under this challenge is possible only for the key of `addr` and only after the +`Ack`, so the first valid frame proves the key and the session at once, and nothing else +has to. There is **no reply**: the publisher sends its first frame and the next back to +back, and learns the outcome from whether the stream survives. + +A frame whose `challenge` is not the stream's is not of this session: it is dropped and +counted (`wrong_challenge`), and a broker MAY reset the stream — it is what an update the +admin's node signed for its previous stream and still had queued looks like, so a +publisher SHOULD discard or re-sign what it had queued for a stream that was reset. A +chunk that does not validate is a protocol violation on any stream: dropped, counted +(`invalid_soc`), the stream reset, the peer blocklisted per the node's policy — one +failed validation is all a connection can cost the broker. So is any frame at all from a +subscriber stream (`wrong_stream`). Several streams MAY be claimed for the admin at once +— the admin from two nodes, or reconnecting before its old stream is torn down: each has +its own challenge, and the cursor arbitrates. **The rest of the handshake.** `FULL` answers a `Join` at a -capacity bound, `REJECTED` one whose spec has a value outside this SWIP. A non-`OK` `Ack` +capacity bound, `REJECTED` one whose spec has a value outside this SWIP or whose `addr` +is not 20 bytes. A non-`OK` `Ack` ends the stream; on `OK` it is retained. A peer whose stream was refused or reset MUST back off before rejoining (randomised exponential, 1 s to 30 s, full jitter); a broker MAY reset a stream that retries faster. Without this every reclaim or reset of a cohort @@ -292,89 +319,93 @@ brings its whole audience back at once. After the handshake every frame is a `Broadcast`: a broker sends deliveries, to subscriber streams; a publisher stream sends publications; a pending stream sends its -claim. A delivery is the accepted publication's bytes, unchanged. +claim. A delivery is the accepted frame, unchanged: the chunk, its kind, the challenge +it was signed for and its index. + +**The chunk is an ordinary single-owner chunk.** Its `id` slot holds its full id, as in +any SOC, and nothing in it is rewritten on receipt. What the receiver needs in order to +know which id the chunk must have — the kind, which selects the feed; the challenge, +which salts its topic; the index — travels in the frame, outside the chunk, and the +receiver derives the id from them and the cohort's topic and requires the chunk's to +equal it. A frame whose three fields do not give the chunk's id is invalid, whoever +altered them. **The frame carries no address.** Recovering a signer always yields *an* address, so the ordinary SOC validation needs an address to hold the chunk against — and every receiver -has one: in BPS-lite the publisher is the admin, so the expected address of an update is -`keccak256(keccak256(topic_s ‖ index) ‖ admin)`, and validating the chunk against it checks -signature, digest and owner in one call, with the owner forced rather than compared -afterwards. Carrying the address would say nothing the receiver does not already know. - -**The `id` slot carries the challenge and the bare index.** A publication's signed id is -`keccak256(topic_s ‖ index)`, `topic_s = keccak256(topic ‖ challenge)`, `index` a uint64 -big-endian in 8 bytes. On the wire the 32-byte `id` slot of `soc` holds -`challenge ‖ index`, and the receiver reconstructs the session topic and the signed id -from the cohort's topic and the slot. This is the carriage of -[SWIP-65](https://github.com/ethersphere/SWIPs/pull/106), salted: under SWIP-65 the slot -holds the index left-padded with 24 zero bytes and the receiver prepends the topic — -that is the channel's own feed; here the 24 bytes are the challenge — that is the -session's. A slot beginning with 24 zero bytes therefore names the channel's own feed: -a broker never issues an all-zero challenge, never accepts such a frame on any stream, -and never sends one in BPS-lite — stored updates reach a subscriber through the -family's history delivery (SWIP-60), verified under the channel's own id. On a publisher -or pending stream, a `Broadcast` whose `id` slot does not begin with the stream's -challenge is not a chunk of this session: it is dropped and counted (`wrong_challenge`), -but it is not a violation — it is what an update the admin signed for an earlier session -and resent after reconnecting looks like, and what a full-protocol admin's service -message looks like at a BPS-lite broker. A subscriber does not check the salt — that was -the broker's check, and a subscriber does not know which stream a delivery came from — -and reads the index from the slot. +has one: in BPS-lite the publisher is the admin, so the expected address of a chunk is +`keccak256(id ‖ admin)`, and validating the chunk against it checks signature, digest +and owner in one call, with the owner forced rather than compared afterwards. Carrying +the address would say nothing the receiver does not already know. -### Validation: the feed cursor +**Kinds.** `DATA` is an update of the session feed; `AUTH` is the empty claim. A frame of +a kind this SWIP does not define is dropped and counted (`unknown_kind`), by a broker and +by a subscriber alike, and is not a violation — it is what a full-protocol admin's roster +or end of stream looks like here. -The broker keeps one **cursor** per cohort: **the lowest index it will accept next**, -initially 0 — index 0 is the first update of every feed. A `Broadcast` arriving at the -broker is accepted iff, in order: - -1. it arrived on a **publisher stream** — or on a **pending stream**, as its claim: a - frame there that passes 2 and 4 upgrades the stream, one that fails 2 is dropped and - counted (`wrong_challenge`), one that passes 2 and fails 4 is a protocol violation: - dropped, counted (`wrong_stream`), the stream reset, the peer blocklisted per the - node's policy — as is any frame from a subscriber stream; -2. its `id` slot begins with **the stream's challenge**; -3. the index `n` in the slot's last 8 bytes is **`n ≥ cursor`**; -4. with `id = keccak256(topic_s ‖ n)`, `topic_s = keccak256(topic ‖ challenge)`, the chunk - **validates as the admin's single-owner chunk**: the receiver forms the expected - address `keccak256(id ‖ admin)` and runs the ordinary SOC validation against it — the - wrapped chunk's BMT address matches `span ‖ payload`, and the signature over - `id ‖ wrappedAddress` recovers an owner that hashes with the id to that address, - which only the admin's does. +**The challenge in the frame** is, at the broker, required to be the stream's own. A +subscriber compares it with nothing — not with its own, and not with the one in earlier +deliveries: a publisher that reconnects publishes under a new one. -On acceptance the broker sets `cursor := n + 1` and enqueues the frame on **every -subscriber stream** in the cohort. Publisher streams receive nothing: a publisher never -gets its own messages back, on whichever of its streams they were sent. +### Validation: the feed cursor -A failure of 4 is an invalid frame: dropped and counted (`invalid_soc`); repeated invalid -frames end the connection (blocklisting policy). A frame failing 3 is a benign -failure — a retransmit, which an admin reconnecting after a reset legitimately sends when -it does not know what the broker last accepted — and is counted separately; a broker MAY -reset a publisher stream whose retransmit rate exceeds its policy. A frame failing 2 on a -publisher or pending stream is dropped and counted (`wrong_challenge`) without counting -towards blocklisting, as the Frames section says. The two cheap checks precede the -signature check. +The broker keeps one **cursor** per cohort: **the lowest `DATA` index it will accept +next**, initially 0 — index 0 is the first update of every feed. A `Broadcast` arriving +at the broker is accepted iff, in order: + +1. it arrived on a **publisher stream** or a **pending stream** — any frame from a + subscriber stream is a protocol violation: dropped, counted (`wrong_stream`), the + stream reset, the peer blocklisted per the node's policy; +2. its `kind` is `DATA` or `AUTH` — another kind is dropped and counted (`unknown_kind`); +3. its `challenge` is **the stream's** — otherwise it is dropped and counted + (`wrong_challenge`), and the broker MAY reset the stream; +4. for `DATA`, its `index` is **`n ≥ cursor`** — otherwise it is a retransmit, dropped + and counted; +5. the chunk's id is the one **derived from the frame** — `keccak256(topic_s ‖ index)` + with `topic_s = keccak256(prefix ‖ topic ‖ challenge)` — and the chunk **validates as + the admin's single-owner chunk**: the receiver forms the expected address + `keccak256(id ‖ admin)` and runs the ordinary SOC validation against it — the wrapped + chunk's BMT address matches `span ‖ payload`, and the signature over + `id ‖ wrappedAddress` recovers an owner that hashes with the id to that address, + which only the admin's does. An `AUTH` chunk moreover has span 0 and no payload. A + failure is a protocol violation: dropped, counted (`invalid_soc`), the stream reset, + the peer blocklisted per the node's policy. + +A frame that passes on a pending stream upgrades it — and there step 4 is applied after +step 5: a `DATA` frame below the cursor is still validated, upgrades the stream if it is +valid, and is then counted as a retransmit and not delivered, so that an admin that +reconnects without knowing the cursor, or publishes from a second node, is not left +pending. An accepted `DATA` frame sets +`cursor := n + 1` and is enqueued, unchanged, on **every subscriber stream** in the +cohort; an `AUTH` does nothing more. Publisher streams receive nothing: a publisher +never gets its own messages back, on whichever of its streams they were sent. + +On a publisher stream the four cheap checks precede the signature check, and on any +stream the signature check is paid at most once in vain per connection. A retransmit is the one failure a conforming admin +produces in the ordinary course — reconnecting after a reset, it does not know what the +broker last accepted — and a broker MAY reset a publisher stream whose retransmit rate +exceeds its policy. **There is no dedup window.** The cursor is total: a message is either at or beyond the next expected index or it is not, and there is no eviction and no edge. **Gaps are allowed** (`n > cursor`): they are the publisher's business, and SWIP-65 makes them detectable and recoverable at the subscriber. A returning admin that continues where it -left off sets the cursor with its first accepted update, never back. This is stricter -than SWIP-65's general carriage rule, under which a broker does not enforce -monotonicity; the restriction is sound here because one publisher on one hop admits no -legitimate reordering. +left off sets the cursor with its first accepted update, never back. A broker enforcing +monotonicity is sound here because one publisher on one hop admits no legitimate +reordering. -Subscribers re-verify every delivery exactly as the broker does (steps 3 and 4, against -the spec they joined with and their own cursor, the index and the session topic read -from the slot — never step 2: the challenge in the slot was the broker's check, and the -subscriber's own challenge is compared with nothing), end to end, whatever the broker -did. +Subscribers re-verify every `DATA` delivery as the broker does — steps 4 and 5, against +the spec they joined with and their own cursor, with the kind, the challenge and the +index read from the frame — end to end, whatever the broker did; they never apply step +3, and drop a kind they do not know. ### Lifetime: inactivity -**A cohort ends by inactivity, and by nothing else.** A cohort on which no publisher -stream has had a message accepted for the **inactivity deadline** is reclaimed: the +**A cohort ends by inactivity — or, once no stream is attached, at the broker's +discretion — and by nothing the admin or its stream does.** A cohort on which no publisher +stream has had a frame accepted — an update or an `AUTH` — for the **inactivity +deadline** is reclaimed: the broker resets every stream in it and forgets it. There is no end-of-stream signal in -BPS-lite — that is one of the admin's service messages, of which this SWIP has none; an +BPS-lite — that is one of the admin's service messages, which this SWIP does not have; an application that needs "over" to be distinguishable from "paused" sends it in its last update, or waits for the service feed the family adds. @@ -382,7 +413,7 @@ service feed the family adds. A publisher stream going away — closed or reset — is therefore not an end: the cohort and its cursor persist, subscribers stay attached, and the admin rejoins — admitted as a pending stream whether or not its old one is still held — receives a fresh challenge, -and claims with its first update under it, continuing at whatever index it is at. Likewise a subscriber-only cohort waits for its admin until the deadline +and claims with its first frame under it, continuing at whatever index it is at. Likewise a subscriber-only cohort waits for its admin until the deadline reclaims it. A cohort with **no attached streams** MAY be reclaimed at once — nothing observes the difference beyond a fresh cursor on the next `Join`. @@ -398,7 +429,7 @@ given RECOMMENDED where a value is given; the last two are MAY: | bound | answer | recommended | |---|---|---| -| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the admin's `addr`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its first publication under its challenge upgrades it, or the **claim deadline** disconnects it, with a short blocklist, counted (`claim_timeout`) | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | +| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the admin's `addr`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its first valid frame upgrades it, or the **claim deadline** disconnects it, with a short blocklist, counted (`claim_timeout`) | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | | live cohorts per broker | `FULL` to a cohort-creating `Join` | implementation-defined | | **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | | **streams per peer connection per cohort** — one connection cannot fill a cohort's fan-out set with declared identities | `FULL` | 2 (?) | @@ -428,9 +459,9 @@ this wire, they never change it**: accepts only what this SWIP defines: a spec value outside this SWIP is `REJECTED`, a field this SWIP does not define is ignored, so a fuller spec is served as the lite spec its known fields spell — a feed-topic cohort whose admin later publishes a roster is, - at a BPS-lite broker, a live stream whose roster is dropped (`wrong_challenge`) and - whose grantees' first updates are violations, their streams reset; an admin that wants - a roster needs a full broker; + at a BPS-lite broker, a live stream whose roster is dropped (`unknown_kind`) and whose + grantees, being subscriber streams here, are reset when they publish; an admin that + wants a roster needs a full broker; - a BPS-lite publisher and subscriber at a full broker are conformant peers of a single-publisher live-stream cohort; the full broker's additions (the admin's service messages) reach them as frames they drop; @@ -438,9 +469,9 @@ this wire, they never change it**: retransmits; no frame a BPS-lite broker accepts is one a full broker refuses. What SWIP-60 adds on this wire: cohort parameters (`publishers`, `history`, `closed`), -the admin's service feed (roster, end of stream) as `Broadcast` frames whose payload is a -service message, the claim by any rostered address and none at all under `ALL`, pending -streams for rostered addresses, and the Bee API. +the admin's service feeds as two more kinds (`EOS`, `ROSTER`), the claim by any rostered +address and none at all under `ALL`, pending streams for rostered addresses, and the Bee +API. ## Rationale @@ -478,32 +509,49 @@ challenge drawn at random for the stream is answered on that stream or nowhere; broker keeps it as long as the stream, which it keeps anyway, and nothing survives the stream to be replayed. -**Why the challenge salts the feed, and the first publication is the claim.** A -challenge answered once, by a claim, protects the claim; the updates that follow are -signed for the channel, and whoever captured them from one session can present them in -another — to a broker whose cursor has restarted after a reclaim, they are fresh. Salting -the topic with the challenge signs every update for the session it was published in, so -there is nothing to replay: an update carries the challenge in the clear in its `id` -slot, the broker requires its own, and a chunk of another session, another broker or the -channel's own feed fails that check before any signature is looked at. Once every update -answers the challenge, the first one is the claim, and a claim that is anything else — a -chunk with a kind, a frame of its own, a credential in `Join` — is a second thing to -specify, sign, verify and replay. Salting the *topic* — `keccak256(topic ‖ challenge)` -is a topic like any other — keeps the session feed an ordinary feed, so that a -subscriber that saves the run has a feed it can replay with ordinary tooling, under a -topic every chunk of it names. What it costs: the publisher signs once per stream it +**Why the challenge salts the feed, and the first frame is the claim.** A challenge +answered once, by a claim, protects the claim; the updates that follow are signed for +the channel, and whoever captured them from one session can present them in another — to +a broker whose cursor has restarted after a reclaim, they are fresh. Salting the topic +with the challenge signs every chunk for the session it was published in, so there is +nothing to replay: a chunk of another session, another broker or the channel's own feed +does not have the id this stream's challenge gives. Because the challenge is specific +to the stream — one peer, one cohort, one session — being under it is all a chunk has to +prove, and nothing else needs signing: no claim chunk, no credential in `Join`, no field +in the payload. Once every chunk answers the challenge, the first one is the claim. +Salting the *topic* — `keccak256(topic ‖ challenge)` is a topic like any other — keeps +the session feed an ordinary feed, so that a subscriber that saves the run has a feed it +can replay with ordinary tooling. What it costs: the publisher signs once per stream it holds, and the live chunks are the session feed's, not the channel's — a publisher that wants the run persisted under the channel's own topic signs the same wrapped payloads again under `keccak256(topic ‖ index)`, which is one signature each over a chunk it already has (?). +**Why the frame carries the kind, the challenge and the index.** A chunk signed under a +salted id can be checked only by someone who can derive that id, and a subscriber knows +neither the publisher's challenge nor, from a hash, the index. They could be packed into +the chunk — the challenge and the index squeezed into the 32 bytes of the `id` slot, the +kind inferred from the shape of what is left — and that is a format of its own to +specify, with a truncated challenge and a slot that is no longer a SOC's. Carried in the +frame they are three plain fields, the chunk stays an ordinary single-owner chunk with +an ordinary id, and nothing is lost: the fields are not signed, and do not need to be, +because the id they must reproduce is. + +**Why `AUTH`, and why an empty chunk.** A publisher that has joined and has nothing to +publish yet — a streamer before the show — would otherwise have to invent an update to +hold its stream. An empty chunk under the stream's challenge proves everything a first +update proves and says nothing; it lives on a feed of its own so that it takes no index +from the stream's. It is the only message of the family's control plane this SWIP needs, +and it is a kind, not a frame, because every kind begins the same way: with the +validation of a single-owner chunk. + **Why the identity is on the stream, and the address is declared.** A peer connection is -a node; a stream is a (node, cohort) pair; the identity that matters is the key that +a node; a stream is a (node, cohort, identity) triple; the identity that matters is the key that signs the messages on that stream. Declaring the address in `Join` admits as pending only a stream that says it is the admin's, and tells the broker which address to hold -the stream's first publication to. Binding the identity to the stream is what lets one +the stream's chunks to. Binding the identity to the stream is what lets one node carry different identities on different streams — and, in SWIP-60, what lets a -subscriber stream be upgraded by its first publication when the admin grants it. +subscriber stream be upgraded by its first frame when the admin grants it. **Why the frame carries no address.** The ordinary SOC validation checks that the signer hashes, with the id, to the chunk's address; a receiver that only recovered a signer @@ -516,7 +564,7 @@ on every frame. **Why no reply to a claim.** After the handshake each direction carries one frame type, and the wire has no envelope. A reply would be a second broker-to-peer type on a stream that carries deliveries, with nothing to tell them apart. It is also unnecessary: streams -are ordered, so a publisher sends its first publication and its second together and +are ordered, so a publisher sends its first frame and its second together and learns the outcome from whether the stream survives. **Why no loopback.** A publisher knows what it published. Echoing it costs a frame per @@ -529,7 +577,7 @@ duplicate paths. One publisher on one hop has neither. The feed index is a total and "at least the next" is both the dedup rule and the memory bound. **Why inactivity, and only inactivity.** An end that is *signed* by the admin is one of the -admin's service messages, of which this SWIP has none; an end that is *inferred* from the +admin's service messages, which this SWIP does not have; an end that is *inferred* from the admin's stream turns every transport hiccup into a stream-ending event for the whole audience. So there is no end: a cohort is a map entry that lives while it is used and is reclaimed when it is @@ -542,19 +590,22 @@ Those of SWIP-60, restricted to its live-stream configuration (admin set, the ad alone publishes, open audience). **The publisher role takes the key, every time, and the session takes it again.** Every -update, the first included, is a chunk signed by the publisher's key under a challenge -that exists on one stream. What can be captured and replayed, by whom, and what stops -it: +chunk, the first included, is signed by the publisher's key under an id that only the +challenge of one stream gives. What can be captured and replayed, by whom, and what +stops it: | replayed | by whom | where | result | stopped by | |---|---|---|---|---| -| a session's updates, the first included | anyone who received them — every subscriber, the node that bridged the admin, a relay, the broker itself | any other stream: another node, the same node reconnecting, this broker after a restart or a reclaim, another broker, another cohort | refused before any signature is checked — dropped and counted on a stream declaring the admin's address, a violation on any other | the challenge in the `id` slot: the receiving stream's differs, and an update is signed for its session's feed | -| a session's updates, at an index the publisher reused in a later session | a broker that carried the earlier session | a subscriber, in place of the later session's update | delivered: the subscriber cannot tell sessions apart | the publisher never reusing an index; then the earlier update is below every returning subscriber's cursor, and history to a first-time one | -| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the claim deadline, then a blocklist | the first publication, not the challenge, is the credential | +| a session's frames, the first included, as they were | anyone who received them — every subscriber, the node that bridged the admin, a relay, the broker itself | any other stream: another node, the same node reconnecting, this broker after a restart or a reclaim, another broker, another cohort | dropped before any signature is checked (`wrong_challenge`) — or, from a subscriber stream, a violation | the frame's challenge is not the receiving stream's | +| a session's chunks, with the receiving stream's challenge put in the frame | the same | the same | a violation: the stream reset, the peer blocklisted | the challenge is in the id's preimage: the derived id is not the chunk's | +| a frame with its kind, challenge or index altered | the broker, towards its subscribers | a subscriber | refused | the same: all three are in the id's preimage | +| a session's update, at an index the publisher reused in a later session | a broker that carried the earlier session | a subscriber, in place of the later session's update | delivered: the subscriber cannot tell sessions apart | the publisher never reusing an index; then the earlier update is below every returning subscriber's cursor, and history to a first-time one | +| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the claim deadline, then a blocklist | the first valid frame, not the challenge, is the credential | | the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it joins the honest broker as the admin, hands the challenge it receives to the publisher as its own, and forwards what the publisher signs under it | the honest broker | the publisher's own updates reach the honest broker, as the publisher's — the relay is a transparent hop that can withhold, not author; the publisher published to the audience it meant to and one more | nothing needed: a relay of a public broadcast is what SWIP-61 makes of every node, and there is no identity to steal, only updates to carry | -| an update of the channel's own feed, `keccak256(topic ‖ index)` | anyone holding it | any stream | refused | the `id` slot: 24 zero bytes are no stream's challenge, and the channel's topic is no session's | +| an update of the channel's own feed, `keccak256(topic ‖ index)` | anyone holding it | any stream | a violation | no challenge gives the channel's own topic: the derived id is not the chunk's | +| an `AUTH` as a `DATA`, or the reverse | anyone | any stream | a violation | the kind selects the feed: the derived id is not the chunk's | | the admin's updates, on the admin's own publisher stream | the admin's node, or the admin | this session | dropped as retransmits, counted | the cursor | -| the admin's updates, on another node's publisher stream | a second node the admin publishes from | this broker | accepted: each stream has its own challenge, and the admin signed for each | nothing needed: that is the admin publishing twice | +| the admin's updates, on another node's publisher stream | a second node the admin publishes from | this broker | valid on each stream — each has its own challenge, and the admin signed for each — and delivered once: the first to arrive moves the cursor, the other is a retransmit | the cursor | | the admin's updates, in the channel's own feed | the admin, a broker or a subscriber that saved them | storage, to first-time viewers | delivered — **history**, genuine updates in order, a late viewer caught up | nothing needed; returning viewers keep their cursor per `(topic, admin)`, first-time freshness is the payload's (SWIP-65) | | a `Join` | anyone | anywhere | attaches or creates, as any join does — no privilege | nothing needed; the bounds and the inactivity deadline | @@ -564,9 +615,11 @@ visible as a gap in the index. **No end signal**: a broker can end a cohort for audience by resetting their streams, which is withholding, nothing more. **Resource bounds are policy; the four capacity bounds, the inactivity deadline and the queue bound are required** (above); the cursor -removes the dedup-window bound and its edge, and the challenge is 24 bytes per stream +removes the dedup-window bound and its edge, and the challenge is 32 bytes per stream the broker holds anyway; a pending stream is a map entry and a challenge for one claim -deadline, and delivers nothing. A subscriber that publishes is a protocol +deadline, and delivers nothing; a chunk that does not validate ends the stream that sent +it, so no connection buys more than one signature check with a forgery. A subscriber +that publishes is a protocol violation and is blocklisted; a squatted cohort is one map entry for one inactivity deadline, and a peer can hold only a bounded number of them and of streams per cohort. @@ -575,53 +628,53 @@ deadline, and a peer can hold only a bounded number of them and of streams per c An implementation is BPS-lite conformant when: 1. a broker accepts a `Join` whose spec is `{topic, FEED_TOPIC, admin}` and answers any - other binding or an absent `admin` with `REJECTED`, ignoring fields it does not define; + other binding, an absent `admin`, or an `addr` that is not 20 bytes with `REJECTED`, + ignoring fields it does not define; 2. a `Join` for a spec with no live cohort creates it, whoever sends it; a `Join` with a byte-identical canonical serialisation attaches to it; no status other than `OK`, `FULL` and `REJECTED` is ever sent; -3. every `Ack{OK}` carries a challenge of exactly 24 bytes, never all zero, drawn at - random for that stream, held for the stream's life and never persisted or reused, - and no `Join` carries or is answered on any credential; +3. every `Ack{OK}` carries a challenge of 32 bytes drawn at random for that stream, held + for the stream's life and never persisted or reused, and no `Join` carries or is + answered on any credential; 4. a `Join` declaring the admin's `addr` is admitted outside the fan-out bound as a - pending stream that receives nothing; its first `Broadcast` under its challenge is - its claim: it upgrades the stream to a publisher stream iff, with - `id = keccak256(keccak256(topic ‖ challenge) ‖ index)`, the chunk validates as a - single-owner chunk at `keccak256(id ‖ admin)`; it is then delivered or counted as a - retransmit like any publication; no reply is sent; -5. a frame on a subscriber stream, and a frame under the challenge on a pending stream - that does not validate, is dropped, counted (`wrong_stream`), the stream reset, the - peer blocklisted; a frame not under the stream's challenge, on a pending or a - publisher stream, is dropped and counted (`wrong_challenge`) and nothing else; -6. a `Broadcast` on a publisher stream is accepted iff its slot begins with the stream's - challenge, the index in the slot is at least the cursor, and the chunk validates as a - single-owner chunk at the address the broker forms itself, - `keccak256(keccak256(keccak256(topic ‖ challenge) ‖ index) ‖ admin)`; accepted frames - set the cursor past their index and are delivered unchanged to every subscriber - stream in the cohort, and to no publisher stream; -7. `n < cursor` is counted as a retransmit, not a violation; a slot not beginning with - the stream's challenge is dropped and counted, not a violation; there is no other - dedup state; gaps are accepted; index 0 is accepted on a fresh cohort; -8. a cohort with no accepted message for the inactivity deadline is reclaimed, every + pending stream that receives nothing; its first valid frame — a `DATA` update, at, above + or below the cursor alike, or an `AUTH` — is its claim and upgrades it to a publisher + stream; no reply is sent; +5. a `Broadcast` is accepted iff it arrives on a publisher or pending stream, its kind is + `DATA` or `AUTH`, its challenge is the stream's, for `DATA` its index is at least the + cursor, and its chunk has the id derived from the frame — + `keccak256(keccak256(prefix ‖ topic ‖ challenge) ‖ index)`, the prefix empty for + `DATA` and `"bps-service:v1" ‖ kind` otherwise — and validates as a single-owner chunk + at `keccak256(id ‖ admin)`; an accepted `DATA` frame sets the cursor past its index + and is delivered unchanged to every subscriber stream in the cohort, and to no + publisher stream; an `AUTH` is never delivered, and an `AUTH` chunk with a payload is + invalid; +6. a frame from a subscriber stream, and a chunk that does not validate on any stream, + is a violation: dropped, counted, the stream reset, the peer blocklisted; a frame of + another kind is dropped and counted; a frame under another challenge is dropped and + counted, and the stream MAY be reset; +7. `n < cursor` is counted as a retransmit, not a violation; there is no other dedup + state; gaps are accepted; index 0 is accepted on a fresh cohort; +8. a cohort with no accepted frame for the inactivity deadline is reclaimed, every stream in it reset; a publisher stream going away does not end the cohort; 9. the capacity bounds are enforced — streams per cohort, cohorts per broker, cohorts per peer connection, streams per peer connection per cohort — `FULL` is issued at capacity and nothing else is; a pending stream is disconnected if it has not claimed - within the claim deadline; -10. a subscriber re-verifies every delivery against the spec it joined with and its own - cursor, reconstructing the session topic and the id from the topic and the slot, and - compares the slot's challenge with nothing; -11. a publisher signs every update under the challenge of the stream it sends it on, + within the claim deadline; a subscriber stream whose outbound queue is full is reset; +10. a subscriber re-verifies every `DATA` delivery against the spec it joined with and + its own cursor, deriving the id from the frame's kind, challenge and index, compares + the frame's challenge with nothing, and drops a kind it does not know; +11. a publisher signs every chunk under the challenge of the stream it sends it on, discards or re-signs what it had queued for a stream that was reset, and never - reuses an index across sessions; + reuses a `DATA` index across sessions; 12. a BPS-lite publisher and subscriber interoperate with a full SWIP-60 broker on a single-publisher cohort. -A broker MUST expose per-cohort counters for the silent outcomes — `wrong_challenge` (a -frame on a publisher or pending stream whose slot does not begin with the stream's -challenge), -`invalid_soc` (a chunk that does not validate at the expected address: bad signature, -wrong owner or wrong digest alike), `wrong_stream` (a frame on a subscriber stream, or a -claim that did not validate), `claim_timeout` (a pending stream +A broker MUST expose per-cohort counters for the silent outcomes — `wrong_stream` (a +frame from a subscriber stream), `unknown_kind`, `wrong_challenge` (a frame whose +challenge is not its stream's), `invalid_soc` (a chunk that does not have the derived +id, or does not validate at the expected address: bad signature, wrong owner or wrong +digest alike), `claim_timeout` (a pending stream disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. @@ -643,7 +696,7 @@ never by changing these; a BPS-lite peer ignores fields it does not define. ## References Full singlehop protocol: [SWIP-60, PR #104](https://github.com/ethersphere/SWIPs/pull/104) -· carriage: [SWIP-65 self-indexed feeds, PR #106](https://github.com/ethersphere/SWIPs/pull/106) +· gap detection and persistence: [SWIP-65 self-indexed feeds, PR #106](https://github.com/ethersphere/SWIPs/pull/106) · first draft of this SWIP: [PR #111](https://github.com/ethersphere/SWIPs/pull/111) · origin: [PR #93](https://github.com/ethersphere/SWIPs/pull/93) "Add: pubsub" · implementation: bee [#5626](https://github.com/ethersphere/bee/pull/5626) (groundwork: From b5d7e68c0343ae7139d415f1f041c3e4e8f9696b Mon Sep 17 00:00:00 2001 From: zelig Date: Sat, 10 Oct 2026 01:33:23 +0200 Subject: [PATCH 11/13] swip-74 rev 9: principal and identity, protocol bps/1.0.0, auth deadline; four points the implementation exposed Field names follow the implementation (bee #5626): principal (was admin), identity (was addr); the libp2p protocol is /swarm/bps/1.0.0/bps; the pending stream's deadline and counter say auth. An index of 2^64-1 is invalid so the cursor never wraps; the frame that upgrades a pending stream counts as activity; a subscriber resets on a delivery that does not validate; a creating principal displaces a publisher-less cohort at the cohort bound. Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 117 ++++++++++++++++++++++++++++------------------- 1 file changed, 70 insertions(+), 47 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index a004e2b8..e3198872 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -28,7 +28,12 @@ stream that says it is the admin's is admitted outside the fan-out bound, silent it claims. Rev 8: the chunk is an ordinary single-owner chunk with its full id, and the frame carries what the id was derived from — the kind, the challenge and the index — next to it; the challenge is 32 bytes; an empty chunk of kind `AUTH` claims a stream -that has nothing to publish yet; a chunk that does not validate disconnects. +that has nothing to publish yet; a chunk that does not validate disconnects. Rev 9, +after the implementation (bee #5626) caught up: the fields are `principal` and +`identity`, the protocol is `bps/1.0.0`, the deadline and its counter say `auth`; an +index of 2⁶⁴−1 is invalid; the frame that upgrades a pending stream counts as activity; +a subscriber resets on a delivery that does not validate; a creating principal displaces +a publisher-less cohort at the cohort bound. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -37,7 +42,7 @@ SWIP-60". Open points are marked (?). --> storage round trip and no polling: video/audio streaming, a price ticker, a game's server-authoritative state, a log tail. The stream is a feed, so the author can persist the same updates for anyone who missed the live run. -- **Dev line**: implement one libp2p protocol, `pubsub/1.0.0`, with the three frames +- **Dev line**: implement one libp2p protocol, `bps/1.0.0`, with the three frames below — one broker, direct streams, one publisher whose every message is signed under a challenge the broker issued for its stream, read-only subscribers — and nothing else: **no history, no bandwidth @@ -99,10 +104,12 @@ subscribers**; they are **feed updates, in increasing index order**. ### Topology and roles - **Broker**: one full node per cohort. Every peer holds one direct libp2p stream per - (peer, cohort, identity) to it, protocol id `pubsub/1.0.0`. Depth is 1 by construction: + (peer, cohort, identity) to it — libp2p protocol `/swarm/bps/1.0.0/bps`: name `bps`, + version `1.0.0`, stream `bps`. Depth is 1 by construction: no relaying, no referral. Broker discovery is out of scope; deployments configure the broker. -- **Admin = the publisher**: the address named in the cohort spec. It is the only +- **Principal = the admin = the publisher**: the address the cohort spec names as + `principal`. It is the only identity whose messages are accepted. It need not be first to join; the cohort does not end when its stream goes away; and it may publish from any node, or from two. - **Subscriber**: joins by naming the cohort and its own address, receives, never @@ -117,13 +124,13 @@ Three fields, no others: |---|---| | `topic` | 32 bytes: the feed topic | | `binding` | `FEED_TOPIC`: the stream is a feed on the topic — on the wire, the session feed on `topic_s = keccak256(topic ‖ challenge)`, whose signed id is `keccak256(topic_s ‖ index)` | -| `admin` | 20-byte address: the publisher | +| `principal` | 20-byte address: the admin, who publishes | **The spec is the cohort's identity.** It has a **canonical serialisation** — `Marshal(spec)`: fields in number order, unset fields not emitted — and cohorts are keyed by it: two peers naming byte-identical specs at a broker are in the same cohort; a spec that differs in any field names a different one. The invite for a stream is therefore the -spec plus the broker: `topic`, `admin`, broker address. No publisher regime, no roster, no +spec plus the broker: `topic`, `principal`, broker address. No publisher regime, no roster, no audience flag, no history flag, no proximity constant: a BPS-lite broker answers any value outside this table with `REJECTED`, and ignores fields it does not define, as proto3 does. Capacity is not in the spec: a cohort cannot dictate a remote node's @@ -152,19 +159,19 @@ enum TopicBinding { message CohortSpec { bytes topic = 1; // 32 bytes TopicBinding binding = 2; // FEED_TOPIC - bytes admin = 3; // 20 bytes, required + bytes principal = 3; // 20 bytes, required: the admin } // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live // cohort has this spec, attaches to it otherwise. Nothing else is ever in it — no -// cursor, no credential: the declared addr makes the stream pending or a subscriber, -// and a pending stream becomes a publisher stream by its first valid frame. +// cursor, no credential: the declared identity makes the stream pending or a +// subscriber, and a pending stream becomes a publisher stream by its first valid frame. message Join { - CohortSpec cohort = 1; - bytes addr = 2; // 20 bytes, required: the stream's identity — the address it - // publishes as if it publishes, which only the admin's may; - // a stream declaring the admin's is admitted outside the - // fan-out bound and receives nothing until it claims + CohortSpec cohort = 1; + bytes identity = 2; // 20 bytes, required: the address the stream publishes as + // if it publishes, which only the principal may; a stream + // declaring the principal is admitted outside the fan-out + // bound and receives nothing until it authenticates } enum Status { @@ -200,7 +207,7 @@ enum Kind { // topic_s = keccak256(prefix || topic || challenge) // id = keccak256(topic_s || index) index as a uint64 big-endian // The receiver derives the id, requires the chunk's to equal it, forms the address the -// chunk must have — keccak256(id || admin) — and validates the chunk against it with +// chunk must have — keccak256(id || principal) — and validates the chunk against it with // the ordinary SOC code. message Broadcast { bytes soc = 1; @@ -223,7 +230,7 @@ reset. A frame that is not what its stream may send is invalid. ### Handshake: `Join`, the challenge, the session feed, the claim Every peer sends `Join` as the first frame on a fresh stream, carrying the full -`CohortSpec` and its identity, `addr`: the address the stream publishes as, if it +`CohortSpec` and its `identity`: the address the stream publishes as, if it publishes — an assertion, proved only by the signatures that follow. Nothing else is in it: no cursor, and no credential, because a credential that could be presented in a `Join` could be presented by whoever captured it. @@ -236,7 +243,7 @@ message. There is nothing to pre-register and no "unknown topic". **The challenge.** With `Ack{OK}` the broker sends a **challenge**: 32 bytes it draws at random for this stream, holds for the stream's life, and never persists or reuses — on -every stream, whatever its `addr`. It is not a secret: it travels in the clear in every +every stream, whatever its `identity`. It is not a secret: it travels in the clear in every frame of the session. It is a **freshness mark**: it exists on this stream only, and nothing signed for another stream, another cohort, another broker or an earlier session of the same peer carries it. Anyone may obtain one by joining, and gains nothing by it. @@ -270,9 +277,10 @@ session's update from standing in for a later one (see Security considerations). same wrapped payload signed again under the channel's own id, `keccak256(topic ‖ index)`, is the update for storage (see Rationale). -**The pending stream.** A `Join` that declares the admin's `addr` is admitted outside -the fan-out bound, as a **pending stream**: attached, in no fan-out set, receiving -nothing, until it claims or the **claim deadline** disconnects it (*Resource bounds*). +**The pending stream.** A `Join` that declares the principal as its `identity` is +admitted outside the fan-out bound, as a **pending stream**: attached, in no fan-out +set, receiving nothing, until it claims — *authenticates*, in the implementation's word +— or the **auth deadline** disconnects it (*Resource bounds*). That is what keeps a full audience, or an old stream of the admin's that the transport has not yet reaped, from locking the admin out, and it is worth nothing to anyone but the admin: a pending stream costs the broker a map entry and a challenge for one @@ -287,11 +295,11 @@ lifetime, and does nothing else. It is never delivered and moves no cursor; a va update claims as well. The broker checks the frame as it checks every one (*Validation* below): that its `challenge` is the stream's, and that the chunk has the id derived from the frame and validates as the admin's -single-owner chunk at `keccak256(id ‖ admin)` — signature, digest and signer in one +single-owner chunk at `keccak256(id ‖ principal)` — signature, digest and signer in one existing call, the owner forced rather than recovered, since recovering a signer always yields *some* address. If it does, the stream **upgrades** to a **publisher stream**, and an update is delivered, or counted as a retransmit, exactly as any publication. A -signature under this challenge is possible only for the key of `addr` and only after the +signature under this challenge is possible only for the key of `identity` and only after the `Ack`, so the first valid frame proves the key and the session at once, and nothing else has to. There is **no reply**: the publisher sends its first frame and the next back to back, and learns the outcome from whether the stream survives. @@ -308,7 +316,7 @@ subscriber stream (`wrong_stream`). Several streams MAY be claimed for the admin its own challenge, and the cursor arbitrates. **The rest of the handshake.** `FULL` answers a `Join` at a -capacity bound, `REJECTED` one whose spec has a value outside this SWIP or whose `addr` +capacity bound, `REJECTED` one whose spec has a value outside this SWIP or whose `identity` is not 20 bytes. A non-`OK` `Ack` ends the stream; on `OK` it is retained. A peer whose stream was refused or reset MUST back off before rejoining (randomised exponential, 1 s to 30 s, full jitter); a broker @@ -333,7 +341,7 @@ altered them. **The frame carries no address.** Recovering a signer always yields *an* address, so the ordinary SOC validation needs an address to hold the chunk against — and every receiver has one: in BPS-lite the publisher is the admin, so the expected address of a chunk is -`keccak256(id ‖ admin)`, and validating the chunk against it checks signature, digest +`keccak256(id ‖ principal)`, and validating the chunk against it checks signature, digest and owner in one call, with the owner forced rather than compared afterwards. Carrying the address would say nothing the receiver does not already know. @@ -359,11 +367,13 @@ at the broker is accepted iff, in order: 3. its `challenge` is **the stream's** — otherwise it is dropped and counted (`wrong_challenge`), and the broker MAY reset the stream; 4. for `DATA`, its `index` is **`n ≥ cursor`** — otherwise it is a retransmit, dropped - and counted; + and counted — and `n < 2⁶⁴−1`: the last index is one no feed has, so that + `cursor := n + 1` never wraps, and an update at it is a protocol violation as under + step 5 (`invalid_soc`), checked before anything upgrades a pending stream; 5. the chunk's id is the one **derived from the frame** — `keccak256(topic_s ‖ index)` with `topic_s = keccak256(prefix ‖ topic ‖ challenge)` — and the chunk **validates as the admin's single-owner chunk**: the receiver forms the expected address - `keccak256(id ‖ admin)` and runs the ordinary SOC validation against it — the wrapped + `keccak256(id ‖ principal)` and runs the ordinary SOC validation against it — the wrapped chunk's BMT address matches `span ‖ payload`, and the signature over `id ‖ wrappedAddress` recovers an owner that hashes with the id to that address, which only the admin's does. An `AUTH` chunk moreover has span 0 and no payload. A @@ -374,7 +384,8 @@ A frame that passes on a pending stream upgrades it — and there step 4 is appl step 5: a `DATA` frame below the cursor is still validated, upgrades the stream if it is valid, and is then counted as a retransmit and not delivered, so that an admin that reconnects without knowing the cursor, or publishes from a second node, is not left -pending. An accepted `DATA` frame sets +pending; it counts as activity for the cohort's lifetime as an `AUTH` does, having +proved the key under this stream's challenge (?). An accepted `DATA` frame sets `cursor := n + 1` and is enqueued, unchanged, on **every subscriber stream** in the cohort; an `AUTH` does nothing more. Publisher streams receive nothing: a publisher never gets its own messages back, on whichever of its streams they were sent. @@ -396,13 +407,16 @@ reordering. Subscribers re-verify every `DATA` delivery as the broker does — steps 4 and 5, against the spec they joined with and their own cursor, with the kind, the challenge and the index read from the frame — end to end, whatever the broker did; they never apply step -3, and drop a kind they do not know. +3, and drop a kind they do not know. A delivery that fails step 5 is a broker forwarding +what it should have refused: the subscriber MUST reset the stream, and SHOULD NOT +rejoin that broker for the cohort (?). ### Lifetime: inactivity **A cohort ends by inactivity — or, once no stream is attached, at the broker's discretion — and by nothing the admin or its stream does.** A cohort on which no publisher -stream has had a frame accepted — an update or an `AUTH` — for the **inactivity +stream has had activity — an accepted update, an `AUTH`, or the frame that upgraded +it — for the **inactivity deadline** is reclaimed: the broker resets every stream in it and forgets it. There is no end-of-stream signal in BPS-lite — that is one of the admin's service messages, which this SWIP does not have; an @@ -429,8 +443,8 @@ given RECOMMENDED where a value is given; the last two are MAY: | bound | answer | recommended | |---|---|---| -| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the admin's `addr`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its first valid frame upgrades it, or the **claim deadline** disconnects it, with a short blocklist, counted (`claim_timeout`) | implementation-defined; claim deadline 30 s, long enough for a wallet prompt (?) | -| live cohorts per broker | `FULL` to a cohort-creating `Join` | implementation-defined | +| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the principal as its `identity`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its first valid frame upgrades it, or the **auth deadline** disconnects it, with a short blocklist, counted (`auth_timeout`) | implementation-defined; auth deadline 30 s, long enough for a wallet prompt (?) | +| live cohorts per broker | `FULL` to a cohort-creating `Join` — except one declaring the principal as its `identity`, for which the broker SHOULD reclaim the oldest cohort that has never had a publisher stream instead, and answers `FULL` when there is none (?) | implementation-defined | | **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | | **streams per peer connection per cohort** — one connection cannot fill a cohort's fan-out set with declared identities | `FULL` | 2 (?) | | **inactivity deadline** — reclaims a cohort, see *Lifetime* | reset | 10 min | @@ -444,8 +458,12 @@ a bounded number of them, and none survives idleness. The pending stream is what an audience that fills the cohort while the admin is away — or an old stream of the admin's that the transport has not yet reaped — from locking the admin out, and it is not a slot anyone can squat: it receives nothing, costs a map entry and a challenge for -one claim deadline, and one connection holds a bounded number of streams per cohort. -Declaring the admin's address buys exactly that. The broker has no +one auth deadline, and one connection holds a bounded number of streams per cohort. +Declaring the principal's address buys exactly that. A cohort without a publisher is +worth less than one whose principal has arrived, which is why a creating principal +displaces one rather than being turned away: otherwise a handful of throwaway peers +could hold every cohort slot for an inactivity deadline and keep a principal out. The +broker has no delivery obligation: a reset for a full queue is a recoverable liveness fault, and blocking fan-out on one slow subscriber would punish the cohort. @@ -600,7 +618,7 @@ stops it: | a session's chunks, with the receiving stream's challenge put in the frame | the same | the same | a violation: the stream reset, the peer blocklisted | the challenge is in the id's preimage: the derived id is not the chunk's | | a frame with its kind, challenge or index altered | the broker, towards its subscribers | a subscriber | refused | the same: all three are in the id's preimage | | a session's update, at an index the publisher reused in a later session | a broker that carried the earlier session | a subscriber, in place of the later session's update | delivered: the subscriber cannot tell sessions apart | the publisher never reusing an index; then the earlier update is below every returning subscriber's cursor, and history to a first-time one | -| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the claim deadline, then a blocklist | the first valid frame, not the challenge, is the credential | +| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the auth deadline, then a blocklist | the first valid frame, not the challenge, is the credential | | the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it joins the honest broker as the admin, hands the challenge it receives to the publisher as its own, and forwards what the publisher signs under it | the honest broker | the publisher's own updates reach the honest broker, as the publisher's — the relay is a transparent hop that can withhold, not author; the publisher published to the audience it meant to and one more | nothing needed: a relay of a public broadcast is what SWIP-61 makes of every node, and there is no identity to steal, only updates to carry | | an update of the channel's own feed, `keccak256(topic ‖ index)` | anyone holding it | any stream | a violation | no challenge gives the channel's own topic: the derived id is not the chunk's | | an `AUTH` as a `DATA`, or the reverse | anyone | any stream | a violation | the kind selects the feed: the derived id is not the chunk's | @@ -616,7 +634,7 @@ audience by resetting their streams, which is withholding, nothing more. **Resou bounds are policy; the four capacity bounds, the inactivity deadline and the queue bound are required** (above); the cursor removes the dedup-window bound and its edge, and the challenge is 32 bytes per stream -the broker holds anyway; a pending stream is a map entry and a challenge for one claim +the broker holds anyway; a pending stream is a map entry and a challenge for one auth deadline, and delivers nothing; a chunk that does not validate ends the stream that sent it, so no connection buys more than one signature check with a forgery. A subscriber that publishes is a protocol @@ -627,8 +645,8 @@ deadline, and a peer can hold only a bounded number of them and of streams per c An implementation is BPS-lite conformant when: -1. a broker accepts a `Join` whose spec is `{topic, FEED_TOPIC, admin}` and answers any - other binding, an absent `admin`, or an `addr` that is not 20 bytes with `REJECTED`, +1. a broker accepts a `Join` whose spec is `{topic, FEED_TOPIC, principal}` and answers any + other binding, an absent `principal`, or an `identity` that is not 20 bytes with `REJECTED`, ignoring fields it does not define; 2. a `Join` for a spec with no live cohort creates it, whoever sends it; a `Join` with a byte-identical canonical serialisation attaches to it; no status other than `OK`, @@ -636,7 +654,7 @@ An implementation is BPS-lite conformant when: 3. every `Ack{OK}` carries a challenge of 32 bytes drawn at random for that stream, held for the stream's life and never persisted or reused, and no `Join` carries or is answered on any credential; -4. a `Join` declaring the admin's `addr` is admitted outside the fan-out bound as a +4. a `Join` declaring the principal as its `identity` is admitted outside the fan-out bound as a pending stream that receives nothing; its first valid frame — a `DATA` update, at, above or below the cursor alike, or an `AUTH` — is its claim and upgrades it to a publisher stream; no reply is sent; @@ -645,7 +663,7 @@ An implementation is BPS-lite conformant when: cursor, and its chunk has the id derived from the frame — `keccak256(keccak256(prefix ‖ topic ‖ challenge) ‖ index)`, the prefix empty for `DATA` and `"bps-service:v1" ‖ kind` otherwise — and validates as a single-owner chunk - at `keccak256(id ‖ admin)`; an accepted `DATA` frame sets the cursor past its index + at `keccak256(id ‖ principal)`; an accepted `DATA` frame sets the cursor past its index and is delivered unchanged to every subscriber stream in the cohort, and to no publisher stream; an `AUTH` is never delivered, and an `AUTH` chunk with a payload is invalid; @@ -654,16 +672,21 @@ An implementation is BPS-lite conformant when: another kind is dropped and counted; a frame under another challenge is dropped and counted, and the stream MAY be reset; 7. `n < cursor` is counted as a retransmit, not a violation; there is no other dedup - state; gaps are accepted; index 0 is accepted on a fresh cohort; -8. a cohort with no accepted frame for the inactivity deadline is reclaimed, every + state; gaps are accepted; index 0 is accepted on a fresh cohort and index 2⁶⁴−1 on + none; +8. a cohort with no activity — no accepted frame and no frame that upgraded a pending + stream — for the inactivity deadline is reclaimed, every stream in it reset; a publisher stream going away does not end the cohort; 9. the capacity bounds are enforced — streams per cohort, cohorts per broker, cohorts per peer connection, streams per peer connection per cohort — `FULL` is issued at - capacity and nothing else is; a pending stream is disconnected if it has not claimed - within the claim deadline; a subscriber stream whose outbound queue is full is reset; + capacity and nothing else is, except that a `Join` declaring the principal as its + `identity` is admitted as a pending stream outside the fan-out bound and, at the + cohort bound, displaces the oldest publisher-less cohort when there is one; a pending stream is disconnected if it has not claimed + within the auth deadline; a subscriber stream whose outbound queue is full is reset; 10. a subscriber re-verifies every `DATA` delivery against the spec it joined with and its own cursor, deriving the id from the frame's kind, challenge and index, compares - the frame's challenge with nothing, and drops a kind it does not know; + the frame's challenge with nothing, drops a kind it does not know, and resets the + stream on a delivery that does not validate; 11. a publisher signs every chunk under the challenge of the stream it sends it on, discards or re-signs what it had queued for a stream that was reset, and never reuses a `DATA` index across sessions; @@ -673,9 +696,9 @@ An implementation is BPS-lite conformant when: A broker MUST expose per-cohort counters for the silent outcomes — `wrong_stream` (a frame from a subscriber stream), `unknown_kind`, `wrong_challenge` (a frame whose challenge is not its stream's), `invalid_soc` (a chunk that does not have the derived -id, or does not validate at the expected address: bad signature, wrong owner or wrong -digest alike), `claim_timeout` (a pending stream -disconnected at the claim deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 +id, or does not validate at the expected address — bad signature, wrong owner or wrong +digest alike — or a `DATA` at index 2⁶⁴−1), `auth_timeout` (a pending stream +disconnected at the auth deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. ## Out of scope (deliberately) From 8dd0dbf3b0a569f7ba4f4a6e6e97a62e8a9655e6 Mon Sep 17 00:00:00 2001 From: zelig Date: Sat, 10 Oct 2026 11:23:27 +0200 Subject: [PATCH 12/13] swip-74 rev 10: the AUTH is the only auth; a pending stream sends nothing else A pending stream accepts exactly one frame, its AUTH: the empty chunk at index 0 of the session's AUTH feed. Anything else from it is wrong_stream before any signature check; DATA is accepted on a publisher stream only, so the checks run in one order on every stream and nothing upgrades a stream from the data path (the step-4-after-step-5 inversion and the (?) on upgrade-by-retransmit are gone). An AUTH on a publisher stream is a heartbeat. An AUTH at another index or with a payload is invalid_soc. The subscriber drops an AUTH delivery. What earlier revisions called the claim is the auth throughout. Conformance 9 softened to SHOULD (?) on displacing a publisher-less cohort. Co-Authored-By: Claude Fable 5.1 --- SWIPs/swip-74.md | 222 ++++++++++++++++++++++++++--------------------- 1 file changed, 122 insertions(+), 100 deletions(-) diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index e3198872..f6172970 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -33,7 +33,11 @@ after the implementation (bee #5626) caught up: the fields are `principal` and `identity`, the protocol is `bps/1.0.0`, the deadline and its counter say `auth`; an index of 2⁶⁴−1 is invalid; the frame that upgrades a pending stream counts as activity; a subscriber resets on a delivery that does not validate; a creating principal displaces -a publisher-less cohort at the cohort bound. +a publisher-less cohort at the cohort bound. Rev 10: the `AUTH` — the empty chunk at +index 0 of the session's `AUTH` feed — is the only way a stream authenticates, and a +pending stream sends nothing else; an update is accepted on a publisher stream only, so +nothing upgrades a stream from the data path; an `AUTH` on a publisher stream is a +heartbeat; what earlier revisions called the claim is the auth. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -67,8 +71,9 @@ with a **challenge** drawn at random for that stream, and for the life of the st channel's feed topic is salted with it: a publisher's messages are ordinary single-owner chunks that are the higher-index updates of that session feed, each sent in a frame that names its kind, the challenge and its index, so that any receiver derives the id the -chunk must have. The first valid frame on the stream is the publisher's claim — an -update, or an empty chunk sent for the purpose. The broker accepts an update iff the +chunk must have. The stream is authenticated by an **`AUTH`**: the empty chunk at index 0 +of the session's `AUTH` feed, signed under the same challenge; until it arrives the +stream receives nothing and may send nothing else. The broker accepts an update iff the frame carries the stream's challenge, its index is at least the channel's cursor, and the chunk validates as the admin's single-owner chunk at the derived id, and delivers the frame unchanged to every subscriber stream — never to a publisher's. All other @@ -165,7 +170,7 @@ message CohortSpec { // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live // cohort has this spec, attaches to it otherwise. Nothing else is ever in it — no // cursor, no credential: the declared identity makes the stream pending or a -// subscriber, and a pending stream becomes a publisher stream by its first valid frame. +// subscriber, and a pending stream becomes a publisher stream by its AUTH. message Join { CohortSpec cohort = 1; bytes identity = 2; // 20 bytes, required: the address the stream publishes as @@ -193,11 +198,12 @@ message Ack { enum Kind { KIND_UNSPECIFIED = 0; // invalid on the wire DATA = 1; // an update of the stream's feed - AUTH = 2; // an empty chunk: claims the stream, says nothing, is never - // delivered + AUTH = 2; // the empty chunk at index 0 of the session's AUTH feed: + // authenticates a pending stream, is a heartbeat on a + // publisher stream, is never delivered } -// Every frame after the handshake: publisher -> broker a publication or a claim, +// Every frame after the handshake: publisher -> broker a publication or an auth, // broker -> subscriber a delivery of the same frame. The chunk is an ordinary // single-owner chunk, travelling as its chunk data with its full id: // soc = id (32) || signature (65) || span (8, LE) || payload (<= 4096) @@ -223,11 +229,11 @@ and role, and what a chunk is from the frame's `kind`** — which, like the chal the index beside it, is not taken on trust: all three are in the preimage of the id the chunk is signed under. The first peer-to-broker frame is a `Join` and the first broker-to-peer frame an `Ack`; after that every frame is a `Broadcast`: a publisher -stream sends publications, the broker sends deliveries, and a stream that has not yet -published sends its claim — its first valid frame — and is a publisher stream or is -reset. A frame that is not what its stream may send is invalid. +stream sends publications and heartbeats, the broker sends deliveries, and a pending +stream sends its `AUTH` and is a publisher stream or is reset. A frame that is not what +its stream may send is invalid. -### Handshake: `Join`, the challenge, the session feed, the claim +### Handshake: `Join`, the challenge, the session feed, the auth Every peer sends `Join` as the first frame on a fresh stream, carrying the full `CohortSpec` and its `identity`: the address the stream publishes as, if it @@ -279,39 +285,43 @@ is the update for storage (see Rationale). **The pending stream.** A `Join` that declares the principal as its `identity` is admitted outside the fan-out bound, as a **pending stream**: attached, in no fan-out -set, receiving nothing, until it claims — *authenticates*, in the implementation's word -— or the **auth deadline** disconnects it (*Resource bounds*). +set, receiving nothing and sending nothing but its `AUTH`, until that **authenticates** +it or the **auth deadline** disconnects it (*Resource bounds*). That is what keeps a full audience, or an old stream of the admin's that the transport has not yet reaped, from locking the admin out, and it is worth nothing to anyone but the admin: a pending stream costs the broker a map entry and a challenge for one deadline and delivers nothing. Every other `Join` is a **subscriber stream**, within the bound. -**The claim.** A pending stream claims the publisher role with its **first valid -frame**: an update — a publication like every one that follows — or, for a publisher -with nothing to say yet, an **`AUTH`**: an empty chunk, no payload, on the session's -`AUTH` feed at any index, which claims the stream, counts as activity for the cohort's -lifetime, and does nothing else. It is never delivered and moves no cursor; a valid -update claims as well. The broker checks the -frame as it checks every one (*Validation* below): that its `challenge` is the stream's, -and that the chunk has the id derived from the frame and validates as the admin's -single-owner chunk at `keccak256(id ‖ principal)` — signature, digest and signer in one -existing call, the owner forced rather than recovered, since recovering a signer always -yields *some* address. If it does, the stream **upgrades** to a **publisher stream**, and -an update is delivered, or counted as a retransmit, exactly as any publication. A -signature under this challenge is possible only for the key of `identity` and only after the -`Ack`, so the first valid frame proves the key and the session at once, and nothing else -has to. There is **no reply**: the publisher sends its first frame and the next back to -back, and learns the outcome from whether the stream survives. +**The auth.** A pending stream authenticates as the publisher with its **`AUTH`**: the +empty chunk — span 0, no payload — at **index 0** of the session's `AUTH` feed, +`keccak256("bps-service:v1" ‖ 0x02 ‖ topic ‖ challenge)`. A session has exactly one such +chunk, and it is the only frame a pending stream may send: an update from a pending +stream is a violation, like a frame from a subscriber. The broker checks the `AUTH` as +it checks every frame (*Validation* below): that its `challenge` is the stream's, that +the chunk has the id derived from the frame and validates as the admin's single-owner +chunk at `keccak256(id ‖ principal)` — signature, digest and signer in one existing call, +the owner forced rather than recovered, since recovering a signer always yields *some* +address — and that it is empty. If it does, the stream **upgrades** to a **publisher +stream**; the `AUTH` is never delivered, moves no cursor, and counts as activity for the +cohort's lifetime. A signature under this challenge is possible only for the key of +`identity` and only after the `Ack`, so the `AUTH` proves the key and the session at +once, and nothing else has to. There is **no reply**: the publisher sends its `AUTH` and +its first update back to back, and learns the outcome from whether the stream survives. +On a publisher stream the same `AUTH`, sent again, is a **heartbeat**: it counts as +activity and does nothing else, so a publisher that pauses holds its cohort past the +inactivity deadline without inventing an update. A frame whose `challenge` is not the stream's is not of this session: it is dropped and counted (`wrong_challenge`), and a broker MAY reset the stream — it is what an update the -admin's node signed for its previous stream and still had queued looks like, so a -publisher SHOULD discard or re-sign what it had queued for a stream that was reset. A +admin's node signed for its previous stream and still had queued looks like when it +arrives after the `AUTH` — before it, the same update is a violation (`wrong_stream`) — +so a publisher SHOULD discard or re-sign what it had queued for a stream that was reset, +and sends nothing on the new stream before its `AUTH`. A chunk that does not validate is a protocol violation on any stream: dropped, counted (`invalid_soc`), the stream reset, the peer blocklisted per the node's policy — one failed validation is all a connection can cost the broker. So is any frame at all from a -subscriber stream (`wrong_stream`). Several streams MAY be claimed for the admin at once +subscriber stream (`wrong_stream`). Several streams MAY be authenticated for the admin at once — the admin from two nodes, or reconnecting before its old stream is torn down: each has its own challenge, and the cursor arbitrates. @@ -326,8 +336,8 @@ brings its whole audience back at once. ### Frames After the handshake every frame is a `Broadcast`: a broker sends deliveries, to -subscriber streams; a publisher stream sends publications; a pending stream sends its -claim. A delivery is the accepted frame, unchanged: the chunk, its kind, the challenge +subscriber streams; a publisher stream sends publications and heartbeats; a pending +stream sends its `AUTH`. A delivery is the accepted frame, unchanged: the chunk, its kind, the challenge it was signed for and its index. **The chunk is an ordinary single-owner chunk.** Its `id` slot holds its full id, as in @@ -345,10 +355,12 @@ has one: in BPS-lite the publisher is the admin, so the expected address of a ch and owner in one call, with the owner forced rather than compared afterwards. Carrying the address would say nothing the receiver does not already know. -**Kinds.** `DATA` is an update of the session feed; `AUTH` is the empty claim. A frame of -a kind this SWIP does not define is dropped and counted (`unknown_kind`), by a broker and -by a subscriber alike, and is not a violation — it is what a full-protocol admin's roster -or end of stream looks like here. +**Kinds.** `DATA` is an update of the session feed; `AUTH` is the empty auth, and the +heartbeat. A frame of +a kind this SWIP does not define is dropped and counted (`unknown_kind`), by a broker on a +publisher stream and by a subscriber alike, and is not a violation — it is what a +full-protocol admin's roster or end of stream looks like here; on a pending stream, which +may send nothing but its `AUTH`, it is one (`wrong_stream`). **The challenge in the frame** is, at the broker, required to be the stream's own. A subscriber compares it with nothing — not with its own, and not with the one in earlier @@ -360,16 +372,17 @@ The broker keeps one **cursor** per cohort: **the lowest `DATA` index it will ac next**, initially 0 — index 0 is the first update of every feed. A `Broadcast` arriving at the broker is accepted iff, in order: -1. it arrived on a **publisher stream** or a **pending stream** — any frame from a - subscriber stream is a protocol violation: dropped, counted (`wrong_stream`), the - stream reset, the peer blocklisted per the node's policy; +1. it arrived on a **publisher stream**, or it is an `AUTH` on a **pending stream** — any + other frame is not what its stream may send, a protocol violation: dropped, counted + (`wrong_stream`), the stream reset, the peer blocklisted per the node's policy; 2. its `kind` is `DATA` or `AUTH` — another kind is dropped and counted (`unknown_kind`); 3. its `challenge` is **the stream's** — otherwise it is dropped and counted (`wrong_challenge`), and the broker MAY reset the stream; 4. for `DATA`, its `index` is **`n ≥ cursor`** — otherwise it is a retransmit, dropped and counted — and `n < 2⁶⁴−1`: the last index is one no feed has, so that `cursor := n + 1` never wraps, and an update at it is a protocol violation as under - step 5 (`invalid_soc`), checked before anything upgrades a pending stream; + step 5 (`invalid_soc`); for `AUTH`, its `index` is 0 — any other is likewise a + violation under step 5; 5. the chunk's id is the one **derived from the frame** — `keccak256(topic_s ‖ index)` with `topic_s = keccak256(prefix ‖ topic ‖ challenge)` — and the chunk **validates as the admin's single-owner chunk**: the receiver forms the expected address @@ -380,18 +393,14 @@ at the broker is accepted iff, in order: failure is a protocol violation: dropped, counted (`invalid_soc`), the stream reset, the peer blocklisted per the node's policy. -A frame that passes on a pending stream upgrades it — and there step 4 is applied after -step 5: a `DATA` frame below the cursor is still validated, upgrades the stream if it is -valid, and is then counted as a retransmit and not delivered, so that an admin that -reconnects without knowing the cursor, or publishes from a second node, is not left -pending; it counts as activity for the cohort's lifetime as an `AUTH` does, having -proved the key under this stream's challenge (?). An accepted `DATA` frame sets -`cursor := n + 1` and is enqueued, unchanged, on **every subscriber stream** in the -cohort; an `AUTH` does nothing more. Publisher streams receive nothing: a publisher +An accepted `AUTH` upgrades a pending stream to a publisher stream, counts as activity +for the cohort's lifetime, and does nothing more — on a publisher stream it is a +heartbeat. An accepted `DATA` frame sets `cursor := n + 1` and is enqueued, unchanged, on +**every subscriber stream** in the cohort. Publisher streams receive nothing: a publisher never gets its own messages back, on whichever of its streams they were sent. -On a publisher stream the four cheap checks precede the signature check, and on any -stream the signature check is paid at most once in vain per connection. A retransmit is the one failure a conforming admin +The four cheap checks precede the signature check on every stream, and the signature +check is paid at most once in vain per connection. A retransmit is the one failure a conforming admin produces in the ordinary course — reconnecting after a reset, it does not know what the broker last accepted — and a broker MAY reset a publisher stream whose retransmit rate exceeds its policy. @@ -407,7 +416,8 @@ reordering. Subscribers re-verify every `DATA` delivery as the broker does — steps 4 and 5, against the spec they joined with and their own cursor, with the kind, the challenge and the index read from the frame — end to end, whatever the broker did; they never apply step -3, and drop a kind they do not know. A delivery that fails step 5 is a broker forwarding +3, and drop any frame that is not `DATA` — a kind they do not know, or an `AUTH`, which +is never a delivery. A delivery that fails step 5 is a broker forwarding what it should have refused: the subscriber MUST reset the stream, and SHOULD NOT rejoin that broker for the cohort (?). @@ -415,8 +425,7 @@ rejoin that broker for the cohort (?). **A cohort ends by inactivity — or, once no stream is attached, at the broker's discretion — and by nothing the admin or its stream does.** A cohort on which no publisher -stream has had activity — an accepted update, an `AUTH`, or the frame that upgraded -it — for the **inactivity +stream has had activity — an accepted update or `AUTH` — for the **inactivity deadline** is reclaimed: the broker resets every stream in it and forgets it. There is no end-of-stream signal in BPS-lite — that is one of the admin's service messages, which this SWIP does not have; an @@ -427,7 +436,7 @@ service feed the family adds. A publisher stream going away — closed or reset — is therefore not an end: the cohort and its cursor persist, subscribers stay attached, and the admin rejoins — admitted as a pending stream whether or not its old one is still held — receives a fresh challenge, -and claims with its first frame under it, continuing at whatever index it is at. Likewise a subscriber-only cohort waits for its admin until the deadline +authenticates with an `AUTH` under it, and continues at whatever index it is at. Likewise a subscriber-only cohort waits for its admin until the deadline reclaims it. A cohort with **no attached streams** MAY be reclaimed at once — nothing observes the difference beyond a fresh cursor on the next `Join`. @@ -443,7 +452,7 @@ given RECOMMENDED where a value is given; the last two are MAY: | bound | answer | recommended | |---|---|---| -| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the principal as its `identity`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its first valid frame upgrades it, or the **auth deadline** disconnects it, with a short blocklist, counted (`auth_timeout`) | implementation-defined; auth deadline 30 s, long enough for a wallet prompt (?) | +| subscriber streams per cohort — the fan-out set | `FULL` to the next `Join` — except a `Join` declaring the principal as its `identity`, admitted **outside the bound as a pending stream**: attached, in no fan-out set, receiving nothing, until its `AUTH` upgrades it, or the **auth deadline** disconnects it, with a short blocklist, counted (`auth_timeout`) | implementation-defined; auth deadline 30 s, long enough for a wallet prompt (?) | | live cohorts per broker | `FULL` to a cohort-creating `Join` — except one declaring the principal as its `identity`, for which the broker SHOULD reclaim the oldest cohort that has never had a publisher stream instead, and answers `FULL` when there is none (?) | implementation-defined | | **cohorts per peer connection** — a peer cannot flood the broker with bogus cohorts while keeping one legitimate stream open | `FULL` | 16 | | **streams per peer connection per cohort** — one connection cannot fill a cohort's fan-out set with declared identities | `FULL` | 2 (?) | @@ -457,8 +466,9 @@ third and fifth bounds exist together: a cohort costs a map entry, a peer can ho a bounded number of them, and none survives idleness. The pending stream is what keeps an audience that fills the cohort while the admin is away — or an old stream of the admin's that the transport has not yet reaped — from locking the admin out, and it is -not a slot anyone can squat: it receives nothing, costs a map entry and a challenge for -one auth deadline, and one connection holds a bounded number of streams per cohort. +not a slot anyone can squat: it receives nothing, may send nothing but its `AUTH`, costs +a map entry and a challenge for one auth deadline, and one connection holds a bounded +number of streams per cohort. Declaring the principal's address buys exactly that. A cohort without a publisher is worth less than one whose principal has arrived, which is why a creating principal displaces one rather than being turned away: otherwise a handful of throwaway peers @@ -478,7 +488,7 @@ this wire, they never change it**: field this SWIP does not define is ignored, so a fuller spec is served as the lite spec its known fields spell — a feed-topic cohort whose admin later publishes a roster is, at a BPS-lite broker, a live stream whose roster is dropped (`unknown_kind`) and whose - grantees, being subscriber streams here, are reset when they publish; an admin that + grantees, being subscriber streams here, are reset at their `AUTH`; an admin that wants a roster needs a full broker; - a BPS-lite publisher and subscriber at a full broker are conformant peers of a single-publisher live-stream cohort; the full broker's additions (the admin's service @@ -487,7 +497,7 @@ this wire, they never change it**: retransmits; no frame a BPS-lite broker accepts is one a full broker refuses. What SWIP-60 adds on this wire: cohort parameters (`publishers`, `history`, `closed`), -the admin's service feeds as two more kinds (`EOS`, `ROSTER`), the claim by any rostered +the admin's service feeds as two more kinds (`EOS`, `ROSTER`), the auth of any rostered address and none at all under `ALL`, pending streams for rostered addresses, and the Bee API. @@ -527,16 +537,16 @@ challenge drawn at random for the stream is answered on that stream or nowhere; broker keeps it as long as the stream, which it keeps anyway, and nothing survives the stream to be replayed. -**Why the challenge salts the feed, and the first frame is the claim.** A challenge -answered once, by a claim, protects the claim; the updates that follow are signed for +**Why the challenge salts the feed.** A challenge +answered once, by the auth, protects the auth; the updates that follow are signed for the channel, and whoever captured them from one session can present them in another — to a broker whose cursor has restarted after a reclaim, they are fresh. Salting the topic with the challenge signs every chunk for the session it was published in, so there is nothing to replay: a chunk of another session, another broker or the channel's own feed does not have the id this stream's challenge gives. Because the challenge is specific to the stream — one peer, one cohort, one session — being under it is all a chunk has to -prove, and nothing else needs signing: no claim chunk, no credential in `Join`, no field -in the payload. Once every chunk answers the challenge, the first one is the claim. +prove, and nothing else needs signing: no credential in `Join`, no field in the payload; +the `AUTH` is a chunk under it like every other. Salting the *topic* — `keccak256(topic ‖ challenge)` is a topic like any other — keeps the session feed an ordinary feed, so that a subscriber that saves the run has a feed it can replay with ordinary tooling. What it costs: the publisher signs once per stream it @@ -555,13 +565,20 @@ frame they are three plain fields, the chunk stays an ordinary single-owner chun an ordinary id, and nothing is lost: the fields are not signed, and do not need to be, because the id they must reproduce is. -**Why `AUTH`, and why an empty chunk.** A publisher that has joined and has nothing to -publish yet — a streamer before the show — would otherwise have to invent an update to -hold its stream. An empty chunk under the stream's challenge proves everything a first -update proves and says nothing; it lives on a feed of its own so that it takes no index -from the stream's. It is the only message of the family's control plane this SWIP needs, -and it is a kind, not a frame, because every kind begins the same way: with the -validation of a single-owner chunk. +**Why the `AUTH` is the only auth, and why an empty chunk at index 0.** Any chunk under +the stream's challenge proves what an auth has to prove, and revisions 7–9 let the first +update be it. That saved one frame and cost two wrinkles: a pending stream's update had +to be validated before the cursor was consulted, so that an admin reconnecting below the +cursor still authenticated, and the role transition sat inside the data path, where +whether an upgrade-by-retransmit counts as activity had to be decided. With one auth +frame the pending stream is a one-frame state machine — it accepts its `AUTH` and +nothing else — the checks run in one order on every stream, and the dApp signs one +well-defined chunk per session before its first update, for which it had to wait for the +challenge anyway. The chunk is empty because the auth has nothing to say, and at index 0 +because a session has one: the `AUTH` feed is the salted topic's, so index 0 is fresh on +every stream, and a heartbeat is the same chunk sent again. It lives on a feed of its own +so that it takes no index from the stream's, and it is a kind, not a frame, because +every kind begins the same way: with the validation of a single-owner chunk. **Why the identity is on the stream, and the address is declared.** A peer connection is a node; a stream is a (node, cohort, identity) triple; the identity that matters is the key that @@ -569,20 +586,20 @@ signs the messages on that stream. Declaring the address in `Join` admits as pen only a stream that says it is the admin's, and tells the broker which address to hold the stream's chunks to. Binding the identity to the stream is what lets one node carry different identities on different streams — and, in SWIP-60, what lets a -subscriber stream be upgraded by its first frame when the admin grants it. +subscriber stream be upgraded by its `AUTH` when the admin grants it. **Why the frame carries no address.** The ordinary SOC validation checks that the signer hashes, with the id, to the chunk's address; a receiver that only recovered a signer would accept any signature at all. But the address is not information the sender has and -the receiver lacks: publishing is restricted to known owners — the admin here, a claimed -or declared address in SWIP-60 — so every receiver forms the expected address itself and +the receiver lacks: publishing is restricted to known owners — the admin here, an +authenticated or declared address in SWIP-60 — so every receiver forms the expected address itself and holds the chunk to it. That is the same check with the owner forced, and 32 bytes fewer on every frame. -**Why no reply to a claim.** After the handshake each direction carries one frame type, +**Why no reply to an auth.** After the handshake each direction carries one frame type, and the wire has no envelope. A reply would be a second broker-to-peer type on a stream that carries deliveries, with nothing to tell them apart. It is also unnecessary: streams -are ordered, so a publisher sends its first frame and its second together and +are ordered, so a publisher sends its `AUTH` and its first update together and learns the outcome from whether the stream survives. **Why no loopback.** A publisher knows what it published. Echoing it costs a frame per @@ -614,11 +631,11 @@ stops it: | replayed | by whom | where | result | stopped by | |---|---|---|---|---| -| a session's frames, the first included, as they were | anyone who received them — every subscriber, the node that bridged the admin, a relay, the broker itself | any other stream: another node, the same node reconnecting, this broker after a restart or a reclaim, another broker, another cohort | dropped before any signature is checked (`wrong_challenge`) — or, from a subscriber stream, a violation | the frame's challenge is not the receiving stream's | +| a session's frames, the first included, as they were | anyone who received them — every subscriber, the node that bridged the admin, a relay, the broker itself | any other stream: another node, the same node reconnecting, this broker after a restart or a reclaim, another broker, another cohort | dropped before any signature is checked (`wrong_challenge`) — or, from a subscriber stream, or as anything but an `AUTH` from a pending stream, a violation (`wrong_stream`) | the frame's challenge is not the receiving stream's | | a session's chunks, with the receiving stream's challenge put in the frame | the same | the same | a violation: the stream reset, the peer blocklisted | the challenge is in the id's preimage: the derived id is not the chunk's | | a frame with its kind, challenge or index altered | the broker, towards its subscribers | a subscriber | refused | the same: all three are in the id's preimage | | a session's update, at an index the publisher reused in a later session | a broker that carried the earlier session | a subscriber, in place of the later session's update | delivered: the subscriber cannot tell sessions apart | the publisher never reusing an index; then the earlier update is below every returning subscriber's cursor, and history to a first-time one | -| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the auth deadline, then a blocklist | the first valid frame, not the challenge, is the credential | +| the challenge | anyone, by joining and declaring the admin's address | this broker | harmless: it proves nothing without the key; the declaration buys a pending stream, which receives nothing, until the auth deadline, then a blocklist | the `AUTH`, not the challenge, is the credential | | the challenge, forwarded | a relay or impostor broker the publisher was pointed at: it joins the honest broker as the admin, hands the challenge it receives to the publisher as its own, and forwards what the publisher signs under it | the honest broker | the publisher's own updates reach the honest broker, as the publisher's — the relay is a transparent hop that can withhold, not author; the publisher published to the audience it meant to and one more | nothing needed: a relay of a public broadcast is what SWIP-61 makes of every node, and there is no identity to steal, only updates to carry | | an update of the channel's own feed, `keccak256(topic ‖ index)` | anyone holding it | any stream | a violation | no challenge gives the channel's own topic: the derived id is not the chunk's | | an `AUTH` as a `DATA`, or the reverse | anyone | any stream | a violation | the kind selects the feed: the derived id is not the chunk's | @@ -655,56 +672,61 @@ An implementation is BPS-lite conformant when: for the stream's life and never persisted or reused, and no `Join` carries or is answered on any credential; 4. a `Join` declaring the principal as its `identity` is admitted outside the fan-out bound as a - pending stream that receives nothing; its first valid frame — a `DATA` update, at, above - or below the cursor alike, or an `AUTH` — is its claim and upgrades it to a publisher - stream; no reply is sent; -5. a `Broadcast` is accepted iff it arrives on a publisher or pending stream, its kind is - `DATA` or `AUTH`, its challenge is the stream's, for `DATA` its index is at least the - cursor, and its chunk has the id derived from the frame — + pending stream that receives nothing and may send nothing but its `AUTH` — the empty + chunk at index 0 of the session's `AUTH` feed — which authenticates it and upgrades + it to a publisher stream; no reply is sent; +5. a `Broadcast` is accepted iff it arrives on a publisher stream or is an `AUTH` on a + pending stream, its kind is `DATA` or `AUTH`, its challenge is the stream's, for + `DATA` its index is at least the cursor and for `AUTH` it is 0, and its chunk has the + id derived from the frame — `keccak256(keccak256(prefix ‖ topic ‖ challenge) ‖ index)`, the prefix empty for `DATA` and `"bps-service:v1" ‖ kind` otherwise — and validates as a single-owner chunk at `keccak256(id ‖ principal)`; an accepted `DATA` frame sets the cursor past its index and is delivered unchanged to every subscriber stream in the cohort, and to no - publisher stream; an `AUTH` is never delivered, and an `AUTH` chunk with a payload is - invalid; -6. a frame from a subscriber stream, and a chunk that does not validate on any stream, - is a violation: dropped, counted, the stream reset, the peer blocklisted; a frame of + publisher stream; an `AUTH` is never delivered, counts as activity, and with a + payload or at another index is invalid; +6. a frame from a subscriber stream, a frame other than an `AUTH` from a pending stream, + and a chunk that does not validate on any stream, is a violation: dropped, counted, + the stream reset, the peer blocklisted; a frame of another kind is dropped and counted; a frame under another challenge is dropped and counted, and the stream MAY be reset; 7. `n < cursor` is counted as a retransmit, not a violation; there is no other dedup state; gaps are accepted; index 0 is accepted on a fresh cohort and index 2⁶⁴−1 on none; -8. a cohort with no activity — no accepted frame and no frame that upgraded a pending - stream — for the inactivity deadline is reclaimed, every +8. a cohort with no accepted frame for the inactivity deadline is reclaimed, every stream in it reset; a publisher stream going away does not end the cohort; 9. the capacity bounds are enforced — streams per cohort, cohorts per broker, cohorts per peer connection, streams per peer connection per cohort — `FULL` is issued at capacity and nothing else is, except that a `Join` declaring the principal as its `identity` is admitted as a pending stream outside the fan-out bound and, at the - cohort bound, displaces the oldest publisher-less cohort when there is one; a pending stream is disconnected if it has not claimed + cohort bound, SHOULD displace the oldest publisher-less cohort when there is one (?); a pending stream is disconnected if it has not authenticated within the auth deadline; a subscriber stream whose outbound queue is full is reset; 10. a subscriber re-verifies every `DATA` delivery against the spec it joined with and its own cursor, deriving the id from the frame's kind, challenge and index, compares - the frame's challenge with nothing, drops a kind it does not know, and resets the - stream on a delivery that does not validate; -11. a publisher signs every chunk under the challenge of the stream it sends it on, - discards or re-signs what it had queued for a stream that was reset, and never - reuses a `DATA` index across sessions; + the frame's challenge with nothing, drops any frame that is not `DATA` — a kind it + does not know, or an `AUTH` — and resets the stream on a delivery that does not + validate; +11. a publisher authenticates every stream with its `AUTH` before it publishes on it, + signs every chunk under the challenge of the stream it sends it on, discards or + re-signs what it had queued for a stream that was reset, and never reuses a `DATA` + index across sessions; 12. a BPS-lite publisher and subscriber interoperate with a full SWIP-60 broker on a single-publisher cohort. A broker MUST expose per-cohort counters for the silent outcomes — `wrong_stream` (a -frame from a subscriber stream), `unknown_kind`, `wrong_challenge` (a frame whose +frame from a subscriber stream, or anything but an `AUTH` from a pending stream), +`unknown_kind`, `wrong_challenge` (a frame whose challenge is not its stream's), `invalid_soc` (a chunk that does not have the derived id, or does not validate at the expected address — bad signature, wrong owner or wrong -digest alike — or a `DATA` at index 2⁶⁴−1), `auth_timeout` (a pending stream +digest alike — or a `DATA` at index 2⁶⁴−1, or an `AUTH` with a payload or at an index +other than 0), `auth_timeout` (a pending stream disconnected at the auth deadline), `retransmit`, `queue_reset` — since items 5–7 and 9 are unobservable from the wire without them. ## Out of scope (deliberately) The admin's service messages (roster, end of stream) and the Bee API bridge; multiple -publishers and the claim by a rostered address; the self-indexed payload construction, +publishers and the auth of a rostered address; the self-indexed payload construction, gap recovery and persistence of [SWIP-65](https://github.com/ethersphere/SWIPs/pull/106); multihop ([SWIP-61](https://github.com/ethersphere/SWIPs/pull/105)); history; bandwidth incentives; broker discovery ([SWIP-59](https://github.com/ethersphere/SWIPs/pull/103)); From cbb06866c7daa72c56b35c75e59aea887a73c752 Mon Sep 17 00:00:00 2001 From: zelig Date: Sat, 10 Oct 2026 11:39:59 +0200 Subject: [PATCH 13/13] swip-74 rev 10: glossary, outcomes table, id-derivation figure Terms at the head of the specification; an Outcomes table after Validation listing every outcome a frame can have, its counter and what the sender sees; a figure of how the id is derived from the frame and where the chunk is validated (assets/swip-74/id-derivation.svg). Co-Authored-By: Claude Fable 5.1 --- SWIPs/assets/swip-74/id-derivation.svg | 66 ++++++++++++++++++++++++++ SWIPs/swip-74.md | 65 ++++++++++++++++++++++++- 2 files changed, 130 insertions(+), 1 deletion(-) create mode 100644 SWIPs/assets/swip-74/id-derivation.svg diff --git a/SWIPs/assets/swip-74/id-derivation.svg b/SWIPs/assets/swip-74/id-derivation.svg new file mode 100644 index 00000000..4cf2f8a1 --- /dev/null +++ b/SWIPs/assets/swip-74/id-derivation.svg @@ -0,0 +1,66 @@ + + + +How a chunk's id is derived from the frame, and where it is validated + +Broadcast (the frame) + +kind +1 byte, enum + +challenge +32 bytes, the stream's + +index +8 bytes, big-endian + +soc +the chunk, 105 + payload + +CohortSpec (from Join) + +topic +32 bytes + +principal +20 bytes, eth address + +1 prefix <- kind +DATA -> (empty) +AUTH -> "bps-service:v1" || 0x02 + +2 topic_s = keccak256( prefix || topic || challenge ) +the session feed of this kind: the topic salted with the challenge the broker drew for the stream; one feed per kind + +3 id = keccak256( topic_s || index ) +index as 8 bytes big-endian: the AUTH is at index 0 of its feed, a DATA update at its feed index + +4 the chunk must carry that id: soc = id || signature || span || payload + +id 32 + +signature 65 + +span 8, LE + +payload <= 4096 (AUTH: span 0, no payload) +the receiver derives the id from the frame and requires soc[0:32] to equal it: +kind, challenge and index are not trusted, they are bound through the id's preimage + +5 address = keccak256( id || principal ) validate the SOC at this address +BMT(span || payload) = wrapped address; the signature over keccak256(id || wrapped) recovers an owner; +keccak256(id || owner) must equal the address: the owner is forced to the principal, not read from the key + + + + + + + + + + +Nothing in the frame is signed. The kind, the challenge and the index are bound because the id the chunk +is signed under is derived from them: change any one and the derived id is not the chunk's, so the frame +is refused before the signature is checked. + \ No newline at end of file diff --git a/SWIPs/swip-74.md b/SWIPs/swip-74.md index f6172970..411ac718 100644 --- a/SWIPs/swip-74.md +++ b/SWIPs/swip-74.md @@ -37,7 +37,8 @@ a publisher-less cohort at the cohort bound. Rev 10: the `AUTH` — the empty ch index 0 of the session's `AUTH` feed — is the only way a stream authenticates, and a pending stream sends nothing else; an update is accepted on a publisher stream only, so nothing upgrades a stream from the data path; an `AUTH` on a publisher stream is a -heartbeat; what earlier revisions called the claim is the auth. +heartbeat; what earlier revisions called the claim is the auth; a glossary, an outcomes +table and a figure of the id derivation are added. SWIP-60 (PR #104) is the full singlehop protocol and extends this wire without changing it; see "Relation to SWIP-60". Open points are marked (?). --> @@ -101,6 +102,37 @@ The key words MUST, MUST NOT, REQUIRED, SHOULD, RECOMMENDED and MAY are to be in as described in RFC 2119. Terms — cohort (a channel's set of nodes), broker, admin, subscriber — are SWIP-60's. +### Terms + +| term | meaning | +|---|---| +| **channel** | what an application streams on: a feed topic and its author's address, `(topic, principal)`, independent of any broker | +| **cohort** | the live state of one channel at one broker: the streams attached under one spec, created by the first `Join` that names the spec, reclaimed by inactivity | +| **cohort spec**, `CohortSpec` | `{topic, binding = FEED_TOPIC, principal}`: the cohort's identity, carried in full in every `Join` | +| **topic** | 32 bytes: the channel's feed topic | +| **principal** | the 20-byte address whose single-owner chunks the cohort carries — the publisher; where the family's control plane is meant, the **admin** | +| **broker** | the full node every stream of the cohort is attached to: it validates, keeps the cursor and fans out; it can withhold, never forge | +| **peer** | a node connected to the broker; one peer connection may hold several streams | +| **stream** | one libp2p stream `/swarm/bps/1.0.0/bps` from a peer to the broker, for one cohort and one identity: the unit of role, flow control and teardown | +| **identity** | the address a stream declares in `Join`: the one it publishes as, if it may | +| **role** | what a stream may send and receive, settled by its identity and its auth: **subscriber stream** — any identity but the principal's; receives every delivery, sends nothing. **Pending stream** — declared the principal, not yet authenticated; receives nothing, sends nothing but its `AUTH`, admitted outside the fan-out bound, disconnected at the auth deadline. **Publisher stream** — a pending stream after its `AUTH`; sends publications and heartbeats, receives nothing | +| **frame** | one protobuf message on a stream: `Join`, `Ack` or `Broadcast`; what a frame is follows from the stream's direction and role, never from an envelope | +| **handshake** | the first frame each way: `Join{cohort, identity}` from the peer, `Ack{status, challenge}` from the broker | +| **challenge** | 32 bytes the broker draws at random for each stream and sends in the `Ack`: it salts the session feed and is never persisted or reused | +| **single-owner chunk**, SOC | `id (32) ‖ signature (65) ‖ span (8) ‖ payload (≤ 4096)`, stored and validated at `keccak256(id ‖ owner)`; every chunk on this wire is one | +| **kind** | what a chunk is: `DATA` or `AUTH` (SWIP-60 adds `EOS`, `ROSTER`); carried in the frame, it selects the feed's prefix | +| **session feed** | the channel's feed salted for one stream: topic `keccak256(prefix ‖ topic ‖ challenge)`, one feed per kind, each with its own index sequence; the chunk id is `keccak256(topic_s ‖ index)` | +| **index** | the chunk's position on its feed: 8 bytes big-endian in the frame and in the id's preimage | +| **update**, publication | a `DATA` frame from a publisher stream: one chunk of the session's `DATA` feed | +| **auth** | the act and the frame that authenticate a stream: the **`AUTH`**, an empty SOC at index 0 of the session's `AUTH` feed, which upgrades a pending stream and is never delivered | +| **heartbeat** | an `AUTH` on a publisher stream: counts as activity, does nothing else | +| **delivery** | an accepted update forwarded unchanged — chunk, kind, challenge, index — to every subscriber stream | +| **fan-out set** | the subscriber streams of a cohort, bounded per cohort | +| **cursor** | the lowest `DATA` index accepted next: the broker keeps one per cohort, a subscriber one per channel | +| **retransmit** | a `DATA` frame below the cursor: dropped and counted, not a violation | +| **violation** | a frame a stream may not send, or a chunk that does not validate: dropped, counted, the stream reset, the peer blocklisted | +| **inactivity deadline** | how long a cohort survives without an accepted frame; the **auth deadline** is how long a pending stream has to send its `AUTH` | + ### The contract Per cohort: messages come from **the admin, and the admin only**; they arrive at **all @@ -268,6 +300,8 @@ topic_s = keccak256(prefix ‖ topic ‖ challenge) id = keccak256(topic_s ‖ index) topic 32, challenge 32, index 8 bytes big-endian ``` +![How a chunk's id is derived from the frame, and where it is validated](assets/swip-74/id-derivation.svg) + — an ordinary feed, addressable as `(admin, topic_s)` by ordinary feed tooling, and one of its own per kind, each with its own index sequence. The chunk is an ordinary single-owner chunk with that id in its `id` slot, and the frame carries beside it the @@ -421,6 +455,35 @@ is never a delivery. A delivery that fails step 5 is a broker forwarding what it should have refused: the subscriber MUST reset the stream, and SHOULD NOT rejoin that broker for the cohort (?). +### Outcomes + +Every frame the broker receives ends in exactly one of the outcomes below. The wire +carries no error frame: what a peer learns, it learns from the `Ack` status, from its +stream being reset, or from nothing happening. A reset tells the peer nothing about its +cause — a violation, the auth deadline, the inactivity reclaim and a broker restart look +the same from the peer, which backs off before rejoining (*Resource bounds*); the broker +tells them apart in its counters. + +| what arrived | from | outcome | counter | the sender sees | +|---|---|---|---|---| +| a `Join` with a binding other than `FEED_TOPIC`, an absent `principal`, or an `identity` that is not 20 bytes | anyone | `Ack{REJECTED}`, stream closed | — | `REJECTED` | +| a `Join` at a capacity bound — fan-out set full, cohorts per broker, cohorts or streams per connection | anyone, except as below | `Ack{FULL}`, stream closed | — | `FULL` | +| a `Join` declaring the principal when the fan-out set is full | the principal's node | admitted as a pending stream outside the bound; at the cohort bound the broker SHOULD reclaim the oldest publisher-less cohort for it (?) | — | `OK` and a challenge | +| any `Broadcast` | a subscriber stream | violation: dropped, stream reset, peer blocklisted | `wrong_stream` | the stream is reset | +| anything but an `AUTH` | a pending stream | violation: dropped, stream reset, peer blocklisted | `wrong_stream` | the stream is reset | +| a kind this SWIP does not define | a publisher stream | dropped | `unknown_kind` | nothing | +| a frame whose challenge is not the stream's | a publisher stream, or an `AUTH` on a pending one | dropped; the broker MAY reset the stream | `wrong_challenge` | nothing, or a reset | +| a `DATA` below the cursor | a publisher stream | dropped | `retransmit` | nothing | +| a chunk whose id is not the one derived from the frame, or that does not validate at `keccak256(id ‖ principal)`; an `AUTH` with a payload or not at index 0; a `DATA` at index 2⁶⁴−1 | a publisher or pending stream | violation: dropped, stream reset, peer blocklisted | `invalid_soc` | the stream is reset | +| no `AUTH` within the auth deadline | a pending stream | stream reset, peer blocklisted for a short time | `auth_timeout` | the stream is reset | +| a valid `AUTH` | a pending stream | the stream upgrades to a publisher stream; activity | — | nothing | +| a valid `AUTH` | a publisher stream | activity, nothing else | — | nothing | +| a valid `DATA` at or above the cursor | a publisher stream | cursor set past it; delivered to every subscriber stream | — | nothing | +| a delivery a subscriber stream cannot take — its outbound queue is full | the broker, towards a subscriber | that stream is reset | `queue_reset` | the stream is reset | +| no accepted frame for the inactivity deadline | — | the cohort is reclaimed, every stream in it reset | — | the stream is reset | +| a delivery of a kind other than `DATA`, or below the subscriber's cursor | the broker, at a subscriber | dropped by the subscriber | — | — | +| a delivery that does not validate | the broker, at a subscriber | the subscriber resets the stream | — | — | + ### Lifetime: inactivity **A cohort ends by inactivity — or, once no stream is attached, at the broker's