Conversation
Full-tier review (5 lenses, run serially, blind-ish: each lens committed to its findings before the next started)Consensus/WASM surface, so Full tier per the review rule. No external models (opt-in only). 1. code-first — intent vs. implementationRead the diff before the description. Implementation matches intent. Notes:
2. merge-semantics — the critical lensQuestion asked: does this change the merge OUTPUT for any input, is the merge still a commutative/associative monoid, are all signatures still verified, is serialization byte-identical?
This lens found the one real defect, and it was fixed before the PR opened. Both memos were originally keyed on Also checked, and correct: the cache is keyed on the resolved banner key, so it stays sound across the member-set mutations 3. skeptical — assume bugs existEdge cases worked through: empty room (early return, cache unused); single member; duplicate identical member entries (memo hit, same verdict); over-cap ban set at step 0-cap followed by Two findings, both handled:
Determinism: both caches are Memory: cache size is bounded by ban count and member count. 4. testing — would each test fail if the fix were reverted?Answered empirically rather than by inspection. Six mutations, each reverting one part of the fix:
Each caught by exactly one test; nothing vacuous. The source pins are deliberately whitespace-insensitive (so Deliberate choice: the committed tests count verifications instead of asserting wall-clock budgets. A timing assertion would be flaky across CI hardware, and flaky tests are broken tests. The wall-clock evidence lives in the PR body, produced by throwaway probes. 5. big-picture — does this actually fix the symptom at real scale?Yes, and structurally rather than by shaving a constant: after the change the cost is flat in invite depth (149-156 ms across depths 1 → 200 at 400 members), where before it grew linearly to 16.9 s. The remaining cost is one verification per signed object, which is the irreducible floor for validating a state. Honest residuals, in the PR body: the room-configuration caps have no hard ceiling, so an owner can still configure past any budget (you would now need ~19,000 signed objects rather than a few hundred members); and Scope check: the triage assessment's StatusDraft, per the standing guardrail: this touches [AI-assisted - Claude] |
Security review findings addressed — pushed as
|
…y apply The room contract re-verifies signatures it has already verified, several times per call. Three independent redundancies, all removable without changing what the contract accepts: * `post_apply_cleanup` asks `ban_signature_matches_current_key` about every stored ban at steps 0, 2 and 5 (plus 0-cap when over the cap), and `MembersV1::apply_delta` asks once more. Every ask was an independent verification. Measured at the live Freenet Official room's shape (496 members, 200 bans, 2000 messages): one `update_state` did 800 ban-signature verifications, now 400 — the remaining pair being the two that straddle `ComposableState::apply_delta`, which cannot share a cache without a trait-level change. * `MembersV1::verify` walked from each member to the owner verifying every link, costing `O(members x depth)` where `O(members)` suffices. At the Official room's measured depth (max 4, mean 2.02) that is 1003 -> 496 verifications; on a synthetic 400-member tree of depth 200 it is 40,700 -> 900. * `opt-level = 'z'` leaves curve25519-dalek's field arithmetic un-inlined, costing 1.6x on every verification (0.4224 ms -> 0.2622 ms under wasmtime with Cranelift at `OptLevel::None`, which is what freenet-core runs contracts under). Both caches key on the WHOLE object plus the verifying key it is checked against, never on the 64-bit `BanId` / `MemberId` — those are hashes of attacker-chosen bytes, so keying on them would let a ~2^33 birthday collision inherit a genuine object's verified verdict. `InviteChainCache` additionally binds the room parameters and the member lookup at construction, so one instance cannot span two rooms, or span `verify` and `apply_delta` whose lookups differ, and hand out an `Ok` earned elsewhere; the borrow checker enforces that in release builds where a `debug_assert` would be compiled out. A walk starting at a NON-CANONICAL duplicate member bypasses the memo entirely. The start's `MemberId` seeds the cycle guard, so for that one class of start a memo hit could suppress a circular-chain error the original raised — the only behavioural divergence in the change, found by differential testing against `MembersV1::get_invite_chain` and fixed rather than accepted. `AuthorizedUserBan`'s `Hash` now covers every field rather than the signature alone, since caching made bucketing load-bearing and a signature is attacker-supplied and replayable across bodies. This does NOT demonstrate a fix for the 5s-budget breach #422 reports: no measured real-room shape comes close to that budget. See the PR for what is and is not explained. Refs #422 [AI-assisted - Claude]
Output-identity restored, and both magnitude claims corrected. Head
|
| before | after | ||
|---|---|---|---|
update_state, 1-msg delta |
800 | 400 | 2.0x, 338 -> 169 ms |
validate_state |
3440 | 3192 | 1.08x |
So F2 confirmed — "≈4s" was wrong; it is 338 ms. The comment now carries the measured count. F4 confirmed — 4 passes -> 2, not 4 -> 1, and I've documented why the remaining pair cannot share a cache (they straddle ComposableState::apply_delta; sharing needs a trait-level change). F3 fixed — the doc comments now say which entry point each half serves. F1 fixed — the table is relabelled as a synthetic worst case, with the measured room first.
The uncomfortable consequence, now stated plainly at the top of the PR: validate_state gains 1.08x at the real room, because its 2000 message signatures dominate and the chain fix cannot touch them. And since the #422 contract is 154 KB — an order of magnitude smaller than the Official room — this PR does not demonstrate a fix for the reported timeout. I've retitled the commit Refs #422, not Closes, and written up what is and is not explained.
F5 checked and eliminated rather than left suspected. I added a fixture where bans target real members with subtrees instead of absent users: validate_state 1435 -> 537 ms, update_state 207 -> 74 ms. No blow-up — it is self-limiting, because banning a member with a large subtree cascade-removes it and the walk shortens. F6 fixed (~272 bytes, the decompressed EdwardsPoint). F7 fixed — harness committed as common/tests/room_scale_bench.rs, #[ignore]d, with the wasmtime invocation in its header.
4. The delegate — settled by measurement, and you were right to insist
origin/main : 6f65e45c… 737,365 bytes (identical to committed chat_delegate.wasm)
this PR : 447e3f3d… 754,652 bytes (+17,287, +2.3%)
The delegate WASM does move. The green check-delegate-migration is not evidence: it diffs the committed artifact, which no source change regenerates. So publishing needs a legacy_delegates.toml entry as well as legacy_room_contracts.toml — now stated in both the PR body and the Cargo.toml comment. A useful side-finding: the baseline delegate build reproduces the committed hash exactly, so unlike the room contract it is byte-reproducible from a standalone -p chat-delegate build.
5. Testing gaps from the testing review — all four closed
AuthorizedMember's structural Eq is now pinned (your mutation passed the whole suite; it now fails one test). The Err-memo path is exercised by a branching tree with a broken shared ancestor. The shared ban cache is exercised across an actual member-set mutation — banner pruned mid-cleanup, asked again at the step-5 sweep. And the fails-open literal pin is replaced with argument-name-insensitive assertions.
Mutation count is now 10, each caught by exactly one test or by the compiler. Suite: 428 passed, 0 failed. clippy back to 11, matching origin/main.
Rebased onto current origin/main (#546/#547/#552 picked up). Filed #554 for the UI-side copy of the same redundancy on the render path.
Still DRAFT. Auto-merge not enabled. Stopping for Ian.
[AI-assisted - Claude]
Independent review — four lenses (output-identity, security, testing, performance)Reviewed by four reviewers that did not write the code, each blind to the others, reading the source before the PR description. Several claims settled by execution. Summary: the code is sound and now genuinely output-identical. The original justification was not, and it shrank substantially under review. The author's own final position — Output-identity — was FALSE, now restored (both confirmed by execution)The claim at Trigger: the walk's start is a non-canonical duplicate ( Characterised precisely by the reviewer: the change was a strict relaxation ( Verified fixed at Note that moving the memo lookup below the cycle guard does not fix this — the hit lands before the cycle node is reached. The non-canonical-start bypass is the correct fix. Why four rounds of self-review missed itThe equivalence tests used Security — clean (confirmed)No forged or unverified object can inherit a cached positive verdict, attacked from four angles, each closed by construction. The structural re-keying is complete (no Confirmed by execution before the fix: weakening Performance — mechanism real, magnitudes were wrongThe O(M×D) mechanism is confirmed by reading, the fix is genuinely O(M), the cache build is O(M) not O(M×D), and there is no cold/warm distinction (every call builds a fresh cache, so the "after" numbers reproduce on every invocation). What was wrong: the "≈ 4s of WASM CPU per update" figure (a room at 200/200 skips the cap pass, so 3 passes ≈ 253 ms), the 4→1 pass claim (it is 4→2), and the headline table's depth-50/200 shape, which contradicted the PR's own fixture comment describing the live room as a two-level star. Measured on the live room — depth max 4, mean 2.02 over 496 members — the real gains are 2.0x on Migration implications (both required to publish)
Filed separately rather than bundled
Scope correctionThis PR does not demonstrate a fix for #422. The reported contract is 154 KB, an order of magnitude smaller than the Official room, and at these measured rates neither redundancy approaches 5s at that size. Three candidate causes were checked and eliminated. #422 should stay open. [AI-assisted - Claude] |
What this actually is
A complexity cleanup to the room contract's signature verification, worth roughly 2x on
update_stateand 1.1x onvalidate_stateat the live Freenet Official room's measured shape, and up to 45x onvalidate_statefor invite trees far deeper than any room measured so far.It does not demonstrate a fix for the 5-second-budget breach #422 reports. See "What remains unexplained".
Measurements
All under wasmtime with Cranelift at
OptLevel::None, which is what freenet-core sets for contracts (crates/core/src/wasm_runtime/engine/wasmtime_engine.rs). One Ed25519 verification costs 0.4224 ms there. Native numbers are not usable for this — see the note at the bottom.Verification counts are exact and machine-independent, so they are the primary evidence; times are
count x 0.4224 ms.The live Freenet Official room, measured 2026-07-29 with
cli/examples/invite_depth_probe.rs(in this PR): 496 members, 200 bans, 2000 messages, 497 member_info, 1.44 MB, invite depth max 4 / mean 2.02 / median 2 (histogram: 1 member at depth 1, 484 at depth 2, 10 at depth 3, 1 at depth 4).update_state(1-message delta)validate_statevalidate_statebarely moves because that room's 2000 message signatures dominate, and the chain fix cannot touch them — each message signature was already verified exactly once.Synthetic depth sweep (400 members, 100 messages, no bans),
validate_state:Only the depth-50 and depth-200 rows breach the 5 s budget, and no measured room has a tree remotely that deep. Treat them as the shape of the curve this change flattens, not as a description of any real room. An earlier revision of this PR led with them, which was misleading.
Harness committed as
common/tests/room_scale_bench.rs(#[ignore]d; it is a measurement, not an assertion) so the numbers can be re-run and disputed.What remains unexplained
The contract in #422 was frozen at 154,375 bytes with every merge exceeding 5 s. At that size — an order of magnitude smaller than the Official room's 1.44 MB — neither redundancy I found gets near 5 s by these measurements. So the cause of the reported timeouts is still open. Candidates checked and eliminated:
member_info.rs): measured at 0.06–0.14 ms at 200 members. Not the driver. Left untouched deliberately.get_downstream_membersbeingO(subtree x members)(raised in review as the one thing a verification-count metric is structurally blind to): added a fixture where bans target real members with subtrees rather than absent users.validate_state1435 -> 537 ms,update_state207 -> 74 ms — no blow-up. Self-limiting, because banning a member with a large subtree cascade-removes it and the walk shortens.The honest position: this removes real, measurable redundancy and raises the ceiling substantially for deep trees, but #422 should stay open until something reproduces its actual shape.
The behavioural divergence found in review, and fixed
The earlier claim of output-identity rested on this, in
InviteChainCache's doc:That is false. The start's
MemberIdseeds the cycle guard'svisitedset, and the memo lookup sat before that guard, so a memo hit could short-circuit a walk the original terminated with a circular-chain error. Same input, two verdicts:Trigger: the walk's start is a non-canonical duplicate (
members_by_id[start.member.id()]resolves to a differentAuthorizedMember) and some node on the walk is already memoizedOk. Reachable from both call sites —verify's map is last-wins,apply_delta's isor_insertover wire-supplied deltas. It needs only an ordinary member's own key: B, invited by A, mints a second entry for A's key claiming B invited it; withmembers = [B, A', A], walking B memoizesB -> Ok, and A''s walk then hits that memo instead of walking into A's already-visited id.Fixed by bypassing the memo entirely for a non-canonical start — it neither reads nor writes a verdict, so it runs as the untouched original walk. Every node reached after the start comes out of
members_by_idand is canonical by construction, so the start is the only place this arises. Note that moving the memo lookup below the cycle guard does not fix it: the hit lands on B before the walk ever reaches A.With that, the change is output-identical again — and now it is tested rather than asserted.
Why four rounds of self-review missed it
Every equivalence test used
InviteChainCache::new(...)as the "fresh walk" oracle — the same new implementation with an empty cache. A one-member cache reproduces the identical short-circuit, so the divergence was structurally invisible to them.MembersV1::get_invite_chainis still live andpuband is the real oracle: the untouched original. A differential against it catches this on the first run, and is nowdifferential_against_the_original_walk_on_a_straight_chain/differential_when_a_walk_starts_at_a_non_canonical_duplicate/verify_agrees_with_the_original_walk_on_a_non_canonical_duplicate. This is the single most valuable test in the PR —validate_chainis a hand re-transcription of a security-critical loop, and nothing else compares it to what it replaced.Approach
Memoize within a single operation; nothing is cached across calls, as the contract is stateless.
BanSignatureCache—post_apply_cleanupbuilds one and threads it through all its ban passes. Keyed on(whole ban, resolved verifying key), the complete input toverify_signature, so a hit answers a byte-identical question. Safe across the member-set mutations the cleanup performs: a banner resolving to a different key is a different cache entry, and one that stops resolving returnsfalsewithout consulting the cache.InviteChainCache—MembersV1::verifyandMembersV1::apply_deltaeach build one, so each member's invite signature is verified once. Keyed on the wholeAuthorizedMember, and bound to(parameters, members_by_id)at construction.[profile.release.package.curve25519-dalek] opt-level = 3.Two hazards closed by construction rather than convention:
BanIdisfast_hash(signature),MemberIdisfast_hash(verifying key)— 64-bit hashes of attacker-chosen bytes. Keying on them would let an attacker who grinds a collision (a birthday search over candidates they generate themselves, ~2^33 work, not 2^64) have a forgery inherit a genuine object's "verified" verdict, reopening the forged-ban enforcement hole feat: member deputies for ban authority (invite-subtree moderation) #411 round 4 A closed and letting a forged member entry claim an unearned position in the invite tree. Pinned by two tests that construct the collisions exactly — two bans sharing a signature necessarily share aBanId; two entries sharing a verifying key necessarily share aMemberId— so no grinding is needed.verifyandapply_delta(whose map deliberately includes not-yet-verified delta members and can therefore grant a weakerOk). Chosen over a storedowner_id+debug_assertbecause the contract ships as a release build, where debug assertions are compiled out — that would have documented the hazard while defending nothing. Verified rather than assumed: a cache-sharing test fails to compile witherror[E0716].AuthorizedUserBan'sHashnow covers every field rather than the signature alone, since caching made bucketing load-bearing and a signature is attacker-supplied and replayable across bodies. Safe to change: no map keyed on the type is ever iterated into state, so bucket order cannot reach the wire or affect convergence.Testing
common/tests/signature_verification_cost_test.rs, 22 tests. They count verifications rather than measuring wall-clock time — exact, machine-independent, and not flaky.MembersV1::get_invite_chain, the implementation this replaced, over every fixture including the non-canonical-duplicate shape and throughMembersV1::verifyitself.1+2+...+N; four passes over N bans cost N verifications.AuthorizedMember's structuralEq, which the memo key depends on and which nothing previously pinned — weakening it to ignore the signature passed the entire suite before this test existed.Err-memo path, via a branching tree with a broken shared ancestor. Previously every fixture was a straight chain, so the fan-out case that makes memoization worthwhile was only tested when all-valid.Mutation-tested, 10 independent reversions, each caught by exactly one test — or by the compiler: disable either memoization; revert a
post_apply_cleanupcall site; drop theCargo.tomlexception; revert either memo key to the id; revert theHashimpl; weakenAuthorizedMember'sEq; revert the non-canonical-start bypass; unbind the chain cache (fails to build). No test is vacuous.Full
river-coresuite: 428 passed, 0 failed, including the retention-monoid proptests, convergence and deputy-ban suites, all unmodified.cargo fmtclean; clippy warning count identical toorigin/main(11, all pre-existing).Publishing implications — needs @sanity
Both WASM artifacts move, so a publish needs two migration entries. Measured against
origin/mainwithcargo build --locked --profile release --target wasm32-unknown-unknown:room_contract.wasmchat_delegate.wasmThe delegate moves because it also depends on
ed25519-dalekand is built--profile release. The room contract key isBLAKE3(wasm, params); the delegate key isBLAKE3(BLAKE3(wasm) || params). So publishing requires alegacy_room_contracts.tomlentry and alegacy_delegates.tomlentry.check-room-contract-migrationandcheck-delegate-migrationboth pass on this PR, and that is not evidence they would catch it: they diff the committed WASM, which no source change regenerates. Stated here rather than discovered at publish time. (The baseline delegate build reproduces the committedchat_delegate.wasmhash exactly,6f65e45c…, so that artifact is byte-reproducible from a standalone build — unlike the room contract, which needscargo make sync-wasm.)Also for Ian: the
curve25519-dalekexception is inherited by thewasm-releaseprofile, so the UI WASM gains the same faster verification and roughly the same +16 KB.Filed separately rather than bundled
MembersV1::apply_deltaputs no bound ondelta.added.len(), so a member can force unbounded signature verification. The ban path has had this guard since feat: member deputies for ban authority (invite-subtree moderation) #411 round 3 item C; the member path never got it. Kept out of this PR because a length bound rejects deltas that previously succeeded, which is a merge-semantics change with a rollout story, and bundling it would destroy the one property that makes this PR reviewable.banned_member_idson the render path, paying 200 Ed25519 verifications per call on the Official room. Same redundancy, UI-side.Note on native profiling
Do not characterise this contract from a native build. Natively the profile change measures as 46x (2.27 ms vs 0.05 ms per verification, while signing is unaffected at 0.03 ms — which is why it stayed invisible, since the contract only ever verifies). In WASM it is 1.6x, because Cranelift re-optimises the module on load and recovers most of the lost inlining. An earlier revision of this PR quoted 46x. Correcting it is what redirected the investigation from the constant factor to the algorithmic cause.
Refs #422
[AI-assisted - Claude]