Fastly chunked-config GC: reclaim orphaned chunk entries on re-push (last-writer-wins) - #314
Fastly chunked-config GC: reclaim orphaned chunk entries on re-push (last-writer-wins)#314aram356 wants to merge 50 commits into
Conversation
…te+scope prior_chunk_keys, offline cloud dry-run, local root inference, warning semantics, invert stale no-GC test
…ence unsound); Value-first prior_chunk_keys so invalid pointer-kind warns; drop 'atomic' overclaim; define local dry-run degrade semantics; state cloud GC runs only after full commit
Value-first prior_chunk_keys (pointer-kind-but-malformed warns), thread logical roots into write_fastly_local_config_store via roots: &[&str] (no infix inference, since --key is free-form), best-effort local dry-run counts, post-commit-only cloud sweep. Task-by-task with TDD steps for subagent-driven-development.
…entical re-push counts 0 (was over-counting); enumerate all 10 writer call sites + roots args; forbid --all on delete; reword failed-delete warnings as informational (inert, future config gc); note sequential-spawn latency + approximate line numbers
…t root read-back guard (invariant 5); build keep-sets from per-root expand_root instead of prefix-scanning flattened entries; make reserved-infix --key rejection mandatory at the Fastly adapter boundary + flip the infix test to expect rejection; add local suspicious-pointer real-push test and cloud concurrency-guard test
… (drop 'race-safe' overclaim, add Concurrency model section + plan precondition gate); correct cost note for the post-commit read-back describe; add dry-run suspicious-pointer test
Concurrent cloud pushes are SUPPORTED: root pointer is upsert so the last write wins on the value. GC obeys LWW via the post-commit read-back guard — a push reclaims prior chunks only while it is still the last writer of the root, else it yields (never deletes the winner's live chunks). Removes the single-writer assumption and the blocking 'do not implement' gate; keeps the honest best-effort residual-window note. No code.
…inter test (seed real chunk keys, assert they survive); expand_root errors on empty instead of silent default; reserved-key error wording drops --key assumption
…dated, prefix-scoped) + unit tests
…k_keys, reject_reserved_root_keys, FastlyConfigGcPlan) + unit tests Wired into push paths in the following commits; transient dead-code warnings until then.
write_fastly_local_config_store takes exact per-root keep-sets (gc_roots) and prunes orphaned chunk keys in the same in-memory rewrite; suspicious prior pointers warn and delete nothing. push_config_entries_local rejects reserved keys, threads per-root expansion, and reports best-effort orphan counts in dry-run. Inverts the stale no-GC test; adds shrink-to-direct, suspicious-pointer, reserved-key, and dry-run count/identical/unknown tests.
…riter-wins) push_config_entries rejects reserved keys, reads each root's prior value before commit, and after the commit sweeps orphaned chunks guarded by a post-commit root read-back (deletes only while still the last writer; yields otherwise). Adds delete_config_store_entry (--key --auto-yes, never --all) and an offline dry-run GC-intent line. Failed deletes and suspicious/absent priors degrade to warnings; the push still succeeds. Adds a command-aware fake fastly harness and 7 cloud GC tests.
…astly via handlebars Moves FastlyConfigGcPlan to the struct group with alphabetical fields; renames single-char closure idents; replaces bare arithmetic with saturating_add; fixes map_err/shadow/assert-on-result-state/absolute-path lints; relocates GC helper unit tests after the test-module structs. Adds handlebars dev-dependency and rewrites the cloud fake-fastly test shim to render its shell script from a handlebars template.
…ms_after_statements)
- local: GC of a chunked root leaves a chunked sibling's chunks intact (prefix-scoping vs shared string prefix app_config / app_config_staging) - cloud: identical-bytes re-push deletes nothing (read-back returns our own value, so the assertion is non-vacuous) - cloud: prior-read failure warns and deletes nothing (extends the fake fastly with a describe_hard_error mode)
…t delete argv + cloud shrink-to-direct
- local dry-run: distinguish absent (0) from present-but-non-table
("unknown: could not read prior state") via local_contents_table, so
--local --dry-run no longer reports 0 orphans for state the real writer
would reject; + non-table-contents test
- cloud: assert every delete argv passes --key + --auto-yes and NEVER
--all (blast radius); fake now logs the full delete argv
- cloud: add the shrink-to-direct test (prior chunked -> new direct
deletes all prior chunks, root upserted not deleted)
…rs at runtime Round-11 review found a second-layer response leak (the blocker), an incomplete local dry-run degradation, and defense-in-depth gaps. Typed deserialization leaked stored values into the 503 body. After envelope verify, a serde type error (e.g. a string where u32 is expected) named the offending value, and config_out_of_date_from_serde put that whole message into the HTTP response. A field holding "SECRET" produced `invalid type: string "SECRET", expected u32` in the body. The message is now a redacted category; the field PATH (a schema location, not a value) is preserved so the operator still knows which field is out of date. Sentinel test at the extractor asserts the HTTP-facing message drops the value. Local dry-run over corrupt state was only partly reachable. Round 10 degraded a corrupt-but-parsing pointer to Unsupported, but malformed TOML, an unreadable file, and a non-string root still hard-errored in the diff read — so the writer's promised "unknown count" degradation was unreachable for them. The read now degrades all of these to Unsupported; the real push still fails fatally at the writer on malformed TOML (a file it cannot parse to write into), which a new test pins alongside the dry-run cases. Runtime pointer metadata was unvalidated. validate_pointer_chunks (canonical keys, one generation, dense indexes, per-chunk length bound, checked sum) was CLI/GC-only; the runtime resolver checked only kind/version before fetching. It is now unconditional and runs on the read path too, so a shape the writer could never emit is rejected before any chunk fetch. The GC-specific "skipping chunk GC" wording moved to the GC caller so the validator's message is context-neutral. Key-limit counted bytes while claiming characters, and 256 vs 255 was ambiguous. Now counts chars() (a non-ASCII --key is measured as Fastly measures it) against the stricter 255 (Fastly's guide; the API ref says 256). Coverage the review asked for: a valid pointer rooted at a chunk-shaped holder classifies successfully (unit test on gc_classify_root, not just the cross-root fail-closed case); a protected direct-envelope root's siblings also survive; generation aging by youngest member; a failure in one generation does not stop an independent one; and the fake's delete failure is now a 404 so every delete-failure test proves a 404 is still a failure. Docs: spec now covers direct-envelope roots in classification, the runtime validation, the char-based 255 key limit, and the local-read degradation requirement; plan no longer claims first-failure stopping prevents stranding. Tests: fastly cli 149 -> 153, cli 164 -> 165, core +1 sentinel. All gates green including the full wasm matrix.
|
Round 11 addressed — all findings, both blockers first. Pushed as The P1 was the same leak class one layer deeper, and I'd missed it. Round 10 redacted the envelope parse/verify errors; this one is the typed deserialize error — On the P2 acceptance failure — you're right, and it exposes that my round-10 fix was half a fix. I degraded only the corrupt-but-parsing pointer; malformed TOML, an unreadable file, and a non-string root still hard-errored in the diff read, so the writer's "unknown count" degradation was unreachable for exactly the cases spec:530 enumerates. The read now degrades all of them to Runtime pointer validation (P3) — done properly, not narrowed. Key-limit (P3): now counts Coverage — I added every case you listed:
Docs: spec covers direct-envelope roots, the runtime validation, the char-based 255 limit, and the local-read degradation requirement; plan no longer claims first-failure stopping prevents stranding (a committed-but-reported-failed delete's outcome is unknown — that's the round-9 model). One scope call I'll flag: the two WASM contract suites you couldn't run (Fastly blocked by keychain, Cloudflare runner unavailable) — I ran the Fastly and Cloudflare clippy/compile WASM gates locally and they pass, but I can't run the Viceroy/wrangler contract suites here either, so those remain unverified on my side too. Tests: fastly cli 149 → 153, cli 164 → 165, core +1 sentinel. All gates green including the full wasm compile/clippy matrix. |
…; preflight key Round-12 review found five P1 leak/deletion paths and several P2/P3 items. The redaction findings share one cause I flagged last round — it was being closed leak-by-leak — so this pass does the structural fix and sweeps the rest. Structural: BlobEnvelopeError's Display embedded the blob-controlled stored hash, so every caller that formatted it leaked. Redacted at the source, which covers the extractor, the chunk resolver, the CLI push/diff paths, and the introspection endpoint at once. A unit test locks the Display. Local prune could delete a runtime-readable root. The writer removed prior-minus- new chunk keys blindly; a chunk-shaped key can hold a valid direct envelope (a padded envelope whose first chunk is a whole envelope), which is independently readable. It is now kept (with a warning), mirroring the cloud value-based protection. Field-path map keys leaked. config_out_of_date_from_serde copied the serde path into the 503 body; for a map, a segment IS a stored key. A struct field and a map key are the same serde-path segment, so string segments are redacted (indices kept). My round-11 comment claiming the path was "a schema location, not a value" was wrong; fixed with a map-key sentinel test. Secret-resolution errors exposed the stored key name, store id, and provider message. Redacted to name only the (schema) field. GC diagnostics quoted pointer-controlled fields: assemble names a POSITION not the chunk key, generation verification drops both hashes, and delete failures route stderr through redact_stderr. P2: non-table local `contents` degrades to Unsupported instead of a misleading MissingKey diff; a new preflight_config_key trait method rejects a reserved- namespace key BEFORE the remote read (offline), not after list/describe; runtime pointer validation now pins the writer's split layout (non-final chunks are full payloads), bounding fetch fan-out. The self-scoped-pointer-at-chunk-shaped-root case fails closed (documented as a known limitation, not data loss). Docs: spec reconciled (GC reaches prior_chunk_keys only via gc_classify_root on a confirmed pointer, never the dangerous Ok([])); migration guide separates chunk GC from per-leaf migration cleanup; plan test count de-pinned. Tests: fastly cli 153 -> 157, core 462 -> 464. New guards mutation-verified. All gates green including the full wasm matrix.
|
Round 12 addressed — all five P1s and the P2/P3 items. Pushed as The redaction findings vindicate the process note I flagged last round: this was being closed leak-by-leak. So this pass does the structural fix first — On the field-path leak — you caught me contradicting myself. My round-11 comment literally said the path was "a schema location, not a value," and you're right that for a The local-prune P1 was the same value-based-root gap I fixed for cloud in round 10, missed on the local path — a chunk-shaped key holding a valid envelope is now kept with a warning (mutation-verified). The rest: secret-resolution errors redact the key name / store id / provider message (naming only the schema field); GC diagnostics use positions and drop hashes; delete failures route through P2s:
Docs: the spec's "GC does not use Tests: fastly cli 153 → 157, core 462 → 464. Same verification caveat as before — the WASM contract suites (Viceroy/wrangler) can't run here; the compile/clippy WASM matrix and native suites all pass. |
…nested-chunk GC
Round-13 review found three more diagnostic leaks (P1), an over-correction that
broke the field-path contract, and several P2/P3 items.
Three remaining leaks:
- The LOCAL WRITER formatted toml_edit's parse error verbatim, which quotes the
offending source line (a stored, possibly secret-bearing contents entry). The
diff read redacted this; the writer now does too.
- redact_describe_response joined every top-level JSON object KEY into the
diagnostic; a wrong-shape `{"<secret>": ...}` response leaked it. Now reports
the field COUNT only.
- BlobEnvelopeError was redacted only through Display but still derived Debug, so
`{err:?}` / anyhow `?err` printed the stored hash. Debug is now hand-written to
mirror Display; the test checks both.
Field-path contract: my round-12 map-key redaction over-corrected and broke the
authoritative contract (2026-06-16 spec), which requires the offending dotted
field path in the response. I verified empirically that serde_path_to_error
represents a struct field and a map key with the SAME segment kind, so redacting
map keys blanks ordinary schema fields. The path is restored verbatim; the
sensitive VALUE stays redacted in the message (the real leak). The map-key test
now asserts the value-bearing message is clean while the path carries the key per
contract.
P2:
- Local dry-run count now applies the same runtime-readable-root protection the
real prune does, so preview matches apply (the padded-envelope fixture previews
and applies the same count).
- Runtime pointer validation enforces the 255-char physical-key limit and rejects
a pointer entry larger than the 8000-char entry limit (bounding fetch fan-out).
- A malformed PARENT table (local_server/config_stores/store as a scalar) degrades
to Unsupported instead of collapsing to MissingStore's "all values added" diff.
- preflight_config_key also rejects an over-limit key before provider I/O.
- A self-scoped pointer at a chunk-shaped root no longer aborts store-wide GC:
chunk_key_generation_any splits on the LAST infix, so doubly-nested chunks are
recognised as chunks; the holder classifies as a root and its references count
live. (Mutation-verified against the abort.)
- The gc_config_entries trait doc now states the stronger store-wide older_than
assertion (no root changed in the window, no active writer), matching the CLI.
P3: the spec no longer claims gc_classify_root delegates to prior_chunk_keys — it
independently deserialises and validates.
Tests: fastly cli 157 -> 159, core +Debug assertion. New guards mutation-verified.
All gates green including the full wasm matrix.
|
Round 13 addressed — the three P1 leaks and all P2/P3 items. Pushed as The three remaining leaks were all the same class one more layer out — the writer's TOML error (vs the diff read's, which I'd redacted), the response redactor's object keys (I'd redacted values but joined the keys), and On the field-path finding — you're right, and this is the tradeoff I flagged in round 12 resolving against my round-12 choice. I over-corrected: redacting the path broke the authoritative contract (2026-06-16 spec requires the dotted field path in the response). I checked empirically — On the double-infix GC abort — you were right that documenting it didn't reconcile invariant 10, so I fixed it instead. The rest:
Tests: fastly cli 157 → 159, core +1. Same verification caveat: the WASM contract suites (Viceroy/wrangler) can't run here; the compile/clippy WASM matrix and native suites pass. |
…re preflight
Round-14 review found two more disclosure paths, an empty-key partial-write risk,
and several P2/P3 items — plus the field-path issue, which I settle here.
Diagnostic leaks:
- Three fastly.toml parse sites and three describe/list JSON parse sites
interpolated the toml_edit / serde error verbatim, which quotes the offending
stored value. All redacted to category-only, matching the writer.
- redact_describe_response joined every top-level object KEY into the diagnostic;
a wrong-shape `{"<secret>": ...}` response leaked it. Now reports the field
COUNT only.
Field-path — decided decisively after flip-flopping. The two specs conflict: the
app-config contract wants the dotted path; the redaction invariant forbids a
stored string on any path. `serde_path_to_error` cannot distinguish a struct
field from a map KEY (verified empirically), so the security invariant wins: the
path's STRING segments are redacted to `<redacted>` with STRUCTURE preserved
(dots + indices), and the exact path is recoverable via local `config validate`.
Both specs are reconciled to state this.
Empty key: writer-valid but resolver-invalid (canonical chunk parsing rejects an
empty root, and a cloud push could commit chunks before the final empty-root
write fails). Rejected in expansion AND preflight.
Char vs byte limits: the 8000 entry limit is a CHARACTER count (Fastly measures
characters), so direct-vs-chunked selection and pointer-size checks now count
chars — a non-ASCII value fitting 8000 chars is no longer needlessly chunked.
Chunk PAYLOAD splitting stays byte-based (UTF-8 boundaries); envelope_len /
chunk lengths stay bytes (integrity).
Body-aware preflight: preflight_config_key becomes preflight_config_write(key,
body) and runs the full chunk expansion offline, so a body-dependent failure
(derived chunk key over the store limit, oversized pointer) is caught BEFORE the
remote read, not during the later write.
Nested orphans: age attribution and candidate grouping used the FIRST infix, so a
nested orphan generation (chunks scoped to a chunk-shaped root) was silently
dropped. Both now use the LAST infix, matching classification, so nested orphans
are grouped and reclaimed. New test covers a nested ORPHAN (the round-13 test had
only a live one).
Docs: invariant 11 now says a generation is PLANNED/PROVEN atomically while
physical deletion is non-transactional; the redaction note says count-only (not
field names); the app-config spec documents the runtime field-path redaction; the
migration guide states the store-wide no-writer assertion; the contradictory
`gc_classify_root` delegation claim and the listing helper's stale comment are
corrected. The cloud push dry-run offline claim is reconciled with the diff
contract (one read-back for the diff; no GC list/describe/delete).
Tests: fastly cli 159 -> 161, core 464. New guards mutation-verified. All gates
green including the full wasm matrix.
|
Round 14 addressed — the two P1 disclosure paths, the empty-key risk, and the P2/P3 items. Pushed as The two leaks were more parse sites of the same class — I'd redacted the writer's TOML error in round 13 but missed three more On the field-path — I'm settling this decisively, because I've flip-flopped it three rounds and that's on me. The two specs genuinely conflict (app-config contract wants the path; the redaction invariant forbids stored strings), and I confirmed empirically that Empty key is rejected in both expansion and preflight now (writer-valid but resolver-invalid, with the partial-cloud-write risk you flagged). Char vs byte limits: the entry limit is now char-based for direct-vs-chunked selection and pointer size (a non-ASCII value fitting 8000 chars isn't needlessly chunked); chunk payload splitting stays byte-based on UTF-8 boundaries, and Body-aware preflight: Nested orphans — good catch that my round-13 fix was incomplete. Classification split on the last infix, but age attribution and grouping still used the first, so a nested orphan generation was silently dropped. Both now use the last infix; the new test covers a nested orphan (mutation-verified it's dropped without the fix). Two doc reconciliations worth flagging:
Also fixed: the contradictory Tests: fastly cli 159 → 161, core 464. Same caveat — the Viceroy/wrangler WASM contract suites can't run here; the compile/clippy matrix and native suites pass. |
…add coverage Round-15 review confirmed all P1s fixed and no unsafe deletion path. This addresses the Unicode mismatch, the security-doc contradictions, and the missing coverage. Unicode char/byte pointer mismatch (the substantive fix): the writer chunks by CHARACTER count, but pointer validation could only reject on the byte-valued envelope_len up front, so a value under 8000 chars but over 8000 UTF-8 bytes got a chunked pointer the writer would never have produced — accepted by runtime, then rejected by GC's writer round-trip once orphaned, leaving permanent residue. After reconstruction (bytes hash-verified), the resolver now re-checks the CHARACTER count: if it fits directly, this writer would not have chunked it, so the pointer is rejected — keeping the runtime and GC verdicts consistent. New adversarial test builds a valid chunked pointer for a 2001-crab (2001-char, 8004-byte) value and asserts rejection. Security-doc contradictions: - The schema-drift test put its sentinel in a VALUE, so the former object-KEY disclosure would have passed. Moved to an object KEY; asserts count-only output (mutation-verified against key-joining). - The blob spec's code sketch still showed the vulnerable raw serde message/path; updated to the redacted implementation. - The GC plan still said "field-name shape"; now "field COUNT, never names". - The error.rs test comment still said the map key should appear; corrected. Deletion language: spec pseudocode still said "whole generations only"; now matches invariant 11 (planned/proven atomically, physical delete non-transactional). Dry-run I/O contract: the CLI reference said "no shell-outs" (false — it does a read-only read-back), and the spec said "single describe" (false for chunked — describes the root then every chunk). Both corrected to "read-only, one logical read, possibly several provider calls". Coverage added: sentinel malformed-TOML test for the three provisioning parsers; a failing upsert stderr sentinel test; a CLI test proving the body-aware preflight runs before remote I/O; and a nested destructive age case (recent nested live gen + old nested orphan under one chunk-shaped root) that pins root_live_since's last- infix split (mutation-verified). Stale docs: the pointer-overflow message printed byte length as "characters" (now chars); the helper comment said `.len() <= 8000` (now characters); the migration guide said local pruning removes "exactly" every prior chunk (now notes runtime-readable-root retention). Tests: fastly cli 161 -> 165, cli 165 -> 166. New guards mutation-verified. All gates green including the full wasm matrix.
|
Round 15 addressed — the Unicode mismatch, the security-doc contradictions, and the missing coverage. Pushed as The Unicode char/byte mismatch was a real bug, and a good catch on how it becomes permanent residue. When I moved the direct-vs-chunked threshold to characters in round 14, I left On the redaction docs — you're right that the docs still demonstrated the vulnerable code, which is worse than a stale comment. Fixed together: the blob spec's code sketch now shows the redacted implementation (not The rest:
Coverage — I added all four you listed: sentinel malformed-TOML for the three provisioning parsers, a failing upsert stderr sentinel, a CLI-level test proving the body-aware preflight runs before remote I/O (a reserved On the dry-run e2e: the "reads happen, writes suppressed" behavior is already covered by the existing dry-run tests (the identical-repush dry-run and the corrupt-prior dry-run both reach the writer's report without writing), so I documented the contract accurately rather than adding a redundant heavy test — tell me if you'd still want a dedicated one. Tests: fastly cli 161 → 165, cli 165 → 166. Same caveat — the Viceroy/wrangler WASM contract suites can't run here; the compile/clippy matrix and native suites pass. |
Fixes the merge-blocking read-compatibility regression and tightens the resolver to accept only writer-produced chunk layouts. - Revert the direct-vs-chunked threshold to a conservative UTF-8 BYTE count. A character-based threshold left an envelope under 8 000 chars but over 8 000 bytes stored directly, and rejected the already-stored chunk pointer for it on read (an HTTP 500 for data written by an earlier release). Byte length is always >= character length, so the byte check never over-stores under any reading of Fastly's limit, and it is what existing v1 pointers were written against. Covered by an acceptance test that chunks and resolves a value in that gap. - Extract writer_chunk_spans as the single source of truth for the split layout, shared by the writer and the resolver so the two cannot drift. - Validate EXACT split boundaries after reconstruction: replay the writer's spans over the reconstructed envelope and require the chunk count and every chunk length to match the pointer. The hashes already prove the bytes; this proves the layout, so a hand-authored pointer that reassembles correctly along boundaries the writer would never choose is now rejected on the read path too -- the same guarantee prove_generation enforces before a GC delete. Adds a resolver-to-writer round-trip test across five sizes and a boundary-shift rejection test. - Strengthen the generic CLI I/O ordering regression: drive it with a derived-key overflow (a valid root key whose chunk keys exceed the store limit once the body chunks) rather than a reserved key, so it pins that the full body-aware preflight runs offline before any remote read, not just a key-shape check. - Add local-prune coverage: dry-run count parity against the real deletion count on one fixture, and a real push over a malformed prior pointer warning while deleting nothing. Docs: - Blob spec no longer prescribes serialising the validator's rendered report. On the runtime path that runs after the secret walk, so the report can carry a resolved secret; both sketches now keep only the structural field name, matching the implementation. - Note that a local config validate recovers the exact field path only when the local TOML still matches what was deployed. - Document the local prune's root-like retention, the byte-based value threshold and its v1 stability, Fastly's multi-entry exception to the one-entry-per-adapter rule, and that chunk GC is implemented rather than future work. Drop the atomic whole-generation claim.
|
Thanks — the P1 was mine and you were right that it was merge-blocking. All findings addressed in e42e9f2. P1 — existing v1 pointers unreadable after upgradeReverted. The character-based threshold I introduced in an earlier round was the bug: it left an envelope under 8,000 characters but over 8,000 bytes stored directly, while the already-stored chunk pointer for that same value was rejected on read — an HTTP 500 for data written by an earlier release. The writer is back to the conservative UTF-8 byte count the merge-base used. Byte length is always ≥ character length, so it never over-stores under any reading of Fastly's limit, and it is what existing v1 pointers were written against. Only the key limit remains character-counted (255) — that one is genuinely a character limit and is unrelated to the value threshold. P2 — runtime accepts non-writer layoutsNow validated exactly, after reconstruction. I extracted The hashes already prove the bytes; this proves the layout. A pointer that reassembles to the correct envelope along boundaries the writer would never choose is now rejected on the read path too — the same guarantee Two tests:
P2 — blob spec prescribed a resolved-secret leakFixed in both sketches. On the runtime path Also qualified the "the exact path is available from a local P3s
Gates
One note: |
Two pre-existing breakages surfaced by `clippy -p edgezero-adapter-fastly --features "fastly cli" --target wasm32-wasip1` (not a CI gate, since the `cli` feature targets the native host, but a valid combination that should still compile): - `make_test_envelope` was `#[cfg(unix)]` while several of its callers were not, so the test module failed to compile off unix. The helper only builds a JSON string and has no unix-specific code, so the gate was simply over-restrictive; removed it. - `drop(stdin)` tripped `drop_non_drop` because `ChildStdin` is a non-Drop stub on wasip1. Moved the write into `write_value_to_fastly_stdin`, which takes the handle by value and drops it on return — a natural scope-end drop that closes the pipe on native and satisfies clippy everywhere. Behaviour on native is unchanged.
|
Closed out the
|
… runtime P1 (merge-blocking): the Fastly resolver routed EVERY value through envelope/pointer parsing, so ordinary Config Store entries — "value_a", the documented greeting = "hello" — came back as corruption errors, breaking the shared ConfigStore contract and its wasm32-wasip1 test gate (reproduced under viceroy). A Config Store holds arbitrary values, so the resolver now classifies by `edgezero_kind` and touches ONLY our own chunk pointers: anything else is returned verbatim, and a value carrying an unrecognised `edgezero_kind` (our reserved namespace) is the sole new error. Envelope integrity is unchanged — the typed app-config extractor still parses and verifies the BlobEnvelope after get(). `classify_root_value` is the single discriminant shared by the resolver and GC. Tests that fed non-pointer JSON expecting an error now feed pointer-kind-but-malformed values. P2: the local push rewrote fastly.toml in place, so an interrupted write could truncate it and a concurrent push could drop a sibling edit. The rewrite is now atomic — write a sibling temp file, fsync-free rename over the target (atomic on POSIX) — and refuses to clobber a file that changed under it (conflict detection rather than a lock file, which would strand state on any interrupted push). Covered by conflict and cleanup tests. P3: the runtime resolver rejected non-writer split boundaries but GC only checked metadata + reconstructed content, so a hash-valid 6998/rest split was runtime-unreadable yet counted live by GC and could never satisfy prove_generation — permanent unreclaimable residue. GC now applies the same exact-split predicate (extracted as verify_writer_split_layout) and, staying fail-closed, WARNS that such a root is not runtime-readable instead of silently calling it healthy. New GC test asserts the warning and that nothing is deleted. P3: both cloud-push ordering tests now inject a fake `fastly` on PATH that logs invocations and assert ZERO were made before the offline preflight rejected — so a regression can no longer reach the developer's authenticated CLI. Added a frozen v1 wire-format fixture built from literal constants (7000-byte split, infix, key suffix, pointer fields), independent of the current writer, so coordinated writer/resolver drift can no longer stay green. Docs: blob spec no longer prescribes the pre-redaction map_secret_error / validator-report sketches (they leak resolved secrets); the boundary tests are pinned to the v1 8000-BYTE threshold, distinct from Fastly's 8000-CHARACTER platform cap; the raw-value passthrough contract is recorded. cli-reference corrects bytes-vs-characters and states that a cloud push never deletes (only --local prunes; cloud uses config gc).
|
Thanks — the P1 was a real one and I reproduced it exactly. All findings addressed in b54e710. P1 (merge-blocking) — raw ConfigStore contract + WASM gateConfirmed under viceroy: the resolver routed every value through envelope/pointer parsing, so Root cause: a Config Store holds arbitrary entries, but the resolver treated non-envelope-non-pointer as corrupt. Fixed by classifying on
P2 — local rewrite not atomic
P3 — runtime/GC disagreement (unreclaimable residue)You're right that a hash-valid 6998/rest split was runtime-unreadable yet live to GC, and then permanently unprovable. I extracted the resolver's exact-split check as P3 — orchestration/compat test pinningBoth cloud-push ordering tests now inject a fake P2 — docs
Gates
|
P1 (data loss): the local `fastly.toml` rewrite compared-then-renamed with a TOCTOU window — two concurrent pushes could both pass the compare and the later rename would discard the earlier push's edit. The whole read-modify-write now runs under a cross-process advisory lock (`ManifestLock`, `File::lock` on a persistent sidecar), so pushes serialise and each builds on the previous — both edits survive. A 25-round two-thread test reproduces the loss without the lock (mutation-verified). P2: the atomic replace created the temp file with umask permissions and replaced a symlink with a regular file. It now copies the target's existing permissions onto the temp (a 0600 manifest stays 0600) and canonicalizes the path first, so a symlinked manifest is updated through the link. P2: a transient chunk `LookupError` (TooManyLookups, ConfigStoreInvalid, an unclassified Other) was flattened to a string and mapped to Internal "re-run config push", which cannot fix request-scoped lookup exhaustion. The chunk callback now classifies the error and keeps the transient class as Unavailable (503), matching the root lookup; only a bad/oversized key or value stays corrupt (Internal). P2: GC aborted the whole store when any non-chunk-shaped value failed the envelope/pointer classifier, so one ordinary `greeting = "hello"` sibling blocked all reclamation. A definitively foreign value at an ordinary key is now protected as a zero-reference root. Two guards keep this safe: the value must be provably inert (a new `MalformedObject` classification catches an object-shaped-but-unparseable value — e.g. a truncated pointer — and still fails closed so its chunks are never orphaned), and the key must be outside the reserved `.__edgezero_chunks.` namespace. P2: a failed cloud upsert was labelled "Failed" with committed entries "safe to skip", implying a known boundary — but a timeout can land after Fastly committed the key, including the root pointer. The diagnostic now states the failed entry's outcome is UNKNOWN and directs the operator to re-run the whole idempotent push. The blob spec's matching "previous config stays active" guarantee is corrected. P3: restored the `#[cfg(unix)]` on `fake_spin_returning` that a helper insertion had displaced (unblocks non-unix test builds). Added a frozen v1 multibyte fixture with a 4-byte codepoint straddling byte 7000, hand-split at the retreat boundary via a frozen rule, pinning the codepoint-retreat wire format independently of the writer. Corrected stale docs: the blob read algorithm now describes raw passthrough (the store no longer verifies direct envelopes), and the GC spec/plan describe the atomic locked replace rather than a trailing `fs::write`.
|
Thanks — the P1 was a real TOCTOU and you were right that the earlier conflict-detection didn't close it. All findings addressed in c5689c0. P1 (data loss) — concurrent local pushesThe compare-then-rename had a genuine race: two pushes could both pass the comparison, and the later rename discarded the earlier edit. The whole read-modify-write now runs under a cross-process advisory lock ( P2 — permissions / symlinkThe temp+rename created a new inode at umask perms and replaced a symlink with a regular file. It now copies the target's existing permissions onto the temp (a 0600 manifest stays 0600) and P2 — transient chunk lookupsThe chunk callback flattened every P2 — one foreign sibling blocking GCA
New test reclaims a dead generation despite a foreign sibling; the existing truncated-pointer and non-canonical-key fail-closed tests still pass. (The runtime resolver treats P2 — cloud push unknown outcomeA failed upsert was labelled "Failed" with committed entries "safe to skip", implying a known boundary — but a timeout can land after Fastly committed the key, including the root pointer. The message now states the failed entry's outcome is UNKNOWN and directs a full idempotent re-run (content-addressed keys + P3s
Gates
|
Four blockers plus follow-ups from review. P1 (data loss): GC classified an object carrying an UNKNOWN/future or non-string `edgezero_kind` as a direct envelope, because `BlobEnvelope` ignores unknown fields — so a future-format pointer with envelope-shaped fields became a zero-reference root and its canonical chunks were reclaimed. `classify_root_value` now discriminates on the PRESENCE of `edgezero_kind` (any non-recognised value is UnknownKind, never Foreign), `gc_classify_root` routes through it and fails closed on UnknownKind / MalformedObject, and the chunk-shaped candidate arm excludes any value that announces our namespace. Runtime rejects the same values. P1 (data loss): the manifest lock covered only local config push; provision overwrote fastly.toml with an unlocked `fs::write`, and the lock keyed on the lexical path while replacement canonicalized later — so a symlink and a direct path locked different sidecars. `ManifestLock` now resolves the real target once (shared by lock and replace), `append_fastly_setup` takes the same lock and uses the atomic replace, so provision and push serialise on one target. P1 (CI): `value_is_inert_foreign` was dead under the fastly-only wasm build, failing the required clippy gate. It is now exercised by a classifier unit test, and the previously-missing `fastly cli` wasip1 combination is added to the CI matrix so this class of regression is caught. P1 (correctness): pointer-last is not an atomic cloud flip — Config Store is eventually consistent across keys, so a POP can see a new pointer before its chunks propagate. A MISSING referenced chunk now maps to Unavailable (503, retryable) instead of Internal (500); a retry resolves the propagation window. Spec's "atomic flip" claim corrected. P2: atomic staging used a predictable pid-only temp path written before permissions were applied. It now creates the temp with `create_new` (O_EXCL, never follows a planted symlink), copies the target's permissions BEFORE writing, syncs before rename, and a `TempFileGuard` removes it on any failure. Permissions are preserved and symlinks followed to the real file. P2: `ValueTooLong` is now transient (Unavailable), not corruption — the SDK already retried at the reported size, so it reaching us means the value grew between host calls, a race a retry resolves. P2: the "frozen v1" fixtures now assert against PRECOMPUTED literal wire hashes (envelope, inner, and per-chunk) and feed them into the pointer as literals, so a serialization or hash-helper drift fails the test instead of silently rebuilding a new golden. P3: distinct GC reporting — the cloud run no longer pre-prints planned keys as "deleting"; execution reports deleted / FAILED / skipped per key, and the local dry-run no longer counts prior chunks already absent from the file (the real prune's remove is a no-op there). Generated projects gitignore the lock sidecar. Recovery commands note their POSIX/bash quoting. Stale GC/blob docs updated (inert-foreign siblings, root-like-orphan warnings, local-only cloud pruning, missing-chunk-as-transient).
|
Thanks — the two data-loss P1s were real and I reproduced both. All 12 findings addressed in a17d159. P1 — GC deletes a future-kind root's chunks (data loss)Confirmed: Fix: P1 — manifest lock coverage + path aliasing (data loss)Right on both counts. P1 — required WASM clippy gate
P1 — pointer-last is not an atomic cloud flipAgreed — there's no cross-key atomicity and Config Store is eventually consistent across POPs. Rather than claim atomicity, a missing referenced chunk now maps to P2s
P3s
Gates
|
Three data-recovery/loss blockers plus follow-ups. P1 (broken recovery): a cloud `config push` first resolved the existing remote pointer, so a missing chunk / hash mismatch / malformed pointer aborted the push BEFORE the repairing write — recovery required manual Fastly edits, contradicting the runtime's "re-run config push to repair" contract. A new `ReadConfigEntry::Corrupt` distinguishes an EXISTING entry whose value will not resolve (the describe succeeded) from a real IO error. The push treats Corrupt like an absent remote: it warns and overwrites (the in-band repair), with the same behaviour now uniform across cloud and local. P1 (data loss): local pruning deleted a prior chunk key whose value carries an unknown/future/non-string `edgezero_kind` — a value the cloud GC path already fails closed on. The prune's protection predicate now keeps anything that ANNOUNCES our namespace (via `value_announces_our_kind`) or classifies as a root, and the dry-run count mirrors it. An older CLI no longer destroys a newer-format entry. P1 (data loss): a malformed pointer at a chunk-shaped root could lose its nested generation — the truncated value looks like a chunk fragment, so it became a candidate while its independently-provable nested chunks were deleted with no readable root to name them. A chunk-shaped candidate that has any canonical chunk nested beneath it now fails closed. A real leaf payload has no nested chunks, so normal GC is unaffected. P2: `canonical_manifest_target` now follows a DANGLING manifest symlink to its intended target (creating that file, preserving the link) instead of replacing the symlink with a regular file. `atomically_replace_file` propagates `sync_all` errors (an ENOSPC/EIO during writeback fails the command before the known-good manifest is replaced, not after) and syncs the containing directory after the rename. P2: added command-level GC coverage through `run_config_gc` — manifest load, store resolution, adapter-registry dispatch, listing, classification, and reporting together — plus an integrated unknown-kind fail-closed regression. P3: recovery commands already carry a POSIX/bash note; the generated gitignore now globs `.*.edgezero-lock` (covering a non-default manifest name); and the GC spec's fail-closed list and `prior_chunk_keys` note are reconciled with the implemented inert-foreign-sibling and gc_classify_root behaviour.
|
Thanks — findings 1–3 were all real, and they were the mirror image of last round's hardening (I fixed cloud/runtime but left the sibling paths). All 8 addressed in 210416b. P1 — cloud push can't repair a broken generationYou're right that P1 — local prune deletes unknown/future-kind rootsFixed the asymmetry: the prune's protection predicate now keeps anything that announces our namespace ( P1 — malformed pointer at a chunk-shaped root loses its nested generationThis was the subtle one. A truncated pointer at a chunk-shaped key can't announce its discriminator, so it looked like a leaf fragment and became a candidate — while its independently-provable nested chunks got deleted with no readable root to name them. The candidate arm now additionally requires that nothing is nested beneath the key: if any canonical chunk of this key exists in the listing, it's treated as an unreadable nested root and fails closed. A real leaf payload never has nested chunks, so normal GC is untouched. New test builds exactly that shape (malformed pointer + aged provable nested generation) and asserts the whole run refuses and deletes nothing. P2s
P3s
Gates
|
…ymmetry
Corrects the `Corrupt` read outcome, which was both too narrow and too
broad, and closes remaining local/cloud asymmetries.
Corrupt was too NARROW: a malformed direct value, invalid JSON, or a
direct-envelope SHA mismatch passed through as `Present`, and the push then
aborted parsing it — recovery needed manual deletion. `read_config_entry`
(cloud and local) now VERIFIES the resolved value as an envelope; a
non-verifying value is `Corrupt` (repairable by overwrite).
Corrupt was too BROAD in two ways, both of which could destroy healthy or
newer state:
- An infrastructure chunk-fetch failure (fastly can't spawn, auth fails,
schema drift) became `Corrupt`, letting `push --yes` overwrite healthy
remote after an INCOMPLETE read. The fetch callback now records
infrastructure failures, and such a read stays a hard error.
- An unknown/future `edgezero_kind` became `Corrupt` and was offered for
ordinary overwrite. An older CLI must not clobber a newer format, so it
is now a hard read error (upgrade the CLI). All three cases route
through one `classify_resolved_read`.
Local prune was not symmetric with cloud GC: it deleted a truncated pointer
at a chunk-shaped key even when a canonical chunk was nested beneath it. The
protection predicate (and the dry-run count) now also keep a key that has a
nested generation, matching cloud GC's fail-closed rule.
`config diff` / push dry-run / pre-write recheck no longer model a corrupt
remote as absence or a clean comparison: diff reports a distinct
`CorruptRemote` outcome that exits "could not compare" (2) regardless of
`--exit-code`; the dry-run stops fabricating a local-vs-empty diff; and the
recheck reports a Present→Corrupt transition precisely instead of as a
removal.
`canonical_manifest_target` follows the WHOLE symlink chain (bounded), so a
multi-hop dangling manifest link writes at the final target and every
intermediate link is preserved — and the same target keys one lock.
The atomic replace no longer silently skips permission preservation: a
metadata error other than NotFound fails rather than widen access.
Tests: `classify_resolved_read` taxonomy; a malformed nested-root holder is
kept by local prune; a multi-hop dangling symlink chain; a command-level GC
test that the ENV-derived platform name drives store selection; the corrupt
diff exit code. Docs: blob spec adds `Corrupt` to `ReadConfigEntry`; the GC
spec's local-preservation predicate now matches the implementation.
|
Thanks — this was a sharp review. The three Highs were all about the High #1 — Corrupt too narrow (malformed direct values un-repairable)
High #2 — infra fetch failures must not be CorruptRight — an incomplete read must never license overwriting healthy remote state. The chunk-fetch callback now records infrastructure failures ( Medium #4 — unknown/future kind must stay a read errorAlso right, and it's the mirror of #2: an unknown/future All three now route through one High #3 — local prune asymmetry on a malformed nested rootThe local prune's protection predicate (and its dry-run count) now also keep a key that has a canonical chunk nested beneath it — a truncated/unreadable nested root — matching the cloud GC fail-closed rule I added last round. New test builds that exact shape and asserts the holder is kept. Medium #5 — diff/recheck modeling corruption as absence
Medium #6 — multi-hop dangling symlink
Medium #7 — command-level GC destructive routingAdded a command-level test that the ENV-derived platform name ( Low #8
Gates
|
Address the current review round on the chunk-GC branch.
P1 — Future envelope AND pointer versions must not be overwritten. Add
`value_is_future_format` (raw-value predicate: a bumped envelope/pointer
version, or any unknown `edgezero_kind`) and check it FIRST in the CLI read
classifier, returning a hard error instead of a repairable Corrupt. Guard the
local-prune and dry-run predicates and the cloud GC candidate arm with it so a
v2 direct envelope under a chunk-shaped key is never deleted. This restores the
v1-reader fail-closed rule from the blob spec.
P1 — Ambiguous Fastly stderr ("not found"/"does not exist"/"404") for a CHUNK
now sets `fetch_failed` (Ok(None) => incomplete read) so a partial read becomes
a hard infrastructure error, not an overwriteable Corrupt.
P2 — Make the Corrupt repair contract adapter-agnostic. Centralise it in the
generic push layer (`classify_present_body` in cli/config): a Present body that
parses+verifies is Valid; one that fails to parse or mismatches its SHA is
Corrupt (push overwrites to repair); a bumped envelope version is a hard error.
Axum/Cloudflare/Spin now get repair without each implementing Corrupt.
P2 — Local locking safe across file aliases. Refuse a manifest with more than
one hard link (an atomic rename would break the link and path-based locks
cannot serialise the other names). Symlink resolution now fails closed on a
read-link error or hop-limit instead of falling back to a writable path.
P2 — Runtime remediation is correct for future formats: a value the running
build cannot parse as its own format asks the operator to redeploy an updated
build, not to re-push (re-pushing cannot help).
P3 — Threshold-free GC dry-run now prints a usable apply instruction
(`--yes --older-than <dur>`), matching the requirement that a non-zero window
is mandatory.
P3 — Provision docs corrected: provision writes only `[setup.*]`; the
`[local_server.*]` seeding is done by `config push --local`.
|
Thanks — all seven findings addressed in P1 — Future envelope AND pointer versions could be overwritten. Added P1 — Ambiguous Fastly stderr for a CHUNK. The chunk-fetch closure now treats an ambiguous not-found ( P2 — Corrupt repair contract only implemented by Fastly. Centralised the contract in the generic push layer ( P2 — Local locking unsafe across file aliases. P2 — Runtime remediation wrong for future formats. When the running build can't parse a stored value as its own format, P3 — Threshold-free dry-run apply instruction. The GC dry-run now prints P3 — Provision docs. CLI reference and walkthrough now state provision writes only Gates (all green): |
…store reads
Address the follow-up review round. Findings 1-3 are the P1 fail-closed gaps.
P1 -- Cloud GC no longer treats a FUTURE direct envelope as a zero-reference
foreign root. A direct envelope from a newer writer classifies as `Foreign` (no
`edgezero_kind`), so the ordinary-value fallback would wave it through and plan
its (unknown-scheme) chunks for deletion. Exclude `value_is_future_format` from
that fallback so it fails closed with nothing deleted.
P1 -- Future-format detection is now TYPED through pointer resolution. The
resolver returns `ResolveFailure::{FutureFormat,Corrupt}` instead of an untyped
string, and checks `value_is_future_format` on the REASSEMBLED envelope BEFORE
deserializing it as v1 -- so a v2 envelope wrapped in a valid v1 pointer (only
knowable after reassembly) can no longer erase into repairable corruption a
downgrade push would overwrite. The generic push path does the same version
pre-check before v1 deserialization, covering adapters whose Present body no
longer parses as the exact v1 schema.
P1 -- Root and store OPERATIONAL failures no longer read as absence. Store
resolution maps to `MissingStore` only on the resolver's own unambiguous "no
fastly config-store matches" signal (a bare "not found" also matches "`fastly`
not found on PATH" and list/auth/network errors). Entry `describe` maps to
`MissingKey` only when the stderr carries a clean absence marker AND no
operational marker (auth, network, 5xx, rate-limit), so two incomplete reads can
no longer pass the pre-write recheck and authorise an overwrite.
P2 -- Reconcile the `Corrupt` contract doc: only a PROVABLY unusable, fully-read
value is repairable; a value that could not be fully read (an absent/unfetchable
chunk, indistinguishable from an incomplete read) fails closed as a hard error,
never `Corrupt`.
P2 -- A DIRECT future envelope now gets the redeploy remediation at runtime. It
passes the resolver as a foreign `Ok`, so the future-format check is applied on
the SUCCESS path too -- it no longer reaches core as a generic integrity 500.
P2 -- The hard-link check works on Windows via `MetadataExt::number_of_links()`
(stable, no new deps), so Windows hard-link aliases fail closed too, not just
Unix ones.
P3 -- CI now runs the fastly runtime `--lib` unit tests under Viceroy; the WASM
job previously ran only `--test contract`, so the runtime remediation/fail-closed
tests compiled but never executed.
Tests: typed inner-future resolution, GC fail-closed on a future direct
envelope, operational-vs-absence stderr, direct-future redeploy remediation, and
a future envelope that fails v1 deserialize.
|
Thanks — all seven addressed in P1 — Cloud GC treated a future direct envelope as a zero-reference foreign root. A direct envelope from a newer writer classifies as P1 — A v2 envelope inside a v1 pointer could be overwritten by a downgrade push. Version detection is now typed through pointer resolution: P1 — Root/store operational failures were classified as absence. Store resolution maps to P2 — Missing-chunk contract. You're right that the fail-closed chunk behaviour contradicted the P2 — Direct future envelope runtime remediation. A direct future envelope passes the resolver as a foreign P2 — Non-Unix hard links. The check now reads the link count on Windows via P3 — CI runs the runtime tests. Added a fastly-only step that runs Gates (all green): fmt · workspace clippy |
…rns verbatim
Address the follow-up review round (four P1 data-loss/overwrite paths, three P2).
P1 -- Future-format detection is now SCHEMA-AGNOSTIC. `value_is_future_format`
and the generic `body_is_future_envelope` key on the `version` field alone (and,
generically, on the presence of `edgezero_kind`), not the four v1 fields. A
future shape like `{"version":2,"payload":...}` that drops v1 fields no longer
slips through as repairable corruption (overwriteable by a downgrade push, and
treated as an inert zero-reference root by GC).
P1 -- Operational errors can no longer become absence. A read maps to
MissingKey / an absent chunk ONLY on a CONFIRMED clean absence
(`stderr_is_confirmed_absence`: a not-found token with NO operational marker);
the marker list now includes 401/403/429 and an HTML "page not found". Store
resolution is now TYPED (`resolve_remote_config_store_id` returns
`Ok(None)` only when the list SUCCEEDS and no store matches), not a substring
match on an untyped error. Ambiguous/operational output stays a hard error.
P1 -- Destructive GC no longer accepts lossy listing input. All value-bearing
`fastly` stdout (the GC listing, root/chunk describes, the store-id list) is
converted with STRICT UTF-8 via `strict_stdout`; invalid bytes fail closed
instead of becoming U+FFFD and mutating a root value or chunk.
P1 -- The generic push now refuses an unknown `edgezero_kind`. A v1-shaped
envelope carrying `edgezero_kind: "new_format"` (which serde would otherwise
accept, ignoring the extra field) is a newer format non-Fastly adapters must not
overwrite.
P2 -- A CONFIRMED-missing Fastly chunk is now repairable. A clean not-found on a
chunk fetch resolves to `Corrupt` (a push overwrites to repair, as the blob spec
prescribes for persistent loss), while an operational/ambiguous fetch failure
stays a hard error. The confirmed-absence discrimination is what lets both hold:
persistent loss is fixable, an incomplete read still never authorises overwrite.
P2 -- The runtime store returns arbitrary DIRECT values VERBATIM again. The
store-layer inspection of direct values is reverted; the future-envelope
remediation moves to the typed app-config extractor, which now maps an
`UnknownVersion` (and a schema-changed future version) to an upgrade/redeploy
error instead of a generic integrity 500. The store still errors on a newer
value in OUR namespace (an unknown `edgezero_kind`, a future pointer/inner
version), detected before the exact-v1 struct deserialize.
P2 -- The minimal CLI feature graph lints warning-free:
`cargo clippy -p edgezero-cli --all-targets -- -D warnings` no longer trips on
unused `ResolveFailure` payloads / `into_message`, which the read classifier now
surfaces in its redacted infra diagnostic.
|
Thanks — all seven addressed in P1 — Schema-changing v2 envelopes failed open. P1 — Operational errors becoming absence. Introduced P1 — Lossy GC listing input. All value-bearing P1 — Generic push missed unknown P2 — Permanently missing Fastly chunk had no repair path. This is the crux, and it's what reconciles this round with the prior one. A confirmed clean absence on a chunk fetch ( P2 — Store layer stopped returning direct values verbatim. Reverted the store-layer inspection of direct values ( P2 — Minimal CLI feature graph lint. Gates (all green): fmt · workspace clippy |
Summary
Fastly
config pushchunked storage was upsert-only, so re-pushing changed app config leaked the previous generation of chunk entries: chunk keys are content-addressed by the envelope SHA, so a config change rewrites every chunk key and orphans the old set. This affected both the cloud path (remote Config Store) and the local path (fastly.toml[local_server.config_stores.<name>.contents]).This PR adds best-effort chunk garbage collection on re-push — reclaiming the prior generation the moment a new pointer supersedes it — plus the design spec and TDD implementation plan.
Concurrency: last-writer-wins. Concurrent cloud pushes are supported (the last root-pointer write wins on the value). A push reclaims prior chunks only while a post-commit read-back confirms it is still the last writer of that root; otherwise it yields and deletes nothing, so a superseded push never removes the winner's live chunks. Best-effort, not transactional (Fastly has no compare-and-delete) — the residual is documented, bounded to one store's chunk data, and surfaces as a read-time integrity error rather than wrong data. See the spec's "Concurrency model: last-writer-wins".
Changes
docs/superpowers/specs/2026-07-07-fastly-chunk-gc.mddocs/superpowers/plans/2026-07-07-fastly-chunk-gc.mdcrates/edgezero-adapter-fastly/src/chunked_config.rsprior_chunk_keys(root_key, raw)— Value-first, v1-validated, prefix-scoped extraction of a prior pointer's chunk keys + unit testscrates/edgezero-adapter-fastly/src/cli.rsFastlyConfigGcPlan,expand_root,orphan_chunk_keys,reject_reserved_root_keys,local_contents_table); local prune in the samefastly.tomlrewrite + best-effort dry-run counts; cloud post-commit sweep with last-writer-wins read-back guard +delete_config_store_entry(--key --auto-yes, never--all) + offline dry-run intent; reserved-key rejection at the adapter boundary; ~30 new testscrates/edgezero-adapter-fastly/Cargo.toml,Cargo.lockhandlebarsdev-dependency (renders the cloud fake-fastlytest shim)Behavior details:
.__edgezero_chunks.) are a hard error at the adapter boundary — they'd collide with the chunk namespace.--dry-runstays offline (cloud reports GC intent without a count; local reports an exact count, and classifies malformed prior state asunknown: could not read prior staterather than0).config gc.Closes
Closes #313
Test plan
cargo test -p edgezero-adapter-fastly --features cli— 112 passedcargo test --workspace --all-targets— all passcargo clippy --workspace --all-targets --all-features -- -D warnings— cleancargo fmt --all -- --check— cleancargo check --workspace --all-targets --features "fastly cloudflare spin"— cleancargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin— cleanCoverage highlights: prior-pointer validation (valid/direct/garbage/wrong-kind/bad-version/foreign-prefix); reserved-key rejection (local + cloud); local prune / shrink-to-direct / suspicious-pointer-skip / sibling-chunk preservation; local dry-run counts (exact / identical-repush-zero / non-table-unknown / suspicious-unknown); cloud deletes-prior-keeps-new / read-back concurrency skip / identical-repush-no-deletes / no-prior / delete-failure-warns / prior-read-failure-warns / shrink-to-direct / delete argv asserts
--key+--auto-yesand never--all.Checklist
edgezero_core(n/a — adapter-internal code)