From e54f066552f09a77f0d77f73fd9d0b57893c8cd2 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 00:06:07 -0700 Subject: [PATCH 01/14] Add design specs for the provider architecture and permission model epic Five specs covering the work proposed in PR #838 (issues #777-#782), written so the next implementation pass has a normative behavioral contract: - Pluggable providers: identity lifecycle contract (mint/recognize/hash/ tombstone), trait minimalism, adapter parity, validation table - Permission model: signal precedence (opt-out over TCF), fail-closed jurisdiction resolution, policy file validation, decision-matrix testing - Migration and rollout: behavior-preservation matrix, ID stability vectors, loud-failure requirements, operator recipes - Client-cycle EC resolve endpoint: threat model and prerequisites; on hold until its open questions get an issue - Integration response-header hook: #782 contract with ordering and collision policy, ships only with a real consumer --- ...26-07-30-client-cycle-ec-resolve-design.md | 120 +++++++++ ...integration-response-header-hook-design.md | 69 +++++ .../2026-07-30-permission-model-design.md | 254 ++++++++++++++++++ .../2026-07-30-pluggable-providers-design.md | 239 ++++++++++++++++ ...07-30-provider-migration-rollout-design.md | 161 +++++++++++ 5 files changed, 843 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md create mode 100644 docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md create mode 100644 docs/superpowers/specs/2026-07-30-permission-model-design.md create mode 100644 docs/superpowers/specs/2026-07-30-pluggable-providers-design.md create mode 100644 docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md new file mode 100644 index 000000000..674de01ff --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -0,0 +1,120 @@ +# Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint + +**Status:** Draft — **prerequisites unmet; do not implement against this spec +until its open questions (§7) are resolved in a dedicated issue** +**Author:** Engineering +**Issue references:** none yet (this spec exists to force one; #778 does not +cover this feature) +**Related specs:** `2026-07-30-pluggable-providers-design.md` +**Last updated:** 2026-07-30 + +> **Context.** PR #838 shipped, undeclared and unspec'd, a second provider +> _type_: a "client-cycle" EC provider whose identifier is established by a +> browser POST to a new public endpoint (`POST /_ts/api/v1/ec/resolve`), +> plus a demo provider (`client-fixed`) and a JS bundle. Review found the +> endpoint accepted cross-origin identity-setting posts with no origin +> check, minted cookies with no identity-graph row (violating an invariant +> the organic path enforces explicitly), was registered on only one of four +> adapters, and could never round-trip because the core did not recognize +> non-HMAC identifiers. None of that is an argument the feature is a bad +> idea — vendor identity systems with a browser leg (e.g. signed-envelope +> schemes) are a real integration target. It is an argument that the feature +> needs a threat model before an implementation. This spec is that threat +> model and the bar an implementation must clear. + +--- + +## 1. Overview + +A **client-cycle** EC provider establishes the identifier via a browser +round trip: server-injected first-party JS obtains or derives a value in the +page (typically a signed envelope from a vendor identity system), posts it to +a Trusted Server endpoint, and the endpoint — after provider-specific +verification — sets the first-party `ts-ec` cookie. + +This differs from server-side providers in one security-critical way: **the +identifier is attacker-influenceable input**, not server-derived evidence. +Everything in this spec follows from that. + +## 2. Threat model + +| Threat | Vector | Consequence if unmitigated | +| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar — login-CSRF for the ad-identity layer; the victim's activity accretes to an attacker-controlled ID | +| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to mint payloads | +| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | +| **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity — a compliance failure, not just a bug | +| **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | + +## 3. Requirements on the endpoint + +`POST /_ts/api/v1/ec/resolve` (final path TBD) MUST: + +1. **Reject cross-site requests.** Require a same-site assertion: `Origin` + (or `Sec-Fetch-Site: same-origin/same-site`) validated against the + publisher's origin set; requests without a validating header are + rejected. CSRF-token designs are acceptable but not required if + origin-based rejection is enforced. +2. **Verify the payload cryptographically per provider.** The provider's + `resolve_from_client` accepts only payloads that are signed by an + expected party, **audience-bound** to this publisher, and **expiring** + (bounded lifetime, single-use where the scheme allows). A provider whose + payloads are replayable constants fails this bar by construction. +3. **Preserve the identity-graph invariant.** The cookie is set only after + the corresponding graph row is written, mirroring the organic path. Graph + unavailable → no cookie, same as organic generation. +4. **Round-trip through the lifecycle contract.** The identifier set here + must be recognized, hashed, and tombstonable by the selected provider + (providers spec §3). The conformance suite runs against every + client-cycle provider. +5. **Exist on every adapter.** Route registration goes through shared route + wiring; the parity suite asserts the endpoint's presence and behavior on + all four adapters. (PR #838 registered it on Fastly only, so the same + config on the Axum dev server proxied the POST to the publisher origin.) +6. **Be uncacheable and permission-gated.** `Cache-Control: no-store`; + the same `store-on-device` permission gate as organic EC creation runs + before any cookie is set. + +## 4. Requirements on the page script + +- The re-post guard must not depend on reading an HttpOnly cookie. Either + the server injects a "resolved" marker the script _can_ read (a + non-identity companion cookie or an injected page variable), or the + endpoint is cheap-idempotent and rate-limited per session; the design must + state which and test it. +- The JS module ships through the standard integration bundle mechanism, + loaded only when a client-cycle provider is the selected EC provider. +- Any constant shared between Rust and TS (endpoint path, marker name) is + asserted equal by a test, not "kept in sync by hand". + +## 5. Demo providers + +A demonstration provider (fixed identifier, no verification) fails §3.2 by +design and therefore MUST NOT be selectable in a production build: gate it +behind a cargo feature or `#[cfg(test)]` so the settings validator does not +accept its key in release artifacts. PR #838's `client-fixed` was selectable +in any production config, giving every visitor the same identity, with a doc +sentence as the only guardrail. + +## 6. Testing + +- Endpoint: origin-rejection, expired/replayed/foreign-audience payload + rejection, graph-unavailable refusal, permission-gate refusal — each as an + integration test, not only unit tests. +- Browser round trip (JS → POST → Set-Cookie → next request recognized) in + the integration suite; PR #838 shipped the JS with in-process unit tests + only, including one asserting a state (reading the HttpOnly cookie) that + cannot occur in a real browser. +- Parity: all four adapters. + +## 7. Open questions — to be settled in the feature's issue before any code + +1. Which concrete vendor scheme is the first real consumer, and does its + envelope format satisfy §3.2 (audience binding, expiry)? If no concrete + consumer exists, the feature waits — the demo provider is not a + consumer. +2. Does the resolve flow need consent-state echo in its response (so the + page can react), and if so what is the minimal disclosure? +3. Rate limiting / abuse posture at the edge for an unauthenticated POST. +4. Whether the endpoint should be versioned separately from the identify + API family it sits beside. diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md new file mode 100644 index 000000000..59d354547 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -0,0 +1,69 @@ +# Design Spec: Integration Response-Header Hook + +**Status:** Draft +**Author:** Engineering +**Issue references:** #782 +**Related specs:** `2026-07-30-pluggable-providers-design.md` +**Last updated:** 2026-07-30 + +> **Context.** Issue #782 already specifies this feature well; its done-when +> is the contract. PR #838 shipped the trait and registry wiring with **no +> adapter call site** — `apply_response_headers` had zero production +> callers, so the feature existed only in its own unit test. This short spec +> restates the contract plus the two details the issue left open (ordering +> and collision policy), and adds the rule that prevents a repeat: the hook +> lands with a consumer or not at all. + +--- + +## 1. Overview + +Integrations can today rewrite request-path behavior (proxies, attribute +rewriters, head injectors) but cannot mutate **response** headers. The hook +adds that: an integration registers a response-header mutator via its +`IntegrationRegistration` builder, and every adapter applies all registered +mutators to the outbound response for HTML document responses it processed. + +## 2. Contract + +- `IntegrationRegistration::builder(ID).with_response_mutator(...)` registers + a mutator; `IntegrationRegistry::apply_response_headers(...)` applies all + registered mutators in registration order. +- **Every adapter calls the apply point** on its outbound-response path for + processed documents. The call site lives in shared response-finalization + code where one exists; where adapters finalize independently, each adapter + gains the call and a test proving it. +- Mutators run **after** Trusted Server's own response-header handling + (EC Set-Cookie emission, EC header clearing, privacy headers) so a mutator + cannot be silently clobbered by later core steps — in PR #838's ordering, + provider-supplied headers were inserted before the EC header-clearing pass + and could be stripped by it. + +## 3. Collision policy + +- Mutators may not touch **reserved headers**: `Set-Cookie` for the EC + cookie, the `x-ts-*` namespace, and the consent/privacy headers core + emits. Attempts are dropped and logged at `warn` with the integration id. +- For non-reserved headers, the mutator API distinguishes **append** from + **replace** explicitly; the default is append. Replacing a header the + origin set is a deliberate act, visible in the mutator's code. +- Later registrations see earlier mutations (order = registration order, + which is deterministic). + +## 4. Done-when (from #782, sharpened) + +1. Trait + builder + registry application, each public item documented. +2. **At least one real consumer ships in the same PR** — an existing + integration registering a mutator for a real need (or, failing a real + need, the feature waits; scaffolding with only self-referential tests is + dead code and will be removed). +3. Every adapter applies mutations on its outbound path, with a per-adapter + route test asserting an integration-set header appears in the response. +4. A parity-suite case asserts identical mutation behavior across adapters. +5. Reserved-header and append/replace semantics covered by unit tests. + +## 5. Size + +This is a ~150-line feature plus tests. It has zero coupling to the provider +architecture or the permission model and should land as its own small PR, +first in the epic's sequence. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md new file mode 100644 index 000000000..f3504a04d --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -0,0 +1,254 @@ +# Design Spec: Jurisdiction Permission Model + +**Status:** Draft +**Author:** Engineering +**Issue references:** #779 +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-provider-migration-rollout-design.md` +**Last updated:** 2026-07-30 + +> **Context.** PR #838 proposed a permission model whose review surfaced two +> classes of defect this spec exists to prevent in the next pass: (1) silent +> behavioral inversions of consent-signal precedence — most seriously, a +> present TCF string short-circuiting GPC/GPP/US-Privacy opt-outs — and +> (2) fail-open jurisdiction resolution when geolocation is disabled. The +> precedence table (§4) and the failure-mode matrix (§6) are the two +> documents whose absence allowed those defects to hide in a 67-file diff. +> They are normative: an implementation whose behavior differs from these +> tables is wrong, whatever its tests say. + +--- + +## 1. Overview + +The permission model replaces the hard-wired jurisdiction gate +(`allows_ec_creation` and its companions) with a single resolved +**permission set** per request. Every data decision Trusted Server itself +makes — EC creation, EC withdrawal, EID transmission into the bidstream, +and provider execution (see providers spec §5) — reads that set. + +The set is resolved from three inputs: + +1. **Jurisdiction** — the country/region the request resolves to (§5). +2. **Policy** — a declarative, version-controlled map from jurisdiction to a + baseline acquisition rule per permission (§3). +3. **Signals** — the request's privacy signals: TCF, GPP, GPC, US Privacy + (§4). + +Scope: the model governs decisions Trusted Server makes. Downstream RTB +partners receive the full, unmodified regulatory context and make their own +compliance decisions. + +## 2. Vocabulary: enforced permissions only + +Permissions are named by IAB TCF Europe purpose identifiers, used strictly as +technical identifiers (no CMP or TCF policy is implemented by naming them). + +**Rule: a purpose appears in the model only when it has both a signal mapping +and an enforcement point.** PR #838 shipped 11 purposes of which 9 were +inert — computed into the bitset and consumed by nothing but a startup log — +while the policy file invited operators to set flags (e.g. +`market-research: denied`) that changed nothing. A policy vocabulary that +overstates what is enforced is a compliance hazard, not forward +compatibility. + +The initial vocabulary is therefore exactly: + +| Identifier | TCF purpose | Enforcement points | +| ------------------------- | ----------- | -------------------------------------------------------------------- | +| `store-on-device` | 1 | EC provider execution; EC creation; withdrawal/tombstone eligibility | +| `select-personalised-ads` | 4 | EID transmission into the bidstream (jointly with `store-on-device`) | + +(The identifier strings are the IAB names verbatim, including their original +spelling.) The extension procedure — add the signal mapping, add the +enforcement point, add the policy vocabulary entry, in one change — is +documented in the policy file header. Policy validation **rejects** a rule +that references an identifier outside the current vocabulary, so the file can +never promise more than the code enforces. + +## 3. Policy + +### 3.1 Format + +A YAML map, embedded at build time, of named **groups** (baselines) and +**rules** keying countries (`FR`) and country/state pairs (`US/CA`) to a +group with optional per-permission overrides. Each permission resolves to an +**acquisition rule**: + +- `granted` — set without any signal, +- `requires_signal` — set only when a signal grants it (opt-in), +- `denied` — never set, even when a signal grants it. + +Overrides support all three targets: `+perm` (granted), `-perm` (denied), and +`~perm` (requires_signal). PR #838 supported only `+`/`-`, making the most +common real-world override — "this state requires a signal for personalized +ads" — inexpressible without duplicating a whole group. Groups may use a +`default:` shorthand for unlisted permissions. + +### 3.2 Validation — at build time, not request time + +The embedded file is validated by a `build.rs` step (or an equivalent +always-run CI test that asserts the parse explicitly): a malformed committed +file fails the **build**, never a request. PR #838's file was parsed lazily +behind a `OnceLock` with an `expect`, meaning a bad edit that escaped unit +tests became a 500 on every request. + +Validation rejects: + +- unknown fields anywhere (`deny_unknown_fields` on every deserialized + struct — PR #838's untagged rule enum silently swallowed a misspelled + `permission:` key, dropping the operator's override with no diagnostic); +- rule keys that are not plausible ISO 3166-1 alpha-2 / ISO 3166-2 codes; +- references to permissions outside the enforced vocabulary (§2); +- groups that neither list every permission nor provide `default:`. + +### 3.3 One source of jurisdiction truth + +The codebase currently carries a second, runtime-configurable jurisdiction +list (`consent.gdpr.applies_in`) used by the auction consent gate. Two +independently maintained country tables that both express "where GDPR +applies" will drift (in PR #838, adding `CH` to one had no effect on the +other). Requirement: either the auction gate derives its jurisdiction class +from the same resolved policy, or a CI test asserts that every country in +`applies_in` resolves to an opt-in (`requires_signal`) baseline in the policy +file, and vice versa for the shipped defaults. + +### 3.4 Shipped table coverage + +A CI test asserts every member of the GDPR country list resolves to an opt-in +baseline (this is what catches a `DK:` typoed as `DL:` — a defect that in +PR #838 survived parse, startup, and all tests, silently dropping Denmark to +the operator default). Countries intentionally not listed are governed by +§5's default-country rules; the policy header documents that this is the +fallback, and the shipped example default is the most protective baseline. + +## 4. Signal precedence — normative table + +Signals are classified: + +- **Opt-out signals** (affirmative withdrawal): GPC header; GPP sections + carrying a sale/sharing opt-out; US Privacy opt-out. +- **Consent records**: a decodable TCF string (standalone or embedded in + GPP), which may grant or refuse individual purposes. + +**Precedence, highest first:** + +1. Policy `denied` — never set, regardless of any signal. +2. **Opt-out signal — always revokes**, regardless of any consent record + present. A GPC header revokes `store-on-device` and + `select-personalised-ads` even when an accompanying TCF string consents to + them. _(This is the rule PR #838 inverted: its resolution returned from + inside the TCF branch before ever reaching the opt-out check, so a + consenting CMP string made the browser's GPC signal a no-op — a + CCPA-facing regression. The pre-existing tests pinning this rule — + `ec_blocked_us_state_gpc_overrides_tcf` and companions — are reinstated + against the new API, not deleted.)_ +3. Consent record refusal — a TCF record present and refusing the purpose + revokes it. +4. Consent record grant — a TCF record present and consenting grants it + (subject to 1–2). +5. No signal — the policy baseline decides: `granted` sets it, + `requires_signal` leaves it unset. + +### 4.1 Decision matrix + +For each enforced permission, with baseline _B_ ∈ {granted, +requires_signal, denied}: + +| Opt-out present | TCF present | TCF consents | Result | +| --------------- | ----------- | ------------ | -------------------------------------------------- | +| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | no | unset (withdrawal applies only where §4.2 says so) | +| no | yes | yes | set, unless B = denied | +| no | no | — | set iff B = granted | + +### 4.2 Withdrawal vs. absence + +Two distinct outcomes, never conflated: + +- **Withdrawal** (destructive: expire the EC cookie, write revocation + tombstones) requires an **affirmative** signal: an opt-out signal, or a + TCF record refusing `store-on-device` **in a jurisdiction whose baseline is + opt-in** (`requires_signal`). A visitor who has simply not yet made a + choice is never stripped of an existing identity. +- In a jurisdiction whose baseline is `granted`, a TCF refusal prevents + _new_ grants but does not tombstone: tombstones are irreversible + revocation markers, and PR #838 wrote them for visitors in unregulated + jurisdictions whose global CMP emitted a purpose-refusing string — + permanent identity loss under a regime the deployment never opted into. +- Withdrawal checking follows the same precedence as §4: an opt-out signal + triggers withdrawal even when a consenting TCF record is present. + +`ec_storage_withdrawn` (or its successor) gets direct unit coverage for every +row above; in PR #838 the headline "withdrawal expires identity" behavior had +no unit test at all. + +## 5. Jurisdiction resolution + +1. A selected geo provider resolves country and optional region; rules match + `country/region` first, then `country`, case-insensitively. +2. **Provider selected, lookup fails for a request** → the configured + `[geo] default_country` rules apply (per #779). +3. **No geo provider selected** → every request resolves to + `default_country`. This turns jurisdiction into a static constant, which + is only honest when the operator can genuinely assert single-jurisdiction + traffic. Constraint: **startup fails** when no geo provider is selected + _and_ the default country's baseline resolves any permission to `granted`, + unless the operator sets an explicit acknowledgment + (`[geo] assume_single_jurisdiction = true`). Without this, the natural + migration config (`default_country = "US"`, geo unset) silently grants + `store-on-device` and EID transmission to every EU visitor — the + highest-severity finding of the PR #838 review. The startup log always + prints the effective baseline and whether geo is live. +4. `default_country` is required; startup fails without it (per #779). The + shipped example uses the most protective baseline. + +## 6. Failure-mode matrix — normative + +| Condition | Resolution behavior | +| ---------------------------------------------------- | ------------------------------------------------ | +| Geo lookup fails at request time (provider selected) | `default_country` baseline | +| No geo provider configured | `default_country` baseline, gated by §5.3 | +| Country resolved, no matching rule | `default_country` baseline | +| Region resolved, no region rule | Country rule | +| Malformed policy file | Build failure (§3.2) — unreachable at runtime | +| No `default_country` | Startup failure | +| Undecodable TCF/GPP string | Treated as absent; opt-out signals still honored | +| Signals contradict (opt-out + consent) | Opt-out wins (§4) | + +The overall posture is **fail-closed**: every ambiguous state resolves to the +configured baseline or more restrictive, and the one configuration that could +convert "no information" into "granted" (§5.3) requires an explicit operator +assertion to exist. + +## 7. Enforcement points + +Exactly three consumers in this epic, all reading the same resolved set: + +1. **Provider execution** (providers spec §5) — all three provider kinds. +2. **EC lifecycle** — creation requires `store-on-device`; withdrawal per + §4.2. +3. **Bidstream EIDs** — transmission requires `store-on-device` ∧ + `select-personalised-ads`. + +## 8. Testing strategy + +- **The decision matrix is the test plan.** Every row of §4.1 and §4.2 × + each baseline, plus every row of §6, as table-driven tests. The ~24-case + matrix deleted by PR #838 (net −18 tests in the consent module, replaced + by happy-path cases only) is restored in equivalent form against the new + API; signal-precedence conflicts (opt-out + consenting TCF) are mandatory + cases, not optional ones. +- Policy validation tests for every §3.2 rejection. +- Shipped-table coverage test (§3.4) and split-brain consistency test + (§3.3). +- One end-to-end integration scenario per posture: opt-in jurisdiction with + and without consent, opt-out jurisdiction with GPC (including GPC + a + consenting TCF string), and the no-geo/default-country path. + +## 9. Out of scope + +- Additional purposes (extension procedure in §2). +- Runtime-loadable policy (the embedded file is deliberate: policy changes + are code reviews). If runtime policy is wanted later, it is its own spec + with its own validation story. diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md new file mode 100644 index 000000000..04663b58c --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -0,0 +1,239 @@ +# Design Spec: Pluggable Edge Cookie, Device, and Geo Providers + +**Status:** Draft +**Author:** Engineering +**Issue references:** #777, #778, #780, #781 +**Related specs:** `2026-07-30-permission-model-design.md`, +`2026-07-30-provider-migration-rollout-design.md`, +`2026-07-30-client-cycle-ec-resolve-design.md` +**Last updated:** 2026-07-30 + +> **Context.** PR #838 proposed a first implementation of this epic in a single +> change. Review of that PR surfaced design gaps this spec exists to close +> before a second implementation pass: an identity abstraction that owned +> minting but not recognition, per-adapter divergence in provider selection, +> silent misconfiguration modes, and speculative trait surface with no +> production caller. This spec is the authoritative statement of what the +> provider architecture must do; where it contradicts PR #838, this spec wins. + +--- + +## 1. Overview and goals + +Trusted Server makes three per-request data decisions that are currently +hard-wired: whether to create or keep an Edge Cookie (EC) identity, how to +classify the requesting device, and whether to resolve geolocation. Each +becomes a **provider**: a selectable component chosen in operator +configuration, with a deliberately neutral default. + +Goals: + +- A deployment picks an implementation per concern (including none) without a + code change to Trusted Server core. +- Defaults are neutral: with no configuration, no EC is created, device + classification uses only the User-Agent, and no geolocation is performed. A + default deployment makes no third-party or host-specific call. +- A provider **declares** the permissions its data use requires (see the + permission model spec); **core enforces** that declaration. A provider + cannot authorize itself. +- All adapters (Fastly, Axum, Cloudflare, Spin) behave identically for + identical configuration, or fail loudly at startup where a host cannot + satisfy the selected provider. + +Non-goals: + +- No vendor provider ships in this epic beyond the host-platform + implementations named below. +- The client-cycle (browser round-trip) provider type is **out of scope** + here; it has its own spec and must clear that spec's requirements first. + +## 2. Provider taxonomy + +| Concern | Trait | Built-in default | Opt-in host implementation | +| ----------- | -------------------- | --------------------------- | ----------------------------------------------------------------- | +| EC identity | `EdgeCookieProvider` | none (stateless) | `hmac` (in core; HMAC over client IP, preserves today's identity) | +| Device | `DeviceProvider` | `builtin` (User-Agent only) | `fastly` (JA4 / HTTP-2 fingerprints) | +| Geo | `GeoProvider` | none (no location) | `platform` (host geo lookup) | + +Selection keys are strings in operator configuration: + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "example-passphrase" + +[device] +provider = "builtin" + +[geo] +provider = "platform" +``` + +## 3. The identity lifecycle contract + +This is the section PR #838 lacked, and the source of its most structural +defect: the trait abstracted **minting** an identifier but left +**recognition** (`is_valid_ec_id`), **hashing** (`ec_hash`), and **KV key +normalization** hard-coded to the built-in HMAC shape. Any provider whose +identifiers do not match `{64hex}.{6alnum}` minted cookies that the very next +request discarded, and whose identities could never be tombstoned on +withdrawal. + +An `EdgeCookieProvider` owns the **complete lifecycle** of the identifiers it +mints. Every lifecycle operation core performs on an EC value MUST be routed +through the selected provider: + +| Lifecycle operation | Where core uses it today | Contract | +| -------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | +| **Recognize** | Reading `ts-ec` back from the request; deciding `ec_was_present` | Provider validates that a returned cookie value is one of its identifiers. A value the selected provider does not recognize is treated as absent. | +| **Hash / normalize** | KV identity-graph keys, log redaction | Provider (or a provider-supplied codec) maps an identifier to its stable KV key form. | +| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider recognizes — never a shape-gated subset. | + +**Invariant:** for every provider `P` and every identifier `id` minted by `P`, +`P.recognize(id)` is true, `P` produces a stable KV key for `id`, and a +withdrawal request carrying `id` tombstones it. A conformance test suite MUST +assert this round-trip for every shipped provider, and the suite MUST be +written so a future provider crate can run it against its own implementation. + +## 4. Trait surface: minimalism rule + +Every trait method MUST have at least one production (non-test) caller in the +same PR that introduces it. Speculative surface observed in PR #838 that MUST +NOT ship without a caller: + +- `keys_equal` (no production caller; existed to serve a unit test), +- `GeneratedEdgeCookie::response_headers` (empty in all built-ins, plumbed + through three layers), +- `IdentityInput.permissions` / `IdentityInput.consent` (ignored by all + built-ins), +- `DeviceProvider::required_permissions` / `GeoProvider::required_permissions` + **unless** the enforcement point of §5 lands in the same change. + +If a future feature needs one of these, it arrives with that feature. + +The minimal `EdgeCookieProvider` surface implied by §3 is: + +```rust +pub trait EdgeCookieProvider { + /// Stable configuration key ("hmac"). + fn id(&self) -> &'static str; + /// Permissions this provider's data use requires. Enforced by core. + fn required_permissions(&self) -> PermissionSet; + /// Mint an identifier from request evidence. + fn generate(&self, input: &IdentityInput<'_>) -> Result>; + /// Whether `value` is an identifier this provider minted. + fn recognize(&self, value: &str) -> bool; + /// Stable KV key form of a recognized identifier. + fn kv_key(&self, id: &EcId) -> KvKey; +} +``` + +(Names indicative; the shape is normative.) + +## 5. Permission enforcement is core's job — for all three provider kinds + +Before executing **any** provider (EC, device, or geo), core resolves the +request's permission set (see the permission model spec) and refuses to run a +provider whose `required_permissions()` are not all set. PR #838 declared this +method on all three traits but consulted it only for the EC provider; the +device and geo declarations were decorative. That is worse than absent — it +reads as a gate and is not one. The enforcement point MUST be a single shared +code path used by all three provider kinds, with a test per kind proving a +provider declaring an unset permission does not execute. + +## 6. Selection, validation, and failure modes + +All validation happens at **settings construction** — a misconfiguration is a +startup error, never a request-time error and never a silent behavior change. + +| Configuration state | Behavior | +| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `provider` names an unknown key | Startup error listing valid keys. | +| `provider` set, its `[ec.providers.]` block missing | Startup error. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | +| No `provider`, no providers block | Valid: the neutral default for that concern. | + +Unknown fields inside every provider config block are rejected +(`deny_unknown_fields` on all new settings structs — PR #838 applied it to +`Ec` but not to `EcProviders`, `DeviceConfig`, or `GeoConfig`, so a typo like +`providr` was silently ignored). + +## 7. Composition root and adapter parity + +Provider construction happens in exactly one place per concern +(`build_ec_provider`, `build_device_provider`, `build_geo_provider`), called +by **every** adapter. No adapter may wire a concrete implementation directly: +in PR #838 the Cloudflare adapter installed its host geo unconditionally, +so identical configuration produced different jurisdictions on different +adapters — which the permission model then turned into different privacy +outcomes. + +Requirements: + +- Each adapter's runtime-services setup routes through the shared builders. +- Providers are constructed **once** per application instance and stored in + app state; PR #838 rebuilt the provider (cloning the secret into a fresh + `Box`) up to three times per request. +- The cross-adapter parity suite gains cases asserting: (a) the selected + provider is honored on every adapter, (b) the neutral default performs no + host call on every adapter, and (c) a capability-unsatisfiable selection + fails startup on the adapters that cannot satisfy it. + +## 8. Crate layout and CI + +Provider crates live flat under `crates/` following the existing naming +convention: `crates/trusted-server-geo-fastly`, +`crates/trusted-server-device-fastly`. (PR #838 introduced a nested +`crates/geo/fastly` layout that broke the directory–package correspondence +every other member follows.) No placeholder directories: a `crates/…/README.md` +with no crate ships when the first crate does. + +Every new crate is added to the `.cargo/config.toml` aliases +(`check-fastly`, `clippy-fastly`, `test-fastly`, `build-fastly`) in the same +PR that adds the crate, and to the CI gate list in `CLAUDE.md`. PR #838's new +crates compiled only transitively and were never linted with `-D warnings` +nor had a single test. + +## 9. Behavior preservation notes + +Two defaults chosen for neutrality change effective behavior on existing +Fastly deployments; both are called out in the migration spec and must be +prominent in release notes: + +- **Bot gate.** The pre-provider EC bot gate required JA4 _and_ platform + class. With `device.provider = "builtin"` the gate degrades to User-Agent + heuristics. Restoring the stronger gate requires `[device] provider = +"fastly"`; the migration guide lists this as a behavior-preserving step for + Fastly deployments. +- **Geo.** With no geo provider, jurisdiction resolution falls to the + configured default country. The permission model spec (§5) constrains this + combination so it cannot silently grant permissions to mis-attributed + traffic. + +## 10. Testing strategy + +- Provider conformance suite (§3 invariant) run against every shipped + provider. +- Enforcement tests per provider kind (§5). +- Settings validation tests for every row of the §6 table, including the + block-without-selector rejection. +- Parity suite additions of §7. +- Unit tests inside each provider crate; crates with no native-target tests + still get clippy coverage via the alias wiring of §8. + +## 11. Implementation order + +1. Traits + lifecycle contract + conformance suite, `hmac` provider + passing it (behavior-identical to today; see migration spec §3 for the + ID-stability vectors). +2. Settings selection + validation table. +3. Composition root + all four adapters wired through it, parity cases. +4. Device and geo providers with the shared enforcement point of §5. + +Each step is independently reviewable; none depends on the permission model +landing first (the EC gate keeps its current jurisdiction logic until the +permission model PR replaces it). diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md new file mode 100644 index 000000000..c00f8c348 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -0,0 +1,161 @@ +# Design Spec: Provider and Permission Model — Migration and Rollout + +**Status:** Draft +**Author:** Engineering +**Issue references:** #777–#781 (epic) +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-permission-model-design.md` +**Last updated:** 2026-07-30 + +> **Context.** The provider/permission epic is a breaking change to a live +> identity system. PR #838's review showed that the riskiest part of such a +> change is not the new code but the transition: silent misconfiguration +> modes, undeclared behavior changes discovered by deleted tests, and no +> written statement of which pre-change behaviors were guaranteed to +> survive. This spec is that statement. Any implementation PR in the epic +> must reconcile its diff against §2's matrix and list every deliberate +> divergence in its description. + +--- + +## 1. Scope + +Covers the transition of existing deployments from the hard-wired EC / +device / geo behavior to the provider architecture and permission model. +Applies to every implementation PR in the epic, and to the operator-facing +migration guide that ships with the last of them. + +## 2. Behavior-preservation matrix + +For each decision the system makes today, the target behavior after the epic, +and whether that is a preservation or a declared change. **Silent changes are +defects.** PR #838 changed six of these without declaring any; each was +discoverable only because a deleted test had pinned the old behavior. + +| # | Decision (today) | After epic | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Policy decision: the shipped `US` baseline decides. If the baseline grants `store-on-device` without a signal, that is a **declared change** requiring sign-off in the policy file review, with rationale in the file itself | Declared change (if made) | +| 4 | UK request, no TCF record → no EC | Same, unless the policy file deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | Unknown jurisdiction (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC allowed, never tombstoned | Same (withdrawal only where baseline is opt-in; permission spec §4.2) | Preserved | +| 7 | EID transmission requires storage + personalization consent where regulated | Same via `store-on-device` ∧ `select-personalised-ads` | Preserved | +| 8 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 9 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"` | Declared change with a documented restore step (§5) | + +Rows 3 and 4 are policy decisions, not code decisions: they belong in the +`permissions.yaml` review, made explicitly by maintainers — not implied by an +implementation. + +## 3. Identity stability guarantee + +For a deployment that selects `provider = "hmac"` and carries its passphrase +over verbatim: + +- The minted identifier is **bit-identical** to today's: + `HMAC-SHA256(passphrase, normalized_ip)` in the existing encoding. +- Cookie name, attributes, and max-age are unchanged; existing `ts-ec` + cookies are recognized by the provider. +- KV identity-graph keys (`ec_hash`, normalization) are unchanged; no + existing graph row is orphaned. + +Enforced by **pinned known-answer tests**: fixed passphrase + IP → exact +expected identifier, cookie string, and KV key, committed as vectors so any +divergence fails CI rather than rotating a production identity base. + +## 4. Configuration migration + +Old shape: + +```toml +[ec] +passphrase = "example-passphrase" +``` + +New shape: + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "example-passphrase" +``` + +Requirements: + +1. **Old key fails loud.** `[ec] passphrase` is rejected at startup with a + message naming the new location — not a generic unknown-field error. +2. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no + `provider = "hmac"` selector is a startup error (providers spec §6). In + PR #838 this configuration — the exact state an operator following the + docs reaches if they miss one line — validated green and silently minted + zero ECs. +3. **The example config ships the migrated happy path**, uncommented: + `provider = "hmac"` with its block, `[geo] default_country`, and (for + Fastly) the behavior-preserving `[device] provider = "fastly"` and + `[geo] provider = "platform"` lines present with a comment stating what + removing them changes. PR #838's example shipped the passphrase block + uncommented with the selector commented out — steering operators directly + into the silent-stateless state. +4. Every misconfiguration in the providers spec §6 table fails at + **startup**. Request-time failure for a configuration error is a defect. +5. Config-store payload validation (`ts config push`) applies the same + rules, so a bad config is rejected at push time, before any instance + restarts into it. + +## 5. Behavior-preserving migration recipe (operator-facing) + +The migration guide (a new `docs/guide/` page, linked from the release notes) +gives one copy-pasteable recipe per adapter for "keep exactly today's +behavior": + +```toml +[ec] +provider = "hmac" +[ec.providers.hmac] +passphrase = "" + +[device] +provider = "fastly" # Fastly deployments: preserves the JA4 bot gate + +[geo] +provider = "platform" # preserves per-request jurisdiction detection +default_country = "FR" # used only when the host lookup fails +``` + +and separately documents the neutral configuration and what it does _not_ do. +The guide states explicitly that `default_country` alone does not replace geo +lookup, and why the permissive-default + no-geo combination requires the +explicit acknowledgment flag (permission spec §5.3). + +## 6. Rollout sequence and observability + +1. Implementation PRs land in the epic's order (providers first, permission + model second); each is reviewable against §2 in isolation. +2. Before/after deploy, operators watch **EC issuance rate** and EID + attachment rate; the migration guide names these as the canary metrics, + because the failure mode of a bad migration is a silent drop to zero (or a + silent grant to everyone), not an error rate. +3. Startup logs always print: selected provider per concern, whether geo is + live, the effective default baseline, and the count of granted-without- + signal permissions. One line, greppable, stable format. +4. Rollback is config-only where possible: reverting to the previous + config version restores the previous behavior on the previous binary. The + one irreversible artifact is withdrawal tombstones — which is why §2 row 6 + (no tombstones without affirmative withdrawal in an opt-in jurisdiction) + is non-negotiable. + +## 7. Documentation deliverables + +- Migration guide page (§5), linked from `CHANGELOG.md` and the release + notes. +- `configuration.md` documents **every** valid `provider` value for all + three concerns, and documents environment-variable overrides only if they + actually work in production builds (in PR #838 the documented + `TRUSTED_SERVER__EC__PROVIDER` override existed only under `#[cfg(test)]`). +- The permission model page states the §4 precedence rules of the permission + spec verbatim — operator docs and normative spec must not diverge on + precedence, and prose like "signals are mapped as a grant or a revoke" + without stating which wins is insufficient. From a35f2ca78759a77f89a43fd4b61806b57c2e2209 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 01:00:07 -0700 Subject: [PATCH 02/14] Address self-review findings and move policy into trusted-server.toml Self-review of the five specs (own pass plus an adversarial fresh-eyes pass) surfaced fixes applied here: - Policy location reversed per maintainer decision: the permission policy is a [permissions] section of trusted-server.toml flowing through the config-store pipeline, not a build-time-embedded YAML. Overrides name acquisition rules directly instead of +/- sigils, and a rules.default entry separates resolved-but-unlisted countries from geo default_country. - Removed a circular requirement: geo and device providers are inputs to permission resolution and cannot be gated on its output; the enforcement gate is EC-only and the decorative required_permissions declarations are dropped from those traits. - Fixed the identity-stability guarantee: the EC id has a random per-mint suffix, so known-answer vectors pin the deterministic 64-hex prefix, recognition of existing cookies, and hash-prefix semantics instead of full identifiers. - Split the KV key contract into the verbatim graph-row key and the deliberately-colliding hash prefix that IP-cluster trust depends on. - Declared previously silent behavior changes in the migration matrix: global opt-out honoring (including tombstones) and the fate of non-regulated countries, with a preserving recipe for the latter. - Sequenced the geo neutral-default flip into the permission model PR so no intermediate step zeroes EC issuance under the current fail-closed gate. - Resolved smaller contradictions: withdrawal triggers made exhaustive (including denied-baseline and policy-edit cases), jurisdiction consistency requirement now covers both legacy lists with explicit exceptions, ISO rule-key validation made decidable, response-header reserved surface defined at cookie-name granularity for Set-Cookie, and the host-signals provider's removal made explicit with config rejection. --- ...26-07-30-client-cycle-ec-resolve-design.md | 22 +- ...integration-response-header-hook-design.md | 40 +- .../2026-07-30-permission-model-design.md | 384 ++++++++++++------ .../2026-07-30-pluggable-providers-design.md | 111 +++-- ...07-30-provider-migration-rollout-design.md | 121 ++++-- 5 files changed, 451 insertions(+), 227 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 674de01ff..1b9eacc12 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -6,7 +6,7 @@ until its open questions (§7) are resolved in a dedicated issue** **Issue references:** none yet (this spec exists to force one; #778 does not cover this feature) **Related specs:** `2026-07-30-pluggable-providers-design.md` -**Last updated:** 2026-07-30 +**Last updated:** 2026-07-31 > **Context.** PR #838 shipped, undeclared and unspec'd, a second provider > _type_: a "client-cycle" EC provider whose identifier is established by a @@ -52,9 +52,10 @@ Everything in this spec follows from that. 1. **Reject cross-site requests.** Require a same-site assertion: `Origin` (or `Sec-Fetch-Site: same-origin/same-site`) validated against the - publisher's origin set; requests without a validating header are - rejected. CSRF-token designs are acceptable but not required if - origin-based rejection is enforced. + publisher's origin allowlist — configuration that does not exist yet and + must be defined by this feature (§7, question 5); requests without a + validating header are rejected. CSRF-token designs are acceptable but not + required if origin-based rejection is enforced. 2. **Verify the payload cryptographically per provider.** The provider's `resolve_from_client` accepts only payloads that are signed by an expected party, **audience-bound** to this publisher, and **expiring** @@ -83,7 +84,11 @@ Everything in this spec follows from that. endpoint is cheap-idempotent and rate-limited per session; the design must state which and test it. - The JS module ships through the standard integration bundle mechanism, - loaded only when a client-cycle provider is the selected EC provider. + loaded only when a client-cycle provider is the selected EC provider. Note + the consequence: bundle content becomes a function of EC configuration, + which interacts with the bundle's content-hash/SRI pinning and caching — + the mechanism today keys off the integration registry, not EC provider + selection (open question, §7). - Any constant shared between Rust and TS (endpoint path, marker name) is asserted equal by a test, not "kept in sync by hand". @@ -118,3 +123,10 @@ sentence as the only guardrail. 3. Rate limiting / abuse posture at the edge for an unauthenticated POST. 4. Whether the endpoint should be versioned separately from the identify API family it sits beside. +5. The shape of the publisher **origin allowlist** that §3.1 validates + against — no such configuration exists today (`publisher.origin_url` is + a single upstream and the cookie domain is a cookie scope, not an origin + set), so it is new settings surface this feature must define. +6. How JS module selection keyed off EC provider configuration coexists + with content-hashed/SRI-pinned bundles (§4) — per-config hashes, cache + keying, and the config-push story for them. diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 59d354547..01aea582d 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -4,7 +4,7 @@ **Author:** Engineering **Issue references:** #782 **Related specs:** `2026-07-30-pluggable-providers-design.md` -**Last updated:** 2026-07-30 +**Last updated:** 2026-07-31 > **Context.** Issue #782 already specifies this feature well; its done-when > is the contract. PR #838 shipped the trait and registry wiring with **no @@ -34,19 +34,28 @@ mutators to the outbound response for HTML document responses it processed. code where one exists; where adapters finalize independently, each adapter gains the call and a test proving it. - Mutators run **after** Trusted Server's own response-header handling - (EC Set-Cookie emission, EC header clearing, privacy headers) so a mutator - cannot be silently clobbered by later core steps — in PR #838's ordering, - provider-supplied headers were inserted before the EC header-clearing pass - and could be stripped by it. + (EC Set-Cookie emission, EC header clearing, privacy headers) so a + mutation cannot be silently stripped by a later core pass. The ordering is + a fresh decision this spec makes — PR #838 never wired the hook, so there + is no existing insertion point to inherit; the implementer places the call + at the end of each adapter's response finalization, and the §4.3 tests pin + it there. ## 3. Collision policy -- Mutators may not touch **reserved headers**: `Set-Cookie` for the EC - cookie, the `x-ts-*` namespace, and the consent/privacy headers core - emits. Attempts are dropped and logged at `warn` with the integration id. +- Mutators may not touch **reserved surface**, which is defined at two + granularities because `Set-Cookie` is multi-valued: (a) reserved header + _names_ — the `x-ts-*` namespace and the consent/privacy headers core + emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, + `ts-eids`, and the other `ts-*` cookies core owns. An integration may + append its own `Set-Cookie` values; it may not set or expire a reserved + cookie name. Violations are dropped and logged at `warn` with the + integration id. The reserved-cookie list is a single constant next to the + cookie definitions, not duplicated in the hook. - For non-reserved headers, the mutator API distinguishes **append** from - **replace** explicitly; the default is append. Replacing a header the - origin set is a deliberate act, visible in the mutator's code. + **replace** explicitly; the default is append (for `Set-Cookie`, append is + the only non-reserved operation — replace is not offered). Replacing a + header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). @@ -62,8 +71,11 @@ mutators to the outbound response for HTML document responses it processed. 4. A parity-suite case asserts identical mutation behavior across adapters. 5. Reserved-header and append/replace semantics covered by unit tests. -## 5. Size +## 5. Size and sequencing -This is a ~150-line feature plus tests. It has zero coupling to the provider -architecture or the permission model and should land as its own small PR, -first in the epic's sequence. +This is a ~150-line feature plus tests, with zero coupling to the provider +architecture or the permission model. It lands as its own small PR **when +its first real consumer is identified** (§4.2) — at any point in the epic's +sequence, blocking nothing and blocked by nothing. If no consumer +materializes, it does not land; being unblocked is not a reason to ship +scaffolding. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index f3504a04d..bd5da97c9 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -5,17 +5,19 @@ **Issue references:** #779 **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md` -**Last updated:** 2026-07-30 +**Last updated:** 2026-07-31 > **Context.** PR #838 proposed a permission model whose review surfaced two > classes of defect this spec exists to prevent in the next pass: (1) silent > behavioral inversions of consent-signal precedence — most seriously, a > present TCF string short-circuiting GPC/GPP/US-Privacy opt-outs — and > (2) fail-open jurisdiction resolution when geolocation is disabled. The -> precedence table (§4) and the failure-mode matrix (§6) are the two +> precedence rules (§4) and the failure-mode matrix (§6) are the two > documents whose absence allowed those defects to hide in a 67-file diff. > They are normative: an implementation whose behavior differs from these -> tables is wrong, whatever its tests say. +> tables is wrong, whatever its tests say. This spec also reverses one +> PR #838 structural decision: policy lives in `trusted-server.toml`, not in +> a build-time-embedded YAML file (§3.1). --- @@ -24,14 +26,14 @@ The permission model replaces the hard-wired jurisdiction gate (`allows_ec_creation` and its companions) with a single resolved **permission set** per request. Every data decision Trusted Server itself -makes — EC creation, EC withdrawal, EID transmission into the bidstream, -and provider execution (see providers spec §5) — reads that set. +makes — EC provider execution, EC creation and withdrawal, and EID +transmission into the bidstream — reads that set (§7). The set is resolved from three inputs: 1. **Jurisdiction** — the country/region the request resolves to (§5). -2. **Policy** — a declarative, version-controlled map from jurisdiction to a - baseline acquisition rule per permission (§3). +2. **Policy** — a declarative map from jurisdiction to a baseline + acquisition rule per permission (§3). 3. **Signals** — the request's privacy signals: TCF, GPP, GPC, US Privacy (§4). @@ -62,72 +64,138 @@ The initial vocabulary is therefore exactly: (The identifier strings are the IAB names verbatim, including their original spelling.) The extension procedure — add the signal mapping, add the enforcement point, add the policy vocabulary entry, in one change — is -documented in the policy file header. Policy validation **rejects** a rule -that references an identifier outside the current vocabulary, so the file can +documented alongside the policy schema. Policy validation **rejects** a rule +that references an identifier outside the current vocabulary, so a policy can never promise more than the code enforces. ## 3. Policy -### 3.1 Format +### 3.1 Location: `[permissions]` in `trusted-server.toml` -A YAML map, embedded at build time, of named **groups** (baselines) and -**rules** keying countries (`FR`) and country/state pairs (`US/CA`) to a -group with optional per-permission overrides. Each permission resolves to an -**acquisition rule**: +Policy is operator-owned runtime configuration, expressed as a +`[permissions]` section of `trusted-server.toml`, flowing through the same +pipeline as every other setting (`ts config push` publishes it as part of the +config blob envelope; instances pick it up like any config change). -- `granted` — set without any signal, -- `requires_signal` — set only when a signal grants it (opt-in), -- `denied` — never set, even when a signal grants it. - -Overrides support all three targets: `+perm` (granted), `-perm` (denied), and -`~perm` (requires_signal). PR #838 supported only `+`/`-`, making the most -common real-world override — "this state requires a signal for personalized -ads" — inexpressible without duplicating a whole group. Groups may use a -`default:` shorthand for unlisted permissions. +This deliberately reverses PR #838, which embedded a `permissions.yaml` at +build time via `include_str!`. That design was rejected because: -### 3.2 Validation — at build time, not request time +- a policy edit — the operation the whole model exists to make easy — + required recompiling and redeploying the binary, cutting against the + runtime config-store pipeline the project has standardized on; +- it introduced a second configuration language and a second validation + path next to the TOML settings machinery that already exists; +- the `include_str!` reached two directory levels above the crate root, + breaking crate packaging; +- validation ran lazily at first use behind a `OnceLock` + `expect`, so a + bad edit that escaped unit tests became a 500 on every request. -The embedded file is validated by a `build.rs` step (or an equivalent -always-run CI test that asserts the parse explicitly): a malformed committed -file fails the **build**, never a request. PR #838's file was parsed lazily -behind a `OnceLock` with an `expect`, meaning a bad edit that escaped unit -tests became a 500 on every request. +Auditability is preserved where it actually lives: the source-controlled +`trusted-server.example.toml` ships the complete recommended policy table +(the reviewable reference artifact), and the operator's own config history — +git for the file, config-store versions for pushes — is the change log. -Validation rejects: +**Compiled-in fallback:** when a config has no `[permissions]` section, a +minimal compiled-in policy applies in which **every permission is +`requires_signal` for every jurisdiction** — the most protective posture. +Absence of policy is always safe; there is no fail-open default. -- unknown fields anywhere (`deny_unknown_fields` on every deserialized - struct — PR #838's untagged rule enum silently swallowed a misspelled - `permission:` key, dropping the operator's override with no diagnostic); -- rule keys that are not plausible ISO 3166-1 alpha-2 / ISO 3166-2 codes; -- references to permissions outside the enforced vocabulary (§2); -- groups that neither list every permission nor provide `default:`. +### 3.2 Format -### 3.3 One source of jurisdiction truth +Named **groups** (baselines) and **rules** mapping a country (`FR`) or +country/state pair (`"US/CA"`) to a group, with optional per-permission +overrides. Each permission resolves to an **acquisition rule**: -The codebase currently carries a second, runtime-configurable jurisdiction -list (`consent.gdpr.applies_in`) used by the auction consent gate. Two -independently maintained country tables that both express "where GDPR -applies" will drift (in PR #838, adding `CH` to one had no effect on the -other). Requirement: either the auction gate derives its jurisdiction class -from the same resolved policy, or a CI test asserts that every country in -`applies_in` resolves to an opt-in (`requires_signal`) baseline in the policy -file, and vice versa for the shipped defaults. +- `granted` — set without any signal, +- `requires_signal` — set only when a signal grants it (opt-in), +- `denied` — never set, even when a signal grants it. -### 3.4 Shipped table coverage +```toml +[permissions.groups.gdpr-eu] +default = "requires_signal" + +[permissions.groups.us-opt-out] +default = "granted" + +[permissions.rules] +FR = "gdpr-eu" +US = "us-opt-out" +# Overrides name explicit acquisition rules — no +/- sigil syntax; TOML +# expresses the target state directly. +"US/CA" = { group = "us-opt-out", overrides = { select-personalised-ads = "requires_signal" } } +# Reserved key: countries that resolve but match no rule. Distinct from +# [geo] default_country, which handles requests that resolve no country at +# all (§5.4). +default = "gdpr-eu" +``` + +A group's `default` covers unlisted permissions; a group may also name +permissions explicitly. Overrides map identifier → acquisition rule, so any +target state (including `requires_signal`) is expressible — PR #838's +`+`/`-` sigil scheme could not express "requires a signal", the most common +real-world override. + +### 3.3 Validation — at config acceptance, not request time + +Policy is validated where every other setting is: at `ts config push` (a bad +policy is rejected before publication) and at settings construction on +startup (a bad stored config produces the startup-error state, never a +per-request failure). -A CI test asserts every member of the GDPR country list resolves to an opt-in -baseline (this is what catches a `DK:` typoed as `DL:` — a defect that in -PR #838 survived parse, startup, and all tests, silently dropping Denmark to -the operator default). Countries intentionally not listed are governed by -§5's default-country rules; the policy header documents that this is the -fallback, and the shipped example default is the most protective baseline. +Validation rejects: -## 4. Signal precedence — normative table +- unknown fields anywhere (`deny_unknown_fields` on every deserialized + struct — PR #838's untagged rule type silently swallowed a misspelled + override key, dropping the operator's override with no diagnostic); +- rule keys whose country part is not in the embedded **assigned** ISO + 3166-1 alpha-2 list (not merely `[A-Z]{2}` — an unassigned code is + almost certainly a typo silently diverting a country to the fallback); + the region part matches `[A-Z0-9]{1,3}`. The `US/CA` slash form is the + house rule-key format corresponding to ISO 3166-2 `US-CA`; +- references to permissions outside the enforced vocabulary (§2); +- references to undefined groups; +- groups that neither list every permission nor provide `default`. + +### 3.4 One source of jurisdiction truth + +Today, `detect_jurisdiction` — driven by the runtime lists +`consent.gdpr.applies_in` and `consent.us_privacy.states` — is the sole +jurisdiction source for **both** the auction consent gate and the EC gate. +The permission model replaces the EC side; if the auction gate keeps reading +the old lists while EC reads policy rules, the two will drift (adding a +country to one has no effect on the other, and an operator has no signal +that they disagree). + +Requirement: the auction gate's jurisdiction class derives from the same +resolved policy (a country is GDPR-class when its rule resolves to an +opt-in baseline for `select-personalised-ads`). Where the legacy lists must +survive an interim period, a CI test asserts consistency between each list +and the policy table, with deliberate divergences recorded as explicit, +commented exceptions in the test — never silent. Both legacy lists are in +scope, not only the GDPR one. + +### 3.5 Shipped-table coverage + +A CI test asserts every member of the GDPR country list resolves to a +GDPR-class baseline in the example policy. This closes a defect class +nothing in PR #838's validation covered: a mistyped country key (`DL:` for +`DK:`) parses cleanly, starts cleanly, and silently drops a member state to +the fallback rule. Countries intentionally unlisted are governed by the +`rules.default` entry (§3.2); the example policy documents that fallback +inline, and ships it as the most protective baseline. + +## 4. Signal precedence — normative Signals are classified: - **Opt-out signals** (affirmative withdrawal): GPC header; GPP sections - carrying a sale/sharing opt-out; US Privacy opt-out. + carrying a sale/sharing opt-out; US Privacy opt-out. Opt-out signals are + honored **globally**, not only in the jurisdictions whose law defines + them — a deliberate, more-protective simplification: scoping a browser's + explicit opt-out to a geolocation guess would honor it for some visitors + and ignore it for others based on IP evidence. (For jurisdictions outside + US states this is a declared behavior change; migration spec §2 records + it.) - **Consent records**: a decodable TCF string (standalone or embedded in GPP), which may grant or refuse individual purposes. @@ -144,7 +212,13 @@ Signals are classified: `ec_blocked_us_state_gpc_overrides_tcf` and companions — are reinstated against the new API, not deleted.)_ 3. Consent record refusal — a TCF record present and refusing the purpose - revokes it. + revokes it. This applies in **every** jurisdiction, including + `granted`-baseline ones: an expressed refusal always beats a policy + default. Note this is a declared, more-protective divergence from the + pre-epic gate, which ignored consent records entirely outside regulated + jurisdictions — the migration spec's matrix (row 6) records it. Refusal + revokes new grants only; whether it also destroys existing identity is + governed strictly by §4.2. 4. Consent record grant — a TCF record present and consenting grants it (subject to 1–2). 5. No signal — the policy baseline decides: `granted` sets it, @@ -155,100 +229,156 @@ Signals are classified: For each enforced permission, with baseline _B_ ∈ {granted, requires_signal, denied}: -| Opt-out present | TCF present | TCF consents | Result | -| --------------- | ----------- | ------------ | -------------------------------------------------- | -| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | -| no | yes | no | unset (withdrawal applies only where §4.2 says so) | -| no | yes | yes | set, unless B = denied | -| no | no | — | set iff B = granted | +| Opt-out present | TCF present | TCF consents | Result | +| --------------- | ----------- | ------------ | ------------------------------------------------ | +| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | no | unset (withdrawal per §4.2, trigger 2) | +| no | yes | yes | set, unless B = denied | +| no | no | — | set iff B = granted | ### 4.2 Withdrawal vs. absence -Two distinct outcomes, never conflated: - -- **Withdrawal** (destructive: expire the EC cookie, write revocation - tombstones) requires an **affirmative** signal: an opt-out signal, or a - TCF record refusing `store-on-device` **in a jurisdiction whose baseline is - opt-in** (`requires_signal`). A visitor who has simply not yet made a - choice is never stripped of an existing identity. -- In a jurisdiction whose baseline is `granted`, a TCF refusal prevents - _new_ grants but does not tombstone: tombstones are irreversible - revocation markers, and PR #838 wrote them for visitors in unregulated - jurisdictions whose global CMP emitted a purpose-refusing string — - permanent identity loss under a regime the deployment never opted into. -- Withdrawal checking follows the same precedence as §4: an opt-out signal - triggers withdrawal even when a consenting TCF record is present. - -`ec_storage_withdrawn` (or its successor) gets direct unit coverage for every -row above; in PR #838 the headline "withdrawal expires identity" behavior had -no unit test at all. +Withdrawal (destructive: expire the EC cookie, write revocation tombstones) +and non-grant (the permission is simply unset) are distinct outcomes, never +conflated. "Baseline" below always means the **resolved acquisition rule for +`store-on-device` in the request's jurisdiction, after overrides** — never a +group label, since a group can mix rules across permissions. + +The triggers, exhaustively — nothing else withdraws: + +1. **An opt-out signal withdraws in every jurisdiction, whatever the + baseline.** (For US states this preserves today's behavior; elsewhere it + is the declared change of §4's global-opt-out rule.) +2. **A TCF record refusing `store-on-device` withdraws iff the baseline is + `requires_signal`.** Where the baseline is `granted`, refusal blocks + _new_ grants but never tombstones: tombstones are irreversible, and + PR #838 wrote them for visitors in unregulated jurisdictions whose + global CMP emitted a purpose-refusing string — permanent identity loss + under a regime the deployment never opted into. +3. **A policy edit is not a user signal.** Tightening a baseline to + `denied` stops new identity but does not itself tombstone identities + minted before the change; cleaning those up is an operational action + (migration spec §6). An affirmative user signal (trigger 1 or 2) still + withdraws them. +4. **Absence of signal never destroys identity.** A visitor who has not yet + made a choice is never stripped of an existing identity. + +Withdrawal checking follows §4 precedence: an opt-out signal triggers +withdrawal even when a consenting TCF record is present. +`ec_storage_withdrawn` (or its successor) gets direct unit coverage for +every trigger above; in PR #838 the headline "withdrawal expires identity" +behavior had no unit test at all. ## 5. Jurisdiction resolution -1. A selected geo provider resolves country and optional region; rules match - `country/region` first, then `country`, case-insensitively. -2. **Provider selected, lookup fails for a request** → the configured - `[geo] default_country` rules apply (per #779). -3. **No geo provider selected** → every request resolves to - `default_country`. This turns jurisdiction into a static constant, which - is only honest when the operator can genuinely assert single-jurisdiction - traffic. Constraint: **startup fails** when no geo provider is selected - _and_ the default country's baseline resolves any permission to `granted`, - unless the operator sets an explicit acknowledgment - (`[geo] assume_single_jurisdiction = true`). Without this, the natural - migration config (`default_country = "US"`, geo unset) silently grants - `store-on-device` and EID transmission to every EU visitor — the - highest-severity finding of the PR #838 review. The startup log always - prints the effective baseline and whether geo is live. -4. `default_country` is required; startup fails without it (per #779). The - shipped example uses the most protective baseline. +### 5.1 Order + +Geo resolution runs **before** permission resolution — jurisdiction is an +input to the permission set, which is why geo providers cannot themselves be +gated on it (providers spec §5). A selected geo provider resolves country +and optional region; rules match `country/region` first, then `country`, +case-insensitively. + +### 5.2 Lookup failure + +Provider selected, lookup resolves nothing for a request → the configured +`[geo] default_country` rules apply (per #779). An adapter whose geo +implementation can never resolve anything must not accept the selection at +all — that is the capability check of providers spec §6, and it prevents a +"selected but always empty" provider from silently converting every request +to §5.3 semantics without §5.3's guard. + +### 5.3 No geo provider selected + +Every request resolves to `default_country` — jurisdiction becomes a static +constant, which is only honest when the operator can genuinely assert +single-jurisdiction traffic. It is not only `granted` baselines that make +this dangerous: with a `requires_signal` baseline, a page-global CMP that +emits a consenting TCF string grants permissions for every mis-attributed +visitor just as effectively. + +Constraint: **startup fails** when an EC provider is selected and no geo +provider is, unless the operator sets an explicit acknowledgment +(`[geo] assume_single_jurisdiction = true`). Stateless deployments (no EC +provider) are exempt. Without this guard, the natural migration config +(`default_country = "US"`, geo unset) silently grants `store-on-device` and +EID transmission to every EU visitor — the highest-severity finding of the +PR #838 review. The startup log always prints the effective baseline and +whether geo is live. + +### 5.4 Defaults, two distinct fallbacks + +`[geo] default_country` is required; startup fails without it (per #779). +It covers requests that resolve **no country at all**. Countries that +resolve but match no rule fall to the policy's `rules.default` entry +(§3.2). The two fallbacks are deliberately separate: "we could not place +this request" and "we placed it somewhere we have no rule for" are +different states, and pre-epic behavior treated them differently (fail +closed vs. non-regulated) — collapsing them is what made PR #838's +migration story unresolvable (migration spec §2, rows 5 and 7). ## 6. Failure-mode matrix — normative -| Condition | Resolution behavior | -| ---------------------------------------------------- | ------------------------------------------------ | -| Geo lookup fails at request time (provider selected) | `default_country` baseline | -| No geo provider configured | `default_country` baseline, gated by §5.3 | -| Country resolved, no matching rule | `default_country` baseline | -| Region resolved, no region rule | Country rule | -| Malformed policy file | Build failure (§3.2) — unreachable at runtime | -| No `default_country` | Startup failure | -| Undecodable TCF/GPP string | Treated as absent; opt-out signals still honored | -| Signals contradict (opt-out + consent) | Opt-out wins (§4) | - -The overall posture is **fail-closed**: every ambiguous state resolves to the -configured baseline or more restrictive, and the one configuration that could -convert "no information" into "granted" (§5.3) requires an explicit operator -assertion to exist. +| Condition | Resolution behavior | +| ---------------------------------------------------- | ------------------------------------------------------------ | +| Geo lookup fails at request time (provider selected) | `default_country` baseline | +| No geo provider configured | `default_country` baseline, guarded by §5.3 | +| Country resolved, no matching rule | Policy `rules.default` | +| Region resolved, no region rule | Country rule | +| No `[permissions]` section | Compiled-in fallback: everything `requires_signal` | +| Malformed policy | Rejected at config push / startup (§3.3) — never per request | +| No `default_country` | Startup failure | +| Undecodable TCF/GPP string | Treated as absent; opt-out signals still honored | +| Signals contradict (opt-out + consent) | Opt-out wins (§4) | + +The overall posture is **fail-closed**: every ambiguous state resolves to +the configured baseline or more restrictive, and the one configuration that +turns "no information" into a static jurisdiction assertion (§5.3) requires +an explicit operator acknowledgment to exist. ## 7. Enforcement points -Exactly three consumers in this epic, all reading the same resolved set: - -1. **Provider execution** (providers spec §5) — all three provider kinds. +Consumers of the resolved set in this epic: + +1. **EC provider execution** (providers spec §5) — the provider's declared + `required_permissions()` must all be set. This gate applies to EC + providers only: geo and device providers execute **before** permission + resolution as its inputs, so gating them on its output would be + circular. Their governance is explicit selection, the capability checks + of providers spec §6, and §2's vocabulary rule — if a future vocabulary + adds a purpose covering geolocation or fingerprinting, gating those + providers will require a two-phase resolution that must be specified + then, not improvised. 2. **EC lifecycle** — creation requires `store-on-device`; withdrawal per §4.2. 3. **Bidstream EIDs** — transmission requires `store-on-device` ∧ `select-personalised-ads`. +The client-cycle resolve endpoint (own spec, currently on hold) would be a +fourth consumer if and when it proceeds. + ## 8. Testing strategy -- **The decision matrix is the test plan.** Every row of §4.1 and §4.2 × - each baseline, plus every row of §6, as table-driven tests. The ~24-case - matrix deleted by PR #838 (net −18 tests in the consent module, replaced - by happy-path cases only) is restored in equivalent form against the new - API; signal-precedence conflicts (opt-out + consenting TCF) are mandatory - cases, not optional ones. -- Policy validation tests for every §3.2 rejection. -- Shipped-table coverage test (§3.4) and split-brain consistency test - (§3.3). +- **The decision matrix is the test plan.** Every row of §4.1 × each + baseline, every trigger of §4.2, and every row of §6, as table-driven + tests. The ~24-case matrix deleted by PR #838 (net −18 tests in the + consent module, replaced by happy-path cases only) is restored in + equivalent form against the new API; signal-precedence conflicts + (opt-out + consenting TCF) are mandatory cases, not optional ones. +- Policy validation tests for every §3.3 rejection, exercised through both + acceptance paths (push-time and startup). +- Shipped-table coverage test (§3.5) and jurisdiction-consistency test + (§3.4) covering both legacy lists. - One end-to-end integration scenario per posture: opt-in jurisdiction with and without consent, opt-out jurisdiction with GPC (including GPC + a - consenting TCF string), and the no-geo/default-country path. + consenting TCF string), the no-geo/default-country path, and the + no-policy compiled fallback. ## 9. Out of scope - Additional purposes (extension procedure in §2). -- Runtime-loadable policy (the embedded file is deliberate: policy changes - are code reviews). If runtime policy is wanted later, it is its own spec - with its own validation story. +- A build-time-embedded policy file (PR #838's approach) — rejected for the + reasons in §3.1, not deferred. +- Per-signal jurisdiction scoping (honoring GPC only where a law defines + it): rejected in favor of the global rule in §4; revisiting it is a + policy-model change requiring its own review. diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 04663b58c..ea0ac765a 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -6,7 +6,7 @@ **Related specs:** `2026-07-30-permission-model-design.md`, `2026-07-30-provider-migration-rollout-design.md`, `2026-07-30-client-cycle-ec-resolve-design.md` -**Last updated:** 2026-07-30 +**Last updated:** 2026-07-31 > **Context.** PR #838 proposed a first implementation of this epic in a single > change. Review of that PR surfaced design gaps this spec exists to close @@ -33,9 +33,11 @@ Goals: - Defaults are neutral: with no configuration, no EC is created, device classification uses only the User-Agent, and no geolocation is performed. A default deployment makes no third-party or host-specific call. -- A provider **declares** the permissions its data use requires (see the +- An **EC provider declares** the permissions its data use requires (see the permission model spec); **core enforces** that declaration. A provider - cannot authorize itself. + cannot authorize itself. (Geo and device providers are governed + differently — they execute as _inputs_ to permission resolution and cannot + be gated on its output; see §5.) - All adapters (Fastly, Axum, Cloudflare, Spin) behave identically for identical configuration, or fail loudly at startup where a host cannot satisfy the selected provider. @@ -71,6 +73,14 @@ provider = "builtin" provider = "platform" ``` +**Deliberately not carried over from PR #838:** the `host-signals` EC +provider (identity from HMAC over JA4/HTTP-2 TLS fingerprints plus client +IP). Minting _identity_ from TLS fingerprints is a different privacy +proposition from device _classification_ (#780) and was specified by no +issue; if wanted, it returns with its own spec and its own vocabulary +discussion. A config selecting `provider = "host-signals"` is rejected at +startup like any unknown key (migration spec §4). + ## 3. The identity lifecycle contract This is the section PR #838 lacked, and the source of its most structural @@ -85,15 +95,17 @@ An `EdgeCookieProvider` owns the **complete lifecycle** of the identifiers it mints. Every lifecycle operation core performs on an EC value MUST be routed through the selected provider: -| Lifecycle operation | Where core uses it today | Contract | -| -------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | -| **Recognize** | Reading `ts-ec` back from the request; deciding `ec_was_present` | Provider validates that a returned cookie value is one of its identifiers. A value the selected provider does not recognize is treated as absent. | -| **Hash / normalize** | KV identity-graph keys, log redaction | Provider (or a provider-supplied codec) maps an identifier to its stable KV key form. | -| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider recognizes — never a shape-gated subset. | +| Lifecycle operation | Where core uses it today | Contract | +| ------------------------- | --------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | +| **Recognize** | Reading `ts-ec` back from the request; deciding `ec_was_present` | Provider validates that a returned cookie value is one of its identifiers. A value the selected provider does not recognize is treated as absent. | +| **Key for the graph row** | KV identity-graph row reads/writes | The row key is the identifier **verbatim**; the provider guarantees its identifiers are stable and KV-safe. | +| **Hash prefix** | IP-cluster sizing (`cluster_trust_threshold` prefix listing), pull-sync dedupe, log redaction | Provider maps an identifier to its hash prefix. This prefix **deliberately collides** across identifiers minted from the same client evidence — the collision is load-bearing for cluster-trust counting, and a provider that returns a unique-per-identifier value silently breaks it. | +| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider recognizes — never a shape-gated subset. | **Invariant:** for every provider `P` and every identifier `id` minted by `P`, -`P.recognize(id)` is true, `P` produces a stable KV key for `id`, and a +`P.recognize(id)` is true, `P` produces a stable hash prefix for `id` (and +two identifiers minted from the same client evidence share it), and a withdrawal request carrying `id` tombstones it. A conformance test suite MUST assert this round-trip for every shipped provider, and the suite MUST be written so a future provider crate can run it against its own implementation. @@ -110,7 +122,8 @@ NOT ship without a caller: - `IdentityInput.permissions` / `IdentityInput.consent` (ignored by all built-ins), - `DeviceProvider::required_permissions` / `GeoProvider::required_permissions` - **unless** the enforcement point of §5 lands in the same change. + — dropped entirely, not deferred: §5 explains why these two kinds cannot + be permission-gated at all. If a future feature needs one of these, it arrives with that feature. @@ -126,23 +139,35 @@ pub trait EdgeCookieProvider { fn generate(&self, input: &IdentityInput<'_>) -> Result>; /// Whether `value` is an identifier this provider minted. fn recognize(&self, value: &str) -> bool; - /// Stable KV key form of a recognized identifier. - fn kv_key(&self, id: &EcId) -> KvKey; + /// Hash prefix of a recognized identifier (see §3: collides by design + /// across identifiers minted from the same client evidence). + fn hash_prefix(&self, id: &EcId) -> HashPrefix; } ``` -(Names indicative; the shape is normative.) - -## 5. Permission enforcement is core's job — for all three provider kinds - -Before executing **any** provider (EC, device, or geo), core resolves the -request's permission set (see the permission model spec) and refuses to run a -provider whose `required_permissions()` are not all set. PR #838 declared this -method on all three traits but consulted it only for the EC provider; the -device and geo declarations were decorative. That is worse than absent — it -reads as a gate and is not one. The enforcement point MUST be a single shared -code path used by all three provider kinds, with a test per kind proving a -provider declaring an unset permission does not execute. +(Names indicative; the shape is normative. `required_permissions` joins the +trait at step 5 of §11, together with its enforcement point.) + +## 5. Permission enforcement is core's job — for EC providers + +Before executing an **EC provider**, core resolves the request's permission +set (see the permission model spec) and refuses to run a provider whose +`required_permissions()` are not all set, with a test proving a provider +declaring an unset permission does not execute. + +This gate applies to EC providers **only**, and the reason is structural, +not convenience: the permission set is resolved _from_ jurisdiction, which +is resolved _by_ the geo provider — gating geo (or device, which runs in +the same pre-resolution phase) on the resolved set would be circular. +PR #838 declared `required_permissions` on all three traits but consulted +it only for the EC provider; the geo and device declarations were +decorative — worse than absent, because they read as a gate and are not +one. This spec resolves that by **not having** the method on those traits +(§4). Geo and device providers are governed by explicit operator selection, +the capability checks of §6, and the permission model's vocabulary rule: if +a future vocabulary adds a purpose covering geolocation or fingerprinting, +gating those providers will require a two-phase resolution design specified +at that time (permission model spec §7). ## 6. Selection, validation, and failure modes @@ -158,9 +183,9 @@ startup error, never a request-time error and never a silent behavior change. | No `provider`, no providers block | Valid: the neutral default for that concern. | Unknown fields inside every provider config block are rejected -(`deny_unknown_fields` on all new settings structs — PR #838 applied it to -`Ec` but not to `EcProviders`, `DeviceConfig`, or `GeoConfig`, so a typo like -`providr` was silently ignored). +(`deny_unknown_fields` on all new settings structs — the pre-existing `Ec` +struct already has it, but PR #838 shipped `EcProviders`, `DeviceConfig`, +and `GeoConfig` without it, so a typo like `providr` was silently ignored). ## 7. Composition root and adapter parity @@ -210,15 +235,16 @@ prominent in release notes: "fastly"`; the migration guide lists this as a behavior-preserving step for Fastly deployments. - **Geo.** With no geo provider, jurisdiction resolution falls to the - configured default country. The permission model spec (§5) constrains this - combination so it cannot silently grant permissions to mis-attributed - traffic. + configured default country. The permission model spec (§5.3) constrains + this combination so it cannot silently grant permissions to mis-attributed + traffic, and §11 below sequences the default flip so the constraint exists + before the flip does. ## 10. Testing strategy - Provider conformance suite (§3 invariant) run against every shipped provider. -- Enforcement tests per provider kind (§5). +- EC permission-enforcement tests (§5). - Settings validation tests for every row of the §6 table, including the block-without-selector rejection. - Parity suite additions of §7. @@ -232,8 +258,19 @@ prominent in release notes: ID-stability vectors). 2. Settings selection + validation table. 3. Composition root + all four adapters wired through it, parity cases. -4. Device and geo providers with the shared enforcement point of §5. - -Each step is independently reviewable; none depends on the permission model -landing first (the EC gate keeps its current jurisdiction logic until the -permission model PR replaces it). +4. Device and geo provider selection. **The geo neutral default does not + flip in this step**: under the current jurisdiction gate, absent geo + resolves to `Unknown`, which fails closed — flipping the default here + would zero EC issuance for every deployment that had not yet opted into + `[geo] provider = "platform"`. Until step 5, the Fastly adapter's geo + selection defaults to `platform` (today's always-on behavior); the + selector exists, only its default is held back. +5. The permission model PR: flips the geo default to none **in the same + change** that introduces the `default_country` fallback and the §5.3 + acknowledgment guard, and adds the EC permission-enforcement point of + §5; `required_permissions()` appears on the EC trait in this step, not + before (per the §4 minimalism rule). + +Steps 1–4 are independently reviewable, behavior-preserving, and do not +depend on the permission model: the EC gate keeps its current jurisdiction +logic until the permission model PR replaces it. diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index c00f8c348..5e7a327c1 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -5,7 +5,7 @@ **Issue references:** #777–#781 (epic) **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-permission-model-design.md` -**Last updated:** 2026-07-30 +**Last updated:** 2026-07-31 > **Context.** The provider/permission epic is a breaking change to a live > identity system. PR #838's review showed that the riskiest part of such a @@ -32,37 +32,45 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Policy decision: the shipped `US` baseline decides. If the baseline grants `store-on-device` without a signal, that is a **declared change** requiring sign-off in the policy file review, with rationale in the file itself | Declared change (if made) | -| 4 | UK request, no TCF record → no EC | Same, unless the policy file deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | Unknown jurisdiction (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC allowed, never tombstoned | Same (withdrawal only where baseline is opt-in; permission spec §4.2) | Preserved | -| 7 | EID transmission requires storage + personalization consent where regulated | Same via `store-on-device` ∧ `select-personalised-ads` | Preserved | -| 8 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 9 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"` | Declared change with a documented restore step (§5) | - -Rows 3 and 4 are policy decisions, not code decisions: they belong in the -`permissions.yaml` review, made explicitly by maintainers — not implied by an -implementation. +| # | Decision (today) | After epic | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Policy decision: the shipped `US` baseline decides. If the baseline grants `store-on-device` without a signal, that is a **declared change** requiring sign-off in the policy review, with rationale in the policy itself | Declared change (if made) | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | + +Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in +the `[permissions]` policy review, made explicitly by maintainers — not +implied by an implementation. ## 3. Identity stability guarantee -For a deployment that selects `provider = "hmac"` and carries its passphrase -over verbatim: - -- The minted identifier is **bit-identical** to today's: - `HMAC-SHA256(passphrase, normalized_ip)` in the existing encoding. -- Cookie name, attributes, and max-age are unchanged; existing `ts-ec` - cookies are recognized by the provider. -- KV identity-graph keys (`ec_hash`, normalization) are unchanged; no - existing graph row is orphaned. - -Enforced by **pinned known-answer tests**: fixed passphrase + IP → exact -expected identifier, cookie string, and KV key, committed as vectors so any -divergence fails CI rather than rotating a production identity base. +Today's EC identifier is `{64-hex}.{6-char}` where the 64-hex part is +deterministic — `HMAC-SHA256(passphrase, normalized_ip)` — and the 6-char +suffix is **random per mint** (an existing test asserts two mints differ). +Full identifiers are therefore not reproducible by design, and no test may +pretend otherwise. What stability means, precisely, for a deployment that +selects `provider = "hmac"` and carries its passphrase over verbatim: + +- **The deterministic prefix is bit-identical.** Pinned known-answer + vectors: fixed passphrase + IP → exact expected 64-hex prefix, committed + so any divergence fails CI rather than rotating the production identity + base. +- **Existing cookies stay recognized.** Fixture `ts-ec` values minted by + the pre-epic code pass the provider's `recognize`, and their graph rows + (keyed by the identifier verbatim) remain reachable — no row is orphaned. +- **The hash prefix keeps its semantics.** `ec_hash` remains the 64-hex + prefix, preserving both its stability and its deliberate collision across + identifiers minted from the same IP — the property IP-cluster trust + counting depends on (providers spec §3). +- **Cookie name, attributes, and max-age are unchanged** (the domain + remains config-derived, as today). ## 4. Configuration migration @@ -87,22 +95,32 @@ Requirements: 1. **Old key fails loud.** `[ec] passphrase` is rejected at startup with a message naming the new location — not a generic unknown-field error. + Implementation note: `Ec` already carries `deny_unknown_fields`, which + would reject the key generically; producing the actionable message means + keeping a deprecated `passphrase` field whose presence triggers the + custom error. 2. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the docs reaches if they miss one line — validated green and silently minted zero ECs. -3. **The example config ships the migrated happy path**, uncommented: +3. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by + PR #838, deliberately not carried into this epic — providers spec §2) + and `provider = "client-fixed"` are unknown keys and rejected like any + other, so a config written against the PR #838 example cannot silently + select a provider that no longer exists. +4. **The example config ships the migrated happy path**, uncommented: `provider = "hmac"` with its block, `[geo] default_country`, and (for Fastly) the behavior-preserving `[device] provider = "fastly"` and `[geo] provider = "platform"` lines present with a comment stating what removing them changes. PR #838's example shipped the passphrase block uncommented with the selector commented out — steering operators directly into the silent-stateless state. -4. Every misconfiguration in the providers spec §6 table fails at +5. Every misconfiguration in the providers spec §6 table fails at **startup**. Request-time failure for a configuration error is a defect. -5. Config-store payload validation (`ts config push`) applies the same - rules, so a bad config is rejected at push time, before any instance +6. Config-store payload validation (`ts config push`) applies the same + rules — including `[permissions]` policy validation (permission spec + §3.3) — so a bad config is rejected at push time, before any instance restarts into it. ## 5. Behavior-preserving migration recipe (operator-facing) @@ -122,18 +140,30 @@ provider = "fastly" # Fastly deployments: preserves the JA4 bot gate [geo] provider = "platform" # preserves per-request jurisdiction detection -default_country = "FR" # used only when the host lookup fails +default_country = "FR" # used only when the host lookup fails (fail-closed) + +# Preserves today's treatment of countries outside the regulation lists +# ("non-regulated" → identity allowed). Omit this section to adopt the +# protective default instead: signal required worldwide (§2 row 7). +[permissions.groups.non-regulated] +default = "granted" + +[permissions.rules] +default = "non-regulated" ``` and separately documents the neutral configuration and what it does _not_ do. The guide states explicitly that `default_country` alone does not replace geo -lookup, and why the permissive-default + no-geo combination requires the -explicit acknowledgment flag (permission spec §5.3). +lookup, why the no-geo combination requires the explicit acknowledgment flag +(permission spec §5.3), and that no recipe preserves row 8 of §2 — the +global honoring of opt-out signals is unconditional. ## 6. Rollout sequence and observability -1. Implementation PRs land in the epic's order (providers first, permission - model second); each is reviewable against §2 in isolation. +1. Implementation PRs land in the epic's order (providers spec §11: + providers first with the geo default held at today's behavior, the + permission model PR flipping it together with its guard); each PR is + reviewable against §2 in isolation and states which rows it touches. 2. Before/after deploy, operators watch **EC issuance rate** and EID attachment rate; the migration guide names these as the canary metrics, because the failure mode of a bad migration is a silent drop to zero (or a @@ -143,18 +173,21 @@ explicit acknowledgment flag (permission spec §5.3). signal permissions. One line, greppable, stable format. 4. Rollback is config-only where possible: reverting to the previous config version restores the previous behavior on the previous binary. The - one irreversible artifact is withdrawal tombstones — which is why §2 row 6 - (no tombstones without affirmative withdrawal in an opt-in jurisdiction) - is non-negotiable. + one irreversible artifact is withdrawal tombstones — which is why the + withdrawal triggers (permission spec §4.2) are exhaustive and why §2 + rows 6 and 8 call out tombstoning explicitly. Cleanup of identities + minted before a policy tightening (permission spec §4.2 trigger 3) is an + operational action documented in the guide, not an automatic one. ## 7. Documentation deliverables - Migration guide page (§5), linked from `CHANGELOG.md` and the release notes. - `configuration.md` documents **every** valid `provider` value for all - three concerns, and documents environment-variable overrides only if they - actually work in production builds (in PR #838 the documented - `TRUSTED_SERVER__EC__PROVIDER` override existed only under `#[cfg(test)]`). + three concerns, the full `[permissions]` schema, and environment-variable + overrides only if they actually work in production builds (in PR #838 the + documented `TRUSTED_SERVER__EC__PROVIDER` override existed only under + `#[cfg(test)]`). - The permission model page states the §4 precedence rules of the permission spec verbatim — operator docs and normative spec must not diverge on precedence, and prose like "signals are mapped as a grant or a revoke" From 9886091e5ca843071cfd0ca38f32fbd881a9a33e Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 01:41:14 -0700 Subject: [PATCH 03/14] Address review: close identity/privacy gaps in the provider and permission specs Blocking findings from review of PR #986, all addressed: - Raw EC egress is now a first-class enforcement point with a mandatory egress inventory (user.id, derived request IDs, page bids, proxy/click forwarding, identify, pull/batch sync, graph access): bidstream egress requires both purposes, first-party identity operations require store-on-device, revocation is exempt, and no-provider mode never vacuously allows an existing cookie to egress. - The behavior-preserving recipe carries the complete policy table plus a delta; a partial permissive-default-only policy is called out as the trap it is, and the exact recipe text becomes a CI fixture run through the decision matrix. - The EC permission gate is split: it covers minting and identity use only; parse, canonicalization, and tombstoning always run, with a spy-provider test - a blanket gate would block withdrawal in exactly the state an opt-out produces. - Provider switching gets active-writer/legacy-readers semantics ([ec] legacy_providers) so old identities keep resolving and stay withdrawable; unmatched cookies never egress. - The lifecycle contract now distinguishes the canonical graph key (provider-owned canonicalization, equivalent envelopes collapse) from the cluster prefix, which must be a literal byte prefix of the graph key because cluster sizing is a KV prefix listing; cluster support is an optional capability with an explicit degradation policy. - Device gating rationale corrected: only geo is circular; device is ungated by decision (security classification authorized by operator selection), with the boundary stated - uses beyond security classification need a vocabulary extension and a gate. - Withdrawal triggers made consistent (TCF refusal withdraws under requires_signal or denied), and a withdrawal-durability contract added: tombstones first, cookie expiry only on success, browser-side durable signals as the retry queue, fault-injection tests. - A signal-normalization matrix is now required (dual-TCF conflict modes, expiry, proxy mode, KV fallback, exact GPP fields), and malformed-but-present records fail closed for acquisition instead of degrading to absent. - Auction jurisdiction class is an explicit per-group regime attribute (gdpr / us-privacy / none), never inferred from purpose flags, and a first-class enforcement point. - The no-geo acknowledgment guard now keys on any enabled jurisdiction consumer, not only EC-provider selection. - Policy validation additionally requires rules.default when the section is present, rejects empty sections and case-insensitive duplicate keys, and canonicalizes default_country. - Resolve endpoint: exact Origin-allowlist membership or session-bound CSRF token (Sec-Fetch-Site demoted to defense-in-depth), real replay mitigation (session nonce or one-time consumption), and bounded-input requirements with 413 boundary tests. - Response hook: structured mutation operations that core validates and attributes (no raw header-map access), framing/hop-by-hop headers reserved, and a normative response-eligibility matrix. - Each spec now carries an explicit divergence table against its issue (#778, #779, #782) so there is one acceptance contract. - Non-blocking clarifications folded in: geo lookup-failure residual declared and metered, deterministic entropy required in conformance tests. --- ...26-07-30-client-cycle-ec-resolve-design.md | 46 ++-- ...integration-response-header-hook-design.md | 50 +++- .../2026-07-30-permission-model-design.md | 218 ++++++++++++++---- .../2026-07-30-pluggable-providers-design.md | 157 ++++++++++--- ...07-30-provider-migration-rollout-design.md | 85 ++++--- 5 files changed, 432 insertions(+), 124 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 1b9eacc12..e0a5529fe 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -50,17 +50,28 @@ Everything in this spec follows from that. `POST /_ts/api/v1/ec/resolve` (final path TBD) MUST: -1. **Reject cross-site requests.** Require a same-site assertion: `Origin` - (or `Sec-Fetch-Site: same-origin/same-site`) validated against the - publisher's origin allowlist — configuration that does not exist yet and - must be defined by this feature (§7, question 5); requests without a - validating header are rejected. CSRF-token designs are acceptable but not - required if origin-based rejection is enforced. -2. **Verify the payload cryptographically per provider.** The provider's - `resolve_from_client` accepts only payloads that are signed by an - expected party, **audience-bound** to this publisher, and **expiring** - (bounded lifetime, single-use where the scheme allows). A provider whose - payloads are replayable constants fails this bar by construction. +1. **Reject cross-site requests with an exact origin check.** The request + is authorized only by **exact membership of the `Origin` header value in + the publisher's origin allowlist** — configuration that does not exist + yet and must be defined by this feature (§7, question 5) — or by a + **session-bound CSRF token**. `Sec-Fetch-Site` is **defense-in-depth + only, never an authorizing alternative**: it carries no origin value to + compare against an allowlist, and `same-site` admits every sibling + subdomain — one compromised or attacker-registered subdomain would be + enough to set identity. Requests with no `Origin` and no valid token are + rejected. +2. **Verify the payload cryptographically per provider — including against + replay.** The provider's `resolve_from_client` accepts only payloads + that are signed by an expected party, **audience-bound** to this + publisher, and **expiring**. Audience binding and expiry alone do not + mitigate replay — a captured token installs in another browser for the + whole validity window — so one of the following is additionally + required: **binding to the requesting browser session** (a server-issued + nonce the payload must embed), or **server-side one-time consumption** + (a replay cache on the payload's unique id). A scheme that can support + neither may only ship if its residual replay window is quantified and + explicitly accepted in the feature's issue — "single-use where the + scheme allows" is not a mitigation. 3. **Preserve the identity-graph invariant.** The cookie is set only after the corresponding graph row is written, mirroring the organic path. Graph unavailable → no cookie, same as organic generation. @@ -75,6 +86,13 @@ Everything in this spec follows from that. 6. **Be uncacheable and permission-gated.** `Cache-Control: no-store`; the same `store-on-device` permission gate as organic EC creation runs before any cookie is set. +7. **Bound every input.** A maximum request-body size (order of the 64 KiB + limit PR #838 at least had), enforced by a **bounded read independent of + `Content-Length`** — a missing, false, or chunked length must not bypass + it; a `Content-Type` allowlist; and a length/character-set constraint on + the resulting identifier that keeps it cookie-safe and within the KV + limits of the providers spec §3. Tests exercise the exact 413 boundary + and the missing/false/chunked-length cases. ## 4. Requirements on the page script @@ -103,8 +121,10 @@ sentence as the only guardrail. ## 6. Testing -- Endpoint: origin-rejection, expired/replayed/foreign-audience payload - rejection, graph-unavailable refusal, permission-gate refusal — each as an +- Endpoint: origin-rejection (including `Sec-Fetch-Site`-only requests, + which must fail), expired/replayed/foreign-audience payload rejection, + graph-unavailable refusal, permission-gate refusal, and the §3.7 body / + content-type / identifier limits at their exact boundaries — each as an integration test, not only unit tests. - Browser round trip (JS → POST → Set-Cookie → next request recognized) in the integration suite; PR #838 shipped the JS with in-process unit tests diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 01aea582d..b91846222 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -29,6 +29,15 @@ mutators to the outbound response for HTML document responses it processed. - `IntegrationRegistration::builder(ID).with_response_mutator(...)` registers a mutator; `IntegrationRegistry::apply_response_headers(...)` applies all registered mutators in registration order. +- **The mutator API is structured operations, not header-map access.** A + mutator returns (or is handed a recorder for) typed operations — + `append(name, value)`, `replace(name, value)`, + `append_set_cookie(cookie)` — which **core validates and applies**, + attributing each to its integration id. PR #838's shape handed the + integration an unrestricted `&mut HeaderMap`, which makes §3's collision + policy unenforceable by construction: core cannot validate or attribute + writes it never sees. An API that cannot express a violation beats one + that promises to catch it. - **Every adapter calls the apply point** on its outbound-response path for processed documents. The call site lives in shared response-finalization code where one exists; where adapters finalize independently, each adapter @@ -45,13 +54,16 @@ mutators to the outbound response for HTML document responses it processed. - Mutators may not touch **reserved surface**, which is defined at two granularities because `Set-Cookie` is multi-valued: (a) reserved header - _names_ — the `x-ts-*` namespace and the consent/privacy headers core - emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, + _names_ — HTTP framing and hop-by-hop headers (`Content-Length`, + `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, + `Keep-Alive`), the `x-ts-*` namespace, and the consent/privacy headers + core emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, `ts-eids`, and the other `ts-*` cookies core owns. An integration may append its own `Set-Cookie` values; it may not set or expire a reserved - cookie name. Violations are dropped and logged at `warn` with the - integration id. The reserved-cookie list is a single constant next to the - cookie definitions, not duplicated in the hook. + cookie name. Violations are rejected at the operation layer (§2) and + logged at `warn` with the integration id. The reserved lists are single + constants next to the definitions they protect, not duplicated in the + hook. - For non-reserved headers, the mutator API distinguishes **append** from **replace** explicitly; the default is append (for `Set-Cookie`, append is the only non-reserved operation — replace is not offered). Replacing a @@ -59,6 +71,24 @@ mutators to the outbound response for HTML document responses it processed. - Later registrations see earlier mutations (order = registration order, which is deterministic). +## 3a. Response eligibility — normative + +Which responses the hook runs on, enumerated so two implementations cannot +diverge silently: + +| Response | Hook runs? | +| --------------------------------------------- | ------------------------------------------------------------ | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes — operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | +| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` | No | + +This deliberately narrows #782's general "outbound response" phrasing to +processed documents (§6). + ## 4. Done-when (from #782, sharpened) 1. Trait + builder + registry application, each public item documented. @@ -79,3 +109,13 @@ its first real consumer is identified** (§4.2) — at any point in the epic's sequence, blocking nothing and blocked by nothing. If no consumer materializes, it does not land; being unblocked is not a reason to ship scaffolding. + +## 6. Divergences from issue #782 + +This spec supersedes #782 on the following points; the issue is updated to +reference this spec when the PR merges: + +| #782 says | This spec says | Why | +| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter; enumerating beats implying | +| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4.2) | PR #838 shipped the trait with zero call sites; an unrestricted `&mut HeaderMap` cannot enforce any collision policy | diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index bd5da97c9..a66037a7e 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -112,9 +112,15 @@ overrides. Each permission resolves to an **acquisition rule**: ```toml [permissions.groups.gdpr-eu] +regime = "gdpr" default = "requires_signal" [permissions.groups.us-opt-out] +regime = "us-privacy" +default = "granted" + +[permissions.groups.non-regulated] +regime = "none" default = "granted" [permissions.rules] @@ -123,10 +129,10 @@ US = "us-opt-out" # Overrides name explicit acquisition rules — no +/- sigil syntax; TOML # expresses the target state directly. "US/CA" = { group = "us-opt-out", overrides = { select-personalised-ads = "requires_signal" } } -# Reserved key: countries that resolve but match no rule. Distinct from -# [geo] default_country, which handles requests that resolve no country at -# all (§5.4). -default = "gdpr-eu" +# Reserved key: countries that resolve but match no rule. Required whenever +# the [permissions] section is present. Distinct from [geo] default_country, +# which handles requests that resolve no country at all (§5.4). +default = "non-regulated" ``` A group's `default` covers unlisted permissions; a group may also name @@ -135,6 +141,14 @@ target state (including `requires_signal`) is expressible — PR #838's `+`/`-` sigil scheme could not express "requires a signal", the most common real-world override. +Each group carries a required **`regime`** class (`gdpr`, `us-privacy`, or +`none`). This is the explicit legal-classification channel: consumers that +need a jurisdiction _class_ — above all server-side auction dispatch — read +`regime`, never infer a class from purpose flags. Inference is lossy +(Purpose 1 and Purpose 4 may legitimately carry different rules, and a +non-GDPR operator may choose an opt-in Purpose 4) and would smuggle legal +meaning back into identifiers this spec declares purely technical (§2). + ### 3.3 Validation — at config acceptance, not request time Policy is validated where every other setting is: at `ts config push` (a bad @@ -153,8 +167,17 @@ Validation rejects: the region part matches `[A-Z0-9]{1,3}`. The `US/CA` slash form is the house rule-key format corresponding to ISO 3166-2 `US-CA`; - references to permissions outside the enforced vocabulary (§2); -- references to undefined groups; -- groups that neither list every permission nor provide `default`. +- references to undefined groups, and groups missing the `regime` class; +- groups that neither list every permission nor provide `default`; +- a present `[permissions]` section without a `rules.default` entry (§5.4 + depends on it existing — its absence must be a validation error, not a + runtime surprise); +- an empty `[permissions]` section (ambiguous intent: an operator who wants + the compiled-in fallback omits the section entirely); +- duplicate rule keys under case-insensitive comparison (`FR` and `fr`); +- a `[geo] default_country` that is not an assigned ISO code; it is + canonicalized to uppercase, and startup logs which rule (or + `rules.default`) it resolves to. ### 3.4 One source of jurisdiction truth @@ -167,12 +190,13 @@ country to one has no effect on the other, and an operator has no signal that they disagree). Requirement: the auction gate's jurisdiction class derives from the same -resolved policy (a country is GDPR-class when its rule resolves to an -opt-in baseline for `select-personalised-ads`). Where the legacy lists must +resolved policy, reading the rule's explicit **`regime`** class (§3.2) — a +country is GDPR-class when its rule resolves to a `regime = "gdpr"` group. +The class is never inferred from purpose flags. Where the legacy lists must survive an interim period, a CI test asserts consistency between each list -and the policy table, with deliberate divergences recorded as explicit, -commented exceptions in the test — never silent. Both legacy lists are in -scope, not only the GDPR one. +and the policy's regime classes, with deliberate divergences recorded as +explicit, commented exceptions in the test — never silent. Both legacy +lists are in scope, not only the GDPR one. ### 3.5 Shipped-table coverage @@ -250,16 +274,21 @@ The triggers, exhaustively — nothing else withdraws: baseline.** (For US states this preserves today's behavior; elsewhere it is the declared change of §4's global-opt-out rule.) 2. **A TCF record refusing `store-on-device` withdraws iff the baseline is - `requires_signal`.** Where the baseline is `granted`, refusal blocks - _new_ grants but never tombstones: tombstones are irreversible, and - PR #838 wrote them for visitors in unregulated jurisdictions whose - global CMP emitted a purpose-refusing string — permanent identity loss - under a regime the deployment never opted into. + `requires_signal` or `denied`.** Where the baseline is `granted`, + refusal blocks _new_ grants but never tombstones: tombstones are + irreversible, and PR #838 wrote them for visitors in unregulated + jurisdictions whose global CMP emitted a purpose-refusing string — + permanent identity loss under a regime the deployment never opted into. + (The `denied` arm exists so trigger 3 is coherent: after a policy + tightens to `denied`, an affirmative refusal must still be able to + withdraw a pre-existing identity.) 3. **A policy edit is not a user signal.** Tightening a baseline to `denied` stops new identity but does not itself tombstone identities minted before the change; cleaning those up is an operational action - (migration spec §6). An affirmative user signal (trigger 1 or 2) still - withdraws them. + (migration spec §6). An affirmative user signal (trigger 1, or trigger 2 + under the now-`denied` baseline) still withdraws them — with a test + pinning exactly this sequence: existing EC → policy tightens to + `denied` → refusal arrives → tombstone. 4. **Absence of signal never destroys identity.** A visitor who has not yet made a choice is never stripped of an existing identity. @@ -269,6 +298,54 @@ withdrawal even when a consenting TCF record is present. every trigger above; in PR #838 the headline "withdrawal expires identity" behavior had no unit test at all. +### 4.3 Withdrawal durability + +Withdrawal is two writes — the KV tombstones and the cookie expiry — and +the contract for partial failure is explicit (PR #838 expired the cookie +first and logged-and-swallowed tombstone-write failures, which can leave a +live graph identity with no browser handle pointing at it): + +- **Order: tombstones first, cookie expiry second.** The cookie is expired + only after the tombstone writes succeed. +- **On tombstone-write failure, the cookie is left in place** and the + failure is logged at `error` with a metric. This is deliberately + self-healing: every withdrawal trigger is durable client-side (GPC is a + browser setting, the TCF record lives in the CMP's storage), so the next + request re-presents the signal and retries the whole withdrawal. No + quarantine queue is needed; the browser is the retry queue. +- Identify, batch-sync, and pull-sync treat a tombstone as authoritative + revocation (as today); a row whose withdrawal is pending retry is simply + still live until the retry lands, and never partially withdrawn. +- Fault-injection tests cover: tombstone write fails → cookie untouched, + error logged; subsequent request with the same signal → withdrawal + completes. + +### 4.4 Signal normalization + +§4's precedence operates on normalized inputs: one effective consent +record and one effective opt-out state per request. The normalization +layer is where today's real-world mess lives, and PR #838 collapsed it +silently. The implementation ships a **normalization matrix** — a +table-driven spec-and-test artifact — covering at minimum: + +- **Dual consent records**: standalone TCF cookie vs. GPP-embedded TCF, + including per-purpose disagreement, resolved per the existing configured + conflict modes (restrictive / permissive / newest). Each mode is either + preserved or explicitly retired in the migration matrix — not dropped. +- **Record expiry** and the persisted-KV consent fallback: when a stored + record substitutes for an absent live one, and how staleness is bounded. +- **Proxy/mirror mode** (CMP consent mirrored server-side): where the + mirrored state enters precedence. +- **Exact GPP fields**: which section fields constitute a sale/sharing/ + targeted-advertising opt-out, enumerated per supported section — "GPP + opt-out" is not a single bit. +- **Malformed-but-present records fail closed for acquisition**: a consent + record that is present but undecodable blocks grants (it does not + degrade to "absent", which under a `granted` baseline would turn garbage + into a grant — the fail-open path in both #838 and the first draft of + this spec). A malformed record never triggers withdrawal — destruction + requires an affirmative, decodable signal (§4.2). + ## 5. Jurisdiction resolution ### 5.1 Order @@ -288,6 +365,13 @@ all — that is the capability check of providers spec §6, and it prevents a "selected but always empty" provider from silently converting every request to §5.3 semantics without §5.3's guard. +Declared residual: when the default country's baseline is permissive, a +per-request lookup failure is a per-request grant to traffic of unknown +origin — this path is not fail-closed, and the spec does not pretend it is. +The lookup-failure rate is exported as a metric and logged, so an elevated +rate (a degraded geo backend silently converting traffic to the default) is +observable rather than invisible. + ### 5.3 No geo provider selected Every request resolves to `default_country` — jurisdiction becomes a static @@ -297,14 +381,20 @@ this dangerous: with a `requires_signal` baseline, a page-global CMP that emits a consenting TCF string grants permissions for every mis-attributed visitor just as effectively. -Constraint: **startup fails** when an EC provider is selected and no geo -provider is, unless the operator sets an explicit acknowledgment -(`[geo] assume_single_jurisdiction = true`). Stateless deployments (no EC -provider) are exempt. Without this guard, the natural migration config -(`default_country = "US"`, geo unset) silently grants `store-on-device` and -EID transmission to every EU visitor — the highest-severity finding of the -PR #838 review. The startup log always prints the effective baseline and -whether geo is live. +Constraint: **startup fails** when no geo provider is selected and any +**jurisdiction consumer** is enabled, unless the operator sets an explicit +acknowledgment (`[geo] assume_single_jurisdiction = true`). Jurisdiction +consumers are enumerated, not implied: an EC provider is selected, +server-side auction dispatch is gated on `regime` (§7), or any raw-EC / +EID egress path is active. An EC-provider-only exemption would be too +narrow — a stateless deployment still dispatches auctions off the policy's +regime class, and no geo + a permissive static jurisdiction misclassifies +EU traffic for that decision just as it would for identity. Only a +deployment with **no** jurisdiction-sensitive behavior is exempt. Without +this guard, the natural migration config (`default_country = "US"`, geo +unset) silently grants `store-on-device` and EID transmission to every EU +visitor — the highest-severity finding of the PR #838 review. The startup +log always prints the effective baseline and whether geo is live. ### 5.4 Defaults, two distinct fallbacks @@ -319,17 +409,17 @@ migration story unresolvable (migration spec §2, rows 5 and 7). ## 6. Failure-mode matrix — normative -| Condition | Resolution behavior | -| ---------------------------------------------------- | ------------------------------------------------------------ | -| Geo lookup fails at request time (provider selected) | `default_country` baseline | -| No geo provider configured | `default_country` baseline, guarded by §5.3 | -| Country resolved, no matching rule | Policy `rules.default` | -| Region resolved, no region rule | Country rule | -| No `[permissions]` section | Compiled-in fallback: everything `requires_signal` | -| Malformed policy | Rejected at config push / startup (§3.3) — never per request | -| No `default_country` | Startup failure | -| Undecodable TCF/GPP string | Treated as absent; opt-out signals still honored | -| Signals contradict (opt-out + consent) | Opt-out wins (§4) | +| Condition | Resolution behavior | +| ---------------------------------------------------- | --------------------------------------------------------------------------------------------- | +| Geo lookup fails at request time (provider selected) | `default_country` baseline | +| No geo provider configured | `default_country` baseline, guarded by §5.3 | +| Country resolved, no matching rule | Policy `rules.default` | +| Region resolved, no region rule | Country rule | +| No `[permissions]` section | Compiled-in fallback: everything `requires_signal` | +| Malformed policy | Rejected at config push / startup (§3.3) — never per request | +| No `default_country` | Startup failure | +| Undecodable TCF/GPP record (present but malformed) | Blocks grants (fail-closed acquisition, §4.4); never withdraws; opt-out signals still honored | +| Signals contradict (opt-out + consent) | Opt-out wins (§4) | The overall posture is **fail-closed**: every ambiguous state resolves to the configured baseline or more restrictive, and the one configuration that @@ -350,12 +440,41 @@ Consumers of the resolved set in this epic: providers will require a two-phase resolution that must be specified then, not improvised. 2. **EC lifecycle** — creation requires `store-on-device`; withdrawal per - §4.2. -3. **Bidstream EIDs** — transmission requires `store-on-device` ∧ - `select-personalised-ads`. + §4.2. Recognition, canonicalization, and revocation of an existing + identifier are **never** permission-gated — they must run precisely when + permissions are withdrawn (providers spec §5). +3. **Every raw-EC egress**, not only EIDs. The raw EC identifier leaves the + process today as OpenRTB `user.id`, inside derived auction request IDs, + on the page-bids path, on proxied/click/Testlight forwarding, through the + identify endpoint, through pull/batch sync, and via identity-graph + reads/writes. Gating EIDs alone (PR #838's shape) leaves the raw EC + reaching bidders when Purpose 1 is granted and Purpose 4 refused; worse, + #838's `ec_allowed` was **vacuously true with no provider configured** + (`is_none_or`), so an existing canonical cookie still escaped in + stateless mode. The contract: + - The implementation maintains an **egress inventory**: every code path + where a raw EC (or a value derived from it) leaves the process is + enumerated in one table, each mapped to its required permissions, with + a test per row. + - **Bidstream egress** (`user.id`, EC-derived request IDs, page bids, + EIDs) requires `store-on-device` ∧ `select-personalised-ads` — the raw + EC is identity in the bidstream and is gated exactly as EIDs are. + - **First-party identity operations** (identify, pull/batch sync, + graph reads/writes other than revocation) require `store-on-device`. + - **Revocation paths are exempt** — tombstoning must work when + permissions are unset. + - With **no EC provider configured**, identity use fails closed: a + cookie value present on the request never egresses anywhere; it is + never vacuously allowed. +4. **Bidstream EIDs** — transmission requires `store-on-device` ∧ + `select-personalised-ads` (subsumed by point 3's inventory; listed + separately because it is the one gate PR #838 had). +5. **Server-side auction dispatch** — gated on the explicit policy `regime` + class (§3.4), a first-class enforcement point, not inferred from purpose + flags. The client-cycle resolve endpoint (own spec, currently on hold) would be a -fourth consumer if and when it proceeds. +further consumer if and when it proceeds. ## 8. Testing strategy @@ -365,6 +484,11 @@ fourth consumer if and when it proceeds. consent module, replaced by happy-path cases only) is restored in equivalent form against the new API; signal-precedence conflicts (opt-out + consenting TCF) are mandatory cases, not optional ones. +- The §4.4 normalization matrix as table-driven tests, including every + configured conflict mode and the malformed-record rows. +- The §7 raw-EC egress inventory: one test per inventoried egress proving + the gate, plus a denylist-style check that no ungated egress exists. +- §4.3 fault-injection cases. - Policy validation tests for every §3.3 rejection, exercised through both acceptance paths (push-time and startup). - Shipped-table coverage test (§3.5) and jurisdiction-consistency test @@ -382,3 +506,15 @@ fourth consumer if and when it proceeds. - Per-signal jurisdiction scoping (honoring GPC only where a law defines it): rejected in favor of the global rule in §4; revisiting it is a policy-model change requiring its own review. + +## 10. Divergences from issue #779 + +This spec supersedes #779 on the following points; the issue is updated to +reference this spec when the PR merges, so there is one acceptance contract, +not two: + +| #779 says | This spec says | Why | +| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Unmatched-but-resolved countries fall to the policy's `rules.default`; `default_country` covers only unresolved requests | The two states had different pre-epic behavior; collapsing them made migration unresolvable (§5.4) | +| The full TCF purpose vocabulary is modeled | Only enforced purposes appear (§2) | Nine inert purposes in a policy file are a compliance hazard, not forward compatibility | +| Policy is an embedded file | Policy is `[permissions]` in `trusted-server.toml` (§3.1) | Runtime config-store pipeline; validation at push time | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index ea0ac765a..88035ea91 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -36,8 +36,7 @@ Goals: - An **EC provider declares** the permissions its data use requires (see the permission model spec); **core enforces** that declaration. A provider cannot authorize itself. (Geo and device providers are governed - differently — they execute as _inputs_ to permission resolution and cannot - be gated on its output; see §5.) + differently, for two different reasons spelled out in §5.) - All adapters (Fastly, Axum, Cloudflare, Spin) behave identically for identical configuration, or fail loudly at startup where a host cannot satisfy the selected provider. @@ -95,20 +94,26 @@ An `EdgeCookieProvider` owns the **complete lifecycle** of the identifiers it mints. Every lifecycle operation core performs on an EC value MUST be routed through the selected provider: -| Lifecycle operation | Where core uses it today | Contract | -| ------------------------- | --------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | -| **Recognize** | Reading `ts-ec` back from the request; deciding `ec_was_present` | Provider validates that a returned cookie value is one of its identifiers. A value the selected provider does not recognize is treated as absent. | -| **Key for the graph row** | KV identity-graph row reads/writes | The row key is the identifier **verbatim**; the provider guarantees its identifiers are stable and KV-safe. | -| **Hash prefix** | IP-cluster sizing (`cluster_trust_threshold` prefix listing), pull-sync dedupe, log redaction | Provider maps an identifier to its hash prefix. This prefix **deliberately collides** across identifiers minted from the same client evidence — the collision is load-bearing for cluster-trust counting, and a provider that returns a unique-per-identifier value silently breaks it. | -| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider recognizes — never a shape-gated subset. | +| Lifecycle operation | Where core uses it today | Contract | +| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | +| **Parse / canonicalize** | Reading `ts-ec` back from the request; deciding `ec_was_present`; batch-sync ingestion | Provider parses a cookie value into its **canonical** identifier, or rejects it. Canonicalization is provider-owned: case variants and equivalent envelopes of the same identity (per #778) parse to the same canonical identifier. A value the selected provider does not recognize is treated as absent (but see §6.1 legacy readers). | +| **Canonical graph key** | KV identity-graph row reads/writes | The provider maps a canonical identifier to its graph key: stable, KV-safe (within KV length and character-set limits), collision-free across the provider's identifier space, and namespaced so two providers' key spaces cannot collide. Two equivalent envelopes of one identity map to one key — verbatim cookie bytes as the key would fork graph rows on canonicalization differences and discard today's batch-sync canonicalization. | +| **Cluster prefix** (optional capability) | IP-cluster sizing (`cluster_trust_threshold`, implemented as a **KV prefix listing**), pull-sync dedupe, log redaction | A provider declaring cluster support returns a prefix that is a **literal byte prefix of the canonical graph key** — the cluster count lists keys by prefix, so an independently derived hash that is not an actual key prefix silently reports the wrong cluster size. The prefix deliberately collides across identifiers minted from the same client evidence. A provider without the capability declares so, and cluster-dependent gating follows a configured degradation policy (treat cluster size as unknown, with the KV-write decision that implies made explicit in config) instead of counting garbage. | +| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider parses — never a shape-gated subset. | **Invariant:** for every provider `P` and every identifier `id` minted by `P`, -`P.recognize(id)` is true, `P` produces a stable hash prefix for `id` (and -two identifiers minted from the same client evidence share it), and a -withdrawal request carrying `id` tombstones it. A conformance test suite MUST -assert this round-trip for every shipped provider, and the suite MUST be -written so a future provider crate can run it against its own implementation. +`P.parse` round-trips `id` (including its case variants and equivalent +envelopes, which all canonicalize to the same identifier and graph key); +where `P` declares cluster support, `cluster_prefix(id)` is a literal prefix +of `graph_key(id)` and is shared by identifiers minted from the same client +evidence; and a withdrawal request carrying `id` tombstones it. A conformance +test suite MUST assert this round-trip for every shipped provider — including +case-variant, equivalent-envelope, cross-provider key-namespace, and KV +length/charset cases — and the suite MUST be written so a future provider +crate can run it against its own implementation. Conformance tests inject +deterministic entropy; probabilistic assertions ("two random suffixes +differ") are not accepted. ## 4. Trait surface: minimalism rule @@ -116,7 +121,10 @@ Every trait method MUST have at least one production (non-test) caller in the same PR that introduces it. Speculative surface observed in PR #838 that MUST NOT ship without a caller: -- `keys_equal` (no production caller; existed to serve a unit test), +- `keys_equal` (no production caller; existed to serve a unit test — its + legitimate purpose, #778's equivalent-envelope comparison, is satisfied + structurally by §3's canonicalizing `parse` instead: equivalents + canonicalize to the same identifier, so no comparison method is needed), - `GeneratedEdgeCookie::response_headers` (empty in all built-ins, plumbed through three layers), - `IdentityInput.permissions` / `IdentityInput.consent` (ignored by all @@ -133,15 +141,21 @@ The minimal `EdgeCookieProvider` surface implied by §3 is: pub trait EdgeCookieProvider { /// Stable configuration key ("hmac"). fn id(&self) -> &'static str; - /// Permissions this provider's data use requires. Enforced by core. + /// Permissions this provider's data use requires. Enforced by core for + /// minting and identity use — never for parse/tombstone (§5). fn required_permissions(&self) -> PermissionSet; /// Mint an identifier from request evidence. fn generate(&self, input: &IdentityInput<'_>) -> Result>; - /// Whether `value` is an identifier this provider minted. - fn recognize(&self, value: &str) -> bool; - /// Hash prefix of a recognized identifier (see §3: collides by design - /// across identifiers minted from the same client evidence). - fn hash_prefix(&self, id: &EcId) -> HashPrefix; + /// Parse and canonicalize a cookie value into this provider's + /// identifier; None when unrecognized. Equivalent envelopes and case + /// variants canonicalize to the same identifier. + fn parse(&self, value: &str) -> Option; + /// Canonical KV graph key for a parsed identifier. + fn graph_key(&self, id: &EcId) -> GraphKey; + /// Cluster capability: a literal byte prefix of `graph_key(id)`, shared + /// across identifiers minted from the same client evidence. None when + /// the provider does not support IP-cluster semantics (§3). + fn cluster_prefix(&self, id: &EcId) -> Option; } ``` @@ -150,24 +164,45 @@ trait at step 5 of §11, together with its enforcement point.) ## 5. Permission enforcement is core's job — for EC providers -Before executing an **EC provider**, core resolves the request's permission -set (see the permission model spec) and refuses to run a provider whose -`required_permissions()` are not all set, with a test proving a provider -declaring an unset permission does not execute. +The gate is on **minting and identity use, never on the lifecycle +operations that withdrawal depends on**. Before minting through an EC +provider or using an identity (raw-EC egress, permission model spec §7), +core resolves the request's permission set and refuses when the provider's +`required_permissions()` are not all set. **Parse, canonicalization, graph +lookup for revocation, and tombstoning always run**, permissions or not — a +blanket execution gate would refuse to run the provider in exactly the +state an opt-out produces, making the withdrawal it demands impossible. A +spy-provider test pins the split: with `store-on-device` unset, `generate` +is never called while a withdrawal request still parses the cookie and +writes tombstones. + +The gate applies to EC providers **only**. Geo and device are ungated for +two _different_ reasons, stated separately because only one of them is +structural: + +- **Geo: circularity.** The permission set is resolved _from_ jurisdiction, + which is resolved _by_ the geo provider. Gating geo on the resolved set + is unsatisfiable. +- **Device: a decision, not a circularity.** Device classification is not + an input to permission resolution (the inputs are jurisdiction, policy, + and signals), so ordering geo → resolution → device → EC and gating + device is perfectly implementable. This spec deliberately does not: + the shipped device providers process technical request metadata (UA, + JA4/HTTP-2 fingerprints) for **security classification** — the bot gate + protecting KV-backed identity writes — which must run precisely for + traffic that has granted nothing. The authorization for that processing + is the operator's explicit `[device] provider` selection, and this spec + records that as the decision, with its privacy implication stated: a + device provider whose data use goes beyond security classification (for + example feeding fingerprints into targeting or identity) is **not + authorized by selection alone** and requires a vocabulary extension plus + a gate before it may ship. -This gate applies to EC providers **only**, and the reason is structural, -not convenience: the permission set is resolved _from_ jurisdiction, which -is resolved _by_ the geo provider — gating geo (or device, which runs in -the same pre-resolution phase) on the resolved set would be circular. PR #838 declared `required_permissions` on all three traits but consulted it only for the EC provider; the geo and device declarations were decorative — worse than absent, because they read as a gate and are not one. This spec resolves that by **not having** the method on those traits -(§4). Geo and device providers are governed by explicit operator selection, -the capability checks of §6, and the permission model's vocabulary rule: if -a future vocabulary adds a purpose covering geolocation or fingerprinting, -gating those providers will require a two-phase resolution design specified -at that time (permission model spec §7). +(§4), with the two rationales above in place of the pretense. ## 6. Selection, validation, and failure modes @@ -187,6 +222,37 @@ Unknown fields inside every provider config block are rejected struct already has it, but PR #838 shipped `EcProviders`, `DeviceConfig`, and `GeoConfig` without it, so a typo like `providr` was silently ignored). +### 6.1 Provider switching: active writer, legacy readers + +Switching `[ec] provider` must not strand the identities the previous +provider minted: with only the selected provider recognizing cookies, an +`hmac` → vendor switch turns every existing cookie into "absent", orphans +its graph row, and — worst — makes a later opt-out unable to tombstone it. +The contract: + +- `[ec] provider` names the **active writer**: the only provider that + mints. +- `[ec] legacy_providers = ["hmac"]` (optional list) names **legacy + readers**: providers consulted, in order, for parse, graph lookup, and + tombstoning when the active writer does not recognize a value. Legacy + readers never mint. Each listed key must have its `[ec.providers.]` + block, validated like the active one (§6 table). +- A cookie recognized by a legacy reader is a live identity for + read/withdrawal purposes; whether it is transparently re-minted under the + active writer is a per-deployment choice + (`[ec] rewrite_legacy = true|false`), and re-minting is subject to the + full minting gate of §5. +- Retiring a legacy reader is the explicit end of those identities: + the migration guide documents the cleanup procedure (migration spec §6). +- Tests: switch active provider → request with old cookie → identity still + resolves and a withdrawal tombstones it; old cookie with no matching + legacy reader → treated as absent and **never egresses**. + +Cluster degradation config (referenced from §3): when the active writer +lacks the cluster capability, `[ec] cluster_fallback = "allow" | "deny"` +decides whether KV-backed writes gated on cluster trust proceed; there is +no implicit default — the operator chooses. + ## 7. Composition root and adapter parity Provider construction happens in exactly one place per concern @@ -242,11 +308,14 @@ prominent in release notes: ## 10. Testing strategy -- Provider conformance suite (§3 invariant) run against every shipped - provider. -- EC permission-enforcement tests (§5). +- Provider conformance suite (§3 invariant, deterministic entropy) run + against every shipped provider. +- EC minting-gate tests (§5), including the spy-provider case: permission + unset → `generate` never called, withdrawal still tombstones. +- Legacy-reader tests (§6.1): provider switch → old cookie resolves and + withdraws; unmatched old cookie never egresses. - Settings validation tests for every row of the §6 table, including the - block-without-selector rejection. + block-without-selector rejection and the `legacy_providers` rules. - Parity suite additions of §7. - Unit tests inside each provider crate; crates with no native-target tests still get clippy coverage via the alias wiring of §8. @@ -274,3 +343,15 @@ prominent in release notes: Steps 1–4 are independently reviewable, behavior-preserving, and do not depend on the permission model: the EC gate keeps its current jurisdiction logic until the permission model PR replaces it. + +## 12. Divergences from issue #778 + +This spec supersedes #778 on the following points; the issue is updated to +reference this spec when the PR merges, so implementation has one +acceptance contract: + +| #778 says | This spec says | Why | +| ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| Identifier comparison is a provider operation (`keys_equal`) | Comparison is structural: `parse` canonicalizes, so equivalent envelopes become the same identifier and graph key (§3) | Satisfies the same requirement with no comparison method to leave uncalled | +| A provider can return response headers | Dropped (§4) | Empty in every built-in in PR #838, plumbed through three layers with no consumer; returns with the first feature that needs it | +| One built-in provider (HMAC) preserving today's behavior | Same, plus explicit legacy-reader semantics for later switches (§6.1) | Switching was unspecified in #778 and stranded identities in the #838 shape | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 5e7a327c1..88b2d99a3 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -32,18 +32,19 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Policy decision: the shipped `US` baseline decides. If the baseline grants `store-on-device` without a signal, that is a **declared change** requiring sign-off in the policy review, with rationale in the policy itself | Declared change (if made) | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | -| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | -| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | -| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| # | Decision (today) | After epic | Status | +| --- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Policy decision: the shipped `US` baseline decides. If the baseline grants `store-on-device` without a signal, that is a **declared change** requiring sign-off in the policy review, with rationale in the policy itself | Declared change (if made) | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11 | Raw EC egress (OpenRTB `user.id`, derived request IDs, page bids, proxy/click/Testlight forwarding, identify, pull/batch sync) is gated by the jurisdiction gate today | Gated by the egress inventory (permission spec §7): bidstream egress requires both purposes, first-party identity operations require `store-on-device`, revocation exempt — at least as strict as today for every inventoried path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -109,16 +110,22 @@ Requirements: and `provider = "client-fixed"` are unknown keys and rejected like any other, so a config written against the PR #838 example cannot silently select a provider that no longer exists. -4. **The example config ships the migrated happy path**, uncommented: +4. **Provider switches go through legacy readers.** Changing + `[ec] provider` on a deployment with live identities requires listing + the outgoing provider in `[ec] legacy_providers` (providers spec §6.1) + so existing cookies keep resolving and stay withdrawable; the guide + documents the switch sequence and the retirement/cleanup step that ends + it. +5. **The example config ships the migrated happy path**, uncommented: `provider = "hmac"` with its block, `[geo] default_country`, and (for Fastly) the behavior-preserving `[device] provider = "fastly"` and `[geo] provider = "platform"` lines present with a comment stating what removing them changes. PR #838's example shipped the passphrase block uncommented with the selector commented out — steering operators directly into the silent-stateless state. -5. Every misconfiguration in the providers spec §6 table fails at +6. Every misconfiguration in the providers spec §6 table fails at **startup**. Request-time failure for a configuration error is a defect. -6. Config-store payload validation (`ts config push`) applies the same +7. Config-store payload validation (`ts config push`) applies the same rules — including `[permissions]` policy validation (permission spec §3.3) — so a bad config is rejected at push time, before any instance restarts into it. @@ -129,6 +136,10 @@ The migration guide (a new `docs/guide/` page, linked from the release notes) gives one copy-pasteable recipe per adapter for "keep exactly today's behavior": +The recipe is the **complete recommended policy table from +`trusted-server.example.toml`** — the full GDPR/UK/US jurisdiction rules, +copied, not referenced by omission — plus this delta: + ```toml [ec] provider = "hmac" @@ -140,22 +151,38 @@ provider = "fastly" # Fastly deployments: preserves the JA4 bot gate [geo] provider = "platform" # preserves per-request jurisdiction detection -default_country = "FR" # used only when the host lookup fails (fail-closed) +default_country = "FR" # used only when the host lookup fails (fail-closed + # because FR resolves to the gdpr-eu rule below) + +# ... the full example policy table goes here: gdpr-eu / gdpr-uk / +# us-opt-out groups and their country rules, verbatim ... -# Preserves today's treatment of countries outside the regulation lists -# ("non-regulated" → identity allowed). Omit this section to adopt the -# protective default instead: signal required worldwide (§2 row 7). +# Delta vs. the protective example: preserve today's treatment of countries +# outside the regulation lists ("non-regulated" → identity allowed). Keep +# the example's `default = "gdpr-eu"` instead to require a signal worldwide +# (§2 row 7). [permissions.groups.non-regulated] +regime = "none" default = "granted" [permissions.rules] default = "non-regulated" ``` -and separately documents the neutral configuration and what it does _not_ do. -The guide states explicitly that `default_country` alone does not replace geo -lookup, why the no-geo combination requires the explicit acknowledgment flag -(permission spec §5.3), and that no recipe preserves row 8 of §2 — the +A partial policy is a trap the first draft of this spec fell into: a +`[permissions]` section containing **only** the permissive +default — with no GDPR/US rules — sends _every_ jurisdiction, France +included, to the permissive fallback, because `default_country` selects a +rule like any other country and finds none. The recipe therefore always +carries the full table, and CI pins it: **the exact documented recipe text +is a fixture**, loaded and run through the complete §4.1/§4.2 decision +matrix of the permission spec, asserting per-jurisdiction outcomes match +the pre-epic gate for every preservation row of §2. + +The guide separately documents the neutral configuration and what it does +_not_ do, states explicitly that `default_country` alone does not replace +geo lookup, why the no-geo combination requires the explicit acknowledgment +flag (permission spec §5.3), and that no recipe preserves row 8 of §2 — the global honoring of opt-out signals is unconditional. ## 6. Rollout sequence and observability @@ -174,10 +201,14 @@ global honoring of opt-out signals is unconditional. 4. Rollback is config-only where possible: reverting to the previous config version restores the previous behavior on the previous binary. The one irreversible artifact is withdrawal tombstones — which is why the - withdrawal triggers (permission spec §4.2) are exhaustive and why §2 - rows 6 and 8 call out tombstoning explicitly. Cleanup of identities - minted before a policy tightening (permission spec §4.2 trigger 3) is an - operational action documented in the guide, not an automatic one. + withdrawal triggers (permission spec §4.2) are exhaustive, why partial + withdrawal failure has an explicit tombstones-first, browser-retries + contract (permission spec §4.3), and why §2 rows 6 and 8 call out + tombstoning explicitly. Two operational procedures are documented in the + guide, not automated: cleanup of identities minted before a policy + tightening (permission spec §4.2 trigger 3), and retirement of a legacy + reader after a provider switch (providers spec §6.1), which is the + deliberate end of the identities only that reader can resolve. ## 7. Documentation deliverables From 5c8c2e8930391ac0c834fb69c545bdf4de595a1d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 07:36:31 -0700 Subject: [PATCH 04/14] Address second review: close permission-algebra, egress, durability, and rollout gaps Blocking findings from the second review of PR #986: - Signal taxonomy gains a grant-signal class (TCF consent, explicit GPP non-opt-out, US Privacy present-and-not-opted-out including N/A) so a requires_signal US rule reproduces today's no-signal-blocks / explicit-non-opt-out-grants behavior, which the two-class model could not express; migration matrix gains rows 3a-3c and the example US group changes to requires_signal. - Auction dispatch gets a normative regime matrix (gdpr / us-privacy / none across consent, opt-out, malformed, expired, absent states), the compiled fallback gains regime = gdpr, and blocked dispatch means no outbound request at all. - The normalization matrix now states outcomes instead of subjects: restrictive/permissive synthesize per purpose, newest selects whole records, expired records are absent entirely, valid-beats-malformed within a family, KV fallback is live-wins with TTL-bounded staleness and an exempt consent-state lookup, mirror mode loses to request records, and GPP fields are enumerated per section. - The egress inventory is a concrete path -> permission table: proxy / click / Testlight forwarding assigned (both purposes, declared as new hardening in split row 11a/11b since those paths are ungated today), identify and pull/batch sync classified as partner exchange (both purposes), and S2S sync authorized by stored provider/version-tagged provenance re-validated against current policy. - Withdrawal drops the false atomicity claim: revocation families are idempotent independent writes, readers fail closed on any present member, and fault-injection covers the Nth-write failure. - Equivalence is provider-declared via fixtures (hmac: hex prefix case-insensitive, suffix case-preserved) instead of a universal case rule; the legacy HMAC grammar is formally reserved as the hmac namespace so verbatim row keys and provider namespacing coexist; global cookie-safe identifier bounds added; non-cluster providers get defined dedupe and redaction. - Legacy readers get full semantics: first-match parse with namespace-overlap validation, recognizing provider's permissions govern, provider/version-tagged provenance, transactional linking rewrite with dual revocation, and provider = "none" as an explicit stateless state that keeps revoke-only legacy readers. - Graph store required at startup when any provider can mint or read; a minted identity is not active until its row commits; a runtime failure matrix covers provider, graph, cluster, rewrite, and geo/device runtime failures. - Rollout gains a dual-read release (N+1 accepts both config shapes, N+2 rejects loudly) since no config is accepted by both current main and a rejecting binary; the preserving recipe becomes one committed valid TOML fixture (the prose delta reopened [permissions.rules], which is invalid TOML); metrics extended with retirement thresholds. - Resolve endpoint defines same-identity no-op / different-identity rejection and an atomic single-key reservation tying replay consumption to graph persistence. - Hook ordering becomes core -> integrations -> inviolable cache/privacy invariant pass (an appended cookie plus replaced public Cache-Control can no longer produce a shared-cacheable cookie response); generic ops reject Set-Cookie; per-integration operation limits and erroring-mutator semantics defined; every eligibility row tested. - Stale device-circularity wording removed from the permission spec; region-form default_country (US/CA) restored; the source-agnostic permission-source requirement of #777/#779 explicitly deferred in the divergence table; fail-closed and most-protective labels qualified with their stated exceptions. --- ...26-07-30-client-cycle-ec-resolve-design.md | 19 ++ ...integration-response-header-hook-design.md | 38 ++- .../2026-07-30-permission-model-design.md | 255 +++++++++++------- .../2026-07-30-pluggable-providers-design.md | 134 +++++++-- ...07-30-provider-migration-rollout-design.md | 123 +++++---- 5 files changed, 383 insertions(+), 186 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index e0a5529fe..88b1fa66b 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -93,6 +93,25 @@ Everything in this spec follows from that. the resulting identifier that keeps it cookie-safe and within the KV limits of the providers spec §3. Tests exercise the exact 413 boundary and the missing/false/chunked-length cases. +8. **Define behavior against an existing identity — no silent + replacement.** When the request already carries a recognized EC: + resolving to the **same** identity is an idempotent no-op (cookie + refreshed, same response); resolving to a **different** identity is + **rejected** — replacing a live identity via an unauthenticated POST is + identity takeover, and any legitimate re-identification flow (account + link, vendor migration) is an explicit linking design this spec does + not authorize (own open question, §7). +9. **Make replay consumption and graph persistence one idempotent + sequence.** Neither naive order works: consume-the-nonce-first makes a + subsequent graph failure unretryable (the token is spent, the identity + never existed); graph-first lets the losers of a replay race leave + residual rows. Required shape: consumption is an **atomic single-key + reservation** keyed by the payload's unique id, recording outcome; the + graph write happens under that reservation and is retried under the + same key; duplicate or racing requests observe the reservation and + receive the original outcome — no second identity, no spent-but-unused + token, no orphan row. Tests cover crash-between-steps and two + concurrent requests with the same payload. ## 4. Requirements on the page script diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index b91846222..1c6251288 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -42,13 +42,20 @@ mutators to the outbound response for HTML document responses it processed. processed documents. The call site lives in shared response-finalization code where one exists; where adapters finalize independently, each adapter gains the call and a test proving it. -- Mutators run **after** Trusted Server's own response-header handling - (EC Set-Cookie emission, EC header clearing, privacy headers) so a - mutation cannot be silently stripped by a later core pass. The ordering is - a fresh decision this spec makes — PR #838 never wired the hook, so there - is no existing insertion point to inherit; the implementer places the call - at the end of each adapter's response finalization, and the §4.3 tests pin - it there. +- **Ordering is three stages, and the last one is inviolable:** core + response-header handling (EC Set-Cookie emission, EC header clearing, + privacy headers) → integration operations → **final cache/privacy + invariant enforcement**, which no integration operation can override. + Running the hook dead-last would be wrong: current `main` deliberately + runs cookie-cache protection _after_ arbitrary header changes, stripping + surrogate caching and forcing private/no-store on any response that sets + a cookie — a hook applied after that recheck could combine an appended + `Set-Cookie` with a replaced public `Cache-Control` into a + **shared-cacheable cookie response**. The invariant pass therefore runs + after all mutations, unconditionally. Middle-stage placement also keeps + the earlier property: an integration mutation is not silently stripped + by ordinary core handling — only by the invariant pass, which logs the + downgrade it applies. ## 3. Collision policy @@ -70,6 +77,14 @@ mutators to the outbound response for HTML document responses it processed. header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). +- **Operation-layer hygiene:** generic `append`/`replace` reject the + `Set-Cookie` header name outright — cookies go only through + `append_set_cookie`, so its validation cannot be bypassed by spelling + the header name in a generic op. Per-integration limits bound total + operations, added header count, and added header bytes; exceeding a + limit rejects the excess operations (logged, attributed), never the + response. A mutator that panics or errors is skipped in full — its + operations are all-or-nothing — and the response proceeds without it. ## 3a. Response eligibility — normative @@ -99,7 +114,14 @@ processed documents (§6). 3. Every adapter applies mutations on its outbound path, with a per-adapter route test asserting an integration-set header appears in the response. 4. A parity-suite case asserts identical mutation behavior across adapters. -5. Reserved-header and append/replace semantics covered by unit tests. +5. Reserved-surface, append/replace, operation-limit, and erroring-mutator + semantics covered by unit tests. +6. **Every row of the §3a eligibility matrix has a test** — streaming, + cache-hit, pass-through, redirect, error, and 304 each proven to run or + not run the hook — not merely one positive header test per adapter. +7. The cache/privacy invariant test: an integration appends a cookie and + replaces `Cache-Control` with a public/surrogate-cacheable value → the + final response is private/no-store with surrogate caching stripped. ## 5. Size and sequencing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index a66037a7e..820426ab5 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -37,6 +37,12 @@ The set is resolved from three inputs: 3. **Signals** — the request's privacy signals: TCF, GPP, GPC, US Privacy (§4). +These are the initial sources. #777/#779 also envision publisher +interaction and external services as permission sources; that +source-interface is **explicitly deferred**, not silently dropped — §10 +records the divergence, and adding a source later means adding a grant- or +opt-out-class input to §4's taxonomy, not a new resolution algorithm. + Scope: the model governs decisions Trusted Server makes. Downstream RTB partners receive the full, unmodified regulatory context and make their own compliance decisions. @@ -56,10 +62,10 @@ compatibility. The initial vocabulary is therefore exactly: -| Identifier | TCF purpose | Enforcement points | -| ------------------------- | ----------- | -------------------------------------------------------------------- | -| `store-on-device` | 1 | EC provider execution; EC creation; withdrawal/tombstone eligibility | -| `select-personalised-ads` | 4 | EID transmission into the bidstream (jointly with `store-on-device`) | +| Identifier | TCF purpose | Enforcement points | +| ------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `store-on-device` | 1 | EC provider execution; EC creation; withdrawal/tombstone eligibility | +| `select-personalised-ads` | 4 | All bidstream and partner identity egress — raw EC in `user.id`, derived request IDs, page bids, EIDs, identify, pull/batch sync (jointly with `store-on-device`; the full path table is §7) | (The identifier strings are the IAB names verbatim, including their original spelling.) The extension procedure — add the signal mapping, add the @@ -97,8 +103,12 @@ git for the file, config-store versions for pushes — is the change log. **Compiled-in fallback:** when a config has no `[permissions]` section, a minimal compiled-in policy applies in which **every permission is -`requires_signal` for every jurisdiction** — the most protective posture. -Absence of policy is always safe; there is no fail-open default. +`requires_signal` for every jurisdiction**, with **`regime = "gdpr"`** so +auction dispatch (§7) is defined and maximally protective too. (This is +the most protective posture that still admits consent — `denied` would be +stricter but would make a signal-carrying deployment inoperable by +default; the distinction is stated, not glossed.) Absence of policy is +always safe; there is no fail-open default. ### 3.2 Format @@ -115,9 +125,13 @@ overrides. Each permission resolves to an **acquisition rule**: regime = "gdpr" default = "requires_signal" +# Opt-out regime, expressed as requires_signal: explicit non-opt-out +# values are grant-class signals (§4), so signal-carrying traffic is +# granted while no-signal traffic stays blocked — matching today's US +# behavior, which `granted` cannot express. [permissions.groups.us-opt-out] regime = "us-privacy" -default = "granted" +default = "requires_signal" [permissions.groups.non-regulated] regime = "none" @@ -175,7 +189,10 @@ Validation rejects: - an empty `[permissions]` section (ambiguous intent: an operator who wants the compiled-in fallback omits the section entirely); - duplicate rule keys under case-insensitive comparison (`FR` and `fr`); -- a `[geo] default_country` that is not an assigned ISO code; it is +- a `[geo] default_country` whose country part is not an assigned ISO + code; it accepts either a country (`FR`) or a country/region key + (`US/CA`) — PR #838 supported region defaults, and a no-geo, + single-state deployment must be able to select its state rule. It is canonicalized to uppercase, and startup logs which rule (or `rules.default`) it resolves to. @@ -210,7 +227,9 @@ inline, and ships it as the most protective baseline. ## 4. Signal precedence — normative -Signals are classified: +Signals are classified into three classes — a two-class model (TCF grant / +opt-out) cannot reproduce today's US behavior, where no-signal traffic is +blocked but an **explicit non-opt-out** value grants: - **Opt-out signals** (affirmative withdrawal): GPC header; GPP sections carrying a sale/sharing opt-out; US Privacy opt-out. Opt-out signals are @@ -220,8 +239,17 @@ Signals are classified: and ignore it for others based on IP evidence. (For jurisdictions outside US states this is a declared behavior change; migration spec §2 records it.) -- **Consent records**: a decodable TCF string (standalone or embedded in - GPP), which may grant or refuse individual purposes. +- **Grant signals** (affirmative permission): a decodable TCF record + consenting to the purpose; an **explicit GPP non-opt-out value** (e.g. + `sale_opt_out = false`); a **US Privacy string present and not opting + out** — including the "not applicable" flag, which today's tests pin as + allowing. Any grant signal satisfies a `requires_signal` baseline; this + is what lets a `requires_signal` US rule preserve today's "no signal → + block, explicit non-opt-out → allow" behavior, which neither `granted` + nor a TCF-only grant class could express. +- **Refusals**: a decodable TCF record refusing the purpose. A refusal is + neither a grant nor an opt-out — it blocks acquisition (precedence 3) + and withdraws only per §4.2. **Precedence, highest first:** @@ -243,8 +271,11 @@ Signals are classified: jurisdictions — the migration spec's matrix (row 6) records it. Refusal revokes new grants only; whether it also destroys existing identity is governed strictly by §4.2. -4. Consent record grant — a TCF record present and consenting grants it - (subject to 1–2). +4. Grant signal — any grant-class signal (TCF consent, explicit GPP + non-opt-out, present-and-not-opted-out US Privacy) grants the permission + (subject to 1–3: a coexisting TCF refusal beats a non-TCF grant signal, + matching today's US-state ordering where a present TCF record decides + before GPP/USP values are consulted). 5. No signal — the policy baseline decides: `granted` sets it, `requires_signal` leaves it unset. @@ -253,12 +284,12 @@ Signals are classified: For each enforced permission, with baseline _B_ ∈ {granted, requires_signal, denied}: -| Opt-out present | TCF present | TCF consents | Result | -| --------------- | ----------- | ------------ | ------------------------------------------------ | -| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | -| no | yes | no | unset (withdrawal per §4.2, trigger 2) | -| no | yes | yes | set, unless B = denied | -| no | no | — | set iff B = granted | +| Opt-out present | TCF refusal present | Grant signal present | Result | +| --------------- | ------------------- | -------------------- | ------------------------------------------------ | +| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | — | unset (withdrawal per §4.2, trigger 2) | +| no | no | yes | set, unless B = denied | +| no | no | no | set iff B = granted | ### 4.2 Withdrawal vs. absence @@ -313,38 +344,45 @@ live graph identity with no browser handle pointing at it): browser setting, the TCF record lives in the CMP's storage), so the next request re-presents the signal and retries the whole withdrawal. No quarantine queue is needed; the browser is the retry queue. -- Identify, batch-sync, and pull-sync treat a tombstone as authoritative - revocation (as today); a row whose withdrawal is pending retry is simply - still live until the retry lands, and never partially withdrawn. -- Fault-injection tests cover: tombstone write fails → cookie untouched, - error logged; subsequent request with the same signal → withdrawal +- **Partial progress is explicit, not atomic.** The tombstone writes are a + **revocation family**: an enumerable, ordered set of independent KV + writes (cookie hash, active-EC hashes), each idempotent, with no + multi-key atomicity assumed — the current storage offers none, and the + spec does not pretend otherwise. A retry resumes the family from the + start (idempotent writes make re-writing completed members harmless). + Withdrawal is **complete** only when every family member is committed; + the cookie expires only then. +- **Reads fail closed on partial families**: every consumer (identify, + batch-sync, pull-sync, egress gates) treats an identity as revoked when + **any** member of its revocation family is present — a partially + withdrawn identity is unusable immediately, even before the family completes. +- Fault-injection tests cover failure at the **Nth** family write (not + only total failure): first write lands, second fails → cookie untouched, + identity already treated as revoked by readers, error logged; subsequent + request with the same signal → family completes and the cookie expires. -### 4.4 Signal normalization +### 4.4 Signal normalization — normative matrix §4's precedence operates on normalized inputs: one effective consent record and one effective opt-out state per request. The normalization layer is where today's real-world mess lives, and PR #838 collapsed it -silently. The implementation ships a **normalization matrix** — a -table-driven spec-and-test artifact — covering at minimum: - -- **Dual consent records**: standalone TCF cookie vs. GPP-embedded TCF, - including per-purpose disagreement, resolved per the existing configured - conflict modes (restrictive / permissive / newest). Each mode is either - preserved or explicitly retired in the migration matrix — not dropped. -- **Record expiry** and the persisted-KV consent fallback: when a stored - record substitutes for an absent live one, and how staleness is bounded. -- **Proxy/mirror mode** (CMP consent mirrored server-side): where the - mirrored state enters precedence. -- **Exact GPP fields**: which section fields constitute a sale/sharing/ - targeted-advertising opt-out, enumerated per supported section — "GPP - opt-out" is not a single bit. -- **Malformed-but-present records fail closed for acquisition**: a consent - record that is present but undecodable blocks grants (it does not - degrade to "absent", which under a `granted` baseline would turn garbage - into a grant — the fail-open path in both #838 and the first draft of - this spec). A malformed record never triggers withdrawal — destruction - requires an affirmative, decodable signal (§4.2). +silently. These are the outcomes — decided here, not delegated to the +implementation; each row marked **changed** also appears in the migration +matrix: + +| Input state | Effective record / outcome | Status | +| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | +| Standalone TCF and GPP-embedded TCF disagree per purpose, mode `restrictive` | Per-purpose **synthesis**: a purpose is consented only when **both** records consent (AND) | Preserved (mode semantics pinned against current tests) | +| Same, mode `permissive` | Per-purpose synthesis: consented when **either** record consents (OR) | Preserved (same pinning) | +| Same, mode `newest` | **Whole-record selection** by `Created` timestamp; tie → the GPP-embedded record | Preserved (same pinning) | +| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | +| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | +| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | +| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | Preserved, circularity resolved | +| Proxy/mirror mode (CMP state mirrored server-side) | Mirror-sourced record enters as a live record; when both the mirror and the request carry records, the **request's record wins** (closer to the user) | Decided here | +| GPP opt-out fields | Enumerated per supported section: the sale, sharing, and targeted-advertising opt-out fields each independently constitute an opt-out signal when set; **all present-and-false** constitutes a grant signal (§4); absent/N-A fields contribute nothing. The exact field list per section ID is an appendix of the implementation PR, reviewed against the GPP spec | Decided here | +| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | ## 5. Jurisdiction resolution @@ -415,63 +453,88 @@ migration story unresolvable (migration spec §2, rows 5 and 7). | No geo provider configured | `default_country` baseline, guarded by §5.3 | | Country resolved, no matching rule | Policy `rules.default` | | Region resolved, no region rule | Country rule | -| No `[permissions]` section | Compiled-in fallback: everything `requires_signal` | +| No `[permissions]` section | Compiled-in fallback: everything `requires_signal`, `regime = "gdpr"` | +| S2S sync request (no user signals) | Authorized by stored provenance re-validated against current policy (§7) | | Malformed policy | Rejected at config push / startup (§3.3) — never per request | | No `default_country` | Startup failure | | Undecodable TCF/GPP record (present but malformed) | Blocks grants (fail-closed acquisition, §4.4); never withdraws; opt-out signals still honored | | Signals contradict (opt-out + consent) | Opt-out wins (§4) | -The overall posture is **fail-closed**: every ambiguous state resolves to -the configured baseline or more restrictive, and the one configuration that -turns "no information" into a static jurisdiction assertion (§5.3) requires -an explicit operator acknowledgment to exist. +The intended posture is fail-closed, with its two exceptions stated rather +than glossed: geo lookup failure resolves to the configured default (§5.2's +declared, metered residual — permissive defaults make this path fail-open), +and the §5.3 static-jurisdiction configuration exists only behind an +explicit operator acknowledgment. Every other ambiguous state resolves to +the configured baseline or more restrictive. ## 7. Enforcement points Consumers of the resolved set in this epic: 1. **EC provider execution** (providers spec §5) — the provider's declared - `required_permissions()` must all be set. This gate applies to EC - providers only: geo and device providers execute **before** permission - resolution as its inputs, so gating them on its output would be - circular. Their governance is explicit selection, the capability checks - of providers spec §6, and §2's vocabulary rule — if a future vocabulary - adds a purpose covering geolocation or fingerprinting, gating those - providers will require a two-phase resolution that must be specified - then, not improvised. + `required_permissions()` must all be set for minting and identity use. + This gate applies to EC providers only. **Geo** is ungated because + gating it is circular — jurisdiction is an input to permission + resolution. **Device** is ungated by a different, deliberate decision + (it is _not_ a resolution input): its security-classification role must + run for traffic that has granted nothing, and operator selection is the + recorded authorization — providers spec §5 states the decision and its + boundary. If a future vocabulary adds a purpose covering geolocation or + fingerprinting, gating those providers will require a two-phase + resolution specified then, not improvised. 2. **EC lifecycle** — creation requires `store-on-device`; withdrawal per §4.2. Recognition, canonicalization, and revocation of an existing identifier are **never** permission-gated — they must run precisely when permissions are withdrawn (providers spec §5). -3. **Every raw-EC egress**, not only EIDs. The raw EC identifier leaves the - process today as OpenRTB `user.id`, inside derived auction request IDs, - on the page-bids path, on proxied/click/Testlight forwarding, through the - identify endpoint, through pull/batch sync, and via identity-graph - reads/writes. Gating EIDs alone (PR #838's shape) leaves the raw EC - reaching bidders when Purpose 1 is granted and Purpose 4 refused; worse, - #838's `ec_allowed` was **vacuously true with no provider configured** - (`is_none_or`), so an existing canonical cookie still escaped in - stateless mode. The contract: - - The implementation maintains an **egress inventory**: every code path - where a raw EC (or a value derived from it) leaves the process is - enumerated in one table, each mapped to its required permissions, with - a test per row. - - **Bidstream egress** (`user.id`, EC-derived request IDs, page bids, - EIDs) requires `store-on-device` ∧ `select-personalised-ads` — the raw - EC is identity in the bidstream and is gated exactly as EIDs are. - - **First-party identity operations** (identify, pull/batch sync, - graph reads/writes other than revocation) require `store-on-device`. - - **Revocation paths are exempt** — tombstoning must work when - permissions are unset. - - With **no EC provider configured**, identity use fails closed: a - cookie value present on the request never egresses anywhere; it is - never vacuously allowed. -4. **Bidstream EIDs** — transmission requires `store-on-device` ∧ - `select-personalised-ads` (subsumed by point 3's inventory; listed - separately because it is the one gate PR #838 had). -5. **Server-side auction dispatch** — gated on the explicit policy `regime` - class (§3.4), a first-class enforcement point, not inferred from purpose - flags. +3. **Every raw-EC egress and identity operation** — the concrete + inventory, normative per path (one test per row; a denylist check + proves no ungated egress exists): + + | Path | Required permissions | Notes | + | ---------------------------------------------------------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | + | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | + | EC-derived auction request IDs | both purposes | Derived values are identity | + | Page-bids path | both purposes | | + | Bidstream EIDs | both purposes | The one gate PR #838 had | + | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | + | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | + | Pull sync / batch sync (partner identity exchange) | both purposes | Authority source for S2S requests: stored provenance, below | + | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | + | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | + | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | + + With **no EC provider configured**, identity use fails closed: a cookie + value present on the request never egresses anywhere — never vacuously + allowed (#838's `ec_allowed` was `is_none_or`, vacuously true with no + provider). + + **S2S authority (batch/pull sync).** A server-to-server request carries + no user signals, geo, or `EcContext` to resolve permissions from. Its + authority is the identity's **stored provenance**: a record written at + mint/update time carrying the resolved jurisdiction, regime, grant + basis, and provider/version (providers spec §6.1). A sync request + re-validates that provenance against the **current** policy revision: + if the stored jurisdiction now resolves to `denied` for the required + permission, the row is not updated and is flagged for the operational + cleanup of §4.2 trigger 3. Sync never mints authority of its own. + +4. **Server-side auction dispatch** — gated on the policy `regime` class, + normatively: + + | Regime | Dispatch rule | Preserves | + | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | + | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | + | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | + | `none` | Dispatch proceeds. | Today's non-regulated arm | + + The **compiled-in fallback policy has `regime = "gdpr"`** (§3.1) — the + no-policy posture must be the most protective for dispatch too, and a + regime-less fallback would leave dispatch undefined. When dispatch is + blocked, nothing leaves for that request: no PBS/APS call, no UA/IP/geo + forwarding to bidders. When dispatch proceeds, what the request may + carry is governed row-by-row by the egress inventory; the full + regulatory context (consent strings) is always forwarded so downstream + partners make their own decisions (§1). The client-cycle resolve endpoint (own spec, currently on hold) would be a further consumer if and when it proceeds. @@ -488,6 +551,13 @@ further consumer if and when it proceeds. configured conflict mode and the malformed-record rows. - The §7 raw-EC egress inventory: one test per inventoried egress proving the gate, plus a denylist-style check that no ungated egress exists. +- The §7 auction-dispatch matrix: every regime × signal state + (consent, opt-out, malformed, expired, absent), including the + no-policy fallback regime, asserting both the dispatch decision and + that a blocked dispatch emits no outbound request. +- The §7 S2S authority path: sync against stored provenance, including + the policy-tightened-to-denied case (no update, flagged for cleanup) + and the exempt consent-state lookup. - §4.3 fault-injection cases. - Policy validation tests for every §3.3 rejection, exercised through both acceptance paths (push-time and startup). @@ -513,8 +583,9 @@ This spec supersedes #779 on the following points; the issue is updated to reference this spec when the PR merges, so there is one acceptance contract, not two: -| #779 says | This spec says | Why | -| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- | -| Unmatched countries fall to `default_country` | Unmatched-but-resolved countries fall to the policy's `rules.default`; `default_country` covers only unresolved requests | The two states had different pre-epic behavior; collapsing them made migration unresolvable (§5.4) | -| The full TCF purpose vocabulary is modeled | Only enforced purposes appear (§2) | Nine inert purposes in a policy file are a compliance hazard, not forward compatibility | -| Policy is an embedded file | Policy is `[permissions]` in `trusted-server.toml` (§3.1) | Runtime config-store pipeline; validation at push time | +| #779 says | This spec says | Why | +| -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Unmatched-but-resolved countries fall to the policy's `rules.default`; `default_country` covers only unresolved requests | The two states had different pre-epic behavior; collapsing them made migration unresolvable (§5.4) | +| The full TCF purpose vocabulary is modeled | Only enforced purposes appear (§2) | Nine inert purposes in a policy file are a compliance hazard, not forward compatibility | +| Policy is an embedded file | Policy is `[permissions]` in `trusted-server.toml` (§3.1) | Runtime config-store pipeline; validation at push time | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signal taxonomy; a pluggable source interface is **deferred** (§1) | Shipping an interface with no second source repeats the inert-surface mistake; the extension path (a new §4 signal class) is defined instead | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 88035ea91..c4dc9cc37 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -94,26 +94,49 @@ An `EdgeCookieProvider` owns the **complete lifecycle** of the identifiers it mints. Every lifecycle operation core performs on an EC value MUST be routed through the selected provider: -| Lifecycle operation | Where core uses it today | Contract | -| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | -| **Parse / canonicalize** | Reading `ts-ec` back from the request; deciding `ec_was_present`; batch-sync ingestion | Provider parses a cookie value into its **canonical** identifier, or rejects it. Canonicalization is provider-owned: case variants and equivalent envelopes of the same identity (per #778) parse to the same canonical identifier. A value the selected provider does not recognize is treated as absent (but see §6.1 legacy readers). | -| **Canonical graph key** | KV identity-graph row reads/writes | The provider maps a canonical identifier to its graph key: stable, KV-safe (within KV length and character-set limits), collision-free across the provider's identifier space, and namespaced so two providers' key spaces cannot collide. Two equivalent envelopes of one identity map to one key — verbatim cookie bytes as the key would fork graph rows on canonicalization differences and discard today's batch-sync canonicalization. | -| **Cluster prefix** (optional capability) | IP-cluster sizing (`cluster_trust_threshold`, implemented as a **KV prefix listing**), pull-sync dedupe, log redaction | A provider declaring cluster support returns a prefix that is a **literal byte prefix of the canonical graph key** — the cluster count lists keys by prefix, so an independently derived hash that is not an actual key prefix silently reports the wrong cluster size. The prefix deliberately collides across identifiers minted from the same client evidence. A provider without the capability declares so, and cluster-dependent gating follows a configured degradation policy (treat cluster size as unknown, with the KV-write decision that implies made explicit in config) instead of counting garbage. | -| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider parses — never a shape-gated subset. | +| Lifecycle operation | Where core uses it today | Contract | +| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | +| **Parse / canonicalize** | Reading `ts-ec` back from the request; deciding `ec_was_present`; batch-sync ingestion | Provider parses a cookie value into its **canonical** identifier, or rejects it. Canonicalization and **equivalence are provider-declared, never imposed globally**: each provider ships equivalence fixtures naming exactly which variants are the same identity — case sensitivity is provider-specific (signed/base64-style envelopes are case-sensitive; even the built-in HMAC id is case-insensitive only in its hex prefix, with a case-preserved suffix). Declared-equivalent values parse to the same canonical identifier (satisfying #778). A value the selected provider does not recognize is treated as absent (but see §6.1 legacy readers). | +| **Canonical graph key** | KV identity-graph row reads/writes | The provider maps a canonical identifier to its graph key: stable, KV-safe (within KV length and character-set limits), collision-free across the provider's identifier space, and namespaced so two providers' key spaces cannot collide. Two equivalent envelopes of one identity map to one key — verbatim cookie bytes as the key would fork graph rows on canonicalization differences and discard today's batch-sync canonicalization. | +| **Cluster prefix** (optional capability) | IP-cluster sizing (`cluster_trust_threshold`, implemented as a **KV prefix listing**), pull-sync dedupe, log redaction | A provider declaring cluster support returns a prefix that is a **literal byte prefix of the canonical graph key** — the cluster count lists keys by prefix, so an independently derived hash that is not an actual key prefix silently reports the wrong cluster size. The prefix deliberately collides across identifiers minted from the same client evidence. A provider without the capability declares so, and cluster-dependent gating follows a configured degradation policy (treat cluster size as unknown, with the KV-write decision that implies made explicit in config) instead of counting garbage. | +| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider parses — never a shape-gated subset. | **Invariant:** for every provider `P` and every identifier `id` minted by `P`, -`P.parse` round-trips `id` (including its case variants and equivalent -envelopes, which all canonicalize to the same identifier and graph key); -where `P` declares cluster support, `cluster_prefix(id)` is a literal prefix -of `graph_key(id)` and is shared by identifiers minted from the same client -evidence; and a withdrawal request carrying `id` tombstones it. A conformance -test suite MUST assert this round-trip for every shipped provider — including -case-variant, equivalent-envelope, cross-provider key-namespace, and KV -length/charset cases — and the suite MUST be written so a future provider -crate can run it against its own implementation. Conformance tests inject -deterministic entropy; probabilistic assertions ("two random suffixes -differ") are not accepted. +`P.parse` round-trips `id` — including every variant `P`'s declared +equivalence fixtures name, all of which canonicalize to the same identifier +and graph key; where `P` declares cluster support, `cluster_prefix(id)` is a +literal prefix of `graph_key(id)` and is shared by identifiers minted from +the same client evidence; and a withdrawal request carrying `id` tombstones +it. A conformance test suite MUST assert this round-trip for every shipped +provider — driven by each provider's equivalence fixtures, plus +cross-provider key-namespace and KV length/charset cases — and the suite +MUST be written so a future provider crate can run it against its own +implementation. Conformance tests inject deterministic entropy; +probabilistic assertions ("two random suffixes differ") are not accepted. + +Three global rules sit above every provider: + +- **Identifier bounds.** A minted identifier obeys a global cookie-safe + alphabet (valid cookie-octets: no separators, whitespace, or control + characters) and a global maximum length — for the identifier itself, not + only the graph key — enforced by core at mint and at parse, so no + provider can emit a value the cookie layer or logs cannot carry. +- **Namespace reservation.** The legacy HMAC grammar `{64hex}.{6alnum}` is + formally **reserved as the `hmac` provider's namespace**. `hmac`'s graph + key is the identifier verbatim and its cluster prefix is the 64-hex + prefix, so every pre-epic row stays reachable and every prefix listing + intact (migration spec §3) — and **no other provider may mint + identifiers or produce graph keys matching that grammar**, which is what + makes verbatim-compatibility and provider-namespacing coexist. + Conformance fixtures include an existing pre-epic row (reachability) and + a prefix-listing case. For `hmac`, the equivalence fixtures pin: + uppercase/lowercase hex-prefix variants are equivalent; suffix case is + preserved and significant. +- **No-cluster behavior is still defined.** A provider without cluster + support deduplicates pull-sync by canonical graph key and redacts logs + with a fixed-length hash of the graph key; `cluster_fallback` (§6.1) + governs only the trust/write decision, not these. ## 4. Trait surface: minimalism rule @@ -147,8 +170,8 @@ pub trait EdgeCookieProvider { /// Mint an identifier from request evidence. fn generate(&self, input: &IdentityInput<'_>) -> Result>; /// Parse and canonicalize a cookie value into this provider's - /// identifier; None when unrecognized. Equivalent envelopes and case - /// variants canonicalize to the same identifier. + /// identifier; None when unrecognized. Values the provider's declared + /// equivalence fixtures name as equivalent canonicalize identically. fn parse(&self, value: &str) -> Option; /// Canonical KV graph key for a parsed identifier. fn graph_key(&self, id: &EcId) -> GraphKey; @@ -176,6 +199,15 @@ spy-provider test pins the split: with `store-on-device` unset, `generate` is never called while a withdrawal request still parses the cookie and writes tombstones. +**A generated identity is not active until its graph row commits.** No +cookie write, no egress, no auction use may observe a minted identifier +before its graph row (with provenance, §6.1) has committed — PR #838 let a +generated EC reach an auction before finalization refused the cookie, +producing an identity that existed for one request and nowhere else. The +normative order is: gate → `generate` → graph-row commit → cookie write → +eligible for egress. A graph-commit failure means the mint never happened: +no cookie, no egress, error logged, the next request retries. + The gate applies to EC providers **only**. Geo and device are ungated for two _different_ reasons, stated separately because only one of them is structural: @@ -209,13 +241,15 @@ one. This spec resolves that by **not having** the method on those traits All validation happens at **settings construction** — a misconfiguration is a startup error, never a request-time error and never a silent behavior change. -| Configuration state | Behavior | -| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `provider` names an unknown key | Startup error listing valid keys. | -| `provider` set, its `[ec.providers.]` block missing | Startup error. | -| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | -| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | -| No `provider`, no providers block | Valid: the neutral default for that concern. | +| Configuration state | Behavior | +| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `provider` names an unknown key | Startup error listing valid keys. | +| `provider` set, its `[ec.providers.]` block missing | Startup error. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | +| No `provider`, no providers block | Valid: the neutral default for that concern. | +| `provider = "none"` (explicit stateless) | Valid, and the only way to combine statelessness with `legacy_providers`: minting stops, legacy readers keep existing identities resolvable and **withdrawable** (§6.1). Without this state, `hmac` → stateless would strand every live row in revoke-proof limbo. | +| A minting provider (or any `legacy_providers`) configured, but no identity-graph store configured or openable | **Startup error.** The lifecycle contract assumes graph persistence (§5); discovering its absence at first mint would be a request-time config failure, which this table exists to forbid. | Unknown fields inside every provider config block are rejected (`deny_unknown_fields` on all new settings structs — the pre-existing `Ec` @@ -237,22 +271,64 @@ The contract: tombstoning when the active writer does not recognize a value. Legacy readers never mint. Each listed key must have its `[ec.providers.]` block, validated like the active one (§6 table). +- **Parse order and ambiguity.** The active writer parses first; the first + match wins. Overlapping recognition is not resolved at request time but + **forbidden at startup**: provider namespaces (§3) may not overlap, and + configuring an active/legacy pair whose grammars intersect is a + validation error. +- **The recognizing provider governs.** A legacy-owned identity is gated + by the **legacy provider's** `required_permissions()` for identity use — + the provider that minted under a declared data-use contract is the one + whose contract applies. +- **Provenance is provider- and version-tagged.** Every graph row carries + the minting provider id and its configuration version (this is the same + provenance record the S2S sync authority reads, permission model spec + §7). Same-provider key/passphrase rotation is a version entry, not a + provider switch: parse consults all configured versions of the active + provider. - A cookie recognized by a legacy reader is a live identity for read/withdrawal purposes; whether it is transparently re-minted under the active writer is a per-deployment choice (`[ec] rewrite_legacy = true|false`), and re-minting is subject to the full minting gate of §5. +- **Rewrite is transactional and linking, not fire-and-forget.** Order: + new row commits first, carrying a link to the old row and a copy of the + old row's consent metadata and partner mappings; only then does the + cookie swap; the old row is tombstoned (or link-retired) only after the + new row and cookie are in place. An interrupted rewrite leaves the old + cookie valid and simply retries — no state in which neither identity + works. **Withdrawal of either linked row tombstones both.** - Retiring a legacy reader is the explicit end of those identities: the migration guide documents the cleanup procedure (migration spec §6). - Tests: switch active provider → request with old cookie → identity still - resolves and a withdrawal tombstones it; old cookie with no matching - legacy reader → treated as absent and **never egresses**. + resolves and a withdrawal tombstones it (both linked rows when + rewritten); old cookie with no matching legacy reader → treated as + absent and **never egresses**; interrupted rewrite → old cookie still + live, retry completes; `provider = "none"` + legacy reader → no mints, + withdrawal still works. Cluster degradation config (referenced from §3): when the active writer lacks the cluster capability, `[ec] cluster_fallback = "allow" | "deny"` decides whether KV-backed writes gated on cluster trust proceed; there is no implicit default — the operator chooses. +### 6.2 Runtime failure matrix — normative + +Startup validation (§6) covers configuration; this covers what happens +when a healthy configuration meets an unhealthy runtime. Every row logs at +`error` with a metric; none is silent: + +| Failure | Behavior | +| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| `generate` returns an error | No identity this request; request proceeds stateless; no cookie written | +| Graph-row commit fails at mint | Mint never happened (§5): no cookie, no egress; next request retries | +| Graph read fails on an existing identity | Identity unusable this request (fail closed for egress); cookie untouched | +| Cluster prefix listing fails | Treated as cluster-size-unknown → `cluster_fallback` policy applies | +| Tombstone write fails | Permission model spec §4.3: family retries, readers fail closed on partial families | +| Legacy rewrite fails mid-flight | Old cookie remains live; rewrite retries (§6.1) | +| Geo provider returns invalid output (unparseable country) at runtime | Treated as lookup failure → `default_country` (permission model spec §5.2), counted in the lookup-failure metric | +| Device provider signals unavailable at runtime (e.g. no JA4 on a request) | Classification degrades per the provider's declared fallback, never silently upgrades `looks_like_browser` | + ## 7. Composition root and adapter parity Provider construction happens in exactly one place per concern diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 88b2d99a3..2ad71526a 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -32,19 +32,23 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Policy decision: the shipped `US` baseline decides. If the baseline grants `store-on-device` without a signal, that is a **declared change** requiring sign-off in the policy review, with rationale in the policy itself | Declared change (if made) | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | -| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | -| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | -| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | -| 11 | Raw EC egress (OpenRTB `user.id`, derived request IDs, page bids, proxy/click/Testlight forwarding, identify, pull/batch sync) is gated by the jurisdiction gate today | Gated by the egress inventory (permission spec §7): bidstream egress requires both purposes, first-party identity operations require `store-on-device`, revocation exempt — at least as strict as today for every inventoried path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| # | Decision (today) | After epic | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | +| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | +| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | +| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull/batch sync) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -94,12 +98,22 @@ passphrase = "example-passphrase" Requirements: -1. **Old key fails loud.** `[ec] passphrase` is rejected at startup with a - message naming the new location — not a generic unknown-field error. - Implementation note: `Ec` already carries `deny_unknown_fields`, which - would reject the key generically; producing the actionable message means - keeping a deprecated `passphrase` field whose presence triggers the - custom error. +1. **The transition has a dual-read release; loud rejection comes one + release later.** Today's binary _requires_ `[ec] passphrase` and — via + `deny_unknown_fields` — _rejects_ `[ec] provider` and + `[ec.providers.*]`; a binary that rejects the old shape outright would + mean **no config both binaries accept**, and a config-store fleet + cannot flip config and binaries atomically. Sequence: + - **Release N+1 (dual-read):** accepts the old shape (mapping + `[ec] passphrase` to the `hmac` provider internally, logging a + deprecation warning per startup) _and_ the new shape. Fleet rolls + binaries to N+1 with config unchanged; then config flips to the new + shape via `ts config push`; either order is safe at every instant. + - **Release N+2:** rejects `[ec] passphrase` at startup with a message + naming the new location — not a generic unknown-field error + (implementation note: producing the actionable message means keeping + a deprecated `passphrase` field whose presence triggers the custom + error). 2. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the @@ -136,48 +150,36 @@ The migration guide (a new `docs/guide/` page, linked from the release notes) gives one copy-pasteable recipe per adapter for "keep exactly today's behavior": -The recipe is the **complete recommended policy table from -`trusted-server.example.toml`** — the full GDPR/UK/US jurisdiction rules, -copied, not referenced by omission — plus this delta: - -```toml -[ec] -provider = "hmac" -[ec.providers.hmac] -passphrase = "" - -[device] -provider = "fastly" # Fastly deployments: preserves the JA4 bot gate - -[geo] -provider = "platform" # preserves per-request jurisdiction detection -default_country = "FR" # used only when the host lookup fails (fail-closed - # because FR resolves to the gdpr-eu rule below) - -# ... the full example policy table goes here: gdpr-eu / gdpr-uk / -# us-opt-out groups and their country rules, verbatim ... - -# Delta vs. the protective example: preserve today's treatment of countries -# outside the regulation lists ("non-regulated" → identity allowed). Keep -# the example's `default = "gdpr-eu"` instead to require a signal worldwide -# (§2 row 7). -[permissions.groups.non-regulated] -regime = "none" -default = "granted" - -[permissions.rules] -default = "non-regulated" -``` +The recipe is **one complete, valid TOML fixture, committed to the +repository** (e.g. `docs/guide/fixtures/migration-preserving.toml`) and +included in the guide verbatim — never described as a textual delta +against the example file. (An earlier draft said "copy the example table, +then set `[permissions.rules] default`" — but the copied table already +declares `[permissions.rules]`, and reopening a TOML table is a parse +error; a prose delta cannot be validated, a committed fixture can.) The +fixture contains, in one document: + +- `[ec] provider = "hmac"` with its passphrase block; +- `[device] provider = "fastly"` (Fastly deployments: preserves the JA4 + bot gate); +- `[geo] provider = "platform"` and `default_country = "FR"` (per-request + jurisdiction detection preserved; the default is fail-closed because FR + resolves to the `gdpr-eu` rule); +- the full `gdpr-eu` / `gdpr-uk` / `us-opt-out` groups and country rules + from the example policy (US as `requires_signal` with the grant-signal + class — §2 rows 3–3b), plus the `non-regulated` group with + `rules.default = "non-regulated"` (row 7). Operators who prefer the + protective worldwide default use the example file itself instead. A partial policy is a trap the first draft of this spec fell into: a `[permissions]` section containing **only** the permissive default — with no GDPR/US rules — sends _every_ jurisdiction, France included, to the permissive fallback, because `default_country` selects a -rule like any other country and finds none. The recipe therefore always -carries the full table, and CI pins it: **the exact documented recipe text -is a fixture**, loaded and run through the complete §4.1/§4.2 decision -matrix of the permission spec, asserting per-jurisdiction outcomes match -the pre-epic gate for every preservation row of §2. +rule like any other country and finds none. The committed fixture is +therefore always complete, and CI pins it: **the fixture file itself** is +loaded and run through the complete §4.1/§4.2 decision matrix of the +permission spec, asserting per-jurisdiction outcomes match the pre-epic +gate for every preservation row of §2. The guide separately documents the neutral configuration and what it does _not_ do, states explicitly that `default_country` alone does not replace @@ -194,7 +196,14 @@ global honoring of opt-out signals is unconditional. 2. Before/after deploy, operators watch **EC issuance rate** and EID attachment rate; the migration guide names these as the canary metrics, because the failure mode of a bad migration is a silent drop to zero (or a - silent grant to everyone), not an error rate. + silent grant to everyone), not an error rate. The full metric set, each + with a stated healthy range: geo lookup-failure/fallback rate (permission + spec §5.2), raw-egress denials by path, tombstone family retries, + legacy-reader hit rate, rewrite failures, and cluster-fallback + engagements. Two of these carry thresholds, not just ranges: + legacy-reader hits trending to ~zero is the **retirement-readiness** + signal for a legacy provider, and a nonzero rewrite-failure rate blocks + retirement outright. 3. Startup logs always print: selected provider per concern, whether geo is live, the effective default baseline, and the count of granted-without- signal permissions. One line, greppable, stable format. From 572b104c796928d8abe645c0eb13e622574fada5 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 08:11:36 -0700 Subject: [PATCH 05/14] Address third review: regime-scoped grants, storage migration, and lifecycle repairs Blocking findings from the third review of PR #986: - Grant evidence is now regime- and permission-scoped: gdpr rules accept only TCF consent for the specific purpose, us-privacy accepts TCF or explicit GPP/USP non-opt-out, none accepts any grant class - closing the hole where sale_opt_out=false in France would have authorized identity and partner egress with no TCF. Opt-outs and refusals stay regime-agnostic. - The graph schema change gets an expand-contract rollout: reader/ preserver release (unknown fields preserved through read-modify-write), fleet-convergence gate, then writer activation, with schema versions, lazy backfill, and mixed-version tests. - S2S batch-sync authority is a full recompute of both permissions from stored per-permission, time-bounded evidence (grant basis, timestamp, jurisdiction, policy revision, provider/version) - failing closed on denied, tightened baselines without acceptable stored evidence, expired evidence, or regime-rejected grant sources. Legacy pre-epic rows are hmac-v0 with no grant evidence and fail closed until lazily backfilled. - Pull sync split from batch sync: pull is browser-request-scoped and keeps using the live P1/P4 decision plus revocation state; only batch is provenance-authorized, and its gate is declared hardening (new matrix row 11c; row 11a corrected). - Withdrawal centers on a family revocation record: a stable family ID in every member row, one record written first that is simultaneously the durable intent, the sibling-discovery mechanism, and the fail-closed marker; member tombstones become cleanup; degraded-graph mode fails S2S closed while writes fail; the healthy-graph residual is declared. - Legacy rewrite is confirmed by presentation: both linked rows stay live until a later request presents the new cookie (the server cannot observe Set-Cookie acceptance); linked rows share the revocation family. - The normalization matrix now preserves actual current semantics: whole-record selection by combined P1/P4 eligibility for restrictive/ permissive, LastUpdated with freshness threshold and restrictive tie-break for newest, proxy mode skips decoding, one-valid/one-expired row added, and GPP section fields enumerated normatively. - The auction matrix regains the raw-signal arm: a decodable TCF record applies the gdpr dispatch rule in every regime, so a P1 refusal on US or non-regulated traffic still blocks dispatch. - Provider namespaces become declarative descriptors core can prove pairwise disjoint at startup (opaque parse cannot be); version rotation gets a schema (versions entries, mint_version, newest-first parse, retirement rules). - Client resolve reservations get pending/committed/failed states, lease takeover, retention through token expiry, deterministic graph idempotency, adapter CAS capability - and duplicates never receive Set-Cookie unless the reservation is session-bound, closing the idempotent-replay fixation hole. - The hook invariant pass preserves any pre-hook private/no-store classification (cookieless personalized HTML cannot be made publicly cacheable) and strips CDN directives; panics are declared forbidden and fatal on wasm32-wasip1 (panic=abort - recovery was unimplementable); a cumulative final-response header budget with deterministic rejection order added. - Rollout ordering corrected to strictly reader-first with a convergence gate (the previous either-order claim was false against binaries that reject the new shape); mixed old/new config shapes rejected; rollback sequencing defined; one preserving fixture per adapter since device/geo selections are capability-gated. - Device-provider authorization reconciled with persisted use: new rows stop carrying fingerprint-derived buyer-facing fields (declared change); a field-level graph contract table is a required implementation deliverable. - Non-blockers folded in: adapter-capability matrix, assigned- subdivision region validation, retirement quiet period no shorter than max cookie/row lifetime plus skew, expanded telemetry, and the stale recognize/hashed/eligibility terms corrected. --- ...26-07-30-client-cycle-ec-resolve-design.md | 30 ++- ...integration-response-header-hook-design.md | 24 +- .../2026-07-30-permission-model-design.md | 206 ++++++++++-------- .../2026-07-30-pluggable-providers-design.md | 78 +++++-- ...07-30-provider-migration-rollout-design.md | 109 +++++---- 5 files changed, 295 insertions(+), 152 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 88b1fa66b..faed2b32d 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -76,7 +76,7 @@ Everything in this spec follows from that. the corresponding graph row is written, mirroring the organic path. Graph unavailable → no cookie, same as organic generation. 4. **Round-trip through the lifecycle contract.** The identifier set here - must be recognized, hashed, and tombstonable by the selected provider + must be parseable, graph-keyed, and tombstonable by the selected provider (providers spec §3). The conformance suite runs against every client-cycle provider. 5. **Exist on every adapter.** Route registration goes through shared route @@ -102,16 +102,30 @@ Everything in this spec follows from that. link, vendor migration) is an explicit linking design this spec does not authorize (own open question, §7). 9. **Make replay consumption and graph persistence one idempotent - sequence.** Neither naive order works: consume-the-nonce-first makes a + sequence — without letting idempotency reinstall the identity + elsewhere.** Neither naive order works: consume-the-nonce-first makes a subsequent graph failure unretryable (the token is spent, the identity never existed); graph-first lets the losers of a replay race leave residual rows. Required shape: consumption is an **atomic single-key - reservation** keyed by the payload's unique id, recording outcome; the - graph write happens under that reservation and is retried under the - same key; duplicate or racing requests observe the reservation and - receive the original outcome — no second identity, no spent-but-unused - token, no orphan row. Tests cover crash-between-steps and two - concurrent requests with the same payload. + reservation** (CAS — an adapter capability the composition root checks, + providers spec §7) keyed by the payload's unique id, with explicit + states: `pending` → `committed` | `failed`. The graph write happens + under the reservation and is retried under the same key; a `pending` + reservation older than its **lease** may be taken over by a retry; + reservations are retained at least through the token's expiry; the + graph write is deterministic under the reservation key so a retry + converges on the same row. + + **A duplicate must never receive the cookie unless the reservation is + session-bound.** "Duplicates observe the recorded outcome" cannot mean + replaying `Set-Cookie` — that would hand a captured token's identity to + a second browser, recreating the fixation §2 exists to prevent. In + one-time mode without session binding, a duplicate gets a terminal + response with **no cookie**; only a requester that proves the original + session binding (the §3.2 nonce) may have the `Set-Cookie` re-emitted. + Tests cover crash-between-steps, lease takeover, two concurrent + requests with the same payload, and a duplicate from a second client + receiving no cookie. ## 4. Requirements on the page script diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 1c6251288..f82f15dfb 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -52,7 +52,14 @@ mutators to the outbound response for HTML document responses it processed. a cookie — a hook applied after that recheck could combine an appended `Set-Cookie` with a replaced public `Cache-Control` into a **shared-cacheable cookie response**. The invariant pass therefore runs - after all mutations, unconditionally. Middle-stage placement also keeps + after all mutations, unconditionally — and it enforces more than the + cookie rule: **any private/no-store classification core assigned before + the hook is preserved** (processed auction HTML is marked private even + when no cookie is emitted — today's final helper returns early without + `Set-Cookie`, so cookie-triggered enforcement alone would let an + integration make cookieless personalized HTML publicly cacheable), and + every CDN/surrogate cache directive is stripped from any response so + classified. Middle-stage placement also keeps the earlier property: an integration mutation is not silently stripped by ordinary core handling — only by the invariant pass, which logs the downgrade it applies. @@ -81,10 +88,19 @@ mutators to the outbound response for HTML document responses it processed. `Set-Cookie` header name outright — cookies go only through `append_set_cookie`, so its validation cannot be bypassed by spelling the header name in a generic op. Per-integration limits bound total - operations, added header count, and added header bytes; exceeding a - limit rejects the excess operations (logged, attributed), never the - response. A mutator that panics or errors is skipped in full — its + operations, added header count, and added header bytes, and a + **cumulative final-response budget** (total header count and bytes) + bounds the sum across integrations — enforced in registration order, so + which operations are rejected when the budget trips is deterministic. + Exceeding a limit rejects the excess operations (logged, attributed), + never the response. A mutator that returns an error is skipped in full — its operations are all-or-nothing — and the response proceeds without it. + **Panics are forbidden and fatal, not recoverable**: the primary target + (`wasm32-wasip1`) builds with `panic = "abort"`, so there is no unwind + boundary to catch at — a spec that promised panic recovery would be + unimplementable there. Mutators are infallible-by-construction or + return `Result`; a panic is a bug that takes the instance down, same as + anywhere else in the request path. ## 3a. Response eligibility — normative diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 820426ab5..f509e8bee 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -64,7 +64,7 @@ The initial vocabulary is therefore exactly: | Identifier | TCF purpose | Enforcement points | | ------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `store-on-device` | 1 | EC provider execution; EC creation; withdrawal/tombstone eligibility | +| `store-on-device` | 1 | EC provider execution; EC creation; input to the §4.2 withdrawal decision (revocation itself is never permission-gated, §7) | | `select-personalised-ads` | 4 | All bidstream and partner identity egress — raw EC in `user.id`, derived request IDs, page bids, EIDs, identify, pull/batch sync (jointly with `store-on-device`; the full path table is §7) | (The identifier strings are the IAB names verbatim, including their original @@ -178,7 +178,7 @@ Validation rejects: - rule keys whose country part is not in the embedded **assigned** ISO 3166-1 alpha-2 list (not merely `[A-Z]{2}` — an unassigned code is almost certainly a typo silently diverting a country to the fallback); - the region part matches `[A-Z0-9]{1,3}`. The `US/CA` slash form is the + the region part must be an assigned ISO 3166-2 subdivision of that country (not merely a shape check — `US/ZZ` would parse but can never match a request), unless the selected geo provider declares its own region vocabulary, in which case validation uses that declaration. The `US/CA` slash form is the house rule-key format corresponding to ISO 3166-2 `US-CA`; - references to permissions outside the enforced vocabulary (§2); - references to undefined groups, and groups missing the `regime` class; @@ -243,10 +243,27 @@ blocked but an **explicit non-opt-out** value grants: consenting to the purpose; an **explicit GPP non-opt-out value** (e.g. `sale_opt_out = false`); a **US Privacy string present and not opting out** — including the "not applicable" flag, which today's tests pin as - allowing. Any grant signal satisfies a `requires_signal` baseline; this - is what lets a `requires_signal` US rule preserve today's "no signal → - block, explicit non-opt-out → allow" behavior, which neither `granted` - nor a TCF-only grant class could express. + allowing. Grant signals are what let a `requires_signal` US rule + preserve today's "no signal → block, explicit non-opt-out → allow" + behavior, which neither `granted` nor a TCF-only grant class could + express. **Which grant evidence a rule accepts is regime- and + permission-scoped** — grant signals are NOT interchangeable across + regimes: + + | Regime of the resolved rule | Evidence accepted as a grant for a `requires_signal` permission | + | --------------------------- | --------------------------------------------------------------- | + | `gdpr` | **Only** a TCF record consenting to that specific purpose | + | `us-privacy` | TCF consent for the purpose, or an explicit GPP/USP non-opt-out | + | `none` | Any grant-class signal | + + Without this scoping, a US-style `sale_opt_out = false` would satisfy a + French `requires_signal` rule — no TCF, both purposes granted, EC minted, + partner egress authorized — contradicting the GDPR preservation row of + the migration matrix. Auction dispatch blocking separately would not + help; identity use would already be authorized. Opt-out signals and + refusals remain regime-agnostic (global), as before: scoping applies + only to what can _grant_, never to what can _revoke_. + - **Refusals**: a decodable TCF record refusing the purpose. A refusal is neither a grant nor an opt-out — it blocks acquisition (precedence 3) and withdraws only per §4.2. @@ -271,11 +288,11 @@ blocked but an **explicit non-opt-out** value grants: jurisdictions — the migration spec's matrix (row 6) records it. Refusal revokes new grants only; whether it also destroys existing identity is governed strictly by §4.2. -4. Grant signal — any grant-class signal (TCF consent, explicit GPP - non-opt-out, present-and-not-opted-out US Privacy) grants the permission - (subject to 1–3: a coexisting TCF refusal beats a non-TCF grant signal, - matching today's US-state ordering where a present TCF record decides - before GPP/USP values are consulted). +4. Grant signal — a grant-class signal **accepted by the resolved rule's + regime for that permission** (table above) grants it (subject to 1–3: a + coexisting TCF refusal beats a non-TCF grant signal, matching today's + US-state ordering where a present TCF record decides before GPP/USP + values are consulted). 5. No signal — the policy baseline decides: `granted` sets it, `requires_signal` leaves it unset. @@ -284,12 +301,12 @@ blocked but an **explicit non-opt-out** value grants: For each enforced permission, with baseline _B_ ∈ {granted, requires_signal, denied}: -| Opt-out present | TCF refusal present | Grant signal present | Result | -| --------------- | ------------------- | -------------------- | ------------------------------------------------ | -| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | -| no | yes | — | unset (withdrawal per §4.2, trigger 2) | -| no | no | yes | set, unless B = denied | -| no | no | no | set iff B = granted | +| Opt-out present | TCF refusal present | Accepted grant present (regime-scoped) | Result | +| --------------- | ------------------- | -------------------------------------- | ------------------------------------------------ | +| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | — | unset (withdrawal per §4.2, trigger 2) | +| no | no | yes | set, unless B = denied | +| no | no | no | set iff B = granted | ### 4.2 Withdrawal vs. absence @@ -331,36 +348,39 @@ behavior had no unit test at all. ### 4.3 Withdrawal durability -Withdrawal is two writes — the KV tombstones and the cookie expiry — and -the contract for partial failure is explicit (PR #838 expired the cookie -first and logged-and-swallowed tombstone-write failures, which can leave a -live graph identity with no browser handle pointing at it): - -- **Order: tombstones first, cookie expiry second.** The cookie is expired - only after the tombstone writes succeed. -- **On tombstone-write failure, the cookie is left in place** and the - failure is logged at `error` with a metric. This is deliberately - self-healing: every withdrawal trigger is durable client-side (GPC is a - browser setting, the TCF record lives in the CMP's storage), so the next - request re-presents the signal and retries the whole withdrawal. No - quarantine queue is needed; the browser is the retry queue. -- **Partial progress is explicit, not atomic.** The tombstone writes are a - **revocation family**: an enumerable, ordered set of independent KV - writes (cookie hash, active-EC hashes), each idempotent, with no - multi-key atomicity assumed — the current storage offers none, and the - spec does not pretend otherwise. A retry resumes the family from the - start (idempotent writes make re-writing completed members harmless). - Withdrawal is **complete** only when every family member is committed; - the cookie expires only then. -- **Reads fail closed on partial families**: every consumer (identify, - batch-sync, pull-sync, egress gates) treats an identity as revoked when - **any** member of its revocation family is present — a partially - withdrawn identity is unusable immediately, even before the family - completes. -- Fault-injection tests cover failure at the **Nth** family write (not - only total failure): first write lands, second fails → cookie untouched, - identity already treated as revoked by readers, error logged; subsequent - request with the same signal → family completes and the cookie expires. +Withdrawal spans multiple KV writes and a cookie expiry; the contract for +partial failure is explicit (PR #838 expired the cookie first and +logged-and-swallowed tombstone-write failures, which can leave a live graph +identity with no browser handle pointing at it). The design centers on one +record that is simultaneously the durable intent, the discovery mechanism, +and the fail-closed marker: + +- **The family revocation record is written first.** Every identity carries + a stable **family ID**, minted with the identity and stored in every + member row (including rows linked by a legacy rewrite, providers spec + §6.1). Revocation writes one record keyed by the family ID. That single + write is the withdrawal: per-member tombstones are cleanup that follows, + idempotent and retried. +- **Every consumer checks the family record, not per-member tombstones.** + A reader arriving through any still-live member row finds the family ID + in the row and the revocation record under it — partial revocation is + discoverable from every member, and the record survives member-tombstone + replacement (which today discards the original row's identity and + metadata, making sibling discovery impossible). +- **The cookie expires only after the family record commits.** +- **If the family-record write itself fails, nothing durable exists** — + the cookie stays and the durable client-side signal (GPC, CMP-stored TCF) + retries the whole withdrawal on the next request. Two mitigations bound + the S2S residual in the meantime: while graph **writes are degraded** + (health signal), S2S partner egress and sync updates fail closed + (providers spec §6.2); and the failure is logged at `error` with a + metric feeding the operational repair path. The residual that remains — + a single failed write on an otherwise healthy graph, for a user who + never returns — is declared here, not hidden. +- Fault-injection tests cover: family-record write fails → cookie + untouched, S2S behavior per degraded mode, retry completes; member + tombstone N fails after the family record → identity already revoked for + every reader, cleanup retries; the same-signal retry path end to end. ### 4.4 Signal normalization — normative matrix @@ -371,18 +391,19 @@ silently. These are the outcomes — decided here, not delegated to the implementation; each row marked **changed** also appears in the migration matrix: -| Input state | Effective record / outcome | Status | -| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | -| Standalone TCF and GPP-embedded TCF disagree per purpose, mode `restrictive` | Per-purpose **synthesis**: a purpose is consented only when **both** records consent (AND) | Preserved (mode semantics pinned against current tests) | -| Same, mode `permissive` | Per-purpose synthesis: consented when **either** record consents (OR) | Preserved (same pinning) | -| Same, mode `newest` | **Whole-record selection** by `Created` timestamp; tie → the GPP-embedded record | Preserved (same pinning) | -| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | -| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | -| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | -| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | Preserved, circularity resolved | -| Proxy/mirror mode (CMP state mirrored server-side) | Mirror-sourced record enters as a live record; when both the mirror and the request carry records, the **request's record wins** (closer to the user) | Decided here | -| GPP opt-out fields | Enumerated per supported section: the sale, sharing, and targeted-advertising opt-out fields each independently constitute an opt-out signal when set; **all present-and-false** constitutes a grant signal (§4); absent/N-A fields contribute nothing. The exact field list per section ID is an appendix of the implementation PR, reviewed against the GPP spec | Decided here | -| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | +| Input state | Effective record / outcome | Status | +| ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | +| Standalone TCF and GPP-embedded TCF disagree, mode `restrictive` | **Whole-record selection** (today's semantics — an earlier draft specified per-purpose synthesis, which is _not_ what the code does): the record whose combined P1 ∧ P4 eligibility is more restrictive governs in full | Preserved (mode semantics pinned against current tests) | +| Same, mode `permissive` | Whole-record selection: the record whose combined P1 ∧ P4 eligibility is more permissive governs in full | Preserved (same pinning) | +| Same, mode `newest` | Whole-record selection by **`LastUpdated`** (not `Created`), subject to the existing freshness threshold; a tie or inconclusive comparison falls back to the **restrictive** selection | Preserved (same pinning) | +| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | +| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | +| One valid record + one **expired** record of the same family | The valid record governs; the expired record is absent entirely (consistent with the expiry row) | Decided here | +| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | +| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | Preserved, circularity resolved | +| Proxy/mirror mode | **Consent decoding is skipped entirely** — today's behavior, preserved as-is; no mirror-sourced record is synthesized (an earlier draft invented one). Retiring proxy mode, if ever wanted, is its own declared change | Decided here | +| GPP opt-out fields | Normative, not deferred: in the US-National section, `SaleOptOut`, `SharingOptOut`, and `TargetedAdvertisingOptOut` each independently constitute an opt-out signal when set to opted-out; each supported US state section maps its correspondingly named fields identically; a field explicitly set to not-opted-out is grant-class evidence (§4, regime-scoped); absent or N/A fields contribute nothing; **unsupported sections contribute nothing** (neither grant nor revoke). Adding a section is a spec change to this row | Decided here | +| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | ## 5. Jurisdiction resolution @@ -490,42 +511,57 @@ Consumers of the resolved set in this epic: inventory, normative per path (one test per row; a denylist check proves no ungated egress exists): - | Path | Required permissions | Notes | - | ---------------------------------------------------------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | - | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | - | EC-derived auction request IDs | both purposes | Derived values are identity | - | Page-bids path | both purposes | | - | Bidstream EIDs | both purposes | The one gate PR #838 had | - | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | - | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | - | Pull sync / batch sync (partner identity exchange) | both purposes | Authority source for S2S requests: stored provenance, below | - | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | - | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | - | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | + | Path | Required permissions | Notes | + | ---------------------------------------------------------------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | + | EC-derived auction request IDs | both purposes | Derived values are identity | + | Page-bids path | both purposes | | + | Bidstream EIDs | both purposes | The one gate PR #838 had | + | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | + | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | + | Pull sync (browser-request-scoped partner exchange) | both purposes, from the **live** request resolution | Pull sync is created from a browser request and checks the live `EcContext` today — it keeps using the live P1 ∧ P4 decision plus the family revocation state (§4.3); stored provenance is never a substitute for available live evidence | + | Batch sync (context-free S2S partner exchange) | both purposes, from **stored provenance** | The only truly signal-less path; authority rules below. Today's handler only authenticates and checks row state, so this gate is **declared hardening** (migration spec §2) | + | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | + | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | + | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | With **no EC provider configured**, identity use fails closed: a cookie value present on the request never egresses anywhere — never vacuously allowed (#838's `ec_allowed` was `is_none_or`, vacuously true with no provider). - **S2S authority (batch/pull sync).** A server-to-server request carries - no user signals, geo, or `EcContext` to resolve permissions from. Its - authority is the identity's **stored provenance**: a record written at - mint/update time carrying the resolved jurisdiction, regime, grant - basis, and provider/version (providers spec §6.1). A sync request - re-validates that provenance against the **current** policy revision: - if the stored jurisdiction now resolves to `denied` for the required - permission, the row is not updated and is flagged for the operational - cleanup of §4.2 trigger 3. Sync never mints authority of its own. + **S2S authority (batch sync).** A context-free server-to-server request + carries no user signals, geo, or `EcContext`. Its authority is the + identity's **stored provenance**: per-permission, time-bounded evidence + written at mint and refreshed on later live requests — grant basis + (which signal class granted, per permission), evidence timestamp, + resolved jurisdiction, policy revision, and provider/version (providers + spec §6.1). A sync request performs a **full recompute of both + permissions** from that stored evidence against the _current_ policy: + it fails closed when the stored jurisdiction's rule is now `denied`, + when a `granted` baseline tightened to `requires_signal` and the stored + evidence contains no accepted grant for that permission, when the + stored evidence has **expired**, or when the regime no longer accepts + the stored grant's source class (§4's regime-scoped table). Any of + these → no update, row flagged for the operational cleanup of §4.2 + trigger 3. Sync never mints authority of its own. + + **Legacy (pre-epic) rows** carry none of these fields. They are treated + as reserved `hmac-v0` provenance with **no stored grant evidence**, so + they **fail closed for partner egress and batch updates** until a live + browser request lazily backfills provenance from a fresh resolution. + Failing open here would grandfather every pre-epic identity past the + permission model indefinitely. 4. **Server-side auction dispatch** — gated on the policy `regime` class, normatively: - | Regime | Dispatch rule | Preserves | - | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | - | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | - | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | - | `none` | Dispatch proceeds. | Today's non-regulated arm | + | Regime | Dispatch rule | Preserves | + | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | + | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | + | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | + | `none` | Dispatch proceeds. | Today's non-regulated arm | + | **Any regime, decodable TCF record present** | The `gdpr` row applies: dispatch requires that record to consent to Purpose 1 — a raw TCF signal makes the request GDPR-relevant regardless of geolocation, so a US or non-regulated request carrying a Purpose 1 refusal is blocked. | Today's raw-signal arm — **must not regress** | The **compiled-in fallback policy has `regime = "gdpr"`** (§3.1) — the no-policy posture must be the most protective for dispatch too, and a diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index c4dc9cc37..cb17b604d 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -122,8 +122,17 @@ Three global rules sit above every provider: characters) and a global maximum length — for the identifier itself, not only the graph key — enforced by core at mint and at parse, so no provider can emit a value the cookie layer or logs cannot carry. +- **Namespaces are declarative and core-proven.** Disjointness of two + opaque `parse` functions is not provable, so every provider declares a + **static namespace descriptor** in a core-owned declarative form — a set + of literal prefixes and/or fixed-shape grammars (alphabet + length + segments), never arbitrary parser logic. Core proves pairwise + disjointness of all configured descriptors at startup (§6.1), and the + conformance suite asserts each provider's `parse` accepts **only** + values matching its declared descriptor — so the declaration, not the + parser, is the authority the overlap check rests on. - **Namespace reservation.** The legacy HMAC grammar `{64hex}.{6alnum}` is - formally **reserved as the `hmac` provider's namespace**. `hmac`'s graph + formally **reserved as the `hmac` provider's namespace descriptor**. `hmac`'s graph key is the identifier verbatim and its cluster prefix is the 64-hex prefix, so every pre-epic row stays reachable and every prefix listing intact (migration spec §3) — and **no other provider may mint @@ -228,7 +237,17 @@ structural: device provider whose data use goes beyond security classification (for example feeding fingerprints into targeting or identity) is **not authorized by selection alone** and requires a vocabulary extension plus - a gate before it may ship. + a gate before it may ship. This bites immediately, not hypothetically: + today's graph rows persist the JA4 class, an HTTP/2 fingerprint hash, + and buyer-facing quality metadata — persistence and scoring that exceed + security classification. The epic therefore **stops writing + fingerprint-derived buyer-facing fields into new rows** (a declared + change, migration spec §2); the boolean security classification outcome + may be persisted. Re-adding them is the vocabulary-extension route. + Relatedly, the implementation PR must deliver a **field-level graph + contract table** — for every persisted row field: purpose, source, + gating permission, TTL, rewrite behavior, egress paths, and tombstone + scrubbing — reviewed against the egress inventory. PR #838 declared `required_permissions` on all three traits but consulted it only for the EC provider; the geo and device declarations were @@ -273,31 +292,44 @@ The contract: block, validated like the active one (§6 table). - **Parse order and ambiguity.** The active writer parses first; the first match wins. Overlapping recognition is not resolved at request time but - **forbidden at startup**: provider namespaces (§3) may not overlap, and - configuring an active/legacy pair whose grammars intersect is a - validation error. + **forbidden at startup**: the declared namespace descriptors (§3) of the + active writer and every legacy reader must be pairwise disjoint — a + check core can actually perform, because descriptors are declarative; + configuring a pair whose descriptors intersect is a validation error. - **The recognizing provider governs.** A legacy-owned identity is gated by the **legacy provider's** `required_permissions()` for identity use — the provider that minted under a declared data-use contract is the one whose contract applies. -- **Provenance is provider- and version-tagged.** Every graph row carries - the minting provider id and its configuration version (this is the same - provenance record the S2S sync authority reads, permission model spec - §7). Same-provider key/passphrase rotation is a version entry, not a - provider switch: parse consults all configured versions of the active - provider. +- **Provenance is provider- and version-tagged, with a defined rotation + schema.** Every graph row carries the minting provider id, its + configuration version, and the per-permission grant evidence (grant + basis, evidence timestamp, resolved jurisdiction, policy revision) that + the S2S sync authority recomputes from (permission model spec §7). + Same-provider key/passphrase rotation is configuration, not a provider + switch: a provider block may hold multiple `versions` entries + (`[ec.providers.hmac.versions.v2] passphrase = …`) with + `mint_version = "v2"` selecting the writer; `parse` consults versions in + declared order, newest first; removing a version entry is a retirement + subject to the same evidence rules as retiring a legacy reader + (migration spec §6). - A cookie recognized by a legacy reader is a live identity for read/withdrawal purposes; whether it is transparently re-minted under the active writer is a per-deployment choice (`[ec] rewrite_legacy = true|false`), and re-minting is subject to the full minting gate of §5. -- **Rewrite is transactional and linking, not fire-and-forget.** Order: - new row commits first, carrying a link to the old row and a copy of the - old row's consent metadata and partner mappings; only then does the - cookie swap; the old row is tombstoned (or link-retired) only after the - new row and cookie are in place. An interrupted rewrite leaves the old - cookie valid and simply retries — no state in which neither identity - works. **Withdrawal of either linked row tombstones both.** +- **Rewrite is transactional, linking, and confirmed by presentation.** + Order: new row commits first, carrying a link to the old row (sharing + its revocation family ID, permission model spec §4.3) and a copy of the + old row's consent metadata and partner mappings; then the new cookie is + emitted. The server only emits `Set-Cookie` — it cannot observe delivery + or acceptance, so **both linked rows stay live** until a later request + **presents the new cookie** (confirmation by presentation); only then is + the old row retired. A deployment may additionally cap the window with a + grace period no shorter than the old cookie's maximum lifetime plus + rollout skew. An interrupted or unconfirmed rewrite leaves the old + cookie fully valid — no state in which neither identity works. + **Withdrawal of either linked row revokes the shared family, i.e. + both.** - Retiring a legacy reader is the explicit end of those identities: the migration guide documents the cleanup procedure (migration spec §6). - Tests: switch active provider → request with old cookie → identity still @@ -341,6 +373,16 @@ outcomes. Requirements: +- **Adapters declare capabilities against an explicit matrix.** The + capability set the composition root checks selections against is + enumerated, not ad hoc: identity-graph persistence, atomic single-key + reservation (CAS — required by the client-cycle reservation and any + future compare-and-set use), KV prefix listing (cluster support), + platform geo, device host evidence (JA4/HTTP-2), and legacy-rewrite + support. Each adapter's declaration is part of its wiring, and the §6 + capability-mismatch startup error is driven by this matrix. Every §6.2 + runtime-failure row gets fault-injection coverage on every adapter that + declares the corresponding capability. - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in app state; PR #838 rebuilt the provider (cloning the secret into a fresh diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 2ad71526a..99ed06ef7 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -32,23 +32,24 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | -| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | -| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | -| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | -| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | -| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | -| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | -| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull/batch sync) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | -| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | +| # | Decision (today) | After epic | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | +| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | +| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | +| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | +| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -67,8 +68,8 @@ selects `provider = "hmac"` and carries its passphrase over verbatim: vectors: fixed passphrase + IP → exact expected 64-hex prefix, committed so any divergence fails CI rather than rotating the production identity base. -- **Existing cookies stay recognized.** Fixture `ts-ec` values minted by - the pre-epic code pass the provider's `recognize`, and their graph rows +- **Existing cookies stay parseable.** Fixture `ts-ec` values minted by + the pre-epic code pass the provider's `parse`, and their graph rows (keyed by the identifier verbatim) remain reachable — no row is orphaned. - **The hash prefix keeps its semantics.** `ec_hash` remains the 64-hex prefix, preserving both its stability and its deliberate collision across @@ -106,40 +107,63 @@ Requirements: cannot flip config and binaries atomically. Sequence: - **Release N+1 (dual-read):** accepts the old shape (mapping `[ec] passphrase` to the `hmac` provider internally, logging a - deprecation warning per startup) _and_ the new shape. Fleet rolls - binaries to N+1 with config unchanged; then config flips to the new - shape via `ts config push`; either order is safe at every instant. + deprecation warning per startup) _and_ the new shape. Ordering is + **strictly reader-first, never "either order"**: current binaries + reject the new shape (and reject the new `[permissions]` / `[device]` + / `[geo]` additions as unknown fields), so the config may flip only + after **fleet convergence on N+1 is confirmed** — binaries first, + convergence gate, then `ts config push`. A config mixing old and new + fields (`[ec] passphrase` alongside `[ec] provider`) is **rejected** + by N+1, not reconciled. Rollback runs the sequence in reverse: config + back to the old shape first, binaries only after config convergence. + Every new config section introduced by the epic follows this same + compatibility rule, not only `[ec]`. - **Release N+2:** rejects `[ec] passphrase` at startup with a message naming the new location — not a generic unknown-field error (implementation note: producing the actionable message means keeping a deprecated `passphrase` field whose presence triggers the custom error). -2. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no +2. **The graph schema change is expand-contract, in lockstep with the + binary sequence.** New rows carry fields v1 rows never had — provider/ + version, per-permission grant evidence, policy revision, family ID, + rewrite links — and two failure modes must be engineered away: a naive + schema-version bump makes old readers fail closed on new rows, and an + old worker that reads, modifies, and reserializes a row **silently + drops** fields it does not model. Sequence: (a) a **reader/preserver + release** ships first — it understands the new fields and, critically, + preserves unknown fields verbatim through read-modify-write; (b) a + **fleet-convergence gate**; (c) only then does **writer activation** + begin emitting the new fields. Rows carry an explicit schema version; + backfill is lazy via live requests (the same pass that backfills legacy + provenance, permission spec §7). Mixed-version tests are mandatory: + old-reader/new-row, new-reader/old-row, and old-worker + read-modify-write preserving new fields byte-for-byte. +3. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the docs reaches if they miss one line — validated green and silently minted zero ECs. -3. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by +4. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by PR #838, deliberately not carried into this epic — providers spec §2) and `provider = "client-fixed"` are unknown keys and rejected like any other, so a config written against the PR #838 example cannot silently select a provider that no longer exists. -4. **Provider switches go through legacy readers.** Changing +5. **Provider switches go through legacy readers.** Changing `[ec] provider` on a deployment with live identities requires listing the outgoing provider in `[ec] legacy_providers` (providers spec §6.1) so existing cookies keep resolving and stay withdrawable; the guide documents the switch sequence and the retirement/cleanup step that ends it. -5. **The example config ships the migrated happy path**, uncommented: +6. **The example config ships the migrated happy path**, uncommented: `provider = "hmac"` with its block, `[geo] default_country`, and (for Fastly) the behavior-preserving `[device] provider = "fastly"` and `[geo] provider = "platform"` lines present with a comment stating what removing them changes. PR #838's example shipped the passphrase block uncommented with the selector commented out — steering operators directly into the silent-stateless state. -6. Every misconfiguration in the providers spec §6 table fails at +7. Every misconfiguration in the providers spec §6 table fails at **startup**. Request-time failure for a configuration error is a defect. -7. Config-store payload validation (`ts config push`) applies the same +8. Config-store payload validation (`ts config push`) applies the same rules — including `[permissions]` policy validation (permission spec §3.3) — so a bad config is rejected at push time, before any instance restarts into it. @@ -150,10 +174,15 @@ The migration guide (a new `docs/guide/` page, linked from the release notes) gives one copy-pasteable recipe per adapter for "keep exactly today's behavior": -The recipe is **one complete, valid TOML fixture, committed to the -repository** (e.g. `docs/guide/fixtures/migration-preserving.toml`) and -included in the guide verbatim — never described as a textual delta -against the example file. (An earlier draft said "copy the example table, +The recipe is a **complete, valid TOML fixture per adapter, committed to +the repository** (e.g. `docs/guide/fixtures/migration-preserving-fastly.toml` +and siblings) and included in the guide verbatim — never described as a +textual delta against the example file. Per-adapter because a single +fixture cannot be: `[device] provider = "fastly"` and +`[geo] provider = "platform"` are capability-gated selections that the +Axum/Cloudflare/Spin adapters reject at startup (providers spec §6); each +adapter's fixture carries the selections valid for it, and each is +CI-validated against its adapter. (An earlier draft said "copy the example table, then set `[permissions.rules] default`" — but the copied table already declares `[permissions.rules]`, and reopening a TOML table is a parse error; a prose delta cannot be validated, a committed fixture can.) The @@ -165,7 +194,8 @@ fixture contains, in one document: - `[geo] provider = "platform"` and `default_country = "FR"` (per-request jurisdiction detection preserved; the default is fail-closed because FR resolves to the `gdpr-eu` rule); -- the full `gdpr-eu` / `gdpr-uk` / `us-opt-out` groups and country rules +- (Fastly fixture; other adapters substitute their valid selections) + the full `gdpr-eu` / `gdpr-uk` / `us-opt-out` groups and country rules from the example policy (US as `requires_signal` with the grant-signal class — §2 rows 3–3b), plus the `non-regulated` group with `rules.default = "non-regulated"` (row 7). Operators who prefer the @@ -175,7 +205,7 @@ A partial policy is a trap the first draft of this spec fell into: a `[permissions]` section containing **only** the permissive default — with no GDPR/US rules — sends _every_ jurisdiction, France included, to the permissive fallback, because `default_country` selects a -rule like any other country and finds none. The committed fixture is +rule like any other country and finds none. Each committed fixture is therefore always complete, and CI pins it: **the fixture file itself** is loaded and run through the complete §4.1/§4.2 decision matrix of the permission spec, asserting per-jurisdiction outcomes match the pre-epic @@ -201,9 +231,14 @@ global honoring of opt-out signals is unconditional. spec §5.2), raw-egress denials by path, tombstone family retries, legacy-reader hit rate, rewrite failures, and cluster-fallback engagements. Two of these carry thresholds, not just ranges: - legacy-reader hits trending to ~zero is the **retirement-readiness** - signal for a legacy provider, and a nonzero rewrite-failure rate blocks - retirement outright. + legacy-reader hits at zero for a **quiet period no shorter than the + maximum cookie/row lifetime plus rollout skew** — or provable + rewrite/backfill completion — is the **retirement-readiness** bar for a + legacy provider ("trending to ~zero" is not evidence; a yearly visitor + is not churn), and a nonzero rewrite-failure rate blocks retirement + outright. The telemetry set also includes: graph read/commit failures, + stored-provenance denials, schema-migration failures, and + replay-reservation recoveries. 3. Startup logs always print: selected provider per concern, whether geo is live, the effective default baseline, and the count of granted-without- signal permissions. One line, greppable, stable format. From 2b4d776b65790aea0c5596b7c45acd6ba4f7c1ac Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 12:24:21 -0700 Subject: [PATCH 06/14] Address architecture review: legacy revocation, consent-field semantics, and distributed contracts P0: legacy identities can now enter the family-revocation protocol. Rows lacking a family ID derive one deterministically from (record kind, provider namespace, canonical graph key), so a first-post-upgrade withdrawal is discoverable by every future reader even if the writer crashes before touching the v1 row; random IDs are called out as recreating the orphan the design exists to eliminate. Withdrawal never depends on backfill; tests pin the first-request-is-withdrawal and crash-between-writes cases. P1 groups: - US signal fields get a normative field x value x permission x destructive table (4.5): sale maps to P1+P4 and is destructive, sharing and targeted-advertising map to P4 only and never destroy identity, USP carries no targeted field, absent/N-A grants nothing, state sections override national, opt-out beats grant across sections. The regime evidence table now defers to this mapping. - Normalization is a six-state machine (valid-grant, valid-refusal, opt-out, malformed-present, expired, absent) wired into precedence and the decision matrix (malformed blocks the granted baseline); proxy mode keeps a syntax pass so malformed is distinguishable, blocks record-derived grants, and is declared as a change from today's fail-open skip. - Conflict resolution is deterministic: whole-record selection over the (P1, P4) tuple ordered lexicographically P1-first (split-purpose records decided), newest uses LastUpdated with the freshness threshold and falls back to restrictive; expiry drops sources before conflict resolution - a declared change from today's conflict-first ordering. - The auction raw-signal arm triggers on raw TC-string presence or a GPP section-2 hint before decoding, so malformed raw TCF still blocks dispatch outside GDPR regions. - Stored provenance ages: authoritative timestamp + valid_until per evidence class, re-presentation does not reset age, and every live resolution atomically replaces the full per-permission snapshot so a later refusal clears older positive authority; rewrite provenance is the fresh live resolution, partner mappings keep original expiries. - A per-record-class consistency matrix: replay reservations need linearizable CAS with fencing (Durable-Object-class on Cloudflare, not Workers KV), family revocation records need strong reads plus a declared bounded visibility lag with read-failure failing closed, identity rows may be eventual; retention outlives every dependent lifetime (today's 24h tombstone TTL explicitly does not carry over). - The US policy enumerates US/ rules for configured privacy states with country-level US non-regulated, preserving Wyoming-class traffic; regionless-geo degradation is declared; the states-list consistency test is region-shaped. - The graph field contract is normative in-spec (providers 6.3): every v1 and new field with purpose, source, gating permission, TTL, rewrite, and revocation treatment - including discontinuing fingerprint-derived buyer-facing fields; releases unified as N/N+1/N+2 with semantic (not byte) unknown-field preservation, a hard rollback floor at N+1 after writer activation, and stated mixed-version expectations. - Identity boundaries are structural: core constructs physical graph keys with record-kind/provider/version prefixes (legacy hmac verbatim excepted) and an AuthorizedIdentity newtype - constructible only after parse, permission, graph, and family checks - is the only type outbound serializers accept. - Legacy rewrite aliases to one canonical row via fenced CAS (no dual-write divergence), with confirmation by presentation and a finite retirement deadline. - Client-cycle: session binding is required for production schemes (one-time consumption demoted to defense-in-depth; at-most-once only as an explicitly recorded posture with orphan cleanup); reservations carry owner hash and monotonic lease epochs with fenced transitions; owner-hash retry re-emits the cookie so lost responses do not orphan rows; resolve checks the family revocation record and loses races to revocation. - The hook snapshots all pre-hook cache restrictions (origin-supplied included) and allows only equal-or-stronger privacy; Content-Encoding and Content-Range join the reserved surface; the test set covers origin-private and core-private cookieless HTML, cache hits, Vary, every CDN directive, and body encoding. P2/P3: mint 'cookie write' clarified as scheduled-on-final-response with egress eligibility at graph commit; degraded-health is a per-instance in-memory state machine with hysteresis; HMAC versions resolve from row provenance (untagged = hmac-v0), not parse; client limits are exact (65,536-byte body, content-type allowlist, 256-byte identifier, 128-byte reservation key, per-code statuses); migration matrix gains rows 3d-3g; fixtures include the graph-store config; every rollout metric ships with threshold, window, and action; batch-sync's coverage dip is operationalized with the provenance-coverage metric; FR default relabeled a protective opt-in fallback; device-selection authorization qualified to the opt-in fingerprint provider; the illustrative policy example is labeled as such. --- ...26-07-30-client-cycle-ec-resolve-design.md | 82 +++++--- ...integration-response-header-hook-design.md | 38 ++-- .../2026-07-30-permission-model-design.md | 194 +++++++++++++----- .../2026-07-30-pluggable-providers-design.md | 126 +++++++++--- ...07-30-provider-migration-rollout-design.md | 62 ++++-- 5 files changed, 366 insertions(+), 136 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index faed2b32d..2f27f1ab6 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -65,13 +65,15 @@ Everything in this spec follows from that. that are signed by an expected party, **audience-bound** to this publisher, and **expiring**. Audience binding and expiry alone do not mitigate replay — a captured token installs in another browser for the - whole validity window — so one of the following is additionally - required: **binding to the requesting browser session** (a server-issued - nonce the payload must embed), or **server-side one-time consumption** - (a replay cache on the payload's unique id). A scheme that can support - neither may only ship if its residual replay window is quantified and - explicitly accepted in the feature's issue — "single-use where the - scheme allows" is not a mitigation. + whole validity window. **Production schemes require session binding** + (a server-issued nonce the payload must embed): one-time consumption + alone limits multiplicity but proves nothing about _which_ browser + redeems first — a captured bearer payload can simply win the race — so + it is defense-in-depth, not the mitigation. First-presenter + at-most-once semantics may ship **only** as an explicitly accepted + posture recorded in the feature's issue, together with a specified + orphan-row cleanup path. "Single-use where the scheme allows" is not a + mitigation. 3. **Preserve the identity-graph invariant.** The cookie is set only after the corresponding graph row is written, mirroring the organic path. Graph unavailable → no cookie, same as organic generation. @@ -86,13 +88,18 @@ Everything in this spec follows from that. 6. **Be uncacheable and permission-gated.** `Cache-Control: no-store`; the same `store-on-device` permission gate as organic EC creation runs before any cookie is set. -7. **Bound every input.** A maximum request-body size (order of the 64 KiB - limit PR #838 at least had), enforced by a **bounded read independent of - `Content-Length`** — a missing, false, or chunked length must not bypass - it; a `Content-Type` allowlist; and a length/character-set constraint on - the resulting identifier that keeps it cookie-safe and within the KV - limits of the providers spec §3. Tests exercise the exact 413 boundary - and the missing/false/chunked-length cases. +7. **Bound every input — exact values, testable boundaries.** Request + body: at most **65,536 bytes** (inclusive; byte 65,537 → `413`), + enforced by a bounded read independent of `Content-Length` — a + missing, false, or chunked length must not bypass it. `Content-Type` + allowlist: `text/plain` and `application/json`; anything else → `415`. + The resulting identifier: at most **256 bytes**, cookie-safe alphabet + (providers spec §3 global bounds); violation → `400`. Reservation key + (payload unique id): at most **128 bytes**. Status codes are part of + the contract: `400` malformed payload/identifier, `403` origin/token + rejection, `409` different-identity or revoked-family conflict (§3.8), + `413` body, `415` content type. Tests exercise each boundary at its + exact edge, including the missing/false/chunked-length cases. 8. **Define behavior against an existing identity — no silent replacement.** When the request already carries a recognized EC: resolving to the **same** identity is an idempotent no-op (cookie @@ -109,23 +116,36 @@ Everything in this spec follows from that. residual rows. Required shape: consumption is an **atomic single-key reservation** (CAS — an adapter capability the composition root checks, providers spec §7) keyed by the payload's unique id, with explicit - states: `pending` → `committed` | `failed`. The graph write happens - under the reservation and is retried under the same key; a `pending` - reservation older than its **lease** may be taken over by a retry; - reservations are retained at least through the token's expiry; the - graph write is deterministic under the reservation key so a retry - converges on the same row. - - **A duplicate must never receive the cookie unless the reservation is - session-bound.** "Duplicates observe the recorded outcome" cannot mean - replaying `Set-Cookie` — that would hand a captured token's identity to - a second browser, recreating the fixation §2 exists to prevent. In - one-time mode without session binding, a duplicate gets a terminal - response with **no cookie**; only a requester that proves the original - session binding (the §3.2 nonce) may have the `Set-Cookie` re-emitted. - Tests cover crash-between-steps, lease takeover, two concurrent - requests with the same payload, and a duplicate from a second client - receiving no cookie. + states: `pending` → `committed` | `failed`, each carrying an **owner + hash** (the session binding) and a **monotonic lease epoch**. Takeover + of an expired `pending` lease increments the epoch, and every state + transition is a fenced CAS on (state, epoch) — a stale owner resuming + after its lease expired cannot commit over the takeover's work, because + its epoch no longer matches. `failed` is retryable: the same owner may + supersede it with a fresh `pending` at a higher epoch. The graph write + happens under the reservation and is deterministic under its key, so + any retry converges on the same row; reservations are retained at least + through the token's expiry. + + **A duplicate must never receive the cookie unless it proves the + original session binding.** "Duplicates observe the recorded outcome" + cannot mean replaying `Set-Cookie` — that would hand a captured token's + identity to a second browser, recreating the fixation §2 exists to + prevent. A requester matching the reservation's owner hash **does** + have the `Set-Cookie` re-emitted — which is precisely how a legitimate + browser whose original response was lost recovers on retry, so a + committed graph row never strands as an orphan for the intended + browser; anyone else gets a terminal response with no cookie. In the + explicitly-accepted at-most-once posture (no owner hash), a lost + response is an **orphan row** handled by the specified cleanup path. + The **same-identity no-op of §3.8 first checks the family revocation + record** (permission model spec §4.3): a resolve against a revoked + family is rejected, never refreshed, and a create racing a revocation + loses — revocation wins. Tests cover crash-between-steps, lease + takeover with a stale-epoch commit attempt, two concurrent requests + with the same payload, a duplicate from a second client receiving no + cookie, owner-hash recovery receiving the cookie, and + resolve-vs-revocation races. ## 4. Requirements on the page script diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index f82f15dfb..8ec5b687d 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -53,13 +53,17 @@ mutators to the outbound response for HTML document responses it processed. `Set-Cookie` with a replaced public `Cache-Control` into a **shared-cacheable cookie response**. The invariant pass therefore runs after all mutations, unconditionally — and it enforces more than the - cookie rule: **any private/no-store classification core assigned before - the hook is preserved** (processed auction HTML is marked private even - when no cookie is emitted — today's final helper returns early without - `Set-Cookie`, so cookie-triggered enforcement alone would let an - integration make cookieless personalized HTML publicly cacheable), and - every CDN/surrogate cache directive is stripped from any response so - classified. Middle-stage placement also keeps + cookie rule. Core **snapshots the complete pre-hook cache restriction + state** — whether the restriction came from core's own classification + (processed auction HTML is marked private even when no cookie is + emitted; today's final helper returns early without `Set-Cookie`) **or + from the origin** (an origin-supplied `private, no-store` that core + merely passed through) — and the post-hook response may only be + **equal or stronger** on the privacy axis: integrations can tighten + caching, never loosen it, regardless of which header they replaced. + Every CDN/surrogate cache directive (`Surrogate-Control`, + `CDN-Cache-Control`, host-specific equivalents) is stripped from any + restricted response. Middle-stage placement also keeps the earlier property: an integration mutation is not silently stripped by ordinary core handling — only by the invariant pass, which logs the downgrade it applies. @@ -70,8 +74,11 @@ mutators to the outbound response for HTML document responses it processed. granularities because `Set-Cookie` is multi-valued: (a) reserved header _names_ — HTTP framing and hop-by-hop headers (`Content-Length`, `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, - `Keep-Alive`), the `x-ts-*` namespace, and the consent/privacy headers - core emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, + `Keep-Alive`), **representation headers coupled to body bytes the hook + cannot see** (`Content-Encoding`, `Content-Range` — relabeling + uncompressed bytes as Brotli, or stripping the encoding from compressed + bytes, corrupts the response), the `x-ts-*` namespace, and the + consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, `ts-eids`, and the other `ts-*` cookies core owns. An integration may append its own `Set-Cookie` values; it may not set or expire a reserved cookie name. Violations are rejected at the operation layer (§2) and @@ -135,9 +142,16 @@ processed documents (§6). 6. **Every row of the §3a eligibility matrix has a test** — streaming, cache-hit, pass-through, redirect, error, and 304 each proven to run or not run the hook — not merely one positive header test per adapter. -7. The cache/privacy invariant test: an integration appends a cookie and - replaces `Cache-Control` with a public/surrogate-cacheable value → the - final response is private/no-store with surrogate caching stripped. +7. Cache/privacy invariant tests, one per restriction source and shape: + cookie appended + public `Cache-Control` replacement → private/no-store, + surrogate stripped; **core-private cookieless** processed HTML + + public replacement → restriction preserved; **origin-private + cookieless** pass-through-classified content + public replacement → + restriction preserved; a cache-hit serve re-applying mutations without + weakening the stored classification; a `Vary` mutation neither + dropping core-required values nor bypassing the snapshot; each CDN + directive (`Surrogate-Control`, `CDN-Cache-Control`, host equivalents) + individually stripped; and a rejected `Content-Encoding` mutation. ## 5. Size and sequencing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index f509e8bee..f47f6673d 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -121,6 +121,10 @@ overrides. Each permission resolves to an **acquisition rule**: - `denied` — never set, even when a signal grants it. ```toml +# Illustrative schema example — NOT the shipped policy. The shipped +# example (trusted-server.example.toml) pairs these groups with the most +# protective rules.default; the permissive default below demonstrates the +# reserved key. [permissions.groups.gdpr-eu] regime = "gdpr" default = "requires_signal" @@ -139,10 +143,15 @@ default = "granted" [permissions.rules] FR = "gdpr-eu" -US = "us-opt-out" +# US privacy gating applies per configured privacy state, matching today's +# state-list behavior; country-level US traffic (a Wyoming request, or one +# whose geo provider yields no region) stays non-regulated. One US/ +# rule per configured privacy state: +"US/CA" = "us-opt-out" +US = "non-regulated" # Overrides name explicit acquisition rules — no +/- sigil syntax; TOML # expresses the target state directly. -"US/CA" = { group = "us-opt-out", overrides = { select-personalised-ads = "requires_signal" } } +"US/CO" = { group = "us-opt-out", overrides = { select-personalised-ads = "requires_signal" } } # Reserved key: countries that resolve but match no rule. Required whenever # the [permissions] section is present. Distinct from [geo] default_country, # which handles requests that resolve no country at all (§5.4). @@ -190,7 +199,9 @@ Validation rejects: the compiled-in fallback omits the section entirely); - duplicate rule keys under case-insensitive comparison (`FR` and `fr`); - a `[geo] default_country` whose country part is not an assigned ISO - code; it accepts either a country (`FR`) or a country/region key + code; it accepts either a country (`FR`) or a country/region key whose + region part is validated as an assigned subdivision exactly like rule + keys (`US/ZZ` is rejected here too) (`US/CA`) — PR #838 supported region defaults, and a no-geo, single-state deployment must be able to select its state rule. It is canonicalized to uppercase, and startup logs which rule (or @@ -213,7 +224,16 @@ The class is never inferred from purpose flags. Where the legacy lists must survive an interim period, a CI test asserts consistency between each list and the policy's regime classes, with deliberate divergences recorded as explicit, commented exceptions in the test — never silent. Both legacy -lists are in scope, not only the GDPR one. +lists are in scope, not only the GDPR one — and the US check is +region-shaped: **every configured `consent.us_privacy.states` entry must +have a matching `US/` rule**, and the country-level `US` rule must +resolve non-regulated (today applies privacy gating only to the configured +states), or the divergence is an explicit commented exception. An adapter +whose geo provider cannot resolve regions degrades **intentionally and +declaredly**: regionless US traffic hits the country rule — non-regulated, +today's behavior for non-privacy-state traffic; an operator preferring +protective country-wide gating writes `US = "us-opt-out"` as their own +declared choice. ### 3.5 Shipped-table coverage @@ -250,11 +270,11 @@ blocked but an **explicit non-opt-out** value grants: permission-scoped** — grant signals are NOT interchangeable across regimes: - | Regime of the resolved rule | Evidence accepted as a grant for a `requires_signal` permission | - | --------------------------- | --------------------------------------------------------------- | - | `gdpr` | **Only** a TCF record consenting to that specific purpose | - | `us-privacy` | TCF consent for the purpose, or an explicit GPP/USP non-opt-out | - | `none` | Any grant-class signal | + | Regime of the resolved rule | Evidence accepted as a grant for a `requires_signal` permission | + | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | + | `gdpr` | **Only** a TCF record consenting to that specific purpose | + | `us-privacy` | TCF consent for the purpose, or GPP/USP evidence **per the §4.5 field mapping** — a field grants only the permissions it maps to | + | `none` | Any grant-class signal | Without this scoping, a US-style `sale_opt_out = false` would satisfy a French `requires_signal` rule — no TCF, both purposes granted, EC minted, @@ -293,20 +313,29 @@ blocked but an **explicit non-opt-out** value grants: coexisting TCF refusal beats a non-TCF grant signal, matching today's US-state ordering where a present TCF record decides before GPP/USP values are consulted). -5. No signal — the policy baseline decides: `granted` sets it, +5. Malformed-present, no valid record of that family (§4.4) — **blocks + the baseline grant**: the permission is unset even under `granted`. + Never withdraws. +6. No signal — the policy baseline decides: `granted` sets it, `requires_signal` leaves it unset. +Normalization (§4.4) reduces each record family to exactly one of six +states — **valid-grant, valid-refusal, opt-out, malformed-present, +expired, absent** — and the precedence above plus the §4.1 matrix are +defined over those states, so no input state is unmapped. + ### 4.1 Decision matrix For each enforced permission, with baseline _B_ ∈ {granted, requires_signal, denied}: -| Opt-out present | TCF refusal present | Accepted grant present (regime-scoped) | Result | -| --------------- | ------------------- | -------------------------------------- | ------------------------------------------------ | -| yes | — | — | **unset** (and withdrawal semantics apply, §4.2) | -| no | yes | — | unset (withdrawal per §4.2, trigger 2) | -| no | no | yes | set, unless B = denied | -| no | no | no | set iff B = granted | +| Opt-out present | TCF refusal present | Accepted grant present (regime-scoped) | Malformed-present | Result | +| --------------- | ------------------- | -------------------------------------- | ----------------- | ------------------------------------------------ | +| yes | — | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | — | — | unset (withdrawal per §4.2, trigger 2) | +| no | no | yes | — | set, unless B = denied | +| no | no | no | yes | **unset** (precedence 5 — blocks baseline grant) | +| no | no | no | no | set iff B = granted | ### 4.2 Withdrawal vs. absence @@ -356,11 +385,19 @@ record that is simultaneously the durable intent, the discovery mechanism, and the fail-closed marker: - **The family revocation record is written first.** Every identity carries - a stable **family ID**, minted with the identity and stored in every - member row (including rows linked by a legacy rewrite, providers spec - §6.1). Revocation writes one record keyed by the family ID. That single - write is the withdrawal: per-member tombstones are cleanup that follows, - idempotent and retried. + a stable **family ID**: minted rows store it, and — the case that makes + or breaks the protocol — **rows that lack the field derive it + deterministically** as a function of (record kind, provider namespace, + canonical graph key), e.g. `fam:v0:hmac:`. Determinism is the + point: a withdrawal arriving on the **first post-upgrade request** — a + GPC-carrying visitor whose v1 row has no family field and has never been + backfilled — computes the same family ID that every future reader of + that row computes, so the revocation record is discoverable even if the + writer crashes before ever touching the member row. A **random** ID + would recreate the exact partial-withdrawal orphan this design exists to + eliminate. Revocation writes one record keyed by the family ID; that + single write is the withdrawal. Per-member tombstones are cleanup that + follows, idempotent and retried. - **Every consumer checks the family record, not per-member tombstones.** A reader arriving through any still-live member row finds the family ID in the row and the revocation record under it — partial revocation is @@ -377,10 +414,22 @@ and the fail-closed marker: metric feeding the operational repair path. The residual that remains — a single failed write on an otherwise healthy graph, for a user who never returns — is declared here, not hidden. +- **Consistency and retention are backend contracts**, defined in the + providers spec consistency matrix (§7): revocation-record reads use the + strongest read the backend offers, adapters declare a bounded + revocation-visibility lag (an eventually-consistent store that cannot + bound it fails startup for identity features), a **failed family-record + read fails closed** for egress (revoked-unknown ≠ live), and revocation + records are retained beyond the maximum of cookie lifetime, row TTL, + rewrite grace, and downstream retry horizon — note today's 24-hour + tombstone TTL is far below this bar and does not carry over. - Fault-injection tests cover: family-record write fails → cookie untouched, S2S behavior per degraded mode, retry completes; member tombstone N fails after the family record → identity already revoked for - every reader, cleanup retries; the same-signal retry path end to end. + every reader, cleanup retries; the same-signal retry path end to end; + **first post-upgrade request is a withdrawal** (v1 row, no family field, + derived ID; crash between family record and row write; reader of the + untouched v1 row still sees the revocation). ### 4.4 Signal normalization — normative matrix @@ -389,21 +438,55 @@ record and one effective opt-out state per request. The normalization layer is where today's real-world mess lives, and PR #838 collapsed it silently. These are the outcomes — decided here, not delegated to the implementation; each row marked **changed** also appears in the migration -matrix: - -| Input state | Effective record / outcome | Status | -| ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | -| Standalone TCF and GPP-embedded TCF disagree, mode `restrictive` | **Whole-record selection** (today's semantics — an earlier draft specified per-purpose synthesis, which is _not_ what the code does): the record whose combined P1 ∧ P4 eligibility is more restrictive governs in full | Preserved (mode semantics pinned against current tests) | -| Same, mode `permissive` | Whole-record selection: the record whose combined P1 ∧ P4 eligibility is more permissive governs in full | Preserved (same pinning) | -| Same, mode `newest` | Whole-record selection by **`LastUpdated`** (not `Created`), subject to the existing freshness threshold; a tie or inconclusive comparison falls back to the **restrictive** selection | Preserved (same pinning) | -| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | -| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | -| One valid record + one **expired** record of the same family | The valid record governs; the expired record is absent entirely (consistent with the expiry row) | Decided here | -| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | -| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | Preserved, circularity resolved | -| Proxy/mirror mode | **Consent decoding is skipped entirely** — today's behavior, preserved as-is; no mirror-sourced record is synthesized (an earlier draft invented one). Retiring proxy mode, if ever wanted, is its own declared change | Decided here | -| GPP opt-out fields | Normative, not deferred: in the US-National section, `SaleOptOut`, `SharingOptOut`, and `TargetedAdvertisingOptOut` each independently constitute an opt-out signal when set to opted-out; each supported US state section maps its correspondingly named fields identically; a field explicitly set to not-opted-out is grant-class evidence (§4, regime-scoped); absent or N/A fields contribute nothing; **unsupported sections contribute nothing** (neither grant nor revoke). Adding a section is a spec change to this row | Decided here | -| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | +matrix. The pipeline order is itself normative: **(1) syntax validation +per source, (2) expiry per source — expired sources drop to absent +_before_ conflict resolution, (3) conflict resolution over the remaining +valid sources.** Current runtime resolves conflicts first and can select +an expired record before clearing both sources; expiry-first is a +**declared change** (migration matrix) that removes that path: + +| Input state | Effective record / outcome | Status | +| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| Standalone TCF and GPP-embedded TCF disagree, mode `restrictive` | Whole-record selection over the **(P1, P4) outcome tuple, compared lexicographically with P1 first** (refusal < grant): the lesser tuple governs in full. Split-purpose records are thereby decided — (grant P1, refuse P4) vs (refuse P1, grant P4) selects the latter. Identical tuples → outcomes are identical; the GPP-embedded record is named for determinism. (An earlier draft specified per-purpose synthesis, which is _not_ what the code does; the tuple order is decided here and pinned against current tests) | Preserved — pinned against current tests | +| Same, mode `permissive` | Same tuple comparison; the **greater** tuple governs in full | Preserved — same pinning | +| Same, mode `newest` | Whole-record selection by **`LastUpdated`** (not `Created`), subject to the existing freshness threshold; a tie, an incomparable pair, or timestamps inside the threshold fall back to the fully deterministic `restrictive` rule above | Preserved — same pinning | +| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | +| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | +| One valid record + one **expired** record of the same family | The valid record governs — the expired one dropped at pipeline step 2, before conflict resolution ever saw it | **Changed (declared)** — current runtime resolves the conflict first and can select the expired record | +| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | +| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | Preserved, circularity resolved | +| Proxy/mirror mode | **Syntax validation still runs; semantic decoding is skipped.** A present record (well- or mal-formed) is _present, undecoded_: it blocks grants (a record TS will not read cannot vouch for consent) and never withdraws; absent → baseline. Header-carried opt-outs (GPC) are unaffected — they need no decoding. Without the syntax pass, malformed-present would be indistinguishable from absent, contradicting the fail-closed rule below | **Changed (declared)**: today proxy mode skips decoding entirely, which under a permissive baseline is fail-open | +| GPP / US Privacy fields | Per the normative field mapping of §4.5 — fields are not interchangeable signals, and absent/N-A fields grant nothing | Decided here (§4.5) | +| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | + +### 4.5 US signal field mapping — normative + +GPP and US Privacy fields map to specific permissions with specific +effects; they are never interchangeable, a field's absence or N/A value +contributes nothing, and only the fields marked destructive trigger +withdrawal. Section IDs and versions are those of the IAB GPP +specification current at implementation time; adding a section or field is +a change to this table. + +| Source · field | Value | `store-on-device` (P1) | `select-personalised-ads` (P4) | Destructive withdrawal? | +| -------------------------------------------- | ------------- | ---------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------ | +| GPP US section · `SaleOptOut` | opted out | opt-out | opt-out | **Yes** (preserves today) | +| GPP US section · `SaleOptOut` | not opted out | grant | grant | — | +| GPP US section · `SharingOptOut` | opted out | — | opt-out | No | +| GPP US section · `SharingOptOut` | not opted out | — | grant | — | +| GPP US section · `TargetedAdvertisingOptOut` | opted out | — | opt-out | **No** — a targeted-advertising choice must never destroy the stored identity | +| GPP US section · `TargetedAdvertisingOptOut` | not opted out | — | grant | — | +| US Privacy · `opt_out_sale` | `Y` | opt-out | opt-out | **Yes** (preserves today) | +| US Privacy · present, `N` or N/A | — | grant | grant | — (today's tests pin N/A as allowing; USP carries no distinct targeted-advertising field, so it never maps to one) | +| Any field | absent / N-A | — | — | — | + +**Multi-section aggregation:** when both a national and an applicable +state section are present, the state section governs for the fields it +carries; across whatever sections apply, **an opt-out in any applicable +section beats a grant in another** (restrictive aggregation). `SharingOptOut` +and `TargetedAdvertisingOptOut` are new enforcement inputs — current code +consults only the sale field — and are declared as such in the migration +matrix. ## 5. Jurisdiction resolution @@ -533,10 +616,17 @@ Consumers of the resolved set in this epic: **S2S authority (batch sync).** A context-free server-to-server request carries no user signals, geo, or `EcContext`. Its authority is the identity's **stored provenance**: per-permission, time-bounded evidence - written at mint and refreshed on later live requests — grant basis - (which signal class granted, per permission), evidence timestamp, + written at mint and replaced on later live requests — grant basis + (which signal class granted, per permission), the evidence's + **authoritative timestamp and `valid_until`** (per evidence class), resolved jurisdiction, policy revision, and provider/version (providers - spec §6.1). A sync request performs a **full recompute of both + spec §6.1). Two aging rules prevent perpetual renewal: **re-presenting + an unchanged signal does not reset evidence age** — only a record + carrying a newer authoritative timestamp (e.g. TCF `LastUpdated`) does; + and every live resolution **atomically replaces the complete + per-permission snapshot**, never merges — a refusal, opt-out, malformed + or absent state in the fresh resolution clears prior positive authority + for its scope, so an old P4 grant cannot survive a later P4 refusal. A sync request performs a **full recompute of both permissions** from that stored evidence against the _current_ policy: it fails closed when the stored jurisdiction's rule is now `denied`, when a `granted` baseline tightened to `requires_signal` and the stored @@ -556,12 +646,12 @@ Consumers of the resolved set in this epic: 4. **Server-side auction dispatch** — gated on the policy `regime` class, normatively: - | Regime | Dispatch rule | Preserves | - | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | - | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | - | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | - | `none` | Dispatch proceeds. | Today's non-regulated arm | - | **Any regime, decodable TCF record present** | The `gdpr` row applies: dispatch requires that record to consent to Purpose 1 — a raw TCF signal makes the request GDPR-relevant regardless of geolocation, so a US or non-regulated request carrying a Purpose 1 refusal is blocked. | Today's raw-signal arm — **must not regress** | + | Regime | Dispatch rule | Preserves | + | ------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | + | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | + | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | + | `none` | Dispatch proceeds. | Today's non-regulated arm | + | **Any regime, raw TCF signal present** — a TC string on the request or a GPP section-2 hint, detected **before decoding** | The `gdpr` row applies: dispatch requires the _effective_ record to be decodable, unexpired, and consenting to Purpose 1. A **malformed or expired** raw signal therefore blocks dispatch — today a malformed raw TCF blocks, and gating this arm on decodability would have silently relaxed that. A US or non-regulated request carrying a Purpose 1 refusal is likewise blocked. | Today's raw-signal arm — **must not regress** | The **compiled-in fallback policy has `regime = "gdpr"`** (§3.1) — the no-policy posture must be the most protective for dispatch too, and a @@ -591,9 +681,15 @@ further consumer if and when it proceeds. (consent, opt-out, malformed, expired, absent), including the no-policy fallback regime, asserting both the dispatch decision and that a blocked dispatch emits no outbound request. -- The §7 S2S authority path: sync against stored provenance, including - the policy-tightened-to-denied case (no update, flagged for cleanup) - and the exempt consent-state lookup. +- The §7 S2S authority path: **every denial reason individually** — + denied rule, tightened baseline without acceptable stored evidence, + expired evidence, regime-rejected grant source — plus the exempt + consent-state lookup, stale-evidence re-presentation (age must not + reset), and legacy-row fail-closed-then-backfill. +- The full cross-product **regime × permission × evidence source** from + §4's acceptance table and §4.5's field mapping, including multi-section + aggregation conflicts. +- Legacy-row withdrawal end to end (§4.3's derived family ID). - §4.3 fault-injection cases. - Policy validation tests for every §3.3 rejection, exercised through both acceptance paths (push-time and startup). diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index cb17b604d..8c934bc31 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -213,9 +213,24 @@ cookie write, no egress, no auction use may observe a minted identifier before its graph row (with provenance, §6.1) has committed — PR #838 let a generated EC reach an auction before finalization refused the cookie, producing an identity that existed for one request and nowhere else. The -normative order is: gate → `generate` → graph-row commit → cookie write → -eligible for egress. A graph-commit failure means the mint never happened: -no cookie, no egress, error logged, the next request retries. +normative order is: gate → `generate` → graph-row commit → cookie +scheduled → eligible for egress. "Cookie scheduled" means queued onto the +final response — `Set-Cookie` is physically emitted after first-request +processing, so egress eligibility begins at **graph commit**, not at +header emission; the identity exists durably from that moment. A +graph-commit failure means the mint never happened: no cookie, no egress, +error logged, the next request retries. + +**Egress is typed, not policed.** The inventory-and-denylist test +(permission model spec §7) is a backstop, but conventions do not survive +new code — the ungated proxy/click/Testlight paths happened precisely +because raw EC values circulate as ordinary strings. Core therefore +introduces an **`AuthorizedIdentity`** newtype constructible only by core, +only after parse + permission gate + graph/family-revocation check; +outbound serializers (ORTB builder, page bids, sync, identify, forwarding) +accept `AuthorizedIdentity`, never `&str`/`EcId`. A future bypass then +requires deliberately reconstructing the raw string — visible in review — +rather than passing along what was already in hand. The gate applies to EC providers **only**. Geo and device are ungated for two _different_ reasons, stated separately because only one of them is @@ -244,10 +259,8 @@ structural: fingerprint-derived buyer-facing fields into new rows** (a declared change, migration spec §2); the boolean security classification outcome may be persisted. Re-adding them is the vocabulary-extension route. - Relatedly, the implementation PR must deliver a **field-level graph - contract table** — for every persisted row field: purpose, source, - gating permission, TTL, rewrite behavior, egress paths, and tombstone - scrubbing — reviewed against the egress inventory. + The field-level graph contract itself is normative in this spec — + §6.3 — not deferred to the implementation. PR #838 declared `required_permissions` on all three traits but consulted it only for the EC provider; the geo and device declarations were @@ -317,19 +330,26 @@ The contract: active writer is a per-deployment choice (`[ec] rewrite_legacy = true|false`), and re-minting is subject to the full minting gate of §5. -- **Rewrite is transactional, linking, and confirmed by presentation.** - Order: new row commits first, carrying a link to the old row (sharing - its revocation family ID, permission model spec §4.3) and a copy of the - old row's consent metadata and partner mappings; then the new cookie is - emitted. The server only emits `Set-Cookie` — it cannot observe delivery - or acceptance, so **both linked rows stay live** until a later request - **presents the new cookie** (confirmation by presentation); only then is - the old row retired. A deployment may additionally cap the window with a - grace period no shorter than the old cookie's maximum lifetime plus - rollout skew. An interrupted or unconfirmed rewrite leaves the old - cookie fully valid — no state in which neither identity works. - **Withdrawal of either linked row revokes the shared family, i.e. - both.** +- **Rewrite aliases to one canonical row — no dual-write window.** Order: + the new canonical row commits first, sharing the old row's revocation + family ID (permission model spec §4.3); its provenance is the **current + live resolution** (rewrite happens on a live request — copying old + consent evidence would rejuvenate stale authority), while partner + mappings copy **with their original per-field timestamps and expiry**. + Then a fenced CAS **replaces the old row with an alias record** pointing + at the canonical row; from that moment every read or update through + either cookie chases the alias (single hop) to the one canonical row — + a concurrent pull/batch/identify update cannot land on a row about to + be discarded, because after the CAS there is only one row to land on, + and an update racing the CAS itself retries against the canonical. Then + the new cookie is emitted. The server cannot observe `Set-Cookie` + acceptance, so the **alias stays live** until a later request presents + the new cookie (confirmation by presentation), and in any case until a + **finite retirement deadline** no shorter than the old cookie's maximum + lifetime plus rollout skew. An interrupted rewrite leaves the old + cookie resolving (directly or via the alias) — no state in which + neither identity works. **Withdrawal through either cookie revokes the + shared family.** - Retiring a legacy reader is the explicit end of those identities: the migration guide documents the cleanup procedure (migration spec §6). - Tests: switch active provider → request with old cookie → identity still @@ -361,6 +381,43 @@ when a healthy configuration meets an unhealthy runtime. Every row logs at | Geo provider returns invalid output (unparseable country) at runtime | Treated as lookup failure → `default_country` (permission model spec §5.2), counted in the lookup-failure metric | | Device provider signals unavailable at runtime (e.g. no JA4 on a request) | Classification degrades per the provider's declared fallback, never silently upgrades `looks_like_browser` | +The **degraded-graph health signal** referenced above and by the +withdrawal contract is a defined state machine, not a vibe: it is +**per-instance and in-memory** (no shared propagation, no stored health +record whose own read could fail), entered when graph-write failures cross +a sliding-window threshold (N failures within window W), and exited with +hysteresis after M consecutive successes. While degraded: S2S partner +egress and sync updates fail closed; organic requests continue stateless. +The thresholds ship as constants with the implementation and are printed +in the startup log. + +### 6.3 Graph row contract — normative + +The per-field contract for identity rows, covering today's v1 fields and +the fields this epic adds. Serialization is JSON with the existing `v` +schema-version discriminator; from release N+1 onward (migration spec §4), +readers round-trip unknown keys **semantically** (values preserved through +read-modify-write; byte-identical output is not required and not +achievable through a structured serializer). + +| Field | Purpose | Source | Gating permission (egress) | TTL / refresh | Rewrite | On revocation | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| key (v1: identifier verbatim; v2: core-constructed, §4) | Row identity | Provider/core | — | Row TTL (1 y today) | New canonical row; old key becomes alias | Family record governs; member tombstone as cleanup | +| `v` | Schema discriminator | Core | — | — | Written at current version | Retained | +| `created` | Row age | Core | P1 (first-party ops) | Never refreshed | Preserved (no rejuvenation) | Retained in tombstone | +| `consent.tcf` / `consent.gpp` | Raw signal snapshot for audit; superseded as authority by provenance | Request | Never egressed to partners | Replaced on live resolution (§7 snapshot rule, permission spec) | Fresh live values | Scrubbed | +| `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | +| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, provider/version) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | +| New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | +| `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | +| `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | +| `pub_properties` (origin/seen domains) | Creation context | Core at mint | P1 | Write-once | Preserved | Scrubbed | +| `device.*` (JA4 class, H2 hash, quality metadata) | **Discontinued for new rows** (§5): fingerprint-derived, buyer-facing — beyond security-classification authorization. v1 rows retain them read-only; they are never egressed post-epic and are dropped at rewrite | Fastly device provider | None grants egress | Write-once (v1) | **Dropped** | Scrubbed | +| New: security classification outcome (boolean) | Bot-gate result | Device provider | — (never egressed) | Written at mint | Fresh | Scrubbed | +| `network.*` | Cluster disambiguation | Platform at mint | P1 | Write-once | Fresh | Scrubbed | +| `ids` (partner → UID map) | Partner identity graph | Pixel/pull/batch sync | P1 ∧ P4 (partner egress) | Per-mapping timestamps; bounded count/length | Copied **with original timestamps/expiry** | Scrubbed | +| New: alias record kind | Rewrite indirection (§6.1) | Core | — | Retirement deadline | Is the mechanism | Family-revoked like any member | + ## 7. Composition root and adapter parity Provider construction happens in exactly one place per concern @@ -373,16 +430,25 @@ outcomes. Requirements: -- **Adapters declare capabilities against an explicit matrix.** The - capability set the composition root checks selections against is - enumerated, not ad hoc: identity-graph persistence, atomic single-key - reservation (CAS — required by the client-cycle reservation and any - future compare-and-set use), KV prefix listing (cluster support), - platform geo, device host evidence (JA4/HTTP-2), and legacy-rewrite - support. Each adapter's declaration is part of its wiring, and the §6 - capability-mismatch startup error is driven by this matrix. Every §6.2 - runtime-failure row gets fault-injection coverage on every adapter that - declares the corresponding capability. +- **Adapters declare capabilities against an explicit matrix — with + consistency semantics, not just feature bits.** The capability set: + identity-graph persistence, atomic single-key reservation, KV prefix + listing (cluster support), platform geo, device host evidence + (JA4/HTTP-2), and legacy-rewrite support. Persistence capabilities carry + **per-record-class consistency requirements**, because "has KV" says + nothing about whether revocation is observable: + + | Record class | Required semantics | + | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | + | Family revocation records | Strongest read the backend offers, plus a **declared, bounded revocation-visibility lag** (Workers KV documents up to ~60 s eventual propagation — that bound must be declared, and the residual it implies stated in operator docs). Unboundable lag → startup failure for identity features. A **failed or erroring revocation-record read fails closed** for egress | + | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | + + Each adapter's declaration is part of its wiring, drives the §6 + capability-mismatch startup error, and every §6.2 runtime-failure row + gets fault-injection coverage on every adapter declaring the + corresponding capability. + - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in app state; PR #838 rebuilt the provider (cloning the secret into a fresh diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 99ed06ef7..eaf6bb133 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -40,6 +40,10 @@ discoverable only because a deleted test had pinned the old behavior. | 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | | 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | | 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | +| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — strictly more protective | +| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | | 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | | 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | | 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | @@ -129,15 +133,26 @@ Requirements: rewrite links — and two failure modes must be engineered away: a naive schema-version bump makes old readers fail closed on new rows, and an old worker that reads, modifies, and reserializes a row **silently - drops** fields it does not model. Sequence: (a) a **reader/preserver - release** ships first — it understands the new fields and, critically, - preserves unknown fields verbatim through read-modify-write; (b) a - **fleet-convergence gate**; (c) only then does **writer activation** - begin emitting the new fields. Rows carry an explicit schema version; - backfill is lazy via live requests (the same pass that backfills legacy - provenance, permission spec §7). Mixed-version tests are mandatory: - old-reader/new-row, new-reader/old-row, and old-worker - read-modify-write preserving new fields byte-for-byte. + drops** fields it does not model. The sequence shares the config + release names: **N+1 is the reader/preserver release** — it understands + the new fields and preserves unknown keys **semantically** through + read-modify-write (values round-trip; byte-identical JSON is neither + required nor achievable through a structured serializer — and a + genuinely pre-N+1 worker cannot preserve at all, which is exactly why + the floor exists); after the **fleet-convergence gate**, **N+2 + activates the writer** and begins emitting the new fields. **Rollback + below N+1 is prohibited once any new-format row exists** — a pre-floor + binary would silently strip the new fields from every row it touches. + Rows carry the existing `v` schema discriminator; backfill is lazy via + live requests (the same pass that backfills legacy provenance, + permission spec §7) — and, critically, **withdrawal never depends on + backfill**: the family ID for an untouched v1 row is derived + deterministically (permission spec §4.3), so a first-post-upgrade + GPC request withdraws correctly with zero migrated state. Mixed-version tests with stated expected results: + N+1-reader/old-row → full function; old-reader/new-row → v1 semantics, + new fields untouched if read-only, preserved semantically if + read-modify-write on N+1, **test-proven lost on pre-N+1** (documenting + why the floor is a floor); N+2-reader/N+1-written-row → full function. 3. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the @@ -188,12 +203,16 @@ declares `[permissions.rules]`, and reopening a TOML table is a parse error; a prose delta cannot be validated, a committed fixture can.) The fixture contains, in one document: -- `[ec] provider = "hmac"` with its passphrase block; +- `[ec] provider = "hmac"` with its passphrase block, **and the + identity-graph store configuration** — selecting a minting provider + without an openable graph store is a startup error (providers spec §6), + so a fixture omitting it would not start; - `[device] provider = "fastly"` (Fastly deployments: preserves the JA4 bot gate); - `[geo] provider = "platform"` and `default_country = "FR"` (per-request - jurisdiction detection preserved; the default is fail-closed because FR - resolves to the `gdpr-eu` rule); + jurisdiction detection preserved; the FR default is a **protective + opt-in fallback**, not fail-closed — valid TCF consent still grants, + where today's unresolved-geo path always denies); - (Fastly fixture; other adapters substitute their valid selections) the full `gdpr-eu` / `gdpr-uk` / `us-opt-out` groups and country rules from the example policy (US as `requires_signal` with the grant-signal @@ -238,11 +257,26 @@ global honoring of opt-out signals is unconditional. is not churn), and a nonzero rewrite-failure rate blocks retirement outright. The telemetry set also includes: graph read/commit failures, stored-provenance denials, schema-migration failures, and - replay-reservation recoveries. + replay-reservation recoveries. **Each rollout-gate metric ships with a + threshold, an evaluation window, and a named action** (pause rollout / + roll back / block retirement) in the migration guide — a metric with a + "healthy range" but no action is dashboard decoration; the two already + specified (legacy-reader quiet period, rewrite failures) are the + pattern the rest follow. 3. Startup logs always print: selected provider per concern, whether geo is live, the effective default baseline, and the count of granted-without- signal permissions. One line, greppable, stable format. -4. Rollback is config-only where possible: reverting to the previous +4. **The batch-sync coverage dip is operationalized, not discovered.** + Because legacy rows fail closed for batch updates until backfilled + (permission spec §7), batch-sync acceptance drops toward zero at + cutover and recovers along the live-traffic backfill curve. The + **provenance-coverage metric** (share of active rows carrying + provenance) is the tracking signal; the migration guide states the + expected recovery shape, tells operators to notify batch-sync partners + of the transient rejection rate, and defines no fail-open shortcut — + the alternative (grandfathering pre-epic identities past the + permission model) is rejected in the permission spec. +5. Rollback is config-only where possible: reverting to the previous config version restores the previous behavior on the previous binary. The one irreversible artifact is withdrawal tombstones — which is why the withdrawal triggers (permission spec §4.2) are exhaustive, why partial From de70ca931138ed21f7d7a3a44fe9c70aecac3f94 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 14:20:41 -0700 Subject: [PATCH 07/14] Address fifth review: opt-out subclasses, GPP applicability, consistency eligibility, and rollout closure P1 fixes: - Opt-out signals split into destructive (GPC, sale, USP - withdraw) and non-destructive (sharing, targeted-advertising - revoke P4 only, never tombstone) subclasses assigned by the 4.5 mapping, resolving the 4.2-vs-4.5 contradiction. - Proxy mode performs minimal opt-out extraction (the 4.5-mapped fields and USP only) so globally authoritative opt-outs are never suppressed; still no record-derived grants; declared as a change from today's opt-out-blind skip. - GPP applicability is an ordered algorithm with a pinned jurisdiction -> section-ID map (usnat 7, usca 8, usva 9, usco 10, usut 11, usct 12): applicability from resolved jurisdiction, state-over-national per field, restrictive aggregation; foreign and non-applicable sections contribute nothing; N/A preserved as not-opted-out (declared, correcting the earlier contributes-nothing rule). - TCF conflict selection reverts to today's algorithm - P1-and-P4 conjunction comparison with standalone winning equal conjunctions (including split-purpose) - replacing the invented lexicographic tuple and keeping the Preserved label honest. - S2S freshness is a per-evidence-class contract: TCF ages by LastUpdated, GPP/USP by first-seen with an equality digest (re-presentation keeps original first-seen), baseline grants re-derive from the current policy revision; clock-skew clamping. - Degraded-mode protection is declared local-only with the cross-instance residual quantified (bounded by user return latency, metered), instead of implying fleet-wide fail-closed. - Family revocation records require a strongly consistent primitive; Workers KV is explicitly ineligible ('60 seconds or more' is not a bound); alias/rewrite records join reservations in the linearizable CAS class and rewrite_legacy is rejected without it. - The trait now really returns graph_key_suffix (a round-4 batch loss), core owns the 6.3 physical key grammar (id/, alias/, fam/, rwx/, resv/ prefixes + reserved legacy grammar) with wire schemas and TTLs per record class. - HMAC version attribution really resolves from immutable row provenance (also a round-4 batch loss); parse identifies namespace only. - AuthorizedIdentity is scope-parameterized (GraphOps vs PartnerEgress) so a P1-only identity cannot reach an ORTB serializer. - The provider contract gains acquisition modes (ServerMint / ClientResolve carrying resolve_from_client and the JS module id); the resolve endpoint enforces the provider's full required_permissions. - Revocation-wins at the resolve endpoint holds because the family check runs through the linearizable class client-cycle already requires. - Rewrite is a persistent fenced transaction: pinned target on retry, reconciliation of updates that won the old-row CAS, orphan GC by absent transaction, and fenced alias retargeting keeping chains single-hop. - Release protocol: rollback is binaries-first (N+2 -> N+1 keeping the new config); N+1 rejects provider/version selections it cannot encode; a pre-N+1 graph-store readiness step plus matrix row 12 covers graphless HMAC deployments (breaking, declared). - The abstract capability list becomes a concrete adapter matrix (Fastly/Axum/Cloudflare/Spin) with honest cells - including that Cloudflare supports platform geo country-only (the migration text claiming it rejects platform geo was wrong) and that no CAS-class primitive is currently wired anywhere but the dev adapter. - Integration cookies enter the permission model: registration-declared names with purpose and retention, persistent cookies gated on store-on-device, session cookies as the narrow exemption. - Cache monotonicity is a defined lattice (no-store > no-cache > private > public, shrink-only ages, snapshot-gated stale directives, protected Vary union) in the contract, not the tests. P2/P3: persisted-KV consent now flows through the full pipeline (declared change); provenance transition and mid-replacement fault tests; the recipe renamed minimal-divergence with its unavoidable divergences enumerated; batch-sync coverage is a gated stage with thresholds and a pause action; policy-revision activation defined (stamped revisions, bounded mixing, no tombstone resurrection); representation surface extended (Content-Type, ETag, Last-Modified, Accept-Ranges, digests); append restricted to list-valued headers; exact budgets and snapshot read semantics; reservation namespacing and per-state ownership conflicts; media-type matching ignores parameters; the pass-through test wording fixed; and a product-decision sign-off list (9 items) added to the migration spec for explicit maintainer ratification. --- ...26-07-30-client-cycle-ec-resolve-design.md | 42 +++- ...integration-response-header-hook-design.md | 60 ++++-- .../2026-07-30-permission-model-design.md | 126 ++++++++---- .../2026-07-30-pluggable-providers-design.md | 186 +++++++++++++----- ...07-30-provider-migration-rollout-design.md | 113 ++++++++--- 5 files changed, 398 insertions(+), 129 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 2f27f1ab6..1eeb61e14 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -61,7 +61,11 @@ Everything in this spec follows from that. enough to set identity. Requests with no `Origin` and no valid token are rejected. 2. **Verify the payload cryptographically per provider — including against - replay.** The provider's `resolve_from_client` accepts only payloads + replay.** `resolve_from_client` is the client-resolve acquisition mode + of the provider contract (providers spec §4 + `Acquisition::ClientResolve`, which also carries the JS module the + page leg needs) — no longer an undeclared method this spec invents. It + accepts only payloads that are signed by an expected party, **audience-bound** to this publisher, and **expiring**. Audience binding and expiry alone do not mitigate replay — a captured token installs in another browser for the @@ -85,14 +89,20 @@ Everything in this spec follows from that. wiring; the parity suite asserts the endpoint's presence and behavior on all four adapters. (PR #838 registered it on Fastly only, so the same config on the Axum dev server proxied the POST to the publisher origin.) -6. **Be uncacheable and permission-gated.** `Cache-Control: no-store`; - the same `store-on-device` permission gate as organic EC creation runs - before any cookie is set. +6. **Be uncacheable and permission-gated on the provider's full + declaration.** `Cache-Control: no-store`; before any cookie is set, + the endpoint enforces the selected provider's complete + `required_permissions()` — not a hard-coded `store-on-device` check; a + client-resolve provider declaring more than P1 gets all of it + enforced, exactly as organic minting does (providers spec §5). 7. **Bound every input — exact values, testable boundaries.** Request body: at most **65,536 bytes** (inclusive; byte 65,537 → `413`), enforced by a bounded read independent of `Content-Length` — a missing, false, or chunked length must not bypass it. `Content-Type` - allowlist: `text/plain` and `application/json`; anything else → `415`. + allowlist: `text/plain` and `application/json`, matched on the media + type alone — case-insensitively, ignoring parameters, so the browser's + default `text/plain;charset=UTF-8` passes; duplicate `Content-Type` + headers → `400`; anything else → `415`. The resulting identifier: at most **256 bytes**, cookie-safe alphabet (providers spec §3 global bounds); violation → `400`. Reservation key (payload unique id): at most **128 bytes**. Status codes are part of @@ -124,8 +134,16 @@ Everything in this spec follows from that. its epoch no longer matches. `failed` is retryable: the same owner may supersede it with a fresh `pending` at a higher epoch. The graph write happens under the reservation and is deterministic under its key, so - any retry converges on the same row; reservations are retained at least - through the token's expiry. + any retry converges on the same row; reservations are retained at + least through the token's expiry. Reservation keys are **namespaced** + per the providers spec §6.3 grammar + (`resv////`), so + payloads cannot collide across publishers, providers, or versions. + Ownership conflicts are terminal per state: a non-owner hitting + `pending` gets `409` (retry only after lease expiry); a non-owner + hitting `committed`/`failed` gets the no-cookie terminal response; an + owner hitting `failed` may supersede it (higher epoch); cleanup + deletes reservations after retention, never before token expiry. **A duplicate must never receive the cookie unless it proves the original session binding.** "Duplicates observe the recorded outcome" @@ -139,9 +157,13 @@ Everything in this spec follows from that. explicitly-accepted at-most-once posture (no owner hash), a lost response is an **orphan row** handled by the specified cleanup path. The **same-identity no-op of §3.8 first checks the family revocation - record** (permission model spec §4.3): a resolve against a revoked - family is rejected, never refreshed, and a create racing a revocation - loses — revocation wins. Tests cover crash-between-steps, lease + record** (permission model spec §4.3), and it does so **through the + linearizable primitive class this feature already requires** for + reservations (providers spec §7 matrix) — which is what makes + "revocation wins" true rather than aspirational: on an eventually + consistent read, a racing create could observe a stale absence and + emit a cookie for a revoked family. A resolve against a revoked family + is rejected, never refreshed, and a create racing a revocation loses. Tests cover crash-between-steps, lease takeover with a stale-epoch commit attempt, two concurrent requests with the same payload, a duplicate from a second client receiving no cookie, owner-hash recovery receiving the cookie, and diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 8ec5b687d..23d1241f2 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -61,9 +61,17 @@ mutators to the outbound response for HTML document responses it processed. merely passed through) — and the post-hook response may only be **equal or stronger** on the privacy axis: integrations can tighten caching, never loosen it, regardless of which header they replaced. - Every CDN/surrogate cache directive (`Surrogate-Control`, - `CDN-Cache-Control`, host-specific equivalents) is stripped from any - restricted response. Middle-stage placement also keeps + "Equal or stronger" is a defined merge, not a vibe: restriction + strength is ordered `no-store` > `no-cache` > `private` > `public`, + and the final value per axis is the **stronger of snapshot and + mutation**; `max-age`/`s-maxage` may only shrink relative to the + snapshot; `stale-while-revalidate`/`stale-if-error` may appear only if + the snapshot had them; every CDN/surrogate directive + (`Surrogate-Control`, `CDN-Cache-Control`, host-specific equivalents) + is stripped from any restricted response; and **core-required `Vary` + members are protected in the contract, not just the tests** — the + final `Vary` is the union of the snapshot's required members and the + mutation. Middle-stage placement also keeps the earlier property: an integration mutation is not silently stripped by ordinary core handling — only by the invariant pass, which logs the downgrade it applies. @@ -75,18 +83,31 @@ mutators to the outbound response for HTML document responses it processed. _names_ — HTTP framing and hop-by-hop headers (`Content-Length`, `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, `Keep-Alive`), **representation headers coupled to body bytes the hook - cannot see** (`Content-Encoding`, `Content-Range` — relabeling - uncompressed bytes as Brotli, or stripping the encoding from compressed - bytes, corrupts the response), the `x-ts-*` namespace, and the + cannot see** (`Content-Encoding`, `Content-Range`, `Content-Type`, + `ETag`, `Last-Modified`, `Accept-Ranges`, and digest headers — + relabeling uncompressed bytes as Brotli, or advertising a validator or + digest for bytes the hook never saw, corrupts responses or poisons + caches), the `x-ts-*` namespace, and the consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, - `ts-eids`, and the other `ts-*` cookies core owns. An integration may - append its own `Set-Cookie` values; it may not set or expire a reserved - cookie name. Violations are rejected at the operation layer (§2) and + `ts-eids`, and the other `ts-*` cookies core owns. Integration cookies are **inside the permission model, not beside it** + (product sign-off item 9, migration spec §8) — otherwise the hook is a + door around the EC gate: an integration could write a durable + identifier while `store-on-device` is denied. `append_set_cookie` + therefore requires the cookie name to be **declared at registration** + with a stated purpose and maximum retention; a **persistent** cookie + (any `Max-Age`/`Expires`) is applied only when the request's resolved + permissions include `store-on-device`, while **session cookies** (no + persistence attributes) are the narrow, documented exemption. + Undeclared cookie names are rejected like reserved ones. An integration + may never set or expire a reserved cookie name. Violations are rejected at the operation layer (§2) and logged at `warn` with the integration id. The reserved lists are single constants next to the definitions they protect, not duplicated in the hook. - For non-reserved headers, the mutator API distinguishes **append** from - **replace** explicitly; the default is append (for `Set-Cookie`, append is + **replace** explicitly; **append is valid only for genuinely + list-valued headers** (a singleton header accepts only replace — two + values of a singleton header by append is a malformed response, not a + merge); the default is append where legal (for `Set-Cookie`, append is the only non-reserved operation — replace is not offered). Replacing a header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, @@ -95,10 +116,15 @@ mutators to the outbound response for HTML document responses it processed. `Set-Cookie` header name outright — cookies go only through `append_set_cookie`, so its validation cannot be bypassed by spelling the header name in a generic op. Per-integration limits bound total - operations, added header count, and added header bytes, and a - **cumulative final-response budget** (total header count and bytes) - bounds the sum across integrations — enforced in registration order, so - which operations are rejected when the budget trips is deterministic. + operations (≤ 32), added headers (≤ 16), and added bytes (≤ 8 KiB), and + a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, + counting `name: value` plus separators, within any lower adapter + ceiling) bounds the sum across integrations — enforced in registration + order, so which operations are rejected when a budget trips is + deterministic. Each mutator receives an **immutable snapshot of the + response head** (status and headers as of its turn, prior integrations' + accepted operations applied) as its read context; it never holds a + mutable reference (§2). Exceeding a limit rejects the excess operations (logged, attributed), never the response. A mutator that returns an error is skipped in full — its operations are all-or-nothing — and the response proceeds without it. @@ -145,9 +171,9 @@ processed documents (§6). 7. Cache/privacy invariant tests, one per restriction source and shape: cookie appended + public `Cache-Control` replacement → private/no-store, surrogate stripped; **core-private cookieless** processed HTML + - public replacement → restriction preserved; **origin-private - cookieless** pass-through-classified content + public replacement → - restriction preserved; a cache-hit serve re-applying mutations without + public replacement → restriction preserved; **origin-private cookieless** processed HTML that retained the + origin's cache restrictions + public replacement → restriction + preserved (pass-through responses never run the hook, §3a); a cache-hit serve re-applying mutations without weakening the stored classification; a `Vary` mutation neither dropping core-required values nor bypassing the snapshot; each CDN directive (`Surrogate-Control`, `CDN-Cache-Control`, host equivalents) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index f47f6673d..6154de7c7 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -251,9 +251,12 @@ Signals are classified into three classes — a two-class model (TCF grant / opt-out) cannot reproduce today's US behavior, where no-signal traffic is blocked but an **explicit non-opt-out** value grants: -- **Opt-out signals** (affirmative withdrawal): GPC header; GPP sections - carrying a sale/sharing opt-out; US Privacy opt-out. Opt-out signals are - honored **globally**, not only in the jurisdictions whose law defines +- **Opt-out signals**, in two subclasses assigned by the §4.5 mapping: + **destructive** opt-outs (GPC; sale opt-outs; USP opt-out) revoke and + trigger withdrawal; **non-destructive** opt-outs (sharing, + targeted-advertising) revoke the permissions they map to but never + destroy the stored identity — a targeted-ads choice must not tombstone. + Both subclasses are honored **globally**, not only in the jurisdictions whose law defines them — a deliberate, more-protective simplification: scoping a browser's explicit opt-out to a geolocation guess would honor it for some visitors and ignore it for others based on IP evidence. (For jurisdictions outside @@ -347,9 +350,12 @@ group label, since a group can mix rules across permissions. The triggers, exhaustively — nothing else withdraws: -1. **An opt-out signal withdraws in every jurisdiction, whatever the - baseline.** (For US states this preserves today's behavior; elsewhere it - is the declared change of §4's global-opt-out rule.) +1. **A destructive opt-out signal (per §4.5's destructive column: GPC, + sale opt-outs, USP opt-out) withdraws in every jurisdiction, whatever + the baseline.** Non-destructive opt-outs (sharing, + targeted-advertising) never trigger this — they revoke acquisition + only. (For US states this preserves today's behavior; elsewhere it is + the declared change of §4's global-opt-out rule.) 2. **A TCF record refusing `store-on-device` withdraws iff the baseline is `requires_signal` or `denied`.** Where the baseline is `granted`, refusal blocks _new_ grants but never tombstones: tombstones are @@ -445,19 +451,19 @@ valid sources.** Current runtime resolves conflicts first and can select an expired record before clearing both sources; expiry-first is a **declared change** (migration matrix) that removes that path: -| Input state | Effective record / outcome | Status | -| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| Standalone TCF and GPP-embedded TCF disagree, mode `restrictive` | Whole-record selection over the **(P1, P4) outcome tuple, compared lexicographically with P1 first** (refusal < grant): the lesser tuple governs in full. Split-purpose records are thereby decided — (grant P1, refuse P4) vs (refuse P1, grant P4) selects the latter. Identical tuples → outcomes are identical; the GPP-embedded record is named for determinism. (An earlier draft specified per-purpose synthesis, which is _not_ what the code does; the tuple order is decided here and pinned against current tests) | Preserved — pinned against current tests | -| Same, mode `permissive` | Same tuple comparison; the **greater** tuple governs in full | Preserved — same pinning | -| Same, mode `newest` | Whole-record selection by **`LastUpdated`** (not `Created`), subject to the existing freshness threshold; a tie, an incomparable pair, or timestamps inside the threshold fall back to the fully deterministic `restrictive` rule above | Preserved — same pinning | -| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | -| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | -| One valid record + one **expired** record of the same family | The valid record governs — the expired one dropped at pipeline step 2, before conflict resolution ever saw it | **Changed (declared)** — current runtime resolves the conflict first and can select the expired record | -| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | -| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | Preserved, circularity resolved | -| Proxy/mirror mode | **Syntax validation still runs; semantic decoding is skipped.** A present record (well- or mal-formed) is _present, undecoded_: it blocks grants (a record TS will not read cannot vouch for consent) and never withdraws; absent → baseline. Header-carried opt-outs (GPC) are unaffected — they need no decoding. Without the syntax pass, malformed-present would be indistinguishable from absent, contradicting the fail-closed rule below | **Changed (declared)**: today proxy mode skips decoding entirely, which under a permissive baseline is fail-open | -| GPP / US Privacy fields | Per the normative field mapping of §4.5 — fields are not interchangeable signals, and absent/N-A fields grant nothing | Decided here (§4.5) | -| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | +| Input state | Effective record / outcome | Status | +| ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| Standalone TCF and GPP-embedded TCF disagree, mode `restrictive` | **Whole-record selection comparing the P1 ∧ P4 conjunction only** — today's algorithm, preserved (an earlier draft's lexicographic (P1, P4) tuple would have changed split-purpose outcomes): if exactly one record's conjunction is false, `restrictive` selects it; **equal conjunctions — including split-purpose disagreements — keep the standalone record**, as current code does | Preserved — pinned against current tests | +| Same, mode `permissive` | Same conjunction comparison, selecting the record whose conjunction is true; equal conjunctions keep the standalone record | Preserved — same pinning | +| Same, mode `newest` | Whole-record selection by **`LastUpdated`** subject to the existing freshness threshold; a tie, an incomparable pair, or timestamps inside the threshold fall back to the `restrictive` rule above (itself deterministic) | Preserved — same pinning | +| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | +| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | +| One valid record + one **expired** record of the same family | The valid record governs — the expired one dropped at pipeline step 2, before conflict resolution ever saw it | **Changed (declared)** — current runtime resolves the conflict first and can select the expired record | +| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | +| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**, then flows through the full normalization pipeline (syntax, expiry, conflict) like any live record; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | **Changed (declared)**: current code returns immediately after the KV load, bypassing expiry and conflict normalization | +| Proxy/mirror mode | **Minimal opt-out extraction still runs; full semantic decoding is skipped.** Because opt-outs are globally authoritative (§4), proxy mode must not suppress them: the §4.5-mapped opt-out fields (GPP US sections) and the US Privacy string are decoded — nothing else — alongside syntax validation, so a valid SaleOptOut or USP opt-out revokes and withdraws exactly as outside proxy mode. No grants are ever derived from records in proxy mode; a present record otherwise blocks grants (fail-closed); absent → baseline. GPC needs no decoding | **Changed (declared)**: today proxy mode skips decoding entirely — fail-open under permissive baselines and, worse, opt-out-blind | +| GPP / US Privacy fields | Per the normative field mapping of §4.5 — fields are not interchangeable signals, and absent/N-A fields grant nothing | Decided here (§4.5) | +| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | ### 4.5 US signal field mapping — normative @@ -480,13 +486,36 @@ a change to this table. | US Privacy · present, `N` or N/A | — | grant | grant | — (today's tests pin N/A as allowing; USP carries no distinct targeted-advertising field, so it never maps to one) | | Any field | absent / N-A | — | — | — | -**Multi-section aggregation:** when both a national and an applicable -state section are present, the state section governs for the fields it -carries; across whatever sections apply, **an opt-out in any applicable -section beats a grant in another** (restrictive aggregation). `SharingOptOut` -and `TargetedAdvertisingOptOut` are new enforcement inputs — current code -consults only the sale field — and are declared as such in the migration -matrix. +**N/A vs absent:** a field explicitly set to _Not Applicable_ is treated +as not-opted-out (grant-class) — pinned by today's USP tests and matching +current GPP `NotApplicable` handling, and declared as such since an +earlier draft said N/A contributes nothing. A field **absent** from an +applicable section, or any field of a non-applicable section, contributes +nothing. + +**Applicability and aggregation — ordered algorithm:** + +1. **Section map (normative, pinned here — not "whatever GPP is current"):** + `US` national ↔ GPP section 7 (usnat); `US/CA` ↔ 8 (usca); `US/VA` ↔ 9 + (usva); `US/CO` ↔ 10 (usco); `US/UT` ↔ 11 (usut); `US/CT` ↔ 12 (usct). + Section versions are those published at this spec's date; adding a + section or version is a change to this map. +2. **Determine applicability from the resolved jurisdiction:** the + national section is applicable to any `us-privacy`-regime request; a + state section is applicable iff it maps to the resolved `US/`. + Foreign-state sections (a `usca` string on a `US/CO` request) and all + sections on non-`us-privacy` requests are **not applicable** and + contribute nothing. Regionless US traffic: national section only. +3. **State-over-national, per field:** where an applicable state section + carries a field, it governs that field; the national section fills only + fields the state section lacks. +4. **Aggregate across what remains applicable:** an opt-out (of either + subclass) in any applicable field beats a grant from another — + restrictive aggregation. + +`SharingOptOut` and `TargetedAdvertisingOptOut` are new enforcement +inputs — current code consults only the sale field — and are declared as +such in the migration matrix. ## 5. Jurisdiction resolution @@ -549,6 +578,22 @@ different states, and pre-epic behavior treated them differently (fail closed vs. non-regulated) — collapsing them is what made PR #838's migration story unresolvable (migration spec §2, rows 5 and 7). +### 5.5 Policy revision activation + +A policy edit propagates through the config store, so a fleet briefly +mixes revisions. The contract: instances stamp every resolution and every +provenance write with the policy revision they used (already required by +§7); the mixing window is bounded by config propagation and observable via +the config-version metric; and mixed revisions cannot cause irreversible +harm, because **destructive withdrawal triggers are user signals, never +policy** (§4.2 trigger 3) — the one revision-sensitive destructive case +(trigger 2 under a now-`denied` baseline) requires an affirmative user +refusal at the evaluating instance, which is safe under either revision. +S2S recomputation always evaluates against the instance's current +revision and records it. Rolling a policy revision back restores +acquisition rules but **cannot resurrect tombstoned identities**; the +migration guide says so where operators will read it. + ## 6. Failure-mode matrix — normative | Condition | Resolution behavior | @@ -620,13 +665,21 @@ Consumers of the resolved set in this epic: (which signal class granted, per permission), the evidence's **authoritative timestamp and `valid_until`** (per evidence class), resolved jurisdiction, policy revision, and provider/version (providers - spec §6.1). Two aging rules prevent perpetual renewal: **re-presenting - an unchanged signal does not reset evidence age** — only a record - carrying a newer authoritative timestamp (e.g. TCF `LastUpdated`) does; - and every live resolution **atomically replaces the complete - per-permission snapshot**, never merges — a refusal, opt-out, malformed - or absent state in the fresh resolution clears prior positive authority - for its scope, so an old P4 grant cannot survive a later P4 refusal. A sync request performs a **full recompute of both + spec §6.1). Freshness is a **per-evidence-class contract**, because not + every source carries a timestamp: + + | Evidence class | Authoritative timestamp | Age reset | Max age | + | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | + | TCF consent | The record's `LastUpdated` | Only a record with a **newer** `LastUpdated` | Existing TCF expiry TTL | + | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (equality digest stored in provenance) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | + | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | + + Timestamps are compared with bounded clock-skew tolerance and + future-dated values are clamped to receipt time. And every live + resolution **atomically replaces the complete per-permission + snapshot**, never merges — a refusal, opt-out, malformed or absent + state in the fresh resolution clears prior positive authority for its + scope, so an old P4 grant cannot survive a later P4 refusal. A sync request performs a **full recompute of both permissions** from that stored evidence against the _current_ policy: it fails closed when the stored jurisdiction's rule is now `denied`, when a `granted` baseline tightened to `requires_signal` and the stored @@ -688,7 +741,12 @@ further consumer if and when it proceeds. reset), and legacy-row fail-closed-then-backfill. - The full cross-product **regime × permission × evidence source** from §4's acceptance table and §4.5's field mapping, including multi-section - aggregation conflicts. + aggregation conflicts and the applicability algorithm's foreign-section + and regionless rows. +- Provenance snapshot-replacement transitions per permission: prior grant + → refusal, → opt-out, → malformed, → absent — plus a mid-replacement + fault proving the surviving state is the complete old **or** complete + new snapshot, never a merged mixture. - Legacy-row withdrawal end to end (§4.3's derived family ID). - §4.3 fault-injection cases. - Policy validation tests for every §3.3 rejection, exercised through both diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 8c934bc31..2d3d58e58 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -98,7 +98,7 @@ through the selected provider: | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | | **Parse / canonicalize** | Reading `ts-ec` back from the request; deciding `ec_was_present`; batch-sync ingestion | Provider parses a cookie value into its **canonical** identifier, or rejects it. Canonicalization and **equivalence are provider-declared, never imposed globally**: each provider ships equivalence fixtures naming exactly which variants are the same identity — case sensitivity is provider-specific (signed/base64-style envelopes are case-sensitive; even the built-in HMAC id is case-insensitive only in its hex prefix, with a case-preserved suffix). Declared-equivalent values parse to the same canonical identifier (satisfying #778). A value the selected provider does not recognize is treated as absent (but see §6.1 legacy readers). | -| **Canonical graph key** | KV identity-graph row reads/writes | The provider maps a canonical identifier to its graph key: stable, KV-safe (within KV length and character-set limits), collision-free across the provider's identifier space, and namespaced so two providers' key spaces cannot collide. Two equivalent envelopes of one identity map to one key — verbatim cookie bytes as the key would fork graph rows on canonicalization differences and discard today's batch-sync canonicalization. | +| **Canonical graph key** | KV identity-graph row reads/writes | The provider supplies a canonical key **suffix**; **core constructs the physical key** per the §6.3 key grammar (legacy-HMAC verbatim keys excepted), so cross-provider and cross-record-kind isolation is enforced by construction rather than promised by provider code. Suffixes are stable, KV-safe (length and character-set limits), and collision-free within the provider's space. Two equivalent envelopes of one identity map to one key — verbatim cookie bytes as the key would fork graph rows on canonicalization differences and discard today's batch-sync canonicalization. | | **Cluster prefix** (optional capability) | IP-cluster sizing (`cluster_trust_threshold`, implemented as a **KV prefix listing**), pull-sync dedupe, log redaction | A provider declaring cluster support returns a prefix that is a **literal byte prefix of the canonical graph key** — the cluster count lists keys by prefix, so an independently derived hash that is not an actual key prefix silently reports the wrong cluster size. The prefix deliberately collides across identifiers minted from the same client evidence. A provider without the capability declares so, and cluster-dependent gating follows a configured degradation policy (treat cluster size as unknown, with the KV-write decision that implies made explicit in config) instead of counting garbage. | | **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider parses — never a shape-gated subset. | @@ -176,19 +176,33 @@ pub trait EdgeCookieProvider { /// Permissions this provider's data use requires. Enforced by core for /// minting and identity use — never for parse/tombstone (§5). fn required_permissions(&self) -> PermissionSet; - /// Mint an identifier from request evidence. - fn generate(&self, input: &IdentityInput<'_>) -> Result>; /// Parse and canonicalize a cookie value into this provider's - /// identifier; None when unrecognized. Values the provider's declared + /// identifier; None when unrecognized. Identifies the provider + /// NAMESPACE only — never a configuration version (§6.1: versions + /// resolve from row provenance). Values the provider's declared /// equivalence fixtures name as equivalent canonicalize identically. fn parse(&self, value: &str) -> Option; - /// Canonical KV graph key for a parsed identifier. - fn graph_key(&self, id: &EcId) -> GraphKey; - /// Cluster capability: a literal byte prefix of `graph_key(id)`, shared - /// across identifiers minted from the same client evidence. None when - /// the provider does not support IP-cluster semantics (§3). + /// Canonical graph-key SUFFIX (bounded length, KV-safe). Core — not + /// the provider — constructs the physical key (§6.3 key grammar), so + /// cross-provider and cross-record-kind isolation is structural. + /// Sole exception: hmac v0 keys are the identifier verbatim. + fn graph_key_suffix(&self, id: &EcId) -> GraphKeySuffix; + /// Cluster capability: a literal byte prefix of the physical graph + /// key, shared across identifiers minted from the same client + /// evidence. None when the provider lacks IP-cluster semantics (§3). fn cluster_prefix(&self, id: &EcId) -> Option; + /// Acquisition mode — exactly one: + fn acquisition(&self) -> Acquisition<'_>; } + +/// How a provider's identifiers come into being. Server-mint providers +/// generate from request evidence; client-resolve providers verify a +/// browser-posted payload (client-cycle spec) and declare the JS module +/// their page leg needs. One provider implements exactly one mode. +pub enum Acquisition<'a> { + ServerMint(&'a dyn ServerMint), // fn generate(&IdentityInput) -> EcId + ClientResolve(&'a dyn ClientResolve),// fn resolve_from_client(&Payload) -> EcId +} // + fn js_module_id() -> &str ``` (Names indicative; the shape is normative. `required_permissions` joins the @@ -225,10 +239,14 @@ error logged, the next request retries. (permission model spec §7) is a backstop, but conventions do not survive new code — the ungated proxy/click/Testlight paths happened precisely because raw EC values circulate as ordinary strings. Core therefore -introduces an **`AuthorizedIdentity`** newtype constructible only by core, -only after parse + permission gate + graph/family-revocation check; -outbound serializers (ORTB builder, page bids, sync, identify, forwarding) -accept `AuthorizedIdentity`, never `&str`/`EcId`. A future bypass then +introduces a **scope-parameterized `AuthorizedIdentity`**, +constructible only by core, only after the checks _for that exact scope_: +`AuthorizedIdentity` after parse + `store-on-device` + +family-revocation check; `AuthorizedIdentity` additionally +after `select-personalised-ads`. Outbound serializers (ORTB builder, page +bids, sync, identify, forwarding) accept `AuthorizedIdentity` +and nothing weaker — an unparameterized wrapper would let a P1-only +identity flow into an ORTB request. A future bypass then requires deliberately reconstructing the raw string — visible in review — rather than passing along what was already in hand. @@ -247,8 +265,11 @@ structural: JA4/HTTP-2 fingerprints) for **security classification** — the bot gate protecting KV-backed identity writes — which must run precisely for traffic that has granted nothing. The authorization for that processing - is the operator's explicit `[device] provider` selection, and this spec - records that as the decision, with its privacy implication stated: a + is the operator's explicit `[device] provider` selection — a statement + about the **opt-in host-fingerprint provider**; the `builtin` UA-only + default processes nothing beyond the User-Agent every request already + carries and needs no such authorization — and this spec records that as + the decision, with its privacy implication stated: a device provider whose data use goes beyond security classification (for example feeding fingerprints into targeting or identity) is **not authorized by selection alone** and requires a vocabulary extension plus @@ -321,35 +342,55 @@ The contract: Same-provider key/passphrase rotation is configuration, not a provider switch: a provider block may hold multiple `versions` entries (`[ec.providers.hmac.versions.v2] passphrase = …`) with - `mint_version = "v2"` selecting the writer; `parse` consults versions in - declared order, newest first; removing a version entry is a retirement - subject to the same evidence rules as retiring a legacy reader - (migration spec §6). + `mint_version = "v2"` selecting the writer. **`parse` cannot identify a + version** — every HMAC version shares one grammar and `parse` returns no + version — so the mint version lives in **immutable row provenance** + (rows without a tag are `hmac-v0`), and cryptographic verification + consults configured versions newest-first only where provenance is + unavailable (a cookie with no reachable row). Removing a version entry + is a retirement subject to the same evidence rules as retiring a legacy + reader (migration spec §6). - A cookie recognized by a legacy reader is a live identity for read/withdrawal purposes; whether it is transparently re-minted under the active writer is a per-deployment choice (`[ec] rewrite_legacy = true|false`), and re-minting is subject to the full minting gate of §5. -- **Rewrite aliases to one canonical row — no dual-write window.** Order: - the new canonical row commits first, sharing the old row's revocation - family ID (permission model spec §4.3); its provenance is the **current - live resolution** (rewrite happens on a live request — copying old - consent evidence would rejuvenate stale authority), while partner - mappings copy **with their original per-field timestamps and expiry**. - Then a fenced CAS **replaces the old row with an alias record** pointing - at the canonical row; from that moment every read or update through - either cookie chases the alias (single hop) to the one canonical row — - a concurrent pull/batch/identify update cannot land on a row about to - be discarded, because after the CAS there is only one row to land on, - and an update racing the CAS itself retries against the canonical. Then - the new cookie is emitted. The server cannot observe `Set-Cookie` - acceptance, so the **alias stays live** until a later request presents - the new cookie (confirmation by presentation), and in any case until a +- **Rewrite is a persistent fenced transaction aliasing to one canonical + row — no dual-write window, no duplicate targets, no lost updates.** + The steps, each resumable because the transaction record (its own + linearizable record class, §7 matrix) is written **first** and pins the + chosen target key and fencing epoch: + 1. **Transaction record** commits: source key, target key, epoch, + state. A crashed rewrite retried later reads it and resumes with + the **same** target — a fresh random target (and an orphaned first + one) cannot exist, and any target row without a committed transaction + pointing at it is garbage-collectable by that absence. + 2. **Canonical row** commits under the pinned target key, sharing the + old row's revocation family ID (permission model spec §4.3); + provenance is the **current live resolution** (copying old consent + evidence would rejuvenate stale authority); partner mappings copy + with their **original per-field timestamps and expiry**, and the + copy point is recorded in the transaction. + 3. **Fenced CAS replaces the old row with an alias record** targeting + the canonical. If the CAS loses to a concurrent pull/batch/identify + update, the rewrite **re-runs a reconciliation pass** under its + epoch — merging updates newer than the recorded copy point into the + canonical — and retries the CAS; an update that won the old row is + therefore never lost. + 4. The new cookie is emitted; the transaction marks complete. + + From step 3 on, every read or update through either cookie chases the + alias (one hop) to the single canonical row. **Chains stay single-hop**: + a later rewrite B→C retargets every alias pointing at B (the canonical + row records its inbound aliases; alias records are in the linearizable + class, so retargeting is fenced) so A points directly at C. The server + cannot observe `Set-Cookie` acceptance, so the alias stays live until a + later request **presents the new cookie**, and in any case until a **finite retirement deadline** no shorter than the old cookie's maximum - lifetime plus rollout skew. An interrupted rewrite leaves the old - cookie resolving (directly or via the alias) — no state in which - neither identity works. **Withdrawal through either cookie revokes the - shared family.** + lifetime plus rollout skew. An interrupted rewrite at any step leaves + the old cookie resolving — no state in which neither identity works. + **Withdrawal through either cookie revokes the shared family.** + - Retiring a legacy reader is the explicit end of those identities: the migration guide documents the cleanup procedure (migration spec §6). - Tests: switch active provider → request with old cookie → identity still @@ -391,7 +432,49 @@ egress and sync updates fail closed; organic requests continue stateless. The thresholds ship as constants with the implementation and are printed in the startup log. -### 6.3 Graph row contract — normative +Its protection is therefore **local-only, and the spec says so**: a +backend-wide outage degrades every instance through its own observations +within one window, but an instance-local family-write failure leaves +other instances — which have no record to find, and healthy backends of +their own — serving S2S egress until the browser's durable signal retries +successfully. That residual is bounded by the user's return latency, is +counted (failed family writes are a first-class metric), and is accepted +in place of a deployment-wide shared fail-closed channel, whose own +availability and freshness would be a harder problem than the one it +solves. + +### 6.3 Storage contract — normative + +**Physical key grammar.** Core constructs every key; providers supply only +the bounded suffix: + +| Record class | Key | Notes | +| ----------------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| Identity row (v2+) | `id///` | Suffix from `graph_key_suffix`, ≤ 128 bytes, KV-safe alphabet | +| Identity row (legacy hmac-v0) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar; no other class or provider may produce a matching key | +| Alias | `alias///` | Same suffix as the row it replaced | +| Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | +| Rewrite transaction | `rwx/` | One in-flight rewrite per family | +| Replay reservation | `resv////` | Client-cycle spec; payload id ≤ 128 bytes | + +Grammars are pairwise non-intersecting by their literal prefixes (plus the +reserved legacy grammar), which is what makes cross-class collision +impossible rather than unlikely. + +**Wire schemas** (JSON, like identity rows; every class carries a schema +version): the **alias record** holds target key, created-at, retirement +deadline, and fencing epoch; the **family revocation record** holds the +family ID, revoked-at, triggering signal class (§4.5 destructive column), +and epoch — deliberately no identity data, so it can outlive its members; +the **rewrite transaction** holds source key, target key, copy point, +state, and epoch; the **reservation** holds state, owner hash, lease +epoch, outcome, and created-at (client-cycle spec). Field validation and +TTLs: aliases live to their retirement deadline; family records to the +§7 retention rule (beyond every member, cookie, rewrite, and retry +lifetime); transactions to completion plus an audit window; reservations +at least through token expiry. + +#### Graph row contract The per-field contract for identity rows, covering today's v1 fields and the fields this epic adds. Serialization is JSON with the existing `v` @@ -438,16 +521,29 @@ Requirements: **per-record-class consistency requirements**, because "has KV" says nothing about whether revocation is observable: - | Record class | Required semantics | - | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | - | Family revocation records | Strongest read the backend offers, plus a **declared, bounded revocation-visibility lag** (Workers KV documents up to ~60 s eventual propagation — that bound must be declared, and the residual it implies stated in operator docs). Unboundable lag → startup failure for identity features. A **failed or erroring revocation-record read fails closed** for egress | - | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | + | Record class | Required semantics | + | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | + | Family revocation records | **Strongly consistent (read-after-write) primitive required.** Cloudflare Workers KV is **not eligible** — its documentation says propagation may take "60 seconds or more", an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | + | Alias / rewrite-transaction records | **Linearizable fenced CAS required** (same primitive class as reservations). `rewrite_legacy = true` is rejected at startup on adapters lacking it (§6) | + | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | Each adapter's declaration is part of its wiring, drives the §6 capability-mismatch startup error, and every §6.2 runtime-failure row gets fault-injection coverage on every adapter declaring the - corresponding capability. + corresponding capability. The **concrete per-adapter values** — the + actual matrix, not the abstract capability list — as known today; a + cell marked _verify_ must be established before the depending feature + is selectable on that adapter, and the filled matrix is normative: + + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | + | Graph persistence (eventual OK) | KV Store: yes | Local store: yes (dev-grade) | Workers KV: yes (eventually consistent) | Key-value: yes | + | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — client-cycle and `rewrite_legacy` unselectable until a primitive exists | Yes (in-process) | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index eaf6bb133..54f7fe4d4 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -54,6 +54,7 @@ discoverable only because a deleted test had pinned the old behavior. | 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | | 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | +| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -118,16 +119,32 @@ Requirements: after **fleet convergence on N+1 is confirmed** — binaries first, convergence gate, then `ts config push`. A config mixing old and new fields (`[ec] passphrase` alongside `[ec] provider`) is **rejected** - by N+1, not reconciled. Rollback runs the sequence in reverse: config - back to the old shape first, binaries only after config convergence. - Every new config section introduced by the epic follows this same - compatibility rule, not only `[ec]`. + by N+1, not reconciled. **Rollback is binaries-first too, in the + other direction**: N+2 → N+1 binaries roll back **keeping the new + config** (N+1 reads it fully — reverting config first would hand the + old shape to N+2 binaries that reject it). N+1 additionally + **rejects provider or version selections whose provenance it cannot + yet encode** — new-provider adoption waits for N+2, so no row is + minted that N+2 would misclassify. Every new config section + introduced by the epic follows this same compatibility rule, not + only `[ec]`. - **Release N+2:** rejects `[ec] passphrase` at startup with a message naming the new location — not a generic unknown-field error (implementation note: producing the actionable message means keeping a deprecated `passphrase` field whose presence triggers the custom error). -2. **The graph schema change is expand-contract, in lockstep with the +2. **Graph-store readiness precedes everything.** Today the graph store + is optional and EC generation succeeds without one; the epic's + no-active-until-commit invariant (providers spec §5) makes it + mandatory wherever a minting provider is configured — so a currently + valid graphless HMAC deployment would **startup-fail on N+1's + dual-read mapping** without a preparatory step. The migration + therefore begins with a **pre-N+1 readiness step**: provision and + verify an openable graph store (and confirm the adapter's capability + row supports the features in use, providers spec §7) _before_ rolling + N+1. This is a **declared breaking change** for graphless deployments + (matrix row 12), not a side effect discovered at boot. +3. **The graph schema change is expand-contract, in lockstep with the binary sequence.** New rows carry fields v1 rows never had — provider/ version, per-permission grant evidence, policy revision, family ID, rewrite links — and two failure modes must be engineered away: a naive @@ -153,49 +170,62 @@ Requirements: new fields untouched if read-only, preserved semantically if read-modify-write on N+1, **test-proven lost on pre-N+1** (documenting why the floor is a floor); N+2-reader/N+1-written-row → full function. -3. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no +4. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the docs reaches if they miss one line — validated green and silently minted zero ECs. -4. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by +5. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by PR #838, deliberately not carried into this epic — providers spec §2) and `provider = "client-fixed"` are unknown keys and rejected like any other, so a config written against the PR #838 example cannot silently select a provider that no longer exists. -5. **Provider switches go through legacy readers.** Changing +6. **Provider switches go through legacy readers.** Changing `[ec] provider` on a deployment with live identities requires listing the outgoing provider in `[ec] legacy_providers` (providers spec §6.1) so existing cookies keep resolving and stay withdrawable; the guide documents the switch sequence and the retirement/cleanup step that ends it. -6. **The example config ships the migrated happy path**, uncommented: +7. **The example config ships the migrated happy path**, uncommented: `provider = "hmac"` with its block, `[geo] default_country`, and (for Fastly) the behavior-preserving `[device] provider = "fastly"` and `[geo] provider = "platform"` lines present with a comment stating what removing them changes. PR #838's example shipped the passphrase block uncommented with the selector commented out — steering operators directly into the silent-stateless state. -7. Every misconfiguration in the providers spec §6 table fails at +8. Every misconfiguration in the providers spec §6 table fails at **startup**. Request-time failure for a configuration error is a defect. -8. Config-store payload validation (`ts config push`) applies the same +9. Config-store payload validation (`ts config push`) applies the same rules — including `[permissions]` policy validation (permission spec §3.3) — so a bad config is rejected at push time, before any instance restarts into it. -## 5. Behavior-preserving migration recipe (operator-facing) +## 5. Minimal-divergence migration recipe (operator-facing) + +"Keep exactly today's behavior" is not fully achievable, and the recipe's +name says so. The unavoidable divergences, enumerated (each also a matrix +row): global opt-out honoring (row 8); refusal blocking new grants +everywhere (row 6); newly enforced GPP sharing/targeted fields, which can +also **grant** P4 where nothing granted before (row 3e); the FR +unresolved-geo fallback, where valid TCF consent can grant while today's +unresolved-geo path always denies (row 5); malformed-present blocking +acquisition (§4.4); proxy-mode opt-out extraction; and the batch-sync +provenance gate (row 11c). Everything else the recipe preserves. The migration guide (a new `docs/guide/` page, linked from the release notes) -gives one copy-pasteable recipe per adapter for "keep exactly today's -behavior": +gives one copy-pasteable recipe per adapter for the minimal-divergence +posture: The recipe is a **complete, valid TOML fixture per adapter, committed to the repository** (e.g. `docs/guide/fixtures/migration-preserving-fastly.toml` and siblings) and included in the guide verbatim — never described as a textual delta against the example file. Per-adapter because a single -fixture cannot be: `[device] provider = "fastly"` and -`[geo] provider = "platform"` are capability-gated selections that the -Axum/Cloudflare/Spin adapters reject at startup (providers spec §6); each +fixture cannot be: `[device] provider = "fastly"` is Fastly-only, and +`[geo] provider = "platform"` varies by host — Cloudflare **does** +support platform geo but resolves **country only, no region** (per the +providers spec adapter matrix), which changes state-level US privacy +outcomes and engages the declared regionless degradation; Axum and Spin +have no platform geo and reject the selection (providers spec §6). Each adapter's fixture carries the selections valid for it, and each is CI-validated against its adapter. (An earlier draft said "copy the example table, then set `[permissions.rules] default`" — but the copied table already @@ -266,16 +296,24 @@ global honoring of opt-out signals is unconditional. 3. Startup logs always print: selected provider per concern, whether geo is live, the effective default baseline, and the count of granted-without- signal permissions. One line, greppable, stable format. -4. **The batch-sync coverage dip is operationalized, not discovered.** +4. **The batch-sync coverage dip is a gated rollout stage, not a + notification.** Provenance-coverage thresholds are normative gate + criteria: the guide defines a target coverage level and evaluation + window; recovery stalling below threshold for the window triggers the + **pause action** — investigate backfill (traffic mix, dormant rows), + never disable the gate; and staging is explicit: provenance + **writing** begins the moment N+2 activates, enforcement is already + in force (there is no fail-open stage), so the only stageable knob is + partner communication and the cleanup cadence for rows that never + recover. Because legacy rows fail closed for batch updates until backfilled (permission spec §7), batch-sync acceptance drops toward zero at cutover and recovers along the live-traffic backfill curve. The **provenance-coverage metric** (share of active rows carrying - provenance) is the tracking signal; the migration guide states the - expected recovery shape, tells operators to notify batch-sync partners - of the transient rejection rate, and defines no fail-open shortcut — - the alternative (grandfathering pre-epic identities past the - permission model) is rejected in the permission spec. + provenance) is the gate signal; operators notify batch-sync partners + of the transient rejection rate. There is no fail-open shortcut — the + alternative (grandfathering pre-epic identities past the permission + model) is rejected in the permission spec. 5. Rollback is config-only where possible: reverting to the previous config version restores the previous behavior on the previous binary. The one irreversible artifact is withdrawal tombstones — which is why the @@ -301,3 +339,32 @@ global honoring of opt-out signals is unconditional. spec verbatim — operator docs and normative spec must not diverge on precedence, and prose like "signals are mapped as a grant or a revoke" without stating which wins is insufficient. + +## 8. Product decisions requiring explicit sign-off + +These are decisions this spec set makes that #838 had not already made (or +made differently). Each must be ratified by maintainers before +implementation — an unratified row reverts to open, not to silently +implemented: + +1. Opt-outs are honored globally and destructive ones irreversibly + withdraw identities outside the jurisdiction defining the signal + (permission spec §4, §4.2). +2. Sale opt-outs (GPP and USP) control both P1 and P4 and destroy the + identity (§4.5). +3. Sharing / targeted-advertising opt-outs remove P4 but intentionally + retain the stored identity (§4.5). +4. US contextual auctions continue during opt-out, with identity removed + (permission spec §7 dispatch matrix). +5. Regionless US traffic is treated as non-regulated unless the operator + chooses country-wide gating (permission spec §3.4). +6. Full consent strings continue downstream, and raw consent snapshots + are retained in graph rows for audit (providers spec §6.3). +7. Legacy batch-sync traffic is rejected until live-browser provenance + backfill occurs (§6.4 of this spec; permission spec §7). +8. Proxy / click / Testlight forwarding becomes newly gated by P1 ∧ P4 + (§2 row 11b). +9. Integration-owned response cookies are inside the permission model: + persistent cookies require `store-on-device` at apply time and a + declared registration; session cookies are the narrow exemption + (response-hook spec §3). From c8b4b849edb110fb94ae6fb433e6038fa35c3bc7 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 19:44:28 -0700 Subject: [PATCH 08/14] Address sixth review: global opt-out aggregation, negative authority, and storage-protocol coherence P1 fixes: - GPP applicability now gates grants only: mapped opt-out fields (either subclass) aggregate globally from any section on any request, resolving the section-4-vs-4.5 contradiction where a French visitor's usnat SaleOptOut was simultaneously mandatory and ignored. - The section map covers everything current code recognizes (7-23, through usmn), not 7-12; Texas section 16 named as what the truncated map would have silently lost; states without a state section (MD, IN, KY, RI) use the national section. - Destructive TCF-refusal withdrawal requires the refusal to be carried by the live request; persisted-KV records participate in acquisition only - closing the path where a years-old stored refusal tombstones on the first signal-less request after a policy tightens to denied, and repairing the mixed-revision safety claim. - Negative authority gets its own record: a permission-exempt, strongly consistent suppression record (sup/) with per-permission entries that every S2S recompute and partner-egress check consults - resolving the circularity where clearing P1 provenance required the P1 the refusal just unset, and the eventual-row edge where a stale replica restored P4 after a targeted-advertising opt-out. - The consistency requirement has one normative home (the providers matrix, strong read-after-write); the permission spec's bounded-lag leftover is gone. - The never-returning-visitor residual is stated as unbounded and becomes sign-off item 11, replacing the false bounded-by-return- latency claim. - Aliases live at the source identity key with a kind discriminator in the value envelope - a separate alias/ address could neither be found by old-cookie lookups nor installed by a single-key CAS. - Identity keys drop the version segment (id//); version lives in the row envelope, killing the read-the-row-to-learn- how-to-read-the-row circularity. All HMAC versions stay on the verbatim key scheme, keeping the 64-hex cluster prefix a literal key prefix for every HMAC row; rotation-induced cluster splits are declared as inherent to rotation. - Rewrite requires the row store itself to provide per-key CAS with read-your-writes (alias installs happen there); adapters with purely eventual row stores cannot host rewrite. Chains use bounded traversal (4 hops, cycle detection, fail closed) with opportunistic path compression instead of an undefined inbound-alias index. - Revocation-wins at the resolve endpoint is an explicit linearization point: family records carry an epoch and the cookie-emitting commit is a CAS conditioned on it - linearizable reads alone lose the race. - resolve_from_client takes a core-built ClientResolveContext (canonical audience, verified session owner, clock, bounded payload) and returns a verified identity with reservation id and expiry. - The mutator snapshot is redacted: all Set-Cookie values and reserved identity/consent/privacy header values withheld, so the hook cannot leak the raw EC around AuthorizedIdentity. - The cache merge is over independent sticky directives per RFC 9111 (no-cache and private are orthogonal; the ordered-lattice version could make a personalized response shared-storable), and the complete origin Vary set is preserved, not only core-required members. - Hook cookie coupling acknowledged: the persistent-cookie gate is a listed enforcement point in the permission spec inventory; cookie operations activate only after the permission model lands; a typed cookie builder enforces declared lifetime/scope/security attributes; deletion cookies work when P1 is denied. - N+1 is a full semantic reader and enforcer for every N+2 record kind (aliases, family revocation, suppression, provenance fail-closed), with rollback tests on N+1 against N+2 data; binaries-first rollback gains its precondition (converge to an N+1-compatible config first after N+2-only adoption, retaining new-provider secrets as legacy readers rather than reverting config). - Adapters without revocation-eligible storage migrate with explicitly stateless fixtures (sign-off item 12) instead of invalid HMAC fixtures. P2: explicit NotApplicable rows (grant-class, preserved) separated from absent; per-permission first-seen digests over only applicable aggregated fields; consent.us_states.privacy_states path corrected; provider-switch rollback keeps the new provider as a legacy reader; rewrite_legacy with a client-resolve writer is a startup error; client parity redefined as identical startup rejection on ungated adapters; the capability matrix distinguishes platform availability from wiring (Spin: available, not wired); row 3e's effects classified in both directions; and the sign-off list is a ratification table (owner/status per row, implementation blocked while any row is open) extended with items 10-14. --- ...26-07-30-client-cycle-ec-resolve-design.md | 32 ++-- ...integration-response-header-hook-design.md | 59 ++++--- .../2026-07-30-permission-model-design.md | 132 ++++++++++----- .../2026-07-30-pluggable-providers-design.md | 126 ++++++++------ ...07-30-provider-migration-rollout-design.md | 157 ++++++++++-------- 5 files changed, 314 insertions(+), 192 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 1eeb61e14..3cfeb09d4 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -85,10 +85,15 @@ Everything in this spec follows from that. must be parseable, graph-keyed, and tombstonable by the selected provider (providers spec §3). The conformance suite runs against every client-cycle provider. -5. **Exist on every adapter.** Route registration goes through shared route - wiring; the parity suite asserts the endpoint's presence and behavior on - all four adapters. (PR #838 registered it on Fastly only, so the same - config on the Axum dev server proxied the POST to the publisher origin.) +5. **Exist on every adapter — where parity means identical behavior, + including identical refusal.** Route registration goes through shared + route wiring. On adapters whose capability matrix rows are green + (today only the dev adapter has the required CAS class — providers + spec §7), the parity suite asserts identical endpoint behavior; on + adapters without them, parity means **identical startup rejection of + the client-cycle selection** — not a proxied 404 (PR #838's failure + mode: Fastly-only registration let the Axum dev server proxy the POST + to the publisher origin), and not a silently absent route. 6. **Be uncacheable and permission-gated on the provider's full declaration.** `Cache-Control: no-store`; before any cookie is set, the endpoint enforces the selected provider's complete @@ -157,13 +162,18 @@ Everything in this spec follows from that. explicitly-accepted at-most-once posture (no owner hash), a lost response is an **orphan row** handled by the specified cleanup path. The **same-identity no-op of §3.8 first checks the family revocation - record** (permission model spec §4.3), and it does so **through the - linearizable primitive class this feature already requires** for - reservations (providers spec §7 matrix) — which is what makes - "revocation wins" true rather than aspirational: on an eventually - consistent read, a racing create could observe a stale absence and - emit a cookie for a revoked family. A resolve against a revoked family - is rejected, never refreshed, and a create racing a revocation loses. Tests cover crash-between-steps, lease + record**, and "revocation wins" is enforced by an explicit + linearization point, not by read strength alone — a linearizable read + followed by a separate commit still loses the race (read "not + revoked" → withdrawal commits → resolve emits a cookie for a revoked + family). The family record carries an **epoch** (providers spec + §6.3), bumped by every revocation-state change; the resolve reads + epoch _e_ at the start, and its cookie-emitting commit is a **CAS in + the strong class conditioned on the family epoch still being _e_**. + A withdrawal landing between read and commit bumps the epoch, the + commit fails, and the resolve is rejected — the CAS is the + linearization point. A resolve against an already-revoked family is + rejected, never refreshed. Tests cover crash-between-steps, lease takeover with a stale-epoch commit attempt, two concurrent requests with the same payload, a duplicate from a second client receiving no cookie, owner-hash recovery receiving the cookie, and diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 23d1241f2..5d35f23a5 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -61,17 +61,22 @@ mutators to the outbound response for HTML document responses it processed. merely passed through) — and the post-hook response may only be **equal or stronger** on the privacy axis: integrations can tighten caching, never loosen it, regardless of which header they replaced. - "Equal or stronger" is a defined merge, not a vibe: restriction - strength is ordered `no-store` > `no-cache` > `private` > `public`, - and the final value per axis is the **stronger of snapshot and - mutation**; `max-age`/`s-maxage` may only shrink relative to the - snapshot; `stale-while-revalidate`/`stale-if-error` may appear only if - the snapshot had them; every CDN/surrogate directive - (`Surrogate-Control`, `CDN-Cache-Control`, host-specific equivalents) - is stripped from any restricted response; and **core-required `Vary` - members are protected in the contract, not just the tests** — the - final `Vary` is the union of the snapshot's required members and the - mutation. Middle-stage placement also keeps + "Equal or stronger" is a defined merge over **independent sticky + directives, not a totally ordered lattice** — `no-cache` and `private` + are orthogonal constraints (RFC 9111: `no-cache` permits shared + storage subject to revalidation; `private` forbids shared storage), so + "replace `private` with the stronger `no-cache`" would make a + personalized response shared-storable. The merge: each of `no-store`, + `no-cache`, `private` is **sticky** — present in the snapshot or the + mutation ⇒ present in the final response, independently; `public` is + dropped whenever any restriction is present; `max-age`/`s-maxage` may + only shrink relative to the snapshot; `stale-while-revalidate`/ + `stale-if-error` may appear only if the snapshot had them; every + CDN/surrogate directive (`Surrogate-Control`, `CDN-Cache-Control`, + host equivalents) is stripped from any restricted response; and the + final `Vary` is the **union of the complete snapshot `Vary` set** — + origin-supplied members included, not only core-required ones — and + the mutation. Middle-stage placement also keeps the earlier property: an integration mutation is not silently stripped by ordinary core handling — only by the invariant pass, which logs the downgrade it applies. @@ -98,8 +103,14 @@ mutators to the outbound response for HTML document responses it processed. (any `Max-Age`/`Expires`) is applied only when the request's resolved permissions include `store-on-device`, while **session cookies** (no persistence attributes) are the narrow, documented exemption. - Undeclared cookie names are rejected like reserved ones. An integration - may never set or expire a reserved cookie name. Violations are rejected at the operation layer (§2) and + Cookie operations go through a **typed cookie builder** that enforces + the declared lifetime ceiling, domain/path scope, and security + attributes (`Secure`, `SameSite`) — not a free-form string; **deletion + cookies (expiry of the integration's own declared names) remain + possible when `store-on-device` is denied**, since removing state must + never require the permission to keep it. Undeclared cookie names are + rejected like reserved ones. An integration may never set or expire a + reserved cookie name. Violations are rejected at the operation layer (§2) and logged at `warn` with the integration id. The reserved lists are single constants next to the definitions they protect, not duplicated in the hook. @@ -121,10 +132,17 @@ mutators to the outbound response for HTML document responses it processed. counting `name: value` plus separators, within any lower adapter ceiling) bounds the sum across integrations — enforced in registration order, so which operations are rejected when a budget trips is - deterministic. Each mutator receives an **immutable snapshot of the + deterministic. Each mutator receives an **immutable, redacted snapshot of the response head** (status and headers as of its turn, prior integrations' accepted operations applied) as its read context; it never holds a - mutable reference (§2). + mutable reference (§2). Redaction is a security boundary, not + tidiness: the hook runs after core queues the EC `Set-Cookie`, so an + unredacted view would hand a mutator the raw EC to copy into + `X-Vendor-Identity` or its own cookie — walking around + `AuthorizedIdentity` entirely. The snapshot therefore + **excludes every `Set-Cookie` value and every reserved identity, + consent, and privacy header value** (names may be listed as present; + values are withheld). Exceeding a limit rejects the excess operations (logged, attributed), never the response. A mutator that returns an error is skipped in full — its operations are all-or-nothing — and the response proceeds without it. @@ -181,10 +199,13 @@ processed documents (§6). ## 5. Size and sequencing -This is a ~150-line feature plus tests, with zero coupling to the provider -architecture or the permission model. It lands as its own small PR **when -its first real consumer is identified** (§4.2) — at any point in the epic's -sequence, blocking nothing and blocked by nothing. If no consumer +This is a modest feature plus tests, with zero coupling to the provider +architecture — but its **cookie operations are coupled to the permission +model** (§3; the gate is a listed enforcement point in the permission +spec §7 inventory), so the claim of total independence is retired: the +header-mutation portion may land whenever its first real consumer is +identified (§4.2), while `append_set_cookie` activates only **after** the +permission model PR, and registers as unavailable before it. If no consumer materializes, it does not land; being unblocked is not a reason to ship scaffolding. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 6154de7c7..8a0818166 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -210,7 +210,7 @@ Validation rejects: ### 3.4 One source of jurisdiction truth Today, `detect_jurisdiction` — driven by the runtime lists -`consent.gdpr.applies_in` and `consent.us_privacy.states` — is the sole +`consent.gdpr.applies_in` and `consent.us_states.privacy_states` — is the sole jurisdiction source for **both** the auction consent gate and the EC gate. The permission model replaces the EC side; if the auction gate keeps reading the old lists while EC reads policy rules, the two will drift (adding a @@ -225,7 +225,7 @@ survive an interim period, a CI test asserts consistency between each list and the policy's regime classes, with deliberate divergences recorded as explicit, commented exceptions in the test — never silent. Both legacy lists are in scope, not only the GDPR one — and the US check is -region-shaped: **every configured `consent.us_privacy.states` entry must +region-shaped: **every configured `consent.us_states.privacy_states` entry must have a matching `US/` rule**, and the country-level `US` rule must resolve non-regulated (today applies privacy gating only to the configured states), or the divergence is an explicit commented exception. An adapter @@ -356,8 +356,15 @@ The triggers, exhaustively — nothing else withdraws: targeted-advertising) never trigger this — they revoke acquisition only. (For US states this preserves today's behavior; elsewhere it is the declared change of §4's global-opt-out rule.) -2. **A TCF record refusing `store-on-device` withdraws iff the baseline is - `requires_signal` or `denied`.** Where the baseline is `granted`, +2. **A TCF record refusing `store-on-device` withdraws iff the baseline + is `requires_signal` or `denied` — and only when the refusal is + carried by the live request.** A persisted-KV consent record + participates in acquisition only and **never triggers withdrawal**: + without this, a refusal stored years ago under a `granted` policy + would destructively fire on the first signal-less request after the + policy tightens to `denied` — a policy edit tombstoning by proxy, + which trigger 3 forbids, and the counterexample to §5.5's + mixed-revision safety claim. Where the baseline is `granted`, refusal blocks _new_ grants but never tombstones: tombstones are irreversible, and PR #838 wrote them for visitors in unregulated jurisdictions whose global CMP emitted a purpose-refusing string — @@ -410,25 +417,43 @@ and the fail-closed marker: discoverable from every member, and the record survives member-tombstone replacement (which today discards the original row's identity and metadata, making sibling discovery impossible). +- **Negative authority has its own permission-exempt record.** A live + refusal or non-destructive opt-out must clear prior positive + provenance — but the row write that would do it requires `store-on-device`, + which the refusal just unset, and identity rows may be eventually + consistent, so a stale replica could resurrect a P4 grant after a + targeted-advertising opt-out. The fix is a **suppression record** in + the strongly consistent class (providers spec §6.3: `sup/`), + carrying per-permission suppression entries with timestamps. Writing it + is **permission-exempt** (clearing authority is protective, like + revocation), and **every S2S recompute and partner-egress check + consults it**: a suppressed permission is unset whatever the row's + provenance says, so no eventual-consistency edge can restore it. - **The cookie expires only after the family record commits.** - **If the family-record write itself fails, nothing durable exists** — - the cookie stays and the durable client-side signal (GPC, CMP-stored TCF) - retries the whole withdrawal on the next request. Two mitigations bound - the S2S residual in the meantime: while graph **writes are degraded** - (health signal), S2S partner egress and sync updates fail closed - (providers spec §6.2); and the failure is logged at `error` with a - metric feeding the operational repair path. The residual that remains — - a single failed write on an otherwise healthy graph, for a user who - never returns — is declared here, not hidden. -- **Consistency and retention are backend contracts**, defined in the - providers spec consistency matrix (§7): revocation-record reads use the - strongest read the backend offers, adapters declare a bounded - revocation-visibility lag (an eventually-consistent store that cannot - bound it fails startup for identity features), a **failed family-record + the cookie stays and the durable client-side signal (GPC, CMP-stored + TCF) retries the whole withdrawal on the next request. Mitigations: + while graph **writes are degraded** (health signal), S2S partner egress + and sync updates fail closed on that instance (providers spec §6.2); + the failure is logged at `error` with a metric feeding the operational + repair path. The residual that remains — a single failed write on an + otherwise healthy graph, for a visitor who **never returns** — is + **unbounded**, not "bounded by return latency": return latency has no + bound for a non-returning visitor, and the per-instance breaker does + not reach other instances. Accepting this residual instead of building + a durable external retry queue is **product sign-off item 11** + (migration spec §8), not a footnote. +- **Consistency and retention are backend contracts with a single + normative home**: the providers spec consistency matrix (§7). It — not + this spec — states the requirement, and it requires a **strongly + consistent (read-after-write) primitive** for revocation records; no + bounded-lag alternative exists (an earlier draft here permitted one, + which contradicted the matrix — an adapter with a two-second lag would + have passed one spec and failed the other). A **failed family-record read fails closed** for egress (revoked-unknown ≠ live), and revocation records are retained beyond the maximum of cookie lifetime, row TTL, - rewrite grace, and downstream retry horizon — note today's 24-hour - tombstone TTL is far below this bar and does not carry over. + rewrite grace, and downstream retry horizon — today's 24-hour tombstone + TTL is far below this bar and does not carry over. - Fault-injection tests cover: family-record write fails → cookie untouched, S2S behavior per degraded mode, retry completes; member tombstone N fails after the family record → identity already revoked for @@ -474,17 +499,18 @@ withdrawal. Section IDs and versions are those of the IAB GPP specification current at implementation time; adding a section or field is a change to this table. -| Source · field | Value | `store-on-device` (P1) | `select-personalised-ads` (P4) | Destructive withdrawal? | -| -------------------------------------------- | ------------- | ---------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------ | -| GPP US section · `SaleOptOut` | opted out | opt-out | opt-out | **Yes** (preserves today) | -| GPP US section · `SaleOptOut` | not opted out | grant | grant | — | -| GPP US section · `SharingOptOut` | opted out | — | opt-out | No | -| GPP US section · `SharingOptOut` | not opted out | — | grant | — | -| GPP US section · `TargetedAdvertisingOptOut` | opted out | — | opt-out | **No** — a targeted-advertising choice must never destroy the stored identity | -| GPP US section · `TargetedAdvertisingOptOut` | not opted out | — | grant | — | -| US Privacy · `opt_out_sale` | `Y` | opt-out | opt-out | **Yes** (preserves today) | -| US Privacy · present, `N` or N/A | — | grant | grant | — (today's tests pin N/A as allowing; USP carries no distinct targeted-advertising field, so it never maps to one) | -| Any field | absent / N-A | — | — | — | +| Source · field | Value | `store-on-device` (P1) | `select-personalised-ads` (P4) | Destructive withdrawal? | +| -------------------------------------------- | --------------------------- | -------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | +| GPP US section · `SaleOptOut` | opted out | opt-out | opt-out | **Yes** (preserves today) | +| GPP US section · `SaleOptOut` | not opted out | grant | grant | — | +| GPP US section · `SharingOptOut` | opted out | — | opt-out | No | +| GPP US section · `SharingOptOut` | not opted out | — | grant | — | +| GPP US section · `TargetedAdvertisingOptOut` | opted out | — | opt-out | **No** — a targeted-advertising choice must never destroy the stored identity | +| GPP US section · `TargetedAdvertisingOptOut` | not opted out | — | grant | — | +| US Privacy · `opt_out_sale` | `Y` | opt-out | opt-out | **Yes** (preserves today) | +| US Privacy · present, `N` or N/A | — | grant | grant | — (today's tests pin N/A as allowing; USP carries no distinct targeted-advertising field, so it never maps to one) | +| Any field | explicitly _Not Applicable_ | as the field's not-opted-out row above | as the field's not-opted-out row above | — | +| Any field | absent | — | — | — | **N/A vs absent:** a field explicitly set to _Not Applicable_ is treated as not-opted-out (grant-class) — pinned by today's USP tests and matching @@ -495,17 +521,29 @@ nothing. **Applicability and aggregation — ordered algorithm:** -1. **Section map (normative, pinned here — not "whatever GPP is current"):** - `US` national ↔ GPP section 7 (usnat); `US/CA` ↔ 8 (usca); `US/VA` ↔ 9 - (usva); `US/CO` ↔ 10 (usco); `US/UT` ↔ 11 (usut); `US/CT` ↔ 12 (usct). - Section versions are those published at this spec's date; adding a - section or version is a change to this map. -2. **Determine applicability from the resolved jurisdiction:** the - national section is applicable to any `us-privacy`-regime request; a - state section is applicable iff it maps to the resolved `US/`. - Foreign-state sections (a `usca` string on a `US/CO` request) and all - sections on non-`us-privacy` requests are **not applicable** and - contribute nothing. Regionless US traffic: national section only. +1. **Section map (normative, pinned here — not "whatever GPP is + current"), covering every section current code recognizes (7–23), not + a subset:** `US` national ↔ 7 (usnat); then the state sections — + `US/CA` ↔ 8, `US/VA` ↔ 9, `US/CO` ↔ 10, `US/UT` ↔ 11, `US/CT` ↔ 12, + `US/FL` ↔ 13, `US/MT` ↔ 14, `US/OR` ↔ 15, `US/TX` ↔ 16, `US/DE` ↔ 17, + `US/IA` ↔ 18, `US/NE` ↔ 19, `US/NH` ↔ 20, `US/NJ` ↔ 21, `US/TN` ↔ 22, + `US/MN` ↔ 23. Dropping to 7–12 would silently lose, e.g., a Texas + (section 16) sale opt-out. The implementation PR cross-checks this + list against the current decoder's section set; versions are those + published at this spec's date; adding a section or version is a change + to this map. +2. **Applicability gates grants only — never opt-outs.** A mapped + **opt-out** field (either subclass) is honored from **any** section on + **any** request, whatever the regime — this is §4's global-opt-out + rule, and filtering it by jurisdiction would make a French visitor's + `usnat SaleOptOut` simultaneously mandatory (§4) and ignored (here). + For **grants**: the national section is applicable to any + `us-privacy`-regime request; a state section is applicable iff it maps + to the resolved `US/`; foreign-state sections and all sections + on non-`us-privacy` requests grant nothing. Regionless US traffic: + national section only. A configured privacy state with no + state-specific section (e.g. MD, IN, KY, RI today) uses the national + section alone. 3. **State-over-national, per field:** where an applicable state section carries a field, it governs that field; the national section fills only fields the state section lacks. @@ -652,6 +690,8 @@ Consumers of the resolved set in this epic: | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | + | Integration persistent response cookies (hook spec §3) | `store-on-device` | Applied at mutation time from the request's resolved permissions; session cookies are the declared exemption (sign-off items 9–10) | + | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | With **no EC provider configured**, identity use fails closed: a cookie value present on the request never egresses anywhere — never vacuously @@ -668,11 +708,11 @@ Consumers of the resolved set in this epic: spec §6.1). Freshness is a **per-evidence-class contract**, because not every source carries a timestamp: - | Evidence class | Authoritative timestamp | Age reset | Max age | - | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | - | TCF consent | The record's `LastUpdated` | Only a record with a **newer** `LastUpdated` | Existing TCF expiry TTL | - | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (equality digest stored in provenance) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | - | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | + | Evidence class | Authoritative timestamp | Age reset | Max age | + | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | + | TCF consent | The record's `LastUpdated` | Only a record with a **newer** `LastUpdated` | Existing TCF expiry TTL | + | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (a **per-permission equality digest computed over only the applicable, aggregated §4.5 fields for that permission** — never the whole GPP record, or a CMP touching an unrelated notice field would mint a new digest and reset first-seen forever) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | + | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | Timestamps are compared with bounded clock-skew tolerance and future-dated values are clamped to receipt time. And every live diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 2d3d58e58..d34382907 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -201,8 +201,14 @@ pub trait EdgeCookieProvider { /// their page leg needs. One provider implements exactly one mode. pub enum Acquisition<'a> { ServerMint(&'a dyn ServerMint), // fn generate(&IdentityInput) -> EcId - ClientResolve(&'a dyn ClientResolve),// fn resolve_from_client(&Payload) -> EcId -} // + fn js_module_id() -> &str + ClientResolve(&'a dyn ClientResolve), +} +// ClientResolve::resolve_from_client(&ClientResolveContext) -> Result +// ctx is core-built: canonical publisher audience, verified session +// owner hash, clock, and the bounded payload — a bare payload could +// not verify audience binding, session binding, or expiry. +// VerifiedIdentity carries the identifier, reservation id, and expiry. +// ClientResolve::js_module_id() -> &str ``` (Names indicative; the shape is normative. `required_permissions` joins the @@ -294,15 +300,16 @@ one. This spec resolves that by **not having** the method on those traits All validation happens at **settings construction** — a misconfiguration is a startup error, never a request-time error and never a silent behavior change. -| Configuration state | Behavior | -| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `provider` names an unknown key | Startup error listing valid keys. | -| `provider` set, its `[ec.providers.]` block missing | Startup error. | -| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | -| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | -| No `provider`, no providers block | Valid: the neutral default for that concern. | -| `provider = "none"` (explicit stateless) | Valid, and the only way to combine statelessness with `legacy_providers`: minting stops, legacy readers keep existing identities resolvable and **withdrawable** (§6.1). Without this state, `hmac` → stateless would strand every live row in revoke-proof limbo. | -| A minting provider (or any `legacy_providers`) configured, but no identity-graph store configured or openable | **Startup error.** The lifecycle contract assumes graph persistence (§5); discovering its absence at first mint would be a request-time config failure, which this table exists to forbid. | +| Configuration state | Behavior | +| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `provider` names an unknown key | Startup error listing valid keys. | +| `provider` set, its `[ec.providers.]` block missing | Startup error. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | +| No `provider`, no providers block | Valid: the neutral default for that concern. | +| `rewrite_legacy = true` with a **client-resolve** active writer | **Startup error.** An organic request carries no signed client payload to mint from, and a later resolve POST meeting a different existing identity is a `409` by the client-cycle spec — the combination is incoherent until an authenticated linking/migration flow is specified. | +| `provider = "none"` (explicit stateless) | Valid, and the only way to combine statelessness with `legacy_providers`: minting stops, legacy readers keep existing identities resolvable and **withdrawable** (§6.1). Without this state, `hmac` → stateless would strand every live row in revoke-proof limbo. | +| A minting provider (or any `legacy_providers`) configured, but no identity-graph store configured or openable | **Startup error.** The lifecycle contract assumes graph persistence (§5); discovering its absence at first mint would be a request-time config failure, which this table exists to forbid. | Unknown fields inside every provider config block are rejected (`deny_unknown_fields` on all new settings structs — the pre-existing `Ec` @@ -371,19 +378,28 @@ The contract: evidence would rejuvenate stale authority); partner mappings copy with their **original per-field timestamps and expiry**, and the copy point is recorded in the transaction. - 3. **Fenced CAS replaces the old row with an alias record** targeting - the canonical. If the CAS loses to a concurrent pull/batch/identify - update, the rewrite **re-runs a reconciliation pass** under its - epoch — merging updates newer than the recorded copy point into the - canonical — and retries the CAS; an update that won the old row is - therefore never lost. + 3. **A per-key CAS on the source identity key replaces the row value + with the alias** (same address, `kind` discriminator — §6.3). This + is why rewrite requires the row store itself to offer CAS with + read-your-writes (§7 matrix): the participants must share + transactional primitives, or a source update landing through an + eventual replica after the copy could be silently dropped. If the + CAS loses to a concurrent pull/batch/identify update, the rewrite + **re-runs a reconciliation pass** under its epoch — re-reading the + source with the store's strongest read, merging updates newer than + the recorded copy point into the canonical — and retries; an update + that won the old row is therefore never lost. 4. The new cookie is emitted; the transaction marks complete. From step 3 on, every read or update through either cookie chases the - alias (one hop) to the single canonical row. **Chains stay single-hop**: - a later rewrite B→C retargets every alias pointing at B (the canonical - row records its inbound aliases; alias records are in the linearizable - class, so retargeting is fenced) so A points directly at C. The server + alias to the single canonical row. Chains are handled by **bounded + traversal with path compression**, not an inbound-alias index (an + index would need its own fenced schema, bounds, and concurrency rules + that nothing defined): traversal follows at most **4** hops with + visited-set cycle detection (deeper or cyclic → treated as row-read + failure, fail closed per §6.2); whenever a traversal crosses more than + one hop, it opportunistically CASes the first alias to point at the + final canonical, so chains converge to one hop without coordination. The server cannot observe `Set-Cookie` acceptance, so the alias stays live until a later request **presents the new cookie**, and in any case until a **finite retirement deadline** no shorter than the old cookie's maximum @@ -448,24 +464,31 @@ solves. **Physical key grammar.** Core constructs every key; providers supply only the bounded suffix: -| Record class | Key | Notes | -| ----------------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------- | -| Identity row (v2+) | `id///` | Suffix from `graph_key_suffix`, ≤ 128 bytes, KV-safe alphabet | -| Identity row (legacy hmac-v0) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar; no other class or provider may produce a matching key | -| Alias | `alias///` | Same suffix as the row it replaced | -| Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | -| Rewrite transaction | `rwx/` | One in-flight rewrite per family | -| Replay reservation | `resv////` | Client-cycle spec; payload id ≤ 128 bytes | - -Grammars are pairwise non-intersecting by their literal prefixes (plus the -reserved legacy grammar), which is what makes cross-class collision -impossible rather than unlikely. +| Record class | Key | Notes | +| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Identity row (non-hmac) | `id//` | Suffix from `graph_key_suffix`, ≤ 128 bytes, KV-safe alphabet. **No version segment** — the mint version lives in the row envelope; a versioned key would be circular (core would need the version to read the row that states the version) | +| Identity row (hmac, **every** version) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar for the whole provider, not just v0 — keeping all HMAC versions on the verbatim scheme is what keeps the 64-hex cluster prefix a literal key prefix for every HMAC row. (Passphrase rotation still changes a given IP's HMAC, so clusters split across rotation until old identities expire — inherent to rotation, declared, not a key-scheme artifact) | +| Alias | **The source identity key itself** — the alias is a value written _at_ the replaced row's address, distinguished by a `kind` discriminator in the JSON envelope | A separate `alias/…` address could never work: lookups by the old cookie hit the source key, and a single-key CAS cannot install a record at a different address | +| Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | +| Suppression (negative authority) | `sup/` | Per-permission suppression entries + timestamps; permission-exempt writes; consulted by every S2S recompute and partner-egress check (permission spec §4.3) | +| Rewrite transaction | `rwx/` | One in-flight rewrite per family | +| Replay reservation | `resv////` | Client-cycle spec; payload id ≤ 128 bytes | + +Grammars are pairwise non-intersecting by their literal prefixes (plus +the reserved hmac grammar), and every record value carries a `kind` +discriminator alongside its schema version — so a reader always knows +what it fetched, including where two classes deliberately share an +address (row vs. alias). **Wire schemas** (JSON, like identity rows; every class carries a schema version): the **alias record** holds target key, created-at, retirement deadline, and fencing epoch; the **family revocation record** holds the family ID, revoked-at, triggering signal class (§4.5 destructive column), -and epoch — deliberately no identity data, so it can outlive its members; +and a **family epoch** bumped on every revocation-state change (the +client-cycle commit CAS is conditioned on it) — deliberately no identity +data, so it can outlive its members; the **suppression record** holds +per-permission suppression entries with timestamps (strong class, +permission-exempt writes, permission model spec §4.3); the **rewrite transaction** holds source key, target key, copy point, state, and epoch; the **reservation** holds state, owner hash, lease epoch, outcome, and created-at (client-cycle spec). Field validation and @@ -521,12 +544,14 @@ Requirements: **per-record-class consistency requirements**, because "has KV" says nothing about whether revocation is observable: - | Record class | Required semantics | - | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | - | Family revocation records | **Strongly consistent (read-after-write) primitive required.** Cloudflare Workers KV is **not eligible** — its documentation says propagation may take "60 seconds or more", an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | - | Alias / rewrite-transaction records | **Linearizable fenced CAS required** (same primitive class as reservations). `rewrite_legacy = true` is rejected at startup on adapters lacking it (§6) | - | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | + | Record class | Required semantics | + | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | + | Family revocation records | **Strongly consistent (read-after-write) primitive required.** Cloudflare Workers KV is **not eligible** — its documentation says propagation may take "60 seconds or more", an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | + | Family suppression records | Same strong class as family revocation — negative authority must not lose races to stale replicas | + | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | + | Alias installs | Row-store **per-key CAS with read-your-writes** — the alias lives at the source identity key (§6.3), so the _row store itself_ must supply the CAS; a purely eventual row store cannot host rewrite. `rewrite_legacy = true` is rejected at startup unless both this and the transaction class are available (§6) | + | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | Each adapter's declaration is part of its wiring, drives the §6 capability-mismatch startup error, and every §6.2 runtime-failure row @@ -536,14 +561,19 @@ Requirements: cell marked _verify_ must be established before the depending feature is selectable on that adapter, and the filled matrix is normative: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | - | Graph persistence (eventual OK) | KV Store: yes | Local store: yes (dev-grade) | Workers KV: yes (eventually consistent) | Key-value: yes | - | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | - | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | - | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — client-cycle and `rewrite_legacy` unselectable until a primitive exists | Yes (in-process) | Durable Objects: possible, not wired | **No** | - | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | - | Device host evidence (JA4/H2) | Yes | No | No | No | + Cells distinguish **platform availability** (the host offers a + primitive) from **wired** (Trusted Server integrates it) — conflating + them is how a "yes" cell hides an unusable feature. Feature eligibility + requires wired, not merely available: + + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | + | Graph persistence (eventual OK) | KV Store: available + wired | Local store: available + wired (dev-grade) | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | + | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — client-cycle and `rewrite_legacy` unselectable until a primitive exists | Yes (in-process) | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 54f7fe4d4..ccd48ac0b 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -32,29 +32,29 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | -| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | -| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | -| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | -| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | -| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — strictly more protective | -| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | -| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | -| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | -| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | -| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | -| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | -| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | -| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | -| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | +| # | Decision (today) | After epic | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | +| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | +| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | +| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | +| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — opt-out effects are more protective; the same fields' not-opted-out values can also **newly grant P4**, which is not (both effects classified in permission spec §4.5) | +| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | +| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | +| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -119,21 +119,47 @@ Requirements: after **fleet convergence on N+1 is confirmed** — binaries first, convergence gate, then `ts config push`. A config mixing old and new fields (`[ec] passphrase` alongside `[ec] provider`) is **rejected** - by N+1, not reconciled. **Rollback is binaries-first too, in the - other direction**: N+2 → N+1 binaries roll back **keeping the new - config** (N+1 reads it fully — reverting config first would hand the - old shape to N+2 binaries that reject it). N+1 additionally - **rejects provider or version selections whose provenance it cannot - yet encode** — new-provider adoption waits for N+2, so no row is - minted that N+2 would misclassify. Every new config section - introduced by the epic follows this same compatibility rule, not - only `[ec]`. + by N+1, not reconciled. **N+1 is a full semantic reader and + enforcer for every N+2 record kind — not a field preserver.** + Preserving unknown JSON does not chase aliases, consult family + revocations, honor suppression records, or fail closed on + provenance; an N+1 that merely preserved would, after rollback, + treat aliased rows as missing and revoked identities as live. + Rollback tests therefore run the alias, family-revocation, + suppression, and provenance paths **on N+1** against N+2-written + data. **Rollback is binaries-first too, in the other direction** — + N+2 → N+1 binaries roll back keeping the new config (N+1 reads it + fully; reverting config first would hand the old shape to N+2 + binaries that reject it) — **with one precondition**: if an + N+2-only provider or version has been adopted, the fleet must first + converge on an N+1-compatible new-shape config (deselecting what + N+1 rejects, retaining the new provider's secrets as a **legacy + reader** so its minted identities keep resolving and stay + withdrawable until they expire — never "revert to the previous + config", which would strand them); only then do binaries roll back. + N+1 additionally **rejects provider or version selections whose + provenance it cannot yet encode** — new-provider adoption waits for + N+2, so no row is minted that N+2 would misclassify. Every new + config section introduced by the epic follows this same + compatibility rule, not only `[ec]`. - **Release N+2:** rejects `[ec] passphrase` at startup with a message naming the new location — not a generic unknown-field error (implementation note: producing the actionable message means keeping a deprecated `passphrase` field whose presence triggers the custom error). -2. **Graph-store readiness precedes everything.** Today the graph store +2. **Revocation-eligible storage is a per-adapter gate, and ungated + adapters migrate stateless.** Identity features require the adapter's + strong-consistency rows in the capability matrix (providers spec §7) + to be green: today that means Fastly must _verify_ its KV read + semantics, Cloudflare must wire a Durable-Object-class primitive, and + Spin must wire storage at all. Until an adapter passes the gate, its + migration fixture is **explicitly stateless** (`provider = "none"`, + no `[permissions]`-gated identity features) — calling an HMAC fixture + "valid" on an adapter that must reject identity features at startup + would make the required fixtures self-contradictory. Whether ungated + adapters go stateless or block the release is product sign-off + item 12. +3. **Graph-store readiness precedes everything.** Today the graph store is optional and EC generation succeeds without one; the epic's no-active-until-commit invariant (providers spec §5) makes it mandatory wherever a minting provider is configured — so a currently @@ -144,7 +170,7 @@ Requirements: row supports the features in use, providers spec §7) _before_ rolling N+1. This is a **declared breaking change** for graphless deployments (matrix row 12), not a side effect discovered at boot. -3. **The graph schema change is expand-contract, in lockstep with the +4. **The graph schema change is expand-contract, in lockstep with the binary sequence.** New rows carry fields v1 rows never had — provider/ version, per-permission grant evidence, policy revision, family ID, rewrite links — and two failure modes must be engineered away: a naive @@ -170,35 +196,35 @@ Requirements: new fields untouched if read-only, preserved semantically if read-modify-write on N+1, **test-proven lost on pre-N+1** (documenting why the floor is a floor); N+2-reader/N+1-written-row → full function. -4. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no +5. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the docs reaches if they miss one line — validated green and silently minted zero ECs. -5. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by +6. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by PR #838, deliberately not carried into this epic — providers spec §2) and `provider = "client-fixed"` are unknown keys and rejected like any other, so a config written against the PR #838 example cannot silently select a provider that no longer exists. -6. **Provider switches go through legacy readers.** Changing +7. **Provider switches go through legacy readers.** Changing `[ec] provider` on a deployment with live identities requires listing the outgoing provider in `[ec] legacy_providers` (providers spec §6.1) so existing cookies keep resolving and stay withdrawable; the guide documents the switch sequence and the retirement/cleanup step that ends it. -7. **The example config ships the migrated happy path**, uncommented: +8. **The example config ships the migrated happy path**, uncommented: `provider = "hmac"` with its block, `[geo] default_country`, and (for Fastly) the behavior-preserving `[device] provider = "fastly"` and `[geo] provider = "platform"` lines present with a comment stating what removing them changes. PR #838's example shipped the passphrase block uncommented with the selector commented out — steering operators directly into the silent-stateless state. -8. Every misconfiguration in the providers spec §6 table fails at +9. Every misconfiguration in the providers spec §6 table fails at **startup**. Request-time failure for a configuration error is a defect. -9. Config-store payload validation (`ts config push`) applies the same - rules — including `[permissions]` policy validation (permission spec - §3.3) — so a bad config is rejected at push time, before any instance - restarts into it. +10. Config-store payload validation (`ts config push`) applies the same + rules — including `[permissions]` policy validation (permission spec + §3.3) — so a bad config is rejected at push time, before any instance + restarts into it. ## 5. Minimal-divergence migration recipe (operator-facing) @@ -343,28 +369,23 @@ global honoring of opt-out signals is unconditional. ## 8. Product decisions requiring explicit sign-off These are decisions this spec set makes that #838 had not already made (or -made differently). Each must be ratified by maintainers before -implementation — an unratified row reverts to open, not to silently -implemented: - -1. Opt-outs are honored globally and destructive ones irreversibly - withdraw identities outside the jurisdiction defining the signal - (permission spec §4, §4.2). -2. Sale opt-outs (GPP and USP) control both P1 and P4 and destroy the - identity (§4.5). -3. Sharing / targeted-advertising opt-outs remove P4 but intentionally - retain the stored identity (§4.5). -4. US contextual auctions continue during opt-out, with identity removed - (permission spec §7 dispatch matrix). -5. Regionless US traffic is treated as non-regulated unless the operator - chooses country-wide gating (permission spec §3.4). -6. Full consent strings continue downstream, and raw consent snapshots - are retained in graph rows for audit (providers spec §6.3). -7. Legacy batch-sync traffic is rejected until live-browser provenance - backfill occurs (§6.4 of this spec; permission spec §7). -8. Proxy / click / Testlight forwarding becomes newly gated by P1 ∧ P4 - (§2 row 11b). -9. Integration-owned response cookies are inside the permission model: - persistent cookies require `store-on-device` at apply time and a - declared registration; session cookies are the narrow exemption - (response-hook spec §3). +made differently). **Implementation is blocked while any row is `open`**; +each row needs an owner, a status, and a link to its decision record — +an unratified row reverts to open, not to silently implemented. + +| # | Decision | Where | Owner | Status | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | --------------------- | ------ | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising opt-outs remove P4 but retain the stored identity | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration persistent cookies inside the permission model (P1-gated, declared registration) | hook §3; permission §7 | maintainers | open | +| 10 | Session cookies exempt from `store-on-device` even when they carry a stable identifier | hook §3 | maintainers + legal | open | +| 11 | A single failed destructive withdrawal may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | From 09e54e96a97562bc9f5af6b72b67c1b5081cc1af Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 31 Jul 2026 22:28:47 -0700 Subject: [PATCH 09/14] Address seventh review: descope the epic periphery, close core findings, add a review ledger The recurring theme - browser-side acquisition, integration-owned identifiers, rewrite, and pre-existing state generating blockers while the core holds - is answered structurally this round: Descope (sign-off item 15, ratify or veto): - The client-cycle spec is demoted to a deferred informative draft: no production adapter has its CAS-class primitive, it has no consumer, and its findings no longer block core ratification. Within it: the ownerless first-presenter mode is removed outright (risk acceptance does not make a security invariant true, and its orphan cleanup was unimplementable - the server cannot observe Set-Cookie acceptance); the page leg is permission-gated before the module executes; the cross-key commit atomicity gap is recorded as open question 0. - rewrite_legacy is cut from the epic into a recorded deferral carrying its open problems (retention lineage, eventual-store visibility, chain stranding, cluster inflation) as the entry bar for a future spec; provider switching is served by legacy readers alone; the key is rejected as unknown. - The hook ships headers-only: the write-side cookie gate never modeled reading, using, forwarding, or withdrawing an integration cookie (or its P4 nature), so cookie operations defer to a follow-up spec with that full model as entry bar; sign-off items 9/10 updated. Core fixes (new P1/P2): - Recognized rowless legacy cookies (graphless deployments) get a permission-gated, race-safe adoption transaction; no egress before adoption; withdrawal needs no adoption (derived family ID); matrix row 13. - Unreferenced [ec.providers.*] blocks are startup errors - a dropped legacy_providers entry must not silently strand identities. - Physical key delimiters are backend-safe and validated per adapter (Fastly forbids / in prefix queries); cluster eligibility requires a queryable physical prefix, checked at startup. - Cluster size means live identity rows: kind/liveness filtering, short-TTL tombstone inflation declared conservative. - Validation split into structural (push + startup) and deployment (startup; optional push pre-check via a machine-readable capability profile) - 'same validation at push' was unimplementable. - The rollback floor is N+2 writer activation itself, recorded as a durable schema-floor marker - not an unobservable first-row fact. - Integration IDs are startup-unique. Previously-open items closed: - GPP map completed against the official registry: section 6 (US Privacy as GPP section) and 24-27 (MD/IN/KY/RI - the earlier claim they had no sections was wrong). - State-over-national applies to grants only; a national opt-out can never be erased by a state field. - Suppression records completed: full negative-state coverage, monotonic per-permission ordering, re-consent clearing, write-failure semantics. - The raw-TCF dispatch arm triggers on TCF-sourced effective records including the persisted-KV fallback. - N+1 writes the safety-critical record kinds (family, suppression) and accepts N+2-only providers as legacy readers, making the rollout boundary safe in both directions. - Request-side integration views are identity-redacted; the legacy RequestFilterEffects.response_headers channel is folded into the hook. - must-understand and friends join the sticky directive set; the Axum matrix cell is honest (in-process, non-durable, dev-only). Process: docs/superpowers/specs/pr986-review-ledger.md records the disposition of every finding from all seven review rounds (fixed / reapplied-after-batch-loss / partial-refixed / superseded / deferred / open), so coverage is auditable per finding rather than claimed in summaries. --- ...26-07-30-client-cycle-ec-resolve-design.md | 51 ++++-- ...integration-response-header-hook-design.md | 63 ++++---- .../2026-07-30-permission-model-design.md | 99 +++++++----- .../2026-07-30-pluggable-providers-design.md | 149 +++++++++--------- ...07-30-provider-migration-rollout-design.md | 74 +++++---- docs/superpowers/specs/pr986-review-ledger.md | 132 ++++++++++++++++ 6 files changed, 385 insertions(+), 183 deletions(-) create mode 100644 docs/superpowers/specs/pr986-review-ledger.md diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 3cfeb09d4..c1ea840ed 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -1,7 +1,12 @@ # Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint -**Status:** Draft — **prerequisites unmet; do not implement against this spec -until its open questions (§7) are resolved in a dedicated issue** +**Status:** **Deferred — informative draft, not part of the epic's +normative set.** No production adapter has the required CAS-class +primitive (providers spec §7 matrix), the feature has no concrete +consumer, and successive reviews keep finding open protocol questions +(§7). It re-enters the epic only through its own dedicated issue, with +this document as the starting bar — findings against this spec do not +block ratification of the core specs. **Author:** Engineering **Issue references:** none yet (this spec exists to force one; #778 does not cover this feature) @@ -69,15 +74,19 @@ Everything in this spec follows from that. that are signed by an expected party, **audience-bound** to this publisher, and **expiring**. Audience binding and expiry alone do not mitigate replay — a captured token installs in another browser for the - whole validity window. **Production schemes require session binding** - (a server-issued nonce the payload must embed): one-time consumption - alone limits multiplicity but proves nothing about _which_ browser - redeems first — a captured bearer payload can simply win the race — so - it is defense-in-depth, not the mitigation. First-presenter - at-most-once semantics may ship **only** as an explicitly accepted - posture recorded in the feature's issue, together with a specified - orphan-row cleanup path. "Single-use where the scheme allows" is not a - mitigation. + whole validity window. **Session binding is required, with no ownerless + escape hatch** (a server-issued nonce the payload must embed): + one-time consumption alone limits multiplicity but proves nothing + about _which_ browser redeems first — a captured bearer payload can + simply win the race — so it is defense-in-depth, not the mitigation. + An earlier draft allowed a "first-presenter, product-accepted" + ownerless mode; it is **removed**: risk acceptance does not make a + security invariant true, and the promised orphan cleanup was + unimplementable anyway — the server cannot observe `Set-Cookie` + acceptance, so it cannot distinguish a lost-response orphan from a + successful-but-dormant identity. A scheme that cannot embed the + session nonce cannot ship. "Single-use where the scheme allows" is + not a mitigation. 3. **Preserve the identity-graph invariant.** The cookie is set only after the corresponding graph row is written, mirroring the organic path. Graph unavailable → no cookie, same as organic generation. @@ -158,10 +167,7 @@ Everything in this spec follows from that. have the `Set-Cookie` re-emitted — which is precisely how a legitimate browser whose original response was lost recovers on retry, so a committed graph row never strands as an orphan for the intended - browser; anyone else gets a terminal response with no cookie. In the - explicitly-accepted at-most-once posture (no owner hash), a lost - response is an **orphan row** handled by the specified cleanup path. - The **same-identity no-op of §3.8 first checks the family revocation + browser; anyone else gets a terminal response with no cookie. The **same-identity no-op of §3.8 first checks the family revocation record**, and "revocation wins" is enforced by an explicit linearization point, not by read strength alone — a linearizable read followed by a separate commit still loses the race (read "not @@ -181,6 +187,16 @@ Everything in this spec follows from that. ## 4. Requirements on the page script +**The page leg is permission-gated before it executes.** The browser +module obtains or derives a vendor identity — vendor contact, stable +identifier in hand — so injecting it whenever the provider is merely +_selected_ would run identity code for a visitor who denied everything, +with only the later POST refused. The module is injected/activated only +when the request's resolved permissions already satisfy the provider's +complete `required_permissions()`, and the page leg is a listed row in +the permission spec's §7 enforcement inventory (deferred alongside this +feature). + - The re-post guard must not depend on reading an HttpOnly cookie. Either the server injects a "resolved" marker the script _can_ read (a non-identity companion cookie or an injected page variable), or the @@ -219,6 +235,11 @@ sentence as the only guardrail. ## 7. Open questions — to be settled in the feature's issue before any code +0. The commit path spans keys (reservation, identity row, family-epoch + CAS): the cross-key atomicity or saga/compensation design is + **undefined** — the single-key CAS steps are specified, their + composition is not. + 1. Which concrete vendor scheme is the first real consumer, and does its envelope format satisfy §3.2 (audience binding, expiry)? If no concrete consumer exists, the feature waits — the demo provider is not a diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 5d35f23a5..a8a7ef7cb 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -67,7 +67,8 @@ mutators to the outbound response for HTML document responses it processed. storage subject to revalidation; `private` forbids shared storage), so "replace `private` with the stronger `no-cache`" would make a personalized response shared-storable. The merge: each of `no-store`, - `no-cache`, `private` is **sticky** — present in the snapshot or the + `no-cache`, `private`, `must-revalidate`, `proxy-revalidate`, + `must-understand`, and `no-transform` is **sticky** — present in the snapshot or the mutation ⇒ present in the final response, independently; `public` is dropped whenever any restriction is present; `max-age`/`s-maxage` may only shrink relative to the snapshot; `stale-while-revalidate`/ @@ -94,22 +95,21 @@ mutators to the outbound response for HTML document responses it processed. digest for bytes the hook never saw, corrupts responses or poisons caches), the `x-ts-*` namespace, and the consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, - `ts-eids`, and the other `ts-*` cookies core owns. Integration cookies are **inside the permission model, not beside it** - (product sign-off item 9, migration spec §8) — otherwise the hook is a - door around the EC gate: an integration could write a durable - identifier while `store-on-device` is denied. `append_set_cookie` - therefore requires the cookie name to be **declared at registration** - with a stated purpose and maximum retention; a **persistent** cookie - (any `Max-Age`/`Expires`) is applied only when the request's resolved - permissions include `store-on-device`, while **session cookies** (no - persistence attributes) are the narrow, documented exemption. - Cookie operations go through a **typed cookie builder** that enforces - the declared lifetime ceiling, domain/path scope, and security - attributes (`Secure`, `SameSite`) — not a free-form string; **deletion - cookies (expiry of the integration's own declared names) remain - possible when `store-on-device` is denied**, since removing state must - never require the permission to keep it. Undeclared cookie names are - rejected like reserved ones. An integration may never set or expire a + `ts-eids`, and the other `ts-*` cookies core owns. **Cookie operations are deferred out of the v1 hook — headers only.** + The write-side gate alone ("persistent cookies require P1") was shown + insufficient: it never modeled reading, using, forwarding, or + withdrawing the cookie — a P1-granted-then-withdrawn integration + cookie would keep arriving on every request with nothing required to + expire, hide, or stop egressing it, and an advertising-identifier + cookie needs P4 the contract never expressed. Rather than ship + "inside the permission model" as a claim the model does not back, + `append_set_cookie` and the typed cookie builder are **deferred** to a + follow-up spec whose entry bar is: declared per-cookie required + permissions, a typed authorized request-side view, stripping from + unauthorized integration/proxy inputs, mandatory expiry on destructive + P1 withdrawal, and startup-unique (name, domain, path) ownership. + Integration IDs are startup-unique regardless. Until then the + operation set is headers-only, and `Set-Cookie` is fully reserved. reserved cookie name. Violations are rejected at the operation layer (§2) and logged at `warn` with the integration id. The reserved lists are single constants next to the definitions they protect, not duplicated in the @@ -174,19 +174,24 @@ processed documents (§6). ## 4. Done-when (from #782, sharpened) 1. Trait + builder + registry application, each public item documented. -2. **At least one real consumer ships in the same PR** — an existing +2. **The pre-existing `RequestFilterEffects.response_headers` channel is + folded into the hook in the same PR** — its outputs become hook + operations subject to the same validation, reserved surface, budgets, + and invariant pass, or the channel is removed; a second, unvalidated + header path bypassing the hook defeats every rule above. +3. **At least one real consumer ships in the same PR** — an existing integration registering a mutator for a real need (or, failing a real need, the feature waits; scaffolding with only self-referential tests is dead code and will be removed). -3. Every adapter applies mutations on its outbound path, with a per-adapter +4. Every adapter applies mutations on its outbound path, with a per-adapter route test asserting an integration-set header appears in the response. -4. A parity-suite case asserts identical mutation behavior across adapters. -5. Reserved-surface, append/replace, operation-limit, and erroring-mutator +5. A parity-suite case asserts identical mutation behavior across adapters. +6. Reserved-surface, append/replace, operation-limit, and erroring-mutator semantics covered by unit tests. -6. **Every row of the §3a eligibility matrix has a test** — streaming, +7. **Every row of the §3a eligibility matrix has a test** — streaming, cache-hit, pass-through, redirect, error, and 304 each proven to run or not run the hook — not merely one positive header test per adapter. -7. Cache/privacy invariant tests, one per restriction source and shape: +8. Cache/privacy invariant tests, one per restriction source and shape: cookie appended + public `Cache-Control` replacement → private/no-store, surrogate stripped; **core-private cookieless** processed HTML + public replacement → restriction preserved; **origin-private cookieless** processed HTML that retained the @@ -199,13 +204,11 @@ processed documents (§6). ## 5. Size and sequencing -This is a modest feature plus tests, with zero coupling to the provider -architecture — but its **cookie operations are coupled to the permission -model** (§3; the gate is a listed enforcement point in the permission -spec §7 inventory), so the claim of total independence is retired: the -header-mutation portion may land whenever its first real consumer is -identified (§4.2), while `append_set_cookie` activates only **after** the -permission model PR, and registers as unavailable before it. If no consumer +This is a modest feature plus tests with zero coupling to the provider +architecture or, in its v1 headers-only form (§3), to the permission +model. It lands whenever its first real consumer is identified (§4.2); +cookie operations arrive only with their own follow-up spec (§3) and its +permission-model coupling. If no consumer materializes, it does not land; being unblocked is not a reason to ship scaffolding. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 8a0818166..4a3feb629 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -423,12 +423,23 @@ and the fail-closed marker: which the refusal just unset, and identity rows may be eventually consistent, so a stale replica could resurrect a P4 grant after a targeted-advertising opt-out. The fix is a **suppression record** in - the strongly consistent class (providers spec §6.3: `sup/`), - carrying per-permission suppression entries with timestamps. Writing it - is **permission-exempt** (clearing authority is protective, like - revocation), and **every S2S recompute and partner-egress check - consults it**: a suppressed permission is unset whatever the row's - provenance says, so no eventual-consistency edge can restore it. + the strongly consistent class (providers spec §6.3: `sup/`). + Its semantics are complete, not sketched: entries are **per permission** + with the triggering state (refusal or non-destructive opt-out — the + full negative-state coverage; absence writes nothing) and an + authoritative timestamp; ordering is **monotonic per permission** — + a write with an older timestamp than the stored entry is a no-op, so + replays cannot regress the state; **re-consent clears**: a live + resolution carrying an accepted grant with a newer authoritative + timestamp than the suppression entry supersedes it (recorded as a + clear entry in the same record — still an exempt write, since it only + ever reflects the live resolution); and **write failure fails closed + for the live request** (the refusal's effect stands for this response) + while S2S may transiently honor prior authority until the retry lands — + logged, metered, and covered by the degraded-mode rule. Every S2S + recompute and partner-egress check consults the record: a suppressed + permission is unset whatever the row's provenance says, so no + eventual-consistency edge can restore it. - **The cookie expires only after the family record commits.** - **If the family-record write itself fails, nothing durable exists** — the cookie stays and the durable client-side signal (GPC, CMP-stored @@ -522,16 +533,19 @@ nothing. **Applicability and aggregation — ordered algorithm:** 1. **Section map (normative, pinned here — not "whatever GPP is - current"), covering every section current code recognizes (7–23), not - a subset:** `US` national ↔ 7 (usnat); then the state sections — - `US/CA` ↔ 8, `US/VA` ↔ 9, `US/CO` ↔ 10, `US/UT` ↔ 11, `US/CT` ↔ 12, - `US/FL` ↔ 13, `US/MT` ↔ 14, `US/OR` ↔ 15, `US/TX` ↔ 16, `US/DE` ↔ 17, - `US/IA` ↔ 18, `US/NE` ↔ 19, `US/NH` ↔ 20, `US/NJ` ↔ 21, `US/TN` ↔ 22, - `US/MN` ↔ 23. Dropping to 7–12 would silently lose, e.g., a Texas - (section 16) sale opt-out. The implementation PR cross-checks this - list against the current decoder's section set; versions are those - published at this spec's date; adding a section or version is a change - to this map. + current"), matching the official IAB registry in full:** section 6 ↔ + the **US Privacy string carried as a GPP section** (it maps to the USP + rows of the field table, not to nothing); `US` national ↔ 7 (usnat); + the state sections — `US/CA` ↔ 8, `US/VA` ↔ 9, `US/CO` ↔ 10, + `US/UT` ↔ 11, `US/CT` ↔ 12, `US/FL` ↔ 13, `US/MT` ↔ 14, `US/OR` ↔ 15, + `US/TX` ↔ 16, `US/DE` ↔ 17, `US/IA` ↔ 18, `US/NE` ↔ 19, `US/NH` ↔ 20, + `US/NJ` ↔ 21, `US/TN` ↔ 22, `US/MN` ↔ 23, **`US/MD` ↔ 24, + `US/IN` ↔ 25, `US/KY` ↔ 26, `US/RI` ↔ 27** (an earlier draft wrongly + claimed MD/IN/KY/RI had no section). A truncated map silently loses + opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. + The implementation PR cross-checks this list against both the current + decoder's section set and the official registry; adding a section or + version is a change to this map. 2. **Applicability gates grants only — never opt-outs.** A mapped **opt-out** field (either subclass) is honored from **any** section on **any** request, whatever the regime — this is §4's global-opt-out @@ -544,9 +558,14 @@ nothing. national section only. A configured privacy state with no state-specific section (e.g. MD, IN, KY, RI today) uses the national section alone. -3. **State-over-national, per field:** where an applicable state section - carries a field, it governs that field; the national section fills only - fields the state section lacks. +3. **State-over-national, per field — for grants only:** where an + applicable state section carries a field, its value governs that + field's **grant** derivation; the national section fills only fields + the state section lacks. This precedence **never suppresses an + opt-out**: a national-section opt-out stands even where the state + section's same field says not-opted-out — step 2's global rule wins, + or a state string could erase a globally authoritative national + opt-out. 4. **Aggregate across what remains applicable:** an opt-out (of either subclass) in any applicable field beats a grant from another — restrictive aggregation. @@ -677,21 +696,21 @@ Consumers of the resolved set in this epic: inventory, normative per path (one test per row; a denylist check proves no ungated egress exists): - | Path | Required permissions | Notes | - | ---------------------------------------------------------------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | - | EC-derived auction request IDs | both purposes | Derived values are identity | - | Page-bids path | both purposes | | - | Bidstream EIDs | both purposes | The one gate PR #838 had | - | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | - | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | - | Pull sync (browser-request-scoped partner exchange) | both purposes, from the **live** request resolution | Pull sync is created from a browser request and checks the live `EcContext` today — it keeps using the live P1 ∧ P4 decision plus the family revocation state (§4.3); stored provenance is never a substitute for available live evidence | - | Batch sync (context-free S2S partner exchange) | both purposes, from **stored provenance** | The only truly signal-less path; authority rules below. Today's handler only authenticates and checks row state, so this gate is **declared hardening** (migration spec §2) | - | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | - | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | - | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | - | Integration persistent response cookies (hook spec §3) | `store-on-device` | Applied at mutation time from the request's resolved permissions; session cookies are the declared exemption (sign-off items 9–10) | - | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | + | Path | Required permissions | Notes | + | ---------------------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | + | EC-derived auction request IDs | both purposes | Derived values are identity | + | Page-bids path | both purposes | | + | Bidstream EIDs | both purposes | The one gate PR #838 had | + | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | + | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | + | Pull sync (browser-request-scoped partner exchange) | both purposes, from the **live** request resolution | Pull sync is created from a browser request and checks the live `EcContext` today — it keeps using the live P1 ∧ P4 decision plus the family revocation state (§4.3); stored provenance is never a substitute for available live evidence | + | Batch sync (context-free S2S partner exchange) | both purposes, from **stored provenance** | The only truly signal-less path; authority rules below. Today's handler only authenticates and checks row state, so this gate is **declared hardening** (migration spec §2) | + | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | + | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | + | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | + | Integration persistent response cookies | `store-on-device` (+ P4 where the cookie is an advertising identifier) | **Deferred with the hook's cookie surface** — the write-side gate alone was insufficient (read/use/forward/withdrawal unmodeled), so cookie operations ship only with the full model; this row and the client-cycle **page leg** (module injection gated on the provider's full declaration) join the inventory when their features do, and the §5.3 no-geo guard's consumer list grows with them | + | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | With **no EC provider configured**, identity use fails closed: a cookie value present on the request never egresses anywhere — never vacuously @@ -739,12 +758,12 @@ Consumers of the resolved set in this epic: 4. **Server-side auction dispatch** — gated on the policy `regime` class, normatively: - | Regime | Dispatch rule | Preserves | - | ------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | - | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | - | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | - | `none` | Dispatch proceeds. | Today's non-regulated arm | - | **Any regime, raw TCF signal present** — a TC string on the request or a GPP section-2 hint, detected **before decoding** | The `gdpr` row applies: dispatch requires the _effective_ record to be decodable, unexpired, and consenting to Purpose 1. A **malformed or expired** raw signal therefore blocks dispatch — today a malformed raw TCF blocks, and gating this arm on decodability would have silently relaxed that. A US or non-regulated request carrying a Purpose 1 refusal is likewise blocked. | Today's raw-signal arm — **must not regress** | + | Regime | Dispatch rule | Preserves | + | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | + | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | + | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | + | `none` | Dispatch proceeds. | Today's non-regulated arm | + | **Any regime, TCF-sourced effective record** — a raw TC string on the request, a GPP section-2 hint (both detected **before decoding**), or a persisted-KV fallback record of TCF origin (§4.4) | The `gdpr` row applies: dispatch requires the _effective_ record to be decodable, unexpired, and consenting to Purpose 1. A **malformed or expired** raw signal therefore blocks dispatch — today a malformed raw TCF blocks, and gating this arm on decodability would have silently relaxed that. A US or non-regulated request carrying a Purpose 1 refusal is likewise blocked. | Today's raw-signal arm — **must not regress** | The **compiled-in fallback policy has `regime = "gdpr"`** (§3.1) — the no-policy posture must be the most protective for dispatch too, and a diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index d34382907..c92253a3d 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -142,6 +142,16 @@ Three global rules sit above every provider: a prefix-listing case. For `hmac`, the equivalence fixtures pin: uppercase/lowercase hex-prefix variants are equivalent; suffix case is preserved and significant. +- **Cluster size means live identity rows.** Prefix counting lists + identity-row keys; family, suppression, reservation, and transaction + records live in other namespaces and never inflate a count. Member + tombstones share the identity key but carry the short cleanup TTL, so + their inflation is transient and biases conservative (an over-count + trips the trust threshold toward denial, never toward extra writes); + the listing filters by the value's `kind`/liveness within the existing + list limit where the backend returns values, and the residual + over-count where it cannot is declared. Aliases are reserved-future + (§6.1) and excluded by `kind` when they exist. - **No-cluster behavior is still defined.** A provider without cluster support deduplicates pull-sync by canonical graph key and redacts logs with a fixed-length hash of the graph key; `cluster_fallback` (§6.1) @@ -241,6 +251,20 @@ header emission; the identity exists durably from that moment. A graph-commit failure means the mint never happened: no cookie, no egress, error logged, the next request retries. +**Pre-existing cookies without rows are adopted, not orphaned.** Current +graphless deployments have minted cookies with no row; under +no-active-until-commit those identities could never be used again and — +without care — never withdrawn. The contract: a recognized legacy cookie +with no reachable row triggers a **permission-gated, race-safe adoption** +on a live request — gated exactly like minting (`store-on-device`), +implemented as create-if-absent on the verbatim key (concurrent adopters +converge: same key, same deterministic family ID), provenance from the +live resolution. Until adoption succeeds the cookie **never egresses**; +**withdrawal works without adoption** — the deterministic family ID +(permission model spec §4.3) needs no row, so a first post-upgrade +request that is an opt-out revokes and expires the cookie with zero +migrated state. Migration matrix row 13 declares this path. + **Egress is typed, not policed.** The inventory-and-denylist test (permission model spec §7) is a backstop, but conventions do not survive new code — the ungated proxy/click/Testlight paths happened precisely @@ -254,7 +278,13 @@ bids, sync, identify, forwarding) accept `AuthorizedIdentity` and nothing weaker — an unparameterized wrapper would let a P1-only identity flow into an ORTB request. A future bypass then requires deliberately reconstructing the raw string — visible in review — -rather than passing along what was already in hand. +rather than passing along what was already in hand. The same boundary +applies **request-side**: integration-facing request views (proxy +interfaces, filter inputs, forwarded header/cookie maps) receive +**identity-redacted** views — the EC cookie and identity headers are +stripped unless the path holds `AuthorizedIdentity` — +because the ungated forwarding paths of PR #838 were exactly integrations +reading the raw request. The gate applies to EC providers **only**. Geo and device are ungated for two _different_ reasons, stated separately because only one of them is @@ -300,16 +330,17 @@ one. This spec resolves that by **not having** the method on those traits All validation happens at **settings construction** — a misconfiguration is a startup error, never a request-time error and never a silent behavior change. -| Configuration state | Behavior | -| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `provider` names an unknown key | Startup error listing valid keys. | -| `provider` set, its `[ec.providers.]` block missing | Startup error. | -| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | -| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | -| No `provider`, no providers block | Valid: the neutral default for that concern. | -| `rewrite_legacy = true` with a **client-resolve** active writer | **Startup error.** An organic request carries no signed client payload to mint from, and a later resolve POST meeting a different existing identity is a `409` by the client-cycle spec — the combination is incoherent until an authenticated linking/migration flow is specified. | -| `provider = "none"` (explicit stateless) | Valid, and the only way to combine statelessness with `legacy_providers`: minting stops, legacy readers keep existing identities resolvable and **withdrawable** (§6.1). Without this state, `hmac` → stateless would strand every live row in revoke-proof limbo. | -| A minting provider (or any `legacy_providers`) configured, but no identity-graph store configured or openable | **Startup error.** The lifecycle contract assumes graph persistence (§5); discovering its absence at first mint would be a request-time config failure, which this table exists to forbid. | +| Configuration state | Behavior | +| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `provider` names an unknown key | Startup error listing valid keys. | +| `provider` set, its `[ec.providers.]` block missing | Startup error. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | +| A `[ec.providers.]` block referenced by neither `provider`, `legacy_providers`, nor a `versions`/`mint_version` chain | **Startup error.** An unreferenced block is almost always a dropped `legacy_providers` entry — accepted silently, it strands every identity that provider minted: unresolvable and, worse, non-withdrawable. | +| No `provider`, no providers block | Valid: the neutral default for that concern. | +| `rewrite_legacy` present at all (deferred out of the epic, §6.1) | **Startup error** — unknown key; transparent re-mint returns only with its own spec. | +| `provider = "none"` (explicit stateless) | Valid, and the only way to combine statelessness with `legacy_providers`: minting stops, legacy readers keep existing identities resolvable and **withdrawable** (§6.1). Without this state, `hmac` → stateless would strand every live row in revoke-proof limbo. | +| A minting provider (or any `legacy_providers`) configured, but no identity-graph store configured or openable | **Startup error.** The lifecycle contract assumes graph persistence (§5); discovering its absence at first mint would be a request-time config failure, which this table exists to forbid. | Unknown fields inside every provider config block are rejected (`deny_unknown_fields` on all new settings structs — the pre-existing `Ec` @@ -357,56 +388,22 @@ The contract: unavailable (a cookie with no reachable row). Removing a version entry is a retirement subject to the same evidence rules as retiring a legacy reader (migration spec §6). -- A cookie recognized by a legacy reader is a live identity for - read/withdrawal purposes; whether it is transparently re-minted under the - active writer is a per-deployment choice - (`[ec] rewrite_legacy = true|false`), and re-minting is subject to the - full minting gate of §5. -- **Rewrite is a persistent fenced transaction aliasing to one canonical - row — no dual-write window, no duplicate targets, no lost updates.** - The steps, each resumable because the transaction record (its own - linearizable record class, §7 matrix) is written **first** and pins the - chosen target key and fencing epoch: - 1. **Transaction record** commits: source key, target key, epoch, - state. A crashed rewrite retried later reads it and resumes with - the **same** target — a fresh random target (and an orphaned first - one) cannot exist, and any target row without a committed transaction - pointing at it is garbage-collectable by that absence. - 2. **Canonical row** commits under the pinned target key, sharing the - old row's revocation family ID (permission model spec §4.3); - provenance is the **current live resolution** (copying old consent - evidence would rejuvenate stale authority); partner mappings copy - with their **original per-field timestamps and expiry**, and the - copy point is recorded in the transaction. - 3. **A per-key CAS on the source identity key replaces the row value - with the alias** (same address, `kind` discriminator — §6.3). This - is why rewrite requires the row store itself to offer CAS with - read-your-writes (§7 matrix): the participants must share - transactional primitives, or a source update landing through an - eventual replica after the copy could be silently dropped. If the - CAS loses to a concurrent pull/batch/identify update, the rewrite - **re-runs a reconciliation pass** under its epoch — re-reading the - source with the store's strongest read, merging updates newer than - the recorded copy point into the canonical — and retries; an update - that won the old row is therefore never lost. - 4. The new cookie is emitted; the transaction marks complete. - - From step 3 on, every read or update through either cookie chases the - alias to the single canonical row. Chains are handled by **bounded - traversal with path compression**, not an inbound-alias index (an - index would need its own fenced schema, bounds, and concurrency rules - that nothing defined): traversal follows at most **4** hops with - visited-set cycle detection (deeper or cyclic → treated as row-read - failure, fail closed per §6.2); whenever a traversal crosses more than - one hop, it opportunistically CASes the first alias to point at the - final canonical, so chains converge to one hop without coordination. The server - cannot observe `Set-Cookie` acceptance, so the alias stays live until a - later request **presents the new cookie**, and in any case until a - **finite retirement deadline** no shorter than the old cookie's maximum - lifetime plus rollout skew. An interrupted rewrite at any step leaves - the old cookie resolving — no state in which neither identity works. - **Withdrawal through either cookie revokes the shared family.** - +- **`rewrite_legacy` is deferred out of the epic.** Transparent re-mint + under the active writer required primitives no production adapter has + (row-store CAS with read-your-writes plus a linearizable transaction + class — §7 matrix), and successive reviews kept surfacing open protocol + problems: retention lineage (a rewritten 364-day-old row either + rejuvenates the identity or leaves a year-long cookie pointing at an + expiring row — a lineage expiry must be pinned across canonical row, + alias, family record, and emitted cookie), alias visibility under + eventual stores, chain stranding after repeated migrations, and + cluster-count inflation by alias keys. Those are recorded here as the + entry bar for a future `rewrite_legacy` spec. Within the epic, provider + switching is served by **legacy readers alone**: old identities keep + resolving and stay withdrawable; they are never transparently + re-minted. The `rewrite_legacy` key is rejected at startup as unknown, + and the alias record class exists in the key grammar (§6.3) only as + reserved-for-future — nothing in the epic writes one. - Retiring a legacy reader is the explicit end of those identities: the migration guide documents the cleanup procedure (migration spec §6). - Tests: switch active provider → request with old cookie → identity still @@ -478,7 +475,17 @@ Grammars are pairwise non-intersecting by their literal prefixes (plus the reserved hmac grammar), and every record value carries a `kind` discriminator alongside its schema version — so a reader always knows what it fetched, including where two classes deliberately share an -address (row vs. alias). +address (row vs. alias). The `/` shown in key sketches is **notation, +not the wire byte**: the physical segment delimiter is a +**backend-safe character validated per adapter** — Fastly permits `/` in +keys but not in prefix _queries_, so a slash-delimited `id//…` +key could never be cluster-listed there despite the matrix marking +prefix listing supported. The reference delimiter is `:`; each adapter's +capability declaration includes which delimiter its prefix queries +accept, and cluster-capability eligibility for a provider requires its +physical prefix to be queryable on that backend — checked at startup, +not discovered at the first cluster count. (hmac verbatim keys contain +no delimiter before the 64-hex prefix and are unaffected.) **Wire schemas** (JSON, like identity rows; every class carries a schema version): the **alias record** holds target key, created-at, retirement @@ -550,7 +557,7 @@ Requirements: | Family revocation records | **Strongly consistent (read-after-write) primitive required.** Cloudflare Workers KV is **not eligible** — its documentation says propagation may take "60 seconds or more", an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | | Family suppression records | Same strong class as family revocation — negative authority must not lose races to stale replicas | | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | - | Alias installs | Row-store **per-key CAS with read-your-writes** — the alias lives at the source identity key (§6.3), so the _row store itself_ must supply the CAS; a purely eventual row store cannot host rewrite. `rewrite_legacy = true` is rejected at startup unless both this and the transaction class are available (§6) | + | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | Each adapter's declaration is part of its wiring, drives the §6 @@ -566,14 +573,14 @@ Requirements: them is how a "yes" cell hides an unusable feature. Feature eligibility requires wired, not merely available: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | - | Graph persistence (eventual OK) | KV Store: available + wired | Local store: available + wired (dev-grade) | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | - | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | - | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | - | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — client-cycle and `rewrite_legacy` unselectable until a primitive exists | Yes (in-process) | Durable Objects: possible, not wired | **No** | - | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | - | Device host evidence (JA4/H2) | Yes | No | No | No | + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | + | Graph persistence (eventual OK) | KV Store: available + wired | Local store: available + wired (dev-grade) | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | + | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — the client-cycle feature (deferred) would need it | Yes — in-process only: linearizable but **non-durable**, dev-eligibility only, not a production persistence claim | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index ccd48ac0b..4be41a00a 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -55,6 +55,7 @@ discoverable only because a deleted test had pinned the old behavior. | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | | 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | | 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | +| 13 | Cookies minted by graphless deployments have no graph row | Recognized rowless legacy cookies are adopted via a permission-gated, race-safe create-if-absent on a live request (providers spec §5); never egress before adoption; withdrawal works without adoption via the derived family ID | **Declared** — identity use of pre-existing cookies pauses until adopted | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -124,10 +125,19 @@ Requirements: Preserving unknown JSON does not chase aliases, consult family revocations, honor suppression records, or fail closed on provenance; an N+1 that merely preserved would, after rollback, - treat aliased rows as missing and revoked identities as live. - Rollback tests therefore run the alias, family-revocation, - suppression, and provenance paths **on N+1** against N+2-written - data. **Rollback is binaries-first too, in the other direction** — + treat revoked identities as live (aliases are reserved-future with + the rewrite deferral, providers spec §6.1). N+1 must also **write** + the safety-critical record kinds — family revocation and + suppression — not only read them: a withdrawal arriving on a + rolled-back N+1 fleet must still revoke; only provenance _writing_ + is deferred to N+2, which is what makes the boundary safe in both + directions. N+1 further **accepts an N+2-only provider in the + `legacy_providers` position** (parse/withdraw need no provenance + encoding) while rejecting it as active writer — otherwise the + rollback rule "retain the new provider as a legacy reader" would be + unsatisfiable on the very release it targets. Rollback tests + therefore run the family-revocation, suppression, and provenance + paths — read **and write** — on N+1 against N+2-written data. **Rollback is binaries-first too, in the other direction** — N+2 → N+1 binaries roll back keeping the new config (N+1 reads it fully; reverting config first would hand the old shape to N+2 binaries that reject it) — **with one precondition**: if an @@ -183,9 +193,12 @@ Requirements: required nor achievable through a structured serializer — and a genuinely pre-N+1 worker cannot preserve at all, which is exactly why the floor exists); after the **fleet-convergence gate**, **N+2 - activates the writer** and begins emitting the new fields. **Rollback - below N+1 is prohibited once any new-format row exists** — a pre-floor - binary would silently strip the new fields from every row it touches. + activates the writer** and begins emitting the new fields. **The rollback floor is crossed at N+2 writer activation itself** — an + observable deploy event, recorded as a durable schema-floor marker in + the config store before writes enable — not at "any new-format row + exists", which no operator can disprove. Below-floor rollback is + prohibited from that marker on; a pre-floor binary would silently + strip the new fields from every row it touches. Rows carry the existing `v` schema discriminator; backfill is lazy via live requests (the same pass that backfills legacy provenance, permission spec §7) — and, critically, **withdrawal never depends on @@ -221,10 +234,16 @@ Requirements: into the silent-stateless state. 9. Every misconfiguration in the providers spec §6 table fails at **startup**. Request-time failure for a configuration error is a defect. -10. Config-store payload validation (`ts config push`) applies the same - rules — including `[permissions]` policy validation (permission spec - §3.3) — so a bad config is rejected at push time, before any instance - restarts into it. +10. Validation is split into two named layers, because "the same + validation at push and startup" is not implementable: **structural + validation** (schema, types, `[permissions]` policy — permission + spec §3.3) runs at `ts config push` and again at startup; + **deployment validation** (adapter capabilities, store bindings, + store openability — a structurally valid selection can still be one + an adapter must reject) runs at startup, where those facts exist. + Push may additionally pre-check deployment facts when given a + **machine-readable adapter capability profile** (the providers §7 + matrix, serialized), but startup remains the authority. ## 5. Minimal-divergence migration recipe (operator-facing) @@ -373,19 +392,20 @@ made differently). **Implementation is blocked while any row is `open`**; each row needs an owner, a status, and a link to its decision record — an unratified row reverts to open, not to silently implemented. -| # | Decision | Where | Owner | Status | -| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | --------------------- | ------ | -| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | -| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | -| 3 | Sharing / targeted-advertising opt-outs remove P4 but retain the stored identity | permission §4.5 | maintainers + legal | open | -| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | -| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | -| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | -| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | -| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | -| 9 | Integration persistent cookies inside the permission model (P1-gated, declared registration) | hook §3; permission §7 | maintainers | open | -| 10 | Session cookies exempt from `store-on-device` even when they carry a stable identifier | hook §3 | maintainers + legal | open | -| 11 | A single failed destructive withdrawal may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | -| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | -| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | -| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | +| # | Decision | Where | Owner | Status | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising opt-outs remove P4 but retain the stored identity | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive withdrawal may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md new file mode 100644 index 000000000..fcfcd400f --- /dev/null +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -0,0 +1,132 @@ +# PR #986 review-finding ledger + +Disposition of every review finding against the provider/permission spec +set, by round. Statuses: **fixed** (commit noted) · **reapplied** (fix was +lost to a failed edit batch and re-landed — the round-4 script loss is +called out where it happened) · **partial → refixed** (a later round showed +the fix incomplete; both commits noted) · **superseded** (descope or a +later design change removed the surface) · **deferred** (moves with a +deferred feature; recorded as its entry bar) · **open** (sign-off table, +migration spec §8). + +Commits: R1 `a35f2ca78` · R2 `9886091e5` · R3 `5c8c2e893` · R4 `2b4d776b6` +· R5 `de70ca931` · R6 `c8b4b849e` · R7 (this commit). + +## Round 1 — adversarial self-review (22 findings) + +All 22 fixed in R1, three later shown partial and refixed: geo/device +gating circularity (refixed R3 — device half was wrong again), identity +stability vectors (refixed R3 — random suffix), §5.3 citations (fixed R1). +Policy moved YAML → TOML in R1 (maintainer decision). No open remnants. + +## Round 2 — first maintainer review (15 blocking + 1 + 4) + +| Finding | Status | +| -------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | +| B1 raw-EC egress ungated | fixed R2; egress table concrete R3; typed R5; scoped types R6 | +| B2 recipe grants everywhere | fixed R2; fixture-not-delta R4; per-adapter R5; minimal-divergence R6 | +| B3 blanket gate blocks withdrawal | fixed R2 (split gate, spy test) | +| B4 provider switch strands identities | fixed R2 (legacy readers); rewrite portion superseded R7 (descope) | +| B5 graph-key/prefix incomplete | fixed R2; literal-prefix R3; namespace R3; core-constructed R5→R6; delimiter R7 | +| B6 device gating not circular | fixed R2; reasoning corrected R3; qualifier R5 (reapplied R6 after batch loss) | +| B7 withdrawal trigger contradiction | fixed R2 (requires_signal ∨ denied) | +| B8 withdrawal storage failure | fixed R2; family record R3; idempotent families R4; unbounded residual honesty R7 (sign-off 11) | +| B9 signal normalization missing | fixed R2 (subjects); outcomes R4; preserved semantics R5; state machine R6 | +| B10 auction class inference lossy | fixed R2 (regime); dispatch matrix R4; raw arm R5; persisted-TCF arm R7 | +| B11 no-geo guard too narrow | fixed R2 (all jurisdiction consumers); cookie consumer deferred R7 | +| B12 validation incomplete | fixed R2; rules.default/dupes R3; region assigned R5 | +| B13 Sec-Fetch-Site insufficient | fixed R2 (exact Origin / CSRF); deferred with client spec R7 | +| B14 replay unmitigated | fixed R2; session binding required R4; ownerless mode removed R7 | +| B15 unbounded inputs | fixed R2; exact limits R5; media-type matching R6 — all deferred with client spec R7 | +| ❓ issue contradictions | fixed R2 (divergence tables per spec) | +| Hook &mut HeaderMap + framing headers | fixed R2 (structured ops, reserved list) | +| 4 non-blocking (geo residual, eligibility matrix, cluster capability, deterministic entropy) | all fixed R2 | + +## Round 3 — second maintainer review (15 blocking + hook + gaps) + +| Finding | Status | +| --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | +| Permission algebra can't preserve US | fixed R3 (grant class); regime-scoped R4; field-scoped R5→R6 | +| Auction dispatch undefined | fixed R3 (regime matrix + fallback regime) | +| Normalization delegated | fixed R3→R5 (in-spec outcomes, preserved semantics R6) | +| Egress inventory incomplete/wrong | fixed R3 (path table, 11a/11b split); pull/batch split R5 | +| Batch sync no authority source | fixed R3 (stored provenance); full recompute R5; live-refusal rule R7 | +| Hook ordering vs cache protection | fixed R3 (invariant pass); snapshot monotonic R5; sticky axes R6; must-understand R7 | +| Universal case equivalence | fixed R3 (provider-declared fixtures) | +| Namespacing vs HMAC stability | fixed R3 (reserved grammar); all-versions-verbatim R7 | +| Legacy-reader gaps | fixed R3→R5 (namespaces, governing permissions, provenance, provider=none); rewrite parts superseded R7 | +| Graph prerequisites/runtime failures | fixed R3 (startup requirement, runtime matrix, active-after-commit); adoption path R7 | +| Withdrawal atomicity | fixed R3 (idempotent families); family record R4; suppression R6→R7 | +| No dual-compatible config | fixed R3 (dual-read N+1); ordering corrected R5; both-direction rollback R6→R7 | +| Source-agnostic sources dropped | fixed R3 (explicit deferral, divergence row) | +| Client resolve replacement/replay | fixed R3→R5; deferred with client spec R7 | +| Device contract / region default | fixed R3 (stale wording, US/CA default) | +| Hook API/eligibility | fixed R3 (ops API, eligibility matrix) | +| Completeness gaps (EcId bounds, non-cluster dedupe, mutator limits, per-row tests, runtime behavior, metrics, fail-closed labels) | all fixed R3→R5 | + +## Round 4 — architecture review (1 P0, 12 P1 groups, 9 P2, 3 P3) + +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | +| P0 legacy family-ID protocol hole | fixed R4 (deterministic derivation) | +| US GPP/USP not permission-scoped | fixed R4 (§4.5 table); regime scope R5; global aggregation R7 | +| Malformed state machine | fixed R4; six-state machine + matrix column R6 | +| TCF conflict nondeterminism | fixed R4; **wrongly "preserved" — refixed R6** (conjunction algorithm) | +| Raw-TCF arm excludes malformed | fixed R4 (raw-presence trigger); persisted fallback R7 | +| Stale authority renewal | fixed R4 (valid_until, snapshot replace); evidence classes R6; digest scope R6 | +| Degraded mode cross-instance | fixed R4 (state machine); **local-only honesty R7** (sign-off 11) | +| Workers KV "bounded" lag | **partial R4 → refixed R6/R7** (strong primitive required; single normative home) | +| Physical keys/schemas contradictory | fixed R4→R6; alias-at-source-key + versionless keys R7 | +| HMAC version via parse | fixed R4 (**lost in failed batch, reapplied R6**; provenance-resolved) | +| AuthorizedIdentity unscoped | fixed R6 (GraphOps/PartnerEgress); request-side redaction R7 | +| Client contract missing | fixed R5 (Acquisition modes); ClientResolveContext R7; deferred R7 | +| Revocation-wins impossible | fixed R6 (family epoch CAS); cross-key atomicity recorded open, deferred R7 | +| Rewrite storage/transactionality | fixed R5→R6; **superseded R7 — rewrite_legacy cut from epic** | +| Release/rollback inconsistent | fixed R5 (dual-read); reader-first R5; preconditions R6; floor marker + N+1 duties R7 | +| Graphless deployments | fixed R6 (readiness step); adoption path R7 (matrix row 13) | +| No concrete adapter matrix | fixed R5→R6; availability-vs-wiring split + Axum honesty R7 | +| Hook cookies bypass model | fixed R5 (write gate); **shown insufficient → cookie ops deferred R7** | +| Cache lattice invalid | fixed R5; **ordered lattice wrong → sticky axes R6**; full Vary R6 | +| P2/P3 (mint ordering, health machine, client limits, fixtures graph config, thresholds, N/A, KV pipeline, transitions, protective labels, FR wording, device qualifier, illustrative label) | all fixed R4–R6 (device qualifier reapplied R6 after batch loss) | + +## Round 5 — package re-review + +All 15 P1 and 4 P2 dispositioned above where they refined earlier rows; +net-new: sections map (fixed R5; **registry-corrected R7**: +6, +24–27), +live-vs-stored refusal (fixed R7), suppression record (fixed R6; +completeness R7), fixture invalidity (stateless fixtures R7). + +## Round 6 — re-review at de70ca9 + +All 18 P1 and 4 P2 fixed in R6 except where R7 shows partials (tracked in +the R7 table below). Sign-off table with owners/status introduced R6. + +## Round 7 — current + +| Finding | Status | +| ------------------------------------------------- | ---------------------------------------------------------------------------------- | +| Client page leg pre-gate | fixed (page-leg gating; deferred with client spec) | +| Cookie read/use/withdraw unmodeled | **cookie ops deferred out of v1 hook** (entry bar recorded; sign-off 9/10 updated) | +| Ownerless mode reintroduces fixation | fixed — ownerless mode removed outright | +| Graphless cookie adoption | fixed (adoption transaction, matrix row 13) | +| Rewrite retention lineage | superseded — rewrite_legacy cut; finding recorded as entry bar | +| Unreferenced provider blocks | fixed (startup error) | +| Fastly prefix-query delimiter | fixed (backend-safe delimiter, per-adapter query validation) | +| P2 alias/tombstone cluster counting | fixed (liveness/kind filtering; aliases reserved-future) | +| P2 push-vs-deploy validation | fixed (two named layers, capability profile) | +| P2 rollback floor unobservable | fixed (floor = writer activation, durable marker) | +| P2 cookie ownership uniqueness | integration-ID uniqueness kept; cookie ownership deferred with cookie ops | +| Still-open: GPP 6/24–27 | fixed (registry-complete map) | +| Still-open: state-over-national opt-out erasure | fixed (grants-only precedence) | +| Still-open: suppression completeness | fixed (monotonic ordering, re-consent clear, write-failure semantics) | +| Still-open: family-epoch cross-key CAS | recorded as client-spec open question 0; deferred | +| Still-open: eventual rows vs alias guarantees | superseded (rewrite cut) | +| Still-open: 4-hop stranding | superseded (rewrite cut) | +| Still-open: N+1 enforce vs N+2 write boundary | fixed (N+1 writes safety-critical records) | +| Still-open: N+2-only legacy reader on N+1 | fixed (accepted in legacy position) | +| Still-open: no-geo guard cookie consumers | deferred with cookie ops (inventory row updated) | +| Still-open: persisted TCF in raw arm | fixed (TCF-sourced effective record triggers arm) | +| Still-open: request-side raw identity | fixed (identity-redacted integration request views) | +| Still-open: RequestFilterEffects.response_headers | fixed (folded into hook, done-when item) | +| Still-open: must-understand | fixed (sticky set extended) | +| Still-open: Axum persistence overstated | fixed (in-process, non-durable, dev-only cell) | From ff1e113e689ce16dae6ad77ff59e0d07db9b047d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 1 Aug 2026 12:58:27 -0700 Subject: [PATCH 10/14] Address eighth review: suppression realizability, N+1 interim semantics, and descope propagation P1 fixes: - Suppression is realizable: writes require linearizable per-key CAS with the record's own version counter ordering transitions (strong reads alone let an older clear overwrite a newer suppress); coverage is every positive-to-unset delta regardless of cause (the refusal/opt-out-only list left malformed-present and applicable absence leaking stale authority to batch sync); timestamp-less GPP/USP sources get sticky opt-out - only an authoritatively timestamped newer grant clears, since opt-out/consent/opt-out(same value) is indistinguishable from replay (sign-off 16); and a failed suppression write is an unbounded residual for a never-returning visitor, sharing sign-off 11 - not 'transient'. - N/A has one meaning everywhere: explicit Not Applicable is grant-class (preserving pinned USP tests and GPP NotApplicable handling), absent grants nothing; the P4-authorizing consequence is sign-off 17. - Adoption authenticates: ServerMint providers gain verify(id, evidence); rowless cookies failing verification are expired, not adopted (including the declared roaming false-negative); adoption gates on the provider's complete required_permissions, needs an atomic create-if-absent capability (Workers KV ineligible), distinguishes read errors from not-found, and bounds adopted-row TTL by a migration cutoff instead of granting a fresh year (sign-off 21). - N+1 has a valid write behavior: it mints v1 rows with today's semantics, and old-shape config runs the pre-epic consent gate unchanged (dual-read = dual-behavior), so neither the active-after-commit contract nor the compiled protective fallback fires mid-convergence; the new contracts activate with N+2/new-shape config (sign-off 20). - Providers ship compiled-in dormant one release before selectability - there is no dynamic provider ABI, so 'N+2-only provider readable by N+1' was impossible as written; new providers get their own reader-first rollout. - The hook's cookie deferral is contradiction-free: the operation list is headers-only, the reserved-surface and generic-op remnants are swept, and the cache test uses a core-owned queued cookie. - The RequestFilterEffects.response_headers channel is NOT folded in - that would break DataDome's challenge/deny flows (headers + cookies on 200/301/302/401/403/429, response classes the hook never runs on). It stays a distinct core-owned security channel with core-mediated security cookies, adopting the shared validation and cache-invariant layers. - Age, Date, and Expires are reserved: replacing Age:59 with Age:0 or extending Expires re-extends freshness in exactly the way the monotonic merge forbids. - Client-cycle types (Acquisition/ClientResolve/reservations) are out of the normative trait surface per the spec's own minimalism rule; the epic's only acquisition mode is server mint. - Request-side redaction is a specified boundary: typed RedactedRequestView with an enumerated strip set, same-PR migration of the raw filter/proxy inputs, and denied/withdrawn tests. P2/P3: rewrite residue swept (tests, runtime row, metrics, retirement gate; alias schema marked reserved); physical keys become one portable delimiter-free fixed-width grammar (class tag + 4-char registry provider code - Fastly rejects both / and : in prefix queries, and per-adapter delimiters would fork physical keys across adapters); the Axum matrix cell reflects UnavailableKvStore; stored cluster sizes cannot outlive their inputs; GPP applicability leftovers reconciled (MD/IN/KY/RI sentence removed, section-6 grants defined, regime-none row aligned); mixed-revision divergence explicitly accepted (sign-off 19); the schema floor lives in write-once/CAS deployment metadata that config rollback cannot erase; sign-off rows 16-21 added and rows 3/11 amended; duplicate integration IDs rejected at startup; GPP versions enumerated with unknown-version-as-malformed; custom geo region vocabularies require a canonical ISO mapping; and the review ledger's overstated R7 dispositions are corrected (suppression, delimiter, redaction, Axum, header-channel) with a full R8 section. --- ...integration-response-header-hook-design.md | 50 ++++-- .../2026-07-30-permission-model-design.md | 117 ++++++++----- .../2026-07-30-pluggable-providers-design.md | 156 +++++++++++------- ...07-30-provider-migration-rollout-design.md | 110 +++++++----- docs/superpowers/specs/pr986-review-ledger.md | 84 +++++++--- 5 files changed, 331 insertions(+), 186 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index a8a7ef7cb..b16350ac2 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -31,8 +31,9 @@ mutators to the outbound response for HTML document responses it processed. registered mutators in registration order. - **The mutator API is structured operations, not header-map access.** A mutator returns (or is handed a recorder for) typed operations — - `append(name, value)`, `replace(name, value)`, - `append_set_cookie(cookie)` — which **core validates and applies**, + `append(name, value)` and `replace(name, value)` (v1 is headers-only; + the cookie operation arrives with the deferred cookie surface, §3) — + which **core validates and applies**, attributing each to its integration id. PR #838's shape handed the integration an unrestricted `&mut HeaderMap`, which makes §3's collision policy unenforceable by construction: core cannot validate or attribute @@ -88,9 +89,14 @@ mutators to the outbound response for HTML document responses it processed. granularities because `Set-Cookie` is multi-valued: (a) reserved header _names_ — HTTP framing and hop-by-hop headers (`Content-Length`, `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, - `Keep-Alive`), **representation headers coupled to body bytes the hook - cannot see** (`Content-Encoding`, `Content-Range`, `Content-Type`, - `ETag`, `Last-Modified`, `Accept-Ranges`, and digest headers — + `Keep-Alive`), **freshness metadata** (`Age`, `Date`, `Expires` — replacing `Age: 59` + with `Age: 0` on a cached `max-age=60` response, or pushing `Expires` + into the future, extends downstream freshness in exactly the way the + monotonic merge forbids for `max-age`, so these are reserved outright + rather than merged), **representation headers coupled to body bytes + the hook cannot see** (`Content-Encoding`, `Content-Range`, + `Content-Type`, `ETag`, `Last-Modified`, `Accept-Ranges`, and digest + headers — relabeling uncompressed bytes as Brotli, or advertising a validator or digest for bytes the hook never saw, corrupts responses or poisons caches), the `x-ts-*` namespace, and the @@ -108,9 +114,14 @@ mutators to the outbound response for HTML document responses it processed. permissions, a typed authorized request-side view, stripping from unauthorized integration/proxy inputs, mandatory expiry on destructive P1 withdrawal, and startup-unique (name, domain, path) ownership. - Integration IDs are startup-unique regardless. Until then the + Integration IDs are **startup-unique, enforced**: registry + construction rejects a duplicate ID (current code silently coalesces, + which corrupts attribution and budgets), with a duplicate-ID test in + the done-when. Until then the operation set is headers-only, and `Set-Cookie` is fully reserved. - reserved cookie name. Violations are rejected at the operation layer (§2) and + reserved cookie name — in v1 that is every cookie name, since + `Set-Cookie` is fully reserved (§3 deferral). Violations are rejected + at the operation layer (§2) and logged at `warn` with the integration id. The reserved lists are single constants next to the definitions they protect, not duplicated in the hook. @@ -125,7 +136,8 @@ mutators to the outbound response for HTML document responses it processed. which is deterministic). - **Operation-layer hygiene:** generic `append`/`replace` reject the `Set-Cookie` header name outright — cookies go only through - `append_set_cookie`, so its validation cannot be bypassed by spelling + the deferred cookie builder (when it exists), so cookie validation + cannot be bypassed by spelling the header name in a generic op. Per-integration limits bound total operations (≤ 32), added headers (≤ 16), and added bytes (≤ 8 KiB), and a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, @@ -174,11 +186,20 @@ processed documents (§6). ## 4. Done-when (from #782, sharpened) 1. Trait + builder + registry application, each public item documented. -2. **The pre-existing `RequestFilterEffects.response_headers` channel is - folded into the hook in the same PR** — its outputs become hook - operations subject to the same validation, reserved surface, budgets, - and invariant pass, or the channel is removed; a second, unvalidated - header path bypassing the hook defeats every rule above. +2. **The pre-existing `RequestFilterEffects.response_headers` channel + remains a distinct, core-owned security channel — not folded in, and + not left unvalidated.** Folding it into this hook would break its one + real consumer: DataDome sets headers **and cookies** on 200, 301/302, + 401, 403, and 429 responses — challenge and deny flows on exactly the + response classes (§3a) this hook never runs on, and with cookie + emission v1 reserves. Instead, the channel keeps its own eligibility + (security-integration responses of any status), its cookies are + **core-mediated security cookies** (explicitly outside the deferred + integration-cookie surface, migrated deliberately when that surface + lands), and it adopts the **shared validation layers**: the + structured-operation checks, reserved header names, budgets, and the + final cache/privacy invariant pass. One invariant enforcer, two + eligibility domains. 3. **At least one real consumer ships in the same PR** — an existing integration registering a mutator for a real need (or, failing a real need, the feature waits; scaffolding with only self-referential tests is @@ -192,7 +213,8 @@ processed documents (§6). cache-hit, pass-through, redirect, error, and 304 each proven to run or not run the hook — not merely one positive header test per adapter. 8. Cache/privacy invariant tests, one per restriction source and shape: - cookie appended + public `Cache-Control` replacement → private/no-store, + a **core-owned** cookie already queued before the hook + an + integration's public `Cache-Control` replacement → private/no-store, surrogate stripped; **core-private cookieless** processed HTML + public replacement → restriction preserved; **origin-private cookieless** processed HTML that retained the origin's cache restrictions + public replacement → restriction diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 4a3feb629..21889eb54 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -187,7 +187,7 @@ Validation rejects: - rule keys whose country part is not in the embedded **assigned** ISO 3166-1 alpha-2 list (not merely `[A-Z]{2}` — an unassigned code is almost certainly a typo silently diverting a country to the fallback); - the region part must be an assigned ISO 3166-2 subdivision of that country (not merely a shape check — `US/ZZ` would parse but can never match a request), unless the selected geo provider declares its own region vocabulary, in which case validation uses that declaration. The `US/CA` slash form is the + the region part must be an assigned ISO 3166-2 subdivision of that country (not merely a shape check — `US/ZZ` would parse but can never match a request), unless the selected geo provider declares its own region vocabulary **together with a canonical mapping to ISO subdivisions** — §4.5 applicability and policy rule keys operate on canonical `US/CA`-form keys, so a provider emitting anything else must declare the translation, validated at startup. The `US/CA` slash form is the house rule-key format corresponding to ISO 3166-2 `US-CA`; - references to permissions outside the enforced vocabulary (§2); - references to undefined groups, and groups missing the `regime` class; @@ -273,11 +273,11 @@ blocked but an **explicit non-opt-out** value grants: permission-scoped** — grant signals are NOT interchangeable across regimes: - | Regime of the resolved rule | Evidence accepted as a grant for a `requires_signal` permission | - | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | - | `gdpr` | **Only** a TCF record consenting to that specific purpose | - | `us-privacy` | TCF consent for the purpose, or GPP/USP evidence **per the §4.5 field mapping** — a field grants only the permissions it maps to | - | `none` | Any grant-class signal | + | Regime of the resolved rule | Evidence accepted as a grant for a `requires_signal` permission | + | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | `gdpr` | **Only** a TCF record consenting to that specific purpose | + | `us-privacy` | TCF consent for the purpose, or GPP/USP evidence **per the §4.5 field mapping** — a field grants only the permissions it maps to | + | `none` | TCF consent; GPP/USP grants only where §4.5 applicability yields one (the national section applies under `us-privacy` regimes, so in practice `none` grants via TCF — moot in the shipped policy, whose `none` baseline is `granted`) | Without this scoping, a US-style `sale_opt_out = false` would satisfy a French `requires_signal` rule — no TCF, both purposes granted, EC minted, @@ -423,23 +423,48 @@ and the fail-closed marker: which the refusal just unset, and identity rows may be eventually consistent, so a stale replica could resurrect a P4 grant after a targeted-advertising opt-out. The fix is a **suppression record** in - the strongly consistent class (providers spec §6.3: `sup/`). - Its semantics are complete, not sketched: entries are **per permission** - with the triggering state (refusal or non-destructive opt-out — the - full negative-state coverage; absence writes nothing) and an - authoritative timestamp; ordering is **monotonic per permission** — - a write with an older timestamp than the stored entry is a no-op, so - replays cannot regress the state; **re-consent clears**: a live - resolution carrying an accepted grant with a newer authoritative - timestamp than the suppression entry supersedes it (recorded as a - clear entry in the same record — still an exempt write, since it only - ever reflects the live resolution); and **write failure fails closed - for the live request** (the refusal's effect stands for this response) - while S2S may transiently honor prior authority until the retry lands — - logged, metered, and covered by the degraded-mode rule. Every S2S - recompute and partner-egress check consults the record: a suppressed - permission is unset whatever the row's provenance says, so no - eventual-consistency edge can restore it. + the strongly consistent class, and — because monotonicity is a + read-modify-write property, not a read property — suppression writes + require **linearizable per-key CAS** (providers spec §6.3 + `sup/`, §7 matrix): with plain read-after-write, two + writers can both read the record and an older clear can overwrite a + newer suppress. The record carries its own **version counter, + incremented through the CAS**, so transitions are ordered by + serialization, not by comparing wall-clock timestamps. + + Coverage is **every authority-clearing delta, not an enumerated cause + list**: after each live resolution, any permission whose new state is + unset while its stored provenance is positive gets a suppression + entry — refusal, non-destructive opt-out, malformed-present, and + applicable absence alike. (The earlier refusal/opt-out-only list left + a hole: a malformed record unsets P1, the P1-gated row update is + thereby forbidden, no suppression is written, and batch sync later + honors the stale grant.) + + **Timestamp-less sources get sticky opt-out.** GPP/USP values carry no + intrinsic timestamp, and opt-out → consent → opt-out(same value) is + information-theoretically indistinguishable from a replay of the first + opt-out. Latest-observation semantics would let a replayed old consent + string clear a newer opt-out, so: a suppression from a timestamp-less + source is cleared **only** by a grant carrying an authoritative + timestamp newer than the suppression's observation (TCF + `LastUpdated`) — a timestamp-less re-consent alone does not clear it. + The consequence (a genuine GPP-only re-consent does not restore + authority until a timestamped source or policy provides it) is + declared and is sign-off item 16. Fixtures: suppress-vs-clear race + under concurrent writers; repeated-value opt-out/consent/opt-out. + + **Write failure fails closed for the live request** (the refusal's + effect stands for this response), but the S2S residual is **unbounded + for a never-returning visitor** — exactly like a failed destructive + revocation, not "transient": other instances continue honoring old + provenance, the breaker is per-instance, and no durable retry exists. + This shares sign-off item 11 (extended to cover suppression) and gets + its own fault test. Every S2S recompute and partner-egress check + consults the record: a suppressed permission is unset whatever the + row's provenance says, so no eventual-consistency edge can restore + it. + - **The cookie expires only after the family record commits.** - **If the family-record write itself fails, nothing durable exists** — the cookie stays and the durable client-side signal (GPC, CMP-stored @@ -498,14 +523,19 @@ an expired record before clearing both sources; expiry-first is a | Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | | Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**, then flows through the full normalization pipeline (syntax, expiry, conflict) like any live record; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | **Changed (declared)**: current code returns immediately after the KV load, bypassing expiry and conflict normalization | | Proxy/mirror mode | **Minimal opt-out extraction still runs; full semantic decoding is skipped.** Because opt-outs are globally authoritative (§4), proxy mode must not suppress them: the §4.5-mapped opt-out fields (GPP US sections) and the US Privacy string are decoded — nothing else — alongside syntax validation, so a valid SaleOptOut or USP opt-out revokes and withdraws exactly as outside proxy mode. No grants are ever derived from records in proxy mode; a present record otherwise blocks grants (fail-closed); absent → baseline. GPC needs no decoding | **Changed (declared)**: today proxy mode skips decoding entirely — fail-open under permissive baselines and, worse, opt-out-blind | -| GPP / US Privacy fields | Per the normative field mapping of §4.5 — fields are not interchangeable signals, and absent/N-A fields grant nothing | Decided here (§4.5) | +| GPP / US Privacy fields | Per the normative field mapping of §4.5 — fields are not interchangeable signals; **explicit N/A is grant-class (not-opted-out), absent grants nothing** — one meaning, everywhere | Decided here (§4.5) | | Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | ### 4.5 US signal field mapping — normative GPP and US Privacy fields map to specific permissions with specific effects; they are never interchangeable, a field's absence or N/A value -contributes nothing, and only the fields marked destructive trigger +behaves per its table row — **explicit _Not Applicable_ is grant-class +(not-opted-out), preserving current USP tests and GPP `NotApplicable` +handling; only a genuinely absent field contributes nothing** (this is +the single normative statement; an earlier "N/A contributes nothing" +rule is dead, and the P4-authorizing consequence is sign-off item 17) — +and only the fields marked destructive trigger withdrawal. Section IDs and versions are those of the IAB GPP specification current at implementation time; adding a section or field is a change to this table. @@ -523,19 +553,17 @@ a change to this table. | Any field | explicitly _Not Applicable_ | as the field's not-opted-out row above | as the field's not-opted-out row above | — | | Any field | absent | — | — | — | -**N/A vs absent:** a field explicitly set to _Not Applicable_ is treated -as not-opted-out (grant-class) — pinned by today's USP tests and matching -current GPP `NotApplicable` handling, and declared as such since an -earlier draft said N/A contributes nothing. A field **absent** from an -applicable section, or any field of a non-applicable section, contributes -nothing. +**N/A vs absent (restating the single rule):** explicit _Not +Applicable_ = grant-class; absent = nothing; a non-applicable section's +fields grant nothing (their opt-outs still count, per step 2). **Applicability and aggregation — ordered algorithm:** 1. **Section map (normative, pinned here — not "whatever GPP is current"), matching the official IAB registry in full:** section 6 ↔ - the **US Privacy string carried as a GPP section** (it maps to the USP - rows of the field table, not to nothing); `US` national ↔ 7 (usnat); + the **US Privacy string carried as a GPP section** — it maps to the USP + rows of the field table in full, opt-outs _and_ grant-class values, + under the same applicability rules as the national section; `US` national ↔ 7 (usnat); the state sections — `US/CA` ↔ 8, `US/VA` ↔ 9, `US/CO` ↔ 10, `US/UT` ↔ 11, `US/CT` ↔ 12, `US/FL` ↔ 13, `US/MT` ↔ 14, `US/OR` ↔ 15, `US/TX` ↔ 16, `US/DE` ↔ 17, `US/IA` ↔ 18, `US/NE` ↔ 19, `US/NH` ↔ 20, @@ -544,8 +572,11 @@ nothing. claimed MD/IN/KY/RI had no section). A truncated map silently loses opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. The implementation PR cross-checks this list against both the current - decoder's section set and the official registry; adding a section or - version is a change to this map. + decoder's section set and the official registry, and **enumerates the + accepted version per section**; a mapped section carrying an unknown + version is treated as malformed-present (blocks grants, never + withdraws — §4.4), not as absent. Adding a section or version is a + change to this map. 2. **Applicability gates grants only — never opt-outs.** A mapped **opt-out** field (either subclass) is honored from **any** section on **any** request, whatever the regime — this is §4's global-opt-out @@ -556,8 +587,7 @@ nothing. to the resolved `US/`; foreign-state sections and all sections on non-`us-privacy` requests grant nothing. Regionless US traffic: national section only. A configured privacy state with no - state-specific section (e.g. MD, IN, KY, RI today) uses the national - section alone. + state-specific section uses the national section alone. 3. **State-over-national, per field — for grants only:** where an applicable state section carries a field, its value governs that field's **grant** derivation; the national section fills only fields @@ -647,9 +677,16 @@ policy** (§4.2 trigger 3) — the one revision-sensitive destructive case (trigger 2 under a now-`denied` baseline) requires an affirmative user refusal at the evaluating instance, which is safe under either revision. S2S recomputation always evaluates against the instance's current -revision and records it. Rolling a policy revision back restores -acquisition rules but **cannot resurrect tombstoned identities**; the -migration guide says so where operators will read it. +revision and records it. One divergence is explicitly accepted rather +than fenced: during convergence, a live refusal under a +`granted`-revision instance suppresses while the same refusal under a +tightened-revision instance destroys (trigger 2) — the destructive +outcome is the target revision's intended behavior arriving early on +part of the fleet, coordinated activation fencing is not worth its +machinery, and the acceptance is sign-off item 19. Rolling a policy +revision back restores acquisition rules but **cannot resurrect +tombstoned identities**; the migration guide says so where operators +will read it. ## 6. Failure-mode matrix — normative diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index c92253a3d..f231601e9 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -150,7 +150,12 @@ Three global rules sit above every provider: trips the trust threshold toward denial, never toward extra writes); the listing filters by the value's `kind`/liveness within the existing list limit where the backend returns values, and the residual - over-count where it cannot is declared. Aliases are reserved-future + over-count where it cannot is declared. A computed cluster size is + **not persisted beyond its inputs' lifetime**: today's code stores the + calculated `cluster_size` in the row and reuses it for the row's full + TTL, which would freeze a tombstone-inflated count for up to a year — + stored values carry a short validity (within the tombstone-TTL + horizon) or are recomputed on use. Aliases are reserved-future (§6.1) and excluded by `kind` when they exist. - **No-cluster behavior is still defined.** A provider without cluster support deduplicates pull-sync by canonical graph key and redacts logs @@ -201,26 +206,20 @@ pub trait EdgeCookieProvider { /// key, shared across identifiers minted from the same client /// evidence. None when the provider lacks IP-cluster semantics (§3). fn cluster_prefix(&self, id: &EcId) -> Option; - /// Acquisition mode — exactly one: - fn acquisition(&self) -> Acquisition<'_>; + /// Cryptographic verification of a parsed identifier against request + /// evidence — recognition (`parse`) is not authentication; adoption + /// (§5) and any rowless acceptance require this. + fn verify(&self, id: &EcId, input: &IdentityInput<'_>) -> bool; } - -/// How a provider's identifiers come into being. Server-mint providers -/// generate from request evidence; client-resolve providers verify a -/// browser-posted payload (client-cycle spec) and declare the JS module -/// their page leg needs. One provider implements exactly one mode. -pub enum Acquisition<'a> { - ServerMint(&'a dyn ServerMint), // fn generate(&IdentityInput) -> EcId - ClientResolve(&'a dyn ClientResolve), -} -// ClientResolve::resolve_from_client(&ClientResolveContext) -> Result -// ctx is core-built: canonical publisher audience, verified session -// owner hash, clock, and the bounded payload — a bare payload could -// not verify audience binding, session binding, or expiry. -// VerifiedIdentity carries the identifier, reservation id, and expiry. -// ClientResolve::js_module_id() -> &str ``` +The acquisition-mode enum (`ServerMint` / `ClientResolve`), the +`ClientResolveContext` contract, replay-reservation schemas, and the +reservation capability rows are **not part of this normative surface** — +they live in the deferred client-cycle document and return with that +feature, per this spec's own minimalism rule (§4): the epic's only +acquisition mode is server mint, expressed directly as `generate`. + (Names indicative; the shape is normative. `required_permissions` joins the trait at step 5 of §11, together with its enforcement point.) @@ -251,19 +250,40 @@ header emission; the identity exists durably from that moment. A graph-commit failure means the mint never happened: no cookie, no egress, error logged, the next request retries. -**Pre-existing cookies without rows are adopted, not orphaned.** Current -graphless deployments have minted cookies with no row; under -no-active-until-commit those identities could never be used again and — -without care — never withdrawn. The contract: a recognized legacy cookie -with no reachable row triggers a **permission-gated, race-safe adoption** -on a live request — gated exactly like minting (`store-on-device`), -implemented as create-if-absent on the verbatim key (concurrent adopters -converge: same key, same deterministic family ID), provenance from the -live resolution. Until adoption succeeds the cookie **never egresses**; -**withdrawal works without adoption** — the deterministic family ID -(permission model spec §4.3) needs no row, so a first post-upgrade -request that is an opt-out revokes and expires the cookie with zero -migrated state. Migration matrix row 13 declares this path. +**Pre-existing cookies without rows are verified, then adopted — parse +is recognition, not authentication.** A syntactically valid +`{64hex}.{6alnum}` string is constructible by anyone; adopting it on +shape alone would let an attacker mint durable rows. The contract: + +- ServerMint providers implement `verify(id, &IdentityInput) -> bool` — + cryptographic verification against **request evidence** (for `hmac`: + recompute over the request's evidence with each configured version's + passphrase and compare to the 64-hex prefix). A recognized rowless + cookie that fails verification is **expired, not adopted** — including + the honest false-negative: a legitimate cookie presented from a + changed network no longer verifies and is expired; the affected + population is graphless deployments only, declared in migration row 13. +- Adoption is gated on the provider's **complete + `required_permissions()`** — exactly like minting, not a hard-coded + `store-on-device`. +- The row write is **atomic create-if-absent** — a distinct capability + row in the §7 matrix (strong class; Workers KV's concurrent same-key + writes can overwrite each other, so it is ineligible, consistent with + its revocation ineligibility). +- **Read errors are not "not found"**: adoption proceeds only on an + authoritative not-found; a failed graph read means no adoption this + request, fail closed. +- Adopted rows do **not** get a fresh full TTL — a nearly expired legacy + identity must not gain a year (the exact rejuvenation problem that + deferred rewrite). Expiry is `min(adopted_at + standard TTL, +migration_cutoff + grace)` with the cutoff configured; sign-off + item 21. + +Until adoption succeeds the cookie **never egresses**; **withdrawal +works without adoption** — the deterministic family ID (permission model +spec §4.3) needs no row, so a first post-upgrade opt-out revokes and +expires the cookie with zero migrated state. Migration matrix row 13 +declares this path. **Egress is typed, not policed.** The inventory-and-denylist test (permission model spec §7) is a backstop, but conventions do not survive @@ -279,12 +299,20 @@ and nothing weaker — an unparameterized wrapper would let a P1-only identity flow into an ORTB request. A future bypass then requires deliberately reconstructing the raw string — visible in review — rather than passing along what was already in hand. The same boundary -applies **request-side**: integration-facing request views (proxy -interfaces, filter inputs, forwarded header/cookie maps) receive -**identity-redacted** views — the EC cookie and identity headers are -stripped unless the path holds `AuthorizedIdentity` — -because the ungated forwarding paths of PR #838 were exactly integrations -reading the raw request. +applies **request-side, as a concrete API transition, not an +assertion**: the current filter/proxy inputs expose the raw request +(cookies included), so a filter can read `ts-ec`, copy it into +`X-Vendor-Identity`, and return it through response effects — response +snapshot redaction cannot undo that. The contract: integration-facing +request access moves to a typed **`RedactedRequestView`** whose stripped +set is enumerated — the `ts-ec` cookie and every `ts-*` cookie, `x-ts-*` +identity/consent headers, and the EIDs header — with identity reachable +only through a scoped `AuthorizedIdentity` parameter; the raw-request +filter/proxy interfaces are migrated in the **same PR** as the typed +egress boundary (they are the same boundary), and the tests are +enumerated: a denied/withdrawn request through a filter, a proxy, and a +forwarding path, each asserting no identity value is readable or +emittable. The gate applies to EC providers **only**. Geo and device are ungated for two _different_ reasons, stated separately because only one of them is @@ -406,12 +434,11 @@ The contract: reserved-for-future — nothing in the epic writes one. - Retiring a legacy reader is the explicit end of those identities: the migration guide documents the cleanup procedure (migration spec §6). -- Tests: switch active provider → request with old cookie → identity still - resolves and a withdrawal tombstones it (both linked rows when - rewritten); old cookie with no matching legacy reader → treated as - absent and **never egresses**; interrupted rewrite → old cookie still - live, retry completes; `provider = "none"` + legacy reader → no mints, - withdrawal still works. +- Tests: switch active provider → request with old cookie → identity + still resolves and a withdrawal tombstones it; old cookie with no + matching legacy reader → treated as absent and **never egresses**; + `provider = "none"` + legacy reader → no mints, withdrawal still + works. (Rewrite-specific tests left with the rewrite deferral.) Cluster degradation config (referenced from §3): when the active writer lacks the cluster capability, `[ec] cluster_fallback = "allow" | "deny"` @@ -431,7 +458,6 @@ when a healthy configuration meets an unhealthy runtime. Every row logs at | Graph read fails on an existing identity | Identity unusable this request (fail closed for egress); cookie untouched | | Cluster prefix listing fails | Treated as cluster-size-unknown → `cluster_fallback` policy applies | | Tombstone write fails | Permission model spec §4.3: family retries, readers fail closed on partial families | -| Legacy rewrite fails mid-flight | Old cookie remains live; rewrite retries (§6.1) | | Geo provider returns invalid output (unparseable country) at runtime | Treated as lookup failure → `default_country` (permission model spec §5.2), counted in the lookup-failure metric | | Device provider signals unavailable at runtime (e.g. no JA4 on a request) | Classification degrades per the provider's declared fallback, never silently upgrades `looks_like_browser` | @@ -476,19 +502,23 @@ the reserved hmac grammar), and every record value carries a `kind` discriminator alongside its schema version — so a reader always knows what it fetched, including where two classes deliberately share an address (row vs. alias). The `/` shown in key sketches is **notation, -not the wire byte**: the physical segment delimiter is a -**backend-safe character validated per adapter** — Fastly permits `/` in -keys but not in prefix _queries_, so a slash-delimited `id//…` -key could never be cluster-listed there despite the matrix marking -prefix listing supported. The reference delimiter is `:`; each adapter's -capability declaration includes which delimiter its prefix queries -accept, and cluster-capability eligibility for a provider requires its -physical prefix to be queryable on that backend — checked at startup, -not discovered at the first cluster count. (hmac verbatim keys contain -no delimiter before the 64-hex prefix and are unaffected.) +not the wire byte** — and the wire form is **one portable grammar, not +per-adapter delimiters** (per-adapter delimiters would give the same +logical identity different physical keys on different adapters, breaking +migration, shared storage, and parity; and Fastly's prefix queries +reject both `/` and `:`, so no delimiter character is safely portable). +Physical keys are **delimiter-free with fixed-width segments**: a +1-character class tag (`i` row, `f` family, `s` suppression, `x` +transaction), a **4-character registry-assigned provider code** +(zero-padded, `[a-z0-9]`), then the suffix — segment boundaries are +positional, so no segment can contain or escape a delimiter, prefix +queries are plain string prefixes on every backend, and cluster +eligibility needs no per-adapter delimiter negotiation. (hmac verbatim +keys remain the reserved exception, with the 64-hex cluster prefix at +position zero.) **Wire schemas** (JSON, like identity rows; every class carries a schema -version): the **alias record** holds target key, created-at, retirement +version): the **alias record** (reserved-future, with rewrite) holds target key, created-at, retirement deadline, and fencing epoch; the **family revocation record** holds the family ID, revoked-at, triggering signal class (§4.5 destructive column), and a **family epoch** bumped on every revocation-state change (the @@ -573,14 +603,14 @@ Requirements: them is how a "yes" cell hides an unusable feature. Feature eligibility requires wired, not merely available: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | ----------------------------------------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | - | Graph persistence (eventual OK) | KV Store: available + wired | Local store: available + wired (dev-grade) | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | - | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | - | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | - | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — the client-cycle feature (deferred) would need it | Yes — in-process only: linearizable but **non-durable**, dev-eligibility only, not a production persistence claim | Durable Objects: possible, not wired | **No** | - | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | - | Device host evidence (JA4/H2) | Yes | No | No | No | + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | + | Graph persistence (eventual OK) | KV Store: available + wired | **Not wired** — the current adapter installs `UnavailableKvStore`; an in-process store is dev-feasible but does not exist yet | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | + | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — the client-cycle feature (deferred) would need it | Yes — in-process only: linearizable but **non-durable**, dev-eligibility only, not a production persistence claim | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 4be41a00a..6299d5448 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -129,34 +129,52 @@ Requirements: the rewrite deferral, providers spec §6.1). N+1 must also **write** the safety-critical record kinds — family revocation and suppression — not only read them: a withdrawal arriving on a - rolled-back N+1 fleet must still revoke; only provenance _writing_ - is deferred to N+2, which is what makes the boundary safe in both - directions. N+1 further **accepts an N+2-only provider in the - `legacy_providers` position** (parse/withdraw need no provenance - encoding) while rejecting it as active writer — otherwise the - rollback rule "retain the new provider as a legacy reader" would be - unsatisfiable on the very release it targets. Rollback tests - therefore run the family-revocation, suppression, and provenance - paths — read **and write** — on N+1 against N+2-written data. **Rollback is binaries-first too, in the other direction** — + rolled-back N+1 fleet must still revoke. + + **N+1's identity-write behavior is v1, explicitly** — this resolves + what was an impossible trilemma (write rows without provenance, + violating active-after-commit; write provenance, violating the + N+2-only writer boundary; or stop minting, an undeclared outage): + N+1 **keeps minting v1 rows with today's semantics**, and the new + active-after-commit/provenance contract activates **with the N+2 + writer**, not before. Likewise the permission model itself: + **old-shape config on N+1 runs the pre-epic consent gate + unchanged** — dual-read means dual-behavior — so the compiled + protective fallback cannot flip behavior mid-convergence before the + operator pushes the new-shape policy; the new model engages only + with new-shape config. The interim (N+1 minting v1 rows, S2S + running today's checks) is declared as sign-off item 20, not + discovered. + + Rollback tests therefore run the family-revocation and suppression + paths — read **and write** — plus v1-minting behavior, on N+1 + against N+2-written data. **Rollback is binaries-first too, in the other direction** — N+2 → N+1 binaries roll back keeping the new config (N+1 reads it fully; reverting config first would hand the old shape to N+2 - binaries that reject it) — **with one precondition**: if an - N+2-only provider or version has been adopted, the fleet must first - converge on an N+1-compatible new-shape config (deselecting what - N+1 rejects, retaining the new provider's secrets as a **legacy - reader** so its minted identities keep resolving and stay - withdrawable until they expire — never "revert to the previous - config", which would strand them); only then do binaries roll back. - N+1 additionally **rejects provider or version selections whose - provenance it cannot yet encode** — new-provider adoption waits for - N+2, so no row is minted that N+2 would misclassify. Every new + binaries that reject it) — **with one structural rule that makes it possible at all**: + providers are compiled into the composition root — there is no + dynamic provider ABI — so an N+1 binary can only read what it + shipped with. Therefore **every provider selectable in release R + must ship compiled-in (dormant: registered, parseable, + configurable, not selectable as writer) in R−1**; adopting a + genuinely new provider gets its own reader-first rollout, exactly + like the epic itself. With that rule, rolling N+2 → N+1 keeps the + new provider's identities resolvable and withdrawable through the + dormant registration ("retain as legacy reader" is now satisfiable + because N+1 physically contains the code); the fleet still first + converges on a config selecting only what N+1 accepts as writer. + N+1 additionally **rejects writer selections whose provenance it + cannot yet encode** — new-writer adoption waits for N+2, so no row + is minted that N+2 would misclassify. Every new config section introduced by the epic follows this same compatibility rule, not only `[ec]`. + - **Release N+2:** rejects `[ec] passphrase` at startup with a message naming the new location — not a generic unknown-field error (implementation note: producing the actionable message means keeping a deprecated `passphrase` field whose presence triggers the custom error). + 2. **Revocation-eligible storage is a per-adapter gate, and ungated adapters migrate stateless.** Identity features require the adapter's strong-consistency rows in the capability matrix (providers spec §7) @@ -194,8 +212,11 @@ Requirements: genuinely pre-N+1 worker cannot preserve at all, which is exactly why the floor exists); after the **fleet-convergence gate**, **N+2 activates the writer** and begins emitting the new fields. **The rollback floor is crossed at N+2 writer activation itself** — an - observable deploy event, recorded as a durable schema-floor marker in - the config store before writes enable — not at "any new-format row + observable deploy event, recorded as a durable schema-floor marker + **in write-once/CAS deployment metadata that ordinary config rollback + cannot touch** — floor-in-rollbackable-config would let "restore the + previous config version" erase the floor after new-format rows exist, + which is exactly the state it guards — not at "any new-format row exists", which no operator can disprove. Below-floor rollback is prohibited from that marker on; a pre-floor binary would silently strip the new fields from every row it touches. @@ -323,14 +344,13 @@ global honoring of opt-out signals is unconditional. silent grant to everyone), not an error rate. The full metric set, each with a stated healthy range: geo lookup-failure/fallback rate (permission spec §5.2), raw-egress denials by path, tombstone family retries, - legacy-reader hit rate, rewrite failures, and cluster-fallback - engagements. Two of these carry thresholds, not just ranges: + legacy-reader hit rate, and cluster-fallback engagements. Two of these carry thresholds, not just ranges: legacy-reader hits at zero for a **quiet period no shorter than the maximum cookie/row lifetime plus rollout skew** — or provable rewrite/backfill completion — is the **retirement-readiness** bar for a legacy provider ("trending to ~zero" is not evidence; a yearly visitor - is not churn), and a nonzero rewrite-failure rate blocks retirement - outright. The telemetry set also includes: graph read/commit failures, + is not churn), (rewrite-based backfill and its metrics left with the rewrite + deferral). The telemetry set also includes: graph read/commit failures, stored-provenance denials, schema-migration failures, and replay-reservation recoveries. **Each rollout-gate metric ships with a threshold, an evaluation window, and a named action** (pause rollout / @@ -392,20 +412,26 @@ made differently). **Implementation is blocked while any row is `open`**; each row needs an owner, a status, and a link to its decision record — an unratified row reverts to open, not to silently implemented. -| # | Decision | Where | Owner | Status | -| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | -| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | -| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | -| 3 | Sharing / targeted-advertising opt-outs remove P4 but retain the stored identity | permission §4.5 | maintainers + legal | open | -| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | -| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | -| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | -| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | -| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | -| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | -| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | -| 11 | A single failed destructive withdrawal may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | -| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | -| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | -| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | -| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | +| # | Decision | Where | Owner | Status | +| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | +| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | +| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | +| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | +| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | +| 21 | Adopted legacy rows are bounded by a migration cutoff, not a fresh full TTL | providers §5 | maintainers + product | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md index fcfcd400f..fa5a68756 100644 --- a/docs/superpowers/specs/pr986-review-ledger.md +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -103,30 +103,60 @@ the R7 table below). Sign-off table with owners/status introduced R6. ## Round 7 — current -| Finding | Status | -| ------------------------------------------------- | ---------------------------------------------------------------------------------- | -| Client page leg pre-gate | fixed (page-leg gating; deferred with client spec) | -| Cookie read/use/withdraw unmodeled | **cookie ops deferred out of v1 hook** (entry bar recorded; sign-off 9/10 updated) | -| Ownerless mode reintroduces fixation | fixed — ownerless mode removed outright | -| Graphless cookie adoption | fixed (adoption transaction, matrix row 13) | -| Rewrite retention lineage | superseded — rewrite_legacy cut; finding recorded as entry bar | -| Unreferenced provider blocks | fixed (startup error) | -| Fastly prefix-query delimiter | fixed (backend-safe delimiter, per-adapter query validation) | -| P2 alias/tombstone cluster counting | fixed (liveness/kind filtering; aliases reserved-future) | -| P2 push-vs-deploy validation | fixed (two named layers, capability profile) | -| P2 rollback floor unobservable | fixed (floor = writer activation, durable marker) | -| P2 cookie ownership uniqueness | integration-ID uniqueness kept; cookie ownership deferred with cookie ops | -| Still-open: GPP 6/24–27 | fixed (registry-complete map) | -| Still-open: state-over-national opt-out erasure | fixed (grants-only precedence) | -| Still-open: suppression completeness | fixed (monotonic ordering, re-consent clear, write-failure semantics) | -| Still-open: family-epoch cross-key CAS | recorded as client-spec open question 0; deferred | -| Still-open: eventual rows vs alias guarantees | superseded (rewrite cut) | -| Still-open: 4-hop stranding | superseded (rewrite cut) | -| Still-open: N+1 enforce vs N+2 write boundary | fixed (N+1 writes safety-critical records) | -| Still-open: N+2-only legacy reader on N+1 | fixed (accepted in legacy position) | -| Still-open: no-geo guard cookie consumers | deferred with cookie ops (inventory row updated) | -| Still-open: persisted TCF in raw arm | fixed (TCF-sourced effective record triggers arm) | -| Still-open: request-side raw identity | fixed (identity-redacted integration request views) | -| Still-open: RequestFilterEffects.response_headers | fixed (folded into hook, done-when item) | -| Still-open: must-understand | fixed (sticky set extended) | -| Still-open: Axum persistence overstated | fixed (in-process, non-durable, dev-only cell) | +| Finding | Status | +| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Client page leg pre-gate | fixed (page-leg gating; deferred with client spec) | +| Cookie read/use/withdraw unmodeled | **cookie ops deferred out of v1 hook** (entry bar recorded; sign-off 9/10 updated) | +| Ownerless mode reintroduces fixation | fixed — ownerless mode removed outright | +| Graphless cookie adoption | fixed (adoption transaction, matrix row 13) | +| Rewrite retention lineage | superseded — rewrite_legacy cut; finding recorded as entry bar | +| Unreferenced provider blocks | fixed (startup error) | +| Fastly prefix-query delimiter | R7's per-adapter `:` delimiter was itself Fastly-rejected and non-portable → **refixed R8**: delimiter-free fixed-width grammar (class tag + registry provider code) | +| P2 alias/tombstone cluster counting | fixed (liveness/kind filtering; aliases reserved-future) | +| P2 push-vs-deploy validation | fixed (two named layers, capability profile) | +| P2 rollback floor unobservable | fixed (floor = writer activation, durable marker) | +| P2 cookie ownership uniqueness | integration-ID uniqueness kept; cookie ownership deferred with cookie ops | +| Still-open: GPP 6/24–27 | fixed (registry-complete map) | +| Still-open: state-over-national opt-out erasure | fixed (grants-only precedence) | +| Still-open: suppression completeness | partial R7 (ordering claimed without CAS; cause-list coverage; timestamp-less unrealizable) → **refixed R8** (CAS + version counter, delta coverage, sticky opt-out) | +| Still-open: family-epoch cross-key CAS | recorded as client-spec open question 0; deferred | +| Still-open: eventual rows vs alias guarantees | superseded (rewrite cut) | +| Still-open: 4-hop stranding | superseded (rewrite cut) | +| Still-open: N+1 enforce vs N+2 write boundary | fixed (N+1 writes safety-critical records) | +| Still-open: N+2-only legacy reader on N+1 | fixed (accepted in legacy position) | +| Still-open: no-geo guard cookie consumers | deferred with cookie ops (inventory row updated) | +| Still-open: persisted TCF in raw arm | fixed (TCF-sourced effective record triggers arm) | +| Still-open: request-side raw identity | asserted R7 without API/tests → **specified R8** (`RedactedRequestView`, enumerated strip set, same-PR migration, denied/withdrawn tests) | +| Still-open: RequestFilterEffects.response_headers | R7 fold-in would have **broken DataDome** (302/401/403/429 + cookies) → **refixed R8**: distinct core-owned security channel sharing validation + invariant layers | +| Still-open: must-understand | fixed (sticky set extended) | +| Still-open: Axum persistence overstated | partial R7 (still marked wired; head installs `UnavailableKvStore`) → **refixed R8** (not wired) | + +## Round 8 — re-review at 09e54e96 + +| Finding | Status | +| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| P1.1 suppression monotonicity unprovidable | fixed (linearizable CAS + record version counter; sticky opt-out for timestamp-less sources — sign-off 16; race fixtures) | +| P1.2 malformed/absent leave stale authority | fixed (suppression on every positive→unset delta, cause-agnostic) | +| P1.3 suppression-write failure "transient" | fixed (unbounded residual, shares sign-off 11, fault test) | +| P1.4 N/A double meaning | fixed (single rule: explicit N/A = grant-class, absent = nothing; sign-off 17) | +| P1.5 adoption = syntax-as-authentication | fixed (`verify` against request evidence; expire on failure; full required_permissions; atomic create-if-absent capability; read-error ≠ not-found) | +| P1.6 N+1 impossible write behavior | fixed (N+1 mints v1 with today's semantics; old-shape config runs pre-epic gate; new contracts activate at N+2/new-shape — sign-off 20) | +| P1.7 N+2-only provider readable by N+1 | fixed (providers ship compiled-in dormant one release early; reader-first per provider) | +| P1.8 cookie deferral contradictions | fixed (ops list, reserved remnant, generic-op wording, core-owned-cookie test) | +| P1.9 DataDome fold-in breakage | fixed (distinct security channel, shared validation/invariant layers, core-mediated security cookies) | +| P1.10 freshness metadata weakening | fixed (`Age`/`Date`/`Expires` reserved, rationale in-spec) | +| P1.11 client-cycle in normative core | fixed (Acquisition enum/ClientResolve/reservations removed from trait surface; `verify` added; deferred doc holds the rest) | +| P1.12 redaction unspecified | fixed (see corrected R7 row above) | +| P2.1 rewrite residue | fixed (tests, runtime row, metrics/retirement swept; alias schema marked reserved) | +| P2.2 delimiter portability | fixed (see corrected R7 row above) | +| P2.3 Axum matrix | fixed (see corrected R7 row above) | +| P2.4 adoption rejuvenation | fixed (migration-cutoff-bounded TTL; sign-off 21) | +| P2.5 stored cluster overcount | fixed (no persistence beyond inputs' lifetime) | +| P2.6 GPP applicability leftovers | fixed (MD/IN/KY/RI sentence removed; section-6 grants defined; regime-`none` row reconciled with applicability) | +| P2.7 mixed-revision divergence | accepted explicitly (sign-off 19) | +| P2.8 floor marker rollbackable | fixed (write-once/CAS deployment metadata) | +| P2.9 sign-off gaps | fixed (rows 16–21 added; rows 3 and 11 amended) | +| P2.10 duplicate integration IDs | fixed (startup rejection + test; current silent coalescing named) | +| P3 GPP version pinning | fixed (accepted versions enumerated; unknown version = malformed-present) | +| P3 geo region vocabulary | fixed (ISO output or declared canonical mapping) | +| P3 stale fragments + ledger overstatement | fixed (this ledger corrected; hook remnants swept) | From 43422b5fc5a3f2c09edddc8a31c1c74caea6ea7a Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 1 Aug 2026 14:22:24 -0700 Subject: [PATCH 11/14] Address ninth and tenth reviews: authentication-sound rowless handling, complete suppression contract, closed security channel Both review passes against ff1e113e, dispositioned together. Trait and rowless identity: - generate is restored to the normative trait (lost in an R8 editing accident, making the required mint sequence unimplementable); verify returns VerifiedIdentity carrying the matched configuration version. - Rowless legacy cookies are expired and re-minted, never adopted: prefix-only HMAC verification cannot authenticate the random suffix, so adoption would let H.aaaaab, H.aaaaac, ... each mint a durable row/family. The rowless family ID derives from the authenticated 64-hex prefix only, collapsing all suffix variants into one withdrawable family; matrix row 13 and sign-off 21 updated; the migration cutoff disappears with adoption. Suppression, completed: - Creation is cause-aware and read-free for signal causes (refusal, opt-out, malformed) - conditioning on observing positive provenance through an eventual row loses the stale-replica race; absence uses a narrow permission-exempt suppression-decision read (the P1-gated read circularity); policy-only tightening writes nothing. - CAS fences writes; authoritative evidence recency decides semantics, with a per-cause transition table: sticky clearing only for opt-out causes, malformed/absence clear on any newer valid grant (sign-off 24), delayed older grants never clear, policy never clears. - Anti-replay: beyond-skew future timestamps are malformed; a digest's first normalized timestamp is pinned and never advanced by re-presentation; digests cover the canonical per-permission semantic result, so equivalent encodings cannot renew authority. - Suppression is inside both AuthorizedIdentity constructors, reads fail closed, retention outlives masked authority, and clearing is fenced on visibility of the matching provenance generation. Storage and capabilities: - New capability rows with per-adapter values: linearizable per-key CAS (suppression), generation-CAS row mutation (rows are heavily mutable, not accretive - Fastly generation markers eligible, Workers KV last-write-wins ineligible), atomic create-if-absent, write-once deployment metadata; revocation reads require globally observable strong consistency, not writer-session read-your-writes; per-class durability and maximum-retention proofs at startup. - Rows carry an absolute expires_at pinned at mint (updates write remaining lifetime - cluster refresh can no longer immortalize an identity) and an immutable mint tag split from the replaceable evidence snapshot; network evidence split from refreshable derived cluster state; cluster_trust_threshold validated against listing caps. - The key grammar's class tags are all non-hex (family tag f -> r) so legacy-grammar disjointness is provable, and provider codes come from a checked-in append-only never-reused registry; the identifier bound is numeric (256 bytes) in the normative contract. Migration: - Once new-shape config is active, N+1 batch sync fails closed on provenance-less rows - the fail-closed rule cannot activate later than the model it protects. - The schema floor has a protocol: dedicated deployment-metadata primitive, create-or-CAS with read-back before enabling writes, startup enforcement, fail-closed on unreadable. - Provider rollback is config-first (writer back, retain as legacy reader, then binaries), distinct from binaries-first schema rollback. - Irreversible artifacts enumerated (revocation, floor, sticky suppression) with recovery/administrative procedures; fixtures branch on capability eligibility; sign-off rows 22-24 added. Hook and security channel: - CDN cache fields reserved outright; stale-* durations shrink-only; the last Set-Cookie contradiction removed; append/replace legality from a core-owned field registry with unknown-fields-reject-append; operations are attributed batches, validated and budgeted atomically (a security 302 can never keep its cookie but lose Location). - Section 4a closes the security channel: typed owned-name cookie operation (ts-* rejected, sign-off 23 for the identifier lifecycle), direction-scoped request-header allowlists applied to a scoped upstream overlay (no credential/identity/routing injection), decision-scoped representation (a challenge owns its body; Continue cannot touch publisher bytes), one global order with the invariant pass unconditionally last. - Eligibility matrix gains HEAD (header parity with GET mandatory) and explicit 1xx/204/205/206 rows. Deferred drafts: the client-cycle page leg gains a pre-vendor-contact live permission check with BFCache abort (TOCTOU); its stale references to the removed acquisition API and old key grammar are marked for renormalization. Ledger: the five overstated R8 dispositions are reopened and corrected, and a Rounds 9-10 section records every finding above. --- ...26-07-30-client-cycle-ec-resolve-design.md | 19 +- ...integration-response-header-hook-design.md | 115 +++++++--- .../2026-07-30-permission-model-design.md | 142 +++++++----- .../2026-07-30-pluggable-providers-design.md | 207 ++++++++++-------- ...07-30-provider-migration-rollout-design.md | 155 +++++++------ docs/superpowers/specs/pr986-review-ledger.md | 85 ++++--- 6 files changed, 451 insertions(+), 272 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index c1ea840ed..eda8268eb 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -66,10 +66,13 @@ Everything in this spec follows from that. enough to set identity. Requests with no `Origin` and no valid token are rejected. 2. **Verify the payload cryptographically per provider — including against - replay.** `resolve_from_client` is the client-resolve acquisition mode - of the provider contract (providers spec §4 - `Acquisition::ClientResolve`, which also carries the JS module the - page leg needs) — no longer an undeclared method this spec invents. It + replay.** `resolve_from_client` belongs to the client-resolve acquisition + surface that was **removed from the normative provider contract with + this feature's deferral** (providers spec §4) — it, the acquisition + enum, and the reservation schemas live only in this informative + draft, to be renormalized (against the current trait and the + delimiter-free key grammar, which superseded the `resv/…` sketch + here) when the feature gets its issue. It accepts only payloads that are signed by an expected party, **audience-bound** to this publisher, and **expiring**. Audience binding and expiry alone do not @@ -195,7 +198,13 @@ with only the later POST refused. The module is injected/activated only when the request's resolved permissions already satisfy the provider's complete `required_permissions()`, and the page leg is a listed row in the permission spec's §7 enforcement inventory (deferred alongside this -feature). +feature). Injection-time gating alone has a TOCTOU gap: consent can be +withdrawn between document delivery and asynchronous vendor contact, or +the document can be restored from BFCache long after its permissions +were resolved — so the module must additionally perform a **live +CMP/permission check immediately before vendor contact** and abort on +permission change or BFCache restoration; endpoint rejection is too late +to undo browser-side identity derivation or vendor egress. - The re-post guard must not depend on reading an HttpOnly cookie. Either the server injects a "resolved" marker the script _can_ read (a diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index b16350ac2..cee462fe4 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -73,9 +73,13 @@ mutators to the outbound response for HTML document responses it processed. mutation ⇒ present in the final response, independently; `public` is dropped whenever any restriction is present; `max-age`/`s-maxage` may only shrink relative to the snapshot; `stale-while-revalidate`/ - `stale-if-error` may appear only if the snapshot had them; every - CDN/surrogate directive (`Surrogate-Control`, `CDN-Cache-Control`, - host equivalents) is stripped from any restricted response; and the + `stale-if-error` may appear only if the snapshot had them **and their + durations may only shrink** (present-at-1s must not become + present-at-1y); CDN-specific cache fields (`Surrogate-Control`, + `CDN-Cache-Control`, host equivalents) are **reserved outright** — + merging them per-directive on unrestricted responses was a hole (an + unrestricted `CDN-Cache-Control: max-age=60` could become a year), and + they are additionally stripped from any restricted response; and the final `Vary` is the **union of the complete snapshot `Vary` set** — origin-supplied members included, not only core-required ones — and the mutation. Middle-stage placement also keeps @@ -126,11 +130,13 @@ mutators to the outbound response for HTML document responses it processed. constants next to the definitions they protect, not duplicated in the hook. - For non-reserved headers, the mutator API distinguishes **append** from - **replace** explicitly; **append is valid only for genuinely - list-valued headers** (a singleton header accepts only replace — two - values of a singleton header by append is a malformed response, not a - merge); the default is append where legal (for `Set-Cookie`, append is - the only non-reserved operation — replace is not offered). Replacing a + **replace** explicitly; append/replace legality comes from a **core-owned field registry**, + not adapter judgment: each known field is classified + append-legal (genuinely list-valued: `Link`, CSP report groups, …), + replace-only (singletons: `Content-Language`, …), or rejected; + **unknown extension fields reject append by default** (replace only) — + "genuinely list-valued" is not a decision four adapters can make + independently and identically (`Set-Cookie` is fully reserved in v1 — neither append nor replace). Replacing a header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). @@ -155,8 +161,16 @@ mutators to the outbound response for HTML document responses it processed. **excludes every `Set-Cookie` value and every reserved identity, consent, and privacy header value** (names may be listed as present; values are withheld). - Exceeding a limit rejects the excess operations (logged, attributed), - never the response. A mutator that returns an error is skipped in full — its + Operations arrive as **attributed batches bound to a registration + ID** — one batch per integration per response, ordered by + registration, with the security channel's batch (§4a) ordered before + response mutators; the current flat effects vector satisfies neither + attribution nor budgets and is restructured accordingly. Validation + and budgeting are **atomic per batch**: a batch that exceeds its + budget is rejected whole (logged, attributed), never partially + applied — item-by-item rejection could apply a security 302's + `Set-Cookie` while dropping its `Location`. The response itself is + never rejected. A mutator that returns an error is skipped in full — its operations are all-or-nothing — and the response proceeds without it. **Panics are forbidden and fatal, not recoverable**: the primary target (`wasm32-wasip1`) builds with `panic = "abort"`, so there is no unwind @@ -170,36 +184,73 @@ mutators to the outbound response for HTML document responses it processed. Which responses the hook runs on, enumerated so two implementations cannot diverge silently: -| Response | Hook runs? | -| --------------------------------------------- | ------------------------------------------------------------ | -| Processed HTML document (rewritten by TS) | Yes | -| Streamed processed document | Yes — operations apply to the header block before first byte | -| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | -| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | -| Redirect (3xx) | No | -| Error responses TS itself generates (4xx/5xx) | No | -| `304 Not Modified` | No | +| Response | Hook runs? | +| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes — operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | +| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` | No | +| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | +| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to processed documents (§6). +## 4a. The security channel — normative closed boundary + +The security channel (today: DataDome) is not a general exception; every +degree of freedom is closed: + +- **Typed security-cookie operation, not header strings.** The channel + emits cookies only through a typed operation whose cookie **names come + from the integration's registered ownership list** (for DataDome, its + documented cookie); every `ts-*` name is rejected; domain/path scope, + attributes, size, and lifetime are constrained by the registration. + Read/vendor-egress/withdrawal semantics of the resulting identifier + are a ratified security-purpose carve-out — **sign-off item 23** — + because the tag-injection → cookie/ClientID read → vendor-send + lifecycle otherwise hands a permission-denied visitor a stable, + exported identifier. No other request filter inherits the cookie + capability. +- **Request-header pointers are direction-scoped allowlists.** Values a + security response names for copying into the request (DataDome's + header-pointer mechanism) are accepted only from a **documented + enrichment-header allowlist**; authentication, `Cookie`, + `Forwarded`/`X-Forwarded-*`, identity, consent, and routing-authority + fields are rejected by name and by class — a compromised endpoint must + not replace origin credentials, inject `ts-ec`, or spoof client + location — and accepted values apply to a **narrowly scoped upstream + overlay**, never the shared request that later integrations read. +- **Representation rules are decision-scoped.** A _Respond_ decision + (challenge/deny) **owns its body** and may set representation headers + (`Content-Type`, encoding, validators) for it — the hook's + representation reservation exists because ordinary mutators do not own + the body, and this one does. A _Continue_ decision may not touch + representation metadata of publisher bytes. +- **One global order, no "wins" exception:** core finalization → + hook/security effects → **final cache/privacy invariant pass, + unconditionally last**. The prior DataDome contract's "applies last + and wins" holds only _within_ the effects layer; nothing outranks the + invariant pass, or a challenge could combine `Set-Cookie` with public + caching. +- The channel adopts the shared layers: structured attributed batches + (§3, atomic per batch — a 302 must never lose `Location` to a budget + while keeping its cookie; on rejection the channel follows DataDome's + specified fail-open), reserved header names, budgets, and the + invariant pass. + ## 4. Done-when (from #782, sharpened) 1. Trait + builder + registry application, each public item documented. 2. **The pre-existing `RequestFilterEffects.response_headers` channel - remains a distinct, core-owned security channel — not folded in, and - not left unvalidated.** Folding it into this hook would break its one - real consumer: DataDome sets headers **and cookies** on 200, 301/302, - 401, 403, and 429 responses — challenge and deny flows on exactly the - response classes (§3a) this hook never runs on, and with cookie - emission v1 reserves. Instead, the channel keeps its own eligibility - (security-integration responses of any status), its cookies are - **core-mediated security cookies** (explicitly outside the deferred - integration-cookie surface, migrated deliberately when that surface - lands), and it adopts the **shared validation layers**: the - structured-operation checks, reserved header names, budgets, and the - final cache/privacy invariant pass. One invariant enforcer, two - eligibility domains. + remains a distinct, core-owned security channel — §4a defines its + closed boundary.** Folding it into this hook would break its one real + consumer: DataDome sets headers **and cookies** on 200, 301/302, 401, + 403, and 429 responses — response classes (§3a) this hook never runs + on, with cookie emission v1 reserves. 3. **At least one real consumer ships in the same PR** — an existing integration registering a mutator for a real need (or, failing a real need, the feature waits; scaffolding with only self-referential tests is diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 21889eb54..360a3f2c1 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -417,53 +417,76 @@ and the fail-closed marker: discoverable from every member, and the record survives member-tombstone replacement (which today discards the original row's identity and metadata, making sibling discovery impossible). -- **Negative authority has its own permission-exempt record.** A live - refusal or non-destructive opt-out must clear prior positive - provenance — but the row write that would do it requires `store-on-device`, - which the refusal just unset, and identity rows may be eventually - consistent, so a stale replica could resurrect a P4 grant after a - targeted-advertising opt-out. The fix is a **suppression record** in - the strongly consistent class, and — because monotonicity is a - read-modify-write property, not a read property — suppression writes - require **linearizable per-key CAS** (providers spec §6.3 - `sup/`, §7 matrix): with plain read-after-write, two - writers can both read the record and an older clear can overwrite a - newer suppress. The record carries its own **version counter, - incremented through the CAS**, so transitions are ordered by - serialization, not by comparing wall-clock timestamps. - - Coverage is **every authority-clearing delta, not an enumerated cause - list**: after each live resolution, any permission whose new state is - unset while its stored provenance is positive gets a suppression - entry — refusal, non-destructive opt-out, malformed-present, and - applicable absence alike. (The earlier refusal/opt-out-only list left - a hole: a malformed record unsets P1, the P1-gated row update is - thereby forbidden, no suppression is written, and batch sync later - honors the stale grant.) - - **Timestamp-less sources get sticky opt-out.** GPP/USP values carry no - intrinsic timestamp, and opt-out → consent → opt-out(same value) is - information-theoretically indistinguishable from a replay of the first - opt-out. Latest-observation semantics would let a replayed old consent - string clear a newer opt-out, so: a suppression from a timestamp-less - source is cleared **only** by a grant carrying an authoritative - timestamp newer than the suppression's observation (TCF - `LastUpdated`) — a timestamp-less re-consent alone does not clear it. - The consequence (a genuine GPP-only re-consent does not restore - authority until a timestamped source or policy provides it) is - declared and is sign-off item 16. Fixtures: suppress-vs-clear race - under concurrent writers; repeated-value opt-out/consent/opt-out. - - **Write failure fails closed for the live request** (the refusal's - effect stands for this response), but the S2S residual is **unbounded - for a never-returning visitor** — exactly like a failed destructive - revocation, not "transient": other instances continue honoring old - provenance, the breaker is per-instance, and no durable retry exists. - This shares sign-off item 11 (extended to cover suppression) and gets - its own fault test. Every S2S recompute and partner-egress check - consults the record: a suppressed permission is unset whatever the - row's provenance says, so no eventual-consistency edge can restore - it. +- **Negative authority has its own permission-exempt record, with a + complete transition contract.** A live refusal or opt-out must clear + prior positive provenance, but the row write that would do it requires + `store-on-device` — which the refusal just unset — and identity rows + may be eventually consistent. The **suppression record** + (`s`-class key per family, providers spec §6.3) resolves this. Its + contract: + + **Creation is cause-aware and mostly read-free.** A live resolution + whose outcome for a permission is unset writes suppression when the + cause is a **signal state** — refusal, non-destructive opt-out, + malformed-present — **unconditionally**, with no row read: conditioning + on observing positive provenance through an eventually consistent row + loses the race where a stale replica hides a just-committed grant. The + one cause that inherently needs prior state — applicable **absence** + clearing a previously positive permission — uses a narrow + **permission-exempt suppression-decision read** exposing only the + family ID and authority metadata (an undeclared exempt read was the + alternative, and skipping it leaves stale S2S authority). + **Policy-only tightening writes nothing**: a policy edit is not a user + signal (§4.2 trigger 3), and a signal-less request after + granted→denied must not create sticky user suppression that a policy + rollback cannot undo. + + **Writes are CAS-fenced; evidence recency decides semantics.** The + record requires linearizable per-key CAS (providers spec §7): CAS + serialization prevents lost updates, but arrival order does **not** + decide outcomes — each per-permission entry stores its cause, source + class, and authoritative evidence timestamp (first-seen normalization + for timestamp-less sources), and an incoming transition applies only + when its evidence timestamp is **newer than or equal to** the stored + entry's; ties resolve to the more restrictive state. So a delayed + grant with `LastUpdated = 100` never clears a suppression whose + refusal carried `200`, while a genuine re-consent at `300` does. The + transition table, by stored cause: **opt-out from a timestamp-less + source** — cleared only by a grant with an authoritative timestamp + newer than the suppression's observation (sticky opt-out, sign-off + 16); **TCF refusal** — cleared by any regime-accepted grant with newer + authoritative evidence; **malformed-present / absence** — cleared by + any regime-accepted valid grant with newer evidence, including a + timestamp-less grant whose first-seen is newer (these causes are not + user opt-outs, so stickiness does not apply — without this rule, one + truncated request would permanently deny a GPP-only user). Policy + changes never clear user-signal suppressions. + + **Anti-replay for timestamps.** A future-dated record is rejected as + malformed beyond the skew window; within it, the record's digest is + stored with its **first normalized timestamp, which re-presentation + never advances** — otherwise a future-dated TCF string replayed after + an opt-out would keep re-normalizing to "now" and clear it. Equality + digests are computed over the **canonical per-permission semantic + result** of §4.5 aggregation, not the raw encoding — two encodings + (or `N` vs explicit N/A) with the same meaning are the same evidence + and keep the original first-seen, so alternating equivalent values + cannot renew authority. + + **Boundary, retention, ordering.** Suppression is checked by **every + `AuthorizedIdentity` constructor** (both scopes), by pull sync's live + path, and by every S2S recompute — not only "partner egress" prose; a + suppression read failure **fails closed** like a revocation read + failure; retention must outlive the positive authority it masks + (providers spec durability/retention capability); and **clearing is + fenced on provenance visibility**: a clear entry records the + provenance generation it reflects, and S2S honors the clear only when + it can read that generation or newer — clearing first would expose the + _older_ positive snapshot through an eventual read. **Write failure + fails closed for the live request**, and the S2S residual is unbounded + for a never-returning visitor (sign-off 11), with fault tests for + suppress-vs-clear races, repeated-value sequences, and the + stale-provenance-read case. - **The cookie expires only after the family record commits.** - **If the family-record write itself fails, nothing durable exists** — @@ -572,10 +595,13 @@ fields grant nothing (their opt-outs still count, per step 2). claimed MD/IN/KY/RI had no section). A truncated map silently loses opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. The implementation PR cross-checks this list against both the current - decoder's section set and the official registry, and **enumerates the - accepted version per section**; a mapped section carrying an unknown - version is treated as malformed-present (blocks grants, never - withdraws — §4.4), not as absent. Adding a section or version is a + decoder's section set and the official registry, and the accepted version per + section is **pinned normatively to the named IAB GPP registry + revision current at this spec's date (2026-08-01)** — "enumerated by + the implementation PR" was two-implementations-diverge territory; a + mapped section carrying a version outside the pinned revision is + treated as malformed-present (blocks grants, never withdraws — §4.4), + not as absent. Adding a section or version is a change to this map. 2. **Applicability gates grants only — never opt-outs.** A mapped **opt-out** field (either subclass) is honored from **any** section on @@ -671,9 +697,9 @@ A policy edit propagates through the config store, so a fleet briefly mixes revisions. The contract: instances stamp every resolution and every provenance write with the policy revision they used (already required by §7); the mixing window is bounded by config propagation and observable via -the config-version metric; and mixed revisions cannot cause irreversible -harm, because **destructive withdrawal triggers are user signals, never -policy** (§4.2 trigger 3) — the one revision-sensitive destructive case +the config-version metric; and mixed-revision irreversibility is bounded and **accepted, not +denied** (sign-off 19): destructive withdrawal triggers are user +signals, never policy (§4.2 trigger 3) — the one revision-sensitive destructive case (trigger 2 under a now-`denied` baseline) requires an affirmative user refusal at the evaluating instance, which is safe under either revision. S2S recomputation always evaluates against the instance's current @@ -770,8 +796,12 @@ Consumers of the resolved set in this epic: | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (a **per-permission equality digest computed over only the applicable, aggregated §4.5 fields for that permission** — never the whole GPP record, or a CMP touching an unrelated notice field would mint a new digest and reset first-seen forever) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | - Timestamps are compared with bounded clock-skew tolerance and - future-dated values are clamped to receipt time. And every live + Timestamps are compared with bounded clock-skew tolerance; + beyond-window future-dated records are **rejected as malformed**, and + within the window a record's first normalized timestamp is pinned to + its digest and never advanced by re-presentation (§4.3's anti-replay + rule — clamping every presentation to "now" would make a future-dated + string perpetually fresh). And every live resolution **atomically replaces the complete per-permission snapshot**, never merges — a refusal, opt-out, malformed or absent state in the fresh resolution clears prior positive authority for its diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index f231601e9..5302a7251 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -119,7 +119,9 @@ Three global rules sit above every provider: - **Identifier bounds.** A minted identifier obeys a global cookie-safe alphabet (valid cookie-octets: no separators, whitespace, or control - characters) and a global maximum length — for the identifier itself, not + characters; normatively `[A-Za-z0-9._~-]`) and a global maximum of + **256 bytes** — stated here, in the normative contract, so dependent + documents reference one number instead of assuming their own — for the identifier itself, not only the graph key — enforced by core at mint and at parse, so no provider can emit a value the cookie layer or logs cannot carry. - **Namespaces are declarative and core-proven.** Disjointness of two @@ -150,7 +152,10 @@ Three global rules sit above every provider: trips the trust threshold toward denial, never toward extra writes); the listing filters by the value's `kind`/liveness within the existing list limit where the backend returns values, and the residual - over-count where it cannot is declared. A computed cluster size is + over-count where it cannot is declared. `cluster_trust_threshold` is validated against the backend's listing + cap at startup — a threshold of 200 against a 100-key listing cap + would make every capped count look trusted; the count must page or + saturate at threshold + 1. A computed cluster size is **not persisted beyond its inputs' lifetime**: today's code stores the calculated `cluster_size` in the row and reuses it for the row's full TTL, which would freeze a tombstone-inflated count for up to a year — @@ -206,11 +211,20 @@ pub trait EdgeCookieProvider { /// key, shared across identifiers minted from the same client /// evidence. None when the provider lacks IP-cluster semantics (§3). fn cluster_prefix(&self, id: &EcId) -> Option; + /// Mint an identifier from request evidence. The one acquisition + /// operation of the epic (server mint); failure means no identity + /// this request (§6.2). Lost from an earlier revision by editing + /// accident — its absence made the required gate → generate → + /// graph-commit sequence unimplementable. + fn generate(&self, input: &IdentityInput<'_>) -> Result>; /// Cryptographic verification of a parsed identifier against request - /// evidence — recognition (`parse`) is not authentication; adoption - /// (§5) and any rowless acceptance require this. - fn verify(&self, id: &EcId, input: &IdentityInput<'_>) -> bool; + /// evidence. Recognition (`parse`) is not authentication; rowless + /// handling (§5) requires this. Returns the matched configuration + /// version — provenance needs it and a bool cannot carry it — or + /// None when nothing verifies. + fn verify(&self, id: &EcId, input: &IdentityInput<'_>) -> Option; } +// VerifiedIdentity { version: ProviderVersion /* … */ } ``` The acquisition-mode enum (`ServerMint` / `ClientResolve`), the @@ -250,40 +264,31 @@ header emission; the identity exists durably from that moment. A graph-commit failure means the mint never happened: no cookie, no egress, error logged, the next request retries. -**Pre-existing cookies without rows are verified, then adopted — parse -is recognition, not authentication.** A syntactically valid -`{64hex}.{6alnum}` string is constructible by anyone; adopting it on -shape alone would let an attacker mint durable rows. The contract: - -- ServerMint providers implement `verify(id, &IdentityInput) -> bool` — - cryptographic verification against **request evidence** (for `hmac`: - recompute over the request's evidence with each configured version's - passphrase and compare to the 64-hex prefix). A recognized rowless - cookie that fails verification is **expired, not adopted** — including - the honest false-negative: a legitimate cookie presented from a - changed network no longer verifies and is expired; the affected - population is graphless deployments only, declared in migration row 13. -- Adoption is gated on the provider's **complete - `required_permissions()`** — exactly like minting, not a hard-coded - `store-on-device`. -- The row write is **atomic create-if-absent** — a distinct capability - row in the §7 matrix (strong class; Workers KV's concurrent same-key - writes can overwrite each other, so it is ineligible, consistent with - its revocation ineligibility). -- **Read errors are not "not found"**: adoption proceeds only on an - authoritative not-found; a failed graph read means no adoption this - request, fail closed. -- Adopted rows do **not** get a fresh full TTL — a nearly expired legacy - identity must not gain a year (the exact rejuvenation problem that - deferred rewrite). Expiry is `min(adopted_at + standard TTL, -migration_cutoff + grace)` with the cutoff configured; sign-off - item 21. - -Until adoption succeeds the cookie **never egresses**; **withdrawal -works without adoption** — the deterministic family ID (permission model -spec §4.3) needs no row, so a first post-upgrade opt-out revokes and -expires the cookie with zero migrated state. Migration matrix row 13 -declares this path. +**Pre-existing rowless cookies are expired and re-minted — never +adopted.** An earlier adoption design failed on an authentication limit: +HMAC verification can authenticate only the 64-hex prefix; the 6-char +suffix is independent randomness, so a client holding `H.aaaaaa` can +present `H.aaaaab`, `H.aaaaac`, … — every variant prefix-verifies, and +an adopt path would mint a **separate durable row and family per +variant**. Therefore: + +- A recognized rowless cookie whose prefix verifies + (`verify → VerifiedIdentity`, carrying the matched version for + provenance) is **expired and replaced by a fresh mint through the + ordinary graph-backed path** (gate → `generate` → commit) when the + request's permissions allow one; continuity with the old identifier is + deliberately not preserved (migration matrix row 13, sign-off 21). A + cookie whose prefix does not verify (including the declared roaming + false-negative) is simply expired. +- **Rowless withdrawal cannot be a row/family-minting oracle**: for + rowless legacy HMAC cookies the derived family ID is a function of the + **authenticated 64-hex prefix only**, so every suffix variant maps to + the _same_ family — one family record withdraws them all, and + attacker-generated variants create nothing new. +- Read errors are still not "not found": a failed graph read means the + cookie is treated as absent this request, fail closed, no expiry + emitted (the row may exist). + declares this path. **Egress is typed, not policed.** The inventory-and-denylist test (permission model spec §7) is a backstop, but conventions do not survive @@ -292,8 +297,10 @@ because raw EC values circulate as ordinary strings. Core therefore introduces a **scope-parameterized `AuthorizedIdentity`**, constructible only by core, only after the checks _for that exact scope_: `AuthorizedIdentity` after parse + `store-on-device` + -family-revocation check; `AuthorizedIdentity` additionally -after `select-personalised-ads`. Outbound serializers (ORTB builder, page +family-revocation **and suppression** checks; +`AuthorizedIdentity` additionally after +`select-personalised-ads` — suppression is part of both constructors, not +a separate prose obligation on S2S callers. Outbound serializers (ORTB builder, page bids, sync, identify, forwarding) accept `AuthorizedIdentity` and nothing weaker — an unparameterized wrapper would let a P1-only identity flow into an ORTB request. A future bypass then @@ -433,7 +440,11 @@ The contract: and the alias record class exists in the key grammar (§6.3) only as reserved-for-future — nothing in the epic writes one. - Retiring a legacy reader is the explicit end of those identities: - the migration guide documents the cleanup procedure (migration spec §6). + the migration guide documents the cleanup procedure (migration spec + §6). **Provenance backfill is not retirement evidence** — a backfilled + row still lives under the legacy cookie namespace and still needs that + provider's parser; only a quiet period spanning the full cookie/row + lifetime justifies removal. - Tests: switch active provider → request with old cookie → identity still resolves and a withdrawal tombstones it; old cookie with no matching legacy reader → treated as absent and **never egresses**; @@ -476,8 +487,10 @@ backend-wide outage degrades every instance through its own observations within one window, but an instance-local family-write failure leaves other instances — which have no record to find, and healthy backends of their own — serving S2S egress until the browser's durable signal retries -successfully. That residual is bounded by the user's return latency, is -counted (failed family writes are a first-class metric), and is accepted +successfully. That residual is **unbounded for a never-returning visitor** +(sign-off item 11 — the permission and migration specs state this and +this spec must not undercut them), is counted (failed family writes are +a first-class metric), and is accepted in place of a deployment-wide shared fail-closed channel, whose own availability and freshness would be a harder problem than the one it solves. @@ -487,15 +500,15 @@ solves. **Physical key grammar.** Core constructs every key; providers supply only the bounded suffix: -| Record class | Key | Notes | -| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Identity row (non-hmac) | `id//` | Suffix from `graph_key_suffix`, ≤ 128 bytes, KV-safe alphabet. **No version segment** — the mint version lives in the row envelope; a versioned key would be circular (core would need the version to read the row that states the version) | -| Identity row (hmac, **every** version) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar for the whole provider, not just v0 — keeping all HMAC versions on the verbatim scheme is what keeps the 64-hex cluster prefix a literal key prefix for every HMAC row. (Passphrase rotation still changes a given IP's HMAC, so clusters split across rotation until old identities expire — inherent to rotation, declared, not a key-scheme artifact) | -| Alias | **The source identity key itself** — the alias is a value written _at_ the replaced row's address, distinguished by a `kind` discriminator in the JSON envelope | A separate `alias/…` address could never work: lookups by the old cookie hit the source key, and a single-key CAS cannot install a record at a different address | -| Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | -| Suppression (negative authority) | `sup/` | Per-permission suppression entries + timestamps; permission-exempt writes; consulted by every S2S recompute and partner-egress check (permission spec §4.3) | -| Rewrite transaction | `rwx/` | One in-flight rewrite per family | -| Replay reservation | `resv////` | Client-cycle spec; payload id ≤ 128 bytes | +| Record class | Key | Notes | +| -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Identity row (non-hmac) | `id//` | Suffix from `graph_key_suffix`, ≤ 128 bytes, KV-safe alphabet. **No version segment** — the mint version lives in the row envelope; a versioned key would be circular (core would need the version to read the row that states the version) | +| Identity row (hmac, **every** version) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar for the whole provider, not just v0 — keeping all HMAC versions on the verbatim scheme is what keeps the 64-hex cluster prefix a literal key prefix for every HMAC row. (Passphrase rotation still changes a given IP's HMAC, so clusters split across rotation until old identities expire — inherent to rotation, declared, not a key-scheme artifact) | +| Alias | **The source identity key itself** — the alias is a value written _at_ the replaced row's address, distinguished by a `kind` discriminator in the JSON envelope | A separate `alias/…` address could never work: lookups by the old cookie hit the source key, and a single-key CAS cannot install a record at a different address | +| Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | +| Suppression (negative authority) | `sup/` | Per-permission suppression entries + timestamps; permission-exempt writes; consulted by every S2S recompute and partner-egress check (permission spec §4.3) | +| Rewrite transaction | `rwx/` | One in-flight rewrite per family | +| Replay reservation _(informative — deferred with client-cycle; not normative surface)_ | `resv/…` | Deferred client-cycle draft | Grammars are pairwise non-intersecting by their literal prefixes (plus the reserved hmac grammar), and every record value carries a `kind` @@ -508,14 +521,19 @@ logical identity different physical keys on different adapters, breaking migration, shared storage, and parity; and Fastly's prefix queries reject both `/` and `:`, so no delimiter character is safely portable). Physical keys are **delimiter-free with fixed-width segments**: a -1-character class tag (`i` row, `f` family, `s` suppression, `x` -transaction), a **4-character registry-assigned provider code** -(zero-padded, `[a-z0-9]`), then the suffix — segment boundaries are -positional, so no segment can contain or escape a delimiter, prefix -queries are plain string prefixes on every backend, and cluster -eligibility needs no per-adapter delimiter negotiation. (hmac verbatim -keys remain the reserved exception, with the 64-hex cluster prefix at -position zero.) +1-character class tag — `i` row, `r` family revocation, `s` suppression, +`x` transaction, every tag chosen **outside the hex alphabet** so no +generated key can begin with 64 hex characters, which is what makes +disjointness from the legacy `{64hex}.{6alnum}` grammar _provable_ +rather than asserted (an earlier `f` tag was itself a hex digit) — then +a **4-character provider code from a checked-in, append-only, +never-reused registry file** (allocation is a reviewed commit; +codes are immutable and never recycled, including for retired +providers), then the suffix. Segment boundaries are positional, so no +segment can contain or escape a delimiter, prefix queries are plain +string prefixes on every backend, and a grammar-disjointness test covers +every class against the legacy grammar. (hmac verbatim keys remain the +reserved exception, with the 64-hex cluster prefix at position zero.) **Wire schemas** (JSON, like identity rows; every class carries a schema version): the **alias record** (reserved-future, with rewrite) holds target key, created-at, retirement @@ -543,23 +561,25 @@ readers round-trip unknown keys **semantically** (values preserved through read-modify-write; byte-identical output is not required and not achievable through a structured serializer). -| Field | Purpose | Source | Gating permission (egress) | TTL / refresh | Rewrite | On revocation | -| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | -| key (v1: identifier verbatim; v2: core-constructed, §4) | Row identity | Provider/core | — | Row TTL (1 y today) | New canonical row; old key becomes alias | Family record governs; member tombstone as cleanup | -| `v` | Schema discriminator | Core | — | — | Written at current version | Retained | -| `created` | Row age | Core | P1 (first-party ops) | Never refreshed | Preserved (no rejuvenation) | Retained in tombstone | -| `consent.tcf` / `consent.gpp` | Raw signal snapshot for audit; superseded as authority by provenance | Request | Never egressed to partners | Replaced on live resolution (§7 snapshot rule, permission spec) | Fresh live values | Scrubbed | -| `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | -| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, provider/version) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | -| New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | -| `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | -| `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | -| `pub_properties` (origin/seen domains) | Creation context | Core at mint | P1 | Write-once | Preserved | Scrubbed | -| `device.*` (JA4 class, H2 hash, quality metadata) | **Discontinued for new rows** (§5): fingerprint-derived, buyer-facing — beyond security-classification authorization. v1 rows retain them read-only; they are never egressed post-epic and are dropped at rewrite | Fastly device provider | None grants egress | Write-once (v1) | **Dropped** | Scrubbed | -| New: security classification outcome (boolean) | Bot-gate result | Device provider | — (never egressed) | Written at mint | Fresh | Scrubbed | -| `network.*` | Cluster disambiguation | Platform at mint | P1 | Write-once | Fresh | Scrubbed | -| `ids` (partner → UID map) | Partner identity graph | Pixel/pull/batch sync | P1 ∧ P4 (partner egress) | Per-mapping timestamps; bounded count/length | Copied **with original timestamps/expiry** | Scrubbed | -| New: alias record kind | Rewrite indirection (§6.1) | Core | — | Retirement deadline | Is the mechanism | Family-revoked like any member | +| Field | Purpose | Source | Gating permission (egress) | TTL / refresh | Rewrite | On revocation | +| ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| key (v1: identifier verbatim; v2: core-constructed, §4) | Row identity | Provider/core | — | Row TTL (1 y today) | New canonical row; old key becomes alias | Family record governs; member tombstone as cleanup | +| `v` | Schema discriminator | Core | — | — | Written at current version | Retained | +| `created` / **`expires_at`** | Row age and the **absolute retention deadline, pinned at mint** — every update writes with the _remaining_ lifetime, never a fresh full TTL (today's full-TTL rewrite lets a frequently visited identity live forever; refreshable derived state must never rejuvenate the identity) | Core | P1 (first-party ops) | Never extended | Preserved (no rejuvenation) | Retained in tombstone | +| `consent.tcf` / `consent.gpp` | Raw signal snapshot for audit; superseded as authority by provenance | Request | Never egressed to partners | Replaced on live resolution (§7 snapshot rule, permission spec) | Fresh live values | Scrubbed | +| `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | +| New: **immutable mint tag** (`mint_provider`, `mint_version`) | Credential retirement and audit — write-once at mint (legacy backfill may populate a missing tag once); **never part of the replaceable snapshot**, or a v1 identity revisited after rotation would be restamped v2 | Mint (or one-time backfill) | — | Immutable | — | Retained | +| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, provider/version) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | +| New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | +| `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | +| `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | +| `pub_properties` (origin/seen domains) | Creation context | Core at mint | P1 | Write-once | Preserved | Scrubbed | +| `device.*` (JA4 class, H2 hash, quality metadata) | **Discontinued for new rows** (§5): fingerprint-derived, buyer-facing — beyond security-classification authorization. v1 rows retain them read-only; they are never egressed post-epic and are dropped at rewrite | Fastly device provider | None grants egress | Write-once (v1) | **Dropped** | Scrubbed | +| New: security classification outcome (boolean) | Bot-gate result | Device provider | — (never egressed) | Written at mint | Fresh | Scrubbed | +| `network.*` (immutable evidence: ASN etc.) | Cluster disambiguation | Platform at mint | P1 | Write-once | Fresh | Scrubbed | +| Derived cluster state (`cluster_size`, computed-at) | Trust gating | Computed | — | **Refreshable, short validity; generation-CAS update; never touches `expires_at`** | Recomputed | Scrubbed | +| `ids` (partner → UID map) | Partner identity graph | Pixel/pull/batch sync | P1 ∧ P4 (partner egress) | Per-mapping timestamps; bounded count/length | Copied **with original timestamps/expiry** | Scrubbed | +| New: alias record kind | Rewrite indirection (§6.1) | Core | — | Retirement deadline | Is the mechanism | Family-revoked like any member | ## 7. Composition root and adapter parity @@ -581,17 +601,26 @@ Requirements: **per-record-class consistency requirements**, because "has KV" says nothing about whether revocation is observable: - | Record class | Required semantics | - | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | - | Family revocation records | **Strongly consistent (read-after-write) primitive required.** Cloudflare Workers KV is **not eligible** — its documentation says propagation may take "60 seconds or more", an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | - | Family suppression records | Same strong class as family revocation — negative authority must not lose races to stale replicas | - | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | - | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | - | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | - - Each adapter's declaration is part of its wiring, drives the §6 - capability-mismatch startup error, and every §6.2 runtime-failure row + | Record class | Required semantics | + | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | + | Family revocation records | **Globally observable strong consistency** — every instance's read observes a committed revocation, not merely the writing session's own writes (writer-scoped read-your-writes is insufficient for a fleet). Cloudflare Workers KV is **not eligible** — "60 seconds or more" is an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | + | Family suppression records | **Linearizable per-key CAS** — read-after-write alone cannot provide read-modify-write monotonicity: two writers both read, and an older clear overwrites a newer suppress | + | Identity-row mutation | **Generation CAS** (conditional write on row generation) with reread/recompute on conflict — rows are heavily mutable (snapshots replaced, partner IDs merged, derived state refreshed), so unordered last-writer-wins loses newer evidence and mappings; _visibility_ may stay eventual, unordered _mutation_ may not. Fastly KV offers generation-marker conditional writes; Workers KV's documented concurrent last-write-wins is ineligible for mutation-bearing rows | + | Row creation | **Atomic create-if-absent** (fresh mints), same primitive family | + | Deployment metadata (schema floor) | **Write-once/CAS**, outside ordinary config storage (migration spec §4) | + | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | + | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | + | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | + + Every record class additionally declares **durability and maximum + retention**: a store passing the consistency check but capping TTLs + below the computed revocation/suppression horizon (e.g. a 30-day + maximum against one-year rows) would let identities become usable + again when their revocation expires — startup proves the configured + store meets each class's computed horizon, and persistence across + restart is part of the declaration. Each adapter's declaration is part + of its wiring, drives the §6 capability-mismatch startup error, and every §6.2 runtime-failure row gets fault-injection coverage on every adapter declaring the corresponding capability. The **concrete per-adapter values** — the actual matrix, not the abstract capability list — as known today; a diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 6299d5448..82fe5779b 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -32,30 +32,30 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | -| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | -| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | -| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | -| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | -| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — opt-out effects are more protective; the same fields' not-opted-out values can also **newly grant P4**, which is not (both effects classified in permission spec §4.5) | -| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | -| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | -| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | -| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | -| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | -| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | -| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | -| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | -| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | -| 13 | Cookies minted by graphless deployments have no graph row | Recognized rowless legacy cookies are adopted via a permission-gated, race-safe create-if-absent on a live request (providers spec §5); never egress before adoption; withdrawal works without adoption via the derived family ID | **Declared** — identity use of pre-existing cookies pauses until adopted | +| # | Decision (today) | After epic | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | +| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | +| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | +| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | +| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — opt-out effects are more protective; the same fields' not-opted-out values can also **newly grant P4**, which is not (both effects classified in permission spec §4.5) | +| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | +| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | +| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | +| 13 | Cookies minted by graphless deployments have no graph row | Recognized rowless cookies are **expired and re-minted** through the ordinary graph-backed path (providers spec §5) — never adopted, since prefix-only verification cannot authenticate suffix variants; identity continuity is deliberately lost; withdrawal works without any row via the prefix-derived family ID | **Declared** — pre-existing identities restart rather than carry over | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -142,9 +142,15 @@ Requirements: unchanged** — dual-read means dual-behavior — so the compiled protective fallback cannot flip behavior mid-convergence before the operator pushes the new-shape policy; the new model engages only - with new-shape config. The interim (N+1 minting v1 rows, S2S - running today's checks) is declared as sign-off item 20, not - discovered. + with new-shape config. The interim is declared as sign-off item 20 — with one + boundary that does **not** wait for N+2: once new-shape config is + active, **context-free partner egress (batch sync) on N+1 fails + closed for rows without provenance**, exactly as the permission + spec's legacy rule requires. Otherwise N+1 would mint a P1-only v1 + row under the new model and then release it through today's + row-state-only batch check — the fail-closed rule cannot activate + later than the model it protects. Live-request paths keep v1 + semantics until N+2. Rollback tests therefore run the family-revocation and suppression paths — read **and write** — plus v1-minting behavior, on N+1 @@ -158,11 +164,15 @@ Requirements: must ship compiled-in (dormant: registered, parseable, configurable, not selectable as writer) in R−1**; adopting a genuinely new provider gets its own reader-first rollout, exactly - like the epic itself. With that rule, rolling N+2 → N+1 keeps the - new provider's identities resolvable and withdrawable through the - dormant registration ("retain as legacy reader" is now satisfiable - because N+1 physically contains the code); the fleet still first - converges on a config selecting only what N+1 accepts as writer. + like the epic itself. With that rule, **schema rollback and provider rollback are + distinct sequences**: schema rollback is binaries-first (above); + **provider rollback is config-first** — a fleet whose config + _selects_ the new provider as writer cannot roll binaries first, + because the older binary rejects that active writer even while + containing its dormant code. The order: switch the current fleet's + writer back to the older provider (retaining the new one in + `legacy_providers`, satisfiable because N+1 physically contains the + code), converge, then roll binaries. N+1 additionally **rejects writer selections whose provenance it cannot yet encode** — new-writer adoption waits for N+2, so no row is minted that N+2 would misclassify. Every new @@ -212,12 +222,20 @@ Requirements: genuinely pre-N+1 worker cannot preserve at all, which is exactly why the floor exists); after the **fleet-convergence gate**, **N+2 activates the writer** and begins emitting the new fields. **The rollback floor is crossed at N+2 writer activation itself** — an - observable deploy event, recorded as a durable schema-floor marker - **in write-once/CAS deployment metadata that ordinary config rollback - cannot touch** — floor-in-rollbackable-config would let "restore the - previous config version" erase the floor after new-format rows exist, - which is exactly the state it guards — not at "any new-format row - exists", which no operator can disprove. Below-floor rollback is + observable deploy event, recorded in the **deployment-metadata + primitive** (providers spec §7 capability row; the existing + config-store interface exposes ordinary put/delete and cannot express + a monotonic floor) with a specified protocol, not an assertion: the + marker lives in a dedicated namespace outside rollbackable config; + the **first N+2 instance to activate creates/advances it via + create-or-CAS** (the creation race resolves to one winner), **reads + it back, and only then enables new-format writes**; every binary + reads the floor at startup and a binary below the floor **fails + startup**; an unreadable floor fails closed (writer stays disabled). + Floor-in-rollbackable-config would let "restore the previous config + version" erase the marker after new-format rows exist — exactly the + state it guards — and "any new-format row exists" is a fact no + operator can disprove. Below-floor rollback is prohibited from that marker on; a pre-floor binary would silently strip the new fields from every row it touches. Rows carry the existing `v` schema discriminator; backfill is lazy via @@ -280,7 +298,10 @@ provenance gate (row 11c). Everything else the recipe preserves. The migration guide (a new `docs/guide/` page, linked from the release notes) gives one copy-pasteable recipe per adapter for the minimal-divergence -posture: +posture — **branching on capability eligibility**: adapters passing the +revocation-storage gate get the HMAC + graph fixture below; ungated +adapters get the explicitly stateless fixture of §4.2, and no universal +HMAC requirement contradicts that: The recipe is a **complete, valid TOML fixture per adapter, committed to the repository** (e.g. `docs/guide/fixtures/migration-preserving-fastly.toml` @@ -381,7 +402,12 @@ global honoring of opt-out signals is unconditional. model) is rejected in the permission spec. 5. Rollback is config-only where possible: reverting to the previous config version restores the previous behavior on the previous binary. The - one irreversible artifact is withdrawal tombstones — which is why the + irreversible artifacts are enumerated — not "one": **family + revocation records and member tombstones** (no recovery; that is + their purpose), the **schema-floor marker** (write-once by design; + no administrative clear), and **sticky timestamp-less suppression** + (administrative clear procedure documented in the guide, requiring + recorded operator intent). Withdrawal tombstones — which is why the withdrawal triggers (permission spec §4.2) are exhaustive, why partial withdrawal failure has an explicit tombstones-first, browser-retries contract (permission spec §4.3), and why §2 rows 6 and 8 call out @@ -412,26 +438,29 @@ made differently). **Implementation is blocked while any row is `open`**; each row needs an owner, a status, and a link to its decision record — an unratified row reverts to open, not to silently implemented. -| # | Decision | Where | Owner | Status | -| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | -| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | -| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | -| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | -| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | -| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | -| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | -| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | -| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | -| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | -| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | -| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | -| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | -| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | -| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | -| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | -| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | -| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | -| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | -| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | -| 21 | Adopted legacy rows are bounded by a migration cutoff, not a fresh full TTL | providers §5 | maintainers + product | open | -| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | +| # | Decision | Where | Owner | Status | +| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | +| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | +| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | +| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | +| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | +| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | +| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | +| 23 | DataDome security exemption: tag injection, cookie/ClientID read, vendor egress, and cross-integration visibility operate outside the permission model as a ratified security-purpose carve-out with owned cookie names, scope, and withdrawal semantics | hook §4a; permission §7 | maintainers + legal | open | +| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky); opt-out stickiness applies only to opt-out causes | permission §4.3 | maintainers + legal | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md index fa5a68756..85a35e965 100644 --- a/docs/superpowers/specs/pr986-review-ledger.md +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -133,30 +133,61 @@ the R7 table below). Sign-off table with owners/status introduced R6. ## Round 8 — re-review at 09e54e96 -| Finding | Status | -| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| P1.1 suppression monotonicity unprovidable | fixed (linearizable CAS + record version counter; sticky opt-out for timestamp-less sources — sign-off 16; race fixtures) | -| P1.2 malformed/absent leave stale authority | fixed (suppression on every positive→unset delta, cause-agnostic) | -| P1.3 suppression-write failure "transient" | fixed (unbounded residual, shares sign-off 11, fault test) | -| P1.4 N/A double meaning | fixed (single rule: explicit N/A = grant-class, absent = nothing; sign-off 17) | -| P1.5 adoption = syntax-as-authentication | fixed (`verify` against request evidence; expire on failure; full required_permissions; atomic create-if-absent capability; read-error ≠ not-found) | -| P1.6 N+1 impossible write behavior | fixed (N+1 mints v1 with today's semantics; old-shape config runs pre-epic gate; new contracts activate at N+2/new-shape — sign-off 20) | -| P1.7 N+2-only provider readable by N+1 | fixed (providers ship compiled-in dormant one release early; reader-first per provider) | -| P1.8 cookie deferral contradictions | fixed (ops list, reserved remnant, generic-op wording, core-owned-cookie test) | -| P1.9 DataDome fold-in breakage | fixed (distinct security channel, shared validation/invariant layers, core-mediated security cookies) | -| P1.10 freshness metadata weakening | fixed (`Age`/`Date`/`Expires` reserved, rationale in-spec) | -| P1.11 client-cycle in normative core | fixed (Acquisition enum/ClientResolve/reservations removed from trait surface; `verify` added; deferred doc holds the rest) | -| P1.12 redaction unspecified | fixed (see corrected R7 row above) | -| P2.1 rewrite residue | fixed (tests, runtime row, metrics/retirement swept; alias schema marked reserved) | -| P2.2 delimiter portability | fixed (see corrected R7 row above) | -| P2.3 Axum matrix | fixed (see corrected R7 row above) | -| P2.4 adoption rejuvenation | fixed (migration-cutoff-bounded TTL; sign-off 21) | -| P2.5 stored cluster overcount | fixed (no persistence beyond inputs' lifetime) | -| P2.6 GPP applicability leftovers | fixed (MD/IN/KY/RI sentence removed; section-6 grants defined; regime-`none` row reconciled with applicability) | -| P2.7 mixed-revision divergence | accepted explicitly (sign-off 19) | -| P2.8 floor marker rollbackable | fixed (write-once/CAS deployment metadata) | -| P2.9 sign-off gaps | fixed (rows 16–21 added; rows 3 and 11 amended) | -| P2.10 duplicate integration IDs | fixed (startup rejection + test; current silent coalescing named) | -| P3 GPP version pinning | fixed (accepted versions enumerated; unknown version = malformed-present) | -| P3 geo region vocabulary | fixed (ISO output or declared canonical mapping) | -| P3 stale fragments + ledger overstatement | fixed (this ledger corrected; hook remnants swept) | +| Finding | Status | +| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| P1.1 suppression monotonicity unprovidable | R8 partial (CAS ordered arrival, not evidence recency; creation still row-observation-dependent) → **refixed R9/R10**: evidence-recency transition table, cause-aware read-free creation, exempt decision read, boundary/retention/fenced clearing | +| P1.2 malformed/absent leave stale authority | fixed (suppression on every positive→unset delta, cause-agnostic) | +| P1.3 suppression-write failure "transient" | fixed (unbounded residual, shares sign-off 11, fault test) | +| P1.4 N/A double meaning | fixed (single rule: explicit N/A = grant-class, absent = nothing; sign-off 17) | +| P1.5 adoption = syntax-as-authentication | R8 partial (prefix verify cannot authenticate suffixes — adoption itself was unsound) → **refixed R9/R10**: adoption removed; rowless cookies expire-and-re-mint; prefix-derived family collapses suffix variants; `verify → VerifiedIdentity{version}` | +| P1.6 N+1 impossible write behavior | fixed (N+1 mints v1 with today's semantics; old-shape config runs pre-epic gate; new contracts activate at N+2/new-shape — sign-off 20) | +| P1.7 N+2-only provider readable by N+1 | fixed (providers ship compiled-in dormant one release early; reader-first per provider) | +| P1.8 cookie deferral contradictions | R8 partial (append-only-non-reserved remnant survived) → **swept R9/R10** | +| P1.9 DataDome fold-in breakage | R8 partial (channel kept but boundary open: untyped cookies, header pointers, ordering conflict) → **refixed R9/R10**: §4a closed boundary — typed owned-name cookie op, direction-scoped allowlists, decision-scoped representation, invariant-last ordering, atomic batches; sign-off 23 | +| P1.10 freshness metadata weakening | fixed (`Age`/`Date`/`Expires` reserved, rationale in-spec) | +| P1.11 client-cycle in normative core | fixed (Acquisition enum/ClientResolve/reservations removed from trait surface; `verify` added; deferred doc holds the rest) | +| P1.12 redaction unspecified | fixed (see corrected R7 row above) | +| P2.1 rewrite residue | fixed (tests, runtime row, metrics/retirement swept; alias schema marked reserved) | +| P2.2 delimiter portability | fixed (see corrected R7 row above) | +| P2.3 Axum matrix | fixed (see corrected R7 row above) | +| P2.4 adoption rejuvenation | fixed (migration-cutoff-bounded TTL; sign-off 21) | +| P2.5 stored cluster overcount | fixed (no persistence beyond inputs' lifetime) | +| P2.6 GPP applicability leftovers | fixed (MD/IN/KY/RI sentence removed; section-6 grants defined; regime-`none` row reconciled with applicability) | +| P2.7 mixed-revision divergence | accepted explicitly (sign-off 19) | +| P2.8 floor marker rollbackable | fixed (write-once/CAS deployment metadata) | +| P2.9 sign-off gaps | fixed (rows 16–21 added; rows 3 and 11 amended) | +| P2.10 duplicate integration IDs | fixed (startup rejection + test; current silent coalescing named) | +| P3 GPP version pinning | R8 partial (still implementation-enumerated) → **refixed R9/R10**: pinned to a named registry revision in the normative spec | +| P3 geo region vocabulary | fixed (ISO output or declared canonical mapping) | +| P3 stale fragments + ledger overstatement | fixed (this ledger corrected; hook remnants swept) | + +## Rounds 9–10 — dual review at ff1e113e + +R9 (10 P1, 13 P2, 1 P3) and the same-head re-audit R10 (9 P1, 9 P2, 1 P3) +are dispositioned together; R10's "previously open" list = R9's P1s, +tracked once. + +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | +| R9-1 trait cannot mint (`generate` lost in an R8 edit) | fixed — restored with failure semantics | +| R9-2 / R10-open rowless HMAC unauthenticatable suffixes | fixed — expire-and-re-mint, prefix-derived family, `VerifiedIdentity{version}` | +| R9-3 missing capability rows (suppression CAS, create-if-absent, floor metadata) + global visibility | fixed — distinct rows with per-adapter values (Fastly generation markers; Workers KV ineligible), globally observable revocation reads | +| R9-4 CAS orders arrival not recency | fixed — evidence-recency transition table with per-cause clearing | +| R9-5 creation condition fails both directions | fixed — cause-aware read-free creation; policy-only tightening writes nothing; absence uses the exempt decision read | +| R9-6 suppression outside the authorization boundary | fixed — both constructors, fail-closed reads, retention rule, provenance-generation-fenced clearing | +| R9-7 N+1 new-shape S2S unsafe | fixed — context-free partner egress fails closed on provenance-less rows once new-shape config is active | +| R9-8 schema-floor protocol unspecified | fixed — dedicated primitive, create-or-CAS, read-back-then-enable, startup enforcement, fail-closed | +| R9-9 security-cookie exception reopens cookie surface | fixed — §4a typed owned-name operation, ts-\* rejected, sign-off 23 | +| R9-10 DataDome representation/ordering conflicts | fixed — decision-scoped representation; one global order, invariant last | +| R9-P2 batch (residual wording; deferred residue markers; registry + non-hex tags; cluster cap; retirement evidence; matrix/fixture branching; cutoff removed with adoption; irreversible artifacts enumerated; mixed-policy absolute removed; version pin; effect attribution; Set-Cookie remnants; network split) | all fixed | +| R9-P3 / R10-P2.8 eligibility rows (HEAD, 1xx, 204, 205, 206) | fixed — HEAD mirrors GET; others enumerated No | +| R10-1 future-dated TCF replay | fixed — beyond-skew rejected as malformed; digest-pinned first normalization | +| R10-2 mutable rows vs eventual/accretive claim | fixed — generation-CAS mutation, eventual visibility only | +| R10-3 cluster refresh extends retention | fixed — absolute `expires_at`, remaining-lifetime writes | +| R10-4 mint version inside replaceable snapshot | fixed — immutable mint tag split from evidence | +| R10-5 suppression needs forbidden read | fixed — read-free signal causes; narrow exempt decision read for absence | +| R10-6 durability/retention not validated | fixed — per-class durability + max-retention capabilities, startup horizon proof | +| R10-7 CDN-header weakening | fixed — CDN cache fields reserved outright; stale-\* durations shrink-only | +| R10-8 DataDome request-header injection | fixed — direction-scoped allowlist, scoped upstream overlay | +| R10-9 DataDome identifier outside permission model | ratification — sign-off 23 | +| R10-P2 batch (semantic digests; malformed/absence clearing + sign-off 24; atomic security batches; field registry; 256-byte bound in normative spec; device sign-off 22; provider-vs-schema rollback; TOCTOU live-check) | all fixed | From bf684e5d4a56756ed70c0e3188353062a587abf5 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 1 Aug 2026 22:41:14 -0700 Subject: [PATCH 12/14] Address eleventh review: suppression recovery, authoritative rowless handling, and the closed DataDome contract P1 fixes: - Suppression cannot deadlock its own recovery: AuthorityRefresh is a permission-exempt write path strictly scoped to committing provenance from the current live resolution while suppression stays effective; the clear then references that provenance's application-level monotonic revision (backend generation markers detect change and carry no order - per Fastly's own contract - so revisions are app-level counters). - Rowless classification is safe: it activates only under a deployment-metadata graphless-migration flag with a strongest-read existence check; otherwise (and on any read error) the state is indeterminate - no identity use, no mint, no cookie expiry. Rowless withdrawal writes nothing: there is no server-side state to revoke, and the prefix-derived family record is removed - it let unauthenticated suffix variants mint records and, because the HMAC prefix is per-IP, would have revoked every identity behind one IP. Family derivation is now one rule everywhere (full graph key, row-backed identities only). - The absence decision reads a strong record: the per-family record is now the authority-state record, carrying a per-permission positive-authority summary CAS-updated by every provenance write - never the eventual identity row, whose stale not-found loses the fresh-grant race. - The concrete adapter matrix gains cells for every mandatory capability (suppression CAS, row-mutation CAS, create-if-absent, deployment metadata, durability/retention) across all four adapters; the accretive claim is deleted (eventual visibility only after a generation-CAS mutation); Axum storage-dependent cells read Unavailable until a store exists. - The suppression wire schema is complete (state, cause, source class, evidence/observation timestamp, referenced provenance revision, positive summary, CAS version, schema version). - generate returns GeneratedIdentity { id, mint_version } - core cannot otherwise record the immutable mint tag; provider/version is removed from the mutable snapshot in both specs. - The DataDome contradictions are resolved: X-DataDome-ClientID is positively enumerated (with the documented X-DataDome-* set) and applies to an owner-scoped upstream overlay, never the shared view, egress under sign-off 23; the cookie carve-out has a concrete lifecycle (name datadome, apex scope, Secure/SameSite=Lax, 13-month ceiling, 4 KiB, owner-only read, deletion always, withdrawal semantics = the open half of item 23, which is pending ratification, not ratified); ordering is one global order (core -> ordinary mutators -> security effects -> invariant pass) with the older DataDome doc marked superseded; challenge batches validate and budget before Respond commits, so rejection can still fail open to Continue. - Headers-only v1 is permission-neutral by construction: the field registry admits inert fields only - Link preload, Reporting- Endpoints/NEL, CSP reporting, and Refresh cause vendor contact and are rejected until permission-declared mutation exists; unknown fields are rejected entirely. P2/P3: exempt-read and AuthorityRefresh rows added to the enforcement inventory with field enumeration; TCF digests include LastUpdated while GPP/USP digest semantics only (genuine CMP renewal refreshes, replays do not); malformed/absence causes get observation timestamps with cross-source comparison rules; the 4.1 matrix gains the suppression condition (one malformed request denies later no-signal requests under granted - sign-off 24 expanded); graph-read errors are indeterminate, not absent; the rewrite/backfill retirement alternative is removed (full-lifetime quiet period only); GPP versions pin to a vendored registry snapshot file; 304 for processed representations gets a 304-safe metadata pass; CDN cache fields are enumerated by name (Surrogate-Control, CDN-Cache-Control, Cloudflare-CDN-Cache-Control, Edge-Control); deferred reservation/CAS material is bracketed informative; the verbatim-key comment covers every hmac version; Content-Language replaced by true singletons; docs/superpowers/specs/ decisions/ created as the sign-off table's decision-record home. Ledger: R9-3, R10-2, R10-4, and GPP-pinning dispositions corrected to partial-then-refixed; full Round 11 section added. --- ...integration-response-header-hook-design.md | 128 +++++++++++------- .../2026-07-30-permission-model-design.md | 92 +++++++++---- .../2026-07-30-pluggable-providers-design.md | 93 ++++++++----- ...07-30-provider-migration-rollout-design.md | 66 ++++----- docs/superpowers/specs/decisions/README.md | 6 + docs/superpowers/specs/pr986-review-ledger.md | 67 +++++---- 6 files changed, 285 insertions(+), 167 deletions(-) create mode 100644 docs/superpowers/specs/decisions/README.md diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index cee462fe4..97dd5d484 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -75,8 +75,11 @@ mutators to the outbound response for HTML document responses it processed. only shrink relative to the snapshot; `stale-while-revalidate`/ `stale-if-error` may appear only if the snapshot had them **and their durations may only shrink** (present-at-1s must not become - present-at-1y); CDN-specific cache fields (`Surrogate-Control`, - `CDN-Cache-Control`, host equivalents) are **reserved outright** — + present-at-1y); CDN-specific cache fields are **reserved outright, by enumerated + name in the field registry** — `Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, and + `Edge-Control`, each individually tested ("host equivalents" was not + a matching rule four adapters would implement identically) — merging them per-directive on unrestricted responses was a hole (an unrestricted `CDN-Cache-Control: max-age=60` could become a year), and they are additionally stripped from any restricted response; and the @@ -131,12 +134,18 @@ mutators to the outbound response for HTML document responses it processed. hook. - For non-reserved headers, the mutator API distinguishes **append** from **replace** explicitly; append/replace legality comes from a **core-owned field registry**, - not adapter judgment: each known field is classified - append-legal (genuinely list-valued: `Link`, CSP report groups, …), - replace-only (singletons: `Content-Language`, …), or rejected; - **unknown extension fields reject append by default** (replace only) — - "genuinely list-valued" is not a decision four adapters can make - independently and identically (`Set-Cookie` is fully reserved in v1 — neither append nor replace). Replacing a + not adapter judgment — and the v1 registry admits **inert fields + only**: "headers-only" is not automatically permission-neutral, since + `Link` (preload/prefetch), `Reporting-Endpoints`/NEL, CSP report + directives, and `Refresh` cause browser-initiated vendor contact on + requests that granted nothing. Fields with active egress side effects + are **rejected in v1**; a follow-up may admit them behind declared + required permissions gated at mutation time. Within the inert set, + each field is classified append-legal (genuinely list-valued), + replace-only (true singletons — e.g. `Content-Location`, `Retry-After`; + an earlier draft miscited `Content-Language`, which is list-valued), + or rejected; **unknown extension fields are rejected entirely in v1** + (neither append nor replace — their side-effect class is unknowable) (`Set-Cookie` is fully reserved in v1 — neither append nor replace). Replacing a header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). @@ -163,8 +172,11 @@ mutators to the outbound response for HTML document responses it processed. values are withheld). Operations arrive as **attributed batches bound to a registration ID** — one batch per integration per response, ordered by - registration, with the security channel's batch (§4a) ordered before - response mutators; the current flat effects vector satisfies neither + registration, with the security channel's batch (§4a) ordered **after** + ordinary response mutators — one global order, core finalization → + ordinary mutators → security effects → invariant pass — so the + security layer's precedence over publisher-facing mutations holds + without a second ordering claim; the current flat effects vector satisfies neither attribution nor budgets and is restructured accordingly. Validation and budgeting are **atomic per batch**: a batch that exceeds its budget is rejected whole (logged, attributed), never partially @@ -184,17 +196,17 @@ mutators to the outbound response for HTML document responses it processed. Which responses the hook runs on, enumerated so two implementations cannot diverge silently: -| Response | Hook runs? | -| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Processed HTML document (rewritten by TS) | Yes | -| Streamed processed document | Yes — operations apply to the header block before first byte | -| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | -| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | -| Redirect (3xx) | No | -| Error responses TS itself generates (4xx/5xx) | No | -| `304 Not Modified` | No | -| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | -| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | +| Response | Hook runs? | +| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes — operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | +| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` for a processed representation | **304-safe metadata pass**: the hook's header mutations for the corresponding processed 200 are re-applied (a 304 updates stored `Cache-Control`/`Vary` — excluding it while running on HEAD contradicted the cache-metadata rationale); where mutations cannot be reproduced, respond 200 instead | +| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | +| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to processed documents (§6). @@ -204,43 +216,59 @@ processed documents (§6). The security channel (today: DataDome) is not a general exception; every degree of freedom is closed: -- **Typed security-cookie operation, not header strings.** The channel - emits cookies only through a typed operation whose cookie **names come - from the integration's registered ownership list** (for DataDome, its - documented cookie); every `ts-*` name is rejected; domain/path scope, - attributes, size, and lifetime are constrained by the registration. - Read/vendor-egress/withdrawal semantics of the resulting identifier - are a ratified security-purpose carve-out — **sign-off item 23** — - because the tag-injection → cookie/ClientID read → vendor-send - lifecycle otherwise hands a permission-denied visitor a stable, - exported identifier. No other request filter inherits the cookie - capability. -- **Request-header pointers are direction-scoped allowlists.** Values a - security response names for copying into the request (DataDome's - header-pointer mechanism) are accepted only from a **documented - enrichment-header allowlist**; authentication, `Cookie`, - `Forwarded`/`X-Forwarded-*`, identity, consent, and routing-authority - fields are rejected by name and by class — a compromised endpoint must - not replace origin credentials, inject `ts-ec`, or spoof client - location — and accepted values apply to a **narrowly scoped upstream - overlay**, never the shared request that later integrations read. +- **Typed security-cookie operation with a concrete lifecycle, not + header strings.** The channel emits cookies only through a typed + operation, and the registration is not a placeholder — for DataDome + it pins: cookie name exactly `datadome`; scope the publisher apex, + path `/`; mandatory `Secure` and `SameSite=Lax`; lifetime at most + DataDome's documented maximum (thirteen months ceiling); size ≤ 4 KiB; + a violating operation is rejected whole (the batch rule). Every + `ts-*` name is rejected. **Read is owner-only** — the cookie is + visible to the security channel and stripped from every other + integration's request view; vendor egress goes only to DataDome + endpoints; deletion is always possible; and whether TS's own + destructive withdrawal also expires it is exactly the open half of + **sign-off item 23** — the carve-out is _pending ratification_, not + ratified, and the permission inventory's cookie deferral stands until + it closes. No other request filter inherits the cookie capability. +- **Request-header pointers are a positive, enumerated allowlist.** + "Documented enrichment headers" is not enforceable; the registration + enumerates the exact names — for DataDome today that is + **`X-DataDome-ClientID` and the documented `X-DataDome-*` enrichment + set, listed one by one** — resolving what was a contradiction: + ClientID propagation is required by the existing DataDome contract + and test, and its identity-class nature is precisely why it applies + only to an **owner-scoped publisher-upstream overlay**, never the + shared request that later integrations read, with its vendor egress + ratified under sign-off 23. Everything else — authentication, + `Cookie`, `Forwarded`/`X-Forwarded-*`, other identity, consent, and + routing-authority fields — is rejected by name and by class: a + compromised endpoint must not replace origin credentials, inject + `ts-ec`, or spoof client location. - **Representation rules are decision-scoped.** A _Respond_ decision (challenge/deny) **owns its body** and may set representation headers (`Content-Type`, encoding, validators) for it — the hook's representation reservation exists because ordinary mutators do not own the body, and this one does. A _Continue_ decision may not touch representation metadata of publisher bytes. -- **One global order, no "wins" exception:** core finalization → - hook/security effects → **final cache/privacy invariant pass, - unconditionally last**. The prior DataDome contract's "applies last - and wins" holds only _within_ the effects layer; nothing outranks the - invariant pass, or a challenge could combine `Set-Cookie` with public - caching. +- **One global order:** core finalization → ordinary mutators → + security effects → **final cache/privacy invariant pass, + unconditionally last**. Security precedence over publisher-facing + mutations comes from its position, not a "wins" rule; nothing outranks + the invariant pass, or a challenge could combine `Set-Cookie` with + public caching. The older DataDome spec's "applies last, after + finalization" wording is **superseded by this order** — updating that + document is a done-when item, since as written it would place DataDome + after the invariant pass and reopen the public-cache-plus-cookie bug. - The channel adopts the shared layers: structured attributed batches (§3, atomic per batch — a 302 must never lose `Location` to a budget - while keeping its cookie; on rejection the channel follows DataDome's - specified fail-open), reserved header names, budgets, and the - invariant pass. + while keeping its cookie), reserved header names, budgets, and the + invariant pass — with one sequencing rule fail-open depends on: the + complete challenge batch is **validated and budgeted before the + Respond decision commits**, so a rejection converts to Continue while + the publisher route is still available; discovering the rejection + after Respond has short-circuited routing would leave nothing to fail + open _to_. ## 4. Done-when (from #782, sharpened) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 360a3f2c1..f049b38c6 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -332,13 +332,13 @@ defined over those states, so no input state is unmapped. For each enforced permission, with baseline _B_ ∈ {granted, requires_signal, denied}: -| Opt-out present | TCF refusal present | Accepted grant present (regime-scoped) | Malformed-present | Result | -| --------------- | ------------------- | -------------------------------------- | ----------------- | ------------------------------------------------ | -| yes | — | — | — | **unset** (and withdrawal semantics apply, §4.2) | -| no | yes | — | — | unset (withdrawal per §4.2, trigger 2) | -| no | no | yes | — | set, unless B = denied | -| no | no | no | yes | **unset** (precedence 5 — blocks baseline grant) | -| no | no | no | no | set iff B = granted | +| Opt-out present | TCF refusal present | Accepted grant present (regime-scoped) | Malformed-present | Result | +| --------------- | ------------------- | -------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| yes | — | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | — | — | unset (withdrawal per §4.2, trigger 2) | +| no | no | yes | — | set, unless B = denied | +| no | no | no | yes | **unset** (precedence 5 — blocks baseline grant) | +| no | no | no | no | set iff B = granted **and no suppression entry stands** — an active suppression (§4.3) beats the baseline, so one malformed request denies later no-signal requests under `granted` until newer valid grant evidence clears it; a policy-baseline grant alone does **not** clear non-user suppression (sign-off 24 covers this consequence) | ### 4.2 Withdrawal vs. absence @@ -451,7 +451,12 @@ and the fail-closed marker: entry's; ties resolve to the more restrictive state. So a delayed grant with `LastUpdated = 100` never clears a suppression whose refusal carried `200`, while a genuine re-consent at `300` does. The - transition table, by stored cause: **opt-out from a timestamp-less + transition table (causes without an intrinsic timestamp — malformed + records decode no `LastUpdated`, absence has no source — use their + **observation timestamp**, server receipt on the shared clock basis + within the skew window; cross-source comparison uses the authoritative + timestamp where one exists, else the observation timestamp, ties + restrictive), by stored cause: **opt-out from a timestamp-less source** — cleared only by a grant with an authoritative timestamp newer than the suppression's observation (sticky opt-out, sign-off 16); **TCF refusal** — cleared by any regime-accepted grant with newer @@ -466,22 +471,45 @@ and the fail-closed marker: malformed beyond the skew window; within it, the record's digest is stored with its **first normalized timestamp, which re-presentation never advances** — otherwise a future-dated TCF string replayed after - an opt-out would keep re-normalizing to "now" and clear it. Equality - digests are computed over the **canonical per-permission semantic - result** of §4.5 aggregation, not the raw encoding — two encodings - (or `N` vs explicit N/A) with the same meaning are the same evidence - and keep the original first-seen, so alternating equivalent values - cannot renew authority. - - **Boundary, retention, ordering.** Suppression is checked by **every - `AuthorizedIdentity` constructor** (both scopes), by pull sync's live - path, and by every S2S recompute — not only "partner egress" prose; a - suppression read failure **fails closed** like a revocation read - failure; retention must outlive the positive authority it masks - (providers spec durability/retention capability); and **clearing is - fenced on provenance visibility**: a clear entry records the - provenance generation it reflects, and S2S honors the clear only when - it can read that generation or newer — clearing first would expose the + an opt-out would keep re-normalizing to "now" and clear it. Equality is + **source-specific**: for GPP/USP the digest is the **canonical + per-permission semantic result** of §4.5 aggregation alone — two + encodings (or `N` vs explicit N/A) with the same meaning are the same + evidence and keep the original first-seen, so alternating equivalent + values cannot renew authority; for TCF the digest is the semantic + result **plus the authoritative `LastUpdated`** — a genuine CMP + renewal with unchanged purposes carries a newer `LastUpdated` and + legitimately refreshes authority age, which a semantics-only digest + would wrongly ignore. + + **Boundary, retention, ordering — without deadlocking recovery.** + Suppression is checked by **every `AuthorizedIdentity` constructor** + (both scopes), by pull sync's live path, and by every S2S recompute. + That gate plus clear-after-provenance would deadlock re-consent — + fresh P1 provenance cannot be written while P1 suppression blocks + `GraphOps`, and clearing first is forbidden — so recovery has its own + narrow write path: **`AuthorityRefresh`**, permission-exempt but + strictly scoped to committing provenance from the _current live + resolution_ (nothing else: no partner writes, no egress, no reads + beyond the row being refreshed) while suppression remains effective; + the clear then references that provenance's revision. Revisions are an + **application-level monotonic counter written with the row** — never + backend generation markers, which (per Fastly's own contract) only + detect change and carry no order — and S2S honors a clear only when it + can read that revision or newer; clearing first would expose the + _older_ positive snapshot through an eventual read. + + **The strong record carries positive-authority state too.** The + per-family record doubles as the **authority-state record**: alongside + negative entries it stores a per-permission positive-authority summary + (revision, evidence timestamp), CAS-updated by every provenance write. + The **absence decision reads this strong summary, never the eventual + identity row** — deciding "no prior authority" from an eventual + not-found loses the race where a just-committed grant is invisible on + a stale replica. A suppression/authority read failure **fails closed** + like a revocation read failure; retention must outlive the positive + authority it masks (providers spec durability/retention capability). + _older_ positive snapshot through an eventual read. **Write failure fails closed for the live request**, and the S2S residual is unbounded for a never-returning visitor (sign-off 11), with fault tests for @@ -596,9 +624,13 @@ fields grant nothing (their opt-outs still count, per step 2). opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. The implementation PR cross-checks this list against both the current decoder's section set and the official registry, and the accepted version per - section is **pinned normatively to the named IAB GPP registry - revision current at this spec's date (2026-08-01)** — "enumerated by - the implementation PR" was two-implementations-diverge territory; a + section is **pinned to a registry snapshot vendored into this + repository** — a checked-in file enumerating, per mapped section, the + accepted version(s), taken from the IAB registry at ratification (a + date is not an immutable identifier, and "enumerated by the + implementation PR" was two-implementations-diverge territory; the + vendored file is the single reproducible authority, and updating it + is a reviewed spec change); a mapped section carrying a version outside the pinned revision is treated as malformed-present (blocks grants, never withdraws — §4.4), not as absent. Adding a section or version is a @@ -774,6 +806,8 @@ Consumers of the resolved set in this epic: | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | | Integration persistent response cookies | `store-on-device` (+ P4 where the cookie is an advertising identifier) | **Deferred with the hook's cookie surface** — the write-side gate alone was insufficient (read/use/forward/withdrawal unmodeled), so cookie operations ship only with the full model; this row and the client-cycle **page leg** (module injection gated on the provider's full declaration) join the inventory when their features do, and the §5.3 no-geo guard's consumer list grows with them | | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | + | Authority-state / suppression-decision read (§4.3) | **exempt**, narrowly scoped | Returns only family ID, per-permission authority summary, and suppression entries — no identity values, no partner data; a test proves nothing else escapes | + | `AuthorityRefresh` provenance write (§4.3) | **exempt**, strictly scoped | Commits current-live-resolution provenance only; enables suppression recovery without reopening `GraphOps` | With **no EC provider configured**, identity use fails closed: a cookie value present on the request never egresses anywhere — never vacuously @@ -786,7 +820,9 @@ Consumers of the resolved set in this epic: written at mint and replaced on later live requests — grant basis (which signal class granted, per permission), the evidence's **authoritative timestamp and `valid_until`** (per evidence class), - resolved jurisdiction, policy revision, and provider/version (providers + resolved jurisdiction, and policy revision — **not** provider/version, + which lives only in the immutable mint tag, or a post-rotation visit + would restamp a v1 identity as v2 (providers spec §6.1). Freshness is a **per-evidence-class contract**, because not every source carries a timestamp: diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 5302a7251..f8ebc0ddf 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -205,7 +205,8 @@ pub trait EdgeCookieProvider { /// Canonical graph-key SUFFIX (bounded length, KV-safe). Core — not /// the provider — constructs the physical key (§6.3 key grammar), so /// cross-provider and cross-record-kind isolation is structural. - /// Sole exception: hmac v0 keys are the identifier verbatim. + /// Sole exception: hmac keys (every version) are the identifier + /// verbatim — the reserved legacy grammar. fn graph_key_suffix(&self, id: &EcId) -> GraphKeySuffix; /// Cluster capability: a literal byte prefix of the physical graph /// key, shared across identifiers minted from the same client @@ -213,10 +214,12 @@ pub trait EdgeCookieProvider { fn cluster_prefix(&self, id: &EcId) -> Option; /// Mint an identifier from request evidence. The one acquisition /// operation of the epic (server mint); failure means no identity - /// this request (§6.2). Lost from an earlier revision by editing - /// accident — its absence made the required gate → generate → - /// graph-commit sequence unimplementable. - fn generate(&self, input: &IdentityInput<'_>) -> Result>; + /// this request (§6.2). Returns the identifier WITH the active + /// configuration version — the immutable mint tag needs it and core + /// cannot reach into provider-specific configuration to learn it. + fn generate(&self, input: &IdentityInput<'_>) + -> Result>; + // GeneratedIdentity { id: EcId, mint_version: ProviderVersion } /// Cryptographic verification of a parsed identifier against request /// evidence. Recognition (`parse`) is not authentication; rowless /// handling (§5) requires this. Returns the matched configuration @@ -272,23 +275,34 @@ present `H.aaaaab`, `H.aaaaac`, … — every variant prefix-verifies, and an adopt path would mint a **separate durable row and family per variant**. Therefore: -- A recognized rowless cookie whose prefix verifies - (`verify → VerifiedIdentity`, carrying the matched version for - provenance) is **expired and replaced by a fresh mint through the - ordinary graph-backed path** (gate → `generate` → commit) when the - request's permissions allow one; continuity with the old identifier is - deliberately not preserved (migration matrix row 13, sign-off 21). A - cookie whose prefix does not verify (including the declared roaming - false-negative) is simply expired. -- **Rowless withdrawal cannot be a row/family-minting oracle**: for - rowless legacy HMAC cookies the derived family ID is a function of the - **authenticated 64-hex prefix only**, so every suffix variant maps to - the _same_ family — one family record withdraws them all, and - attacker-generated variants create nothing new. -- Read errors are still not "not found": a failed graph read means the - cookie is treated as absent this request, fail closed, no expiry - emitted (the row may exist). - declares this path. +- **"Rowless" requires an authoritative not-found, and only in + migration mode.** Identity-row visibility may be eventual, so a plain + not-found proves nothing — a just-minted row invisible on a stale + replica would classify its own cookie as rowless and expire/re-mint + it, forking the identity. The rowless path therefore activates only + when the deployment-metadata **graphless-migration flag** is set (set + by the §4.2 readiness step for deployments that actually ran + graphless; permanently-graphed deployments never classify anything + rowless), and the existence check uses the backend's strongest read. + Outside migration mode, or on any read error, the state is + **indeterminate**: no identity use, no mint, no cookie expiry — + "treated as absent" was the wrong contract, since absence feeds the + fresh-mint path. +- A verified rowless cookie (`verify → VerifiedIdentity`, carrying the + matched version) is **expired and replaced by a fresh mint through the + ordinary graph-backed path** when permissions allow; continuity is + deliberately not preserved (migration matrix row 13, sign-off 21). An + unverifiable cookie (including the declared roaming false-negative) is + simply expired. +- **Rowless withdrawal writes nothing** — there is no server-side state + to revoke: no row, no partner mappings, no S2S surface. The cookie is + expired, and that is the entire withdrawal. (An earlier prefix-derived + family record was over-engineering with two defects: unauthenticated + suffix variants could mint records, and — because the HMAC prefix is + per-IP — one visitor's withdrawal would have revoked every identity + behind the same IP. Family records exist only for row-backed + identities, derived from the full graph key, one derivation + everywhere.) **Egress is typed, not policed.** The inventory-and-denylist test (permission model spec §7) is a backstop, but conventions do not survive @@ -541,9 +555,15 @@ deadline, and fencing epoch; the **family revocation record** holds the family ID, revoked-at, triggering signal class (§4.5 destructive column), and a **family epoch** bumped on every revocation-state change (the client-cycle commit CAS is conditioned on it) — deliberately no identity -data, so it can outlive its members; the **suppression record** holds -per-permission suppression entries with timestamps (strong class, -permission-exempt writes, permission model spec §4.3); +data, so it can outlive its members; the **authority-state (suppression) record** holds, per permission: +state (`suppressed`/`cleared`), cause, source class, authoritative or +observation evidence timestamp, the **application-level provenance +revision** a clear references, and the positive-authority summary +(revision + evidence timestamp) — plus the record-level CAS version +counter and schema version; unknown-field and range validation apply +like every class (strong class, permission-exempt writes per the +permission spec's inventory; revisions are app-level counters because +backend generation markers detect change without ordering); the **rewrite transaction** holds source key, target key, copy point, state, and epoch; the **reservation** holds state, owner hash, lease epoch, outcome, and created-at (client-cycle spec). Field validation and @@ -611,7 +631,7 @@ Requirements: | Deployment metadata (schema floor) | **Write-once/CAS**, outside ordinary config storage (migration spec §4) | | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | - | Identity rows | Eventual consistency acceptable — rows are accretive, and the family-record check (strong, per above) governs use | + | Identity rows | Eventual **visibility** acceptable _after_ a generation-CAS mutation commits (see Identity-row mutation above) — the earlier "rows are accretive" claim is deleted: rows replace snapshots, merge partner IDs, and refresh derived state, and unordered last-writer-wins loses newer evidence | Every record class additionally declares **durability and maximum retention**: a store passing the consistency check but capping TTLs @@ -632,14 +652,19 @@ Requirements: them is how a "yes" cell hides an unusable feature. Feature eligibility requires wired, not merely available: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | ----------------------------------------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | - | Graph persistence (eventual OK) | KV Store: available + wired | **Not wired** — the current adapter installs `UnavailableKvStore`; an in-process store is dev-feasible but does not exist yet | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | - | Prefix listing (cluster) | Yes (used today) | Yes | Yes (eventual) | _verify_ | - | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | Yes (in-process) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | - | Linearizable fenced CAS (reservations, alias/rewrite) | **Not currently available** — the client-cycle feature (deferred) would need it | Yes — in-process only: linearizable but **non-durable**, dev-eligibility only, not a production persistence claim | Durable Objects: possible, not wired | **No** | - | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | - | Device host evidence (JA4/H2) | Yes | No | No | No | + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | + | Graph persistence (eventual OK) | KV Store: available + wired | **Not wired** — the current adapter installs `UnavailableKvStore`; an in-process store is dev-feasible but does not exist yet | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | + | Prefix listing (cluster) | Yes (used today) | **Unavailable** (no store wired — in-process feasibility is a note, not a cell) | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | **Unavailable** (no store wired) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS _(informative — deferred features)_ | **Not currently available** | **Unavailable** (no store wired) | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | + | Suppression / authority-state CAS | Generation-marker conditional write: available, **wiring to verify** | **Unavailable** (no store wired) | Workers KV: **ineligible** (last-write-wins); Durable Objects: feasible, not wired | **Unavailable** | + | Identity-row generation-CAS mutation | Same primitive as above | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | + | Row create-if-absent | Generation-marker create: available, **wiring to verify** | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | + | Deployment metadata (write-once/CAS) | **Not wired** — needs a primitive distinct from the config store | **Unavailable** | DO: feasible, not wired | **Unavailable** | + | Durability / max-retention proof | KV durable; TTL ceilings **to verify** against computed horizons | **Unavailable** | Workers KV TTLs: to verify; DO storage: feasible | **Unavailable** | - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 82fe5779b..eb0ffe8b5 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -367,8 +367,8 @@ global honoring of opt-out signals is unconditional. spec §5.2), raw-egress denials by path, tombstone family retries, legacy-reader hit rate, and cluster-fallback engagements. Two of these carry thresholds, not just ranges: legacy-reader hits at zero for a **quiet period no shorter than the - maximum cookie/row lifetime plus rollout skew** — or provable - rewrite/backfill completion — is the **retirement-readiness** bar for a + maximum cookie/row lifetime plus rollout skew** is the **only + retirement-readiness** bar for a legacy provider ("trending to ~zero" is not evidence; a yearly visitor is not churn), (rewrite-based backfill and its metrics left with the rewrite deferral). The telemetry set also includes: graph read/commit failures, @@ -435,32 +435,36 @@ global honoring of opt-out signals is unconditional. These are decisions this spec set makes that #838 had not already made (or made differently). **Implementation is blocked while any row is `open`**; -each row needs an owner, a status, and a link to its decision record — -an unratified row reverts to open, not to silently implemented. - -| # | Decision | Where | Owner | Status | -| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | -| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | -| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | -| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | -| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | -| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | -| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | -| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | -| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | -| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | -| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | -| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | -| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | -| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | -| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | -| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | -| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | -| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | -| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | -| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | -| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | -| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | -| 23 | DataDome security exemption: tag injection, cookie/ClientID read, vendor egress, and cross-integration visibility operate outside the permission model as a ratified security-purpose carve-out with owned cookie names, scope, and withdrawal semantics | hook §4a; permission §7 | maintainers + legal | open | -| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky); opt-out stickiness applies only to opt-out causes | permission §4.3 | maintainers + legal | open | -| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | +each row needs an owner, a status, and a link to its decision record. +Decision records live as files under +`docs/superpowers/specs/decisions/` (one per row, `NN-title.md`, +recording the decision, the deciders, and the date) — the table links +them as rows close; an unratified row reverts to open, not to silently +implemented. + +| # | Decision | Where | Owner | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | +| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | +| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | +| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | +| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | +| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | +| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | +| 23 | DataDome security exemption: tag injection, cookie/ClientID read, vendor egress, and cross-integration visibility operate outside the permission model as a ratified security-purpose carve-out with owned cookie names, scope, and withdrawal semantics | hook §4a; permission §7 | maintainers + legal | open | +| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky; opt-out stickiness applies only to opt-out causes) — and an active suppression **overrides a `granted` baseline**: one malformed request denies later no-signal requests until valid evidence clears it, and a policy-baseline grant alone never clears | permission §4.3, §4.1 | maintainers + legal | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/decisions/README.md b/docs/superpowers/specs/decisions/README.md new file mode 100644 index 000000000..fd3d0e1e2 --- /dev/null +++ b/docs/superpowers/specs/decisions/README.md @@ -0,0 +1,6 @@ +# PR #986 product-decision records + +One file per open row of the migration spec §8 sign-off table +(`NN-title.md`), recording the decision, the deciders, and the date. The +table links each record as its row closes; a row without a record here is +open, and implementation is blocked while any row is open. diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md index 85a35e965..e58443d90 100644 --- a/docs/superpowers/specs/pr986-review-ledger.md +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -167,27 +167,46 @@ R9 (10 P1, 13 P2, 1 P3) and the same-head re-audit R10 (9 P1, 9 P2, 1 P3) are dispositioned together; R10's "previously open" list = R9's P1s, tracked once. -| Finding | Status | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | -| R9-1 trait cannot mint (`generate` lost in an R8 edit) | fixed — restored with failure semantics | -| R9-2 / R10-open rowless HMAC unauthenticatable suffixes | fixed — expire-and-re-mint, prefix-derived family, `VerifiedIdentity{version}` | -| R9-3 missing capability rows (suppression CAS, create-if-absent, floor metadata) + global visibility | fixed — distinct rows with per-adapter values (Fastly generation markers; Workers KV ineligible), globally observable revocation reads | -| R9-4 CAS orders arrival not recency | fixed — evidence-recency transition table with per-cause clearing | -| R9-5 creation condition fails both directions | fixed — cause-aware read-free creation; policy-only tightening writes nothing; absence uses the exempt decision read | -| R9-6 suppression outside the authorization boundary | fixed — both constructors, fail-closed reads, retention rule, provenance-generation-fenced clearing | -| R9-7 N+1 new-shape S2S unsafe | fixed — context-free partner egress fails closed on provenance-less rows once new-shape config is active | -| R9-8 schema-floor protocol unspecified | fixed — dedicated primitive, create-or-CAS, read-back-then-enable, startup enforcement, fail-closed | -| R9-9 security-cookie exception reopens cookie surface | fixed — §4a typed owned-name operation, ts-\* rejected, sign-off 23 | -| R9-10 DataDome representation/ordering conflicts | fixed — decision-scoped representation; one global order, invariant last | -| R9-P2 batch (residual wording; deferred residue markers; registry + non-hex tags; cluster cap; retirement evidence; matrix/fixture branching; cutoff removed with adoption; irreversible artifacts enumerated; mixed-policy absolute removed; version pin; effect attribution; Set-Cookie remnants; network split) | all fixed | -| R9-P3 / R10-P2.8 eligibility rows (HEAD, 1xx, 204, 205, 206) | fixed — HEAD mirrors GET; others enumerated No | -| R10-1 future-dated TCF replay | fixed — beyond-skew rejected as malformed; digest-pinned first normalization | -| R10-2 mutable rows vs eventual/accretive claim | fixed — generation-CAS mutation, eventual visibility only | -| R10-3 cluster refresh extends retention | fixed — absolute `expires_at`, remaining-lifetime writes | -| R10-4 mint version inside replaceable snapshot | fixed — immutable mint tag split from evidence | -| R10-5 suppression needs forbidden read | fixed — read-free signal causes; narrow exempt decision read for absence | -| R10-6 durability/retention not validated | fixed — per-class durability + max-retention capabilities, startup horizon proof | -| R10-7 CDN-header weakening | fixed — CDN cache fields reserved outright; stale-\* durations shrink-only | -| R10-8 DataDome request-header injection | fixed — direction-scoped allowlist, scoped upstream overlay | -| R10-9 DataDome identifier outside permission model | ratification — sign-off 23 | -| R10-P2 batch (semantic digests; malformed/absence clearing + sign-off 24; atomic security batches; field registry; 256-byte bound in normative spec; device sign-off 22; provider-vs-schema rollback; TOCTOU live-check) | all fixed | +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| R9-1 trait cannot mint (`generate` lost in an R8 edit) | fixed — restored with failure semantics | +| R9-2 / R10-open rowless HMAC unauthenticatable suffixes | fixed — expire-and-re-mint, prefix-derived family, `VerifiedIdentity{version}` | +| R9-3 missing capability rows (suppression CAS, create-if-absent, floor metadata) + global visibility | R9/R10 partial (abstract rows only — the concrete matrix had no cells) → **refixed R11**: concrete per-adapter cells for all five primitives | +| R9-4 CAS orders arrival not recency | fixed — evidence-recency transition table with per-cause clearing | +| R9-5 creation condition fails both directions | fixed — cause-aware read-free creation; policy-only tightening writes nothing; absence uses the exempt decision read | +| R9-6 suppression outside the authorization boundary | fixed — both constructors, fail-closed reads, retention rule, provenance-generation-fenced clearing | +| R9-7 N+1 new-shape S2S unsafe | fixed — context-free partner egress fails closed on provenance-less rows once new-shape config is active | +| R9-8 schema-floor protocol unspecified | fixed — dedicated primitive, create-or-CAS, read-back-then-enable, startup enforcement, fail-closed | +| R9-9 security-cookie exception reopens cookie surface | fixed — §4a typed owned-name operation, ts-\* rejected, sign-off 23 | +| R9-10 DataDome representation/ordering conflicts | fixed — decision-scoped representation; one global order, invariant last | +| R9-P2 batch (residual wording; deferred residue markers; registry + non-hex tags; cluster cap; retirement evidence; matrix/fixture branching; cutoff removed with adoption; irreversible artifacts enumerated; mixed-policy absolute removed; version pin; effect attribution; Set-Cookie remnants; network split) | all fixed | +| R9-P3 / R10-P2.8 eligibility rows (HEAD, 1xx, 204, 205, 206) | fixed — HEAD mirrors GET; others enumerated No | +| R10-1 future-dated TCF replay | fixed — beyond-skew rejected as malformed; digest-pinned first normalization | +| R10-2 mutable rows vs eventual/accretive claim | R9/R10 partial (accretive claim survived in the matrix) → **refixed R11** (claim deleted; eventual visibility only after generation-CAS) | +| R10-3 cluster refresh extends retention | fixed — absolute `expires_at`, remaining-lifetime writes | +| R10-4 mint version inside replaceable snapshot | R9/R10 partial (mutable snapshot still listed provider/version) → **refixed R11** (removed from snapshot and from permission §7's field list) | +| R10-5 suppression needs forbidden read | fixed — read-free signal causes; narrow exempt decision read for absence | +| R10-6 durability/retention not validated | fixed — per-class durability + max-retention capabilities, startup horizon proof | +| R10-7 CDN-header weakening | fixed — CDN cache fields reserved outright; stale-\* durations shrink-only | +| R10-8 DataDome request-header injection | fixed — direction-scoped allowlist, scoped upstream overlay | +| R10-9 DataDome identifier outside permission model | ratification — sign-off 23 | +| R10-P2 batch (semantic digests; malformed/absence clearing + sign-off 24; atomic security batches; field registry; 256-byte bound in normative spec; device sign-off 22; provider-vs-schema rollback; TOCTOU live-check) | all fixed | + +## Round 11 — review at 43422b5f + +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| P1 suppression recovery deadlock | fixed — `AuthorityRefresh` scoped write path; clear references app-level provenance revision | +| P1 eventual not-found → rowless misclassification | fixed — rowless only under the graphless-migration deployment flag with strongest-read existence check; otherwise indeterminate (no use, no mint, no expiry); rowless withdrawal writes nothing (prefix-family removed — it also would have revoked whole IPs) | +| P1 absence suppression misses fresh grants | fixed — authority-state record carries a strong positive-authority summary; absence decision never reads the eventual row | +| P1 concrete matrix omits mandatory capabilities | fixed — see corrected R9-3 row above | +| P1 mutable vs accretive | fixed — see corrected R10-2 row above | +| P1 suppression wire schema + generation ordering | fixed — full field schema; app-level monotonic revisions (Fastly markers detect change, no order) | +| P1 `generate` cannot carry mint version | fixed — `GeneratedIdentity { id, mint_version }` | +| P1 ClientID required and prohibited | fixed — positively enumerated allowlist including `X-DataDome-ClientID`, owner-scoped upstream overlay, egress under sign-off 23 | +| P1 cookie exception without lifecycle | fixed — concrete registration (name `datadome`, scope, attributes, 13-month ceiling, 4 KiB, owner-only read, deletion always); "ratified" corrected to _pending ratification_ | +| P1 ordering contradiction | fixed — one order: core → mutators → security → invariant; older DataDome doc marked superseded, update in done-when | +| P1 headers-only not permission-neutral | fixed — v1 field registry admits inert fields only; active-egress fields rejected; unknown fields fully rejected | +| P2 batch (inventory rows for the exempt read and AuthorityRefresh; TCF-vs-GPP source-specific digests; observation timestamps; granted-baseline suppression row in §4.1 + sign-off 24 expanded; snapshot field cleanup; indeterminate read errors; Axum cells unavailable; rewrite/backfill retirement alternative removed; GPP vendored snapshot; 304-safe metadata pass; Respond-first validation; CDN names enumerated; deferred residue bracketed) | all fixed | +| P3 batch (verbatim comment covers every hmac version; `Content-Language` example corrected; stale fragments swept; this ledger corrected) | fixed | +| Ratification note | decisions directory created (`docs/superpowers/specs/decisions/`), table text points to it; item 23 wording no longer claims ratification | From ba25ba85c2b97877e2aa423b44ad7e8c443da10f Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sun, 2 Aug 2026 15:24:12 -0700 Subject: [PATCH 13/14] Address twelfth review: two-record commit protocol, strong-class rowless proof, and bounded DataDome surfaces P1 fixes: - The two-record authority protocol has an explicit commit point: the identity row commits at revision r (generation-CAS), then the authority-state record CAS-updates to r, and r is usable - by S2S, by the absence decision, for egress eligibility - only when the strong record reports it. A crash between the writes is a recoverable intermediate (AuthorityRefresh re-runs step 2), never divergence. Mint eligibility begins at the authority-state commit, not the row commit. - Authority-state requires globally observable strong reads AND linearizable CAS - a stale successful read would authorize egress after a committed suppression, and read-failure-fails-closed does not cover stale successes; matrix and Fastly cells updated (read semantics to verify). - Rowless classification is proven from the strong class: every post-upgrade identity has an authority-state record under its derivable family ID, so rowless = graphless-migration flag AND no such record on a strong read - no eventual storage participates. The flag has a defined wire key, setter, capability requirement, rollback survival, and explicit operator clearing. - Rowless withdrawal is one contract everywhere: an exact-cookie family record (full-graph-key derivation - no per-IP blast) written by prefix-verified cookies only (attackers can spend withdrawal only on their own prefix), then cookie expiry - aligned with the family-record-first rule and migration row 13; the prefix mechanism is gone from every document, and cookie-only best-effort withdrawal (lost response = live cookie) is rejected. - N+1 neither creates nor clears authority-state records: clearing requires the AuthorityRefresh fence over revision-bearing rows a v1 writer cannot produce. N+1 reads fully and fails closed; suppression persists through rollback and recovery waits for roll-forward - a declared protective limitation. N+1 still writes family revocations. - The graph-row table gains the provenance-revision field (init 1, monotonic u64, overflow is an error, CAS'd with row generation) and loses the provider/version leftover from mutable provenance. - The positive-authority summary carries kind (user evidence vs policy baseline), grant basis/source class, policy revision, and valid_until - the absence decision distinguishes vanished user evidence from policy-only change without touching the eventual row. - The DataDome request-header allowlist is a checked-in file (datadome-header-allowlist.md) pinned to X-DataDome-ClientID alone; the cookie strip inventory is exhaustive (origin forwarding, proxy/click/Testlight upstreams, auction serialization, logs - each a tested row), not just integration views. - The 304 pass re-emits the persisted final post-hook header set stored with the cached representation; absent metadata means cache miss - the where-does-the-200-come-from gap is closed. P2/P3: gpp-registry-snapshot.md vendored (sections 6-27, versions, ratification re-verification note); the v1 field registry is enumerated in-spec; the datadome cookie pins PSL-computed registrable Domain and Max-Age <= 34,214,400 s; the clock-skew window is a normative 300 s constant; sign-off 23 is an open question enumerating observers, not 'ratified'; the permission spec repeats the globally-observable revocation wording verbatim instead of paraphrasing; rewrite/reservation wire schemas are bracketed informative and rewrite links leave the migration expansion; the dangling duplicate sentence, the reserved- cookie-name phrasing, and the negative-authority-only key-table description are gone; and the ledger adds Round 12 with a mechanical-anchor rule so closure claims are greppable rather than trusted. --- ...integration-response-header-hook-design.md | 64 ++++--- .../2026-07-30-permission-model-design.md | 49 +++++- .../2026-07-30-pluggable-providers-design.md | 159 ++++++++++-------- ...07-30-provider-migration-rollout-design.md | 120 +++++++------ .../specs/datadome-header-allowlist.md | 10 ++ .../specs/gpp-registry-snapshot.md | 35 ++++ docs/superpowers/specs/pr986-review-ledger.md | 30 ++++ 7 files changed, 309 insertions(+), 158 deletions(-) create mode 100644 docs/superpowers/specs/datadome-header-allowlist.md create mode 100644 docs/superpowers/specs/gpp-registry-snapshot.md diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 97dd5d484..3d5caa061 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -126,8 +126,8 @@ mutators to the outbound response for HTML document responses it processed. which corrupts attribution and budgets), with a duplicate-ID test in the done-when. Until then the operation set is headers-only, and `Set-Cookie` is fully reserved. - reserved cookie name — in v1 that is every cookie name, since - `Set-Cookie` is fully reserved (§3 deferral). Violations are rejected + cookie via any operation — `Set-Cookie` is fully reserved in v1 (§3 + deferral). Violations are rejected at the operation layer (§2) and logged at `warn` with the integration id. The reserved lists are single constants next to the definitions they protect, not duplicated in the @@ -145,7 +145,13 @@ mutators to the outbound response for HTML document responses it processed. replace-only (true singletons — e.g. `Content-Location`, `Retry-After`; an earlier draft miscited `Content-Language`, which is list-valued), or rejected; **unknown extension fields are rejected entirely in v1** - (neither append nor replace — their side-effect class is unknowable) (`Set-Cookie` is fully reserved in v1 — neither append nor replace). Replacing a + (neither append nor replace — their side-effect class is unknowable). + The v1 registry is enumerated here, not delegated: **admitted** — + `Cache-Control` (monotonic merge per this section), `Vary` (union + merge), `Content-Language` (append), `X-Robots-Tag` (append), + `Retry-After` (replace-only), `Content-Location` (replace-only); + everything else known is classified reserved or rejected by the rules + above, and growing the admitted set is a spec change to this list (`Set-Cookie` is fully reserved in v1 — neither append nor replace). Replacing a header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). @@ -196,17 +202,17 @@ mutators to the outbound response for HTML document responses it processed. Which responses the hook runs on, enumerated so two implementations cannot diverge silently: -| Response | Hook runs? | -| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Processed HTML document (rewritten by TS) | Yes | -| Streamed processed document | Yes — operations apply to the header block before first byte | -| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | -| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | -| Redirect (3xx) | No | -| Error responses TS itself generates (4xx/5xx) | No | -| `304 Not Modified` for a processed representation | **304-safe metadata pass**: the hook's header mutations for the corresponding processed 200 are re-applied (a 304 updates stored `Cache-Control`/`Vary` — excluding it while running on HEAD contradicted the cache-metadata rationale); where mutations cannot be reproduced, respond 200 instead | -| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | -| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | +| Response | Hook runs? | +| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes — operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | +| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: when the processed 200 was cached, its **final post-hook header set** (the cache-relevant fields) was persisted alongside the representation; a 304 re-emits those persisted finals — deterministic, no mutator re-run, no representation reconstruction. If no persisted metadata exists, the conditional request is treated as a **cache miss** (full 200 fetched and processed); the earlier "respond 200 instead" without saying whence was not implementable | +| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | +| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to processed documents (§6). @@ -219,13 +225,21 @@ degree of freedom is closed: - **Typed security-cookie operation with a concrete lifecycle, not header strings.** The channel emits cookies only through a typed operation, and the registration is not a placeholder — for DataDome - it pins: cookie name exactly `datadome`; scope the publisher apex, - path `/`; mandatory `Secure` and `SameSite=Lax`; lifetime at most - DataDome's documented maximum (thirteen months ceiling); size ≤ 4 KiB; + it pins: cookie name exactly `datadome`; `Domain` set to the + registrable domain computed against the **Mozilla Public Suffix List** + (vendored revision named by the implementation; host-only is not used + because DataDome requires site-wide scope), path `/`; mandatory + `Secure` and `SameSite=Lax`; `Max-Age` at most **34,214,400 seconds** + (396 days — the thirteen-month ceiling, as an exact number); size ≤ + 4 KiB; a violating operation is rejected whole (the batch rule). Every - `ts-*` name is rejected. **Read is owner-only** — the cookie is - visible to the security channel and stripped from every other - integration's request view; vendor egress goes only to DataDome + `ts-*` name is rejected. **Read is owner-only, and the strip inventory is exhaustive, not + integration-scoped** — the browser sends `datadome` in the ordinary + `Cookie` header, so it is removed from **every non-DataDome surface**: + other integrations' request views, publisher-origin proxy forwarding, + proxy/click/Testlight upstreams, auction/page-bids request + serialization, and logs (redaction list) — each surface a tested row + of the inventory; only the security channel itself observes it; vendor egress goes only to DataDome endpoints; deletion is always possible; and whether TS's own destructive withdrawal also expires it is exactly the open half of **sign-off item 23** — the carve-out is _pending ratification_, not @@ -233,9 +247,13 @@ degree of freedom is closed: it closes. No other request filter inherits the cookie capability. - **Request-header pointers are a positive, enumerated allowlist.** "Documented enrichment headers" is not enforceable; the registration - enumerates the exact names — for DataDome today that is - **`X-DataDome-ClientID` and the documented `X-DataDome-*` enrichment - set, listed one by one** — resolving what was a contradiction: + enumerates the exact names from the **checked-in allowlist file + `docs/superpowers/specs/datadome-header-allowlist.md`** — spec-pinned + today to exactly **`X-DataDome-ClientID`**; every other `X-DataDome-*` + field is rejected until a reviewed commit adds it to that file + ("documented enrichment set, listed one by one" without an actual list + was a wildcard whose contents could change outside the spec) — + resolving what was a contradiction: ClientID propagation is required by the existing DataDome contract and test, and its identity-class nature is precisely why it applies only to an **owner-scoped publisher-upstream overlay**, never the diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index f049b38c6..360df42ad 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -502,7 +502,15 @@ and the fail-closed marker: **The strong record carries positive-authority state too.** The per-family record doubles as the **authority-state record**: alongside negative entries it stores a per-permission positive-authority summary - (revision, evidence timestamp), CAS-updated by every provenance write. + — **kind** (user evidence vs. policy-baseline), grant basis / source + class, policy revision, `valid_until`, provenance revision, and + evidence timestamp — CAS-updated by every provenance write. The kind + and policy revision are load-bearing: the absence decision must + distinguish vanished _user_ evidence (suppress) from a + policy-baseline grant that disappeared because the _policy_ changed + (never suppress — trigger 3), and a revision-and-timestamp-only + summary would force exactly the eventual row read this record exists + to eliminate. The **absence decision reads this strong summary, never the eventual identity row** — deciding "no prior authority" from an eventual not-found loses the race where a just-committed grant is invisible on @@ -510,8 +518,23 @@ and the fail-closed marker: like a revocation read failure; retention must outlive the positive authority it masks (providers spec durability/retention capability). - _older_ positive snapshot through an eventual read. **Write failure - fails closed for the live request**, and the S2S residual is unbounded + **The strong record is the commit point — the two-record protocol is + explicit.** Every provenance-bearing write spans the eventual identity + row and the strong authority-state record, in a fixed order with + defined intermediate states: (1) the row commits at revision _r_ + (generation-CAS); (2) the authority-state record CAS-updates its + summary to _r_. **Revision _r_ is committed — usable by S2S, visible + to the absence decision — only when the strong record reports it**; a + row at _r_ whose summary still reads _r−1_ is simply uncommitted + detail, and a crash between the writes leaves a recoverable state (the + next live resolution re-runs step 2 via `AuthorityRefresh`), never a + divergent one. This ordering is why the absence decision can trust the + summary: there is no state in which the row authorizes something the + strong record has never heard of. Minting follows the same rule — + see the providers spec §5 order, where eligibility begins at the + **authority-state commit**, not the row commit. + + **Write failure fails closed for the live request**, and the S2S residual is unbounded for a never-returning visitor (sign-off 11), with fault tests for suppress-vs-clear races, repeated-value sequences, and the stale-provenance-read case. @@ -532,9 +555,12 @@ and the fail-closed marker: (migration spec §8), not a footnote. - **Consistency and retention are backend contracts with a single normative home**: the providers spec consistency matrix (§7). It — not - this spec — states the requirement, and it requires a **strongly - consistent (read-after-write) primitive** for revocation records; no - bounded-lag alternative exists (an earlier draft here permitted one, + this spec — states the requirement, and it requires **globally observable + strong consistency** for revocation records — every instance's read + observes a committed revocation, never merely the writer's own + session (this spec deliberately repeats the provider contract's exact + wording rather than paraphrasing it into the weaker "read-after-write"); + no bounded-lag alternative exists (an earlier draft here permitted one, which contradicted the matrix — an adapter with a two-second lag would have passed one spec and failed the other). A **failed family-record read fails closed** for egress (revoked-unknown ≠ live), and revocation @@ -624,8 +650,8 @@ fields grant nothing (their opt-outs still count, per step 2). opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. The implementation PR cross-checks this list against both the current decoder's section set and the official registry, and the accepted version per - section is **pinned to a registry snapshot vendored into this - repository** — a checked-in file enumerating, per mapped section, the + section is **pinned to the vendored registry snapshot + `docs/superpowers/specs/gpp-registry-snapshot.md`** — a checked-in file enumerating, per mapped section, the accepted version(s), taken from the IAB registry at ratification (a date is not an immutable identifier, and "enumerated by the implementation PR" was two-implementations-diverge territory; the @@ -832,7 +858,12 @@ Consumers of the resolved set in this epic: | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (a **per-permission equality digest computed over only the applicable, aggregated §4.5 fields for that permission** — never the whole GPP record, or a CMP touching an unrelated notice field would mint a new digest and reset first-seen forever) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | - Timestamps are compared with bounded clock-skew tolerance; + Timestamps are compared with a bounded clock-skew tolerance that is a + **normative constant — 300 seconds** (five minutes, applied + symmetrically; a spec-level value because suppression precedence, + malformed classification, and future-date handling all hinge on it, + and per-deployment values would give the same input different privacy + outcomes); beyond-window future-dated records are **rejected as malformed**, and within the window a record's first normalized timestamp is pinned to its digest and never advanced by re-presentation (§4.3's anti-replay diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index f8ebc0ddf..fbe362fcf 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -259,8 +259,13 @@ cookie write, no egress, no auction use may observe a minted identifier before its graph row (with provenance, §6.1) has committed — PR #838 let a generated EC reach an auction before finalization refused the cookie, producing an identity that existed for one request and nowhere else. The -normative order is: gate → `generate` → graph-row commit → cookie -scheduled → eligible for egress. "Cookie scheduled" means queued onto the +normative order is: gate → `generate` → graph-row commit → +**authority-state commit** (the strong record reporting the row's +revision — the commit point of the two-record protocol, permission model +spec §4.3) → cookie scheduled → eligible for egress. Eligibility begins +at the authority-state commit, not the row commit: a row whose revision +the strong record has not reported is uncommitted detail, which is what +keeps S2S authorization and the absence decision consistent. "Cookie scheduled" means queued onto the final response — `Set-Cookie` is physically emitted after first-request processing, so egress eligibility begins at **graph commit**, not at header emission; the identity exists durably from that moment. A @@ -275,34 +280,47 @@ present `H.aaaaab`, `H.aaaaac`, … — every variant prefix-verifies, and an adopt path would mint a **separate durable row and family per variant**. Therefore: -- **"Rowless" requires an authoritative not-found, and only in - migration mode.** Identity-row visibility may be eventual, so a plain +- **"Rowless" is proven from the strong class, never from eventual + storage.** Identity-row visibility may be eventual, so a plain not-found proves nothing — a just-minted row invisible on a stale - replica would classify its own cookie as rowless and expire/re-mint - it, forking the identity. The rowless path therefore activates only - when the deployment-metadata **graphless-migration flag** is set (set - by the §4.2 readiness step for deployments that actually ran - graphless; permanently-graphed deployments never classify anything - rowless), and the existence check uses the backend's strongest read. - Outside migration mode, or on any read error, the state is - **indeterminate**: no identity use, no mint, no cookie expiry — - "treated as absent" was the wrong contract, since absence feeds the - fresh-mint path. + replica would classify its own cookie as rowless and fork the + identity, and "the backend's strongest read" over eventual storage is + not an authoritative primitive. The proof uses what the protocol + already guarantees: **every post-upgrade identity has an + authority-state record** (the commit point, §5 mint order) under its + derivable family ID, in the globally-strong class — so _rowless_ = + the deployment-metadata **graphless-migration flag** is set AND the + strong read finds **no authority-state record** for the cookie's + derived family ID. Graphless-era cookies never had one; no eventual + read participates. The flag itself is specified: a named + deployment-metadata key (write-once/CAS class), set by the §4.2 + readiness step only on deployments that actually ran graphless + (requires the deployment-metadata capability), surviving binary + rollback, and **cleared by an explicit operator action** once the + migration window closes (quiet-period criterion in the guide) — + clearing ends rowless classification permanently. Outside the flag, + or on any read error, the state is **indeterminate**: no identity + use, no mint, no cookie expiry — "treated as absent" was the wrong + contract, since absence feeds the fresh-mint path. - A verified rowless cookie (`verify → VerifiedIdentity`, carrying the matched version) is **expired and replaced by a fresh mint through the ordinary graph-backed path** when permissions allow; continuity is deliberately not preserved (migration matrix row 13, sign-off 21). An unverifiable cookie (including the declared roaming false-negative) is simply expired. -- **Rowless withdrawal writes nothing** — there is no server-side state - to revoke: no row, no partner mappings, no S2S surface. The cookie is - expired, and that is the entire withdrawal. (An earlier prefix-derived - family record was over-engineering with two defects: unauthenticated - suffix variants could mint records, and — because the HMAC prefix is - per-IP — one visitor's withdrawal would have revoked every identity - behind the same IP. Family records exist only for row-backed - identities, derived from the full graph key, one derivation - everywhere.) +- **Rowless withdrawal writes an exact-cookie family record, then + expires the cookie** — one contract, aligned with the family-first + rule of the permission spec (cookie-only expiry would be best-effort: + a lost response leaves the "withdrawn" cookie usable on its next + presentation). The family ID uses the **same full-graph-key derivation + as row-backed identities** — one derivation everywhere — so the record + revokes exactly the presented cookie value: no per-IP blast (the + earlier prefix derivation would have revoked every identity behind one + IP), and bounded minting, because only **prefix-verified** cookies may + write one — an attacker can fabricate suffix variants only for their + own evidence's prefix, spending their own withdrawal on themselves. + A re-presented withdrawn variant finds its family record and stays + dead. **Egress is typed, not policed.** The inventory-and-denylist test (permission model spec §7) is a backstop, but conventions do not survive @@ -520,7 +538,7 @@ the bounded suffix: | Identity row (hmac, **every** version) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar for the whole provider, not just v0 — keeping all HMAC versions on the verbatim scheme is what keeps the 64-hex cluster prefix a literal key prefix for every HMAC row. (Passphrase rotation still changes a given IP's HMAC, so clusters split across rotation until old identities expire — inherent to rotation, declared, not a key-scheme artifact) | | Alias | **The source identity key itself** — the alias is a value written _at_ the replaced row's address, distinguished by a `kind` discriminator in the JSON envelope | A separate `alias/…` address could never work: lookups by the old cookie hit the source key, and a single-key CAS cannot install a record at a different address | | Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | -| Suppression (negative authority) | `sup/` | Per-permission suppression entries + timestamps; permission-exempt writes; consulted by every S2S recompute and partner-egress check (permission spec §4.3) | +| Authority-state (suppression + positive-authority summary) | `s` + family ID (fixed-width grammar) | Per-permission negative entries **and** the positive summary (permission spec §4.3); permission-exempt writes; consulted by every constructor and S2S recompute | | Rewrite transaction | `rwx/` | One in-flight rewrite per family | | Replay reservation _(informative — deferred with client-cycle; not normative surface)_ | `resv/…` | Deferred client-cycle draft | @@ -564,9 +582,9 @@ counter and schema version; unknown-field and range validation apply like every class (strong class, permission-exempt writes per the permission spec's inventory; revisions are app-level counters because backend generation markers detect change without ordering); -the **rewrite transaction** holds source key, target key, copy point, -state, and epoch; the **reservation** holds state, owner hash, lease -epoch, outcome, and created-at (client-cycle spec). Field validation and +the **rewrite transaction** _(informative — deferred with rewrite)_ +holds source key, target key, copy point, state, and epoch; the **reservation** _(informative — deferred with client-cycle)_ holds +state, owner hash, lease epoch, outcome, and created-at. Field validation and TTLs: aliases live to their retirement deadline; family records to the §7 retention rule (beyond every member, cookie, rewrite, and retry lifetime); transactions to completion plus an audit window; reservations @@ -581,25 +599,26 @@ readers round-trip unknown keys **semantically** (values preserved through read-modify-write; byte-identical output is not required and not achievable through a structured serializer). -| Field | Purpose | Source | Gating permission (egress) | TTL / refresh | Rewrite | On revocation | -| ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | -| key (v1: identifier verbatim; v2: core-constructed, §4) | Row identity | Provider/core | — | Row TTL (1 y today) | New canonical row; old key becomes alias | Family record governs; member tombstone as cleanup | -| `v` | Schema discriminator | Core | — | — | Written at current version | Retained | -| `created` / **`expires_at`** | Row age and the **absolute retention deadline, pinned at mint** — every update writes with the _remaining_ lifetime, never a fresh full TTL (today's full-TTL rewrite lets a frequently visited identity live forever; refreshable derived state must never rejuvenate the identity) | Core | P1 (first-party ops) | Never extended | Preserved (no rejuvenation) | Retained in tombstone | -| `consent.tcf` / `consent.gpp` | Raw signal snapshot for audit; superseded as authority by provenance | Request | Never egressed to partners | Replaced on live resolution (§7 snapshot rule, permission spec) | Fresh live values | Scrubbed | -| `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | -| New: **immutable mint tag** (`mint_provider`, `mint_version`) | Credential retirement and audit — write-once at mint (legacy backfill may populate a missing tag once); **never part of the replaceable snapshot**, or a v1 identity revisited after rotation would be restamped v2 | Mint (or one-time backfill) | — | Immutable | — | Retained | -| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, provider/version) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | -| New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | -| `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | -| `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | -| `pub_properties` (origin/seen domains) | Creation context | Core at mint | P1 | Write-once | Preserved | Scrubbed | -| `device.*` (JA4 class, H2 hash, quality metadata) | **Discontinued for new rows** (§5): fingerprint-derived, buyer-facing — beyond security-classification authorization. v1 rows retain them read-only; they are never egressed post-epic and are dropped at rewrite | Fastly device provider | None grants egress | Write-once (v1) | **Dropped** | Scrubbed | -| New: security classification outcome (boolean) | Bot-gate result | Device provider | — (never egressed) | Written at mint | Fresh | Scrubbed | -| `network.*` (immutable evidence: ASN etc.) | Cluster disambiguation | Platform at mint | P1 | Write-once | Fresh | Scrubbed | -| Derived cluster state (`cluster_size`, computed-at) | Trust gating | Computed | — | **Refreshable, short validity; generation-CAS update; never touches `expires_at`** | Recomputed | Scrubbed | -| `ids` (partner → UID map) | Partner identity graph | Pixel/pull/batch sync | P1 ∧ P4 (partner egress) | Per-mapping timestamps; bounded count/length | Copied **with original timestamps/expiry** | Scrubbed | -| New: alias record kind | Rewrite indirection (§6.1) | Core | — | Retirement deadline | Is the mechanism | Family-revoked like any member | +| Field | Purpose | Source | Gating permission (egress) | TTL / refresh | Rewrite | On revocation | +| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| key (v1: identifier verbatim; v2: core-constructed, §4) | Row identity | Provider/core | — | Row TTL (1 y today) | New canonical row; old key becomes alias | Family record governs; member tombstone as cleanup | +| `v` | Schema discriminator | Core | — | — | Written at current version | Retained | +| `created` / **`expires_at`** | Row age and the **absolute retention deadline, pinned at mint** — every update writes with the _remaining_ lifetime, never a fresh full TTL (today's full-TTL rewrite lets a frequently visited identity live forever; refreshable derived state must never rejuvenate the identity) | Core | P1 (first-party ops) | Never extended | Preserved (no rejuvenation) | Retained in tombstone | +| `consent.tcf` / `consent.gpp` | Raw signal snapshot for audit; superseded as authority by provenance | Request | Never egressed to partners | Replaced on live resolution (§7 snapshot rule, permission spec) | Fresh live values | Scrubbed | +| `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | +| New: **immutable mint tag** (`mint_provider`, `mint_version`) | Credential retirement and audit — write-once at mint (legacy backfill may populate a missing tag once); **never part of the replaceable snapshot**, or a v1 identity revisited after rotation would be restamped v2 | Mint (or one-time backfill) | — | Immutable | — | Retained | +| New: **provenance revision** (application-level monotonic counter, u64, initialized at 1, incremented by every provenance-bearing write, CAS'd with the row generation, serialized as an integer; overflow is a hard error, not a wrap) | Orders clears vs. snapshots (permission spec §4.3) | Core | Read by S2S/clears | Monotonic | — | Retained | +| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, ) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | +| New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | +| `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | +| `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | +| `pub_properties` (origin/seen domains) | Creation context | Core at mint | P1 | Write-once | Preserved | Scrubbed | +| `device.*` (JA4 class, H2 hash, quality metadata) | **Discontinued for new rows** (§5): fingerprint-derived, buyer-facing — beyond security-classification authorization. v1 rows retain them read-only; they are never egressed post-epic and are dropped at rewrite | Fastly device provider | None grants egress | Write-once (v1) | **Dropped** | Scrubbed | +| New: security classification outcome (boolean) | Bot-gate result | Device provider | — (never egressed) | Written at mint | Fresh | Scrubbed | +| `network.*` (immutable evidence: ASN etc.) | Cluster disambiguation | Platform at mint | P1 | Write-once | Fresh | Scrubbed | +| Derived cluster state (`cluster_size`, computed-at) | Trust gating | Computed | — | **Refreshable, short validity; generation-CAS update; never touches `expires_at`** | Recomputed | Scrubbed | +| `ids` (partner → UID map) | Partner identity graph | Pixel/pull/batch sync | P1 ∧ P4 (partner egress) | Per-mapping timestamps; bounded count/length | Copied **with original timestamps/expiry** | Scrubbed | +| New: alias record kind | Rewrite indirection (§6.1) | Core | — | Retirement deadline | Is the mechanism | Family-revoked like any member | ## 7. Composition root and adapter parity @@ -621,17 +640,17 @@ Requirements: **per-record-class consistency requirements**, because "has KV" says nothing about whether revocation is observable: - | Record class | Required semantics | - | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | - | Family revocation records | **Globally observable strong consistency** — every instance's read observes a committed revocation, not merely the writing session's own writes (writer-scoped read-your-writes is insufficient for a fleet). Cloudflare Workers KV is **not eligible** — "60 seconds or more" is an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | - | Family suppression records | **Linearizable per-key CAS** — read-after-write alone cannot provide read-modify-write monotonicity: two writers both read, and an older clear overwrites a newer suppress | - | Identity-row mutation | **Generation CAS** (conditional write on row generation) with reread/recompute on conflict — rows are heavily mutable (snapshots replaced, partner IDs merged, derived state refreshed), so unordered last-writer-wins loses newer evidence and mappings; _visibility_ may stay eventual, unordered _mutation_ may not. Fastly KV offers generation-marker conditional writes; Workers KV's documented concurrent last-write-wins is ineligible for mutation-bearing rows | - | Row creation | **Atomic create-if-absent** (fresh mints), same primitive family | - | Deployment metadata (schema floor) | **Write-once/CAS**, outside ordinary config storage (migration spec §4) | - | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | - | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | - | Identity rows | Eventual **visibility** acceptable _after_ a generation-CAS mutation commits (see Identity-row mutation above) — the earlier "rows are accretive" claim is deleted: rows replace snapshots, merge partner IDs, and refresh derived state, and unordered last-writer-wins loses newer evidence | + | Record class | Required semantics | + | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | + | Family revocation records | **Globally observable strong consistency** — every instance's read observes a committed revocation, not merely the writing session's own writes (writer-scoped read-your-writes is insufficient for a fleet). Cloudflare Workers KV is **not eligible** — "60 seconds or more" is an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | + | Authority-state records (suppression + positive summary) | **Globally observable strong reads AND linearizable per-key CAS** — CAS alone orders writes, but a stale successful _read_ on another instance would authorize egress after a committed suppression ("read failures fail closed" does not cover stale successes); both properties are adapter eligibility gates | + | Identity-row mutation | **Generation CAS** (conditional write on row generation) with reread/recompute on conflict — rows are heavily mutable (snapshots replaced, partner IDs merged, derived state refreshed), so unordered last-writer-wins loses newer evidence and mappings; _visibility_ may stay eventual, unordered _mutation_ may not. Fastly KV offers generation-marker conditional writes; Workers KV's documented concurrent last-write-wins is ineligible for mutation-bearing rows | + | Row creation | **Atomic create-if-absent** (fresh mints), same primitive family | + | Deployment metadata (schema floor) | **Write-once/CAS**, outside ordinary config storage (migration spec §4) | + | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | + | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | + | Identity rows | Eventual **visibility** acceptable _after_ a generation-CAS mutation commits (see Identity-row mutation above) — the earlier "rows are accretive" claim is deleted: rows replace snapshots, merge partner IDs, and refresh derived state, and unordered last-writer-wins loses newer evidence | Every record class additionally declares **durability and maximum retention**: a store passing the consistency check but capping TTLs @@ -652,19 +671,19 @@ Requirements: them is how a "yes" cell hides an unusable feature. Feature eligibility requires wired, not merely available: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | ----------------------------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | - | Graph persistence (eventual OK) | KV Store: available + wired | **Not wired** — the current adapter installs `UnavailableKvStore`; an in-process store is dev-feasible but does not exist yet | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | - | Prefix listing (cluster) | Yes (used today) | **Unavailable** (no store wired — in-process feasibility is a note, not a cell) | Yes (eventual) | _verify_ | - | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | **Unavailable** (no store wired) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | - | Linearizable fenced CAS _(informative — deferred features)_ | **Not currently available** | **Unavailable** (no store wired) | Durable Objects: possible, not wired | **No** | - | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | - | Device host evidence (JA4/H2) | Yes | No | No | No | - | Suppression / authority-state CAS | Generation-marker conditional write: available, **wiring to verify** | **Unavailable** (no store wired) | Workers KV: **ineligible** (last-write-wins); Durable Objects: feasible, not wired | **Unavailable** | - | Identity-row generation-CAS mutation | Same primitive as above | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | - | Row create-if-absent | Generation-marker create: available, **wiring to verify** | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | - | Deployment metadata (write-once/CAS) | **Not wired** — needs a primitive distinct from the config store | **Unavailable** | DO: feasible, not wired | **Unavailable** | - | Durability / max-retention proof | KV durable; TTL ceilings **to verify** against computed horizons | **Unavailable** | Workers KV TTLs: to verify; DO storage: feasible | **Unavailable** | + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | + | Graph persistence (eventual OK) | KV Store: available + wired | **Not wired** — the current adapter installs `UnavailableKvStore`; an in-process store is dev-feasible but does not exist yet | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | + | Prefix listing (cluster) | Yes (used today) | **Unavailable** (no store wired — in-process feasibility is a note, not a cell) | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | **Unavailable** (no store wired) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS _(informative — deferred features)_ | **Not currently available** | **Unavailable** (no store wired) | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | + | Authority-state: global strong reads + CAS | Conditional writes available (generation marker); **globally current read semantics to verify** — both required, wiring to verify | **Unavailable** (no store wired) | Workers KV: **ineligible**; Durable Objects: feasible, not wired | **Unavailable** | + | Identity-row generation-CAS mutation | Same primitive as above | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | + | Row create-if-absent | Generation-marker create: available, **wiring to verify** | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | + | Deployment metadata (write-once/CAS) | **Not wired** — needs a primitive distinct from the config store | **Unavailable** | DO: feasible, not wired | **Unavailable** | + | Durability / max-retention proof | KV durable; TTL ceilings **to verify** against computed horizons | **Unavailable** | Workers KV TTLs: to verify; DO storage: feasible | **Unavailable** | - Each adapter's runtime-services setup routes through the shared builders. - Providers are constructed **once** per application instance and stored in diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index eb0ffe8b5..474275616 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -32,30 +32,30 @@ and whether that is a preservation or a declared change. **Silent changes are defects.** PR #838 changed six of these without declaring any; each was discoverable only because a deleted test had pinned the old behavior. -| # | Decision (today) | After epic | Status | -| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | -| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | -| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | -| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | -| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | -| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | -| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | -| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — opt-out effects are more protective; the same fields' not-opted-out values can also **newly grant P4**, which is not (both effects classified in permission spec §4.5) | -| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | -| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | -| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | -| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | -| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | -| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | -| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | -| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | -| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | -| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | -| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | -| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | -| 13 | Cookies minted by graphless deployments have no graph row | Recognized rowless cookies are **expired and re-minted** through the ordinary graph-backed path (providers spec §5) — never adopted, since prefix-only verification cannot authenticate suffix variants; identity continuity is deliberately lost; withdrawal works without any row via the prefix-derived family ID | **Declared** — pre-existing identities restart rather than carry over | +| # | Decision (today) | After epic | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | +| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | +| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | +| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | +| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — opt-out effects are more protective; the same fields' not-opted-out values can also **newly grant P4**, which is not (both effects classified in permission spec §4.5) | +| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | +| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | +| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | +| 13 | Cookies minted by graphless deployments have no graph row | Rowless proof via the strong authority-state class under the graphless-migration flag (providers spec §5); verified cookies are expired and re-minted without continuity; **rowless withdrawal writes an exact-cookie family record, then expires** — full-key derivation, no prefix mechanism | **Declared** — pre-existing identities restart rather than carry over | Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in the `[permissions]` policy review, made explicitly by maintainers — not @@ -126,10 +126,19 @@ Requirements: revocations, honor suppression records, or fail closed on provenance; an N+1 that merely preserved would, after rollback, treat revoked identities as live (aliases are reserved-future with - the rewrite deferral, providers spec §6.1). N+1 must also **write** - the safety-critical record kinds — family revocation and - suppression — not only read them: a withdrawal arriving on a - rolled-back N+1 fleet must still revoke. + the rewrite deferral, providers spec §6.1). N+1 must also **write + family revocation records** — a withdrawal arriving on a + rolled-back N+1 fleet must still revoke. **Authority-state + (suppression) is different: N+1 neither creates nor clears it.** + Creating would be safe, but clearing now requires the + `AuthorityRefresh` provenance protocol over revision-bearing rows + that N+1 (a v1 writer) cannot produce — so an N+1 clearing without + the fence would expose stale positive snapshots, and one clearing + with it would need the whole N+2 write model. Instead: N+1 + **reads** authority-state fully and fails closed on suppressed + permissions; suppression created by N+2 stays in force during a + rollback, and **recovery (clearing) waits for roll-forward** — a + protective, declared limitation, not an undefined one. **N+1's identity-write behavior is v1, explicitly** — this resolves what was an impossible trilemma (write rows without provenance, @@ -210,8 +219,7 @@ Requirements: (matrix row 12), not a side effect discovered at boot. 4. **The graph schema change is expand-contract, in lockstep with the binary sequence.** New rows carry fields v1 rows never had — provider/ - version, per-permission grant evidence, policy revision, family ID, - rewrite links — and two failure modes must be engineered away: a naive + version, per-permission grant evidence, policy revision, family ID — and two failure modes must be engineered away: a naive schema-version bump makes old readers fail closed on new rows, and an old worker that reads, modifies, and reserializes a row **silently drops** fields it does not model. The sequence shares the config @@ -442,29 +450,29 @@ recording the decision, the deciders, and the date) — the table links them as rows close; an unratified row reverts to open, not to silently implemented. -| # | Decision | Where | Owner | Status | -| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | -| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | -| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | -| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | -| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | -| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | -| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | -| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | -| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | -| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | -| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | -| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | -| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | -| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | -| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | -| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | -| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | -| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | -| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | -| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | -| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | -| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | -| 23 | DataDome security exemption: tag injection, cookie/ClientID read, vendor egress, and cross-integration visibility operate outside the permission model as a ratified security-purpose carve-out with owned cookie names, scope, and withdrawal semantics | hook §4a; permission §7 | maintainers + legal | open | -| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky; opt-out stickiness applies only to opt-out causes) — and an active suppression **overrides a `granted` baseline**: one malformed request denies later no-signal requests until valid evidence clears it, and a policy-baseline grant alone never clears | permission §4.3, §4.1 | maintainers + legal | open | -| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | +| # | Decision | Where | Owner | Status | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | +| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | +| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | +| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | +| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | +| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | +| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | +| 23 | **Open question, not ratified**: may DataDome's security identifier (tag injection, `datadome` cookie, `X-DataDome-ClientID` read and vendor egress) operate outside the permission model? The decision must enumerate exactly which consumers may observe the cookie/ClientID — today's spec allows only the security channel itself and strips every other surface (hook §4a) — plus retention and whether TS withdrawal expires it | hook §4a; permission §7 | maintainers + legal | open | +| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky; opt-out stickiness applies only to opt-out causes) — and an active suppression **overrides a `granted` baseline**: one malformed request denies later no-signal requests until valid evidence clears it, and a policy-baseline grant alone never clears | permission §4.3, §4.1 | maintainers + legal | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/datadome-header-allowlist.md b/docs/superpowers/specs/datadome-header-allowlist.md new file mode 100644 index 000000000..9acb8c225 --- /dev/null +++ b/docs/superpowers/specs/datadome-header-allowlist.md @@ -0,0 +1,10 @@ +# DataDome request-header allowlist (normative, checked-in) + +The complete set of response-named header pointers the security channel +(hook spec §4a) may copy into the owner-scoped publisher-upstream +overlay. Every `X-DataDome-*` name not listed here is rejected. Adding a +name is a reviewed commit to this file and a spec change. + +| Header | Direction | Scope | +| --------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------- | +| `X-DataDome-ClientID` | response → upstream overlay | Owner-scoped overlay only; never the shared request view; vendor egress governed by sign-off item 23 | diff --git a/docs/superpowers/specs/gpp-registry-snapshot.md b/docs/superpowers/specs/gpp-registry-snapshot.md new file mode 100644 index 000000000..103e8ca78 --- /dev/null +++ b/docs/superpowers/specs/gpp-registry-snapshot.md @@ -0,0 +1,35 @@ +# GPP registry snapshot (normative, vendored) + +The pinned per-section accepted versions for the permission spec's §4.5 +map. This file is the single reproducible authority; updating it is a +reviewed spec change. A mapped section presenting a version not listed +here is treated as malformed-present (permission spec §4.4). + +| GPP section ID | Section | Accepted version(s) | +| -------------- | --------------------------------------------------- | ------------------- | +| 6 | US Privacy string (uspv1, carried as a GPP section) | 1 | +| 7 | usnat | 1 | +| 8 | usca | 1 | +| 9 | usva | 1 | +| 10 | usco | 1 | +| 11 | usut | 1 | +| 12 | usct | 1 | +| 13 | usfl | 1 | +| 14 | usmt | 1 | +| 15 | usor | 1 | +| 16 | ustx | 1 | +| 17 | usde | 1 | +| 18 | usia | 1 | +| 19 | usne | 1 | +| 20 | usnh | 1 | +| 21 | usnj | 1 | +| 22 | ustn | 1 | +| 23 | usmn | 1 | +| 24 | usmd | 1 | +| 25 | usin | 1 | +| 26 | usky | 1 | +| 27 | usri | 1 | + +Version values were captured from the IAB registry at the time of +writing and are re-verified against the official registry as part of +ratification review; any correction is a change to this file. diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md index e58443d90..f0c1291af 100644 --- a/docs/superpowers/specs/pr986-review-ledger.md +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -210,3 +210,33 @@ tracked once. | P2 batch (inventory rows for the exempt read and AuthorityRefresh; TCF-vs-GPP source-specific digests; observation timestamps; granted-baseline suppression row in §4.1 + sign-off 24 expanded; snapshot field cleanup; indeterminate read errors; Axum cells unavailable; rewrite/backfill retirement alternative removed; GPP vendored snapshot; 304-safe metadata pass; Respond-first validation; CDN names enumerated; deferred residue bracketed) | all fixed | | P3 batch (verbatim comment covers every hmac version; `Content-Language` example corrected; stale fragments swept; this ledger corrected) | fixed | | Ratification note | decisions directory created (`docs/superpowers/specs/decisions/`), table text points to it; item 23 wording no longer claims ratification | + +## Round 12 — review at bf684e5 + +The R11 rows this round showed as overstated (provider/version leftover +in the graph table, no GPP snapshot file, sign-off 23 wording, dangling +fragments) are hereby corrected below — and from this round on, ledger +"fixed" claims are **mechanically greppable**: each row's parenthetical +names an anchor phrase present in the tree, so `grep` can audit closure +instead of trusting prose. + +| Finding | Status | +| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| P1 two-record atomicity | fixed ("The strong record is the commit point": row commits at r, then authority-state CAS to r; S2S/absence use r only when the strong record reports it; mint eligibility begins at authority-state commit) | +| P1 suppression reads not globally current | fixed ("Globally observable strong reads AND linearizable per-key CAS" — matrix row and Fastly cell updated with reads-to-verify) | +| P1 rowless authoritative not-found | fixed ("proven from the strong class": no authority-state record under the derived family ID + graphless-migration flag; flag wire/lifecycle defined; no eventual read participates) | +| P1 rowless withdrawal three ways | fixed (one contract: "exact-cookie family record, then expires" — full-key derivation, verified-cookies-only writes; migration row 13 aligned; prefix mechanism gone from every spec) | +| P1 N+1 cannot run suppression recovery | fixed ("N+1 neither creates nor clears" authority-state; reads fail closed; clears wait for roll-forward — declared protective limitation) | +| P1 row schema missing revision / stale provider-version | fixed (provenance-revision field row added with init/overflow/CAS rules; provider/version stripped from the mutable provenance row) | +| P1 summary insufficient for policy-only rule | fixed (summary carries kind, grant basis/source class, policy revision, valid_until — absence decision reproducible from the strong record) | +| P1 DataDome allowlist not enumerated | fixed (checked-in `datadome-header-allowlist.md`, spec-pinned to `X-DataDome-ClientID` alone; other `X-DataDome-*` rejected) | +| P1 cookie confinement misses upstream/log surfaces | fixed (exhaustive strip inventory: origin forwarding, proxy/click/Testlight upstreams, auction serialization, logs — each a tested row) | +| P1 304 not implementable | fixed (persisted final post-hook header set re-emitted; absent metadata → cache miss) | +| P2 GPP snapshot missing | fixed (`gpp-registry-snapshot.md` vendored, sections 6–27, ratification re-verification note) | +| P2 field registry not enumerated | fixed (v1 admitted set enumerated in-spec; growth is a spec change) | +| P2 domain/lifetime irreproducible | fixed (PSL-computed registrable domain, vendored PSL revision, Max-Age ≤ 34,214,400 s) | +| P2 skew window unvalued | fixed (normative 300 s constant with rationale) | +| P2 sign-off 23 says ratified | fixed (rewritten as an open question enumerating observers) | +| P2 revocation wording paraphrase | fixed (permission spec repeats "globally observable" verbatim) | +| P2 deferred residue in normative schemas | fixed (rewrite transaction and reservation wire schemas bracketed informative; rewrite links out of the migration expansion item) | +| P3 dangling sentences / key-table description / ledger reliability | fixed (fragments removed; key table says authority-state with positive summary; this section's mechanical-anchor rule) | From 3523b362c0daf41184c2b52a19a55d92010158ff Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sun, 2 Aug 2026 23:47:07 -0700 Subject: [PATCH 14/14] Address thirteenth review: coherent suppression expiry, provable rowless prerequisites, honest failure semantics P1 fixes: - Suppression entries carry their evidence class's valid_until and go inert at expiry (lazily GC'd) - resolving the contradiction where an expired TCF refusal under a granted baseline would deny forever while the normalization table promised the baseline grant; normalization wins. - Rowless classification gains real prerequisites: the flag may be set only after full N+2 convergence (an N+1 fleet still minting v1 rows never classifies rowless) and an idempotent stub-backfill has stamped an authority-state existence stub on every existing row - only then does 'no record' actually mean graphless-era. A per-deployment flag alone cannot prove a per-cookie fact. - The mint-path recovery claim is retracted: a failed authority-state commit after the row commit leaves an orphan no later request can find (no cookie was emitted); it authorizes nothing, expires by TTL, is counted, and has its own runtime-failure-matrix row. The eligibility-at-graph-commit leftover is swept. - The authority-state wire schema now carries every field the permission protocol consumes: negative entries with cause/source/ timestamps/valid_until/referenced revision; the positive summary with kind, grant basis, policy identity, valid_until, revision, evidence timestamp, and the semantic digest with pinned first-seen; plus the backfill stub marker. - Negative-record creation is admission-controlled (existing family on a strong read, or a verified identifier - fabrications write nothing), and rowless withdrawal collapses to one capped per-prefix record (8 suffix hashes; saturation escalates to prefix-wide rowless revocation as the declared abuse response, harming only the abuser's own same-IP graphless cohort) - closing the storage-amplification surface that per-variant exact-cookie records opened. - Embedded GPP GPC is mapped: Gpc=true in any section is the same destructive global opt-out as the header (OR-aggregated); GpcSegmentIncluded=false/absent contributes nothing; malformed GPC segments render the section malformed-present. Sign-off 26. - Batch S2S jurisdiction ages: stored jurisdiction older than the consent-TTL horizon fails closed pending a live refresh; the horizon and its two-sided trade-off are sign-off 25. - Observability sinks join the egress inventory: raw EC values never reach logs/traces/metrics/errors, logging boundaries take hash-only types, the existing PR #838 logging site is cited, and a log-schema denylist test enforces the row. - 304 persisted metadata is versioned by (integration-registry, config, invariant) revisions with mismatch = cache miss, so normal and conditional hits cannot serve different policy metadata; cache-relevant fields are defined. - The DataDome contract aligns with documented vendor behavior where hardening was not intended (configurable SameSite, one-year 31,536,000 s cap replacing the over-vendor 396-day figure, 512-byte size per the current Fastly module, Domain per vendor guidance PSL-validated); the deliberately reduced pointer allowlist requires product AND vendor acceptance (sign-off 28); and the publisher origin is named in sign-off 23 as a ClientID observer - the overlay is the mechanism, the row now names the recipient. P2/P3: deployment-metadata 'm' key class with the graphless flag's full wire lifecycle (N+2 + backfill attested in the value, operator CAS clearing); N+1 rollback tests aligned to read-and-fail-closed for authority state; AuthorityRefresh's access set enumerated (row CAS + authority-record CAS - the old wording forbade a read its own protocol needs); expired-live-plus-persisted fallback decided (expired live does not suppress fallback); the skew constant became an algorithm (beyond-window malformed, no clamping, expiry grace, within-window equality routing to the restrictive tie rule); policy revisions have canonical identity (content digest + activation generation); provider-code-registry.md (hmac allocated) and psl-snapshot-ref.md (ICANN+private, IDNA, IP/single-label host-only) created; the GPP decoder gap (sections 24-27, usnat-v2-decodable-but-unpinned) is an explicit prerequisite; sign-off rows 25-28 added; adapter qualification is a pre-ratification prerequisite; telemetry residue removed; the hook fragment, provenance comma, test host-equivalents, and client-draft Axum claim are fixed. Ledger: Round 13 adopts the added-vs-verified vocabulary - rows are 'text-added' until a subsequent review declines to reopen them; all R12 rows retroactively so marked. --- ...26-07-30-client-cycle-ec-resolve-design.md | 4 +- ...integration-response-header-hook-design.md | 53 ++++---- .../2026-07-30-permission-model-design.md | 105 +++++++++++----- .../2026-07-30-pluggable-providers-design.md | 117 +++++++++++------- ...07-30-provider-migration-rollout-design.md | 96 +++++++------- docs/superpowers/specs/pr986-review-ledger.md | 26 ++++ .../specs/provider-code-registry.md | 10 ++ docs/superpowers/specs/psl-snapshot-ref.md | 13 ++ 8 files changed, 287 insertions(+), 137 deletions(-) create mode 100644 docs/superpowers/specs/provider-code-registry.md create mode 100644 docs/superpowers/specs/psl-snapshot-ref.md diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index eda8268eb..d6a5a487d 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -100,8 +100,8 @@ Everything in this spec follows from that. 5. **Exist on every adapter — where parity means identical behavior, including identical refusal.** Route registration goes through shared route wiring. On adapters whose capability matrix rows are green - (today only the dev adapter has the required CAS class — providers - spec §7), the parity suite asserts identical endpoint behavior; on + (today **no adapter** has the required CAS class — the normative + matrix marks even Axum's storage unavailable; providers spec §7), the parity suite asserts identical endpoint behavior; on adapters without them, parity means **identical startup rejection of the client-cycle selection** — not a proxied 404 (PR #838's failure mode: Fastly-only registration let the Axum dev server proxy the POST diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 3d5caa061..c290b0c3d 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -126,8 +126,8 @@ mutators to the outbound response for HTML document responses it processed. which corrupts attribution and budgets), with a duplicate-ID test in the done-when. Until then the operation set is headers-only, and `Set-Cookie` is fully reserved. - cookie via any operation — `Set-Cookie` is fully reserved in v1 (§3 - deferral). Violations are rejected + `Set-Cookie` is fully reserved in v1 (§3 deferral). Violations are + rejected at the operation layer (§2) and logged at `warn` with the integration id. The reserved lists are single constants next to the definitions they protect, not duplicated in the @@ -202,17 +202,17 @@ mutators to the outbound response for HTML document responses it processed. Which responses the hook runs on, enumerated so two implementations cannot diverge silently: -| Response | Hook runs? | -| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Processed HTML document (rewritten by TS) | Yes | -| Streamed processed document | Yes — operations apply to the header block before first byte | -| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | -| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | -| Redirect (3xx) | No | -| Error responses TS itself generates (4xx/5xx) | No | -| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: when the processed 200 was cached, its **final post-hook header set** (the cache-relevant fields) was persisted alongside the representation; a 304 re-emits those persisted finals — deterministic, no mutator re-run, no representation reconstruction. If no persisted metadata exists, the conditional request is treated as a **cache miss** (full 200 fetched and processed); the earlier "respond 200 instead" without saying whence was not implementable | -| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | -| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | +| Response | Hook runs? | +| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes — operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | +| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: when the processed 200 was cached, its **final post-hook header set** (the cache-relevant fields) was persisted alongside the representation, **versioned by (integration-registry revision, config revision, invariant revision)**; a 304 re-emits those persisted finals only when all three match the serving instance — a mismatch (integration or config changed since caching) is a **cache miss**, so a normal hit and a conditional hit can never return different policy metadata for one representation. "Cache-relevant fields" is defined: the registry-admitted mutable set plus the Cache-Control family and `Vary`; `Set-Cookie` and validators follow ordinary 304 rules and are never replayed. Absent metadata → cache miss | +| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | +| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to processed documents (§6). @@ -225,13 +225,22 @@ degree of freedom is closed: - **Typed security-cookie operation with a concrete lifecycle, not header strings.** The channel emits cookies only through a typed operation, and the registration is not a placeholder — for DataDome - it pins: cookie name exactly `datadome`; `Domain` set to the - registrable domain computed against the **Mozilla Public Suffix List** - (vendored revision named by the implementation; host-only is not used - because DataDome requires site-wide scope), path `/`; mandatory - `Secure` and `SameSite=Lax`; `Max-Age` at most **34,214,400 seconds** - (396 days — the thirteen-month ceiling, as an exact number); size ≤ - 4 KiB; + it pins, **aligned to documented vendor behavior where hardening was + not intended**: cookie name exactly `datadome`; `Domain` per + DataDome's own guidance (the module sets it; TS validates it does not + exceed the registrable domain, computed against the **vendored + Mozilla PSL snapshot** `docs/superpowers/specs/psl-snapshot-ref.md` — + ICANN + private sections, IDNA-mapped; IP-literal or single-label + hosts fall back to host-only), path `/`; `Secure` mandatory; + `SameSite` configurable `Lax` (default) / `Strict` / `None` + (`None` requires `Secure`), matching the vendor's endpoint options; + `Max-Age` at most **31,536,000 seconds** (the vendor's one-year cap — + the earlier 396-day figure exceeded it); size ≤ **512 bytes** + (DataDome's current Fastly-module limit; 4 KiB was ours, not theirs). + Where the contract **is** deliberately narrower than the vendor — the + spec-pinned pointer allowlist starting at ClientID-only against + DataDome's mandatory response-directed mapping set — that reduction + needs explicit product **and vendor** acceptance: sign-off item 28; a violating operation is rejected whole (the batch rule). Every `ts-*` name is rejected. **Read is owner-only, and the strip inventory is exhaustive, not integration-scoped** — the browser sends `datadome` in the ordinary @@ -317,8 +326,8 @@ degree of freedom is closed: origin's cache restrictions + public replacement → restriction preserved (pass-through responses never run the hook, §3a); a cache-hit serve re-applying mutations without weakening the stored classification; a `Vary` mutation neither - dropping core-required values nor bypassing the snapshot; each CDN - directive (`Surrogate-Control`, `CDN-Cache-Control`, host equivalents) + dropping core-required values nor bypassing the snapshot; each of the four enumerated CDN fields (`Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, `Edge-Control`) individually stripped; and a rejected `Content-Encoding` mutation. ## 5. Size and sequencing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 360df42ad..b9614430f 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -451,6 +451,16 @@ and the fail-closed marker: entry's; ties resolve to the more restrictive state. So a delayed grant with `LastUpdated = 100` never clears a suppression whose refusal carried `200`, while a genuine re-consent at `300` does. The + **Every suppression entry carries its own `valid_until`, derived from + its evidence class's TTL, and an expired entry is inert** — treated as + cleared without a write, lazily garbage-collected. Without this, an + expired TCF refusal under a `granted` baseline would deny forever: + normalization says an expired record is absent and "must not revoke + indefinitely", yet the surviving suppression would block the baseline + grant that same table promises — the two contracts now agree, in the + normalization table's favor. (Destructive opt-outs tombstone and need + no suppression longevity; non-destructive opt-out entries expire on + the consent-TTL horizon of the evidence that created them.) The transition table (causes without an intrinsic timestamp — malformed records decode no `LastUpdated`, absence has no source — use their **observation timestamp**, server receipt on the shared clock basis @@ -490,8 +500,12 @@ and the fail-closed marker: `GraphOps`, and clearing first is forbidden — so recovery has its own narrow write path: **`AuthorityRefresh`**, permission-exempt but strictly scoped to committing provenance from the _current live - resolution_ (nothing else: no partner writes, no egress, no reads - beyond the row being refreshed) while suppression remains effective; + resolution_ — its exact access set: **read + generation-CAS of the row + being refreshed, and read + CAS of the family's authority-state + record** (its own clearing protocol requires both; the earlier "no + reads beyond the row" wording forbade a read its own CAS needs); + nothing else — no partner writes, no egress — while suppression + remains effective; the clear then references that provenance's revision. Revisions are an **application-level monotonic counter written with the row** — never backend generation markers, which (per Fastly's own contract) only @@ -597,6 +611,7 @@ an expired record before clearing both sources; expiry-first is a | Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | | One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | | One valid record + one **expired** record of the same family | The valid record governs — the expired one dropped at pipeline step 2, before conflict resolution ever saw it | **Changed (declared)** — current runtime resolves the conflict first and can select the expired record | +| **Expired** live record + still-valid persisted-KV record | The expired live record is absent entirely (step 2), so it does **not** suppress the fallback: the persisted record substitutes, subject to its own TTL and the full pipeline — "live wins" applies to live records that still exist after expiry filtering | Decided here | | Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | | Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**, then flows through the full normalization pipeline (syntax, expiry, conflict) like any live record; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | **Changed (declared)**: current code returns immediately after the KV load, bypassing expiry and conflict normalization | | Proxy/mirror mode | **Minimal opt-out extraction still runs; full semantic decoding is skipped.** Because opt-outs are globally authoritative (§4), proxy mode must not suppress them: the §4.5-mapped opt-out fields (GPP US sections) and the US Privacy string are decoded — nothing else — alongside syntax validation, so a valid SaleOptOut or USP opt-out revokes and withdraws exactly as outside proxy mode. No grants are ever derived from records in proxy mode; a present record otherwise blocks grants (fail-closed); absent → baseline. GPC needs no decoding | **Changed (declared)**: today proxy mode skips decoding entirely — fail-open under permissive baselines and, worse, opt-out-blind | @@ -614,7 +629,7 @@ the single normative statement; an earlier "N/A contributes nothing" rule is dead, and the P4-authorizing consequence is sign-off item 17) — and only the fields marked destructive trigger withdrawal. Section IDs and versions are those of the IAB GPP -specification current at implementation time; adding a section or field is +specification pinned by the vendored snapshot; adding a section or field is a change to this table. | Source · field | Value | `store-on-device` (P1) | `select-personalised-ads` (P4) | Destructive withdrawal? | @@ -634,6 +649,16 @@ a change to this table. Applicable_ = grant-class; absent = nothing; a non-applicable section's fields grant nothing (their opt-outs still count, per step 2). +**Embedded GPC is mapped, not ignored.** The US sections carry +`GpcSegmentIncluded` and `Gpc` fields; a request with embedded +`Gpc = true` and no `Sec-GPC` header was previously unspecified despite +the global-GPC rule. Normatively: embedded `Gpc = true` in **any** +section is the same **destructive global opt-out** as the header +(aggregated with it by OR — opt-outs are never jurisdiction-filtered); +`GpcSegmentIncluded = false`, an absent segment, or `Gpc = false` +contributes nothing; a malformed optional GPC segment renders that +section malformed-present (blocks grants, never withdraws). + **Applicability and aggregation — ordered algorithm:** 1. **Section map (normative, pinned here — not "whatever GPP is @@ -647,7 +672,12 @@ fields grant nothing (their opt-outs still count, per step 2). `US/NJ` ↔ 21, `US/TN` ↔ 22, `US/MN` ↔ 23, **`US/MD` ↔ 24, `US/IN` ↔ 25, `US/KY` ↔ 26, `US/RI` ↔ 27** (an earlier draft wrongly claimed MD/IN/KY/RI had no section). A truncated map silently loses - opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. + opt-outs — a Texas (16) or Maryland (24) sale opt-out must not + vanish. **The current decoder is an explicit prerequisite gap**: it + (and `iab_gpp` 0.1.2) supports sections 7–23 only and models `usnat` + v2 while the snapshot pins v1 — implementation must extend or replace + the decoder for 24–27 _and_ reject versions the library happens to + decode but the snapshot disallows. The implementation PR cross-checks this list against both the current decoder's section set and the official registry, and the accepted version per section is **pinned to the vendored registry snapshot @@ -751,6 +781,14 @@ migration story unresolvable (migration spec §2, rows 5 and 7). ### 5.5 Policy revision activation +A **policy revision** has a defined identity: the canonical content +digest of the `[permissions]` section (identity — republishing identical +policy yields the same digest) paired with the config-store activation +generation (ordering — strictly monotonic per instance). Provenance +stores both; comparisons order by generation and equate by digest, so a +rollback is a _new_ generation carrying an _old_ digest, with defined +semantics on both axes. + A policy edit propagates through the config store, so a fleet briefly mixes revisions. The contract: instances stamp every resolution and every provenance write with the policy revision they used (already required by @@ -817,23 +855,24 @@ Consumers of the resolved set in this epic: inventory, normative per path (one test per row; a denylist check proves no ungated egress exists): - | Path | Required permissions | Notes | - | ---------------------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | - | EC-derived auction request IDs | both purposes | Derived values are identity | - | Page-bids path | both purposes | | - | Bidstream EIDs | both purposes | The one gate PR #838 had | - | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | - | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | - | Pull sync (browser-request-scoped partner exchange) | both purposes, from the **live** request resolution | Pull sync is created from a browser request and checks the live `EcContext` today — it keeps using the live P1 ∧ P4 decision plus the family revocation state (§4.3); stored provenance is never a substitute for available live evidence | - | Batch sync (context-free S2S partner exchange) | both purposes, from **stored provenance** | The only truly signal-less path; authority rules below. Today's handler only authenticates and checks row state, so this gate is **declared hardening** (migration spec §2) | - | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | - | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | - | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | - | Integration persistent response cookies | `store-on-device` (+ P4 where the cookie is an advertising identifier) | **Deferred with the hook's cookie surface** — the write-side gate alone was insufficient (read/use/forward/withdrawal unmodeled), so cookie operations ship only with the full model; this row and the client-cycle **page leg** (module injection gated on the provider's full declaration) join the inventory when their features do, and the §5.3 no-geo guard's consumer list grows with them | - | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | - | Authority-state / suppression-decision read (§4.3) | **exempt**, narrowly scoped | Returns only family ID, per-permission authority summary, and suppression entries — no identity values, no partner data; a test proves nothing else escapes | - | `AuthorityRefresh` provenance write (§4.3) | **exempt**, strictly scoped | Commits current-live-resolution provenance only; enables suppression recovery without reopening `GraphOps` | + | Path | Required permissions | Notes | + | ------------------------------------------------------------------ | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | + | EC-derived auction request IDs | both purposes | Derived values are identity | + | Page-bids path | both purposes | | + | Bidstream EIDs | both purposes | The one gate PR #838 had | + | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | + | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | + | Pull sync (browser-request-scoped partner exchange) | both purposes, from the **live** request resolution | Pull sync is created from a browser request and checks the live `EcContext` today — it keeps using the live P1 ∧ P4 decision plus the family revocation state (§4.3); stored provenance is never a substitute for available live evidence | + | Batch sync (context-free S2S partner exchange) | both purposes, from **stored provenance** | The only truly signal-less path; authority rules below. Today's handler only authenticates and checks row state, so this gate is **declared hardening** (migration spec §2) | + | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | + | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | + | **Observability sinks — logs, traces, metrics, error attachments** | Never — no permission authorizes them | Raw EC values (and derived URLs embedding them) must not reach any observability sink: logging boundaries accept redacted/hash-only types, not `&str` (PR #838 logs a redirect URL containing the EC and the raw `ec_id` field — the motivating counterexample); a log-schema denylist test enforces the row | + | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | + | Integration persistent response cookies | `store-on-device` (+ P4 where the cookie is an advertising identifier) | **Deferred with the hook's cookie surface** — the write-side gate alone was insufficient (read/use/forward/withdrawal unmodeled), so cookie operations ship only with the full model; this row and the client-cycle **page leg** (module injection gated on the provider's full declaration) join the inventory when their features do, and the §5.3 no-geo guard's consumer list grows with them | + | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | + | Authority-state / suppression-decision read (§4.3) | **exempt**, narrowly scoped | Returns only family ID, per-permission authority summary, and suppression entries — no identity values, no partner data; a test proves nothing else escapes | + | `AuthorityRefresh` provenance write (§4.3) | **exempt**, strictly scoped | Commits current-live-resolution provenance only; enables suppression recovery without reopening `GraphOps` | With **no EC provider configured**, identity use fails closed: a cookie value present on the request never egresses anywhere — never vacuously @@ -858,12 +897,14 @@ Consumers of the resolved set in this epic: | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (a **per-permission equality digest computed over only the applicable, aggregated §4.5 fields for that permission** — never the whole GPP record, or a CMP touching an unrelated notice field would mint a new digest and reset first-seen forever) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | - Timestamps are compared with a bounded clock-skew tolerance that is a - **normative constant — 300 seconds** (five minutes, applied - symmetrically; a spec-level value because suppression precedence, - malformed classification, and future-date handling all hinge on it, - and per-deployment values would give the same input different privacy - outcomes); + Timestamp handling is an **algorithm, not just a constant**: with + skew S = 300 s (normative), a timestamp `t > now + S` renders its + record malformed-present; `t` in `(now, now + S]` is used as-is (not + clamped — clamping re-freshens replays); expiry checks grant a grace + of S (`expired` means `valid_until < now − S`); and two evidence + timestamps within S of each other **compare equal**, which routes + the comparison to the tie rule (restrictive) — so a slightly + future-dated consent cannot out-order a just-observed opt-out; beyond-window future-dated records are **rejected as malformed**, and within the window a record's first normalized timestamp is pinned to its digest and never advanced by re-presentation (§4.3's anti-replay @@ -880,7 +921,15 @@ Consumers of the resolved set in this epic: stored evidence has **expired**, or when the regime no longer accepts the stored grant's source class (§4's regime-scoped table). Any of these → no update, row flagged for the operational cleanup of §4.2 - trigger 3. Sync never mints authority of its own. + trigger 3. Sync never mints authority of its own. **Stored + jurisdiction ages too**: batch sync has no live geo, so the + jurisdiction it recomputes against is the one from the last browser + visit — and a visitor who moved from a permissive into a GDPR + jurisdiction would otherwise keep old-rule egress for up to the row + lifetime. A stored jurisdiction older than the **consent-TTL + horizon** fails closed pending a live refresh (the inverse — denying + a visitor who moved the other way — is the accepted cost); the + horizon choice and its legal trade-off are **sign-off item 25**. **Legacy (pre-epic) rows** carry none of these fields. They are treated as reserved `hmac-v0` provenance with **no stored grant evidence**, so diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index fbe362fcf..05e231e95 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -263,12 +263,18 @@ normative order is: gate → `generate` → graph-row commit → **authority-state commit** (the strong record reporting the row's revision — the commit point of the two-record protocol, permission model spec §4.3) → cookie scheduled → eligible for egress. Eligibility begins -at the authority-state commit, not the row commit: a row whose revision -the strong record has not reported is uncommitted detail, which is what -keeps S2S authorization and the absence decision consistent. "Cookie scheduled" means queued onto the +at the authority-state commit and nowhere earlier. **Mint-path failure +between the two writes is declared, not "recovered"**: no cookie was +emitted, so no later request can identify the orphan — the earlier +claim that recovery "re-runs step 2" is false for the mint path (it +holds only for _presented_ identities via `AuthorityRefresh`). An +orphaned row (or orphaned pending record, if the order's first write +succeeded alone) authorizes nothing — the strong record never reported +it — and is bounded by its `expires_at`/retention TTL; the failure is +counted (a first-class metric) and appears in the §6.2 runtime matrix. "Cookie scheduled" means queued onto the final response — `Set-Cookie` is physically emitted after first-request -processing, so egress eligibility begins at **graph commit**, not at -header emission; the identity exists durably from that moment. A +processing, so egress eligibility begins at the **authority-state commit** +(§5 mint order — not graph commit, and not header emission). A graph-commit failure means the mint never happened: no cookie, no egress, error logged, the next request retries. @@ -285,14 +291,21 @@ variant**. Therefore: not-found proves nothing — a just-minted row invisible on a stale replica would classify its own cookie as rowless and fork the identity, and "the backend's strongest read" over eventual storage is - not an authoritative primitive. The proof uses what the protocol - already guarantees: **every post-upgrade identity has an - authority-state record** (the commit point, §5 mint order) under its + not an authoritative primitive. The proof needs more than the flag — + a per-deployment flag cannot prove a per-cookie fact, and "no + authority-state record" alone would misclassify every graph-backed + legacy row and every N+1-minted v1 row (neither has one). The + protocol closes both gaps with **prerequisites for setting the + flag**: (1) the fleet has fully converged on **N+2** (no v1 minting + anywhere — an N+1 fleet must never run rowless classification), and + (2) an idempotent **stub-backfill scan** has stamped a minimal + authority-state existence stub onto **every existing identity row** + (legacy and N+1-minted alike). Only then does the invariant hold: + every row-backed identity has an authority-state record under its derivable family ID, in the globally-strong class — so _rowless_ = - the deployment-metadata **graphless-migration flag** is set AND the - strong read finds **no authority-state record** for the cookie's - derived family ID. Graphless-era cookies never had one; no eventual - read participates. The flag itself is specified: a named + flag set AND the strong read finds **no record** for the cookie's + derived family ID. Graphless-era cookies never got a stub because + they have no row for the scan to find; no eventual read participates. The flag itself is specified: a named deployment-metadata key (write-once/CAS class), set by the §4.2 readiness step only on deployments that actually ran graphless (requires the deployment-metadata capability), surviving binary @@ -308,19 +321,29 @@ variant**. Therefore: deliberately not preserved (migration matrix row 13, sign-off 21). An unverifiable cookie (including the declared roaming false-negative) is simply expired. -- **Rowless withdrawal writes an exact-cookie family record, then - expires the cookie** — one contract, aligned with the family-first - rule of the permission spec (cookie-only expiry would be best-effort: - a lost response leaves the "withdrawn" cookie usable on its next - presentation). The family ID uses the **same full-graph-key derivation - as row-backed identities** — one derivation everywhere — so the record - revokes exactly the presented cookie value: no per-IP blast (the - earlier prefix derivation would have revoked every identity behind one - IP), and bounded minting, because only **prefix-verified** cookies may - write one — an attacker can fabricate suffix variants only for their - own evidence's prefix, spending their own withdrawal on themselves. - A re-presented withdrawn variant finds its family record and stays - dead. +- **Rowless withdrawal writes into one capped per-prefix record, then + expires the cookie** — durable (cookie-only expiry is best-effort: a + lost response leaves the "withdrawn" cookie usable), and **bounded in + storage**, which separate exact-cookie records were not: a holder of + one valid prefix can fabricate billions of suffix variants, and + per-variant records would be attacker-priced strong storage. The + record (strong class, keyed on the verified prefix) holds a bounded + list (cap 8) of withdrawn-suffix hashes; writes are admitted only for + **prefix-verified** cookies; **saturation escalates to prefix-wide + rowless revocation** — every rowless cookie under that prefix is + treated withdrawn, which harms only the abuser's own same-IP graphless + cohort and is the declared abuse response (legitimate users hold one + or two variants ever). A re-presented withdrawn variant finds its + entry (or the saturated record) and stays dead. Row-backed + withdrawal is untouched: full-graph-key family records, one derivation + everywhere. +- **Negative-record creation has admission rules everywhere**: + suppression and family records may be written only for (a) an + existing family — the authority-state record exists on a strong + read — or (b) a **verified** identifier (`verify`); a fabricated, + unverifiable cookie writes nothing. This bounds record creation to + real identities plus the writer's own evidence, and per-prefix + rate limits apply to the rowless path above. **Egress is typed, not policed.** The inventory-and-denylist test (permission model spec §7) is a backstop, but conventions do not survive @@ -494,15 +517,16 @@ Startup validation (§6) covers configuration; this covers what happens when a healthy configuration meets an unhealthy runtime. Every row logs at `error` with a metric; none is silent: -| Failure | Behavior | -| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| `generate` returns an error | No identity this request; request proceeds stateless; no cookie written | -| Graph-row commit fails at mint | Mint never happened (§5): no cookie, no egress; next request retries | -| Graph read fails on an existing identity | Identity unusable this request (fail closed for egress); cookie untouched | -| Cluster prefix listing fails | Treated as cluster-size-unknown → `cluster_fallback` policy applies | -| Tombstone write fails | Permission model spec §4.3: family retries, readers fail closed on partial families | -| Geo provider returns invalid output (unparseable country) at runtime | Treated as lookup failure → `default_country` (permission model spec §5.2), counted in the lookup-failure metric | -| Device provider signals unavailable at runtime (e.g. no JA4 on a request) | Classification degrades per the provider's declared fallback, never silently upgrades `looks_like_browser` | +| Failure | Behavior | +| ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `generate` returns an error | No identity this request; request proceeds stateless; no cookie written | +| Graph-row commit fails at mint | Mint never happened (§5): no cookie, no egress; next request retries | +| Authority-state commit fails after the row commit at mint | No cookie, no eligibility (the strong record never reported the revision); the orphan row expires by TTL; counted — **no recovery claim**, nothing can find it | +| Graph read fails on an existing identity | Identity unusable this request (fail closed for egress); cookie untouched | +| Cluster prefix listing fails | Treated as cluster-size-unknown → `cluster_fallback` policy applies | +| Tombstone write fails | Permission model spec §4.3: family retries, readers fail closed on partial families | +| Geo provider returns invalid output (unparseable country) at runtime | Treated as lookup failure → `default_country` (permission model spec §5.2), counted in the lookup-failure metric | +| Device provider signals unavailable at runtime (e.g. no JA4 on a request) | Classification degrades per the provider's declared fallback, never silently upgrades `looks_like_browser` | The **degraded-graph health signal** referenced above and by the withdrawal contract is a defined state machine, not a vibe: it is @@ -539,7 +563,8 @@ the bounded suffix: | Alias | **The source identity key itself** — the alias is a value written _at_ the replaced row's address, distinguished by a `kind` discriminator in the JSON envelope | A separate `alias/…` address could never work: lookups by the old cookie hit the source key, and a single-key CAS cannot install a record at a different address | | Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | | Authority-state (suppression + positive-authority summary) | `s` + family ID (fixed-width grammar) | Per-permission negative entries **and** the positive summary (permission spec §4.3); permission-exempt writes; consulted by every constructor and S2S recompute | -| Rewrite transaction | `rwx/` | One in-flight rewrite per family | +| Deployment metadata | `m` + fixed metadata name (fixed-width grammar; schema floor, graphless-migration flag) | Write-once/CAS class; value carries schema version, state, epoch, set-at; the graphless flag's lifecycle: created by the migration readiness step **after** N+2 convergence + stub-backfill completion (both attested in the value), cleared by explicit operator CAS with the quiet-period criterion recorded | +| Rewrite transaction _(informative — deferred)_ | `rwx/` | One in-flight rewrite per family | | Replay reservation _(informative — deferred with client-cycle; not normative surface)_ | `resv/…` | Deferred client-cycle draft | Grammars are pairwise non-intersecting by their literal prefixes (plus @@ -558,8 +583,8 @@ Physical keys are **delimiter-free with fixed-width segments**: a generated key can begin with 64 hex characters, which is what makes disjointness from the legacy `{64hex}.{6alnum}` grammar _provable_ rather than asserted (an earlier `f` tag was itself a hex digit) — then -a **4-character provider code from a checked-in, append-only, -never-reused registry file** (allocation is a reviewed commit; +a **4-character provider code from the checked-in, append-only, +never-reused registry `docs/superpowers/specs/provider-code-registry.md`** (allocation is a reviewed commit; codes are immutable and never recycled, including for retired providers), then the suffix. Segment boundaries are positional, so no segment can contain or escape a delimiter, prefix queries are plain @@ -573,12 +598,18 @@ deadline, and fencing epoch; the **family revocation record** holds the family ID, revoked-at, triggering signal class (§4.5 destructive column), and a **family epoch** bumped on every revocation-state change (the client-cycle commit CAS is conditioned on it) — deliberately no identity -data, so it can outlive its members; the **authority-state (suppression) record** holds, per permission: +data, so it can outlive its members; the **authority-state record** holds, per permission — negative side: state (`suppressed`/`cleared`), cause, source class, authoritative or -observation evidence timestamp, the **application-level provenance -revision** a clear references, and the positive-authority summary -(revision + evidence timestamp) — plus the record-level CAS version -counter and schema version; unknown-field and range validation apply +observation evidence timestamp, entry `valid_until` (evidence-class TTL; +expired entries are inert), and the provenance revision a clear +references; positive side (the summary, **every field the permission +spec's absence/replay decisions consume — a reduced schema cannot +reproduce them**): kind (user evidence vs policy baseline), grant +basis/source class, policy revision (digest + activation generation), +`valid_until`, provenance revision, evidence timestamp, and the +per-permission **semantic digest with its pinned first-seen/ +first-normalized timestamps** (anti-replay); record level: family ID, +CAS version counter, schema version, stub marker (backfill, §5); unknown-field and range validation apply like every class (strong class, permission-exempt writes per the permission spec's inventory; revisions are app-level counters because backend generation markers detect change without ordering); @@ -608,7 +639,7 @@ achievable through a structured serializer). | `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | | New: **immutable mint tag** (`mint_provider`, `mint_version`) | Credential retirement and audit — write-once at mint (legacy backfill may populate a missing tag once); **never part of the replaceable snapshot**, or a v1 identity revisited after rotation would be restamped v2 | Mint (or one-time backfill) | — | Immutable | — | Retained | | New: **provenance revision** (application-level monotonic counter, u64, initialized at 1, incremented by every provenance-bearing write, CAS'd with the row generation, serialized as an integer; overflow is a hard error, not a wrap) | Orders clears vs. snapshots (permission spec §4.3) | Core | Read by S2S/clears | Monotonic | — | Retained | -| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, ) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | +| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | | New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | | `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | | `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 474275616..f9bdbb36f 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -161,8 +161,10 @@ Requirements: later than the model it protects. Live-request paths keep v1 semantics until N+2. - Rollback tests therefore run the family-revocation and suppression - paths — read **and write** — plus v1-minting behavior, on N+1 + Rollback tests therefore run: family-revocation read **and + write**; authority-state/suppression **read-and-fail-closed only** + (N+1 writes none — the earlier read-and-write test requirement + contradicted this contract); and v1-minting behavior — all on N+1 against N+2-written data. **Rollback is binaries-first too, in the other direction** — N+2 → N+1 binaries roll back keeping the new config (N+1 reads it fully; reverting config first would hand the old shape to N+2 @@ -194,7 +196,15 @@ Requirements: a deprecated `passphrase` field whose presence triggers the custom error). -2. **Revocation-eligible storage is a per-adapter gate, and ungated +2. **Adapter qualification is a pre-ratification prerequisite, not a + footnote.** No adapter is presently proven eligible for the complete + identity protocol — Fastly's global-read/retention cells are + unverified and its deployment-metadata primitive unwired; every + other adapter is unavailable or needs a new primitive. Ratifying + before at least one adapter qualifies risks an epic with no + selectable identity provider, so Fastly qualification (or an + explicit decision to proceed without it) gates ratification. +3. **Revocation-eligible storage is a per-adapter gate, and ungated adapters migrate stateless.** Identity features require the adapter's strong-consistency rows in the capability matrix (providers spec §7) to be green: today that means Fastly must _verify_ its KV read @@ -206,7 +216,7 @@ Requirements: would make the required fixtures self-contradictory. Whether ungated adapters go stateless or block the release is product sign-off item 12. -3. **Graph-store readiness precedes everything.** Today the graph store +4. **Graph-store readiness precedes everything.** Today the graph store is optional and EC generation succeeds without one; the epic's no-active-until-commit invariant (providers spec §5) makes it mandatory wherever a minting provider is configured — so a currently @@ -217,7 +227,7 @@ Requirements: row supports the features in use, providers spec §7) _before_ rolling N+1. This is a **declared breaking change** for graphless deployments (matrix row 12), not a side effect discovered at boot. -4. **The graph schema change is expand-contract, in lockstep with the +5. **The graph schema change is expand-contract, in lockstep with the binary sequence.** New rows carry fields v1 rows never had — provider/ version, per-permission grant evidence, policy revision, family ID — and two failure modes must be engineered away: a naive schema-version bump makes old readers fail closed on new rows, and an @@ -256,32 +266,32 @@ Requirements: new fields untouched if read-only, preserved semantically if read-modify-write on N+1, **test-proven lost on pre-N+1** (documenting why the floor is a floor); N+2-reader/N+1-written-row → full function. -5. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no +6. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a startup error (providers spec §6). In PR #838 this configuration — the exact state an operator following the docs reaches if they miss one line — validated green and silently minted zero ECs. -6. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by +7. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by PR #838, deliberately not carried into this epic — providers spec §2) and `provider = "client-fixed"` are unknown keys and rejected like any other, so a config written against the PR #838 example cannot silently select a provider that no longer exists. -7. **Provider switches go through legacy readers.** Changing +8. **Provider switches go through legacy readers.** Changing `[ec] provider` on a deployment with live identities requires listing the outgoing provider in `[ec] legacy_providers` (providers spec §6.1) so existing cookies keep resolving and stay withdrawable; the guide documents the switch sequence and the retirement/cleanup step that ends it. -8. **The example config ships the migrated happy path**, uncommented: +9. **The example config ships the migrated happy path**, uncommented: `provider = "hmac"` with its block, `[geo] default_country`, and (for Fastly) the behavior-preserving `[device] provider = "fastly"` and `[geo] provider = "platform"` lines present with a comment stating what removing them changes. PR #838's example shipped the passphrase block uncommented with the selector commented out — steering operators directly into the silent-stateless state. -9. Every misconfiguration in the providers spec §6 table fails at - **startup**. Request-time failure for a configuration error is a defect. -10. Validation is split into two named layers, because "the same +10. Every misconfiguration in the providers spec §6 table fails at + **startup**. Request-time failure for a configuration error is a defect. +11. Validation is split into two named layers, because "the same validation at push and startup" is not implementable: **structural validation** (schema, types, `[permissions]` policy — permission spec §3.3) runs at `ts config push` and again at startup; @@ -378,10 +388,8 @@ global honoring of opt-out signals is unconditional. maximum cookie/row lifetime plus rollout skew** is the **only retirement-readiness** bar for a legacy provider ("trending to ~zero" is not evidence; a yearly visitor - is not churn), (rewrite-based backfill and its metrics left with the rewrite - deferral). The telemetry set also includes: graph read/commit failures, - stored-provenance denials, schema-migration failures, and - replay-reservation recoveries. **Each rollout-gate metric ships with a + is not churn), . The telemetry set also includes: graph read/commit failures, + stored-provenance denials, and schema-migration failures. **Each rollout-gate metric ships with a threshold, an evaluation window, and a named action** (pause rollout / roll back / block retirement) in the migration guide — a metric with a "healthy range" but no action is dashboard decoration; the two already @@ -450,29 +458,33 @@ recording the decision, the deciders, and the date) — the table links them as rows close; an unratified row reverts to open, not to silently implemented. -| # | Decision | Where | Owner | Status | -| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | -| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | -| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | -| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | -| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | -| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | -| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | -| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | -| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | -| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | -| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | -| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | -| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | -| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | -| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | -| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | -| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | -| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | -| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | -| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | -| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | -| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | -| 23 | **Open question, not ratified**: may DataDome's security identifier (tag injection, `datadome` cookie, `X-DataDome-ClientID` read and vendor egress) operate outside the permission model? The decision must enumerate exactly which consumers may observe the cookie/ClientID — today's spec allows only the security channel itself and strips every other surface (hook §4a) — plus retention and whether TS withdrawal expires it | hook §4a; permission §7 | maintainers + legal | open | -| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky; opt-out stickiness applies only to opt-out causes) — and an active suppression **overrides a `granted` baseline**: one malformed request denies later no-signal requests until valid evidence clears it, and a policy-baseline grant alone never clears | permission §4.3, §4.1 | maintainers + legal | open | -| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | +| # | Decision | Where | Owner | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | +| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | +| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | +| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | +| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | +| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | +| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | +| 23 | **Open question, not ratified**: may DataDome's security identifier (tag injection, `datadome` cookie, `X-DataDome-ClientID` read and vendor egress) operate outside the permission model? The decision must enumerate exactly which consumers may observe the cookie/ClientID — the enumerated observers are the security channel **and the publisher origin, which receives `X-DataDome-ClientID` via the upstream overlay** — "owner-scoped overlay" names the mechanism, this row names the recipient — plus retention and whether TS withdrawal expires it | hook §4a; permission §7 | maintainers + legal | open | +| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky; opt-out stickiness applies only to opt-out causes) — and an active suppression **overrides a `granted` baseline**: one malformed request denies later no-signal requests until valid evidence clears it, and a policy-baseline grant alone never clears | permission §4.3, §4.1 | maintainers + legal | open | +| 25 | Batch-sync stored-jurisdiction maximum age (consent-TTL horizon): a mover into GDPR stops old-rule egress at the horizon; a mover out is denied until a live visit | permission §7 | maintainers + legal | open | +| 26 | Embedded GPP GPC maps to the destructive global opt-out (header-OR-embedded aggregation) | permission §4.5 | maintainers + legal | open | +| 27 | Proxy-mode minimal opt-out extraction (decode only §4.5-mapped opt-out fields; no grants) | permission §4.4 | maintainers + legal | open | +| 28 | DataDome integration is deliberately reduced relative to vendor defaults (spec-pinned pointer allowlist starting at ClientID-only; hardened cookie attributes) — requires product **and vendor** acceptance | hook §4a; `datadome-header-allowlist.md` | maintainers + product | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md index f0c1291af..9f2b04669 100644 --- a/docs/superpowers/specs/pr986-review-ledger.md +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -240,3 +240,29 @@ instead of trusting prose. | P2 revocation wording paraphrase | fixed (permission spec repeats "globally observable" verbatim) | | P2 deferred residue in normative schemas | fixed (rewrite transaction and reservation wire schemas bracketed informative; rewrite links out of the migration expansion item) | | P3 dangling sentences / key-table description / ledger reliability | fixed (fragments removed; key table says authority-state with positive summary; this section's mechanical-anchor rule) | + +## Round 13 — review at ba25ba85 + +**Status vocabulary change (per this round's P3):** ledger rows now +distinguish **text-added** (normative text landed; cross-document +coherence pending the next review) from **verified-closed** (a later +review re-examined and did not reopen). A greppable anchor proves +presence, not coherence — R12's own rows demonstrated the difference. +All R12 rows are retroactively **text-added**; rows below are +text-added unless marked otherwise. + +| Finding | Status | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| P1 expired refusal suppresses forever | text-added: suppression entries carry evidence-class `valid_until`, expired entries inert — normalization wins | +| P1 graphless flag can't prove per-cookie fact | text-added: rowless gated on N+2 convergence + idempotent stub-backfill of every existing row; N+1 fleets never classify rowless | +| P1 mint authority-write failure unrecoverable | text-added: false recovery claim retracted; declared TTL-bounded orphan + failure-matrix row; eligibility-at-graph-commit leftover swept | +| P1 wire schema can't represent the protocol | text-added: full authority-state schema (negative entries with valid_until; positive summary with kind/basis/policy identity/digest/first-seen; stub marker) | +| P1 negative-record amplification | text-added: admission rules (existing family or verified id); rowless withdrawal = one capped per-prefix record (8), saturation → prefix-wide rowless revocation as declared abuse response | +| P1 embedded GPP GPC unmapped | text-added: Gpc=true in any section = destructive global opt-out, header-OR-embedded; GpcSegmentIncluded/malformed rules; sign-off 26 | +| P1 stale batch jurisdiction | text-added: consent-TTL jurisdiction-age horizon, fail closed pending live refresh; sign-off 25 | +| P1 observability egress | text-added: inventory row, hash-only logging types, #838 logging site cited, denylist test | +| P1 200-vs-304 divergence | text-added: persisted metadata versioned by (registry, config, invariant) revisions; mismatch = miss; cache-relevant fields defined | +| P1 DataDome vendor incompatibility | text-added: SameSite configurable, 1-year cap (31,536,000 s), 512-byte size, Domain per vendor guidance PSL-validated; reduced pointer set → product **and vendor** acceptance (sign-off 28) | +| P1 publisher origin as ClientID observer | text-added: sign-off 23 names the recipient | +| P2 batch (flag wire contract + `m` class; N+1 test alignment; AuthorityRefresh read-set; expired-live fallback row; skew algorithm; policy-revision identity = digest + activation generation; provider-code registry + PSL reference files created; decoder prerequisite noted; sign-off rows 25–28; adapter qualification as pre-ratification prerequisite; telemetry residue swept) | text-added | +| P3 batch (hook fragment; provenance comma; host-equivalents in tests; client-draft Axum claim; this vocabulary change) | text-added | diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md new file mode 100644 index 000000000..fb373f32e --- /dev/null +++ b/docs/superpowers/specs/provider-code-registry.md @@ -0,0 +1,10 @@ +# Provider-code registry (normative, append-only, never reused) + +Four-character codes (`[a-z0-9]`, zero-padded) used in physical graph +keys (providers spec §6.3). Allocation is a reviewed commit to this +file; codes are immutable and never recycled, including for retired +providers. + +| Code | Provider | Allocated | Status | +| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ------ | +| `hmac` | Built-in HMAC EC provider (note: hmac identities use the reserved verbatim key grammar, so this code appears in non-key contexts — provenance, registries — not in physical keys) | 2026-08-02 | active | diff --git a/docs/superpowers/specs/psl-snapshot-ref.md b/docs/superpowers/specs/psl-snapshot-ref.md new file mode 100644 index 000000000..72bb3a519 --- /dev/null +++ b/docs/superpowers/specs/psl-snapshot-ref.md @@ -0,0 +1,13 @@ +# Public Suffix List snapshot reference (normative) + +The vendored Mozilla PSL revision used for registrable-domain +computation (hook spec §4a): the implementation PR vendors the list file +and records its upstream commit hash here. Rules: ICANN **and** private +sections apply; hostnames are IDNA-mapped before matching; IP literals +and single-label hosts have no registrable domain (cookie falls back to +host-only). Updating the snapshot is a reviewed spec change. + +| Field | Value | +| --------------- | --------------------------------------------------------- | +| Upstream commit | _recorded by the implementation PR that vendors the list_ | +| Vendored path | _recorded alongside_ |