Skip to content

The as_of conformance check enforces a rule the spec never states, and covers half of it #184

Description

@macanderson

as-of-temporal is the only conformance check in the suite that enforces a rule SPEC.md does not state. A provider implemented from the spec alone — which §1 says is sufficient — can fail it without having broken any published requirement.

In plain words

The automated conformance suite rejects a provider for returning content that wasn't valid yet at the query's pinned time (as_of) — but the written spec never actually states that rule anywhere with a MUST or SHOULD, so a provider built strictly from the spec can fail a test for violating a requirement that was never published. At the same time, the check only looks at half the relevant window: it never checks whether the returned content had already expired (gone stale) by the pinned time, only whether it started too late.

contextgraph-conformance/src/lib.rs:1144 requires a provider to exclude frames whose valid_from is later than the query's as_of pin, and fails it by name when it does not:

provider returned N frame(s) whose valid_from is after as_of=... — content that
was not yet true at the pinned instant (§6.1)

That citation is to §6.1, whose entire statement about as_of is one clause of prose in a paragraph headed Semantics: "as_of pins retrieval to an instant." No MUST, no SHOULD, no anchor. Every other check in the suite cites a numbered requirement — Q1, B3, F5, E1, H4, G4. This one cannot, because there is none.

The asymmetry with kinds is the sharp part. §5.1 exists specifically to close this defect for kinds, and describes it exactly: "not an unreachable field, but a reachable one that silently does nothing". kinds got Q1 and a filter check. as_of got the check and no requirement. Issue #10 added the probe; the spec anchor never followed.

The check also covers only half of what §6.1 describes

§6.1 gives valid_from/valid_to as a window: "valid_from/valid_to bound when the content was true in the world". check_as_of reads valid_from only. A frame whose valid_to is 2020-01-01T00:00:00Z — content that had stopped being true years before the pin — is returned for as_of: 2026-07-01T00:00:00Z and the check passes, reporting "none of the returned frame(s) is dated after the pin".

So the half of temporal validity that catches stale facts is unchecked, while the half that catches premature ones is enforced without being specified.

Why it matters beyond tidiness

README.md lists Temporal validity as one of the seven guarantees — "A query can pin retrieval to a point in time with as_of" — enforced by "ContextFrame temporal fields". The frame's temporal fields carry F4, which constrains only their format. So the guarantee a reader is offered is that as_of is well-formed, not that it does anything, and the suite meanwhile enforces a stricter rule than either document states.

Reproduce

rg -n 'as_of' SPEC.md              # three hits: an example, F4's format rule, one prose clause
rg -n 'CHECK_AS_OF' contextgraph-conformance/src/lib.rs

Then: write a provider that serves a frame with valid_to in the past and valid_from absent. contextgraph-inspect reports as-of-temporal green. Add valid_from after the pin instead, and it fails — citing a section that imposes no requirement.

Why this needs a decision rather than a patch

Writing the requirement means deciding the semantics, and the choices are not obvious:

  • Is as_of a MUST or a SHOULD? A provider with no temporal index cannot honour it at all, which argues for a capabilities flag — but §8.3 and ADR 0004 are explicit that a capability nothing exercises is the surface this project removes.
  • Does as_of bind valid_to as well as valid_from — i.e. is it a point-in-window predicate, or only a not-yet-true filter?
  • How does it interact with recorded_at? §6.1 distinguishes when content was true from when the provider learned it, which is a bitemporal model; as_of currently pins neither axis normatively.
  • What does a provider that cannot serve a pinned query reply — zero frames, or a new error code? unsupported_kind set the precedent for kinds; there is no unsupported_as_of.

Pillar

Reliability and maintainability. A conformance suite that enforces unpublished rules is the self-attestation §11.1 rejects, inverted: the spec under-claims and the suite over-enforces, and a provider author cannot reconcile them from the documents they were told to implement from.

Done looks like

  • A requirement anchor for as_of in §5 or §6.1, stating the predicate a provider must apply and against which fields.
  • check_as_of extended to the full window if the requirement covers valid_to, and a --misbehave mode that trips it — the suite's standing rule is that every check has an adversarial witness.
  • README's "Temporal validity" row pointing at the new anchor rather than at the fields.
  • The kinds precedent followed: if a provider may decline, say what it replies.
  • Lands in contextgraph-conformance (lib.rs) plus the SPEC.md edit, without growing lib.rs into a new god file under this workspace's size conventions.
  • Any new SPEC.md prose cites code by symbol (check_as_of, CHECK_AS_OF) rather than a pinned line number, matching this org's citation convention.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions