Skip to content

Format 0.2: NaN errors on present data; stricter bundle root name - #22

Merged
alexkjames merged 4 commits into
mainfrom
format-0.2
Aug 13, 2026
Merged

alexkjames merged 4 commits into
mainfrom
format-0.2

Conversation

@alexkjames

Copy link
Copy Markdown
Contributor

Format changes (normative)

NaN errors on present data (§9, §3.8, §3.12). A present datum may now carry NaN in both err_real and err_imag, meaning the producer delivered no uncertainty estimates — the honest representation for contractor processing chains (e.g. Quantec per-receiver deliverables) that report responses without uncertainties. Rules:

  • Errors are pairwise all-or-nothing: one finite and one NaN is invalid.
  • A missing datum still requires NaN errors; a finite or zero error on a missing datum stays invalid.
  • Present DC data requires err_imag = 0 only when errors are delivered.
  • Producers must not use NaN errors as a quality flag (that is use = 0) and must not fabricate errors; consumers needing uncertainties apply their own error model or floors.

Stricter bundle root name (§2). The top-level directory name must start with [A-Za-z0-9_] — excluding ., .., hidden directories, and option-like --prefixed names — and must not end with a dot (stripped by Windows). Charset unchanged.

Rationale for both is recorded in csemx-rationale.md.

Mechanics

  • Beta minor per §12: readers accept only the exact format version, so 0.1 bundles need re-export as 0.2.
  • Validator: scalar reference (validate_data_row) and streamed fast path updated in lockstep; the parity test class cross-checks them, with new coverage for NaN-error present rows (including DC), reversed mixed-error rows, and the root-name rule.
  • Version bumped consistently: spec title/§12, both schema copies, both validator-metadata copies, all 7 example manifests + rebuilt example.csemx.zip, template, quickstart, web status line, pyproject.toml → 0.2.0.

Verification

  • 106 unit tests pass with and without optional deps; check_examples.sh --full and rebuild_example_zip.py --check pass.
  • Real-world check: a Quantec-exported Quellaveco bundle with 678,974 present rows all carrying NaN errors validates clean under 0.2 (after correcting its format.version), and fails only on the error rule under 0.1.

🤖 Generated with Claude Code

alexkjames and others added 4 commits August 13, 2026 12:14
A present datum may now carry both err_real/err_imag as NaN, meaning the
producer delivered no uncertainty estimates (contractor processing chains
commonly report responses without them). Errors are pairwise all-or-nothing;
mixed finite/NaN stays invalid, as does a finite error on a missing datum.
For present DC data err_imag = 0 is required only when errors are delivered.

The bundle top-level directory name must now start with [A-Za-z0-9_] (no
'.', '..', hidden or option-like names) and must not end with a dot
(stripped by Windows); charset unchanged.

Spec, rationale, schemas, examples, quickstart, and web status bumped to
0.2; validator fast path and scalar reference updated in lockstep with
parity coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alexkjames
alexkjames merged commit 624cd4b into main Aug 13, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant