Skip to content

feat(core): keep DOSY results in the project and give ILT a value lifecycle#20

Merged
nmrtist merged 1 commit into
mainfrom
feat/dosy-result-persistence
Jul 26, 2026
Merged

feat(core): keep DOSY results in the project and give ILT a value lifecycle#20
nmrtist merged 1 commit into
mainfrom
feat/dosy-result-persistence

Conversation

@nmrtist

@nmrtist nmrtist commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Why

Pseudo-2D analysis state lived only in memory. Reopening a project always returned to the stack display and forgot the DOSY method, its parameters and both maps. Because figure() falls back to the stack plot whenever the selected map is absent, the failure mode was a silently different picture, not an empty one — the worst shape a failure can take in a plotting tool.

What is persisted

Both maps go into one binary blob under a new plotx_dosy_v1 storage kind, next to the display, the method, and a provenance record naming the algorithm, its version, the invocation inputs and a fingerprint of the numeric data the fit consumed. The encoder follows the existing AFM payload shape: magic, length-prefixed JSON header, length-prefixed arrays, and a reader that proves every extent before it trusts it.

Cached contour geometry stays derived, like processed_figure. It is rebuilt from the restored numbers.

A mismatched fingerprint keeps and shows the stored map; it never recomputes. fit_ilt is the only implementation and there is no multi-version algorithm registry, so recomputing would silently redraw every stored project after any future kernel change. A DOSY map gets published; a contour ladder does not. That asymmetry is the whole reason the numbers are stored rather than regenerated.

Invariants worth keeping

  • The fingerprint covers every numeric input the fit consumes. Referencing moves the chemical-shift axis without touching a single trace sample, and the diffusion metadata reaches the per-column fit through the b-factor conversion. Both are in the digest, and both methods hash the same inputs. Adding an input to a DOSY fit means adding it here too, or the guarantee quietly narrows.
  • Restoration happens after retransform(). rebuild() unconditionally clears every map, and the load path retransforms after applying the recipe. A restore placed earlier is erased without a trace. A test fails if it moves.
  • User-visible explanations are derived where they can go stale, stored only where they cannot. Whether the selected map is missing is a function of the display, the method and which maps exist, so it is computed on demand; a stored copy outlives the condition and ends up claiming a fallback while the map is on screen. Only genuine load-time facts are stored.
  • What is drawn and what is reported must agree. A view snapshot is bypassed when the map it depicts did not survive the load, and snapshot_backed follows that branch so the rebuilt figure still gets axis overrides and viewport sync.
  • Every mutation of persisted state marks the document dirty, including builds that fit nothing — they still install the method, the map and its provenance.

Value lifecycle

The ILT regularization weight now has all three stages: a persistent application default, an explicit input for the next build, and the stored result provenance, resolved in that order.

The explicit input is scoped to one dataset and absent by default. That absence is load-bearing: it is what keeps the other two stages reachable from every entry point rather than only from a painted panel. A resolution whose fallbacks only fire in the UI validates the panel, not the model.

Bounds moved out of an egui widget into settings/model.rs, shared by the Preferences dialog, the Experiment card and the property schema. dataset.analysis.ilt.result_lambda exposes the stored value read-only, refused before any provider is consulted.

Because one stage reads a project file, the parameters are validated before the inversion. The grid size sizes a dense n_grid x n_grid system, so an unchecked value from a malformed project would size an unbounded allocation instead of producing an error. Those bounds are safety limits for external input, not preferences, and deliberately have no catalog properties.

Project-file float fidelity

serde_json gains the float_roundtrip feature. Its default parser returns a value one ULP away for roughly a tenth of all f64s — measured at 20798 of 200000 — and every number in a project travels that path: spectral widths, carrier frequencies, phases, region bounds, peak positions, fit results. For DOSY it made an untouched project report a false provenance mismatch on reopen; more broadly it broke the plain promise that saving and reopening does not change the data. One workspace-level feature fixes it everywhere, and a round-trip test pins it so it cannot come back unnoticed.

Values already written by earlier development builds keep whatever they drifted to. The drift is one ULP, far below any measurement uncertainty, so no data repair is attempted.

Adding to this

A new persisted analysis result needs four things, in this order: a shape validated on both encode and decode; a fingerprint over everything its kernel reads; a restore placed after retransform(); and a diagnostic that reaches the load report rather than only a tool panel. Skipping the last one is how a silent swap gets back in.

Deliberately not done

  • Undo does not restore a discarded map. Undoing a processing change drops it exactly as any reprocessing does, and the user is told. Making maps survive undo means putting them in the processing snapshot, which is a larger change than this one.
  • d_min, d_max and n_grid gained validation bounds but no catalog properties. They define a grid rather than express a scientific choice; only the regularization weight was migrated.
  • The provenance property targets the dataset, not a field. A DOSY map is a 2D scalar field and belongs in the field catalog, but making it one turns a fixed key set into a conditional one and pulls in capabilities and a figure adapter. The dataset is the truthful address today.
  • An older build can still open a project it wrote and drop the new extension on resave. That is a property of the additive-extension model rather than of this change — plotx.regions and plotx.analysis behave identically — so the schema version is unchanged. The structural fix is preserving unknown extension keys across a round trip, which would benefit every additive extension at once.

Housekeeping

Three modules were split first, as pure moves with no behavior change, to stay under the source-size limit: view conversion out of project/convert.rs, the Dataset dispatch out of state/datasets.rs, and the table workflow out of state/app_impl_analysis.rs.

Documentation is updated in both languages: the pseudo-2D guide, the file-format reference and the preferences reference.

…ecycle

Pseudo-2D analysis state lived only in memory: reopening a project always
returned to the stack display and forgot the DOSY method, its parameters and
both maps. Because `figure()` falls back to the stack plot when the selected
map is absent, the failure was a silently different picture rather than an
empty one.

Both maps now persist as one binary blob under a new `plotx_dosy_v1` storage
kind, alongside the display, the method and a provenance record naming the
algorithm, its version, the invocation inputs and a fingerprint of the numeric
data the fit consumed. Cached contour geometry stays derived, like
`processed_figure`. On load a mismatched fingerprint keeps and shows the stored
map rather than recomputing it: `fit_ilt` is the only implementation and there
is no multi-version algorithm registry, so recomputing would silently redraw
every stored project after any kernel change.

The fingerprint covers every numeric input the fit consumes, not only the trace
samples. Referencing moves the chemical-shift axis without touching a sample,
and the diffusion metadata reaches the per-column fit through the b-factor
conversion, so both are in the digest and both methods hash the same inputs.

The ILT regularization weight now has all three lifecycle stages in place: a
persistent application default, an explicit input for the next build, and the
stored result provenance, resolved in that priority order. The explicit input
is scoped to one dataset and absent by default, so the fallbacks stay reachable
from every entry point rather than only from the panel. Its bounds move out of
an egui widget into `settings/model.rs`, shared by the Preferences dialog, the
Experiment card and the property schema; `dataset.analysis.ilt.result_lambda`
exposes the stored value read-only. Because one of those stages reads a project
file, the parameters are validated before the inversion: the grid size sizes a
dense `n_grid x n_grid` system, so an unchecked value would size an unbounded
allocation instead of producing an error.

Anything the user must be told is derived where it can go stale and stored only
where it cannot. Whether the selected map is missing is a function of the
display, the method and which maps exist, so it is computed on demand; only
genuine load-time facts are stored. Both reach the load report and the status
bar, not just the Experiment card, because a project that opens showing a
different plot than it saved is exactly what must not be quiet. A view snapshot
is bypassed when the map it depicts did not survive the load, so the canvas and
the report cannot disagree.

`serde_json` gains the `float_roundtrip` feature. Its default parser returns a
value one ULP away for roughly a tenth of all `f64`s, and every number in a
project travels that path, so a reopened but unmodified project could report a
false provenance mismatch. A round-trip test pins it.

Three modules were split first, as pure moves, to stay under the source-size
limit: view conversion out of `project/convert.rs`, the `Dataset` dispatch out
of `state/datasets.rs`, and the table workflow out of `state/app_impl_analysis.rs`.

Notes for later work:
- Undoing a processing change discards a built map, as any reprocessing does.
  Making maps survive undo means putting them in the processing snapshot.
- `d_min`, `d_max` and `n_grid` gained validation bounds but no catalog
  properties; only the regularization weight was migrated.
- A DOSY map is a 2D scalar field and would be better addressed as a dataset
  field than through the dataset itself, which is why its provenance property
  targets the dataset for now.
@nmrtist
nmrtist merged commit e5b87bc into main Jul 26, 2026
11 checks passed
@nmrtist
nmrtist deleted the feat/dosy-result-persistence branch July 26, 2026 12:26
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant