Prepare OM4 snapshot dataset generation

[alxmrs]

Summary

• add a backward-compatible DATA_VARIANT=snapshots mode to the Torch preprocessing and normalization harnesses
• select om4_5daily_snapshots.zarr and derive om4_<resolution>_snapshots output directories
• make the OM4 variable contract explicit, retaining the issue-approved five-day-mean wfo forcing while excluding undeclared raw diagnostics
• make preprocessing schema failures fatal and validate the flattened, publishable output contract at every resolution
• explicitly write Zarr v2 and fix clusterless dry runs
• document the snapshot temporal semantics and the planned v2026-08 four-resolution workflow

The source and temporal semantics follow the discussion in #450, especially:

• Re-run preprocessing on OM4 data #450 (comment)
• Re-run preprocessing on OM4 data #450 (comment)
• Re-run preprocessing on OM4 data #450 (comment)

Validation

• uv run --project data pytest data/tests -q — 31 passed, 1 skipped
• targeted pre-commit hooks — passed
• bash -n scripts/slurm_preprocess_om4.sbatch scripts/slurm_make_norm_om4.sbatch — passed

Deployment after review

No preprocessing jobs or public-bucket writes were performed for this PR. After approval/merge, the intended sequence is:

1. run the real snapshot-source, ten-timestep, no-write Torch smoke test
2. publish 2°, 1°, 1/2°, and 1/4° unfiltered datasets under s3://m2lines-pubs/Samudra/v2026-08/om4_<resolution>_snapshots/
3. generate normalization stores only after each processed dataset succeeds
4. validate the public stores and update the available-datasets table with measured sizes and links

[alxmrs]

Self-review, noticed a new source for freshwater fluxes.

[alxmrs]

🤖 Self-review follow-up is in 2568fe0.

Changes made:

• centralized the OM4 variable contract so preprocessing, flattening, source selection, and validation cannot drift independently;
• replaced the hard-coded 1-degree input schema with shared resolution-independent validation for both depth-resolved and flattened outputs;
• require both provenance hashes, validate grid type, coordinate/variable dimensions, dtypes, required variables, and allowed flattened variables;
• fixed final validation for --skip_flattening and corrected native --skip_regridding output metadata from gaussian to tripolar;
• centralized averaged/snapshot source and output naming in one shell helper used by both Slurm harnesses, with tests for both variants and invalid input;
• added regression tests for provenance, Gaussian versus tripolar metadata, dtypes, and the shell policy.

I intentionally left wfo optional pending the discussion with Will about a matching freshwater-flux source for the averaged run. Snapshot input continues to preserve wfo when present.

[alxmrs]

@fomo-bot Will you review this PR?

[fomo-bot]

Reviewed current head 2568fe09.

Findings: no blocking code issues. The snapshot/averaged source selection, OM4 variable contract, final output validation, Zarr v2 write path, and Slurm harness changes look coherent against the current diff. The remaining wfo optionality looks like an external data-source decision tracked in #838, not a correctness blocker in this PR.

Validation:

• Live PR: MERGEABLE, draft, REVIEW_REQUIRED, mergeStateStatus=BLOCKED.
• Review threads: all resolved.
• CI: current head checks are green, with conditional jobs skipped.
• Local: uv run --project data --extra test pytest data/tests -q passed 39 passed, 1 skipped; bash -n scripts/slurm_preprocess_om4.sbatch scripts/slurm_make_norm_om4.sbatch scripts/om4_data_variant.sh passed; git diff --check origin/main...HEAD passed.

Non-blocking note: the PR body’s validation command omits the data package test extra; in this fresh checkout, uv run --project data pytest data/tests -q did not have pytest available, while the --extra test form passed.

I’d be comfortable with this after it is marked ready for review and the normal review gate is satisfied. Keep the post-merge real-source smoke test before any public bucket writes.

Debug info

• Repository: m2lines/Samudra
• Trigger: Prepare OM4 snapshot dataset generation #857 (comment)
• Request ID: 10671167-0014-445c-80d8-d3041121b003
• Accepted: true