Skip to content

Add opt-in error-aware emulator training - #9

Draft
botjean wants to merge 1 commit into
JETSCAPE:mainfrom
botjean:agent/emulator-statistical-uncertainty
Draft

Add opt-in error-aware emulator training#9
botjean wants to merge 1 commit into
JETSCAPE:mainfrom
botjean:agent/emulator-statistical-uncertainty

Conversation

@botjean

@botjean botjean commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Dependency

This PR depends on #8. Keep it draft until #8 is merged, then update it onto main and rerun the combined tests. The two branches already merge cleanly in a local integration checkout.

Summary

  • Add opt-in normalize_y and deterministic random_state settings for scikit-learn GPs.
  • Add opt-in model-prediction statistical uncertainty in emulator training.
  • Project independent per-bin prediction variances through feature scaling and PCA to a heterogeneous alpha for every design point and retained PC.
  • Record exact training-data, design, parameterization, and GP-setting metadata in emulator caches.
  • Reject legacy or incompatible caches with a one-time force_retrain: true migration message.
  • Allow the HDF5 prediction-matrix loader to select fields such as y_err_stat.

Methodological scope

Prediction statistical errors are treated as independent between observable bins before PCA. Their marginal variances are projected into retained-PC space; off-diagonal PC-noise covariance is discarded because the retained PCs are modeled by independent GPs. Ordinary PCA is still fitted to central predictions.

All new training options default to disabled, so freshly trained emulators retain the previous method unless explicitly configured. Cache validation is intentionally stricter: metadata-free legacy pickles must be retrained once.

Validation

  • Focused methodology/data-loading tests: 15 passed
  • Full clean-branch suite: 25 passed
  • Independent agent re-review: no remaining findings
  • Clean merge with Fix emulator covariance batching and group cache collisions #8: 31 passed
  • End-to-end synthetic HDF5 fit verifies projected per-design/per-PC alpha
  • Real cache write/reuse/invalidation path is tested
  • git diff --check: clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant