Finalize publication evidence with independent OPE, clustered stats, and HTML report

[vtavakkoli]

Summary

Final publication-evidence hardening for the full COMMAG benchmark. This PR addresses the methodological issues found in the first full run and adds a self-contained visual report.

Scientific corrections

Independent policy evaluation

• splits the tr0–tr11 training pool into disjoint policy-fit and independent-OPE-fit configuration subsets
• chooses the OPE subset deterministically to maximize action coverage on both sides
• trains separate HistGradientBoosting outcome evaluators for SLA, energy proxy, and stability
• validates the evaluator with grouped cross-fitting on scenario × training configuration × experiment
• scores proposed/FQI/CQL/HGB counterfactual actions with the independent evaluator instead of the policy critics
• keeps logged outcomes explicitly labeled as observed

Validation is now used

tr12–tr14 now freezes, before final testing:

• SLA threshold
• uncertainty threshold
• planning weight
• switching/hysteresis margin
• OOD threshold calibrated to a target false-positive rate on seen validation only
• linear-CQL alpha

The full validation grid and selected setting are exported. Seen/unseen final-test rows are not used during calibration.

Clustered statistical inference

Primary paired inference aggregates by scenario × training configuration × experiment and reports equal-cluster mean utility delta, cluster bootstrap 95% CI, paired random-sign permutation p-value, Cohen's dz, and row-weighted delta as a secondary descriptive measure. UE/episode statistics are exported only as episode_statistics_secondary.json.

OOD evaluation

Adds AUROC, AUPRC, FPR@95%TPR, validation-calibrated threshold rates, per-unseen-scenario discrimination, threshold sensitivity, and existing mean/p95 OOD/fallback evidence.

Transition / shortcut audit

Adds scheduler, PRB, and joint scheduler+PRB change counts/rates per scenario/config/experiment/BS. If no within-episode joint-action changes exist, policy-change-only classifier metrics remain unavailable and the report labels feature-removal results as persistence/configuration shortcut sensitivity.

Churn / hysteresis

Adds a validation-tuned switching margin. Proposed changes with insufficient score improvement hold the current supported action. Validation records the utility/SLA/churn/rollback trade-off used to select the margin.

Latency evidence

Adds critic/OOD/selection component probes plus p50/p95/p99 host/container per-decision latency and runtime metadata. This is explicitly not presented as end-to-end RIC-to-gNB latency.

Readiness semantics

Successful execution no longer means publication readiness. Final summary separates:

• run_status: EXPERIMENT-COMPLETE
• evidence_status: READY-FOR-MANUSCRIPT or REVIEW-REQUIRED

The evidence status comes from explicit methodological gates and is not conditioned on favorable statistical significance.

Self-contained HTML report

Generates results/publication/report.html with no external JavaScript, remote asset, font, or image dependency. Inline SVG figures cover:

• experimental architecture
• independent-OPE utility/SLA/energy/churn comparisons
• cluster-level effect-size forest plot
• OOD behavior by scenario
• validation calibration
• transition/shortcut audit
• latency/runtime evidence
• readiness gates and claim restrictions

Main outputs

• report.html
• publication_summary.json
• publication_baselines.csv
• publication_decisions.csv.gz
• clustered_statistics.json
• paired_statistics.json (compatibility alias for primary clustered inference)
• episode_statistics_secondary.json
• independent_ope_fit.json
• partition.json
• validation_calibration.csv
• validation_selection.json
• cql_validation.csv
• ood_detection.json
• ood_generalization.json
• ood_threshold_sensitivity.csv
• transition_audit.csv
• policy_shortcut_test.csv
• latency_profile.json
• literature_reference.json
• publication_models.joblib

Run

docker compose -f docker-compose.publication.yml up --build --force-recreate publication-test

Tests

The PR includes deterministic unit tests for partitioning, config validation, independent-OPE scoring, clustered inference, OOD metrics, transition-change auditing, and self-contained SVG reporting, plus a synthetic end-to-end publication test that exercises partition → policy/OPE fitting → validation calibration → FQI/CQL/proposed scoring → clustered statistics → OOD audit → HTML generation.

The full 2M+ row COMMAG experiment must still be rerun on the target Docker host before manuscript numbers are finalized. This PR intentionally does not reuse the earlier direct-method result table as if it came from the corrected independent evaluator.