Add multi-objective COMMAG offline policy study

[vtavakkoli]

Summary

Extends the existing pinned COMMAG/Fitted-Q benchmark with a paper-oriented, experiment-separated offline RAN policy study while leaving the current COMMAG workflow intact.

Added

• multi-objective policy selection over SLA, energy proxy, stability and uncertainty
• independent SLA, energy and stability critics plus an uncertainty critic trained from grouped out-of-fold residuals
• Isolation-Forest OOD detection with a conservative sticky fallback
• bounded Fitted-Q planning term
• train-only residual adaptation with an explicit without_real_data_adaptation ablation
• Logistic Regression, Random Forest, ExtraTrees and HistGradientBoosting baselines
• matched ablations:
  • without safety gate
  • without OOD gate
  • without planning
  • without real-data adaptation
• default 30-seed robustness study with mean/std/95% CI exports
• latency, throughput, SLA violation, energy proxy, stability, policy churn, rollback/fallback, OOD and uncertainty metrics
• paper-ready CSV/JSON/HTML evidence in results/prepare-data, results/train and results/test
• isolated Docker workflow in docker-compose.offline-study.yml with prepare-data, train and test services
• comprehensive methodology/reproducibility documentation

Scientific guardrails

The reports keep observed logged outcomes separate from counterfactual direct-method estimates. Selected-policy SLA/energy/stability/utility and uplift are explicitly labeled as model estimates from fixed traces, not causal online intervention effects. Energy is explicitly labeled as a normalized proxy rather than joules.

exp1 remains the training side and exp2 remains held out; episode overlap and unsupported held-out actions fail validation.

Validation performed

• python -m compileall -q agentic_ran tests
• YAML parsing for configs/offline_policy.yaml and docker-compose.offline-study.yml
• PYTHONPATH=. pytest -q tests/test_offline_policy.py → 3 passed
• full 30-seed integration study on a deterministic COMMAG-shaped fixture → BENCHMARK-READY; all structural gates passed
• published Git blob hashes verified against the locally tested files

The current execution environment does not provide Docker or direct network access to download the public COMMAG files, so I did not claim new real-COMMAG performance numbers in this PR. The Docker workflow is wired to the repository's existing pinned COMMAG downloader for reproducible real-data execution.

Run

mkdir -p data/raw/commag data/prepared/commag artifacts/offline-policy \
  results/prepare-data results/train results/test

LOCAL_UID=$(id -u) LOCAL_GID=$(id -g) \
  docker compose -f docker-compose.offline-study.yml up --build --abort-on-container-exit test

See docs/OFFLINE_POLICY_STUDY.md for output files and paper-use guidance.