Consolidate rift_O4c + master + junior/rift_O4d into rift_O4d (and port the open O4c portfolio-driver fix)

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,37 @@
+<!--
+Delete any section that does not apply.  The checklist below is the CHEAP pass; the
+integrator-specific invariants and the blast-radius rule live in
+MonteCarloMarginalizeCode/Code/RIFT/integrators/REVIEW_CHECKLIST.md
+-->
+
+## What this changes, and why
+
+<!-- The defect or the goal.  If a number changed, say which measurement says so. -->
+
+## Evidence
+
+<!-- What was measured, in WHICH REGIME, over how many replicates.  A single run that
+     completed and looked sane is not evidence about a degenerate-regime fix. -->
+
+## Checklist
+
+- [ ] New/changed tests are **named in `.github/workflows/ci.yml`** (a test file existing does
+      not mean CI runs it — grep the workflow)
+- [ ] Tests assert the precondition they are about, so they cannot pass vacuously
+- [ ] **Read the diff — `git show HEAD` — and confirmed every hunk is mine.** These checkouts
+      are shared between concurrent sessions and `git add -A` has twice swept another branch's
+      uncommitted work into a commit. `--stat` does not answer this: it is blind to an
+      unrelated edit inside a file you also changed, which is exactly what happened both
+      times. (See `RIFT/integrators/REVIEW_CHECKLIST.md` §1.)
+- [ ] Any changed default is justified by a measurement, and the previous behaviour is still
+      reachable by flag
+
+### If this touches the integrators, the ILE, the likelihood, any evidence/weight/gate, **or changes a default that production inherits**
+
+- [ ] Read `RIFT/integrators/REVIEW_CHECKLIST.md` §2–3 (population-vs-sample; side effects
+      when the feature is off)
+- [ ] **Requested a full code review** — high blast radius: a wrong answer here is plausible,
+      not loud
+- [ ] Shape-recovery merge gate run per `RIFT/integrators/TESTING.md` — required whenever the
+      change can move a posterior, which includes a changed default that production inherits,
+      not only edits under `RIFT/integrators/` (the fast CI integral test does not subsume it)

.github/workflows/ci.yml

@@ -196,8 +196,16 @@ jobs:
           python -m pip install -r requirements.txt --break-system-packages
           python -m pip install coverage pytest --break-system-packages
           python -m pip install --editable . --break-system-packages
-      - name: Run probe confirm-on-fail accounting tests
-        run: python -m pytest -q MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/test_probe_confirm.py
+      - name: Run probe confirm-on-fail accounting and CIP intrinsic-prior tests
+        # The CIP prior suite rides along here for the same reason: it is pure logic that
+        # runs in seconds (it reads CIP's source with ast instead of importing it, so it
+        # needs only numpy/scipy), and a wrong prior density or a mis-wired
+        # --eccentricity-prior coordinate raises nothing -- it merely reweights the
+        # posterior. Without a job invoking it the fix can regress behind green checks.
+        run: |
+          python -m pytest -q \
+            MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/test_probe_confirm.py \
+            MonteCarloMarginalizeCode/Code/test/test_cip_priors.py
 
   lisa-check:
     needs: install
@@ -341,6 +349,13 @@ jobs:
         # because this job is the only one matrixed over BOTH numpy lanes, and these tests
         # are the kind that break on numpy API removals (e.g. np.trapz -> np.trapezoid).
         run: python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_limit_cosine_samplers.py
+      - name: Run NAL / supplementary-likelihood hook tests
+        # Also run by .travis/test-integrate.sh (integration-check, py3.10). Repeated here for the
+        # same reason as above: nal_io is pure numpy and has a legacy-numpy fallback for
+        # default_rng, so it must be exercised in BOTH numpy lanes.
+        run: |
+          python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_nal_io.py \
+                             MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py
       - name: Run adaptive-volume collapse / evidence-accounting regressions
         # These were unguarded.  The AV live-volume-collapse family, the L0 rescue's warm
         # seed, and the portfolio fair-draw backend each shipped a regression suite that NO
@@ -353,7 +368,18 @@ jobs:
           python -m pytest -q \
             MonteCarloMarginalizeCode/Code/test/test_av_empty_live_volume.py \
             MonteCarloMarginalizeCode/Code/test/test_l0_rescue_seed.py \
+            MonteCarloMarginalizeCode/Code/test/test_seq_warmstart_seed.py \
+            MonteCarloMarginalizeCode/Code/test/test_fairdraw_double_weighting.py \
             MonteCarloMarginalizeCode/Code/test/test_portfolio_fairdraw_backend.py
+      - name: Audit _rvs consumers against the fair-draw rebind
+        # sampler._rvs is rebound to an EXPORT resample at the end of integrate_log, and five
+        # separate defects have come from a consumer reading it afterwards as though it were
+        # the sample set.  This does not assert that post-rebind reads are bugs -- most are
+        # legitimately per-row -- it asserts that every one of them carries a recorded human
+        # verdict, so a NEW or MOVED consumer fails the build instead of being found later by
+        # whoever happens to edit the surrounding code.
+        run: |
+          python MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/audit_rvs_fairdraw.py --check
       - name: Run test scripts
         run: |
           . .travis/test-coord.sh

.github/workflows/private-review-dispatch.yml

@@ -0,0 +1,109 @@
+name: Private upstream RIFT review dispatch
+
+on:
+  pull_request_target:
+    types:
+      - opened
+      - reopened
+      - ready_for_review
+      - synchronize
+      - converted_to_draft
+      - closed
+
+permissions:
+  id-token: write
+
+concurrency:
+  group: private-review-dispatch-upstream-rift-${{ github.event.pull_request.number }}
+  cancel-in-progress: true
+
+jobs:
+  dispatch:
+    if: >-
+      github.event.pull_request.user.login == 'oshaughnessy-junior' &&
+      github.event.pull_request.head.repo.owner.login == 'oshaughnessy-junior' &&
+      (github.event.pull_request.base.ref == 'rift_O4c' ||
+       github.event.pull_request.base.ref == 'master' ||
+       github.event.pull_request.base.ref == 'rift_O4d')
+    name: Dispatch exact upstream RIFT PR generation
+    runs-on: ubuntu-24.04
+    environment: private-review-dispatch-rift-upstream
+    timeout-minutes: 5
+    steps:
+      # SECURITY: pull_request_target runs trusted default-branch code. This
+      # workflow must never check out, fetch, cache, download, interpret, or
+      # execute pull-request-controlled content.
+      - name: Join review-dispatch tailnet segment
+        uses: tailscale/github-action@306e68a486fd2350f2bfc3b19fcd143891a4a2d8 # v4
+        with:
+          oauth-client-id: ${{ vars.TS_WIF_CLIENT_ID }}
+          audience: ${{ vars.TS_WIF_AUDIENCE }}
+          tags: tag:review-dispatcher
+          version: 1.98.10
+          ping: ${{ vars.REVIEW_COORDINATOR_HOST }}
+          use-cache: "false"
+
+      - name: Verify dispatcher cannot reach blocked coordinator ports
+        shell: bash
+        env:
+          REVIEW_COORDINATOR_HOST: ${{ vars.REVIEW_COORDINATOR_HOST }}
+        run: |
+          set -euo pipefail
+          if [[ ! "$REVIEW_COORDINATOR_HOST" =~ ^([a-z0-9-]+\.)*[a-z0-9-]+$ ]]; then
+            echo "Coordinator host variable is invalid" >&2
+            exit 1
+          fi
+          for port in 22 443 18789 3000; do
+            if timeout 3 bash -c 'exec 3<>"/dev/tcp/$1/$2"' \
+              _ "$REVIEW_COORDINATOR_HOST" "$port" 2>/dev/null; then
+              echo "Dispatcher unexpectedly reached blocked coordinator port $port" >&2
+              exit 1
+            fi
+          done
+
+      - name: Admit PR event through private coordinator
+        shell: bash
+        env:
+          DISPATCH_ACTION: ${{ github.event.action }}
+          DISPATCH_REPOSITORY: ${{ github.repository }}
+          DISPATCH_REPOSITORY_ID: ${{ github.repository_id }}
+          DISPATCH_PR: ${{ github.event.pull_request.number }}
+          DISPATCH_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+          DISPATCH_DRAFT: ${{ github.event.pull_request.draft }}
+          REVIEW_COORDINATOR_HOST: ${{ vars.REVIEW_COORDINATOR_HOST }}
+          REVIEW_COORDINATOR_PORT: ${{ vars.REVIEW_COORDINATOR_PORT }}
+          OIDC_AUDIENCE: interhost-cross-review-upstream-rift
+        run: |
+          set -euo pipefail
+          if [[ ! "$REVIEW_COORDINATOR_HOST" =~ ^([a-z0-9-]+\.)*[a-z0-9-]+$ ]] ||
+             [[ ! "$REVIEW_COORDINATOR_PORT" =~ ^[0-9]{2,5}$ ]]; then
+            echo "Coordinator address variables are invalid" >&2
+            exit 1
+          fi
+
+          oidc_response="$(curl --fail --silent --show-error \
+            --header "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
+            "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=${OIDC_AUDIENCE}")"
+          oidc_token="$(jq -er '.value | select(type == "string" and length > 0)' <<<"$oidc_response")"
+          echo "::add-mask::$oidc_token"
+
+          payload="$(jq -cn \
+            --arg schema "interhost-cross-review.github-actions-dispatch.v1" \
+            --arg action "$DISPATCH_ACTION" \
+            --arg repository "$DISPATCH_REPOSITORY" \
+            --arg repository_id "$DISPATCH_REPOSITORY_ID" \
+            --arg pr "$DISPATCH_PR" \
+            --arg head_sha "$DISPATCH_HEAD_SHA" \
+            --arg draft "$DISPATCH_DRAFT" \
+            '{schema:$schema, action:$action, repository:$repository,
+              repository_id:($repository_id | tonumber), pr:($pr | tonumber),
+              head_sha:$head_sha, draft:($draft == "true")}')"
+
+          curl --fail-with-body --silent --show-error \
+            --proto '=https' \
+            --retry 3 --retry-all-errors --retry-delay 2 \
+            --connect-timeout 10 --max-time 30 \
+            --header "Authorization: Bearer $oidc_token" \
+            --header "Content-Type: application/json" \
+            --data-binary "$payload" \
+            "https://${REVIEW_COORDINATOR_HOST}:${REVIEW_COORDINATOR_PORT}/github/actions/upstream-rift-dispatch"

.gitlab-ci.yml

@@ -125,6 +125,10 @@ integrator_gate_accounting_check:
 test_run:
   stage: system tests
   script:
+    # CIP prior densities + the --eccentricity-prior coordinate wiring (pure
+    # numpy/scipy, seconds). Runs before the heavy scripts so a wrong prior fails
+    # fast rather than after the end-to-end runs.
+    - python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_cip_priors.py
     - . .travis/test-coord.sh
     - bash .travis/test-integrate.sh
     - . .travis/test-posterior.sh

.travis/test-asimov.sh

@@ -5,4 +5,11 @@ set -euo pipefail
 # Asimov 0.5 series.  This test skips cleanly for unsupported/future series
 # from inside pytest, so developers can preflight 0.6/0.7 environments without
 # editing the test.
-python -m pytest -q MonteCarloMarginalizeCode/Code/test/asimov_integration
+# Bootstrap-source selection ("scheduler: bootstrap file:") is driven against a stub
+# production rather than a project on disk, so it lives outside asimov_integration/.
+# It still needs asimov importable, and this is the only lane that installs it, so run
+# it here: otherwise the suite is never invoked and the behaviour can regress while the
+# required checks stay green.
+python -m pytest -q \
+    MonteCarloMarginalizeCode/Code/test/asimov_integration \
+    MonteCarloMarginalizeCode/Code/test/test_asimov_bootstrap_source.py

.travis/test-integrate.sh

@@ -37,6 +37,13 @@ fi
 # coordinate-transform + prior-mass identities, so they belong with the integrator gate
 # rather than with the end-to-end run tests.
 python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_limit_cosine_samplers.py
+python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_mcsampler_ensemble_log_contract.py
+
+# Supplementary-likelihood plugin hook: the NAL reader/evaluator (pure numpy, no data) and the
+# static guard on the drivers' prepare-hook wiring, which is what makes the plugin receive the
+# SAMPLING basis at all. Both are seconds-long and protect a silent-wrong-answer path.
+python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_nal_io.py \
+                   MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py
 
 python MonteCarloMarginalizeCode/Code/test/test_mcsamplerEnsemble_extended.py --as-test --n-max 100000
 

CHANGES.rst

@@ -148,6 +148,43 @@ development tree is rift_O4d.
     (the flat-mode no-op master is preserved); ILE honours --srate-resample-time-marginalization instead of
     always doubling, and recovers the requested export rate EXACTLY rather than an integer multiple; Virgo
     calibration correction convention fixed; multi-GPU ILE fan-out with a multi-container example in one ini.
+  - CIP eccentricity priors: ``--eccentricity-prior log_uniform`` (ingested from rift_O4c
+    0.0.17.12, rapidpe-rift/rift!54) called ``np.ln``, which does not exist in numpy, so the
+    option raised AttributeError as soon as the prior was evaluated; its normalization was also
+    the uniform prior's, ``ln(ECC_MAX-ECC_MIN)``, where a density uniform in ln(e) needs
+    ``ln(ECC_MAX/ECC_MIN)`` (negative for the shipped 0.001/0.4 defaults).  Both fixed.  The
+    option also reached only the ``eccentricity`` coordinate, so a run sampling
+    ``eccentricity_squared`` -- which is what ``--use-eccentricity-squared`` asks for, and what
+    iteration 0 of an eccentric pseudo_pipe run uses -- silently kept the uniform-in-e^2
+    density; the log-uniform prior is now installed for that coordinate too (as the same
+    distribution written in e^2), and the ``--ecc-min 0`` floor correction now reaches its
+    range.  Separately, ``eccentricity_ln`` is a logarithmic coordinate under EVERY prior, so
+    the shipped ``--ecc-min`` default of 0.0 left a run sampling it with a ``[log(0), ...]``
+    range and a prior that divided by zero; the floor correction is now keyed on the sampled
+    coordinate as well as on the prior, while a run that neither asks for the log-uniform
+    prior nor samples a log coordinate keeps ``--ecc-min`` exactly as given.  Finally,
+    ``--eccentricity-prior`` is restricted to ``uniform``/``log_uniform`` in both CIP and
+    pseudo_pipe, which forwards the value verbatim: only ``log_uniform`` is branched on, so an
+    unsupported value ran the uniform prior while reporting the requested one.  With this, the
+    CIP prior densities get their first test suite: test_cip_priors.py
+    extracts the shipped ``def`` blocks from CIP with ast rather than transcribing them, then
+    checks that all 39 priors evaluate finite and non-negative, that the 20 claiming a
+    normalized density integrate to 1 against their stated measure, and that
+    ``--eccentricity-prior`` selects a normalized density for every eccentricity coordinate,
+    including at the CLI defaults read out of CIP's own argparse calls rather than assumed.
+    Also fixed on rift_O4c (PR #174).
+  - ILE portfolio driver: a portfolio the driver cannot build now FAILS instead of silently
+    integrating with a different sampler.  ``--sampler-method portfolio`` no longer carries a
+    dead ok-flag test that fell through to the plain mcsampler.MCSampler; the terminal
+    diagnostic and the plugin-pipeline branch no longer dereference mcsamplerPortfolio when its
+    import failed (that raised NameError from the diagnostic itself on a torch-free container);
+    an unrecognized ``--sampler-portfolio`` member and an omitted/empty member list are now
+    errors naming the known members.  Ported from rift_O4c (PR #172).
+  - asimov: the RIFT bootstrap source can be named explicitly (``scheduler: bootstrap file:``), bypassing
+    the dependency scan, with ``{event}``/``{analysis}`` substitution and single-match globbing; the
+    PESummary analysis label is auto-derived and an ambiguous or raw-bilby metafile now fails loudly
+    instead of silently bootstrapping from whichever label sorted first.  Ingested from rift_O4c 0.0.17.11
+    (rapidpe-rift/rift!53).
   - containers, docs, CI: container survey/warmup tooling (containers/survey_scan) with GPU-inventory
     profiles and tests; container canaries fixed after setuptools 84; an upstream dependency-compatibility
     check run on both GitLab and GitHub CI; new docs for distance-grid workflows, the demo catalog, the
@@ -164,13 +201,32 @@ development tree is rift_O4d.
     certify correctness; k-hat does not catch confidently-wrong runs from support mismatch; the L0 'doubles
     landed fraction' claim and the cap24 lnZ-bias claim are retracted).
 
+0.0.17.13
+---------
+MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/55  , for ln(e) parameter access in pipeline
+
+0.0.17.12
+---------
+MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/54  , for eccentricity prior (log-uniform)
+
+0.0.17.11
+---------
+MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/53  , so user can specify the bootstrap file in asimov yaml
+
+0.0.17.10
+------------
+MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/51 , https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/49
+so output sampling rate is precisely what is requested
+
 0.0.17.9
 ------------
 development tree is rift_O4c_staging -> rift_O4c; draft MR notes at
 https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/49
    - (rc0) write_bilby_pickle: shutil.copyfile threw error if cache file already existed (copy into same file error) in
      code that protected against duplicate IFO entries.
-
+   - (rc1) V calibration convention sign (rapidpe-rift/rift!50); multi-container capability; multi-GPU 'fanout' capability
+release is rc1
+
 
 0.0.17.8
 ------------