diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 000000000..b7d56ea87 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,37 @@ + + +## What this changes, and why + + + +## Evidence + + + +## Checklist + +- [ ] New/changed tests are **named in `.github/workflows/ci.yml`** (a test file existing does + not mean CI runs it — grep the workflow) +- [ ] Tests assert the precondition they are about, so they cannot pass vacuously +- [ ] **Read the diff — `git show HEAD` — and confirmed every hunk is mine.** These checkouts + are shared between concurrent sessions and `git add -A` has twice swept another branch's + uncommitted work into a commit. `--stat` does not answer this: it is blind to an + unrelated edit inside a file you also changed, which is exactly what happened both + times. (See `RIFT/integrators/REVIEW_CHECKLIST.md` §1.) +- [ ] Any changed default is justified by a measurement, and the previous behaviour is still + reachable by flag + +### If this touches the integrators, the ILE, the likelihood, any evidence/weight/gate, **or changes a default that production inherits** + +- [ ] Read `RIFT/integrators/REVIEW_CHECKLIST.md` §2–3 (population-vs-sample; side effects + when the feature is off) +- [ ] **Requested a full code review** — high blast radius: a wrong answer here is plausible, + not loud +- [ ] Shape-recovery merge gate run per `RIFT/integrators/TESTING.md` — required whenever the + change can move a posterior, which includes a changed default that production inherits, + not only edits under `RIFT/integrators/` (the fast CI integral test does not subsume it) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index de08d4934..d981f7ac0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -196,8 +196,16 @@ jobs: python -m pip install -r requirements.txt --break-system-packages python -m pip install coverage pytest --break-system-packages python -m pip install --editable . --break-system-packages - - name: Run probe confirm-on-fail accounting tests - run: python -m pytest -q MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/test_probe_confirm.py + - name: Run probe confirm-on-fail accounting and CIP intrinsic-prior tests + # The CIP prior suite rides along here for the same reason: it is pure logic that + # runs in seconds (it reads CIP's source with ast instead of importing it, so it + # needs only numpy/scipy), and a wrong prior density or a mis-wired + # --eccentricity-prior coordinate raises nothing -- it merely reweights the + # posterior. Without a job invoking it the fix can regress behind green checks. + run: | + python -m pytest -q \ + MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/test_probe_confirm.py \ + MonteCarloMarginalizeCode/Code/test/test_cip_priors.py lisa-check: needs: install @@ -341,6 +349,13 @@ jobs: # because this job is the only one matrixed over BOTH numpy lanes, and these tests # are the kind that break on numpy API removals (e.g. np.trapz -> np.trapezoid). run: python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_limit_cosine_samplers.py + - name: Run NAL / supplementary-likelihood hook tests + # Also run by .travis/test-integrate.sh (integration-check, py3.10). Repeated here for the + # same reason as above: nal_io is pure numpy and has a legacy-numpy fallback for + # default_rng, so it must be exercised in BOTH numpy lanes. + run: | + python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_nal_io.py \ + MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py - name: Run adaptive-volume collapse / evidence-accounting regressions # These were unguarded. The AV live-volume-collapse family, the L0 rescue's warm # seed, and the portfolio fair-draw backend each shipped a regression suite that NO @@ -353,7 +368,18 @@ jobs: python -m pytest -q \ MonteCarloMarginalizeCode/Code/test/test_av_empty_live_volume.py \ MonteCarloMarginalizeCode/Code/test/test_l0_rescue_seed.py \ + MonteCarloMarginalizeCode/Code/test/test_seq_warmstart_seed.py \ + MonteCarloMarginalizeCode/Code/test/test_fairdraw_double_weighting.py \ MonteCarloMarginalizeCode/Code/test/test_portfolio_fairdraw_backend.py + - name: Audit _rvs consumers against the fair-draw rebind + # sampler._rvs is rebound to an EXPORT resample at the end of integrate_log, and five + # separate defects have come from a consumer reading it afterwards as though it were + # the sample set. This does not assert that post-rebind reads are bugs -- most are + # legitimately per-row -- it asserts that every one of them carries a recorded human + # verdict, so a NEW or MOVED consumer fails the build instead of being found later by + # whoever happens to edit the surrounding code. + run: | + python MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/audit_rvs_fairdraw.py --check - name: Run test scripts run: | . .travis/test-coord.sh diff --git a/.github/workflows/private-review-dispatch.yml b/.github/workflows/private-review-dispatch.yml new file mode 100644 index 000000000..e5aa0c28a --- /dev/null +++ b/.github/workflows/private-review-dispatch.yml @@ -0,0 +1,109 @@ +name: Private upstream RIFT review dispatch + +on: + pull_request_target: + types: + - opened + - reopened + - ready_for_review + - synchronize + - converted_to_draft + - closed + +permissions: + id-token: write + +concurrency: + group: private-review-dispatch-upstream-rift-${{ github.event.pull_request.number }} + cancel-in-progress: true + +jobs: + dispatch: + if: >- + github.event.pull_request.user.login == 'oshaughnessy-junior' && + github.event.pull_request.head.repo.owner.login == 'oshaughnessy-junior' && + (github.event.pull_request.base.ref == 'rift_O4c' || + github.event.pull_request.base.ref == 'master' || + github.event.pull_request.base.ref == 'rift_O4d') + name: Dispatch exact upstream RIFT PR generation + runs-on: ubuntu-24.04 + environment: private-review-dispatch-rift-upstream + timeout-minutes: 5 + steps: + # SECURITY: pull_request_target runs trusted default-branch code. This + # workflow must never check out, fetch, cache, download, interpret, or + # execute pull-request-controlled content. + - name: Join review-dispatch tailnet segment + uses: tailscale/github-action@306e68a486fd2350f2bfc3b19fcd143891a4a2d8 # v4 + with: + oauth-client-id: ${{ vars.TS_WIF_CLIENT_ID }} + audience: ${{ vars.TS_WIF_AUDIENCE }} + tags: tag:review-dispatcher + version: 1.98.10 + ping: ${{ vars.REVIEW_COORDINATOR_HOST }} + use-cache: "false" + + - name: Verify dispatcher cannot reach blocked coordinator ports + shell: bash + env: + REVIEW_COORDINATOR_HOST: ${{ vars.REVIEW_COORDINATOR_HOST }} + run: | + set -euo pipefail + if [[ ! "$REVIEW_COORDINATOR_HOST" =~ ^([a-z0-9-]+\.)*[a-z0-9-]+$ ]]; then + echo "Coordinator host variable is invalid" >&2 + exit 1 + fi + for port in 22 443 18789 3000; do + if timeout 3 bash -c 'exec 3<>"/dev/tcp/$1/$2"' \ + _ "$REVIEW_COORDINATOR_HOST" "$port" 2>/dev/null; then + echo "Dispatcher unexpectedly reached blocked coordinator port $port" >&2 + exit 1 + fi + done + + - name: Admit PR event through private coordinator + shell: bash + env: + DISPATCH_ACTION: ${{ github.event.action }} + DISPATCH_REPOSITORY: ${{ github.repository }} + DISPATCH_REPOSITORY_ID: ${{ github.repository_id }} + DISPATCH_PR: ${{ github.event.pull_request.number }} + DISPATCH_HEAD_SHA: ${{ github.event.pull_request.head.sha }} + DISPATCH_DRAFT: ${{ github.event.pull_request.draft }} + REVIEW_COORDINATOR_HOST: ${{ vars.REVIEW_COORDINATOR_HOST }} + REVIEW_COORDINATOR_PORT: ${{ vars.REVIEW_COORDINATOR_PORT }} + OIDC_AUDIENCE: interhost-cross-review-upstream-rift + run: | + set -euo pipefail + if [[ ! "$REVIEW_COORDINATOR_HOST" =~ ^([a-z0-9-]+\.)*[a-z0-9-]+$ ]] || + [[ ! "$REVIEW_COORDINATOR_PORT" =~ ^[0-9]{2,5}$ ]]; then + echo "Coordinator address variables are invalid" >&2 + exit 1 + fi + + oidc_response="$(curl --fail --silent --show-error \ + --header "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ + "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=${OIDC_AUDIENCE}")" + oidc_token="$(jq -er '.value | select(type == "string" and length > 0)' <<<"$oidc_response")" + echo "::add-mask::$oidc_token" + + payload="$(jq -cn \ + --arg schema "interhost-cross-review.github-actions-dispatch.v1" \ + --arg action "$DISPATCH_ACTION" \ + --arg repository "$DISPATCH_REPOSITORY" \ + --arg repository_id "$DISPATCH_REPOSITORY_ID" \ + --arg pr "$DISPATCH_PR" \ + --arg head_sha "$DISPATCH_HEAD_SHA" \ + --arg draft "$DISPATCH_DRAFT" \ + '{schema:$schema, action:$action, repository:$repository, + repository_id:($repository_id | tonumber), pr:($pr | tonumber), + head_sha:$head_sha, draft:($draft == "true")}')" + + curl --fail-with-body --silent --show-error \ + --proto '=https' \ + --retry 3 --retry-all-errors --retry-delay 2 \ + --connect-timeout 10 --max-time 30 \ + --header "Authorization: Bearer $oidc_token" \ + --header "Content-Type: application/json" \ + --data-binary "$payload" \ + "https://${REVIEW_COORDINATOR_HOST}:${REVIEW_COORDINATOR_PORT}/github/actions/upstream-rift-dispatch" diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 78a887b9f..43c660f20 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -125,6 +125,10 @@ integrator_gate_accounting_check: test_run: stage: system tests script: + # CIP prior densities + the --eccentricity-prior coordinate wiring (pure + # numpy/scipy, seconds). Runs before the heavy scripts so a wrong prior fails + # fast rather than after the end-to-end runs. + - python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_cip_priors.py - . .travis/test-coord.sh - bash .travis/test-integrate.sh - . .travis/test-posterior.sh diff --git a/.travis/test-asimov.sh b/.travis/test-asimov.sh index 9a89ddd95..4a1fda9ef 100644 --- a/.travis/test-asimov.sh +++ b/.travis/test-asimov.sh @@ -5,4 +5,11 @@ set -euo pipefail # Asimov 0.5 series. This test skips cleanly for unsupported/future series # from inside pytest, so developers can preflight 0.6/0.7 environments without # editing the test. -python -m pytest -q MonteCarloMarginalizeCode/Code/test/asimov_integration +# Bootstrap-source selection ("scheduler: bootstrap file:") is driven against a stub +# production rather than a project on disk, so it lives outside asimov_integration/. +# It still needs asimov importable, and this is the only lane that installs it, so run +# it here: otherwise the suite is never invoked and the behaviour can regress while the +# required checks stay green. +python -m pytest -q \ + MonteCarloMarginalizeCode/Code/test/asimov_integration \ + MonteCarloMarginalizeCode/Code/test/test_asimov_bootstrap_source.py diff --git a/.travis/test-integrate.sh b/.travis/test-integrate.sh index ebeaf3f21..51cdd1b26 100755 --- a/.travis/test-integrate.sh +++ b/.travis/test-integrate.sh @@ -37,6 +37,13 @@ fi # coordinate-transform + prior-mass identities, so they belong with the integrator gate # rather than with the end-to-end run tests. python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_limit_cosine_samplers.py +python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_mcsampler_ensemble_log_contract.py + +# Supplementary-likelihood plugin hook: the NAL reader/evaluator (pure numpy, no data) and the +# static guard on the drivers' prepare-hook wiring, which is what makes the plugin receive the +# SAMPLING basis at all. Both are seconds-long and protect a silent-wrong-answer path. +python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_nal_io.py \ + MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py python MonteCarloMarginalizeCode/Code/test/test_mcsamplerEnsemble_extended.py --as-test --n-max 100000 diff --git a/CHANGES.rst b/CHANGES.rst index ca7751a71..efbe8b7be 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -148,6 +148,43 @@ development tree is rift_O4d. (the flat-mode no-op master is preserved); ILE honours --srate-resample-time-marginalization instead of always doubling, and recovers the requested export rate EXACTLY rather than an integer multiple; Virgo calibration correction convention fixed; multi-GPU ILE fan-out with a multi-container example in one ini. + - CIP eccentricity priors: ``--eccentricity-prior log_uniform`` (ingested from rift_O4c + 0.0.17.12, rapidpe-rift/rift!54) called ``np.ln``, which does not exist in numpy, so the + option raised AttributeError as soon as the prior was evaluated; its normalization was also + the uniform prior's, ``ln(ECC_MAX-ECC_MIN)``, where a density uniform in ln(e) needs + ``ln(ECC_MAX/ECC_MIN)`` (negative for the shipped 0.001/0.4 defaults). Both fixed. The + option also reached only the ``eccentricity`` coordinate, so a run sampling + ``eccentricity_squared`` -- which is what ``--use-eccentricity-squared`` asks for, and what + iteration 0 of an eccentric pseudo_pipe run uses -- silently kept the uniform-in-e^2 + density; the log-uniform prior is now installed for that coordinate too (as the same + distribution written in e^2), and the ``--ecc-min 0`` floor correction now reaches its + range. Separately, ``eccentricity_ln`` is a logarithmic coordinate under EVERY prior, so + the shipped ``--ecc-min`` default of 0.0 left a run sampling it with a ``[log(0), ...]`` + range and a prior that divided by zero; the floor correction is now keyed on the sampled + coordinate as well as on the prior, while a run that neither asks for the log-uniform + prior nor samples a log coordinate keeps ``--ecc-min`` exactly as given. Finally, + ``--eccentricity-prior`` is restricted to ``uniform``/``log_uniform`` in both CIP and + pseudo_pipe, which forwards the value verbatim: only ``log_uniform`` is branched on, so an + unsupported value ran the uniform prior while reporting the requested one. With this, the + CIP prior densities get their first test suite: test_cip_priors.py + extracts the shipped ``def`` blocks from CIP with ast rather than transcribing them, then + checks that all 39 priors evaluate finite and non-negative, that the 20 claiming a + normalized density integrate to 1 against their stated measure, and that + ``--eccentricity-prior`` selects a normalized density for every eccentricity coordinate, + including at the CLI defaults read out of CIP's own argparse calls rather than assumed. + Also fixed on rift_O4c (PR #174). + - ILE portfolio driver: a portfolio the driver cannot build now FAILS instead of silently + integrating with a different sampler. ``--sampler-method portfolio`` no longer carries a + dead ok-flag test that fell through to the plain mcsampler.MCSampler; the terminal + diagnostic and the plugin-pipeline branch no longer dereference mcsamplerPortfolio when its + import failed (that raised NameError from the diagnostic itself on a torch-free container); + an unrecognized ``--sampler-portfolio`` member and an omitted/empty member list are now + errors naming the known members. Ported from rift_O4c (PR #172). + - asimov: the RIFT bootstrap source can be named explicitly (``scheduler: bootstrap file:``), bypassing + the dependency scan, with ``{event}``/``{analysis}`` substitution and single-match globbing; the + PESummary analysis label is auto-derived and an ambiguous or raw-bilby metafile now fails loudly + instead of silently bootstrapping from whichever label sorted first. Ingested from rift_O4c 0.0.17.11 + (rapidpe-rift/rift!53). - containers, docs, CI: container survey/warmup tooling (containers/survey_scan) with GPU-inventory profiles and tests; container canaries fixed after setuptools 84; an upstream dependency-compatibility check run on both GitLab and GitHub CI; new docs for distance-grid workflows, the demo catalog, the @@ -164,13 +201,32 @@ development tree is rift_O4d. certify correctness; k-hat does not catch confidently-wrong runs from support mismatch; the L0 'doubles landed fraction' claim and the cap24 lnZ-bias claim are retracted). +0.0.17.13 +--------- +MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/55 , for ln(e) parameter access in pipeline + +0.0.17.12 +--------- +MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/54 , for eccentricity prior (log-uniform) + +0.0.17.11 +--------- +MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/53 , so user can specify the bootstrap file in asimov yaml + +0.0.17.10 +------------ +MR https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/51 , https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/49 +so output sampling rate is precisely what is requested + 0.0.17.9 ------------ development tree is rift_O4c_staging -> rift_O4c; draft MR notes at https://git.ligo.org/rapidpe-rift/rift/-/merge_requests/49 - (rc0) write_bilby_pickle: shutil.copyfile threw error if cache file already existed (copy into same file error) in code that protected against duplicate IFO entries. - + - (rc1) V calibration convention sign (rapidpe-rift/rift!50); multi-container capability; multi-GPU 'fanout' capability +release is rc1 + 0.0.17.8 ------------ diff --git a/MonteCarloMarginalizeCode/Code/RIFT/asimov/rift.py b/MonteCarloMarginalizeCode/Code/RIFT/asimov/rift.py index 9036ca652..b373aa84c 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/asimov/rift.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/asimov/rift.py @@ -78,10 +78,115 @@ def _create_ledger_entries(self): for section_arg in required_args[section]: if section_arg not in section_data: section_data[section_arg] = {} + # Top-level groups a PESummary metafile carries that are not analysis labels + _PESUMMARY_RESERVED = ('version', 'history') + + def _resolve_bootstrap_file(self): + """ + Resolve an explicitly-specified bootstrap posterior file, if any. + + Set ``scheduler: bootstrap file:`` to point RIFT straight at a + PESummary metafile, bypassing the dependency scan entirely. Use this + when the file is already on disk and you do not want to express it as + an asimov dependency - notably when bootstrapping from an offline PE + run that RIFT does not otherwise need to wait for. + + The string may contain ``{event}``/```` and + ``{analysis}``/````, and may contain shell wildcards, in + which case exactly one match is required. + + Returns None if the setting is absent. Raises PipelineException if it + is present but does not resolve to exactly one existing file - an + explicit request that cannot be honoured must never fall back silently. + """ + template = self.production.meta.get('scheduler', {}).get('bootstrap file') + if not template: + return None + + for token, value in (('event', self.production.event.name), + ('analysis', self.production.name)): + template = template.replace('{%s}' % token, value) + template = template.replace('<%s>' % token, value) + + if any(ch in template for ch in '*?['): + import glob as _glob + matches = sorted(_glob.glob(template)) + if len(matches) != 1: + raise PipelineException( + "RIFT bootstrap: 'bootstrap file' pattern {} matched {} files, " + "need exactly 1: {}".format(template, len(matches), matches), + production=self.production.name) + template = matches[0] + + if not os.path.exists(template): + raise PipelineException( + "RIFT bootstrap: 'bootstrap file' {} does not exist".format(template), + production=self.production.name) + + self.logger.info("RIFT bootstrap: using explicit file {}".format(template)) + return template + + def _dataset_label(self, posterior_file): + """ + The PESummary analysis label to read out of ``posterior_file``. + + An explicit ``dataset:`` is returned as-is without opening the file, + preserving the previous ``if "dataset" not in self.production.meta`` + short-circuit: existing ledgers that pin a dataset must keep building + even if the source metafile has since moved or become unreadable, as + long as the bootstrap grid itself is already present. + + Otherwise auto-derive, requiring exactly one candidate so an ambiguous + metafile fails here rather than by silently bootstrapping from whichever + label sorted first. + """ + requested = self.production.meta.get('dataset') + if requested: + return requested + + import h5py + + with h5py.File(posterior_file, 'r') as handle: + keys = list(handle.keys()) + # A PESummary analysis label is a group holding the samples; other + # root-level entries (metadata, and the raw-bilby layout below) are + # not candidates. + labels = [k for k in keys + if k not in self._PESUMMARY_RESERVED + and hasattr(handle[k], 'keys') + and ('posterior_samples' in handle[k] or 'posterior' in handle[k])] + root_is_samples = 'posterior' in keys and not labels + + if root_is_samples: + # A raw bilby result file: samples live at the root rather than + # under an analysis label, so the PESummary reader cannot consume + # it. Say so, instead of raising 'Unknown key in file'. + raise PipelineException( + "RIFT bootstrap: {} looks like a raw bilby result file, not a " + "PESummary metafile. Point 'bootstrap file' at the PESummary " + "output (.../pesummary/samples/posterior_samples.h5).".format( + posterior_file), + production=self.production.name) + + if len(labels) != 1: + raise PipelineException( + "RIFT bootstrap: {} has {} analysis labels ({}); set 'dataset:' " + "to choose one".format(posterior_file, len(labels), labels), + production=self.production.name) + return labels[0] + def _find_posterior(self): """ Find the input posterior samples. + + An explicit ``scheduler: bootstrap file:`` wins; otherwise fall back to + scanning dependencies for the first that publishes a 'samples' asset. """ + posterior_file = self._resolve_bootstrap_file() + if posterior_file: + self.production.meta['dataset'] = self._dataset_label(posterior_file) + return posterior_file + if self.production.dependencies: productions = {} for production in self.production.event.productions: @@ -91,16 +196,18 @@ def _find_posterior(self): try: if "samples" in productions[previous_job].pipeline.collect_assets(): posterior_file = productions[previous_job].pipeline.collect_assets()['samples'] - if "dataset" not in self.production.meta: - import h5py - with h5py.File(posterior_file,'r') as f: - keys = list(f.keys()) - keys.remove('version') - keys.remove('history') - self.production.meta['dataset'] = keys[0] + self.production.meta['dataset'] = self._dataset_label(posterior_file) return posterior_file - except Exception: - pass + except PipelineException: + raise + except Exception as exc: + # Historically silent. A dependency that publishes 'samples' + # in a form we cannot read (e.g. the bilby pipeline, which + # returns a *list* of raw result files) used to leave the + # run with no bootstrap and no message. + self.logger.warning( + "RIFT bootstrap: could not use samples from {}: {}".format( + previous_job, exc)) else: self.logger.error("Could not find an analysis providing posterior samples to analyse.") @@ -367,8 +474,10 @@ def build_dag(self, user=None, dryrun=False): if self.production.meta["waveform"]["non-spin"]: command += ["--assume-nospin"] - # Generate initial samples, based on previous PE results - if 'bootstrap upstream' in self.production.meta['scheduler']: + # Generate initial samples, based on previous PE results (or an + # explicitly-specified posterior, via scheduler: bootstrap file:) + if ('bootstrap upstream' in self.production.meta['scheduler'] + or 'bootstrap file' in self.production.meta['scheduler']): # get posterior file posterior_file = self._find_posterior() self.logger.info(" Bootstrap requested, attempting with file {}".format(posterior_file) ) @@ -381,6 +490,13 @@ def build_dag(self, user=None, dryrun=False): ) bootstrap_file_ascii = str(bootstrap_file) + "_ascii" # test if bootstrap file already exists + if os.path.exists(bootstrap_file): + # Rebuilding an analysis under the same name reuses this + # silently, so a changed bootstrap source has no effect. + self.logger.warning( + "RIFT bootstrap: reusing existing grid {} and IGNORING {}; " + "delete it (and its _ascii) to rebuild".format( + bootstrap_file, posterior_file)) if not(os.path.exists(bootstrap_file)): import RIFT.misc.samples_utils RIFT.misc.samples_utils.dump_pesummary_samples_to_file_as_rift(posterior_file, self.production.meta['dataset'], bootstrap_file_ascii) diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/REVIEW_CHECKLIST.md b/MonteCarloMarginalizeCode/Code/RIFT/integrators/REVIEW_CHECKLIST.md new file mode 100644 index 000000000..0f56e4c14 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/REVIEW_CHECKLIST.md @@ -0,0 +1,143 @@ +# Pre-merge review checklist for sampler / estimator changes + +Companion to `TESTING.md` (which owns the expensive shape-recovery merge gate). This file is +the **cheap** pass: things to check by reading, before a PR is opened, that have repeatedly +survived CI, a green test suite, and a successful GPU run. + +Why it exists: two consecutive integrator PRs each needed **four review rounds**, and in both +cases every finding was the same shape. + +* **`oshaughnessy-junior#63`** (AV collapse reporting) — three findings that were all *"a status changes after + the thing consuming it has already run"*: the caller never read the verdict; the empty-cycle + test used the cumulative count; replica status was discarded and the reject gate checked the + pre-replication value. +* **`oshaughnessy-junior#78` / `#84`** (L0 rescue warm seed) — four findings that were all *"an array or a count + looks like the population but is a filtered or resampled subset of it"*: the seed read the + fair-draw export resample; the reject gate differenced two differently-resampled lnZ values; + the reserve's cap kept `-inf`-dominated rows in proportion (2 of 10 finite rows survived) + and consumed the global RNG; and the capped estimator's *logarithm* carried sampling error + though its linear total was unbiased. + +Every one of those returns a plausible-looking number when it is wrong. None of them crashes. + +--- + +## 1. Always, and cheap + +- [ ] **Is the new test file named in `.github/workflows/ci.yml`?** The workflow lists test + files individually, so a test file existing does not mean CI runs it. **Grep the + workflow for the filename** — do not rely on any coverage claim, here or elsewhere, + including this one; the routing drifts. + + *History, as the reason this item exists:* the AV collapse-detection suites + (`test_av_empty_live_volume.py`, `test_l0_rescue_seed.py`, + `test_portfolio_fairdraw_backend.py`) were reachable from no CI job at all until + 2026-08-12, one of them having shipped with an already-merged PR. They are routed now, + in the numpy-matrix sampler job. +- [ ] **Does a test assert the precondition it is about?** For a degenerate-regime fix, a test + that silently runs in the healthy regime passes vacuously. Open with the assertion — + e.g. `assert r['n_finite'] < r['n_retained'], 'this target did not underflow'`. +- [ ] **Does the diff contain only your hunks?** Read the diff — `git show HEAD` — before + pushing, or `git diff --cached` before committing. Several checkouts here are shared + between concurrent sessions, and `git add -A` has twice swept another branch's + uncommitted work into a commit here. + + **`--stat` does not answer this question and is not a cheaper way to.** It reports + filenames and line counts, so it catches an unexpected *file* while being blind to an + unrelated edit *inside a file you also changed* — which is the case that actually + occurred, both times, because the swept-in work was in `mcsamplerPortfolio.py` and the + commit legitimately touched `mcsamplerPortfolio.py`. `--stat` showed one expected + filename and a slightly larger line count. Nor is it a useful companion: `git show` + already names every file in its diff headers, so running `--stat` first adds a step and + an opportunity to stop early. +- [ ] **If a default changed, is the previous behaviour still reachable by flag?** and is the + new default's justification a measurement, not a preference? + +## 2. The population-vs-sample invariants + +Ask these of every array and every count the change reads: + +- [ ] **Is this the population, or a sample of it? What is the denominator?** An importance + estimate is `mean(w)` over the draws that were *made*; a draw whose likelihood + underflowed to `-inf` is a real draw contributing a real zero. Filtering those out and + then averaging divides by the wrong `n`. +- [ ] **`sampler._rvs` may be the fair-draw EXPORT resample rather than the sample set — check + which.** Regenerate the list rather than trusting the one below, which has already been + wrong once by omission: + + grep -l 'bFairdraw and not(n_extr is None)' mcsampler*.py # who rebinds + grep -l '_warm_seed_reserve' mcsampler*.py # who keeps the draws + + As of 2026-08-13 the first returns **all six** samplers — `mcsampler`, `mcsamplerGPU`, + `mcsamplerEnsemble`, `mcsamplerNFlow`, `mcsamplerAdaptiveVolume`, `mcsamplerPortfolio`. + Each rebinds `_rvs` **only** when all three hold: + + bFairdraw # kwargs['igrand_fairdraw_samples'] + and n_extr is not None # kwargs['igrand_fairdraw_samples_max'] + and n_extr < len(retained) # n_extr = min(n_extr, 1.5*eff_samp, 1.5*neff) + + Otherwise `_rvs` is still the retained set and reading it is correct. When it *does* + fire, the rows are drawn **with replacement, proportional to weight**, so on a collapsed + pass it is *one row* while the live set held 1000 — and duplicates are why a "2-point" + cloud can have affine rank 0. The ILE sets both kwargs from + `--fairdraw-extrinsic-output` / `--fairdraw-extrinsic-output-n-max` (see the + `igrand_fairdraw_samples` entries in `integrate_likelihood_extrinsic_batchmode`), so on + the extrinsic-export path assume it fired unless you have checked. + **Do not infer the condition from the sampler class; find the caller's kwargs.** +- [ ] **If you need the draws, is a pre-fair-draw record actually available for this sampler?** + `_warm_seed_reserve` (snapshotted before the fair draw, carrying `n_retained`, + `n_finite` and the exact `ln_sum_w_finite`) is **not** a generic sampler facility. The + second grep above returns only **`mcsamplerAdaptiveVolume` and `mcsamplerPortfolio`**; + `mcsampler`, `mcsamplerGPU`, `mcsamplerEnsemble` and `mcsamplerNFlow` have no + equivalent. Code that must work across samplers needs its own fallback, and should + report which reading it used rather than leaving it inferable. +- [ ] **Is the code path reachable at all?** `mcsamplerPortfolio` drives its members through + `draw_simplified()`, never their `integrate_log()`, so a member never executes anything + that lives there. Name the caller before assuming a branch runs. +- [ ] **Is the *logarithm* of the estimator unbiased, or only the linear total?** A uniformly + capped (Horvitz–Thompson) sum is unbiased in `Z` and biased in `ln Z`, and every gate + here compares logarithms against a nats threshold. Where an exact total is available, + record it rather than re-estimating it. +- [ ] **Does an error cancel between the two things being compared?** It does not if the two + passes have different `eff_samp`, different finite fractions, or different subsample + sizes — which, on the paths that trigger a rescue, they always do. +- [ ] **Is a status written before the thing that consumes it runs?** (the collapse-reporting + shape above.) + +## 3. Side effects when the feature is OFF + +- [ ] **Does the change consume the global RNG?** Anything built unconditionally must not + advance `np.random` — it moves the fair draw, the exported posterior, and every later + event and replica, so an opt-in feature that is switched off changes a seeded run. + Use a private `RandomState`. +- [ ] **Does it mutate shared state in place** (`_rvs`, a member's grid, a cached proposal) + that a later point or replica will read? +- [ ] **Is per-point state cleared on ENTRY**, so "present" means "this pass wrote it" rather + than "the previous point left it"? + +## 4. Blast radius — when to spawn a full code review + +Run the cheap list above on everything. **Additionally request a full code review** (the +`code-review` skill, or `/code-review ultra` for a multi-agent pass) when the diff touches: + +* anything under `RIFT/integrators/`; +* `bin/integrate_likelihood_extrinsic_batchmode` or the other ILE entry points; +* `RIFT/likelihood/factored_likelihood.py` or the CUDA kernels; +* evidence, weights, normalization, `n_eff`/ESS, or any collapse/rejection gate; +* any changed default that production configurations inherit. + +These are the paths where a wrong answer is *plausible* rather than *loud*. The expensive +shape-recovery gate in `TESTING.md` is still required for them — it catches a different class +(posterior shape wrong while the integral is right) and does not subsume this list. + +## 5. What does not count as verification + +A single run that completed and looks sane is not evidence about the regime a fix targets. +These integrators degrade continuously: a run in the healthy part of the range executes the +same lines and prints the same-looking diagnostics as one in the failing part. Measured +example — a GPU replicate quoted as confirming the warm-seed fix had 239 seed points from a +20001-row reserve, i.e. it never entered the collapsed regime, while the same code kept **2 of +10** finite rows when finite rows were rare. + +State which regime the verification run was in, and prefer a replicate campaign over a single +run wherever the failure is a lottery (as it is above rho_net ~ 100). diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/TESTING.md b/MonteCarloMarginalizeCode/Code/RIFT/integrators/TESTING.md index 7e5e909f8..b0ac5cedf 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/TESTING.md +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/TESTING.md @@ -1,5 +1,10 @@ # Before merging changes to this directory +> See also **`REVIEW_CHECKLIST.md`** in this directory — the cheap, read-it-yourself pass, and +> the rule for when a change's blast radius warrants a full code review. It catches a +> different class from the gate below: defects that leave the integral, the posterior shape +> AND the diagnostics all looking reasonable while a count or a normalization is wrong. + **Any PR that touches the integrators must pass the posterior SHAPE-recovery merge gate**, not just the fast CI integral test: diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsampler.py b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsampler.py index 56c20586a..2f01140dd 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsampler.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsampler.py @@ -447,6 +447,11 @@ def integrate(self, func, *args, **kwargs): deltalnL = kwargs['igrand_threshold_deltalnL'] if 'igrand_threshold_deltalnL' in kwargs else float("Inf") # default is to return all deltaP = kwargs["igrand_threshold_p"] if 'igrand_threshold_p' in kwargs else 0 # default is to omit 1e-7 of probability bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None tripwire_fraction = kwargs["tripwire_fraction"] if "tripwire_fraction" in kwargs else 2 # make it impossible to trigger @@ -792,6 +797,7 @@ def integrate(self, func, *args, **kwargs): else: self._rvs[key] = self._rvs[key][indx_list] + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w # Create extra dictionary to return things dict_return ={} if convergence_tests is not None: diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerAdaptiveVolume.py b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerAdaptiveVolume.py index ac367d1f7..865a09917 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerAdaptiveVolume.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerAdaptiveVolume.py @@ -389,6 +389,80 @@ def make_warm_seed_reserve(X, lnL, params_ordered, n_max=20000, return out +def lnZ_from_reserve(reserve): + """lnZ implied by a warm-seed reserve, on the ORIGINAL PROPOSAL-DRAW normalization. + + -> float, or None if the reserve cannot support the estimate. + + THE DENOMINATOR IS THE POINT. An importance estimate is mean(w) over the draws that were + MADE, and a draw whose likelihood underflowed to -inf is a real draw contributing a real + zero. make_warm_seed_reserve drops those rows -- they are ballast for every other + consumer -- and may then cap what survives, so the stored array is neither the draw set + nor a uniform sample of it. Averaging over the stored rows silently divides by the wrong + n, overestimating by log(n_retained/n_finite). For a PORTFOLIO, whose _rvs holds every + draw and whose finite fraction on a collapsed pass is ~1e-5, that is ~11 nats. + + It does not cancel in the L0 reject gate, which is what makes it dangerous rather than + merely wrong: the cold and warm passes have DIFFERENT finite fractions, so the gate would + see the difference of two different-sized errors and could again keep a mass-losing warm + pass or discard a good one -- the failure the gate change exists to remove, reappearing + one layer down. + + So rebuild the estimate explicitly. With `m` rows kept, out of `n_finite` finite draws, + out of `n_retained` draws made: + + sum over the finite population ~= (n_finite / m) * sum over kept rows + lnZ = logsumexp(lw_kept) + log(n_finite) - log(m) - log(n_retained) + + which collapses to logsumexp(lw) - log(m) exactly when nothing was dropped and nothing + was capped -- i.e. for AV, whose retained set is already all finite, leaving its + behaviour unchanged. + + One wrinkle, disclosed rather than corrected: make_warm_seed_reserve force-appends the + peak row, so the kept rows are very slightly peak-biased rather than uniform. At one row + in n_max (default 20000) that sits far below the MC error on either pass, and correcting + it exactly would mean tracking whether the peak was already in the subsample. + """ + if not isinstance(reserve, dict): + return None + # EXACT PATH. make_warm_seed_reserve records the finite-population weight total before + # the cap, precisely so this does not have to be estimated from the subsample. Prefer it + # whenever it is there: the estimate below is a Horvitz-Thompson realization whose + # LOGARITHM carries sampling error, and this gate is a comparison of logarithms against a + # 0.5-nat threshold. See the cap discussion in make_warm_seed_reserve. + _exact = reserve.get('ln_sum_w_finite') + _n_ret_exact = reserve.get('n_retained') + if _exact is not None and _n_ret_exact: + try: + if np.isfinite(float(_exact)) and float(_n_ret_exact) > 0: + return float(float(_exact) - np.log(float(_n_ret_exact))) + except (TypeError, ValueError): + pass + # FALLBACK: a reserve built without the prior components, or by an older writer. Scale + # the kept rows back up to the finite population and then to the draws made -- unbiased + # in the linear total, but exposed to the cap sampling error described above. + try: + lnL = np.asarray(reserve['lnL'], dtype=float).ravel() + lp = np.asarray(reserve['log_joint_prior'], dtype=float).ravel() + ls = np.asarray(reserve['log_joint_s_prior'], dtype=float).ravel() + except (KeyError, TypeError, ValueError): + return None + m = len(lnL) + if m == 0 or len(lp) != m or len(ls) != m: + return None + lw = lnL + lp - ls + lw = lw[np.isfinite(lw)] + if lw.size == 0: + return None + n_fin = float(reserve.get('n_finite', m) or m) + n_ret = float(reserve.get('n_retained', n_fin) or n_fin) + if not (n_fin > 0 and n_ret > 0): + return None + mx = np.max(lw) + tot = mx + np.log(np.sum(np.exp(lw - mx))) + return float(tot + np.log(n_fin) - np.log(float(m)) - np.log(n_ret)) + + def warm_seed_scale_from_finite_points(points, lnL, box_lo, box_hi, axes, eig_lo=1e-5, eig_hi=0.5): """Estimate the POSTERIOR scale (a box-scaled covariance over `axes`) from the finite @@ -1496,6 +1570,11 @@ def integrate_log(self, lnF, *args, xpy=xpy_default,**kwargs): deltalnL = kwargs['igrand_threshold_deltalnL'] if 'igrand_threshold_deltalnL' in kwargs else float("Inf") # default is to return all deltaP = kwargs["igrand_threshold_p"] if 'igrand_threshold_p' in kwargs else 0 # default is to omit 1e-7 of probability bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None bShowEvaluationLog = kwargs['verbose'] if 'verbose' in kwargs else False @@ -1806,6 +1885,11 @@ def _eval_integrand(samples): # Bounded, because this is the array the surrounding code calls a memory hog: a # uniform subsample without replacement (plus the peak row, which the seed needs and # a subsample can drop) is all a seed or a scale estimate can use. + # The two PRIOR components ride along as well, so a consumer can rebuild the + # importance weight -- and therefore lnZ -- from the retained set. Without them the + # only reconstructable lnZ is the one from the fair-drawn _rvs, and on this pass that + # is a SINGLE row drawn proportional to weight, i.e. the largest weight rather than + # the mean: an estimate high by ~log(n_retained / eff_samp), about 7 nats here. try: self._warm_seed_reserve = make_warm_seed_reserve( allx, allloglkl - allp, self.params_ordered, @@ -1868,6 +1952,7 @@ def _eval_integrand(samples): self._rvs[key] = arr[indx_host] + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w # perform type conversion of all stored variables. VERY LARGE -- should only do this if we need it! if cupy_ok: for name in self._rvs: diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerEnsemble.py b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerEnsemble.py index 780cbd1b6..a590caa4e 100755 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerEnsemble.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerEnsemble.py @@ -30,6 +30,7 @@ import functools import scipy.special +import scipy.stats as stats #from statutils import cumvar @@ -588,7 +589,7 @@ def integrate_log(self, func, *args,**kwargs): args_passed.update(kwargs) args_passed['use_lnL']=True args_passed['return_lnI']=True - return integrate(func, *args, args_passed) + return self.integrate(func, *args, **args_passed) def integrate(self, func, *args,**kwargs): nmax = kwargs["nmax"] if "nmax" in kwargs else 1e6 @@ -636,6 +637,11 @@ def integrate(self, func, *args,**kwargs): return_lnI = kwargs["return_lnI"] if "return_lnI" in kwargs else False bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None self.func = func @@ -726,6 +732,16 @@ def integrate(self, func, *args,**kwargs): # Store sample history on the host so downstream (CPU) consumers -- # weights, CDFs, posterior plots -- work regardless of backend. + # A sampler can be reused across log- and linear-space integrations. + # Remove mode-specific results from the previous run before exporting + # this one so convergence checks cannot consume stale log weights. + for key in ( + 'log_integrand', + 'log_joint_prior', + 'log_joint_s_prior', + 'log_weights', + ): + self._rvs.pop(key, None) index = 0 for param in args: self._rvs[param] = self.identity_convert(sample_array[:,index]) @@ -733,6 +749,18 @@ def integrate(self, func, *args,**kwargs): self._rvs['joint_prior'] = self.identity_convert(prior_array) self._rvs['joint_s_prior'] = self.identity_convert(p_array) self._rvs['integrand'] = self.identity_convert(value_array) + if use_lnL: + # Preserve the historical ``integrand`` alias for direct callers, + # while exposing an unambiguous log-space contract to downstream + # consumers. In log mode ``value_array`` is already ln(L). + self._rvs['log_integrand'] = self.identity_convert(integrator.cumulative_values) + self._rvs['log_joint_prior'] = self.identity_convert(self.xpy.log(prior_array)) + self._rvs['log_joint_s_prior'] = self.identity_convert(self.xpy.log(p_array)) + self._rvs['log_weights'] = self.identity_convert( + integrator.cumulative_values + + self.xpy.log(prior_array) + - self.xpy.log(p_array) + ) if bFairdraw and not(n_extr is None): # scalars: use Python min on floats. self.xpy.min([list]) fails on cupy @@ -755,6 +783,7 @@ def integrate(self, func, *args,**kwargs): else: self._rvs[key] = self._rvs[key][indx_list] + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w dict_return = {} if dict_return_q: dict_return["integrator"] = integrator @@ -920,17 +949,29 @@ def sanityCheckSamplerIntegrateUnity(sampler,*args,**kwargs): return sampler.integrate(lambda *args: 1,*args,**kwargs) def convergence_test_MostSignificantPoint(pcut, rvs, params): - weights = rvs["weights"] + if "log_weights" in rvs: + log_weights = np.asarray(rvs["log_weights"]) + return np.exp(np.max(log_weights) - scipy.special.logsumexp(log_weights)) < pcut + weights = rvs.get("weights") + if weights is None: + weights = rvs["integrand"]*rvs["joint_prior"]/rvs["joint_s_prior"] indxmax = np.argmax(weights) wtSum = np.sum(weights) return weights[indxmax]/wtSum < pcut def convergence_test_NormalSubIntegrals(ncopies, pcutNormalTest, sigmaCutRelativeErrorThreshold, rvs, params): - weights = rvs["integrand"]* rvs["joint_prior"]/rvs["joint_s_prior"] igrandValues = np.zeros(ncopies) - len_part = int(len(weights)/ncopies) - for indx in np.arange(ncopies): - igrandValues[indx] = np.log(np.mean(weights[indx*len_part:(indx+1)*len_part])) + if "log_weights" in rvs: + log_weights = np.asarray(rvs["log_weights"]) + len_part = int(len(log_weights)/ncopies) + for indx in np.arange(ncopies): + log_weights_here = log_weights[indx*len_part:(indx+1)*len_part] + igrandValues[indx] = scipy.special.logsumexp(log_weights_here) - np.log(len(log_weights_here)) + else: + weights = rvs["integrand"]*rvs["joint_prior"]/rvs["joint_s_prior"] + len_part = int(len(weights)/ncopies) + for indx in np.arange(ncopies): + igrandValues[indx] = np.log(np.mean(weights[indx*len_part:(indx+1)*len_part])) igrandValues= np.sort(igrandValues) valTest = stats.normaltest(igrandValues)[1] igrandSigma = (np.std(igrandValues))/np.sqrt(ncopies) diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerGPU.py b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerGPU.py index ba9af2cc1..e247ee719 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerGPU.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerGPU.py @@ -671,6 +671,11 @@ def integrate_log(self, lnF, *args, xpy=xpy_default,**kwargs): deltalnL = kwargs['igrand_threshold_deltalnL'] if 'igrand_threshold_deltalnL' in kwargs else float("Inf") # default is to return all deltaP = kwargs["igrand_threshold_p"] if 'igrand_threshold_p' in kwargs else 0 # default is to omit 1e-7 of probability bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None bShowEvaluationLog = kwargs['verbose'] if 'verbose' in kwargs else False @@ -949,6 +954,7 @@ def inner(arg): self._rvs[key] = identity_convert(self._rvs[key][indx_list]) + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w # Create extra dictionary to return things dict_return ={} if convergence_tests is not None: @@ -1085,6 +1091,11 @@ def integrate(self, func, *args, **kwargs): deltalnL = kwargs['igrand_threshold_deltalnL'] if 'igrand_threshold_deltalnL' in kwargs else float("Inf") # default is to return all deltaP = kwargs["igrand_threshold_p"] if 'igrand_threshold_p' in kwargs else 0 # default is to omit 1e-7 of probability bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None bShowEvaluationLog = kwargs['verbose'] if 'verbose' in kwargs else False @@ -1374,6 +1385,7 @@ def inner(arg): else: self._rvs[key] = identity_convert(self._rvs[key][indx_list]) + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w # Create extra dictionary to return things dict_return ={} if convergence_tests is not None: diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerNFlow.py b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerNFlow.py index f91991740..a0e716ae1 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerNFlow.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerNFlow.py @@ -808,6 +808,11 @@ def integrate_log(self, lnF, *args, xpy=xpy_default,**kwargs): deltalnL = kwargs['igrand_threshold_deltalnL'] if 'igrand_threshold_deltalnL' in kwargs else float("Inf") # default is to return all deltaP = kwargs["igrand_threshold_p"] if 'igrand_threshold_p' in kwargs else 0 # default is to omit 1e-7 of probability bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None bShowEvaluationLog = kwargs['verbose'] if 'verbose' in kwargs else True @@ -977,6 +982,7 @@ def _eval_integrand(cols): self._rvs[key] = identity_convert(self._rvs[key][indx_list]) + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w # perform type conversion of all stored variables. VERY LARGE -- should only do this if we need it! if cupy_ok: for name in self._rvs: diff --git a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerPortfolio.py b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerPortfolio.py index 8b58a5c60..579584a0e 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerPortfolio.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/integrators/mcsamplerPortfolio.py @@ -1291,6 +1291,11 @@ def integrate_log(self, lnF, *args, xpy=xpy_default,**kwargs): deltalnL = kwargs['igrand_threshold_deltalnL'] if 'igrand_threshold_deltalnL' in kwargs else float("Inf") # default is to return all deltaP = kwargs["igrand_threshold_p"] if 'igrand_threshold_p' in kwargs else 0 # default is to omit 1e-7 of probability bFairdraw = kwargs["igrand_fairdraw_samples"] if "igrand_fairdraw_samples" in kwargs else False + # The fair draw below REPLACES _rvs with an export resample; a consumer that then + # weights those rows applies w twice. Record whether it actually FIRED -- the CLI + # flag is not the same predicate, since the draw is skipped when it would not + # shrink the record. Reset per pass: samplers are reused across events. + self._rvs_is_fairdraw = False n_extr = kwargs["igrand_fairdraw_samples_max"] if "igrand_fairdraw_samples_max" in kwargs else None bShowEvaluationLog = kwargs['verbose'] if 'verbose' in kwargs else False @@ -1326,9 +1331,15 @@ def integrate_log(self, lnF, *args, xpy=xpy_default,**kwargs): # the support-mismatch diagnostic describes THIS integral: a second point reusing the same # sampler object must not inherit the previous point's escaped mass. self._reset_support_diagnostics() - if 'integrand' in self._rvs: - # remove conflict - del self._rvs['integrand'] + # The aggregate sample record belongs to ONE integration pass. A second call on the + # same portfolio object (notably the ILE L0 warm retry) deliberately reuses the MEMBER + # proposals, but must not append its draws to the first pass's exported cloud. Besides + # mixing two estimates in _rvs, that made the second pass's warm-seed reserve contain + # cold fair-draw rows and inflated its retained-sample evidence. Proposal/adaptation + # state lives on the members, so clearing this portfolio-level cache keeps the warm + # start while giving the new pass an independent sample record. The L0 driver snapshots + # and restores the cold cache if the warm pass is rejected or raises. + self._rvs = {} # Same reasoning for the warm-seed reserve: a pass that raises part-way would # otherwise leave the PREVIOUS point's retained samples here, and an L0 rescue would # then seed this point's live volume from a different point's peak. Drop it on @@ -1945,6 +1956,7 @@ def _eval_integrand(cols): self._rvs[key] = arr[indx_host] + self._rvs_is_fairdraw = True # _rvs is now an EXPORT resample, rows already ~ w # Create extra dictionary to return things dict_return ={} # if convergence_tests is not None: diff --git a/MonteCarloMarginalizeCode/Code/RIFT/interpolators/nal_io.py b/MonteCarloMarginalizeCode/Code/RIFT/interpolators/nal_io.py new file mode 100644 index 000000000..5ec9fbfbf --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/RIFT/interpolators/nal_io.py @@ -0,0 +1,893 @@ +"""nal_io -- generic reader/evaluator for Normal-Approximate-Likelihood (NAL) artifacts. + +Motivation. RIFT can already CONSUME a stored quadratic, but only through ad-hoc h5py reads inside +`util_ConstructIntrinsicPosterior_GenericCoordinates.fit_quadratic_stored` (`--fit-load-quadratic`): +the coordinates are implicit in the run configuration, nothing declares a frame or a cosmology, and +the reader is not importable from anywhere else. This module is the generic form -- a NAL is just +a bounded multivariate normal in a NAMED chart, so it should be loadable, evaluable and summable +without going through a driver. + +The intended entry point is RIFT's existing plugin hook, which both +`util_ConstructIntrinsicPosterior_GenericCoordinates.py` and `util_ConstructEOSPosterior.py` +expose identically: + + --supplementary-likelihood-factor-code RIFT.interpolators.nal_io + --supplementary-likelihood-factor-function nal_lnL + --supplementary-likelihood-factor-ini my_nal.ini + +The driver calls `prepare_nal_lnL(config=, coords=)` once, then adds +`nal_lnL(*x)` to its own lnL, where `x` is one array per sampling coordinate in `coord_names` +order. Because the contribution is purely additive, a run whose own data file is a DUMMY (a +placeholder net-marg grid) and whose entire likelihood comes from this plugin is a legitimate and +already-used configuration -- that is how the hook itself has been exercised inside CIP. + +NOTE the prepare hook is only reached on RIFT >= the fix in `test_supplementary_likelihood_hook.py`; +before that commit both drivers misspelt the assignment and `--supplementary-likelihood-factor-ini` +raised `TypeError: 'NoneType' object is not callable`. A plugin with no `prepare_` function was +unaffected. On such a RIFT the artifacts can still be supplied through RIFT_NAL_ARTIFACTS, but the +SAMPLING BASIS must then be declared too, as RIFT_NAL_SAMPLER_COORDS='mc,eta,...': without it the +plugin does not know what the arrays it is handed are called, and refuses to guess (see +`nal_lnL`). Guessing is not conservative -- an artifact in (mc, delta_mc) fed a sampler in +(mc, eta) passes every dimension check while evaluating eta as delta_mc. + +WHAT A NAL IS HERE, precisely + ln L(theta) = lnL_peak - 1/2 (theta - mu)^T Gamma (theta - mu), theta in B (bounds box) + zero outside B. +Gamma is the Fisher matrix (-Hessian of lnL), matching `BayesianLeastSquares.fit_quadratic` and the +`lnL_gamma.dat` sidecar. Equivalently Sigma = Gamma^-1. + +TRUNCATION. A NAL is used as a TRUNCATED normal: the consumer only ever evaluates it at physically +realisable parameters. Two consequences that are easy to get wrong and are handled explicitly here: + * the truncation normalisation is a per-event CONSTANT (it depends only on that event's own mu and + Gamma, not on the hyperparameters being sampled), so it cancels in any hyper-posterior. It is + therefore NOT applied by default; `renormalize=True` is available for standalone use where the + absolute scale matters. + * the fitted `mu` may legitimately lie OUTSIDE the physical domain. That is the standard device + for representing a likelihood that rails against a boundary (e.g. eta -> 1/4), not a sign of a + broken fit, and `mu` must never be read as "the measured value". Nothing here rejects an + artifact on that basis. + +THE RUN'S OWN CHART. Artifact metadata cannot say what the SAMPLER is walking in, and the +coordinate names do not distinguish it: 'mc' and 'delta_mc' are spelt identically in a +detector-frame and a source-frame chart. Comparing artifacts with each other (`check_set_compatible`) +therefore does not protect a run, and a single artifact is not compared with anything at all. The +run must declare its own frame and chart -- `[nal] sampler_frame` / `sampler_chart`, or +RIFT_NAL_SAMPLER_FRAME / RIFT_NAL_SAMPLER_CHART -- and they are checked against EVERY artifact, +including a lone one (see `check_sampler_compatible`). No conversion is attempted: mapping between +frames needs a redshift per sample, which the plugin is never handed. + +SCALE OF THE CONTRIBUTION. `nal_lnL` returns the artifacts' lnL with their summed peak SUBTRACTED, +so it is never positive. Both drivers' default path evaluates +`likelihood_function(*x) * np.exp(supplemental_ln_likelihood(*x))`, and float64 exp overflows above +~709: a real loud-event artifact (lnL_peak ~ SNR^2/2, e.g. 3386 for SNR ~ 82) would silently become +inf there, and the drivers' own `lnL_shift` does not reach this separate exponentiation. The +subtracted constant is a fixed multiplicative factor on the likelihood: it cancels in any posterior +but it does NOT cancel in an absolute lnL or an evidence, which is what `integral_result.dat` and +the `_lnL.dat` sidecar are read as -- an odds ratio against a run without this factor would be +wrong by exp(offset). It is reported at preparation and exposed as `nal_lnL_offset()`, following +the same `_...` naming convention as the `prepare_` hook; both drivers look for +that function and ADD the constant back into every absolute likelihood and evidence they write, so +the centring stays inside the sampler where it is needed. + +Environment: pure numpy; h5py only if the gwalk view is used. +""" +import glob as _glob +import json +import os + +import numpy as np + +def _rng(seed): + """numpy Generator when available, else a legacy RandomState. + + `default_rng` needs numpy >= 1.17; RIFT is deployed into a range of environments and this + module is otherwise dependency-free, so it should not be the thing that breaks on an old one. + Only used for the truncation-mass Monte Carlo, where stream equivalence does not matter. + """ + try: + return np.random.default_rng(seed) + except AttributeError: # pragma: no cover - old numpy + return np.random.RandomState(seed) + + +__all__ = ["NAL", "NALSet", "load_nal", "load_nal_dir", "write_nal", + "nal_lnL", "nal_lnL_offset", "prepare_nal_lnL", "write_gwalk_view", + "check_frame_invariant", "check_artifact_frame_invariant", + "check_set_compatible", "check_sampler_compatible", + "SCHEMA_VERSION"] + +SCHEMA_VERSION = 2 + +# A dropped coordinate whose bounds sit at least this many marginal sigma from mu on BOTH sides is +# treated as unbounded when marginalising: the Gaussian tail beyond 5 sigma is 2.9e-7 per side. +_UNBOUNDED_SIGMA = 5.0 +# Correlation below this counts as none: the truncation factor is then a constant, not theta-dependent. +_CORR_TOL = 1e-8 +# An eigenvalue of gamma at or below this fraction of the largest one counts as zero (see +# NAL._check_positive_definite). Relative, so it is invariant under a rescaling of lnL. +_PD_RTOL = 1e-12 + +# Charts this module knows how to build from RIFT's native parameters. Definitions are taken from +# RIFT/lalsimutils.py, NOT from any design document: +# xi (:961-966) dot(Lhat, m1*chi1Vec + m2*chi2Vec)/(m1+m2), Lhat = zhat +# chiMinus (:971-976) dot(Lhat, m1*chi1Vec - m2*chi2Vec)/(m1+m2) <- MASS WEIGHTED +# delta_mc (:587-590) eta = (1 - delta^2)/4, i.e. delta = sqrt(1-4 eta) +KNOWN_COORDS = ("mc", "eta", "delta_mc", "xi", "chiMinus", + "s1z", "s2z", "s1x_bar", "s1y_bar", "s2x_bar", "s2y_bar", "u_d", "dist") + +# Names that carry the luminosity distance. BOTH of them: `_derive` treats u_d = 1/dist as +# interchangeable, so a chart carrying `dist` says exactly as much about the distance as one +# carrying `u_d`, and the frame invariant must recognise either. +_DISTANCE_COORDS = ("u_d", "dist") + +# Metadata keys `write_nal` owns -- either validated here (frame/cosmology/d_prior, through +# check_frame_invariant) or derived from the artifact itself. `extra` may not overwrite them: the +# value that was checked and the value that is recorded must be the same value. +_RESERVED_META_KEYS = ("schema", "method", "chart", "coord_names", "frame", "cosmology", "d_prior", + "lnL_peak", "lnL_ref", "symmetry", "unconstrained_dirs", "parents", + "run_id", "git_sha", "validation") + + +def _derive(name, have): + """Best-effort derivation of one chart coordinate from a dict of available arrays. + + Deliberately narrow: it covers the mass/aligned-spin identities CIP actually samples in, and + raises a NAMED error otherwise rather than silently returning something plausible. + """ + if name in have: + return have[name] + g = have.get + if name == "delta_mc" and "eta" in have: + return np.sqrt(np.maximum(1.0 - 4.0 * np.asarray(g("eta"), float), 0.0)) + if name == "eta" and "delta_mc" in have: + return 0.25 * (1.0 - np.asarray(g("delta_mc"), float) ** 2) + if name in ("xi", "chiMinus") and all(k in have for k in ("s1z", "s2z")) \ + and ("q" in have or "delta_mc" in have or "eta" in have): + if "q" in have: + q = np.asarray(g("q"), float) + else: + d = _derive("delta_mc", have) + q = (1.0 - d) / (1.0 + d) + s1z, s2z = np.asarray(g("s1z"), float), np.asarray(g("s2z"), float) + # m1/M = 1/(1+q), m2/M = q/(1+q) + return (s1z + q * s2z) / (1.0 + q) if name == "xi" else (s1z - q * s2z) / (1.0 + q) + if name == "u_d" and "dist" in have: + return 1.0 / np.asarray(g("dist"), float) + if name == "dist" and "u_d" in have: + return 1.0 / np.asarray(g("u_d"), float) + raise KeyError( + "nal_io: cannot build chart coordinate %r from the sampler coordinates %s. Either sample " + "in a chart that contains it, or extend _derive()." % (name, sorted(have))) + + +class NAL(object): + """One event's bounded multivariate-normal likelihood in a declared chart.""" + + def __init__(self, mu, gamma, coord_names, lnL_peak=0.0, bounds=None, meta=None): + self.mu = np.asarray(mu, float).ravel() + self.gamma = np.asarray(gamma, float) + self.gamma = 0.5 * (self.gamma + self.gamma.T) + self.coord_names = list(coord_names) + self.lnL_peak = float(lnL_peak) + self.bounds = None if bounds is None else np.asarray(bounds, float) + self.meta = dict(meta or {}) + self.source = None # set by load_nal(); for error messages + self._log_mass_cache = None # (settings, value); see log_mass() + d = len(self.mu) + if self.gamma.shape != (d, d): + raise ValueError("nal_io: gamma shape %s does not match mu (%d)" + % (self.gamma.shape, d)) + if len(self.coord_names) != d: + raise ValueError("nal_io: %d coord_names for %d-dimensional NAL" + % (len(self.coord_names), d)) + self._check_positive_definite() + + def _check_positive_definite(self): + """`gamma` must be finite and positive definite, or this is not a peaked likelihood. + + Not a formality. A negative eigenvalue makes lnL INCREASE away from mu along that + direction: the object is a saddle, `lnL_peak` is not the peak, and `_peak_offset` -- which + the plugin relies on to keep the drivers' `np.exp(supplemental)` in range -- is no longer an + upper bound on anything, so the overflow protection silently stops protecting. Nothing + downstream can notice: the arithmetic is finite and the array shapes are right. A fitter + that has not converged, or one that fitted a boundary-railed direction badly, produces + exactly this, so it is a realistic input rather than a hypothetical one. + + SINGULAR IS ALSO REJECTED, explicitly. `cov()` -- hence `marginal`, `log_mass`, + `write_gwalk_view` and the truncation machinery -- needs Gamma^-1, and a numerically + singular Gamma inverts to garbage rather than to an error. A genuinely unconstrained + direction is a real thing, but it is not representable as a normalizable likelihood without + bounds; it belongs in the artifact's `unconstrained_dirs` metadata, with the direction + removed from the chart. + + The threshold is RELATIVE to the largest eigenvalue, so it is invariant under an overall + rescaling of lnL and does not depend on the units of the chart. It is set far above the + ~1e-16 relative error of a symmetric eigendecomposition and far below the conditioning of + any usable fit, so it separates "zero" from "small" without rejecting the honestly + ill-conditioned mass/spin blocks these fits produce. + """ + if not np.all(np.isfinite(self.gamma)): + raise ValueError("nal_io: gamma contains non-finite entries -- an unconverged or " + "failed fit, not a likelihood") + if not np.all(np.isfinite(self.mu)): + raise ValueError("nal_io: mu contains non-finite entries") + w = np.linalg.eigvalsh(self.gamma) + scale = float(np.max(np.abs(w))) if len(w) else 0.0 + if scale <= 0.0: + raise ValueError("nal_io: gamma is identically zero -- no likelihood is defined") + if w[0] < -_PD_RTOL * scale: + raise ValueError( + "nal_io: gamma is not positive definite (eigenvalues %s): lnL would INCREASE away " + "from mu along the negative direction, so this is a saddle and lnL_peak is not the " + "peak. The plugin's overflow guard subtracts a bound built from lnL_peak, which " + "such an artifact does not respect. Refit, or drop the unconverged direction." + % np.array2string(w, precision=4)) + if w[0] <= _PD_RTOL * scale: + raise ValueError( + "nal_io: gamma is numerically singular (eigenvalues %s, smallest is %.3g of the " + "largest): it has an unconstrained direction, and Gamma^-1 -- needed by cov(), " + "marginal(), log_mass() and the gwalk view -- would be numerically meaningless " + "rather than an error. A flat direction is not a normalizable likelihood: remove it " + "from the chart and record it in the artifact's 'unconstrained_dirs', or bound it " + "and refit." % (np.array2string(w, precision=4), w[0] / scale)) + + @property + def ndim(self): + return len(self.mu) + + def cov(self): + return np.linalg.inv(self.gamma) + + def marginal(self, keep, ignore_truncation=False, shape_only=False): + """Marginal NAL over a subset of coordinates, by name or index. + + Uses Sigma = Gamma^-1 and takes the SUB-BLOCK -- equivalently the Schur complement + Gamma_AA - Gamma_AB Gamma_BB^-1 Gamma_BA. This is the MARGINAL. Taking `Gamma_AA` + instead would give the CONDITIONAL (nuisance held fixed), which is systematically too + narrow; they are easy to confuse and are not the same object. + + ABSOLUTE SCALE. Marginalising is an INTEGRAL, so it changes the peak as well as the + shape, and the module promises absolute lnL elsewhere -- so the constant is computed, not + dropped: + + lnL_peak_marg = lnL_peak + (k/2) ln(2 pi) - 1/2 ln det Gamma_BB [+ ln P(B in bounds)] + + for the k dropped coordinates, where Gamma_BB is the DROPPED sub-block of Gamma (the + conditional precision, not a sub-block of Sigma). Dropping one independent unit-variance + coordinate therefore raises the peak by 0.5 ln(2 pi) = 0.919, not by nothing: a marginal + that kept `lnL_peak` would be low by that much per coordinate, and the error compounds -- + it is a factor 1e3 after 15 coordinates, applied to a quantity read as an evidence. + The bracketed term is the enclosed mass of the dropped block, present only when its bounds + bite; under the conditions this method allows (below) it is a constant, and it is evaluated + by the same controlled Monte Carlo as `log_mass`, so it can RAISE if it is too small to + resolve. `shape_only=True` restores the projection-with-the-original-peak behaviour, for a + caller who wants the shape and will supply the normalisation themselves. + + TRUNCATION. That identity is the UNTRUNCATED marginal. Integrating out a coordinate that + is genuinely truncated multiplies the result by the mass of that coordinate's CONDITIONAL + distribution inside its own bounds -- and the conditional mean slides with the retained + coordinates whenever the two are correlated, so the factor is a theta-DEPENDENT + conditional-CDF ratio. It is not Gaussian and cannot be absorbed into (mu, Gamma), so the + result would have the wrong SHAPE, not merely the wrong normalisation. Boundary-railing + fits -- the case this module exists to represent -- are exactly where it bites, so it is + REJECTED rather than silently approximated. Two situations are provably safe and are + allowed: a dropped coordinate whose bounds lie at least 5 marginal sigma from mu on both + sides (factor 1 to ~1e-6), or one uncorrelated with every retained coordinate (factor + constant, so only lnL_peak moves -- which the integration constant above accounts for). + Pass `ignore_truncation=True` to take the untruncated marginal anyway; the constant is then + the untruncated one, since that is what was asked for. + """ + idx = [self.coord_names.index(k) if isinstance(k, str) else int(k) for k in keep] + drop = [i for i in range(self.ndim) if i not in idx] + Sigma = self.cov() + if drop and self.bounds is not None and not ignore_truncation: + self._reject_theta_dependent_truncation(idx, drop, Sigma) + S = Sigma[np.ix_(idx, idx)] + b = None if self.bounds is None else self.bounds[idx] + peak = self.lnL_peak + if drop and not shape_only: + peak = peak + self._marginalization_constant(idx, drop, Sigma, ignore_truncation) + return NAL(self.mu[idx], np.linalg.inv(S), [self.coord_names[i] for i in idx], + lnL_peak=peak, bounds=b, meta=self.meta) + + def _marginalization_constant(self, keep_idx, drop_idx, Sigma, ignore_truncation): + """ln of the factor picked up by integrating exp(lnL) over the dropped coordinates. + + Completing the square in the joint quadratic gives + + int exp(-1/2 d^T Gamma d) d(theta_B) + = (2 pi)^(k/2) |Gamma_BB|^(-1/2) exp(-1/2 d_A^T (Gamma/Gamma_BB) d_A) + + -- the Schur complement in the exponent (the shape `marginal` already returns) and a + constant in front. Gamma_BB is the sub-block of GAMMA, not of Sigma: it is the precision + of B CONDITIONED on A, which is what completing the square produces. + + With bounds, the integral over the box multiplies this by P(theta_B in B_B | theta_A). + `_reject_theta_dependent_truncation` has already established that this probability does not + depend on theta_A -- every dropped coordinate is either effectively unbounded or + uncorrelated with everything retained -- so it may be evaluated once, at theta_A = mu_A, + where theta_B ~ N(mu_B, Gamma_BB^-1). It is skipped entirely when no dropped bound bites, + which is both the common case and the one where the Monte Carlo would be a waste (the + answer is 1 to ~1e-6 per side by the 5-sigma criterion that let it through). + """ + k = len(drop_idx) + G_BB = self.gamma[np.ix_(drop_idx, drop_idx)] + sign, logdet = np.linalg.slogdet(G_BB) + if sign <= 0: # unreachable for a positive-definite + raise ValueError("nal_io: dropped block of gamma is not positive definite") + const = 0.5 * k * np.log(2 * np.pi) - 0.5 * logdet + if self.bounds is None or ignore_truncation: + return const + sd = np.sqrt(np.diag(Sigma)) + bites = [j for j in drop_idx + if (self.mu[j] - self.bounds[j][0]) < _UNBOUNDED_SIGMA * sd[j] + or (self.bounds[j][1] - self.mu[j]) < _UNBOUNDED_SIGMA * sd[j]] + if not bites: + return const + block = NAL(self.mu[drop_idx], G_BB, [self.coord_names[j] for j in drop_idx], + bounds=self.bounds[drop_idx]) + return const + block.log_mass() + + def _reject_theta_dependent_truncation(self, keep_idx, drop_idx, Sigma): + """Raise unless every dropped coordinate is effectively unbounded or uncorrelated.""" + sd = np.sqrt(np.diag(Sigma)) + bad = [] + for j in drop_idx: + lo, hi = self.bounds[j] + if (self.mu[j] - lo) >= _UNBOUNDED_SIGMA * sd[j] and \ + (hi - self.mu[j]) >= _UNBOUNDED_SIGMA * sd[j]: + continue # bounds do not bite + rho = np.abs(Sigma[j, keep_idx]) / (sd[j] * sd[keep_idx]) + if np.all(rho <= _CORR_TOL): + continue # factor is a constant + bad.append(self.coord_names[j]) + if bad: + raise ValueError( + "nal_io: cannot marginalise over %s: those coordinates are truncated by `bounds` " + "AND correlated with the ones kept, so the exact marginal carries a " + "theta-dependent conditional-CDF factor that a Gaussian sub-block cannot " + "represent -- the marginal would have the wrong shape, most severely for the " + "boundary-railing fits this module is written for. Keep them, widen their bounds " + "if the truncation is not physical, or pass ignore_truncation=True if you have " + "established the factor is harmless." % bad) + + def lnL(self, theta, renormalize=False): + """ln L at theta, shape (N, ndim) or (ndim,). Outside `bounds` returns -inf.""" + X = np.atleast_2d(np.asarray(theta, float)) + d = X - self.mu + out = self.lnL_peak - 0.5 * np.einsum("ij,jk,ik->i", d, self.gamma, d) + if self.bounds is not None: + inside = np.all((X >= self.bounds[:, 0]) & (X <= self.bounds[:, 1]), axis=1) + out = np.where(inside, out, -np.inf) + if renormalize: + out = out - self.log_mass() + return out + + def log_mass(self, rel_tol=0.01, max_draws=4000000, batch=200000, seed=0): + """log of the Gaussian mass inside `bounds` -- computed ONCE per artifact, then cached. + + Deliberately NOT a product of 1-D marginal masses. That factorisation ignores correlations + and is badly biased for a correlated fit -- measured against brute force on a 3-D Gaussian + with rho = 0.9 pairwise, the factorised value is 42% low. This is a per-event CONSTANT and + cancels in any hyper-posterior, which is why `renormalize` defaults to False. + + Monte Carlo, but to a CONTROLLED error: batches are drawn until the relative standard error + of the hit fraction, sqrt((1-f)/hits), is at or below `rel_tol`, and a mass too small to + resolve within `max_draws` RAISES. Flooring the estimate at 1/n instead reports a number + with no error bar as if it were a measurement: for a 1-D standard normal truncated to + [6, 7] the true mass is ~1e-9 while a 200k-draw floor returns 5e-6, shifting the + renormalized lnL by ~8.5 nat. A known value may be declared as + meta['log_truncation_mass'] and is then used verbatim. + + Caching is not an optimisation but a requirement: `lnL(..., renormalize=True)` is called + once per likelihood evaluation, and re-running a 200k-draw Monte Carlo inside it makes the + opt-in path unusable from any sampler. + """ + if self.bounds is None: + return 0.0 + if self.meta.get("log_truncation_mass") is not None: + return float(self.meta["log_truncation_mass"]) + key = (rel_tol, max_draws, batch, seed) + if self._log_mass_cache is not None and self._log_mass_cache[0] == key: + return self._log_mass_cache[1] + rng = _rng(seed) + cov = self.cov() + hits = 0 + drawn = 0 + rse = np.inf + while drawn < max_draws: + k = int(min(batch, max_draws - drawn)) + G = rng.multivariate_normal(self.mu, cov, size=k) + hits += int(np.count_nonzero( + np.all((G >= self.bounds[:, 0]) & (G <= self.bounds[:, 1]), axis=1))) + drawn += k + if hits > 0: + f = hits / float(drawn) + rse = float(np.sqrt((1.0 - f) / hits)) # relative standard error of f + if rse <= rel_tol: + break + if hits == 0 or rse > rel_tol: + raise ValueError( + "nal_io: truncation mass unresolved -- %d of %d draws landed inside `bounds`, a " + "relative error of %s against the requested %g. The normalisation is too small to " + "estimate here and a floored guess would be wrong by an unknown amount, so it is " + "not returned. Record the known value as meta['log_truncation_mass'], raise " + "max_draws, widen `bounds` if the truncation is not physical, or leave " + "renormalize=False -- the constant cancels in any hyper-posterior." + % (hits, drawn, ("undefined" if hits == 0 else "%.3g" % rse), rel_tol)) + self._log_mass_cache = (key, float(np.log(hits / float(drawn)))) + return self._log_mass_cache[1] + + +def _canon(value): + """Comparable form of a metadata value (dicts compare by content, not by key order).""" + if value is None: + return None + if isinstance(value, (dict, list, tuple)): + return json.dumps(value, sort_keys=True, default=str) + return value + + +def check_set_compatible(nals): + """Refuse to ADD artifacts whose metadata does not establish that they share a chart. + + Equal `coord_names` is NOT enough. 'mc' is a detector-frame chirp mass in one artifact and a + source-frame one in another; two source-frame artifacts built under different cosmologies, or + with different distance priors integrated out, are likewise in different charts wearing the + same labels. Summing any of those evaluates a joint likelihood at a theta that means + something different in each term -- silently, because the arithmetic is perfectly well defined + and only the physics is wrong. + + Fails closed. An artifact that does not declare its `frame` cannot be SHOWN compatible with + another, so it is rejected rather than assumed. `chart` is required of EVERY member of a set + for the same reason: matching coordinate names in a matching frame still do not establish + matching coordinate CONVENTIONS -- which spin basis, which mass pairing, which angle + reference -- so a set in which nobody declares a chart is not a set that has been shown + compatible, it is one in which the question was never asked. `cosmology` and `d_prior` must + agree whenever the frame is 'source' (there they define what the masses mean) or whenever any + member of the set declares them at all. A single artifact is never checked: nothing is being + added to it. + """ + nals = list(nals) + if len(nals) < 2: + return + frames = [n.meta.get("frame") for n in nals] + undeclared = [i for i, f in enumerate(frames) if not f] + if undeclared: + raise ValueError( + "nal_io: artifact(s) at position(s) %s declare no 'frame', so they cannot be shown to " + "be in the same chart as the rest of the set -- detector- and source-frame masses " + "carry identical coordinate names. Write them with write_nal(), which records the " + "frame, before summing." % undeclared) + if len(set(frames)) != 1: + raise ValueError("nal_io: cannot sum artifacts across frames %s -- the same coordinate " + "names denote different physical quantities" % sorted(set(frames))) + for key in ("chart", "cosmology", "d_prior"): + vals = [_canon(n.meta.get(key)) for n in nals] + declared = [v for v in vals if v is not None and v != ""] + # `chart` is never optional in a set: "nobody declared one" is not evidence of agreement, + # and write_nal(chart=None) will happily produce a whole catalogue of such artifacts. + required = key == "chart" or (frames[0] == "source" and key in ("cosmology", "d_prior")) + if not (declared or required): + continue # nobody claims it; nothing to reconcile + if len(declared) != len(vals) or len(set(declared)) != 1: + raise ValueError( + "nal_io: artifacts in a set must agree on a non-empty %r before they may be " + "added; got %d of %d declaring it, %d distinct value(s). Artifacts that differ " + "in %r are in different charts even when their coordinate names match, and one " + "that does not declare it cannot be shown to agree. Record it at write time as " + "write_nal(..., %s=...)." + % (key, len(declared), len(vals), len(set(declared)), key, key)) + + +def check_sampler_compatible(nals, frame, chart): + """Refuse to evaluate artifacts against a run whose chart is not DECLARED to match them. + + `check_set_compatible` only compares artifacts with each other, and it is skipped entirely for + a single artifact -- nothing is being added to it. Neither fact says anything about the + SAMPLER: `coords` carries names alone, and 'mc'/'delta_mc' are spelt identically whether the + run walks in detector-frame or source-frame masses. A source-frame NAL evaluated at + detector-frame samples (or the reverse) has the right array count, the right names and no + error -- only a wrong answer, biased by the redshift of the event. + + Conversion is not an option here: it needs a redshift per sample, which the plugin is never + handed. So the run declares its frame and its chart and they are compared, and every artifact + must state its own. Fails closed in both directions -- undeclared on either side is a + mismatch, not a pass. + """ + nals = list(nals) + if not frame: + raise ValueError( + "nal_io: the run's sampling frame is undeclared, so these artifacts cannot be shown " + "to be in the chart the sampler is walking in -- detector- and source-frame masses " + "wear identical coordinate names, and no dimension or name check can tell them " + "apart. Declare it as [nal] sampler_frame = detector|source in the ini, or " + "RIFT_NAL_SAMPLER_FRAME.") + if frame not in ("detector", "source"): + raise ValueError("nal_io: sampler_frame must be 'detector' or 'source', got %r" % (frame,)) + if not chart: + raise ValueError( + "nal_io: the run's sampling chart is undeclared. Matching coordinate names in a " + "matching frame still do not establish matching coordinate CONVENTIONS -- which spin " + "basis, which mass pairing, which angle reference. Declare it as [nal] sampler_chart " + "in the ini, or RIFT_NAL_SAMPLER_CHART, naming the same chart the artifacts do.") + for key, want in (("frame", frame), ("chart", chart)): + got = sorted({(n.meta.get(key) or "") for n in nals + if n.meta.get(key) != want}) + if got: + raise ValueError( + "nal_io: artifact %s %s does not match the run's declared %s %r. The artifacts " + "would be evaluated at coordinates that mean something else in the chart they " + "were fitted in, silently: the array count and the coordinate names are " + "identical either way. Use artifacts built for this run's chart, or correct the " + "declaration." % (key, got, key, want)) + + +class NALSet(object): + """A catalogue of NALs, summed. Each event contributes additively in lnL.""" + + def __init__(self, nals, require_compatible=True): + """`require_compatible=False` skips `check_set_compatible` -- only for a caller who has + established equivalence of the charts by other means.""" + self.nals = list(nals) + if not self.nals: + raise ValueError("nal_io: empty NALSet") + ch = {tuple(n.coord_names) for n in self.nals} + if len(ch) != 1: + raise ValueError("nal_io: NALSet requires one common chart, got %s" % sorted(ch)) + if require_compatible: + check_set_compatible(self.nals) + self.coord_names = list(self.nals[0].coord_names) + + def lnL(self, theta, renormalize=False): + X = np.atleast_2d(np.asarray(theta, float)) + tot = np.zeros(len(X)) + for n in self.nals: + tot = tot + n.lnL(X, renormalize=renormalize) + return tot + + +def load_nal(path): + """Load one artifact. Accepts .npz (with sidecar .meta.json) or the .meta.json.""" + base = path[:-len(".meta.json")] if path.endswith(".meta.json") else \ + (path[:-4] if path.endswith(".npz") else path) + meta = {} + if os.path.exists(base + ".meta.json"): + meta = json.load(open(base + ".meta.json")) + d = np.load(base + ".npz", allow_pickle=False) + names = meta.get("coord_names") + if names is None: + raise KeyError("nal_io: %s.meta.json must declare coord_names -- a NAL without a named " + "chart is not interpretable" % base) + g = d["gamma"] if "gamma" in d else np.linalg.inv(d["cov"]) + out = NAL(d["theta_star"], g, names, + lnL_peak=float(meta.get("lnL_peak", 0.0)), + bounds=d["bounds"] if "bounds" in d else None, meta=meta) + out.source = base + # Enforce the frame invariant on the CONSUMER side too: most artifacts a run loads were not + # written by write_nal(), so a check that only runs in the writer never sees them. + check_artifact_frame_invariant(out, where=base) + return out + + +def load_nal_dir(pattern): + """Load every artifact matching a glob (e.g. '/path/*.npz'), sorted for reproducibility.""" + out = [load_nal(p) for p in sorted(_glob.glob(pattern))] + if not out: + raise IOError("nal_io: no artifacts matched %r" % pattern) + return out + + +def _sha256(path, chunk=1 << 20): + import hashlib + h = hashlib.sha256() + with open(path, "rb") as f: + for blk in iter(lambda: f.read(chunk), b""): + h.update(blk) + return h.hexdigest() + + +def _git_sha(path): + """Short git sha of the tree `path` lives in, or None. Best-effort provenance only.""" + import subprocess + try: + d = path if os.path.isdir(path) else os.path.dirname(os.path.abspath(path)) or "." + out = subprocess.run(["git", "-C", d, "rev-parse", "--short", "HEAD"], + capture_output=True, text=True, timeout=10) + return out.stdout.strip() or None + except Exception: + return None + + +def check_frame_invariant(coord_names, frame, cosmology=None, d_prior=None): + """An artifact may carry EITHER the distance coordinate OR source-frame masses, never both. + + A distance-marginalised DETECTOR-frame quadratic is not reusable: it has silently integrated + the mass-redshift degeneracy against one particular distance prior, and nothing downstream can + undo that or even detect it. So: + + * a distance coordinate present => frame must be 'detector' (distance has not been + marginalised yet). BOTH spellings count: `_derive` makes u_d and dist + interchangeable, so a chart carrying `dist` is exactly as + distance-carrying as one carrying `u_d`. + * frame source => no distance coordinate at all, AND the cosmology and the distance prior + that was integrated must both be recorded, since the source-frame masses + are meaningless without them. + + Raises ValueError rather than warning: an artifact that cannot state its own frame honestly + should not be written at all. (The shipped O3/O4 NAL catalogue records none of this -- its + npz carries only names/labels/centers/sigs/covs/ess/method/kl -- so a consumer cannot tell + which cosmology produced it.) + """ + dist_coords = [c for c in coord_names if c in _DISTANCE_COORDS] + if frame not in ("detector", "source"): + raise ValueError("nal_io: frame must be 'detector' or 'source', got %r" % (frame,)) + if dist_coords and frame != "detector": + raise ValueError("nal_io: chart carries %s, so masses are detector-frame; got frame=%r" + % (dist_coords, frame)) + if frame == "source": + if dist_coords: + raise ValueError("nal_io: frame='source' must not also carry a distance coordinate " + "(%s)" % dist_coords) + if not cosmology: + raise ValueError("nal_io: frame='source' requires a declared cosmology") + if not d_prior: + raise ValueError("nal_io: frame='source' requires the distance prior that was " + "integrated out (name and range)") + + +def check_artifact_frame_invariant(nal, where=None, require_frame=False): + """Apply the frame invariant to an artifact this module did not write. + + `write_nal` enforces `check_frame_invariant` at WRITE time, but nothing a consumer is handed + has necessarily been through it. Artifacts arrive from exporters that predate this module -- + the shipped O3/O4 NAL catalogue records no frame at all, only names/labels/centers/sigs/covs -- + and from fitting scripts that assemble the npz and the meta.json themselves. Such an artifact + can perfectly well declare `frame='source'` while still carrying `u_d`, or declare it with no + cosmology and no distance prior: a distance-marginalised quadratic whose mass-redshift + degeneracy was integrated against a prior nobody recorded, which no consumer can undo and none + can detect from the numbers. A check that only runs in the writer is a check the artifacts + that matter never meet, so it is re-run on LOAD against the artifact's own recorded metadata. + + An artifact declaring no frame at all is left to `check_set_compatible` / + `check_sampler_compatible`, which reject it with a message naming the comparison that cannot be + made. `require_frame=True` makes it an error here instead, so the plugin entry point fails + closed whatever the order of the checks around it. + """ + meta = nal.meta or {} + frame = meta.get("frame") + where = where or getattr(nal, "source", None) or "" + if not frame: + if require_frame: + raise ValueError( + "nal_io: artifact %s declares no 'frame', so its own consistency cannot be " + "established: whether its masses are detector- or source-frame decides whether " + "carrying a distance coordinate is normal or means the distance has already been " + "integrated out against an unrecorded prior. Rewrite it with write_nal(), which " + "records the frame and checks the invariant." % where) + return + try: + check_frame_invariant(nal.coord_names, frame, meta.get("cosmology"), meta.get("d_prior")) + except ValueError as exc: + raise ValueError( + "nal_io: artifact %s fails the frame invariant on its own recorded metadata (%s). It " + "was not written by write_nal(), or was edited afterwards; the run cannot interpret " + "it and no downstream step can detect the error from the numbers alone." % (where, exc)) + + +def write_nal(base, nal, chart=None, frame="detector", cosmology=None, d_prior=None, + symmetry=None, unconstrained_dirs=None, validation=None, parents=None, + run_id=None, lnL_ref="noise_hypothesis_ratio", extra=None): + """Write .npz + .meta.json. + + `parents` should be the source files the fit consumed (all.net / all_dslice.dat); their + sha256 is recorded so an artifact can always be traced to the grid that produced it. + + `chart` is optional here -- a lone artifact is self-consistent without one -- but an artifact + written without it cannot later be ADDED to another: `check_set_compatible` requires every + member of a set to declare the same non-empty chart. Name it now if the artifact is destined + for a catalogue. + + `extra` may only ADD metadata. It is applied after the frame invariant has been checked, so + allowing it to overwrite a validated key would let frame='detector' pass the check while + frame='source' is what gets recorded -- an artifact claiming source-frame masses with no + cosmology, no distance prior, possibly carrying u_d. Collisions raise, before any file is + written. + """ + coord_names = list(nal.coord_names) + check_frame_invariant(coord_names, frame, cosmology, d_prior) + clash = sorted(set(extra or ()) & set(_RESERVED_META_KEYS)) + if clash: + raise ValueError( + "nal_io: extra=%s would overwrite metadata this writer validates or derives. Those " + "keys are checked BEFORE extra is applied, so overwriting them records something " + "that was never validated -- notably a frame whose cosmology, distance prior and " + "distance coordinate went unchecked. Pass them as the named arguments instead " + "(write_nal(..., frame=..., cosmology=...)), or rename the extra key." % clash) + bounds = nal.bounds + if bounds is None: + sd = np.sqrt(np.diag(nal.cov())) + bounds = np.stack([nal.mu - 10 * sd, nal.mu + 10 * sd], 1) + np.savez(base + ".npz", theta_star=nal.mu, gamma=nal.gamma, bounds=np.asarray(bounds)) + meta = { + "schema": SCHEMA_VERSION, "method": "nal", + "chart": chart or nal.meta.get("chart"), "coord_names": coord_names, + "frame": frame, "cosmology": cosmology, "d_prior": d_prior, + "lnL_peak": float(nal.lnL_peak), "lnL_ref": lnL_ref, + "symmetry": symmetry, "unconstrained_dirs": list(unconstrained_dirs or []), + "parents": [{"path": p, "sha256": _sha256(p)} for p in (parents or []) + if os.path.exists(p)], + "run_id": run_id, "git_sha": _git_sha(__file__), + "validation": dict(validation or {}), + } + if extra: + meta.update(extra) + with open(base + ".meta.json", "w") as f: + json.dump(meta, f, indent=1, sort_keys=True) + return base + ".npz", base + ".meta.json" + + +def write_gwalk_view(path, nal, label, scale_max=None): + """Emit the gwalk HDF5 view for one NAL. + + Two gwalk behaviours this deliberately works around, both verified against gwalk 2.3.0: + * `offset` is asserted into [-scale_max, scale_max] with scale_max defaulting to 500, and a + loud event's lnL_peak-derived offset exceeds that (lnL_peak ~ SNR^2/2, so SNR > ~32 trips + it). scale_max is therefore written explicitly, sized to the value being stored. + * gwalk's own `normalize()` recomputes `offset` from a product of 1-D marginal masses that + ignores correlations, and OVERWRITES whatever was stored. Consumers must not call it; + the exact lnL_peak is carried in `offset` here instead. + """ + import h5py + Sig = nal.cov() + sd = np.sqrt(np.diag(Sig)) + cor = Sig / np.outer(sd, sd) + D = nal.ndim + sign, logdet = np.linalg.slogdet(nal.gamma) + offset = nal.lnL_peak + 0.5 * D * np.log(2 * np.pi) - 0.5 * logdet + if scale_max is None: + scale_max = max(500.0, 2.0 * abs(offset)) + with h5py.File(path, "a") as f: + grp = f.require_group(label) + for k, v in (("mu", nal.mu), ("std", sd), ("cor", cor), ("cov", Sig), + ("limits", nal.bounds if nal.bounds is not None + else np.stack([nal.mu - 10 * sd, nal.mu + 10 * sd], 1)), + ("offset", np.array([offset])), ("scale", np.array([1.0]))): + if k in grp: + del grp[k] + grp.create_dataset(k, data=np.asarray(v)) + grp.attrs["ndim"] = D + grp.attrs["scale_max"] = scale_max + grp.attrs["coord_names"] = json.dumps(nal.coord_names) + return offset + + +# ----------------------------------------------------------------- RIFT plugin hook entry points +_STATE = {"set": None, "coords": None, "renormalize": False, "offset": 0.0} + + +def _peak_offset(nals, renormalize): + """Largest value the summed contribution can take: sum of the per-artifact peaks. + + Each term is at most its own peak, so subtracting this makes `nal_lnL` non-positive + everywhere and `np.exp` of it safe. With `renormalize` the per-artifact peak is + lnL_peak - log_mass (log_mass <= 0, so the peak RISES); computing it here also fails early and + fills the cache rather than surprising the sampler on its first call. + """ + return float(sum(n.lnL_peak - (n.log_mass() if renormalize else 0.0) for n in nals)) + + +def prepare_nal_lnL(config=None, coords=None): + """Called once by CIP / EOSPosterior with the parsed ini and the run's SAMPLING coordinates. + + ini section: + [nal] + artifacts = /path/to/*.npz ; glob, or a comma-separated list + sampler_frame = detector ; REQUIRED: the frame the RUN samples in + sampler_chart = NAL:aligned ; REQUIRED: the chart the RUN samples in + renormalize = false ; per-event constant, cancels in a hyper-posterior + sampler_coords = mc,eta ; only needed when the driver cannot pass coords= + Falls back to the environment variables RIFT_NAL_ARTIFACTS, RIFT_NAL_SAMPLER_FRAME, + RIFT_NAL_SAMPLER_CHART and RIFT_NAL_SAMPLER_COORDS when no ini is supplied, so the plugin also + works on RIFT versions predating the prepare-hook fix. `coords` from the driver always wins: + it is the authoritative sampling basis. + + The frame and chart of the RUN cannot be read off the artifacts or the coordinate names, and + the driver does not pass them, so they must be declared and are then checked against every + artifact -- including a single one, which no set check ever examines. + """ + pat = os.environ.get("RIFT_NAL_ARTIFACTS") + declared = os.environ.get("RIFT_NAL_SAMPLER_COORDS") + frame = os.environ.get("RIFT_NAL_SAMPLER_FRAME") + chart = os.environ.get("RIFT_NAL_SAMPLER_CHART") + if config is not None and config.has_section("nal"): + if config.has_option("nal", "artifacts"): + pat = config.get("nal", "artifacts") + if config.has_option("nal", "renormalize"): + _STATE["renormalize"] = config.get("nal", "renormalize").strip().lower() \ + in ("1", "true", "yes") + if config.has_option("nal", "sampler_coords"): + declared = config.get("nal", "sampler_coords") + if config.has_option("nal", "sampler_frame"): + frame = config.get("nal", "sampler_frame") + if config.has_option("nal", "sampler_chart"): + chart = config.get("nal", "sampler_chart") + if not pat: + raise ValueError("nal_io: no artifacts configured -- set [nal] artifacts in the ini or " + "the RIFT_NAL_ARTIFACTS environment variable") + nals = [] + for part in pat.split(","): + part = part.strip() + nals += load_nal_dir(part) if any(c in part for c in "*?[") else [load_nal(part)] + check_sampler_compatible(nals, (frame or "").strip(), (chart or "").strip()) + # Every artifact must ALSO be self-consistent, not merely consistent with the run: agreeing + # with a declared frame says nothing about whether the artifact's own chart and metadata are + # compatible with that frame. load_nal() already enforces this for a declared frame; repeated + # here with require_frame=True so the entry point fails closed regardless of check order, and + # for NALs assembled in memory rather than loaded from disk. + for n in nals: + check_artifact_frame_invariant(n, require_frame=True) + _STATE["set"] = NALSet(nals) + if coords is not None: + _STATE["coords"] = list(coords) + elif declared: + _STATE["coords"] = [s.strip() for s in declared.split(",") if s.strip()] + else: + _STATE["coords"] = None + _STATE["offset"] = _peak_offset(nals, _STATE["renormalize"]) + print("nal_io: loaded %d NAL artifact(s), chart %s (run frame %s, chart %s); sampler coords " + "%s; contribution centred by %.6g" + % (len(nals), _STATE["set"].coord_names, frame, chart, _STATE["coords"], + _STATE["offset"])) + + +def nal_lnL_offset(): + """The constant `nal_lnL` subtracts (sum of the artifacts' peak lnL, less the truncation mass + when `renormalize` is on -- i.e. exactly the constant that was removed, whichever mode is in + force). + + A fixed multiplicative factor on the likelihood. It cancels in any posterior, but NOT in an + absolute lnL or an evidence: reporting the centred value makes `integral_result.dat` low by + this amount and any odds ratio against a run without the factor wrong by exp(offset). Both + drivers query `_offset`, which is this function for + the `nal_lnL` entry point, and add it back to the absolute quantities they write. Zero before + `prepare_nal_lnL` has run, so a driver may call it unconditionally. + """ + return _STATE["offset"] + + +def nal_lnL(*x): + """Additive lnL contribution. `x` is one array per sampling coordinate, in coord_names order. + + Matches the calling convention in util_ConstructIntrinsicPosterior_GenericCoordinates.py:2952 + and util_ConstructEOSPosterior.py:946 (`log_likelihood_function(*x) + supplemental(*x)`). + + CENTRED: the artifacts' summed peak (`nal_lnL_offset()`) is subtracted, so the return value is + never positive. The drivers' DEFAULT path is not the log one above but + `likelihood_function(*x) * np.exp(supplemental(*x))`, where float64 overflows past ~709 and a + perfectly valid loud-event artifact (lnL_peak ~ SNR^2/2) would return inf for every sample; + the drivers' lnL_shift rescales their own fit, not this separate exponentiation. The + subtracted constant multiplies the likelihood by exp(-offset) and so cancels in any posterior. + """ + if _STATE["set"] is None: + prepare_nal_lnL(config=None, coords=None) # legacy: environment-only configuration + S = _STATE["set"] + names = _STATE["coords"] + arrs = [np.atleast_1d(np.asarray(a, float)).ravel() for a in x] + if names is None: + # Fail closed. Assuming the sampler integrates in the artifact's own chart is NOT the + # safe default: a sampler in (mc, eta) against an artifact in (mc, delta_mc) has the right + # number of arrays, so nothing would raise -- eta would simply be evaluated as delta_mc. + raise ValueError( + "nal_io: the sampling basis is unknown, so the arrays handed to nal_lnL cannot be " + "named. The driver should call prepare_nal_lnL(config=..., coords=); " + "on a RIFT that never reaches the prepare hook, declare the basis explicitly as " + "RIFT_NAL_SAMPLER_COORDS='name1,name2,...' (or [nal] sampler_coords in the ini). " + "Refusing to assume the sampler integrates in the artifact chart %s." % S.coord_names) + if len(arrs) != len(names): + raise ValueError("nal_io: called with %d coordinate array(s) but the declared sampling " + "basis %s has %d -- the arrays would be mislabelled" + % (len(arrs), names, len(names))) + have = dict(zip(names, arrs)) + theta = np.stack([_derive(k, have) for k in S.coord_names], 1) + return S.lnL(theta, renormalize=_STATE["renormalize"]) - _STATE["offset"] diff --git a/MonteCarloMarginalizeCode/Code/RIFT/lalsimutils.py b/MonteCarloMarginalizeCode/Code/RIFT/lalsimutils.py index 2d4b531cc..7804f3162 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/lalsimutils.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/lalsimutils.py @@ -347,7 +347,7 @@ def lsu_StringFromPNOrder(order): # Class to hold arguments of ChooseWaveform functions # -valid_params = ['m1', 'm2', 's1x', 's1y', 's1z', 's2x', 's2y', 's2z', 'chi1_perp', 'chi2_perp', 'chi1_perp_bar', 'chi2_perp_bar','chi1_perp_u', 'chi2_perp_u', 's1z_bar', 's2z_bar', 'lambda1', 'lambda2', 'theta','phi', 'phiref', 'psi', 'incl', 'tref', 'dist', 'mc', 'mc_ecc', 'eta', 'delta_mc', 'chi1', 'chi2', 'thetaJN', 'phiJL', 'theta1', 'theta2', 'cos_theta1', 'cos_theta2', 'theta1_Jfix', 'theta2_Jfix', 'psiJ', 'beta', 'cos_beta', 'sin_phiJL', 'cos_phiJL', 'phi12', 'phi1', 'phi2', 'LambdaTilde', 'DeltaLambdaTilde', 'lambda_plus', 'lambda_minus', 'q', 'mtot','xi','chiz_plus', 'chiz_minus', 'chieff_aligned','fmin','fref', "SOverM2_perp", "SOverM2_L", "DeltaOverM2_perp", "DeltaOverM2_L", "shu","ampO", "phaseO",'eccentricity','eccentricity_squared', 'chi_pavg','mu1','mu2','eos_table_index','meanPerAno'] +valid_params = ['m1', 'm2', 's1x', 's1y', 's1z', 's2x', 's2y', 's2z', 'chi1_perp', 'chi2_perp', 'chi1_perp_bar', 'chi2_perp_bar','chi1_perp_u', 'chi2_perp_u', 's1z_bar', 's2z_bar', 'lambda1', 'lambda2', 'theta','phi', 'phiref', 'psi', 'incl', 'tref', 'dist', 'mc', 'mc_ecc', 'eta', 'delta_mc', 'chi1', 'chi2', 'thetaJN', 'phiJL', 'theta1', 'theta2', 'cos_theta1', 'cos_theta2', 'theta1_Jfix', 'theta2_Jfix', 'psiJ', 'beta', 'cos_beta', 'sin_phiJL', 'cos_phiJL', 'phi12', 'phi1', 'phi2', 'LambdaTilde', 'DeltaLambdaTilde', 'lambda_plus', 'lambda_minus', 'q', 'mtot','xi','chiz_plus', 'chiz_minus', 'chieff_aligned','fmin','fref', "SOverM2_perp", "SOverM2_L", "DeltaOverM2_perp", "DeltaOverM2_L", "shu","ampO", "phaseO",'eccentricity','eccentricity_squared','eccentricity_ln', 'chi_pavg','mu1','mu2','eos_table_index','meanPerAno'] # so far, used for puffball, to prevent insanity (infinite growth) and/or death to downselect # - note we also provide for extrinsic: RA (phi), phiref, psi, just in case we need it in the future @@ -885,6 +885,9 @@ def assign_param(self,p,val): if p == 'eccentricity_squared': self.eccentricity = np.sqrt(val) # value is eccentricity squared return self + if p == 'eccentricity_ln': + self.eccentricity = np.exp(val) # value is ln(eccentricity) + return self # assign an attribute if hasattr(self,p): setattr(self,p,val) @@ -1313,6 +1316,8 @@ def integrand_denominator(S): return dLt if p == 'eccentricity_squared': return self.eccentricity**2 + if p == 'eccentricity_ln': + return np.log(self.eccentricity) if p == 'ecc_cos_meanPerAno': return self.eccentricity*np.cos(self.meanPerAno) if p == 'ecc_sin_meanPerAno': diff --git a/MonteCarloMarginalizeCode/Code/RIFT/misc/container_manifest.py b/MonteCarloMarginalizeCode/Code/RIFT/misc/container_manifest.py index 49076edd6..8559663a0 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/misc/container_manifest.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/misc/container_manifest.py @@ -105,6 +105,14 @@ def _image_runtime_path(image): return image +def _image_basename(image): + """The bare file name an image has once transferred into the job scratch dir. + + Used by the container-universe selector, which MUST NOT contain a ``/``. + """ + return image.rstrip("/").split("/")[-1] + + def _fmt_cap(value): """Format a capability number for a ClassAd expression (e.g. 7.0 -> '7.0').""" return repr(float(value)) @@ -311,17 +319,36 @@ def build_container_image_select(manifest, request_gpu=True): GPU jobs (``request_gpu=True``, the default) get a per-machine selection: an unquoted ``$$([ ... ])`` token. ``$$()`` is HTCondor's *match-time machine-ad substitution* -- the schedd evaluates the bracketed expression against the - matched machine ad and substitutes a literal image string into - ``container_image`` before the job reaches the execution point. Unlike - :func:`build_singularity_image_expr` (an execute-side ClassAd expression that - OSPool glidein pilots read as a literal string and hold on), the pilot only - ever sees a literal URL. ``$$`` in ``container_image`` is HTCondor's - documented mechanism for per-GPU-capability image selection, and it works on - both the CIT-local pool and OSPool glideins. The branch value is the manifest - image *verbatim* (an ``osdf://`` URL the container-universe file-transfer - plugin fetches, or a CVMFS/local path used in place) -- NOT a ``./basename`` - rewrite. ``container_image`` is a single submit command (not a comma list), - so the comma-bearing ``ifThenElse`` form is fine. + matched machine ad and substitutes a literal string before the job reaches the + execution point. Unlike :func:`build_singularity_image_expr` (an execute-side + ClassAd expression that OSPool glidein pilots read as a literal string and hold + on), the pilot only ever sees a literal image name. ``container_image`` is a + single submit command (not a comma list), so the comma-bearing ``ifThenElse`` + form is fine here. + + **The branch values are BASENAMES, not full URLs.** ``condor_submit`` parses + ``container_image`` *before* any ``$$`` expansion and derives the job ad's + ``ContainerImage`` -- the name the image will have in the job scratch dir -- as + the text after the **last** ``/``. A selector containing full paths therefore + gets cut in half, and what survives is not even a valid image name. This is not + theoretical: submitting the full-URL form to the IGWN pool holds the job at the + execute point with:: + + PREPARE_JOB (prepare-hook) failed (reported status 001): + Unable to download or build singularity image cutest_busybox_...sif") ]) + + With no ``/`` in the value, that derivation is a no-op, the whole ``$$`` token + survives into ``ContainerImage``, and the schedd expands it at match time + (``MATCH_EXP_ContainerImage = "rift_container_modern.sif"``) -- verified end to + end on an OSPool glidein. + + Because the selector now names only basenames, the caller MUST also deliver the + matched image itself: add :func:`build_transfer_input_expr` (the comma-free + ternary over the full URLs) to ``transfer_input_files`` **and** emit it as + ``MY.TransferInput`` so it overrides the entry ``condor_submit`` would otherwise + derive from ``container_image``. All images in the family must therefore be + transferable URLs; a family that references an image in place (CVMFS/local path) + cannot be selected this way and raises :class:`ContainerManifestError`. **Non-GPU jobs (``request_gpu=False``) collapse to a SINGLE fixed container**: the plain ``fallback`` image (a literal ``container_image``, no ``$$()``). @@ -339,9 +366,20 @@ def build_container_image_select(manifest, request_gpu=True): by_label = {c["label"]: c for c in manifest["containers"]} fb_image = by_label[manifest["fallback"]]["image"] if not request_gpu: - # Single fixed container: no capability, no $$() -- a plain literal. + # Single fixed container: no capability, no $$() -- a plain literal. This is + # the ordinary single-image path condor_submit handles correctly (it derives + # ContainerImage as the basename, which is exactly right). return fb_image - selector = _build_selector(manifest, lambda c: '"{}"'.format(c["image"])) + in_place = [c["label"] for c in manifest["containers"] if not _image_needs_transfer(c["image"])] + if in_place: + raise ContainerManifestError( + "container universe per-machine selection requires every image in the family " + "to be a transferable URL (e.g. osdf://), because the selector may not contain " + "a '/' -- condor_submit would truncate it. In-place image(s): {}. Either " + "stage those images at a URL, or use RIFT_CONTAINER_RUNTIME_SELECT=1 instead " + "of RIFT_CONTAINER_UNIVERSE=1.".format(", ".join(sorted(in_place))) + ) + selector = _build_selector(manifest, lambda c: '"{}"'.format(_image_basename(c["image"]))) return "$$([ {} ])".format(selector) diff --git a/MonteCarloMarginalizeCode/Code/RIFT/misc/dag_utils_generic.py b/MonteCarloMarginalizeCode/Code/RIFT/misc/dag_utils_generic.py index edb1e94ba..f3f516ce0 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/misc/dag_utils_generic.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/misc/dag_utils_generic.py @@ -2524,11 +2524,14 @@ def write_ILE_sub_simple(tag='integrate', exe=None, log_dir=None, use_eos=False, # Selective transfer: only the matched osdf image is fetched (via the # $$() token, which is comma-free so it survives transfer_input_files # comma-splitting). CVMFS/local images are referenced in place and - # never transferred, so the whole family is never pulled. In container- - # universe mode the image is delivered via container_image itself; in - # runtime-select mode the wrapper self-fetches. In both cases do NOT add - # the match-time transfer token. - if singularity_transfer_expr and not singularity_container_universe and not singularity_runtime_select: + # never transferred, so the whole family is never pulled. + # + # Container universe needs this token too: its container_image selector + # names BASENAMES (it may not contain a '/', or condor_submit truncates + # it -- see build_container_image_select), so the image itself must be + # delivered by file transfer. Runtime-select mode self-fetches inside + # the wrapper, so it is the only mode that skips the token. + if singularity_transfer_expr and not singularity_runtime_select: extra_files += [singularity_transfer_expr] elif singularity_image: if 'osdf:' in singularity_image: @@ -2874,6 +2877,15 @@ def write_ILE_sub_simple(tag='integrate', exe=None, log_dir=None, use_eos=False, fname_str=fname_str.strip() ile_job.add_condor_cmd('transfer_input_files', fname_str) ile_job.add_condor_cmd('should_transfer_files','YES') + if singularity_container_universe: + # condor_submit APPENDS the container_image value to the derived + # TransferInput. Our selector names basenames (it may not contain a + # '/'), so that appended entry would ask the execute point to fetch a + # bare file name from the access point and fail. Set TransferInput + # directly -- emitted after transfer_input_files, it wins -- so the + # list is exactly ours, with the matched image supplied by the + # comma-free $$() ternary already in extra_files. + ile_job.add_condor_cmd('MY.TransferInput', '"' + fname_str.replace('"', '\\"') + '"') if not transfer_output_files is None: if not isinstance(transfer_output_files, list): @@ -3107,13 +3119,14 @@ def write_calpilot_sub(tag='calpilot', exe=None, log_dir=None, universe="vanilla singularity_container_universe = bool(use_singularity and os.environ.get('RIFT_CONTAINER_UNIVERSE')) if singularity_container_universe: singularity_container_image_select = build_container_image_select(_manifest) - else: - # Selective ($$()) transfer of only the matched osdf image (comma-free so - # it survives transfer_input_files comma-splitting). In container-universe - # mode the image is delivered via container_image itself, so skip this. - _transfer_expr = build_transfer_input_expr(_manifest) - if on_osg and _transfer_expr: - transfer_files += [_transfer_expr] + # Selective ($$()) transfer of only the matched osdf image (comma-free so it + # survives transfer_input_files comma-splitting). Container universe needs it + # too: its container_image selector names BASENAMES (it may not contain a '/', + # or condor_submit truncates it), so the image arrives by file transfer. + # (container universe requires use_singularity, which already implies on_osg) + _transfer_expr = build_transfer_input_expr(_manifest) + if on_osg and _transfer_expr: + transfer_files += [_transfer_expr] if use_singularity: base = os.environ.get('SINGULARITY_BASE_EXE_DIR', '/usr/bin/') @@ -3220,8 +3233,14 @@ def write_calpilot_sub(tag='calpilot', exe=None, log_dir=None, universe="vanilla # absolute paths -> condor transfers each to the worker scratch dir by basename, # which is what the stage args (basenames) reference. transfer_files += [wd + "/consolidated_$(macroiteration).composite", ile_args_file] - job.add_condor_cmd('transfer_input_files', ','.join(transfer_files)) + _tif_str = ','.join(transfer_files) + job.add_condor_cmd('transfer_input_files', _tif_str) job.add_condor_cmd('should_transfer_files', 'YES') + if singularity_container_universe: + # condor_submit APPENDS the container_image value to the derived + # TransferInput; our selector names basenames, so that entry would ask + # the execute point to fetch a bare file name and fail. Pin the list. + job.add_condor_cmd('MY.TransferInput', '"' + _tif_str.replace('"', '\\"') + '"') job.add_condor_cmd('when_to_transfer_output', 'ON_EXIT') job.add_condor_cmd('transfer_output_files', 'cal_consolidated_$(macroiteration).npz') # Container-family GPU jobs (CALPILOT runs ILE on a GPU): exclude slots that diff --git a/MonteCarloMarginalizeCode/Code/RIFT/physics/GWSignal.py b/MonteCarloMarginalizeCode/Code/RIFT/physics/GWSignal.py index c83479696..9ada39b95 100644 --- a/MonteCarloMarginalizeCode/Code/RIFT/physics/GWSignal.py +++ b/MonteCarloMarginalizeCode/Code/RIFT/physics/GWSignal.py @@ -198,7 +198,19 @@ def hlmoft(P, Lmax=2,approx_string=None,no_trust_align_method=None,internal_phas .value ) for mode in hlmT: - hlmT[mode].data.data = distance_rescaling*hlmT[mode].data.data + # NOTE THE SIGN. gwsignal's TEOBResumSDALI modes are MINUS the + # polarization convention, not plus. Getting this wrong is not + # visible as a bad fit: a global sign on h is exactly + # psi -> psi + pi/2, so it silently displaces the polarization + # angle by a quarter turn and leaves every other parameter, and + # the peak likelihood, looking fine. + # + # Separately, and NOT corrected here: TEOB's modes also do not + # want the exp(i m phi_shift) applied above, which leaves RIFT's + # reported `phase` for this approximant offset by pi/2 from the + # polarization path. That is a phase-convention difference, not + # a sign error, and it is unchanged by this fix. + hlmT[mode].data.data = -distance_rescaling*hlmT[mode].data.data return hlmT diff --git a/MonteCarloMarginalizeCode/Code/bin/create_event_parameter_pipeline_BasicIteration b/MonteCarloMarginalizeCode/Code/bin/create_event_parameter_pipeline_BasicIteration index f3fb2c707..eba932a0b 100755 --- a/MonteCarloMarginalizeCode/Code/bin/create_event_parameter_pipeline_BasicIteration +++ b/MonteCarloMarginalizeCode/Code/bin/create_event_parameter_pipeline_BasicIteration @@ -294,6 +294,7 @@ parser.add_argument("--cip-exe-G",default=None,help="filename of CIP or equivale parser.add_argument("--use-eccentricity",default=False,action='store_true') parser.add_argument("--use-meanPerAno",default=False,action='store_true') parser.add_argument("--use-eccentricity-squared-sampling",default=False,action='store_true') +parser.add_argument("--use-eccentricity-ln-sampling",default=False,action='store_true') parser.add_argument("--use-tabular-eos-file",default=False,action='store_true') parser.add_argument("--test-exe",default=None,help="filename of test code or equivalent executable. Must have a --test-output argument. Used for convergence testing or other termination. NOT ACTIVE; see 'convergence_test_samples.py' for example") parser.add_argument("--plot-exe",default=None,help="filename of plot code or equivalent executable. Will default to `which plot_posterior_corner.py`. Default is to plot last set of samples") @@ -1434,6 +1435,8 @@ else: cip_args_extra +=" --n-eff {} --n-output-samples {} ".format(n_samples_per_job,n_samples_per_job) if not (opts.use_eccentricity_squared_sampling): cip_args_lines[indx] = cip_args_lines[indx].replace(' --parameter eccentricity_squared ',' --parameter-implied eccentricity_squared --parameter-nofit eccentricity ') + if not (opts.use_eccentricity_ln_sampling): + cip_args_lines[indx] = cip_args_lines[indx].replace(' --parameter eccentricity_ln ',' --parameter-implied eccentricity_ln --parameter-nofit eccentricity ') transfer_files_cip=['../all.net'] if opts.use_osg_cip and 'fit-method gp' in cip_args_base: transfer_files_cip += ['my_fit.pkl'] # transfer current working directory fit diff --git a/MonteCarloMarginalizeCode/Code/bin/integrate_likelihood_extrinsic_batchmode b/MonteCarloMarginalizeCode/Code/bin/integrate_likelihood_extrinsic_batchmode index 359ab921a..a8b40987c 100755 --- a/MonteCarloMarginalizeCode/Code/bin/integrate_likelihood_extrinsic_batchmode +++ b/MonteCarloMarginalizeCode/Code/bin/integrate_likelihood_extrinsic_batchmode @@ -341,7 +341,7 @@ integration_params.add_option("--manual-logarithm-offset",type=float,default=0,h integration_params.add_option("--auto-logarithm-offset",action='store_true',help="Use the 'guess_snr' field returned in the precompute stage to change --manual-logarithm-offset for each event.") integration_params.add_option("--internal-use-lnL",action='store_true',help="likelihood returns lnL, and integrator integrates lnL") integration_params.add_option("--sampler-method",default="adaptive_cartesian_gpu",help="adaptive_cartesian|GMM|adaptive_cartesian_gpu") -integration_params.add_option("--sampler-portfolio",default=None,action='append',type=str,help="comma-separated strings, matching sampler methods other than portfolio") +integration_params.add_option("--sampler-portfolio",default=None,action='append',type=str,help="Portfolio member sampler, one of AV / GMM / AC (adaptive_cartesian_gpu) or a discovered plugin. Repeat the option per member, or give one comma-separated list; both forms may be mixed. An unrecognized name is an error.") integration_params.add_option("--sampler-portfolio-args",default=None, action='append', type=str, help='eval-able dictionaryo to be passed to that sampler') # Portfolio freeze-policy knobs (only meaningful with --sampler-method portfolio). A member # whose balance weight drops below --portfolio-freeze-wt normally stops updating its proposal; @@ -377,7 +377,7 @@ integration_params.add_option("--sampler-sequential-warmstart",action='store_tru integration_params.add_option("--sampler-sequential-warmstart-cover-frac",type=float,default=0.5,help="Coverage floor for --sampler-sequential-warmstart: fraction of full-prior coverage mixed into the seed so the warm live volume always contains a cold start (a mis-matched proposal then only costs efficiency, never bias). Default 0.5, the measured-safe floor (see --sampler-warmstart-cover-frac); 0.1 is under-covered.") integration_params.add_option("--sampler-sequential-warmstart-deltalnL",type=float,default=15.0,help="Keep previous-point samples within this lnL of the max as the warm seed for the next point. Default 15.") integration_params.add_option("--sampler-l0-rescue-accept-truncated", action='store_true', default=False, help="Report the L0 rescue's warm pass even when it lands well below the full-support cold pass (see --sampler-l0-rescue-reject-dlnZ). Default OFF: on that evidence the cold result is kept instead, since the warm pass is confined to the seeded peak and may be missing a mode. The rescue itself still runs either way.") -integration_params.add_option("--sampler-l0-rescue-reject-dlnZ", type=float, default=0.5, help="Evidence threshold (nats) for rejecting the L0 rescue's warm pass: reject when the full-support cold pass reports lnZ this much HIGHER, which indicates the seed missed mass. Larger = more permissive.") +integration_params.add_option("--sampler-l0-rescue-reject-dlnZ", type=float, default=3.0, help="Evidence threshold (nats) for rejecting the L0 rescue's warm pass: reject when the full-support cold pass reports lnZ this much HIGHER, which would indicate the seed missed mass. Larger = more permissive. DEFAULT RAISED 0.5 -> 3.0 ON MEASUREMENT (see test/expensive_before_merging/integrators/L0_REJECT_DLNZ_MEASUREMENT.md): across 160 known-lnZ passes the gate caught 0 of 55 genuinely truncated warm passes at EVERY threshold, while at 0.5 it binned 25% of GOOD portfolio warm passes. 0.5 was therefore strictly dominated -- it bought no detection and cost one good pass in four. 3.0 keeps a safety net for a genuinely large discrepancy at ~0% false-positive rate. This gate is NOT a working truncation detector; do not rely on it as one.") integration_params.add_option("--sampler-l0-rescue-puff-scale", type='choice', choices=['fixed','auto'], default='auto', help="How wide to puff the L0 rescue's seed when it is rank-deficient in the adaptive dimensions. 'auto' (default) measures the posterior scale AND correlations from every finite lnL the collapsed pass already drew; 'fixed' uses --sampler-l0-rescue-puff-width-frac of each parameter's prior range, which is the historical behaviour and knows nothing about the posterior (which narrows as 1/rho). 'auto' falls back to 'fixed' when there are too few finite points to estimate a covariance.") integration_params.add_option("--sampler-l0-rescue-puff-width-frac", type=float, default=0.005, help="Isotropic puff width for the L0 rescue's rank-deficient seed, as a fraction of each parameter's prior range. Used by --sampler-l0-rescue-puff-scale fixed, and as the 'auto' fallback. Default 0.005 = the historical hardcoded 1/200.") integration_params.add_option("--sampler-l0-rescue-puff-factor", type=float, default=2.0, help="Multiply the L0 rescue's puff width by this factor. Default 2 is the measured optimum on a known-lnZ 6-D target (mean lnZ error +0.08 nats, ESS 52); BOTH tails are wrong, so do not treat wide as free -- x0.5 truncates (-8.5 nats), x6 biases high (+3.0) and costs efficiency, x12 is a cold start in all but name and re-collapses (-30).") @@ -1236,7 +1236,11 @@ elif opts.sampler_method == 'AV': mcsampler.set_xpy_to_numpy() sampler.xpy= numpy sampler.identity_convert= lambda x: x -elif opts.sampler_method == "portfolio" and mcsampler_Portfolio_ok: +elif opts.sampler_method == "portfolio": + # NB the `and mcsampler_Portfolio_ok` that used to be part of this test made the raise + # below dead code AND sent an unavailable portfolio to the `else` fallback at the end of + # this chain, which silently runs the plain mcsampler.MCSampler instead. Requesting a + # sampler that cannot be built must fail, not quietly become a different sampler. if not(mcsampler_Portfolio_ok): raise Exception(" Portfolio integrator requested but not available") use_portfolio=True @@ -1246,7 +1250,11 @@ elif opts.sampler_method == "portfolio" and mcsampler_Portfolio_ok: # flatten the appended list and split every element on ',' (e.g. ['AV,GMM'] -> ['AV','GMM'], # ['AV','GMM'] -> ['AV','GMM']). Without this, 'AV,GMM' was one bogus member name that matched # no branch, silently yielding a single-member portfolio. - sampler_types = [s.strip() for item in opts.sampler_portfolio for s in str(item).split(',') if s.strip()] + # The `or []` matters because the option defaults to None: omitting it entirely used to raise + # TypeError from this comprehension rather than saying what the user actually got wrong. + sampler_types = [s.strip() for item in (opts.sampler_portfolio or []) for s in str(item).split(',') if s.strip()] + if not sampler_types: + raise Exception(" --sampler-method portfolio requires at least one --sampler-portfolio member") # prep xpy, etc my_xpy = xpy_default @@ -1277,6 +1285,12 @@ elif opts.sampler_method == "portfolio" and mcsampler_Portfolio_ok: mcsampler = mcsamplerGPU # force use of routines in that file, for properly configured GPU-accelerated code as needed elif name in mcsamplerPortfolio.known_pipelines: # everything else, including nflow sampler = mcsamplerPortfolio.known_pipelines[name]() + else: + # No else clause here meant an unrecognized name left `sampler` bound to its + # previous value -- the plain MCSampler built before this chain, or, on the second + # and later iterations, the PREVIOUS member -- and appended it silently. The + # portfolio then ran with a member the user never asked for. + raise Exception(" --sampler-portfolio: unknown member '{}'. Known: AV, GMM, AC/adaptive_cartesian_gpu, {}".format(name, sorted(mcsamplerPortfolio.known_pipelines))) print('PORTFOLIO: adding {} '.format(name)) # enable xpy for low level sampler as needed if hasattr(sampler, 'xpy'): @@ -1289,7 +1303,7 @@ elif opts.sampler_method == "portfolio" and mcsampler_Portfolio_ok: sampler.identity_convert= my_identity_convert sampler.identity_convert_togpu= my_identity_convert_togpu # sampler weights will be CPU-typed, so don't change them -elif opts.sampler_method in mcsamplerPortfolio.known_pipelines: # access from plugins +elif mcsampler_Portfolio_ok and opts.sampler_method in mcsamplerPortfolio.known_pipelines: # access from plugins sampler = mcsamplerPortfolio.known_pipelines[opts.sampler_method]() # prep xpy, etc my_xpy = xpy_default @@ -1300,7 +1314,10 @@ elif opts.sampler_method in mcsamplerPortfolio.known_pipelines: # access from pl sampler.identity_convert_togpu= my_identity_convert_togpu else: print(" ILE: **original sampler** ") - print(" ILE requested: {}".format(opts.sampler_method), " compare to ", mcsamplerPortfolio.known_pipelines) + # mcsamplerPortfolio is only bound if its import succeeded; reaching this line with a + # failed import used to raise NameError from the diagnostic itself. + print(" ILE requested: {}".format(opts.sampler_method), " compare to ", + sorted(mcsamplerPortfolio.known_pipelines) if mcsampler_Portfolio_ok else "") # # Psi -- polarization angle @@ -2123,6 +2140,67 @@ def _rvs_len(rvs): return 0 +def _rvs_is_export_resample(sampler): + """True when the ROWS of _rvs were drawn in proportion to weight. + + Set by the samplers at the rebind itself, so it means "the draw FIRED", which is not the + same predicate as `opts.fairdraw_extrinsic_output`: the draw is skipped when it would not + shrink the record (n_extr >= len(_rvs)), and then the rows are still the retained set with + real importance weights. Keying off the CLI flag would flatten those, which is the same + class of error in the other direction. + + SURVIVES POOLING. A pooled record built from fair-drawn replicas still has + posterior-resampled rows, so anything that must not re-weight such rows -- the .dslice + reweight core -- has to keep seeing True here. Whether the record is GLOBALLY equal-weight + is a different question with a different answer; see _rvs_is_equal_weight. + """ + return bool(getattr(sampler, '_rvs_is_fairdraw', False)) + + +def _rvs_is_equal_weight(sampler): + """True when EVERY row of _rvs carries the same posterior weight. + + Two properties were briefly conflated here, and separating them is the whole point: + + rows resampled -- each row was drawn proportional to w (per-BLOCK property) + equal weight -- the record as a whole is uniform (property of the WHOLE record) + + A single fair draw has both. A POOLED record has the first and not the second: + _pool_replica_rvs gives block k weights summing to Z_k/K, equal within a block but + differing between blocks by exactly the replica evidences. Answering the second question + with the first flag made .dgrid and the proposal breadcrumb mix replicas by exported row + count instead of by evidence; answering the first with the second made the .dslice + safeguard and the block-Kish n_eff branch unreachable. Both are wrong, in opposite + directions, from one boolean. + """ + return (bool(getattr(sampler, '_rvs_is_fairdraw', False)) + and not bool(getattr(sampler, '_rvs_is_pooled', False))) + + +def ln_weights_for_posterior(rvs, sampler, convert=None, use_lnL=None): + """The weights to use when treating an _rvs record as a POSTERIOR SAMPLE SET. + + NOT the same question as `ln_weights_from_rvs`, which answers "what is the importance + weight of this record" and is always right about that. The question here is "how should + these rows be weighted to represent the posterior", and the answer depends on whether the + fair draw already did it. + + A fair-drawn record was resampled WITH REPLACEMENT proportional to w, so its rows are + already an equal-weight draw from the posterior. Weighting them by w a second time + applies w^2 and over-concentrates the result -- measured at a 13% shift in the posterior + mean of a weight-correlated coordinate (verify_skew.py). `_pool_replica_rvs` has guarded + against exactly this since the replica work, via `already_resampled`; the .dgrid exporter + and the extrinsic-proposal breadcrumb did not, and both fed science products. + + So: uniform (zero log-weight) for a fair-drawn record, the derived importance weight + otherwise. Returns a float array the length of the record. + """ + if _rvs_is_equal_weight(sampler): + return numpy.zeros(_rvs_len(rvs), dtype=float) + return numpy.asarray(ln_weights_from_rvs(rvs, convert=convert, use_lnL=use_lnL), + dtype=float) + + def _pool_replica_rvs(rep_rvs, sampler, rep_lnZ=None, already_resampled=False, use_lnL=None): """Concatenate the replicas' samples into one correctly-weighted set. @@ -2141,7 +2219,29 @@ def _pool_replica_rvs(rep_rvs, sampler, rep_lnZ=None, already_resampled=False, u block's weights, and the equation to solve is convention-dependent (see below). """ _lnL_here = _rvs_lnL_convention(use_lnL) - rep_rvs = [r for r in rep_rvs if r] + # `already_resampled` may be a single bool or a PER-REPLICA sequence. It has to be the + # latter in general: each pass decides independently whether to fair-draw (the draw is + # skipped when it would not shrink that pass's record), so one global flag either flattens + # a replica whose weights are genuine, or leaves a resampled replica double-weighted. A + # mixture of raw and resampled replicas is the normal case near the n_extr boundary. + _ar_list = (list(already_resampled) + if isinstance(already_resampled, (list, tuple, numpy.ndarray)) + else None) + # Drop empty records in LOCKSTEP with their metadata. The filter used to run on rep_rvs + # alone, so a single empty replica shifted every later block against its own lnZ -- and + # would now shift it against its own resampled flag too. + _keep = [i for i, r in enumerate(rep_rvs) if r] + rep_rvs = [rep_rvs[i] for i in _keep] + if rep_lnZ is not None: + rep_lnZ = [rep_lnZ[i] for i in _keep if i < len(rep_lnZ)] + if _ar_list is not None: + _ar_list = [_ar_list[i] for i in _keep if i < len(_ar_list)] + + def _block_resampled(i): + if _ar_list is not None: + return bool(_ar_list[i]) if i < len(_ar_list) else False + return bool(already_resampled) + if len(rep_rvs) <= 1: return rep_rvs[0] if rep_rvs else {} # `already_resampled` -- the records are FAIRDRAW output. Those samples were already drawn in @@ -2176,7 +2276,7 @@ def _pool_replica_rvs(rep_rvs, sampler, rep_lnZ=None, already_resampled=False, u if n_k <= 0: continue _flat_block = False - if already_resampled and rep_lnZ is not None and _i < len(rep_lnZ) \ + if _block_resampled(_i) and rep_lnZ is not None and _i < len(rep_lnZ) \ and numpy.isfinite(rep_lnZ[_i]): # equal weights within the block, summing to Z_k/K _flat_block = True @@ -2305,6 +2405,131 @@ def _kish_neff_of_rvs(rvs, use_lnL=None): return None +def _lnZ_of_reserve_or_rvs(sampler, rvs, reserve=None): + """lnZ of a completed pass, from the points it RETAINED where that is available. + + The L0 rescue's reject gate compares the warm pass's lnZ against the cold pass's, and + both were read out of _rvs -- which the fair draw has already replaced with + min(n_extr, 1.5*eff_samp, 1.5*neff) rows resampled WITH REPLACEMENT, proportional to + weight. That is not a smaller unbiased sample of the same estimator, it is a DIFFERENT + and biased one: _lnZ_of_rvs forms logsumexp(w)/n, so drawing n rows proportional to w + returns something near max(w) rather than mean(w), high by roughly + + log(n_retained / eff_samp) + + and the two passes are drawn at wildly different n and eff_samp. On the collapsed cold + pass at rho_net 146.8 that is ONE row out of 1000 retained at eff_samp ~ 1 -- about 7 + nats high -- against 5 rows at eff_samp ~ 8 for the warm pass. So the gate was reading a + 4.2-nat artifact of its own two subsample sizes as evidence that the warm seed had missed + mass, and rejecting the warm pass in 10 of 12 replicates on it. + + Falls back to the old _rvs reading when no reserve was kept (a sampler that does not + keep one, or a pass that raised), so the comparison degrades to the previous behaviour + rather than to no gate at all. + + Returns (lnZ, source) -- the caller MUST check that both sides of the comparison came + from the same source, because the two readings are not interchangeable: mixing them is + the same apples-to-oranges error in a new place. + """ + _res = reserve if reserve is not None else getattr(sampler, '_warm_seed_reserve', None) + if isinstance(_res, dict) and 'log_joint_prior' in _res and 'log_joint_s_prior' in _res: + try: + # NOT _lnZ_of_rvs: it averages over the rows it is handed, and the reserve is + # neither the draw set nor a uniform sample of it -- non-finite rows were dropped + # and the remainder may have been capped. lnZ_from_reserve restores the original + # proposal-draw normalization from n_finite/n_retained. Without it a PORTFOLIO + # reading is high by ~log(n_retained/n_finite), ~11 nats on a collapsed pass, and + # the error does NOT cancel in the gate: the cold and warm passes have different + # finite fractions, so it is the difference of two different-sized errors. + _v = mcsamplerAdaptiveVolume.lnZ_from_reserve(_res) + if _v is not None and numpy.isfinite(_v): + return _v, 'retained' + except Exception: + pass + return _lnZ_of_rvs(rvs, already_pooled=False), 'fairdraw' + + +def _snapshot_pass_state(sampler, res, var, neff, dict_return, rvs=None): + """Everything that must move TOGETHER when a completed pass is put back -> dict. + + THE POINT IS THE WORD "everything". A pass is described by more than its samples, and the + L0 rescue's reject path restored only some of it: `_rvs`, the estimate and `dict_return` + went back to the cold pass while `_warm_seed_reserve` was left holding the REJECTED warm + cloud. Nothing read that attribute at the time, so it was latent -- until + --sampler-sequential-warmstart began seeding the next intrinsic point from the reserve, + at which point a rejected, truncated warm pass became the seed for the following point. + That is the exact failure the rescue's own reject gate exists to prevent, reintroduced one + attribute over. + + So the snapshot carries the reserve and the fair-draw marker as well, including the + per-member reserves: `_warm_seed_reserve_for` falls through to `portfolio_realizations`, + so restoring only the aggregate would leave that fallback pointing at the warm pass. + """ + return dict( + rvs=(dict(sampler._rvs) if rvs is None else rvs), + res=res, var=var, neff=neff, dict_return=dict_return, + warm_seed_reserve=getattr(sampler, '_warm_seed_reserve', None), + rvs_is_fairdraw=bool(getattr(sampler, '_rvs_is_fairdraw', False)), + rvs_is_pooled=bool(getattr(sampler, '_rvs_is_pooled', False)), + member_reserves=[getattr(_m, '_warm_seed_reserve', None) + for _m in list(getattr(sampler, 'portfolio_realizations', []) or [])], + ) + + +def _restore_pass_state(sampler, state): + """Undo of _snapshot_pass_state -> (res, var, neff, dict_return). + + Both callers (the reject path and the exception handler) go through here, so the set of + attributes that travels with a restored pass cannot drift between them. + """ + sampler._rvs = state['rvs'] + sampler._warm_seed_reserve = state['warm_seed_reserve'] + sampler._rvs_is_fairdraw = state['rvs_is_fairdraw'] + sampler._rvs_is_pooled = state['rvs_is_pooled'] + _members = list(getattr(sampler, 'portfolio_realizations', []) or []) + for _m, _r in zip(_members, state.get('member_reserves', [])): + _m._warm_seed_reserve = _r + return state['res'], state['var'], state['neff'], state['dict_return'] + + +def _warm_seed_reserve_for(sampler): + """The retained-sample reserve a completed pass left behind, or None. + + THE ONE LOOKUP FOR SEED CONSUMERS, because two of them need exactly this record and must + not drift: the L0 auto-rescue (which re-seeds a collapsed pass from its own peak) and the + --sampler-sequential-warmstart capture (which seeds the NEXT intrinsic point). Both + otherwise fall back to sampler._rvs, which by then has been rebound to a fair-draw + subset of min(n_extr, 1.5*eff_samp, 1.5*neff) rows taken WITH REPLACEMENT -- and the + whole point of the reserve is that on the collapsed pass a warm start exists for, that + subset is a handful of rows several of which are the same point twice. + + A PORTFOLIO keeps the reserve on the aggregate, not on its members, but a bare AV member + can be the one that has it; check the sampler first, then its realizations. + + COLUMN ORDER MUST MATCH or the seed is scrambled: the reserve stores X in the column + order of the sampler that built it, and a seed handed to bootstrap_from_samples is read + positionally against params_ordered. A mismatch is silent and produces a seed in the + wrong coordinates, so decline the reserve rather than use it. + + NOT SHARED WITH `_lnZ_of_reserve_or_rvs` ABOVE, deliberately. That one wants a different + record: it reads only lnL and the two prior columns, never X, so a column-order mismatch + is harmless to it and declining on one would throw away a perfectly good lnZ reading and + silently downgrade the gate to its fair-draw fallback. Two lookups with two different + admissibility rules is correct; collapsing them would be the kind of false unification + that produces the next defect. (`_lnZ_of_reserve_or_rvs` does not do the portfolio-member + fallback either -- worth a look, but that is a change to just-merged #79, not a rebase.) + """ + _res = getattr(sampler, '_warm_seed_reserve', None) + if _res is None: + for _m in list(getattr(sampler, 'portfolio_realizations', []) or []): + _res = getattr(_m, '_warm_seed_reserve', None) + if _res is not None: + break + if _res is not None and list(_res.get('params_ordered', [])) != list(sampler.params_ordered): + return None + return _res + + def _warm_seed_geometry(sampler): """Which columns a warm seed must span, and the box it must lie in -> (axes, lo, hi). @@ -2350,6 +2575,22 @@ def _clear_warm_state(sampler): def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_trunc_Q=inv_spec_trunc_Q, T_spec=T_spec): nEvals=0 + # PROVENANCE RESET, ON ENTRY, BEFORE ANYTHING CAN FAIL. + # + # _rvs_is_pooled describes the record this call is about to build, and it is set by THIS + # function (the replica block) rather than by a sampler, so no sampler-side per-pass reset + # can clear it. Clearing it only on the normal return is not enough: _reject_if_collapsed + # RAISES after pooling, the caller's `except Exception` swallows that and moves to the next + # event, and the marker survives. The next ordinary fair draw is then read as "pooled", + # _rvs_is_equal_weight goes False, and .dgrid and the extrinsic-proposal breadcrumb apply + # importance weights to rows that already carry them -- the w^2 defect this whole change + # exists to remove, resurrected on the event after any failure. + # + # On ENTRY rather than in a `finally`: entry is reached on every call by construction, + # needs no restructuring of a 2000-line function, and leaves the state correct even for a + # caller that never returns normally at all. The end-of-function clear stays as well, so + # a sampler handed to anything else afterwards is not carrying a stale marker. + sampler._rvs_is_pooled = False P = P_list[indx_event] # if pin-distance-to-sim, change the distance prior accordingly if opts.pin_distance_to_sim: @@ -3275,14 +3516,9 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # is where "5 seed points of affine rank 2" came from. The live set held a # thousand. integrate_log now stashes a bounded copy of the retained points # before that overwrite; fall back to _rvs for a sampler that does not keep one. - _res_l0 = getattr(sampler, '_warm_seed_reserve', None) - if _res_l0 is None: - for _m in list(getattr(sampler, 'portfolio_realizations', []) or []): - _res_l0 = getattr(_m, '_warm_seed_reserve', None) - if _res_l0 is not None: - break - if _res_l0 is not None and list(_res_l0.get('params_ordered', [])) != list(sampler.params_ordered): - _res_l0 = None # column order must match, or the seed is scrambled + # Shared with the --sampler-sequential-warmstart capture below; see + # _warm_seed_reserve_for for the portfolio fallback and the column-order guard. + _res_l0 = _warm_seed_reserve_for(sampler) if _res_l0 is not None: _cols = np.asarray(_res_l0['X'], dtype=float) _lnv = np.asarray(_res_l0['lnL'], dtype=float).ravel() @@ -3362,15 +3598,35 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # restore below ran -- i.e. the reject path would report the cold lnZ while # exporting the warm cloud, exactly what it exists to prevent. _cold_rvs = dict(sampler._rvs) - _cold_lnZ = _lnZ_of_rvs(_cold_rvs, already_pooled=False) + # Snapshot the RESERVE for the same reason and at the same moment: the warm + # pass's integrate_log clears and rewrites it, so reading it after the fact + # would compare the warm pass against itself. + _cold_reserve_l0 = getattr(sampler, '_warm_seed_reserve', None) + _cold_lnZ, _cold_src = _lnZ_of_reserve_or_rvs(sampler, _cold_rvs, + reserve=_cold_reserve_l0) # dict_return too: khat, block scatter, ESS, the confidence interval and the # replica trigger downstream all read it, so keeping the warm pass's diagnostics - # beside a restored cold result would describe a run we did not report. - _cold_res, _cold_var, _cold_neff, _cold_dict = res, var, neff, dict_return - _cold_state_l0 = (_cold_rvs, _cold_res, _cold_var, _cold_neff, _cold_dict) + # beside a restored cold result would describe a run we did not report. And the + # RESERVE and the fair-draw marker, for the same reason one level out -- see + # _snapshot_pass_state. + _cold_state_l0 = _snapshot_pass_state(sampler, res, var, neff, dict_return, + rvs=_cold_rvs) sampler.bootstrap_from_samples(_seed, cover_frac=0.0) res, var, neff, dict_return = sampler.integrate(like_to_integrate, *unpinned_params, **pinned_params) - _warm_lnZ = _lnZ_of_rvs(sampler._rvs, already_pooled=False) + _warm_lnZ, _warm_src = _lnZ_of_reserve_or_rvs(sampler, sampler._rvs) + # BOTH SIDES FROM THE SAME READING, or the difference is not a difference. + # A fair-drawn lnZ sits ~log(n_retained/eff_samp) above a retained-set one, so + # a mixed comparison manufactures a gap of several nats in whichever direction + # the mismatch happens to fall. If the two passes did not produce the same + # kind of estimate, fall back to reading BOTH from _rvs -- the old behaviour, + # which is at least self-consistent -- rather than compare across conventions. + if _cold_src != _warm_src: + print(" [L0 auto-rescue] lnZ provenance differs (cold={}, warm={});" + " re-reading both from the fair-draw record so the comparison is" + " like-for-like.".format(_cold_src, _warm_src)) + _cold_lnZ = _lnZ_of_rvs(_cold_rvs, already_pooled=False) + _warm_lnZ = _lnZ_of_rvs(sampler._rvs, already_pooled=False) + _cold_src = _warm_src = 'fairdraw' _evidence_of_loss = ( (_cold_lnZ is not None) and (_warm_lnZ is not None) and numpy.isfinite(_cold_lnZ) and numpy.isfinite(_warm_lnZ) @@ -3389,8 +3645,11 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t print(" [L0 auto-rescue] keeping the COLD (full-support) result; its n_eff" " is lower but it is not missing mass. A portfolio avoids this" " trade entirely -- its GMM member carries a defensive component.") - sampler._rvs = _cold_rvs - res, var, neff, dict_return = _cold_res, _cold_var, _cold_neff, _cold_dict + # The RESERVE goes back too. Without it --sampler-sequential-warmstart + # seeds the next intrinsic point from the warm cloud this gate just + # rejected: _warm_seed_reserve_for would return the warm pass's record + # while _rvs, the estimate and the diagnostics all describe the cold one. + res, var, neff, dict_return = _restore_pass_state(sampler, _cold_state_l0) _clear_warm_state(sampler) except Exception as _e_l0: # "skipped" is only true if the warm pass never started. If it raised PARTWAY THROUGH @@ -3406,7 +3665,7 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t print(" [L0 auto-rescue] the warm pass may already have replaced the stored" " samples; restoring the COLD pass so the reported diagnostics and the" " exported samples describe the same integral.") - sampler._rvs, res, var, neff, dict_return = _cold_state_l0 + res, var, neff, dict_return = _restore_pass_state(sampler, _cold_state_l0) _clear_warm_state(sampler) # Persist adapted state / trained flow for reuse by later instances. @@ -3513,6 +3772,13 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # replica scores HIGHEST and would be the one exported. (Measured elsewhere in this work: # the copy with the highest n_eff in its arm was the most biased, 11 nats low.) _rep_rvs = [sampler._rvs] + # Provenance PER REPLICA, captured beside the record it describes. The CLI flag is + # not this: each pass decides independently whether to fair-draw (skipped when it + # would not shrink that pass's record), so passing opts.fairdraw_extrinsic_output to + # the pooler either flattens a replica whose importance weights are genuine, or leaves + # a resampled replica double-weighted. Near the n_extr boundary a run can produce a + # MIXTURE of raw and resampled replicas, which one global boolean cannot describe. + _rep_fairdraw = [bool(getattr(sampler, '_rvs_is_fairdraw', False))] # Collapse status must be aggregated over EVERY replica that ends up in the pool. # The exported posterior is the pooled mixture, so one collapsed replica taints it # even if the first run was healthy -- and the status sidecar is written from @@ -3570,6 +3836,7 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t _sig2 = max(float(_sig2), float(_sb2)) _rep_lnZ.append(float(_lr2)); _rep_sig.append(float(_sig2)); _rep_neff.append(float(_neff2)) _rep_rvs.append(sampler._rvs) + _rep_fairdraw.append(bool(getattr(sampler, '_rvs_is_fairdraw', False))) _rep_collapsed.append(bool(_dd2.get('live_volume_collapsed', False)) if isinstance(_dd2, dict) else False) if isinstance(_dd2, dict) and _dd2.get('collapse_reason'): @@ -3582,9 +3849,35 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # Folding the factor into log_joint_s_prior is therefore a statement of the real pooled # sampling density, not a fudge -- and it leaves every downstream weight computation # (which all form log_integrand + log_joint_prior - log_joint_s_prior) correct untouched. - sampler._rvs = _pool_replica_rvs(_rep_rvs, sampler, rep_lnZ=_rep_lnZ, - already_resampled=bool(opts.fairdraw_extrinsic_output), - use_lnL=rvs_integrand_is_lnL) + _pooled_rvs = _pool_replica_rvs(_rep_rvs, sampler, rep_lnZ=_rep_lnZ, + already_resampled=_rep_fairdraw, + use_lnL=rvs_integrand_is_lnL) + # A POOLED RECORD IS NOT A FAIR DRAW, even when every block that went into it was. + # _pool_replica_rvs gives block k weights summing to Z_k/K: equal WITHIN a block (each + # block really is an equal-weight draw from its own posterior) but differing BETWEEN + # blocks by exactly the replica evidences. Leaving the marker set would make + # ln_weights_for_posterior return zeros, and .dgrid and the proposal breadcrumb would + # then mix the replicas by exported ROW COUNT instead of by evidence -- silently + # discarding the disagreement the replicas were run to measure. The reconstructed + # per-row weights already encode it, so clear the marker and let them be read. + # + # Only when it actually pooled: every fallback path in _pool_replica_rvs returns one of + # its INPUT records unchanged (too few replicas, no sampling-prior column, an exception), + # and such a record is still the fair draw it arrived as. Identity, not length, is the + # reliable test for that. + _did_pool = not any(_pooled_rvs is _r for _r in _rep_rvs) + if _did_pool: + # POOLED, not equal-weight. The rows are still posterior-resampled wherever their + # block was (so _rvs_is_fairdraw stays, and the .dslice safeguard keeps firing), + # but the record as a whole is a mixture weighted by the replica evidences, so + # ln_weights_for_posterior must read the reconstructed per-row weights. + sampler._rvs_is_pooled = True + sampler._rvs_is_fairdraw = any(_rep_fairdraw) + # Did pooling FLATTEN any block? That, not "is the record resampled", is what makes + # the pooled Kish n_eff meaningless below -- a flattened block's rows carry its export + # size rather than its integration quality. + _blocks_flattened = bool(_did_pool and any(_rep_fairdraw)) + sampler._rvs = _pooled_rvs # The pooled export is a mixture over every replica in _rep_rvs, so its collapse # status is the OR over them: one collapsed member taints the pool. Fold that back # into dict_return, which is what the status sidecar and the downstream reporting @@ -3625,12 +3918,38 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # combined effective sample size of K independent runs, which is only true if they # agree; when they disagree -- the case these replicas exist to detect -- the pooled # Kish n_eff is smaller, and that disagreement is exactly what should show up here. - _neff_pooled = _kish_neff_of_rvs(sampler._rvs) + # + # ...but NOT the Kish n_eff OF THE POOLED RECORD when that record is the fair-draw + # export. _pool_replica_rvs deliberately FLATTENS each block in that case (equal + # weights within a block, summing to Z_k/K), and the Kish n_eff of piecewise-constant + # weights is just the row count -- i.e. K*min(n_max, 1.5*eff_samp, 1.5*neff), the size + # of the EXPORT, which says nothing about how well the integral converged. With + # --fairdraw-extrinsic-output-n-max at its default of 5 that reports n_eff = 5K. + # + # Do the same computation one level up, where the quantities are still meaningful: + # Kish over the BLOCKS, each carrying its own Z_k and its own n_eff, + # + # neff_pooled = (sum_k Z_k)^2 / sum_k (Z_k^2 / neff_k) + # + # which has exactly the property the paragraph above asks for: it reduces to + # sum_k neff_k when the replicas agree, and falls below it when they disagree -- + # the disagreement these replicas exist to detect. + if _blocks_flattened: + _l_rel = numpy.asarray(_rep_lnZ, dtype=float) - float(numpy.max(_rep_lnZ)) + _Zk = numpy.exp(_l_rel) + _nk = numpy.asarray(_rep_neff, dtype=float) + _ok = numpy.isfinite(_Zk) & numpy.isfinite(_nk) & (_nk > 0) + _neff_pooled = (float(numpy.sum(_Zk[_ok]) ** 2 / numpy.sum(_Zk[_ok] ** 2 / _nk[_ok])) + if numpy.any(_ok) else None) + _neff_how = 'block Kish over replicas (the export is fair-drawn)' + else: + _neff_pooled = _kish_neff_of_rvs(sampler._rvs) + _neff_how = 'Kish over the pooled samples' neff = float(_neff_pooled) if _neff_pooled is not None else float(numpy.sum(_rep_neff)) if _neff_pooled is not None: - print(" [mc error] pooled posterior: {} samples, Kish n_eff {:.1f} (sum over replicas was {:.1f})".format( + print(" [mc error] pooled posterior: {} samples, n_eff {:.1f} via {} (sum over replicas was {:.1f})".format( len(numpy.atleast_1d(list(sampler._rvs.values())[0])) if sampler._rvs else 0, - float(_neff_pooled), float(numpy.sum(_rep_neff)))) + float(_neff_pooled), _neff_how, float(numpy.sum(_rep_neff)))) # keep the (res, var) pair consistent for any downstream reader if not(opts.internal_use_lnL): res = numpy.exp(log_res); var = (sqrt_var_over_res*res)**2 @@ -3674,8 +3993,13 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # Use the ONE canonical derivation. The inline copy that used to live here carried the # same linear-only assumption ln_weights_from_rvs was just fixed for, so a GMM run storing # lnL in 'integrand' fitted the handoff proposal to log(lnL)-flattened weights. - _lw = np.asarray(ln_weights_from_rvs(_rvs, convert=identity_convert, - use_lnL=rvs_integrand_is_lnL), dtype=float) + # POSTERIOR weights, not importance weights: under --fairdraw-extrinsic-output these + # rows are already a w-proportional draw, and fitting the GMM with w on top of that + # gives a proposal shaped like w^2 -- over-concentrated. It is then handed to the NEXT + # iteration via --extrinsic-proposal-breadcrumb, so the truncation compounds across + # iterations rather than staying inside one run. + _lw = np.asarray(ln_weights_for_posterior(_rvs, sampler, convert=identity_convert, + use_lnL=rvs_integrand_is_lnL), dtype=float) # extrinsic samples + bounds for the standard groups that this run actually sampled. _ext_params = [p for grp in _ehmod.STANDARD_GROUPS for p in grp] _ext_samples = {p: np.array(_rvs[p], dtype=float).reshape(-1) for p in _ext_params if p in _rvs} @@ -3832,7 +4156,9 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t fname_output_dgrid = opts.output_file +"_"+str(indx_event)+"_" + ".dgrid" dL = np.array(sampler._rvs["distance"]) rvs = sampler._rvs - ln_wts = ln_weights_from_rvs(rvs, use_lnL=rvs_integrand_is_lnL) + # POSTERIOR weights: a fair-drawn record is already an equal-weight posterior draw, + # and build_distance_grid weights the samples again to form the per-bin mass. + ln_wts = ln_weights_for_posterior(rvs, sampler, use_lnL=rvs_integrand_is_lnL) # Distance prior at each sample. Use the sampler's stored prior_pdf # callable; this matches whatever ILE actually integrated against # (volumetric, pseudo_cosmo, redshift, ...). @@ -3914,12 +4240,29 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t # .dgrid. rvs = sampler._rvs _rvs = lambda k: np.asarray(identity_convert(rvs[k]), float) # cupy-safe column read - ln_w_full = ln_weights_from_rvs(rvs, convert=identity_convert, use_lnL=rvs_integrand_is_lnL) + ln_w_full = ln_weights_for_posterior(rvs, sampler, convert=identity_convert, + use_lnL=rvs_integrand_is_lnL) # Split K into core (reweight) and wing (fresh) slices. --distance-slice-all-fresh # forces zero reweight core: EVERY slice is a fresh fixed-d integration. Use it # when the main-loop n_eff is small -- the reweight core is then starved (the same # MC noise as the .dgrid fair-draw histogram), while each fresh slice is honest. all_fresh = bool(getattr(opts, "distance_slice_all_fresh", False)) + # THE REWEIGHT CORE CANNOT RUN ON A FAIR-DRAWN RECORD. It reuses the Omega samples + # as an importance sample at each slice distance, applying pi_Omega/q_Omega on top of + # rows that were already resampled proportional to the full weight -- so the + # prior/proposal ratio is counted twice, and N = len(rvs['distance']) is the resample + # size rather than the number of draws. Unlike .dgrid this is not a single spurious + # factor that can be divided out: the correct estimator would need the pre-draw + # record, which by then is gone. + # + # The fresh path is exact and already supported, so use it rather than reporting a + # plausible wrong number -- every slice becomes an independent fixed-d integration. + # It costs more likelihood evaluations; say so, rather than changing cost silently. + if not all_fresh and _rvs_is_export_resample(sampler): + print(" [dslice] _rvs is the fair-draw export; forcing --distance-slice-all-fresh" + " (the reweight core would double-count pi_Omega/q_Omega on resampled rows)." + " K fresh fixed-d integrations instead of a reweighted core.") + all_fresh = True if all_fresh: n_core, n_wing = 0, K else: @@ -4303,18 +4646,74 @@ def analyze_event(P_list, indx_event, data_dict, psd_dict, fmax, opts,inv_spec_t _SEQ_WS_PENDING = None if getattr(opts, 'sampler_sequential_warmstart', False) and hasattr(sampler, 'bootstrap_from_samples'): try: - _lnkey = 'log_integrand' if 'log_integrand' in sampler._rvs else ('integrand' if 'integrand' in sampler._rvs else None) - if _lnkey is not None and all(p in sampler._rvs for p in sampler.params_ordered): - _lnv = np.asarray(sampler.identity_convert(sampler._rvs[_lnkey]), dtype=float).ravel() - if _lnv.size >= 2 and np.any(np.isfinite(_lnv)): - _cols = np.vstack([np.asarray(sampler.identity_convert(sampler._rvs[p]), dtype=float).ravel() - for p in sampler.params_ordered]).T - _keep = _lnv > (np.nanmax(_lnv) - opts.sampler_sequential_warmstart_deltalnL) - _SEQ_WS_PENDING = _cols[_keep] if np.sum(_keep) >= 2 else (_cols if _cols.shape[0] >= 2 else None) + # SEED FROM THE RETAINED POINTS, AND JUDGE THE SEED BY RANK -- the same two rules + # the L0 auto-rescue was given (PR #78), for the same reasons, because this is the + # same construction one code path away. Both were wrong here: + # + # 1. IT READ THE FAIR DRAW. sampler._rvs has been rebound to + # min(n_extr, 1.5*eff_samp, 1.5*neff) rows resampled WITH REPLACEMENT for + # EXPORT. --fairdraw-extrinsic-output is not an exotic setting: every + # extrinsic stage built by create_event_parameter_pipeline_BasicIteration, + # cepp_basic_htcondor and create_event_nr_pipeline_with_cip passes it + # unconditionally. So on the collapsed high-amplitude pass this feature is + # most wanted for, the seed for the NEXT intrinsic point was a handful of + # rows -- one, in the rho_net 146.8 logs -- several of them the same point + # twice, while the live set held a thousand. + # + # 2. THE GUARD WAS A COUNT (`_lnv.size >= 2`, `np.sum(_keep) >= 2`), which is + # exactly the rule build_warm_seed exists to replace: n points span at most + # n-1 affine dimensions, so two rows drawn with replacement can be rank 0 in + # 6 and still pass. The next point then warm-starts inside a degenerate + # sliver and reports a healthy n_eff over truncated support -- the quiet + # failure, not the loud one. + # + # Fall back to _rvs only for a sampler that keeps no reserve, so the feature + # degrades to its previous behaviour rather than to no seed at all. + _res_ws = _warm_seed_reserve_for(sampler) + if _res_ws is not None: + _cols = np.asarray(_res_ws['X'], dtype=float) + _lnv = np.asarray(_res_ws['lnL'], dtype=float).ravel() + _src_ws = 'retained' + else: + _lnkey = 'log_integrand' if 'log_integrand' in sampler._rvs else ('integrand' if 'integrand' in sampler._rvs else None) + _lnv = (np.asarray(sampler.identity_convert(sampler._rvs[_lnkey]), dtype=float).ravel() + if (_lnkey is not None and all(p in sampler._rvs for p in sampler.params_ordered)) + else np.array([])) + _cols = (np.vstack([np.asarray(sampler.identity_convert(sampler._rvs[p]), dtype=float).ravel() + for p in sampler.params_ordered]).T + if _lnv.size else np.zeros((0, len(sampler.params_ordered)))) + _src_ws = 'fairdraw' + if _lnv.size >= 1 and np.any(np.isfinite(_lnv)): + # build_warm_seed applies the deltalnL window, the affine-rank test and the + # puff-to-full-rank. Reuse the L0 rescue's puff knobs rather than adding a + # parallel set: it is the same geometry problem (a rank-deficient cloud about + # a known peak) and two independently-tuned widths would be one more pair to + # keep in step. + _ax_ws, _lo_ws, _hi_ws = _warm_seed_geometry(sampler) + _seed_ws, _info_ws = mcsamplerAdaptiveVolume.build_warm_seed( + _cols, _lnv, _lo_ws, _hi_ws, _ax_ws, + deltalnL=opts.sampler_sequential_warmstart_deltalnL, + puff_scale=opts.sampler_l0_rescue_puff_scale, + puff_width_frac=opts.sampler_l0_rescue_puff_width_frac, + puff_factor=opts.sampler_l0_rescue_puff_factor) + _SEQ_WS_PENDING = _seed_ws if len(_seed_ws) >= 2 else None + if _info_ws['puffed']: + print(" [seq warm-start] seed from {} source: {} point(s) had affine rank" + " {}/{}: PUFFED to rank {}/{} with {} points".format( + _src_ws, _info_ws['n_core'], _info_ws['rank_core'], _info_ws['dim'], + _info_ws['rank_final'], _info_ws['dim'], _info_ws['n_puff'])) + if _info_ws['rank_final'] < _info_ws['dim']: + print(" [seq warm-start] *** the puffed seed is STILL rank-deficient" + " ({}/{}); the next point may be reported as collapsed.".format( + _info_ws['rank_final'], _info_ws['dim'])) except Exception as _e_cap: print(" [seq warm-start] could not capture seed ({})".format(_e_cap)) sampler._rvs = {} + # _rvs_is_pooled is set by THIS function (the replica block), not by the sampler, so the + # sampler's own per-pass reset cannot clear it. Drop it with the record it describes, or + # the next event inherits "pooled" and its exports lose the equal-weight treatment. + sampler._rvs_is_pooled = False return res diff --git a/MonteCarloMarginalizeCode/Code/bin/util_ConstructEOSPosterior.py b/MonteCarloMarginalizeCode/Code/bin/util_ConstructEOSPosterior.py index af9c1e0fa..11cb7b645 100755 --- a/MonteCarloMarginalizeCode/Code/bin/util_ConstructEOSPosterior.py +++ b/MonteCarloMarginalizeCode/Code/bin/util_ConstructEOSPosterior.py @@ -347,6 +347,8 @@ def uniform_prior(x): supplemental_ln_likelihood= None supplemental_ln_likelihood_prep =None supplemental_ln_likelihood_parsed_ini=None +supplemental_ln_likelihood_offset_fn=None +supplemental_ln_likelihood_offset=0.0 # Supplemental likelihood factor. Must have identical call sequence to 'likelihood_function'. Called with identical raw inputs (including cosines/etc) if opts.supplementary_likelihood_factor_code and opts.supplementary_likelihood_factor_function: print(" EXTERNAL SUPPLEMENTARY LIKELIHOOD FACTOR : {}.{} ".format(opts.supplementary_likelihood_factor_code,opts.supplementary_likelihood_factor_function)) @@ -354,18 +356,37 @@ def uniform_prior(x): external_likelihood_module = sys.modules[opts.supplementary_likelihood_factor_code] supplemental_ln_likelihood = getattr(external_likelihood_module,opts.supplementary_likelihood_factor_function) name_prep = "prepare_"+opts.supplementary_likelihood_factor_function + # Optional _offset hook, same naming convention as prepare_. A plugin whose + # contribution is large must return a CENTRED lnL -- the default path here exponentiates it + # (likelihood_function*np.exp(supplemental)), and float64 overflows past ~709, which a single + # loud-event quadratic (lnL_peak ~ SNR^2/2) exceeds on its own. That centring is a constant + # multiplicative factor: harmless in the posterior, but it would otherwise leak into the ABSOLUTE + # evidence written below and make it wrong by exactly that constant. The plugin reports what it + # removed; we add it back at the write site. Queried after integration, not here: the value is + # only known once the plugin has been prepared/loaded. + name_offset = opts.supplementary_likelihood_factor_function+"_offset" + if hasattr(external_likelihood_module,name_offset): + supplemental_ln_likelihood_offset_fn=getattr(external_likelihood_module,name_offset) if hasattr(external_likelihood_module,name_prep): - supplemental_ln_likelhood_prep=getattr(external_likelihood_module,name_prep) + supplemental_ln_likelihood_prep=getattr(external_likelihood_module,name_prep) # Check for and load in ini file associated with external library if opts.supplementary_likelihood_factor_ini: import configparser as ConfigParser config = ConfigParser.ConfigParser() - config.optionxform=str # force preserve case! + config.optionxform=str # force preserve case! config.read(opts.supplementary_likelihood_factor_ini) - supplemental_ln_likelhood_parsed_ini=config - - # Call the ini file, tell it what coordinates we are using by name - supplemental_ln_likelihood_prep(config=supplemental_ln_likelihood_parsed_ini,coords=coord_names) + supplemental_ln_likelihood_parsed_ini=config + + # Prepare the plugin, telling it what coordinates we are using by name. Called whether or not + # an ini was supplied (config=None then): a plugin configured entirely by environment still + # needs the basis, and without it the arrays it is handed are anonymous -- same count, same + # order, no error, wrong coordinates. + # coords MUST be low_level_coord_names, not coord_names: the sampler integrates over + # low_level_coord_names and so calls supplemental_ln_likelihood(*x) with one array per + # SAMPLING coordinate, in that order. With --parameter-implied/--parameter-nofit the two + # lists differ, and handing over the fit basis would make the plugin label those arrays + # wrongly -- evaluating at the wrong coordinates without any error. + supplemental_ln_likelihood_prep(config=supplemental_ln_likelihood_parsed_ini,coords=low_level_coord_names) supplemental_coordinate_convert = None supplemental_coordinate_inverse = None @@ -785,6 +806,37 @@ def convert_coords(x_in, _low=low_level_coord_names, _coord=coord_names): sampler = mcsamplerPortfolio.MCSampler(portfolio=sampler_list) +# Does this sampler hand back ln(integral), or the integral itself? +# +# --internal-use-lnL alone does NOT answer that. 'use_lnL'/'return_lnI' ride in as +# **kwargs to integrate(), and several samplers ignore one or both of them: +# mcsamplerAdaptiveVolume, mcsamplerNFlow, mcsamplerPortfolio +# -- integrate() ALWAYS delegates to integrate_log(); the return is +# ln(integral) whether or not the flags were passed +# mcsamplerGPU -- delegates to integrate_log() only when use_lnL is set +# mcsamplerEnsemble -- the only sampler that reads return_lnI +# mcsampler -- the legacy 'adaptive_cartesian' backend: reads NEITHER flag, +# and always returns the linear integral +# Key off the class actually constructed above rather than off --sampler-method: the +# portfolio branch rebinds sampler, and an unrecognized --sampler-method silently falls +# through to the default mcsampler. +_sampler_module = type(sampler).__module__.split('.')[-1] +if _sampler_module in ['mcsamplerAdaptiveVolume', 'mcsamplerNFlow', 'mcsamplerPortfolio']: + sampler_returns_ln_integral = True +elif _sampler_module in ['mcsamplerGPU', 'mcsamplerEnsemble']: + sampler_returns_ln_integral = bool(opts.internal_use_lnL) # exactly when use_lnL/return_lnI are set below +else: # mcsampler, the legacy adaptive_cartesian backend + sampler_returns_ln_integral = False + if opts.internal_use_lnL: + # This backend ignores use_lnL, so it would integrate lnL as though it were L: + # the returned value is not an evidence in either convention, and neither + # res nor log(res) can repair it. Refuse rather than write a meaningless + # number to --fname-output-integral. Compare ok_lnL_methods/bad_lnL_methods + # in util_ConstructIntrinsicPosterior_GenericCoordinates.py. + print(" OPTION MISMATCH : --internal-use-lnL needs a sampler that honors it; --sampler-method {} builds {}, which ignores it. Use GMM, AV, adaptive_cartesian_gpu, or portfolio.".format(opts.sampler_method, _sampler_module)) + sys.exit(99) + + ## ## Loop over param names ## @@ -950,9 +1002,27 @@ def parse_corr_params(my_str): res, var, neff, dict_return = sampler.integrate(fn_passed, *low_level_coord_names, verbose=True,nmax=int(opts.n_max),n=n_step,neff=opts.n_eff, save_intg=True,tempering_adapt=True, floor_level=1e-3,igrand_threshold_p=1e-3,convergence_tests=test_converged,adapt_weight_exponent=my_exp,no_protect_names=True,**extra_args) # MC integrates in the SAMPLING basis (low_level_coord_names); convert_coords routes each sample into the fit basis (coord_names) before evaluating the GP/RF +# result value: be careful, if the sampler returns lnI, then must not take log twice! +# See sampler_returns_ln_integral where the sampler is constructed: this is a property of +# the backend, NOT of --internal-use-lnL. +ln_integrand_value = None +if sampler_returns_ln_integral: + ln_integrand_value = res +else: + ln_integrand_value = np.log(res) # Save result -- needed for odds ratios, etc. -np.savetxt(opts.fname_output_integral, [np.log(res)]) +# Absolute scale: a supplementary-likelihood plugin may subtract a constant from its own lnL to keep +# the exponentiated integrand in float64 range (see the _offset note at the import above). +# That constant divides out of the posterior but not out of an evidence -- which is exactly what +# this file is read as -- so it is restored here. Queried now rather than next to the prepare call +# because a plugin configured entirely by environment prepares itself lazily on its first +# evaluation. Stays 0.0 for plugins that do not centre and for runs with no supplementary factor. +if supplemental_ln_likelihood_offset_fn: + supplemental_ln_likelihood_offset = float(supplemental_ln_likelihood_offset_fn()) + print(" EXTERNAL SUPPLEMENTARY LIKELIHOOD FACTOR : restoring offset {} in reported evidence ".format(supplemental_ln_likelihood_offset)) +ln_integrand_value_absolute = ln_integrand_value + supplemental_ln_likelihood_offset +np.savetxt(opts.fname_output_integral, [ln_integrand_value_absolute]) if neff < len(coord_names): print(" PLOTS WILL FAIL ") @@ -1092,4 +1162,3 @@ def parse_corr_params(my_str): print(" Saving to ", opts.fname_output_samples+".dat") np.savetxt(opts.fname_output_samples+".dat",dat_out,header=" lnL sigma_lnL " + ' '.join(dat_orig_names)) - diff --git a/MonteCarloMarginalizeCode/Code/bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py b/MonteCarloMarginalizeCode/Code/bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py index c2caf26a4..82f4f3f56 100755 --- a/MonteCarloMarginalizeCode/Code/bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py +++ b/MonteCarloMarginalizeCode/Code/bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py @@ -290,6 +290,7 @@ def extract_combination_from_LI(samples_LI, p): parser.add_argument("--cap-points",default=-1,type=int,help="Maximum number of points in the sample, if positive. Useful to cap the number of points ued for GP. See also lnLoffset. Note points are selected AT RANDOM") parser.add_argument("--chi-max", default=1,type=float,help="Maximum range of 'a' allowed. Use when comparing to models that aren't calibrated to go to the Kerr limit.") parser.add_argument("--chi-small-max", default=None,type=float,help="Maximum range of 'a' allowed on the smaller body. If not specified, defaults to chi_max") +parser.add_argument("--eccentricity-prior", default="uniform",choices=['uniform','log_uniform'],help="Options are 'uniform' and 'log_uniform'") # constrained: only 'log_uniform' is branched on below, so anything else would quietly fall through to the uniform prior parser.add_argument("--ecc-max", default=0.9,type=float,help="Maximum range of 'eccentricity' allowed.") parser.add_argument("--ecc-min", default=0.0,type=float,help="Minimum range of 'eccentricity' allowed.") parser.add_argument("--meanPerAno-max", default=2*np.pi,type=float,help="Maximum range of 'meanPerAno' allowed.") @@ -738,6 +739,8 @@ def extract_combination_from_LI(samples_LI, p): supplemental_ln_likelihood= None supplemental_ln_likelihood_prep =None supplemental_ln_likelihood_parsed_ini=None +supplemental_ln_likelihood_offset_fn=None +supplemental_ln_likelihood_offset=0.0 # Supplemental likelihood factor. Must have identical call sequence to 'likelihood_function'. Called with identical raw inputs (including cosines/etc) if opts.supplementary_likelihood_factor_code and opts.supplementary_likelihood_factor_function: print(" EXTERNAL SUPPLEMENTARY LIKELIHOOD FACTOR : {}.{} ".format(opts.supplementary_likelihood_factor_code,opts.supplementary_likelihood_factor_function)) @@ -745,6 +748,17 @@ def extract_combination_from_LI(samples_LI, p): external_likelihood_module = sys.modules[opts.supplementary_likelihood_factor_code] supplemental_ln_likelihood = getattr(external_likelihood_module,opts.supplementary_likelihood_factor_function) name_prep = "prepare_"+opts.supplementary_likelihood_factor_function + # Optional _offset hook, same naming convention as prepare_. A plugin whose + # contribution is large must return a CENTRED lnL -- the default path here exponentiates it + # (likelihood_function*np.exp(supplemental)), and float64 overflows past ~709, which a single + # loud-event quadratic (lnL_peak ~ SNR^2/2) exceeds on its own. That centring is a constant + # multiplicative factor: harmless in the posterior, but it would otherwise leak into every + # ABSOLUTE lnL and evidence written below and make them wrong by exactly that constant. The + # plugin reports what it removed; we add it back at the write sites. Queried after integration, + # not here: the value is only known once the plugin has been prepared/loaded. + name_offset = opts.supplementary_likelihood_factor_function+"_offset" + if hasattr(external_likelihood_module,name_offset): + supplemental_ln_likelihood_offset_fn=getattr(external_likelihood_module,name_offset) if opts.using_eos_for_prior: # Load in filename fname = opts.using_eos.replace('file:', '') @@ -762,17 +776,18 @@ def extract_combination_from_LI(samples_LI, p): if hasattr(external_likelihood_module,name_prep): - supplemental_ln_likelhood_prep=getattr(external_likelihood_module,name_prep) + supplemental_ln_likelihood_prep=getattr(external_likelihood_module,name_prep) # Check for and load in ini file associated with external library if opts.supplementary_likelihood_factor_ini: import configparser as ConfigParser config = ConfigParser.ConfigParser() - config.optionxform=str # force preserve case! + config.optionxform=str # force preserve case! config.read(opts.supplementary_likelihood_factor_ini) - supplemental_ln_likelhood_parsed_ini=config + supplemental_ln_likelihood_parsed_ini=config - # Call the ini file, tell it what coordinates we are using by name - supplemental_ln_likelihood_prep(config=supplemental_ln_likelihood_parsed_ini,coords=coord_names) + # NOTE the plugin is NOT prepared here: low_level_coord_names is not final at this point (the + # tabular-EOS path appends 'ordering' to it further down). See the preparation call just + # before the sampling loop below. @@ -907,9 +922,30 @@ def tapered_magnitude_prior_alt(x,loc=0.8,kappa=20.): # def eccentricity_prior(x): return np.ones(x.shape) / (ECC_MAX-ECC_MIN) # uniform over the interval [0.0, ECC_MAX] +def log_eccentricity_prior(x): + # Density in e for a distribution uniform in ln(e) on [ECC_MIN, ECC_MAX]. + # Was np.ln (does not exist in numpy, so --eccentricity-prior log_uniform raised + # AttributeError as soon as the prior was evaluated) with a (ECC_MAX-ECC_MIN) + # normalization, which is the uniform prior's normalization, not this one's: + # \int_ECC_MIN^ECC_MAX dx/(x*C) = 1 => C = ln(ECC_MAX/ECC_MIN). + # The statement below is byte-identical to rift_O4c 0.0.17.13, which reached the same + # fix independently; keep it that way so future O4c->O4d merges do not conflict here. + return np.ones(x.shape) / (x*np.log(ECC_MAX/ECC_MIN)) # log uniform over the interval [ECC_MIN, ECC_MAX]; if ECC_MIN=0.0, auto corrects to ECC_MIN=0.001 + +def uniform_eccentricity_ln_prior(x): + return np.ones(x.shape) / ((np.log(ECC_MAX/ECC_MIN))) # log uniform over the interval [ECC_MIN, ECC_MAX]; if ECC_MIN=0.0, auto corrects to ECC_MIN=0.001 + def eccentricity_squared_prior(x): # note this is INCONSISTENT with the prior above -- we are designed to give a CDF = (e/emax)^2 for example here, or more generally (e^2 - emin^2)/(emax^2-emin^2) return np.ones(x.shape) / (ECC_MAX**2-ECC_MIN**2) # uniform over the interval [ECC_MIN, ECC_MAX] +def log_eccentricity_squared_prior(x): + # The log_eccentricity_prior distribution expressed in the eccentricity_squared + # coordinate, for runs that sample e^2 (--parameter eccentricity_squared) but asked + # for --eccentricity-prior log_uniform. With u = e^2, + # p(u) = p(e) |de/du| = 1/(2 e^2 ln(ECC_MAX/ECC_MIN)) = 1/(u ln(ECC_MAX^2/ECC_MIN^2)), + # i.e. log-uniform in u as well, as a log-uniform variable must be under a power map. + return np.ones(x.shape) / (2*x*np.log(ECC_MAX/ECC_MIN)) # log uniform, as a density in e^2 on [ECC_MIN^2, ECC_MAX^2] + def meanPerAno_prior(x): return np.ones(x.shape) / (MEANPERANO_MAX-MEANPERANO_MIN) # uniform over the interval [MEANPERANO_MIN, MEANPERANO_MAX] @@ -962,6 +998,7 @@ def normalized_zbar_prior(z): 's2z_bar':normalized_zbar_prior, # Other priors 'eccentricity':eccentricity_prior, + 'eccentricity_ln':uniform_eccentricity_ln_prior, 'eccentricity_squared':eccentricity_squared_prior, 'meanPerAno':meanPerAno_prior, 'chi_pavg':precession_prior, @@ -985,6 +1022,7 @@ def normalized_zbar_prior(z): 'lambda_plus':[lambda_min,lambda_plus_max], 'lambda_minus':[-lambda_max,lambda_max], # will include the true region always...lots of overcoverage for small lambda, but adaptation will save us. 'eccentricity':[ECC_MIN, ECC_MAX], + 'eccentricity_ln':[np.log(ECC_MIN), np.log(ECC_MAX)], 'eccentricity_squared':[ECC_MIN**2, ECC_MAX**2], 'meanPerAno':[MEANPERANO_MIN, MEANPERANO_MAX], 'chi_pavg':[0.0,2.0], @@ -1159,7 +1197,48 @@ def normalized_zbar_prior(z): prior_map['lambda1'] = functools.partial(mcsampler.power_down_samp,xmin=0,xmax=lambda_max,alpha=opts.prior_lambda_power+1) prior_map['lambda2'] = functools.partial(mcsampler.power_down_samp,xmin=0,xmax=lambda_small_max,alpha=opts.prior_lambda_power+1) - +if opts.eccentricity_prior == 'log_uniform': + # Apply the requested prior to EVERY eccentricity coordinate CIP can sample, not just + # 'eccentricity': --parameter eccentricity_squared is a supported choice (and what + # pseudo_pipe uses for iteration 0 of an eccentric run), so setting only + # prior_map['eccentricity'] left such a run silently sampling the uniform-in-e^2 + # density while reporting a log-uniform prior. + # 'eccentricity_ln' needs no entry here: uniform_eccentricity_ln_prior is already + # uniform in ln(e), i.e. this same distribution written in that coordinate. + prior_map['eccentricity'] = log_eccentricity_prior + prior_map['eccentricity_squared'] = log_eccentricity_squared_prior + +# A zero lower edge is unusable for anything logarithmic in e, and that is a property of +# the COORDINATE as much as of the prior. Sampling in 'eccentricity_ln' is logarithmic +# under the default uniform prior too: with --ecc-min at its default of 0.0 that +# coordinate's range is [log(0), log(ECC_MAX)] = [-inf, ...], and +# uniform_eccentricity_ln_prior evaluates log(ECC_MAX/ECC_MIN), i.e. divides by zero. +# So the correction -- and the domain check that follows it -- is keyed on either +# trigger, not on --eccentricity-prior alone. Runs that neither ask for a log-uniform +# prior nor sample a log coordinate keep the ecc range exactly as given. +if opts.eccentricity_prior == 'log_uniform' or 'eccentricity_ln' in low_level_coord_names: + if ECC_MIN == 0.0: + print("Warning: You passed 0.0 as ecc-min with a logarithmic eccentricity prior or coordinate. Changing ecc-min to 0.001") + ECC_MIN = 0.001 + # Zero is only the most common invalid floor, not the only one. Everything + # logarithmic in e needs a strictly positive, correctly ordered interval: np.log of a + # non-positive edge is nan (or -inf), and log(ECC_MAX/ECC_MIN) is nan for either edge + # negative and zero for ECC_MAX == ECC_MIN. None of that raises -- it propagates + # into the sampler as nan/inf coordinate bounds and nan prior densities, i.e. a run + # that reports a prior it never had. The zero floor corrected just above is the one + # documented exception; every other invalid range is a configuration error, so say so + # and exit nonzero rather than hand the sampler numbers it cannot use. + if not (0 < ECC_MIN < ECC_MAX): + print("Incompatible options: a logarithmic eccentricity prior or coordinate requires 0 < ecc-min < ecc-max, but received ecc-min = {} and ecc-max = {}".format(ECC_MIN,ECC_MAX)) + sys.exit(1) + prior_range_map['eccentricity'] = [ECC_MIN,ECC_MAX] + prior_range_map['eccentricity_ln'] = [np.log(ECC_MIN),np.log(ECC_MAX)] + # 1/u is no more integrable down to u=0 than 1/e is down to e=0, so the squared + # coordinate's range has to follow the corrected floor too + prior_range_map['eccentricity_squared'] = [ECC_MIN**2,ECC_MAX**2] + print("Eccentricity range: ",prior_range_map['eccentricity']) + print("ln(eccentricity) range: ",prior_range_map['eccentricity_ln']) + print("eccentricity^2 range: ",prior_range_map['eccentricity_squared']) # tex_dictionary = { # "mtot": '$M$', # "mc": '${\cal M}_c$', @@ -2597,6 +2676,25 @@ def convert_coords(x_in): sampler = mcsamplerPortfolio.known_pipelines[opts.sampler_method]() +### +### Supplemental likelihood: prepare the plugin, now that the sampling basis is FINAL +### +# Deliberately here and not next to the import above. low_level_coord_names is still being built +# at that point -- the tabular-EOS branch appends 'ordering' to it -- and the plugin RECORDS the +# basis it is handed, so a list snapshotted earlier is short by that coordinate while the sampler +# still calls supplemental_ln_likelihood(*x) with one array per FINAL sampling coordinate. Nothing +# below this line changes the list, so this is the first point at which it can be declared. +# Called whether or not an ini was supplied (config=None then): a plugin configured entirely by +# environment still needs the basis, and without it the arrays it is handed are anonymous -- same +# count, same order, no error, wrong coordinates. +# coords MUST be low_level_coord_names, not coord_names: the sampler integrates over +# low_level_coord_names and so calls supplemental_ln_likelihood(*x) with one array per SAMPLING +# coordinate, in that order. With --parameter-implied/--parameter-nofit the two lists differ, and +# handing over the fit basis would make the plugin label those arrays wrongly -- evaluating at the +# wrong coordinates without any error. +if supplemental_ln_likelihood_prep: + supplemental_ln_likelihood_prep(config=supplemental_ln_likelihood_parsed_ini,coords=low_level_coord_names) + ## ## Loop over param names ## @@ -3001,6 +3099,18 @@ def parse_corr_params(my_str): else: ln_integrand_value = np.log(res) +# Absolute scale of everything reported below. A supplementary-likelihood plugin may subtract a +# constant from its own lnL to keep the exponentiated integrand in float64 range (see the +# _offset note at the import above); that constant divides out of the posterior but not +# out of an evidence or an absolute lnL. Queried here rather than next to the prepare call because +# a plugin configured entirely by environment prepares itself lazily on its first evaluation, which +# has certainly happened by now. Stays 0.0 for every plugin that does not centre, and for runs +# with no supplementary factor at all -- so nothing else changes. +if supplemental_ln_likelihood_offset_fn: + supplemental_ln_likelihood_offset = float(supplemental_ln_likelihood_offset_fn()) + print(" EXTERNAL SUPPLEMENTARY LIKELIHOOD FACTOR : restoring offset {} in reported lnL/evidence ".format(supplemental_ln_likelihood_offset)) +ln_integrand_value_absolute = ln_integrand_value + supplemental_ln_likelihood_offset + # Test n_eff threshold if not (opts.fail_unless_n_eff is None): if neff < opts.fail_unless_n_eff and not(opts.not_worker): # if we need the output to continue: @@ -3064,7 +3174,7 @@ def parse_corr_params(my_str): # Save result -- needed for odds ratios, etc. # Warning: integral_result.dat uses *original* prior, before any reweighting -np.savetxt(opts.fname_output_integral+".dat", [ln_integrand_value+lnL_shift]) +np.savetxt(opts.fname_output_integral+".dat", [ln_integrand_value_absolute+lnL_shift]) @@ -3090,12 +3200,12 @@ def parse_corr_params(my_str): annotation_header = linefirst # this will/must be lnL sigma_lnL and then parameter names, which we want to preserve with open(opts.fname_output_integral+"+annotation.dat", 'w') as file_out: if not(opts.using_eos) or not(opts.using_eos.startswith('file:')): - str_out =list( map(str,[ln_integrand_value, np.sqrt(var)/res, neff])) + str_out =list( map(str,[ln_integrand_value_absolute, np.sqrt(var)/res, neff])) file_out.write("# " + annotation_header + "\n") file_out.write(' '.join( str_out + eos_extra + ["\n"])) else: file_out.write("# " + annotation_header + "\n") - file_out.write(" {} {} ".format(ln_integrand_value, np.sqrt(var)/res) + ' '.join(map(str,params_here))) + file_out.write(" {} {} ".format(ln_integrand_value_absolute, np.sqrt(var)/res) + ' '.join(map(str,params_here))) #np.savetxt(opts.fname_output_integral+"+annotation.dat", np.array([[np.log(res), np.sqrt(var)/res, neff]]), header=eos_extra) # since not EOS, can just use np.savetxt # with open(opts.fname_output_integral+"+annotation_ESS.dat", 'w') as file_out: @@ -3148,7 +3258,7 @@ def parse_corr_params(my_str): weights_scaled = weights_scaled/np.max(weights_scaled) # try to reduce dynamic range n_ESS = np.sum(weights_scaled)**2/np.sum(weights_scaled**2) print(" n_eff n_ESS ", neff, n_ESS) -np.savetxt(opts.fname_output_integral+"+annotation_ESS.dat",[[ln_integrand_value, np.sqrt(var)/res, neff, n_ESS]],header=" lnL sigmaL neff n_ESS ") +np.savetxt(opts.fname_output_integral+"+annotation_ESS.dat",[[ln_integrand_value_absolute, np.sqrt(var)/res, neff, n_ESS]],header=" lnL sigmaL neff n_ESS ") # Throw away stupid points that don't impact the posterior @@ -3261,7 +3371,10 @@ def parse_corr_params(my_str): # Integral result v2: using modified prior. # Note also downselects NOT applied: no range cuts, unless applied as part of aligned_prior, etc. # - use for Bayes factors with GREAT CARE for this reason; should correct for with indx_ok -log_res_reweighted = lnLmax + np.log(np.mean(weights)) +# Same absolute-scale restoration as for the integral above: lnLmax here is a maximum of the +# CENTRED integrand, and this file is documented to agree with integral_result.dat -- so leaving the +# plugin's constant out of one and not the other turns a check into a spurious disagreement. +log_res_reweighted = lnLmax + np.log(np.mean(weights)) + supplemental_ln_likelihood_offset sigma_reweighted= np.std(weights,dtype=RiftFloat)/np.mean(weights) neff_reweighted = np.sum(weights)/np.max(weights) np.savetxt(opts.fname_output_integral+"_withpriorchange.dat", [log_res_reweighted]) # should agree with the usual result, if no prior changes @@ -3583,6 +3696,11 @@ def parse_corr_params(my_str): n_delivered_unique = len(np.unique(kept_indx_list[:n_output_size])) print(" export supply: requested {} delivered {} distinct {} unique-draw bound {} ".format(opts.n_output_samples, n_output_size, n_delivered_unique, n_unique_bound)) np.savetxt(opts.fname_output_samples+"+annotation_export.dat", [[opts.n_output_samples, n_output_size, n_delivered_unique, n_unique_bound]], header=" n_requested n_delivered n_distinct unique_draw_bound ") +# Absolute lnL for every exported product. Same correction as for the evidence above, and needed +# for the same reason: the exported lnL column, the _lnL.dat sidecar and best_point_by_lnL_value.dat +# are all read as absolute lnL by the next stage, so a plugin's internal centring must not reach +# them. Adds 0.0 unless a supplementary-likelihood plugin reported an offset. +lnL_list = np.array(lnL_list,dtype=internal_dtype) + supplemental_ln_likelihood_offset # Hyperpipeline ASCII grid writer (opt-in via env var). See note above the # earlier identical writer site -- same rationale. if _hpio.is_active(): @@ -3598,7 +3716,6 @@ def parse_corr_params(my_str): lnL_values=lnL_list[:n_output_size]) else: lalsimutils.ChooseWaveformParams_array_to_xml(P_list[:n_output_size],fname=opts.fname_output_samples,fref=P.fref) -lnL_list = np.array(lnL_list,dtype=internal_dtype) np.savetxt(opts.fname_output_samples+"_lnL.dat", lnL_list) diff --git a/MonteCarloMarginalizeCode/Code/bin/util_RIFT_pseudo_pipe.py b/MonteCarloMarginalizeCode/Code/bin/util_RIFT_pseudo_pipe.py index 2dcf54291..1ff73c88c 100755 --- a/MonteCarloMarginalizeCode/Code/bin/util_RIFT_pseudo_pipe.py +++ b/MonteCarloMarginalizeCode/Code/bin/util_RIFT_pseudo_pipe.py @@ -428,6 +428,8 @@ def run_lisa_known_sky_surface(opts): parser.add_argument("--internal-tabular-eos-file",type=str,default=None,help="Tabular file of EOS to use. The default prior will be UNIFORM in this table!") parser.add_argument("--sample-eccentricity-squared",action='store_true', help="Option for sampling as well as fitting in eccentricity_squared instead of fitting in eccentricity_squared and sampling in eccentricity (also need option --use-eccentricity-squared") parser.add_argument("--use-eccentricity-squared",action='store_true', help="Allows for fitting and sampling in eccentricity_squared instead of eccentricity") +parser.add_argument("--sample-eccentricity-ln",action='store_true', help="Option for sampling as well as fitting in eccentricity_ln instead of fitting in eccentricity_ln and sampling in eccentricity (also need option --use-eccentricity-ln") +parser.add_argument("--use-eccentricity-ln",action='store_true', help="Allows for fitting and sampling in eccentricity_ln instead of eccentricity") parser.add_argument("--assume-eccentric",action='store_true', help="Add eccentric options for each part of analysis") parser.add_argument("--use-meanPerAno",action='store_true', help="Add meanPerAno options for each part of analysis") parser.add_argument("--internal-cip-use-periodic-ecc-vars",action='store_true', help="use e cos ell, e sin ell as fitting variables ") @@ -493,6 +495,7 @@ def run_lisa_known_sky_surface(opts): parser.add_argument("--ile-gpu-fanout",default=None,help="Multi-GPU ILE fan-out: split each ILE batch's intrinsic-grid range across N GPUs on the node (one shard per GPU). Integer N (also requests N GPUs+CPUs) or 'auto' (split across whatever GPUs are visible at runtime). Baked into the generated ile_pre.sh, so it needs no runtime environment. Equivalent to setting RIFT_ILE_GPU_FANOUT. Requires --ile-force-gpu.") parser.add_argument("--fake-data-cache",type=str) parser.add_argument("--spin-magnitude-prior",default='default',type=str,help="options are default [uniform mag for precessing, zprior for aligned], volumetric, uniform_mag_prec, uniform_mag_aligned, zprior_aligned") +parser.add_argument("--eccentricity-prior",default='uniform',type=str,choices=['uniform','log_uniform'],help="options are uniform in e ('uniform') and uniform in log(e) ('log_uniform')") # constrained: the value is forwarded verbatim to CIP, which only branches on the exact string 'log_uniform', so an unrecognized value here would silently run the uniform prior instead of failing parser.add_argument("--force-lambda-max",default=None,type=float,help="Provide this value to override the value of lambda-max provided") parser.add_argument("--force-lambda-small-max",default=None,type=float,help="Provide this value to override the value of lambda-small-max provided") parser.add_argument("--force-lambda-no-linear-init",action='store_true',help="Disables use of priors focused towards small lambda for initial iterations. Designed for PP plot tests with wide/uniform priors.") @@ -602,6 +605,13 @@ def run_lisa_known_sky_surface(opts): parser.add_argument("--internal-force-puff-iterations", default=4, type=int, help="Number of iterations to be puffed") opts= parser.parse_args() +# Multi-GPU ILE fan-out: --ile-gpu-fanout funnels through RIFT_ILE_GPU_FANOUT, which +# create_event_parameter_pipeline_BasicIteration (run via os.system, inheriting this +# environment) and dag_utils read at DAG-build time to size request_GPUs/CPUs and bake +# the value into ile_pre.sh. A CLI value wins over any inherited environment value. +if opts.ile_gpu_fanout is not None: + os.environ['RIFT_ILE_GPU_FANOUT'] = str(opts.ile_gpu_fanout) + config_stored=None; config_dict=None ile_condor_commands = None if (opts.use_ini): @@ -1610,6 +1620,7 @@ def approx_supports_precession(approx_name): if opts.internal_cip_tripwire: line += " --tripwire-fraction {} ".format(opts.internal_cip_tripwire) line += prior_args_lookup[opts.spin_magnitude_prior] + if opts.cip_internal_use_eta_in_sampler: line = line.replace('parameter delta_mc','parameter eta') if opts.cip_fit_method == 'quadratic' or opts.cip_fit_method == 'polynomial': @@ -1691,11 +1702,14 @@ def approx_supports_precession(approx_name): if opts.fit_save_gp: line += " --fit-save-gp my_gp " # fiducial filename, stored in each iteration + line += " --eccentricity-prior {}".format(opts.eccentricity_prior) if opts.assume_eccentric: if opts.use_meanPerAno: line += " --parameter meanPerAno --use-meanPerAno " if opts.use_eccentricity_squared: line += " --use-eccentricity --parameter eccentricity_squared " + elif opts.use_eccentricity_ln: + line += " --use-eccentricity --parameter eccentricity_ln " else: line += " --use-eccentricity --parameter eccentricity " # if opts.use_eccentricity_squared: @@ -1988,6 +2002,8 @@ def approx_supports_precession(approx_name): cmd += " --use-eccentricity " if opts.sample_eccentricity_squared: cmd += " --use-eccentricity-squared-sampling " + if opts.sample_eccentricity_ln: + cmd += " --use-eccentricity-ln-sampling " if opts.use_meanPerAno: cmd += " --use-meanPerAno " if opts.calibration_reweighting and (not opts.bilby_pickle_file): diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_MEASUREMENT.md b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_MEASUREMENT.md new file mode 100644 index 000000000..1069b0896 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_MEASUREMENT.md @@ -0,0 +1,97 @@ +# Measuring `--sampler-l0-rescue-reject-dlnZ` after #79 + +**Result: the default is raised 0.5 -> 3.0, and the gate is not a working truncation +detector.** Across 160 known-lnZ passes on two samplers it caught **0 of 55** genuinely +truncated warm passes at *every* threshold from 0.25 to 4.0 nats, while at 0.5 it binned +**25%** of good portfolio warm passes. 0.5 was strictly dominated: no detection, real cost. + +Reproduce: + +``` +python3 measure_l0_reject_dlnZ.py --sampler AV --reps 20 --out-json av.json +python3 measure_l0_reject_dlnZ.py --sampler portfolio --reps 20 --out-json pf.json +``` + +Run logs for the numbers below: `L0_REJECT_DLNZ_run_AV_2026-08-13.log`, +`L0_REJECT_DLNZ_run_portfolio_2026-08-13.log`. + +## Why this had to wait for #79 + +The 0.5 default was chosen when both sides of the comparison were read from the fair-drawn +`_rvs`, which carries a `log(n_retained/eff_samp)` offset that does not cancel between passes +at different `n_eff` (+3.48 nats measured, `verify_skew.py`). Tuning against that would have +been calibrating to the bug. With #79 in (re-landed as #86) both sides come from the retained +reserve, so the number means what its help text says. + +## Method + +Known-lnZ targets (sums of Gaussians on the unit cube, uniform priors, rho=100, 6-D), driving +the real rescue sequence: cold pass -> reserve -> `build_warm_seed` -> `bootstrap_from_samples` +-> warm pass, with both lnZ readings taken through the ILE's own `_lnZ_of_reserve_or_rvs`. + +**Rejection rates are conditioned on what the warm pass ACTUALLY did**, not on the intended +condition -- read out of its own retained set. This matters: on a portfolio the defensive GMM +component means a seeded warm pass usually still reaches every mode, so most "bimodal" +replicates are not truncated at all and a rejection there is a *false* positive. Scoring +against the label rather than the outcome would have reported those as successes. + +## The cold reference is unusable on AV, and fine on a portfolio + +| | cold lnZ − true | warm lnZ − true | +|---|---|---| +| **AV**, unimodal | **−76.2** (sd 37.9) | −0.75 (sd 0.43) | +| **AV**, bimodal f=0.50 | **−91.6** (sd 29.5) | −0.87 (sd 0.24) | +| **portfolio**, unimodal | −0.23 (sd 16.3) | −0.07 (sd 0.51) | +| **portfolio**, bimodal f=0.50 | −1.02 (sd 0.95) | −0.31 (sd 0.37) | + +The gate's premise is *"the cold pass had full support, so a lower warm evidence means the seed +missed mass."* On **AV that premise is false**: the collapsed cold pass reports 70–92 nats below +truth. `cold − warm` is then about −70 to −91 and no positive threshold is ever crossed. This is +not a helper bug -- `lnZ_from_reserve` reproduces `integrate_log`'s own reported lnZ *to the +digit* in all conditions. It is what ESS 1.00 with k-hat ~51 means. + +On a **portfolio the premise holds** (cold within ~1 nat), because the GMM member carries a +defensive component -- which is also why its warm pass is rarely truncated in the first place: +it reached every mode in **77 of 80** replicates. The configuration where the gate *could* work +is the configuration that barely needs it. + +## Rejection rates, by what actually happened + +``` + AV (28 good, 52 truncated) portfolio (77 good, 3 truncated) + dlnZ FPR TPR FPR TPR + 0.25 0% 0% 34% 0% + 0.50 0% 0% 25% 0% + 1.00 0% 0% 12% 0% + 2.00 0% 0% 5% 0% + 3.00 0% 0% 0% 0% +``` + +`TPR` is zero everywhere, on both samplers, across all 55 truncated passes. The AV column is +zero in both directions -- the gate is simply inert there. The portfolio column is *all cost*: +at the old 0.5 default, one good warm pass in four was discarded in favour of a collapsed cold +result. + +## What changed, and what did not + +**Changed:** the default, 0.5 -> 3.0. That removes the measured false-positive cost (~0% at 3.0) +without giving up any detection, because there is none to give up. It keeps a safety net for a +genuinely large (>3 nat) discrepancy, which no measurement here rules out. + +**Not changed:** the gate itself. Replacing it properly means dropping the evidence comparison +for a direct test -- **support containment**: the cold reserve already holds the pass's finite +draws, so ask whether the warm live volume contains them. That needs no second trustworthy lnZ, +which is exactly the resource the collapsed regime cannot supply. Recommended as the follow-up; +deliberately not attempted here, since it is a new detector and wants its own validation. + +There is also an irreducible blind spot independent of all this: a threshold `T` can never catch +a missed mode carrying less than `1 - exp(-T)` of the mass -- 95% at T=3.0. That alone means the +evidence comparison cannot be the primary truncation detector at any usable threshold. + +## Scope + +Synthetic Gaussian / two-Gaussian targets; rho=100; uniform priors on the unit cube; one bimodal +geometry (modes at 0.25 and 0.75 in one coordinate); 20 replicates per condition per sampler +(160 passes total). Not measured: real GW likelihoods, other separations, other rho, GPU +backends. The portfolio truncated-pass count is only 3, so its `TPR` column is weak evidence on +its own -- the strong statement comes from AV's 52. diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_run_AV_2026-08-13.log b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_run_AV_2026-08-13.log new file mode 100644 index 000000000..fb2ff42fb --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_run_AV_2026-08-13.log @@ -0,0 +1,77 @@ +============================================================================== +Measuring --sampler-l0-rescue-reject-dlnZ on the POST-#79 gate +sampler = AV, rho = 100.0, 20 replicates per condition +============================================================================== +### NULL (unimodal -- a good warm pass) (20 usable of 20 replicates) + true lnZ 4973.7237 mass fractions [1.] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 1.00 warm n_eff median 12.23 + cold_lnZ - TRUE: median -76.195 sd 37.856 <- the gate's reference + warm_lnZ - TRUE: median -0.748 sd 0.431 + cold lnZ REPORTED BY integrate_log - TRUE: median -76.195 sd 37.856 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.748 sd 0.431 + gap = cold_lnZ - warm_lnZ: median -75.393 mean -79.820 sd 37.565 [p05 -131.585, p95 -19.345] +### SIGNAL (bimodal, seeded mode holds f=0.50; true deficit -log f = +0.693) (20 usable of 20 replicates) + true lnZ 4974.4169 mass fractions [0.5 0.5] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 1.00 warm n_eff median 12.12 + warm pass reached only ONE mode in 16/20 replicates + cold_lnZ - TRUE: median -91.636 sd 29.522 <- the gate's reference + warm_lnZ - TRUE: median -0.865 sd 0.242 + cold lnZ REPORTED BY integrate_log - TRUE: median -91.636 sd 29.522 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.865 sd 0.242 + gap = cold_lnZ - warm_lnZ: median -90.890 mean -79.847 sd 29.497 [p05 -111.838, p95 -35.963] +### SIGNAL (bimodal, seeded mode holds f=0.75; true deficit -log f = +0.288) (20 usable of 20 replicates) + true lnZ 4974.0114 mass fractions [0.75 0.25] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 1.00 warm n_eff median 14.84 + warm pass reached only ONE mode in 18/20 replicates + cold_lnZ - TRUE: median -74.869 sd 27.048 <- the gate's reference + warm_lnZ - TRUE: median -0.790 sd 0.523 + cold lnZ REPORTED BY integrate_log - TRUE: median -74.869 sd 27.048 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.790 sd 0.523 + gap = cold_lnZ - warm_lnZ: median -73.703 mean -69.325 sd 27.067 [p05 -114.286, p95 -25.926] +### SIGNAL (bimodal, seeded mode holds f=0.90; true deficit -log f = +0.105) (20 usable of 20 replicates) + true lnZ 4973.8291 mass fractions [0.9 0.1] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 1.00 warm n_eff median 13.46 + warm pass reached only ONE mode in 18/20 replicates + cold_lnZ - TRUE: median -69.682 sd 33.712 <- the gate's reference + warm_lnZ - TRUE: median -0.322 sd 1.099 + cold lnZ REPORTED BY integrate_log - TRUE: median -69.682 sd 33.712 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.322 sd 1.099 + gap = cold_lnZ - warm_lnZ: median -69.560 mean -68.529 sd 33.037 [p05 -110.970, p95 -22.656] +per-replicate records -> /tmp/claude-40428/knob_AV.json +============================================================================== +REJECTION RATE BY WHAT THE WARM PASS ACTUALLY DID + good = warm pass reached every mode (28 replicates) + trunc = warm pass reached fewer (52 replicates) +============================================================================== + dlnZ FPR (good pass binned) TPR (truncation caught) + 0.25 0% 0% + 0.50 0% 0% + 0.75 0% 0% + 1.00 0% 0% + 1.50 0% 0% + 2.00 0% 0% + 2.50 0% 0% + 3.00 0% 0% + 4.00 0% 0% +============================================================================== +THRESHOLD TABLE + FPR = a GOOD warm pass rejected (loud failure: collapsed cold result reported) + TPR = a TRUNCATED warm pass rejected (the catch we want) +============================================================================== + dlnZ FPR TPR f=0.50 TPR f=0.75 TPR f=0.90 + 0.25 0% 0% 0% 0% + 0.50 0% 0% 0% 0% + 0.75 0% 0% 0% 0% + 1.00 0% 0% 0% 0% + 1.50 0% 0% 0% 0% + 2.00 0% 0% 0% 0% + 3.00 0% 0% 0% 0% +NULL spread sets the floor: sd 37.565 nats, p95 -19.345. +STRUCTURAL BLIND SPOT: a threshold T can never catch a missed mode carrying less + T=0.5 -> blind to any mode holding < 39% of the total mass + T=1.0 -> blind to any mode holding < 63% of the total mass + T=2.0 -> blind to any mode holding < 86% of the total mass diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_run_portfolio_2026-08-13.log b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_run_portfolio_2026-08-13.log new file mode 100644 index 000000000..1de413a8f --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/L0_REJECT_DLNZ_run_portfolio_2026-08-13.log @@ -0,0 +1,77 @@ +============================================================================== +Measuring --sampler-l0-rescue-reject-dlnZ on the POST-#79 gate +sampler = portfolio, rho = 100.0, 20 replicates per condition +============================================================================== +### NULL (unimodal -- a good warm pass) (20 usable of 20 replicates) + true lnZ 4973.7237 mass fractions [1.] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 2.18 warm n_eff median 80.34 + cold_lnZ - TRUE: median -0.233 sd 16.263 <- the gate's reference + warm_lnZ - TRUE: median -0.066 sd 0.507 + cold lnZ REPORTED BY integrate_log - TRUE: median -0.233 sd 16.263 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.066 sd 0.507 + gap = cold_lnZ - warm_lnZ: median -0.229 mean -6.414 sd 15.775 [p05 -28.492, p95 +0.529] +### SIGNAL (bimodal, seeded mode holds f=0.50; true deficit -log f = +0.693) (20 usable of 20 replicates) + true lnZ 4974.4169 mass fractions [0.5 0.5] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 2.42 warm n_eff median 11.58 + warm pass reached only ONE mode in 2/20 replicates + cold_lnZ - TRUE: median -1.023 sd 0.947 <- the gate's reference + warm_lnZ - TRUE: median -0.310 sd 0.371 + cold lnZ REPORTED BY integrate_log - TRUE: median -1.023 sd 0.947 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.310 sd 0.371 + gap = cold_lnZ - warm_lnZ: median -0.478 mean -0.329 sd 1.038 [p05 -1.589, p95 +1.571] +### SIGNAL (bimodal, seeded mode holds f=0.75; true deficit -log f = +0.288) (20 usable of 20 replicates) + true lnZ 4974.0114 mass fractions [0.75 0.25] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 1.82 warm n_eff median 16.57 + warm pass reached only ONE mode in 0/20 replicates + cold_lnZ - TRUE: median +0.042 sd 1.123 <- the gate's reference + warm_lnZ - TRUE: median -0.246 sd 0.339 + cold lnZ REPORTED BY integrate_log - TRUE: median +0.042 sd 1.123 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.246 sd 0.339 + gap = cold_lnZ - warm_lnZ: median +0.268 mean +0.535 sd 1.148 [p05 -0.906, p95 +2.515] +### SIGNAL (bimodal, seeded mode holds f=0.90; true deficit -log f = +0.105) (20 usable of 20 replicates) + true lnZ 4973.8291 mass fractions [0.9 0.1] + lnZ sources (cold,warm): [('retained', 'retained')] + cold n_eff median 1.67 warm n_eff median 10.74 + warm pass reached only ONE mode in 1/20 replicates + cold_lnZ - TRUE: median -1.043 sd 1.168 <- the gate's reference + warm_lnZ - TRUE: median -0.094 sd 0.561 + cold lnZ REPORTED BY integrate_log - TRUE: median -1.043 sd 1.168 + warm lnZ REPORTED BY integrate_log - TRUE: median -0.094 sd 0.561 + gap = cold_lnZ - warm_lnZ: median -0.905 mean -0.641 sd 1.060 [p05 -1.882, p95 +0.860] +per-replicate records -> /tmp/claude-40428/knob_PF.json +============================================================================== +REJECTION RATE BY WHAT THE WARM PASS ACTUALLY DID + good = warm pass reached every mode (77 replicates) + trunc = warm pass reached fewer (3 replicates) +============================================================================== + dlnZ FPR (good pass binned) TPR (truncation caught) + 0.25 34% 0% + 0.50 25% 0% + 0.75 18% 0% + 1.00 12% 0% + 1.50 8% 0% + 2.00 5% 0% + 2.50 4% 0% + 3.00 0% 0% + 4.00 0% 0% +============================================================================== +THRESHOLD TABLE + FPR = a GOOD warm pass rejected (loud failure: collapsed cold result reported) + TPR = a TRUNCATED warm pass rejected (the catch we want) +============================================================================== + dlnZ FPR TPR f=0.50 TPR f=0.75 TPR f=0.90 + 0.25 30% 20% 55% 25% + 0.50 10% 20% 45% 20% + 0.75 0% 15% 40% 15% + 1.00 0% 10% 30% 5% + 1.50 0% 10% 15% 5% + 2.00 0% 5% 15% 0% + 3.00 0% 0% 0% 0% +NULL spread sets the floor: sd 15.775 nats, p95 +0.529. +STRUCTURAL BLIND SPOT: a threshold T can never catch a missed mode carrying less + T=0.5 -> blind to any mode holding < 39% of the total mass + T=1.0 -> blind to any mode holding < 63% of the total mass + T=2.0 -> blind to any mode holding < 86% of the total mass diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/RVS_FAIRDRAW_AUDIT.md b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/RVS_FAIRDRAW_AUDIT.md new file mode 100644 index 000000000..659eaad17 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/RVS_FAIRDRAW_AUDIT.md @@ -0,0 +1,287 @@ +# Audit: every `_rvs` consumer, against the fair-draw rebind + +Mandate: *"Every other consumer of `_rvs` rows or lengths in the ILE is unaudited. Everything +found here was in code this change happened to touch."* This is that sweep, done +mechanically so it can be re-run rather than repeated by hand. + +Different axis from `RVS_CACHE_AUDIT.md` in this directory. That one asks *"cached column, +or canonical components?"*; this one asks *"before or after the rebind?"*. A site can be +wrong on either axis independently, and the two sweeps agree on nothing except the file list. + +## How to re-run it + +``` +python3 audit_rvs_fairdraw.py --summary # counts per file and phase +python3 audit_rvs_fairdraw.py --post-rebind # only the sites that see the resample +python3 audit_rvs_fairdraw.py --check # CI gate: every such site has a verdict +python3 make_rvs_fairdraw_ledger.py # regenerate the verdicts from the rules +``` + +`--check` runs in CI. It does **not** assert that post-rebind reads are bugs -- most are +legitimately per-row. It asserts that each one carries a recorded human verdict, keyed by a +whitespace-normalized hash of the read itself. Moving or reindenting a line keeps its +verdict; changing what it reads does not, and a new consumer fails the build. Function-level +keying was rejected deliberately: `analyze_event` alone holds 40 sites, and every one of the +five known defects was added *next to* correct code. + +## The census + +306 reads of `_rvs` across the seven integrators, the three ILE scripts, the two CIP scripts +and `RIFT/misc/distance_slices.py`. 131 of them see the resample. (Counts move as the code +is edited -- `--summary` is the authority, not this table.) + +| | sites | +|---|---| +| `BEFORE` the rebind, inside the rebinding function | 160 | +| `AFTER`, inside it | 26 | +| `POST_INTEGRATE` (bin/ scripts, after `integrate` returned) | 104 | +| post-rebind reads carrying a recorded verdict | 131 / 131 | + +Those 131 reads collapse to 123 distinct ledger entries (identical reads share a key): +`PER_ROW` 54, `BENIGN` 61, `FIXED` 3, `NO_FAIRDRAW` 4, **`BROKEN` 1** -- down from 8 when the +sweep started. The one remaining is #79's cross-source fallback, described under Finding 0. + +**Inside the integrators, nothing post-rebind is broken.** All 26 lexically-`AFTER` sites are +the rebind's own right-hand side or the element-wise cupy→numpy conversion loop that follows +it. That is a real result and it narrows the search: the hazard lives entirely in the +consumers, not in the samplers. + +`--fairdraw-extrinsic-output` is not exotic. `create_event_parameter_pipeline_BasicIteration`, +`cepp_basic_htcondor` and `create_event_nr_pipeline_with_cip` all append it to the extrinsic +stage unconditionally, so every one of these paths runs on the resample in production. + +## Measured, not asserted + +`verify_skew.py` drives the ILE's own `_lnZ_of_rvs` / `_kish_neff_of_rvs`: + +| claim | result | +|---|---| +| `_lnZ_of_rvs(..., already_pooled=False)` on a fair-drawn record is high by `log(n/n_eff)` | excess `+4.509` vs predicted `+4.503` at n_eff 44; `+7.33` vs `+7.47` at n_eff 2.3 | +| the error does not cancel between two passes at different n_eff | two passes with **identical true lnZ**, n_eff 1.8 vs 53.3: the gate reads `+3.48` nats | +| ...and therefore rejects a good warm pass | **100%** of the time at the 0.5 default; 94% at 2.0 | +| re-weighting an already fair-drawn record shifts a posterior mean | **13%** of the true value on a weight-correlated coordinate | + +## Finding 0 -- PR #79 was not in the development line (RESOLVED by #86) + +> **Resolved 2026-08-13.** Re-landed as PR #86 (merge `5a2965ba`), cherry-picking `23be21ab` +> and `9fc806f8` onto `0ae3f48f` -- both clean, and the resulting diff byte-identical to the +> original, so #79's approval carried over rather than being re-reviewed. A sweep of all 74 +> merged PRs found **no other orphans**: the only other non-ancestors are four `master`-based +> PRs touching solely `.github/workflows/private-review-dispatch.yml`, which correctly do not +> belong in `rift_O4d`. +> +> Two lessons worth keeping, since ancestry alone answers neither: +> * `git merge-base --is-ancestor` gives FALSE ALARMS when content re-lands by cherry-pick +> (#79 still fails it; its patch-ids are present). Confirm with `git cherry` / `patch-id`. +> * `git cherry` gives false alarms of its own when a change lands FOLDED into another commit +> -- `7c986d63` reports `+` while its content is in `rift_O4d`, verified by diffing the +> function. Neither tool is sufficient alone; the file is the authority. +> +> One residual survives #79 and is recorded as `BROKEN` in the ledger: its cross-source +> fallback (`_cold_src != _warm_src`) re-reads both sides from the fair-draw record, which is +> self-consistent but not unbiased, since the two passes sit at different `n_eff`. Bounded to +> the mismatch case and documented at the site. Closing it needs a reserve for the samplers +> that keep none. + +The original finding, kept because it explains how this happens and how to detect it. Past +tense throughout: this described the tree before #86. + +The reject-gate fix was **not an ancestor of `junior/rift_O4d`**, so the gate in the tree we +developed from was the pre-#79 code, and the measurement above described production. + +``` +7ca5c8df warm-seed reserve: record the exact pre-cap weight total (PR #84 branch tip) +68987b26 Merge PR #79 into that branch <- has the fix +bc210833 Merge PR #84 ... parents are ad426d9f and 7ca5c8df <- took the branch BEFORE 68987b26 +``` + +`junior/claude/l0-rescue-exact-total` pointed at `68987b26`, but `bc210833` merged `7ca5c8df`. +So #79 was merged into the #84 branch *after* #84 had already gone to `rift_O4d`, and never +followed. Verified three ways: `merge-base --is-ancestor` fails for `23be21ab`, `7efdd229`, +`9fc806f8` and `68987b26`; `git show 0ae3f48f:...batchmode | grep _lnZ_of_reserve_or_rvs` +returns nothing; and `lnZ_from_reserve` is absent from `mcsamplerAdaptiveVolume.py`. + +Note `rift_O4d_wt_ralph`, a **pinned measurement tree**, sat at `7efdd229` -- a copy of the #79 +work -- so measurements had been running against code the development line did not have. + +The `--sampler-l0-rescue-reject-dlnZ` re-tune was blocked on this, since re-tuning against the +`log(n/n_eff)` artifact would have calibrated to the bug. With #86 in it was measurable, and +was measured: see Finding 4. + +## Finding 1 -- the sequential warm-start seed (FIXED here) + +`--sampler-sequential-warmstart` captures a seed for the next intrinsic point. It read +`sampler._rvs` and guarded with `_lnv.size >= 2` / `np.sum(_keep) >= 2`. That is PR #78's +defect exactly, one code path away: the resample, and a **count** where a **rank** is needed. + +Both failure directions are real and neither is loud. On the measured pass the fair draw +keeps **one** row (`Fairdraw size : 1`), so the count declines and the feature is *silently +inert* -- the user asked for a warm start and got none. At the rho_net 102.8 regime it keeps +~5 and the count *accepts* a rank-2-of-6 seed, so the next point warm-starts into a sliver and +reports a healthy n_eff over truncated support. + +Fixed by routing through `_warm_seed_reserve_for` (new shared lookup) and `build_warm_seed` +(rank test + puff), mirroring the L0 rescue. The L0 rescue's inline reserve lookup was +replaced by the shared one, so the pair cannot drift. 13 tests in +`test/test_seq_warmstart_seed.py`, added to CI; `test_l0_rescue_seed.py` + +`test_av_empty_live_volume.py` still pass (100 passed, 2 skipped). + +## Finding 2 -- three double-weighting sites (FIXED) + +All three took rows that are already a `w`-proportional draw and weighted them by `w` again. +`_pool_replica_rvs` has guarded against exactly this since the replica work, via +`already_resampled`; none of these had an equivalent. + +The predicate matters, and it is not the CLI flag: the samplers skip the draw when it would not +shrink the record, and those rows still carry real importance weights. So each sampler now marks +the rebind ITSELF (`self._rvs_is_fairdraw`, set at all seven rebind sites, reset per pass because +samplers are reused across events), and the ILE reads it through `_rvs_is_export_resample`. + +1. **`--extrinsic-proposal-output` breadcrumb** -- fitted the GMM proposal with `w` on top of a + `w`-proportional draw, so the proposal came out shaped like `w^2` and was then handed to the + NEXT iteration via `--extrinsic-proposal-breadcrumb`. The worst of the three, because the + truncation compounded across iterations. Now takes `ln_weights_for_posterior`. +2. **`.dgrid`** -- same shape, one product. Now takes `ln_weights_for_posterior`. +3. **`.dslice` reweight core** -- a *different* shape: it double-counts `pi_Omega/q_Omega` and + takes `N` from the resample, and cannot be corrected after the fact because the pre-draw + record is gone. Routed instead to the exact all-fresh path (K independent fixed-d + integrations), loudly, since that changes cost. + +`ln_weights_for_posterior` is the new single answer to "how should these rows be weighted to +represent the posterior", as distinct from `ln_weights_from_rvs`, which answers "what is the +importance weight of this record" and was always right about that. + +## Finding 3 -- pooled n_eff (FIXED) + +`_kish_neff_of_rvs` on the pooled record, which overwrites the reported `neff`. When the export +is fair-drawn `_pool_replica_rvs` deliberately flattens each block, and the Kish n_eff of +piecewise-constant weights is just the row count -- `K*min(n_max, 1.5*eff_samp, 1.5*neff)`, the +size of the EXPORT. At the default `--fairdraw-extrinsic-output-n-max 5` that reports `n_eff = 5K`. + +Now computed one level up where the quantities are still meaningful -- Kish over the BLOCKS, + + neff_pooled = (sum_k Z_k)^2 / sum_k (Z_k^2 / neff_k) + +which is exactly the property the original comment asked for: it reduces to `sum_k neff_k` when +the replicas agree and falls below it when they disagree. The pooled Kish is still used when the +export was NOT fair-drawn, where per-row weights are real and finer-grained. + +## Finding 5 -- two composition defects, from review (FIXED) + +Both are the same shape, and it is the shape worth remembering: **a fix that is correct in +isolation and wrong once another code path runs after it.** The unit tests for Findings 1-3 +all passed with both bugs present, because each tested its helper rather than the composition. + +**The fair-draw marker survived pooling.** `_pool_replica_rvs` builds a record that is +equal-weight *within* each block but weighted *between* blocks by exactly the replica +evidences `Z_k/K`. Leaving the marker set made `ln_weights_for_posterior` return zeros, so +`.dgrid` and the proposal breadcrumb would have mixed replicas by exported **row count** +instead of by evidence -- silently discarding the disagreement the replicas exist to measure. +The marker is now cleared after pooling, and only when pooling actually happened: every +fallback in `_pool_replica_rvs` returns one of its *input* records, which is still the fair +draw it arrived as, so the test is object identity rather than length. + +**A rejected warm rescue left its reserve behind.** The reject path restored `_rvs`, the +estimate and `dict_return`, but not `_warm_seed_reserve` -- which still described the rejected +warm pass. Latent until Finding 1 made the sequential warm start read the reserve, at which +point the next intrinsic point would have been seeded from the very truncated cloud the +evidence gate had just thrown away. Note the direction: **Finding 1's fix is what activated +this**; before it, the capture read `_rvs`, which the reject path does restore. Snapshot and +restore now travel through `_snapshot_pass_state` / `_restore_pass_state`, which carry the +reserve, the fair-draw marker and the per-member reserves (`_warm_seed_reserve_for` falls +through to `portfolio_realizations`, so restoring only the aggregate would leave that fallback +pointing at the warm pass). Both restore sites -- the reject path and the exception handler -- +go through the one helper. + +A note on the tests, because it is a real limitation: `analyze_event` needs data, PSDs and a +waveform, so the call sites cannot be exercised from a unit test. The behavioural tests pin +the contracts the call sites depend on; the wiring is pinned at source level. Verified by +reverting each fix -- only the wiring tests fail. If `analyze_event` ever becomes callable in +pieces, promote them. + +## Finding 6 -- one flag was answering two questions (review round 2; FIXED) + +The Finding-5 fix cleared `_rvs_is_fairdraw` after pooling. That fixed the weighting question +and broke two others, because the flag was carrying two distinct properties: + +| property | scope | true for a pooled record? | +|---|---|---| +| rows were drawn proportional to `w` | per **block** | **yes** | +| the record is globally equal-weight | whole **record** | **no** | + +Clearing it made the `.dslice` all-fresh safeguard stop firing on a pooled record whose blocks +*are* resampled, and made the new block-Kish `n_eff` branch **unreachable** -- it sat below the +line that cleared the flag it tested, and is only ever reached when pooling happened. + +Now two predicates over two markers: `_rvs_is_export_resample` (rows resampled; survives +pooling) and `_rvs_is_equal_weight` (`fairdraw and not pooled`). The block-Kish branch keys on +neither -- it uses a local computed where pooling happens, since "did pooling flatten a block" +is a fact about that step, not a property of the record. + +**And `already_resampled` was the CLI option, not what each pass did.** The draw is skipped per +pass when it would not shrink that pass's record, so one global boolean either flattens a +replica whose importance weights are genuine, or leaves a resampled replica double-weighted -- +and a run near the `n_extr` boundary produces a *mixture*, which a boolean cannot describe at +all. `_pool_replica_rvs` now accepts a per-replica sequence and decides per block; the ILE +captures each replica's marker beside the record it describes. While there, the empty-record +filter was made lockstep: it used to drop from `rep_rvs` alone, shifting every later block +against its own `lnZ`. + +The lesson is the same one as Finding 5, one level up: **a boolean that is true for two +different reasons will eventually be read for the wrong one.** + +## Finding 7 -- the pooled marker outlived a FAILED event (review round 3; FIXED) + +`_rvs_is_pooled` was cleared on the normal return of `analyze_event`. But `_reject_if_collapsed` +**raises** after pooling, the caller's `except Exception` swallows it and moves to the next +event, and the marker survived. The next ordinary fair draw was then read as pooled, +`_rvs_is_equal_weight` went False, and `.dgrid` and the extrinsic-proposal breadcrumb applied +importance weights to rows that already carried them -- the `w^2` defect of Finding 2, +resurrected on the event after any failure. + +Reset on ENTRY instead. Entry is reached on every call by construction, needs no restructuring +of a 2000-line function, and is correct even for a caller that never returns normally. The +end-of-function clear stays too, so a sampler handed elsewhere afterwards is not carrying a +stale marker. + +**Fourth round, fourth defect in the provenance bookkeeping rather than in the physics.** The +recurring shapes, for whoever touches this next: + +1. correct in isolation, wrong in composition (Finding 5); +2. one flag answering two questions (Finding 6); +3. the CLI option is not "what the pass actually did" (Finding 6); +4. cleared only on the happy path (this one). + +All four are the same underlying problem: a boolean describing mutable shared state is a second +source of truth that every site touching the first must maintain. That is the argument for the +naming change in the Recommendation below, and the reason it is a separate draft rather than a +rider on this PR. + +## Finding 4 -- the reject knob (MEASURED; default raised) + +See `L0_REJECT_DLNZ_MEASUREMENT.md`. Across 160 known-lnZ passes the gate caught **0 of 55** +truncated warm passes at every threshold, while at the old 0.5 default it binned **25%** of good +portfolio warm passes. Default raised to 3.0 -- strictly better, since there was no detection to +trade away. The gate is documented as not being a truncation detector; support containment is +the recommended replacement and is deliberately left as follow-up. + +## Recommendation: make the rebind unable to do this again + +Five defects -- now six -- of one shape in one attribute is an API problem. Ranked by +benefit per unit of blast radius: + +1. **Keep the retained set under its own name (recommended).** `_warm_seed_reserve` already + is this, for two callers. Generalize it: have `integrate_log` always leave + `self._retained` (or keep `_rvs` and expose `self._export`), and migrate consumers one at + a time. Cheap, incremental, and each migration is independently testable. It does not + *prevent* the mistake, but it gives every future author a correct thing to reach for. +2. **Have the fair draw return a new object instead of mutating in place.** The correct fix + in principle and the one that makes the error unrepresentable. Blast radius is large: + `_rvs` is read at 307 sites, and every `bin/` consumer would need to be told which object + it wants. Worth doing behind the naming change above, not instead of it. +3. **Keep `--check` in CI regardless.** It is the only one of the three that catches the + *next* consumer rather than fixing the current ones, and it is already green. + +A cheaper partial: set a flag on the record (`_rvs['__fairdrawn__'] = True`) and have +`ln_weights_from_rvs` warn when a caller weights a flagged record. That would have caught all +three Finding-2 sites at runtime, and it is a dozen lines. diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/audit_rvs_fairdraw.py b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/audit_rvs_fairdraw.py new file mode 100644 index 000000000..9ec8bb942 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/audit_rvs_fairdraw.py @@ -0,0 +1,426 @@ +#!/usr/bin/env python3 +"""Enumerate every read of ``_rvs`` (and of the columns derived from it) and classify +each one against the fair-draw rebind at the end of ``integrate_log``/``integrate``. + +WHY THIS EXISTS +--------------- +``sampler._rvs`` is not the sample set. At the end of the integrator the fair-draw +export rebinds *every* ``_rvs`` key to a subset drawn WITH REPLACEMENT, proportional to +weight, of size ``min(n_extr, 1.5*eff_samp, 1.5*neff)``. Any consumer that reads +``_rvs`` rows, lengths, or statistics after that point is reading an EXPORT RESAMPLE and +is skewed by roughly ``log(n_retained/eff_samp)`` -- catastrophically so on a collapsed +pass, where the resample can be a single row. + +That shape has produced five separate defects (CIP posterior export; L0 rescue seed, +PR #78; rescue reject gate, PR #79; warm-seed reserve cap and its logarithm, PR #84). +This script exists so the sixth is found mechanically rather than by whoever happens to +be editing the surrounding code. + +This audits a DIFFERENT axis from ``RVS_CACHE_AUDIT.md`` in this directory. That one +asks "cached column, or canonical components?"; this one asks "before or after the +rebind?". A site can be wrong on either axis independently. + +PHASES +------ +``BEFORE`` / ``AFTER`` read lexically inside a function that performs the rebind. +``CALLED_BEFORE/AFTER`` read in a helper, classified by where that helper is called + from inside the rebinding function (intra-file call graph). +``POST_INTEGRATE`` read on a sampler outside any rebinding function -- i.e. in + bin/ scripts and utilities that run after ``integrate`` has + returned. Every one of these sees the resample. +``NO_REBIND`` read in a file/scope with no rebind at all (helpers, tests). + +USAGE +----- + python3 audit_rvs_fairdraw.py # human-readable report + python3 audit_rvs_fairdraw.py --json # machine-readable + python3 audit_rvs_fairdraw.py --summary # counts per file/phase + python3 audit_rvs_fairdraw.py --check # exit 1 on an unclassified site + +``--check`` is the CI form. It does NOT assert that every post-rebind site is a bug -- +many are legitimately per-row. It asserts that every post-rebind site appears in +``VERDICTS`` below with a recorded human judgement. A new or moved consumer therefore +fails the build and has to be classified by a person, which is the property we want. + +Needs Python >= 3.8 for end_lineno; degrades gracefully on 3.6/3.7. +""" +import argparse +import ast +import hashlib +import json +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +CODE_ROOT = os.path.abspath(os.path.join(HERE, "..", "..", "..")) + +TARGETS = [ + "RIFT/integrators/mcsampler.py", + "RIFT/integrators/mcsamplerAdaptiveVolume.py", + "RIFT/integrators/mcsamplerEnsemble.py", + "RIFT/integrators/mcsamplerGPU.py", + "RIFT/integrators/mcsamplerNFlow.py", + "RIFT/integrators/mcsamplerPortfolio.py", + "RIFT/integrators/mcsampler_generic.py", + "RIFT/misc/distance_slices.py", + "bin/integrate_likelihood_extrinsic_batchmode", + "bin/integrate_likelihood_extrinsic_batchmode_lisa", + "bin/integrate_likelihood_extrinsic", + "bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py", + "bin/util_ConstructEOSPosterior.py", +] + +# Columns whose per-row VALUES survive the rebind but whose LENGTH and population +# statistics do not. +DERIVED_KEYS = { + "log_integrand", "log_joint_prior", "log_joint_s_prior", "log_weights", + "weights", "joint_prior", "joint_s_prior", "integrand", "sample_n", +} + +# Calls that turn rows into a population statistic. Used to raise the severity of a +# report, never to decide it. +POPULATION_CALLS = { + "sum", "mean", "cov", "std", "var", "median", "average", "argmax", "argmin", + "max", "min", "len", "logsumexp", "percentile", "quantile", "corrcoef", + "histogram", "cumsum", "count_nonzero", "vstack", "column_stack", "shape", +} + +POST_REBIND_PHASES = ("AFTER", "CALLED_AFTER", "POST_INTEGRATE") + + +# --------------------------------------------------------------------------- AST compat +def _const_str(node): + """String value of a subscript slice, across 3.6-3.12 AST shapes.""" + if hasattr(ast, "Index") and isinstance(node, getattr(ast, "Index")): + node = node.value + if isinstance(node, ast.Constant) and isinstance(node.value, str): + return node.value + if hasattr(ast, "Str") and isinstance(node, getattr(ast, "Str")): + return node.s + return None + + +def _end_lineno(node, fallback): + end = getattr(node, "end_lineno", None) + if end is not None: + return end + return max([n.lineno for n in ast.walk(node) if hasattr(n, "lineno")] or [fallback]) + + +# --------------------------------------------------------------------------- analysis +def _rebind_lines(tree): + """{function name: line of the fair-draw rebind}. The rebind is the assignment to + ``self._rvs[key]`` inside the branch guarded by ``bFairdraw``.""" + out = {} + for node in ast.walk(tree): + if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + continue + for sub in ast.walk(node): + if not isinstance(sub, ast.If): + continue + cond = {n.id for n in ast.walk(sub.test) if isinstance(n, ast.Name)} + if "bFairdraw" not in cond: + continue + writes = [ + s.lineno + for s in ast.walk(sub) + if isinstance(s, ast.Assign) + for t in s.targets + if isinstance(t, ast.Subscript) + and isinstance(t.value, ast.Attribute) + and t.value.attr == "_rvs" + ] + if writes: + # innermost rebind wins; first write line is where _rvs stops being the + # retained set + out[node.name] = min(out.get(node.name, 10 ** 9), min(writes)) + return out + + +def _function_spans(tree): + spans = [] + for node in ast.walk(tree): + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + spans.append((node.lineno, _end_lineno(node, node.lineno), node.name)) + spans.sort(key=lambda s: s[1] - s[0]) # innermost first + return spans + + +def _call_sites(tree): + """{callee name: [(line, enclosing function), ...]} for intra-file calls.""" + spans = _function_spans(tree) + + def enclosing(lineno): + for lo, hi, name in spans: + if lo <= lineno <= hi: + return name + return "" + + out = {} + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + fn = node.func + name = fn.attr if isinstance(fn, ast.Attribute) else getattr(fn, "id", None) + if name: + out.setdefault(name, []).append((node.lineno, enclosing(node.lineno))) + return out + + +def scan_file(relpath): + path = os.path.join(CODE_ROOT, relpath) + if not os.path.exists(path): + return [{"file": relpath, "line": 0, "phase": "MISSING", "source": "file not found"}] + with open(path, "r", errors="replace") as f: + src = f.read() + try: + tree = ast.parse(src) + except SyntaxError as e: + return [{"file": relpath, "line": 0, "phase": "UNPARSEABLE", "source": str(e)}] + + parents = {} + for node in ast.walk(tree): + for child in ast.iter_child_nodes(node): + parents[id(child)] = node + + rebinds = _rebind_lines(tree) + spans = _function_spans(tree) + calls = _call_sites(tree) + lines = src.splitlines() + + def enclosing(lineno): + for lo, hi, name in spans: + if lo <= lineno <= hi: + return name + return "" + + write_nodes = set() + for node in ast.walk(tree): + if isinstance(node, (ast.Assign, ast.AugAssign)) or type(node).__name__ == "AnnAssign": + targets = node.targets if isinstance(node, ast.Assign) else [node.target] + for t in targets: + for sub in ast.walk(t): + write_nodes.add(id(sub)) + + def phase_of(fn_name, lineno): + """Classify a read, following one level of intra-file call graph.""" + rebind = rebinds.get(fn_name) + if rebind is not None: + return ("BEFORE" if lineno < rebind else "AFTER"), rebind + # helper: where is it called from? + verdicts = set() + rb_line = None + for call_line, caller in calls.get(fn_name, []): + caller_rebind = rebinds.get(caller) + if caller_rebind is None: + continue + rb_line = caller_rebind + verdicts.add("CALLED_BEFORE" if call_line < caller_rebind else "CALLED_AFTER") + if verdicts == {"CALLED_BEFORE"}: + return "CALLED_BEFORE", rb_line + if verdicts == {"CALLED_AFTER"}: + return "CALLED_AFTER", rb_line + if verdicts: + return "CALLED_BOTH", rb_line + # No intra-file caller inside a rebinding function. If this file has any + # rebind at all it is an integrator, and an unlinked helper is ambiguous; + # otherwise this is a downstream consumer of a returned sampler. + return ("NO_REBIND" if rebinds else "POST_INTEGRATE"), None + + hits = [] + for node in ast.walk(tree): + if not (isinstance(node, ast.Attribute) and node.attr == "_rvs"): + continue + parent = parents.get(id(node)) + subscript = parent if isinstance(parent, ast.Subscript) and parent.value is node else None + target = subscript if subscript is not None else node + if id(target) in write_nodes: + continue # a write, not a read + + key = _const_str(subscript.slice) if subscript is not None else None + + pop = None + p = parents.get(id(target)) + depth = 0 + while p is not None and depth < 5: + if isinstance(p, ast.Call): + fn = p.func + name = fn.attr if isinstance(fn, ast.Attribute) else getattr(fn, "id", None) + if name in POPULATION_CALLS: + pop = name + break + p = parents.get(id(p)) + depth += 1 + + fn_name = enclosing(node.lineno) + phase, rebind = phase_of(fn_name, node.lineno) + + hits.append({ + "file": relpath, + "line": node.lineno, + "function": fn_name, + "key": key, + "phase": phase, + "rebind_line": rebind, + "population_call": pop, + "derived_key": (key in DERIVED_KEYS) if key else None, + "source": lines[node.lineno - 1].strip()[:200], + }) + return hits + + +# --------------------------------------------------------------------------- ledger +# Human verdicts for post-rebind sites, in rvs_fairdraw_verdicts.json. Verdict is one of: +# PER_ROW -- reads a per-row value it independently owns. The resample changes WHICH +# rows are present, never the value carried by a row, so this is correct. +# FIXED -- was broken; now reads the retained set or an exact recorded total. +# BROKEN -- reads a population statistic of the resample. Must name its follow-up. +# BENIGN -- a population read whose consumer is documented as describing the EXPORT +# rather than the integral, so the resample is the right input. +# NO_FAIRDRAW -- this caller never sets igrand_fairdraw_samples, so no rebind happens on +# its sampler and _rvs is still the retained set. VERIFY BY GREP, not by +# assumption: the flag arrives from the caller, not from the sampler. +LEDGER_PATH = os.path.join(HERE, "rvs_fairdraw_verdicts.json") +VALID_VERDICTS = ("PER_ROW", "FIXED", "BROKEN", "BENIGN", "NO_FAIRDRAW") + + +def site_key(hit): + """A stable identifier for one post-rebind read. + + NOT the line number -- these files are edited constantly and a line-keyed ledger would be + stale within a week. NOT the enclosing function either: analyze_event alone holds 40 of + these, so a function-keyed ledger would let a NEW consumer be added beside 39 approved + ones without tripping anything -- exactly the failure mode this gate exists to prevent, + since every one of the five known defects was added next to correct code. + + So: file, function, and a hash of the READ ITSELF with whitespace squeezed out. Moving + or reindenting a line keeps its verdict; changing what it reads does not. + """ + norm = "".join((hit.get("source") or "").split()) + h = hashlib.sha1(norm.encode("utf-8")).hexdigest()[:10] + return "{}:{}:{}".format(hit["file"], hit["function"], h) + + +def load_verdicts(): + """Verdicts live in a sidecar JSON so the ledger can be edited without touching the + scanner. Missing file => empty ledger => --check reports everything, which is the + correct behaviour for a gate: absent evidence is not a pass.""" + if not os.path.exists(LEDGER_PATH): + return {} + with open(LEDGER_PATH) as f: + return json.load(f) + + +# --------------------------------------------------------------------------- main +def main(): + ap = argparse.ArgumentParser( + description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--json", action="store_true") + ap.add_argument("--summary", action="store_true") + ap.add_argument("--check", action="store_true") + ap.add_argument("--emit-ledger", action="store_true", + help="print a verdict ledger skeleton, preserving existing verdicts") + ap.add_argument("--phase", help="comma-separated phases to keep") + ap.add_argument("--post-rebind", action="store_true", + help="only sites that see the resample") + args = ap.parse_args() + + hits = [] + for t in TARGETS: + hits.extend(scan_file(t)) + if args.post_rebind: + hits = [h for h in hits if h.get("phase") in POST_REBIND_PHASES] + if args.phase: + keep = set(args.phase.split(",")) + hits = [h for h in hits if h.get("phase") in keep] + + if args.json: + json.dump(hits, sys.stdout, indent=2) + sys.stdout.write("\n") + return 0 + + if args.summary: + tally = {} + for h in hits: + tally.setdefault(h["file"], {}).setdefault(h["phase"], 0) + tally[h["file"]][h["phase"]] += 1 + phases = sorted({p for v in tally.values() for p in v}) + print("{:<58} {}".format("file", " ".join("{:>14}".format(p) for p in phases))) + for f in sorted(tally): + print("{:<58} {}".format( + f, " ".join("{:>14}".format(tally[f].get(p, "")) for p in phases))) + print("\ntotal sites: {}".format(len(hits))) + return 0 + + if args.emit_ledger: + ledger = load_verdicts() + out = {} + for h in hits: + if h.get("phase") not in POST_REBIND_PHASES: + continue + k = site_key(h) + out[k] = ledger.get(k, { + "verdict": "TODO", + "why": "", + "source": h.get("source", ""), + }) + out[k]["source"] = h.get("source", "") + json.dump(out, sys.stdout, indent=2, sort_keys=True) + sys.stdout.write("\n") + return 0 + + if args.check: + ledger = load_verdicts() + unclassified, bad = [], [] + n_post = 0 + for h in hits: + if h.get("phase") not in POST_REBIND_PHASES: + continue + n_post += 1 + k = site_key(h) + entry = ledger.get(k) + if entry is None: + unclassified.append((k, h["line"], h["source"])) + elif entry.get("verdict") not in VALID_VERDICTS: + bad.append((k, h["line"], entry.get("verdict"))) + if unclassified or bad: + if unclassified: + print("UNCLASSIFIED post-rebind _rvs reads ({}):".format(len(unclassified))) + for k, line, src in unclassified: + print(" {}\n line {}: {}".format(k, line, src)) + if bad: + print("\nSites with no usable verdict ({}); must be one of {}:".format( + len(bad), ", ".join(VALID_VERDICTS))) + for k, line, v in bad: + print(" {} (line {}) -> {!r}".format(k, line, v)) + print("\n`_rvs` is an EXPORT resample by this point. For each site above, decide") + print("whether it reads a per-row value it owns (fine) or a population statistic") + print("of the resample (broken), then record it:") + print(" python3 {} --emit-ledger > {}".format( + os.path.basename(__file__), os.path.basename(LEDGER_PATH))) + return 1 + print("OK: all {} post-rebind _rvs reads carry a recorded verdict.".format(n_post)) + return 0 + + by_file = {} + for h in hits: + by_file.setdefault(h["file"], []).append(h) + n_flag = 0 + for f in sorted(by_file): + print("\n=== {} ===".format(f)) + for h in sorted(by_file[f], key=lambda x: x["line"]): + flag = "" + if h.get("phase") in POST_REBIND_PHASES and h.get("population_call"): + flag = " <== post-rebind + {}()".format(h["population_call"]) + n_flag += 1 + print(" {:>5} {:<34} {:<15} key={:<22}{}".format( + h["line"], str(h.get("function"))[:34], str(h.get("phase")), + str(h.get("key")), flag)) + print(" {}".format(h.get("source", ""))) + print("\n{} sites; {} post-rebind reads feed a population-shaped call.".format( + len(hits), n_flag)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/make_rvs_fairdraw_ledger.py b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/make_rvs_fairdraw_ledger.py new file mode 100644 index 000000000..5dec706af --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/make_rvs_fairdraw_ledger.py @@ -0,0 +1,212 @@ +"""Regenerate rvs_fairdraw_verdicts.json from the classification RULES. + +Run after moving or editing a post-rebind _rvs consumer: + + python3 make_rvs_fairdraw_ledger.py && python3 audit_rvs_fairdraw.py --check + +The rules are the audit's reasoning in executable form -- ship the thing that +regenerates the list, not just the list. + +Every rule below corresponds to a code path that was READ during the audit; the rule is +how the verdict is applied to each of that path's sites, not a substitute for having +looked. Sites that match nothing stay TODO and fail --check, which is the intended +behaviour for anything this pass did not actually reach. +""" +import json +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +import audit_rvs_fairdraw as A # noqa: E402 + +hits = [h for t in A.TARGETS for h in A.scan_file(t) + if h.get("phase") in A.POST_REBIND_PHASES] + + +def verdict(h): + f, fn, src, line = h["file"], h["function"], h["source"], h["line"] + s = " ".join(src.split()) + + # --- the integrators themselves ------------------------------------------------ + if f.startswith("RIFT/integrators/"): + if "indx_list" in s: + return ("PER_ROW", + "The rebind's own right-hand side: this IS the fair draw, gathering each " + "key at the drawn indices. Read before the write it feeds.") + if "identity_convert(self._rvs[name])" in s or "for name in self._rvs" in s \ + or "isinstance(self._rvs[name]" in s: + return ("PER_ROW", + "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows " + "are present. Touches no population statistic; correct on any row set.") + if "self._rvs['integrand'] = self._rvs['log_integrand']" in s.replace('"', "'"): + return ("PER_ROW", + "Aliases one per-row column onto another so raw-field consumers find it. " + "Row-for-row; unaffected by which rows survived.") + + # --- distance exporters: the live defect ---------------------------------------- + if "rvs=(dict(sampler._rvs) if rvs is None else rvs)" in s: + return ("PER_ROW", + "_snapshot_pass_state takes a dict copy of whatever rows are present so a " + "rejected warm pass can be undone. It makes no claim about their statistics, " + "and it snapshots the fair-draw MARKER alongside them so the restored record " + "and the marker describing it cannot disagree.") + if s in ("rvs = sampler._rvs", "_rvs = sampler._rvs") or \ + 'sampler._rvs["distance"]' in s: + return ("PER_ROW", + "Binds the record, or reads one COLUMN of it, for the distance exporters. The " + "resample changes which rows are present, never the value a row carries, and " + "the weighting decision is made separately by ln_weights_for_posterior (.dgrid) " + "or by forcing the all-fresh path (.dslice). Correct on either row set.") + if "ln_weights_for_posterior" in s: + return ("FIXED", + "Asks for POSTERIOR weights, which are uniform when the record is the " + "fair-draw export and the derived importance weights otherwise. The predicate " + "is set by the sampler at the rebind, so it means 'the draw fired' rather than " + "'the flag was passed'. Pinned by test_fairdraw_double_weighting.py.") + if f == "RIFT/misc/distance_slices.py": + return ("FIXED", + "The .dslice reweight core cannot be corrected after the fact -- it " + "double-counts pi_Omega/q_Omega and takes N from the resample, and the " + "pre-draw record is gone by then. The ILE now forces the exact all-fresh path " + "(K independent fixed-d integrations) when the record is fair-drawn, and says " + "so, rather than reporting a plausible wrong number.") + + # --- the L0 rescue and the sequential warm start --------------------------------- + if f == "bin/integrate_likelihood_extrinsic_batchmode": + if "_lnZ_of_reserve_or_rvs" in s: + return ("FIXED", + "PR #79, re-landed as #86. sampler._rvs is passed as the FALLBACK " + "argument only: the helper prefers the retained reserve via " + "lnZ_from_reserve, and the gate refuses to compare across sources " + "(_cold_src != _warm_src forces BOTH back to the fair-draw reading, which " + "is at least self-consistent). Measured before #79: two passes with " + "identical true lnZ at n_eff 1.8 vs 53 produced a +3.48 nat gap and " + "rejected the good warm pass 100% of the time at the 0.5 default.") + if "_lnZ_of_rvs" in s: + return ("BROKEN", + "PR #79's CROSS-SOURCE FALLBACK: fires only when the cold and warm passes " + "produced different reading sources (one had a reserve, the other did " + "not), and then re-reads BOTH sides from the fair-draw record. That is " + "self-consistent, which is what #79 claims for it, but it is not " + "unbiased: the two passes sit at different n_eff, so the log(n/n_eff) " + "artifact does not cancel and this branch is back in the regime measured " + "at +3.48 nats / 100% rejection at the 0.5 default. A known, documented, " + "BOUNDED residual -- not a defect anyone introduced. Closing it needs a " + "retained-set reading on both sides, i.e. a reserve for the samplers that " + "keep none. Follow-up, not a regression.") + if "_kish_neff_of_rvs" in s or "list(sampler._rvs.values())[0]" in s: + return ("FIXED", + "Kish of the pooled record is only used when the export is NOT fair-drawn. " + "When it is, _pool_replica_rvs has flattened each block, and the Kish of " + "piecewise-constant weights is just the row count (5K at the default " + "--fairdraw-extrinsic-output-n-max 5). The ILE now computes the same " + "quantity one level up -- (sum Z_k)^2 / sum(Z_k^2/neff_k), Kish over the " + "BLOCKS -- which reduces to sum(neff_k) when replicas agree and falls " + "below it when they do not. The row count beside it is a deliberate " + "report of the export size.") + if "_warm_seed_reserve_for" in s or "_res_l0" in s or "_seed_ws" in s \ + or "_res_ws" in s or "_SEQ_WS_PENDING = _seed_ws" in s: + return ("FIXED", + "Seeds from the retained-sample reserve, falling back to _rvs only for a " + "sampler that keeps none, and judges the seed by affine RANK via " + "build_warm_seed. PR #78 for the L0 rescue; this change for the sequential " + "warm start. Pinned by test_l0_rescue_seed.py and test_seq_warmstart_seed.py.") + if "_cold_rvs" in s: + return ("PER_ROW", + "Snapshots the cold record so the reject path can restore it. A dict copy " + "of whatever rows exist; makes no claim about their statistics.") + if "_rep_rvs" in s: + return ("PER_ROW", + "Collects each replica's record for pooling. _pool_replica_rvs is told " + "already_resampled=bool(opts.fairdraw_extrinsic_output) and forces flat " + "within-block weights on that path, so the resampling is accounted for " + "THERE rather than here.") + + if "extrinsic_handoff" in s or (s == "_rvs = sampler._rvs"): + return ("FIXED", + "Extrinsic-proposal breadcrumb: fits a GMM to _rvs rows with " + "log_weights=_lw, where _lw is rebuilt from those same rows. Under " + "--fairdraw-extrinsic-output the rows are ALREADY a w-proportional draw, so " + "the proposal is fitted to w^2 and comes out over-concentrated -- and it is " + "then handed to the NEXT iteration via --extrinsic-proposal-breadcrumb, " + "which is the truncated-support failure mode this whole line of work is " + "about. Same class as .dgrid/.dslice; propagates forward, so arguably the " + "worst of the three. FIXED: it now takes ln_weights_for_posterior, " + "which is uniform on a fair-drawn record.") + if "_lnkey else np.array" in s or "_lnv = (np.asarray" in s or \ + ("_cols = (np.vstack" in s): + return ("BENIGN", + "The DELIBERATE no-reserve fallback for a sampler that keeps none: reads " + "_rvs so the feature degrades to its previous behaviour rather than to no " + "seed at all. The rank hazard is still handled -- build_warm_seed puffs the " + "result to full rank -- so what remains is only 'fewer points', which " + "cannot be improved without a reserve.") + if "(fair draw left" in s or ("len(np.asarray(sampler.identity_convert(sampler._rvs['log_integrand']))" in s): + return ("BENIGN", + "Reports how many rows the fair draw left, for the log line that contrasts " + "it with the retained count. Reading the resample's size is the POINT here.") + + # --- MAP seed / coordinate transform / export, in all three ILE scripts ---------- + if f.startswith("bin/integrate_likelihood_extrinsic"): + if "indx_guess" in s: + return ("BENIGN", + "MAP-seed read: argmax over the record, then the SAME row's coordinates. " + "The fair draw picks rows proportional to weight, so the retained argmax is " + "a genuinely-drawn high-likelihood point; it seeds a local search and a " + "printed diagnostic, and no downstream number is a statistic of it. " + "Degraded (it may not be the global MAP), not wrong.") + if "numpy.arccos" in s or "numpy.pi/2 -" in s or 'sampler._rvs[("declination"' in s: + return ("PER_ROW", + "In-place coordinate transform, row for row.") + if "copy.deepcopy(sampler._rvs)" in s or s.startswith("samples = sampler._rvs") \ + or "for key in sampler._rvs.keys()" in s: + return ("BENIGN", + "THE export itself. Under --fairdraw-extrinsic-output the fair draw is " + "precisely what these rows are supposed to be, so the resample is the " + "correct input here rather than a hazard.") + if 'len(sampler._rvs["psi"])' in s: + return ("PER_ROW", + "Builds a constant t_ref column conformal with the exported rows. Uses the " + "length only to match shape, and makes no claim about it.") + if "lnL at start" in s: + return ("BENIGN", + "Printed diagnostic of the best retained sample. Explicitly labelled as " + "what ILE reports including weights; not an input to any result.") + if '"distance" not in sampler._rvs' in s or "in sampler._rvs" in s \ + or 'sampler._rvs["psi"],' in s: + return ("PER_ROW", + "Key-presence / column reference, not a population statistic.") + + # --- CIP: no fair draw happens on these samplers --------------------------------- + if f.startswith("bin/util_Construct"): + return ("NO_FAIRDRAW", + "Verified by grep: neither util_ConstructIntrinsicPosterior_GenericCoordinates " + "nor util_ConstructEOSPosterior passes igrand_fairdraw_samples, so integrate() " + "never runs the rebind and _rvs is still the retained set. (The known CIP " + "posterior-export defect was a different mechanism -- replace=False successive " + "sampling, PR #44 lineage -- not this one.)") + + return (None, None) + + +ledger, todo = {}, [] +for h in hits: + k = A.site_key(h) + v, why = verdict(h) + if v is None: + todo.append((k, h["file"], h["line"], h["source"])) + continue + ledger[k] = {"verdict": v, "why": why, "source": h.get("source", "")} + +path = A.LEDGER_PATH +with open(path, "w") as f: + json.dump(ledger, f, indent=2, sort_keys=True) + f.write("\n") + +from collections import Counter +print("wrote", len(ledger), "verdicts:", dict(Counter(v["verdict"] for v in ledger.values()))) +if todo: + print("\nUNMATCHED (left TODO, will fail --check):") + for k, f_, ln, src in todo: + print(" {}:{} {}".format(f_, ln, " ".join(src.split())[:110])) diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/measure_l0_reject_dlnZ.py b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/measure_l0_reject_dlnZ.py new file mode 100644 index 000000000..2eb64bf24 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/measure_l0_reject_dlnZ.py @@ -0,0 +1,349 @@ +#!/usr/bin/env python3 +"""Measure `--sampler-l0-rescue-reject-dlnZ` against the POST-#79 gate. + +WHY THIS HAD TO WAIT FOR #79 +---------------------------- +The 0.5-nat default was chosen when both sides of the comparison were read out of the +fair-drawn `_rvs`, which carries a `log(n_retained/eff_samp)` offset that does NOT cancel +between two passes at different `n_eff` (measured at +3.48 nats, `verify_skew.py`). Tuning +against that would have been calibrating to the bug. With #79 in (re-landed as #86) both +sides come from the retained reserve via `lnZ_from_reserve`, so the number now means what the +knob's help text says it means, and it can be measured. + +WHAT THE KNOB DECIDES +--------------------- +The L0 rescue re-runs a collapsed pass warm, seeded from that pass's own peak. The warm pass +is precise but confined to the seeded region, so it is biased low by any mode the seed missed. +The gate keeps the COLD result when + + cold_lnZ - warm_lnZ > dlnZ + +The trade is asymmetric and both directions are real: + * TOO SMALL -> a good warm pass is binned in favour of a collapsed cold one. LOUD: the run + reports the collapse. + * TOO LARGE -> a genuinely truncated warm pass is reported. QUIET: it has an excellent ESS + over a sliver of the support, and nothing says so. + +So this measures two distributions on known-lnZ targets: + NULL unimodal. The seed cannot miss a mode because there is only one. Any gap is noise, + and its spread is the floor below which no threshold can go. + SIGNAL bimodal, mass fraction `f` in the mode the cold pass found. A seed confined to that + mode measures f*Z, so the true deficit is -log(f). + +Usage: + python3 measure_l0_reject_dlnZ.py # default: 40 replicates per condition + python3 measure_l0_reject_dlnZ.py --reps 100 +Runs single-threaded on CPU; set OMP_NUM_THREADS=1. +""" +import argparse +import os +import sys + +import numpy as np + +HERE = os.path.dirname(os.path.abspath(__file__)) +CODE = os.path.abspath(os.path.join(HERE, "..", "..", "..")) +sys.path.insert(0, CODE) + +import RIFT.integrators.mcsamplerAdaptiveVolume as mcsamplerAV # noqa: E402 +from RIFT.integrators.mcsamplerAdaptiveVolume import build_warm_seed # noqa: E402 + +NAMES = ['right_ascension', 'declination', 'phi_orb', 'inclination', 'psi', 'distance'] +NDIM = len(NAMES) +LO = np.zeros(NDIM) +HI = np.ones(NDIM) +AX = list(range(NDIM)) +_ILE = os.path.join(CODE, "bin", "integrate_likelihood_extrinsic_batchmode") + + +def _load_ile_helpers(): + """Exec the ILE's lnZ helpers so this measures THE gate, not a copy of it.""" + src = open(_ILE).read() + start = src.index("def ln_weights_from_rvs") + end = src.index("def _warm_seed_geometry") + ns = {"numpy": np, "np": np, "mcsamplerAdaptiveVolume": mcsamplerAV, + "_rvs_lnL_convention": lambda x=None: bool(x)} + exec(compile(src[start:end], "ile_lnZ_helpers", "exec"), ns) + assert "_lnZ_of_reserve_or_rvs" in ns, "PR #79 helper absent -- is #86 merged in this tree?" + return ns + + +H = _load_ile_helpers() +_lnZ_of_reserve_or_rvs = H["_lnZ_of_reserve_or_rvs"] + + +SAMPLER_KIND = 'AV' + + +def _sampler(n_chunk=20000): + if SAMPLER_KIND == 'portfolio': + return _portfolio(n_chunk) + s = mcsamplerAV.MCSampler(n_chunk=n_chunk) + s.xpy = mcsamplerAV.xpy_default + s.identity_convert = mcsamplerAV.identity_convert + for name in NAMES: + s.add_parameter(name, pdf=None, left_limit=0.0, right_limit=1.0, + prior_pdf=lambda x: np.ones(np.shape(x)), adaptive_sampling=True) + return s + + +def _portfolio(n_chunk=20000): + """AV + GMM portfolio, built the way the ILE builds one (see test_l0_rescue_seed). + + The interesting difference for this measurement: the GMM member carries a DEFENSIVE + component, so the portfolio is the configuration the L0 rescue's own docstring says + "avoids this trade entirely". Whether its cold pass is a usable lnZ reference -- which is + what the gate needs and what AV cannot supply -- is exactly the open question. + """ + import RIFT.integrators.mcsamplerPortfolio as mcsamplerPF + import RIFT.integrators.mcsamplerEnsemble as mcsamplerEnsemble + members = [mcsamplerAV.MCSampler(n_chunk=n_chunk), mcsamplerEnsemble.MCSampler()] + s = mcsamplerPF.MCSampler(portfolio=members) + pdf = np.vectorize(lambda x: 1.0) + for name in NAMES: + s.add_parameter(name, pdf, prior_pdf=pdf, left_limit=0.0, right_limit=1.0, + adaptive_sampling=True) + s.setup() + return s + + +class Target(object): + """Sum of Gaussians on the unit cube, with the float64 underflow of the real code. + + Uniform prior of density 1, so Z = sum_k A_k * prod_i (w_ki * sqrt(2 pi)) when every mode + sits well inside the cube -- which is checked at construction. + """ + + def __init__(self, centers, widths, log_amps): + self.c = np.atleast_2d(np.asarray(centers, dtype=float)) + self.w = np.atleast_2d(np.asarray(widths, dtype=float)) + self.a = np.asarray(log_amps, dtype=float).ravel() + # every mode must be >5 sigma from every face, or the analytic Z is wrong + assert np.all(self.c - 5 * self.w > 0) and np.all(self.c + 5 * self.w < 1), \ + "a mode is too close to the boundary for the analytic normalization" + self.ln_mode_Z = self.a + np.sum(np.log(self.w * np.sqrt(2 * np.pi)), axis=1) + m = self.ln_mode_Z.max() + self.lnZ_true = float(m + np.log(np.sum(np.exp(self.ln_mode_Z - m)))) + self.mass_frac = np.exp(self.ln_mode_Z - self.lnZ_true) + self.lnLmax = float(self.a.max()) + # modes carrying a non-negligible share; a 1e-6 mode is not a miss worth flagging + self.n_modes_expected = int(np.sum(self.mass_frac > 0.02)) + + def __call__(self, *args, **kwargs): + x = np.array([np.asarray(v, dtype=float).ravel() for v in args]).T + terms = np.stack([ + self.a[k] - 0.5 * np.sum(((x - self.c[k]) / self.w[k]) ** 2, axis=-1) + for k in range(len(self.a))], axis=0) + m = terms.max(axis=0) + out = m + np.log(np.sum(np.exp(terms - m), axis=0)) + return np.where(out > self.lnLmax - 745.0, out, -np.inf) + + def nearest_mode(self, X): + d = np.stack([np.sum(((X - self.c[k]) / self.w[k]) ** 2, axis=-1) + for k in range(len(self.a))], axis=0) + return np.argmin(d, axis=0), np.sqrt(d.min(axis=0)) + + +def one_replicate(target, seed, neff_target=8, nmax=400000, n=20000, fairdraw_max=200): + """Run the REAL rescue sequence: cold pass, reserve, rank-tested seed, warm pass. + + Returns None when the cold pass did not actually collapse -- a replicate that lands on a + healthy pass is not evidence about a knob that only fires on collapsed ones. + """ + np.random.seed(seed) + s = _sampler() + kw = dict(no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=fairdraw_max) + if SAMPLER_KIND == 'portfolio': + kw['save_intg'] = True + try: + cold = s.integrate_log(target, *NAMES, nmax=nmax, neff=neff_target, n=n, **kw) + except Exception: + return None + cold_neff = None if cold is None or cold[2] is None else float( + mcsamplerAV.identity_convert(cold[2])) + reserve = getattr(s, '_warm_seed_reserve', None) + if reserve is None: + return None + cold_rvs = dict(s._rvs) + cold_lnZ, cold_src = _lnZ_of_reserve_or_rvs(s, cold_rvs, reserve=reserve) + + X = np.asarray(reserve['X'], dtype=float) + lnv = np.asarray(reserve['lnL'], dtype=float).ravel() + if lnv.size < 1 or not np.any(np.isfinite(lnv)): + return None + seed_pts, info = build_warm_seed(X, lnv, LO, HI, AX, deltalnL=15.0) + seed_mode = int(target.nearest_mode(np.atleast_2d(seed_pts))[0][0]) + + try: + s.bootstrap_from_samples(seed_pts, cover_frac=0.0) + warm = s.integrate_log(target, *NAMES, nmax=nmax, neff=neff_target, n=n, **kw) + except Exception: + return None + warm_neff = None if warm is None or warm[2] is None else float( + mcsamplerAV.identity_convert(warm[2])) + warm_lnZ, warm_src = _lnZ_of_reserve_or_rvs(s, s._rvs) + if cold_lnZ is None or warm_lnZ is None: + return None + if not (np.isfinite(cold_lnZ) and np.isfinite(warm_lnZ)): + return None + + # did the warm pass actually reach every mode? (its own retained set is the evidence) + wres = getattr(s, '_warm_seed_reserve', None) + modes_seen = set() + if wres is not None: + idx, _ = target.nearest_mode(np.asarray(wres['X'], dtype=float)) + modes_seen = set(int(v) for v in np.unique(idx)) + _rep = lambda r: (None if (r is None or r[0] is None) + else float(mcsamplerAV.identity_convert(r[0]))) + return dict(cold_reported=_rep(cold), warm_reported=_rep(warm), + gap=float(cold_lnZ - warm_lnZ), cold_lnZ=float(cold_lnZ), + warm_lnZ=float(warm_lnZ), cold_src=cold_src, warm_src=warm_src, + cold_neff=cold_neff, warm_neff=warm_neff, seed_mode=seed_mode, + modes_seen=modes_seen, puffed=bool(info['puffed'])) + + +def run(label, target, reps, seed0): + rows = [] + for i in range(reps): + r = one_replicate(target, seed0 + i) + if r is not None: + rows.append(r) + print("\n### {} ({} usable of {} replicates)".format(label, len(rows), reps)) + if not rows: + print(" no usable replicates") + return rows + g = np.array([r['gap'] for r in rows]) + srcs = set((r['cold_src'], r['warm_src']) for r in rows) + print(" true lnZ {:.4f} mass fractions {}".format( + target.lnZ_true, np.array2string(target.mass_frac, precision=3))) + print(" lnZ sources (cold,warm): {}".format(sorted(srcs))) + print(" cold n_eff median {:.2f} warm n_eff median {:.2f}".format( + float(np.median([r['cold_neff'] or np.nan for r in rows])), + float(np.median([r['warm_neff'] or np.nan for r in rows])))) + if len(target.a) > 1: + conf = sum(1 for r in rows if len(r['modes_seen']) < len(target.a)) + print(" warm pass reached only ONE mode in {}/{} replicates".format(conf, len(rows))) + dc = np.array([r['cold_lnZ'] for r in rows]) - target.lnZ_true + dw = np.array([r['warm_lnZ'] for r in rows]) - target.lnZ_true + print(" cold_lnZ - TRUE: median {:+.3f} sd {:.3f} <- the gate's reference".format( + float(np.median(dc)), float(np.std(dc, ddof=1)))) + print(" warm_lnZ - TRUE: median {:+.3f} sd {:.3f}".format( + float(np.median(dw)), float(np.std(dw, ddof=1)))) + cr = np.array([r['cold_reported'] for r in rows if r['cold_reported'] is not None]) + wr = np.array([r['warm_reported'] for r in rows if r['warm_reported'] is not None]) + if cr.size: + print(" cold lnZ REPORTED BY integrate_log - TRUE: median {:+.3f} sd {:.3f}".format( + float(np.median(cr - target.lnZ_true)), float(np.std(cr - target.lnZ_true, ddof=1)))) + if wr.size: + print(" warm lnZ REPORTED BY integrate_log - TRUE: median {:+.3f} sd {:.3f}".format( + float(np.median(wr - target.lnZ_true)), float(np.std(wr - target.lnZ_true, ddof=1)))) + print(" gap = cold_lnZ - warm_lnZ: median {:+.3f} mean {:+.3f} sd {:.3f}" + " [p05 {:+.3f}, p95 {:+.3f}]".format( + float(np.median(g)), float(np.mean(g)), float(np.std(g, ddof=1)), + float(np.percentile(g, 5)), float(np.percentile(g, 95)))) + return rows + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--reps", type=int, default=40) + ap.add_argument("--rho", type=float, default=100.0) + ap.add_argument("--sampler", choices=("AV", "portfolio"), default="AV") + ap.add_argument("--out-json", default=None, + help="dump per-replicate records so the analysis can be redone without " + "re-measuring (the threshold table is cheap; the passes are not)") + args = ap.parse_args() + global SAMPLER_KIND + SAMPLER_KIND = args.sampler + + rho = args.rho + w = (0.5 / rho) * np.ones(NDIM) + lnA = 0.5 * rho ** 2 + + print("=" * 78) + print("Measuring --sampler-l0-rescue-reject-dlnZ on the POST-#79 gate") + print("sampler = {}, rho = {}, {} replicates per condition".format( + args.sampler, rho, args.reps)) + print("=" * 78) + + conditions = [] + + # NULL: one mode. Any gap here is noise. + uni = Target([0.5 * np.ones(NDIM)], [w], [lnA]) + conditions.append(("NULL (unimodal -- a good warm pass)", uni, + run("NULL (unimodal -- a good warm pass)", uni, args.reps, 1000))) + + # SIGNAL: two modes, varying how much mass sits in the one the cold pass finds. + for f in (0.5, 0.75, 0.9): + c2 = np.copy(0.5 * np.ones(NDIM)); c2[0] = 0.25 + c1 = np.copy(0.5 * np.ones(NDIM)); c1[0] = 0.75 + # equal widths -> amplitude ratio sets the mass ratio + tgt = Target([c1, c2], [w, w], [lnA, lnA + np.log((1 - f) / f)]) + lbl = "SIGNAL (bimodal, seeded mode holds f={:.2f}; true deficit -log f = {:+.3f})".format( + f, -np.log(f)) + conditions.append((lbl, tgt, run(lbl, tgt, args.reps, 2000 + int(1000 * f)))) + + if args.out_json: + import json + with open(args.out_json, "w") as fh: + json.dump([dict(condition=lbl, lnZ_true=t.lnZ_true, + mass_frac=list(map(float, t.mass_frac)), n_modes=len(t.a), + **{k: (sorted(v) if isinstance(v, set) else v) + for k, v in r.items()}) + for lbl, t, rows in conditions for r in rows], fh, indent=1) + print("\nper-replicate records -> {}".format(args.out_json)) + + # ---- rejection rate CONDITIONED ON WHAT ACTUALLY HAPPENED + # The condition label is the INTENT, not the outcome: on a portfolio the defensive GMM + # component means a seeded warm pass usually still reaches every mode, so most "SIGNAL" + # replicates are not truncated and a rejection there is a FALSE positive. Split on the + # warm pass's own retained set instead of on the label. + good = [r for _, t, rows in conditions for r in rows + if len(r['modes_seen']) >= t.n_modes_expected] + trunc = [r for _, t, rows in conditions for r in rows + if len(r['modes_seen']) < t.n_modes_expected] + print("\n" + "=" * 78) + print("REJECTION RATE BY WHAT THE WARM PASS ACTUALLY DID") + print(" good = warm pass reached every mode ({} replicates)".format(len(good))) + print(" trunc = warm pass reached fewer ({} replicates)".format(len(trunc))) + print("=" * 78) + gg = np.array([r['gap'] for r in good]) if good else np.array([]) + tg = np.array([r['gap'] for r in trunc]) if trunc else np.array([]) + print("{:>8} {:>26} {:>26}".format("dlnZ", "FPR (good pass binned)", "TPR (truncation caught)")) + for thr in (0.25, 0.5, 0.75, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0): + f = "{:>25.0f}%".format(100 * np.mean(gg > thr)) if gg.size else "{:>26}".format("n/a") + t_ = "{:>25.0f}%".format(100 * np.mean(tg > thr)) if tg.size else "{:>26}".format("n/a") + print("{:>8.2f} {} {}".format(thr, f, t_)) + + # ---- the decision table + print("\n" + "=" * 78) + print("THRESHOLD TABLE") + print(" FPR = a GOOD warm pass rejected (loud failure: collapsed cold result reported)") + print(" TPR = a TRUNCATED warm pass rejected (the catch we want)") + print("=" * 78) + null = np.array([r['gap'] for r in conditions[0][2]]) if conditions[0][2] else np.array([]) + sig = [(lbl, np.array([r['gap'] for r in rows])) for lbl, _, rows in conditions[1:] if rows] + hdr = "{:>8} {:>8}".format("dlnZ", "FPR") + for lbl, _ in sig: + f = lbl.split("f=")[1][:4] + hdr += " {:>10}".format("TPR f=" + f) + print(hdr) + for thr in (0.25, 0.5, 0.75, 1.0, 1.5, 2.0, 3.0): + line = "{:>8.2f} {:>7.0f}%".format(thr, 100 * np.mean(null > thr) if null.size else np.nan) + for _, gg in sig: + line += " {:>9.0f}%".format(100 * np.mean(gg > thr)) + print(line) + + if null.size: + print("\nNULL spread sets the floor: sd {:.3f} nats, p95 {:+.3f}.".format( + float(np.std(null, ddof=1)), float(np.percentile(null, 95)))) + print("STRUCTURAL BLIND SPOT: a threshold T can never catch a missed mode carrying less") + print("than 1-exp(-T) of the mass, however well tuned --") + for thr in (0.5, 1.0, 2.0): + print(" T={:.1f} -> blind to any mode holding < {:.0f}% of the total mass".format( + thr, 100 * (1 - np.exp(-thr)))) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/rvs_fairdraw_verdicts.json b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/rvs_fairdraw_verdicts.json new file mode 100644 index 000000000..bdecbd3d8 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/rvs_fairdraw_verdicts.json @@ -0,0 +1,622 @@ +{ + "RIFT/integrators/mcsampler.py:integrate:ac2283de73": { + "source": "self._rvs[key] = self._rvs[key][indx_list]", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsampler.py:integrate:dad2084a85": { + "source": "self._rvs[key] = self._rvs[key][:,indx_list]", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerAdaptiveVolume.py:integrate_log:34ee3992c4": { + "source": "if isinstance(self._rvs[name],xpy_default.ndarray):", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerAdaptiveVolume.py:integrate_log:8f476f15d1": { + "source": "for name in self._rvs:", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerAdaptiveVolume.py:integrate_log:c0e27aa48f": { + "source": "self._rvs[name] = identity_convert(self._rvs[name]) # this is trivial if xpy_default is numpy, and a conversion otherwise", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerEnsemble.py:integrate:ac2283de73": { + "source": "self._rvs[key] = self._rvs[key][indx_list]", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerEnsemble.py:integrate:dad2084a85": { + "source": "self._rvs[key] = self._rvs[key][:,indx_list]", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate:2c1f0136f3": { + "source": "self._rvs[key] = identity_convert(self._rvs[key][indx_list])", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate:34ee3992c4": { + "source": "if isinstance(self._rvs[name],xpy_default.ndarray):", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate:8f476f15d1": { + "source": "for name in self._rvs:", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate:c0e27aa48f": { + "source": "self._rvs[name] = identity_convert(self._rvs[name]) # this is trivial if xpy_default is numpy, and a conversion otherwise", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate:cb15a6423d": { + "source": "self._rvs[key] = identity_convert(self._rvs[key][:,indx_list])", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate_log:2c1f0136f3": { + "source": "self._rvs[key] = identity_convert(self._rvs[key][indx_list])", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate_log:34ee3992c4": { + "source": "if isinstance(self._rvs[name],xpy_default.ndarray):", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate_log:8f476f15d1": { + "source": "for name in self._rvs:", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate_log:c0e27aa48f": { + "source": "self._rvs[name] = identity_convert(self._rvs[name]) # this is trivial if xpy_default is numpy, and a conversion otherwise", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerGPU.py:integrate_log:cb15a6423d": { + "source": "self._rvs[key] = identity_convert(self._rvs[key][:,indx_list])", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerNFlow.py:integrate_log:2c1f0136f3": { + "source": "self._rvs[key] = identity_convert(self._rvs[key][indx_list])", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerNFlow.py:integrate_log:34ee3992c4": { + "source": "if isinstance(self._rvs[name],xpy_default.ndarray):", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerNFlow.py:integrate_log:8f476f15d1": { + "source": "for name in self._rvs:", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerNFlow.py:integrate_log:c0e27aa48f": { + "source": "self._rvs[name] = identity_convert(self._rvs[name]) # this is trivial if xpy_default is numpy, and a conversion otherwise", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerNFlow.py:integrate_log:cb15a6423d": { + "source": "self._rvs[key] = identity_convert(self._rvs[key][:,indx_list])", + "verdict": "PER_ROW", + "why": "The rebind's own right-hand side: this IS the fair draw, gathering each key at the drawn indices. Read before the write it feeds." + }, + "RIFT/integrators/mcsamplerPortfolio.py:integrate_log:34ee3992c4": { + "source": "if isinstance(self._rvs[name],xpy_default.ndarray):", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerPortfolio.py:integrate_log:8f476f15d1": { + "source": "for name in self._rvs:", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerPortfolio.py:integrate_log:c0e27aa48f": { + "source": "self._rvs[name] = identity_convert(self._rvs[name]) # this is trivial if xpy_default is numpy, and a conversion otherwise", + "verdict": "PER_ROW", + "why": "Element-wise dtype/backend conversion (cupy -> numpy) over whatever rows are present. Touches no population statistic; correct on any row set." + }, + "RIFT/integrators/mcsamplerPortfolio.py:integrate_log:db92cab991": { + "source": "self._rvs['integrand'] = self._rvs['log_integrand'] # always integrating log function. Match behavior of other routines", + "verdict": "PER_ROW", + "why": "Aliases one per-row column onto another so raw-field consumers find it. Row-for-row; unaffected by which rows survived." + }, + "RIFT/misc/distance_slices.py:importance_reweight_slices:d948a54bfb": { + "source": "rvs = sampler._rvs", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic::0a939664b9": { + "source": "samples = sampler._rvs", + "verdict": "BENIGN", + "why": "THE export itself. Under --fairdraw-extrinsic-output the fair draw is precisely what these rows are supposed to be, so the resample is the correct input here rather than a hazard." + }, + "bin/integrate_likelihood_extrinsic::0d89fb8881": { + "source": "if \"distance\" not in sampler._rvs:", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic::16e8b48c86": { + "source": "(sampler._rvs[\"phi_orb\"][indx_guess]/(2*numpy.pi)), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::1dbc4dbe43": { + "source": "sampler._rvs[\"right_ascension\"][indx_guess]/(2*numpy.pi) , \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::29d2d2ebb6": { + "source": "P.dist = sampler._rvs[\"distance\"][indx_guess]*1e6*lal.PC_SI", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic::31db586050": { + "source": "P.incl = sampler._rvs[\"inclination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::45fa8d2146": { + "source": "sampler._rvs[\"psi\"][indx_guess]/numpy.pi,\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::4dd03bebc6": { + "source": "P.phi = sampler._rvs[\"right_ascension\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::5933ec4fe3": { + "source": "samples[\"t_ref\"] = float(fiducial_epoch)*numpy.ones(len(sampler._rvs[\"psi\"]))", + "verdict": "PER_ROW", + "why": "Builds a constant t_ref column conformal with the exported rows. Uses the length only to match shape, and makes no claim about it." + }, + "bin/integrate_likelihood_extrinsic::59fc582ac2": { + "source": "sampler._rvs[\"declination\"] = numpy.pi/2 - numpy.arccos(sampler._rvs[\"declination\"].astype(numpy.float64))", + "verdict": "PER_ROW", + "why": "In-place coordinate transform, row for row." + }, + "bin/integrate_likelihood_extrinsic::6658553446": { + "source": "P.theta = sampler._rvs[\"declination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::69aea9d541": { + "source": "sampler._rvs[\"inclination\"] = numpy.arccos(sampler._rvs[\"inclination\"].astype(numpy.float64))", + "verdict": "PER_ROW", + "why": "In-place coordinate transform, row for row." + }, + "bin/integrate_likelihood_extrinsic::6c4b180497": { + "source": "(sampler._rvs[\"declination\"][indx_guess]/numpy.pi), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::7226654481": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"integrand\"]) # start search near maximum-likelihood point. (WARNING: can be very close by)", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::7e145a8364": { + "source": "sampler._rvs[\"psi\"],", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic::8a0be56ddc": { + "source": "sampler._rvs[\"distance\"][indx_guess]/dmax\\", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic::9bbe221907": { + "source": "sampler._rvs[\"declination\"], sampler._rvs[\"right_ascension\"] = sampler._rvs[(\"declination\", \"right_ascension\")]", + "verdict": "PER_ROW", + "why": "In-place coordinate transform, row for row." + }, + "bin/integrate_likelihood_extrinsic::9cc74af37f": { + "source": "sampler._rvs[\"inclination\"][indx_guess]/(numpy.pi),\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::9dc941f7cf": { + "source": "P.psi = sampler._rvs[\"psi\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic::dc8bf3e149": { + "source": "print( \" lnL at start :\", lnLstart, \" [note can be lower than peak because of time offset]: best reported by ILE (including weights) is \", numpy.log(numpy.max(sampler._rvs[\"integrand\"])))", + "verdict": "BENIGN", + "why": "Printed diagnostic of the best retained sample. Explicitly labelled as what ILE reports including weights; not an input to any result." + }, + "bin/integrate_likelihood_extrinsic::e0bbb0348e": { + "source": "P.phiref = sampler._rvs[\"phi_orb\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:_snapshot_pass_state:02594db9f0": { + "source": "rvs=(dict(sampler._rvs) if rvs is None else rvs),", + "verdict": "PER_ROW", + "why": "_snapshot_pass_state takes a dict copy of whatever rows are present so a rejected warm pass can be undone. It makes no claim about their statistics, and it snapshots the fair-draw MARKER alongside them so the restored record and the marker describing it cannot disagree." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:011d296b48": { + "source": "_rep_rvs = [sampler._rvs]", + "verdict": "PER_ROW", + "why": "Collects each replica's record for pooling. _pool_replica_rvs is told already_resampled=bool(opts.fairdraw_extrinsic_output) and forces flat within-block weights on that path, so the resampling is accounted for THERE rather than here." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:07f46c212f": { + "source": "_lnv = (np.asarray(sampler.identity_convert(sampler._rvs[_lnkey]), dtype=float).ravel()", + "verdict": "BENIGN", + "why": "The DELIBERATE no-reserve fallback for a sampler that keeps none: reads _rvs so the feature degrades to its previous behaviour rather than to no seed at all. The rank hazard is still handled -- build_warm_seed puffs the result to full rank -- so what remains is only 'fewer points', which cannot be improved without a reserve." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:0ab512f38a": { + "source": "_warm_lnZ = _lnZ_of_rvs(sampler._rvs, already_pooled=False)", + "verdict": "BROKEN", + "why": "PR #79's CROSS-SOURCE FALLBACK: fires only when the cold and warm passes produced different reading sources (one had a reserve, the other did not), and then re-reads BOTH sides from the fair-draw record. That is self-consistent, which is what #79 claims for it, but it is not unbiased: the two passes sit at different n_eff, so the log(n/n_eff) artifact does not cancel and this branch is back in the regime measured at +3.48 nats / 100% rejection at the 0.5 default. A known, documented, BOUNDED residual -- not a defect anyone introduced. Closing it needs a retained-set reading on both sides, i.e. a reserve for the samplers that keep none. Follow-up, not a regression." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:1240e69c24": { + "source": "len(numpy.atleast_1d(list(sampler._rvs.values())[0])) if sampler._rvs else 0,", + "verdict": "FIXED", + "why": "Kish of the pooled record is only used when the export is NOT fair-drawn. When it is, _pool_replica_rvs has flattened each block, and the Kish of piecewise-constant weights is just the row count (5K at the default --fairdraw-extrinsic-output-n-max 5). The ILE now computes the same quantity one level up -- (sum Z_k)^2 / sum(Z_k^2/neff_k), Kish over the BLOCKS -- which reduces to sum(neff_k) when replicas agree and falls below it when they do not. The row count beside it is a deliberate report of the export size." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:1331dd70ea": { + "source": "len(np.asarray(sampler.identity_convert(sampler._rvs['log_integrand'])).ravel())", + "verdict": "BENIGN", + "why": "Reports how many rows the fair draw left, for the log line that contrasts it with the retained count. Reading the resample's size is the POINT here." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:16e8b48c86": { + "source": "(sampler._rvs[\"phi_orb\"][indx_guess]/(2*numpy.pi)), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:1dbc4dbe43": { + "source": "sampler._rvs[\"right_ascension\"][indx_guess]/(2*numpy.pi) , \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:25d9742c4d": { + "source": "_rep_rvs.append(sampler._rvs)", + "verdict": "PER_ROW", + "why": "Collects each replica's record for pooling. _pool_replica_rvs is told already_resampled=bool(opts.fairdraw_extrinsic_output) and forces flat within-block weights on that path, so the resampling is accounted for THERE rather than here." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:29d2d2ebb6": { + "source": "P.dist = sampler._rvs[\"distance\"][indx_guess]*1e6*lal.PC_SI", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:31db586050": { + "source": "P.incl = sampler._rvs[\"inclination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:32c64bcd73": { + "source": "_lnv = np.asarray(sampler.identity_convert(sampler._rvs[_lnkey]), dtype=float).ravel() if _lnkey else np.array([])", + "verdict": "BENIGN", + "why": "The DELIBERATE no-reserve fallback for a sampler that keeps none: reads _rvs so the feature degrades to its previous behaviour rather than to no seed at all. The rank hazard is still handled -- build_warm_seed puffs the result to full rank -- so what remains is only 'fewer points', which cannot be improved without a reserve." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:3600397a2b": { + "source": "if \"integrand\" in sampler._rvs:", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:36e58e97fb": { + "source": "if 'log_integrand' in sampler._rvs else '?'))", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:45fa8d2146": { + "source": "sampler._rvs[\"psi\"][indx_guess]/numpy.pi,\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:4dd03bebc6": { + "source": "P.phi = sampler._rvs[\"right_ascension\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:65d3d3b519": { + "source": "_rvs = sampler._rvs", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:6658553446": { + "source": "P.theta = sampler._rvs[\"declination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:675b92662a": { + "source": "if (_lnkey is not None and all(p in sampler._rvs for p in sampler.params_ordered))", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:6b4e7670c7": { + "source": "dL_samp = np.asarray(identity_convert(sampler._rvs[\"distance\"]), float)", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:6c4b180497": { + "source": "(sampler._rvs[\"declination\"][indx_guess]/numpy.pi), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:7226654481": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"integrand\"]) # start search near maximum-likelihood point. (WARNING: can be very close by)", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:7a4e161eb0": { + "source": "_cold_rvs = dict(sampler._rvs)", + "verdict": "PER_ROW", + "why": "Snapshots the cold record so the reject path can restore it. A dict copy of whatever rows exist; makes no claim about their statistics." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:8a0be56ddc": { + "source": "sampler._rvs[\"distance\"][indx_guess]/dmax\\", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:9cc74af37f": { + "source": "sampler._rvs[\"inclination\"][indx_guess]/(numpy.pi),\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:9dc941f7cf": { + "source": "P.psi = sampler._rvs[\"psi\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:a1d5088a59": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"log_integrand\"])", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:a6ea5209f8": { + "source": "_cols = (np.vstack([np.asarray(sampler.identity_convert(sampler._rvs[p]), dtype=float).ravel()", + "verdict": "BENIGN", + "why": "The DELIBERATE no-reserve fallback for a sampler that keeps none: reads _rvs so the feature degrades to its previous behaviour rather than to no seed at all. The rank hazard is still handled -- build_warm_seed puffs the result to full rank -- so what remains is only 'fewer points', which cannot be improved without a reserve." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:acdd1e28bd": { + "source": "_neff_pooled = _kish_neff_of_rvs(sampler._rvs)", + "verdict": "FIXED", + "why": "Kish of the pooled record is only used when the export is NOT fair-drawn. When it is, _pool_replica_rvs has flattened each block, and the Kish of piecewise-constant weights is just the row count (5K at the default --fairdraw-extrinsic-output-n-max 5). The ILE now computes the same quantity one level up -- (sum Z_k)^2 / sum(Z_k^2/neff_k), Kish over the BLOCKS -- which reduces to sum(neff_k) when replicas agree and falls below it when they do not. The row count beside it is a deliberate report of the export size." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:b8c45b4f30": { + "source": "samples = copy.deepcopy(sampler._rvs) # deep copy: avoid modifying structures and having side effect on integrator, which loops over keys Expensive!", + "verdict": "BENIGN", + "why": "THE export itself. Under --fairdraw-extrinsic-output the fair draw is precisely what these rows are supposed to be, so the resample is the correct input here rather than a hazard." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:c717be4455": { + "source": "dL = np.array(sampler._rvs[\"distance\"])", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:cfe2518e26": { + "source": "_lnkey = 'log_integrand' if 'log_integrand' in sampler._rvs else ('integrand' if 'integrand' in sampler._rvs else None)", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:d2514d9663": { + "source": "_warm_lnZ, _warm_src = _lnZ_of_reserve_or_rvs(sampler, sampler._rvs)", + "verdict": "FIXED", + "why": "PR #79, re-landed as #86. sampler._rvs is passed as the FALLBACK argument only: the helper prefers the retained reserve via lnZ_from_reserve, and the gate refuses to compare across sources (_cold_src != _warm_src forces BOTH back to the fair-draw reading, which is at least self-consistent). Measured before #79: two passes with identical true lnZ at n_eff 1.8 vs 53 produced a +3.48 nat gap and rejected the good warm pass 100% of the time at the 0.5 default." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:d948a54bfb": { + "source": "rvs = sampler._rvs", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:dc8bf3e149": { + "source": "print( \" lnL at start :\", lnLstart, \" [note can be lower than peak because of time offset]: best reported by ILE (including weights) is \", numpy.log(numpy.max(sampler._rvs[\"integrand\"])))", + "verdict": "BENIGN", + "why": "Printed diagnostic of the best retained sample. Explicitly labelled as what ILE reports including weights; not an input to any result." + }, + "bin/integrate_likelihood_extrinsic_batchmode:analyze_event:e0bbb0348e": { + "source": "P.phiref = sampler._rvs[\"phi_orb\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:16e8b48c86": { + "source": "(sampler._rvs[\"phi_orb\"][indx_guess]/(2*numpy.pi)), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:1dbc4dbe43": { + "source": "sampler._rvs[\"right_ascension\"][indx_guess]/(2*numpy.pi) , \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:29d2d2ebb6": { + "source": "P.dist = sampler._rvs[\"distance\"][indx_guess]*1e6*lal.PC_SI", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:31db586050": { + "source": "P.incl = sampler._rvs[\"inclination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:3600397a2b": { + "source": "if \"integrand\" in sampler._rvs:", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:3a7fb52dae": { + "source": "for key in sampler._rvs.keys():", + "verdict": "BENIGN", + "why": "THE export itself. Under --fairdraw-extrinsic-output the fair draw is precisely what these rows are supposed to be, so the resample is the correct input here rather than a hazard." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:45fa8d2146": { + "source": "sampler._rvs[\"psi\"][indx_guess]/numpy.pi,\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:4dd03bebc6": { + "source": "P.phi = sampler._rvs[\"right_ascension\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:6658553446": { + "source": "P.theta = sampler._rvs[\"declination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:6c4b180497": { + "source": "(sampler._rvs[\"declination\"][indx_guess]/numpy.pi), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:7226654481": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"integrand\"]) # start search near maximum-likelihood point. (WARNING: can be very close by)", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:8a0be56ddc": { + "source": "sampler._rvs[\"distance\"][indx_guess]/dmax\\", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:9cc74af37f": { + "source": "sampler._rvs[\"inclination\"][indx_guess]/(numpy.pi),\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:9dc941f7cf": { + "source": "P.psi = sampler._rvs[\"psi\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:a1d5088a59": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"log_integrand\"])", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:b8c45b4f30": { + "source": "samples = copy.deepcopy(sampler._rvs) # deep copy: avoid modifying structures and having side effect on integrator, which loops over keys Expensive!", + "verdict": "BENIGN", + "why": "THE export itself. Under --fairdraw-extrinsic-output the fair draw is precisely what these rows are supposed to be, so the resample is the correct input here rather than a hazard." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:dc8bf3e149": { + "source": "print( \" lnL at start :\", lnLstart, \" [note can be lower than peak because of time offset]: best reported by ILE (including weights) is \", numpy.log(numpy.max(sampler._rvs[\"integrand\"])))", + "verdict": "BENIGN", + "why": "Printed diagnostic of the best retained sample. Explicitly labelled as what ILE reports including weights; not an input to any result." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event:e0bbb0348e": { + "source": "P.phiref = sampler._rvs[\"phi_orb\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:16e8b48c86": { + "source": "(sampler._rvs[\"phi_orb\"][indx_guess]/(2*numpy.pi)), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:1dbc4dbe43": { + "source": "sampler._rvs[\"right_ascension\"][indx_guess]/(2*numpy.pi) , \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:29d2d2ebb6": { + "source": "P.dist = sampler._rvs[\"distance\"][indx_guess]*1e6*lal.PC_SI", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:31db586050": { + "source": "P.incl = sampler._rvs[\"inclination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:3600397a2b": { + "source": "if \"integrand\" in sampler._rvs:", + "verdict": "PER_ROW", + "why": "Key-presence / column reference, not a population statistic." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:3a7fb52dae": { + "source": "for key in sampler._rvs.keys():", + "verdict": "BENIGN", + "why": "THE export itself. Under --fairdraw-extrinsic-output the fair draw is precisely what these rows are supposed to be, so the resample is the correct input here rather than a hazard." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:45fa8d2146": { + "source": "sampler._rvs[\"psi\"][indx_guess]/numpy.pi,\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:4dd03bebc6": { + "source": "P.phi = sampler._rvs[\"right_ascension\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:6658553446": { + "source": "P.theta = sampler._rvs[\"declination\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:6c4b180497": { + "source": "(sampler._rvs[\"declination\"][indx_guess]/numpy.pi), \\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:7226654481": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"integrand\"]) # start search near maximum-likelihood point. (WARNING: can be very close by)", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:8a0be56ddc": { + "source": "sampler._rvs[\"distance\"][indx_guess]/dmax\\", + "verdict": "PER_ROW", + "why": "Binds the record, or reads one COLUMN of it, for the distance exporters. The resample changes which rows are present, never the value a row carries, and the weighting decision is made separately by ln_weights_for_posterior (.dgrid) or by forcing the all-fresh path (.dslice). Correct on either row set." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:9cc74af37f": { + "source": "sampler._rvs[\"inclination\"][indx_guess]/(numpy.pi),\\", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:9dc941f7cf": { + "source": "P.psi = sampler._rvs[\"psi\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:a1d5088a59": { + "source": "indx_guess = numpy.argmax(sampler._rvs[\"log_integrand\"])", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:b8c45b4f30": { + "source": "samples = copy.deepcopy(sampler._rvs) # deep copy: avoid modifying structures and having side effect on integrator, which loops over keys Expensive!", + "verdict": "BENIGN", + "why": "THE export itself. Under --fairdraw-extrinsic-output the fair draw is precisely what these rows are supposed to be, so the resample is the correct input here rather than a hazard." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:dc8bf3e149": { + "source": "print( \" lnL at start :\", lnLstart, \" [note can be lower than peak because of time offset]: best reported by ILE (including weights) is \", numpy.log(numpy.max(sampler._rvs[\"integrand\"])))", + "verdict": "BENIGN", + "why": "Printed diagnostic of the best retained sample. Explicitly labelled as what ILE reports including weights; not an input to any result." + }, + "bin/integrate_likelihood_extrinsic_batchmode_lisa:analyze_event_LISA:e0bbb0348e": { + "source": "P.phiref = sampler._rvs[\"phi_orb\"][indx_guess]", + "verdict": "BENIGN", + "why": "MAP-seed read: argmax over the record, then the SAME row's coordinates. The fair draw picks rows proportional to weight, so the retained argmax is a genuinely-drawn high-likelihood point; it seeds a local search and a printed diagnostic, and no downstream number is a statistic of it. Degraded (it may not be the global MAP), not wrong." + }, + "bin/util_ConstructEOSPosterior.py::0a939664b9": { + "source": "samples = sampler._rvs", + "verdict": "NO_FAIRDRAW", + "why": "Verified by grep: neither util_ConstructIntrinsicPosterior_GenericCoordinates nor util_ConstructEOSPosterior passes igrand_fairdraw_samples, so integrate() never runs the rebind and _rvs is still the retained set. (The known CIP posterior-export defect was a different mechanism -- replace=False successive sampling, PR #44 lineage -- not this one.)" + }, + "bin/util_ConstructEOSPosterior.py::2a70a911c5": { + "source": "weights_scaled = np.exp(dat_logL - lnLmax)*sampler._rvs[\"joint_prior\"]/sampler._rvs[\"joint_s_prior\"]", + "verdict": "NO_FAIRDRAW", + "why": "Verified by grep: neither util_ConstructIntrinsicPosterior_GenericCoordinates nor util_ConstructEOSPosterior passes igrand_fairdraw_samples, so integrate() never runs the rebind and _rvs is still the retained set. (The known CIP posterior-export defect was a different mechanism -- replace=False successive sampling, PR #44 lineage -- not this one.)" + }, + "bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py::0a939664b9": { + "source": "samples = sampler._rvs", + "verdict": "NO_FAIRDRAW", + "why": "Verified by grep: neither util_ConstructIntrinsicPosterior_GenericCoordinates nor util_ConstructEOSPosterior passes igrand_fairdraw_samples, so integrate() never runs the rebind and _rvs is still the retained set. (The known CIP posterior-export defect was a different mechanism -- replace=False successive sampling, PR #44 lineage -- not this one.)" + }, + "bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py::2a70a911c5": { + "source": "weights_scaled = np.exp(dat_logL - lnLmax)*sampler._rvs[\"joint_prior\"]/sampler._rvs[\"joint_s_prior\"]", + "verdict": "NO_FAIRDRAW", + "why": "Verified by grep: neither util_ConstructIntrinsicPosterior_GenericCoordinates nor util_ConstructEOSPosterior passes igrand_fairdraw_samples, so integrate() never runs the rebind and _rvs is still the retained set. (The known CIP posterior-export defect was a different mechanism -- replace=False successive sampling, PR #44 lineage -- not this one.)" + } +} diff --git a/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/verify_skew.py b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/verify_skew.py new file mode 100644 index 000000000..7efc18a1e --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/expensive_before_merging/integrators/verify_skew.py @@ -0,0 +1,143 @@ +"""Numerically confirm (or refute) the three post-rebind claims, using the ILE's OWN +helpers rather than a reimplementation of them.""" +import importlib.util +import os +import sys + +import numpy as np + +CODE = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "..")) +sys.path.insert(0, CODE) + +# The ILE is a script, not a module: load the two helpers by exec'ing just their source. +src = open(os.path.join(CODE, "bin", "integrate_likelihood_extrinsic_batchmode")).read() +ns = {"numpy": np, "np": np} +start = src.index("def ln_weights_from_rvs") +end = src.index("def _warm_seed_geometry") +body = src[start:end] +# _rvs_lnL_convention is defined elsewhere; stub it to "as passed" +ns["_rvs_lnL_convention"] = lambda x=None: bool(x) +exec(compile(body, "ile_helpers", "exec"), ns) +_lnZ_of_rvs = ns["_lnZ_of_rvs"] +_kish_neff_of_rvs = ns["_kish_neff_of_rvs"] + +rng = np.random.default_rng(20260813) + + +def make_pass(n, spread): + """A retained set whose log-weights have the given spread (nats). Large spread = + the collapsed, high-SNR regime; small spread = a healthy pass.""" + lnL = rng.normal(0.0, spread, size=n) + return { + "log_integrand": lnL, + "log_joint_prior": np.zeros(n), + "log_joint_s_prior": np.zeros(n), + "x": rng.normal(size=n), + } + + +def fair_draw(rvs, n_extr, rng): + """The rebind, exactly as integrate_log performs it: n_extr rows WITH REPLACEMENT, + proportional to weight.""" + lw = rvs["log_integrand"] + rvs["log_joint_prior"] - rvs["log_joint_s_prior"] + lw = lw - np.max(lw) + w = np.exp(lw) + w = w / w.sum() + idx = rng.choice(np.arange(len(w)), size=n_extr, replace=True, p=w) + return {k: np.asarray(v)[idx] for k, v in rvs.items()} + + +def kish(rvs): + lw = rvs["log_integrand"] + rvs["log_joint_prior"] - rvs["log_joint_s_prior"] + lw = lw - lw.max() + w = np.exp(lw) + return w.sum() ** 2 / (w ** 2).sum() + + +print("=" * 78) +print("CLAIM 1: _lnZ_of_rvs(already_pooled=False) on a fair-drawn record is HIGH by") +print(" about log(n_retained / n_eff).") +print("=" * 78) +print("{:>8} {:>8} {:>10} {:>10} {:>10} {:>10} {:>10}".format( + "n", "spread", "n_eff", "n_extr", "lnZ_true", "lnZ_draw", "excess")) +for spread in (1.0, 3.0, 6.0, 9.0): + n = 4000 + rvs = make_pass(n, spread) + ne = kish(rvs) + lnZ_true = _lnZ_of_rvs(rvs, already_pooled=False) + n_extr = max(1, int(min(300, 1.5 * ne))) + excess = [] + for _ in range(200): + fd = fair_draw(rvs, n_extr, rng) + excess.append(_lnZ_of_rvs(fd, already_pooled=False) - lnZ_true) + print("{:>8} {:>8.1f} {:>10.1f} {:>10} {:>10.3f} {:>10.3f} {:>+10.3f} (log(n/n_eff)={:+.3f})".format( + n, spread, ne, n_extr, lnZ_true, lnZ_true + np.mean(excess), + float(np.mean(excess)), float(np.log(n / ne)))) + +print() +print("=" * 78) +print("CLAIM 2: the gate's cold-vs-warm difference does NOT cancel, because the two") +print(" passes sit at very different n_eff. Simulates the measured regime:") +print(" collapsed cold (n_eff~1) vs healthy warm (n_eff~20), SAME true lnZ.") +print("=" * 78) +cold = make_pass(1000, 9.0) +warm = make_pass(1000, 2.0) +# force identical true evidence so any gap the gate sees is pure artifact +lnZ_c = _lnZ_of_rvs(cold, already_pooled=False) +lnZ_w = _lnZ_of_rvs(warm, already_pooled=False) +warm["log_integrand"] = warm["log_integrand"] + (lnZ_c - lnZ_w) +print(" true lnZ cold {:.4f} warm {:.4f} (difference by construction {:+.4f})".format( + _lnZ_of_rvs(cold, already_pooled=False), _lnZ_of_rvs(warm, already_pooled=False), + _lnZ_of_rvs(cold, already_pooled=False) - _lnZ_of_rvs(warm, already_pooled=False))) +print(" n_eff cold {:.2f} warm {:.2f}".format(kish(cold), kish(warm))) +gaps = [] +for _ in range(400): + c = fair_draw(cold, max(1, int(1.5 * kish(cold))), rng) + w = fair_draw(warm, max(1, int(1.5 * kish(warm))), rng) + gaps.append(_lnZ_of_rvs(c, already_pooled=False) - _lnZ_of_rvs(w, already_pooled=False)) +gaps = np.array(gaps) +print(" gate reads cold-warm = {:+.3f} nats (median), IQR [{:+.3f}, {:+.3f}]".format( + float(np.median(gaps)), float(np.percentile(gaps, 25)), float(np.percentile(gaps, 75)))) +for thr in (0.5, 2.0, 5.0): + print(" at --sampler-l0-rescue-reject-dlnZ {:.1f}: rejects the (equally good) warm pass" + " {:.0f}% of the time".format(thr, 100.0 * np.mean(gaps > thr))) + +print() +print("=" * 78) +print("CLAIM 3: _kish_neff_of_rvs of a fair-drawn record tracks the ROW COUNT, not the") +print(" pass's true n_eff.") +print("=" * 78) +print("{:>8} {:>12} {:>10} {:>14}".format("spread", "true n_eff", "n_extr", "kish(fairdraw)")) +for spread in (1.0, 3.0, 6.0, 9.0): + rvs = make_pass(4000, spread) + ne = kish(rvs) + n_extr = max(1, int(min(300, 1.5 * ne))) + vals = [_kish_neff_of_rvs(fair_draw(rvs, n_extr, rng)) for _ in range(100)] + print("{:>8.1f} {:>12.1f} {:>10} {:>14.1f}".format(spread, ne, n_extr, float(np.mean(vals)))) + +print() +print("=" * 78) +print("CLAIM 4: re-weighting an already fair-drawn record (the .dgrid / .dslice path)") +print(" applies w twice. Compare the weighted mean of a coordinate.") +print("=" * 78) +n = 4000 +lnL = rng.normal(0.0, 4.0, size=n) +x = lnL * 0.5 + rng.normal(size=n) * 0.5 # x correlated with weight +rvs = {"log_integrand": lnL, "log_joint_prior": np.zeros(n), + "log_joint_s_prior": np.zeros(n), "x": x} +w = np.exp(lnL - lnL.max()); w /= w.sum() +truth = float(np.sum(w * x)) +ne = kish(rvs) +n_extr = max(1, int(min(300, 1.5 * ne))) +naive, correct = [], [] +for _ in range(300): + fd = fair_draw(rvs, n_extr, rng) + lw = fd["log_integrand"] + fd["log_joint_prior"] - fd["log_joint_s_prior"] + ww = np.exp(lw - lw.max()); ww /= ww.sum() + naive.append(float(np.sum(ww * fd["x"]))) # what the exporter does + correct.append(float(np.mean(fd["x"]))) # a fair draw is already equal-weight +print(" posterior mean of x, truth {:+.4f}".format(truth)) +print(" fair draw, UNWEIGHTED (correct) {:+.4f}".format(float(np.mean(correct)))) +print(" fair draw, RE-WEIGHTED by w (the exporter) {:+.4f}".format(float(np.mean(naive)))) +print(" -> re-weighting shifts the estimate by {:+.4f} ({:.0f}% of the truth)".format( + float(np.mean(naive)) - truth, 100 * abs(float(np.mean(naive)) - truth) / abs(truth))) diff --git a/MonteCarloMarginalizeCode/Code/test/test_asimov_bootstrap_source.py b/MonteCarloMarginalizeCode/Code/test/test_asimov_bootstrap_source.py new file mode 100644 index 000000000..d1f7d4b55 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_asimov_bootstrap_source.py @@ -0,0 +1,204 @@ +#!/usr/bin/env python3 +""" +Unit tests for explicit bootstrap-source selection in RIFT.asimov.rift. + +``scheduler: bootstrap file:`` lets a blueprint name the PESummary metafile to +bootstrap from directly, instead of routing it through an asimov dependency. +The dependency route only works for pipelines whose ``collect_assets()`` +returns ``samples`` as a single path to a PESummary metafile; the bilby +pipeline returns a *list* of raw bilby result files, which used to fail +silently and leave the run with no bootstrap at all. + +These tests drive ``Rift._resolve_bootstrap_file`` and ``Rift._dataset_label`` +against a stub production, so they need no asimov project on disk. +""" + +import os + +import numpy as np +import pytest + +h5py = pytest.importorskip("h5py") +# asimov registers RIFT.asimov.rift:Rift as an entry point and loads it while +# building known_pipelines; importing asimov first keeps that from re-entering +# our own in-flight import of the same module. +pytest.importorskip("asimov") +rift_asimov = pytest.importorskip("RIFT.asimov.rift") + +Rift = rift_asimov.Rift +PipelineException = rift_asimov.PipelineException + + +class _StubEvent: + def __init__(self, name): + self.name = name + + +class _StubProduction: + def __init__(self, meta, event="S250202cu", name="rift-SEOBNRv5PHM"): + self.meta = meta + self.name = name + self.event = _StubEvent(event) + + +class _StubRift: + """Bind the methods under test to a stub, bypassing Rift.__init__.""" + + _PESUMMARY_RESERVED = Rift._PESUMMARY_RESERVED + _resolve_bootstrap_file = Rift._resolve_bootstrap_file + _dataset_label = Rift._dataset_label + + def __init__(self, meta, event="S250202cu"): + self.production = _StubProduction(meta, event=event) + self.logger = _NullLogger() + + +class _NullLogger: + def info(self, *args, **kwargs): + pass + + def warning(self, *args, **kwargs): + pass + + def error(self, *args, **kwargs): + pass + + +def _write_metafile(path, labels, n_samples=32): + """A minimal PESummary-shaped metafile.""" + columns = ("mass_1", "mass_2", "chirp_mass", "luminosity_distance", + "geocent_time", "iota", "psi", "ra", "dec", "phase", + "spin_1x", "spin_1y", "spin_1z", "spin_2x", "spin_2y", "spin_2z") + dtype = np.dtype([(name, float) for name in columns]) + samples = np.zeros(n_samples, dtype=dtype) + with h5py.File(path, "w") as handle: + handle.create_group("version") + handle.create_group("history") + for label in labels: + handle.create_group(label).create_dataset("posterior_samples", data=samples) + return str(path) + + +def _write_raw_bilby(path, n_samples=32): + """A raw bilby result file: samples at the root, no analysis label.""" + with h5py.File(path, "w") as handle: + handle.create_dataset("label", data=b"bilby-SEOBNRv5PHM") + posterior = handle.create_group("posterior") + for name in ("mass_1", "mass_2", "chirp_mass"): + posterior.create_dataset(name, data=np.zeros(n_samples)) + return str(path) + + +# --- resolution ----------------------------------------------------------- + +def test_absent_setting_returns_none(): + assert _StubRift({"scheduler": {}})._resolve_bootstrap_file() is None + assert _StubRift({})._resolve_bootstrap_file() is None + + +def test_explicit_path_is_returned(tmp_path): + target = _write_metafile(tmp_path / "posterior_samples.h5", ["bilby-SEOBNRv5PHM"]) + stub = _StubRift({"scheduler": {"bootstrap file": target}}) + assert stub._resolve_bootstrap_file() == target + + +@pytest.mark.parametrize("token", ["{event}", ""]) +def test_event_substitution(tmp_path, token): + event_dir = tmp_path / "S250202cu" + event_dir.mkdir() + target = _write_metafile(event_dir / "posterior_samples.h5", ["bilby-SEOBNRv5PHM"]) + template = os.path.join(str(tmp_path), token, "posterior_samples.h5") + stub = _StubRift({"scheduler": {"bootstrap file": template}}, event="S250202cu") + assert stub._resolve_bootstrap_file() == target + + +def test_analysis_substitution(tmp_path): + analysis_dir = tmp_path / "rift-SEOBNRv5PHM" + analysis_dir.mkdir() + target = _write_metafile(analysis_dir / "posterior_samples.h5", ["x"]) + template = os.path.join(str(tmp_path), "{analysis}", "posterior_samples.h5") + assert _StubRift({"scheduler": {"bootstrap file": template}})._resolve_bootstrap_file() == target + + +def test_unique_glob_is_accepted(tmp_path): + target = _write_metafile(tmp_path / "posterior_samples.h5", ["x"]) + pattern = os.path.join(str(tmp_path), "*.h5") + assert _StubRift({"scheduler": {"bootstrap file": pattern}})._resolve_bootstrap_file() == target + + +def test_ambiguous_glob_raises(tmp_path): + _write_metafile(tmp_path / "a.h5", ["x"]) + _write_metafile(tmp_path / "b.h5", ["x"]) + stub = _StubRift({"scheduler": {"bootstrap file": os.path.join(str(tmp_path), "*.h5")}}) + with pytest.raises(PipelineException): + stub._resolve_bootstrap_file() + + +def test_glob_matching_nothing_raises(tmp_path): + stub = _StubRift({"scheduler": {"bootstrap file": os.path.join(str(tmp_path), "*.h5")}}) + with pytest.raises(PipelineException): + stub._resolve_bootstrap_file() + + +def test_missing_file_raises_rather_than_falling_back(tmp_path): + """An explicit request that cannot be honoured must never be silent.""" + stub = _StubRift({"scheduler": {"bootstrap file": str(tmp_path / "nope.h5")}}) + with pytest.raises(PipelineException): + stub._resolve_bootstrap_file() + + +# --- label selection ------------------------------------------------------ + +def test_single_label_is_auto_derived(tmp_path): + target = _write_metafile(tmp_path / "m.h5", ["bilby-SEOBNRv5PHM"]) + stub = _StubRift({"scheduler": {}}) + assert stub._dataset_label(target) == "bilby-SEOBNRv5PHM" + + +def test_explicit_dataset_is_honoured(tmp_path): + target = _write_metafile(tmp_path / "m.h5", ["a", "b"]) + stub = _StubRift({"scheduler": {}, "dataset": "b"}) + assert stub._dataset_label(target) == "b" + + +def test_explicit_dataset_does_not_open_the_file(): + """ + Backwards compatibility: the previous code only opened the metafile when + `dataset` was absent. A ledger that pins a dataset must keep building even + if the source file has since moved, so long as the grid already exists. + """ + stub = _StubRift({"scheduler": {}, "dataset": "pinned"}) + assert stub._dataset_label("/nonexistent/never/opened.h5") == "pinned" + + +def test_ambiguous_labels_raise_without_explicit_dataset(tmp_path): + target = _write_metafile(tmp_path / "m.h5", ["a", "b"]) + with pytest.raises(PipelineException): + _StubRift({"scheduler": {}})._dataset_label(target) + + +def test_metafile_without_version_and_history_still_works(tmp_path): + """ + The previous implementation used list.remove('version'), which raises + ValueError when those groups are absent - swallowed by a bare except. + """ + path = str(tmp_path / "m.h5") + columns = np.dtype([("mass_1", float)]) + with h5py.File(path, "w") as handle: + handle.create_group("only-label").create_dataset( + "posterior_samples", data=np.zeros(4, dtype=columns) + ) + assert _StubRift({"scheduler": {}})._dataset_label(path) == "only-label" + + +def test_raw_bilby_result_is_rejected_with_a_useful_message(tmp_path): + """RIFT consumes PESummary metafiles, not raw bilby result files.""" + target = _write_raw_bilby(tmp_path / "raw_result.hdf5") + with pytest.raises(PipelineException) as excinfo: + _StubRift({"scheduler": {}})._dataset_label(target) + assert "raw bilby" in str(excinfo.value).lower() + assert "pesummary" in str(excinfo.value).lower() + + +if __name__ == "__main__": + raise SystemExit(pytest.main([__file__, "-v"])) diff --git a/MonteCarloMarginalizeCode/Code/test/test_cip_priors.py b/MonteCarloMarginalizeCode/Code/test/test_cip_priors.py new file mode 100644 index 000000000..03c383db7 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_cip_priors.py @@ -0,0 +1,765 @@ +#!/usr/bin/env python3 +""" +Unit tests for the prior densities defined inside +``bin/util_ConstructIntrinsicPosterior_GenericCoordinates.py`` (CIP). + +Why this file looks the way it does +----------------------------------- +CIP is a script, not an importable module: importing it parses argv, reads the +input grid and runs several thousand lines of module-level setup. So the +priors -- which are otherwise ordinary pure functions of one array -- have +never been reachable from a test, and have never had one. + +That is how ``--eccentricity-prior log_uniform`` shipped in 0.0.17.12 calling +``np.ln``, which does not exist in numpy. The option raised AttributeError the +first time the prior was evaluated, and its normalization was independently +wrong: it used ``log(ECC_MAX-ECC_MIN)``, the *uniform* prior's normalization, +where a density uniform in ln(e) needs ``log(ECC_MAX/ECC_MIN)``. + +Rather than transcribe the priors here -- which lets the test silently drift +away from the shipped code, the usual failure mode of a copied reference +implementation -- this module extracts the actual ``def`` blocks from the CIP +source with ``ast`` and execs them in a namespace holding numpy and the handful +of module-level constants they close over. The functions under test are +therefore byte-identical to the ones CIP runs. + +Three layers of coverage: + +``test_prior_evaluates`` + Every extracted prior must evaluate on a valid array and return finite, + non-negative, correctly-shaped values. This is the cheap generic guard: it + catches the ``np.ln`` class of defect (a name that does not exist) for any + prior, including ones added later, without anyone having to write a new + test. + +``test_prior_is_normalized`` + The subset whose docstring or comment claims a normalized density is + integrated numerically over its stated support and must come to 1. This is + what catches a wrong normalization constant, which evaluates perfectly + happily and silently reweights a posterior. Priors documented in-source as + unnormalized are listed in UNNORMALIZED below and deliberately excluded. + +the ``_eccentricity_setup`` tests + A correct density is worth nothing if the option does not install it for the + coordinate the run actually samples. These execute CIP's own + ``--eccentricity-prior`` block against its own default prior_map / + prior_range_map entries, and check every eccentricity coordinate -- + including eccentricity_squared, which is what an eccentric pseudo_pipe run + samples in iteration 0. They also run the eccentricity_ln coordinate at + CIP's *shipped* ``--ecc-min`` default, read out of the argparse call rather + than assumed here: that coordinate is logarithmic under every prior, so the + default of 0.0 gives it a [-inf, ...] range and a prior that divides by + zero, independently of --eccentricity-prior. + +``test_eccentricity_prior_option_rejects_unknown_values`` + The option value is forwarded verbatim from pseudo_pipe to CIP and only the + exact string 'log_uniform' is branched on, so both parsers must reject + anything else rather than fall through to the uniform prior. +""" + +import ast +import os +import re +import sys +import types + +import numpy as np +import pytest + +scipy_stats = pytest.importorskip("scipy.stats") +from scipy import integrate + +CIP_SCRIPT = os.path.join( + os.path.dirname(os.path.abspath(__file__)), + "..", + "bin", + "util_ConstructIntrinsicPosterior_GenericCoordinates.py", +) + +# The pipeline driver that forwards --eccentricity-prior to CIP. Only its argparse +# spec is inspected (by ast, like everything else here); the script is never imported. +PSEUDO_PIPE_SCRIPT = os.path.join( + os.path.dirname(os.path.abspath(__file__)), + "..", + "bin", + "util_RIFT_pseudo_pipe.py", +) + +# Values the priors close over. Chosen to be ordinary production-shaped +# numbers rather than 1.0 everywhere, so a normalization that happens to be +# right only for the unit interval does not pass by accident. +CHI_MAX = 0.9 +ECC_MIN = 0.001 # CIP's own auto-correction when --ecc-min is 0 +ECC_MAX = 0.4 +LAMBDA_MIN = 0.0 +LAMBDA_MAX = 4000.0 +LAMBDA_SMALL_MAX = 2000.0 +MC_MIN = 5.0 +MC_MAX = 60.0 + +# CIP sets p_Rbar = lalsimutils.p_R. Read out of the lalsimutils SOURCE rather +# than imported: importing lalsimutils pulls in LAL, whose default error handler +# calls abort(), which turns any unrelated numerical complaint raised inside +# scipy.integrate.quad below into a hard core dump instead of a test failure. +# These priors are pure numpy, so the test stays free of that whole stack. +LALSIMUTILS_SOURCE = os.path.join( + os.path.dirname(os.path.abspath(__file__)), "..", "RIFT", "lalsimutils.py") + + +def _p_rbar(default=0.25): + """lalsimutils.p_R, parsed from source; `default` matches the shipped value.""" + try: + with open(LALSIMUTILS_SOURCE) as handle: + for line in handle: + match = re.match(r"^p_R\s*=\s*([0-9.eE+-]+)\s*(#.*)?$", line) + if match: + return float(match.group(1)) + except OSError: + pass + return default + + +# Functions with 'prior' in the name that are NOT one-dimensional densities of +# a parameter, and so are not subject to either test below. +NOT_A_DENSITY = { + # a CDF helper: takes a scalar eta_min, not an array of samples + "unscaled_eta_prior_cdf", + # operate on a whole parameter vector during the fit, not on one coordinate + "my_prior_scale", + "my_log_prior_scale", +} + + +def _parse_script(path): + with open(path) as handle: + return ast.parse(handle.read()) + + +CIP_TREE = _parse_script(CIP_SCRIPT) +PSEUDO_PIPE_TREE = _parse_script(PSEUDO_PIPE_SCRIPT) + + +def _add_argument_kwargs(tree, option): + """The keyword arguments of a shipped ``parser.add_argument(option, ...)`` call. + + Lets a test assert against the CLI as actually shipped -- the real default, the + real `choices` -- instead of a value transcribed into the test, which is the same + drift problem the prior extraction above avoids. + """ + for node in ast.walk(tree): + if not (isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) + and node.func.attr == "add_argument" and node.args): + continue + try: + if ast.literal_eval(node.args[0]) != option: + continue + except (ValueError, SyntaxError): + continue + found = {} + for keyword in node.keywords: + if keyword.arg is None: + continue + try: + found[keyword.arg] = ast.literal_eval(keyword.value) + except (ValueError, SyntaxError): + # e.g. type=float, or a default built from an expression; the tests + # here only read literal defaults and choices + found[keyword.arg] = None + return found + raise AssertionError("no add_argument({!r}) call found".format(option)) + + +# argparse's own default for --ecc-min: what a run gets when the user says nothing. +CIP_ECC_MIN_DEFAULT = _add_argument_kwargs(CIP_TREE, "--ecc-min")["default"] +CIP_ECC_PRIOR_DEFAULT = _add_argument_kwargs( + CIP_TREE, "--eccentricity-prior")["default"] + + +def _exec_in(namespace, *nodes): + """Compile and run the given top-level CIP nodes in `namespace`.""" + module = ast.Module(body=list(nodes), type_ignores=[]) + exec(compile(module, CIP_SCRIPT, "exec"), namespace) + + +def _make_namespace(ecc_min=ECC_MIN, ecc_max=ECC_MAX, eccentricity_prior="uniform", + coords=()): + """The module-level constants the priors and the option wiring close over. + + `coords` stands in for CIP's low_level_coord_names, the coordinates the Monte + Carlo actually samples in; the eccentricity block consults it because the ln + coordinate needs a positive floor whatever the prior is. + """ + return { + "low_level_coord_names": list(coords), + # the eccentricity block exits on an unusable logarithmic range + "sys": sys, + "np": np, + "numpy": np, + "scipy": types.SimpleNamespace(stats=scipy_stats), + "chi_max": CHI_MAX, + "chi_small_max": CHI_MAX, + "ECC_MIN": ecc_min, + "ECC_MAX": ecc_max, + "MEANPERANO_MIN": 0.0, + "MEANPERANO_MAX": 2 * np.pi, + "lambda_min": LAMBDA_MIN, + "lambda_max": LAMBDA_MAX, + "lambda_small_max": LAMBDA_SMALL_MAX, + "mc_min": MC_MIN, + "mc_max": MC_MAX, + "p_Rbar": _p_rbar(), + # lambda_tilde_prior reads opts directly, as does the --eccentricity-prior block + "opts": types.SimpleNamespace(lambda_max=LAMBDA_MAX, + eccentricity_prior=eccentricity_prior), + } + + +def _load_priors(namespace): + """Exec the prior ``def`` blocks out of the CIP source, verbatim. + + Only top-level FunctionDef nodes are taken, so the surrounding script + (argparse, I/O, the fitting machinery) never runs. Selection is on + 'prior' appearing anywhere in the name, NOT a '_prior' suffix: the suffix + rule silently skips s_component_zprior, s_component_zprior_positive and + the two *volumetricprior densities, which is most of the spin sector. + """ + found = {} + for node in CIP_TREE.body: + if not isinstance(node, ast.FunctionDef): + continue + if "prior" not in node.name.lower() or node.name in NOT_A_DENSITY: + continue + _exec_in(namespace, node) + found[node.name] = namespace[node.name] + return found + + +PRIORS = _load_priors(_make_namespace()) + +# Support on which each prior may be evaluated. Only used to feed the smoke +# test valid inputs; priors with an integrable singularity at an endpoint are +# sampled strictly inside. +SUPPORT = { + # masses: the mass priors are normalized against the mc window, and M_prior + # / mc_prior go negative for x < 0, so they must not be fed the default + # spin-shaped interval + "M_prior": (MC_MIN, MC_MAX), + "mc_prior": (MC_MIN, MC_MAX), + "m1_prior": (1.0, 200.0), + "m2_prior": (1.0, 200.0), + "m_prior": (1.0, 1000.0), + "q_prior": (0.0, 1.0), + # eta in (0, 1/4]; both endpoints are singular, and the linspace below + # drops them + "eta_prior": (0.0, 0.25), + # delta_mc = sqrt(1-4 eta) in [0,1); eta -> 0 at the upper end + "delta_mc_prior": (0.0, 1.0), + "gaussian_mass_prior": (-4.0, 4.0), + "eccentricity_prior": (ECC_MIN, ECC_MAX), + "log_eccentricity_prior": (ECC_MIN, ECC_MAX), + "uniform_eccentricity_ln_prior": (ECC_MIN, ECC_MAX), + "eccentricity_squared_prior": (ECC_MIN, ECC_MAX), + # a density in e^2, so it is evaluated on the squared interval + "log_eccentricity_squared_prior": (ECC_MIN ** 2, ECC_MAX ** 2), + "meanPerAno_prior": (0.0, 2 * np.pi), + "precession_prior": (0.0, 2.0), + "lambda_prior": (LAMBDA_MIN, LAMBDA_MAX), + "lambda_small_prior": (LAMBDA_MIN, LAMBDA_SMALL_MAX), + "lambda_tilde_prior": (0.0, LAMBDA_MAX), + "delta_lambda_tilde_prior": (-500.0, 500.0), + "unnormalized_log_prior": (0.1, 10.0), + "normalized_Rbar_prior": (0.0, 1.0), + "normalized_Rbar_singular_prior": (1e-6, 1.0), + "normalized_zbar_prior": (-1.0, 1.0), + "s_component_volumetricprior": (0.0, 1.0), + "s_component_aligned_volumetricprior": (-1.0, 1.0), + "s_magnitude_uniform_prior": (0.0, CHI_MAX), + "s_component_sqrt_prior": (1e-6, CHI_MAX), + "s_component_zprior": (-CHI_MAX, CHI_MAX), + "s_component_zprior_positive": (0.0, CHI_MAX), +} +DEFAULT_SUPPORT = (-CHI_MAX, CHI_MAX) + +# Documented in-source as not normalized (or normalized only up to a factor the +# caller supplies). Excluded from the normalization test on purpose, not by +# oversight -- see the comments on each in CIP. +UNNORMALIZED = { + "unnormalized_uniform_prior", + "unnormalized_log_prior", + "xi_uniform_prior", + "M_prior", + "m_prior", + "m1_prior", + "m2_prior", + "mc_prior", + "q_prior", + "eta_prior", + "delta_mc_prior", + "s1z_prior", + "s2z_prior", + "lambda_tilde_prior", + "delta_lambda_tilde_prior", + "tapered_magnitude_prior", + "tapered_magnitude_prior_alt", + # p(a) for a volumetric spin MAGNITUDE prior; carries the 1/3 of the + # 3-d measure, so it is not a normalized 1-d density on its own. + "s_component_volumetricprior", +} + +# (prior, lower, upper, change of variable, interior singular points) for every +# prior that claims a normalized density. The 4th entry names the measure the +# density is defined against: 'x' integrates dx directly, 'log' integrates +# d(ln x), 'square' integrates d(x^2). Getting this wrong is exactly the bug +# being tested for, so each is spelled out rather than inferred. +# +# The 5th entry lists interior points where the integrand is singular; they are +# handed to quad's `points` so QUADPACK subdivides there. Without it an +# integrand that returns inf at a node aborts the process rather than raising. +NORMALIZED = [ + ("eccentricity_prior", ECC_MIN, ECC_MAX, "x", ()), + # The regression target: log-uniform in e over [ECC_MIN, ECC_MAX]. + ("log_eccentricity_prior", ECC_MIN, ECC_MAX, "x", ()), + # Density against d(ln e), so it must integrate to 1 over ln-space. + ("uniform_eccentricity_ln_prior", ECC_MIN, ECC_MAX, "log", ()), + # Density against d(e^2); see the INCONSISTENT note in CIP. + ("eccentricity_squared_prior", ECC_MIN, ECC_MAX, "square", ()), + # Already written as a function of u=e^2, so it integrates du over the squared + # interval directly rather than through the 'square' substitution. + ("log_eccentricity_squared_prior", ECC_MIN ** 2, ECC_MAX ** 2, "x", ()), + ("meanPerAno_prior", 0.0, 2 * np.pi, "x", ()), + ("precession_prior", 0.0, 2.0, "x", ()), + ("triangle_prior", -CHI_MAX, CHI_MAX, "x", ()), + ("s_component_uniform_prior", -CHI_MAX, CHI_MAX, "x", ()), + ("s_magnitude_uniform_prior", 0.0, CHI_MAX, "x", ()), + # 1/sqrt(|x|) singularity at the origin, integrable + ("s_component_sqrt_prior", -CHI_MAX, CHI_MAX, "x", (0.0,)), + ("s_component_zprior", -CHI_MAX, CHI_MAX, "x", (0.0,)), + ("s_component_zprior_positive", 0.0, CHI_MAX, "x", ()), + ("s_component_gaussian_prior", -CHI_MAX, CHI_MAX, "x", ()), + ("s_component_aligned_volumetricprior", -1.0, 1.0, "x", ()), + ("normalized_Rbar_prior", 0.0, 1.0, "x", ()), + ("normalized_Rbar_singular_prior", 0.0, 1.0, "x", ()), + ("normalized_zbar_prior", -1.0, 1.0, "x", ()), + ("lambda_prior", LAMBDA_MIN, LAMBDA_MAX, "x", ()), + ("lambda_small_prior", LAMBDA_MIN, LAMBDA_SMALL_MAX, "x", ()), +] + + +def test_priors_were_actually_extracted(): + """Guard against the extraction silently finding nothing. + + If CIP is refactored so the priors are no longer top-level '*_prior' + functions, every parametrized test below would collect zero cases and the + suite would go green while testing nothing. Fail loudly instead. + """ + assert len(PRIORS) > 25, "only found {} priors in CIP: {}".format( + len(PRIORS), sorted(PRIORS)) + for name in ("eccentricity_prior", "log_eccentricity_prior", + "uniform_eccentricity_ln_prior", "eccentricity_squared_prior", + "log_eccentricity_squared_prior"): + assert name in PRIORS, "{} not extracted from CIP".format(name) + + +@pytest.mark.parametrize("name", sorted(PRIORS)) +def test_prior_evaluates(name): + """Every prior evaluates on its support without raising, and returns + finite non-negative densities of the input shape. + + This is the check that would have caught np.ln at the point it was written: + the call raises AttributeError rather than returning a number. + """ + lo, hi = SUPPORT.get(name, DEFAULT_SUPPORT) + # strictly interior, so an integrable endpoint singularity is not the thing + # under test here + x = np.linspace(lo, hi, 17)[1:-1] + + value = np.asarray(PRIORS[name](x), dtype=float) + + # A constant prior may legitimately return a bare scalar rather than an + # array (m1_prior, m2_prior, m_prior, s1z_prior, s2z_prior all do), and + # callers rely on numpy broadcasting it. Require broadcastability, not an + # exact shape match. + try: + broadcast = np.broadcast_to(value, x.shape) + except ValueError: + pytest.fail("{}: returned shape {} does not broadcast to input {}".format( + name, value.shape, x.shape)) + + assert np.all(np.isfinite(broadcast)), "{}: non-finite densities".format(name) + assert np.all(broadcast >= 0), "{}: negative density".format(name) + + +@pytest.mark.parametrize("name,lo,hi,measure,singular", + NORMALIZED, ids=[row[0] for row in NORMALIZED]) +def test_prior_is_normalized(name, lo, hi, measure, singular): + """Priors that claim a normalized density must integrate to 1. + + Catches a wrong normalization constant, which -- unlike a wrong function + name -- raises nothing and merely reweights the posterior. With the + 0.0.17.12 log(ECC_MAX-ECC_MIN) constant this integrates to about -0.13 + rather than 1. + """ + prior = PRIORS[name] + + if measure == "log": + # density against d(ln x): substitute u = ln x + integrand = lambda u: float(prior(np.array([np.exp(u)]))[0]) + lo_t, hi_t = np.log(lo), np.log(hi) + elif measure == "square": + # density against d(x^2): substitute u = x^2 + integrand = lambda u: float(prior(np.array([np.sqrt(u)]))[0]) + lo_t, hi_t = lo ** 2, hi ** 2 + else: + integrand = lambda u: float(prior(np.array([u]))[0]) + lo_t, hi_t = lo, hi + + if singular: + total, err = integrate.quad(integrand, lo_t, hi_t, limit=200, + points=list(singular)) + else: + total, err = integrate.quad(integrand, lo_t, hi_t, limit=200) + + assert err < 1e-4, "{}: quadrature did not converge (err={})".format(name, err) + assert total == pytest.approx(1.0, rel=2e-3), ( + "{} integrates to {:.6f} over [{}, {}] d{}, not 1".format( + name, total, lo, hi, measure)) + + +def test_log_eccentricity_prior_is_log_uniform(): + """The shape check behind the normalization: e*p(e) is constant. + + A density uniform in ln(e) is p(e) = 1/(e * ln(emax/emin)), so e*p(e) does + not depend on e. This pins the 1/e, independently of the constant, and + distinguishes it from the flat eccentricity_prior. + """ + prior = PRIORS["log_eccentricity_prior"] + e = np.geomspace(ECC_MIN, ECC_MAX, 25) + + scaled = e * np.asarray(prior(e), dtype=float) + + assert np.allclose(scaled, scaled[0], rtol=1e-10), ( + "e*p(e) is not constant, so p is not log-uniform: {}".format(scaled)) + assert scaled[0] == pytest.approx(1.0 / np.log(ECC_MAX / ECC_MIN), rel=1e-10) + + +def test_uniform_and_log_eccentricity_priors_differ(): + """The two eccentricity priors must not be the same function. + + --eccentricity-prior selects between them; if a refactor collapsed one onto + the other the option would silently stop doing anything. + """ + e = np.linspace(ECC_MIN, ECC_MAX, 11) + + flat = np.asarray(PRIORS["eccentricity_prior"](e), dtype=float) + log_uniform = np.asarray(PRIORS["log_eccentricity_prior"](e), dtype=float) + + assert not np.allclose(flat, log_uniform) + # log-uniform puts more weight at small e, which is the entire point + assert log_uniform[0] > flat[0] + assert log_uniform[-1] < flat[-1] + + +### +### End-to-end coordinate selection: which density --eccentricity-prior actually +### installs for the coordinate a run samples in. +### +### CIP can sample eccentricity in three coordinates, and pseudo_pipe chooses among +### them: --parameter eccentricity, --parameter eccentricity_squared (what +### --use-eccentricity-squared asks for, and what iteration 0 of an eccentric run uses), +### and eccentricity_ln. The prior is looked up by coordinate name -- prior_map[p] with +### the range prior_range_map[p] -- so an option that rewrites only one entry silently +### leaves the other coordinates on their default density. +### + +ECC_COORDS = ("eccentricity", "eccentricity_ln", "eccentricity_squared") + + +def _eccentricity_dict_entries(name, namespace): + """Exec only the eccentricity entries of a shipped top-level dict literal. + + CIP's prior_map / prior_range_map also hold mcsampler callables, functools partials + and mass/spin/matter constants that this test has no business constructing. + Rebuilding the literal with just the eccentricity keys keeps the entries under test + identical to the shipped ones, while leaving the rest of the script out and not + breaking when an unrelated sector gains an entry. + """ + for node in CIP_TREE.body: + if not isinstance(node, ast.Assign) or not isinstance(node.value, ast.Dict): + continue + if not (len(node.targets) == 1 and isinstance(node.targets[0], ast.Name) + and node.targets[0].id == name): + continue + keys, values = [], [] + for key, value in zip(node.value.keys, node.value.values): + # IGWN production hosts still provide Python 3.6, where parsed + # string literals are ast.Str rather than ast.Constant. + key_value = key.s if isinstance(key, ast.Str) else getattr(key, "value", None) + if key_value in ECC_COORDS: + keys.append(key) + values.append(value) + assert keys, "no eccentricity entries in CIP's {}".format(name) + trimmed = ast.Assign(targets=node.targets, + value=ast.Dict(keys=keys, values=values)) + _exec_in(namespace, ast.fix_missing_locations( + ast.copy_location(trimmed, node))) + return namespace[name] + raise AssertionError("could not find the {} dict in CIP".format(name)) + + +def _selects_eccentricity(test): + """Does this `if` test steer the eccentricity setup? + + Matched on `opts.eccentricity_prior` or `ECC_MIN` appearing anywhere in the test, + rather than on one exact comparison: the prior selection and the zero-floor + correction are separate top-level conditions with different triggers, and a test + that recognised only the first would silently stop running the second. + """ + for node in ast.walk(test): + if (isinstance(node, ast.Attribute) and isinstance(node.value, ast.Name) + and node.value.id == "opts" and node.attr == "eccentricity_prior"): + return True + if isinstance(node, ast.Name) and node.id == "ECC_MIN": + return True + return False + + +def _exec_eccentricity_option_block(namespace): + """Run CIP's top-level eccentricity `if` blocks, verbatim and in source order.""" + found = 0 + for node in CIP_TREE.body: + if isinstance(node, ast.If) and _selects_eccentricity(node.test): + _exec_in(namespace, node) + found += 1 + assert found, "could not find the --eccentricity-prior block in CIP" + + +def _eccentricity_setup(ecc_min=ECC_MIN, ecc_max=ECC_MAX, eccentricity_prior="uniform", + coords=()): + """Reproduce CIP's eccentricity prior selection: defaults, then the option block.""" + namespace = _make_namespace(ecc_min=ecc_min, + ecc_max=ecc_max, + eccentricity_prior=eccentricity_prior, + coords=coords) + _load_priors(namespace) + # ln(ECC_MIN) with --ecc-min 0 is -inf here exactly as it is in CIP (and nan for a + # negative one); the option block is what repairs or rejects it, and that is the + # thing under test + with np.errstate(divide="ignore", invalid="ignore"): + prior_map = _eccentricity_dict_entries("prior_map", namespace) + prior_range_map = _eccentricity_dict_entries("prior_range_map", namespace) + _exec_eccentricity_option_block(namespace) + return namespace, prior_map, prior_range_map + + +def _integral_over_range(density, bounds): + """Integrate a coordinate's density over that coordinate's sampling range.""" + lo, hi = bounds + integrand = lambda u: float(np.asarray(density(np.array([u])), dtype=float)[0]) + return integrate.quad(integrand, lo, hi, limit=200) + + +def test_uniform_eccentricity_prior_leaves_the_shipped_defaults(): + """--eccentricity-prior uniform (the default) must not touch any entry.""" + namespace, prior_map, _ = _eccentricity_setup(eccentricity_prior="uniform") + + assert prior_map["eccentricity"] is namespace["eccentricity_prior"] + assert prior_map["eccentricity_squared"] is namespace["eccentricity_squared_prior"] + assert prior_map["eccentricity_ln"] is namespace["uniform_eccentricity_ln_prior"] + + +def test_log_uniform_selects_a_log_uniform_density_for_every_coordinate(): + """--eccentricity-prior log_uniform must reach the coordinate actually sampled. + + Setting only prior_map['eccentricity'] left a --parameter eccentricity_squared run + on the flat-in-e^2 default: no error, no warning, a different posterior than the + one requested. + """ + namespace, prior_map, prior_range_map = _eccentricity_setup( + eccentricity_prior="log_uniform") + + assert prior_map["eccentricity"] is namespace["log_eccentricity_prior"] + assert prior_map["eccentricity_squared"] is namespace["log_eccentricity_squared_prior"] + # uniform in ln(e) already IS this distribution written in that coordinate, so the + # default entry is correct and deliberately left alone + assert prior_map["eccentricity_ln"] is namespace["uniform_eccentricity_ln_prior"] + + for coord in ECC_COORDS: + total, err = _integral_over_range(prior_map[coord], prior_range_map[coord]) + assert err < 1e-4, "{}: quadrature did not converge".format(coord) + assert total == pytest.approx(1.0, rel=2e-3), ( + "{}: selected density integrates to {:.6f} over its sampling range {}, " + "not 1".format(coord, total, prior_range_map[coord])) + + +def test_log_uniform_is_one_distribution_in_e_and_in_e_squared(): + """The e and e^2 coordinates must describe the SAME distribution. + + Equal densities are not the requirement -- equal probability is. P(e < E) computed + in the e coordinate must equal P(e^2 < E^2) computed in the e^2 coordinate, which is + what fails if the e^2 entry keeps a density of a different family. + """ + _, prior_map, _ = _eccentricity_setup(eccentricity_prior="log_uniform") + + for cut in np.geomspace(1.5 * ECC_MIN, 0.9 * ECC_MAX, 5): + cdf_e, _ = _integral_over_range(prior_map["eccentricity"], (ECC_MIN, cut)) + cdf_u, _ = _integral_over_range(prior_map["eccentricity_squared"], + (ECC_MIN ** 2, cut ** 2)) + assert cdf_u == pytest.approx(cdf_e, rel=1e-6), ( + "P(e<{:.4f}) is {:.6f} sampling in e but {:.6f} sampling in e^2".format( + cut, cdf_e, cdf_u)) + + +def test_ecc_min_zero_correction_reaches_every_coordinate_range(): + """--ecc-min 0 with log_uniform: the 0.001 floor must reach every range. + + A log-uniform density is not integrable down to zero in ANY of these coordinates, so + a range whose lower edge is left at 0 gives a divergent normalization rather than a + prior. + """ + namespace, prior_map, prior_range_map = _eccentricity_setup( + ecc_min=0.0, eccentricity_prior="log_uniform") + + assert namespace["ECC_MIN"] == 0.001 + + for coord in ECC_COORDS: + bounds = prior_range_map[coord] + assert np.all(np.isfinite(bounds)), ( + "{}: sampling range {} still has a zero-eccentricity edge".format( + coord, bounds)) + total, err = _integral_over_range(prior_map[coord], bounds) + assert err < 1e-4, "{}: quadrature did not converge".format(coord) + assert total == pytest.approx(1.0, rel=2e-3), ( + "{}: selected density integrates to {:.6f} over its sampling range {}, " + "not 1".format(coord, total, bounds)) + + +def test_ln_coordinate_is_usable_at_the_shipped_cli_defaults(): + """--parameter eccentricity_ln with no --ecc-min and no --eccentricity-prior. + + eccentricity_ln is a logarithmic coordinate under EVERY prior, so the shipped + --ecc-min default hits it whatever --eccentricity-prior says: the range is + [log(0), log(ECC_MAX)] and uniform_eccentricity_ln_prior divides by log(ECC_MAX/0). + The floor therefore has to be keyed on the coordinate as well as on the prior. + + Both defaults are read out of CIP's own argparse calls rather than written here, so + this exercises the real default invocation and keeps following it if it changes. + """ + namespace, prior_map, prior_range_map = _eccentricity_setup( + ecc_min=CIP_ECC_MIN_DEFAULT, + eccentricity_prior=CIP_ECC_PRIOR_DEFAULT, + coords=("mc", "eta", "eccentricity_ln")) + + assert namespace["ECC_MIN"] > 0, ( + "ecc-min is still {} for a run sampling ln(e)".format(namespace["ECC_MIN"])) + + bounds = prior_range_map["eccentricity_ln"] + assert np.all(np.isfinite(bounds)), ( + "eccentricity_ln sampling range {} still has a log(0) edge".format(bounds)) + + # evaluating at all is the point: with ECC_MIN left at 0.0 this raises + # ZeroDivisionError inside the prior rather than returning a density + density = prior_map["eccentricity_ln"] + values = np.asarray(density(np.linspace(bounds[0], bounds[1], 9)), dtype=float) + assert np.all(np.isfinite(values)) and np.all(values > 0) + + total, err = _integral_over_range(density, bounds) + assert err < 1e-4, "eccentricity_ln: quadrature did not converge" + assert total == pytest.approx(1.0, rel=2e-3), ( + "eccentricity_ln: density integrates to {:.6f} over its sampling range {}, " + "not 1".format(total, bounds)) + + +def test_ecc_min_zero_is_left_alone_without_a_log_prior_or_log_coordinate(): + """The floor is a repair, not a policy: a linear-in-e run keeps the ecc-min given. + + --parameter eccentricity under the uniform prior is perfectly well defined down to + e=0, so raising its lower edge would move a boundary the user set. + """ + namespace, _, prior_range_map = _eccentricity_setup( + ecc_min=0.0, eccentricity_prior="uniform", coords=("mc", "eccentricity")) + + assert namespace["ECC_MIN"] == 0.0 + assert prior_range_map["eccentricity"][0] == 0.0 + + +# Every way a logarithmic eccentricity range can be unusable other than the zero floor, +# which is repaired rather than rejected. All of these reach np.log of a non-positive +# number or a zero/negative log(ECC_MAX/ECC_MIN), i.e. nan or inf bounds and densities. +INVALID_LOG_RANGES = [ + (-0.1, ECC_MAX, "negative ecc-min"), + (0.0, -0.1, "zero ecc-min floored, negative ecc-max"), + (0.1, 0.0, "zero ecc-max"), + (0.3, 0.2, "ecc-max below ecc-min"), + (0.2, 0.2, "empty interval"), +] + +# The two independent ways a run becomes logarithmic in e; both must validate. +LOG_TRIGGERS = [ + ({"eccentricity_prior": "log_uniform", "coords": ("mc", "eccentricity")}, + "log_uniform prior"), + ({"eccentricity_prior": "uniform", "coords": ("mc", "eccentricity_ln")}, + "ln coordinate"), +] + + +@pytest.mark.parametrize("trigger,trigger_id", LOG_TRIGGERS, + ids=[row[1] for row in LOG_TRIGGERS]) +@pytest.mark.parametrize("ecc_min,ecc_max,case", INVALID_LOG_RANGES, + ids=[row[2] for row in INVALID_LOG_RANGES]) +def test_logarithmic_eccentricity_rejects_an_unusable_range(ecc_min, ecc_max, case, + trigger, trigger_id): + """An invalid logarithmic range must fail the run, not produce nan priors. + + Only an exactly-zero ecc-min was ever checked, so e.g. --ecc-min -0.1 walked past the + floor correction into np.log of a negative number: the sampling bounds and the prior + densities come out nan, nothing raises, and the run reports a prior it does not have. + """ + with pytest.raises(SystemExit) as excinfo: + _eccentricity_setup(ecc_min=ecc_min, ecc_max=ecc_max, **trigger) + + assert excinfo.value.code not in (0, None), ( + "{} with {}: exited {}, which reads as success".format( + case, trigger_id, excinfo.value.code)) + + +@pytest.mark.parametrize("trigger,trigger_id", LOG_TRIGGERS, + ids=[row[1] for row in LOG_TRIGGERS]) +def test_valid_logarithmic_eccentricity_range_is_accepted_untouched(trigger, trigger_id): + """The rejection above must not catch an ordinary 0 < ecc-min < ecc-max run. + + A validity check that also refuses good input would take out every eccentric run. + """ + namespace, prior_map, prior_range_map = _eccentricity_setup( + ecc_min=0.01, ecc_max=ECC_MAX, **trigger) + + assert namespace["ECC_MIN"] == 0.01, "a valid ecc-min was moved" + for coord in ECC_COORDS: + bounds = prior_range_map[coord] + assert np.all(np.isfinite(bounds)), "{}: non-finite range {}".format(coord, bounds) + assert bounds[0] < bounds[1], "{}: inverted range {}".format(coord, bounds) + values = np.asarray(prior_map[coord](np.linspace(bounds[0], bounds[1], 9)[1:-1]), + dtype=float) + assert np.all(np.isfinite(values)) and np.all(values > 0), ( + "{}: non-finite or non-positive densities".format(coord)) + + +@pytest.mark.parametrize("tree,script", [(CIP_TREE, "CIP"), + (PSEUDO_PIPE_TREE, "pseudo_pipe")], + ids=["CIP", "pseudo_pipe"]) +def test_eccentricity_prior_option_rejects_unknown_values(tree, script): + """Both parsers must constrain --eccentricity-prior to the values CIP implements. + + pseudo_pipe forwards the string verbatim and CIP branches on exactly 'log_uniform', + so an unconstrained option turns a typo -- or an unimplemented value -- into a run + that silently uses the uniform prior and reports the requested one. + """ + kwargs = _add_argument_kwargs(tree, "--eccentricity-prior") + choices = kwargs.get("choices") + + assert choices is not None, ( + "{}: --eccentricity-prior accepts any string".format(script)) + assert sorted(choices) == sorted(["uniform", "log_uniform"]), ( + "{}: --eccentricity-prior choices are {}".format(script, choices)) + assert kwargs.get("default") in choices, ( + "{}: default {!r} is not one of the accepted values".format( + script, kwargs.get("default"))) diff --git a/MonteCarloMarginalizeCode/Code/test/test_container_manifest.py b/MonteCarloMarginalizeCode/Code/test/test_container_manifest.py index 0b30d32da..177983a8f 100644 --- a/MonteCarloMarginalizeCode/Code/test/test_container_manifest.py +++ b/MonteCarloMarginalizeCode/Code/test/test_container_manifest.py @@ -58,6 +58,22 @@ ) +ALL_OSDF_MANIFEST = textwrap.dedent( + """ + version: 1 + fallback: ancient + containers: + - label: ancient + image: osdf:///igwn/sw/rift_ancient_cuda11.sif + cuda_capability_min: 3.0 + cuda_capability_max: 7.0 + - label: modern + image: osdf:///igwn/sw/rift_modern_cuda12.sif + cuda_capability_min: 7.0 + """ +) + + def _write(tmp_path, text, name="fam.yaml"): p = tmp_path / name p.write_text(text) @@ -147,7 +163,8 @@ def test_selectors_are_not_undefined_guarded(tmp_path): m = cm.load_container_manifest(_write(tmp_path, MIXED_MANIFEST)) assert "=?= undefined" not in cm.build_singularity_image_expr(m) assert "=?= undefined" not in cm.build_transfer_input_expr(m) - assert "=?= undefined" not in cm.build_container_image_select(m) + assert "=?= undefined" not in cm.build_container_image_select( + cm.load_container_manifest(_write(tmp_path, ALL_OSDF_MANIFEST, "osdf.yaml"))) def test_capability_defined_requirement(tmp_path): @@ -257,17 +274,33 @@ def test_backward_compat_single_sif(tmp_path, monkeypatch): # --------------------------------------------------------------------------- def test_container_image_select_expression(tmp_path): - m = cm.load_container_manifest(_write(tmp_path, MIXED_MANIFEST)) + m = cm.load_container_manifest(_write(tmp_path, ALL_OSDF_MANIFEST)) expr = cm.build_container_image_select(m) - # a $$() match-time substitution token with VERBATIM image values (osdf URL - # fetched by container universe; cvmfs path used in place) -- NOT a ./basename - # rewrite, and NOT undefined-guarded (Requirements exclusion is used instead) + # A $$() match-time substitution token over BASENAMES. condor_submit derives the + # job ad's ContainerImage as the text after the LAST '/', *before* any $$ + # expansion, so a selector containing a path is truncated and the job holds at the + # execute point ("Unable to download or build singularity image ...sif\") ])", + # observed live on an OSPool glidein). With no '/' the token survives intact and + # the schedd expands it at match time. assert expr.startswith("$$([ ") and expr.endswith(" ])") + assert "/" not in expr # THE invariant assert "=?= undefined" not in expr # not a guess-guard assert "ifThenElse(TARGET.GPUs_Capability >= 7.0," in expr - assert '"osdf:///igwn/rift_modern_cuda12.sif"' in expr # raw osdf URL - assert '"/cvmfs/sw/rift_ancient_cuda11.sif"' in expr # fallback verbatim - assert "./rift_modern_cuda12.sif" not in expr # no basename rewrite + assert '"rift_modern_cuda12.sif"' in expr # basename branch + assert '"rift_ancient_cuda11.sif"' in expr # fallback basename + + +def test_container_image_select_rejects_in_place_images(tmp_path): + # An in-place (CVMFS/local) image can only be named by its full path, which would + # reintroduce the '/' truncation. Refuse loudly rather than emit a submit file + # that holds every job. + m = cm.load_container_manifest(_write(tmp_path, MIXED_MANIFEST)) + with pytest.raises(cm.ContainerManifestError) as exc: + cm.build_container_image_select(m) + assert "ancient" in str(exc.value) + # ... but the CPU-only single-image path is unaffected: it is a plain literal that + # condor_submit handles correctly. + assert cm.build_container_image_select(m, request_gpu=False) == "/cvmfs/sw/rift_ancient_cuda11.sif" def test_integration_container_universe(tmp_path, monkeypatch): @@ -285,7 +318,7 @@ def test_integration_container_universe(tmp_path, monkeypatch): arg_str="--foo bar", transfer_files=["../all.net"], use_singularity=True, - singularity_image=_write(tmp_path, MIXED_MANIFEST), + singularity_image=_write(tmp_path, ALL_OSDF_MANIFEST), request_gpu=True, cache_file="local.cache", ) @@ -294,9 +327,19 @@ def test_integration_container_universe(tmp_path, monkeypatch): ci = cmds["container_image"] assert ci.startswith("$$([") # match-time substitution, unquoted assert not ci.startswith('"') + assert "/" not in ci # else condor_submit truncates it assert "MY.SingularityImage" not in cmds # the OSG-breaking attr is gone assert "MY.SingularityBindCVMFS" not in cmds - assert "$$([" not in cmds.get("transfer_input_files", "") # image via container_image, not transfer + + # container_image names only a basename, so the image must arrive by transfer: + # exactly one comma-free $$() token carrying the full URLs. + tif = cmds["transfer_input_files"] + assert tif.count("$$([") == 1 + assert "osdf:///igwn/sw/rift_modern_cuda12.sif" in tif + # ... and TransferInput is pinned, so condor_submit does not append the basename + # selector to it as a bogus extra input file. + assert cmds["MY.TransferInput"] == '"' + tif.replace('"', '\\"') + '"' + assert "Capability >= 3.0" in cmds["require_gpus"] # floor still steers GPUs # GPU family job: still excludes slots that don't advertise the capability attr assert "TARGET.GPUs_Capability =!= undefined" in cmds["requirements"] diff --git a/MonteCarloMarginalizeCode/Code/test/test_fairdraw_double_weighting.py b/MonteCarloMarginalizeCode/Code/test/test_fairdraw_double_weighting.py new file mode 100644 index 000000000..ba62f61c7 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_fairdraw_double_weighting.py @@ -0,0 +1,682 @@ +#!/usr/bin/env python +""" +Regression tests for DOUBLE-WEIGHTING a fair-drawn `_rvs` record +(bin/integrate_likelihood_extrinsic_batchmode, RIFT/integrators/*). + +Found by the mechanical `_rvs` audit +(test/expensive_before_merging/integrators/audit_rvs_fairdraw.py). + +THE DEFECT. At the end of integrate_log the fair draw replaces every `_rvs` key with rows +resampled WITH REPLACEMENT proportional to the importance weight `w`. Those rows are then an +EQUAL-WEIGHT draw from the posterior. Three consumers went on to weight them by `w` again -- +so the product follows `w^2` and is over-concentrated: + + * the `--extrinsic-proposal-output` GMM breadcrumb, whose proposal is handed to the NEXT + iteration, so the truncation compounds across iterations; + * the `.dgrid` distance-posterior exporter; + * the `.dslice` reweight core (a different shape: it double-counts pi_Omega/q_Omega and + takes N from the resample, and cannot be corrected after the fact -- so it is routed to + the exact all-fresh path instead). + +`_pool_replica_rvs` has guarded against exactly this since the replica work, via +`already_resampled`. None of these three had an equivalent. + +THE PREDICATE MATTERS. `opts.fairdraw_extrinsic_output` is NOT the same question as "did the +draw fire": the samplers skip it when it would not shrink the record, and then the rows still +carry real importance weights that must be applied. So the samplers mark the rebind itself. +""" + +import os + +import numpy as np +import pytest + +import RIFT.integrators.mcsamplerAdaptiveVolume as mcsamplerAV + +NAMES = ['right_ascension', 'declination', 'phi_orb', 'inclination', 'psi', 'distance'] +NDIM = len(NAMES) + +_ILE = os.path.join(os.path.dirname(os.path.abspath(__file__)), + '..', 'bin', 'integrate_likelihood_extrinsic_batchmode') +_INTEGRATORS = os.path.join(os.path.dirname(os.path.abspath(__file__)), + '..', 'RIFT', 'integrators') + +SAMPLER_FILES = ['mcsampler.py', 'mcsamplerAdaptiveVolume.py', 'mcsamplerEnsemble.py', + 'mcsamplerGPU.py', 'mcsamplerNFlow.py', 'mcsamplerPortfolio.py'] + + +def _load_ile_helpers(): + """Exec the weight helpers out of the ILE script (not importable: it parses argv).""" + src = open(_ILE).read() + start = src.index("def ln_weights_from_rvs") + end = src.index("def _pool_replica_rvs") + ns = {"numpy": np, "np": np, "_rvs_lnL_convention": lambda x=None: bool(x)} + exec(compile(src[start:end], "ile_weight_helpers", "exec"), ns) + return ns + + +H = _load_ile_helpers() +ln_weights_from_rvs = H["ln_weights_from_rvs"] +ln_weights_for_posterior = H["ln_weights_for_posterior"] +_rvs_is_export_resample = H["_rvs_is_export_resample"] + + +class _FakeSampler(object): + def __init__(self, fairdrawn): + self._rvs_is_fairdraw = fairdrawn + + +def _record(n=200, seed=3): + rng = np.random.default_rng(seed) + lnL = rng.normal(0.0, 3.0, size=n) + return {"log_integrand": lnL, + "log_joint_prior": np.zeros(n), + "log_joint_s_prior": np.zeros(n), + "x": rng.normal(size=n)} + + +def _sampler(n_chunk=20000): + s = mcsamplerAV.MCSampler(n_chunk=n_chunk) + s.xpy = mcsamplerAV.xpy_default + s.identity_convert = mcsamplerAV.identity_convert + for name in NAMES: + s.add_parameter(name, pdf=None, left_limit=0.0, right_limit=1.0, + prior_pdf=lambda x: np.ones(np.shape(x)), adaptive_sampling=True) + return s + + +def _peaked(rho): + x0 = 0.5 * np.ones(NDIM) + w = (0.5 / rho) * np.ones(NDIM) + lnLmax = 0.5 * rho ** 2 + + def lnL(*args, **kwargs): + x = np.array([np.asarray(a, dtype=float).ravel() for a in args]).T + out = lnLmax - 0.5 * np.sum(((x - x0) / w) ** 2, axis=-1) + return np.where(out > lnLmax - 745.0, out, -np.inf) + return lnL + + +### +### 1. the helper itself +### + +def test_a_fair_drawn_record_gets_uniform_weights(): + r = _record() + lw = ln_weights_for_posterior(r, _FakeSampler(True)) + assert lw.shape == (len(r["log_integrand"]),) + assert np.allclose(lw, 0.0), 'a fair draw is already equal-weight; w must not be reapplied' + + +def test_a_retained_record_still_gets_its_importance_weights(): + """The other direction is just as wrong: flattening a record the draw never touched.""" + r = _record() + lw = ln_weights_for_posterior(r, _FakeSampler(False)) + assert np.allclose(lw, ln_weights_from_rvs(r)), \ + 'a record that was NOT resampled must keep its importance weights' + assert np.std(lw) > 1.0, 'these weights are not degenerate; flattening them loses the shape' + + +def test_the_helper_defaults_to_importance_weights_when_the_flag_is_absent(): + """An older sampler, or one hand-built in a test, must not be silently flattened.""" + class _Bare(object): + pass + r = _record() + assert not _rvs_is_export_resample(_Bare()) + assert np.allclose(ln_weights_for_posterior(r, _Bare()), ln_weights_from_rvs(r)) + + +### +### 2. the numbers: re-weighting a fair draw shifts the answer +### + +def test_reweighting_a_fair_draw_biases_a_posterior_mean(): + """The measurement behind the fix, at small scale: on a coordinate correlated with the + weight, applying w twice moves the posterior mean well outside its own MC error.""" + rng = np.random.default_rng(11) + n = 20000 + lnL = rng.normal(0.0, 4.0, size=n) + x = 0.5 * lnL + 0.5 * rng.normal(size=n) + rvs = {"log_integrand": lnL, "log_joint_prior": np.zeros(n), + "log_joint_s_prior": np.zeros(n), "x": x} + w = np.exp(lnL - lnL.max()); w /= w.sum() + truth = float(np.sum(w * x)) + + idx = rng.choice(np.arange(n), size=4000, replace=True, p=w) # the rebind + drawn = {k: np.asarray(v)[idx] for k, v in rvs.items()} + + correct = float(np.mean(ln_weights_for_posterior(drawn, _FakeSampler(True)) * 0 + + drawn["x"])) + lw = ln_weights_from_rvs(drawn) + ww = np.exp(lw - lw.max()); ww /= ww.sum() + doubled = float(np.sum(ww * drawn["x"])) + + assert abs(correct - truth) < 0.1, 'the unweighted fair draw should recover the truth' + assert doubled - truth > 0.5, \ + 'expected a clear upward bias from w^2, got {:+.3f}'.format(doubled - truth) + + +### +### 3. the samplers mark the rebind, and only when it fires +### + +def test_the_sampler_marks_the_record_when_the_fair_draw_fires(): + np.random.seed(20260813) + s = _sampler() + s.integrate_log(_peaked(100.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + assert _rvs_is_export_resample(s), 'the fair draw fired but left no marker' + + +def test_the_sampler_does_not_mark_the_record_when_no_fair_draw_was_asked_for(): + np.random.seed(20260813) + s = _sampler() + s.integrate_log(_peaked(100.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False) + assert not _rvs_is_export_resample(s) + + +def test_the_marker_is_reset_per_pass_rather_than_latching(): + """Samplers are reused across events (--n-events-to-analyze), so a latched True would + flatten the weights of every later pass that did not resample.""" + np.random.seed(20260813) + s = _sampler() + s.integrate_log(_peaked(100.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + assert _rvs_is_export_resample(s) + s.integrate_log(_peaked(100.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False) + assert not _rvs_is_export_resample(s), 'the marker latched across passes' + + +@pytest.mark.parametrize('fname', SAMPLER_FILES) +def test_every_sampler_with_a_fair_draw_marks_it(fname): + """Fix the twin: all seven rebind sites, not just the one that was being edited.""" + src = open(os.path.join(_INTEGRATORS, fname)).read() + if 'bFairdraw' not in src: + pytest.skip('{} has no fair-draw block'.format(fname)) + n_anchor = src.count('bFairdraw = kwargs[') + src.count('bFairdraw = kwargs[') + assert src.count('self._rvs_is_fairdraw = True') == n_anchor, \ + '{}: {} fair-draw block(s) but {} marker(s) -- a rebind is unmarked'.format( + fname, n_anchor, src.count('self._rvs_is_fairdraw = True')) + assert src.count('self._rvs_is_fairdraw = False') == n_anchor, \ + '{}: a fair-draw block never resets the marker, so it latches across passes'.format(fname) + + +### +### 4. the consumers actually use it +### + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +@pytest.mark.parametrize('anchor', [ + '_ext = _ehmod.fit_extrinsic_proposal', # extrinsic-proposal breadcrumb + 'dgrid = build_distance_grid(', # .dgrid +]) +def test_the_weighted_exporters_ask_for_posterior_weights(anchor): + src = open(_ILE).read() + i = src.index(anchor) + block = src[max(0, i - 2500):i] + assert 'ln_weights_for_posterior' in block, \ + 'this exporter still applies importance weights to a possibly fair-drawn record' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_dslice_reweight_core_refuses_a_fair_drawn_record(): + src = open(_ILE).read() + i = src.index('all_fresh = bool(getattr(opts, "distance_slice_all_fresh"') + block = src[i:i + 1800] + assert '_rvs_is_export_resample(sampler)' in block, \ + 'the .dslice reweight core still runs on the fair-draw export' + assert 'all_fresh = True' in block, 'it does not route to the exact fresh path' + + +### +### 5. pooled n_eff after replica pooling +### + +def _block_kish(rep_lnZ, rep_neff): + """The formula the ILE now uses when the pooled export is fair-drawn.""" + l = np.asarray(rep_lnZ, float) - float(np.max(rep_lnZ)) + Z = np.exp(l) + n = np.asarray(rep_neff, float) + ok = np.isfinite(Z) & np.isfinite(n) & (n > 0) + return float(np.sum(Z[ok]) ** 2 / np.sum(Z[ok] ** 2 / n[ok])) + + +def test_pooled_neff_reduces_to_the_sum_when_replicas_agree(): + """The property the original comment asked for, and the sanity check on the formula.""" + for K, ne in ((3, 40.0), (5, 12.5), (2, 100.0)): + assert _block_kish([7.0] * K, [ne] * K) == pytest.approx(K * ne, rel=1e-9) + + +def test_pooled_neff_falls_below_the_sum_when_replicas_disagree(): + """Disagreement is the whole reason the replicas are run; it must show up here.""" + agree = _block_kish([7.0, 7.0, 7.0], [40.0] * 3) + disagree = _block_kish([7.0, 7.0, 11.0], [40.0] * 3) # one replica 4 nats high + assert disagree < agree + assert disagree < 0.5 * agree, 'a 4-nat outlier barely moved the pooled n_eff' + + +def test_pooled_neff_is_not_the_export_row_count(): + """The defect: Kish of the FLATTENED pooled record is just its row count. + + _pool_replica_rvs forces equal weights within each block when the input is fair-drawn, so + the Kish n_eff of that record equals the number of exported rows -- which is + K*min(n_max, 1.5*eff_samp, 1.5*neff), the size of the export, not the quality of the + integral. At the default --fairdraw-extrinsic-output-n-max 5 that is 5K. + """ + K, n_k = 4, 5 # 5 exported rows per replica, the default cap + lnZ_k, neff_k = 7.0, 60.0 + flat_lw = np.concatenate([ + np.full(n_k, lnZ_k - np.log(K) - np.log(n_k)) for _ in range(K)]) + w = np.exp(flat_lw - flat_lw.max()) + kish_of_export = float(np.sum(w) ** 2 / np.sum(w ** 2)) + assert kish_of_export == pytest.approx(K * n_k), 'flat weights: Kish IS the row count' + assert kish_of_export == pytest.approx(20.0) + assert _block_kish([lnZ_k] * K, [neff_k] * K) == pytest.approx(K * neff_k) + assert _block_kish([lnZ_k] * K, [neff_k] * K) > 10 * kish_of_export, \ + 'the export row count understates the pooled n_eff by more than an order of magnitude' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_ile_uses_the_block_form_only_for_a_fair_drawn_export(): + """A record that was never resampled still has real per-row weights, and the pooled Kish + over them is finer-grained than the block form -- so the switch must be conditional.""" + src = open(_ILE).read() + i = src.index('_neff_pooled') + block = src[i - 1800:i + 2000] + # keyed on whether pooling FLATTENED any block -- not on a record-level flag, which the + # pooling step two hundred lines above clears, making this branch dead + assert '_blocks_flattened' in block, 'the switch is unconditional or dead' + assert '_kish_neff_of_rvs(sampler._rvs)' in block, \ + 'the non-flattened path no longer uses the pooled Kish' + + +### +### 6. the L0 reject threshold, pinned to its measurement +### + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_l0_reject_threshold_is_the_measured_default(): + """0.5 was strictly dominated and is not a safe value to drift back to. + + Measured over 160 known-lnZ passes (L0_REJECT_DLNZ_MEASUREMENT.md): the gate caught 0 of + 55 genuinely truncated warm passes at EVERY threshold from 0.25 to 4.0 nats, on both AV + and portfolio. Meanwhile at 0.5 it rejected 25% of GOOD portfolio warm passes, keeping a + collapsed cold result instead. So 0.5 bought no detection and cost one good pass in four; + 3.0 costs ~0% and buys the same nothing, while still catching a genuinely large gap. + """ + src = open(_ILE).read() + i = src.index('--sampler-l0-rescue-reject-dlnZ') + decl = src[i:i + 400] + assert 'default=3.0' in decl, \ + 'the L0 reject threshold moved off its measured value; re-measure before changing it' + assert 'default=0.5' not in decl + + +### +### 7. COMPOSITION: the marker and the reserve must agree with the record beside them +### +### Both defects below were found in review, and both are the same shape: a fix that is +### correct in isolation and wrong once another code path runs after it. The tests above +### exercise the pieces separately and would pass with both bugs present. +### +### NOTE ON WHAT CARRIES THE WEIGHT HERE. The helpers below (_pool_replica_rvs, +### _snapshot_pass_state / _restore_pass_state) are correct in isolation and were correct +### with both bugs present -- the bugs were at the CALL SITES, in analyze_event, which needs +### data, PSDs and a waveform to run and cannot be exercised from a unit test. So the +### behavioural tests pin the contracts the call sites depend on, and the source-level tests +### pin the wiring. Verified by reverting each fix: only the wiring tests fail. If a future +### change makes analyze_event callable in pieces, promote these. +### + +def _load_pool_helpers(): + """Exec ln_weights_* plus _pool_replica_rvs and its dependencies.""" + src = open(_ILE).read() + start = src.index("def ln_weights_from_rvs") + end = src.index("def _warm_seed_geometry") + ns = {"numpy": np, "np": np, "_rvs_lnL_convention": lambda x=None: bool(x), + "mcsamplerAdaptiveVolume": mcsamplerAV} + exec(compile(src[start:end], "ile_pool_helpers", "exec"), ns) + return ns + + +P = _load_pool_helpers() + + +class _ConvSampler(object): + """Minimal stand-in for the sampler interface _pool_replica_rvs uses.""" + def __init__(self): + self._rvs_is_fairdraw = True + + @staticmethod + def identity_convert(x): + return x + + +def _fairdrawn_block(n, seed): + """An equal-weight posterior draw: the rows a fair draw leaves behind.""" + rng = np.random.default_rng(seed) + lnL = rng.normal(0.0, 2.0, size=n) + return {"log_integrand": lnL, + "log_joint_prior": np.zeros(n), + "log_joint_s_prior": np.zeros(n), + "x": rng.normal(size=n)} + + +def test_a_pooled_record_is_not_globally_equal_weight(): + """P1: each block is equal-weight WITHIN itself, but blocks differ by their evidences. + + Treating the pooled record as a fair draw makes .dgrid and the proposal breadcrumb mix the + replicas by exported ROW COUNT instead of by evidence -- discarding exactly the replica + disagreement the replicas were run to measure. + """ + rep_lnZ = [7.0, 9.0] # a 2-nat disagreement, i.e. e^2 in evidence + reps = [_fairdrawn_block(60, 1), _fairdrawn_block(60, 2)] + pooled = P["_pool_replica_rvs"](reps, _ConvSampler(), rep_lnZ=rep_lnZ, + already_resampled=True, use_lnL=False) + lw = P["ln_weights_from_rvs"](pooled) + assert np.ptp(lw) > 1.0, 'the pooled record came out globally flat; the evidences are gone' + a, b = lw[:60], lw[60:] + assert np.allclose(a, a[0]) and np.allclose(b, b[0]), 'within a block weights must be equal' + assert b[0] - a[0] == pytest.approx(rep_lnZ[1] - rep_lnZ[0], abs=1e-9), \ + 'the between-block offset must be exactly the evidence difference' + + +def test_the_fairdraw_marker_is_cleared_once_the_record_is_pooled(): + """...so ln_weights_for_posterior reads those reconstructed block weights.""" + src = open(_ILE).read() + i = src.index('_pool_replica_rvs(_rep_rvs') + block = src[i:i + 2600] + assert '_rvs_is_pooled = True' in block, \ + 'pooling leaves no mark; the pooled record would be treated as globally equal-weight' + assert 'is _r for _r in _rep_rvs' in block, \ + 'the mark must only be set when pooling actually happened -- every fallback in ' \ + '_pool_replica_rvs returns an INPUT record, which is still a plain fair draw' + assert '_rvs_is_fairdraw = False' not in block, \ + 'pooling must not clear rows-resampled: the .dslice guard and the block-Kish branch ' \ + 'both depend on it and would go dead' + + +def test_posterior_weights_of_a_pooled_record_keep_the_replica_evidences(): + """End to end through the helper the exporters actually call.""" + rep_lnZ = [7.0, 9.0] + reps = [_fairdrawn_block(40, 5), _fairdrawn_block(40, 6)] + s = _ConvSampler() + pooled = P["_pool_replica_rvs"](reps, s, rep_lnZ=rep_lnZ, + already_resampled=True, use_lnL=False) + s._rvs_is_fairdraw = False # what the ILE now does after pooling + lw = P["ln_weights_for_posterior"](pooled, s) + assert lw[40] - lw[0] == pytest.approx(rep_lnZ[1] - rep_lnZ[0], abs=1e-9) + # and the bug: had the marker survived, every row would weigh the same + s._rvs_is_fairdraw = True + assert np.allclose(P["ln_weights_for_posterior"](pooled, s), 0.0) + + +def test_a_fallback_pool_keeps_the_marker(): + """One replica, or a record with no sampling-prior column, comes back unchanged and is + still the fair draw it arrived as.""" + one = [_fairdrawn_block(30, 9)] + out = P["_pool_replica_rvs"](one, _ConvSampler(), rep_lnZ=[7.0], + already_resampled=True, use_lnL=False) + assert out is one[0], 'a single replica must come back as the same object' + + +def test_rejecting_the_warm_pass_restores_the_cold_reserve(): + """P1: the reject path put back _rvs, the estimate and the diagnostics, but left + _warm_seed_reserve holding the REJECTED warm cloud -- which --sampler-sequential-warmstart + then seeds the next intrinsic point from. Snapshot and restore must move together.""" + ns = {} + src = open(_ILE).read() + start = src.index("def _snapshot_pass_state") + end = src.index("def _warm_seed_geometry") + exec(compile(src[start:end], "ile_state_helpers", "exec"), ns) + + class _S(object): + pass + s = _S() + s._rvs = {"x": np.arange(5)} + s._warm_seed_reserve = {"tag": "cold"} + s._rvs_is_fairdraw = True + s.portfolio_realizations = [] + state = ns["_snapshot_pass_state"](s, 1.0, 2.0, 3.0, {"d": "cold"}) + + # the warm pass runs and overwrites everything in place + s._rvs = {"x": np.arange(2)} + s._warm_seed_reserve = {"tag": "warm"} + s._rvs_is_fairdraw = False + + res, var, neff, dd = ns["_restore_pass_state"](s, state) + assert (res, var, neff, dd) == (1.0, 2.0, 3.0, {"d": "cold"}) + assert s._warm_seed_reserve == {"tag": "cold"}, \ + 'the rejected warm reserve survived; the next point would seed from it' + assert s._rvs_is_fairdraw is True, 'the marker must describe the restored record' + assert ns["_warm_seed_reserve_for"] is not None + + +def test_the_restore_reaches_portfolio_member_reserves_too(): + """_warm_seed_reserve_for falls through to portfolio_realizations, so restoring only the + aggregate would leave that fallback pointing at the rejected warm pass.""" + ns = {} + src = open(_ILE).read() + start = src.index("def _snapshot_pass_state") + end = src.index("def _warm_seed_geometry") + exec(compile(src[start:end], "ile_state_helpers", "exec"), ns) + + class _S(object): + pass + m = _S(); m._warm_seed_reserve = {"tag": "cold-member"} + s = _S(); s._rvs = {}; s._warm_seed_reserve = None + s._rvs_is_fairdraw = False; s.portfolio_realizations = [m] + state = ns["_snapshot_pass_state"](s, 0, 0, 0, {}) + m._warm_seed_reserve = {"tag": "warm-member"} + ns["_restore_pass_state"](s, state) + assert m._warm_seed_reserve == {"tag": "cold-member"} + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_both_l0_restore_paths_go_through_the_shared_helper(): + """The reject path and the exception handler must not drift in WHAT they put back.""" + src = open(_ILE).read() + assert src.count('_restore_pass_state(sampler, _cold_state_l0)') == 2, \ + 'expected the reject path and the failure handler to share one restore' + assert 'sampler._rvs, res, var, neff, dict_return = _cold_state_l0' not in src, \ + 'the old partial tuple restore is back' + # and no hand-rolled partial restore alongside it: the reject branch must not poke _rvs + # directly, which is precisely what left the reserve describing the rejected warm pass. + i = src.index('keeping the COLD (full-support) result') + branch = src[i:i + 1200] + assert 'sampler._rvs =' not in branch, \ + 'the reject branch assigns _rvs directly again; the reserve and marker will not follow' + + +### +### 8. TWO PROPERTIES, TWO FLAGS (review round 2) +### +### "rows were resampled" and "the record is globally equal-weight" are different questions. +### A single fair draw answers yes to both; a POOLED record answers yes to the first and no to +### the second. Collapsing them broke things in both directions: answering the second with the +### first flag made .dgrid/.breadcrumb mix replicas by row count; answering the first with the +### second made the .dslice safeguard and the block-Kish n_eff branch unreachable. +### + +def _raw_block(n, seed, spread=2.0): + """A retained block: real, VARYING importance weights. Not a fair draw.""" + rng = np.random.default_rng(seed) + lnL = rng.normal(0.0, spread, size=n) + return {"log_integrand": lnL, "log_joint_prior": np.zeros(n), + "log_joint_s_prior": np.zeros(n), "x": rng.normal(size=n)} + + +def test_a_pooled_record_still_has_resampled_rows(): + """The .dslice reweight core must keep refusing it: reweighting rows that were already + drawn proportional to w double-counts, whether or not they were later pooled.""" + class _S(object): + _rvs_is_fairdraw = True + _rvs_is_pooled = True + assert _rvs_is_export_resample(_S()) is True + assert P["_rvs_is_equal_weight"](_S()) is False + + +def test_a_plain_fair_draw_answers_yes_to_both(): + class _S(object): + _rvs_is_fairdraw = True + _rvs_is_pooled = False + assert _rvs_is_export_resample(_S()) is True + assert P["_rvs_is_equal_weight"](_S()) is True + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_dslice_safeguard_survives_pooling(): + """It keys on rows-resampled, which pooling must not clear.""" + src = open(_ILE).read() + i = src.index('forcing --distance-slice-all-fresh') + branch = src[max(0, i - 900):i] + assert '_rvs_is_export_resample(sampler)' in branch + assert '_rvs_is_equal_weight(sampler)' not in branch, \ + 'the .dslice guard now asks the equal-weight question, which pooling clears' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_block_kish_branch_is_reachable_after_pooling(): + """It used to key on a flag that the line above it cleared, so it could never fire -- + and it is only reached at all when pooling happened.""" + src = open(_ILE).read() + i = src.index('block Kish over replicas') + branch = src[max(0, i - 1400):i] + assert '_blocks_flattened' in branch, \ + 'the block-Kish branch keys on a flag pooling clears; it is dead code' + assert '_rvs_is_export_resample(sampler)' not in branch + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_posterior_weight_helper_asks_the_equal_weight_question(): + src = open(_ILE).read() + i = src.index('def ln_weights_for_posterior') + body = src[i:i + 2600] + assert '_rvs_is_equal_weight(sampler)' in body + assert '_rvs_is_export_resample(sampler)' not in body + + +### +### 9. MIXED PROVENANCE: replicas decide independently whether to fair-draw +### + +def test_pooling_respects_per_replica_provenance(): + """A raw block must keep its weight SHAPE; a resampled block must be flattened. One + global boolean cannot express this, and gets one of the two wrong.""" + rep_lnZ = [7.0, 7.0] + raw, drawn = _raw_block(50, 21), _fairdrawn_block(50, 22) + pooled = P["_pool_replica_rvs"]([raw, drawn], _ConvSampler(), rep_lnZ=rep_lnZ, + already_resampled=[False, True], use_lnL=False) + lw = P["ln_weights_from_rvs"](pooled) + a, b = lw[:50], lw[50:] + assert np.std(a) > 0.5, 'the RAW block was flattened; genuine importance weights destroyed' + assert np.allclose(b, b[0]), 'the RESAMPLED block was not flattened; it stays double-weighted' + # both blocks must still carry the same total mass, since their evidences are equal + tot = lambda v: float(np.log(np.sum(np.exp(v - np.max(lw))))) + assert tot(a) == pytest.approx(tot(b), abs=1e-6) + + +def test_a_global_flag_would_flatten_a_raw_replica(): + """The failure the per-replica list prevents, pinned so it cannot come back.""" + rep_lnZ = [7.0, 7.0] + raw, drawn = _raw_block(50, 31), _fairdrawn_block(50, 32) + wrong = P["_pool_replica_rvs"]([raw, drawn], _ConvSampler(), rep_lnZ=rep_lnZ, + already_resampled=True, use_lnL=False) + lw = P["ln_weights_from_rvs"](wrong) + assert np.allclose(lw[:50], lw[0]), \ + 'expected the global-True path to flatten the raw block (this is the bug being avoided)' + + +def test_pooling_filters_empty_replicas_in_lockstep_with_their_metadata(): + """An empty record used to be dropped from rep_rvs alone, shifting every later block + against its own lnZ -- and now against its own resampled flag too.""" + rep_lnZ = [7.0, 99.0, 9.0] # the middle entry belongs to the empty one + blocks = [_fairdrawn_block(40, 41), {}, _fairdrawn_block(40, 42)] + pooled = P["_pool_replica_rvs"](blocks, _ConvSampler(), rep_lnZ=rep_lnZ, + already_resampled=[True, True, True], use_lnL=False) + lw = P["ln_weights_from_rvs"](pooled) + assert len(lw) == 80 + # the surviving blocks must be offset by 9.0-7.0, not by anything involving 99.0 + assert lw[40] - lw[0] == pytest.approx(2.0, abs=1e-9) + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_ile_passes_per_replica_provenance_not_the_cli_option(): + src = open(_ILE).read() + i = src.index('_pool_replica_rvs(_rep_rvs') + call = src[i:i + 400] + assert 'already_resampled=_rep_fairdraw' in call, \ + 'the pooler is still told the CLI option instead of what each pass actually did' + assert 'opts.fairdraw_extrinsic_output' not in call + assert src.count('_rep_fairdraw.append(') == 1, 'replica markers are not captured per pass' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_pooled_marker_is_dropped_with_the_record(): + """_rvs_is_pooled is set by analyze_event, not by the sampler, so the sampler's per-pass + reset cannot clear it; a latched True would cost the NEXT event its equal-weight export.""" + src = open(_ILE).read() + i = src.index('# _rvs_is_pooled is set by THIS function') + assert '_rvs_is_pooled = False' in src[i:i + 700], \ + 'the pooled marker outlives the record it describes' + # and it must sit with the _rvs wipe at the END of analyze_event, not with the replica + # loop's per-replica reset, which runs many times per event + assert 'sampler._rvs = {}' in src[max(0, i - 300):i] + + +### +### 10. PROVENANCE LIFECYCLE: the pooled marker must not outlive a FAILED event +### + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_pooled_marker_is_reset_on_entry_not_only_on_the_normal_return(): + """_reject_if_collapsed RAISES after pooling; the caller's `except Exception` swallows it + and moves to the next event. Clearing the marker only on the normal return therefore + leaves it set, and the next ordinary fair draw is read as pooled -- so .dgrid and the + proposal breadcrumb apply importance weights to rows that already carry them. That is the + w^2 defect this suite exists to prevent, resurrected on the event after any failure.""" + src = open(_ILE).read() + i = src.index('def analyze_event(') + head = src[i:i + 1600] + assert '_rvs_is_pooled = False' in head, \ + 'the pooled marker is not reset on entry; it survives a failed event' + # ...and strictly before anything that could raise or export + j_reset = src.index('_rvs_is_pooled = False', i) + for later in ('_pool_replica_rvs(_rep_rvs', 'def _reject_if_collapsed', + 'ln_weights_for_posterior(rvs, sampler'): + if later in src[i:]: + assert j_reset < src.index(later, i), \ + 'the reset happens after {}; a raise before it still leaks'.format(later) + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_raising_collapse_gate_sits_after_pooling_so_the_leak_was_reachable(): + """Pins the precondition, so this test cannot quietly stop testing anything if the order + of the replica block and the gate ever changes.""" + src = open(_ILE).read() + i_pool = src.index('_pool_replica_rvs(_rep_rvs') + i_gate = src.index('_reject_if_collapsed(dict_return, "pooled over') + assert i_pool < i_gate, 'the pooled-verdict gate no longer runs after pooling' + assert 'raise _exc(' in src, 'the collapse gate no longer raises' + + +def test_a_stale_pooled_marker_would_reapply_importance_weights(): + """The consequence, in numbers: with the marker wrongly set, the helper stops returning + uniform weights for a plain fair draw and hands back w -- applied to rows already drawn + proportional to w.""" + r = _record() + + class _S(object): + _rvs_is_fairdraw = True + _rvs_is_pooled = True # stale, inherited from a failed previous event + + lw = ln_weights_for_posterior(r, _S()) + assert not np.allclose(lw, 0.0), 'expected the stale marker to reinstate w (the defect)' + assert np.allclose(lw, ln_weights_from_rvs(r)) + + _S._rvs_is_pooled = False # correctly reset on entry + assert np.allclose(ln_weights_for_posterior(r, _S()), 0.0) diff --git a/MonteCarloMarginalizeCode/Code/test/test_gwsignal_teob_mode_sign.py b/MonteCarloMarginalizeCode/Code/test/test_gwsignal_teob_mode_sign.py new file mode 100644 index 000000000..a9b1c02e8 --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_gwsignal_teob_mode_sign.py @@ -0,0 +1,207 @@ +"""Regression guard: RIFT's gwsignal TEOBResumSDALI templates must not carry +a global sign flip relative to the same plugin's polarizations. + +Background +---------- +RIFT reads TEOBResumSDALI through ``GenerateTDModes``; bilby and dingo read it +through ``GenerateFDWaveform`` (polarizations). For this one plugin the modes +are *minus* the polarization convention, and dimensionless rather than in +strain units. ``GWSignal.hlmoft`` rescales by ``nu M G/(c^2 D)``; that factor +must carry the minus sign. + +Getting it wrong is invisible in a fit. The antenna patterns obey +``F+(psi+pi/2) = -F+(psi)`` and ``Fx(psi+pi/2) = -Fx(psi)`` in every detector +for every mode, so a global sign on h is *exactly* ``psi -> psi + pi/2`` and +nothing else: sky location, distance, masses, inclination and the peak +likelihood all still agree. It displaced psi by a quarter turn in every RIFT +TEOBResumSDALI posterior in the eccentric-PE task force comparison. + +What this test asserts, and what it deliberately does not +-------------------------------------------------------- +It tests the *physics* (is there a sign error?), not the phase convention. + +Fitting a free complex coefficient ``c_m`` per azimuthal index m, RIFT's +templates are correct iff ``c_m = s * exp(i m delta)`` with ``s = +1``: a +common ``delta`` is just a relabelling of ``phi_ref``, but ``s = -1`` is the +psi bug. For a single m the two are indistinguishable -- ``-exp(i m delta)`` +can be reabsorbed into ``delta`` -- which is exactly why the quadrupole alone +cannot see this. Two m values sharing one ``delta`` separate them: + + arg(c_4) - 2 arg(c_2) == 0 if s = +1, == pi if s = -1 + +That combination is invariant under any choice of ``delta``, so this test +passes for either TEOB phase convention and fails only on a true sign error. +It is therefore agnostic about whether the ``exp(i m phi_shift)`` in +``hlmoft`` is right for TEOB -- a separate, open question about making +``phase`` line up across approximants. + +A whole-waveform scalar fit would NOT work here: the buggy and correct forms +differ per mode by ``-exp(i m pi/2)``, which is ``+1`` at ``m = +-2``, so the +dominant quadrupole agrees either way and carries nearly all the power. + +Not run in CI: the runners do not have the plugin packages, so the +backend-availability check skips there. A missing backend package is the +*only* thing allowed to skip -- once the package is present, every later +failure (building the generator, waveform generation, ``hlmoft``, the fit, the +assertion) is a real failure and is reported as one. +Run it where the plugin is installed: + + python -m pytest -q MonteCarloMarginalizeCode/Code/test/test_gwsignal_teob_mode_sign.py +""" +import importlib.util + +import numpy as np +import pytest + +lal = pytest.importorskip("lal") +lalsim = pytest.importorskip("lalsimulation") +u = pytest.importorskip("astropy.units") + +gws = pytest.importorskip("lalsimulation.gwsignal") +wfm = pytest.importorskip("lalsimulation.gwsignal.core.waveform") + +import RIFT.lalsimutils as lsu # noqa: E402 +import RIFT.physics.GWSignal as rgws # noqa: E402 + +M1, M2 = 80.0, 40.0 # q = 2, so odd-m modes are alive +DIST, IOTA = 1500.0, np.pi / 4 +DELTAT, DELTAF = 1.0 / 4096, 1.0 / 16 +F22 = FREF = 20.0 + + +def _pdict(): + return { + "mass1": M1 * u.solMass, "mass2": M2 * u.solMass, + "spin1x": 0.0 * u.dimensionless_unscaled, + "spin1y": 0.0 * u.dimensionless_unscaled, + "spin1z": 0.0 * u.dimensionless_unscaled, + "spin2x": 0.0 * u.dimensionless_unscaled, + "spin2y": 0.0 * u.dimensionless_unscaled, + "spin2z": 0.0 * u.dimensionless_unscaled, + "deltaT": DELTAT * u.s, "f22_start": F22 * u.Hz, + "f22_ref": FREF * u.Hz, "phi_ref": 0.0 * u.rad, + "distance": DIST * u.Mpc, "inclination": IOTA * u.rad, + "eccentricity": 0.0 * u.dimensionless_unscaled, + "longAscNodes": 0.0 * u.rad, "meanPerAno": 0.0 * u.rad, + "condition": 0, + } + + +def _P(): + P = lsu.ChooseWaveformParams() + P.m1, P.m2 = M1 * lal.MSUN_SI, M2 * lal.MSUN_SI + P.s1x = P.s1y = P.s1z = P.s2x = P.s2y = P.s2z = 0.0 + P.dist = DIST * 1e6 * lal.PC_SI + P.incl = IOTA + P.phiref = P.psi = 0.0 + P.fmin, P.fref = F22, FREF + P.deltaT, P.deltaF = DELTAT, DELTAF + P.eccentricity, P.meanPerAno = 0.0, 0.0 + P.taper = lalsim.SIM_INSPIRAL_TAPER_NONE + P.approx = lalsim.IMRPhenomXPHM # unused; approx_string drives gwsignal + return P + + +def _arr(x): + return np.asarray(x.value if hasattr(x, "value") else x) + + +def _times(x): + t = x.times + return np.asarray(t.value if hasattr(t, "value") else t) + + +def _regrid(ts, a, grid): + return a[np.clip(np.searchsorted(ts, grid), 0, len(a) - 1)] + + +# Third-party package each model is implemented by; a missing one is the only +# thing here that means "not this host" rather than "broken". Keyed by +# approximant, so adding an approximant to the parametrization without saying +# what provides it raises KeyError instead of silently skipping. Both are +# top-level modules, which keeps the ``find_spec`` lookup below simple. +_BACKEND_MODULE = { + "TEOBResumSDALI": "EOBRun_module", # pip install teobresums + "SEOBNRv5EHM": "pyseobnr", +} + + +def _generator(approx): + """The gwsignal generator for ``approx``; skip only if its backend is absent. + + Absence is decided on its own terms, before anything is constructed: + ``find_spec`` locates the plugin package without importing it, so it cannot + be confused with a failure of the code under test. The factory call is + then deliberately unguarded. On a host that *has* the backend, an API + incompatibility, a plugin that raises on import, or a regression inside + ``gwsignal_get_waveform_generator`` is a real failure of this guard; + catching it would downgrade the very breakage this file exists to detect + into a green skip that asserts nothing. A gwsignal too old to know the + approximant at all fails here for the same reason -- the package is + installed, so the mismatch is worth reporting. + """ + module = _BACKEND_MODULE[approx] + if importlib.util.find_spec(module) is None: + pytest.skip("%s needs the %s module, which is not installed here" + % (approx, module)) + return gws.models.gwsignal_get_waveform_generator(approx) + + +def _coeffs_per_m(approx, gen): + """Free complex coefficient per azimuthal index, fitting the gwsignal + polarizations with RIFT's own modes.""" + hp, hc = wfm.GenerateTDWaveform(_pdict(), gen) + t_pol = _times(hp) + + hlm = rgws.hlmoft(_P(), Lmax=4, approx_string=approx) + key0 = sorted(hlm)[0] + t_rift = (float(hlm[key0].epoch) + + np.arange(hlm[key0].data.length) * hlm[key0].deltaT) + + grid = np.arange(max(t_rift[0], t_pol[0]), min(t_rift[-1], t_pol[-1]), + DELTAT) + target = (_regrid(t_pol, _arr(hp), grid) + - 1j * _regrid(t_pol, _arr(hc), grid)) + + ms = sorted({k[1] for k in hlm}) + cols = [] + for m in ms: + b = np.zeros(len(t_rift), dtype=complex) + for k in hlm: + if k[1] == m: + b += hlm[k].data.data * lal.SpinWeightedSphericalHarmonic( + IOTA, 0.0, -2, k[0], k[1]) + cols.append(_regrid(t_rift, b, grid)) + + A = np.array(cols).T + keep = np.abs(target) > 0.01 * np.abs(target).max() + c, *_ = np.linalg.lstsq(A[keep], target[keep], rcond=None) + return dict(zip(ms, c)) + + +def _sign_invariant(c): + """arg(c_4) - 2 arg(c_2), in radians, folded to [-pi, pi]. + + 0 => no global sign error (any phi_ref convention). +-pi => psi is + displaced by pi/2. + """ + z = c[4] * np.conj(c[2]) ** 2 + return np.angle(z / np.abs(z)) + + +@pytest.mark.parametrize("approx", ["TEOBResumSDALI", "SEOBNRv5EHM"]) +def test_no_global_sign_error_vs_polarizations(approx): + c = _coeffs_per_m(approx, _generator(approx)) + + for m in (2, 4): + assert m in c, "need m=%d to separate a sign from a phase" % m + + d = _sign_invariant(c) + assert abs(d) < np.radians(45), ( + "%s: global sign error -- RIFT's templates are -1 x the gwsignal " + "polarizations, i.e. psi is displaced by pi/2.\n" + " arg(c_4) - 2 arg(c_2) = %+.1f deg (expected ~0, pi means the bug)\n" + " per-m coefficients: %s" + % (approx, np.degrees(d), + " ".join("m=%+d: %.3f/%+.1fd" % (m, abs(v), np.degrees(np.angle(v))) + for m, v in sorted(c.items())))) diff --git a/MonteCarloMarginalizeCode/Code/test/test_l0_rescue_seed.py b/MonteCarloMarginalizeCode/Code/test/test_l0_rescue_seed.py index c090ef352..bc9c2df1a 100644 --- a/MonteCarloMarginalizeCode/Code/test/test_l0_rescue_seed.py +++ b/MonteCarloMarginalizeCode/Code/test/test_l0_rescue_seed.py @@ -425,6 +425,26 @@ def test_a_portfolio_pass_leaves_a_reserve_of_its_aggregate_retained_points(): assert r['X'].shape[1] == NDIM +def test_a_second_portfolio_pass_reserve_contains_only_second_pass_draws(): + """A warm retry reuses member proposals, not the cold pass's aggregate sample cache.""" + np.random.seed(20260812) + s = _portfolio(256) + + def _marked(value): + return lambda *args: np.full(np.asarray(args[0]).shape, value, dtype=float) + + kwargs = dict(nmax=256, neff=1, n=256, no_protect_names=True, + verbose=False, save_intg=True) + s.integrate_log(_marked(11.0), *NAMES, **kwargs) + assert np.all(s._warm_seed_reserve['lnL'] == 11.0) + + s.integrate_log(_marked(22.0), *NAMES, **kwargs) + reserve = s._warm_seed_reserve + assert reserve['n_retained'] == s.ntotal == 256 + assert np.all(reserve['lnL'] == 22.0), \ + 'the warm reserve contains cold-pass rows from the retained portfolio cache' + + def test_the_portfolio_reserve_is_taken_before_pruning_and_the_fair_draw(): """Order matters: taken after either step it would carry the same starved subset.""" import RIFT.integrators.mcsamplerPortfolio as mcsamplerPF @@ -568,6 +588,204 @@ def test_the_exact_total_is_absent_rather_than_wrong_without_the_prior_component assert r['ln_sum_w_finite'] is None +### +### 4b. the reject gate must not read a fair-draw artifact as evidence of lost mass +### + +def test_a_fair_drawn_lnZ_sits_above_the_retained_set_lnZ_by_a_predictable_amount(): + """WHY the reject gate had to stop reading _rvs. + + _lnZ_of_rvs forms logsumexp(w) - log(n). The fair draw resamples n rows proportional to + w, so its rows cluster at the TOP of the weight distribution and the estimate lands near + max(w) rather than mean(w) -- high by about log(n_retained / eff_samp). The gate compared + a 1-row cold reading against a 5-row warm one and read the difference as lost mass. + """ + rng = np.random.RandomState(20260811) + n = 1000 + lw = np.concatenate([[0.0], -30.0 - 5.0 * rng.rand(n - 1)]) # one dominant weight + w = np.exp(lw - lw.max()) + eff = w.sum() / w.max() + lnZ_all = np.log(np.mean(np.exp(lw))) + drawn = rng.choice(n, size=1, replace=True, p=w / w.sum()) + lnZ_fair = np.log(np.mean(np.exp(lw[drawn]))) + assert lnZ_fair > lnZ_all, 'the fair-drawn reading must be the HIGH one' + predicted = np.log(n / eff) + assert abs((lnZ_fair - lnZ_all) - predicted) < 0.5, \ + 'gap {:.2f} should track log(n/eff_samp) = {:.2f}'.format(lnZ_fair - lnZ_all, predicted) + + +def test_the_reserve_carries_what_is_needed_to_rebuild_the_weight(): + """lnZ needs the two prior components, not just lnL.""" + np.random.seed(20260811) + s = _sampler(20000) + s.integrate_log(_peaked(60.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + r = s._warm_seed_reserve + for k in ('lnL', 'log_joint_prior', 'log_joint_s_prior'): + assert k in r, 'reserve omits {}'.format(k) + assert len(r[k]) == len(r['X']), '{} is not aligned with the points'.format(k) + + +### +### 4e. the reserve's lnZ must be normalized by the DRAWS, not by what survived filtering +### + +def _reserve(lnL, lp=None, ls=None, n_retained=None, n_finite=None): + from RIFT.integrators.mcsamplerAdaptiveVolume import make_warm_seed_reserve + lnL = np.asarray(lnL, dtype=float) + X = np.zeros((len(lnL), NDIM)) + r = make_warm_seed_reserve( + X, lnL, NAMES, n_max=0, + log_joint_prior=np.zeros(len(lnL)) if lp is None else lp, + log_joint_s_prior=np.zeros(len(lnL)) if ls is None else ls) + if n_retained is not None: + r['n_retained'] = n_retained + if n_finite is not None: + r['n_finite'] = n_finite + return r + + +def test_reserve_lnZ_divides_by_the_draws_made_not_by_the_finite_survivors(): + """A -inf draw is a real draw contributing a real zero; dropping it must not renormalize. + + Averaging over the stored rows overestimates by log(n_retained/n_finite) -- ~11 nats for + a portfolio whose finite fraction on a collapsed pass is ~1e-5. + """ + from RIFT.integrators.mcsamplerAdaptiveVolume import lnZ_from_reserve + finite = np.array([10.0, 9.0, 8.0]) + n_draws = 300000 + lnL = np.concatenate([finite, np.full(n_draws - len(finite), -np.inf)]) + r = _reserve(lnL) + assert r['n_retained'] == n_draws and r['n_finite'] == len(finite) + expected = np.log(np.sum(np.exp(finite))) - np.log(n_draws) + assert abs(lnZ_from_reserve(r) - expected) < 1e-9 + + +def test_reserve_lnZ_is_unchanged_for_a_sampler_whose_rows_are_all_finite(): + """AV retains only finite rows, so the correction is exactly zero there.""" + from RIFT.integrators.mcsamplerAdaptiveVolume import lnZ_from_reserve + lw = np.array([3.0, 2.0, 1.0, 0.5]) + r = _reserve(lw) + assert r['n_retained'] == r['n_finite'] == 4 + expected = np.log(np.mean(np.exp(lw))) + assert abs(lnZ_from_reserve(r) - expected) < 1e-9 + + +def test_the_fallback_reading_accounts_for_the_uniform_cap(): + """The FALLBACK path only -- a reserve with no recorded exact total (an older writer, or + one built without the prior components). With m rows kept out of n_finite, the sum must + be scaled back up by n_finite/m. Where the exact total IS recorded it wins, because this + estimate's logarithm carries cap sampling error; see section 4g.""" + from RIFT.integrators.mcsamplerAdaptiveVolume import lnZ_from_reserve + lw = np.zeros(100) # w = 1 each, so the arithmetic is exact + r = dict(lnL=lw, log_joint_prior=np.zeros(100), log_joint_s_prior=np.zeros(100), + n_retained=10000, n_finite=1000, params_ordered=NAMES) + assert 'ln_sum_w_finite' not in r + # 100 kept of 1000 finite of 10000 drawn -> Z = (1000/100)*100*1 / 10000 = 0.1 + assert abs(lnZ_from_reserve(r) - np.log(0.1)) < 1e-9 + + +def test_the_normalization_error_does_not_cancel_between_two_passes(): + """WHY this is a gate bug and not just a wrong number: the fractions differ. + + A cold pass with a 1e-5 finite fraction and a warm pass with 1e-2, on IDENTICAL finite + weights, must give the same lnZ ordering as their draw counts imply -- not an artefact + of how much each one underflowed. + """ + from RIFT.integrators.mcsamplerAdaptiveVolume import lnZ_from_reserve + finite = np.array([10.0, 9.5, 9.0, 8.0]) + cold = _reserve(finite, n_retained=1000000, n_finite=len(finite)) + warm = _reserve(finite, n_retained=1000, n_finite=len(finite)) + gap = lnZ_from_reserve(warm) - lnZ_from_reserve(cold) + assert abs(gap - np.log(1000000.0 / 1000.0)) < 1e-9, \ + 'the draw-count difference is not being carried into lnZ' + # and the naive row-average would have made them IDENTICAL, hiding a 6.9-nat difference + naive = np.log(np.mean(np.exp(finite))) + assert abs(naive - (lnZ_from_reserve(cold) + np.log(1000000.0 / len(finite)))) < 1e-9 + + +def test_a_portfolio_pass_with_underflowed_draws_reports_a_draw_normalized_lnZ(): + """End to end on the sampler that actually holds -inf rows in _rvs.""" + from RIFT.integrators.mcsamplerAdaptiveVolume import lnZ_from_reserve + np.random.seed(20260812) + s = _portfolio(20000) + # a peak sharp enough that most draws underflow, so n_finite << n_retained + s.integrate_log(_peaked(90.0), *NAMES, nmax=200000, neff=8, n=20000, + no_protect_names=True, verbose=False, save_intg=True, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + r = s._warm_seed_reserve + assert r is not None + assert r['n_finite'] < r['n_retained'], \ + 'this target did not underflow; the test would prove nothing' + v = lnZ_from_reserve(r) + assert v is not None and np.isfinite(v) + # it is the EXACT pre-cap total over the draws made -- not an average of stored rows + assert abs(v - (r['ln_sum_w_finite'] - np.log(r['n_retained']))) < 1e-9 + # and the naive row-average, which is what reading the reserve like an _rvs would give, + # is higher. The gap is log(n_retained/n_finite) only when the cap did not bind; with a + # cap the stored rows are a subsample too, so assert the direction, not the exact size. + lw = r['lnL'] + r['log_joint_prior'] - r['log_joint_s_prior'] + lw = lw[np.isfinite(lw)] + naive = np.log(np.sum(np.exp(lw - lw.max()))) + lw.max() - np.log(len(r['lnL'])) + assert naive > v, 'the draw normalization did not lower the estimate' + if len(r['lnL']) >= r['n_finite']: # uncapped: the exact relation holds + assert abs((naive - v) - np.log(r['n_retained'] / float(r['n_finite']))) < 1e-6 + + +### +### 4g. the gate's reading must not depend on whether the cap happened to bind +### + +def test_capping_does_not_move_the_reserve_lnZ(): + """The reviewer's scenario, end to end. + + Two equally dominant rows among 200,000. Uncapped, both are in the reserve; capped at + 2,000 only the force-appended peak is certain and the other is missed ~99% of the time. + Estimating lnZ from the kept rows puts the capped reading log(2) = 0.69 nats low -- + above the 0.5-nat default reject threshold -- so a cold pass that fits under the cap + would reject an otherwise identical warm pass that does not, on subsample luck alone. + """ + from RIFT.integrators.mcsamplerAdaptiveVolume import ( + make_warm_seed_reserve, lnZ_from_reserve) + n = 200000 + lnL = np.full(n, -50.0) + lnL[[7, 9999]] = 0.0 + X = np.zeros((n, NDIM)) + z = np.zeros(n) + cold = make_warm_seed_reserve(X, lnL, NAMES, n_max=0, + log_joint_prior=z, log_joint_s_prior=z) + warm = make_warm_seed_reserve(X, lnL, NAMES, n_max=2000, + log_joint_prior=z, log_joint_s_prior=z) + assert len(warm['lnL']) < len(cold['lnL']), 'the cap did not bind; test proves nothing' + gap = abs(lnZ_from_reserve(cold) - lnZ_from_reserve(warm)) + assert gap < 1e-9, 'cap sampling error of {:.3f} nats reached the gate'.format(gap) + + +def test_the_exact_reading_is_the_draw_normalized_one(): + from RIFT.integrators.mcsamplerAdaptiveVolume import ( + make_warm_seed_reserve, lnZ_from_reserve) + n = 50000 + lnL = np.full(n, -np.inf) + lnL[:4] = np.array([1.0, 0.5, 0.25, 0.0]) + X = np.zeros((n, NDIM)) + z = np.zeros(n) + r = make_warm_seed_reserve(X, lnL, NAMES, n_max=0, + log_joint_prior=z, log_joint_s_prior=z) + expected = np.log(np.sum(np.exp(lnL[:4]))) - np.log(n) + assert abs(lnZ_from_reserve(r) - expected) < 1e-9 + + +def test_the_fallback_still_works_for_a_reserve_without_the_exact_total(): + """An older writer, or one that had no prior components at build time.""" + from RIFT.integrators.mcsamplerAdaptiveVolume import lnZ_from_reserve + lw = np.array([1.0, 0.5, 0.25]) + r = dict(lnL=lw, log_joint_prior=np.zeros(3), log_joint_s_prior=np.zeros(3), + n_retained=300, n_finite=3, params_ordered=NAMES) + expected = np.log(np.sum(np.exp(lw))) - np.log(300) + assert abs(lnZ_from_reserve(r) - expected) < 1e-9 + + ### ### 5. the ILE must actually use all of this ### @@ -601,3 +819,19 @@ def test_the_puff_width_is_configurable_and_defaults_are_the_measured_ones(): assert opt in src, 'missing {}'.format(opt) i = src.index('--sampler-l0-rescue-puff-factor') assert 'default=2.0' in src[i:i + 200], 'the measured optimum is not the default' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_reject_gate_reads_both_sides_from_the_same_record(): + with open(_ILE) as f: + src = f.read() + assert '_lnZ_of_reserve_or_rvs' in src, 'the gate still reads lnZ straight out of _rvs' + i = src.index('_evidence_of_loss = (') + block = src[max(0, i - 2000):i] + assert '_cold_src != _warm_src' in block, \ + 'nothing stops the gate comparing a retained-set lnZ against a fair-drawn one' + assert 'lnZ_from_reserve' in src, \ + 'the reserve reading still averages over stored rows instead of over the draws made' + # the cold reserve must be snapshotted before the warm pass overwrites it + assert block.index('_cold_reserve_l0') < block.index('sampler.integrate('), \ + 'the cold reserve is read after the warm pass has already replaced it' diff --git a/MonteCarloMarginalizeCode/Code/test/test_mcsampler_ensemble_log_contract.py b/MonteCarloMarginalizeCode/Code/test/test_mcsampler_ensemble_log_contract.py new file mode 100644 index 000000000..aeb7619be --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_mcsampler_ensemble_log_contract.py @@ -0,0 +1,139 @@ +import numpy as np + +from RIFT.integrators import mcsamplerEnsemble + + +def test_integrate_log_dispatches_with_log_flags(): + sampler = mcsamplerEnsemble.MCSampler() + seen = {} + + def fake_integrate(func, *args, **kwargs): + seen.update(func=func, args=args, kwargs=kwargs) + return "sentinel" + + sampler.integrate = fake_integrate + func = object() + + assert sampler.integrate_log(func, "x", nmax=17) == "sentinel" + assert seen == { + "func": func, + "args": ("x",), + "kwargs": {"nmax": 17, "use_lnL": True, "return_lnI": True}, + } + + +def test_log_weight_convergence_avoids_linear_underflow(): + rvs = {"log_weights": np.array([-10000.0, -10001.0, -10002.0])} + + expected_fraction = 1.0 / (1.0 + np.exp(-1.0) + np.exp(-2.0)) + assert mcsamplerEnsemble.convergence_test_MostSignificantPoint( + expected_fraction + 1e-12, rvs, None + ) + assert not mcsamplerEnsemble.convergence_test_MostSignificantPoint( + expected_fraction - 1e-12, rvs, None + ) + + +def test_integrate_log_exports_consistent_log_sample_fields(): + sampler = mcsamplerEnsemble.MCSampler() + sampler.add_parameter( + "x", + pdf=lambda x: np.ones_like(x) / 2.0, + prior_pdf=lambda x: np.ones_like(x) / 2.0, + left_limit=-1.0, + right_limit=1.0, + adaptive_sampling=True, + ) + + sampler.integrate_log( + lambda x: -0.5 * np.asarray(x) ** 2, + "x", + n=100, + nmax=100, + neff=1, + min_iter=1, + max_iter=1, + correlate_all_dims=True, + n_comp=1, + ) + + np.testing.assert_allclose( + sampler._rvs["log_weights"], + sampler._rvs["log_integrand"] + + sampler._rvs["log_joint_prior"] + - sampler._rvs["log_joint_s_prior"], + ) + np.testing.assert_array_equal( + sampler._rvs["integrand"], sampler._rvs["log_integrand"] + ) + + +def test_linear_integration_clears_log_fields_when_sampler_is_reused(): + sampler = mcsamplerEnsemble.MCSampler() + sampler.add_parameter( + "x", + pdf=lambda x: np.ones_like(x) / 2.0, + prior_pdf=lambda x: np.ones_like(x) / 2.0, + left_limit=-1.0, + right_limit=1.0, + adaptive_sampling=True, + ) + integration_options = { + "n": 100, + "nmax": 100, + "neff": 1, + "min_iter": 1, + "max_iter": 1, + "correlate_all_dims": True, + "n_comp": 1, + } + + sampler.integrate_log( + lambda x: -0.5 * np.asarray(x) ** 2, + "x", + **integration_options, + ) + assert "log_weights" in sampler._rvs + + sampler.integrate( + lambda x: np.exp(-0.5 * np.asarray(x) ** 2), + "x", + **integration_options, + ) + + log_fields = { + "log_integrand", + "log_joint_prior", + "log_joint_s_prior", + "log_weights", + } + assert log_fields.isdisjoint(sampler._rvs) + + weights = ( + sampler._rvs["integrand"] + * sampler._rvs["joint_prior"] + / sampler._rvs["joint_s_prior"] + ) + expected_fraction = np.max(weights) / np.sum(weights) + assert mcsamplerEnsemble.convergence_test_MostSignificantPoint( + expected_fraction + 1e-12, sampler._rvs, None + ) + assert not mcsamplerEnsemble.convergence_test_MostSignificantPoint( + expected_fraction - 1e-12, sampler._rvs, None + ) + + +def test_normal_subintegrals_accept_log_weights(monkeypatch): + # Eight equal-mass chunks at a scale where conversion to linear weights + # would underflow. Stub only the normality statistic; the test exercises + # the logsumexp chunk reduction and its relative-error gate. + monkeypatch.setattr( + mcsamplerEnsemble.stats, + "normaltest", + lambda values: (0.0, 0.5), + ) + rvs = {"log_weights": np.full(80, -10000.0)} + + assert mcsamplerEnsemble.convergence_test_NormalSubIntegrals( + 8, 0.01, 1e-12, rvs, None + ) diff --git a/MonteCarloMarginalizeCode/Code/test/test_nal_io.py b/MonteCarloMarginalizeCode/Code/test/test_nal_io.py new file mode 100644 index 000000000..1284247ad --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_nal_io.py @@ -0,0 +1,855 @@ +"""Tests for RIFT.interpolators.nal_io -- the generic NAL reader/evaluator. + +Everything is checked against an answer known analytically or by brute force; nothing is +self-referential. +""" +import json +import os +import sys + +import numpy as np +import pytest + +# Import by PATH, not as RIFT.interpolators.nal_io: importing the package pulls in lalsimutils +# and hence glue/lal, which this module does not need. nal_io is deliberately pure-numpy (h5py +# only for the optional gwalk view), and loading it standalone here both keeps the test runnable +# in a bare environment and asserts that independence. +import importlib.util as _ilu # noqa: E402 +_p = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", + "RIFT", "interpolators", "nal_io.py") +_spec = _ilu.spec_from_file_location("nal_io", os.path.abspath(_p)) +nal_io = _ilu.module_from_spec(_spec) +_spec.loader.exec_module(nal_io) + + +def _make(d=4, seed=0): + rng = np.random.default_rng(seed) + A = rng.standard_normal((d, d)) + G = A @ A.T + d * np.eye(d) + mu = rng.standard_normal(d) + return mu, G + + +def _declare_run(monkeypatch, frame="detector", chart="NAL:aligned"): + """Declare the frame and chart the RUN samples in, as an ini-less deployment would.""" + monkeypatch.setenv("RIFT_NAL_SAMPLER_FRAME", frame) + monkeypatch.setenv("RIFT_NAL_SAMPLER_CHART", chart) + + +def test_lnL_matches_the_quadratic_form(): + mu, G = _make() + n = nal_io.NAL(mu, G, ["mc", "delta_mc", "xi", "chiMinus"], lnL_peak=7.5) + X = mu + np.random.default_rng(1).standard_normal((50, 4)) * 0.3 + want = 7.5 - 0.5 * np.einsum("ij,jk,ik->i", X - mu, G, X - mu) + assert np.allclose(n.lnL(X), want, atol=1e-12) + assert np.isclose(n.lnL(mu)[0], 7.5) + + +def test_marginal_is_the_schur_complement_not_the_conditional(): + """The distinction that is easy to get backwards: marginal = sub-block of Gamma^-1.""" + mu, G = _make(d=5, seed=3) + n = nal_io.NAL(mu, G, list("abcde")) + m = n.marginal(["a", "b"]) + Sig = np.linalg.inv(G) + assert np.allclose(m.cov(), Sig[np.ix_([0, 1], [0, 1])], atol=1e-12) + conditional = np.linalg.inv(G[np.ix_([0, 1], [0, 1])]) + # they must genuinely differ, else the test proves nothing + assert not np.allclose(m.cov(), conditional, atol=1e-6) + # and the marginal must be the WIDER of the two + assert np.all(np.diag(m.cov()) > np.diag(conditional)) + + +def test_bounds_zero_the_likelihood_outside(): + mu, G = _make(d=2, seed=5) + b = np.stack([mu - 0.5, mu + 0.5], 1) + n = nal_io.NAL(mu, G, ["mc", "eta"], bounds=b) + assert np.isfinite(n.lnL(mu)[0]) + assert n.lnL(mu + 10.0)[0] == -np.inf + + +def test_renormalization_is_not_a_product_of_1d_marginals(): + """A correlated truncated Gaussian: the factorised mass is biased, the MC one is not.""" + mu = np.zeros(3) + rho = 0.9 + C = np.full((3, 3), rho) + (1 - rho) * np.eye(3) + n = nal_io.NAL(mu, np.linalg.inv(C), list("abc"), + bounds=np.stack([-np.ones(3), np.ones(3)], 1)) + logm = n.log_mass(seed=2) + from scipy.stats import norm + factorised = np.log(np.prod([norm.cdf(1) - norm.cdf(-1)] * 3)) + assert np.exp(logm) > np.exp(factorised) * 1.2 # correlation concentrates mass in the box + # and the MC mass must agree with an independent brute-force estimate + G = np.random.default_rng(9).multivariate_normal(mu, C, 400000) + brute = np.log(np.all(np.abs(G) <= 1, axis=1).mean()) + assert abs(logm - brute) < 0.02 + + +def test_truncation_constant_is_computed_once_and_reused(): + """`renormalize=True` must not re-run the Monte Carlo on every likelihood call.""" + mu = np.zeros(2) # mass in the box ~ 0.466, comfortably < 1 + n = nal_io.NAL(mu, np.eye(2), ["mc", "eta"], bounds=np.stack([mu - 1.0, mu + 1.0], 1)) + + calls = {"n": 0} + real = nal_io._rng + + def counting_rng(seed): + calls["n"] += 1 + return real(seed) + + nal_io._rng = counting_rng + try: + first = n.log_mass() + for _ in range(5): + n.lnL(mu, renormalize=True) + assert calls["n"] == 1, "truncation mass recomputed inside lnL: %d draws sets" % calls["n"] + finally: + nal_io._rng = real + assert n.log_mass() == first < 0.0 + + +def test_unresolvable_truncation_mass_raises_rather_than_guessing(): + """A mass too small to estimate must fail, not come back floored at 1/n. + + 1-D standard normal on [6, 7]: true mass ~1e-9, so no affordable number of draws lands in the + box. The old floor returned ~5e-6 -- an 8.5 nat error presented as a measurement. + """ + n = nal_io.NAL([0.0], [[1.0]], ["mc"], bounds=[[6.0, 7.0]]) + with pytest.raises(ValueError, match="unresolved"): + n.log_mass(max_draws=200000, batch=100000) + with pytest.raises(ValueError, match="unresolved"): + n.lnL(np.array([[6.5]]), renormalize=True) + # ... unless the artifact declares the value it knows + n.meta["log_truncation_mass"] = -20.6 + assert np.isclose(n.log_mass(), -20.6) + assert np.isclose(n.lnL(np.array([[6.5]]), renormalize=True)[0], -0.5 * 6.5 ** 2 + 20.6) + + +def test_log_mass_is_accurate_for_a_small_but_reachable_mass(): + """Against the analytic answer, where the old fixed-n floor would have been consulted.""" + from scipy.stats import norm + n = nal_io.NAL([0.0], [[1.0]], ["mc"], bounds=[[3.0, 4.0]]) + want = np.log(norm.cdf(4.0) - norm.cdf(3.0)) # ~ -6.8 + got = n.log_mass(rel_tol=0.02, max_draws=8000000) + assert abs(got - want) < 0.1 + + +def test_roundtrip_artifact(tmp_path): + mu, G = _make() + names = ["mc", "delta_mc", "xi", "chiMinus"] + base = str(tmp_path / "ev1") + np.savez(base + ".npz", theta_star=mu, gamma=G, + bounds=np.stack([mu - 5, mu + 5], 1)) + json.dump({"coord_names": names, "lnL_peak": 3.25, "chart": "NAL:aligned", + "frame": "detector"}, open(base + ".meta.json", "w")) + n = nal_io.load_nal(base + ".npz") + assert n.coord_names == names and np.isclose(n.lnL_peak, 3.25) + assert np.allclose(n.lnL(mu)[0], 3.25) + + +def test_meta_without_coord_names_is_rejected(tmp_path): + """A NAL with no declared chart is uninterpretable and must not load silently.""" + mu, G = _make(d=2) + base = str(tmp_path / "bad") + np.savez(base + ".npz", theta_star=mu, gamma=G) + json.dump({"lnL_peak": 0.0}, open(base + ".meta.json", "w")) + with pytest.raises(KeyError): + nal_io.load_nal(base + ".npz") + + +def test_plugin_hook_contract(tmp_path, monkeypatch): + """Exercise exactly what CIP/EOSPosterior do: prepare(config, coords) then lnL(*x).""" + mu, G = _make(d=2, seed=11) + names = ["mc", "delta_mc"] + for i in range(2): # two events -> contributions ADD + base = str(tmp_path / ("ev%d" % i)) + np.savez(base + ".npz", theta_star=mu, gamma=G) + json.dump({"coord_names": names, "lnL_peak": 1.0, "chart": "NAL:aligned", + "frame": "detector"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", str(tmp_path / "*.npz")) + _declare_run(monkeypatch) + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=names) + out = nal_io.nal_lnL(np.array([mu[0]]), np.array([mu[1]])) + # 1.0 per event, summed -- then centred by the summed peak, so the peak sits at 0 + assert np.isclose(nal_io.nal_lnL_offset(), 2.0) + assert np.isclose(out[0], 0.0) + + +def test_plugin_derives_delta_mc_from_eta(tmp_path, monkeypatch): + """Sampler in (mc, eta); artifact chart in (mc, delta_mc). Must convert, not fail.""" + mu = np.array([30.0, 0.3]) # delta_mc = 0.3 -> eta = 0.2275 + G = np.diag([1.0, 4.0]) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=G) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch) + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "eta"]) + eta = 0.25 * (1 - 0.3 ** 2) + out = nal_io.nal_lnL(np.array([30.0]), np.array([eta])) + assert np.isclose(out[0], 0.0, atol=1e-10) # lands exactly on the peak + + +def test_unbuildable_coordinate_raises_named_error(tmp_path, monkeypatch): + mu, G = _make(d=2, seed=2) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=G) + json.dump({"coord_names": ["mc", "s1x_bar"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch) + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "eta"]) + with pytest.raises(KeyError, match="s1x_bar"): + nal_io.nal_lnL(np.array([30.0]), np.array([0.2])) + + +def test_wrong_basis_from_the_driver_would_evaluate_at_the_wrong_point(tmp_path, monkeypatch): + """`coords` must be the driver's SAMPLING basis, which is not its FIT basis. + + Concretely, `--parameter mc --parameter eta --parameter-implied delta_mc + --parameter-nofit s1z` gives fit coord_names = [mc, eta, delta_mc] but sampling + low_level_coord_names = [mc, eta, s1z], and the sampler calls the plugin with one array per + SAMPLING coordinate. Declaring the fit basis zips 'delta_mc' onto the s1z array: same length, + no error, silently the wrong point. This test pins both halves -- the right basis lands on + the peak, the wrong one does not -- so the failure mode stays visible rather than numerical. + """ + mu = np.array([30.0, 0.3]) # chart is (mc, delta_mc) + G = np.diag([1.0, 4.0]) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=G) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch) + + mc, eta, s1z = 30.0, 0.25 * (1 - 0.3 ** 2), -0.4 # delta_mc = 0.3 exactly + x = [np.array([mc]), np.array([eta]), np.array([s1z])] + + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "eta", "s1z"]) # sampling basis + assert np.isclose(nal_io.nal_lnL(*x)[0], 0.0, atol=1e-10) + + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "eta", "delta_mc"]) # fit basis: WRONG + wrong = nal_io.nal_lnL(*x)[0] + # s1z has been read as delta_mc: lnL = -1/2 * 4 * (s1z - 0.3)^2 + assert np.isclose(wrong, -0.5 * 4.0 * (s1z - 0.3) ** 2) + assert wrong < -0.5 # and it is nowhere near the peak + + +def test_environment_only_configuration_will_not_guess_the_basis(tmp_path, monkeypatch): + """RIFT_NAL_ARTIFACTS alone must fail closed: the incoming arrays have no names. + + The dangerous case has the RIGHT number of arrays, so no dimension check can catch it: a + sampler in (mc, eta) against an artifact in (mc, delta_mc) would evaluate eta as delta_mc. + """ + mu = np.array([30.0, 0.3]) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=np.diag([1.0, 4.0])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + monkeypatch.delenv("RIFT_NAL_SAMPLER_COORDS", raising=False) + _declare_run(monkeypatch) + + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + eta = 0.25 * (1 - 0.3 ** 2) + with pytest.raises(ValueError, match="sampling basis is unknown"): + nal_io.nal_lnL(np.array([30.0]), np.array([eta])) + + # declaring the basis explicitly is the supported way out, and it then converts eta -> delta_mc + monkeypatch.setenv("RIFT_NAL_SAMPLER_COORDS", "mc, eta") + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + out = nal_io.nal_lnL(np.array([30.0]), np.array([eta])) + assert np.isclose(out[0], 0.0, atol=1e-10) # lands on the peak, not off it + + +def test_driver_coords_override_the_environment_declaration(tmp_path, monkeypatch): + """The driver knows the real sampling basis; a stale environment value must not win.""" + mu = np.array([30.0, 0.3]) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=np.diag([1.0, 4.0])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + monkeypatch.setenv("RIFT_NAL_SAMPLER_COORDS", "mc,delta_mc") + _declare_run(monkeypatch) + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "eta"]) + assert nal_io._STATE["coords"] == ["mc", "eta"] + + +def test_wrong_number_of_arrays_is_rejected(tmp_path, monkeypatch): + mu = np.array([30.0, 0.3]) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=np.diag([1.0, 4.0])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch) + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "eta"]) + with pytest.raises(ValueError, match="sampling basis"): + nal_io.nal_lnL(np.array([30.0])) + + +# ------------------------------------------------------------------- summing artifacts / charts + +def _nal(meta, seed=0): + mu, G = _make(d=2, seed=seed) + return nal_io.NAL(mu, G, ["mc", "delta_mc"], meta=meta) + + +def test_set_refuses_artifacts_in_different_frames(): + """Same coordinate NAMES, different meanings: detector-frame mc is not source-frame mc.""" + with pytest.raises(ValueError, match="frames"): + nal_io.NALSet([_nal({"frame": "detector"}), + _nal({"frame": "source", "cosmology": {"name": "Planck15"}, + "d_prior": {"name": "cosmo_sourceframe"}}, seed=1)]) + + +def test_set_refuses_undeclared_frame(): + """Fail closed: an artifact that will not state its frame cannot be shown compatible.""" + with pytest.raises(ValueError, match="frame"): + nal_io.NALSet([_nal({"frame": "detector"}), _nal({}, seed=1)]) + + +def test_set_refuses_mismatched_cosmology_or_distance_prior(): + a = {"frame": "source", "chart": "NAL:aligned", "cosmology": {"name": "Planck15"}, + "d_prior": {"name": "cosmo_sourceframe", "d_max": 10000.0}} + b = dict(a, cosmology={"name": "Planck18"}) + with pytest.raises(ValueError, match="cosmology"): + nal_io.NALSet([_nal(a), _nal(b, seed=1)]) + c = dict(a, d_prior={"name": "uniform_comoving", "d_max": 10000.0}) + with pytest.raises(ValueError, match="d_prior"): + nal_io.NALSet([_nal(a), _nal(c, seed=1)]) + + +def test_set_requires_cosmology_for_source_frame_artifacts(): + m = {"frame": "source", "chart": "NAL:aligned"} + with pytest.raises(ValueError, match="cosmology"): + nal_io.NALSet([_nal(m), _nal(m, seed=1)]) + + +def test_set_refuses_mismatched_charts(): + with pytest.raises(ValueError, match="chart"): + nal_io.NALSet([_nal({"frame": "detector", "chart": "NAL:aligned"}), + _nal({"frame": "detector", "chart": "NAL:precessing"}, seed=1)]) + + +def test_set_refuses_artifacts_that_all_omit_the_chart(): + """Silence from every artifact is not agreement. + + Same coord_names, same frame, no chart anywhere: nothing establishes that the two were built + in the same coordinate CONVENTIONS (spin basis, mass pairing, angle reference), and the sum + would be well-defined arithmetic over two different meanings of theta. `write_nal(chart=None)` + produces exactly these, so the all-undeclared case is the one that actually occurs -- and it + is the case that a "declared values must match" check would wave through. + """ + with pytest.raises(ValueError, match="chart"): + nal_io.NALSet([_nal({"frame": "detector"}), _nal({"frame": "detector"}, seed=1)]) + # partially declared is no better: one artifact still cannot be shown to agree + with pytest.raises(ValueError, match="chart"): + nal_io.NALSet([_nal({"frame": "detector", "chart": "NAL:aligned"}), + _nal({"frame": "detector"}, seed=1)]) + # nor is a declaration that says nothing + with pytest.raises(ValueError, match="chart"): + nal_io.NALSet([_nal({"frame": "detector", "chart": ""}), + _nal({"frame": "detector", "chart": ""}, seed=1)]) + # and the escape hatch for a caller who has established equivalence by other means still works + nal_io.NALSet([_nal({"frame": "detector"}), _nal({"frame": "detector"}, seed=1)], + require_compatible=False) + + +def test_written_artifacts_without_a_chart_cannot_be_summed(tmp_path): + """End to end: write_nal(chart=None) is loadable and usable alone, but not addable.""" + mu, G = _make(d=2) + paths = [] + for i in range(2): + base = str(tmp_path / ("ev%d" % i)) + nal_io.write_nal(base, nal_io.NAL(mu, G, ["mc", "delta_mc"]), frame="detector") + paths.append(base) + loaded = [nal_io.load_nal(p + ".npz") for p in paths] + nal_io.NALSet([loaded[0]]) # alone: fine, nothing is being added + with pytest.raises(ValueError, match="chart"): + nal_io.NALSet(loaded) + for p in paths: # ... and declaring one fixes it + nal_io.write_nal(p, nal_io.NAL(mu, G, ["mc", "delta_mc"]), chart="NAL:aligned", + frame="detector") + nal_io.NALSet([nal_io.load_nal(p + ".npz") for p in paths]) + + +def test_set_accepts_matching_metadata_and_a_lone_artifact(): + m = {"frame": "detector", "chart": "NAL:aligned"} + s = nal_io.NALSet([_nal(m), _nal(m, seed=1)]) + assert s.coord_names == ["mc", "delta_mc"] + # a single artifact is never checked: nothing is being added to it + assert nal_io.NALSet([_nal({})]).coord_names == ["mc", "delta_mc"] + # dict ordering is not a difference + nal_io.NALSet([_nal({"frame": "source", "chart": "NAL:aligned", + "cosmology": {"name": "Planck15", "h": 0.679}, + "d_prior": {"name": "p", "d_max": 1.0}}), + _nal({"frame": "source", "chart": "NAL:aligned", + "cosmology": {"h": 0.679, "name": "Planck15"}, + "d_prior": {"d_max": 1.0, "name": "p"}}, seed=1)]) + + +# --------------------------------------------------------- the artifacts vs the RUN's own chart + +def test_sampler_check_rejects_an_undeclared_or_mismatched_run_frame(): + """The set check compares artifacts with each other; it says nothing about the sampler.""" + ok = {"frame": "detector", "chart": "NAL:aligned"} + with pytest.raises(ValueError, match="sampling frame is undeclared"): + nal_io.check_sampler_compatible([_nal(ok)], "", "NAL:aligned") + with pytest.raises(ValueError, match="frame"): + nal_io.check_sampler_compatible([_nal(ok)], "source", "NAL:aligned") + with pytest.raises(ValueError, match="sampler_frame"): + nal_io.check_sampler_compatible([_nal(ok)], "geocenter", "NAL:aligned") + nal_io.check_sampler_compatible([_nal(ok)], "detector", "NAL:aligned") + + +def test_sampler_check_rejects_an_undeclared_or_mismatched_run_chart(): + ok = {"frame": "detector", "chart": "NAL:aligned"} + with pytest.raises(ValueError, match="sampling chart is undeclared"): + nal_io.check_sampler_compatible([_nal(ok)], "detector", "") + with pytest.raises(ValueError, match="chart"): + nal_io.check_sampler_compatible([_nal(ok)], "detector", "NAL:precessing") + with pytest.raises(ValueError, match="chart"): # artifact declares nothing + nal_io.check_sampler_compatible([_nal({"frame": "detector"})], "detector", "NAL:aligned") + + +def test_a_lone_source_frame_artifact_is_still_checked_against_the_run(tmp_path, monkeypatch): + """The hole the set check cannot cover: one artifact is never compared with anything. + + Source- and detector-frame charts wear the same coordinate names, so a source-frame NAL fed + the driver's default detector-frame samples raises nothing on names or array count. + """ + mu = np.array([30.0, 0.3]) + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=np.diag([1.0, 4.0])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "source", + "chart": "NAL:aligned", "cosmology": {"name": "Planck15"}, + "d_prior": {"name": "cosmo_sourceframe"}}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch, frame="detector") # the driver's default basis + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + with pytest.raises(ValueError, match="frame"): + nal_io.prepare_nal_lnL(config=None, coords=["mc", "delta_mc"]) + # declaring the run honestly is what makes it usable + _declare_run(monkeypatch, frame="source") + nal_io.prepare_nal_lnL(config=None, coords=["mc", "delta_mc"]) + assert np.isclose(nal_io.nal_lnL(np.array([30.0]), np.array([0.3]))[0], 0.0, atol=1e-10) + + +def test_contribution_is_centred_so_the_drivers_exponentiation_cannot_overflow(tmp_path, + monkeypatch): + """Both drivers' DEFAULT path is likelihood_function(*x) * np.exp(supplemental(*x)). + + float64 exp overflows above ~709, so a loud but perfectly valid artifact (lnL_peak ~ SNR^2/2) + would contribute inf for every sample. The contribution is centred by the summed peak; the + constant is recoverable from nal_lnL_offset(). + """ + mu = np.array([30.0, 0.3]) + loud = 3386.0 # a real SNR~82 event's lnL_peak + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=np.diag([1.0, 4.0])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": loud, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch) + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "delta_mc"]) + + X = [np.array([30.0, 30.2, 29.5]), np.array([0.3, 0.35, 0.2])] + out = nal_io.nal_lnL(*X) + assert np.isclose(nal_io.nal_lnL_offset(), loud) + assert np.all(out <= 0.0) # never positive, so exp never overflows + assert np.all(np.isfinite(np.exp(out))) + # SHAPE is untouched: only a constant has moved + assert np.allclose(out - out[0], [0.0, -0.5 * (0.2 ** 2 + 4 * 0.05 ** 2), + -0.5 * (0.5 ** 2 + 4 * 0.1 ** 2)]) + + +# ------------------------------------------------------------------- validity of the fit itself + +def test_indefinite_gamma_is_rejected(): + """A negative eigenvalue is a saddle, and it breaks the plugin's overflow guard. + + lnL then INCREASES away from mu along that direction, so lnL_peak is not the peak and the + constant `_peak_offset` subtracts is no longer an upper bound on the contribution -- which is + the entire mechanism keeping the drivers' np.exp(supplemental) finite. Demonstrated here + rather than asserted: the quadratic form really does exceed its nominal peak. + """ + G = np.array([[1.0, 0.0], [0.0, -1.0]]) + off_peak = 5.0 - 0.5 * np.array([0.0, 3.0]) @ G @ np.array([0.0, 3.0]) + assert off_peak > 5.0 # 9.5 nat ABOVE the declared peak + with pytest.raises(ValueError, match="positive definite"): + nal_io.NAL(np.zeros(2), G, ["mc", "delta_mc"], lnL_peak=5.0) + + +def test_singular_gamma_is_rejected_explicitly(): + """A flat direction is not a normalizable likelihood, and Gamma^-1 would be garbage, not an error.""" + G = np.array([[1.0, 0.0], [0.0, 0.0]]) + with pytest.raises(ValueError, match="singular"): + nal_io.NAL(np.zeros(2), G, ["mc", "delta_mc"]) + # numerically singular counts too: inverting this loses every significant digit + G = np.diag([1.0, 1e-15]) + with pytest.raises(ValueError, match="singular"): + nal_io.NAL(np.zeros(2), G, ["mc", "delta_mc"]) + + +def test_non_finite_fit_is_rejected(): + with pytest.raises(ValueError, match="non-finite"): + nal_io.NAL(np.zeros(2), np.array([[1.0, np.nan], [np.nan, 1.0]]), ["mc", "delta_mc"]) + with pytest.raises(ValueError, match="non-finite"): + nal_io.NAL([np.inf, 0.0], np.eye(2), ["mc", "delta_mc"]) + + +def test_ill_conditioned_but_valid_fit_is_accepted(): + """The check must not reject the fits this module exists for. + + Mass and spin curvatures differ by many orders of magnitude in a real chart -- a condition + number of 1e8 is ordinary, not a defect -- so the threshold is relative and tight. + """ + n = nal_io.NAL(np.zeros(2), np.diag([1e6, 1e-2]), ["mc", "xi"]) + assert np.allclose(n.cov(), np.diag([1e-6, 1e2])) + # and it is invariant under an overall rescaling of lnL, which changes no eigenvalue RATIO + nal_io.NAL(np.zeros(2), np.diag([1e6, 1e-2]) * 1e-9, ["mc", "xi"]) + + +def test_invalid_gamma_is_caught_when_an_artifact_is_LOADED(tmp_path): + """The check has to hold on the consumer side: the artifact was fitted somewhere else.""" + base = str(tmp_path / "bad") + np.savez(base + ".npz", theta_star=np.zeros(2), gamma=np.diag([1.0, -1.0])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": 0.0, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + with pytest.raises(ValueError, match="positive definite"): + nal_io.load_nal(base + ".npz") + + +# ------------------------------------------------------------------------ bounded marginalization + +def _correlated(rho=0.9, d=2): + C = np.full((d, d), rho) + (1 - rho) * np.eye(d) + return np.zeros(d), np.linalg.inv(C) + + +def test_marginal_rejects_a_bounded_correlated_nuisance(): + """Integrating out a truncated, correlated coordinate is not a covariance sub-block. + + The exact marginal picks up the mass of the dropped coordinate's CONDITIONAL distribution + inside its bounds, whose mean moves with the retained coordinate -- a theta-dependent factor, + so the untruncated answer has the wrong SHAPE, not just the wrong normalisation. + """ + mu, G = _correlated() + n = nal_io.NAL(mu, G, ["mc", "delta_mc"], bounds=np.stack([mu - 0.5, mu + 0.5], 1)) + with pytest.raises(ValueError, match="delta_mc"): + n.marginal(["mc"]) + # and the escape hatch still gives the untruncated sub-block + m = n.marginal(["mc"], ignore_truncation=True) + assert np.allclose(m.cov(), np.linalg.inv(G)[:1, :1]) + + +def test_marginal_allowed_when_the_dropped_bound_does_not_bite(): + """Bounds far outside the fit are a formality: the truncation factor is 1 to ~1e-6.""" + mu, G = _correlated() + wide = np.stack([mu - 50.0, mu + 50.0], 1) + n = nal_io.NAL(mu, G, ["mc", "delta_mc"], bounds=wide) + m = n.marginal(["mc"]) + assert np.allclose(m.cov(), np.linalg.inv(G)[:1, :1]) + assert np.allclose(m.bounds, wide[:1]) + + +def test_marginal_allowed_when_the_dropped_coordinate_is_uncorrelated(): + """No correlation -> the truncation factor is a constant, which only shifts lnL_peak.""" + mu = np.zeros(2) + n = nal_io.NAL(mu, np.diag([1.0, 4.0]), ["mc", "delta_mc"], + bounds=np.stack([mu - 0.1, mu + 0.1], 1)) + m = n.marginal(["mc"]) + assert np.allclose(m.gamma, [[1.0]]) + + +def test_marginal_peak_matches_brute_force_integration(): + """The marginal is an INTEGRAL: check it against one, at several points. + + Compares the returned marginal's absolute lnL with log of the numerically integrated joint + likelihood over the dropped coordinate. This pins the constant AND the shape at once; keeping + the original lnL_peak fails it by 0.0885 nat here, uniformly in x. + """ + C = np.array([[1.0, 0.9], [0.9, 1.0]]) + n = nal_io.NAL(np.zeros(2), np.linalg.inv(C), ["mc", "delta_mc"], lnL_peak=4.0) + m = n.marginal(["mc"]) + y = np.linspace(-14.0, 14.0, 280001) # dy = 1e-4, ~14 sigma each way + for x in (0.0, 0.4, -1.1): + grid = np.stack([np.full_like(y, x), y], 1) + want = np.log(np.sum(np.exp(n.lnL(grid))) * (y[1] - y[0])) + assert np.isclose(m.lnL(np.array([[x]]))[0], want, atol=1e-6) + # and the shape-only projection is exactly the old behaviour, for a caller who normalises + assert np.isclose(n.marginal(["mc"], shape_only=True).lnL_peak, 4.0) + + +def test_marginal_constant_for_one_independent_standard_normal(): + """The textbook case: dropping one independent unit-variance coordinate adds 0.5 ln(2 pi).""" + n = nal_io.NAL(np.zeros(3), np.eye(3), ["mc", "delta_mc", "xi"], lnL_peak=2.0) + assert np.isclose(n.marginal(["mc", "delta_mc"]).lnL_peak, 2.0 + 0.5 * np.log(2 * np.pi)) + assert np.isclose(n.marginal(["mc"]).lnL_peak, 2.0 + np.log(2 * np.pi)) + # the constant uses Gamma_BB -- the CONDITIONAL precision of the dropped block, a sub-block of + # Gamma -- not a sub-block of Sigma. They differ as soon as anything is correlated. + C = np.array([[1.0, 0.9], [0.9, 1.0]]) + n2 = nal_io.NAL(np.zeros(2), np.linalg.inv(C), ["mc", "delta_mc"]) + gamma_bb = np.linalg.inv(C)[1, 1] + assert np.isclose(n2.marginal(["mc"]).lnL_peak, + 0.5 * np.log(2 * np.pi) - 0.5 * np.log(gamma_bb)) + assert not np.isclose(gamma_bb, 1.0 / C[1, 1]) # the two candidates really do differ + + +def test_marginal_constant_includes_the_enclosed_mass_of_a_bounded_nuisance(): + """A bounded (but uncorrelated) dropped coordinate contributes its enclosed mass as well. + + Allowed because the factor is constant -- but constant is not the same as absent, and here it + is 0.38 nat. Checked against the analytic 1-D normal mass. + """ + from scipy.stats import norm + mu = np.zeros(2) + bounds = np.array([[-50.0, 50.0], [-1.0, 1.0]]) # retained wide open, nuisance at 1 sigma + n = nal_io.NAL(mu, np.eye(2), ["mc", "delta_mc"], bounds=bounds) + want = 0.5 * np.log(2 * np.pi) + np.log(norm.cdf(1.0) - norm.cdf(-1.0)) + assert abs(n.marginal(["mc"]).lnL_peak - want) < 0.02 # MC mass, 1% relative by default + # ignore_truncation asks for the untruncated object, so it gets the untruncated constant + assert np.isclose(n.marginal(["mc"], ignore_truncation=True).lnL_peak, + 0.5 * np.log(2 * np.pi)) + + +def test_unbounded_marginal_is_unaffected(): + """The plain (bounds-free) marginal must keep working exactly as before.""" + mu, G = _make(d=5, seed=3) + n = nal_io.NAL(mu, G, list("abcde")) + assert np.allclose(n.marginal(["a", "b"]).cov(), + np.linalg.inv(G)[np.ix_([0, 1], [0, 1])], atol=1e-12) + + +# --------------------------------------------------------------------------- writer / provenance + +def test_write_read_roundtrip_preserves_everything(tmp_path): + mu, G = _make(d=3, seed=7) + names = ["mc", "delta_mc", "xi"] + n = nal_io.NAL(mu, G, names, lnL_peak=12.5, + bounds=np.stack([mu - 3, mu + 3], 1)) + src = tmp_path / "all.net" + src.write_text("dummy grid\n") + base = str(tmp_path / "ev") + nal_io.write_nal(base, n, chart="NAL:aligned", frame="detector", + parents=[str(src)], run_id="unit-test", + validation={"chi2_red": 1.02}) + back = nal_io.load_nal(base + ".npz") + assert back.coord_names == names + assert np.allclose(back.mu, mu) and np.allclose(back.gamma, G) + assert np.isclose(back.lnL_peak, 12.5) + X = mu + 0.1 + assert np.allclose(back.lnL(X), n.lnL(X), atol=1e-12) + meta = json.load(open(base + ".meta.json")) + assert meta["schema"] == nal_io.SCHEMA_VERSION and meta["chart"] == "NAL:aligned" + assert meta["parents"] and len(meta["parents"][0]["sha256"]) == 64 + assert meta["validation"]["chi2_red"] == 1.02 + + +def test_frame_invariant_rejects_source_frame_with_distance(): + """An artifact may carry u_d OR source-frame masses, never both.""" + with pytest.raises(ValueError, match="u_d"): + nal_io.check_frame_invariant(["mc", "delta_mc", "u_d"], "source", + cosmology={"name": "Planck15"}, + d_prior={"name": "cosmo_sourceframe"}) + + +def test_frame_invariant_requires_cosmology_and_prior_for_source_frame(): + with pytest.raises(ValueError, match="cosmology"): + nal_io.check_frame_invariant(["mc", "delta_mc"], "source") + with pytest.raises(ValueError, match="distance prior"): + nal_io.check_frame_invariant(["mc", "delta_mc"], "source", + cosmology={"name": "Planck15"}) + # fully declared: fine + nal_io.check_frame_invariant(["mc", "delta_mc"], "source", + cosmology={"name": "Planck15"}, + d_prior={"name": "cosmo_sourceframe", + "d_min": 1.0, "d_max": 10000.0}) + + +def test_frame_invariant_rejects_bad_frame_name(): + with pytest.raises(ValueError, match="frame"): + nal_io.check_frame_invariant(["mc"], "det") + + +def test_write_refuses_undeclared_source_frame(tmp_path): + """The writer must not emit an artifact that cannot state its own frame honestly.""" + mu, G = _make(d=2, seed=8) + n = nal_io.NAL(mu, G, ["mc", "delta_mc"]) + with pytest.raises(ValueError): + nal_io.write_nal(str(tmp_path / "bad"), n, frame="source") + assert not os.path.exists(str(tmp_path / "bad.npz")) + + +def test_frame_invariant_rejects_source_frame_carrying_dist(tmp_path): + """`dist` is a distance coordinate exactly as much as `u_d` is -- _derive interconverts them. + + Checking only 'u_d' let write_nal emit the artifact this invariant exists to forbid: masses + declared source-frame, a distance prior recorded as already integrated out, and the distance + still sitting in the chart. + """ + with pytest.raises(ValueError, match="dist"): + nal_io.check_frame_invariant(["mc", "delta_mc", "dist"], "source", + cosmology={"name": "Planck15"}, + d_prior={"name": "cosmo_sourceframe"}) + # detector-frame is where a distance coordinate belongs, under either spelling + nal_io.check_frame_invariant(["mc", "delta_mc", "dist"], "detector") + # ... and the writer must refuse it too, without leaving a file behind + mu, G = _make(d=3, seed=12) + n = nal_io.NAL(mu, G, ["mc", "delta_mc", "dist"]) + with pytest.raises(ValueError, match="dist"): + nal_io.write_nal(str(tmp_path / "bad"), n, chart="NAL:aligned", frame="source", + cosmology={"name": "Planck15"}, + d_prior={"name": "cosmo_sourceframe"}) + assert not os.path.exists(str(tmp_path / "bad.npz")) + assert not os.path.exists(str(tmp_path / "bad.meta.json")) + + +def test_extra_may_not_overwrite_validated_metadata(tmp_path): + """`extra` is applied after check_frame_invariant, so it must not reach the checked keys. + + Otherwise frame='detector' is what gets validated and frame='source' is what gets recorded: + a source-frame artifact with no cosmology, no distance prior, and a distance coordinate. + """ + mu, G = _make(d=3, seed=13) + n = nal_io.NAL(mu, G, ["mc", "delta_mc", "u_d"]) + base = str(tmp_path / "ev") + with pytest.raises(ValueError, match="frame"): + nal_io.write_nal(base, n, chart="NAL:aligned", frame="detector", + extra={"frame": "source"}) + assert not os.path.exists(base + ".npz") # rejected before anything is written + with pytest.raises(ValueError, match="cosmology"): + nal_io.write_nal(base, n, chart="NAL:aligned", frame="detector", + extra={"cosmology": {"name": "Planck15"}}) + # extra that only ADDS is still honoured + nal_io.write_nal(base, n, chart="NAL:aligned", frame="detector", + extra={"pipeline_note": "synthetic"}) + meta = json.load(open(base + ".meta.json")) + assert meta["frame"] == "detector" and meta["pipeline_note"] == "synthetic" + + +def _hand_written(tmp_path, meta, name="ev", coord_names=("mc", "delta_mc", "u_d")): + """An artifact assembled by hand, exactly as a foreign exporter produces one. + + Deliberately does NOT go through write_nal: the whole point of the consumer-side check is that + most artifacts a run loads were never near this module's writer. + """ + mu, G = _make(d=len(coord_names), seed=17) + base = str(tmp_path / name) + np.savez(base + ".npz", theta_star=mu, gamma=G) + full = {"coord_names": list(coord_names), "lnL_peak": 0.0, "chart": "NAL:aligned"} + full.update(meta) + json.dump(full, open(base + ".meta.json", "w")) + return base + + +def test_loaded_artifact_frame_invariant_is_enforced_on_the_consumer_side(tmp_path): + """The invariant write_nal enforces must also hold for artifacts it did not write. + + A source-frame artifact still carrying the distance coordinate, or one declaring source-frame + masses with no cosmology and no distance prior, has integrated the mass-redshift degeneracy + against a prior nobody recorded. Loaded, it evaluates perfectly happily: right dimension, + right names, wrong masses, no error anywhere downstream. + """ + base = _hand_written(tmp_path, {"frame": "source", "cosmology": {"name": "Planck15"}, + "d_prior": {"name": "cosmo_sourceframe"}}, name="carries_ud") + with pytest.raises(ValueError, match="u_d"): + nal_io.load_nal(base + ".npz") + + base = _hand_written(tmp_path, {"frame": "source"}, name="no_cosmo", + coord_names=("mc", "delta_mc")) + with pytest.raises(ValueError, match="cosmology"): + nal_io.load_nal(base + ".npz") + + base = _hand_written(tmp_path, {"frame": "sourceframe"}, name="bad_frame", + coord_names=("mc", "delta_mc")) + with pytest.raises(ValueError, match="frame"): + nal_io.load_nal(base + ".npz") + + # ... and a consistent one loads, with the file named on the object for later error messages + base = _hand_written(tmp_path, {"frame": "detector"}, name="ok") + n = nal_io.load_nal(base + ".npz") + assert n.source == base and n.meta["frame"] == "detector" + + +def test_undeclared_frame_loads_but_never_reaches_an_evaluation(tmp_path, monkeypatch): + """The shipped O3/O4 catalogue declares no frame at all; it must stay loadable, not usable. + + Loading is how such an artifact gets inspected and rewritten with its frame recorded, so the + load-time check does not reject it. Every path that would EVALUATE it does: the plugin entry + point, and check_artifact_frame_invariant itself when asked to fail closed. + """ + base = _hand_written(tmp_path, {}, coord_names=("mc", "delta_mc")) # no 'frame' key at all + n = nal_io.load_nal(base + ".npz") # loads: nothing has been evaluated yet + assert n.meta.get("frame") is None + + with pytest.raises(ValueError, match="no 'frame'"): + nal_io.check_artifact_frame_invariant(n, require_frame=True) + + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch, frame="detector") + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + with pytest.raises(ValueError, match="frame"): + nal_io.prepare_nal_lnL(config=None, coords=["mc", "delta_mc"]) + + +def test_offset_restores_the_absolute_likelihood_and_evidence(tmp_path, monkeypatch): + """nal_lnL(x) + nal_lnL_offset() is the artifacts' TRUE lnL -- what the drivers must report. + + The centring keeps the drivers' exponentiation in range, and cancels in the posterior; it does + not cancel in an absolute lnL or an evidence. Adding the offset back must recover the + uncentred value exactly, in both renormalize modes -- the offset tracks whichever constant was + actually removed, so a driver that adds it back needs to know nothing about the plugin's mode. + """ + mu = np.array([30.0, 0.3]) + loud = 3386.0 + base = str(tmp_path / "ev") + np.savez(base + ".npz", theta_star=mu, gamma=np.diag([1.0, 4.0]), + bounds=np.array([[25.0, 35.0], [0.0, 1.0]])) + json.dump({"coord_names": ["mc", "delta_mc"], "lnL_peak": loud, "frame": "detector", + "chart": "NAL:aligned"}, open(base + ".meta.json", "w")) + monkeypatch.setenv("RIFT_NAL_ARTIFACTS", base + ".npz") + _declare_run(monkeypatch) + X = [np.array([30.0, 30.2, 29.5]), np.array([0.3, 0.35, 0.2])] + theta = np.stack(X, 1) + + for renormalize in (False, True): + nal_io._STATE.update(set=None, coords=None, renormalize=renormalize, offset=0.0) + nal_io.prepare_nal_lnL(config=None, coords=["mc", "delta_mc"]) + centred = nal_io.nal_lnL(*X) + want = nal_io._STATE["set"].lnL(theta, renormalize=renormalize) + assert np.all(centred <= 0.0) # exp() stays in range for the driver + assert np.allclose(centred + nal_io.nal_lnL_offset(), want, atol=1e-9) + # the constant is large enough to matter: reporting the centred value would put the + # evidence out by thousands of nat, not by a rounding error + assert nal_io.nal_lnL_offset() > 3000.0 + + # before preparation the offset is a harmless zero, so a driver may query it unconditionally + nal_io._STATE.update(set=None, coords=None, renormalize=False, offset=0.0) + assert nal_io.nal_lnL_offset() == 0.0 + + +def test_gwalk_offset_conversion_and_scale_max(tmp_path): + """offset = lnL_peak + D/2 ln2pi - 1/2 ln|Gamma|, and scale_max must clear gwalk's 500 cap.""" + h5py = pytest.importorskip("h5py") + mu, G = _make(d=3, seed=4) + loud = 3386.0 # a real SNR~82 event's lnL_peak + n = nal_io.NAL(mu, G, ["mc", "delta_mc", "xi"], lnL_peak=loud) + path = str(tmp_path / "view.h5") + off = nal_io.write_gwalk_view(path, n, "S250114ax/NAL:aligned:test:nal") + sign, logdet = np.linalg.slogdet(G) + assert np.isclose(off, loud + 0.5 * 3 * np.log(2 * np.pi) - 0.5 * logdet) + with h5py.File(path, "r") as f: + g = f["S250114ax/NAL:aligned:test:nal"] + assert set(["mu", "std", "cor", "cov", "limits", "offset", "scale"]) <= set(g.keys()) + assert g.attrs["scale_max"] > abs(off) > 500.0 # would trip gwalk's default cap diff --git a/MonteCarloMarginalizeCode/Code/test/test_seq_warmstart_seed.py b/MonteCarloMarginalizeCode/Code/test/test_seq_warmstart_seed.py new file mode 100644 index 000000000..cab52302b --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_seq_warmstart_seed.py @@ -0,0 +1,309 @@ +#!/usr/bin/env python +""" +Regression tests for the SEQUENTIAL WARM-START seed +(--sampler-sequential-warmstart, bin/integrate_likelihood_extrinsic_batchmode). + +This is the same defect PR #78 fixed for the L0 auto-rescue, one code path away, found by +the mechanical `_rvs` audit in test/expensive_before_merging/integrators/audit_rvs_fairdraw.py. +With --n-events-to-analyze > 1, each intrinsic point may seed the next point's extrinsic +integral from the samples it drew. The capture block read them out of `sampler._rvs`, and by +the time it runs, integrate_log has REBOUND every _rvs key to a fair-draw subset of + + min(n_extr, 1.5*eff_samp, 1.5*neff) + +rows taken WITH REPLACEMENT -- a resample built for EXPORT. Two consequences, both of which +this suite pins: + + 1. ON A COLLAPSED PASS THERE IS ALMOST NOTHING LEFT TO SEED FROM. eff_samp ~ 1 gives one + row ("Fairdraw size : 1" in the rho_net 146.8 logs) out of a thousand retained, and + drawing WITH REPLACEMENT is why a "2-point seed" can have affine rank 0 -- two copies of + one point. This is not an exotic configuration: every extrinsic stage built by + create_event_parameter_pipeline_BasicIteration, cepp_basic_htcondor and + create_event_nr_pipeline_with_cip passes --fairdraw-extrinsic-output unconditionally. + + 2. THE GUARD WAS A COUNT (`_lnv.size >= 2`, `np.sum(_keep) >= 2`), which is exactly the + rule build_warm_seed exists to replace. n points span at most n-1 affine dimensions, so + a 2-row seed passes the count and is still rank-deficient in 6. The next point then + warm-starts into a degenerate sliver and reports a healthy n_eff over truncated support: + the QUIET failure mode, which is why this survived while the L0 one was found. + +The requirement is the same one PR #78 set: seed from the points the pass RETAINED, and judge +the seed by RANK, repairing it before it reaches the sampler. +""" + +import os + +import numpy as np +import pytest + +import RIFT.integrators.mcsamplerAdaptiveVolume as mcsamplerAV +from RIFT.integrators.mcsamplerAdaptiveVolume import build_warm_seed, seed_affine_rank + +NAMES = ['right_ascension', 'declination', 'phi_orb', 'inclination', 'psi', 'distance'] +NDIM = len(NAMES) +LO = np.zeros(NDIM) +HI = np.ones(NDIM) +AX = list(range(NDIM)) + +_ILE = os.path.join(os.path.dirname(os.path.abspath(__file__)), + '..', 'bin', 'integrate_likelihood_extrinsic_batchmode') + + +def _sampler(n_chunk=10000): + """Bound to the ACTIVE backend exactly as the ILE does; see test_l0_rescue_seed.""" + s = mcsamplerAV.MCSampler(n_chunk=n_chunk) + s.xpy = mcsamplerAV.xpy_default + s.identity_convert = mcsamplerAV.identity_convert + for name in NAMES: + s.add_parameter(name, pdf=None, left_limit=0.0, right_limit=1.0, + prior_pdf=lambda x: np.ones(np.shape(x)), adaptive_sampling=True) + return s + + +def _peaked(rho, x0=None, widths=None): + """6-D Gaussian at lnL scale rho^2/2, with the float64 underflow of the real code.""" + x0 = 0.5 * np.ones(NDIM) if x0 is None else np.asarray(x0, dtype=float) + w = (0.5 / rho) * np.ones(NDIM) if widths is None else np.asarray(widths, dtype=float) + lnLmax = 0.5 * rho ** 2 + + def lnL(*args, **kwargs): + x = np.array([np.asarray(a, dtype=float).ravel() for a in args]).T + out = lnLmax - 0.5 * np.sum(((x - x0) / w) ** 2, axis=-1) + return np.where(out > lnLmax - 745.0, out, -np.inf) + return lnL + + +def _load_reserve_lookup(): + """Exec just `_warm_seed_reserve_for` out of the ILE script. + + The ILE is an executable, not an importable module, and importing it would run an + argument parser and a stack of optional dependencies. The helper is self-contained + (builtins only), so lifting its source is enough to test its BEHAVIOUR rather than + merely asserting that some words appear in the file. + """ + with open(_ILE) as f: + src = f.read() + start = src.index('def _warm_seed_reserve_for') + end = src.index('def _warm_seed_geometry') + ns = {} + exec(compile(src[start:end], 'ile_warm_seed_reserve_for', 'exec'), ns) + return ns['_warm_seed_reserve_for'] + + +class _FakeSampler(object): + def __init__(self, params_ordered, reserve=None, members=()): + self.params_ordered = list(params_ordered) + if reserve is not None: + self._warm_seed_reserve = reserve + self.portfolio_realizations = list(members) + + +def _reserve(params_ordered, n=10): + return dict(X=np.zeros((n, len(params_ordered))), lnL=np.zeros(n), + n_retained=n, n_finite=n, ln_sum_w_finite=0.0, + params_ordered=list(params_ordered)) + + +### +### 1. the shared reserve lookup -- behaviour, not text +### + +def test_the_reserve_lookup_finds_a_reserve_on_the_sampler_itself(): + look = _load_reserve_lookup() + r = _reserve(NAMES) + assert look(_FakeSampler(NAMES, reserve=r)) is r + + +def test_the_reserve_lookup_falls_through_to_a_portfolio_member(): + """A portfolio keeps the reserve on the aggregate, but a bare AV member can be the one + holding it; the L0 rescue relied on this fallback and the sequential capture must too.""" + look = _load_reserve_lookup() + r = _reserve(NAMES) + s = _FakeSampler(NAMES, members=[_FakeSampler(NAMES), _FakeSampler(NAMES, reserve=r)]) + assert look(s) is r + + +def test_the_reserve_lookup_declines_a_reserve_in_the_wrong_column_order(): + """Silent scrambling is worse than no seed: X is read positionally against + params_ordered, so a permuted reserve seeds the next point in wrong coordinates.""" + look = _load_reserve_lookup() + scrambled = list(reversed(NAMES)) + assert look(_FakeSampler(NAMES, reserve=_reserve(scrambled))) is None + + +def test_the_reserve_lookup_returns_none_when_nobody_kept_one(): + look = _load_reserve_lookup() + assert look(_FakeSampler(NAMES)) is None + assert look(_FakeSampler(NAMES, members=[_FakeSampler(NAMES)])) is None + + +### +### 2. the substance: the fair draw starves this seed, the reserve does not +### + +@pytest.mark.parametrize('rho', [60.0, 100.0]) +def test_the_fair_draw_leaves_far_fewer_rows_than_the_pass_retained(rho): + """The precondition for everything below. Assert it explicitly rather than assume it: + a test that lands on a healthy pass proves nothing about the collapsed one.""" + np.random.seed(20260813) + s = _sampler(20000) + s.integrate_log(_peaked(rho), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + n_rvs = len(np.asarray(mcsamplerAV.identity_convert(s._rvs['log_integrand'])).ravel()) + reserve = s._warm_seed_reserve + assert reserve is not None, 'no reserve to compare against' + assert reserve['n_retained'] > n_rvs, \ + 'fair draw kept {} of {} retained -- this pass did not collapse, so it cannot ' \ + 'exercise the defect'.format(n_rvs, reserve['n_retained']) + + +def test_a_seed_taken_from_the_fair_draw_is_rank_deficient_where_the_reserve_is_not(): + """The defect and its fix, side by side on one real pass.""" + np.random.seed(20260813) + s = _sampler(20000) + s.integrate_log(_peaked(100.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + conv = mcsamplerAV.identity_convert + from_rvs = np.vstack([np.asarray(conv(s._rvs[p]), dtype=float).ravel() + for p in s.params_ordered]).T + rank_rvs, _ = seed_affine_rank(from_rvs, LO, HI, axes=AX) + + reserve = s._warm_seed_reserve + seed, info = build_warm_seed(np.asarray(reserve['X'], dtype=float), + np.asarray(reserve['lnL'], dtype=float).ravel(), + LO, HI, AX, deltalnL=15.0) + assert rank_rvs < NDIM, \ + 'the fair-drawn rows spanned {}/{} -- no defect to fix on this pass'.format(rank_rvs, NDIM) + assert info['rank_final'] >= NDIM, \ + 'the reserve-based seed is still rank {}/{}'.format(info['rank_final'], NDIM) + assert len(seed) >= 2 + + +def test_the_old_count_rule_had_no_good_outcome_on_a_collapsed_pass(): + """The count rule fails in BOTH directions, and which one you get is luck. + + Measured on this pass the fair draw keeps ONE row ("Fairdraw size : 1", the rho_net 146.8 + regime), so the count DECLINES and --sampler-sequential-warmstart is silently inert -- the + user asked for a warm start and got none, with no message. At the rho_net 102.8 regime it + keeps ~5, the count ACCEPTS, and the seed is rank 2-4 of 6: the next point warm-starts into + a sliver and reports a healthy n_eff over truncated support. + + So the assertion is not "the count accepted it". It is that the count rule cannot produce + a usable seed here either way, while the reserve can. + """ + np.random.seed(20260813) + s = _sampler(20000) + s.integrate_log(_peaked(100.0), *NAMES, nmax=400000, neff=8, n=20000, + no_protect_names=True, verbose=False, + igrand_fairdraw_samples=True, igrand_fairdraw_samples_max=200) + conv = mcsamplerAV.identity_convert + lnv = np.asarray(conv(s._rvs['log_integrand']), dtype=float).ravel() + cols = np.vstack([np.asarray(conv(s._rvs[p]), dtype=float).ravel() + for p in s.params_ordered]).T + keep = lnv > (np.nanmax(lnv) - 15.0) + old_seed = cols[keep] if np.sum(keep) >= 2 else (cols if cols.shape[0] >= 2 else None) + if old_seed is None: + declined = True + rank = 0 + else: + declined = False + rank, _ = seed_affine_rank(old_seed, LO, HI, axes=AX) + assert declined or rank < NDIM, \ + 'the fair-drawn rows spanned {}/{} -- this pass does not exercise the defect'.format(rank, NDIM) + + # ...and the reserve, on the same pass, does produce one. + r = s._warm_seed_reserve + _, info = build_warm_seed(np.asarray(r['X'], dtype=float), + np.asarray(r['lnL'], dtype=float).ravel(), + LO, HI, AX, deltalnL=15.0) + assert info['rank_final'] >= NDIM + + +def test_duplicate_rows_from_sampling_with_replacement_span_nothing(): + """Why 'n rows' and 'n distinct points' are not the same quantity after a fair draw.""" + pt = np.full((1, NDIM), 0.5) + assert seed_affine_rank(np.repeat(pt, 5, axis=0), LO, HI, axes=AX)[0] == 0 + + +### +### 3. the ILE actually wires it that way +### + +def _read_ile(): + with open(_ILE) as f: + return f.read() + + +def _read_ile_code(): + """The ILE source, comments and docstrings removed AND all whitespace squeezed out. + + These assertions are about what the code DOES, and the comments that explain why a rule + went quote that rule verbatim -- so a naive substring search reports the explanation as a + regression. (It did, on the first run of this suite.) + + The result is WHITESPACE-FREE, so every needle below must be written whitespace-free too. + That is the point: it also makes the assertions immune to reformatting. + """ + import io + import tokenize + src = _read_ile() + out = [] + prev_end = (1, 0) + try: + toks = list(tokenize.generate_tokens(io.StringIO(src).readline)) + except (tokenize.TokenError, IndentationError): + # never let a tokenizer quirk turn into a false pass + return src + for tok in toks: + if tok.type == tokenize.COMMENT: + continue + if tok.type == tokenize.STRING and tok.start[1] == 0: + continue # module/def-level docstring on its own line + if tok.start[0] != prev_end[0]: + out.append('\n') + out.append(tok.string) + prev_end = tok.end + return ''.join(out) + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_seq_warmstart_capture_seeds_from_the_reserve(): + src = _read_ile() + i = src.index('_SEQ_WS_PENDING = _seed_ws') + block = src[max(0, i - 3000):i] + assert '_warm_seed_reserve_for(sampler)' in block, \ + 'the sequential capture still seeds from _rvs, which the fair draw has truncated' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_seq_warmstart_capture_goes_through_the_rank_tested_builder(): + src = _read_ile() + i = src.index('_SEQ_WS_PENDING = _seed_ws') + block = src[max(0, i - 3000):i] + assert 'build_warm_seed' in block, 'the sequential seed is not rank-tested or puffed' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_seq_warmstart_capture_no_longer_guards_on_a_row_count(): + """The CODE, not the comment that explains why it went.""" + code = _read_ile_code() # whitespace-free; needles must be too + assert 'np.sum(_keep)>=2' not in code, 'the count rule is back' + assert '_keep=_lnv>(np.nanmax(_lnv)' not in code, \ + 'the inline deltalnL window is back; build_warm_seed owns that cut' + + +@pytest.mark.skipif(not os.path.exists(_ILE), reason='ILE executable not in this tree') +def test_the_rescue_and_the_sequential_capture_share_one_reserve_lookup(): + """Five defects of this shape have come from two copies of one idea drifting apart. + Both callers go through the single helper, so a fix to one cannot miss the other.""" + code = _read_ile_code() # whitespace-free; needles must be too + assert code.count('def_warm_seed_reserve_for') == 1 + # one definition + exactly two call sites (the L0 rescue and the sequential capture) + assert code.count('_warm_seed_reserve_for(sampler)') == 3, \ + 'expected the shared lookup to have exactly two callers, found {}'.format( + code.count('_warm_seed_reserve_for(sampler)') - 1) + # and the old inline duplicate is gone + assert "_res_l0=getattr(sampler,'_warm_seed_reserve',None)" not in code, \ + 'the inline reserve lookup is back; it will drift from the shared one' diff --git a/MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py b/MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py new file mode 100644 index 000000000..30fb9c46c --- /dev/null +++ b/MonteCarloMarginalizeCode/Code/test/test_supplementary_likelihood_hook.py @@ -0,0 +1,362 @@ +"""Guard the --supplementary-likelihood-factor plugin hook in CIP and EOSPosterior. + +Motivation: both drivers carried an identical typo for months -- + + supplemental_ln_likelhood_prep = getattr(module, name_prep) # assigned (misspelt) + supplemental_ln_likelhood_parsed_ini = config # assigned (misspelt) + supplemental_ln_likelihood_prep(config=supplemental_ln_likelihood_parsed_ini, ...) # called + +so the CALLED names were never the ASSIGNED ones and remained None from their initialisation. +Anyone supplying --supplementary-likelihood-factor-ini together with a plugin that defines a +prepare_ hook got `TypeError: 'NoneType' object is not callable`. The plain hook (no +ini, or no prepare_) worked, which is why it survived: the hasattr() guard skips the whole block. + +These tests are STATIC (ast-based). The drivers are top-level scripts, not importable modules, so +we cannot exercise the block directly without running a full inference job; parsing catches the +whole class of defect at negligible cost. +""" +import ast +import os + +import pytest + +HERE = os.path.dirname(os.path.abspath(__file__)) +BIN = os.path.abspath(os.path.join(HERE, "..", "bin")) +DRIVERS = [ + "util_ConstructIntrinsicPosterior_GenericCoordinates.py", + "util_ConstructEOSPosterior.py", +] + + +def _tree(fname): + path = os.path.join(BIN, fname) + if not os.path.exists(path): + pytest.skip("driver not present: %s" % fname) + with open(path) as f: + return ast.parse(f.read(), filename=path) + + +def _assigned_names(tree): + out = set() + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + for t in node.targets: + for n in ast.walk(t): + if isinstance(n, ast.Name): + out.add(n.id) + elif isinstance(node, (ast.AugAssign, ast.AnnAssign)) and \ + isinstance(node.target, ast.Name): + out.add(node.target.id) + elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + out.add(node.name) + elif isinstance(node, (ast.Import, ast.ImportFrom)): + for a in node.names: + out.add((a.asname or a.name).split(".")[0]) + elif isinstance(node, ast.For) and isinstance(node.target, ast.Name): + out.add(node.target.id) + elif isinstance(node, ast.withitem) and node.optional_vars is not None: + for n in ast.walk(node.optional_vars): + if isinstance(n, ast.Name): + out.add(n.id) + elif isinstance(node, ast.ExceptHandler) and node.name: + out.add(node.name) + elif isinstance(node, ast.comprehension): + for n in ast.walk(node.target): + if isinstance(n, ast.Name): + out.add(n.id) + return out + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_supplementary_names_are_assigned_before_use(fname): + """Every `supplemental_*` identifier that is READ must also be ASSIGNED in the same file. + + This is the general form of the bug: a misspelt assignment leaves the read name bound to its + initial None, which fails only on the rarely-exercised ini path. + """ + tree = _tree(fname) + assigned = _assigned_names(tree) + used = {n.id for n in ast.walk(tree) + if isinstance(n, ast.Name) and isinstance(n.ctx, ast.Load) + and n.id.startswith("supplemental_")} + missing = sorted(used - assigned) + assert not missing, ( + "%s reads supplementary-hook names it never assigns: %s -- almost certainly a typo in " + "the assignment, which leaves the name None and breaks " + "--supplementary-likelihood-factor-ini" % (fname, missing)) + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_no_dead_supplementary_assignments(fname): + """No `supplemental_*` name may be ASSIGNED and then never READ. + + This is the general signature of the historical bug and the one that matters: the misspelt + names WERE assigned, so a "used but never assigned" check does not see them -- the + correctly-spelt names are assigned at initialisation. What is anomalous is that the misspelt + assignments are dead: nothing ever reads them. A spelling blocklist would only catch this one + typo; this catches any future variant. + """ + tree = _tree(fname) + loaded = {n.id for n in ast.walk(tree) + if isinstance(n, ast.Name) and isinstance(n.ctx, ast.Load)} + stored = {n.id for n in ast.walk(tree) + if isinstance(n, ast.Name) and isinstance(n.ctx, ast.Store) + and n.id.startswith("supplemental_")} + dead = sorted(stored - loaded) + assert not dead, ( + "%s assigns supplementary-hook name(s) that are never read: %s -- a dead assignment here " + "means the value silently never reaches the code that uses it" % (fname, dead)) + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_no_known_likelihood_misspelling(fname): + """Direct guard on the specific historical typo, in identifiers and attributes alike.""" + tree = _tree(fname) + bad = {n.id for n in ast.walk(tree) if isinstance(n, ast.Name) and "likelhood" in n.id} + bad |= {n.attr for n in ast.walk(tree) if isinstance(n, ast.Attribute) and "likelhood" in n.attr} + assert not bad, "%s contains misspelt 'likelhood' identifier(s): %s" % (fname, sorted(bad)) + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_prepare_hook_is_actually_invoked(fname): + """The prepare_ hook must still be called with config= and coords=. + + Guards against 'fixing' the typo by deleting the call rather than repairing the name. + """ + tree = _tree(fname) + calls = [n for n in ast.walk(tree) + if isinstance(n, ast.Call) and isinstance(n.func, ast.Name) + and n.func.id == "supplemental_ln_likelihood_prep"] + assert calls, "%s never calls supplemental_ln_likelihood_prep" % fname + kw = {k.arg for c in calls for k in c.keywords} + assert {"config", "coords"} <= kw, ( + "%s calls the prepare hook without config=/coords=; plugins rely on that signature " + "(got %s)" % (fname, sorted(kw))) + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_prepare_hook_is_invoked_without_an_ini(fname): + """The prepare call must NOT sit inside `if opts.supplementary_likelihood_factor_ini:`. + + A plugin configured entirely by environment gets no ini, so gating preparation on one leaves it + never told the sampling basis -- and a plugin that then has to guess what its input arrays are + called can guess wrong with the right array count and no error. `config=None` is a perfectly + good argument; the basis is the part that cannot be reconstructed later. + """ + tree = _tree(fname) + for node in ast.walk(tree): + if not isinstance(node, ast.If): + continue + gated_on_ini = any(isinstance(a, ast.Attribute) and + a.attr == "supplementary_likelihood_factor_ini" + for a in ast.walk(node.test)) + if not gated_on_ini: + continue + for inner in node.body: + for c in ast.walk(inner): + assert not (isinstance(c, ast.Call) and isinstance(c.func, ast.Name) + and c.func.id == "supplemental_ln_likelihood_prep"), ( + "%s only prepares the supplementary-likelihood plugin when an ini is supplied; " + "it must also be prepared (with config=None) so the plugin is always told " + "coords=" % fname) + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_prepare_hook_is_told_the_sampling_basis(fname): + """`coords=` must name the SAME list the driver splats into `sampler.integrate`. + + The sampler is handed one dimension per name in that starred list, so it calls + `supplemental_ln_likelihood(*x)` with one array per SAMPLING coordinate, in that order. + Declaring any other list -- notably the FIT basis `coord_names`, which differs from + `low_level_coord_names` as soon as --parameter-implied or --parameter-nofit is used -- makes + the plugin attach the wrong name to each array. Nothing raises: the plugin simply evaluates + at coordinates it has mislabelled. Comparing the two identifiers is the whole invariant, and + it is checkable statically; the runtime consequence of getting it wrong is exercised in + test_nal_io.py::test_wrong_basis_from_the_driver_would_evaluate_at_the_wrong_point. + """ + tree = _tree(fname) + sampled = {n.value.id for c in ast.walk(tree) + if isinstance(c, ast.Call) and isinstance(c.func, ast.Attribute) + and c.func.attr == "integrate" + for n in c.args + if isinstance(n, ast.Starred) and isinstance(n.value, ast.Name)} + assert sampled, ( + "%s never splats a coordinate-name list into sampler.integrate(); the basis the " + "supplementary hook must be told can no longer be identified" % fname) + declared = [k.value for c in ast.walk(tree) + if isinstance(c, ast.Call) and isinstance(c.func, ast.Name) + and c.func.id == "supplemental_ln_likelihood_prep" + for k in c.keywords if k.arg == "coords"] + assert declared, "%s never passes coords= to the prepare hook" % fname + for node in declared: + assert isinstance(node, ast.Name) and node.id in sampled, ( + "%s tells the prepare hook coords=%s, but integrates over %s -- the hook must be " + "given the SAMPLING basis, since that is the order the plugin is called with" + % (fname, ast.dump(node) if not isinstance(node, ast.Name) else node.id, + sorted(sampled))) + + +def _adds(node, *names): + """True if `node` is a sum (at any depth) that includes every one of `names` as a bare Name.""" + if not isinstance(node, ast.BinOp) or not isinstance(node.op, ast.Add): + return False + present = {n.id for n in ast.walk(node) if isinstance(n, ast.Name)} + return set(names) <= present + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_supplementary_offset_hook_is_queried(fname): + """The driver must look for the plugin's `_offset` companion. + + A plugin whose contribution is large has to return a CENTRED lnL: the default path here is + `likelihood_function(*x) * np.exp(supplemental(*x))`, and float64 exp overflows past ~709 -- + which a single loud-event quadratic (lnL_peak ~ SNR^2/2) exceeds on its own. The plugin + therefore reports the constant it removed, by the same naming convention as prepare_, + and the driver must ask for it; without that the constant is unrecoverable downstream. + """ + tree = _tree(fname) + built = [c for c in ast.walk(tree) + if isinstance(c, ast.BinOp) and isinstance(c.op, ast.Add) + and any(isinstance(n, ast.Constant) and n.value == "_offset" for n in ast.walk(c))] + assert built, ("%s never builds the '_offset' hook name, so a plugin that centres " + "its contribution has no way to report the constant it removed" % fname) + called = [c for c in ast.walk(tree) + if isinstance(c, ast.Call) and isinstance(c.func, ast.Name) + and c.func.id == "supplemental_ln_likelihood_offset_fn"] + assert called, "%s resolves the offset hook but never calls it" % fname + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_reported_evidence_restores_the_supplementary_offset(fname): + """Absolute evidence outputs must carry `ln_integrand_value + supplemental_..._offset`. + + The centring is a constant multiplicative factor on the integrand. It divides out of the + posterior -- which is why the sampler may have it -- but NOT out of an evidence: `integral_result.dat` + is read as an absolute lnZ and differenced against other runs, so reporting the centred value + makes every such odds ratio wrong by exp(offset), thousands of nat for a loud event, with + nothing anomalous to see in the file. + """ + tree = _tree(fname) + absolute = [n for n in ast.walk(tree) if isinstance(n, ast.Assign) + and any(isinstance(t, ast.Name) and t.id == "ln_integrand_value_absolute" + for t in n.targets)] + assert absolute, ("%s never forms an absolute evidence; the value it writes is whatever the " + "sampler returned, including any constant the plugin subtracted" % fname) + assert all(_adds(a.value, "ln_integrand_value", "supplemental_ln_likelihood_offset") + for a in absolute), ( + "%s defines ln_integrand_value_absolute without adding " + "supplemental_ln_likelihood_offset to ln_integrand_value" % fname) + + # ... and nothing may WRITE the centred value. Restoring the constant in one output and not + # another is the harder bug to see: the files disagree by a constant and each looks plausible. + for c in ast.walk(tree): + if not isinstance(c, ast.Call): + continue + writes = (isinstance(c.func, ast.Attribute) + and c.func.attr in ("savetxt", "write")) + if not writes: + continue + bare = [n for n in ast.walk(c) if isinstance(n, ast.Name) + and n.id == "ln_integrand_value"] + assert not bare, ( + "%s line %d writes the centred ln_integrand_value; absolute lnL/evidence outputs must " + "use ln_integrand_value_absolute" % (fname, c.lineno)) + + +def test_cip_reweighted_evidence_is_on_the_same_scale(): + """`_withpriorchange.dat` is documented to agree with `integral_result.dat`. + + It is built from lnLmax of the CENTRED integrand, so restoring the offset in one file and not + the other would turn that agreement -- the only cross-check the driver ships on its own + evidence -- into a fixed disagreement of exactly the offset, in a file whose stated purpose is + to be compared. + """ + fname = "util_ConstructIntrinsicPosterior_GenericCoordinates.py" + tree = _tree(fname) + assigns = [n for n in ast.walk(tree) if isinstance(n, ast.Assign) + and any(isinstance(t, ast.Name) and t.id == "log_res_reweighted" + for t in n.targets)] + assert assigns, "%s no longer computes log_res_reweighted" % fname + for a in assigns: + assert any(isinstance(x, ast.Name) and x.id == "supplemental_ln_likelihood_offset" + for x in ast.walk(a.value)), ( + "%s line %d computes the reweighted evidence without restoring the supplementary " + "offset, so it disagrees with integral_result.dat by that constant" % (fname, a.lineno)) + + +def test_cip_exported_lnL_columns_are_absolute(): + """CIP's per-sample lnL exports are read as absolute lnL, so they carry the offset too. + + `_lnL.dat`, the hyperpipeline grid's lnL column and best_point_by_lnL_value.dat all + come from `lnL_list`, and each is consumed as a lnL on the same scale as the fit's own -- the + next iteration's grid, the best-point record. The correction is applied once, where the list + becomes an array, so it cannot be applied to some consumers and not others; this checks that it + happens before anything reads the list. + """ + fname = "util_ConstructIntrinsicPosterior_GenericCoordinates.py" + tree = _tree(fname) + corrected = [n.lineno for n in ast.walk(tree) if isinstance(n, ast.Assign) + and any(isinstance(t, ast.Name) and t.id == "lnL_list" for t in n.targets) + and any(isinstance(x, ast.Name) + and x.id == "supplemental_ln_likelihood_offset" + for x in ast.walk(n.value))] + assert len(corrected) == 1, ( + "%s must restore the supplementary-likelihood offset on lnL_list exactly once (found %d " + "site(s)); more than one would double-count it, none leaves the exported lnL columns " + "centred" % (fname, len(corrected))) + reads = [c.lineno for c in ast.walk(tree) if isinstance(c, ast.Call) + for n in ast.walk(c) if isinstance(n, ast.Name) and n.id == "lnL_list" + and isinstance(n.ctx, ast.Load)] + early = sorted(ln for ln in reads if ln < corrected[0]) + # appends while the list is being built are Calls too -- they are attribute calls ON the list + early = [ln for ln in early + if not any(isinstance(c, ast.Call) and isinstance(c.func, ast.Attribute) + and isinstance(c.func.value, ast.Name) and c.func.value.id == "lnL_list" + and c.lineno == ln for c in ast.walk(tree))] + assert not early, ( + "%s reads lnL_list at line(s) %s, before the offset is restored at line %d -- those " + "consumers would get the centred values" % (fname, early, corrected[0])) + + +# List methods that change the contents in place. `+=` on a list is an AugAssign whose target is a +# Store of the same Name, so it is caught by the rebind check rather than this one. +_LIST_MUTATORS = ("append", "extend", "insert", "remove", "pop", "clear", "sort", "reverse") + + +@pytest.mark.parametrize("fname", DRIVERS) +def test_prepare_hook_runs_after_the_sampling_basis_is_final(fname): + """Nothing may change the declared coordinate list after the prepare hook has been told it. + + The plugin RECORDS the basis it is handed (nal_io copies it into module state); the sampler + then calls the plugin with one array per coordinate in the FINAL list. CIP builds that list + in stages -- `--parameter-implied`/`--parameter-nofit` early, then `ordering` appended for a + tabular-EOS run much later -- so preparing next to the plugin import snapshots a list that is + one entry short of what the sampler supplies. The plugin then either raises on the array + count (nal_io does) or, worse, mislabels every array after the missing one. + + Checked by line number rather than by running the driver: these are top-level scripts, and the + ordering is a property of the file, not of any particular run's flags. + """ + tree = _tree(fname) + calls = [c for c in ast.walk(tree) + if isinstance(c, ast.Call) and isinstance(c.func, ast.Name) + and c.func.id == "supplemental_ln_likelihood_prep"] + assert calls, "%s never calls supplemental_ln_likelihood_prep" % fname + prepared_at = min(c.lineno for c in calls) + declared = {k.value.id for c in calls for k in c.keywords + if k.arg == "coords" and isinstance(k.value, ast.Name)} + assert declared, "%s never passes a named coordinate list as coords=" % fname + for name in sorted(declared): + rebound = [n.lineno for n in ast.walk(tree) + if isinstance(n, ast.Name) and n.id == name + and isinstance(n.ctx, ast.Store)] + mutated = [c.lineno for c in ast.walk(tree) + if isinstance(c, ast.Call) and isinstance(c.func, ast.Attribute) + and c.func.attr in _LIST_MUTATORS + and isinstance(c.func.value, ast.Name) and c.func.value.id == name] + late = sorted(ln for ln in rebound + mutated if ln > prepared_at) + assert not late, ( + "%s prepares the supplementary-likelihood plugin at line %d with coords=%s, but that " + "list is still changed afterwards at line(s) %s -- the plugin would be told a basis " + "that is not the one the sampler integrates over. Prepare after the last change." + % (fname, prepared_at, name, late)) diff --git a/containers/README.md b/containers/README.md index e5f501e52..11867dcfd 100644 --- a/containers/README.md +++ b/containers/README.md @@ -125,9 +125,9 @@ example. Schema: For the ILE (and CIP) Condor submit, a manifest produces: - **`MY.SingularityImage`** — an *unquoted* `ifThenElse(...)` expression that - selects the highest-capability image the matched machine can run, defaulting to - the `fallback` image (also used when the capability attribute is `undefined`, - e.g. on a CPU-only CIP slot — hence the fallback must be CPU-safe): + selects the highest-capability image the matched machine can run, with the + `fallback` image as the innermost `else` (used when the machine's capability is + below every threshold): ``` ifThenElse(TARGET.GPUs_Capability >= 8.0, "./rift_container_modern.sif", "/cvmfs/.../rift_container_default.sif") @@ -149,6 +149,59 @@ For the ILE (and CIP) Condor submit, a manifest produces: composed (`&&`) with any user-supplied `RIFT_REQUIRE_GPUS` (which today you use to block incompatible hosts by `DeviceName`). Both apply; neither is dropped. +### OSG: pick a delivery mode + +The expression-valued `MY.SingularityImage` is evaluated *execute-side*. OSPool +glidein pilots read `SingularityImage` as a **literal string**, so an +`ifThenElse` lands verbatim and the job holds. Two opt-in modes fix this, +selected by an environment variable at DAG-build time: + +| env var | behaviour | +|---|---| +| *(unset)* | legacy `universe = vanilla` + expression-valued `MY.SingularityImage`. Correct on a local/CIT pool; **not OSG-safe**. | +| `RIFT_CONTAINER_UNIVERSE=1` | **recommended for OSG.** `universe = container` + `container_image = $$([ ifThenElse(...) ])` over image BASENAMES. `$$()` is HTCondor's match-time (schedd-side) machine-ad substitution, so the pilot only ever sees a literal image name. No `MY.SingularityImage`, no `MY.SingularityBindCVMFS`; the matched image arrives via the `$$()` transfer token with `MY.TransferInput` pinned (see below). Requires every family image to be a transferable URL. GPU access is automatic under `request_gpus`. Works on CIT-local too. | +| `RIFT_CONTAINER_RUNTIME_SELECT=1` | older ILE-only fallback: Condor runs a generated `rift_container_select.sh` on the bare node, which reads the real capability from `nvidia-smi`, fetches only the matching image (`stashcp`/`pelican`) and re-execs under `apptainer exec --nv`. | + +Under asimov set it from the blueprint, not the shell: + +```yaml +scheduler: + singularity image: /path/to/rift_container_family.yaml + singularity base exe directory: /usr/local/bin/ + environment variables: + RIFT_CONTAINER_UNIVERSE: 1 +``` + +With `osdf://` images inside a manifest, the pipeline also enables the matching +transfer credential automatically (`use_oauth_services = scitokens`, or `igwn` +for `igwn+osdf:`) by inspecting the manifest's image URLs — the single-image path +keys off the `SINGULARITY_RIFT_IMAGE` string, which for a family is only a +`.yaml` path. + +> **Why the container-universe selector names basenames, not URLs.** +> `condor_submit` parses `container_image` *before* any `$$` expansion and derives +> the job ad's `ContainerImage` -- the name the image gets in the job scratch dir -- +> as the text after the **last** `/`. A selector containing full paths is cut in +> half, and the fragment that survives is not a valid image name. Submitting that +> form to the IGWN pool holds the job at the execute point: +> `PREPARE_JOB (prepare-hook) failed: Unable to download or build singularity image +> cutest_busybox_...sif") ])`. +> +> So the selector emits **basenames only** (no `/`); the whole `$$` token survives +> into `ContainerImage` and the schedd expands it at match time +> (`MATCH_EXP_ContainerImage = "rift_container_modern.sif"`). The image itself +> arrives via the comma-free `$$()` transfer token, and `MY.TransferInput` is pinned +> so `condor_submit` does not append the basename selector to `TransferInput` as a +> bogus extra input file. Verified end to end on an OSPool glidein against +> `$CondorVersion: 25.11.1`. +> +> Consequence: **every image in a family used with container universe must be a +> transferable URL.** An in-place (CVMFS/local) image can only be named by its full +> path, which reintroduces the truncation, so `build_container_image_select()` +> raises `ContainerManifestError` for such a family. Stage those images at a URL, or +> use `RIFT_CONTAINER_RUNTIME_SELECT=1`. + + ### HTCondor GPU attribute names — important Two different namespaces are in play and are kept separate: diff --git a/containers/rift_container_family.yaml b/containers/rift_container_family.yaml index 1a87b1a5e..fc324736f 100644 --- a/containers/rift_container_family.yaml +++ b/containers/rift_container_family.yaml @@ -34,11 +34,19 @@ capability_attr: GPUs_Capability # CPU-safe / most broadly compatible image. fallback: default +# NOTE for RIFT_CONTAINER_UNIVERSE=1 (the recommended OSG mode): every image below +# must be a transferable URL. condor_submit derives the job's ContainerImage as the +# text after the last "/" of container_image, BEFORE any $$ expansion, so the +# selector may not contain a path -- it names basenames, and the image is delivered +# by file transfer. An in-place CVMFS/local image cannot be named that way and is +# rejected with ContainerManifestError. It is still fine under the legacy and +# runtime-select modes; see containers/README.md. + containers: - # Default, broadly-compatible image for older machines. Referenced in place - # on CVMFS -- never transferred; CVMFS lazy-fetches it only when selected. + # Default, broadly-compatible image for older machines. Delivered via osdf: + # only the matched machine fetches it (selective $$() transfer). - label: default - image: /cvmfs/singularity.opensciencegrid.org/oshaughn/rift_container_default.sif + image: osdf:///igwn/staging/oshaughn/rift_containers/rift_container_default.sif cuda_capability_min: 3.5 cuda_capability_max: 8.0 note: "base=nvidia/cuda:11.8.0-runtime-ubuntu22.04, cupy-cuda11x" diff --git a/docs/private-review-dispatch.md b/docs/private-review-dispatch.md new file mode 100644 index 000000000..a4546d19b --- /dev/null +++ b/docs/private-review-dispatch.md @@ -0,0 +1,22 @@ +# Private upstream RIFT review dispatch + +This workflow is dispatch-only. It never checks out or executes pull-request +content. It is eligible only when all three conditions are true: + +- base repository is `oshaughn/research-projects-RIT` (enforced by workflow + location and exact OIDC repository identity); +- PR author and head repository owner are `oshaughnessy-junior`; +- base branch is `rift_O4c`, `master`, or `rift_O4d`. + +The coordinator must independently re-read the PR and enforce the same author, +head-owner, and base-branch allowlists. Workflow fields are only routing hints. + +Before merge, an upstream owner must create GitHub environment +`private-review-dispatch-rift-upstream`; configure repository-specific +Tailscale WIF variables; install the reviewer App only on this repository with +metadata-read and pull-request-write; and complete the negative WIF/ACL tests. +The coordinator uses a separate ledger, OIDC audience, WIF credential, and +tailnet endpoint from the junior-fork review service. + +Approval and automatic merge are disabled. A successful COMMENT review asks +OpenClaw/main to alert Richard for manual merge. diff --git a/docs/source/containers.rst b/docs/source/containers.rst index f6d811309..9cea32906 100644 --- a/docs/source/containers.rst +++ b/docs/source/containers.rst @@ -118,10 +118,9 @@ What the pipeline generates For a manifest, the ILE (and CIP) Condor submit files get: * **``MY.SingularityImage``** — an *unquoted* ``ifThenElse`` expression that - selects the highest-capability image the matched machine can run, falling back - to the ``fallback`` image (also used when the capability attribute is - ``undefined``, e.g. on a CPU-only CIP slot — hence the fallback must be - CPU-safe):: + selects the highest-capability image the matched machine can run, with the + ``fallback`` image as the innermost ``else`` (used when the machine's + capability is below every threshold):: ifThenElse(TARGET.GPUs_Capability >= 8.0, "./rift_container_modern.sif", "/cvmfs/.../rift_container_default.sif") @@ -141,6 +140,107 @@ For a manifest, the ILE (and CIP) Condor submit files get: supports. +* **A capability-defined ``Requirements`` clause** — + ``TARGET.GPUs_Capability =!= undefined``. The selection above is deliberately + *not* undefined-guarded: a slot that does not advertise the machine-level + capability rollup could be anything (including a Blackwell that hard-fails on + the older fallback image), so the safe action is to **not match it** rather + than guess. Measured on the CIT pool, a large fraction of GPU slots satisfy + the per-GPU ``require_gpus`` floor yet do not advertise the rollup attribute; + without this clause those jobs go on hold with "Cannot expand $$ expression". + +CPU-only jobs are handled differently. **CIP requests no GPU**, so its matched +slot advertises no capability at all and a capability-keyed selection cannot +resolve — it would hold the job. CIP therefore collapses to a **single fixed +container**: the manifest ``fallback`` image (hence the requirement that the +fallback be CPU-safe), quoted as a plain literal, with no ``$$()`` token and no +capability ``Requirements`` clause. + + +.. _osg-container-modes: + +Choosing a delivery mode (legacy vs container universe) +------------------------------------------------------- + +The expression-valued ``MY.SingularityImage`` above is evaluated on the +*execute* side. That works on a local HTCondor pool, but **OSPool glidein +pilots read** ``SingularityImage`` **as a literal string**, so an ``ifThenElse`` +lands verbatim and the job holds. Two opt-in modes solve this; pick one with an +environment variable at DAG-build time. + +.. list-table:: + :header-rows: 1 + :widths: 26 74 + + * - Mode + - Behaviour + * - *(default)* + - Legacy: ``universe = vanilla`` + expression-valued + ``MY.SingularityImage``. Correct on a local/CIT pool. **Not OSG-safe.** + * - ``RIFT_CONTAINER_UNIVERSE=1`` + - **Recommended for OSG.** ``universe = container`` and + ``container_image = $$([ ifThenElse(...) ])``. ``$$()`` is HTCondor's + documented *match-time* (schedd-side) machine-ad substitution, so the + pilot only ever sees a literal image URL. No ``MY.SingularityImage``, + no ``MY.SingularityBindCVMFS``, no ``$$()`` transfer token — the image is + delivered by ``container_image`` itself. GPU access is automatic under + ``request_gpus``. Works on CIT-local too. + * - ``RIFT_CONTAINER_RUNTIME_SELECT=1`` + - Older fallback, ILE only. Condor runs a generated + ``rift_container_select.sh`` on the bare execute node; the wrapper reads + the *real* GPU capability from ``nvidia-smi``, fetches only the matching + image (``stashcp``/``pelican``), and re-execs the command under + ``apptainer exec --nv``. Detects the actual device rather than trusting + an advertised attribute, at the cost of running outside a container. + +Under asimov, set the variable from the blueprint rather than the shell: + +.. code-block:: yaml + + scheduler: + singularity image: /path/to/rift_container_family.yaml + singularity base exe directory: /usr/local/bin/ + environment variables: + RIFT_CONTAINER_UNIVERSE: 1 + +.. note:: + + With a family manifest whose images are ``osdf://`` URLs, the pipeline also + enables the matching transfer credential automatically + (``use_oauth_services = scitokens``, or ``igwn`` for ``igwn+osdf:``). The + single-image code path keys that off the ``SINGULARITY_RIFT_IMAGE`` string + itself, which for a manifest is just a ``.yaml`` path — so the manifest's + image URLs are inspected instead. + +.. warning:: + + **Why the container-universe selector names basenames, not URLs.** + ``condor_submit`` parses ``container_image`` *before* any ``$$`` expansion and + derives the job ad's ``ContainerImage`` — the name the image will have in the + job scratch dir — as the text after the **last** ``/``. A selector containing + full paths is therefore cut in half, and the fragment that survives is not a + valid image name. Submitting that form to the IGWN pool holds the job at the + execute point:: + + PREPARE_JOB (prepare-hook) failed (reported status 001): + Unable to download or build singularity image cutest_busybox_...sif") ]) + + So the selector emits **basenames only** (no ``/``), the whole ``$$`` token + survives into ``ContainerImage``, and the schedd expands it at match time + (``MATCH_EXP_ContainerImage = "rift_container_modern.sif"``). The image itself + is delivered by the comma-free ``$$()`` transfer token, and ``MY.TransferInput`` + is pinned so ``condor_submit`` does not append the basename selector to + ``TransferInput`` as a bogus extra input file. Verified end to end on an OSPool + glidein against ``$CondorVersion: 25.11.1``. + + Consequence: **every image in a family used with container universe must be a + transferable URL.** An image referenced in place (a CVMFS or local path) can + only be named by its full path, which would reintroduce the truncation, so + :func:`~RIFT.misc.container_manifest.build_container_image_select` raises + ``ContainerManifestError`` for such a family. Stage those images at a URL, or + use ``RIFT_CONTAINER_RUNTIME_SELECT=1``. + + GPU attribute names -------------------