fix: validate adapt-image cache against dataset mask (stale cache IndexError)

[Jammy2211]

Overview

The 2026-07-21 census reported an IndexError: index 177 is out of bounds for axis 0 with size 177
in mapper_util.adaptive_pixel_signals_from, diagnosed as an off-by-one in multi-plane
(double-Einstein-ring) adapt-data indexing. That diagnosis is wrong. adaptive_pixel_signals_from
is correct and the bug is not multi-plane specific.

The real defect is in PyAutoGalaxy: galaxy_name_image_dict_via_result_from caches per-galaxy adapt
images to files/galaxy_images_snr.fits, keyed only by the search identifier (model + search).
The dataset mask is not part of that key, so when dataset shaping changes while the model stays
identical, a stale cache from a different mask is silently loaded — producing an adapt image of the
wrong length, which blows up much later as a raw IndexError deep in the inversion.

Plan

• Validate the cached mask against the expected dataset mask when loading the adapt-image cache; treat
  a mismatch as a cache miss so the images are recomputed and the cache overwritten.
• Correct the docstring that currently claims cache staleness is "structurally guarded" — that
  guarantee only covers model/search changes, not dataset/mask changes.
• In PyAutoArray, replace the bare IndexError with a clear exception naming the stale-cache cause, so
  any future mask mismatch is legible at the point it occurs rather than 5 frames deep.
• Add numpy-only unit tests on both sides.
• Clear the stale pre-docs: add nufftax + FINUFFT citation guidance #396 caches from the census output/, confirm the double-Einstein-ring SLaM
  script runs green, and correct the NEEDS_FIX marker with the real diagnosis.

Detailed implementation plan

Affected Repositories

• PyAutoGalaxy (primary)
• PyAutoArray (loud guard)
• autolens_workspace (census marker + stale output cleanup)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoGalaxy	main	clean
./PyAutoArray	main	clean
./autolens_workspace	main	dirty (pre-existing regenerated dataset binaries, unrelated)

Suggested branch: feature/adapt-image-cache-mask-validation

Evidence

Reproduction matrix — scripts/imaging/features/advanced/double_einstein_ring/slam.py,
PYAUTO_TEST_MODE=2:

Data size	output/ state	Result
full (100x100)	cleared	PASS (exit 0, masks equal: True, 2828/2828)
PYAUTO_SMALL_DATASETS=1 (16x16)	cleared	PASS (208/208)
either	census output/ left in place	IndexError: index 177 out of bounds for size 177

The failing expression is the subscript adapt_data[slim_index_for_sub_slim_index], not the
xp.take — the traceback's ~~~~^^^^ marker pins it.

PyAutoArray 656be94b (#396, merged 2026-07-19, current main HEAD) changed the
PYAUTO_SMALL_DATASETS cap from 15x15 to 16x16. The census ran against an output/ holding pre-#396
caches: cached adapt image 15x15 = 177 pixels, current dataset mask 16x16 = 208 pixels.
Identical model => identical search identifier => stale cache silently loaded. Slim indices run to 207
against 177-length data, so the first out-of-range index is 177 — which only looks like an
off-by-one.

Implementation Steps

1. PyAutoGalaxy/autogalaxy/analysis/adapt_images/adapt_images.py
   • _galaxy_image_dict_from_cache(cache_path, mask=None): after rebuilding the cached Mask2D from
     HDU 0, compare against the expected mask; on mismatch return None so the caller recomputes and
     overwrites the cache.
   • galaxy_name_image_dict_via_result_from: pass mask=result.mask. Result.mask resolves to
     analysis.dataset.mask (autogalaxy/analysis/result.py:65), which is cheap and does not
     rebuild the max-log-likelihood fit — the caching win from autolens_profiling#70 is preserved.
   • Fix the staleness claim in the docstring (currently around line 127).
2. PyAutoArray/autoarray/inversion/mappers/abstract.py
   • Mapper.pixel_signals_from: raise a clear exception when
     len(adapt_data) != over_sampler.mask.pixels_in_mask, naming the stale adapt-image cache as the
     likely cause. A loud crash, not a silent guard — no fallback, no clamping.
3. Tests (numpy only — no JAX in library unit tests)
   • PyAutoGalaxy: write a cache under mask A; loading with expected mask B is a miss, with mask A is a
     hit.
   • PyAutoArray: adapt_data whose length disagrees with the mask raises the clear exception.
4. Verification
   • Plant a deliberately stale cache and re-run the DSPL SLaM script; confirm it recovers (recomputes)
     instead of crashing.
   • Re-run clean at full size and under PYAUTO_SMALL_DATASETS=1.
5. Census follow-up
   • Clear pre-docs: add nufftax + FINUFFT citation guidance #396 adapt caches from output/, re-run to confirm green, correct/remove the NEEDS_FIX
     marker with the real diagnosis.

Key Files

• PyAutoGalaxy/autogalaxy/analysis/adapt_images/adapt_images.py — cache read/write + staleness docstring
• PyAutoGalaxy/autogalaxy/analysis/result.py — Result.mask, the cheap expected-mask accessor
• PyAutoArray/autoarray/inversion/mappers/abstract.py — pixel_signals_from, where the mismatch surfaces
• PyAutoArray/autoarray/inversion/mappers/mapper_util.py — adaptive_pixel_signals_from (correct, not changed)

Original Prompt

Click to expand starting prompt

The original census prompt diagnosed this as an off-by-one in PyAutoArray
mapper_util.adaptive_pixel_signals_from, targeting PyAutoArray +
autolens_workspace, and asked to "fix the indexing in mapper_util". That diagnosis was
disproved by reproduction (see Evidence above); the prompt file
PyAutoMind/active/double_einstein_ring_adaptive_pixel_signals_indexerror.md retains the original
report verbatim alongside the corrected analysis.

[Jammy2211]

Library PRs created

• fix: validate adapt-image cache against the dataset mask #517 — cache mask validation (primary fix)
• fix: raise a clear exception when adapt data and the data mask disagree PyAutoArray#399 — clear exception replacing the bare IndexError

Both labelled pending-release.

Verification

Reproduced the census failure deterministically by planting a genuine stale cache — the loose files/ copy
and the archived copy inside each search's .zip, since restore() re-extracts the zip and defeats
poisoning only the loose file. The planted caches are 15x15 / 177 pixels, matching the census exactly.

• stock main: IndexError: index 177 is out of bounds for axis 0 with size 177 — same frame, same numbers
• these branches: exit 0, full pipeline through source_pix[2] with the likelihood actually evaluated

Suites: PyAutoGalaxy 1006 passed, PyAutoArray 927 passed.

Census follow-up

• No NEEDS_FIX marker exists for double_einstein_ring — the parallel chat did not re-add one, so there
  was nothing to remove.
• No stale adapt caches remain anywhere under autolens_workspace/output/.

Spun out

PyAutoLabs/PyAutoFit#1414 — Paths.preserve_in_zip only adds a member absent from the zip and never
replaces one, so a recomputed cache is written loose but the archived stale copy survives restore(). A
stale search therefore misses its cache on every run rather than once. Correctness is unaffected; only the
caching win is lost until the output is cleared. Recorded in the docstring.

Shipped behind Heart YELLOW

Score 52, no RED, acknowledged by the human. Reasons (none related to this change): workspace validation not
passing (10 failed, 2026-07-20); 58 stale parked scripts; 10 slow scripts; PyAutoCTI open PR 1390d old;
release validation stale (source moved since rehearsal).

[Jammy2211]

Shipped

Both PRs merged to main:

• fix: validate adapt-image cache against the dataset mask #517 — 64790e7d (cache mask validation, primary fix)
• fix: raise a clear exception when adapt data and the data mask disagree PyAutoArray#399 — c37e478c (clear exception replacing the bare IndexError)

All CI green on both (unittest 3.12 / 3.13, plus docs-build on PyAutoGalaxy).

No workspace change was required — the double_einstein_ring scripts were never at fault, and no
NEEDS_FIX marker existed for them.

Remaining follow-up: PyAutoLabs/PyAutoFit#1414 (preserve_in_zip never replaces an existing archive
member, so an invalidated cache is recomputed every run instead of being repaired on disk). Correctness is
unaffected by that; only the caching win is lost until the affected output is cleared.

Closing.