maintenance: purge 50 zero-match no_run.yaml entries + 11 dead env_vars patterns

[Jammy2211]

Overview

A full census of every config/build/ path-pattern config across all 10 build targets in PyAutoHands/autobuild/config/workspaces.yaml found that 50 of 122 no_run.yaml entries (41%) match zero files, plus 11 dead env_vars.yaml override patterns. The rot is concentrated in three repos — HowToLens (28/30 dead), HowToGalaxy (15/16), autolens_workspace (7/27) — and the cause is not gradual drift but bidirectional copy-paste between the HowTo* repos and their parent workspaces. One entry, plot/visuals, turned out to be an active silent failure rather than cruft: its skip stopped applying a year ago when the file moved, so a script marked "Bugged" has been running in the build unnoticed ever since.

This closes the "~44 zero-match entries remain, needs a deliberate call before purging" item deliberately left open by the 2026-07-22 ell_comps/HowTo no_run sweep (autogalaxy_workspace#143, HowToGalaxy#34/#35, HowToLens#44).

Plan

• Purge every zero-match entry from no_run.yaml in HowToLens, HowToGalaxy and autolens_workspace, including bare-stem prospective guards — a guard for a script that does not exist is unverifiable, and is exactly what produced this mess.
• Purge the 11 dead env_vars.yaml / env_vars_release.yaml override patterns in HowToLens, HowToGalaxy, HowToFit and autogalaxy_workspace_test.
• Drop plot/visuals as stale rather than re-spelling it: the script was tested under the real build env profile and passes clean (exit 0, no output), so the "Bugged" comment is obsolete.
• Add a header comment to each cleaned no_run.yaml recording the matcher rule and that entries must not be copied between repos — the copy-paste is the actual defect; a purge alone just resets the clock.
• Verify with the real matcher that every surviving entry matches at least one file, then run the build over the two HowTo repos to confirm nothing that had been skipped starts failing.

Detailed implementation plan

Affected Repositories

• HowToLens (primary)
• HowToGalaxy
• autolens_workspace
• HowToFit
• autogalaxy_workspace_test

Branch Survey

Repository	Current Branch	Dirty?	Behind origin/main
./HowToLens	main	clean	0
./HowToGalaxy	main	clean	0
./autolens_workspace	main	clean	0
./HowToFit	main	clean	1 (pull first)
./autogalaxy_workspace_test	main	clean	0

Suggested branch: feature/no-run-config-purge
Worktree: ~/Code/PyAutoLabs-wt/no-run-config-purge/

Concurrent claim (human-approved). All five repos are claimed in active.md by the parked rename-autobuild-to-autohands task (blocked on Heart RED, nothing committed). The file sets were verified disjoint: grep -n 'autobuild|PyAutoHands|PyAutoBuild' across every file this task edits returns nothing. The only build config mentioning PyAutoHands is markdown_examples.yaml, which is already renamed on main and is not edited here (it is clean in every repo). Proceeding concurrently on a separate branch and worktree was explicitly approved.

Brain phase-split override (human-approved). The Feature Agent scored this too-large (19) and proposed a 4-phase split (design / core_api / workspace_examples / docs). The score is repo-count-driven; the change is ~61 lines of YAML deleted across 8 files with no code, API or docs surface. Shipping as a single pass.

Census method

Entries were evaluated with the real matcher, should_skip in PyAutoHands/autobuild/build_util.py:143:

• an entry containing / is a substring match against the full path including extension;
• an entry without / matches the file stem exactly.

run_all walks only scripts/ (PyAutoHands/autobuild/run_all.py:78).

Per-repo result:

Repo	Entries	Zero-match
autolens_workspace	27	7
HowToLens	30	28
HowToGalaxy	16	15
autolens_workspace_test	23	0
autogalaxy_workspace	10	0
autogalaxy_workspace_test	9	0
autofit_workspace	5	0
autofit_workspace_test	2	0
HowToFit	0	0
euclid_strong_lens_modeling_pipeline	0	0

markdown_examples.yaml and visualise_notebooks.yaml are clean in every repo.

Root cause: bidirectional copy-paste

Each dead entry was attributed to the repo where its file actually lives:

• HowToLens' list is a copy of autolens_workspace's. 18 of its 28 dead entries (gui/mask, cluster/modeling, mass_stellar_dark/slam, time_delays, hpc/example_cpu, imaging/features/advanced/subhalo/sensitivity/, …) name real files that exist only in autolens_workspace. HowToLens contains nothing but chapter_*/tutorial_*.py, so none of those paths can ever exist there.
• HowToGalaxy ← autogalaxy_workspace, same mechanism, 11 of 15.
• The copy went back the other way too: autolens_workspace carries tutorial_searches, tutorial_5_borders and tutorial_6_model_fit — HowTo tutorial stems that cannot exist in a workspace.
• Only 3 of the 46 HowTo entries are live: tutorial_searches (both repos) and tutorial_5_borders (HowToLens).

Implementation Steps

1. Create branch + worktree feature/no-run-config-purge across the five repos. Pull HowToFit first (1 behind origin/main).

2. HowToLens/config/build/no_run.yaml — delete 28 of 30 entries, keep only tutorial_searches and tutorial_5_borders. Deleted: guides/results/examples/samples, guides/results/examples/samples_via_aggregator, guides/results/examples/queries, guides/results/database/start_here, tutorial_6_model_fit, gui/extra_galaxies_centres, gui/mask_extra_galaxies, gui/lens_light_centre, gui/mask, gui/positions, deflections, deflections_source_snr, fits_make, png_make, time_delays, hpc/example_cpu, examples/searches, data_fitting, cluster/modeling, cluster/start_here, mass_stellar_dark/modeling, mass_stellar_dark/slam, shapelets/modeling, detect/database, imaging/features/advanced/subhalo/sensitivity/, plot/visuals, point_source/features/double_einstein_cross/simulator, dataset/imaging/slacs1430+4105.

3. HowToGalaxy/config/build/no_run.yaml — delete 15 of 16, keep only tutorial_searches. Deleted: guides/results/start_here, guides/results/examples/galaxies_fit, guides/results/examples/samples, guides/results/examples/models, guides/results/workflow/csv_make, guides/results/database/start_here, gui/light_centre, gui/mask_extra_galaxies, gui/mask, gui/extra_galaxies_centres, fits_make, png_make, examples/searches, data_fitting, ellipse/database.

4. autolens_workspace/config/build/no_run.yaml — delete the 7 zero-match entries: tutorial_searches, tutorial_5_borders, tutorial_6_model_fit (HowTo stems that leaked in), deflections, deflections_source_snr (no such file in any repo or in history), dataset/imaging/slacs1430+4105 (points outside scripts/, so the walker never sees it — its own comment admits this), and plot/visuals (see step 6). The other 20 entries are live and untouched.

5. env_vars.yaml siblings — delete 11 dead override patterns:

   • HowToLens/config/build/env_vars.yaml: imaging/start_here, interferometer/start_here, group/start_here, multi/start_here, guides/results/ — all autolens_workspace paths.
   • HowToGalaxy/config/build/env_vars.yaml: guides/results/.
   • HowToFit/config/build/env_vars.yaml: plot/emcee_plotter, plot/zeus_plotter — legacy, those scripts are gone.
   • autogalaxy_workspace_test/config/build/env_vars.yaml: quantity/visualization.py, quantity/visualization_jax; and env_vars_release.yaml: quantity/visualization_jax.

6. plot/visuals — resolved by test, not by guess. The file moved scripts/plot/visuals.py → scripts/guides/plot/examples/visuals.py in commit 9e1a50dc4 (2025-07-18); the pattern was never updated, so the skip has not applied for a year. Run under the exact build env profile (via autobuild repro_command):

   (cd autolens_workspace && env PYAUTO_TEST_MODE=2 PYAUTO_SKIP_FIT_OUTPUT=1 \
     PYAUTO_SKIP_VISUALIZATION=1 PYAUTO_SKIP_CHECKS=1 PYAUTO_DISABLE_JAX=1 \
     PYAUTO_FAST_PLOTS=1 JAX_ENABLE_X64=True NUMBA_CACHE_DIR=/tmp/numba_cache \
     MPLCONFIGDIR=/tmp/matplotlib python3 scripts/guides/plot/examples/visuals.py)
   

   Result: exit 0, no output, no warnings. The "Bugged, but visuals being refactored soon" comment is obsolete. Entry dropped as stale in both repos; no corrected pattern re-added.

7. Add a header comment to each cleaned no_run.yaml stating the matcher rule (/ = substring incl. extension, no / = exact stem) and that entries are repo-local and must not be copied between repos.

8. Verify. Re-run the census script — every surviving entry must match ≥1 file in its own repo. Then autobuild run_all howtolens and autobuild run_all howtogalaxy to confirm nothing that had been skipped now fails.

9. Ship via ship_workspace → 5 PRs, stopping at PR-open. Heart is currently RED (release validation integrate:fail, 646p/13f/78s, 33 stale parked scripts) — unrelated to this task, but merge stays a human act regardless.

Notes on specific entries

• tutorial_6_model_fit (HowToLens + autolens_workspace) matches no file in any repo's history. tutorial_8_model_fit.py exists in HowToLens chapter_4, but the intended target is unrecoverable, so the entry is deleted as part of the purge rather than repaired.
• Bare-stem guards (fits_make, png_make, data_fitting, deflections, examples/searches) are purged too, per explicit decision. If such a script is ever added, the build fails loudly and the entry can be re-added with a real, verifiable reason.
• dataset/imaging/slacs1430+4105 is inert because it points outside scripts/. The directory itself is untouched — only the useless skip entry is removed.

Key Files

• HowToLens/config/build/no_run.yaml — 30 entries → 2
• HowToGalaxy/config/build/no_run.yaml — 16 entries → 1
• autolens_workspace/config/build/no_run.yaml — 27 entries → 20
• HowToLens/config/build/env_vars.yaml — 5 dead override patterns removed
• HowToGalaxy/config/build/env_vars.yaml — 1 removed
• HowToFit/config/build/env_vars.yaml — 2 removed
• autogalaxy_workspace_test/config/build/env_vars.yaml — 2 removed
• autogalaxy_workspace_test/config/build/env_vars_release.yaml — 1 removed
• PyAutoHands/autobuild/build_util.py:143 — should_skip, the matcher the census was run against (read-only reference, not edited)

Original Prompt

Click to expand starting prompt

no_run.yaml (and sibling build configs) are 41% dead entries from copy-paste

Type: maintenance
Target: HowToLens
Repos:

• HowToLens
• HowToGalaxy
• autolens_workspace
• HowToFit
• autogalaxy_workspace_test
  Difficulty: small
  Autonomy: supervised
  Priority: normal
  Status: formalised

Original request (verbatim)

clean up no_run.yaml fiels in HowTo and other repos, which clearly are not in
sync with their actual contents and have a lot of stuff from legacy old code
etc. do a full census of all configs of this nature.

Census (complete, 2026-07-23)

Evaluated every entry in every config/build/ path-pattern config across all 10
build targets in PyAutoHands/autobuild/config/workspaces.yaml, using the real
matcher (PyAutoHands/autobuild/build_util.py:143 should_skip): an entry
containing / is a substring match against the full path including extension;
an entry without / matches the file stem exactly. run_all walks only
scripts/ (run_all.py:78).

no_run.yaml — 122 entries, 50 zero-match (41%)

Repo	Entries	Zero-match
autolens_workspace	27	7
HowToLens	30	28
HowToGalaxy	16	15
autolens_workspace_test	23	0
autogalaxy_workspace	10	0
autogalaxy_workspace_test	9	0
autofit_workspace	5	0
autofit_workspace_test	2	0
HowToFit	0	0
euclid_strong_lens_modeling_pipeline	0	0

Root cause: bidirectional copy-paste, not gradual drift

Each dead entry was attributed to the repo where its file actually lives:

• HowToLens' list is a copy of autolens_workspace's. 18 of its 28 dead
  entries (gui/mask, cluster/modeling, mass_stellar_dark/slam,
  time_delays, hpc/example_cpu,
  imaging/features/advanced/subhalo/sensitivity/, …) name real files that
  exist only in autolens_workspace. HowToLens contains nothing but
  chapter_*/tutorial_*.py, so none of those paths can ever exist there.
• HowToGalaxy ← autogalaxy_workspace, same mechanism, 11 of 15.
• The copy went back the other way too: autolens_workspace carries
  tutorial_searches, tutorial_5_borders and tutorial_6_model_fit —
  HowTo tutorial stems that cannot exist in a workspace.
• Only 3 of the 46 HowTo entries are live: tutorial_searches (both) and
  tutorial_5_borders (HowToLens).

One entry is an active silent failure, not cruft

plot/visuals (autolens_workspace + HowToLens), commented "Bugged, but visuals
being refactored soon." The file moved scripts/plot/visuals.py →
scripts/guides/plot/examples/visuals.py in commit 9e1a50dc4 (2025-07-18).
The pattern was never updated, so the skip stopped applying a year ago and the
script has been running in the build ever since. Deleting the entry is the wrong
fix — the pattern must be retested, then corrected or dropped on evidence.

dataset/imaging/slacs1430+4105 is inert for a different reason: it points
outside scripts/, so the walker never sees it. Its own comment admits this.

Siblings are contaminated the same way — 11 more dead entries

Same copy-paste mechanism, in env_vars.yaml / env_vars_release.yaml:

• HowToLens: imaging/start_here, interferometer/start_here,
  group/start_here, multi/start_here, guides/results/ — autolens_workspace
  paths.
• HowToGalaxy: guides/results/.
• HowToFit: plot/emcee_plotter, plot/zeus_plotter — legacy, those
  scripts are gone.
• autogalaxy_workspace_test: quantity/visualization.py and
  quantity/visualization_jax (the latter in both env_vars files).

markdown_examples.yaml and visualise_notebooks.yaml are clean in every repo.

Scope decisions already taken by the human

1. Purge every zero-match entry, including bare-stem prospective guards
   (fits_make, png_make, data_fitting, deflections, examples/searches).
   A guard for a script that does not exist is unverifiable and is exactly what
   produced this mess; if such a script is ever added the build fails loudly and
   the entry can be re-added with a real reason.
2. plot/visuals gets tested, not guessed. Run
   scripts/guides/plot/examples/visuals.py under the build env profile; if it
   passes, drop the entry as stale, if it still fails, re-add a correctly-spelled
   path entry with a fresh reason.

Notes

• This closes the "~44 zero-match entries remain, needs a deliberate call before
  purging" item left open by the 2026-07-22 ell_comps/HowTo no_run sweep
  (autogalaxy_workspace#143, HowToGalaxy#34/refactor!: point the Colab bootstrap at autonerves (renamed from autoconf) #35, HowToLens#44).
• tutorial_6_model_fit (HowToLens + autolens_workspace) matches no file in any
  repo's history; it is deleted as part of the purge rather than repaired,
  since the intended target is unrecoverable.
• Config-only change. No script, notebook or library edits expected, beyond
  whatever the plot/visuals test verdict implies.

[Jammy2211]

Implementation complete — PARKED AT THE SHIP GATE on Heart RED

All 8 config files are edited and verified in the worktree ~/Code/PyAutoLabs-wt/no-run-config-purge (branch feature/no-run-config-purge), but nothing is committed or pushed. ship_workspace step 3 is RED → stop, and this task repairs none of the RED reasons, so the narrow corrective-PR exception does not apply.

RED reasons, verbatim from pyauto-heart readiness --json (score 0, 2026-07-23T11:07:05Z):

• PyAutoFit: 1 commit(s) behind origin
• PyAutoLens: 1 commit(s) behind origin
• release validation FAILED (stage integrate)
• workspace validation not passing (13 failed, 2026-07-21T19-05-22Z)
• 33 stale parked script(s)
• manifest drift: tenant firewall (organ code) — 2 mismatch(es) vs PyAutoMind/repos.yaml

None is caused by this task.

What was done

Repo	File	Change
HowToLens	config/build/no_run.yaml	30 entries → 2
HowToGalaxy	config/build/no_run.yaml	16 → 1
autolens_workspace	config/build/no_run.yaml	27 → 20
HowToLens	config/build/env_vars.yaml	5 overrides → overrides: []
HowToGalaxy	config/build/env_vars.yaml	1 → overrides: []
HowToFit	config/build/env_vars.yaml	2 → overrides: []
autogalaxy_workspace_test	config/build/env_vars.yaml	2 removed (15 remain)
autogalaxy_workspace_test	config/build/env_vars_release.yaml	1 removed (8 remain)

Config-only: zero script, notebook or library edits, so no notebook regeneration is needed.

Verification (already done — quote this in the PRs)

1. Census re-run: 0 zero-match entries across all 10 build targets, down from 50 no_run + 11 env_vars.
2. Proven behaviour-neutral, not sampled. For all 532 scripts across the 5 repos, both the resolved should_skip set and the fully-resolved build_env_for_script dict are identical before vs after — because every removed entry matched zero files. A run_all pass would be strictly weaker evidence than this, so it was not run.
3. plot/visuals resolved by test, not by guess: ran clean under the real build env profile (exit 0, no output), so the entry was dropped as stale rather than re-spelled to the moved path.
4. overrides: [] is written explicitly wherever a list emptied — env_config.py:73 reads .get("overrides", []), so a bare overrides: key would load as None and raise on iteration.

Two corrections to the issue body above

• HowToFit's two patterns are not "legacy, those scripts are gone." They are the same copy-paste mechanism: plot/emcee_plotter and plot/zeus_plotter were lifted from autofit_workspace/config/build/env_vars.yaml lines 26/28, where the identical patterns still live and still match. Those scripts exist only as autofit_workspace/scripts/plot/*.py and never existed in HowToFit. This makes the copy-paste finding unanimous across all three HowTo repos.
• autogalaxy_workspace_test's three entries have a different cause from the copy-paste: commit 49e53de (2026-05-22), "chore: remove quantity scripts and config entries", deleted scripts/quantity/ but left three of its own config entries behind — an incomplete removal.

To resume

Re-run pyauto-heart readiness --json. On GREEN (or acknowledged YELLOW), re-run /ship_workspace — the edits are already in the worktree; it only needs to re-verify and open the 5 PRs. Do not git clean or remove that worktree in the meantime.

[Jammy2211]

Shipped — 5 PRs open (human override of the Heart RED gate)

The human explicitly authorized shipping despite Heart RED. Recording that plainly: the gate was not satisfied, it was overridden.

Supporting context found independently by the concurrent rename-autobuild-to-autohands session: the RED is largely stale evidence. release validation FAILED (stage integrate) traces to run 29912642195, whose 3 script failures were fixed by autolens_workspace f582fb7f5 — merged nine hours after that run started. The "13 failed" workspace-validation reason is dated 2026-07-21, also pre-fix. Only the manifest-drift reason is live, and it is unrelated to this task.

PRs

Repo	PR	Change	Smoke
HowToLens	#47	no_run 30→2, env_vars 5→[]	6/6
HowToGalaxy	PyAutoLabs/HowToGalaxy#37	no_run 16→1, env_vars 1→[]	4/4
autolens_workspace	PyAutoLabs/autolens_workspace#319	no_run 27→20	11/11
HowToFit	PyAutoLabs/HowToFit#26	env_vars 2→[]	10/10
autogalaxy_workspace_test	PyAutoLabs/autogalaxy_workspace_test#82	env_vars 2 + env_vars_release 1 removed	34/34

All labelled pending-release. 65/65 smoke scripts passed across the five repos.

Commits: 3eaf95e, ae83995, d77554044, 2fec2ea, ed72a20.

Final verification

• Census re-run: 0 zero-match entries across all 10 build targets — down from 50 no_run + 11 env_vars.
• Proven behaviour-neutral, not sampled: across all 532 scripts in the 5 repos, the resolved should_skip set and the fully-resolved build_env_for_script dict are byte-identical before vs after, because every removed entry matched zero files.
• plot/visuals ran clean (exit 0) under the real build env profile, so it was dropped as stale rather than re-spelled.

Merging is left to a human. Nothing here is merged.

[Jammy2211]

Complete — all 5 PRs merged

Repo	PR	Merged
HowToLens	#47	✅
HowToGalaxy	PyAutoLabs/HowToGalaxy#37	✅
autolens_workspace	PyAutoLabs/autolens_workspace#319	✅
HowToFit	PyAutoLabs/HowToFit#26	✅
autogalaxy_workspace_test	PyAutoLabs/autogalaxy_workspace_test#82	✅

Verified on merged main: the census re-runs at 0 zero-match entries across all 10 build targets and 0 dead sibling patterns — down from 50 dead no_run.yaml entries and 11 dead env_vars override patterns. 65/65 smoke scripts passed before merge.

Worktree removed, feature branches deleted locally and on the remote. Completion record written to PyAutoMind/complete/2026/07/no-run-config-purge.md.

Left open, deliberately

The clean repos are worth understanding: autolens_workspace_test carries 23 entries with zero dead, as do autogalaxy_workspace, autofit_workspace, the other two _test workspaces and the euclid pipeline. Whatever discipline keeps those honest is exactly what the HowTo repos lacked. Not investigated here to avoid widening scope.

Also noted for the record: tutorial_6_model_fit was deleted rather than repaired — it matched no file in any repo's history, and though tutorial_8_model_fit.py exists in HowToLens chapter_4, the intended target was unrecoverable. If that skip was load-bearing for a real script, it will now surface as a failure rather than silence.