feat: in-file '# ENV:' declarations — script env requirements travel with the script

[Jammy2211]

Overview

Phase 1b of the 2026-07-23 test-maintainability plan (Phase 1a: #185, merged). Per-script env requirements currently live in ~45 pattern-keyed override blocks across the workspace smoke profiles; a moved or renamed script silently orphans its override and runs with the wrong env — the failure class behind ~7 of 23 distinct July RED-grind signatures. This task adds a one-line in-file declaration (# ENV: jax full_datasets real_plots ...) resolved by the shared autohands/env_config.py (which every repo now uses, post-#185), migrates the existing overrides into the scripts they describe, and makes the validator reject new pattern-overrides for declarable capabilities.

Plan

• Mechanism first (PyAutoHands, one PR): declaration parser + token→env mapping as ONE function hooked into apply_profile after overrides/derivation (script intent applies last, so no profile override can silently defeat it); validator round-trip + no-new-declarable-overrides guards; tests.
• Then per-repo migration PRs: convert each override block to an in-file declaration next to the script's docstring, moving the override's WHY-comment into the script; delete the override blocks (~21 autolens_workspace_test, ~15 autogalaxy_workspace_test, ~9 autofit_workspace_test, plus any in autocti_workspace_test and the three user workspaces).
• Verification per repo: resolved-env diff (resolve_clean, empty base) over every script must be EMPTY before/after migration.
• no_run.yaml stays out of scope (skip behaviour ≠ env resolution).

Detailed implementation plan

Affected Repositories

• PyAutoHands (primary — mechanism + validator)
• autolens_workspace_test, autogalaxy_workspace_test, autofit_workspace_test, autocti_workspace_test
• autolens_workspace, autofit_workspace, autogalaxy_workspace

Branch Survey

All eight repos on main, clean.

Suggested branch: feature/env-inline-declarations
Worktree root: ~/Code/PyAutoLabs-wt/env-inline-declarations/
Work classification: Workspace + PyAutoHands infra (no PyAuto* library edits)

Implementation Steps

1. Token vocabulary (env_config.py): jax → PYAUTO_DISABLE_JAX=0; full_datasets → PYAUTO_SMALL_DATASETS=0; real_plots → PYAUTO_FAST_PLOTS=0; real_search → PYAUTO_TEST_MODE=0; real_output → all four. Values SET explicitly (not unset) so the declaration is self-contained regardless of profile defaults.
2. Parser: read the first ~30 lines of the script for a # ENV: line (exact regex anchored, one line, space-separated tokens; unknown token = validator error). Reuse the header-scan idiom of navigator.py::_docstring_blocks — no new parsing framework.
3. Hook: in apply_profile after the derive_jax_markers step; the resolver needs the script's real path (runner passes rel path — resolve against the workspace root the same way the marker derivation does; the validator already walks real files).
4. Validator (validate_env_profiles.py): (a) declaration round-trip — declared script resolving the wrong value is an error; (b) strict mode: a profile override whose effect is expressible as a declaration is an error (grandfather nothing — migration removes them all); (c) unknown tokens error.
5. Migration order: autofit_workspace_test (9 overrides) first as the pilot, then autogalaxy_workspace_test (15), autolens_workspace_test (21), autocti_workspace_test (0 overrides — confirm no-op), then the three user workspaces (small profiles). Move each override's rationale comment into the script beside its declaration.
6. Per-repo verification: resolved-env diff over every script, empty base, before/after: EMPTY. Same harness pattern as refactor: collapse Family-B run_smoke env forks onto shared env_config #185 (674/0).
7. Docs: env_profile_redesign.md gains a §on declarations + the triage rule: an env fix must be justified by the script's declared intent, never by making a failure disappear (three July failures mis-diagnosed as env gaps were real library bugs: PyAutoArray#398, PyAutoFit#1415, PyAutoGalaxy#515).

Key Files

• PyAutoHands/autohands/env_config.py — apply_profile (:96), is_jax_marked (:155)
• PyAutoHands/autohands/validate_env_profiles.py — check_profile, resolve_clean
• PyAutoHands/autohands/navigator.py — _docstring_blocks (:87), the header-parse precedent
• */config/build/profile_smoke.yaml (+ release where overrides exist) — override blocks to burn down
• PyAutoHands/docs/env_profile_redesign.md — design doc update

Sizing override (recorded)

Feature Agent scored too-large (24) off repo count with a generic 4-phase split. Overridden (same precedent as #185): one task, mechanism-PR first then per-repo migration PRs, each verified by an empty resolved-env diff. The design is in the prompt; no separate design phase.

Coordination note

Concurrent task rectangular-mesh-consolidation (PyAutoArray#402) plans a FUTURE autolens_workspace_test leg (jax_grad consolidation). Its active claim is PyAutoArray only, so no conflict now; if its workspace leg starts before this task's autolens_workspace_test migration PR, serialise on that repo.

Heart status note

Heart RED (release validation, stage integrate — pre-existing, #185 shipped under authorized exception). Ship gate applies at ship time; fresh authorization or GREEN required.

Original Prompt

Click to expand starting prompt

In-file # ENV: declarations: script env requirements travel with the script (Phase 1b)

Type: feature
Target: workspaces
Repos:

• PyAutoHands
• autolens_workspace_test
• autogalaxy_workspace_test
• autofit_workspace_test
• autocti_workspace_test
• autolens_workspace
• autofit_workspace
• autogalaxy_workspace
  Difficulty: medium
  Autonomy: supervised
  Priority: high
  Status: formalised
  Blocked-by: draft/refactor/workspaces/env_resolver_family_b_unification.md (Phase 1a)

Phase 1b of the 2026-07-23 maintainability plan. DO NOT start until Phase 1a
(resolver unification) has merged — the mechanism only reaches repos that
resolve through the shared env_config.py.

The problem. Per-script env requirements live in pattern-keyed override
blocks in each workspace's config/build/profile_smoke.yaml (~21 blocks in
autolens_workspace_test, 15 autogalaxy, 9 autofit — ~45 total). A moved or
renamed script silently orphans its override: nothing fails at the config
layer, the script just runs with the wrong env. The July failure census
attributes ~7 of 23 distinct failure signatures to exactly this class.
Reading across the 45 blocks, they collapse to a small capability vocabulary:
needs-JAX, needs-full-resolution, needs-real-plots, needs-real-output,
needs-real-search.

The task.

1. Define a one-line header declaration parsed from the script file, e.g.
   # ENV: jax full_datasets real_plots (placed near the top; exact syntax
   at implementer's discretion but greppable and dumb to parse). Token →
   env-var mapping:
   • jax → PYAUTO_DISABLE_JAX=0 (union with the existing
     is_jax_marked name derivation; keep both)
   • full_datasets → unset/0 PYAUTO_SMALL_DATASETS
   • real_plots → unset/0 PYAUTO_FAST_PLOTS
   • real_search → unset/0 PYAUTO_TEST_MODE
   • real_output → all four of the above (the database/scrape shape)
2. Implement as ONE new function in PyAutoHands/autohands/env_config.py,
   called from apply_profile (env_config.py:96) in the same precedence slot
   as the existing derivation hook (:125): scrub → defaults → overrides →
   declarations/markers last, so no profile override can silently defeat a
   script's declared intent. Reuse the header-parsing idiom already proven in
   autohands/navigator.py (_docstring_blocks :87) rather than inventing a
   new parser.
3. Extend validate_env_profiles.py: (a) declarations must round-trip — a
   declared script resolving the wrong value is an error; (b) NEW
   pattern-overrides in profiles that duplicate a declarable capability are
   an error under strict mode (grandfather nothing — see step 4); (c) keep
   the dead-pattern and legacy-name guards as-is.
4. Migrate: convert the ~45 existing override blocks to in-file declarations
   across the listed repos, then delete the override blocks. Each override's
   WHY-comment (many are small essays) moves into the script next to its
   declaration — the knowledge colocates too, not just the flag.
   Verification is the established gate: resolved-env diff over every script
   (resolve_clean, empty base) must be EMPTY before/after per repo.
5. Keep no_run.yaml out of scope — skip-behaviour is a separate path
   (build_util.should_skip), not env resolution. Do not merge the two.

Triage rule to encode in the docs (from the July census): an env fix must
be justified by the script's declared intent, never by making a failure
disappear — at least three July failures first mis-diagnosed as env gaps were
real library bugs (Convolver cap PyAutoArray#398, empty-HDU PyAutoFit#1415,
SersicCore PyAutoGalaxy#515). Declarations make "add an override until it
passes" unavailable; the docs should say why.

[Jammy2211]

Shipped — 6 PRs open (under human-acknowledged Heart YELLOW: workspace validation not passing (13 failed, 2026-07-21T19-05-22Z); 33 stale parked script(s) — both pre-existing).

Repo	PR	Content
PyAutoHands	#188	mechanism: parser + resolver hook + validator (merge FIRST)
autofit_workspace_test	#72	8 overrides → 16 scripts declared (pilot)
autogalaxy_workspace_test	#89	all 15 overrides → 52 scripts, overrides: []
autolens_workspace_test	#207	20 of 21 → 74 scripts (database/scrape/ stays — numpy→JAX release-flip case, validator carve-out)
autolens_workspace	#326	5 migrated + 3 split → 53 scripts; 1 redundant release override removed
autogalaxy_workspace	#152	1 migrated + 3 split → 36 scripts; 1 redundant release override removed

No-change repos (verified clean): autocti_workspace_test (zero overrides), autofit_workspace (both overrides non-migratable: release pins TEST_MODE=1 + non-declarable SKIP_FIT_OUTPUT).

Merge order: #188 first; the five migration PRs in any order after. Post-merge follow-up: one-line PyAutoHeart PR adding --strict-declarations to the reusable smoke workflow (deliberately sequenced after all migrations merge; all seven repos already validate 0-error under it).

Verification recap: 231 scripts declared; smoke resolution byte-identical to HEAD in every repo; release diffs exclusively in the reader-verified "0"→absent class (absent ≡ "0"/off for all four vars, file:line evidence in docs §10); 189 PyAutoHands tests pass.