Skip to content

feat(scheduler): the power axis — the scheduler refuses work it cannot power (bp-154) - #41

Merged
ascalva merged 8 commits into
mainfrom
build/bp-154-power-axis
Aug 12, 2026
Merged

feat(scheduler): the power axis — the scheduler refuses work it cannot power (bp-154)#41
ascalva merged 8 commits into
mainfrom
build/bp-154-power-axis

Conversation

@ascalva

@ascalva ascalva commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Implements bp-154dn-supervision-and-liveness Amendment A1, the power axis. The
supervisor could see memory and could see the owner; it could not see the battery. Now it can.

closes #12

Why (the warrant, glossed inline)

Issue #12 (energy: the scheduler cannot see the battery, migrated from frozen
docs/findings/finding-0279.md) records three measured emergencies, not the two its title
names: Jul 24 — drained to 1% during the v1.18.0 deploy night, the embedder starved under
critical-battery throttle, code_sync wedged, and the daemon died unwitnessed and stayed dead three
days; Jul 28 — 100%→8% in 2h40m during the revival's backlog drain, caught at the wire; Aug 1
— fatal, the machine died mid-run and run #39 came up in recovery.

The battery hardware is healthy and stays healthy: Condition Normal, 95% maximum capacity, 128
cycles, re-measured 2026-08-05. The drain is load, not degradation. The scheduler was the defect.

§2.7 had given the supervisor exactly one resource axis to refuse on — memory (non-negotiable #8,
"the scheduler refuses breaching work"). That principle was never memory-specific; it had simply
never been given a second axis.

What changed

file what
scheduler/power.py (new) the sensor: an injectable probe over pmset -g batt, a ~20% floor, assume_discharging_when_unknown = True
scheduler/supervisor.py power: Power field (defaulted), power_blocked_tiers(), the floor branch in tick, one term added to the union at the claim site
tests/unit/test_power.py (new) 22 sensor tests, every probe injected — no pmset subprocess runs
tests/fixtures/power.py (new) on_ac() / on_battery() / unreadable(), one home for the doubles
tests/integration/test_supervisor.py the predicate, composition, floor and hold tests
tests/integration/test_cron.py, test_research_cron.py, test_chat_sensor_wiring.py, tests/e2e/test_scheduler_live.py inject an on-AC sensor at each real-Supervisor construction site

Five commits, one per plan item, deliberately reviewable in isolation.

⚑ The one hard pin held: three predicates, three questions

The power rule is its own predicate, never folded into blocked_tiers(). That method's own
docstring already gave the reason about the single-model rule — "two different reasons to refuse a
tier, conflated into one predicate, is how a reader later cannot tell which rule refused a job"

and a power refusal answers a different question ("is there energy?") from a presence refusal ("is
the owner here?"). model_blocked_tiers() was the precedent for a sibling; this is the third. Each
now points at the other two. They compose only at the ONE claim site, by union.

HEAVY_TIERS is read, never reshaped — one shed vocabulary, not two. The tripwire
(tests/integrity/test_shadow_isolation.py, which imports it and is deliberately out of scope) is
untouched and green.

⚑ Fail closed, and one named asymmetry

None is an ordinary state here, not a theoretical one: an absent pmset, a failed or hung exec,
unreadable output, or a probe that raises. All four funnel to "discharging", because the failure
being designed against is the machine dying and a sensor that failed open would re-create it exactly
when the system is least healthy.

The one asymmetry is named and defaulted off rather than left implicit: an unknown percentage
does not put the sensor below the floor. Fail-closed on the discharging question costs the heavy
lanes; fail-closed on the floor refuses every tier, so a host with no battery to read (a desktop,
CI, any non-macOS worker) would dispatch nothing ever — the guard against an outage would be the
outage. halt_when_percent_unknown exists, off, asserted in both positions.

The floor: refuse before claim, and that is the clean close

Below ~20% while discharging, tick returns before queue.claim. No running row is minted, so
nothing is stranded for a machine that may not survive to close it, and a following run's
sweep_orphans finds nothing to reclaim or strand-fail — asserted, because the falsifier is the Aug
1 shape. The hold for AC is the absence of dispatch: run() returns 0, the launcher's existing
conditional sleep is the duty cycle, and the next tick re-reads the battery. Nothing in the diff kills
an in-flight job.

How it was verified — the falsifiers were executed, not asserted

  • Deleting | self.power_blocked_tiers() from the union reds exactly one test
    test_a_heavy_job_is_NOT_claimed_while_discharging_and_IS_once_back_on_AC (1 failed, 52 passed) —
    while every sensor and predicate test stays green. That is the finding-0187 shape caught in the act
    (deleting bp-105's sweep call once left 85/85 green), and it is why Items 2 and 3 are separate
    commits: Item 2 lands the predicate inert, Item 3's whole content is "and now it is consumed".
  • Deleting the floor branch reds four tests. The floor test enqueues a light-tier job on
    purpose, so the discharging shed provably does not cover it — the test cannot pass on the shed's
    behaviour.
  • "The hold must not spin hot" is executable: a counting probe proves a ten-tick drain request
    costs exactly ONE battery reading.
  • The test repairs are proven necessary, not assumed: with the probe forced unreadable to
    simulate the CI runner (ubuntu-latest, no pmset),
    test_cron_jobs_are_gated_during_foreground_then_run_in_a_trough fails assert trough.run() == 2
    with 0. With the injections in place under the same simulation the whole model-free tier is green
    but for three known-red worktree tests.

The local gate, each leg run separately, counts exactly as observed

leg result
uv run ruff check . exit 0 — All checks passed
uv run scripts/check_imports.py exit 0 — I2 OK, worker boundary OK
uv run mypy core agents eval ops scheduler scripts exit 0 — no issues in 263 source files
uv run mypy (argless) exit 1 — 69 errors in 20 files, the expected baseline; none in the new files
uv run python -m ops.type_gate exit 0 — membership OK, bare-ignore OK, one pre-existing parked psutil report
uv run pytest -q exit 1 — 5 failed, 2463 passed, 15 skipped in 278 s

The five reds are the three known-red-locally classes and nothing else: the finding-0103
core-self-containment ratchet, tests/e2e/test_dream_v2_live.py, and three
tests/integration/test_worktree_enforcement.py cases (#13 — green in CI).
tests/e2e/test_scheduler_live.py (the known flake, and in this plan's write scope) passed in
the gate run; an earlier full run failed it on assert captured["text"].strip() with run() == 1
and state == DONE already passing — the job dispatched and the live model returned an empty string,
which a power refusal cannot cause (a refusal fails one assertion earlier).

For the reviewer to rule on

  1. Write-scope overrun, flagged not hidden. tests/integration/test_cron.py,
    test_research_cron.py and test_chat_sensor_wiring.py are not in bp-154 §5's list. §5
    anticipated exactly this repair class — "may observe a supervisor constructed without a power
    sensor; carried so a default-construction change can be repaired in-session"
    — but enumerated
    three files where the real set of real-Supervisor construction sites in tests is six. The
    repairs are one line each and are proven necessary above.
  2. A documented deviation from a parked default, filed as the power floor holds by NOT dispatching; the process-level clean stop is deliberately unwired #40. A1's parked hold-for-AC decision
    records "stop cleanly; launchd KeepAlive restarts and re-evaluates". This PR holds by not
    dispatching
    instead, because ThrottleInterval is 10 s and every respawn pays preflight's
    uncosted ~120 s Ollama probe (frozen finding-0195) — a restart loop at the floor would spend the
    reserve the floor exists to protect. ops/lifecycle/launcher.py is also outside this plan's
    write_scope. Consequence stated honestly: the queue ledger is clean, the run ledger is
    unchanged, so a death at 0% still produces a recovery run — with no stranded jobs in it.
  3. Enforcement tier, claimed honestly: tier 5 with a tier-4 test, deliberately identical to what
    §2.7 claims for the memory ceiling. Power is a sampled reading of the physical world, so tier 1 is
    unreachable and claiming it would be the overclaim §0's ladder names as the foot-gun.
  4. The honest limit, recorded not hidden (A1.4): this bounds what is started, never what is
    already running — Jul 24's code_backfill was in flight when the throttle hit, so this would not
    have prevented that emergency outright. In-flight bounding is finding-0178's machinery.

Ready to deskcheck on real hardware: unplug and watch a synthesis-tier job stay QUEUED while the
light lanes drain, then plug in and watch it go. The code path is what the tests exercise; the
deskcheck is what proves the probe reads what we think it reads on the live daemon. (Checked by
reading rather than executing: /usr/bin/pmset is -rwxr-xr-x root:wheel, -g batt needs no
privilege, and neither plist overrides PATH — so launchd's default PATH reaches it.)

Context: #40 (parked, route:orchestrator), #13 (the locally-red enforcement suite),
bp-153 — this plan is cheap insurance for that rebuild, whose Jul 24 wedge was itself caused by
a battery drain.

🤖 Generated with Claude Code

https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U

ascalva and others added 8 commits August 8, 2026 23:36
…robe

bp-154 Item 1 (dn-supervision-and-liveness Amendment A1; issue #12). §2.7 gave
the supervisor exactly one resource axis to refuse on — memory. This is the
sensor for a second: the machine's remaining energy.

`scheduler/power.py` is deliberately shaped like `scheduler/presence.py` rather
than invented — an injectable probe (so the gate is testable with no hardware),
a floor, and the idiom that matters: a fail-CLOSED default. `Presence` reads
`assume_present_when_unknown = True`; this reads
`assume_discharging_when_unknown = True`. An unreadable battery is treated as
discharging, because the failure mode being designed against is the machine
dying and a sensor that failed open would re-create it exactly when the system
is least healthy (A1.7's named falsifier).

`None` is an ORDINARY state here, not a theoretical one, so all four paths to it
are carried from the probe precedent and tested: an absent `pmset`
(`shutil.which`), a failed or hung exec (explicit `timeout=5`, plus the
`(OSError, SubprocessError)` catch), unreadable output, and — added because an
exception escaping into `tick` would be a new crash path rather than a guard — a
probe that raises.

One named asymmetry, deliberate and defaulted off: an unknown PERCENTAGE does
not put the sensor below the floor. Fail-closed on the discharging question
costs the heavy lanes; fail-closed on the floor would refuse every tier, so a
host with no battery to read (a desktop, CI, any non-macOS worker) would
dispatch nothing ever — the guard against an outage would be the outage.
`halt_when_percent_unknown` exists, off, so the decision is visible.

Verification: `uv run pytest tests/unit/test_power.py -q` → 22 passed. No
`pmset` subprocess runs in the unit tests (every probe is injected), and the
real probe is proven not to run at import or by construction — the call counter
is asserted in both directions so the test cannot pass against a dead probe.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U
bp-154 Item 2 (dn-supervision-and-liveness A1.1). The supervisor gains a THIRD
refusal predicate beside `blocked_tiers()` (the foreground gate) and
`model_blocked_tiers()` (the single-model rule): on battery, shed the heavy
lanes.

⚑ The amendment's one load-bearing pin is honored: the power rule is its OWN
predicate and is never folded into `blocked_tiers()`. That method's own
docstring already gives the reason about the model rule — "two different reasons
to refuse a tier, conflated into one predicate, is how a reader later cannot
tell which rule refused a job" — and a power refusal answers a different
question ("is there energy?") from a presence refusal ("is the owner here?").
`model_blocked_tiers` was the precedent for a sibling; this is the third. Each
of the three now points at the other two, so all are findable from any one.

`HEAVY_TIERS` is READ, never reshaped — one shed vocabulary, not two (A1's
parked selector decision; `load_key` was rejected as the default because its
interaction with this set is undesigned). `tests/integrity/test_shadow_
isolation.py`, which imports `HEAVY_TIERS` and is deliberately out of scope,
stays green untouched.

This commit is INERT by construction: the predicate computes a set nobody
consumes yet. `tick` is untouched, so no dispatch decision changes. Item 3's
whole content is "and now it is actually consumed" — separated precisely because
a guard nobody calls is the finding-0187 shape (deleting bp-105's sweep call
left 85/85 green).

`Supervisor.power` is defaulted like `presence`, so every existing construction
site gets the guard with no edit. Because that default FAILS CLOSED, tests that
dispatch a heavy tier must inject an on-AC sensor rather than inherit the host's
real battery — `tests/fixtures/power.py` is the one place those doubles live,
with the rule about which to use (injecting on-AC into a test meant to exercise
a power refusal would hide the feature).

Verification: `uv run pytest tests/integration/test_supervisor.py -q` → 29
passed (28 pre-existing, unchanged in behaviour; the foreground gate is asserted
byte-identical under all four presence × power combinations, and a source-shape
test proves the two predicates read their own sensor and no other's).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U
bp-154 Item 3 (dn-supervision-and-liveness A1.1/A1.4). Item 2 landed the
predicate inert; this is the commit whose whole content is "and now it is
actually consumed". The union in `tick` gains one term —
`| self.power_blocked_tiers()` — and nothing else changes: no new queue API, no
second enforcement point, the ONE claim site pin intact.

⚑ The falsifier is executed, not asserted. With the new term deleted from the
union, exactly one test reds —
`test_a_heavy_job_is_NOT_claimed_while_discharging_and_IS_once_back_on_AC` —
while all 22 sensor tests and all 5 predicate tests stay green. That is the
finding-0187 shape caught in the act: a guard nobody calls passes every test
written about the guard (deleting bp-105's sweep call left 85/85 green), so the
test that matters tests the COMPOSITION.

Non-vacuity is asserted inline before the claim: the foreground gate is open,
the model rule is unarmed, and the floor is not reached (55%), so the power term
is provably the only rule that can have refused the job. The same queue is then
drained by an on-AC supervisor and the held job runs — the rule defers, it never
drops. A companion test pins the other direction: adding a term must not weaken
the terms already there (owner present + on AC still refuses the heavy lane).

Enforcement tier, honestly: tier 5 with a tier-4 test, identical to what §2.7
claims for the memory ceiling. Power is a sampled reading of the physical world,
so tier 1 is unreachable and claiming it would be the overclaim §0's ladder
names as the foot-gun.

Verification: `uv run pytest tests/integration/test_supervisor.py -q` → 31
passed; `uv run ruff check scheduler/ tests/integration/test_supervisor.py` →
clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U
…the ledger clean

bp-154 Item 4 (dn-supervision-and-liveness A1.2). Below ~20% while discharging,
the answer stops being "shed the heavy lanes" and becomes "start nothing at
all": `tick` refuses every tier, including the light ones the shed leaves alone.

The refusal sits BEFORE `claim`, and that placement is the whole content of
"close the ledger clean". No RUNNING row is minted, so nothing is left behind
for a machine that may not survive to close it, and a following run's orphan
sweep finds nothing to reclaim and nothing to strand-fail. Claiming here would
reproduce the Aug 1 shape exactly — the machine died mid-run and run #39 came up
in recovery.

The hold for AC is the ABSENCE of dispatch, not a wait loop. `run` returns 0 on
the first tick and hands control back; the launcher's existing conditional sleep
("sleep only when the drain came back idle") is the duty cycle, and the next
tick re-reads the battery, so mains returning resumes dispatch with no
intervention and no state to reset. An in-process sleep/wait was the rejected
alternative in A1's parked hold-for-AC decision: it would hold the supervisor
lock while doing nothing and would itself drain the battery it protects. That
falsifier is now checkable rather than argued — a counting probe proves a
ten-tick drain request costs exactly ONE reading.

Nothing is killed. This bounds what is STARTED; a job already in flight runs to
its own completion or checkpoint (A1.4's honest limit — in-flight energy
bounding is finding-0178's machinery, not this), and a job that finished before
the floor was reached is left DONE, asserted.

Deleting the floor branch reds four tests; the fixtures are non-vacuous by
construction (the floor test enqueues a LIGHT tier, so the discharging shed
provably does not cover it).

Verification: `uv run pytest tests/integration/test_supervisor.py -q` → 35
passed; `uv run ruff check scheduler/ tests/` → clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U
… built

bp-154 Item 5. `Supervisor.power` is defaulted like `presence` so the live
daemon gets the guard with no wiring — and because that default FAILS CLOSED, a
default-constructed supervisor in a test decides heavy-tier dispatch from the
machine the suite runs on: the developer's actual battery on macOS, and "no
`pmset` therefore discharging" on CI's ubuntu runner.

Repaired by injection, exactly as each of these already injects
`Presence(idle_probe=...)` rather than reading real HID idle time:
`test_cron.py`, `test_research_cron.py` (both dispatch synthesis-tier jobs),
`test_chat_sensor_wiring.py` and `test_scheduler_live.py` (pinned tier — the
shed does not reach them, but the floor would on a draining laptop).

⚑ Proven necessary, not assumed. With the probe forced unreadable to simulate
the CI runner, `test_cron_jobs_are_gated_during_foreground_then_run_in_a_trough`
fails `assert trough.run() == 2` with 0 — the trough half of the foreground test
never runs, because the synthesis tier is shed for want of a battery reading.
With the injection restored and the same simulation in place, the whole
model-free tier is green but for the three known-red worktree-enforcement tests
(issue #13).

Note the scope: `test_cron.py`, `test_research_cron.py` and
`test_chat_sensor_wiring.py` are NOT in bp-154's write_scope. §5 anticipated the
class of repair — "may observe a supervisor constructed without a power sensor;
carried so a default-construction change can be repaired in-session" — but
enumerated only three files; the real set of real-Supervisor construction sites
in tests is six. Flagged for review rather than quietly absorbed.

Item 5's own falsifier is respected: `on_ac()` restores the intended subject of
a test that was never about power. No test meant to exercise a power refusal is
given one — those live in `test_supervisor.py` and inject their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U
…orded

The seal entry carries the read map, the Follow-through block, the gate results
per leg exactly as observed, and the two in-build decisions with their warrant:
unknown-percentage does not halt (a named field, not an omission), and the hold
for AC is the absence of dispatch rather than a process exit (filed as #40 with
its evidence — ThrottleInterval 10 s × preflight's uncosted ~120 s Ollama probe
would out-drain the idle hold).

Also recorded: the write_scope overrun on three test files (§5 anticipated the
repair class but enumerated three of the six real-Supervisor construction sites),
and that `tests/e2e/test_scheduler_live.py`'s earlier red was the known flake —
it failed on the live model returning an empty string, one assertion AFTER
`run() == 1` and `state == DONE` had already passed, and it passed in the gate
run.
PR #41 is the door for this build; the seal's Next action now names it, and
records that the `closes #12` keyword was verified unbackticked after opening
(GitHub does not parse closing keywords inside a code span).
…tion

The module docstring enumerates what the loop does at each step, and it is the
first thing a reader meets. It listed the foreground check and the RAM ceiling
but not the axis that now refuses dispatch outright — an omission that would let
a later reader conclude presence and memory are the only reasons a job is
refused. Adds step 0 (the floor, ahead of the claim) and the power half of step
1's shed.

Comment-only; no behaviour, no test changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U
@ascalva
ascalva merged commit 6a7608b into main Aug 12, 2026
6 checks passed
ascalva added a commit that referenced this pull request Aug 12, 2026
PR #41 is the door for this build; the seal's Next action now names it, and
records that the `closes #12` keyword was verified unbackticked after opening
(GitHub does not parse closing keywords inside a code span).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

energy: the scheduler cannot see the battery — two emergencies in four days, one fatal

1 participant