handbook: a step that produces nothing still reports success - #17
Merged
Conversation
Three classes from a week of live work, none of them in the handbook. F9 — the silent no-op. A skipped `when`, a swallowed call error, a variable resolving to empty: all reach the trace as successful steps with non-zero duration. One pass was placed four times before it applied anything, and every trace looked identical. All four causes were found by comparing a FACT with an expectation, never by reading the trace. F10 — a branch that never fired is dead, not rare. Measured: the only dead branches in a live catalogue sat in one description, and half its branches were never chosen while two fallbacks did the work. The threshold matters — dead branches live where traffic is thin, so count the share rather than the absolute. Context section — an extractor whose output the model reads must be bounded, because it grows with the data. A cluster-wide list killed the next step with a context-length error; the volume has to be made artificially, since stored fixtures are trimmed. Verification section, V7 — code inside steps is checked by none of the first three layers: the linter does not execute it and the step probe visits only steps with a schema. Two cheap layers close it, with the two traps that made each of them pass on any code. The library's own test caught a mistake in this text: it named a trace field as if it were part of the format. Reworded.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three classes from a week of live work, none of them in the handbook.
F9 — the silent no-op. A skipped
when, a call error swallowed byon_error: continue, a variable resolving to empty: all of them reach the traceas successful steps with non-zero duration. One pass was placed four times
before it applied anything, and every trace looked identical — all steps fine,
turn completed. The four causes were a wrong argument name, a gate testing for
the presence of success instead of the absence of failure, a step sitting
outside the branch that owns its variable, and a neighbouring loop wiping that
variable on every clean run. All four were found by comparing a FACT with an
expectation, never by reading the trace.
F10 — a branch that never fired is dead, not rare. Measured on a live
catalogue: the only dead branches sat in one description, where half the
branches were never chosen and two fallbacks did the work. The threshold
matters — a rule of "never fired in twenty runs" missed it, because dead
branches live where traffic is thin.
Context section — an extractor whose output the model reads must be
bounded, because it grows with the data. A cluster-wide list was printed in
full and the next step died with a context-length error. Volume has to be made
artificially: stored fixtures are trimmed, or they weigh too much.
Verification, V7 — code inside steps is checked by none of the first three
layers: the linter does not execute it, and the step probe visits only steps
with a schema. Two cheap layers close the gap, along with the two traps that
made each of them pass on any code.
The package's own test caught a mistake in this text — it named a trace field as
if it belonged to the format. Reworded.
🤖 Generated with Claude Code