Repository navigation
fix: estimate tokens at the measured 2.0 chars/token (figures were 44% low), and stop a failed or Ctrl+C'd script leaving its transcript recording the window - #98
Merged
Conversation
…unmeasured 3.5 Every script's "~N tokens est." footer, token-count.ps1 and the ztokens records divided output characters by 3.5, a prose rule of thumb that was never measured. Anthropic's count_tokens on 8.9M characters of real z-script output gives 1.97 characters per token on Claude Opus 5.5, Fable 5.1, Sonnet 5.5 and Haiku 5.5 (one shared tokenizer), 2.03 on short non-deploy output, and 2.62 on the previous tokenizer. So every figure was about 44% low. - ZHelpers.ps1: one named constant, $global:ZCharsPerToken = 2.0, with the measurement in a comment; records carry the basis "est. chars/2.0". - token-count.ps1 reads that constant instead of keeping its own 3.5. - TOKEN_SAVINGS.md: every measured figure recomputed from the same captured character counts (x1.75; an active dev day is ~46,300 tokens, not ~26,500), a rewritten measurement note, and current prices (Haiku 5.5, Sonnet 5.5, Opus 5.5, Fable 5.1). Raw-orchestration estimates were judged in tokens and are unchanged, apart from the full-rebuild floor. - tests/TokenEstimate.Tests.ps1: the ratio stays in the measured band, and the footer and record agree; it fails at 3.5.
…er leaves its transcript recording the window
Start-ZTracking starts a transcript, and only Stop-ZTracking stopped and
deleted it, on the paths that reached it. A terminating error or Ctrl+C left
the transcript in %TEMP% capturing every later command and its output in that
PowerShell window, never deleted, and the run never recorded.
- Every tracked script (16 here) runs its body in
`try { ... } finally { Stop-ZTracking -IfActive }`. A finally block runs on
an error, an exit and Ctrl+C. The body is not re-indented, so each script's
diff is three lines.
- Stop-ZTracking -IfActive is silent when the script already stopped tracking
itself, so a normal run's footer and trailer print once.
- Start-ZTracking first clears leftovers (Clear-ZTrackLeftovers): this
window's own unstopped transcript, held open and named by the Process ID in
its header (never another window's run, nor a transcript started by hand),
and files whose process has ended, after an hour's grace. A live run's file
is held open and cannot be deleted, so it is skipped.
- tests/TranscriptLeak.Tests.ps1: a parse-tree check that every script that
calls Start-ZTracking wraps the rest in that finally; throw, exit and
already-stopped runs in child processes with an isolated TMP; both kinds of
leftover, including one held open. TokenEstimate.Tests.ps1 now isolates TMP
too, because Start-ZTracking sweeps it.
…'s Stop cannot turn their stderr into a test failure CI failed two TranscriptLeak tests with "RemoteException: boom". The powershell shell in GitHub Actions runs with $ErrorActionPreference = 'Stop', and under 2>&1 Windows PowerShell turns each line a child process writes to stderr into an error record - here the fixture's own deliberate throw - which Stop then made terminate the test. Invoke-Isolated now sets Continue for the child call, local to the function. Reproduced under Stop before the fix; the suite passes under Stop after (305 passed). CHECKSUMS.txt refreshed.
kellymichels
added a commit
that referenced
this pull request
Nov 1, 2026
…p comment in 15 scripts, so they match their source (#100) The Oct 31 zpublish_zscripts run copied this from evo.scripts and left it uncommitted, as the publisher always does. The nightly test run skips a dirty tree, so evo.zscripts went unmeasured on 2026-11-01. PR #98 published this comment without the number, while the private source has carried it since its own fix, so every publish staged it again. Other files here already cite evo.scripts numbers, so this follows the same practice. Comment only; CHECKSUMS.txt refreshed by the publisher.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes to output tracking, one commit each so either can be reverted alone. Both commits pass the suite on their own.
1. Token estimates were about 44% low (
12f286e)Every script's
~N tokens est.footer,token-count.ps1and the ztokens records divided output characters by 3.5. That was a prose rule of thumb that was never measured. Anthropic'scount_tokenson 8.9M characters of real z-script output gives:ZHelpers.ps1: one named constant,$global:ZCharsPerToken = 2.0. Each record carries its basis,est. chars/2.0.token-count.ps1reads that constant instead of keeping its own 3.5.TOKEN_SAVINGS.md(and.txt): every measured figure is recomputed from the same captured character counts (×1.75). An active dev day is ~46,300 tokens, not ~26,500; a cachedzdeployis ~1,420, not ~810. The measurement note is rewritten. Prices are now the current models: Haiku 5.5, Sonnet 5.5, Opus 5.5, Fable 5.1. The raw-orchestration estimates were judged in tokens, so they are unchanged, apart from the full-rebuild floor, which can't be below the output it contains.tests/TokenEstimate.Tests.ps1(new): checks the ratio stays in the measured band and the footer and record agree. It fails at 3.5.2. An error or Ctrl+C left the transcript recording the window (
d9cded9)Tracking ended only on paths that reached
Stop-ZTracking. After a terminating error or Ctrl+C, the transcript in%TEMP%kept capturing every later command and its output in that PowerShell window. It was never deleted, and the run was never recorded.try { … } finally { Stop-ZTracking -IfActive }, which runs on an error, anexitand Ctrl+C. The body is not re-indented, so each script's diff is three lines.-IfActiveis silent when the script already stopped tracking itself, so the footer prints once.Start-ZTrackingfirst clears leftovers (Clear-ZTrackLeftovers). It handles this window's own unstopped transcript, which is held open and named by theProcess ID:in its header, and files whose process has ended, after an hour's grace. A live run's file is held open, can't be deleted, and is skipped. It never stops another window's run, or a transcript started by hand.tests/TranscriptLeak.Tests.ps1(new) contains:Start-ZTrackingwraps the rest in thatfinally;exitand already-stopped runs in child processes with an isolatedTMP;Tested
Invoke-Pester .\testswithTMPisolated: 305 passed, 0 failed (1 skipped). The commit-1 state alone passes too: 296 passed, 0 failed.tryfrom one script, the cleanup call, the-IfActiveguard or the hour's grace each fails exactly its test.CHECKSUMS.txtis refreshed in each commit (zchecksums -Update), and the plain-text twins are regenerated withscripts/plaintext_twins.py.--checkreports them in sync.The release bump comes after merge, as usual.
🤖 Generated with Claude Code