JIT-retrieval token-usage logging is write-side dead: record_read has no CLI caller

[tkcoding]

Context

The JIT-retrieval token-tracking layer landed via PRs #108–#111 (two-tier retrieval
cascade, per-section staleness, and a token-tracking/read-gate layer). Tracing the
actual code on main shows the tracking half is write-side dead:

• Read side works. decision_log.summarize_reads() + cfs usage-report (skills/studio/scripts/studio/commands/usage_report.py) correctly aggregate and display logged "read" events, per method, to the user.
• Write side doesn't exist. decision_log.record_read(method, target, lines, tokens, source) (skills/studio/scripts/studio/utils/decision_log.py:365) is fully implemented and unit-tested, but no CLI command calls it. Compare with check_gate, which has a real CLI wrapper (commands/read_gate.py → cfs read-gate) — record_read has no equivalent write-side wrapper.
• Net effect: cfs usage-report will always report "no read events logged yet," because nothing outside the Python process can currently write one.
• Also confirmed: nothing in skills/ currently calls any of cfs retrieve / heading-nav / tfidf-score / okf-status / read-gate — no skill teaches an agent to run the route→read→log flow yet. That's a separate, larger task (authoring a JIT-retrieval skill) and is explicitly out of scope here — this issue only makes the missing plumbing exist and behave correctly so that later work has something to call into.

What "using it" should look like once this is done

cfs log-read is a passive recorder, not a meter — it cannot measure tokens
itself, only write down a number a caller already has. The only realistic source
of a real number is whatever process actually executed the LLM read-and-answer
call and received real usage stats back from the model provider — in practice, an
orchestrating agent that just dispatched a subagent to do the read and got its
actual token usage in the completion result (this is how every "real, measured"
number in this investigation was obtained — never computed by a CLI). A plain human
at a terminal has no way to produce a real number out of thin air; they'd only have
one if they copied it from somewhere that already measured it.

This means the skill/agent wiring called out as "not in this plan" below isn't a
nice-to-have follow-up — it's the actual mechanism that would let cfs log-read
ever be called with real numbers, since nothing else in this repo has visibility
into real per-call token usage. Without it, --source estimated (an explicitly
labeled guess, computed by whatever caller has one, e.g. bytes÷4 on the read
target) is the only other honest option for populating --tokens at all.

1. An orchestrating agent (not a human) does a real read via a dispatched call and
   receives real token usage back from that dispatch.
2. It runs cfs log-read --method heading-nav --target doc.md --lines 340 --tokens 12480 --source real, passing through the number it just received. This appends the event via record_read (as today) — plus, in the same call:
   • Recomputes the project's cumulative real-token total via summarize_reads().
   • Checks that total against a budget (token_budget.check_budget, new).
   • In human mode, always prints a one-line running total, e.g. ↳ logged: 12,480 tokens (project total: 245,300 / 500,000, 49%). There's no daemon/timer in this CLI (each cfs call is a fresh process), so "periodic" visibility is realized as every time a real read is logged — the natural cadence retrieval actually happens at.
   • If the new total crosses the budget, prints a prominent ui.warn(...) line (the codebase's existing user-facing-warning convention — see commands/validate.py:1498 etc.; read_gate.py currently under-uses this, only returning a flag with no ui.warn even when needs_confirmation is true).
3. cfs retrieve and cfs read-gate (run before a read) get a small addition: alongside the existing line-count gate, they surface the current real cumulative total-so-far and flag when it's already at/near budget, e.g. ⚠ already at 92% of token budget (461,200 / 500,000) — this read has not started yet. This is the honest version of "warn about potential to go over budget": grounded in real prior spend, not a fabricated estimate of a read that hasn't happened yet — deliberately not inventing a bytes→tokens estimator for gating, since a guessed number could look like a measured one once it's driving a warning.
4. cfs usage-report continues to work exactly as today for the full breakdown, any time.

Budget scope: cumulative across the whole project's decision log (same scope usage-report already reports over) — no "session" concept exists in this stateless, one-process-per-invocation CLI. Per-session scoping (a --since <timestamp> filter) is a possible follow-up, not part of this issue.

Budget config: off by default (matches decision_log's own opt-in philosophy for CFS_DECISION_LOG / decisions.off), set via a CFS_TOKEN_BUDGET env var (int; unset or non-numeric = disabled) — consistent with the existing CFS_<NAME> convention (CFS_DECISION_LOG, CFS_GIT_KIT_CACHE_DIR). No project-config-file precedent exists in this codebase for a tunable like this.

Proposed changes

• New skills/studio/scripts/studio/utils/token_budget.py — pure decision logic, mirrors read_gate.py's shape (no I/O):

  def check_budget(cumulative_tokens: int, threshold: Optional[int]) -> Dict[str, Any]:
      # threshold=None -> {"enabled": False, ...}
      # returns {"enabled", "over_budget", "cumulative_tokens", "threshold", "pct"}
  def budget_threshold_from_env() -> Optional[int]:
      # reads CFS_TOKEN_BUDGET, None if unset/non-numeric

• New skills/studio/scripts/studio/commands/log_read.py — CLI wrapper modeled on commands/read_gate.py / commands/usage_report.py (ui.JsonSafeArgumentParser, ui.result(output, human_fn=...)):
  • args: --method, --target, --lines, --tokens, --source (default "real"), optional --budget (overrides CFS_TOKEN_BUDGET for this call, mirrors read-gate's --threshold override).
  • Calls decision_log.record_read(...) → decision_log.summarize_reads() → token_budget.check_budget(...).
  • _human_log_read(data): always prints the running total; ui.warn(...) when over_budget is true.
• Edit skills/studio/scripts/studio/cli.py — register log-read next to read-gate/usage-report/retrieve (help text, _COMMAND_SECTIONS, _COMMAND_HANDLERS, dispatch table).
• Edit skills/studio/scripts/studio/utils/cascade.py — route_query's result gains a usage_so_far block (from summarize_reads() + check_budget()), same pattern as the existing read_gate block it already adds.
• Edit skills/studio/scripts/studio/commands/read_gate.py — same usage_so_far addition; also make _human_read_gate call ui.warn(...) when needs_confirmation is true instead of a plain ui.substep(...), bringing it in line with the codebase's actual warning convention.
• New tests: tests/test_token_budget.py (mirrors tests/test_read_gate.py), plus additions to tests/test_decision_log.py / tests/test_cascade.py, and a new tests/test_log_read.py.

Explicitly out of scope for this issue

• No skill/agent instructions get written — nothing currently calls the JIT-retrieval CLI at all; that's a separate, larger task. Note: as explained above, this means cfs log-read lands in this issue fully working but with no realistic caller for --tokens real values until that follow-up exists — it's a real dependency, not just related work.
• No token-cost estimation before a read (no bytes→tokens heuristic) — only real, already-logged spend drives the budget warning.
• No session-scoped budget (only whole-project cumulative, matching usage-report's existing scope).

Verification

• tests/test_token_budget.py: threshold unset → enabled: False; under/at/over threshold boundaries; negative/garbage input clamped (mirrors read_gate.py's own negative-line clamp test).
• tests/test_log_read.py: a cfs log-read call appends a real event readable back via decision_log.read_events(event="read"); JSON output shape; human output contains the running-total line and, when over budget, the ⚠ warning text; --json mode never prints the human warning (matches ui.warn's existing JSON-suppression).
• Manual: CFS_TOKEN_BUDGET=100 cfs log-read --method baseline --target doc.md --lines 500 --tokens 150 --source real should print the over-budget warning; a follow-up cfs usage-report should show the same event aggregated.
• Full existing suite (pytest tests/) green, no regression in test_decision_log.py, test_cascade.py, test_read_gate.py, test_usage_report.py.

---

Work will happen on a branch in the tkcoding/studio fork; this issue tracks it upstream, PR to follow once ready.