Repository navigation
[Bug]: countTokens builds a new WASM tokenizer on every call (~70 ms each), blocking the host on long sessions #638
Description
Bug Description
Token counting builds a brand-new tokenizer on every call, which blocks OpenCode's event loop for seconds at a time on long sessions.
lib/token-utils.ts uses countTokens from @anthropic-ai/tokenizer. That function constructs a new WASM Tiktoken (loading the full Claude BPE ranks) for every call and frees it afterwards:
function countTokens(text) {
const tokenizer = getTokenizer(); // new Tiktoken(bpe_ranks, special_tokens, pat_str)
const encoded = tokenizer.encode(text.normalize('NFKC'), 'all');
tokenizer.free();
return encoded.length;
}Measured on Linux x64 with Bun 1.4.2, for a ~2,400-character string:
countTokens(text): ~70 ms per call- one
getTokenizer()reused forencode(...): ~0.19 ms per call
The counts are identical.
The call rate makes this visible. syncToolCache counts tokens for each uncached tool part on every chat-messages transform, and the cache is trimmed to MAX_TOOL_CACHE_SIZE (1,000). In a session with more than 1,000 tool calls, the evicted entries are re-counted on every request, each at ~70 ms. Because this runs on the host's main thread, every other session and API request on the same OpenCode server waits too.
We found it with a sampling CPU profile of an OpenCode server that was freezing for 3–13 s at a time during long sessions.
Expected Behavior
Token counting should cost well under a millisecond per call, and should not block the host for seconds on long sessions. Building the tokenizer once and reusing it gives exactly the same counts.
Debug Context Logs
Not applicable: this is a performance problem, not a context/pruning decision.
Sampling-profile excerpt (innermost frame first), one stalled request:
.wasm-function[...]
< Tiktoken tiktoken/lite/tiktoken_bg.cjs
< getTokenizer @anthropic-ai/tokenizer/dist/cjs/index.js
< countTokens @anthropic-ai/tokenizer/dist/cjs/index.js
< countTokens (DCP lib/token-utils.ts)
< syncToolCache (DCP)
< chat messages transform (DCP)
DCP Version
3.2.0 (current master still calls countTokens directly)
Activity
aiev commentedon Oct 19, 2026
Production data from a large session (OpenCode 2.x, DCP 3.2.0, Linux):
countTokensover the whole session text (7.8 MB) took ~3.1 s per call; 1 MB ~0.5 s; 100 KB ~0.25 s. Counting 8 MB missed 299 consecutive 10 ms timer ticks in-process./api/infowent unanswered for 9 s; the clients' health check (2 s timeout × 3 attempts) then declared the service unresponsive — the log has three such warnings in one afternoon (16:40, 16:54, 17:06 UTC), all while a compression was running, plus a SIGKILL of the service (observed via systemd, exit status 9/KILL) after a stalled stream. From the user's side this reads as "the DCP compact keeps restarting the service": the compaction dies mid-pass and is retried every ~2 minutes.countTokens→round(len / 4)), after which there were no further stalls or kills — that is the only hot path we had to neutralise to make the plugin usable on long sessions.Our fix is a superset of #639's approach: reuse one tokenizer, memoize exact counts by content (summaries/system prompt/tool outputs get re-counted on every transform, and again after tool-cache evictions), and bound the work per call (head/middle/tail samples above a threshold, plus a long-single-char-run guard — see #659). Same box, before/after:
Sampled estimates stayed within 0.6% of exact on our corpus. Happy to fold this into #639 or open it separately, whichever you prefer — code and benches live in our fork:
github.com/aiev/opencode-dynamic-context-pruning(branchfix/tokenizer-cost).