fix(analytics): accurate log/cost reporting and always-on cost display - #530
Open
8nevil8 wants to merge 4 commits into
Open
fix(analytics): accurate log/cost reporting and always-on cost display#5308nevil8 wants to merge 4 commits into
8nevil8 wants to merge 4 commits into
Conversation
…Log=true The cost enricher's loadAgentSessionFile trusted a correlation record's agentSessionFile path without checking it still exists. A coding agent's own retention (e.g. Claude Code's default 30-day cleanup) can rotate a transcript away long after the session ran; the correlation record still names it. 122 of 291 real Claude sessions in one report claimed hadLog=true against a deleted file, misrepresenting the Coverage table's "Native log" count. Generated with AI Co-Authored-By: codemie-ai <codemie.ai@gmail.com> Claude-Session: https://claude.ai/code/session_0146TJLVz6A1KcRoGRMn3MJx
…tart payloads hasOwnershipMarker's bounded transcript scan for legacy sessions without a sidecar marker used a 4KB byte budget capped to the first 10 lines. A SessionStart hook can inject a large hookAdditionalContext blob (CLAUDE.md, memory files, ...) as one of the transcript's first lines — observed up to ~100KB on real sessions — pushing codemie_session_start past both limits. 153 CodeMie-owned Claude sessions were misreported as native-external for exactly this reason. Scan is now 256KB, bounded by bytes read rather than line count, since one large early line can consume most of a small budget by itself. Generated with AI Co-Authored-By: codemie-ai <codemie.ai@gmail.com> Claude-Session: https://claude.ai/code/session_0146TJLVz6A1KcRoGRMn3MJx
Two related gaps in the plain console command (not --report): - Session duration only showed the wall-clock span (endTime - startTime), which counts idle/overnight gaps and can overstate active work by an order of magnitude for a resumed session. Now also surfaces CodeMie's own tracked activeDurationMs when the session JSON carries it, labeled separately from the wall-clock span. - Cost enrichment previously only ran when --report was passed, so the plain console command — the one most invocations use — never computed or displayed cost at all. It now always enriches from correlated logs when no source (OTEL) cost is present, and both `Est. Cost` and each session's `Cost:` line are populated on every path; --report reuses the same result rather than re-enriching. Also relabels the HTML report's "Files touched" stat to "File operations" to distinguish it from the report's separate unique-files-changed columns. Generated with AI Co-Authored-By: codemie-ai <codemie.ai@gmail.com> Claude-Session: https://claude.ai/code/session_0146TJLVz6A1KcRoGRMn3MJx
…tion test
The "setup skills" wizard's Global scope installs into the real
~/.claude/skills/ (claude-skill-generator.ts hardcodes os.homedir() there,
since .claude/ is Claude Code's own directory, not something CODEMIE_HOME
redirection reaches). That install escapes the temp CODEMIE_HOME used
elsewhere in this suite and was never cleaned up, leaking a skill dir into
the real user environment on every run. Removes it in afterAll by matching
the on-disk slug prefix (`${skillName}-${project}-global`) rather than
duplicating the exact slug algorithm.
Generated with AI
Co-Authored-By: codemie-ai <codemie.ai@gmail.com>
Claude-Session: https://claude.ai/code/session_0146TJLVz6A1KcRoGRMn3MJx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes three accuracy gaps in
codemie analyticsaround which sessions count as having a usable native log, session duration, and cost visibility — plus a leaked test artifact cleanup.Changes
loadAgentSessionFilenow verifies the correlated/native transcript path still exists on disk before treating a session ashadLog=true. A coding agent's own retention (e.g. Claude Code's default 30-day cleanup) can rotate a transcript away after the session ran while the correlation record still names it — 122 of 291 real Claude sessions in one report were misreported this way.hasOwnershipMarker's legacy-session scan widened from a 4KB/10-line budget to a 256KB byte-bounded scan. A SessionStart hook can inject a largehookAdditionalContextblob (CLAUDE.md, memory files, ...) as an early transcript line — observed up to ~100KB — pushing the ownership marker past the old limits and misclassifying 153 CodeMie-owned Claude sessions as native-external.activeDurationMsalongside the existing wall-clockduration(which counts idle/overnight gaps). Cost enrichment now always runs for the plain console command, not just--report, soEst. Costand per-sessionCost:lines are populated on every invocation. Also relabels the HTML report's "Files touched" stat to "File operations" to distinguish it from the report's unique-files-changed columns.~/.claude/skills/(outside the tempCODEMIE_HOMEthe suite otherwise uses) and never cleaned that up — now removed inafterAll.Testing
npm run typecheck, lint, and secrets scan passed via pre-commit hooks on each commitChecklist
npm run ci)main