Repository navigation
Conversation
Measure real usage and storage HTTP routes, independent status and event probes, phase timings, memory, and result consistency across source checkouts. Keep boundary failures visible and provide deterministic synthetic ledgers plus read-only private snapshots.
|
Thanks @jcs130 for the benchmark. Its main result is useful to us: on the 334 MB synthetic ledger the responses of bd4e35e and v0.2.16 match byte for byte, which confirms that #415 did not change any usage figure. We are closing this PR without merging. For us a harness like this is a comparison we run by hand when the ledger code changes, and we keep those outside the repository. In the repository its 634 lines would have to follow every change to the It has already been used once: for #421 we ran it from your branch to measure before and after the fix in #424, and the numbers are in #421. On the cached count: the drift exists inside |
Large ledgers need a repeatable HTTP and event-loop regression workload. This adds a manual benchmark, deterministic fixture tests, and documentation; it changes no production implementation or dependencies.
Workload
pnpm bench:usagestarts isolated processes for selected source checkouts using the real WebApp routes, UsageLog, aggregateUsage and JSONL event store. It covers first/cached storage scans, all-time/30-day queries, repeated and concurrent queries, preloaded aggregation, and append during a query.Outputs include phase timings; independent HTTP status probes at 25 ms and 500 ms; external event pulses and queued-handler latency; timer gaps and GC; peak RSS and sampled heap; source/input hashes; full-response fingerprints; and correctness gates. No Persona, model or World starts. The runner can generate deterministic ledgers or take a read-only snapshot supplied locally by its operator. Raw data remains in the selected local output directory.
Synthetic verification
The generated workload has 334,000,640 bytes and 130,469 deterministic records, with three repetitions per version/case. Official pre-fix bd4e35e and v0.2.16 9569f55 produce matching full-response fingerprints and independent call/token totals. Source data hashes remain unchanged. On this generated workload, v0.2.16's preloaded aggregation median is 246.4 ms and concurrent-query status latency reaches 420.8 ms. These are environment-specific observations, not universal thresholds;
--max-status-msenables an explicit host-specific gate.The disposable boundary fixture also exposes duplicate-ID count drift: after warming count(), appending an existing recordId increments the cached count even though readAll() deduplicates it. The manual benchmark preserves this as a failed invariant rather than accepting it as expected behavior. This PR contains no production fix for that issue.
Validation
Documentation and commands are in
docs/usage-benchmark.md. No private-ledger measurement results, raw records, deployment configuration, or live identities are included in this PR description.