Skip to content

Add private local telemetry writer - #134

Merged
Doctacon merged 6 commits into
developfrom
work/local-telemetry-writer
Aug 19, 2026
Merged

Add private local telemetry writer#134
Doctacon merged 6 commits into
developfrom
work/local-telemetry-writer

Conversation

@Doctacon

@Doctacon Doctacon commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • move enabled retrieval telemetry persistence out of the short-lived retrieval process into a private, bounded filesystem inbox drained by one Buoy-owned local writer
  • independently revalidate canonical trace-envelope/v1 payloads before idempotent replay into the unchanged DuckDB v1 schema
  • add read-only buoy telemetry status and bounded buoy telemetry flush
  • preserve default-off behavior, retrieval results and errors, stdout/stderr, the local privacy contract, and the existing ~/.buoy/telemetry/telemetry.duckdb analytical surface
  • add no Collector, OTLP endpoint, listener, socket, remote backend, provider access, or cloud destination

Why

The first local telemetry slice appended to DuckDB inside each retrieval process. That made fresh-process DuckDB startup latency part of retrieval completion and could drop observations when multiple processes contended for the single-writer database. The private writer keeps the same local ownership and content-free schema while making producer handoff durable, bounded, eventual, and observable.

User impact

With telemetry disabled, behavior remains a zero-side-effect no-op. With BUOY_TELEMETRY=local, retrieval now returns after a private atomic envelope publication instead of waiting for DuckDB persistence. Users can inspect backlog and health with buoy telemetry status or request a bounded drain with buoy telemetry flush.

Packaging stop and repair

The first PR head correctly built buoy = buoy_search.entrypoint:main, but the repository's distribution validator still required the former buoy_search.cli:main target. The first exact-head Python jobs passed, the distribution job failed on that mismatch, and the PR returned to draft before merge. The repaired validator now uses one exact source/wheel contract, requires the lightweight entry point plus all six telemetry modules, and rejects old, missing, additional, malformed, or non-UTF-8 entry-point metadata. The original packaging PASS is visibly superseded in the records rather than rewritten.

Validation

  • exact runtime implementation: 55f41fd9e2f98fed83fd0619c29a9a5549ae4052; packaging repair: 33c6124180120bb6711e3463556a672789bc134c; repaired closure head: fe40aadf88e6fbe8ad702225a111d2f787291689
  • 100/100 synchronized publications, 100 complete trace graphs, one elected writer, and no trace loss
  • 100/100 paired enabled/disabled retrieval-equivalence checks; producer p99 overhead 3.147 ms and warm visibility p99 90.827 ms
  • 38/38 adversarial crash, replay, privacy, filesystem, lifecycle, status/flush, and zero-network scenarios
  • 166/166 focused telemetry/release tests and 1,030/1,030 full tests on Python 3.11 and Python 3.13
  • repaired wheel/sdist validation, exact installed metadata, negative entry-point fixtures, and clean-wheel telemetry lifecycle
  • source, lock, ranking, C6, compilation, receipt, diff, package-content, and isolated-home checks
  • independently reproduced exact topology, scope, source receipts, acceptance artifacts, and distribution hashes

Records

The completed ticket, diagnostic findings, corrected evidence, specifications, decision, superseded original review, and independent packaging-repair PASS review are included under .10x/. The repaired closure commit changes only evidence, reviews, and the ticket location.

@Doctacon
Doctacon marked this pull request as draft August 19, 2026 21:16
@Doctacon
Doctacon marked this pull request as ready for review August 19, 2026 21:42
@Doctacon
Doctacon merged commit e9c906c into develop Aug 19, 2026
3 checks passed
Doctacon added a commit that referenced this pull request Aug 20, 2026
* develop:
  Record integrated telemetry installation (#136)
  Authorize one-time integrated telemetry installation (#135)
  Add private local telemetry writer (#134)
  Add opt-in local retrieval telemetry (#133)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant