Skip to content

feat: wave 2 — observability (telemetry, SSE logs, /metrics) - #20

Closed
wra-sol wants to merge 1 commit into
feat/wave-1-attach-and-security-baselinefrom
feat/wave-2-observability
Closed

feat: wave 2 — observability (telemetry, SSE logs, /metrics)#20
wra-sol wants to merge 1 commit into
feat/wave-1-attach-and-security-baselinefrom
feat/wave-2-observability

Conversation

@wra-sol

@wra-sol wra-sol commented Jun 29, 2026

Copy link
Copy Markdown
Owner

Summary

Wave 2 of the opencode manager improvement plan. Three slices, one PR. Based on Wave 1 (PR #19).

2.1 — Expose crash count, uptime, last-exit rc, restart history (closes #6)

ChildProcess now tracks started_at, last_exit_rc, last_exit_at, crash_count, and a restart_history deque (last 10 entries with timestamp, reason, and rc). These are surfaced in:

  • /manage/status JSON (new fields: pid, started_at, uptime_s, last_exit_rc, last_exit_at, crash_count, restarts)
  • Dashboard HTML (new rows: Uptime, Crashes, Last exit)

Restart reasons tracked: manual, crash, reconfigure, stopped.

2.2 — SSE log tail + manager events in ring + filter/search (closes #7)

  • Log ring broadened from 400 to 1000 entries, now captures both child stdout (plain strings) and manager events (dicts with ts, source, level, line).
  • Manager events logged: login success/failure (with IP), restart, key rotate, reconfigure, proxy errors, child crash/respawn/give-up.
  • New endpoint GET /manage/logs/stream — SSE stream that tails the log ring live. Sends the current filtered ring as an initial burst, then appends new entries as they arrive (polls at 0.5s).
  • Filter params on both /manage/logs and /manage/logs/stream: ?level=error, ?source=child or ?source=manager, ?q=<substring>.
  • /manage/logs returns structured entries (child stdout normalized to dicts for the API).
  • Dashboard renders both string and dict log entries.

2.3 — /metrics Prometheus endpoint (closes #8)

New endpoint GET /manage/metrics (session-auth required) emitting Prometheus text format:

Counters (monotonic, process-lifetime):

  • opencode_railway_requests_total{path}
  • opencode_railway_login_attempts_total
  • opencode_railway_login_failures_total
  • opencode_railway_child_crashes_total
  • opencode_railway_restarts_total{reason}
  • opencode_railway_proxy_errors_total

Gauges:

  • opencode_railway_child_up (1|0)
  • opencode_railway_log_ring_size
  • opencode_railway_uptime_seconds

Metrics class with thread-safe increments. Request counting in do_GET and do_POST.

Tests

11 new test cases:

  • Status telemetry fields, restart history recording, dashboard telemetry display
  • Log filtering by source, search, manager events in ring, structured log entries
  • /metrics requires auth, returns Prometheus text, counts requests, counts login failures

119 total tests pass. ruff check + ruff format --check clean.

Wave 2.1 — Expose crash count, uptime, last-exit rc, restart history (closes #6)
- ChildProcess tracks started_at, last_exit_rc, last_exit_at, restart_history
- /manage/status returns pid, started_at, uptime_s, last_exit_rc, crash_count, restarts
- Dashboard shows uptime, crash count, last exit in the status card
- Restart history records reason (manual, crash, reconfigure, key-rotate) and rc

Wave 2.2 — SSE log tail + manager events + filter/search (closes #7)
- Log ring broadened to 1000 entries, now captures both child stdout (strings)
  and manager events (dicts with ts, source, level, line)
- Manager events logged: login success/failure, restart, key rotate, reconfigure,
  proxy errors, child crash/respawn
- New endpoint GET /manage/logs/stream — SSE tail with live updates
- Filter params on /manage/logs and /manage/logs/stream: ?level=, ?source=, ?q=
- /manage/logs returns structured entries (normalized to dicts for the API)
- Dashboard renders both string and dict log entries

Wave 2.3 — /metrics Prometheus endpoint (closes #8)
- New endpoint GET /manage/metrics (session-auth required)
- Counters: requests_total{path}, login_attempts, login_failures,
  child_crashes, restarts_total{reason}, proxy_errors
- Gauges: child_up, log_ring_size, uptime_seconds
- Metrics class with thread-safe increments
- Request counting in do_GET and do_POST

Tests: 11 new test cases. 119 total, all passing. Lint + format clean.
@wra-sol wra-sol added enhancement New feature or request wave-2 Wave 2: observability labels Jun 29, 2026
@wra-sol
wra-sol deleted the branch feat/wave-1-attach-and-security-baseline June 29, 2026 02:54
@wra-sol wra-sol closed this Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request wave-2 Wave 2: observability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant