Skip to content

Increase test coverage, starting with the agent-callable tools #2

Description

@leggetter

Coverage is 67% overall, but the distribution is the problem rather than the number. The modules with no Hermes or network dependency are near-complete; the ones that reach the outside world, and the ones an LLM can invoke unsupervised, are the least tested.

hookdeck/state.py            100%
hookdeck/routing.py          100%
hookdeck/verify.py           100%
hookdeck/constants.py        100%
hookdeck/payload.py           98%
hookdeck/provision.py         97%
hookdeck/settings.py          89%
hookdeck/api.py               76%
hookdeck/adapter.py           73%
hookdeck/dashboard/…          64%
hookdeck/tunnel.py            41%
hookdeck/cli.py               34%
hookdeck/tools.py             29%   ← no test file at all

1. tools.py — the priority

Seven tools registered into the hookdeck toolset, with no test file in the repo. These are the ones the agent calls, unsupervised, against a live Hookdeck project:

hookdeck_queue_status reads
hookdeck_list_failed_events reads
hookdeck_get_event_body reads
hookdeck_retry_event writes — redelivers one event
hookdeck_bulk_retry writes — redelivers every failed event matching a query
hookdeck_pause_connection writes — stops delivery
hookdeck_resume_connection writes — resumes it

Worth covering, roughly in this order:

  • _pause_minutes clamping. MAX_PAUSE_MINUTES is the only thing between a confused agent and a connection paused indefinitely. Non-integer, negative, absent, and over-cap inputs should all land inside the bounds.
  • The pause → auto-resume contract, end to end. _schedule_resume writes a deadline to the ledger and adapter._resume_due_connections honours it. Only the adapter half is covered today, so nothing proves the two agree on the path or the row shape — and _with_ledger resolving a different database than the adapter reads is exactly the failure that looks like the feature silently not working.
  • _cancel_scheduled_resume on manual resume. Without it a deadline from an earlier short pause can unpause a later, longer one early.
  • _guard error shaping. A HookdeckAPIError should reach the model as a sentence it can act on, never a traceback.
  • Connection resolution by name vs. id. The web_/con_ prefix check decides whether a lookup happens at all; a name that matches nothing should say so rather than acting on something.
  • _run when a loop is already running. The worker-thread path is what stops the tool exploding when dispatched from async context.

2. cli.py (34%)

Argument parsing and setup --dry-run are covered. Not covered: status, pause, resume, replay, and most of doctor — which is the command an operator reaches for precisely when something is already wrong. The API calls are easy to stub; HookdeckAPI takes an injected client, as tests/test_api.py already does with httpx.MockTransport.

3. tunnel.py (41%)

listen_args and the missing-binary path are covered. The supervisor is not: restart with capped backoff, the _HEALTHY_RUN_SECONDS distinction between a failed start and a healthy session that ended, and stop() terminating then killing. These are testable with a fake asyncio.create_subprocess_exec, and the backoff reset rule in particular is the kind of thing that is easy to get backwards and never notice.

4. dashboard/plugin_api.py (64%)

_require_own_connection is the interesting gap — it is the check that stops this tab pausing a connection belonging to somebody else's production traffic. It deserves a test for the refusal path, not just the allow path. This gets more load-bearing with organisation-level API keys, since it authorises by connection name.

5. The dashboard bundle

dashboard/dist/index.js has no tests of any kind. Lower priority — it is 194 lines of hand-written IIFE, and its failure mode is visible rather than silent — but worth noting it is entirely unverified.


Not a request to chase a coverage number. The specific worry is that the surface an LLM can drive without a human in the loop is the least verified part of the codebase, and it is the part that mutates a live project.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions