Skip to content

Fix three ingest defects, add CI and linting, and document the architecture - #1

Closed
leggetter wants to merge 9 commits into
hookdeck:mainfrom
leggetter:chore/docs-defects-and-ci
Closed

Fix three ingest defects, add CI and linting, and document the architecture#1
leggetter wants to merge 9 commits into
hookdeck:mainfrom
leggetter:chore/docs-defects-and-ci

Conversation

@leggetter

Copy link
Copy Markdown

Came out of a review of the repo. Nine commits, each self-contained and readable on its own — three defect fixes, tooling, and docs. Nothing here changes the design; it fixes things that were wrong, and writes down things that were true but unstated.

Defects

Each fix has a test that fails without it.

[d4ce91e] A malformed signature answered 500 instead of 401. hmac.compare_digest refuses two str arguments unless both are pure ASCII, and the signature header is a value an unauthenticated sender controls outright. One non-ASCII byte raised TypeError out of verify_signature, and aiohttp turned that into a 500.

Wrong twice over. It bypassed assert_declared_status — so the one status the adapter emitted without deciding its retryability was the one an attacker could trigger. And 500 is inside the provisioned retry rule's 500-599 range, so Hookdeck retried a forged request roughly ten times with backoff instead of dropping it on the first refusal. Reproduced over a raw socket before fixing.

[08bc29f] X-Request-ID was standing in for a Hookdeck event id. That header is not a Hookdeck identifier: anything in front of the gateway can set it, it is not subject to header_prefix, and it is not unique per delivery — one Hookdeck request fans out to one event per matching connection, so two routes sharing a source would collide and the second event would be dropped as a duplicate.

It also defeated the branch it was standing in for. _admit deliberately warns that a delivery with no event id is being processed without dedup or retry, and names header_prefix as the likely cause. A substitute id silenced that warning while delivering neither guarantee, since POST /events/{proxy-id}/retry 404s. REQUEST_ID keeps its place in constants.py with the fan-out reason written down.

[4ed091c] The sync-ack marker only cleared on success. _acked_before_completion survived an exhausted retry budget and an abandoned hand-back. The leak is small; the behaviour is the point — a later sync run of the same event id would find its own id already in the set and be treated as having been acked early when it had not, asking for a redelivery the 5xx response was already asking for.

Tooling

[11201f1] Ruff. Lint only — ruff format would rewrite nineteen files and bury the history for no correctness gain, so line-length is set for anyone who chooses to run it and E501 stays off. Most of the diff is --fix: Optional[X]X | None, Dictdict, import ordering.

The one judgement call is selecting BLE001 rather than ignoring it. This adapter degrades instead of aborting in seven places, and two already carried a noqa: BLE001 saying so. Selecting the rule makes those two meaningful, gives the other five the same one-line justification, and means the next blind except has to be argued for where it is written.

[5d8bcb2] CI, releases, and a guard against upstream drift.

  • CI: ruff, pytest across 3.10–3.13, and a packaging job. The third is the one worth arguing for — a wheel missing plugin.yaml, the dashboard bundle or the bundled skill installs perfectly and then registers nothing, and the symptom is a platform that is simply absent. The job builds the artifact, looks inside it for each file, and installs the wheel to confirm the hermes_agent.plugins entry point is really declared.

  • Releases: tag-driven, publishing with PyPI Trusted Publishing so there is no long-lived token in the repo. The version now lives only in hookdeck.__version__ and pyproject reads it from there; the build refuses a tag that disagrees. Needs one-time setup on PyPI — see the checklist below.

  • scripts/check_upstream_contract.py: the suite runs entirely against tests/hermes_stub.py, which is the only way to exercise ingest without a Hermes checkout but cannot notice the real WebhookAdapter renaming something underneath it. This parses upstream and asserts every borrowed name is still there — imports, inherited methods, and the self._… attributes the base classes own. Weekly, not per-PR. Verified green against current hermes-agent and red against a copy with two names deliberately moved.

    It is a smoke alarm, not a type check: a signature that changes while the name stays put still gets through.

Docs

[1fe21f2] What the two halves are, and which Hookdeck this is. The README assumed you knew both products, and assumed "Hookdeck" was unambiguous. It is not — the platform includes Outpost, which points the other way, and a reader arriving from the Hookdeck docs had no way to tell which one this is. This plugin is inbound only and now says so.

"Platform" was the other collision: it is Hermes' word for a source of inbound work, which is why plugin.yaml declares kind: platform and why the opening line read "A Hookdeck platform plugin" — a sentence that looks like it is about the Hookdeck Platform and is not. plugin.yaml, __init__.py and pyproject.toml already said "event gateway"; they now capitalise it as the product name so every surface names the same thing.

[2d7d9df] An architecture section, with the CLI in it. The README went from a comparison table straight to install, so the shape of the thing was only ever assembled from the quickstarts. Two diagrams now carry it: a topology flowchart, and a sequence diagram for the ack and hand-back — the subtlest part of the design and the one prose was working hardest to explain. The attempt-counter rule that lets dedup and retry coexist is visible in it rather than asserted.

This is also where the CLI is explained, which answers a question the docs left open: three different things here are called CLI. The Hookdeck binary the adapter spawns and supervises, the hermes hookdeck operator commands that talk to the REST API instead, and Hermes' own CLI hosting both. The quickstarts keep the operational caveats, because those are things you do rather than things you need to understand the design.

Both blocks were rendered before committing.

[32cb145] What Hookdeck can do that this plugin does not ask it to. The Limitations list read as the boundary of Hookdeck when it is really the boundary of hookdeck/api.py. A new section lists what is unexposed rather than impossible: the Publish API, bulk operation plans and cancellation, request replay, ignored-event retry, issue triggers, transformations, destination auth, the wider metrics.

The durability limitation was also imprecise. It now says push-only in both directions and spells out the consequence that was implicit: an event is recoverable because a delivered-but-failed event stays retryable, not because anything holds a lease on it — which is exactly why boot recovery has to reconcile running ledger rows itself.

Two things that need a decision

1. Is CLI_DISCONNECTED a retryable ignored-event cause? POST /bulk/ignored-events/retry retries ignored events matching a cause. If CLI_DISCONNECTED is accepted, then the sharpest edge in CLI mode — events discarded with no listener attached, which the README currently describes as unrecoverable — has a recovery path, and hermes hookdeck doctor should be pointing at it. The published docs only evidence FILTERED and TRANSFORMATION_FAILED, so it is written as needing confirmation rather than asserted. Someone with a live project can settle this quickly.

2. Trusted Publishing needs a one-time setup before the first tag. On PyPI: Project → Publishing → add a GitHub publisher for hookdeck/hermes-hookdeck, workflow release.yml, environment pypi. Until that exists the publish job will fail; nothing else in CI depends on it.

Not in scope, and still the biggest gap

tools.py sits at 29% coverage with no test file at all — seven agent-invocable tools, including hookdeck_pause_connection and hookdeck_bulk_retry, which act on a live Hookdeck project unsupervised. The pause/auto-resume mechanism spans tools.py → ledger → adapter._resume_due_connections with only the adapter half covered. cli.py (34%) and tunnel.py (41%) are the next two. Worth its own PR.

Checks

  • ruff check . clean
  • 183 tests pass (177 before; 6 added)
  • python -m build + twine check pass, and the wheel carries plugin.yaml, the dashboard bundle and the skill
  • scripts/check_upstream_contract.py green against current hermes-agent

Opened from a fork — I do not have push access to this repo.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo

leggetter and others added 9 commits August 11, 2026 11:52
`hmac.compare_digest` rejects two `str` arguments unless both are pure
ASCII, and the signature header is a value an unauthenticated sender
controls outright. One non-ASCII byte raised `TypeError` out of
`verify_signature`, and aiohttp turned that into a 500.

Wrong twice over. It bypassed `assert_declared_status`, which every other
response goes through precisely so no status escapes without its
retryability being declared — so the one status the adapter emitted
without deciding anything was the one an attacker could trigger. And 500
is inside the provisioned retry rule's `500-599` range, so Hookdeck
retried a forged request roughly ten times with backoff instead of
dropping it on the first refusal.

Compare as bytes, which has no ASCII restriction. Neither encode can
fail: the expected value is base64, and `replace` turns an undecodable
candidate into bytes that simply do not match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
The event id fell back to a bare `X-Request-ID` header, which is not a
Hookdeck identifier at all: anything in front of the gateway can set it,
it is not subject to the configurable `header_prefix`, and it is not
unique per delivery — one Hookdeck request fans out to one event per
matching connection, so two routes sharing a source produce two events
carrying the same request id, and the second would be dropped as a
duplicate.

It also defeated the branch it was standing in for. `_admit` handles a
delivery with no event id deliberately: it warns that the delivery is
being processed without deduplication or retry and names `header_prefix`
as the likely cause. A substitute id silences that warning while
delivering neither guarantee — `POST /events/{id}/retry` with a proxy's
id 404s, so the failed run is never handed back.

Hookdeck's own `x-hookdeck-requestid` is not a substitute either, for the
fan-out reason above; `REQUEST_ID` keeps its place in constants with that
written down, so the next person does not reach for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
`_acked_before_completion` records that a sync run outlasted its timeout
and was acked 202, so its eventual failure gets the explicit hand-back an
async run would get. It was only discarded on success and on a hand-back
that worked — so an event that exhausted its retry budget, or whose
hand-back was abandoned after three failed API calls, left its id in the
set for the life of the process.

Small as a leak. The behavioural half matters more: a later sync-mode run
of the same event id would find its own id already in the set and be
treated as having been acked early when it had not, asking Hookdeck to
redeliver an event the 5xx response was already asking it to redeliver.

Discard once the hand-back decision has been made, which is the point the
marker has done its job, and on the exhausted path that returns before
reaching it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Lint only, not formatting. `ruff format` would rewrite nineteen files at
once and bury the history for no correctness gain, so `line-length` is
set for anyone who chooses to run it and E501 stays off — the long lines
here are argparse calls, log statements and comment prose that read worse
wrapped.

Most of the diff is `ruff --fix`: `Optional[X]` to `X | None`, `Dict` to
`dict`, import ordering, `__all__` sorted. All of it is mechanical, and
`from __future__ import annotations` is already everywhere, so nothing
changes at runtime.

The one judgement call is selecting BLE001 rather than ignoring it. This
adapter degrades instead of aborting in seven places — a body it cannot
read, a skill it cannot load, a CLI whose version it cannot determine, a
supervisor that must restart rather than die — and every one is a
decision the author made on purpose. Two already carried a `noqa: BLE001`
saying so. Selecting the rule makes those two meaningful, gives the other
five the same one-line justification, and means the next blind `except`
has to be argued for at the point it is written.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Three jobs on every push and PR: ruff, pytest across 3.10 to 3.13, and a
packaging job. The third is the one worth arguing for. This plugin is not
just its modules — a wheel missing plugin.yaml, the dashboard bundle or
the bundled skill installs perfectly and then registers nothing, and the
symptom is a platform that is simply absent with no error to debug. So
the job builds the artifact, looks inside it for each of those files, and
installs the wheel to confirm the `hermes_agent.plugins` entry point
Hermes discovers the plugin by is really declared.

Releases are tag-driven and publish to PyPI with Trusted Publishing, so
there is no long-lived API token in the repository. The build job refuses
to proceed if the tag and `hookdeck.__version__` disagree, and the
version now lives only in the module — `pyproject` reads it from there,
because two copies drift and the stale one is always the one nobody
looks at.

Separately, and on a schedule rather than per-PR: the test suite runs
entirely against `tests/hermes_stub.py`, which is the only way to
exercise ingest without a Hermes checkout but cannot notice the real
`WebhookAdapter` renaming something underneath it. The new script parses
upstream and asserts every borrowed name is still there — imports,
inherited methods, and the `self._…` attributes the base classes own. It
is a smoke alarm, not a type check: a signature that changes while the
name stays put still gets through, which is why the README points at a
real end-to-end run as the thing that proves integration. Verified
against the current hermes-agent, and against a copy with two names
deliberately moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
The README assumed you already knew both products. Two paragraphs now say
what Hermes Agent is, what Hookdeck Event Gateway is, and what connecting
them gets you, before the comparison table starts arguing about ingest.

It also assumed "Hookdeck" was unambiguous, and it is not. Hookdeck's
platform includes Outpost, which points the other way — self-hosted
infrastructure for sending your own webhooks to your users — and a reader
arriving from the Hookdeck docs has no way to tell which one this is.
This plugin is inbound only, and now says so.

"Platform" was the other collision. It is Hermes' word for a source of
inbound work, which is why `plugin.yaml` declares `kind: platform` and
why the opening line read "A Hookdeck platform plugin" — a sentence that
looks like it is about the Hookdeck Platform and is not.

`plugin.yaml`, `__init__.py` and `pyproject.toml` already said "event
gateway"; they now capitalise it as the product name, so every surface
that names it names the same thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
The README went from a comparison table straight to install, so the
shape of the thing was only ever assembled from the quickstarts. Two
diagrams now carry it: a topology flowchart for where each piece runs and
what crosses the network boundary, and a sequence diagram for the ack and
hand-back, which is the subtlest part of the design and the one prose was
working hardest to explain.

The sequence diagram earns its place. "Ack 202 before the run finishes,
then ask for redelivery if it fails" is three sentences of prose and one
picture, and the attempt-counter rule that lets dedup and retry coexist
is visible in it rather than asserted.

This is also where the CLI is explained, which answers a question the
docs left open: three different things here are called CLI. The Hookdeck
binary the adapter spawns and supervises, the `hermes hookdeck` operator
commands that talk to the REST API instead, and Hermes' own CLI hosting
both. The quickstarts keep the operational caveats — pausing before a
planned shutdown, the version floor — because those are things you do,
not things you need to know to understand the design.

Both blocks were rendered before committing; the topology one is laid out
so the flow reads left to right, which needed the retry edge lifted out
into the sequence diagram rather than closing a cycle in the flowchart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Two related gaps in the docs. The Limitations list read as the boundary
of Hookdeck when it is really the boundary of `hookdeck/api.py`, and one
of its entries was imprecise about why the durability claim holds.

The durability entry now says push-only in both directions — Hookdeck
pushes to the adapter, the adapter pushes retry requests back — and spells
out the consequence that was implicit: an event is safe because a
delivered-but-failed event stays retryable, not because anything holds a
lock on it. That is exactly why boot recovery has to reconcile `running`
ledger rows itself rather than letting a lease expire.

The new section lists what is unexposed rather than impossible. The
Publish API is the one worth reading first, because it answers "can I
push events in?" with yes — `hermes hookdeck test <route>` would put a
real event through the real connection, and Hermes could enqueue durable
work for itself. Bulk operations have an estimate step and a cancel that
`hookdeck_bulk_retry` skips, which matters more than usual for a tool an
agent can call unsupervised. Requests can be replayed where the plugin
only knows about events. Issue triggers, transformations, destination
auth and the rest of the metrics are all unprovisioned.

One entry is deliberately a question. `POST /bulk/ignored-events/retry`
retries ignored events by `cause`, and if `CLI_DISCONNECTED` is an
accepted cause then the sharpest edge in CLI mode has a recovery path the
README currently describes as absent. The published docs only evidence
`FILTERED` and `TRANSFORMATION_FAILED`, so it is written as needing
confirmation against a live project rather than asserted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
@leggetter

Copy link
Copy Markdown
Author

Superseded by #3, which is the same work on a branch in this repo rather than a fork — I did have write access after all, and my earlier permission check was wrong. #3 also carries three commits added since: the HOOKDECK_EG_* namespacing with project pinning, the answered CLI_DISCONNECTED question, and the removal of the pre-release compatibility shims.

@leggetter leggetter closed this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant