Fix three ingest defects, add CI and linting, and document the architecture - #1
Closed
leggetter wants to merge 9 commits into
Closed
Fix three ingest defects, add CI and linting, and document the architecture#1leggetter wants to merge 9 commits into
leggetter wants to merge 9 commits into
Conversation
`hmac.compare_digest` rejects two `str` arguments unless both are pure ASCII, and the signature header is a value an unauthenticated sender controls outright. One non-ASCII byte raised `TypeError` out of `verify_signature`, and aiohttp turned that into a 500. Wrong twice over. It bypassed `assert_declared_status`, which every other response goes through precisely so no status escapes without its retryability being declared — so the one status the adapter emitted without deciding anything was the one an attacker could trigger. And 500 is inside the provisioned retry rule's `500-599` range, so Hookdeck retried a forged request roughly ten times with backoff instead of dropping it on the first refusal. Compare as bytes, which has no ASCII restriction. Neither encode can fail: the expected value is base64, and `replace` turns an undecodable candidate into bytes that simply do not match. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
The event id fell back to a bare `X-Request-ID` header, which is not a
Hookdeck identifier at all: anything in front of the gateway can set it,
it is not subject to the configurable `header_prefix`, and it is not
unique per delivery — one Hookdeck request fans out to one event per
matching connection, so two routes sharing a source produce two events
carrying the same request id, and the second would be dropped as a
duplicate.
It also defeated the branch it was standing in for. `_admit` handles a
delivery with no event id deliberately: it warns that the delivery is
being processed without deduplication or retry and names `header_prefix`
as the likely cause. A substitute id silences that warning while
delivering neither guarantee — `POST /events/{id}/retry` with a proxy's
id 404s, so the failed run is never handed back.
Hookdeck's own `x-hookdeck-requestid` is not a substitute either, for the
fan-out reason above; `REQUEST_ID` keeps its place in constants with that
written down, so the next person does not reach for it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
`_acked_before_completion` records that a sync run outlasted its timeout and was acked 202, so its eventual failure gets the explicit hand-back an async run would get. It was only discarded on success and on a hand-back that worked — so an event that exhausted its retry budget, or whose hand-back was abandoned after three failed API calls, left its id in the set for the life of the process. Small as a leak. The behavioural half matters more: a later sync-mode run of the same event id would find its own id already in the set and be treated as having been acked early when it had not, asking Hookdeck to redeliver an event the 5xx response was already asking it to redeliver. Discard once the hand-back decision has been made, which is the point the marker has done its job, and on the exhausted path that returns before reaching it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Lint only, not formatting. `ruff format` would rewrite nineteen files at once and bury the history for no correctness gain, so `line-length` is set for anyone who chooses to run it and E501 stays off — the long lines here are argparse calls, log statements and comment prose that read worse wrapped. Most of the diff is `ruff --fix`: `Optional[X]` to `X | None`, `Dict` to `dict`, import ordering, `__all__` sorted. All of it is mechanical, and `from __future__ import annotations` is already everywhere, so nothing changes at runtime. The one judgement call is selecting BLE001 rather than ignoring it. This adapter degrades instead of aborting in seven places — a body it cannot read, a skill it cannot load, a CLI whose version it cannot determine, a supervisor that must restart rather than die — and every one is a decision the author made on purpose. Two already carried a `noqa: BLE001` saying so. Selecting the rule makes those two meaningful, gives the other five the same one-line justification, and means the next blind `except` has to be argued for at the point it is written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Three jobs on every push and PR: ruff, pytest across 3.10 to 3.13, and a packaging job. The third is the one worth arguing for. This plugin is not just its modules — a wheel missing plugin.yaml, the dashboard bundle or the bundled skill installs perfectly and then registers nothing, and the symptom is a platform that is simply absent with no error to debug. So the job builds the artifact, looks inside it for each of those files, and installs the wheel to confirm the `hermes_agent.plugins` entry point Hermes discovers the plugin by is really declared. Releases are tag-driven and publish to PyPI with Trusted Publishing, so there is no long-lived API token in the repository. The build job refuses to proceed if the tag and `hookdeck.__version__` disagree, and the version now lives only in the module — `pyproject` reads it from there, because two copies drift and the stale one is always the one nobody looks at. Separately, and on a schedule rather than per-PR: the test suite runs entirely against `tests/hermes_stub.py`, which is the only way to exercise ingest without a Hermes checkout but cannot notice the real `WebhookAdapter` renaming something underneath it. The new script parses upstream and asserts every borrowed name is still there — imports, inherited methods, and the `self._…` attributes the base classes own. It is a smoke alarm, not a type check: a signature that changes while the name stays put still gets through, which is why the README points at a real end-to-end run as the thing that proves integration. Verified against the current hermes-agent, and against a copy with two names deliberately moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
The README assumed you already knew both products. Two paragraphs now say what Hermes Agent is, what Hookdeck Event Gateway is, and what connecting them gets you, before the comparison table starts arguing about ingest. It also assumed "Hookdeck" was unambiguous, and it is not. Hookdeck's platform includes Outpost, which points the other way — self-hosted infrastructure for sending your own webhooks to your users — and a reader arriving from the Hookdeck docs has no way to tell which one this is. This plugin is inbound only, and now says so. "Platform" was the other collision. It is Hermes' word for a source of inbound work, which is why `plugin.yaml` declares `kind: platform` and why the opening line read "A Hookdeck platform plugin" — a sentence that looks like it is about the Hookdeck Platform and is not. `plugin.yaml`, `__init__.py` and `pyproject.toml` already said "event gateway"; they now capitalise it as the product name, so every surface that names it names the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
The README went from a comparison table straight to install, so the shape of the thing was only ever assembled from the quickstarts. Two diagrams now carry it: a topology flowchart for where each piece runs and what crosses the network boundary, and a sequence diagram for the ack and hand-back, which is the subtlest part of the design and the one prose was working hardest to explain. The sequence diagram earns its place. "Ack 202 before the run finishes, then ask for redelivery if it fails" is three sentences of prose and one picture, and the attempt-counter rule that lets dedup and retry coexist is visible in it rather than asserted. This is also where the CLI is explained, which answers a question the docs left open: three different things here are called CLI. The Hookdeck binary the adapter spawns and supervises, the `hermes hookdeck` operator commands that talk to the REST API instead, and Hermes' own CLI hosting both. The quickstarts keep the operational caveats — pausing before a planned shutdown, the version floor — because those are things you do, not things you need to know to understand the design. Both blocks were rendered before committing; the topology one is laid out so the flow reads left to right, which needed the retry edge lifted out into the sequence diagram rather than closing a cycle in the flowchart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Two related gaps in the docs. The Limitations list read as the boundary of Hookdeck when it is really the boundary of `hookdeck/api.py`, and one of its entries was imprecise about why the durability claim holds. The durability entry now says push-only in both directions — Hookdeck pushes to the adapter, the adapter pushes retry requests back — and spells out the consequence that was implicit: an event is safe because a delivered-but-failed event stays retryable, not because anything holds a lock on it. That is exactly why boot recovery has to reconcile `running` ledger rows itself rather than letting a lease expire. The new section lists what is unexposed rather than impossible. The Publish API is the one worth reading first, because it answers "can I push events in?" with yes — `hermes hookdeck test <route>` would put a real event through the real connection, and Hermes could enqueue durable work for itself. Bulk operations have an estimate step and a cancel that `hookdeck_bulk_retry` skips, which matters more than usual for a tool an agent can call unsupervised. Requests can be replayed where the plugin only knows about events. Issue triggers, transformations, destination auth and the rest of the metrics are all unprovisioned. One entry is deliberately a question. `POST /bulk/ignored-events/retry` retries ignored events by `cause`, and if `CLI_DISCONNECTED` is an accepted cause then the sharpest edge in CLI mode has a recovery path the README currently describes as absent. The published docs only evidence `FILTERED` and `TRANSFORMATION_FAILED`, so it is written as needing confirmation against a live project rather than asserted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Author
|
Superseded by #3, which is the same work on a branch in this repo rather than a fork — I did have write access after all, and my earlier permission check was wrong. #3 also carries three commits added since: the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Came out of a review of the repo. Nine commits, each self-contained and readable on its own — three defect fixes, tooling, and docs. Nothing here changes the design; it fixes things that were wrong, and writes down things that were true but unstated.
Defects
Each fix has a test that fails without it.
[
d4ce91e] A malformed signature answered 500 instead of 401.hmac.compare_digestrefuses twostrarguments unless both are pure ASCII, and the signature header is a value an unauthenticated sender controls outright. One non-ASCII byte raisedTypeErrorout ofverify_signature, and aiohttp turned that into a 500.Wrong twice over. It bypassed
assert_declared_status— so the one status the adapter emitted without deciding its retryability was the one an attacker could trigger. And 500 is inside the provisioned retry rule's500-599range, so Hookdeck retried a forged request roughly ten times with backoff instead of dropping it on the first refusal. Reproduced over a raw socket before fixing.[
08bc29f]X-Request-IDwas standing in for a Hookdeck event id. That header is not a Hookdeck identifier: anything in front of the gateway can set it, it is not subject toheader_prefix, and it is not unique per delivery — one Hookdeck request fans out to one event per matching connection, so two routes sharing a source would collide and the second event would be dropped as a duplicate.It also defeated the branch it was standing in for.
_admitdeliberately warns that a delivery with no event id is being processed without dedup or retry, and namesheader_prefixas the likely cause. A substitute id silenced that warning while delivering neither guarantee, sincePOST /events/{proxy-id}/retry404s.REQUEST_IDkeeps its place inconstants.pywith the fan-out reason written down.[
4ed091c] The sync-ack marker only cleared on success._acked_before_completionsurvived an exhausted retry budget and an abandoned hand-back. The leak is small; the behaviour is the point — a latersyncrun of the same event id would find its own id already in the set and be treated as having been acked early when it had not, asking for a redelivery the 5xx response was already asking for.Tooling
[
11201f1] Ruff. Lint only —ruff formatwould rewrite nineteen files and bury the history for no correctness gain, soline-lengthis set for anyone who chooses to run it andE501stays off. Most of the diff is--fix:Optional[X]→X | None,Dict→dict, import ordering.The one judgement call is selecting
BLE001rather than ignoring it. This adapter degrades instead of aborting in seven places, and two already carried anoqa: BLE001saying so. Selecting the rule makes those two meaningful, gives the other five the same one-line justification, and means the next blindexcepthas to be argued for where it is written.[
5d8bcb2] CI, releases, and a guard against upstream drift.CI: ruff, pytest across 3.10–3.13, and a packaging job. The third is the one worth arguing for — a wheel missing
plugin.yaml, the dashboard bundle or the bundled skill installs perfectly and then registers nothing, and the symptom is a platform that is simply absent. The job builds the artifact, looks inside it for each file, and installs the wheel to confirm thehermes_agent.pluginsentry point is really declared.Releases: tag-driven, publishing with PyPI Trusted Publishing so there is no long-lived token in the repo. The version now lives only in
hookdeck.__version__andpyprojectreads it from there; the build refuses a tag that disagrees. Needs one-time setup on PyPI — see the checklist below.scripts/check_upstream_contract.py: the suite runs entirely againsttests/hermes_stub.py, which is the only way to exercise ingest without a Hermes checkout but cannot notice the realWebhookAdapterrenaming something underneath it. This parses upstream and asserts every borrowed name is still there — imports, inherited methods, and theself._…attributes the base classes own. Weekly, not per-PR. Verified green against currenthermes-agentand red against a copy with two names deliberately moved.It is a smoke alarm, not a type check: a signature that changes while the name stays put still gets through.
Docs
[
1fe21f2] What the two halves are, and which Hookdeck this is. The README assumed you knew both products, and assumed "Hookdeck" was unambiguous. It is not — the platform includes Outpost, which points the other way, and a reader arriving from the Hookdeck docs had no way to tell which one this is. This plugin is inbound only and now says so."Platform" was the other collision: it is Hermes' word for a source of inbound work, which is why
plugin.yamldeclareskind: platformand why the opening line read "A Hookdeck platform plugin" — a sentence that looks like it is about the Hookdeck Platform and is not.plugin.yaml,__init__.pyandpyproject.tomlalready said "event gateway"; they now capitalise it as the product name so every surface names the same thing.[
2d7d9df] An architecture section, with the CLI in it. The README went from a comparison table straight to install, so the shape of the thing was only ever assembled from the quickstarts. Two diagrams now carry it: a topology flowchart, and a sequence diagram for the ack and hand-back — the subtlest part of the design and the one prose was working hardest to explain. The attempt-counter rule that lets dedup and retry coexist is visible in it rather than asserted.This is also where the CLI is explained, which answers a question the docs left open: three different things here are called CLI. The Hookdeck binary the adapter spawns and supervises, the
hermes hookdeckoperator commands that talk to the REST API instead, and Hermes' own CLI hosting both. The quickstarts keep the operational caveats, because those are things you do rather than things you need to understand the design.Both blocks were rendered before committing.
[
32cb145] What Hookdeck can do that this plugin does not ask it to. The Limitations list read as the boundary of Hookdeck when it is really the boundary ofhookdeck/api.py. A new section lists what is unexposed rather than impossible: the Publish API, bulk operation plans and cancellation, request replay, ignored-event retry, issue triggers, transformations, destination auth, the wider metrics.The durability limitation was also imprecise. It now says push-only in both directions and spells out the consequence that was implicit: an event is recoverable because a delivered-but-failed event stays retryable, not because anything holds a lease on it — which is exactly why boot recovery has to reconcile
runningledger rows itself.Two things that need a decision
1. Is
CLI_DISCONNECTEDa retryable ignored-event cause?POST /bulk/ignored-events/retryretries ignored events matching acause. IfCLI_DISCONNECTEDis accepted, then the sharpest edge in CLI mode — events discarded with no listener attached, which the README currently describes as unrecoverable — has a recovery path, andhermes hookdeck doctorshould be pointing at it. The published docs only evidenceFILTEREDandTRANSFORMATION_FAILED, so it is written as needing confirmation rather than asserted. Someone with a live project can settle this quickly.2. Trusted Publishing needs a one-time setup before the first tag. On PyPI: Project → Publishing → add a GitHub publisher for
hookdeck/hermes-hookdeck, workflowrelease.yml, environmentpypi. Until that exists thepublishjob will fail; nothing else in CI depends on it.Not in scope, and still the biggest gap
tools.pysits at 29% coverage with no test file at all — seven agent-invocable tools, includinghookdeck_pause_connectionandhookdeck_bulk_retry, which act on a live Hookdeck project unsupervised. The pause/auto-resume mechanism spanstools.py→ ledger →adapter._resume_due_connectionswith only the adapter half covered.cli.py(34%) andtunnel.py(41%) are the next two. Worth its own PR.Checks
ruff check .cleanpython -m build+twine checkpass, and the wheel carriesplugin.yaml, the dashboard bundle and the skillscripts/check_upstream_contract.pygreen against currenthermes-agentOpened from a fork — I do not have push access to this repo.
🤖 Generated with Claude Code
https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo