Skip to content

Fix four defects, namespace the environment, add CI and publishing, and document the architecture - #3

Draft
leggetter wants to merge 14 commits into
mainfrom
chore/docs-defects-and-ci
Draft

Fix four defects, namespace the environment, add CI and publishing, and document the architecture#3
leggetter wants to merge 14 commits into
mainfrom
chore/docs-defects-and-ci

Conversation

@leggetter

@leggetter leggetter commented Aug 11, 2026

Copy link
Copy Markdown

Fixes #4.

Started as a review of the repo, became the work the review implied. Nineteen commits, each self-contained: three ingest defects, a fourth found by running against a real Hookdeck project, an environment rename, packaging and CI, and the docs to match.

Nothing here changes the design. It fixes things that were wrong, and writes down things that were true but unstated.


Functional changes, before and after

1. A malformed signature answered 500 instead of 401

Before A non-ASCII byte in x-hookdeck-signature raised out of the verification path and became a 500. That skipped assert_declared_status — so the one status emitted without its retryability declared was the one an unauthenticated caller could trigger — and 500 sits inside the provisioned retry rule's 500-599 range, so Hookdeck retried the forged request ~10 times with backoff.
After 401, like any other forged signature. Dropped on the first refusal.

Reproduced over a raw socket before fixing; the test fails without the fix.

2. A proxy's X-Request-ID could become the delivery identity

Before With no x-hookdeck-eventid, the adapter fell back to X-Request-ID. Anything in front of the gateway can set that; it is not subject to header_prefix; and it is not unique per delivery — one Hookdeck request fans out to one event per matching connection, so two routes sharing a source would collide and the second event would be dropped as a duplicate. It also silenced the warning it stood in for, while POST /events/{proxy-id}/retry 404s — so neither dedup nor retry actually worked.
After No fallback. A delivery genuinely missing the header is processed with the existing loud warning that dedup and retry are unavailable, naming header_prefix as the likely cause.

3. The sync-ack marker only cleared on success

Before _acked_before_completion survived an exhausted retry budget and an abandoned hand-back. Beyond the slow leak: a later sync run of the same event id would find its own id already in the set and be treated as acked-early when it was not, asking Hookdeck to redeliver an event the 5xx response was already asking it to redeliver.
After Cleared on every terminal path — success, exhausted, and abandoned hand-back alike.

4. cli mode could forward from the wrong project, silently

Found by the first live run, and the reason live runs are worth doing.

Before "Which project" was two independent settings. The API key decided what setup, status and the retry hand-back acted on; the Hookdeck CLI's own config decided what hookdeck listen forwarded from. Nothing reconciled them. When they differed: setup reported success, the adapter logged mode=cli listening on…, the gateway looked healthy — and the tunnel restart-looped on no connection found matching filter while every event became a CLI_DISCONNECTED ignored event (events: 0, ignored: 1). It also auto-created a stray source in the unintended project.
After The gateway authenticates a CLI session it owns, from the API key it already has, at ~/.hermes/hookdeck/cli-config.toml, and passes --hookdeck-config to every CLI call. Two projects cannot drift apart when only one of them is configurable. doctor compares them regardless, because the no-API-key fallback still exists.

Deliberately not hookdeck ci against the shared config: that switches the CLI's active project for every other use on the machine, and does so even with --local, despite its help text (hookdeck-cli#332, filed with a reproduction and cause). cli_login is removed — its job is now done properly. Set cli_config_path: "" to opt back into your own ambient session.

CLI sessions also identify as hermes-<hostname> rather than inheriting the bare hostname, so a gateway's tunnels are distinguishable from an operator's own in the Hookdeck dashboard.

5. Environment variables are namespaced to the Event Gateway

Before HOOKDECK_API_KEY, HOOKDECK_WEBHOOK_SECRET, HOOKDECK_MODE, … — a bare prefix claiming the whole namespace for whichever Hookdeck integration got there first.
After HOOKDECK_EG_*. Breaking, with no fallback, because nothing is released.

HOOKDECK_API_KEY is the deliberate exception: it is the Hookdeck CLI's own variable and the adapter hands it to the subprocess it spawns, so HOOKDECK_EG_API_KEY wins when set and HOOKDECK_API_KEY stays a first-class alternative rather than a deprecated one.

6. A project can now be pinned — ahead of organisation-level keys

Before Nothing said which project an API key should act on, because a key implies exactly one.
After HOOKDECK_EG_PROJECT_ID (or project_id in config) sends X-Team-Id, the same header the Hookdeck CLI uses.

Optional today, required once an org-level key reaches several projects. Also a safety improvement now: the dashboard authorises pause/resume by matching connection names against configured routes, so an unscoped org key would let a same-named connection in an unrelated project satisfy that check.


Non-functional

Ruff, lint only — ruff format would rewrite nineteen files and bury the history for no correctness gain. Most of the diff is --fix. The one judgement call is selecting BLE001 rather than ignoring it: this adapter degrades instead of aborting in seven places, two already said so with a noqa, and now all seven carry a one-line justification at the point they are written.

CI — ruff, pytest on 3.10–3.13, and a packaging job that builds the wheel, opens it to confirm plugin.yaml, the dashboard bundle and the skill are inside, then installs it to check the hermes_agent.plugins entry point resolves. A wheel missing those installs perfectly and registers nothing.

Releases — tag-driven, PyPI Trusted Publishing, no stored token. The version lives only in hookdeck.__version__; the build refuses a tag that disagrees. Pre-release tags are marked as such.

scripts/check_upstream_contract.py — the suite runs entirely against tests/hermes_stub.py, which cannot notice the real WebhookAdapter renaming something underneath it. This parses upstream weekly and asserts every borrowed name still exists. A smoke alarm, not a type check.

Packaging metadata — the PyPI page would have been bare. Now an author, a [project.urls] sidebar, classifiers matching the tested Python versions, and PEP 639 License-Expression: MIT instead of the whole licence text inlined into a metadata field. Adds the [build-system] table the project never had.

Docs — an opening that says what Hermes Agent and Hookdeck Event Gateway are; that this is the Event Gateway and not Outpost, and that "platform" is Hermes' word rather than Hookdeck's; an architecture section with a topology diagram and a sequence diagram for the ack-and-hand-back; separation of the three different things called "CLI" here; and a section on what Hookdeck can do that this plugin does not ask it to.


Verification

Beyond 192 unit tests and green CI on a real runner:

Against a real Hermes install — 27/27. Plugin discovered through the hermes_agent.plugins entry point (hermes plugins list shows hookdeck | 0.1.0rc1 | entrypoint), every keyword register_platform is given is a real PlatformEntry field, Platform("hookdeck") mints its enum member, the adapter really subclasses core's WebhookAdapter, and a signed delivery comes out as a prompt rendered by Hermes' own templating.

Against a real Hookdeck project — 20/20, over a live hookdeck listen tunnel with events injected through the Publish API. The two claims no stub can prove:

Agent run failed for event evt_MhT0ps… (run 1 of 4) — asked Hookdeck to redeliver
dispatch route=livetest event_id=evt_MhT0ps… attempt=2
  → Hookdeck: status=SUCCESSFUL attempts=2

POST /events/evt_STfcra…/retry "HTTP/1.1 200 OK"
Found 1 run(s) interrupted by a previous shutdown; asked Hookdeck to redeliver 1 of them
dispatch route=livetest event_id=evt_STfcra… attempt=2 → succeeded

A failed run really is handed back and returns with a higher attempt number; a gateway killed mid-run really does recover its orphan at the next start. Both were previously stub-only, and they are why the plugin exists.

0.1.0rc1 published to PyPI as a dry run of the release path — trusted publishing, reviewer gate, v*-tags-only environment — then installed from PyPI into a clean venv to confirm the entry point and all four non-Python files survive packaging.

The README's "Verified end to end" section previously described a run made before this branch existed. It now describes these.


Known gaps

  • tools.py is at 29% coverage with no test file — seven tools an LLM can invoke unsupervised against a live project. Tracked in Increase test coverage, starting with the agent-callable tools #2.
  • pull_request events produce no CI runs in this repo. Push-triggered CI works. Most likely because main has no .github directory yet, which merging this fixes — open a trivial PR afterwards to confirm.
  • Boot recovery and the retry hand-back are now live-verified; admission-control deferral (503) is not — it remains unit-tested only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo

@leggetter leggetter closed this Aug 11, 2026
@leggetter leggetter reopened this Aug 11, 2026
@leggetter leggetter changed the title Fix three ingest defects, namespace the environment, add CI and linting, and document the architecture Fix four defects, namespace the environment, add CI and publishing, and document the architecture Aug 11, 2026
leggetter and others added 14 commits August 11, 2026 16:07
`hmac.compare_digest` rejects two `str` arguments unless both are pure
ASCII, and the signature header is a value an unauthenticated sender
controls outright. One non-ASCII byte raised `TypeError` out of
`verify_signature`, and aiohttp turned that into a 500.

Wrong twice over. It bypassed `assert_declared_status`, which every other
response goes through precisely so no status escapes without its
retryability being declared — so the one status the adapter emitted
without deciding anything was the one an attacker could trigger. And 500
is inside the provisioned retry rule's `500-599` range, so Hookdeck
retried a forged request roughly ten times with backoff instead of
dropping it on the first refusal.

Compare as bytes, which has no ASCII restriction. Neither encode can
fail: the expected value is base64, and `replace` turns an undecodable
candidate into bytes that simply do not match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit d4ce91e)
The event id fell back to a bare `X-Request-ID` header, which is not a
Hookdeck identifier at all: anything in front of the gateway can set it,
it is not subject to the configurable `header_prefix`, and it is not
unique per delivery — one Hookdeck request fans out to one event per
matching connection, so two routes sharing a source produce two events
carrying the same request id, and the second would be dropped as a
duplicate.

It also defeated the branch it was standing in for. `_admit` handles a
delivery with no event id deliberately: it warns that the delivery is
being processed without deduplication or retry and names `header_prefix`
as the likely cause. A substitute id silences that warning while
delivering neither guarantee — `POST /events/{id}/retry` with a proxy's
id 404s, so the failed run is never handed back.

Hookdeck's own `x-hookdeck-requestid` is not a substitute either, for the
fan-out reason above; `REQUEST_ID` keeps its place in constants with that
written down, so the next person does not reach for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 08bc29f)
`_acked_before_completion` records that a sync run outlasted its timeout
and was acked 202, so its eventual failure gets the explicit hand-back an
async run would get. It was only discarded on success and on a hand-back
that worked — so an event that exhausted its retry budget, or whose
hand-back was abandoned after three failed API calls, left its id in the
set for the life of the process.

Small as a leak. The behavioural half matters more: a later sync-mode run
of the same event id would find its own id already in the set and be
treated as having been acked early when it had not, asking Hookdeck to
redeliver an event the 5xx response was already asking it to redeliver.

Discard once the hand-back decision has been made, which is the point the
marker has done its job, and on the exhausted path that returns before
reaching it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 4ed091c)
Lint only, not formatting. `ruff format` would rewrite nineteen files at
once and bury the history for no correctness gain, so `line-length` is
set for anyone who chooses to run it and E501 stays off — the long lines
here are argparse calls, log statements and comment prose that read worse
wrapped.

Most of the diff is `ruff --fix`: `Optional[X]` to `X | None`, `Dict` to
`dict`, import ordering, `__all__` sorted. All of it is mechanical, and
`from __future__ import annotations` is already everywhere, so nothing
changes at runtime.

The one judgement call is selecting BLE001 rather than ignoring it. This
adapter degrades instead of aborting in seven places — a body it cannot
read, a skill it cannot load, a CLI whose version it cannot determine, a
supervisor that must restart rather than die — and every one is a
decision the author made on purpose. Two already carried a `noqa: BLE001`
saying so. Selecting the rule makes those two meaningful, gives the other
five the same one-line justification, and means the next blind `except`
has to be argued for at the point it is written.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 11201f1)
Three jobs on every push and PR: ruff, pytest across 3.10 to 3.13, and a
packaging job. The third is the one worth arguing for. This plugin is not
just its modules — a wheel missing plugin.yaml, the dashboard bundle or
the bundled skill installs perfectly and then registers nothing, and the
symptom is a platform that is simply absent with no error to debug. So
the job builds the artifact, looks inside it for each of those files, and
installs the wheel to confirm the `hermes_agent.plugins` entry point
Hermes discovers the plugin by is really declared.

Releases are tag-driven and publish to PyPI with Trusted Publishing, so
there is no long-lived API token in the repository. The build job refuses
to proceed if the tag and `hookdeck.__version__` disagree, and the
version now lives only in the module — `pyproject` reads it from there,
because two copies drift and the stale one is always the one nobody
looks at.

Separately, and on a schedule rather than per-PR: the test suite runs
entirely against `tests/hermes_stub.py`, which is the only way to
exercise ingest without a Hermes checkout but cannot notice the real
`WebhookAdapter` renaming something underneath it. The new script parses
upstream and asserts every borrowed name is still there — imports,
inherited methods, and the `self._…` attributes the base classes own. It
is a smoke alarm, not a type check: a signature that changes while the
name stays put still gets through, which is why the README points at a
real end-to-end run as the thing that proves integration. Verified
against the current hermes-agent, and against a copy with two names
deliberately moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 5d8bcb2)
…pinned

`HOOKDECK_*` becomes `HOOKDECK_EG_*`. Hookdeck's platform is more than
one product, and a bare `HOOKDECK_` prefix claims the whole namespace for
whichever integration got there first. Every lookup now goes through one
helper in constants, which reads the namespaced name and falls back to
the old one with a deprecation warning naming its replacement — these
names were documented before the plugin was ever published, and a rename
should not be why someone's gateway stops booting. The fallback goes at
1.0.

`HOOKDECK_API_KEY` is deliberately not renamed the same way. It is the
Hookdeck CLI's own documented variable, and this adapter passes it
through to the `hookdeck listen` subprocess it spawns — so demanding a
second name for one secret would be worse than sharing the ecosystem's
convention. `HOOKDECK_EG_API_KEY` wins when set; `HOOKDECK_API_KEY` is a
first-class fallback, not a deprecated one.

The second half is forward compatibility. A Hookdeck API key is scoped to
one project today, so the key implies the project and nothing has to say
it. Organisation-level keys that reach several projects are coming, and
then it does. `HOOKDECK_EG_PROJECT_ID`, or `project_id` in config.yaml,
sends `X-Team-Id` — the same header the Hookdeck CLI sets.

Worth setting before it is required, because it closes a hole that only
opens with org keys: the dashboard authorises pause and resume by
matching connection *names* against the configured routes, so an unscoped
organisation key would let a same-named connection in an unrelated
project satisfy that check and be paused. `doctor` now reports whether
the project is pinned, and says why it does not matter yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 8d1812f)
The rename shipped with a fallback that read the old bare `HOOKDECK_`
names and warned, plus a changelog entry explaining the migration. Both
were written for users who do not exist: the package has never been
published, so there is no installed base to keep working and no upgrade
to document.

So `env()` goes — with no fallback it was pure indirection over
`os.getenv` — along with its warning bookkeeping, the two tests pinning
the deprecation behaviour, and the migration prose in the README. The
`HOOKDECK_EG_` constants stay, and `HOOKDECK_API_KEY` remains a
first-class alternative for the API key, which was never about
compatibility: it is the Hookdeck CLI's own variable and this adapter
hands it to the subprocess it spawns.

CHANGELOG.md goes the same way. The release workflow now uses
`--generate-notes`, so the commit messages are the changelog rather than
a second place to keep the same story in sync.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit e665a72)
The build was valid but the page it would produce was bare: no author, no
links, no classifiers, and `license = {file = "LICENSE"}` inlining the
entire MIT text into a metadata field meant for a short expression.

Now a PEP 639 SPDX expression (`License-Expression: MIT`, with the file
carried separately), an author, keywords, trove classifiers including the
Python versions CI actually tests, and a `[project.urls]` sidebar
pointing at the repo, the issue tracker, the Hookdeck docs and Hermes.

Also adds the `[build-system]` table the project never had. It was
building on whatever the frontend happened to pick, which worked but was
unstated, and the SPDX expression above needs setuptools 77 or newer — a
build against an older pin would otherwise fail confusingly.

Reverts the temporary push trigger from the previous commit. It did its
job: CI ran on a real runner and every job passed — lint, the 3.10 to
3.13 matrix, and the packaging job that opens the wheel to check the
plugin's manifest, dashboard bundle and skill are inside it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 5aaf1ae)
Exercises the real thing rather than a parallel one: the trusted
publisher just configured on PyPI, the actual OIDC exchange, the `pypi`
environment's reviewer gate, and the artefact CI already builds. A
release candidate is the version number you are allowed to spend on
finding out the pipeline is wrong, and pip will not install it by
default.

Also marks pre-release tags as such. `gh release create` would otherwise
publish v0.1.0rc1 as a normal release and point the repository's "latest
release" at a candidate.

(cherry picked from commit 69dc779)
…cts agree

The first run against a real Hookdeck project failed, and the way it
failed is the point: `setup` reported a connection created, the adapter
logged that it was listening, and the gateway looked healthy — while
`hookdeck listen` restart-looped on "no connection found matching filter"
and every event became a CLI_DISCONNECTED ignored event.

The cause is that "which project" is two independent settings. The API
key decides what `setup`, `status` and the retry hand-back act on; the
Hookdeck CLI's own config decides what `hookdeck listen` forwards from.
Nothing reconciled them, and the CLI had been left pointing somewhere
else weeks earlier.

So the adapter now authenticates a CLI config of its own from the API key
it already has, beside the ledger, and passes `--hookdeck-config` to
every CLI call. Two projects cannot drift apart when only one of them is
configurable. `cli_login` is gone: its job is done properly now, and its
old implementation ran `hookdeck ci` against the shared config, which
switches the CLI's active project for every other use on the machine.
`--local` does not avoid that either, despite its help text —
hookdeck-cli#332.

`doctor` compares the two regardless, because the fallback path still
exists: without an API key there is nothing to authenticate with, and the
ambient session is all there is.

Sessions also identify themselves as `hermes-<hostname>` rather than
inheriting the bare hostname, so a gateway's tunnels are distinguishable
from an operator's own `hookdeck listen` in the Hookdeck dashboard.

Verified against the live project: 20/20 with the gateway pinning its own
session from a directory with no local config to fall back on, and the
operator's `~/.config/hookdeck/config.toml` byte-identical afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 873b119)
`hookdeck ci --local` writes credentials to `.hookdeck/config.toml` in
the working directory, and says so in its own output. Nothing in this
repo creates it, but anyone running that command from the checkout would
have a credential file sitting untracked and easy to add by accident.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
(cherry picked from commit 10bd29f)
A 53KB binary full of local absolute paths, committed by accident while
measuring coverage in the working tree.

(cherry picked from commit 895f81693fc0215b514074cbd851fad95bb7891b)
Five things wrong, four of them in code added earlier on this branch.

`cli_config_path:` written as a bare YAML key parses as None, and `str()`
turned that into the literal "None" — so the adapter would pass
`--hookdeck-config None` and write a file by that name into its working
directory, while doctor inspected the default path instead. A bare key is
the most natural way to write the empty value the docs ask for.

`~` was never expanded, and the Hookdeck CLI does not expand it either:
an unexpanded path makes the CLI create a directory literally named `~`.
doctor did expand it, so the two inspected different files.

`hookdeck ci` ran once per route against one shared config file, so with
two or more routes a `listen` process read that file while another `ci`
was writing it — and one gateway minted a session per route. It now runs
once, before any tunnel starts.

A failed `hookdeck ci` used to be logged and ignored, leaving `listen` to
restart-loop forever against a config that was never written. It now
fails the connect, because a tunnel that cannot authenticate is not a
tunnel.

doctor read the first `project_id` in a CLI config, but those files are
multi-section with a top-level `profile` key choosing the active one — so
anyone with more than one profile got a mismatch that was not real. It
now reads the active profile's section, and distinguishes "no such file"
from "file present, names no project", which mean different things.

Project pinning goes entirely. It existed to name a project explicitly,
and the review found it only ever reached the adapter's own API client —
not the agent tools, not the dashboard, not the CLI — so the guarantee it
appeared to offer was not one it delivered. Better absent than
half-present.

The settings test that would have caught two of these had been weakened
to `endswith("cli-config.toml")`, which passes for "None" and for an
unexpanded `~` path. It now asserts the resolved path, and the six doctor
branches that had no coverage at all have tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
Rebasing onto main's README rewrite left the docs describing an older
plugin. Correctness first: the quickstart exported `HOOKDECK_API_KEY` and
`HOOKDECK_WEBHOOK_SECRET`, which the code no longer reads, so following
the README would produce a gateway that refuses to start.
`docs/security.md` named the old allowlist variable, and
`examples/config.yaml` still offered `cli_login`, which no longer exists.

`docs/operations.md` said the adapter deliberately does not authenticate
the CLI. It now does, for reasons that section did not have when it was
written, so it says the opposite and explains why.

New material goes where main put reference material rather than back into
the README. `docs/architecture.md` carries the topology and the
ack-and-hand-back sequence diagram — both rendered before committing —
plus the delivery pipeline and the three things called "CLI".
`docs/limitations.md` gains what Hookdeck can do that this plugin does
not ask it to, and the fact that CLI_DISCONNECTED events are recoverable
with a reconnect-first ordering rule.

The README keeps main's shape: it says which Hookdeck this is, since the
platform is more than one product and Outpost points the other way.

The "Verified end to end" section stays deleted, as main decided. The
evidence lives in the pull request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiavrHa47c8mRR2kekHgVo
@leggetter
leggetter force-pushed the chore/docs-defects-and-ci branch from 10bd29f to 5780f2f Compare August 11, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli mode: API key project and CLI project can differ silently, and the gateway reports healthy

1 participant