diff --git a/.env.example b/.env.example index 2ad53e9..9fcf76d 100644 --- a/.env.example +++ b/.env.example @@ -1,6 +1,6 @@ NODE_ENV=development PORT=3000 -MONGO_URI=mongodb://localhost:27017/htmltrust +MONGO_URI=mongodb://localhost:27017/content-signing # Public base URL of this directory. Used to decide whether a keyid of the # form https://host/api/keys/{id} names a key held here or somewhere else. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9777a7f..07a6b5c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -30,25 +30,8 @@ jobs: with: node-version: "22" - # The package token used to be written to ~/.gitconfig, where every later - # step -- including any dependency lifecycle script -- could read it back. - # It is now passed through GIT_CONFIG_* environment variables, which git - # honours for this process tree only and never persists to disk, and - # --ignore-scripts keeps third-party install hooks from running at all - # while the token is in the environment. - # - # HTMLTRUST_PKG_TOKEN must be a fine-grained PAT scoped to the - # HTMLTrust/htmltrust-canonicalization repository with Contents: Read and - # nothing else. A classic `repo`-scoped token grants write access to every - # repo the owner can reach and must not be used here. - name: Install dependencies - env: - GIT_CONFIG_COUNT: "2" - GIT_CONFIG_KEY_0: url.https://x-access-token:${{ secrets.HTMLTRUST_PKG_TOKEN }}@github.com/.insteadOf - GIT_CONFIG_VALUE_0: https://github.com/ - GIT_CONFIG_KEY_1: url.https://x-access-token:${{ secrets.HTMLTRUST_PKG_TOKEN }}@github.com/.insteadOf - GIT_CONFIG_VALUE_1: ssh://git@github.com/ - run: npm ci --ignore-scripts + run: npm ci --ignore-scripts --no-audit --no-fund - name: Run unit tests run: npm test @@ -79,7 +62,7 @@ jobs: npm run conformance - name: Validate OpenAPI spec - run: npx @redocly/cli lint openapi.yaml --skip-rule no-unused-components || true + run: npm run openapi:lint - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: diff --git a/README.md b/README.md index 48b5b64..533f8ce 100644 --- a/README.md +++ b/README.md @@ -1,228 +1,180 @@ -# HTMLTrust Server Reference (Node.js) +# HTMLTrust Server Reference -Runnable Node.js reference implementation of the HTMLTrust trust directory API. It manages author identities, cryptographic key pairs, content signing and verification, and a federated trust directory with reputation tracking. +This repository contains the runnable Node.js reference server for the HTMLTrust trust directory API. It stores author profiles and public keys, accepts signed content and endorsements, and exposes directory search and reputation data. -This is a companion to the [HTMLTrust specification](https://github.com/HTMLTrust/htmltrust-spec). +The wire contract is documented in [`openapi.yaml`](openapi.yaml). The server is the implementation used by the local development workflow and the end-to-end simulation. -## Current status +## Quick start -This repository is the runnable Node.js reference server used for local development and the end-to-end simulation. The sibling Python and Rust repositories are design scaffolds with no runnable server code and make no conformance claim. +### Requirements -## Prerequisites +For a local Node run, install: -- Node.js 22+ -- MongoDB, local or remote +- Node.js 22 or newer +- MongoDB 7 or a compatible MongoDB deployment -## Quick start +Docker users can run the complete test suite without installing Node.js or MongoDB. See [Test in Docker](#test-in-docker). + +### Checkout, install, and run ```sh git clone https://github.com/HTMLTrust/htmltrust-server-reference.git cd htmltrust-server-reference npm ci -cp .env.example .env # Edit with your values -npm run dev # Starts with nodemon (auto-reload) +cp .env.example .env +npm run dev ``` -The server starts at `http://localhost:3000`. A demo web UI is available at the root URL. See [Environment Variables](#environment-variables) for required production settings. - -Run the unit tests with `npm test`. The full API conformance suite needs a disposable MongoDB instance; see [Tests](#tests). - -## Personality: the "permissive community directory" - -The HTMLTrust protocol is federated, meaning multiple trust directories MAY coexist with different curatorial philosophies. This Node.js implementation is the baseline reference: full-featured, permissive, and neutral, suitable for general-purpose deployment and for exercising the OpenAPI endpoints. - -The sibling reference implementations demonstrate alternative curatorial philosophies using the same protocol: - -- **[`htmltrust-server-reference-python`](../htmltrust-server-reference-python/)** -- planned curated journalism directory. Its repository is a design scaffold. -- **[`htmltrust-server-reference-rust`](../htmltrust-server-reference-rust/)** -- planned rapid-flag public-safety directory. Its repository is a design scaffold. +Set `MONGO_URI` in `.env` to the database used by the server. The default development URI is `mongodb://localhost:27017/content-signing`, and the server listens on port `3000`. Open `http://localhost:3000/` for the demo page. -The planned implementations target the same OpenAPI contract. Only this Node.js repository currently provides a runnable reference for local testing. +`npm run dev` uses nodemon. Use `npm start` for a regular Node process. -## What It Does +### Run tests -This server implements the **Trust Directory** component of the HTMLTrust system: +Unit tests use Node's built-in test runner and require no database: -- **Author Management** — Create and manage author profiles with cryptographic key pairs -- **Content Signing** — Sign content hashes with author private keys, producing verifiable signatures -- **Content Verification** — Verify that content signatures are authentic and untampered -- **Trust Directory** — Search for public keys, track content occurrences across domains, and manage reputation -- **Voting & Reputation** — Community-driven trust/distrust system for authors and content -- **Claims** — Extensible metadata system for content categorization (authorship type, license, AI involvement, etc.) - -## Tech Stack - -- **Node.js** + **Express 5** -- **MongoDB** via **Mongoose** -- **Node.js `crypto`** for key generation, signing, and verification (RSA, ECDSA, Ed25519) +```sh +npm test +npm run openapi:lint +``` -### Tests +Install the conformance runner once, then run the reference server against a disposable in-process MongoDB: ```sh -npm test # unit tests: JCS, claims canonicalization, RFC 9421 verification -npm run conformance # full API conformance suite against a disposable MongoDB +npm --prefix conformance/runner ci +npm run conformance ``` -`npm test` needs no database. `npm run conformance` boots `mongodb-memory-server` and the reference server itself; set `SERVER_PORT` / `MONGO_PORT` if 3000 or 37017 are taken. +The conformance command runs every fixture and the canonical v1 smoke checks. Set `SERVER_PORT` or `MONGO_PORT` when the defaults are occupied. The first run can download a MongoDB binary for `mongodb-memory-server`. -For the Docker-based conformance path, use the repository script. It starts a -MongoDB 7 container bound to loopback, starts this server with Node on the -host, runs all fixtures, and removes the disposable container when complete: +## Test in Docker + +The repository script runs unit and conformance tests inside a disposable Node 22 container. It mounts the checkout read-only, copies sources into the container, and installs dependencies there. Test output and generated runtime files leave no files in the checkout. Checkout-scoped Docker volumes cache npm packages and the MongoDB test binary. ```sh -npm ci -npm run conformance:docker +./scripts/test-in-docker.sh ``` -Use `SERVER_PORT=3100 MONGO_PORT=37018 npm run conformance:docker` when the -default ports are occupied. Add `--keep-running` to leave the server and -MongoDB container running for manual requests. - -### Environment Variables - -See `.env.example` for all options. At minimum you need: +This is the lowest-dependency test path: it requires Docker and a shell. Set `HTMLTRUST_TEST_IMAGE` to use another compatible Node image. The older `npm run conformance:docker` command remains available for developers who want to run the conformance runner with a host Node process and a Docker MongoDB container. -| Variable | Description | -|---|---| -| `MONGO_URI` | MongoDB connection string | -| `AUTHOR_API_KEY_PEPPER` | Pepper for author API key hashing. Required when `NODE_ENV=production`; the server refuses to start without it | -| `GENERAL_API_KEY` | Supplementary demo key for submission endpoints | -| `ADMIN_API_KEY` | Admin key for directory-operator operations (defining claim types, endorsement takedown) | +## Deployment -The CMS integration normally sets `HTMLTRUST_API_URL` to the server origin, -`HTMLTRUST_AUTHOR_ID` to the registered author, and -`HTMLTRUST_AUTHOR_API_KEY` to the one-time key returned by `POST /api/authors`. -The server's compatibility signing endpoint is -`POST /api/content/sign`; it returns `contentHash`, `claimsHash`, `signature`, -`keyid`, and `algorithm`. The CMS passes its publication origin in `domain` and -must preserve the returned algorithm and keyid in the signed section. +1. Provision MongoDB 7, create a database for this service, and set `MONGO_URI` with credentials appropriate for the deployment. +2. Install production dependencies from the lockfile: -## API Overview + ```sh + npm ci --omit=dev + ``` -Full API documentation is in [`openapi.yaml`](openapi.yaml). Key endpoint groups: +3. Set `NODE_ENV=production`, `AUTHOR_API_KEY_PEPPER`, and `DIRECTORY_BASE_URL`. Use a random, long-lived pepper and keep it outside the repository. `DIRECTORY_BASE_URL` must be the public origin clients use to resolve directory key URLs. +4. Set `GENERAL_API_KEY` and `ADMIN_API_KEY` when compatibility or operator routes need them. Static general and author API-key authentication is disabled in production unless `HTMLTRUST_ALLOW_API_KEY_AUTH=1` is set. +5. Start the service with `npm start` behind a TLS-terminating reverse proxy. Set `TRUST_PROXY` to the number of trusted proxy hops when the proxy forwards client addresses. -| Path | Description | Auth | -|---|---|---| -| `GET /api/.well-known/htmltrust` | Discover directory capabilities | Public | -| `GET /api/content/:hash` | Get draft content record by percent-encoded hash | Public | -| `POST /api/content` | Submit a signed content occurrence | HTTP Message Signature | -| `GET /api/content/:hash/endorsements` | List structured endorsements for a content hash | Public | -| `GET /api/keys/:id` | Get draft key document | Public | -| `GET /api/signers/:id/reputation` | Get draft signer reputation | Public | -| `POST /api/authors` | Create author + key pair | General API key | -| `GET /api/authors/:id/public-key` | Get author's public key | Public | -| `POST /api/content/sign` | Compatibility helper: sign contentHash + claimsHash | Author API key | -| `POST /api/content/verify` | Verify a signature (deprecated, see below) | Public | -| `GET /api/directory/keys` | Search public keys | Public | -| `GET /api/directory/content` | Search signed content | Public | -| `GET /api/endorsements?content-hash=...` | List endorsements for a content hash | Public | -| `POST /api/endorsements` | Submit a signed endorsement | HTTP Message Signature | -| `DELETE /api/endorsements/:id` | Delete an endorsement | Endorser's own key, or admin key | -| `POST /api/votes` | Vote trust/distrust | HTTP Message Signature | +Before upgrading a database created by an earlier server version, run the explicit index migration with the same connection string used by the service: -### Deprecated endpoints +```sh +MONGO_URI="mongodb://user:password@db.example/htmltrust" npm run migrate:v1 +``` -`POST /api/content/verify` is deprecated. Per [HTMLTrust spec §3.1](https://htmltrust.dev/spec#section-3-1), cryptographic verification is a local operation: clients MUST verify signatures themselves (e.g. via `SubtleCrypto`) using public keys resolved through the directory's key endpoints. A remote yes/no answer from the directory is by definition not a cryptographic guarantee since the directory is not part of the trust root. The endpoint remains as a low-trust convenience for legacy clients, returns the `Deprecation: true` header (RFC 9745), and will be removed in a future major version. The directory's role is to serve public keys, endorsements, and reputation data — not to act as an oracle for signature validity. +The migration replaces the legacy content identity and endorsement indexes. Run it during a maintenance window and verify backups before changing production data. -### Draft wire-format notes +See [.env.example](.env.example) for every supported setting. Remote `did:` and HTTPS key resolution is disabled by default because dereferencing submitter-provided URLs creates an outbound-request risk. Enable `HTMLTRUST_REMOTE_KEY_RESOLUTION=1` only after reviewing the network policy for the deployment. -Hashes, signatures, and key bytes use canonical unpadded standard Base64, not base64url. JSON fields named `domain` carry the serialized Web origin (`scheme://host[:port]`), not a bare hostname. Content signatures bind `contentHash:claimsHash:domain:signedAt`, where `claimsHash` is the SHA-256 of the draft §4.6 canonical claims serialization over all direct child `meta` claims in the signed section. +## API -Endorsement signatures cover the RFC 8785 JCS serialization of the endorsement document with the `signature` member omitted (draft §10.2). The directory verifies that signature against the endorser's resolved key before storing anything, and serves the stored document back byte-for-byte: it injects no `_id`, `createdAt`, or `contentHash` alias, because §10.1 requires unrecognised members to be included in the signed payload, so any injected member would break verification for the next reader. The identifier of a newly stored endorsement is returned in the `Location` header of the 201 response. `contentHash` appears only on documents stored by earlier versions of this server. +The server exposes two HTTP surfaces. The root routes are the canonical HTMLTrust v1 directory surface. The `/api` routes are compatibility routes used by the demo UI and existing integrations. -Endorsements are append-only. Resubmitting an identical document is idempotent (200 instead of 201); a different document from the same endorser for the same content hash — a revocation, for instance — is stored alongside the original, because §10.3 requires a directory holding both to serve both. +### Canonical v1 routes -### Authentication +| Method and path | Purpose | Authentication | +|---|---|---| +| `GET /.well-known/htmltrust` | Discover directory version, capabilities, algorithms, and profiles | Public | +| `GET /keys/:id` | Retrieve a directory key document | Public | +| `GET /signers/:id/reputation` | Retrieve signer reputation | Public | +| `POST /content` | Submit and re-verify a signed content record | RFC 9421 HTTP Message Signature | +| `GET /content/:hash` | Retrieve a content record by percent-encoded hash | Public | +| `GET /content/:hash/endorsements` | List endorsements for a content hash | Public | +| `GET /endorsements?content-hash=...` | List endorsements for a content hash | Public | +| `POST /endorsements` | Store a signed endorsement | RFC 9421 HTTP Message Signature | +| `DELETE /endorsements/:id` | Delete an endorsement with the endorser key or directory admin key | Endorser signature or admin key | -Draft §9.8 requires POST endpoints to authenticate with an [RFC 9421](https://www.rfc-editor.org/rfc/rfc9421) HTTP Message Signature made with a key the directory can resolve per §8. The signature MUST cover the request target, `host`, `date`, and — for requests with a body — `content-digest`: +Canonical writes require a resolvable key in an RFC 9421 signature. The covered components include `@method`, `@target-uri`, `host`, `date`, and `content-digest` for requests with a body. The `keyid` identifies the key that signed the request. -``` -Signature-Input: sig1=("@method" "@target-uri" "host" "date" "content-digest");\ - created=1770000000;keyid="https://directory.example/api/keys/k-abc123" -Signature: sig1=:MEUCIQD...: -``` +Canonical reads return an HTMLTrust media type by default and accept `application/json`. Responses include `Vary: Accept`; public reads include `Cache-Control` and `ETag`. A matching `If-None-Match` request receives `304 Not Modified`. Canonical JSON submissions accept `application/json` and `application/*+json`; another request media type receives `415`. -The authenticated identity is the resolved key, which is what lets the directory bind a submission, a vote, or an endorsement deletion to a specific signer. +### Compatibility routes -The static API keys below remain as a supplementary demo and operator scheme. A shared secret says nothing about *who* sent a request, so it cannot carry submitter identity; requests authenticated this way vote as a single collapsed identity and cannot delete another party's endorsement. They are refused when `NODE_ENV=production` unless `HTMLTRUST_ALLOW_API_KEY_AUTH=1` is set. +The following routes retain the original `/api` prefix and response shapes: -| Header | Purpose | +| Route group | Operations | |---|---| -| `X-API-KEY` | Demo submission key (creating authors, voting, reporting) | -| `X-AUTHOR-API-KEY` | Author-specific operations (directory-side signing, updating own profile) | -| `X-ADMIN-API-KEY` | Directory-operator operations (managing claim types, endorsement takedown) | +| `/api/authors` | Create and list authors; read, update, or delete an author; read its public key | +| `/api/content` | Sign, verify, submit, and retrieve content; register occurrences; list content endorsements | +| `/api/claims` | Create, list, read, update, and delete claim types | +| `/api/directory` | Search keys and content; read key reputation and occurrences; report keys or content | +| `/api/endorsements` | List, submit, and delete endorsements | +| `/api/votes` | Submit votes; list votes; read vote statistics; delete a vote | +| `/api/keys` and `/api/signers` | Read the compatibility key and signer-reputation documents | +| `/api/.well-known/htmltrust` | Compatibility alias for the discovery document | -Author API keys are stored as an HMAC-SHA-256 under `AUTHOR_API_KEY_PEPPER` and are shown exactly once, at author creation. Deployments upgrading from a version that stored them in plaintext need the one-time migration described in `src/utils/apiKeys.js`; databases created before endorsements became append-only also need the two unique indexes dropped, as described in `src/models/Endorsement.js`. +The compatibility routes use the API-key headers described below for their original protected operations. Content and endorsement submission also accept the RFC 9421 flow, while canonical root submissions require that flow. The OpenAPI file defines the long-term v1 resource shapes and media types; this implementation currently keeps author, claim-management, directory-search, reporting, and voting operations under `/api`. -### Key custody +### Deprecated route -`POST /api/authors` accepts an optional `publicKey` (SPKI PEM). Supply it to register a key you already hold: the directory then stores no private key for that author, and content is signed locally and submitted through `POST /api/content`. Omit it and the directory generates and holds the key pair, acting as the convenience registry of draft §9.6. +`POST /api/content/verify` is deprecated and returns `Deprecation: true` as specified by RFC 9745. Signature verification belongs in the client, using a public key retrieved from the directory and a local cryptographic API such as `SubtleCrypto`. The route remains for legacy clients and will be removed in a future major version. There is no canonical root `/content/verify` route. -Resolving `did:` and `https:` keyids means dereferencing URLs chosen by whoever submits a record, which is a server-side request forgery primitive. It is therefore off by default; only keys held by this directory resolve. Set `HTMLTRUST_REMOTE_KEY_RESOLUTION=1` to enable it. +### Authentication headers -## Project Structure +Use RFC 9421 signatures for canonical writes. The compatibility surface supports these headers when its API-key authentication is enabled: -``` -src/ -├── server.js # Express app entry point -├── config/ -│ └── db.js # MongoDB connection -├── controllers/ # Route handlers -│ ├── authorController.js -│ ├── claimController.js -│ ├── contentController.js -│ ├── directoryController.js -│ ├── endorsementController.js -│ └── voteController.js -├── middleware/ -│ └── auth.js # API key authentication -├── models/ # Mongoose schemas -│ ├── Author.js -│ ├── Claim.js -│ ├── ContentOccurrence.js -│ ├── ContentSignature.js -│ ├── Endorsement.js -│ ├── Key.js -│ └── Vote.js -├── public/ # Demo web UI -│ ├── index.html -│ └── js/main.js -├── routes/ # Express route definitions -│ ├── authors.js -│ ├── claims.js -│ ├── content.js -│ ├── directory.js -│ ├── endorsements.js -│ └── votes.js -└── utils/ - └── crypto.js # Key generation, signing, verification -``` +| Header | Use | +|---|---| +| `X-API-KEY` | General compatibility operations, including author creation, occurrence registration, reporting, and demo submissions | +| `X-AUTHOR-API-KEY` | Author-specific compatibility operations and the compatibility signing helper | +| `X-ADMIN-API-KEY` | Claim-type administration and endorsement takedown | -## Companion Repositories +Author API keys are returned once by `POST /api/authors`. The server stores an HMAC-SHA-256 digest under `AUTHOR_API_KEY_PEPPER`. A deployment that still has plaintext keys must migrate them using the procedure in [`src/utils/apiKeys.js`](src/utils/apiKeys.js), then remove the plaintext field. -| Repository | Description | -|---|---| -| [htmltrust-spec](https://github.com/HTMLTrust/htmltrust-spec) | The HTMLTrust specification and paper | -| [htmltrust-browser-reference](https://github.com/HTMLTrust/htmltrust-browser-reference) | Reference browser extension for signature validation | -| [htmltrust-cms-reference](https://github.com/HTMLTrust/htmltrust-cms-reference) | Reference CMS plugin (WordPress) | -| [htmltrust-website](https://github.com/HTMLTrust/htmltrust-website) | Project website | +### Wire-format details -## License +- Hashes, signatures, and key bytes use unpadded standard Base64. +- `domain` values are serialized web origins such as `https://publisher.example:8443`. +- Content signatures bind `contentHash`, `claimsHash`, `domain`, and `signedAt`. `claimsHash` covers the canonical serialization of direct `meta` claims in the signed section. +- Endorsement signatures cover the RFC 8785 JCS serialization of the endorsement document with `signature` omitted. New endorsements are served as signed, and the 201 response supplies the stored identifier in `Location`. +- Endorsements are append-only. An identical retry on canonical `POST /endorsements` returns `201` with the existing resource in `Location`; the `/api/endorsements` compatibility route returns `200`. A different document from the same endorser and content hash is stored as another record. +### Key custody -This project is licensed under the [PolyForm Noncommercial License 1.0.0](https://polyformproject.org/licenses/noncommercial/1.0.0). You may use, modify, and share the software for any noncommercial purpose with attribution. Commercial use requires a separate agreement with the licensor. +`POST /api/authors` accepts an optional SPKI PEM `publicKey`. Supplying one registers a key held by the caller and leaves its private key outside the directory. Omitting it asks the server to generate and hold a key pair for the convenience registry flow. Content signed by a caller-held key is submitted through `POST /content` or the compatibility `POST /api/content` route. -## Origin & Contributions +## Project structure -HTMLTrust is an idea I (Jason Grey) have been chewing on since 2024. I'm not an academic — I'm an engineer with a day job and a family — so the spec, the reference implementations, and most of this prose have been written with significant help from AI tools acting as research assistant, technical writer, and pair programmer. I wrote the original architectural sketches and reviewed every line; the assistants filled in the gaps and saved me from re-typing the same explanation for the hundredth time. +```text +src/ +├── server.js Express application entry point +├── config/ MongoDB connection setup +├── controllers/ Request handlers +├── middleware/ Authentication and content negotiation +├── models/ Mongoose schemas +├── public/ Demo web UI +├── routes/ Compatibility route definitions +└── utils/ Cryptography and protocol helpers +conformance/ +├── fixtures/ YAML API scenarios +└── runner/ Implementation-agnostic conformance runner +scripts/test-in-docker.sh Docker-only unit and conformance entrypoint +openapi.yaml API contract and response schemas +``` -**Contributions are welcome — human or AI-assisted, doesn't matter to me.** What matters is whether the code, the spec text, or the conformance vectors move the project forward. Open a PR. +## Related repositories -What this project is **not** a forum for: +- [HTMLTrust specification](https://github.com/HTMLTrust/htmltrust-spec) +- [Browser reference](https://github.com/HTMLTrust/htmltrust-browser-reference) +- [CMS reference](https://github.com/HTMLTrust/htmltrust-cms-reference) +- [Project website](https://github.com/HTMLTrust/htmltrust-website) -- Debates about whether AI should be used to write code or specifications. -- Opinions on who is or isn't trustworthy on the web. -- Politics, religion, professional practice, or personal philosophy. +## License and contributions -HTMLTrust is a mechanism — a way for *anyone* to sign content they publish and for *anyone* to decide whom they trust, on their own terms. The project takes no position on what the right answers are; it just provides the tools. If you want to debate the answers, there are entire continents of the internet better suited to it. +This project is licensed under the [PolyForm Noncommercial License 1.0.0](https://polyformproject.org/licenses/noncommercial/). Commercial use requires a separate agreement with the licensor. -If this work is useful to you and you'd like to support it, see [GitHub Sponsors](https://github.com/sponsors/jt55401) or the other channels in [`.github/FUNDING.yml`](.github/FUNDING.yml). +Issues and pull requests are welcome. Contributions may include code, specification text, documentation, or conformance fixtures. Please keep changes focused on improving the protocol and its implementations. diff --git a/conformance/fixtures/03-signed-content-submission.yaml b/conformance/fixtures/03-signed-content-submission.yaml index 3118242..2a6e663 100644 --- a/conformance/fixtures/03-signed-content-submission.yaml +++ b/conformance/fixtures/03-signed-content-submission.yaml @@ -32,7 +32,7 @@ steps: X-AUTHOR-API-KEY: $authorApiKey body: contentHash: "sha256:xJTJYuXl1MuP1EjRLhKtgMUZvvc6qexrTMHyVnVL+Yc" - claimsHash: "sha256:EOlXUVED7G9RI90/iTXXNtY79KQEW6LxLVOVtsjlHWs" + claimsHash: "sha256:4LOflWusiW26FjvAHhwAZpPqZrLblKkYZ1QYKPORKDo" domain: "https://conformance.example.com" signedAt: "2026-05-12T12:00:00.000Z" claims: @@ -60,7 +60,7 @@ steps: path: /content/verify body: contentHash: "sha256:xJTJYuXl1MuP1EjRLhKtgMUZvvc6qexrTMHyVnVL+Yc" - claimsHash: "sha256:EOlXUVED7G9RI90/iTXXNtY79KQEW6LxLVOVtsjlHWs" + claimsHash: "sha256:4LOflWusiW26FjvAHhwAZpPqZrLblKkYZ1QYKPORKDo" domain: "https://conformance.example.com" signedAt: "2026-05-12T12:00:00.000Z" authorId: $authorId diff --git a/conformance/fixtures/04-content-retrieval-by-hash.yaml b/conformance/fixtures/04-content-retrieval-by-hash.yaml index 8ec73b0..31c2c6f 100644 --- a/conformance/fixtures/04-content-retrieval-by-hash.yaml +++ b/conformance/fixtures/04-content-retrieval-by-hash.yaml @@ -33,7 +33,7 @@ steps: X-AUTHOR-API-KEY: $authorApiKey body: contentHash: "sha256:82rHSQ/ThLduI0bbHYOVbxn5mEXR0FxMzn6YsVHSwSs" - claimsHash: "sha256:epUf+9l+yWgGFMHwNw++jCpep5Ib/f4T5dZBDO8nb5o" + claimsHash: "sha256:M9aSt+kY8XVWfoAaE4jJEbxdU2sOw01Kr8BU/vhyoUo" domain: "https://retrieval.example.com" signedAt: "2026-05-12T12:00:00.000Z" claims: diff --git a/conformance/fixtures/07-key-reputation.yaml b/conformance/fixtures/07-key-reputation.yaml index acf745a..a6dd817 100644 --- a/conformance/fixtures/07-key-reputation.yaml +++ b/conformance/fixtures/07-key-reputation.yaml @@ -61,7 +61,7 @@ steps: expect: status: 200 body: - kid: $keyId + kid: "$rootUrl/keys/$keyId" algorithm: "ed25519" publicKey: $nonempty-string diff --git a/conformance/runner/run.mjs b/conformance/runner/run.mjs index 1993c30..59fcb39 100755 --- a/conformance/runner/run.mjs +++ b/conformance/runner/run.mjs @@ -674,6 +674,7 @@ async function runScenario(config, openapi, fixture, opts) { adminApiKey: config.adminApiKey, run_nonce: runNonce, baseUrl: `${config.targetUrl}${config.basePath}`, + rootUrl: config.targetUrl, signerPublicKeyPem: signingKey.publicKey, __signingKey: signingKey, ...(fixture.doc.vars || {}), diff --git a/conformance/runner/v1-smoke.mjs b/conformance/runner/v1-smoke.mjs new file mode 100644 index 0000000..d2d4edb --- /dev/null +++ b/conformance/runner/v1-smoke.mjs @@ -0,0 +1,251 @@ +#!/usr/bin/env node + +import { + createHash, + generateKeyPairSync, + sign as cryptoSign, +} from "node:crypto"; + +const target = (process.argv[2] || "http://localhost:3000").replace(/\/$/, ""); +const generalApiKey = process.argv[3] || process.env.GENERAL_API_KEY || "conformance_general_key"; + +const fail = (message, detail) => { + if (detail !== undefined) console.error(detail); + throw new Error(message); +}; + +const requestJson = async (url, init = {}, expectedStatus = 200) => { + const response = await fetch(url, init); + const text = await response.text(); + let body; + try { + body = text ? JSON.parse(text) : undefined; + } catch { + body = text; + } + if (response.status !== expectedStatus) { + fail(`${init.method || "GET"} ${url} returned ${response.status}, expected ${expectedStatus}`, body); + } + return { response, body }; +}; + +const canonicalize = (value) => { + if (value === null || typeof value !== "object") return JSON.stringify(value); + if (Array.isArray(value)) return `[${value.map(canonicalize).join(",")}]`; + return `{${Object.keys(value) + .sort((left, right) => (left === right ? 0 : left < right ? -1 : 1)) + .map((key) => `${JSON.stringify(key)}:${canonicalize(value[key])}`) + .join(",")}}`; +}; + +const unpadded = (buffer) => buffer.toString("base64").replace(/=+$/, ""); +const prefixedSha256 = (text) => `sha256:${unpadded(createHash("sha256").update(text).digest())}`; + +const signHttpRequest = ({ url, body, keyid, privateKey, nonce }) => { + const parsed = new URL(url); + const date = new Date().toUTCString(); + const created = Math.floor(Date.now() / 1000); + const digest = createHash("sha256").update(body).digest("base64"); + const contentDigest = `sha-256=:${digest}:`; + const parameters = + `("@method" "@target-uri" "host" "date" "content-digest")` + + `;created=${created};keyid="${keyid}";alg="ed25519";nonce="${nonce}"`; + const base = [ + '"@method": POST', + `"@target-uri": ${url}`, + `"host": ${parsed.host}`, + `"date": ${date}`, + `"content-digest": ${contentDigest}`, + `"@signature-params": ${parameters}`, + ].join("\n"); + const signature = unpadded(cryptoSign(null, Buffer.from(base, "utf8"), privateKey)); + return { + "content-type": "application/json", + host: parsed.host, + date, + "content-digest": contentDigest, + "signature-input": `sig1=${parameters}`, + signature: `sig1=:${signature}:`, + }; +}; + +const signedPost = async ({ path, document, keyid, privateKey, nonce, expectedStatus = 201 }) => { + const url = `${target}${path}`; + const body = JSON.stringify(document); + return requestJson(url, { + method: "POST", + headers: signHttpRequest({ url, body, keyid, privateKey, nonce }), + body, + }, expectedStatus); +}; + +const main = async () => { + const signingKey = generateKeyPairSync("ed25519", { + publicKeyEncoding: { type: "spki", format: "pem" }, + privateKeyEncoding: { type: "pkcs8", format: "pem" }, + }); + + const author = await requestJson(`${target}/api/authors`, { + method: "POST", + headers: { + "content-type": "application/json", + "x-api-key": generalApiKey, + }, + body: JSON.stringify({ + name: `V1 smoke ${Date.now()}`, + keyType: "ORGANIZATION", + keyAlgorithm: "ed25519", + publicKey: signingKey.publicKey, + }), + }, 201); + const authorId = author.body.author.id; + const publicKey = await requestJson(`${target}/api/authors/${authorId}/public-key`); + const keyId = publicKey.body.id || publicKey.body._id; + const keyid = `${target}/keys/${keyId}`; + + const discovery = await requestJson(`${target}/.well-known/htmltrust`); + if (!discovery.body.supportedProfiles?.includes("htmltrust-signature-v1")) { + fail("discovery does not advertise htmltrust-signature-v1", discovery.body); + } + if (discovery.body.directory !== `${target}/`) { + fail("discovery directory does not name the canonical root", discovery.body); + } + + const keyDocument = await requestJson(keyid); + if (keyDocument.body.kid !== keyid || keyDocument.body.publicKeyPem !== undefined) { + fail("root key document has the wrong kid or exposes the PEM compatibility field", keyDocument.body); + } + + const signedAt = "2026-01-15T12:00:00Z"; + const claims = [ + { name: "author", content: "Ada Lovelace" }, + { name: "claim:License", content: "CC-BY-4.0" }, + { name: "signed-at", content: signedAt }, + ]; + const canonicalClaims = + "author:Ada Lovelace\nclaim\\:License:CC-BY-4.0\nsigned-at:2026-01-15T12\\:00\\:00Z\n"; + const contentHash = prefixedSha256("HTMLTrust v1 integration content"); + const claimsHash = prefixedSha256(canonicalClaims); + const location = "https://example.com/research/paper?revision=1"; + const signingObject = { + algorithm: "ed25519", + attributeProfile: "htmltrust-attrs-v1", + canonicalizationProfile: "htmltrust-c14n-v1", + claimsHash, + contentHash, + context: "https://htmltrust.org/protocol/signed-section", + keyid, + location, + profile: "htmltrust-signature-v1", + scope: "url", + signedAt, + urlProfile: "htmltrust-safe-url-v1", + }; + const contentSignature = unpadded(cryptoSign( + null, + Buffer.from(canonicalize(signingObject), "utf8"), + signingKey.privateKey, + )); + const submission = { + profile: "htmltrust-signature-v1", + contentHash, + keyid, + algorithm: "ed25519", + signedAt, + scope: "url", + location, + signature: contentSignature, + sourceURL: `${location}#results`, + claims, + }; + + const submitted = await signedPost({ + path: "/content", + document: submission, + keyid, + privateKey: signingKey.privateKey, + nonce: "content-valid", + }); + if (submitted.response.headers.get("location") !== `/content/${encodeURIComponent(contentHash)}`) { + fail("POST /content returned the wrong Location header", submitted.response.headers.get("location")); + } + const signer = submitted.body.signers?.[0]; + if ( + signer?.profile !== "htmltrust-signature-v1" || + signer?.keyid !== keyid || + signer?.location !== location || + signer?.scope !== "url" + ) { + fail("POST /content returned an incomplete v1 signer record", submitted.body); + } + + await requestJson(`${target}/content/${encodeURIComponent(contentHash)}`); + + const badLocation = { ...submission, location: "https://example.com/research/other" }; + const rejectedLocation = await signedPost({ + path: "/content", + document: badLocation, + keyid, + privateKey: signingKey.privateKey, + nonce: "content-bad-location", + expectedStatus: 400, + }); + if (rejectedLocation.body.type !== "https://htmltrust.org/errors/content-submission-invalid") { + fail("POST /content did not reject a mismatched location with problem details", rejectedLocation.body); + } + + const apiKeyOnly = await requestJson(`${target}/content`, { + method: "POST", + headers: { + "content-type": "application/json", + "x-api-key": generalApiKey, + }, + body: JSON.stringify(submission), + }, 401); + if (!apiKeyOnly.response.headers.get("www-authenticate")) { + fail("canonical POST /content accepted API-key fallback or omitted its signature challenge"); + } + + const unsignedEndorsement = { + endorser: keyid, + endorsement: contentHash, + algorithm: "ed25519", + timestamp: "2026-05-10T09:00:00Z", + claim: "Reviewed against the published source.", + }; + const endorsement = { + ...unsignedEndorsement, + signature: unpadded(cryptoSign( + null, + Buffer.from(canonicalize(unsignedEndorsement), "utf8"), + signingKey.privateKey, + )), + }; + await signedPost({ + path: "/endorsements", + document: endorsement, + keyid, + privateKey: signingKey.privateKey, + nonce: "endorsement-valid", + }); + await signedPost({ + path: "/endorsements", + document: endorsement, + keyid, + privateKey: signingKey.privateKey, + nonce: "endorsement-idempotent-repeat", + }); + const endorsements = await requestJson( + `${target}/content/${encodeURIComponent(contentHash)}/endorsements`, + ); + if (!Array.isArray(endorsements.body) || endorsements.body.length !== 1) { + fail("root content endorsement listing did not return the stored document", endorsements.body); + } + + console.log("HTMLTrust v1 directory smoke: 12 checks passed"); +}; + +main().catch((error) => { + console.error(error.stack || error.message); + process.exit(1); +}); diff --git a/conformance/runner/with-reference-server.mjs b/conformance/runner/with-reference-server.mjs index 451fab4..873c080 100644 --- a/conformance/runner/with-reference-server.mjs +++ b/conformance/runner/with-reference-server.mjs @@ -132,8 +132,23 @@ async function main() { runner.on("exit", (code) => resolveExit(code ?? 1)); }); + // Exercise the canonical root endpoints separately from the explicit + // pre-v1 `/api` compatibility fixtures. + const v1Runner = spawn( + process.execPath, + [ + resolve(SELF_DIR, "v1-smoke.mjs"), + `http://localhost:${SERVER_PORT}`, + GENERAL_API_KEY, + ], + { stdio: "inherit" }, + ); + const v1Exit = await new Promise((resolveExit) => { + v1Runner.on("exit", (code) => resolveExit(code ?? 1)); + }); + shuttingDown = true; - await shutdown(runnerExit); + await shutdown(runnerExit === 0 && v1Exit === 0 ? 0 : 1); } let shuttingDown = false; diff --git a/openapi.yaml b/openapi.yaml index 0461328..c0a9153 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -4,18 +4,22 @@ info: description: | Reference implementation of the HTMLTrust trust directory API. Provides endpoints for author management, content signing and verification, claims, and a federated - trust directory with reputation tracking. + trust directory with reputation tracking. The canonical v1 directory paths + use RFC 9421 authentication for writes, negotiate the HTMLTrust JSON media + types (with application/json as a compatibility representation), and use + standard Cache-Control, ETag, and Vary headers on public reads. version: 1.0.0 contact: name: HTMLTrust url: https://github.com/HTMLTrust/htmltrust-server-reference email: jason@jason-grey.com + license: + name: PolyForm Noncommercial License 1.0.0 + url: https://polyformproject.org/licenses/noncommercial/1.0.0/ servers: - - url: https://api.contentsigning.example.com/v1 - description: Production server - - url: https://api.staging.contentsigning.example.com/v1 - description: Staging server + - url: / + description: Same origin as the API document tags: - name: Authors @@ -43,9 +47,17 @@ security: components: securitySchemes: + HttpSignatureInput: + type: apiKey + in: header + name: Signature-Input + description: | + RFC 9421 signature metadata. Send this header together with the + `Signature` header represented by `HttpMessageSignature`. HttpMessageSignature: - type: http - scheme: signature + type: apiKey + in: header + name: Signature description: | RFC 9421 HTTP Message Signature, the scheme draft §9.8 REQUIRES for POST endpoints: "POST endpoints MUST be authenticated using HTTP @@ -55,7 +67,7 @@ components: The request carries `Signature-Input` and `Signature` header fields. The covered components MUST include the request target (`@method` together with `@target-uri`, or `@request-target`), `host`, `date`, - and — for any request with a body — `content-digest` (RFC 9530). The + and, for any request with a body, `content-digest` (RFC 9530). The `keyid` signature parameter is resolved per draft §8 (DID, HTTPS key document, or a directory `/keys/{id}` reference) and the signature is verified against that key, so the authenticated identity is the key @@ -64,7 +76,7 @@ components: Example: Signature-Input: sig1=("@method" "@target-uri" "host" "date" \ - "content-digest");created=1770000000;keyid="https://directory.example/api/keys/k-abc123" + "content-digest");created=1770000000;keyid="https://directory.example/keys/k-abc123";alg="ed25519" Signature: sig1=:MEUCIQD...: Failed verification returns 401 with a @@ -88,8 +100,8 @@ components: description: | Supplementary demo/admin scheme for submission endpoints. A shared secret proves nothing about who submitted a record, so it cannot carry - the submitter identity the protocol depends on; prefer - HttpMessageSignature. Refused when NODE_ENV=production unless + the submitter identity the protocol depends on; prefer the paired + HttpSignatureInput and HttpMessageSignature schemes. Refused when NODE_ENV=production unless HTMLTRUST_ALLOW_API_KEY_AUTH=1 is set. AdminApiKey: @@ -238,7 +250,7 @@ components: DirectoryDiscovery: type: object - required: [directory, version, capabilities, supportedAlgorithms] + required: [directory, version, capabilities, supportedAlgorithms, supportedProfiles] properties: directory: type: string @@ -262,10 +274,13 @@ components: hash: type: array items: { type: string } + supportedProfiles: + type: array + items: { type: string } KeyDocument: type: object - required: [algorithm, publicKey] + required: [algorithm, publicKey, publicKeyEncoding] properties: kid: type: string @@ -276,6 +291,7 @@ components: type: string description: Canonical unpadded standard Base64 SPKI DER public key bytes. publicKeyEncoding: + enum: [spki-der] type: string description: Reference-server extension describing how publicKey was derived from the stored PEM. publicKeyPem: @@ -301,22 +317,67 @@ components: type: array items: type: object - required: [keyid, signedAt, domain, signature] + required: [profile, keyid, algorithm, signedAt, scope, location, signature] properties: + profile: + type: string keyid: type: string + algorithm: + type: string signedAt: type: string format: date-time - domain: + scope: + type: string + enum: [url, origin] + location: type: string - description: Serialized Web origin, not a bare hostname. + format: uri signature: type: string description: Canonical unpadded standard Base64 signature. endorsementCount: type: integer + ContentSubmission: + type: object + required: [profile, contentHash, keyid, algorithm, signedAt, scope, location, signature, sourceURL, claims] + properties: + profile: + type: string + enum: [htmltrust-signature-v1] + contentHash: + type: string + keyid: + type: string + algorithm: + type: string + enum: [ed25519, ecdsa-p256, ecdsa-p384, rsa-pss-sha256, rsa-pkcs1-sha256] + signedAt: + type: string + format: date-time + scope: + type: string + enum: [url, origin] + location: + type: string + format: uri + signature: + type: string + description: Canonical unpadded standard Base64 signature. + sourceURL: + type: string + format: uri + claims: + type: array + items: + type: object + required: [name, content] + properties: + name: { type: string } + content: { type: string } + SignerReputation: type: object required: [keyid, score, asOf, components] @@ -399,7 +460,7 @@ components: description: UTC RFC3339 timestamp from the direct child signed-at claim domain: type: string - description: Serialized Web origin associated with the content, not a bare hostname + description: Serialized Web origin associated with the content; bare hostnames are invalid authorId: type: string format: uuid @@ -427,7 +488,7 @@ components: signedAt: "2026-05-01T10:30:00Z" domain: "https://example.com" authorId: "123e4567-e89b-12d3-a456-426614174000" - keyid: "https://directory.example/api/keys/123e4567-e89b-12d3-a456-426614174001" + keyid: "https://directory.example/keys/123e4567-e89b-12d3-a456-426614174001" algorithm: "ed25519" signature: "MEUCIQD7y5SxmQJ9f0lE9B0BwqIJKKdL5fZMNQOiPnKWUJfmrgIgEbHtPwDxM9xGbCZzW9k2R9jFxwJZQQlPfhgj+0YP7vQ=" claims: @@ -514,8 +575,8 @@ components: The directory verifies the endorser's signature over JCS(document minus `signature`) before storing it (§9.7), and serves - the document back byte-for-byte as submitted. It adds no members of - its own: §10.1 requires unrecognised members to be preserved and + every submitted member without adding fields of its own. Section 10.1 + requires unrecognised members to be preserved and included in the signed payload, so an injected `_id`, `createdAt`, or `contentHash` alias would invalidate the endorser's signature for anyone who recomputes it. The server-assigned identifier is returned @@ -544,7 +605,6 @@ components: algorithm: type: string description: Signature algorithm identifier from draft §7.1 - default: ed25519 claim: type: string description: Free-text human-readable rationale for the endorsement @@ -578,6 +638,13 @@ paths: responses: "200": description: Directory discovery document + headers: + Cache-Control: + schema: { type: string } + ETag: + schema: { type: string } + Vary: + schema: { type: string } content: application/htmltrust-directory+json: schema: @@ -585,6 +652,12 @@ paths: application/json: schema: $ref: "#/components/schemas/DirectoryDiscovery" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /keys/{id}: get: @@ -601,6 +674,13 @@ paths: responses: "200": description: Key document + headers: + Cache-Control: + schema: { type: string } + ETag: + schema: { type: string } + Vary: + schema: { type: string } content: application/htmltrust-key+json: schema: @@ -613,7 +693,25 @@ paths: content: application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "400": + description: Invalid key identifier + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Directory could not read the key document + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /signers/{id}/reputation: get: @@ -631,6 +729,13 @@ paths: responses: "200": description: Signer reputation + headers: + Cache-Control: + schema: { type: string } + ETag: + schema: { type: string } + Vary: + schema: { type: string } content: application/json: schema: @@ -640,7 +745,25 @@ paths: content: application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "400": + description: Invalid signer identifier + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Directory could not read signer reputation + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /content: post: @@ -648,54 +771,32 @@ paths: - Content summary: Submit a signed content occurrence for indexing description: | - Draft directory submission endpoint. The directory re-verifies the - submitted signature over `contentHash:claimsHash:domain:signedAt`. - `domain` is a serialized Web origin. `claimsHash` may be supplied by - clients that already computed the canonical direct-child meta claims - hash; otherwise the directory derives it from the submitted `claims` - array using the draft §4.6 canonical claims serialization (normalized - name/content pairs, sorted by the UTF-8 bytes of the normalized name). + Canonical HTMLTrust v1 directory submission endpoint. The directory + recomputes the claims hash from the complete `claims` array and + re-verifies the v1 JCS signing payload. `location` must be derived + from `sourceURL` according to `scope`; callers cannot submit a + precomputed `claimsHash`. operationId: submitContent security: - - HttpMessageSignature: [] - - GeneralApiKey: [] + - HttpSignatureInput: [] + HttpMessageSignature: [] requestBody: required: true content: application/json: schema: - type: object - required: [contentHash, keyid, signedAt, domain, signature, sourceURL] - properties: - contentHash: - type: string - claimsHash: - type: string - keyid: - type: string - signedAt: - type: string - format: date-time - domain: - type: string - description: Serialized Web origin, not a bare hostname. - signature: - type: string - description: Canonical unpadded standard Base64 signature. - sourceURL: - type: string - format: uri - claims: - type: array - items: - type: object - required: [name, content] - properties: - name: { type: string } - content: { type: string } + $ref: "#/components/schemas/ContentSubmission" + application/htmltrust-content+json: + schema: + $ref: "#/components/schemas/ContentSubmission" responses: "201": description: Content record created + headers: + Cache-Control: + schema: { type: string } + Location: + schema: { type: string } content: application/htmltrust-content+json: schema: @@ -708,7 +809,31 @@ paths: content: application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "401": + description: Missing or invalid HTTP Message Signature + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "415": + description: Request body media type is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Verified content could not be stored + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /content/{hash}: get: @@ -726,6 +851,13 @@ paths: responses: "200": description: Content record + headers: + Cache-Control: + schema: { type: string } + ETag: + schema: { type: string } + Vary: + schema: { type: string } content: application/htmltrust-content+json: schema: @@ -738,7 +870,25 @@ paths: content: application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "400": + description: Invalid content hash + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Directory could not read the content record + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /content/{hash}/endorsements: get: @@ -756,6 +906,13 @@ paths: responses: "200": description: Structured endorsement documents + headers: + Cache-Control: + schema: { type: string } + ETag: + schema: { type: string } + Vary: + schema: { type: string } content: application/htmltrust-endorsement+json: schema: @@ -767,6 +924,24 @@ paths: type: array items: $ref: "#/components/schemas/Endorsement" + "400": + description: Invalid content hash + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Directory could not read endorsements + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /authors: post: @@ -835,9 +1010,9 @@ paths: "400": description: Invalid input content: - application/json: + application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" "401": description: Unauthorized content: @@ -1103,7 +1278,7 @@ paths: description: Hash of all direct child meta claims using canonical unpadded standard Base64 domain: type: string - description: Serialized Web origin associated with the content, not a bare hostname + description: Serialized Web origin associated with the content; bare hostnames are invalid signedAt: type: string format: date-time @@ -1167,7 +1342,7 @@ paths: description: Hash of all direct child meta claims domain: type: string - description: Serialized Web origin associated with the content, not a bare hostname + description: Serialized Web origin associated with the content; bare hostnames are invalid signedAt: type: string format: date-time @@ -1297,6 +1472,12 @@ paths: type: integer limit: type: integer + "429": + description: Request rate limit exceeded + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /claims/{claimId}: get: @@ -1403,6 +1584,12 @@ paths: type: integer limit: type: integer + "429": + description: Request rate limit exceeded + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /directory/keys/{keyId}/reputation: get: @@ -1577,6 +1764,12 @@ paths: type: integer limit: type: integer + "429": + description: Request rate limit exceeded + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /directory/content/{contentHash}/occurrences: get: @@ -1736,7 +1929,19 @@ paths: responses: "200": description: Array of endorsements + headers: + Cache-Control: + schema: { type: string } + ETag: + schema: { type: string } + Vary: + schema: { type: string } content: + application/htmltrust-endorsement+json: + schema: + type: array + items: + $ref: "#/components/schemas/Endorsement" application/json: schema: type: array @@ -1745,9 +1950,21 @@ paths: "400": description: Invalid input content: - application/json: + application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Directory could not read endorsements + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" post: tags: - Endorsements @@ -1755,70 +1972,67 @@ paths: description: | Stores a structured endorsement document. The signed payload is the JSON canonicalization of the endorsement document with `signature` - omitted. The legacy `contentHash` field is accepted as an alias for - the draft `endorsement` field. + omitted. The canonical endpoint requires RFC 9421 authentication; + the `/api/endorsements` compatibility route also accepts the legacy + API-key scheme. operationId: createEndorsement security: - - HttpMessageSignature: [] - - GeneralApiKey: [] + - HttpSignatureInput: [] + HttpMessageSignature: [] requestBody: required: true content: application/json: schema: - type: object - required: - - endorser - - endorsement - - algorithm - - signature - - timestamp - properties: - endorser: - type: string - description: Opaque endorser keyid (e.g. "did:web:publisher.org") - endorsement: - type: string - description: Draft field for the targeted content hash (e.g. "sha256:...") - contentHash: - type: string - description: Legacy alias for endorsement - signature: - type: string - description: Canonical unpadded standard Base64 signature over JCS(document with signature omitted) - timestamp: - type: string - description: ISO-8601 timestamp at which the endorsement was issued - algorithm: - type: string - description: Signature algorithm (default ed25519) - default: ed25519 - rawBlob: - type: string - description: | - The exact bytes the client signed over. Clients SHOULD - supply this so verifiers can re-verify byte-identically. - If omitted, the server constructs a canonical blob in a - stable key order. + $ref: "#/components/schemas/Endorsement" + application/htmltrust-endorsement+json: + schema: + $ref: "#/components/schemas/Endorsement" responses: "201": description: Endorsement stored + headers: + Cache-Control: + schema: { type: string } + Location: + schema: { type: string } content: + application/htmltrust-endorsement+json: + schema: + $ref: "#/components/schemas/Endorsement" application/json: schema: $ref: "#/components/schemas/Endorsement" "400": description: Invalid input content: - application/json: + application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" "401": description: Unauthorized content: - application/json: + application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "406": + description: Requested representation is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "415": + description: Request body media type is not supported + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" + "500": + description: Verified endorsement could not be stored + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" /endorsements/{id}: delete: @@ -1826,13 +2040,12 @@ paths: - Endorsements summary: Delete an endorsement description: | - Removes an endorsement from the directory. MVP: gated behind the - general API key only. Production deployments MUST additionally - require that the caller's authenticated identity match the - endorsement's `endorser` keyid. + Removes an endorsement from the directory. The caller must authenticate + with the endorsement's key or with the directory administrator key. operationId: deleteEndorsement security: - - HttpMessageSignature: [] + - HttpSignatureInput: [] + HttpMessageSignature: [] - AdminApiKey: [] parameters: - name: id @@ -1843,15 +2056,33 @@ paths: responses: "204": description: Endorsement deleted + "400": + description: Invalid endorsement identifier or request + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" "401": description: Unauthorized content: - application/json: + application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "403": + description: The authenticated key is not the endorser + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" "404": description: Endorsement not found content: - application/json: + application/problem+json: schema: - $ref: "#/components/schemas/Error" + $ref: "#/components/schemas/Problem" + "500": + description: Directory could not delete the endorsement + content: + application/problem+json: + schema: + $ref: "#/components/schemas/Problem" diff --git a/package-lock.json b/package-lock.json index bc6931b..2e39ee0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,7 @@ "version": "0.1.0", "license": "LicenseRef-PolyForm-Noncommercial-1.0.0", "dependencies": { - "@htmltrust/canonicalization": "https://github.com/HTMLTrust/htmltrust-canonicalization/archive/refs/tags/v0.2.2.tar.gz", + "@htmltrust/canonicalization": "https://github.com/HTMLTrust/htmltrust-canonicalization/archive/37359dd8a872f8d09fa0e7f7dd75567d92e5bec4.tar.gz", "cors": "^2.8.5", "dotenv": "^16.5.0", "express": "^5.1.0", @@ -19,15 +19,22 @@ "mongoose": "^8.14.1" }, "devDependencies": { + "@redocly/cli": "2.49.0", "mongodb-memory-server": "^11.1.0", "nodemon": "^3.1.10" } }, "node_modules/@htmltrust/canonicalization": { - "version": "0.2.2", - "resolved": "https://github.com/HTMLTrust/htmltrust-canonicalization/archive/refs/tags/v0.2.2.tar.gz", - "integrity": "sha512-qKOx4PipywaLx3R/Bc6S+IWSZZhD/DQ4LQaPC7bHREq40iRIjeVkkSSrHFp2cTyqdPvsPZF2ufvT5YUz9II8RA==", - "license": "LicenseRef-PolyForm-Noncommercial-1.0.0" + "version": "0.3.0", + "resolved": "https://github.com/HTMLTrust/htmltrust-canonicalization/archive/37359dd8a872f8d09fa0e7f7dd75567d92e5bec4.tar.gz", + "integrity": "sha512-IXCQNBj3M5CtOsk8zNncN9LhmDPrOUul3O13hqmgBbGdUDRZ9V2JOigdklAFtxTy3QAoE4SuODW/48svVZwwow==", + "license": "LicenseRef-PolyForm-Noncommercial-1.0.0", + "dependencies": { + "parse5": "7.3.0" + }, + "engines": { + "node": ">=22" + } }, "node_modules/@mongodb-js/saslprep": { "version": "1.4.6", @@ -38,6 +45,21 @@ "sparse-bitfield": "^3.0.3" } }, + "node_modules/@redocly/cli": { + "version": "2.49.0", + "resolved": "https://registry.npmjs.org/@redocly/cli/-/cli-2.49.0.tgz", + "integrity": "sha512-87UuARXYrPIiHhd/UriamL4BApuPgB0E91coBtoPG6PK4giTwiuo3dI4RoyMGhsC7TwgAnL8PJuhCwGymgwbVA==", + "dev": true, + "license": "MIT", + "bin": { + "openapi": "bin/cli.js", + "redocly": "bin/cli.js" + }, + "engines": { + "node": ">=22.12.0 || >=20.19.0 <21.0.0", + "npm": ">=10" + } + }, "node_modules/@types/webidl-conversions": { "version": "7.0.3", "resolved": "https://registry.npmjs.org/@types/webidl-conversions/-/webidl-conversions-7.0.3.tgz", @@ -524,6 +546,18 @@ "node": ">= 0.8" } }, + "node_modules/entities": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz", + "integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==", + "license": "BSD-2-Clause", + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, "node_modules/es-define-property": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz", @@ -1529,6 +1563,18 @@ "node": ">=6" } }, + "node_modules/parse5": { + "version": "7.3.0", + "resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz", + "integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==", + "license": "MIT", + "dependencies": { + "entities": "^6.0.0" + }, + "funding": { + "url": "https://github.com/inikulin/parse5?sponsor=1" + } + }, "node_modules/parseurl": { "version": "1.3.3", "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz", diff --git a/package.json b/package.json index 7ee96d2..379d9fc 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,9 @@ "test": "node --test \"test/**/*.test.js\"", "conformance": "node conformance/runner/with-reference-server.mjs", "conformance:docker": "bash conformance/run-conformance.sh", - "conformance:runner": "node conformance/runner/run.mjs" + "conformance:runner": "node conformance/runner/run.mjs", + "openapi:lint": "redocly lint openapi.yaml --skip-rule no-unused-components", + "migrate:v1": "node scripts/migrate-v1-indexes.js" }, "keywords": [ "htmltrust", @@ -20,7 +22,7 @@ "author": "Jason Grey ", "license": "LicenseRef-PolyForm-Noncommercial-1.0.0", "dependencies": { - "@htmltrust/canonicalization": "https://github.com/HTMLTrust/htmltrust-canonicalization/archive/refs/tags/v0.2.2.tar.gz", + "@htmltrust/canonicalization": "https://github.com/HTMLTrust/htmltrust-canonicalization/archive/37359dd8a872f8d09fa0e7f7dd75567d92e5bec4.tar.gz", "cors": "^2.8.5", "dotenv": "^16.5.0", "express": "^5.1.0", @@ -30,6 +32,7 @@ "mongoose": "^8.14.1" }, "devDependencies": { + "@redocly/cli": "2.49.0", "mongodb-memory-server": "^11.1.0", "nodemon": "^3.1.10" } diff --git a/scripts/migrate-v1-indexes.js b/scripts/migrate-v1-indexes.js new file mode 100644 index 0000000..92bd93e --- /dev/null +++ b/scripts/migrate-v1-indexes.js @@ -0,0 +1,69 @@ +#!/usr/bin/env node + +/** + * Upgrade indexes created by the pre-v1 reference server. + * + * MongoDB cannot replace an index with the same key pattern while its options + * differ. The v1 ContentSignature identity is a partial unique index, so an + * old database must drop the pre-v1 unique index before Mongoose can create + * the v1 definition. Endorsement indexes were unique in an earlier release; + * they are intentionally non-unique now so a revocation can coexist with the + * endorsement it revokes. + * + * Run once with the same MONGO_URI used by the server: + * npm run migrate:v1 + */ +const mongoose = require('mongoose'); +const ContentSignature = require('../src/models/ContentSignature'); +const Endorsement = require('../src/models/Endorsement'); + +const LEGACY_INDEXES = [ + [ContentSignature, 'contentHash_1_domain_1_authorId_1'], + [Endorsement, 'contentHash_1_endorser_1'], + [Endorsement, 'endorsement_1_endorser_1'], +]; + +const dropIfPresent = async (model, name) => { + let indexes; + try { + indexes = await model.collection.indexes(); + } catch (error) { + // A fresh deployment has no collection yet. createIndexes below will + // create it with the current schema, so there is nothing to migrate. + if (error.code === 26 || error.codeName === 'NamespaceNotFound') return false; + throw error; + } + if (!indexes.some((index) => index.name === name)) return false; + await model.collection.dropIndex(name); + console.log(`dropped ${model.collection.name}.${name}`); + return true; +}; + +const migrate = async () => { + const mongoUri = process.env.MONGO_URI || 'mongodb://localhost:27017/content-signing'; + // Do not let Mongoose auto-create the replacement index before the legacy + // one is dropped. MongoDB rejects two indexes with the same key pattern but + // different options, which is the reason this migration exists. + await mongoose.connect(mongoUri, { autoIndex: false }); + try { + for (const [model, indexName] of LEGACY_INDEXES) { + await dropIfPresent(model, indexName); + } + // Recreate the current partial/non-unique definitions without touching + // unrelated indexes owned by an operator or another application. + await ContentSignature.createIndexes(); + await Endorsement.createIndexes(); + console.log('v1 index migration complete'); + } finally { + await mongoose.disconnect(); + } +}; + +if (require.main === module) { + migrate().catch((error) => { + console.error(`v1 index migration failed: ${error.message}`); + process.exitCode = 1; + }); +} + +module.exports = { LEGACY_INDEXES, dropIfPresent, migrate }; diff --git a/scripts/test-in-docker.sh b/scripts/test-in-docker.sh new file mode 100755 index 0000000..7e527d1 --- /dev/null +++ b/scripts/test-in-docker.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# Run the unit and conformance suites inside a disposable Node container. +# The repository is mounted read-only. The test copy and every dependency +# install live in the container, so a test run leaves the checkout unchanged. + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +IMAGE="${HTMLTRUST_TEST_IMAGE:-node:22-bookworm}" +CHECKOUT_ID="$(printf '%s' "$REPO_ROOT" | cksum | awk '{print $1}')" +CONTAINER_NAME="htmltrust-server-${CHECKOUT_ID}-$$" +CACHE_PREFIX="htmltrust-server-${CHECKOUT_ID}" + +echo "Running HTMLTrust tests in ${IMAGE}" +exec docker run --rm --init --interactive \ + --name "${CONTAINER_NAME}" \ + --volume "${REPO_ROOT}:/repo:ro" \ + --volume "${CACHE_PREFIX}-npm:/root/.npm" \ + --volume "${CACHE_PREFIX}-mongodb:/var/cache/mongodb" \ + --env MONGOMS_DOWNLOAD_DIR=/var/cache/mongodb \ + "${IMAGE}" \ + bash -s <<'CONTAINER_SCRIPT' +set -euo pipefail + +mkdir -p /workspace /var/cache/mongodb +# Keep all npm output and generated files in the disposable container. The +# source mount is read-only, which also catches accidental checkout writes. +tar --exclude=.git --exclude=node_modules -cf - -C /repo . | tar -xf - -C /workspace +cd /workspace + +npm ci --ignore-scripts --no-audit --no-fund +npm --prefix conformance/runner ci --ignore-scripts --no-audit --no-fund + +echo "== npm test ==" +npm test + +echo "== npm run openapi:lint ==" +npm run openapi:lint + +echo "== npm run conformance ==" +npm run conformance +CONTAINER_SCRIPT diff --git a/src/controllers/contentController.js b/src/controllers/contentController.js index fdb1af2..44efe22 100644 --- a/src/controllers/contentController.js +++ b/src/controllers/contentController.js @@ -19,6 +19,8 @@ const { } = require('../utils/htmltrustProtocol'); const { canonicalizeClaims } = require('../utils/claims'); const { directoryKeyUrl } = require('../utils/directoryUrl'); +const { validateV1ContentSubmission } = require('../utils/signingProfile'); +const { negotiatedType } = require('../middleware/contentNegotiation'); /** * Build the canonical binding string that is actually signed. @@ -113,18 +115,25 @@ const resolveSubmissionKey = async (req, keyid) => { return resolution; }; -const contentRecord = async (req, contentHash) => { - const signatures = await ContentSignature.find({ contentHash }).sort({ createdAt: 1 }); +const contentRecord = async (req, contentHash, { v1Only = false } = {}) => { + const query = { contentHash }; + if (v1Only) query.profile = 'htmltrust-signature-v1'; + const signatures = await ContentSignature.find(query).sort({ createdAt: 1 }); if (signatures.length === 0) return null; const signers = await Promise.all(signatures.map(async (signature) => { const key = await Key.findById(signature.keyId); - return { - keyid: key ? keyidFor(req, key) : String(signature.keyId), + const record = { + profile: signature.profile || 'legacy-colon-binding', + keyid: signature.keyid || (key ? keyidFor(req, key) : String(signature.keyId)), + algorithm: signature.algorithm || (key ? normalizeAlgorithm(key.algorithm) : undefined), signedAt: signature.signedAt, - domain: signature.domain, + scope: signature.scope, + location: signature.location, signature: signature.signature, }; + if (!signature.profile) record.domain = signature.domain; + return Object.fromEntries(Object.entries(record).filter(([, value]) => value !== undefined)); })); const endorsementCount = await Endorsement.countDocuments({ @@ -454,18 +463,52 @@ exports.registerOccurrence = async (req, res) => { * @access Public */ exports.getContentRecord = async (req, res) => { + let contentHash; + try { + contentHash = assertContentHash(req.params.contentHash, 'contentHash'); + } catch (error) { + return problem(res, 400, 'Invalid content hash', detailFor(error)); + } + try { - const contentHash = assertContentHash(req.params.contentHash, 'contentHash'); const record = await contentRecord(req, contentHash); if (!record) { return problem(res, 404, 'Content not found', 'No content record exists for the requested hash', { contentHash, }); } - res.type('application/htmltrust-content+json').status(200).json(record); + res.type(negotiatedType(req, 'application/htmltrust-content+json')).status(200).json(record); + } catch (error) { + console.error('Get content record error:', error); + return problem(res, 500, 'Directory read failure', 'The directory could not read the content record', { + type: 'https://htmltrust.org/errors/storage-failure', + }); + } +}; + +/** Canonical v1 GET /content/:contentHash. */ +exports.getContentRecordV1 = async (req, res) => { + let contentHash; + try { + contentHash = assertContentHash(req.params.contentHash, 'contentHash'); } catch (error) { return problem(res, 400, 'Invalid content hash', detailFor(error)); } + + try { + const record = await contentRecord(req, contentHash, { v1Only: true }); + if (!record) { + return problem(res, 404, 'Content not found', 'No v1 content record exists for the requested hash', { + contentHash, + }); + } + return res.type(negotiatedType(req, 'application/htmltrust-content+json')).status(200).json(record); + } catch (error) { + console.error('Get v1 content record error:', error); + return problem(res, 500, 'Directory read failure', 'The directory could not read the content record', { + type: 'https://htmltrust.org/errors/storage-failure', + }); + } }; /** @@ -558,25 +601,161 @@ exports.submitContent = async (req, res) => { res .status(201) .location(`/api/content/${encodeURIComponent(contentHash)}`) - .type('application/htmltrust-content+json') + .type(negotiatedType(req, 'application/htmltrust-content+json')) .json(record); } catch (error) { return problem(res, 400, 'Invalid content submission', detailFor(error)); } }; +/** + * Submit and re-verify an htmltrust-signature-v1 occurrence. + * + * This is the canonical POST /content handler. The `/api/content` handler + * above remains the explicit pre-v1 compatibility endpoint. + */ +exports.submitContentV1 = async (req, res) => { + let submission; + try { + submission = await validateV1ContentSubmission(req.body); + } catch (error) { + return problem(res, 400, 'Invalid content submission', detailFor(error), { + type: 'https://htmltrust.org/errors/content-submission-invalid', + }); + } + + try { + const resolution = await resolveUsableKey(submission.keyid, { req, algorithm: submission.algorithm }); + if (!resolution.ok) { + return problem( + res, + 400, + 'Key resolution failed', + `The submitted keyid could not be resolved to a usable key (${resolution.reason})`, + { + type: `https://htmltrust.org/errors/${resolution.reason}`, + keyid: submission.keyid, + }, + ); + } + if (resolution.resolved.algorithm !== submission.algorithm) { + return problem( + res, + 400, + 'Algorithm mismatch', + `The submission declares ${submission.algorithm} but the resolved key uses ${resolution.resolved.algorithm}`, + { type: 'https://htmltrust.org/errors/algorithm-mismatch' }, + ); + } + if (!verifySignature( + submission.payload, + submission.signature, + resolution.resolved.publicKeyPem, + submission.algorithm, + )) { + return problem( + res, + 400, + 'Signature verification failed', + 'The submitted signature did not verify against the canonical signing payload.', + { + type: 'https://htmltrust.org/errors/signature-invalid', + contentHash: submission.contentHash, + }, + ); + } + + const localKey = resolution.resolved.key; + const identity = { + contentHash: submission.contentHash, + profile: submission.profile, + location: submission.location, + keyid: submission.keyid, + }; + let contentSignature = await ContentSignature.findOne(identity); + const storedClaims = Object.fromEntries( + submission.claims.map(({ name, content }) => [name, content]), + ); + if (contentSignature) { + contentSignature.algorithm = submission.algorithm; + contentSignature.claimsHash = submission.claimsHash; + contentSignature.signedAt = submission.signedAt; + contentSignature.scope = submission.scope; + contentSignature.sourceURL = submission.sourceURL; + contentSignature.signature = submission.signature; + contentSignature.claims = storedClaims; + contentSignature.occurrences += 1; + if (localKey) { + contentSignature.authorId = localKey.authorId; + contentSignature.keyId = localKey._id; + } + await contentSignature.save(); + } else { + contentSignature = await ContentSignature.create({ + ...identity, + algorithm: submission.algorithm, + claimsHash: submission.claimsHash, + signedAt: submission.signedAt, + scope: submission.scope, + sourceURL: submission.sourceURL, + signature: submission.signature, + claims: storedClaims, + authorId: localKey && localKey.authorId, + keyId: localKey && localKey._id, + }); + } + + const sourceOrigin = new URL(submission.sourceURL).origin; + await ContentOccurrence.findOneAndUpdate( + { signatureId: contentSignature._id, url: submission.sourceURL }, + { + signatureId: contentSignature._id, + url: submission.sourceURL, + domain: sourceOrigin, + signatureValid: true, + lastSeen: Date.now(), + }, + { upsert: true, setDefaultsOnInsert: true }, + ); + + const record = await contentRecord(req, submission.contentHash, { v1Only: true }); + return res + .status(201) + .location(`/content/${encodeURIComponent(submission.contentHash)}`) + .type(negotiatedType(req, 'application/htmltrust-content+json')) + .json(record); + } catch (error) { + console.error('Submit v1 content error:', error); + return problem(res, 500, 'Content storage failed', detailFor( + error, + 'The directory could not store the verified content record', + ), { + type: 'https://htmltrust.org/errors/content-storage-failed', + }); + } +}; + exports.listContentEndorsements = async (req, res) => { + let contentHash; + try { + contentHash = assertContentHash(req.params.contentHash, 'contentHash'); + } catch (error) { + return problem(res, 400, 'Invalid content hash', detailFor(error)); + } + try { - const contentHash = assertContentHash(req.params.contentHash, 'contentHash'); const endorsements = await Endorsement.find({ $or: [{ endorsement: contentHash }, { contentHash }], }).sort({ createdAt: -1 }); const { toEndorsementDocument } = require('./endorsementController'); res - .type('application/htmltrust-endorsement+json') + .type(negotiatedType(req, 'application/htmltrust-endorsement+json')) .status(200) .json(endorsements.map(toEndorsementDocument)); } catch (error) { - return problem(res, 400, 'Invalid content hash', detailFor(error)); + console.error('List content endorsements error:', error); + return problem(res, 500, 'Directory read failure', 'The directory could not read endorsements', { + type: 'https://htmltrust.org/errors/storage-failure', + }); } }; diff --git a/src/controllers/directoryController.js b/src/controllers/directoryController.js index ff87347..f4e671c 100644 --- a/src/controllers/directoryController.js +++ b/src/controllers/directoryController.js @@ -9,6 +9,8 @@ const { problem, safeSearchRegex, } = require('../utils/htmltrustProtocol'); +const { directoryBaseUrl, directoryKeyUrl } = require('../utils/directoryUrl'); +const { negotiatedType } = require('../middleware/contentNegotiation'); /** * Clamp caller-supplied pagination. An unbounded `limit` turns a public read @@ -26,7 +28,7 @@ const boundedPage = (value) => { return parsed; }; -const baseDirectoryUrl = (req) => `${req.protocol}://${req.get('host')}/api/`; +const baseDirectoryUrl = (req) => `${directoryBaseUrl(req)}/`; const keyIdFromSignerId = (id) => { if (!id || typeof id !== 'string') return null; @@ -44,7 +46,7 @@ const keyIdFromSignerId = (id) => { exports.discovery = async (req, res) => { res - .type('application/htmltrust-directory+json') + .type(negotiatedType(req, 'application/htmltrust-directory+json')) .status(200) .json({ directory: baseDirectoryUrl(req), @@ -56,30 +58,50 @@ exports.discovery = async (req, res) => { reputation: true }, supportedAlgorithms: { - signature: ['ed25519', 'rsa-pkcs1-sha256', 'ecdsa-p256'], - hash: ['sha256'] - } + signature: [ + 'ed25519', + 'ecdsa-p256', + 'ecdsa-p384', + 'rsa-pss-sha256', + 'rsa-pkcs1-sha256' + ], + hash: ['sha256', 'sha384', 'sha512'] + }, + supportedProfiles: ['htmltrust-signature-v1'] }); }; exports.getKeyDocument = async (req, res) => { + if (!/^[0-9a-fA-F]{24}$/.test(req.params.id)) { + return problem(res, 400, 'Invalid key id', 'The key id must be a 24-character hexadecimal identifier'); + } + try { const key = await Key.findById(req.params.id); if (!key) { return problem(res, 404, 'Key not found', 'No key document exists for the requested id'); } res - .type('application/htmltrust-key+json') + .type(negotiatedType(req, 'application/htmltrust-key+json')) .status(200) - .json(keyDocumentFor(key)); + .json(keyDocumentFor(key, directoryKeyUrl(req, key._id))); } catch (error) { - return problem(res, 400, 'Invalid key id', detailFor(error)); + console.error('Get key document error:', error); + return problem(res, 500, 'Directory read failure', 'The directory could not read the key document', { + type: 'https://htmltrust.org/errors/storage-failure', + }); } }; exports.getSignerReputation = async (req, res) => { + let signerId; + try { + signerId = decodeURIComponent(req.params.id); + } catch (error) { + return problem(res, 400, 'Invalid signer id', 'The signer id is not valid percent-encoding'); + } + try { - const signerId = decodeURIComponent(req.params.id); const keyId = keyIdFromSignerId(signerId); let key = null; if (keyId) { @@ -102,7 +124,10 @@ exports.getSignerReputation = async (req, res) => { methodology: `${baseDirectoryUrl(req)}methodology/reputation-v1` }); } catch (error) { - return problem(res, 400, 'Invalid signer id', detailFor(error)); + console.error('Get signer reputation error:', error); + return problem(res, 500, 'Directory read failure', 'The directory could not read signer reputation', { + type: 'https://htmltrust.org/errors/storage-failure', + }); } }; diff --git a/src/controllers/endorsementController.js b/src/controllers/endorsementController.js index 85b86cf..17ecb91 100644 --- a/src/controllers/endorsementController.js +++ b/src/controllers/endorsementController.js @@ -14,9 +14,10 @@ const { const { canonicalizeJcs } = require('../utils/jcs'); const { resolveUsableKey, sameKeyMaterial } = require('../utils/keyResolution'); const { hasAdminApiKey } = require('../middleware/auth'); +const { negotiatedType } = require('../middleware/contentNegotiation'); /** - * Return the stored endorsement document exactly as it was submitted. + * Return the stored endorsement with every submitted member intact. * * Draft §9.5: "The directory MUST NOT alter the endorsement payloads in a * manner that invalidates the endorser's signature", and §10.1 requires @@ -54,16 +55,15 @@ const toEndorsementDocument = (endorsement) => { * canonicalized, verified, and stored, so nothing may be injected into it: * every added member changes the signing payload. */ -const validateEndorsementDocument = (body) => { +const validateEndorsementDocument = (body, { stripLegacyRawBlob = false } = {}) => { if (!body || typeof body !== 'object' || Array.isArray(body)) { throw invalid('The endorsement document must be a JSON object'); } const document = { ...body }; - // `rawBlob` is a legacy compatibility field that was never part of the - // signed document; strip it before verification so old clients that still - // send it do not fail, and keep it out of storage. - delete document.rawBlob; + // `rawBlob` is a pre-v1 compatibility field that was never signed. The + // canonical root endpoint preserves it like every other extension member. + if (stripLegacyRawBlob) delete document.rawBlob; for (const field of ['endorser', 'endorsement', 'signature', 'timestamp']) { if (typeof document[field] !== 'string' || document[field].length === 0) { @@ -104,7 +104,7 @@ const documentHashFor = (document) => * @returns {Promise<{ok: true} | {ok: false, status: number, title: string, detail: string, type?: string}>} */ const verifyEndorsementSignature = async (document, req) => { - const resolution = await resolveUsableKey(document.endorser, { req }); + const resolution = await resolveUsableKey(document.endorser, { req, algorithm: document.algorithm }); if (!resolution.ok) { return { ok: false, @@ -161,15 +161,19 @@ const verifyEndorsementSignature = async (document, req) => { exports.createEndorsement = async (req, res) => { let document; try { - document = validateEndorsementDocument(req.body); + document = validateEndorsementDocument(req.body, { + stripLegacyRawBlob: req.baseUrl === '/api/endorsements', + }); } catch (error) { return problem(res, 400, 'Invalid endorsement', detailFor(error, 'The endorsement document is not valid'), { type: 'https://htmltrust.org/errors/endorsement-invalid', }); } + let verification; + let documentHash; try { - const verification = await verifyEndorsementSignature(document, req); + verification = await verifyEndorsementSignature(document, req); if (!verification.ok) { return problem(res, verification.status, verification.title, verification.detail, { type: verification.type, @@ -177,8 +181,14 @@ exports.createEndorsement = async (req, res) => { }); } - const documentHash = documentHashFor(document); + documentHash = documentHashFor(document); + } catch (error) { + return problem(res, 400, 'Invalid endorsement', detailFor(error, 'The endorsement could not be verified'), { + type: 'https://htmltrust.org/errors/endorsement-invalid', + }); + } + try { // Append-only with idempotent resubmission. A second, different document // from the same endorser for the same content hash (a revocation, an // updated claim) is stored alongside the first: draft §10.3 requires a @@ -202,15 +212,16 @@ exports.createEndorsement = async (req, res) => { created = true; } + const status = req.baseUrl === '/api/endorsements' && !created ? 200 : 201; return res - .status(created ? 201 : 200) - .location(`/api/endorsements/${stored._id}`) - .type('application/htmltrust-endorsement+json') + .status(status) + .location(`/endorsements/${stored._id}`) + .type(negotiatedType(req, 'application/htmltrust-endorsement+json')) .json(toEndorsementDocument(stored)); } catch (error) { console.error('Create endorsement error:', error); - return problem(res, 400, 'Invalid endorsement', detailFor(error, 'The endorsement could not be stored'), { - type: 'https://htmltrust.org/errors/endorsement-invalid', + return problem(res, 500, 'Directory storage failure', 'The directory could not store the endorsement', { + type: 'https://htmltrust.org/errors/storage-failure', }); } }; @@ -221,30 +232,39 @@ exports.createEndorsement = async (req, res) => { * @access Public */ exports.listEndorsements = async (req, res) => { - try { - // Accept both kebab-case (spec-style) and camelCase query parameters. - const contentHash = req.query['content-hash'] || req.query.contentHash || req.query.endorsement; + // Accept both kebab-case (spec-style) and camelCase query parameters. + const contentHash = req.query['content-hash'] || req.query.contentHash || req.query.endorsement; - if (!contentHash) { - return problem(res, 400, 'Invalid request', 'content-hash query parameter is required'); - } + if (!contentHash) { + return problem(res, 400, 'Invalid request', 'content-hash query parameter is required'); + } + + let endorsementHash; + try { + endorsementHash = assertContentHash(String(contentHash), 'content-hash'); + } catch (error) { + return problem(res, 400, 'Invalid content hash', detailFor(error, 'The content-hash parameter is not valid')); + } - const endorsementHash = assertContentHash(String(contentHash), 'content-hash'); + try { const endorsements = await Endorsement.find({ $or: [{ endorsement: endorsementHash }, { contentHash: endorsementHash }] }).sort({ createdAt: -1 }); return res - .type('application/htmltrust-endorsement+json') + .type(negotiatedType(req, 'application/htmltrust-endorsement+json')) .status(200) .json(endorsements.map(toEndorsementDocument)); } catch (error) { console.error('List endorsements error:', error); - return problem(res, 400, 'Invalid content hash', detailFor(error, 'The content-hash parameter is not valid')); + return problem(res, 500, 'Directory read failure', 'The directory could not read endorsements', { + type: 'https://htmltrust.org/errors/storage-failure', + }); } }; exports.toEndorsementDocument = toEndorsementDocument; +exports.validateEndorsementDocument = validateEndorsementDocument; /** * @desc Delete an endorsement @@ -262,6 +282,10 @@ exports.toEndorsementDocument = toEndorsementDocument; * able to delete anyone's endorsements is a censorship primitive. */ exports.deleteEndorsement = async (req, res) => { + if (!/^[0-9a-fA-F]{24}$/.test(req.params.id)) { + return problem(res, 400, 'Invalid endorsement id', 'The endorsement id must be a 24-character hexadecimal identifier'); + } + try { const endorsement = await Endorsement.findById(req.params.id); @@ -282,7 +306,10 @@ exports.deleteEndorsement = async (req, res) => { ); } - const endorserKey = await resolveUsableKey(endorsement.endorser, { req }); + const endorserKey = await resolveUsableKey(endorsement.endorser, { + req, + algorithm: endorsement.algorithm, + }); const sameKeyid = actor.keyid === endorsement.endorser; const sameMaterial = endorserKey.ok && sameKeyMaterial(actor.resolved.publicKeyPem, endorserKey.resolved.publicKeyPem); @@ -298,6 +325,8 @@ exports.deleteEndorsement = async (req, res) => { return res.status(204).send(); } catch (error) { console.error('Delete endorsement error:', error); - return problem(res, 400, 'Invalid request', detailFor(error, 'The endorsement could not be deleted')); + return problem(res, 500, 'Directory storage failure', 'The directory could not delete the endorsement', { + type: 'https://htmltrust.org/errors/storage-failure', + }); } }; diff --git a/src/middleware/contentNegotiation.js b/src/middleware/contentNegotiation.js new file mode 100644 index 0000000..1af7b4c --- /dev/null +++ b/src/middleware/contentNegotiation.js @@ -0,0 +1,56 @@ +const { problem } = require('../utils/htmltrustProtocol'); + +/** + * Apply the HTTP semantics shared by the canonical directory endpoints. + * + * The draft permits application/json as a compatibility representation, but + * a client that explicitly asks for another representation must receive 406 + * rather than silently getting JSON. Public reads are cacheable and Express + * supplies an ETag when the response is serialized; Cache-Control and Vary + * make those validators usable by shared caches. + */ +const negotiate = (responseType, { + cacheControl = 'public, max-age=60, must-revalidate', + requestBody = false, +} = {}) => (req, res, next) => { + res.vary('Accept'); + + const selectedType = req.accepts([responseType, 'application/json']); + if (!selectedType) { + return problem( + res, + 406, + 'Not acceptable', + `The endpoint can return ${responseType} or application/json`, + { + type: 'https://htmltrust.org/errors/not-acceptable', + accepted: [responseType, 'application/json'], + }, + ); + } + + req.htmltrustResponseType = selectedType; + + if (requestBody) { + const contentType = String(req.headers['content-type'] || '').split(';', 1)[0].trim().toLowerCase(); + if (contentType !== 'application/json' && !contentType.endsWith('+json')) { + return problem( + res, + 415, + 'Unsupported media type', + 'Canonical directory submissions require an application/json representation', + { + type: 'https://htmltrust.org/errors/unsupported-media-type', + accepted: ['application/json', 'application/*+json'], + }, + ); + } + } + + res.set('Cache-Control', cacheControl); + return next(); +}; + +const negotiatedType = (req, fallback) => req.htmltrustResponseType || fallback; + +module.exports = { negotiate, negotiatedType }; diff --git a/src/middleware/httpSignature.js b/src/middleware/httpSignature.js index dcbf97a..5bb089d 100644 --- a/src/middleware/httpSignature.js +++ b/src/middleware/httpSignature.js @@ -10,14 +10,10 @@ const { resolveUsableKey } = require("../utils/keyResolution"); * signature input MUST cover the `(request-target)`, `host`, `date`, and * `content-digest` components at a minimum." * - * This is a deliberately minimal verifier, not a general RFC 9421 library. - * It supports exactly what the draft requires: - * - * Covered components: `@method` plus `@target-uri` or `@path` (equivalently - * `@request-target`), `@authority` or `host`, `date`, and — whenever the - * request carries a body — `content-digest`. Requests covering a smaller - * set are rejected; covering more is allowed and the extra components are - * included in the signature base as normal. + * This is a deliberately small verifier, not a general RFC 9421 library. + * Strict mode implements the HTMLTrust v1 request profile. Compatibility + * mode keeps the broader component aliases accepted by the pre-v1 `/api` + * routes. * * Derived components with parameters (`@query-param`, `;req`, `;sf`, `;bs`) * are NOT supported and are rejected rather than silently ignored, because @@ -31,6 +27,8 @@ const { resolveUsableKey } = require("../utils/keyResolution"); // Accepted clock skew for the `created` parameter and the `date` header. const MAX_SKEW_SECONDS = 300; +const V1_COMPONENTS = ["@method", "@target-uri", "host", "date", "content-digest"]; +const IMF_FIXDATE = /^(Mon|Tue|Wed|Thu|Fri|Sat|Sun), \d{2} (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) \d{4} \d{2}:\d{2}:\d{2} GMT$/; // Signatures already seen inside the acceptance window, to stop a captured // request from being replayed verbatim. Bounded so it cannot grow without @@ -95,6 +93,52 @@ const splitLabel = (member) => { return [member.slice(0, eq).trim(), member.slice(eq + 1).trim()]; }; +const splitParameters = (tail) => { + if (tail.length > 0 && !tail.startsWith(";")) { + throw new SignatureError("signature parameters must follow the inner list"); + } + const parts = []; + let inQuotes = false; + let start = 1; + for (let index = 1; index < tail.length; index += 1) { + const ch = tail[index]; + if (inQuotes && ch === "\\") { + index += 1; + } else if (ch === '"') { + inQuotes = !inQuotes; + } else if (ch === ";" && !inQuotes) { + parts.push(tail.slice(start, index)); + start = index + 1; + } + } + if (inQuotes) throw new SignatureError("unterminated string signature parameter"); + if (tail.length > 0) parts.push(tail.slice(start)); + return parts; +}; + +const parseParameterValue = (raw) => { + if (/^"(?:[\x20-\x21\x23-\x5b\x5d-\x7e]|\\["\\])*"$/.test(raw)) { + return { + type: "string", + value: raw.slice(1, -1).replace(/\\(["\\])/g, "$1"), + }; + } + if (/^-?(?:0|[1-9]\d*)$/.test(raw)) { + const value = Number(raw); + if (!Number.isSafeInteger(value)) { + throw new SignatureError("integer signature parameter is outside the safe range"); + } + return { type: "integer", value }; + } + if (/^[A-Za-z*][A-Za-z0-9_.*:\/-]*$/.test(raw)) { + return { type: "token", value: raw }; + } + if (raw === "?0" || raw === "?1") { + return { type: "boolean", value: raw === "?1" }; + } + throw new SignatureError("malformed signature parameter value"); +}; + /** * Parse one `Signature-Input` value: an inner list of quoted component * identifiers followed by `;name=value` parameters. @@ -116,23 +160,32 @@ const parseSignatureInputValue = (raw) => { } const params = {}; + const paramTypes = {}; const tail = raw.slice(close + 1); - for (const part of tail.split(";")) { + for (const part of splitParameters(tail)) { const chunk = part.trim(); if (!chunk) continue; const eq = chunk.indexOf("="); + const rawName = (eq === -1 ? chunk : chunk.slice(0, eq)).trim(); + if (!/^[a-z*][a-z0-9_.*-]*$/.test(rawName)) { + throw new SignatureError(`invalid signature parameter name ${rawName}`); + } + const name = rawName.toLowerCase(); + if (Object.hasOwn(params, name)) { + throw new SignatureError(`duplicate signature parameter ${name}`); + } if (eq === -1) { - params[chunk.toLowerCase()] = true; + params[name] = true; + paramTypes[name] = "boolean"; continue; } - const name = chunk.slice(0, eq).trim().toLowerCase(); - let value = chunk.slice(eq + 1).trim(); - if (value.startsWith('"') && value.endsWith('"')) value = value.slice(1, -1); - else if (/^-?\d+$/.test(value)) value = Number(value); + const parsed = parseParameterValue(chunk.slice(eq + 1).trim()); + const { type, value } = parsed; params[name] = value; + paramTypes[name] = type; } - return { components, params, raw }; + return { components, params, paramTypes, raw }; }; const parseSignatureValue = (raw) => { @@ -241,6 +294,43 @@ const assertRequiredComponents = (components, hasBody) => { } }; +const assertV1Profile = ({ label, components, params, paramTypes, rawSignature }) => { + if (label !== "sig1") { + throw new SignatureError('HTMLTrust v1 requires the signature label "sig1"'); + } + if ( + components.length !== V1_COMPONENTS.length || + components.some((component, index) => component !== V1_COMPONENTS[index]) + ) { + throw new SignatureError( + `HTMLTrust v1 requires exactly these covered components in order: ${V1_COMPONENTS.join(", ")}`, + ); + } + if (paramTypes.created !== "integer" || !Number.isSafeInteger(params.created) || params.created < 0) { + throw new SignatureError("HTMLTrust v1 requires an integer `created` parameter"); + } + if (paramTypes.keyid !== "string" || params.keyid.length === 0) { + throw new SignatureError("HTMLTrust v1 requires a non-empty `keyid` parameter"); + } + if (paramTypes.alg !== "string" || params.alg !== "ed25519") { + throw new SignatureError('HTMLTrust v1 requires `alg="ed25519"`'); + } + if (params.nonce !== undefined && (paramTypes.nonce !== "string" || params.nonce.length === 0)) { + throw new SignatureError("the `nonce` parameter must be a non-empty string"); + } + const encoded = rawSignature.slice(1, -1); + if (!/^[A-Za-z0-9+/]+$/.test(encoded)) { + throw new SignatureError("HTMLTrust v1 signatures must use canonical unpadded Base64"); + } + const decoded = Buffer.from(encoded.padEnd(Math.ceil(encoded.length / 4) * 4, "="), "base64"); + const canonical = decoded.toString("base64").replace(/=+$/, ""); + if (canonical !== encoded || decoded.length !== 64) { + throw new SignatureError( + "HTMLTrust v1 signatures must be a canonical unpadded Base64 encoding of 64 bytes", + ); + } +}; + const assertFreshness = (req, params) => { const now = Math.floor(Date.now() / 1000); if (typeof params.created === "number" && Math.abs(now - params.created) > MAX_SKEW_SECONDS) { @@ -251,7 +341,12 @@ const assertFreshness = (req, params) => { } const date = headerValue(req, "date"); const parsed = date ? Date.parse(date) : NaN; - if (!Number.isFinite(parsed)) throw new SignatureError("date header is missing or unparseable"); + if ( + !date || !IMF_FIXDATE.test(date) || !Number.isFinite(parsed) || + new Date(parsed).toUTCString() !== date + ) { + throw new SignatureError("date header must be a valid IMF-fixdate HTTP date"); + } if (Math.abs(now - Math.floor(parsed / 1000)) > MAX_SKEW_SECONDS) { throw new SignatureError("date header is outside the acceptance window"); } @@ -297,7 +392,10 @@ const verifyBytes = (base, signature, publicKeyPem, algorithm) => { * * @returns {Promise<{ok: true, actor: object} | {ok: false, status: number, title: string, detail: string}>} */ -const verifyHttpMessageSignature = async (req, { resolve = resolveUsableKey } = {}) => { +const verifyHttpMessageSignature = async ( + req, + { resolve = resolveUsableKey, strictV1 = false } = {}, +) => { const inputHeader = headerValue(req, "signature-input"); const signatureHeader = headerValue(req, "signature"); if (!inputHeader || !signatureHeader) { @@ -305,25 +403,45 @@ const verifyHttpMessageSignature = async (req, { resolve = resolveUsableKey } = } try { - const signatures = new Map(splitDictionary(signatureHeader).map((member) => splitLabel(member))); - const hasBody = Boolean(req.rawBody && req.rawBody.length > 0); + const signatureMembers = splitDictionary(signatureHeader).map((member) => splitLabel(member)); + const inputMembers = splitDictionary(inputHeader).map((member) => splitLabel(member)); + if (strictV1) { + if ( + signatureMembers.length !== 1 || + inputMembers.length !== 1 || + signatureMembers[0][0] !== "sig1" || + inputMembers[0][0] !== "sig1" + ) { + throw new SignatureError('HTMLTrust v1 accepts exactly one signature labeled "sig1"'); + } + } + const signatures = new Map(signatureMembers); + // An explicitly captured empty buffer is still a body representation. Its + // content-digest must bind to the empty byte sequence just like any other + // raw body; checking `.length > 0` would skip that verification. + const hasBody = req.rawBody !== undefined && req.rawBody !== null; let lastError = null; - for (const member of splitDictionary(inputHeader)) { - const [label, rawValue] = splitLabel(member); + for (const [label, rawValue] of inputMembers) { const rawSignature = signatures.get(label); if (!rawSignature) continue; try { - const { components, params, raw } = parseSignatureInputValue(rawValue); - if (!params.keyid || typeof params.keyid !== "string") { - throw new SignatureError("signature is missing a keyid parameter"); + const { components, params, paramTypes, raw } = parseSignatureInputValue(rawValue); + if (strictV1) { + assertV1Profile({ label, components, params, paramTypes, rawSignature }); + } else { + if (!params.keyid || typeof params.keyid !== "string") { + throw new SignatureError("signature is missing a keyid parameter"); + } + assertRequiredComponents(components, hasBody); } - assertRequiredComponents(components, hasBody); assertFreshness(req, params); - if (hasBody) verifyContentDigest(req); + // A covered digest always has to be tied back to the exact bytes, + // including an empty sequence when no parser supplied rawBody. + if (components.includes("content-digest")) verifyContentDigest(req); - const resolution = await resolve(params.keyid, { req }); + const resolution = await resolve(params.keyid, { req, algorithm: params.alg }); if (!resolution.ok) { return { ok: false, @@ -334,6 +452,9 @@ const verifyHttpMessageSignature = async (req, { resolve = resolveUsableKey } = }; } const { resolved } = resolution; + if (strictV1 && resolved.algorithm !== "ed25519") { + throw new SignatureError("HTMLTrust v1 request authentication requires an Ed25519 key"); + } if (params.alg && params.alg !== resolved.algorithm) { throw new SignatureError("signature `alg` does not match the resolved key algorithm"); } @@ -378,13 +499,13 @@ const verifyHttpMessageSignature = async (req, { resolve = resolveUsableKey } = * so the legacy static API-key schemes can stay available for the demo UI and * the conformance suite; see `src/middleware/auth.js`. */ -const requireActorSignature = ({ fallback } = {}) => async (req, res, next) => { +const requireActorSignature = ({ fallback, strictV1 = false } = {}) => async (req, res, next) => { const hasSignature = Boolean(req.headers["signature-input"] || req.headers.signature); if (!hasSignature && typeof fallback === "function") { return fallback(req, res, next); } - const result = await verifyHttpMessageSignature(req); + const result = await verifyHttpMessageSignature(req, { strictV1 }); if (!result.ok) { res.set("WWW-Authenticate", 'Signature realm="htmltrust-directory"'); return problem(res, result.status, result.title, result.detail, result.type ? { type: result.type } : {}); @@ -396,5 +517,6 @@ const requireActorSignature = ({ fallback } = {}) => async (req, res, next) => { module.exports = { buildSignatureBase, requireActorSignature, + V1_COMPONENTS, verifyHttpMessageSignature, }; diff --git a/src/models/ContentSignature.js b/src/models/ContentSignature.js index 6c96c52..9b4adcf 100644 --- a/src/models/ContentSignature.js +++ b/src/models/ContentSignature.js @@ -6,33 +6,43 @@ const ContentSignatureSchema = new mongoose.Schema({ required: [true, 'Content hash is required'], index: true }, - // Canonical hash of the claims map (sorted "name:content\n" records, then - // hashed). Part of the signature binding per spec §2.1. + // Canonical hash of the complete direct-child claims array. claimsHash: { type: String, default: '' }, - // ISO-8601 timestamp from the element in the - // signed-section. Part of the signature binding per spec §2.1. + // Exact v1 timestamp from the signed-at claim. signedAt: { type: String, default: '' }, domain: { type: String, - required: [true, 'Domain is required'], index: true }, + profile: { + type: String, + index: true + }, + algorithm: String, + keyid: { + type: String, + index: true + }, + scope: String, + location: { + type: String, + index: true + }, + sourceURL: String, authorId: { type: mongoose.Schema.Types.ObjectId, ref: 'Author', - required: true, index: true }, keyId: { type: mongoose.Schema.Types.ObjectId, - ref: 'Key', - required: true + ref: 'Key' }, signature: { type: String, @@ -53,8 +63,31 @@ const ContentSignatureSchema = new mongoose.Schema({ } }); -// Compound index for faster lookups -ContentSignatureSchema.index({ contentHash: 1, domain: 1, authorId: 1 }, { unique: true }); +// Pre-v1 rows and v1 rows have different identities. A v1 signature may be +// indexed at more than one signed URL, including when its key is remote and +// has no local Author row. +ContentSignatureSchema.index( + { contentHash: 1, domain: 1, authorId: 1 }, + { + unique: true, + partialFilterExpression: { + domain: { $type: 'string' }, + authorId: { $type: 'objectId' }, + // MongoDB partial indexes support `$in`, while `$exists: false` is not + // a supported partial-index predicate. `null` matches the missing + // profile field on pre-v1 documents and keeps v1 rows out of this + // legacy identity index. + profile: { $in: [null] } + } + } +); +ContentSignatureSchema.index( + { contentHash: 1, profile: 1, location: 1, keyid: 1 }, + { + unique: true, + partialFilterExpression: { profile: 'htmltrust-signature-v1' } + } +); // Virtual for content occurrences ContentSignatureSchema.virtual('contentOccurrences', { diff --git a/src/models/Key.js b/src/models/Key.js index a69952f..2a5c294 100644 --- a/src/models/Key.js +++ b/src/models/Key.js @@ -45,6 +45,12 @@ const KeySchema = new mongoose.Schema({ type: Boolean, default: false }, + revokedAt: Date, + supersededBy: String, + previousKeys: { + type: [String], + default: undefined + }, trustScore: { type: Number, min: 0, diff --git a/src/server.js b/src/server.js index fe4682d..0ae4f16 100644 --- a/src/server.js +++ b/src/server.js @@ -57,6 +57,10 @@ app.use(cors()); app.use( express.json({ limit: process.env.MAX_REQUEST_BODY || '256kb', + // Canonical directory resources use vendor JSON media types such as + // application/htmltrust-content+json. Keep application/json accepted for + // compatibility with the pre-v1 API surface. + type: ['application/json', 'application/*+json'], verify: (req, res, buf) => { req.rawBody = buf; }, @@ -112,7 +116,69 @@ const authLimiter = limiter(60 * 1000, Number(process.env.RATE_LIMIT_AUTH || 30) // Routes that mutate reputation or add records are the ones worth flooding. const writeLimiter = limiter(60 * 1000, Number(process.env.RATE_LIMIT_WRITE || 60)); +const { negotiate } = require('./middleware/contentNegotiation'); + +// Canonical HTMLTrust v1 directory surface. POST requests use the exact +// RFC 9421 profile and never fall back to a shared API key. +const { requireActorSignature } = require('./middleware/httpSignature'); +const { + getContentRecordV1, + listContentEndorsements, + submitContentV1, +} = require('./controllers/contentController'); +const { + createEndorsement, + listEndorsements, + deleteEndorsement, +} = require('./controllers/endorsementController'); +const { + getKeyDocument, + getSignerReputation, +} = require('./controllers/directoryController'); + +app.post( + '/content', + writeLimiter, + negotiate('application/htmltrust-content+json', { cacheControl: 'no-store', requestBody: true }), + requireActorSignature({ strictV1: true }), + submitContentV1, +); +app.get( + '/content/:contentHash/endorsements', + negotiate('application/htmltrust-endorsement+json'), + listContentEndorsements, +); +app.get( + '/content/:contentHash', + negotiate('application/htmltrust-content+json'), + getContentRecordV1, +); +app.get( + '/endorsements', + negotiate('application/htmltrust-endorsement+json'), + listEndorsements, +); +app.post( + '/endorsements', + writeLimiter, + negotiate('application/htmltrust-endorsement+json', { cacheControl: 'no-store', requestBody: true }), + requireActorSignature({ strictV1: true }), + createEndorsement, +); +app.delete( + '/endorsements/:id', + requireActorSignature({ fallback: (req, res, next) => next() }), + deleteEndorsement, +); +app.get('/keys/:id', negotiate('application/htmltrust-key+json'), getKeyDocument); +app.get( + '/signers/:id/reputation', + negotiate('application/json'), + getSignerReputation, +); +// Explicit pre-v1 compatibility surface used by the demo UI and the original +// conformance runner. These routes retain their documented API-key fallback. app.use('/api/authors', authLimiter, require('./routes/authors')); app.use('/api/content', writeLimiter, require('./routes/content')); app.use('/api/claims', require('./routes/claims')); @@ -123,8 +189,16 @@ app.use('/api/keys', require('./routes/keys')); app.use('/api/signers', require('./routes/signers')); const { discovery } = require('./controllers/directoryController'); -app.get('/.well-known/htmltrust', discovery); -app.get('/api/.well-known/htmltrust', discovery); +app.get( + '/.well-known/htmltrust', + negotiate('application/htmltrust-directory+json', { cacheControl: 'public, max-age=3600, must-revalidate' }), + discovery, +); +app.get( + '/api/.well-known/htmltrust', + negotiate('application/htmltrust-directory+json', { cacheControl: 'public, max-age=3600, must-revalidate' }), + discovery, +); app.use(express.static(path.join(__dirname, 'public'))); diff --git a/src/utils/claims.js b/src/utils/claims.js index 86a5a68..3826b74 100644 --- a/src/utils/claims.js +++ b/src/utils/claims.js @@ -1,22 +1,14 @@ /** - * Canonical claims serialization, draft §4.6. + * HTMLTrust v1 claim normalization and serialization. * - * The claims hash is part of the signing payload binding (§5), so the - * directory has to reproduce the signer's byte string exactly: - * - * 1. Each claim `name` and `content` is normalized as plain text per §4.4 - * (NFKC, formatting-character strip, whitespace collapse). - * 2. Each pair is serialized as `name` `:` `content` `\n`. - * 3. Lines are sorted by the UTF-8 byte sequence of the *normalized name* - * (not by the whole line, and not by UTF-16 code unit). - * 4. The sorted lines are concatenated. - * - * Text normalization and serialization come from the shared - * `@htmltrust/canonicalization` library so the server cannot drift from the - * signers. Array input is validated before conversion to an object because an - * object cannot represent duplicate claim names. + * Claims are normalized as plain text, checked before serialization, sorted + * by the UTF-8 bytes of their normalized names, and escaped so the record + * grammar stays injective. */ +const MAX_CLAIMS = 64; +const MAX_CLAIM_BYTES = 4 * 1024; + let sharedModulePromise; const loadShared = () => { if (!sharedModulePromise) { @@ -25,42 +17,104 @@ const loadShared = () => { return sharedModulePromise; }; +const claimError = (code, detail) => { + const error = new Error(`${code}: ${detail}`); + error.expose = true; + return error; +}; + +const escapeClaimField = (value) => value.replace(/[\\:\n]/g, (character) => { + if (character === "\\") return "\\\\"; + if (character === ":") return "\\:"; + return "\\n"; +}); + +const asEntries = (claims, strictArray) => { + if (strictArray && !Array.isArray(claims)) { + throw claimError("claim-malformed", "claims must be an array"); + } + if (Array.isArray(claims)) { + return claims.map((claim) => { + if (!claim || typeof claim !== "object" || Array.isArray(claim)) { + throw claimError("claim-malformed", "each claim must be an object"); + } + if (strictArray) { + const keys = Object.keys(claim).sort(); + if (keys.length !== 2 || keys[0] !== "content" || keys[1] !== "name") { + throw claimError("claim-malformed", "each claim must contain only name and content"); + } + if (typeof claim.name !== "string" || typeof claim.content !== "string") { + throw claimError("claim-malformed", "claim name and content must be strings"); + } + } + return [claim.name, claim.content]; + }); + } + if (!claims || typeof claims !== "object") { + throw claimError("claim-malformed", "claims must be an object or array"); + } + return Object.entries(claims); +}; + /** - * Serialize a claims map to its canonical byte string. + * Normalize and validate a complete set of direct-child claim records. * - * @param {Record|Array<{name: string, content: string}>} claims - * @returns {Promise} canonical claims string + * @param {unknown} claims + * @param {{strictArray?: boolean}} options + * @returns {Promise>} */ -const canonicalizeClaims = async (claims) => { - const shared = await loadShared(); - const entries = Array.isArray(claims) - ? claims.map((claim) => [claim && claim.name, claim && claim.content]) - : Object.entries(claims || {}); +const normalizeClaims = async (claims, { strictArray = false } = {}) => { + const entries = asEntries(claims, strictArray); + if (entries.length > MAX_CLAIMS) { + throw claimError("resource-limit-exceeded", `a section may contain at most ${MAX_CLAIMS} claims`); + } + const shared = await loadShared(); const seen = new Set(); - const validated = entries.map(([name, content]) => { + const normalized = entries.map(([name, content]) => { if (name == null || content == null) { - const error = new Error("claim-malformed: each claim needs a name and content"); - error.expose = true; - throw error; + throw claimError("claim-malformed", "each claim needs a name and content"); } - const normalizedName = shared.normalizeText(String(name)).trim(); + const normalizedContent = shared.normalizeText(String(content)).trim(); if (!normalizedName) { - const error = new Error("claim-malformed: claim name normalized to the empty string"); - error.expose = true; - throw error; + throw claimError("claim-malformed", "claim name normalized to the empty string"); + } + if ( + Buffer.byteLength(normalizedName, "utf8") > MAX_CLAIM_BYTES || + Buffer.byteLength(normalizedContent, "utf8") > MAX_CLAIM_BYTES + ) { + throw claimError( + "resource-limit-exceeded", + `normalized claim names and values may be at most ${MAX_CLAIM_BYTES} bytes`, + ); } if (seen.has(normalizedName)) { - const error = new Error(`claim-duplicate: ${normalizedName}`); - error.expose = true; - throw error; + throw claimError("claim-duplicate", normalizedName); } seen.add(normalizedName); - return [normalizedName, content]; + return { name: normalizedName, content: normalizedContent }; }); - return shared.canonicalizeClaims(Object.fromEntries(validated)); + normalized.sort((left, right) => Buffer.compare( + Buffer.from(left.name, "utf8"), + Buffer.from(right.name, "utf8"), + )); + return normalized; +}; + +/** Serialize claims to the v1 canonical byte string. */ +const canonicalizeClaims = async (claims, options) => { + const normalized = await normalizeClaims(claims, options); + return normalized + .map(({ name, content }) => `${escapeClaimField(name)}:${escapeClaimField(content)}\n`) + .join(""); }; -module.exports = { canonicalizeClaims }; +module.exports = { + canonicalizeClaims, + escapeClaimField, + MAX_CLAIMS, + MAX_CLAIM_BYTES, + normalizeClaims, +}; diff --git a/src/utils/directoryUrl.js b/src/utils/directoryUrl.js index 310d613..fa6d9eb 100644 --- a/src/utils/directoryUrl.js +++ b/src/utils/directoryUrl.js @@ -12,6 +12,6 @@ const directoryBaseUrl = (req, env = process.env) => { }; const directoryKeyUrl = (req, keyId, env = process.env) => - `${directoryBaseUrl(req, env)}/api/keys/${encodeURIComponent(String(keyId))}`; + `${directoryBaseUrl(req, env)}/keys/${encodeURIComponent(String(keyId))}`; module.exports = { directoryBaseUrl, directoryKeyUrl }; diff --git a/src/utils/htmltrustProtocol.js b/src/utils/htmltrustProtocol.js index 7bf66bb..7b400fd 100644 --- a/src/utils/htmltrustProtocol.js +++ b/src/utils/htmltrustProtocol.js @@ -157,22 +157,24 @@ const assertRfc3339Utc = (value, field = "timestamp") => { return value; }; -const keyDocumentFor = (key) => { +const keyDocumentFor = (key, kid) => { const publicKeyObject = crypto.createPublicKey(key.publicKey); const der = publicKeyObject.export({ type: "spki", format: "der" }); const doc = { - kid: String(key._id), + kid: kid || String(key._id), algorithm: normalizeAlgorithm(key.algorithm), publicKey: toCanonicalBase64(der), publicKeyEncoding: "spki-der", - // Draft §8.2: a verifier MUST treat `revoked: true` as a key-revoked - // failure, so this has to report the stored state rather than a constant. revoked: Boolean(key.revoked), }; if (key.expiresAt) { doc.expires = key.expiresAt.toISOString(); } - doc.publicKeyPem = key.publicKey; + if (key.revokedAt) doc.revokedAt = key.revokedAt.toISOString(); + if (key.supersededBy) doc.supersededBy = key.supersededBy; + if (Array.isArray(key.previousKeys) && key.previousKeys.length > 0) { + doc.previousKeys = key.previousKeys; + } return doc; }; diff --git a/src/utils/jcs.js b/src/utils/jcs.js index 07aefe1..4aae8f1 100644 --- a/src/utils/jcs.js +++ b/src/utils/jcs.js @@ -9,10 +9,9 @@ * * §3.2.1 Whitespace between tokens is removed. * §3.2.2.1 Literals: `null`, `true`, `false`. - * §3.2.2.2 Strings use the ECMAScript `JSON.stringify` escaping, including - * the ES2019 "well-formed" escaping of lone surrogates. Node's - * `JSON.stringify` implements exactly this, so it is used directly - * rather than reimplemented. + * §3.2.2.2 Strings use ECMAScript JSON escaping. Lone UTF-16 surrogates are + * rejected because RFC 8785 requires I-JSON input and explicitly + * treats them as invalid Unicode data. * §3.2.2.3 Numbers use the ECMAScript `Number::toString` algorithm, which is * what `JSON.stringify` emits for finite numbers (including the * `-0` -> `0` mapping JCS requires). Non-finite numbers are not @@ -24,9 +23,8 @@ * claims canonicalization in draft §4.6, which sorts by UTF-8 * bytes.) * - * Arrays keep their element order. Values that have no JSON representation - * (`undefined`, functions, symbols) are dropped from objects and serialized as - * `null` inside arrays, matching `JSON.stringify` semantics. + * Arrays keep their element order. Values outside the JSON data model are + * rejected rather than silently changing a signed payload. */ /** @@ -37,12 +35,20 @@ const byUtf16CodeUnit = (a, b) => { return a < b ? -1 : 1; }; -// Values JSON.stringify silently omits from objects and replaces with null -// inside arrays. BigInt is deliberately NOT in this set: JSON.stringify -// throws on it, and silently dropping a numeric field would change the -// signed payload without anyone noticing. -const isOmittable = (value) => - value === undefined || typeof value === "function" || typeof value === "symbol"; +const assertUnicodeScalarString = (value) => { + for (let index = 0; index < value.length; index += 1) { + const unit = value.charCodeAt(index); + if (unit >= 0xd800 && unit <= 0xdbff) { + const next = value.charCodeAt(index + 1); + if (!(next >= 0xdc00 && next <= 0xdfff)) { + throw new Error("JCS: string contains a lone UTF-16 surrogate"); + } + index += 1; + } else if (unit >= 0xdc00 && unit <= 0xdfff) { + throw new Error("JCS: string contains a lone UTF-16 surrogate"); + } + } +}; const serialize = (value, out) => { // Honour toJSON() the way JSON.stringify does, so Date and Mongoose @@ -68,6 +74,7 @@ const serialize = (value, out) => { out.push(JSON.stringify(value)); return; case "string": + assertUnicodeScalarString(value); out.push(JSON.stringify(value)); return; case "bigint": @@ -80,22 +87,18 @@ const serialize = (value, out) => { out.push("["); for (let i = 0; i < value.length; i += 1) { if (i > 0) out.push(","); - const element = value[i]; - // JSON.stringify replaces non-representable array elements with null. - if (isOmittable(element)) out.push("null"); - else serialize(element, out); + serialize(value[i], out); } out.push("]"); return; } if (typeof value === "object") { - const names = Object.keys(value) - .filter((name) => !isOmittable(value[name])) - .sort(byUtf16CodeUnit); + const names = Object.keys(value).sort(byUtf16CodeUnit); out.push("{"); for (let i = 0; i < names.length; i += 1) { if (i > 0) out.push(","); + assertUnicodeScalarString(names[i]); out.push(JSON.stringify(names[i])); out.push(":"); serialize(value[names[i]], out); diff --git a/src/utils/keyResolution.js b/src/utils/keyResolution.js index 2136f0d..f39b875 100644 --- a/src/utils/keyResolution.js +++ b/src/utils/keyResolution.js @@ -1,8 +1,13 @@ const crypto = require("crypto"); const dns = require("dns").promises; +const https = require("https"); const net = require("net"); const Key = require("../models/Key"); -const { normalizeAlgorithm } = require("./htmltrustProtocol"); +const { + assertRfc3339Utc, + decodeCanonicalBase64, + normalizeAlgorithm, +} = require("./htmltrustProtocol"); /** * Key resolution, draft §8. @@ -90,19 +95,102 @@ const selfOriginsFor = (req) => { const isExpired = (expires) => { if (!expires) return false; const at = expires instanceof Date ? expires.getTime() : Date.parse(expires); - return Number.isFinite(at) && at <= Date.now(); + return !Number.isFinite(at) || at <= Date.now(); }; -const resolveLocal = async (keyid, selfOrigins) => { +/** + * DID verification methods can carry the same lifecycle fields as an + * HTMLTrust key document. A malformed lifecycle value is unusable as well, + * since accepting it would turn an untrusted DID document into a key grant. + */ +const isUsableDidMethod = (method) => { + if (!method || typeof method !== "object" || Array.isArray(method)) return false; + if (method.revoked === true) return false; + if (method.revoked !== undefined && typeof method.revoked !== "boolean") return false; + + for (const field of ["expires", "expiresAt"]) { + if (method[field] !== undefined) { + const expiresAt = method[field] instanceof Date + ? method[field].getTime() + : typeof method[field] === "string" && method[field].length > 0 + ? Date.parse(method[field]) + : NaN; + if (!Number.isFinite(expiresAt) || expiresAt <= Date.now()) return false; + } + } + if (method.revokedAt !== undefined) { + const revokedAt = method.revokedAt instanceof Date + ? method.revokedAt.getTime() + : typeof method.revokedAt === "string" && method.revokedAt.length > 0 + ? Date.parse(method.revokedAt) + : NaN; + if (!Number.isFinite(revokedAt) || revokedAt <= Date.now()) return false; + } + return true; +}; + +/** + * Restrict DID verification method types to key material this resolver can + * actually verify. DID Core requires `type`; omitting it leaves the verifier + * without the algorithm binding required by the HTMLTrust profile. + */ +const isCompatibleDidMethodType = (method, algorithm) => { + if (method.type === undefined) return false; + const types = Array.isArray(method.type) ? method.type : [method.type]; + if (types.length === 0 || types.some((type) => typeof type !== "string")) return false; + + const keyForm = method.publicKeyJwk || method.kty ? "jwk" : + method.publicKeyMultibase ? "multibase" : + method.publicKeyBase58 ? "base58" : "unknown"; + if (keyForm !== "jwk") return false; + + const compatible = types.map((type) => type.includes("#") ? type.slice(type.lastIndexOf("#") + 1) : type); + if (compatible.some((type) => ![ + "JsonWebKey2020", + "Ed25519VerificationKey2018", + "Ed25519VerificationKey2020", + "EcdsaSecp256r1VerificationKey2019", + "EcdsaSecp256r1VerificationKey2020", + "EcdsaSecp384r1VerificationKey2019", + "EcdsaSecp384r1VerificationKey2020", + "RsaVerificationKey2018", + "RsaVerificationKey2020", + ].includes(type))) return false; + + return compatible.some((type) => { + switch (type) { + case "JsonWebKey2020": + return true; + case "Ed25519VerificationKey2018": + case "Ed25519VerificationKey2020": + return algorithm === "ed25519"; + case "EcdsaSecp256r1VerificationKey2019": + case "EcdsaSecp256r1VerificationKey2020": + return algorithm === "ecdsa-p256"; + case "EcdsaSecp384r1VerificationKey2019": + case "EcdsaSecp384r1VerificationKey2020": + return algorithm === "ecdsa-p384"; + case "RsaVerificationKey2018": + case "RsaVerificationKey2020": + return algorithm === "rsa-pkcs1-sha256" || algorithm === "rsa-pss-sha256"; + default: + return false; + } + }); +}; + +const resolveLocal = async (keyid, selfOrigins, requestedAlgorithm) => { if (!isSelfHosted(keyid, selfOrigins)) return null; const id = directoryKeyIdFrom(keyid); if (!id) return null; const key = await Key.findById(id); if (!key) return null; + const algorithm = normalizeAlgorithm(key.algorithm); + if (requestedAlgorithm && algorithm !== requestedAlgorithm) return null; return { keyid, publicKeyPem: key.publicKey, - algorithm: normalizeAlgorithm(key.algorithm), + algorithm, revoked: Boolean(key.revoked), expires: key.expiresAt || null, source: "directory", @@ -111,109 +199,302 @@ const resolveLocal = async (keyid, selfOrigins) => { }; /** - * Reject hosts that resolve to loopback, link-local, or RFC 1918 space before - * making an outbound request. This is checked before the fetch, so a hostile - * DNS server can still race it (TOCTOU); it raises the cost of SSRF rather - * than eliminating it, which is why remote resolution stays opt-in. + * Return true only for globally routable addresses. This deliberately errs on + * the side of refusing special-use space, including documentation, benchmark, + * multicast, and transition ranges. The address returned here is also pinned + * to the eventual TLS connection by fetchJson's `lookup` callback. */ -const assertPublicHost = async (hostname) => { - const literal = net.isIP(hostname) ? [{ address: hostname }] : await dns.lookup(hostname, { all: true }); - for (const { address } of literal) { - if ( - /^127\./.test(address) || - /^10\./.test(address) || - /^192\.168\./.test(address) || - /^169\.254\./.test(address) || - /^172\.(1[6-9]|2[0-9]|3[01])\./.test(address) || - address === "0.0.0.0" || - address === "::1" || - /^f[cd][0-9a-f]{2}:/i.test(address) || - /^fe80:/i.test(address) - ) { +const ipv4ToNumber = (address) => { + const parts = address.split(".").map(Number); + if (parts.length !== 4 || parts.some((part) => !Number.isInteger(part) || part < 0 || part > 255)) return null; + return (((parts[0] * 256 + parts[1]) * 256 + parts[2]) * 256) + parts[3]; +}; + +const inIpv4Range = (address, network, prefix) => { + const value = ipv4ToNumber(address); + const base = ipv4ToNumber(network); + if (value === null || base === null) return false; + const mask = prefix === 0 ? 0 : (0xffffffff << (32 - prefix)) >>> 0; + return ((value >>> 0) & mask) === ((base >>> 0) & mask); +}; + +const parseIpv6 = (address) => { + let value = address.toLowerCase().replace(/^\[|\]$/g, ""); + if (value.includes("%")) return null; + if (value.includes(".")) { + const split = value.lastIndexOf(":"); + const ipv4 = ipv4ToNumber(value.slice(split + 1)); + if (split < 0 || ipv4 === null) return null; + value = `${value.slice(0, split)}:${(ipv4 >>> 16).toString(16)}:${(ipv4 & 0xffff).toString(16)}`; + } + const halves = value.split("::"); + if (halves.length > 2) return null; + const left = halves[0] ? halves[0].split(":") : []; + const right = halves.length === 2 && halves[1] ? halves[1].split(":") : []; + if (left.concat(right).some((part) => !/^[0-9a-f]{1,4}$/.test(part))) return null; + const missing = 8 - left.length - right.length; + if ((halves.length === 1 && missing !== 0) || (halves.length === 2 && missing < 1)) return null; + const segments = left.concat(Array(missing).fill("0"), right); + return segments.reduce((result, segment) => (result << 16n) | BigInt(parseInt(segment, 16)), 0n); +}; + +const inIpv6Range = (value, network, prefix) => { + const base = parseIpv6(network); + if (value === null || base === null) return false; + return (value >> BigInt(128 - prefix)) === (base >> BigInt(128 - prefix)); +}; + +const isPublicAddress = (address) => { + const family = net.isIP(address); + if (family === 4) { + // RFC 6890 special-use and other non-global IPv4 ranges. + return ![ + ["0.0.0.0", 8], ["10.0.0.0", 8], ["100.64.0.0", 10], ["127.0.0.0", 8], + ["169.254.0.0", 16], ["172.16.0.0", 12], ["192.0.0.0", 24], ["192.0.2.0", 24], + ["192.88.99.0", 24], ["192.168.0.0", 16], ["198.18.0.0", 15], ["198.51.100.0", 24], + ["203.0.113.0", 24], ["224.0.0.0", 4], ["240.0.0.0", 4], + ].some(([network, prefix]) => inIpv4Range(address, network, prefix)); + } + if (family !== 6) return false; + const value = parseIpv6(address); + if (value === null) return false; + // IPv4-mapped addresses inherit the routability of their embedded IPv4. + if (inIpv6Range(value, "::ffff:0:0", 96)) { + const embedded = Number(value & 0xffffffffn); + const ipv4 = `${embedded >>> 24}.${(embedded >>> 16) & 255}.${(embedded >>> 8) & 255}.${embedded & 255}`; + return isPublicAddress(ipv4); + } + // Global unicast is 2000::/3. Everything outside it is special-use, + // including loopback, ULA, link-local, multicast, and unspecified space. + if (!inIpv6Range(value, "2000::", 3)) return false; + return ![ + ["2001:db8::", 32], ["2001:10::", 28], ["2001:20::", 28], + ["2001:2::", 48], ["2001::", 32], ["2002::", 16], + ].some(([network, prefix]) => inIpv6Range(value, network, prefix)); +}; + +const resolvePublicAddresses = async (hostname) => { + const host = hostname.replace(/^\[|\]$/g, ""); + const addresses = net.isIP(host) + ? [{ address: host, family: net.isIP(host) }] + : await dns.lookup(host, { all: true }); + if (!addresses.length) throw new Error("key document host did not resolve"); + for (const { address } of addresses) { + if (!isPublicAddress(address)) { throw new Error(`refusing to resolve a key from non-public address ${address}`); } } + return addresses; }; -const fetchJson = async (url) => { +const fetchJson = async (url, acceptedMediaTypes) => { const target = new URL(url); if (target.protocol !== "https:") { throw new Error("key documents may only be fetched over https"); } - await assertPublicHost(target.hostname); - - const controller = new AbortController(); - const timer = setTimeout(() => controller.abort(), REMOTE_TIMEOUT_MS); - try { - const response = await fetch(target, { - redirect: "error", - signal: controller.signal, + if (target.username || target.password) throw new Error("key document URLs may not contain credentials"); + if (target.port && target.port !== "443") throw new Error("key documents may only use HTTPS port 443"); + const addresses = await resolvePublicAddresses(target.hostname); + return new Promise((resolve, reject) => { + let settled = false; + let request; + const timer = setTimeout(() => request.destroy(new Error("key document fetch timed out")), REMOTE_TIMEOUT_MS); + const fail = (error) => { + if (settled) return; + settled = true; + clearTimeout(timer); + reject(error); + }; + const succeed = (value) => { + if (settled) return; + settled = true; + clearTimeout(timer); + resolve(value); + }; + request = https.request({ + protocol: target.protocol, + hostname: target.hostname.replace(/^\[|\]$/g, ""), + port: target.port || 443, + path: `${target.pathname}${target.search}`, + servername: net.isIP(target.hostname.replace(/^\[|\]$/g, "")) ? undefined : target.hostname, + method: "GET", headers: { accept: "application/htmltrust-key+json, application/jwk+json, application/json" }, - }); - if (!response.ok) throw new Error(`key document fetch returned ${response.status}`); - const declaredLength = Number(response.headers.get("content-length")); - if (Number.isFinite(declaredLength) && declaredLength > REMOTE_MAX_BYTES) { - throw new Error("key document exceeds the size limit"); - } - if (!response.body) throw new Error("key document response has no body"); - const chunks = []; - let received = 0; - for await (const chunk of response.body) { - received += chunk.byteLength; - if (received > REMOTE_MAX_BYTES) { - controller.abort(); - throw new Error("key document exceeds the size limit"); + // The validated address is used for this request, closing the DNS + // validation/request TOCTOU window and DNS rebinding race. + lookup: (_hostname, options, callback) => { + const selected = addresses[0]; + const family = selected.family || net.isIP(selected.address); + if (options?.all) { + callback(null, [{ address: selected.address, family }]); + } else { + callback(null, selected.address, family); + } + }, + }, (response) => { + response.on("error", fail); + const status = response.statusCode || 0; + if (status < 200 || status >= 300) { + response.resume(); + fail(new Error(`key document fetch returned ${status}`)); + return; + } + const headers = response.headers || {}; + const mediaType = String(headers["content-type"] || "") + .split(";", 1)[0].trim().toLowerCase(); + if (!acceptedMediaTypes.includes(mediaType)) { + response.resume(); + fail(new Error(`key document has unsupported media type ${mediaType || "(missing)"}`)); + return; + } + let headerBytes = 0; + for (const [name, value] of Object.entries(headers)) { + headerBytes += Buffer.byteLength(name) + Buffer.byteLength(Array.isArray(value) ? value.join(",") : String(value)) + 4; } - chunks.push(Buffer.from(chunk)); + const declaredLength = Number(headers["content-length"]); + if (headerBytes > REMOTE_MAX_BYTES || (Number.isFinite(declaredLength) && declaredLength + headerBytes > REMOTE_MAX_BYTES)) { + response.resume(); + fail(new Error("key document exceeds the size limit")); + return; + } + const chunks = []; + let received = headerBytes; + response.on("data", (chunk) => { + received += chunk.length; + if (received > REMOTE_MAX_BYTES) { + fail(new Error("key document exceeds the size limit")); + request.destroy(); + return; + } + chunks.push(Buffer.from(chunk)); + }); + response.on("end", () => { + if (received > REMOTE_MAX_BYTES) return; + try { + succeed({ document: JSON.parse(Buffer.concat(chunks).toString("utf8")), mediaType }); + } catch (error) { + fail(error); + } + }); + }); + request.setTimeout(REMOTE_TIMEOUT_MS, () => { + fail(new Error("key document fetch timed out")); + request.destroy(); + }); + request.on("error", fail); + request.end(); + }); +}; + +const algorithmFromJwk = (document, { allowTypeInference = false, requestedAlgorithm } = {}) => { + if (document && typeof document.alg === "string") { + switch (document.alg) { + case "EdDSA": return "ed25519"; + case "ES256": return "ecdsa-p256"; + case "ES384": return "ecdsa-p384"; + case "RS256": return "rsa-pkcs1-sha256"; + case "PS256": return "rsa-pss-sha256"; + default: throw new Error(`JWK declares unsupported alg ${document.alg}`); } - return JSON.parse(Buffer.concat(chunks, received).toString("utf8")); - } finally { - clearTimeout(timer); } + if (allowTypeInference) { + if (document?.kty === "OKP" && document.crv === "Ed25519") return "ed25519"; + if (document?.kty === "EC" && document.crv === "P-256") return "ecdsa-p256"; + if (document?.kty === "EC" && document.crv === "P-384") return "ecdsa-p384"; + // RSA JWKs do not encode the padding/hash choice. A declared requested + // algorithm supplies that missing part when the DID method omits `alg`. + if (document?.kty === "RSA" && requestedAlgorithm?.startsWith("rsa-")) return requestedAlgorithm; + } + throw new Error("JWK must declare a supported alg"); }; -const pemFromKeyDocument = (document) => { - if (document && document.kty) { - // RFC 7517 JSON Web Key (draft §8.2 permits either shape). - return crypto - .createPublicKey({ key: document, format: "jwk" }) - .export({ type: "spki", format: "pem" }); - } - if (document && typeof document.publicKey === "string") { - if (document.publicKey.includes("BEGIN PUBLIC KEY")) return document.publicKey; - const der = Buffer.from(document.publicKey, "base64"); - return crypto - .createPublicKey({ key: der, format: "der", type: "spki" }) - .export({ type: "spki", format: "pem" }); - } - throw new Error("key document does not contain a usable public key"); +const assertKeyMatchesAlgorithm = (keyObject, algorithm) => { + const type = keyObject.asymmetricKeyType; + const details = keyObject.asymmetricKeyDetails || {}; + const matches = { + ed25519: type === "ed25519", + "ecdsa-p256": type === "ec" && details.namedCurve === "prime256v1", + "ecdsa-p384": type === "ec" && details.namedCurve === "secp384r1", + "rsa-pkcs1-sha256": type === "rsa", + "rsa-pss-sha256": type === "rsa" || type === "rsa-pss", + }[algorithm]; + if (!matches) throw new Error(`public key parameters do not match ${algorithm}`); }; -const algorithmFromKeyDocument = (document) => { - if (document && document.algorithm) return normalizeAlgorithm(document.algorithm); - if (document && document.alg) { - switch (document.alg) { - case "EdDSA": - return "ed25519"; - case "ES256": - return "ecdsa-p256"; - case "ES384": - return "ecdsa-p384"; - case "RS256": - return "rsa-pkcs1-sha256"; - case "PS256": - return "rsa-pss-sha256"; - default: - break; +const validateOptionalKeyFields = (document) => { + if (document.expires !== undefined) { + if (typeof document.expires !== "string") throw new Error("expires must be a string"); + assertRfc3339Utc(document.expires, "expires"); + } + if (document.revoked !== undefined && typeof document.revoked !== "boolean") { + throw new Error("revoked must be a boolean"); + } + if (document.revokedAt !== undefined) { + if (typeof document.revokedAt !== "string") throw new Error("revokedAt must be a string"); + assertRfc3339Utc(document.revokedAt, "revokedAt"); + } + if (document.supersededBy !== undefined && typeof document.supersededBy !== "string") { + throw new Error("supersededBy must be a string"); + } + if ( + document.previousKeys !== undefined && + (!Array.isArray(document.previousKeys) || document.previousKeys.some((value) => typeof value !== "string")) + ) { + throw new Error("previousKeys must be an array of strings"); + } +}; + +const resolveHtmlTrustKeyDocument = (document, requestedKeyid, requestedAlgorithm) => { + if (!document || typeof document !== "object" || Array.isArray(document)) { + throw new Error("key document must be a JSON object"); + } + if (document.kid !== undefined) { + if (typeof document.kid !== "string" || document.kid !== requestedKeyid) { + throw new Error("key document kid does not match the requested keyid"); } } - if (document && document.kty === "OKP" && document.crv === "Ed25519") return "ed25519"; - throw new Error("key document does not declare a supported algorithm"); + if (document.publicKeyEncoding !== "spki-der") { + throw new Error('publicKeyEncoding must be exactly "spki-der"'); + } + if (typeof document.algorithm !== "string") { + throw new Error("key document algorithm is required"); + } + const algorithm = normalizeAlgorithm(document.algorithm); + if (algorithm !== document.algorithm) { + throw new Error(`key document algorithm must use the canonical identifier ${algorithm}`); + } + if (requestedAlgorithm && algorithm !== requestedAlgorithm) { + throw new Error(`key document algorithm does not match the requested algorithm ${requestedAlgorithm}`); + } + const der = decodeCanonicalBase64(document.publicKey, "publicKey"); + const keyObject = crypto.createPublicKey({ key: der, format: "der", type: "spki" }); + assertKeyMatchesAlgorithm(keyObject, algorithm); + validateOptionalKeyFields(document); + return { + publicKeyPem: keyObject.export({ type: "spki", format: "pem" }), + algorithm, + }; +}; + +const resolveJwk = (document, options = {}) => { + if (!document || typeof document !== "object" || Array.isArray(document) || !document.kty) { + throw new Error("JWK must be a JSON object with kty"); + } + const algorithm = algorithmFromJwk(document, options); + if (options.requestedAlgorithm && algorithm !== options.requestedAlgorithm) { + throw new Error(`JWK algorithm does not match the requested algorithm ${options.requestedAlgorithm}`); + } + const keyObject = crypto.createPublicKey({ key: document, format: "jwk" }); + assertKeyMatchesAlgorithm(keyObject, algorithm); + return { + publicKeyPem: keyObject.export({ type: "spki", format: "pem" }), + algorithm, + }; }; /** did:web resolution per §8.1, restricted to the did:web method. */ const didWebUrl = (did) => { - const rest = did.slice("did:web:".length); + const documentDid = did.split("#", 1)[0]; + const rest = documentDid.slice("did:web:".length); if (!rest) throw new Error("malformed did:web identifier"); const parts = rest.split(":").map(decodeURIComponent); const host = parts.shift(); @@ -221,43 +502,102 @@ const didWebUrl = (did) => { return `https://${host}${path}`; }; -const resolveDidWeb = async (did) => { - const document = await fetchJson(didWebUrl(did)); - const methods = Array.isArray(document.verificationMethod) ? document.verificationMethod : []; +const resolveDidWeb = async (did, requestedAlgorithm) => { + const { document } = await fetchJson(didWebUrl(did), [ + "application/did+json", + "application/did+ld+json", + "application/ld+json", + "application/json", + ]); + const methods = Array.isArray(document.verificationMethod) + ? document.verificationMethod.filter((method) => method && typeof method === "object" && !Array.isArray(method)) + : []; + const methodsById = new Map(); + const duplicateMethodIds = new Set(); + for (const method of methods) { + if (typeof method.id !== "string") continue; + if (methodsById.has(method.id)) duplicateMethodIds.add(method.id); + else methodsById.set(method.id, method); + } const assertion = Array.isArray(document.assertionMethod) ? document.assertionMethod : []; - const preferred = - methods.find((method) => assertion.includes(method.id)) || methods[0]; - if (!preferred) throw new Error("DID document has no verification method"); - const material = preferred.publicKeyJwk || preferred; - return { - publicKeyPem: pemFromKeyDocument(material), - algorithm: algorithmFromKeyDocument(material), - revoked: false, - expires: null, - source: "did:web", - }; + const authorized = []; + const seenAuthorizationIds = new Set(); + for (const entry of assertion) { + const embedded = entry && typeof entry === "object" && !Array.isArray(entry) ? entry : null; + const id = typeof entry === "string" ? entry : embedded?.id; + if (typeof id !== "string" || seenAuthorizationIds.has(id)) continue; + seenAuthorizationIds.add(id); + if (duplicateMethodIds.has(id)) continue; + // An embedded relationship entry is itself a verification method. If it + // only carries an id, use the full method from verificationMethod. A + // linked method is authoritative, preventing an embedded duplicate from + // replacing its key material under the same id. + authorized.push(methodsById.get(id) || embedded); + } + const fragment = did.includes("#"); + const ordered = authorized.filter((method) => method && (!fragment || method.id === did)); + if (!ordered.length) throw new Error(fragment ? "DID verification method was not found" : "DID document has no verification method"); + let lastError; + const seen = new Set(); + for (const preferred of ordered) { + if (!preferred) continue; + const identity = preferred.id || preferred; + if (seen.has(identity)) continue; + seen.add(identity); + if (!isUsableDidMethod(preferred)) { + lastError = new Error("DID verification method is revoked or expired"); + if (fragment) break; + continue; + } + try { + const material = preferred.publicKeyJwk || preferred; + const resolved = resolveJwk(material, { allowTypeInference: true, requestedAlgorithm }); + if (!isCompatibleDidMethodType(preferred, resolved.algorithm)) { + throw new Error("DID verification method type does not match its key algorithm"); + } + return { + ...resolved, + revoked: false, + expires: preferred.expires || preferred.expiresAt || null, + source: "did:web", + }; + } catch (error) { + lastError = error; + // A fragment names one exact method. Do not silently substitute another + // key if that method is malformed or uses a different algorithm. + if (fragment) break; + } + } + throw lastError || new Error("DID document has no compatible verification method"); }; /** * Resolve a keyid to a public key. * * @param {string} keyid - * @param {{ req?: import('express').Request }} options + * @param {{ req?: import('express').Request, algorithm?: string }} options * @returns {Promise<{keyid: string, publicKeyPem: string, algorithm: string, * revoked: boolean, expires: Date|string|null, source: string, key?: object}|null>} * null when the keyid cannot be resolved. */ -const resolveKeyId = async (keyid, { req } = {}) => { +const resolveKeyId = async (keyid, { req, algorithm } = {}) => { if (typeof keyid !== "string" || keyid.length === 0 || keyid.length > 2048) return null; - const local = await resolveLocal(keyid, selfOriginsFor(req)); + let requestedAlgorithm; + try { + requestedAlgorithm = algorithm === undefined ? undefined : normalizeAlgorithm(algorithm); + } catch { + return null; + } + + const local = await resolveLocal(keyid, selfOriginsFor(req), requestedAlgorithm); if (local) return local; if (!REMOTE_ENABLED()) return null; try { if (keyid.startsWith("did:web:")) { - return { keyid, ...(await resolveDidWeb(keyid)) }; + return { keyid, ...(await resolveDidWeb(keyid, requestedAlgorithm)) }; } if (keyid.startsWith("did:")) { // Other DID methods need a method-specific resolver; §8.1 allows an @@ -266,11 +606,17 @@ const resolveKeyId = async (keyid, { req } = {}) => { return null; } if (keyid.startsWith("https://")) { - const document = await fetchJson(keyid); + const { document, mediaType } = await fetchJson(keyid, [ + "application/htmltrust-key+json", + "application/jwk+json", + "application/json", + ]); + const keyMaterial = mediaType === "application/jwk+json" || document?.kty + ? resolveJwk(document, { requestedAlgorithm }) + : resolveHtmlTrustKeyDocument(document, keyid, requestedAlgorithm); return { keyid, - publicKeyPem: pemFromKeyDocument(document), - algorithm: algorithmFromKeyDocument(document), + ...keyMaterial, revoked: document.revoked === true, expires: document.expires || null, source: "https", diff --git a/src/utils/signingProfile.js b/src/utils/signingProfile.js new file mode 100644 index 0000000..f5b3693 --- /dev/null +++ b/src/utils/signingProfile.js @@ -0,0 +1,220 @@ +const crypto = require("crypto"); +const { canonicalizeJcs } = require("./jcs"); +const { + assertContentHash, + decodeCanonicalBase64, + invalid, + normalizeAlgorithm, +} = require("./htmltrustProtocol"); +const { canonicalizeClaims, normalizeClaims } = require("./claims"); + +const PROFILE = Object.freeze({ + signature: "htmltrust-signature-v1", + canonicalization: "htmltrust-c14n-v1", + attributes: "htmltrust-attrs-v1", + url: "htmltrust-safe-url-v1", + context: "https://htmltrust.org/protocol/signed-section", +}); + +const HASH_NAMES = Object.freeze({ + sha256: "sha256", + sha384: "sha384", + sha512: "sha512", +}); + +const EXACT_TIMESTAMP = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})Z$/; +const ASCII_CONTROL = /[\u0000-\u001f\u007f]/; +const ASCII_EDGE_WHITESPACE = /^[\u0009-\u000d\u0020]|[\u0009-\u000d\u0020]$/; + +const assertProtocolString = (value, field) => { + if (typeof value !== "string" || value.length === 0) { + throw invalid(`${field} must be a non-empty string`); + } + if (ASCII_EDGE_WHITESPACE.test(value) || ASCII_CONTROL.test(value)) { + throw invalid(`${field} contains forbidden ASCII whitespace or control characters`); + } + return value; +}; + +const daysInMonth = (year, month) => { + if (month === 2) { + const leap = year % 4 === 0 && (year % 100 !== 0 || year % 400 === 0); + return leap ? 29 : 28; + } + return [4, 6, 9, 11].includes(month) ? 30 : 31; +}; + +const assertV1Timestamp = (value, field = "signedAt") => { + if (typeof value !== "string") { + throw invalid(`${field} must use YYYY-MM-DDTHH:MM:SSZ`); + } + const match = EXACT_TIMESTAMP.exec(value); + if (!match) { + throw invalid(`${field} must use YYYY-MM-DDTHH:MM:SSZ`); + } + const [, yearRaw, monthRaw, dayRaw, hourRaw, minuteRaw, secondRaw] = match; + const year = Number(yearRaw); + const month = Number(monthRaw); + const day = Number(dayRaw); + const hour = Number(hourRaw); + const minute = Number(minuteRaw); + const second = Number(secondRaw); + if ( + year < 1 || month < 1 || month > 12 || day < 1 || + day > daysInMonth(year, month) || hour > 23 || minute > 59 || second > 59 + ) { + throw invalid(`${field} is not a valid Gregorian UTC date and time`); + } + return value; +}; + +const deriveLocation = (sourceURL, scope) => { + assertProtocolString(sourceURL, "sourceURL"); + if (scope !== "url" && scope !== "origin") { + throw invalid("scope must be exactly url or origin"); + } + let parsed; + try { + parsed = new URL(sourceURL); + } catch { + throw invalid("sourceURL must be an absolute URL"); + } + if (parsed.protocol !== "https:" || parsed.username || parsed.password) { + throw invalid("sourceURL must use HTTPS and must not contain credentials"); + } + if (scope === "origin") return parsed.origin; + parsed.hash = ""; + return parsed.href; +}; + +const hashCanonicalClaims = (canonicalClaims, algorithm) => { + const hashName = HASH_NAMES[algorithm]; + if (!hashName) throw invalid(`unsupported content hash algorithm ${algorithm}`); + const digest = crypto + .createHash(hashName) + .update(canonicalClaims, "utf8") + .digest("base64") + .replace(/=+$/, ""); + return `${algorithm}:${digest}`; +}; + +const signingObjectFor = ({ + algorithm, + claimsHash, + contentHash, + keyid, + location, + scope, + signedAt, +}) => ({ + algorithm, + attributeProfile: PROFILE.attributes, + canonicalizationProfile: PROFILE.canonicalization, + claimsHash, + contentHash, + context: PROFILE.context, + keyid, + location, + profile: PROFILE.signature, + scope, + signedAt, + urlProfile: PROFILE.url, +}); + +const assertCanonicalAlgorithm = (algorithm) => { + assertProtocolString(algorithm, "algorithm"); + const normalized = normalizeAlgorithm(algorithm); + if (normalized !== algorithm) { + throw invalid(`algorithm must use the canonical identifier ${normalized}`); + } + return algorithm; +}; + +const assertSignatureShape = (signature, algorithm) => { + const decoded = decodeCanonicalBase64(signature, "signature"); + const exactLengths = { + ed25519: 64, + "ecdsa-p256": 64, + "ecdsa-p384": 96, + }; + if (exactLengths[algorithm] && decoded.length !== exactLengths[algorithm]) { + throw invalid(`signature must decode to ${exactLengths[algorithm]} bytes for ${algorithm}`); + } + return signature; +}; + +/** + * Validate a POST /content body and construct the exact v1 signing payload. + */ +const validateV1ContentSubmission = async (body) => { + if (!body || typeof body !== "object" || Array.isArray(body)) { + throw invalid("the content submission must be a JSON object"); + } + if (Object.hasOwn(body, "claimsHash")) { + throw invalid("claimsHash is computed by the directory and must not be submitted"); + } + if (body.profile !== PROFILE.signature) { + throw invalid(`profile must be exactly ${PROFILE.signature}`); + } + + const contentHash = assertContentHash(body.contentHash, "contentHash"); + const hashAlgorithm = contentHash.slice(0, contentHash.indexOf(":")); + const algorithm = assertCanonicalAlgorithm(body.algorithm); + const keyid = assertProtocolString(body.keyid, "keyid"); + if (Buffer.byteLength(keyid, "utf8") > 2048) { + throw invalid("keyid must be 2048 bytes or fewer"); + } + const signedAt = assertV1Timestamp(body.signedAt, "signedAt"); + const scope = body.scope; + const location = deriveLocation(body.sourceURL, scope); + if (body.location !== location) { + throw invalid(`location must equal the ${scope} location derived from sourceURL`); + } + const signature = assertSignatureShape(body.signature, algorithm); + + const normalizedClaims = await normalizeClaims(body.claims, { strictArray: true }); + const signedAtClaims = normalizedClaims.filter((claim) => claim.name === "signed-at"); + if (signedAtClaims.length !== 1) { + throw invalid("claims must contain exactly one signed-at record"); + } + assertV1Timestamp(signedAtClaims[0].content, "signed-at claim"); + if (signedAtClaims[0].content !== signedAt) { + throw invalid("the normalized signed-at claim must equal signedAt"); + } + + const canonicalClaims = await canonicalizeClaims(body.claims, { strictArray: true }); + const claimsHash = hashCanonicalClaims(canonicalClaims, hashAlgorithm); + const signingObject = signingObjectFor({ + algorithm, + claimsHash, + contentHash, + keyid, + location, + scope, + signedAt, + }); + return { + algorithm, + canonicalClaims, + claims: normalizedClaims, + claimsHash, + contentHash, + keyid, + location, + payload: canonicalizeJcs(signingObject), + profile: PROFILE.signature, + scope, + signature, + signedAt, + sourceURL: body.sourceURL, + }; +}; + +module.exports = { + assertV1Timestamp, + deriveLocation, + hashCanonicalClaims, + PROFILE, + signingObjectFor, + validateV1ContentSubmission, +}; diff --git a/test/canonicalRoutes.test.js b/test/canonicalRoutes.test.js new file mode 100644 index 0000000..64d7cda --- /dev/null +++ b/test/canonicalRoutes.test.js @@ -0,0 +1,15 @@ +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); + +const serverSource = fs.readFileSync(require.resolve('../src/server'), 'utf8'); +const endorsementSource = fs.readFileSync(require.resolve('../src/controllers/endorsementController'), 'utf8'); + +test('canonical endorsement creation returns a canonical resource Location', () => { + assert.match(endorsementSource, /\.location\(`\/endorsements\/\$\{stored\._id\}`\)/); + assert.doesNotMatch(endorsementSource, /\.location\(`\/api\/endorsements\/\$\{stored\._id\}`\)/); +}); + +test('canonical endorsement deletion is registered', () => { + assert.match(serverSource, /app\.delete\(\s*['"]\/endorsements\/:id['"]/s); +}); diff --git a/test/contentNegotiation.test.js b/test/contentNegotiation.test.js new file mode 100644 index 0000000..1f760df --- /dev/null +++ b/test/contentNegotiation.test.js @@ -0,0 +1,67 @@ +const test = require('node:test'); +const assert = require('node:assert/strict'); +const { negotiate } = require('../src/middleware/contentNegotiation'); + +const response = () => { + const headers = {}; + const res = { + headers, + statusCode: 200, + vary(name) { headers.Vary = name; return this; }, + set(name, value) { headers[name] = value; return this; }, + status(code) { this.statusCode = code; return this; }, + type(value) { headers['Content-Type'] = value; return this; }, + json(value) { this.body = value; return this; }, + }; + return res; +}; + +const request = (accept, contentType) => ({ + headers: { + ...(accept === undefined ? {} : { accept }), + ...(contentType === undefined ? {} : { 'content-type': contentType }), + }, + accepts(types) { + if (!accept) return types[0]; + return types.find((type) => accept === '*/*' || accept.split(',').map((part) => part.trim()).includes(type)); + }, +}); + +test('canonical reads advertise cache semantics and vary on Accept', () => { + const req = request('application/json'); + const res = response(); + let called = false; + negotiate('application/htmltrust-content+json')(req, res, () => { called = true; }); + + assert.equal(called, true); + assert.equal(req.htmltrustResponseType, 'application/json'); + assert.equal(res.headers.Vary, 'Accept'); + assert.equal(res.headers['Cache-Control'], 'public, max-age=60, must-revalidate'); +}); + +test('canonical responses retain the vendor media type by default', () => { + const req = request(undefined); + const res = response(); + negotiate('application/htmltrust-content+json')(req, res, () => {}); + + assert.equal(req.htmltrustResponseType, 'application/htmltrust-content+json'); +}); + +test('canonical endpoints reject an unacceptable representation', () => { + const req = request('text/html'); + const res = response(); + negotiate('application/htmltrust-content+json')(req, res, () => {}); + + assert.equal(res.statusCode, 406); + assert.equal(res.body.type, 'https://htmltrust.org/errors/not-acceptable'); + assert.equal(res.headers.Vary, 'Accept'); +}); + +test('canonical submissions require a JSON representation', () => { + const req = request('application/json', 'text/plain'); + const res = response(); + negotiate('application/htmltrust-content+json', { requestBody: true })(req, res, () => {}); + + assert.equal(res.statusCode, 415); + assert.equal(res.body.type, 'https://htmltrust.org/errors/unsupported-media-type'); +}); diff --git a/test/directoryUrl.test.js b/test/directoryUrl.test.js index 7ba17d3..d3551cc 100644 --- a/test/directoryUrl.test.js +++ b/test/directoryUrl.test.js @@ -12,13 +12,13 @@ const request = { test('directory URLs fall back to the request origin', () => { assert.equal(directoryBaseUrl(request, {}), 'http://localhost:3000'); - assert.equal(directoryKeyUrl(request, 'key 1', {}), 'http://localhost:3000/api/keys/key%201'); + assert.equal(directoryKeyUrl(request, 'key 1', {}), 'http://localhost:3000/keys/key%201'); }); test('directory URLs use the configured public base URL', () => { const env = { DIRECTORY_BASE_URL: 'https://directory.example/' }; assert.equal(directoryBaseUrl(request, env), 'https://directory.example'); - assert.equal(directoryKeyUrl(request, 'abc', env), 'https://directory.example/api/keys/abc'); + assert.equal(directoryKeyUrl(request, 'abc', env), 'https://directory.example/keys/abc'); }); test('directory URLs reject unsupported configured schemes', () => { diff --git a/test/httpSignature.test.js b/test/httpSignature.test.js index 7651d97..a9057e8 100644 --- a/test/httpSignature.test.js +++ b/test/httpSignature.test.js @@ -33,21 +33,41 @@ const digestOf = (body) => `sha-256=:${crypto.createHash("sha256").update(Buffer.from(body, "utf8")).digest("base64")}:`; /** Sign a request the way a conforming client would. */ -const signRequest = ({ body = '{"a":1}', components, keyid = "test-key", created = Math.floor(Date.now() / 1000), date = new Date().toUTCString() } = {}) => { +const signRequest = ({ + body = '{"a":1}', + components, + keyid = "test-key", + created = Math.floor(Date.now() / 1000), + date = new Date().toUTCString(), + alg = "ed25519", + label = "sig1", + nonce, + padded = false, + bareKeyid = false, + bareAlg = false, + bareNonce = false, +} = {}) => { const covered = components || ["@method", "@target-uri", "host", "date", "content-digest"]; - const params = `(${covered.map((c) => `"${c}"`).join(" ")});created=${created};keyid="${keyid}"`; + const parameters = [ + `created=${created}`, + `keyid=${bareKeyid ? keyid : `"${keyid}"`}`, + ...(alg === null ? [] : [`alg=${bareAlg ? alg : `"${alg}"`}`]), + ...(nonce === undefined ? [] : [`nonce=${bareNonce ? nonce : `"${nonce}"`}`]), + ]; + const params = `(${covered.map((c) => `"${c}"`).join(" ")});${parameters.join(";")}`; const req = makeRequest({ body, headers: { host: "directory.example", date, "content-digest": digestOf(body), - "signature-input": `sig1=${params}`, + "signature-input": `${label}=${params}`, }, }); const base = buildSignatureBase(req, covered, params); - const signature = crypto.sign(null, Buffer.from(base, "utf8"), privateKey).toString("base64"); - req.headers.signature = `sig1=:${signature}:`; + let signature = crypto.sign(null, Buffer.from(base, "utf8"), privateKey).toString("base64"); + if (!padded) signature = signature.replace(/=+$/, ""); + req.headers.signature = `${label}=:${signature}:`; return req; }; @@ -76,6 +96,126 @@ test("accepts a correctly signed request", async () => { assert.equal(result.actor.keyid, "test-key"); }); +test("verifies content-digest for an explicitly captured empty raw body", async () => { + const req = signRequest({ body: "", nonce: "empty-body" }); + const result = await verifyHttpMessageSignature(req, { resolve }); + assert.equal(result.ok, true); + + const wrongDigest = signRequest({ body: "", nonce: "empty-body-wrong-digest" }); + wrongDigest.headers["content-digest"] = digestOf("different bytes"); + const rejected = await verifyHttpMessageSignature(wrongDigest, { resolve }); + assert.equal(rejected.ok, false); + assert.match(rejected.detail, /content-digest/); + + const uncaptured = signRequest({ body: "", nonce: "empty-body-not-captured" }); + uncaptured.rawBody = undefined; + uncaptured.headers["content-digest"] = digestOf("different bytes"); + const uncapturedRejected = await verifyHttpMessageSignature(uncaptured, { resolve }); + assert.equal(uncapturedRejected.ok, false); + assert.match(uncapturedRejected.detail, /content-digest/); +}); + +test("strict v1 accepts the exact sig1 request profile", async () => { + let requestedAlgorithm; + const result = await verifyHttpMessageSignature( + signRequest({ body: '{"strict":true}', nonce: "request-1" }), + { + resolve: async (keyid, options) => { + requestedAlgorithm = options.algorithm; + return resolve(keyid, options); + }, + strictV1: true, + }, + ); + assert.equal(result.ok, true); + assert.equal(requestedAlgorithm, "ed25519"); +}); + +test("strict v1 rejects a missing alg parameter", async () => { + const result = await verifyHttpMessageSignature( + signRequest({ body: '{"missing":"alg"}', alg: null }), + { resolve, strictV1: true }, + ); + assert.equal(result.ok, false); + assert.match(result.detail, /alg/); +}); + +test("strict v1 enforces structured-field parameter types", async () => { + for (const options of [ + { body: '{"bare":"keyid"}', bareKeyid: true }, + { body: '{"bare":"alg"}', bareAlg: true }, + { body: '{"bare":"nonce"}', nonce: "request-2", bareNonce: true }, + ]) { + const result = await verifyHttpMessageSignature(signRequest(options), { + resolve, + strictV1: true, + }); + assert.equal(result.ok, false); + assert.match(result.detail, /keyid|alg|nonce/); + } +}); + +test("rejects a Date field that is not an IMF-fixdate HTTP date", async () => { + const result = await verifyHttpMessageSignature( + signRequest({ body: '{"date":"iso"}', date: new Date().toISOString() }), + { resolve, strictV1: true }, + ); + assert.equal(result.ok, false); + assert.match(result.detail, /IMF-fixdate/); +}); + +test("strict v1 rejects reordered, aliased, or additional components", async () => { + const cases = [ + ["@target-uri", "@method", "host", "date", "content-digest"], + ["@method", "@path", "host", "date", "content-digest"], + ["@method", "@target-uri", "host", "date", "content-digest", "@scheme"], + ]; + for (let index = 0; index < cases.length; index += 1) { + const result = await verifyHttpMessageSignature( + signRequest({ body: `{"case":${index}}`, components: cases[index] }), + { resolve, strictV1: true }, + ); + assert.equal(result.ok, false); + assert.match(result.detail, /exactly these covered components/); + } +}); + +test("strict v1 rejects labels other than sig1 and padded signatures", async () => { + const wrongLabel = await verifyHttpMessageSignature( + signRequest({ body: '{"label":false}', label: "other" }), + { resolve, strictV1: true }, + ); + assert.equal(wrongLabel.ok, false); + assert.match(wrongLabel.detail, /sig1/); + + const padded = await verifyHttpMessageSignature( + signRequest({ body: '{"padding":true}', padded: true }), + { resolve, strictV1: true }, + ); + assert.equal(padded.ok, false); + assert.match(padded.detail, /unpadded Base64/); +}); + +test("strict v1 rejects a noncanonical Base64 spelling with altered pad bits", async () => { + const req = signRequest({ body: '{"padBits":true}' }); + const match = /^sig1=:([A-Za-z0-9+/]+):$/.exec(req.headers.signature); + assert.ok(match); + const alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + const encoded = match[1]; + const lastIndex = alphabet.indexOf(encoded.at(-1)); + assert.equal(lastIndex % 16, 0); + const alternate = encoded.slice(0, -1) + alphabet[lastIndex + 1]; + assert.deepEqual( + Buffer.from(alternate, "base64"), + Buffer.from(encoded, "base64"), + ); + req.headers.signature = `sig1=:${alternate}:`; + + const result = await verifyHttpMessageSignature(req, { resolve, strictV1: true }); + assert.equal(result.ok, false); + assert.match(result.detail, /canonical unpadded Base64/); +}); + test("rejects a request whose body was swapped after signing", async () => { const req = signRequest({ body: '{"a":1}' }); // Same headers, different body: the content-digest no longer matches. diff --git a/test/indexes.test.js b/test/indexes.test.js new file mode 100644 index 0000000..b99986f --- /dev/null +++ b/test/indexes.test.js @@ -0,0 +1,30 @@ +const test = require('node:test'); +const assert = require('node:assert/strict'); +const ContentSignature = require('../src/models/ContentSignature'); +const Endorsement = require('../src/models/Endorsement'); +const { LEGACY_INDEXES } = require('../scripts/migrate-v1-indexes'); + +test('v1 content identity uses a partial unique index', () => { + const index = ContentSignature.schema.indexes().find(([keys]) => + keys.contentHash === 1 && keys.profile === 1 && keys.location === 1 && keys.keyid === 1); + assert.ok(index, 'v1 identity index is declared'); + assert.equal(index[1].unique, true); + assert.deepEqual(index[1].partialFilterExpression, { profile: 'htmltrust-signature-v1' }); + const legacy = ContentSignature.schema.indexes().find(([keys]) => + keys.contentHash === 1 && keys.domain === 1 && keys.authorId === 1); + assert.deepEqual(legacy[1].partialFilterExpression.profile, { $in: [null] }); +}); + +test('pre-v1 index names are covered by the explicit migration', () => { + const names = LEGACY_INDEXES.map(([, name]) => name); + assert.deepEqual(names, [ + 'contentHash_1_domain_1_authorId_1', + 'contentHash_1_endorser_1', + 'endorsement_1_endorser_1', + ]); + const endorsementIndexes = Endorsement.schema.indexes(); + assert.ok(endorsementIndexes.some(([keys, options]) => + keys.contentHash === 1 && keys.endorser === 1 && options?.unique !== true)); + assert.ok(endorsementIndexes.some(([keys, options]) => + keys.endorsement === 1 && keys.endorser === 1 && options?.unique !== true)); +}); diff --git a/test/jcs.test.js b/test/jcs.test.js index f6ea153..62195c4 100644 --- a/test/jcs.test.js +++ b/test/jcs.test.js @@ -47,8 +47,12 @@ test("JCS escapes strings exactly like JSON.stringify", () => { assert.equal(canonicalizeJcs({ s: value }), `{"s":${JSON.stringify(value)}}`); assert.equal(canonicalizeJcs(""), '"\\u000f"'); assert.equal(canonicalizeJcs("\n"), '"\\n"'); - // ES2019 well-formed stringify: a lone surrogate is escaped, not emitted raw. - assert.equal(canonicalizeJcs("\ud800"), '"\\ud800"'); +}); + +test("JCS rejects lone UTF-16 surrogates", () => { + assert.throws(() => canonicalizeJcs("\ud800"), /surrogate/); + assert.throws(() => canonicalizeJcs("\udfff"), /surrogate/); + assert.throws(() => canonicalizeJcs({ "\ud800": 1 }), /surrogate/); }); test("JCS emits no insignificant whitespace and preserves array order", () => { @@ -61,17 +65,10 @@ test("JCS is idempotent through a JSON round trip", () => { assert.equal(canonicalizeJcs(JSON.parse(once)), once); }); -test("JCS never emits the token `undefined`", () => { - // The hand-rolled canonicalizer this replaces serialized an undefined - // member as the bare token `undefined`, producing bytes no other - // implementation could reproduce (and that are not JSON at all). - const got = canonicalizeJcs({ a: 1, b: undefined }); - assert.equal(got, '{"a":1}'); - assert.ok(!got.includes("undefined")); -}); - test("JCS rejects values with no JSON representation", () => { assert.throws(() => canonicalizeJcs({ a: Infinity })); assert.throws(() => canonicalizeJcs({ a: NaN })); assert.throws(() => canonicalizeJcs({ a: 1n })); + assert.throws(() => canonicalizeJcs({ a: undefined })); + assert.throws(() => canonicalizeJcs([undefined])); }); diff --git a/test/keyDocument.test.js b/test/keyDocument.test.js new file mode 100644 index 0000000..1ee14cb --- /dev/null +++ b/test/keyDocument.test.js @@ -0,0 +1,41 @@ +const test = require("node:test"); +const assert = require("node:assert/strict"); +const crypto = require("node:crypto"); +const { keyDocumentFor } = require("../src/utils/htmltrustProtocol"); + +test("directory key documents expose canonical SPKI DER and lifecycle metadata", () => { + const { publicKey } = crypto.generateKeyPairSync("ed25519", { + publicKeyEncoding: { type: "spki", format: "pem" }, + privateKeyEncoding: { type: "pkcs8", format: "pem" }, + }); + const key = { + _id: "abc123", + publicKey, + algorithm: "ed25519", + revoked: true, + revokedAt: new Date("2026-08-01T00:00:00Z"), + expiresAt: new Date("2026-12-31T00:00:00Z"), + supersededBy: "https://directory.example/keys/new", + previousKeys: ["https://directory.example/keys/older"], + }; + const kid = "https://directory.example/keys/abc123"; + const document = keyDocumentFor(key, kid); + + assert.equal(document.kid, kid); + assert.equal(document.algorithm, "ed25519"); + assert.equal(document.publicKeyEncoding, "spki-der"); + assert.equal(document.revoked, true); + assert.equal(document.revokedAt, "2026-08-01T00:00:00.000Z"); + assert.equal(document.expires, "2026-12-31T00:00:00.000Z"); + assert.equal(document.supersededBy, "https://directory.example/keys/new"); + assert.deepEqual(document.previousKeys, ["https://directory.example/keys/older"]); + assert.equal(document.publicKeyPem, undefined); + assert.ok(!document.publicKey.includes("=")); + + const decoded = Buffer.from(document.publicKey, "base64"); + assert.equal( + crypto.createPublicKey({ key: decoded, format: "der", type: "spki" }) + .export({ type: "spki", format: "pem" }), + publicKey, + ); +}); diff --git a/test/keyResolution.test.js b/test/keyResolution.test.js index e3abba0..49b7e0c 100644 --- a/test/keyResolution.test.js +++ b/test/keyResolution.test.js @@ -1,35 +1,417 @@ const test = require("node:test"); const assert = require("node:assert/strict"); +const https = require("node:https"); +const dns = require("node:dns").promises; +const { EventEmitter } = require("node:events"); +const { Readable } = require("node:stream"); const { resolveKeyId } = require("../src/utils/keyResolution"); +const mockHttps = ({ body, status = 200, headers = {}, onRequest } = {}) => { + const previous = https.request; + let destroyed = false; + https.request = (options, callback) => { + onRequest?.(options); + const request = new EventEmitter(); + request.setTimeout = () => {}; + request.destroy = (error) => { + destroyed = true; + if (error) process.nextTick(() => request.emit("error", error)); + }; + request.end = () => process.nextTick(() => { + const response = new Readable({ read() {} }); + response.statusCode = typeof status === "function" ? status() : status; + response.headers = typeof headers === "function" ? headers() : headers; + callback(response); + if (!destroyed) { + const value = typeof body === "function" ? body() : body; + response.push(Buffer.isBuffer(value) ? value : Buffer.from(String(value ?? ""))); + response.push(null); + } + }); + return request; + }; + return { restore: () => { https.request = previous; }, wasDestroyed: () => destroyed }; +}; + test("remote key resolution stops reading after the 64 KiB limit", async () => { - const previousFetch = global.fetch; const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; const previousConsoleError = console.error; - let pulls = 0; - let cancelled = false; - const stream = new ReadableStream({ - pull(controller) { - pulls += 1; - controller.enqueue(new Uint8Array(32 * 1024)); - }, - cancel() { - cancelled = true; - }, + const requestMock = mockHttps({ + body: Buffer.alloc(64 * 1024 + 1), + headers: { "content-type": "application/htmltrust-key+json" }, }); - global.fetch = async () => new Response(stream, { status: 200 }); console.error = () => {}; process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; try { const resolved = await resolveKeyId("https://93.184.216.34/key.json"); assert.equal(resolved, null); - assert.ok(pulls <= 4, `expected bounded streaming reads, read ${pulls} chunks`); - assert.equal(cancelled, true); + assert.equal(requestMock.wasDestroyed(), true); + } finally { + requestMock.restore(); + console.error = previousConsoleError; + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("remote key resolution rejects IPv4-mapped loopback literals", async () => { + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + let fetched = false; + const requestMock = mockHttps({ + body: "{}", + headers: { "content-type": "application/json" }, + onRequest: () => { + fetched = true; + }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + try { + assert.equal(await resolveKeyId("https://[::ffff:127.0.0.1]/key.json"), null); + assert.equal(fetched, false); + } finally { + requestMock.restore(); + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("remote key resolution rejects special-use IPv4 and IPv6 ranges", async () => { + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const previousConsoleError = console.error; + const requestMock = mockHttps({ + body: "{}", + headers: { "content-type": "application/json" }, + onRequest: () => assert.fail("a special-use address must not be requested"), + }); + console.error = () => {}; + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + try { + for (const keyid of [ + "https://100.64.0.1/key.json", + "https://198.18.0.1/key.json", + "https://[fc00::1]/key.json", + "https://[fe80::1]/key.json", + "https://[2001:db8::1]/key.json", + "https://[ff02::1]/key.json", + ]) { + assert.equal(await resolveKeyId(keyid), null, keyid); + } + } finally { + requestMock.restore(); + console.error = previousConsoleError; + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("direct HTTPS key documents enforce media type, kid, SPKI encoding, and lifecycle fields", async () => { + const crypto = require("node:crypto"); + const { publicKey } = crypto.generateKeyPairSync("ed25519"); + const der = publicKey.export({ type: "spki", format: "der" }); + const encoded = der.toString("base64").replace(/=+$/, ""); + const keyid = "https://93.184.216.34/key.json"; + const valid = { + kid: keyid, + algorithm: "ed25519", + publicKeyEncoding: "spki-der", + publicKey: encoded, + revoked: false, + }; + + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const previousConsoleError = console.error; + let document = valid; + let contentType = "application/htmltrust-key+json"; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + headers: () => ({ "content-type": contentType }), + }); + console.error = () => {}; + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + + try { + const resolved = await resolveKeyId(keyid); + assert.equal(resolved.algorithm, "ed25519"); + assert.match(resolved.publicKeyPem, /BEGIN PUBLIC KEY/); + + for (const invalidDocument of [ + { ...valid, kid: "https://example.net/other.json" }, + { ...valid, publicKeyEncoding: "pem", publicKey: publicKey.export({ type: "spki", format: "pem" }) }, + { ...valid, publicKey: `${encoded}=` }, + { ...valid, expires: "tomorrow" }, + { ...valid, revoked: "false" }, + ]) { + document = invalidDocument; + assert.equal(await resolveKeyId(keyid), null); + } + + document = valid; + contentType = "text/plain"; + assert.equal(await resolveKeyId(keyid), null); + + const jwk = publicKey.export({ format: "jwk" }); + contentType = "application/jwk+json"; + document = jwk; + assert.equal(await resolveKeyId(keyid), null, "a direct JWK without alg must fail"); + document = { ...jwk, alg: "EdDSA" }; + assert.equal((await resolveKeyId(keyid)).algorithm, "ed25519"); + } finally { + requestMock.restore(); + console.error = previousConsoleError; + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("did:web resolution selects the verification method named by a fragment", async () => { + const crypto = require("node:crypto"); + const { publicKey: first } = crypto.generateKeyPairSync("ed25519"); + const { publicKey: selected } = crypto.generateKeyPairSync("ed25519"); + const did = "did:web:93.184.216.34#key-2"; + const document = { + id: "did:web:93.184.216.34", + verificationMethod: [ + { id: "did:web:93.184.216.34#key-1", type: "JsonWebKey2020", publicKeyJwk: first.export({ format: "jwk" }) }, + { id: did, type: "JsonWebKey2020", publicKeyJwk: selected.export({ format: "jwk" }) }, + ], + assertionMethod: [did], + }; + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + onRequest: (options) => { + assert.equal(options.hostname, "93.184.216.34"); + assert.equal(options.path, "/.well-known/did.json"); + }, + headers: { "content-type": "application/did+json" }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + try { + const resolved = await resolveKeyId(did); + assert.equal( + crypto.createPublicKey(resolved.publicKeyPem).export({ type: "spki", format: "der" }).toString("hex"), + selected.export({ type: "spki", format: "der" }).toString("hex"), + ); + } finally { + requestMock.restore(); + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("did:web resolution selects a verification method compatible with the declared algorithm", async () => { + const crypto = require("node:crypto"); + const { publicKey: ed25519 } = crypto.generateKeyPairSync("ed25519"); + const { publicKey: p256 } = crypto.generateKeyPairSync("ec", { namedCurve: "prime256v1" }); + const did = "did:web:93.184.216.34"; + const document = { + verificationMethod: [ + { id: `${did}#ed25519`, type: "JsonWebKey2020", publicKeyJwk: { ...ed25519.export({ format: "jwk" }), alg: "EdDSA" } }, + { id: `${did}#p256`, type: "JsonWebKey2020", publicKeyJwk: { ...p256.export({ format: "jwk" }), alg: "ES256" } }, + ], + assertionMethod: [`${did}#p256`], + }; + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + headers: { "content-type": "application/did+json" }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + try { + const resolved = await resolveKeyId(did, { algorithm: "ecdsa-p256" }); + assert.equal(resolved.algorithm, "ecdsa-p256"); + assert.deepEqual( + crypto.createPublicKey(resolved.publicKeyPem).export({ type: "spki", format: "der" }), + p256.export({ type: "spki", format: "der" }), + ); + } finally { + requestMock.restore(); + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("did:web resolution rejects a verification method whose type disagrees with its key", async () => { + const crypto = require("node:crypto"); + const { publicKey } = crypto.generateKeyPairSync("ec", { namedCurve: "prime256v1" }); + const did = "did:web:93.184.216.34"; + const keyid = `${did}#wrong-type`; + const document = { + verificationMethod: [{ + id: keyid, + type: "Ed25519VerificationKey2020", + publicKeyJwk: { ...publicKey.export({ format: "jwk" }), alg: "ES256" }, + }], + assertionMethod: [keyid], + }; + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const previousConsoleError = console.error; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + headers: { "content-type": "application/did+json" }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + console.error = () => {}; + try { + assert.equal(await resolveKeyId(did, { algorithm: "ecdsa-p256" }), null); + delete document.verificationMethod[0].type; + assert.equal( + await resolveKeyId(did, { algorithm: "ecdsa-p256" }), + null, + "a DID verification method without a type must fail", + ); } finally { - global.fetch = previousFetch; + requestMock.restore(); console.error = previousConsoleError; if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; } }); + +test("did:web resolution rejects revoked and expired assertion methods", async () => { + const crypto = require("node:crypto"); + const { publicKey: revoked } = crypto.generateKeyPairSync("ed25519"); + const { publicKey: expired } = crypto.generateKeyPairSync("ed25519"); + const did = "did:web:93.184.216.34"; + const revokedId = `${did}#revoked`; + const expiredId = `${did}#expired`; + const document = { + verificationMethod: [ + { + id: revokedId, + type: "Ed25519VerificationKey2020", + publicKeyJwk: revoked.export({ format: "jwk" }), + revoked: true, + }, + { + id: expiredId, + type: "Ed25519VerificationKey2020", + publicKeyJwk: expired.export({ format: "jwk" }), + expires: "2020-01-01T00:00:00Z", + }, + ], + assertionMethod: [revokedId, expiredId], + }; + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const previousConsoleError = console.error; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + headers: { "content-type": "application/did+json" }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + console.error = () => {}; + try { + assert.equal(await resolveKeyId(did, { algorithm: "ed25519" }), null); + } finally { + requestMock.restore(); + console.error = previousConsoleError; + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("did:web resolution accepts an embedded assertion verification method", async () => { + const crypto = require("node:crypto"); + const { publicKey } = crypto.generateKeyPairSync("ed25519"); + const did = "did:web:93.184.216.34"; + const embedded = { + id: `${did}#embedded`, + type: "Ed25519VerificationKey2020", + publicKeyJwk: publicKey.export({ format: "jwk" }), + }; + const document = { verificationMethod: [], assertionMethod: [embedded] }; + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + headers: { "content-type": "application/did+json" }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + try { + const resolved = await resolveKeyId(did, { algorithm: "ed25519" }); + assert.equal(resolved.algorithm, "ed25519"); + assert.deepEqual( + crypto.createPublicKey(resolved.publicKeyPem).export({ type: "spki", format: "der" }), + publicKey.export({ type: "spki", format: "der" }), + ); + } finally { + requestMock.restore(); + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("did:web resolution never falls back to an unauthorized compatible key", async () => { + const crypto = require("node:crypto"); + const { publicKey: authorized } = crypto.generateKeyPairSync("ed25519"); + const { publicKey: unauthorized } = crypto.generateKeyPairSync("ec", { namedCurve: "prime256v1" }); + const did = "did:web:93.184.216.34"; + const authorizedId = `${did}#authorized-ed25519`; + const document = { + verificationMethod: [ + { + id: authorizedId, + type: "Ed25519VerificationKey2020", + publicKeyJwk: { ...authorized.export({ format: "jwk" }), alg: "EdDSA" }, + }, + { + id: `${did}#unauthorized-p256`, + type: "EcdsaSecp256r1VerificationKey2019", + publicKeyJwk: { ...unauthorized.export({ format: "jwk" }), alg: "ES256" }, + }, + ], + assertionMethod: [authorizedId], + }; + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const previousConsoleError = console.error; + const requestMock = mockHttps({ + body: () => JSON.stringify(document), + headers: { "content-type": "application/did+json" }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + console.error = () => {}; + try { + assert.equal(await resolveKeyId(did, { algorithm: "ecdsa-p256" }), null); + } finally { + requestMock.restore(); + console.error = previousConsoleError; + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); + +test("remote key resolution pins the validated DNS address for the HTTPS request", async () => { + const previousEnabled = process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + const previousLookup = dns.lookup; + const pinnedAddress = "93.184.216.34"; + let lookups = 0; + dns.lookup = async () => { + lookups += 1; + return [{ address: pinnedAddress, family: 4 }]; + }; + const requestMock = mockHttps({ + body: "{}", + headers: { "content-type": "application/json" }, + onRequest: (options) => { + options.lookup(options.hostname, {}, (error, address, family) => { + assert.ifError(error); + assert.equal(address, pinnedAddress); + assert.equal(family, 4); + }); + options.lookup(options.hostname, { all: true }, (error, addresses) => { + assert.ifError(error); + assert.deepEqual(addresses, [{ address: pinnedAddress, family: 4 }]); + }); + }, + }); + process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = "1"; + try { + assert.equal(await resolveKeyId("https://rebind.example.test/key.json"), null); + assert.equal(lookups, 1); + } finally { + requestMock.restore(); + dns.lookup = previousLookup; + if (previousEnabled === undefined) delete process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION; + else process.env.HTMLTRUST_REMOTE_KEY_RESOLUTION = previousEnabled; + } +}); diff --git a/test/protocol.test.js b/test/protocol.test.js index bf914c9..2571330 100644 --- a/test/protocol.test.js +++ b/test/protocol.test.js @@ -5,6 +5,7 @@ const { safeSearchRegex, } = require("../src/utils/htmltrustProtocol"); const { canonicalizeClaims } = require("../src/utils/claims"); +const { validateEndorsementDocument } = require("../src/controllers/endorsementController"); test("safeSearchRegex turns caller input into a literal", () => { // Every one of these is regular-expression syntax that used to reach the @@ -56,6 +57,22 @@ test("the endorsement payload omits only the signature", () => { ); }); +test("canonical endorsement validation preserves every extension member", () => { + const document = { + endorser: "did:web:reviewer.example", + endorsement: `sha256:${Buffer.alloc(32).toString("base64").replace(/=+$/, "")}`, + algorithm: "ed25519", + timestamp: "2026-05-10T09:00:00Z", + signature: Buffer.alloc(64).toString("base64").replace(/=+$/, ""), + rawBlob: { extension: true }, + }; + assert.deepEqual(validateEndorsementDocument(document).rawBlob, { extension: true }); + assert.equal( + Object.hasOwn(validateEndorsementDocument(document, { stripLegacyRawBlob: true }), "rawBlob"), + false, + ); +}); + test("claims canonicalization sorts by name in UTF-8 byte order", async () => { // "z" (U+007A) sorts before the astral character in UTF-8 byte order, and // the whole-line sort the previous implementation used would have ordered @@ -65,7 +82,7 @@ test("claims canonicalization sorts by name in UTF-8 byte order", async () => { a: "zzz", "signed-at": "2026-05-12T12:00:00Z", }); - assert.equal(canonical, "a:zzz\nsigned-at:2026-05-12T12:00:00Z\nz:aaa\n"); + assert.equal(canonical, "a:zzz\nsigned-at:2026-05-12T12\\:00\\:00Z\nz:aaa\n"); }); test("claims canonicalization normalizes claim text", async () => { @@ -75,6 +92,11 @@ test("claims canonicalization normalizes claim text", async () => { assert.equal(canonical, "License:CC-BY 4.0\n"); }); +test("claims canonicalization removes boundary whitespace from names and values", async () => { + const canonical = await canonicalizeClaims([{ name: " author ", content: " Ada Lovelace " }]); + assert.equal(canonical, "author:Ada Lovelace\n"); +}); + test("claims canonicalization rejects duplicate normalized names", async () => { await assert.rejects( () => canonicalizeClaims([{ name: "a", content: "1" }, { name: "a", content: "2" }]), diff --git a/test/serverBodyParsing.test.js b/test/serverBodyParsing.test.js new file mode 100644 index 0000000..445d7dd --- /dev/null +++ b/test/serverBodyParsing.test.js @@ -0,0 +1,49 @@ +const test = require('node:test'); +const assert = require('node:assert/strict'); +const express = require('express'); +const fs = require('node:fs'); + +const serverSource = fs.readFileSync(require.resolve('../src/server'), 'utf8'); + +const postJson = async (contentType, body) => { + const app = express(); + app.use(express.json({ + type: ['application/json', 'application/*+json'], + verify: (req, res, rawBody) => { + req.rawBody = rawBody; + }, + })); + app.post('/', (req, res) => res.json({ body: req.body, rawBody: req.rawBody.toString('utf8') })); + + const server = app.listen(0); + try { + const { port } = server.address(); + const response = await fetch(`http://127.0.0.1:${port}/`, { + method: 'POST', + headers: { 'content-type': contentType }, + body, + }); + return { status: response.status, body: await response.json() }; + } finally { + await new Promise((resolve) => server.close(resolve)); + } +}; + +test('server JSON body parsing accepts vendor +json media types', async () => { + assert.match( + serverSource, + /type:\s*\[\s*['"]application\/json['"]\s*,\s*['"]application\/\*\+json['"]\s*\]/, + ); + + const response = await postJson('application/htmltrust-content+json', '{"contentHash":"sha256:test"}'); + assert.equal(response.status, 200); + assert.deepEqual(response.body.body, { contentHash: 'sha256:test' }); + assert.equal(response.body.rawBody, '{"contentHash":"sha256:test"}'); +}); + +test('server JSON body parsing preserves application/json behavior', async () => { + const response = await postJson('application/json', '{"contentHash":"sha256:compat"}'); + assert.equal(response.status, 200); + assert.deepEqual(response.body.body, { contentHash: 'sha256:compat' }); + assert.equal(response.body.rawBody, '{"contentHash":"sha256:compat"}'); +}); diff --git a/test/signingProfile.test.js b/test/signingProfile.test.js new file mode 100644 index 0000000..6f42ebb --- /dev/null +++ b/test/signingProfile.test.js @@ -0,0 +1,127 @@ +const test = require("node:test"); +const assert = require("node:assert/strict"); +const { + assertV1Timestamp, + deriveLocation, + PROFILE, + validateV1ContentSubmission, +} = require("../src/utils/signingProfile"); +const { canonicalizeClaims } = require("../src/utils/claims"); + +const vectorSubmission = () => ({ + profile: PROFILE.signature, + contentHash: "sha256:IVAwpRTDujszmYf76W497alVTtxGCgtJtQlasiFSCM8", + keyid: "https://keys.example/alice-2026.json", + algorithm: "ed25519", + signedAt: "2026-01-15T12:00:00Z", + scope: "url", + location: "https://example.com/essays/engines", + signature: "m0ykSPqUWdyZprUAqosOB2IEK2XsKp7auPIWz80/2ht+LwT1LiNcsLL6cn2IkmTZFG9ptLiUHaB1crPJgBw7BA", + sourceURL: "HTTPS://EXAMPLE.COM:443/essays/engines#analysis", + claims: [ + { name: "author", content: "Ada Lovelace" }, + { name: "claim:License", content: "CC-BY-4.0" }, + { name: "signed-at", content: "2026-01-15T12:00:00Z" }, + ], +}); + +test("constructs the frozen signing-profile vector", async () => { + const validated = await validateV1ContentSubmission(vectorSubmission()); + assert.equal( + validated.canonicalClaims, + "author:Ada Lovelace\nclaim\\:License:CC-BY-4.0\nsigned-at:2026-01-15T12\\:00\\:00Z\n", + ); + assert.equal( + validated.claimsHash, + "sha256:Fk5udwCnu1au8v5oaBsU+aSB5S2zSLqoF0xXO6HrIn4", + ); + assert.equal( + validated.payload, + '{"algorithm":"ed25519","attributeProfile":"htmltrust-attrs-v1",' + + '"canonicalizationProfile":"htmltrust-c14n-v1",' + + '"claimsHash":"sha256:Fk5udwCnu1au8v5oaBsU+aSB5S2zSLqoF0xXO6HrIn4",' + + '"contentHash":"sha256:IVAwpRTDujszmYf76W497alVTtxGCgtJtQlasiFSCM8",' + + '"context":"https://htmltrust.org/protocol/signed-section",' + + '"keyid":"https://keys.example/alice-2026.json",' + + '"location":"https://example.com/essays/engines",' + + '"profile":"htmltrust-signature-v1","scope":"url",' + + '"signedAt":"2026-01-15T12:00:00Z","urlProfile":"htmltrust-safe-url-v1"}', + ); +}); + +test("derives exact URL and origin scope from an HTTPS source URL", () => { + assert.equal( + deriveLocation("https://BÜCHER.example:443/article?q=1#part", "url"), + "https://xn--bcher-kva.example/article?q=1", + ); + assert.equal( + deriveLocation("https://example.org:8443/a?q=1#part", "origin"), + "https://example.org:8443", + ); +}); + +test("rejects non-public v1 locations and hidden URL controls", () => { + assert.throws(() => deriveLocation("http://example.com/a", "url"), /HTTPS/); + assert.throws(() => deriveLocation("https://user@example.com/a", "url"), /credentials/); + assert.throws(() => deriveLocation("https://example.com/\tpath", "url"), /control/); + assert.throws(() => deriveLocation("https://example.com/a", "path"), /scope/); +}); + +test("accepts only the exact v1 timestamp grammar and valid calendar dates", () => { + assert.equal(assertV1Timestamp("2024-02-29T23:59:59Z"), "2024-02-29T23:59:59Z"); + for (const value of [ + "0000-01-01T00:00:00Z", + "2023-02-29T00:00:00Z", + "2026-01-15t12:00:00z", + "2026-01-15T12:00:00.000Z", + "2026-01-15T12:00:60Z", + "2026-01-15T12:00:00+00:00", + ]) { + assert.throws(() => assertV1Timestamp(value)); + } +}); + +test("claims escaping is injective and applies after normalization", async () => { + assert.equal( + await canonicalizeClaims([ + { name: "a:b\\c", content: "x:y\\z" }, + { name: "signed-at", content: "2026-01-15T12:00:00Z" }, + ], { strictArray: true }), + "a\\:b\\\\c:x\\:y\\\\z\nsigned-at:2026-01-15T12\\:00\\:00Z\n", + ); +}); + +test("rejects incomplete, duplicate, oversized, or caller-hashed claims", async () => { + const suppliedHash = { ...vectorSubmission(), claimsHash: "sha256:ignored" }; + await assert.rejects(() => validateV1ContentSubmission(suppliedHash), /must not be submitted/); + + const duplicate = vectorSubmission(); + duplicate.claims = [...duplicate.claims, { name: "author", content: "Other" }]; + await assert.rejects(() => validateV1ContentSubmission(duplicate), /claim-duplicate/); + + const missingTimestamp = vectorSubmission(); + missingTimestamp.claims = missingTimestamp.claims.filter(({ name }) => name !== "signed-at"); + await assert.rejects(() => validateV1ContentSubmission(missingTimestamp), /exactly one signed-at/); + + const extraMember = vectorSubmission(); + extraMember.claims = [{ name: "signed-at", content: extraMember.signedAt, extra: true }]; + await assert.rejects(() => validateV1ContentSubmission(extraMember), /only name and content/); + + const tooMany = vectorSubmission(); + tooMany.claims = Array.from({ length: 65 }, (_, index) => ({ + name: index === 0 ? "signed-at" : `claim-${index}`, + content: index === 0 ? tooMany.signedAt : "value", + })); + await assert.rejects(() => validateV1ContentSubmission(tooMany), /resource-limit-exceeded/); +}); + +test("rejects a location or signed-at claim that does not match the body", async () => { + const wrongLocation = { ...vectorSubmission(), location: "https://example.com/other" }; + await assert.rejects(() => validateV1ContentSubmission(wrongLocation), /derived from sourceURL/); + + const wrongTime = vectorSubmission(); + wrongTime.claims = wrongTime.claims.map((claim) => ( + claim.name === "signed-at" ? { ...claim, content: "2026-01-15T12:00:01Z" } : claim + )); + await assert.rejects(() => validateV1ContentSubmission(wrongTime), /must equal signedAt/); +}); diff --git a/test/storageErrors.test.js b/test/storageErrors.test.js new file mode 100644 index 0000000..df334c7 --- /dev/null +++ b/test/storageErrors.test.js @@ -0,0 +1,103 @@ +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const ContentSignature = require('../src/models/ContentSignature'); +const Endorsement = require('../src/models/Endorsement'); +const Key = require('../src/models/Key'); +const contentController = require('../src/controllers/contentController'); +const directoryController = require('../src/controllers/directoryController'); +const endorsementController = require('../src/controllers/endorsementController'); + +const validHash = `sha256:${Buffer.alloc(32).toString('base64').replace(/=+$/, '')}`; + +const response = () => ({ + statusCode: 200, + headers: {}, + status(code) { this.statusCode = code; return this; }, + set(name, value) { this.headers[name] = value; return this; }, + type(value) { this.headers['Content-Type'] = value; return this; }, + json(value) { this.body = value; return this; }, + send(value) { this.body = value; return this; }, +}); + +const withStub = async (object, property, replacement, callback) => { + const original = object[property]; + object[property] = replacement; + const originalError = console.error; + console.error = () => {}; + try { + await callback(); + } finally { + console.error = originalError; + object[property] = original; + } +}; + +test('content read failures return a generic 500 problem', async () => { + await withStub(ContentSignature, 'find', () => ({ + sort: async () => { throw new Error('database exploded'); }, + }), async () => { + for (const handler of [contentController.getContentRecord, contentController.getContentRecordV1]) { + const res = response(); + await handler({ params: { contentHash: validHash } }, res); + assert.equal(res.statusCode, 500); + assert.equal(res.body.type, 'https://htmltrust.org/errors/storage-failure'); + assert.doesNotMatch(res.body.detail, /database exploded/); + } + }); +}); + +test('endorsement read failures return a generic 500 problem', async () => { + await withStub(Endorsement, 'find', () => ({ + sort: async () => { throw new Error('database exploded'); }, + }), async () => { + const listResponse = response(); + await endorsementController.listEndorsements({ query: { 'content-hash': validHash } }, listResponse); + assert.equal(listResponse.statusCode, 500); + assert.equal(listResponse.body.type, 'https://htmltrust.org/errors/storage-failure'); + + const nestedResponse = response(); + await contentController.listContentEndorsements( + { params: { contentHash: validHash } }, + nestedResponse, + ); + assert.equal(nestedResponse.statusCode, 500); + assert.equal(nestedResponse.body.type, 'https://htmltrust.org/errors/storage-failure'); + }); +}); + +test('key read failures return a generic 500 problem', async () => { + await withStub(Key, 'findById', async () => { throw new Error('database exploded'); }, async () => { + const res = response(); + await directoryController.getKeyDocument({ params: { id: 'a'.repeat(24) } }, res); + assert.equal(res.statusCode, 500); + assert.equal(res.body.type, 'https://htmltrust.org/errors/storage-failure'); + assert.doesNotMatch(res.body.detail, /database exploded/); + }); +}); + +test('endorsement delete storage failures return a generic 500 problem', async () => { + await withStub(Endorsement, 'findById', async () => { throw new Error('database exploded'); }, async () => { + const res = response(); + await endorsementController.deleteEndorsement( + { params: { id: 'b'.repeat(24) } }, + res, + ); + assert.equal(res.statusCode, 500); + assert.equal(res.body.type, 'https://htmltrust.org/errors/storage-failure'); + }); +}); + +test('malformed read identifiers still return 400 before database access', async () => { + let queried = false; + await withStub(Key, 'findById', async () => { queried = true; }, async () => { + const keyResponse = response(); + await directoryController.getKeyDocument({ params: { id: 'not-an-object-id' } }, keyResponse); + assert.equal(keyResponse.statusCode, 400); + assert.equal(queried, false); + }); + + const contentResponse = response(); + await contentController.getContentRecordV1({ params: { contentHash: 'bad' } }, contentResponse); + assert.equal(contentResponse.statusCode, 400); +});