Complete the normative v1 directory surface

README.md

@@ -73,7 +73,7 @@ This is the lowest-dependency test path: it requires Docker and a shell. Set `HT
    npm ci --omit=dev
    ```
 
-3. Set `NODE_ENV=production`, `AUTHOR_API_KEY_PEPPER`, and `DIRECTORY_BASE_URL`. Use a random, long-lived pepper and keep it outside the repository. `DIRECTORY_BASE_URL` must be the public origin clients use to resolve directory key URLs.
+3. Set `NODE_ENV=production`, `AUTHOR_API_KEY_PEPPER`, and `DIRECTORY_BASE_URL`. Use a random, long-lived pepper and keep it outside the repository. `DIRECTORY_BASE_URL` must be the public HTTPS origin clients use to resolve directory key URLs. When the variable is unset, the server falls back to the request origin for local development, including `http://localhost`; that fallback is not a production deployment setting.
 4. Set `GENERAL_API_KEY` and `ADMIN_API_KEY` when compatibility or operator routes need them. Static general and author API-key authentication is disabled in production unless `HTMLTRUST_ALLOW_API_KEY_AUTH=1` is set.
 5. Start the service with `npm start` behind a TLS-terminating reverse proxy. Set `TRUST_PROXY` to the number of trusted proxy hops when the proxy forwards client addresses.
 
@@ -101,11 +101,9 @@ The server exposes two HTTP surfaces. The root routes are the canonical HTMLTrus
 | `POST /content` | Submit and re-verify a signed content record | RFC 9421 HTTP Message Signature |
 | `GET /content/:hash` | Retrieve a content record by percent-encoded hash | Public |
 | `GET /content/:hash/endorsements` | List endorsements for a content hash | Public |
-| `GET /endorsements?content-hash=...` | List endorsements for a content hash | Public |
 | `POST /endorsements` | Store a signed endorsement | RFC 9421 HTTP Message Signature |
-| `DELETE /endorsements/:id` | Delete an endorsement with the endorser key or directory admin key | Endorser signature or admin key |
 
-Canonical writes require a resolvable key in an RFC 9421 signature. The covered components include `@method`, `@target-uri`, `host`, `date`, and `content-digest` for requests with a body. The `keyid` identifies the key that signed the request.
+Canonical writes require a resolvable key in an RFC 9421 signature. The `sig1` input MUST cover exactly `@method`, `@target-uri`, `host`, `date`, and `content-digest`, in that order. The `keyid` identifies the key that signed the request.
 
 Canonical reads return an HTMLTrust media type by default and accept `application/json`. Responses include `Vary: Accept`; public reads include `Cache-Control` and `ETag`. A matching `If-None-Match` request receives `304 Not Modified`. Canonical JSON submissions accept `application/json` and `application/*+json`; another request media type receives `415`.
 
@@ -154,7 +152,7 @@ Author API keys are returned once by `POST /api/authors`. The server stores an H
 
 ### Key custody
 
-`POST /api/authors` accepts an optional SPKI PEM `publicKey`. Supplying one registers a key held by the caller and leaves its private key outside the directory. Omitting it asks the server to generate and hold a key pair for the convenience registry flow. Content signed by a caller-held key is submitted through `POST /content` or the compatibility `POST /api/content` route.
+`POST /api/authors` accepts an optional SPKI PEM `publicKey`. Supplying one registers a key held by the caller and leaves its private key outside the directory. Omitting it asks the server to generate and hold a key pair for the convenience registry flow. The public key document uses an opaque `id` in `/keys/{id}` URLs; existing pre-v1 rows remain readable through their legacy ObjectId URL until migrated. Content signed by a caller-held key is submitted through `POST /content` or the compatibility `POST /api/content` route.
 
 ## Project structure
 

conformance/README.md

@@ -48,7 +48,7 @@ cd conformance/runner
 npm install               # one-time
 node run.mjs \
   --target-url   http://your-server.example \
-  --base-path    /v1 \
+  --base-path    /api \
   --general-api-key YOUR_GENERAL_KEY \
   --admin-api-key   YOUR_ADMIN_KEY
 ```
@@ -58,7 +58,7 @@ All flags:
 | Flag | Default | Description |
 |---|---|---|
 | `--target-url URL` | `http://localhost:3000` | Base URL of the server |
-| `--base-path PATH` | `/api` | Prefix prepended to spec paths. Use `/v1` for spec-conformant servers, `/api` for the Node reference, or `""` for none. |
+| `--base-path PATH` | `/api` | Prefix prepended to the fixture's logical compatibility paths. Use `/api` for the Node reference or `""` when the target mounts those compatibility paths at the origin. Canonical v1 root paths are exercised separately by `runner/v1-smoke.mjs` and are never prefixed. |
 | `--general-api-key KEY` | env `GENERAL_API_KEY` | Value for `X-API-KEY` |
 | `--admin-api-key KEY` | env `ADMIN_API_KEY` | Value for `X-ADMIN-API-KEY` |
 | `--fixtures-dir DIR` | `../fixtures` | Where YAML fixtures live |
@@ -162,28 +162,32 @@ If you need a richer validator, swap in `ajv` — `validate()` in
 
 ## Known deviations of the Node reference
 
-For transparency — these were observed while building the suite. They are
-**not** fixed in this task (out of scope), but the suite is configured to
-work around them when run against the Node reference.
-
-1. **`id` fields are 24-char MongoDB ObjectIds, not RFC-4122 UUIDs.**
-   `openapi.yaml` specifies `format: uuid` for every id field. The Node
-   reference simply emits the underlying Mongoose `_id`. The
-   `--accept-mongo-ids` flag relaxes UUID checks to also accept ObjectIds.
-2. **Mongoose `_id` is exposed alongside the spec's `id` in many responses.**
-   `openapi.yaml` doesn't forbid extra properties (no `additionalProperties:
-   false`), so this is technically tolerable; but it leaks implementation
-   detail and a strict OpenAPI validator could complain. The same flag
-   permits `_id` as a synonym for `id`.
-3. **API is mounted under `/api/…` instead of `/v1/…`** as the spec's
-   `servers` block implies. `--base-path /api` accommodates this; other
-   implementations should use `/v1` or `""` as appropriate.
+The following compatibility notes describe behavior retained on the legacy
+`/api` surface. The suite is configured to work around them when run against
+the Node reference.
+
+1. **Compatibility author ids remain 24-char MongoDB ObjectIds.**
+   Canonical key-document ids are opaque public ids, while the older author
+   and compatibility-resource envelopes still expose their historical
+   ObjectId values. The `--accept-mongo-ids` flag relaxes UUID checks for those
+   compatibility fields.
+2. **Mongoose `_id` is exposed alongside the spec's `id` in some compatibility
+   responses.** `openapi.yaml` does not forbid extra properties, so this is
+   technically tolerable; the `--accept-mongo-ids` flag also accepts `_id` as
+   a capture fallback.
+3. **The fixture files use logical compatibility paths such as `/authors` and
+   `/directory/keys`.** The published OpenAPI contract names their runtime
+   paths with the `/api` prefix. The runner adds `--base-path` to fixture URLs
+   and uses named schemas rather than looking up paths in OpenAPI, so this
+   keeps the fixtures reusable without hiding the deployed route prefix.
 4. **`/votes` endpoints are implemented but not documented in
    `openapi.yaml`.** Fixtures 05, 06, and 09 exercise them anyway since the
    spec text references endorsement/trust voting. A future spec revision
    is expected to formalize them.
 
-Running the suite in **strict** mode against the reference (drop
-`--accept-mongo-ids`, use `--base-path ""`, and target a spec-conformant
-server) will surface deviations 1 and 2 immediately; that is the intended
-behaviour for verifying other implementations.
+For a strict check against another implementation, use a target and
+`--base-path` that match its published compatibility routes, omit
+`--accept-mongo-ids`, and run the canonical v1 smoke against the unprefixed
+root paths. The Node reference intentionally keeps its `/api` routes for
+backward compatibility while its canonical v1 directory surface is rooted at
+`/`.

conformance/runner/v1-smoke.mjs

@@ -29,6 +29,13 @@ const requestJson = async (url, init = {}, expectedStatus = 200) => {
   return { response, body };
 };
 
+const assertMediaType = ({ response }, expected, label) => {
+  const contentType = response.headers.get("content-type") || "";
+  if (!contentType.startsWith(expected)) {
+    fail(`${label} returned ${contentType}, expected ${expected}`);
+  }
+};
+
 const canonicalize = (value) => {
   if (value === null || typeof value !== "object") return JSON.stringify(value);
   if (Array.isArray(value)) return `[${value.map(canonicalize).join(",")}]`;
@@ -41,31 +48,49 @@ const canonicalize = (value) => {
 const unpadded = (buffer) => buffer.toString("base64").replace(/=+$/, "");
 const prefixedSha256 = (text) => `sha256:${unpadded(createHash("sha256").update(text).digest())}`;
 
-const signHttpRequest = ({ url, body, keyid, privateKey, nonce }) => {
+const signHttpRequest = ({
+  url,
+  body,
+  keyid,
+  privateKey,
+  nonce,
+  components = ["@method", "@target-uri", "host", "date", "content-digest"],
+  label = "sig1",
+  includeAlg = true,
+  padded = false,
+}) => {
   const parsed = new URL(url);
   const date = new Date().toUTCString();
   const created = Math.floor(Date.now() / 1000);
   const digest = createHash("sha256").update(body).digest("base64");
   const contentDigest = `sha-256=:${digest}:`;
   const parameters =
-    `("@method" "@target-uri" "host" "date" "content-digest")` +
-    `;created=${created};keyid="${keyid}";alg="ed25519";nonce="${nonce}"`;
+    `(${components.map((component) => `"${component}"`).join(" ")})` +
+    `;created=${created};keyid="${keyid}"` +
+    (includeAlg ? `;alg="ed25519"` : "") +
+    `;nonce="${nonce}"`;
+  const values = {
+    "@method": "POST",
+    "@target-uri": url,
+    host: parsed.host,
+    date,
+    "content-digest": contentDigest,
+    "content-type": "application/json",
+    "@request-target": parsed.pathname + parsed.search,
+  };
   const base = [
-    '"@method": POST',
-    `"@target-uri": ${url}`,
-    `"host": ${parsed.host}`,
-    `"date": ${date}`,
-    `"content-digest": ${contentDigest}`,
+    ...components.map((component) => `"${component}": ${values[component]}`),
     `"@signature-params": ${parameters}`,
   ].join("\n");
-  const signature = unpadded(cryptoSign(null, Buffer.from(base, "utf8"), privateKey));
+  let signature = cryptoSign(null, Buffer.from(base, "utf8"), privateKey).toString("base64");
+  if (!padded) signature = unpadded(Buffer.from(signature, "base64"));
   return {
     "content-type": "application/json",
     host: parsed.host,
     date,
     "content-digest": contentDigest,
-    "signature-input": `sig1=${parameters}`,
-    signature: `sig1=:${signature}:`,
+    "signature-input": `${label}=${parameters}`,
+    signature: `${label}=:${signature}${padded ? "=" : ""}:`,
   };
 };
 
@@ -104,17 +129,28 @@ const main = async () => {
   const keyid = `${target}/keys/${keyId}`;
 
   const discovery = await requestJson(`${target}/.well-known/htmltrust`);
+  assertMediaType(discovery, "application/htmltrust-directory+json", "discovery");
   if (!discovery.body.supportedProfiles?.includes("htmltrust-signature-v1")) {
     fail("discovery does not advertise htmltrust-signature-v1", discovery.body);
   }
   if (discovery.body.directory !== `${target}/`) {
     fail("discovery directory does not name the canonical root", discovery.body);
   }
+  await requestJson(`${target}/endorsements?content-hash=sha256:removed-root-list-route`, {}, 404);
+  await requestJson(`${target}/endorsements/000000000000000000000000`, { method: "DELETE" }, 404);
 
   const keyDocument = await requestJson(keyid);
+  assertMediaType(keyDocument, "application/htmltrust-key+json", "key document");
   if (keyDocument.body.kid !== keyid || keyDocument.body.publicKeyPem !== undefined) {
     fail("root key document has the wrong kid or exposes the PEM compatibility field", keyDocument.body);
   }
+  const reputation = await requestJson(
+    `${target}/signers/${encodeURIComponent(keyId)}/reputation`,
+  );
+  assertMediaType(reputation, "application/json", "signer reputation");
+  if (reputation.body.keyid !== keyId || typeof reputation.body.score !== "number") {
+    fail("root signer reputation has the wrong key identifier or score", reputation.body);
+  }
 
   const signedAt = "2026-01-15T12:00:00Z";
   const claims = [
@@ -166,6 +202,7 @@ const main = async () => {
     privateKey: signingKey.privateKey,
     nonce: "content-valid",
   });
+  assertMediaType(submitted, "application/htmltrust-content+json", "content submission");
   if (submitted.response.headers.get("location") !== `/content/${encodeURIComponent(contentHash)}`) {
     fail("POST /content returned the wrong Location header", submitted.response.headers.get("location"));
   }
@@ -179,7 +216,8 @@ const main = async () => {
     fail("POST /content returned an incomplete v1 signer record", submitted.body);
   }
 
-  await requestJson(`${target}/content/${encodeURIComponent(contentHash)}`);
+  const content = await requestJson(`${target}/content/${encodeURIComponent(contentHash)}`);
+  assertMediaType(content, "application/htmltrust-content+json", "content record");
 
   const badLocation = { ...submission, location: "https://example.com/research/other" };
   const rejectedLocation = await signedPost({
@@ -206,6 +244,56 @@ const main = async () => {
     fail("canonical POST /content accepted API-key fallback or omitted its signature challenge");
   }
 
+  const submissionBody = JSON.stringify(submission);
+  const signatureProfileCases = [
+    {
+      name: "legacy request-target component",
+      components: ["@method", "@request-target", "host", "date", "content-digest"],
+      nonce: "content-legacy-target",
+    },
+    {
+      name: "reordered components",
+      components: ["@target-uri", "@method", "host", "date", "content-digest"],
+      nonce: "content-reordered-components",
+    },
+    {
+      name: "wrong signature label",
+      label: "other",
+      nonce: "content-wrong-label",
+    },
+    {
+      name: "missing alg parameter",
+      includeAlg: false,
+      nonce: "content-missing-alg",
+    },
+    {
+      name: "additional covered component",
+      components: ["@method", "@target-uri", "host", "date", "content-digest", "content-type"],
+      nonce: "content-additional-component",
+    },
+    {
+      name: "padded signature bytes",
+      padded: true,
+      nonce: "content-padded-signature",
+    },
+  ];
+  for (const profileCase of signatureProfileCases) {
+    const rejected = await requestJson(`${target}/content`, {
+      method: "POST",
+      headers: signHttpRequest({
+        url: `${target}/content`,
+        body: submissionBody,
+        keyid,
+        privateKey: signingKey.privateKey,
+        ...profileCase,
+      }),
+      body: submissionBody,
+    }, 401);
+    if (!rejected.response.headers.get("www-authenticate")) {
+      fail(`canonical POST /content accepted ${profileCase.name}`);
+    }
+  }
+
   const unsignedEndorsement = {
     endorser: keyid,
     endorsement: contentHash,
@@ -221,13 +309,14 @@ const main = async () => {
       signingKey.privateKey,
     )),
   };
-  await signedPost({
+  const submittedEndorsement = await signedPost({
     path: "/endorsements",
     document: endorsement,
     keyid,
     privateKey: signingKey.privateKey,
     nonce: "endorsement-valid",
   });
+  assertMediaType(submittedEndorsement, "application/htmltrust-endorsement+json", "endorsement submission");
   await signedPost({
     path: "/endorsements",
     document: endorsement,
@@ -238,11 +327,12 @@ const main = async () => {
   const endorsements = await requestJson(
     `${target}/content/${encodeURIComponent(contentHash)}/endorsements`,
   );
+  assertMediaType(endorsements, "application/htmltrust-endorsement+json", "endorsement listing");
   if (!Array.isArray(endorsements.body) || endorsements.body.length !== 1) {
     fail("root content endorsement listing did not return the stored document", endorsements.body);
   }
 
-  console.log("HTMLTrust v1 directory smoke: 12 checks passed");
+  console.log("HTMLTrust v1 directory smoke: canonical operations and signature profile checks passed");
 };
 
 main().catch((error) => {