Skip to content

Fix Relay tests discard inbox responses, hiding 500 Internal Server Errors - #1047

Open
sij411 wants to merge 4 commits into
fedify-dev:mainfrom
sij411:fix/relay-test
Open

sij411 wants to merge 4 commits into
fedify-dev:mainfrom
sij411:fix/relay-test

Conversation

@sij411

@sij411 sij411 commented Sep 22, 2026

Copy link
Copy Markdown
Member

Fixes #1046

  • Move Alice's inbox fixtures to intercepted IPv6 documentation addresses.
  • Assert 202 Accepted for previously unchecked inbox responses.
  • Verify approved follows send exactly one Accept POST from the relay, referencing the correct Follow

AI assistance

  • I planned the fix and Codex:gpt-6-astra implemented it.

Move Alice's inbox fixtures to separate IPv6 documentation ranges
and keep the document loader and stored actor fixtures consistent.
Add module-level fetch interceptors, which this checkout lacks, so
fixture deliveries receive a local response without DNS or network I/O.

This implements the fixture portion of:
fedify-dev#1046

Validated with mise run check-each relay and mise run test-each relay
(Deno, Node.js, and Bun).

Changelog: none
Assisted-by: Codex:gpt-6-astra
Check for 202 Accepted at all 14 previously discarded inbox responses
in the LitePub and Mastodon relay tests. This prevents follower-store
assertions from hiding errors returned by the inbox handler.

fedify-dev#1046

Validated with mise run test-each relay, including code checks and
Deno, Node.js, and Bun tests.

Changelog: none
Assisted-by: Codex:gpt-6-astra
Record deliveries to Alice's inbox and assert that approved follows
produce exactly one Accept POST from the relay for the expected Follow.
Cover subscription approval, public follows, and malformed follower rows.
Use unique Follow IDs to isolate concurrent tests and loop iterations.

fedify-dev#1046

Validated with mise run test-each relay, including code checks and
Deno, Node.js, and Bun tests.

Changelog: none
Assisted-by: Codex:gpt-6-astra
@netlify

netlify Bot commented Sep 22, 2026

Copy link
Copy Markdown

Deploy Preview for fedify-json-schema canceled.

Name Link
🔨 Latest commit af66039
🔍 Latest deploy log https://app.netlify.com/projects/fedify-json-schema/deploys/6ab226f4d46aa90008c8118e

@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: c9c4946d-bb73-4404-8cad-2d61152cd561

📥 Commits

Reviewing files that changed from the base of the PR and between c5c9d61 and af66039.

📒 Files selected for processing (6)
  • CHANGES.md
  • changes.d/fedify/dns-delivery-failures.md
  • packages/fedify/src/federation/send.test.ts
  • packages/fedify/src/federation/send.ts
  • packages/relay/src/litepub.test.ts
  • packages/relay/src/mastodon.test.ts

Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

Changes

Delivery failure handling and relay test coverage

Layer / File(s) Summary
Classify and record delivery failures
packages/fedify/src/federation/send.ts, packages/fedify/src/federation/send.test.ts, CHANGES.md, changes.d/fedify/dns-delivery-failures.md
DNS failures now raise FetchError, pass through delivery failure handling, and record failed delivery metrics. Private-address and protocol rejections remain UrlError. Tests cover direct and redirected destinations.
Validate LitePub relay responses
packages/relay/src/litepub.test.ts
Tests use recorded inboxes, assert 202 responses, and verify generated Accept deliveries with unique activity IDs.
Validate Mastodon relay responses
packages/relay/src/mastodon.test.ts
Tests use a shared RFC 3849 inbox, assert 202 responses, and verify generated Accept deliveries.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Low

Merge Risk: ⚪ Minimal · up to af660

The change records DNS delivery failures correctly and strengthens relay tests to require successful responses and expected Accept deliveries; no merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR also changes packages/fedify/src/federation/send.ts, adds DNS-failure tests in packages/fedify/src/federation/send.test.ts, and adds CHANGES.md and `changes.d/fedify/dns-delivery-failures… Remove the unrelated send.ts, send.test.ts, CHANGES.md, and changes.d/fedify/dns-delivery-failures.md changes, or provide a directly linked issue that requires the DNS error-classification and metrics behavior.
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 4 files. (2 skipped: 2… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: updating Relay tests so discarded inbox responses no longer hide 500 Internal Server Error results.
Description check ✅ Passed The description directly matches the changeset. It covers intercepted inbox fixtures, 202 response assertions, and verification of relay Accept deliveries.
Linked Issues check ✅ Passed The relay test changes satisfy #1046. packages/relay/src/litepub.test.ts and packages/relay/src/mastodon.test.ts use intercepted RFC 3849 inbox addresses. The affected tests assert 202 responses…
Full details: Out of Scope Changes check

Explanation

The PR also changes packages/fedify/src/federation/send.ts, adds DNS-failure tests in packages/fedify/src/federation/send.test.ts, and adds CHANGES.md and changes.d/fedify/dns-delivery-failures.md entries. These changes implement DNS error classification, delivery metrics, and circuit-breaker behavior. They do not support the relay test requirements in #1046.

Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 4 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sij411 sij411 changed the title Fix/relay test Fix Relay tests discard inbox responses, hiding 500 Internal Server Errors Sep 22, 2026
@codecov

codecov Bot commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Files with missing lines Coverage Δ
packages/fedify/src/federation/send.ts 95.77% <100.00%> (+0.03%) ⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

DNS resolution failures are temporary transport failures, not evidence
that a destination violates the private-address policy. Preserve their
FetchError classification so the existing outbox middleware counts them
against the circuit breaker. Keep policy rejections as UrlError so they
do not count as transport failures or weaken SSRF protection.

The URL validator currently attaches a cause only when DNS resolution
throws; private-address and protocol rejections have no cause. Use that
existing distinction at the delivery boundary to retain the resolver's
original error without changing the shared validator's API or behavior
for its other callers. This relies on that distinction remaining true;
if policy errors gain causes, introduce an explicit discriminator.

Move initial URL validation inside the delivery try/catch so direct DNS
failures record failure metrics just like redirect and fetch failures.
Validation still happens before any request is sent.

Add signed and unsigned regression cases for direct and redirected DNS
failures, checking FetchError, failure metrics, and no fetch to the failed
destination. Existing destination tests retain private-address coverage.

Verified that the regression tests fail before the fix and pass after it.
Focused Deno tests and the full fedify package suite passed on Deno,
Node.js, and Bun. Added a Sacho fragment and regenerated the changelog.

Assisted-by: Codex:gpt-6-astra
@sij411

sij411 commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

@codex review

@sij411
sij411 marked this pull request as ready for review September 22, 2026 15:49
@sij411

sij411 commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@sij411

sij411 commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

af66039 : not 100% sure this is the right solution so if y'all have better solution leave some comments on please

@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@dahlia dahlia self-assigned this Sep 22, 2026
@dahlia dahlia added the component/relay Relay related (@fedify/relay) label Sep 22, 2026
@dahlia dahlia added this to the Fedify 2.4 milestone Sep 22, 2026
@sij411

sij411 commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@dahlia dahlia left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test changes do what #1046 asked for, and the new assertions have teeth: putting Alice's inbox back on remote.example.com fails three tests in litepub.test.ts.

My one structural request is the last commit, “Preserve transport errors for DNS failures,” which changes delivery behavior in @fedify/fedify and is unrelated to this issue. Details are inline. Please drop it from this branch and open a separate issue for it; the relay tests pass without it, so nothing here depends on it.

The rest is minor.

id: new URL("https://remote.example.com/users/alice"),
preferredUsername: "alice",
inbox: new URL("https://remote.example.com/users/alice/inbox"),
inbox: new URL(aliceInbox),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follower2's inbox on line 1050 still points at remote.example.com, so this test mixes two fixture conventions. Nothing fails today because it never delivers, but that is easy to misread later. Either move bob onto the intercepted prefix as well, or note that this fixture is store-only.

strictEqual(accepts[0].body.actor, "https://relay.example.com/users/relay");
strictEqual(accepts[0].body.object.type, "Follow");
strictEqual(
accepts[0].body.object.actor?.id ?? accepts[0].body.object.actor,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This accepts either serialization shape, so it passes whichever one the relay happens to emit. Pinning the shape the relay actually produces would catch a regression that changes it. The same helper in mastodon.test.ts has the same issue.

} catch (error) {
// DNS resolution errors carry the resolver's failure as their cause;
// private-address and protocol policy rejections do not.
if (error instanceof UrlError && error.cause != null) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unrelated to #1046 and is not mentioned in the pull request description. Could you split it into its own issue and pull request?

Two things to settle there:

  • The bug is already released. 68ec7b4 shipped in 2.0.28 through 2.3.8, so under the branch policy the fix belongs on 2.0-maintenance rather than main.
  • error.cause != null relies on validatePublicUrl() attaching a cause only for DNS failures, which is an internal detail of the validator. Please add an explicit property to UrlError and discriminate on that instead. Adding API in a patch release is not ideal, but this is small enough to be acceptable. It would also cover DNS lookup did not return any IP address in packages/vocab-runtime/src/url.ts, which carries no cause and so stays a UrlError under the current heuristic.

}
let response: Response;
try {
await validateUrl(inbox.href);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this inside the try also makes private address rejections record a delivery failure metric and an error log, which is a separate behavior change from the DNS one and is not in the changelog fragment. Worth deciding explicitly when this moves to its own pull request.

ignore: "Deno" in globalThis &&
(await Deno.permissions.query({ name: "net" })).state !== "granted",
async fn(t) {
const destination = "https://fedify-delivery.invalid/inbox";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fedify-delivery.invalid performs a real DNS query on Node.js and Bun. On a network that answers NXDOMAIN with a public address, validation passes, the catch-all mock returns 202 Accepted, and this fails for reasons unrelated to the code. Stubbing the resolver would be better; failing that, a comment would save the next person the debugging.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/relay Relay related (@fedify/relay)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Relay tests discard inbox responses, hiding at least one 500 Internal Server Error

2 participants