Skip to content

Send transactional mail through Resend, not SES - #211

Merged
jodeleeuw merged 1 commit into
testfrom
mail-via-resend
Aug 29, 2026
Merged

Send transactional mail through Resend, not SES#211
jodeleeuw merged 1 commit into
testfrom
mail-via-resend

Conversation

@jodeleeuw

Copy link
Copy Markdown
Member

AWS denied the SES production-access request. That would have left every send restricted to verified recipient addresses — failing for every real researcher, and failing terminally, since mail-delivery.ts classifies a rejected recipient as permanent and does not retry it. Resend has no equivalent gate, so the transport moves there.

Only the transport moved

mail.ts's document contract, the claim/lease machinery, the delivery.* outcome fields, the Firestore TTL policy and purge-user-data.ts's owner query are all untouched. That's what kept this to one file plus its tests — the same property that made the earlier extension→SES swap cheap.

  • buildSendEmailInput emits Resend's JSON body, using reply_to, not replyTo. The Node SDK takes camelCase; the raw REST API takes snake_case and ignores unknown keys silently — so getting this wrong means every notification goes out with no Reply-To and nothing says so. Pinned by its own test.
  • classifySesErrorclassifyMailError, rebuilt on Resend's documented error codes with HTTP status as the fallback.
  • No SDK. One fetch to one endpoint. @aws-sdk/client-sesv2 is gone, and with it the dynamic import that existed to keep the SDK off apidata's cold-start path.
  • Undici unwrapping. Node's fetch reports every network failure as TypeError: fetch failed with the real diagnosis on .cause, so errorName unwraps it. Without that, a momentary DNS blip classifies as unrecognised → terminal → a silently lost notification. Tested in the wrapped form, since that's the only form the code ever receives.

One behavioural change, deliberate

Ambiguous failures (timeouts, ECONNRESET) are now retryable. They were terminal because SESv2 has no idempotency token: a request that went out and never answered couldn't be retried without risking a second copy of a notification whose entire value is arriving once, so the old code chose to lose the mail.

Resend takes an Idempotency-Key, and we send the mail document's id on every attempt — so the retry is a no-op at Resend rather than a second send. We get the retry and exactly-once instead of choosing.

The key expires after 24h. Every retry path here is minutes wide, but that bounds any future retry mechanism, and it's called out in AMBIGUOUS_ERRORS and in both test suites so a later change can't quietly invalidate it.

Config

Six *_SES_* repo secrets become one *_RESEND_API_KEY per environment.

The test site is now expected to send, reversing this runbook's earlier "leave it unset" default. Reason: the test site is the only place delivery is exercised before production — the emulator short-circuits before sending, and the unit suites mock the transport at a function seam. Reputation and daily quota are shared with prod, which is documented in runbook §2(d) along with what to actually smoke-test.

Also corrects three things found while verifying the deploy steps

None caused by this change:

  • The production project is osf-relay, not datapipe-prod.firebaserc's default, and what firebase use default selects. GCP project ids are immutable, so datapipe-prod never existed. The runbook and three migration scripts all had it wrong, including the backfill's FIREBASE_PROJECT_ID || 'datapipe-prod' default.
  • Runbook §4's console instructions named delivery.endTime where the gcloud command says delivery.expireAt. Following the console path creates a policy that reaps every mail document the moment it's delivered — no debugging window, and nothing saying it happened.
  • mail.ts's header still described the extension as the transport and claimed switching providers was a config change rather than a redeploy. Now also explains why the collection exists at all (the tx.create inside the episode transaction) and why the TTL does.

Testing

Full suite green on the rebased tree: 79 suites, 1119 tests, plus tsc and lint.

The two mail suites were rewritten rather than patched — 48 pure tests (request mapping, error taxonomy, claim state machine, config) and 19 emulator-backed ones. No test in the repo can reach api.resend.com: the transport is injected at a function seam, so the real one is never constructed.

Deploy prerequisites — status

  • jspsych.org verified in Resend
  • PROD_RESEND_API_KEY / TEST_RESEND_API_KEY set
  • mail TTL policy on datapipe-testdelivery.expireAt, state ACTIVE
  • mail TTL policy on osf-relay — still absent, needed before prod
  • Post-deploy smoke test: change a contact email on a test account, confirm the mail document reaches delivery.state: "SUCCESS" with delivery.info.transport: "resend"

Known gap, not addressed here

Hitting the daily quota leaves the mail document as a retryable ERROR that nothing retries on a timer — and upload-failure-notify.ts sets notifiedAt in the same transaction that enqueues, so the episode is already armed as "told them." The researcher is never notified and the system believes it did. Closing that needs a scheduled sweeper, which is separable from this change and has its own design question (it must be kind-aware, and it collides with the 24h idempotency window).

🤖 Generated with Claude Code

AWS denied the SES production-access request, which would have left every
send restricted to verified recipient addresses -- failing for every real
researcher, and terminally, since mail-delivery.ts classifies a rejected
recipient as permanent. Resend has no equivalent gate.

Only the transport moved. mail.ts's document contract, the claim/lease
machinery, the delivery.* outcome fields, the TTL policy and
purge-user-data.ts's owner query are all untouched, which is what kept this
to one file plus its tests -- the same property that made the earlier
extension-to-SES swap cheap.

  - buildSendEmailInput emits Resend's JSON body. It uses reply_to, not
    replyTo: the SDK takes camelCase, the raw REST API takes snake_case and
    ignores unknown keys silently, so getting it wrong would mean every
    notification going out with no Reply-To and nothing saying so. Pinned by
    its own test.
  - classifySesError -> classifyMailError, rebuilt on Resend's error codes
    with HTTP status as the fallback.
  - The transport is one fetch to one endpoint. @aws-sdk/client-sesv2 is
    gone, and with it the dynamic import that existed to keep the SDK off
    apidata's cold-start path.
  - Undici reports every network failure as `TypeError: fetch failed` with
    the real diagnosis on .cause, so errorName unwraps it. Without that, a
    momentary DNS blip classifies as unrecognised, which is terminal, which
    silently loses a notification.

One behavioural change, deliberate: ambiguous failures (timeouts,
ECONNRESET) are now retryable. They were terminal because SESv2 has no
idempotency token, so a request that went out and never answered could not
be retried without risking a second copy of a notification whose whole value
is arriving once. Resend takes an Idempotency-Key; we send the mail
document's id on every attempt, so the retry is a no-op at Resend rather
than a second send. The key expires after 24h, which bounds any future retry
mechanism -- noted in AMBIGUOUS_ERRORS and in both test suites.

Six *_SES_* repo secrets become one *_RESEND_API_KEY per environment. The
test site is now expected to send: it is the only place delivery is
exercised before production, since the emulator short-circuits before
sending and the unit suites mock the transport. Reputation and daily quota
are shared with prod -- documented in runbook 2(d).

Also corrects three things found while verifying the deploy steps, none of
them caused by this change:

  - The production project is osf-relay, not datapipe-prod (.firebaserc's
    default, and what `firebase use default` selects). GCP project ids are
    immutable, so datapipe-prod never existed; the runbook and three
    migration scripts had it wrong.
  - Runbook 4's console instructions named delivery.endTime where the gcloud
    command says delivery.expireAt. Following the console path would create
    a policy that reaps every mail document the moment it is delivered.
  - mail.ts's header still described the extension as the transport and
    claimed switching providers was a config change rather than a redeploy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1dNU7EnmBBMTn8MWfojDL
@jodeleeuw
jodeleeuw merged commit 0041d8c into test Aug 29, 2026
1 check passed
@jodeleeuw
jodeleeuw deleted the mail-via-resend branch August 29, 2026 15:02
jodeleeuw added a commit that referenced this pull request Aug 29, 2026
The TTL on mail/delivery.expireAt was created by hand with gcloud, as the
runbook told us to. The next deploy deleted it:

  14:51  gcloud ... --enable-ttl --project=datapipe-test   -> ACTIVE
  15:02  PR #211 merged to `test`
  15:04  firestore: Deleting 1 field overrides...          <- the deploy
  15:10  TTL gone

`firebase deploy --only firestore` reconciles field overrides against
firestore.indexes.json, and that file said `"fieldOverrides": []` -- so a
hand-made TTL was not merely unmanaged, it was something the deploy was
actively instructed to delete. --force suppressed the prompt that would have
said so, and the deploy reported success. The retention promise stops being
kept, silently, while the runbook says it is.

Declared in firestore.indexes.json instead, so it is version-controlled and
recreated by the same deploy that would otherwise remove it. The block keeps
the three default single-field indexes: a fieldOverride replaces the field's
whole index configuration, so `"indexes": []` would additionally turn off
single-field indexing for delivery.expireAt. What is committed here is what
`firebase firestore:indexes` emits for a field with TTL on and default
indexing, read off osf-relay while its policy was still active, so it
round-trips.

Runbook 4 rewritten around the declarative mechanism, keeping the timeline
above -- following the old instructions produced a policy that survived
exactly until the next deploy, and that is worth being unable to rediscover
the hard way.

Also corrects two comments in mail-delivery.ts that said the TTL keys on
delivery.endTime. It keys on delivery.expireAt; endTime only gates whether
expireAt gets written. That same confusion is what put the wrong field in
runbook 4's console instructions.


Claude-Session: https://claude.ai/code/session_01D1dNU7EnmBBMTn8MWfojDL

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant