Keep the mail TTL policy from being deleted by every deploy

[jodeleeuw]

The TTL on mail/delivery.expireAt was created by hand with gcloud, as runbook §4 instructed. The next deploy deleted it.

14:51  gcloud ... --enable-ttl --project=datapipe-test   → ACTIVE
15:02  PR #211 merged to `test`
15:04  firestore: Deleting 1 field overrides...          ← the deploy
15:10  TTL gone

That middle line is from the deploy log, verbatim.

Why

firebase deploy --only firestore reconciles field overrides against firestore.indexes.json, and that file said "fieldOverrides": [] — an explicit assertion that no field overrides should exist. So a hand-made TTL wasn't merely unmanaged; it was something the deploy was actively instructed to delete. --force suppressed the confirmation prompt that would have said so, and the deploy reported success.

A manual TTL therefore survives exactly until the next deploy of any kind, silently. The retention promise stops being kept while the runbook says it is — and the only symptom is a mail collection quietly accumulating researcher email addresses forever.

This currently affects prod too. osf-relay's TTL is ACTIVE right now, but the next deploy to main removes it by the same mechanism.

The fix

Declared in firestore.indexes.json, so it's version-controlled and created by the same deploy that was deleting it. Merging this restores the datapipe-test policy automatically — no gcloud needed — and protects prod ahead of its next deploy.

Two details that aren't arbitrary:

• The block is not hand-written. It's what firebase firestore:indexes emits, read off osf-relay while its policy was still active, so it round-trips rather than being a guess at the schema.
• The three default single-field indexes are deliberate. A fieldOverride replaces the field's entire index configuration, so "indexes": [] would additionally turn off single-field indexing for delivery.expireAt — an unrelated second change nobody asked for.

Also

• Runbook §4 rewritten around the declarative mechanism, keeping the timeline above in it. Following the old instructions produced a policy that survived until the next deploy; that's worth not having to rediscover.
• Two comments in mail-delivery.ts corrected — they said the TTL keys on delivery.endTime. It keys on delivery.expireAt; endTime only gates whether expireAt gets written. That same confusion is what put the wrong field in §4's console instructions, fixed in Send transactional mail through Resend, not SES #211.

Checked for wider drift

No real indexes are at risk. Local firestore.indexes.json has one uploadQueue index (experimentID, status, providerErrorCode) that prod lacks — a create on next deploy, not a delete. Nothing deployed is missing from the file.

Verifying after merge

gcloud firestore fields ttls list --collection-group=mail --project=datapipe-test

state: ACTIVE is the goal. CREATING means the field backfill is still running — several minutes regardless of collection size, since it's a control-plane operation. Listed 0 items would mean this didn't take.

🤖 Generated with Claude Code