Skip to content

fix(address): make region-nullable migration atomic and self-healing - #215

Open
sroussey wants to merge 1 commit into
forms-worklist-streamingfrom
claude/sec-migration-tx-r77055
Open

fix(address): make region-nullable migration atomic and self-healing#215
sroussey wants to merge 1 commit into
forms-worklist-streamingfrom
claude/sec-migration-tx-r77055

Conversation

@sroussey

Copy link
Copy Markdown
Contributor

Summary

The SQLite branch of migrateAddressRegionNullable (added in the parent PR #213 base branch) ran the five rebuild DDL steps at the top level with no transaction boundary — DROP legacy / RENAME / drop-per-index / async setupDatabase() / INSERT ... SELECT / DROP legacy. An interrupt between the RENAME and the final INSERT strands every historical address in addresses__legacy_region, and the migration's own early-return guards (tableExists, notnull === 0) then treat the freshly rebuilt (empty, nullable) addresses as already-migrated on the next boot — so the stranded rows are never recovered. Junction tables reference address_hash_id, so the data cannot simply be re-extracted.

This PR closes both ends of that failure mode without changing the postgres path (already atomic — one DROP NOT NULL DDL).

Changes

  • Atomic rebuild. Wrap the five SQLite mutations in a single BEGIN IMMEDIATE / COMMIT block, with a matching ROLLBACK on any thrown error. The block is manual rather than db.transaction(...) because one of the enclosed steps (ADDRESS_REPOSITORY_TOKEN.setupDatabase()) is async and the wrapper's sync-callback form cannot await. Every DDL step runs on the single getDb() connection, so it enrolls in the transaction.
  • Self-heal a stranded prior interrupt. Ahead of the notnull-shape check, probe sqlite_master for addresses__legacy_region; if present, choose in an atomic block based on the two row counts:
    • legacy has rows AND addresses is missing/empty → restore the pre-migration state (drop the empty addresses, rename legacy back) so the normal flow below rebuilds it,
    • otherwise → drop the stale scratch table.
  • Postgres path unchanged, but the JSDoc now states explicitly that the single-DDL DROP NOT NULL is atomic and does not need wrapping, so a future reader does not add redundant ceremony.

Test plan

  • bun run test src/storage/address/AddressRegionNullableMigration.test.ts — 5 passed (existing 3 preserved, 2 added).
  • bun run test src/storage/address/ — 69 passed across 4 files (regression sweep on the address tier).
  • Tightened the existing "relaxes NOT NULL" test: replaced expect(indexes.length).toBeGreaterThan(0) with an equality check against the exact user-index tuple set a fresh setupAllDatabases() produces on addresses (memoized helper, reused across the two migrated-shape assertions). A silent unindexed rebuild now fails the assertion; the previous threshold would not.
  • Added "recovers from an interrupt between RENAME and INSERT" — seeds the legacy shape with two rows, manually simulates the exact stranded state (rename aside, drop indexes, setupDatabase() a fresh empty addresses), then asserts the next boot reaches every seeded row, has the correct index set, and cleared the legacy table.
  • Added "rolls back a failed migration attempt" — proxies ADDRESS_REPOSITORY_TOKEN so its first setupDatabase() call throws, expects setupAllDatabases() to reject, and asserts the legacy shape (notnull === 1) plus every seeded row still exist, with no addresses__legacy_region leftover.

Stacks on top of #213 (forms-worklist-streaming); rebase after that lands.


Generated by Claude Code

The SQLite `state_or_country` rebuild in `migrateAddressRegionNullable`
issued five top-level DDL statements — DROP legacy / RENAME / drop
per-index / async `setupDatabase()` / INSERT / DROP legacy — with no
transaction boundary. An interrupt between the RENAME and the final
INSERT stranded every historical address in `addresses__legacy_region`,
and the migration's own early-return checks (`tableExists`, `notnull`)
would then treat the freshly rebuilt (empty, nullable) `addresses` as
already-migrated, so the stranded rows were never recovered.

- Wrap the five mutations in a single `BEGIN IMMEDIATE`/`COMMIT` block
  (manual, not `db.transaction(...)`, because `setupDatabase()` is async
  and the wrapper's callback form must be sync). An interrupt inside now
  rolls back to the original `addresses` table.
- Ahead of the existing shape checks, self-heal a stranded
  `addresses__legacy_region` from a prior interrupted run: if legacy
  carries data and `addresses` is missing or empty, restore the
  pre-migration state; otherwise discard the stale scratch table.
- Postgres path is left unchanged (a single-statement `DROP NOT NULL`
  is already atomic); the JSDoc now states that explicitly so a future
  reader does not add redundant wrapping.

Tests:
- Replace the `indexes.length > 0` threshold with an equality check
  against the exact index set a fresh `setupAllDatabases()` produces on
  `addresses` (memoized helper reused across the two migrated-shape
  assertions).
- Add "recovers from an interrupt between RENAME and INSERT" — seeds
  legacy rows, simulates the exact stranded state (rename aside, empty
  rebuilt `addresses`), then asserts the next boot recovers every row.
- Add "rolls back a failed migration attempt" — proxies
  `ADDRESS_REPOSITORY_TOKEN.setupDatabase` to throw once, expects the
  whole call to reject, and asserts the legacy shape and rows survive
  with no leftover scratch table.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVYgNUoS8194CMP9fDCQLL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants