Skip to content

fix(ddl): remove duplicate is_current rows from users - #1010

Closed
rickyrombo wants to merge 1 commit into
mainfrom
fix/users-duplicate-current-rows
Closed

fix(ddl): remove duplicate is_current rows from users#1010
rickyrombo wants to merge 1 commit into
mainfrom
fix/users-duplicate-current-rows

Conversation

@rickyrombo

@rickyrombo rickyrombo commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Pairs with OpenAudio/go-openaudio#433, which adds users_current_uniq_idx to stop this recurring.

What

users_pkey is (user_id, txhash), which admits a second is_current row under a different txhash. Five users have one, spread 2025-06 → 2026-06.

Five rows, but the blast radius isn't five rows — anything joining an entity to its owner's wallet fans out and silently duplicates every entity those users own. Measured on a production clone:

with dedupe plain join phantom rows
tracks 1,955,896 1,955,914 +18
follows 26,067,177 26,067,964 +787

Why the delete is here and not in the ETL migration

ETL migrations run automatically at indexer start (SkipMigrations is left false), so shipping the delete there would make a go get silently remove rows from this database. The index stays in the ETL migration — it's that module's invariant — and the one-time repair lives in the application that owns the data.

Ordering

The index cannot be created while duplicates exist, so this must run first. It does: ddl migrations run in the pre-roll migrate Job that every serving Deployment DependsOn, while the ETL's run at indexer start after that Job completes.

Verified both orders against a fixture:

index first          -> ERROR: could not create unique index ... Key (user_id)=(98311147) is duplicated
backfill then index  -> both apply; violations 0; indisvalid = t
re-running either    -> no-op

Choices

  • Deleted, not demoted. users keeps no versioned history — the indexer writes it in place and production has zero is_current = false rows — so demoting would leave a category of row nothing reads.
  • Safe to delete. No foreign key references users; its triggers are AFTER INSERT (on_user) and AFTER INSERT OR UPDATE (trg_users), so a delete fires neither.
  • Winner is the highest blocknumber, matching how consumers pick the live row. On the clone, blocknumber and updated_at agree in all five cases, and for user 666149592 it keeps is_deactivated = true — the later of that pair's states.

Cause is not established

Both indexer create paths reject an existing user (validateUserCreate and migratedUserCreateHandler both call userExists), so a single writer can't produce these — its check and insert share a transaction. A second writer can, since check-then-act isn't atomic across transactions; three of the five pairs put a bare-hex txhash next to a 0x-prefixed one, which fits but doesn't prove it. The index will surface it if it recurs.

🤖 Generated with Claude Code

users_pkey is (user_id, txhash), which admits a second is_current row under a
different txhash. Five users have one, spread from 2025-06 to 2026-06. Five
rows, but the blast radius is not five rows: anything joining an entity to
its owner's wallet fans out and silently duplicates every entity those users
own — 18 extra tracks and 787 extra follows on a production clone.

This is the backfill half of pkg/etl migration 0035, which adds
users_current_uniq_idx to stop it recurring. The delete lives here rather
than in that migration because ETL migrations run automatically at indexer
start, so shipping it there would make a `go get` silently remove rows from
this database.

The index cannot be created while duplicates exist, so this has to run first,
and it does: ddl migrations run in the pre-roll `migrate` Job that every
serving Deployment depends on, while the ETL's run at indexer start after
that Job completes. Verified both orders against a fixture — backfill then
index succeeds; index first fails with "could not create unique index ...
Key (user_id)=(98311147) is duplicated" rather than repairing anything, which
is the signal that this migration has not run.

Deleted rather than demoted: users keeps no versioned history, so demoting
would leave a category of row nothing reads. No foreign key references users,
and its triggers are INSERT or INSERT OR UPDATE, so a delete fires neither.
Winner is the highest blocknumber, which also keeps is_deactivated = true for
user 666149592. Re-running is a no-op.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rickyrombo

Copy link
Copy Markdown
Contributor Author

Superseded by #1011, which combines this with the other two api-side changes. They're only correct together in one deploy — merging the module bump without the users backfill would fail ETL migration 0035's index creation and stop the indexer starting.

@rickyrombo rickyrombo closed this Aug 4, 2026
rickyrombo added a commit that referenced this pull request Aug 5, 2026
Supersedes #1008, #1009 and #1010, which were the same work split three
ways.

## Why one PR

These three changes are only correct **together, in one deploy**. Split,
each one alone breaks something:

| merged alone | result |
|---|---|
| the bump | ETL `0035` creates `users_current_uniq_idx`, **fails on the
existing duplicates**, `RunMigrations` errors and the indexer won't
start |
| `0236` (album) | the old indexer keeps deriving `album` from
`is_album` and undoes the backfill |
| `0237` (users) | harmless, but pointless without the index that stops
it recurring |

As one PR the deploy is atomic, and the ordering inside it is guaranteed
by existing machinery: `bridge migrate` runs as a pre-roll Job that
every serving Deployment `DependsOn` (serving pods get
`runMigrations=false`), so both ddl migrations complete before the
indexer starts and runs the ETL's.

## Contents

**`0237_users_one_current_row_backfill`** — deletes 5 duplicate
`is_current` rows from `users`. Small count, large blast radius: joins
from an entity to its owner's wallet fan out, measured at **+18 tracks
and +787 follows** on a production clone. Must precede the ETL index.

**`0236_saves_reposts_album_to_playlist`** — 670 saves and 528 reposts
written as `album` collapse to `playlist`. `on_save`/`on_repost` are
disabled for the update (their notification `group_id` embeds the type,
so a plain UPDATE would mint duplicate favourite notifications);
`trg_saves`/`trg_reposts` stay enabled so the search indexer sees the
change.

**`deps: pin pkg/etl v1.6.4`** — brings OpenAudio/go-openaudio#428
(album type), #425 (the `users` invariant + genesis-writer join
simplification) and #433 (`0035` no longer deletes anything).

## The delete moved out of the ETL migration

`v1.6.3`'s `0035` deleted the duplicates itself. Since ETL migrations
run automatically at indexer start, that made a `go get` able to remove
rows from this database. #433 split it: the index stays in the ETL
migration, the repair moved to `0237` here. **`v1.6.4` ships zero
`DELETE` statements** — verified against the resolved module, not just
the tag.

The comment above the ETL config now records that line, and its
corollary: an ETL migration can depend on a ddl one having run, and
`0035` fails loudly if `0237` hasn't.

## Verified

- Resolved module `pkg/etl@v1.6.4` contains `0035` with `CREATE UNIQUE
INDEX` and **0** `DELETE` statements.
- Both migration orders against fixtures: backfill→index applies cleanly
(`violations 0`, `indisvalid = t`); index→backfill fails with `could not
create unique index … Key (user_id)=(98311147) is duplicated`, which is
the intended signal that `0237` hasn't run.
- Both migrations idempotent; re-running is a no-op.
- `0236` fires **zero** `on_save`/`on_repost` triggers against a fixture
with the real wiring, and exactly one `pg_notify` per updated row.
- `go build ./...` and `go vet ./indexer/` clean.
- No FK references `users`; its triggers are INSERT / INSERT OR UPDATE,
so the delete fires neither.
- Cutting `pkg/etl/v1.6.4` did not move `openaudio/go-openaudio:stable`
— still the 2026-07-30 `v1.8.2` digest, so no node-operator rollout.

## Not established

The cause of the duplicate `users` rows. Both indexer create paths
reject an existing user, so a single writer can't produce them; a second
writer can, since check-then-act isn't atomic across transactions. Three
of five pairs put a bare-hex `txhash` next to a `0x`-prefixed one, which
fits but doesn't prove it. The index will surface it if it recurs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant