Skip to content

ref(eap): no-op migrations for tables that are no longer used - #8495

Merged
MeredithAnya merged 3 commits into
masterfrom
meredith/9-25-26-migrations
Sep 25, 2026
Merged

MeredithAnya merged 3 commits into
masterfrom
meredith/9-25-26-migrations

Conversation

@MeredithAnya

@MeredithAnya MeredithAnya commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

The EAP migration group has accumulated 71 migrations across several generations of table design: eap_spans → eap_spans_2 → eap_items_1 → eap_items_2, plus side branches for ourlogs, uptime_monitor_checks, and the abandoned eap_trace_item_attrs autocomplete experiment.

Only 18 tables are still in use — exactly the local_table_name values in snuba/datasets/configuration/events_analytics_platform/storages/. Everything else is created and then never read by any storage, entity, or query path.

This continues the cleanup from #7721, using the same no-op pattern.

Changes

19 migrations converted to no-ops. Files stay in place so the runner doesn't complain about missing entries; forwards_ops() / backwards_ops() return [] and a docstring records what each one used to do.

Migrations Tables they created
0001–0005, 0007–0010, 0016 eap_spans, eap_spans_2, spans_attributes_meta
0019, 0022, 0027 uptime_monitor_checks v1/v2
0020, 0021, 0028 ourlogs v1/v2/v3
0023, 0025, 0029 eap_trace_item_attrs

2 orphaned processors deleted — uptime_monitors_processor.py and ourlogs_processor.py. Neither had a single reference left in snuba/, tests/, or rust_snuba/.

4 manual jobs deleted, all of which read or wrote tables that no longer get created:

Job Dead table
scrub_ips_from_eap_spans eap_spans_2_local
scrub_users_from_eap_spans eap_spans_2_local
scrub_users_from_eap_spans_str_attrs spans_str_attrs_3_local
extract_span_data (+ its test) span data out of eap_spans_2_local

Manual jobs self-register via Job.class_from_name, so there is no registry to update. Only extract_span_data had a test, and it was already @pytest.mark.skip("can't test writing to GCS").

Deliberately kept

  • 0052 / 0053 create ClickHouse WORKLOAD objects, not tables. They back deletes on eap_items — no-oping them would break deletes on fresh installs.
  • The eap_items MV chains, which must stay intact: 0034 → 0039 → 0042/0047 → 0049 → 0057 → 0058 → 0059 → 0060 → 0061 → 0068 (_mv_10), and 0030 → 0051 → 0062 → 0063 → 0072 for co-occurring attrs.

Verification

A clean snuba migrations migrate --force marks all 71 EAP migrations completed and produces exactly these tables, nothing else:

eap_item_co_occurring_attrs_{1,2}_local    eap_items_{1,2}_local
eap_item_co_occurring_attrs_{3,4}_mv       eap_items_{1,2}_downsample_{8,64,512}_local
                                           eap_items_1_downsample_{8,64,512}_mv_10
                                           eap_items_2_downsample_{8,64,512}_mv_1
  • tests/migrations/ + tests/manual_jobs/ — 502 passed (includes test_runner.py, which runs every migration against a live ClickHouse)
  • tests/datasets/, tests/clusters/, test_endpoint_trace_item_attribute_names.py — 357 passed
  • ruff and mypy clean
  • No references to any deleted job or processor remain in snuba/, tests/, or docs/

🤖 Generated with Claude Code

@MeredithAnya
MeredithAnya requested review from a team as code owners September 25, 2026 19:45
MeredithAnya and others added 2 commits September 25, 2026 13:11
scrub_ips_from_eap_spans and scrub_users_from_eap_spans target
eap_spans_2_local; scrub_users_from_eap_spans_str_attrs targets
spans_str_attrs_3_local. Neither table is created any more now that
the migrations that built them are no-ops, so these jobs would fail
on any fresh deployment.

Manual jobs self-register via Job.class_from_name, so there is no
registry to update. None of the three had tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@ElioDiNino ElioDiNino left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 💯

@MeredithAnya
MeredithAnya merged commit 5a04537 into master Sep 25, 2026
68 checks passed
@MeredithAnya
MeredithAnya deleted the meredith/9-25-26-migrations branch September 25, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants