Skip to content

kafka: retire unhosted logs when bridge metadata deletes a topic - #576

Draft
bringhurst wants to merge 1 commit into
3.9-li-bridge/deletion-recoveryfrom
3.9-li-bridge/stray-log-cleanup
Draft

bringhurst wants to merge 1 commit into
3.9-li-bridge/deletion-recoveryfrom
3.9-li-bridge/stray-log-cleanup

Conversation

@bringhurst

Copy link
Copy Markdown
Member

Retire an unhosted local log when metadata reports its deletion, behind the cleanup flag. Preserve hosted replicas, unrelated logs and stale-controller fencing. Point process CI at the companion 3.0 repair.

Verification and release boundary

The latest clean-source mixed migration and process-evidence audit pass locally. The current wrapper suite passes 132 tests, with unchanged Kafka dependencies before and after the run. The reorganized Python suite passes 65 tests. These results apply to the complete candidate stack, not every intermediate commit. Focused tests are included with the relevant layers.

The final requirement audit and remote CI review remain open. Published artifact qualification, live production state, the client/tool floor, the deployed ZooKeeper server, capacity limits and named security/release approvals remain release gates. Do not deploy an intermediate stack commit.

@bringhurst
bringhurst added this pull request to stack #580 September 10, 2026 22:49
@bringhurst
bringhurst force-pushed the 3.9-li-bridge/stray-log-cleanup branch from 1578464 to e9065c4 Compare September 10, 2026 23:07
A canceled reassignment can leave a loaded log behind across broker restart. Behind the cleanup flag, delete that unhosted local log when metadata reports the partition deleted. Keep hosted replicas on the StopReplica path. Test both flag settings, stale controller fencing, unrelated logs and empty name reuse.
@bringhurst
bringhurst removed this pull request from stack #580 September 11, 2026 01:35
@bringhurst
bringhurst changed the base branch from 3.9-li-bridge/runbook to 3.9-li-bridge/deletion-recovery September 11, 2026 01:35
@bringhurst
bringhurst force-pushed the 3.9-li-bridge/stray-log-cleanup branch from e9065c4 to 77da203 Compare September 11, 2026 01:36
@bringhurst
bringhurst added this pull request to stack #582 September 11, 2026 01:36
@bringhurst

Copy link
Copy Markdown
Member Author

Blocking qualification finding: the current passing migration scenario does not cover a former replica that stays offline through reassignment, deletion, and name reuse. A targeted run reached ISR and promoted that replica, but read the old 64-byte record at offset 0 where the recreated topic had a new 128-byte record. The old broker missed the deletion notification. This also affects the companion in #577. I am extending the cleanup gate to reconcile unhosted logs from the complete initial metadata image, preserving assigned leaderless logs and same-epoch incremental behavior, and adding the case to mandatory qualification. Do not treat the existing green checks as approval to deploy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant