kafka: verify 3.9 topic identity before reusing logs - #586
bringhurst wants to merge 2 commits into
Conversation
9aa7416 to
4f73d1a
Compare
|
The deleted-topic batch correction is published with the paired 3.0 change in #585. Both full ReplicaManagerTest suites, dedicated identity tests, the real-ZooKeeper identity test, and Scala 2.13 compilation pass. The current local scenario-4 run has completed all four generation/assignment-timing promotion checks with exact replacement records; the complete run is still active, so this is not a full-migration pass. I also restacked this PR onto the hourly-log retention fix in #584. That only changes the evidence collector and its tests; broker source is unchanged by the restack. The Python suite now has 70 passing tests. Final wrapper qualification still requires the internal-network dependency refresh. |
|
The local scenario-4 run has now finished as a failure, not a pass. All four offline-name-reuse promotion/record checks passed, and the source fingerprint stayed unchanged. At the native checkpoint, the old metadata-churn helper exited on The failure is retained at |
Behind the cleanup flag, read ZooKeeper topic IDs on the controller path. Retire only known obsolete generations. Preserve unidentified data and conflicting current/future copies, validate supplied wire IDs, fence failed controller updates, and retry failed full-image checks. Require an IBP with topic IDs.
Distinguish a missing ZooKeeper znode from an existing topic without identity. Do not admit a deleted topic, but allow unrelated partitions to proceed. Keep unverifiable existing data fail-closed, preserve identity-check retries, and retain the native flag-off path. Fix the read-only Map parameter for Scala 2.13.
4f73d1a to
d67d7b2
Compare
Fix the assigned-before-return variant described in #584. Assignment does not identify a recreated topic generation because bridge control messages omit topic IDs.
Under the existing default-off cleanup flag, read the authoritative ZooKeeper topic IDs on the control path, persist IDs before accepting records, and retire only known obsolete generations. Keep unidentified nonempty logs and conflicting current/future copies for recovery. Validate supplied wire IDs, fence failed controller updates, and retry failed full-image checks. Reject cleanup on an IBP without topic IDs. Flag-off behavior is unchanged.
Both full ReplicaManagerTest suites and dedicated identity/cache tests pass. The assigned-before-return target failed before the repair and passed with the initial repair; the tightened current safety paths have fresh unit coverage. All 68 Python tests pass. Scenario revision 4 now requires both broker generations and both assignment timings; its complete process run is active. Old scenario evidence is rejected.
This is a draft, not rollout approval. The private wrapper dependency refresh requires the internal network/VPN, so current final wrapper qualification is blocked. Production client/state/runtime/capacity and named security/release approvals remain separate gates.