fix(export): refresh the member sha table after in-place member replacement - #99
Merged
Conversation
…cement rebuild_int8_head32 rewrites the encoder/decoder graphs inside a copy of the shipped int8 zip but left metadata.yaml's sha256 table pointing at the old digests — write_parity validates strictly before writing, so every rebuild died at exactly that point (parity itself passed). refresh_member_shas recomputes the table and rewrites the member atomically; the rebuild calls it right after assembling the zip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
Every
rebuild_int8_head32run (khm-latn, urd-g2p, urd-diac, heb-diac,tha-g2p-small) died at the same place despite parity passing:
Cause
The rebuild replaces the encoder/decoder members inside a copy of the
shipped int8 zip but never updates
metadata.yaml's sha256 table.write_parityvalidates strictly before writing, so the stale tablerejects the freshly rebuilt artifact — the zips existed on the volume
but could never receive their parity block or margin report.
Fix
imf.export.refresh_member_shas(zip)— recomputes the sha tablefrom actual member bytes and rewrites the metadata member atomically
(same tempfile + os.replace pattern as
write_parity)rebuild_int8_head32calls it immediately after assembling the zip→ validation must fail on the stale table → refresh → validation
passes and no other member changed
Verification
tests/test_imf_export.py— 6 passed (5 existing + new).