Fix TKey's compression test (#77), refuse RBlob keys, and two key items of #87 - #130
Open
sathabbott wants to merge 1 commit into
Open
sathabbott wants to merge 1 commit into
sathabbott wants to merge 1 commit into
Conversation
- A key payload is compressed iff fObjLen > fNbytes - fKeyLen, the test at all of ROOT's read sites (root-io-spec Compression §1), in read_object and in TKey_header.is_compressed(). A raw payload longer than fObjLen has slack; its first fObjLen bytes are the object. The "!=" form read that slack as a compression block header. - read_object refuses a key of class RBlob. Its fObjLen is decorative and one blob can hold several pages (root-io-spec RNTuple NOTES 1), so even the correct test would silently truncate it; RNTuple bytes are found through the anchor's envelope locators and the page lists. - A key is large when fVersion > 1000, as in TKey.cxx, so is_short() is <= 1000 and update_members uses it (root-io-spec Record §2 and its erratum 9). Differs only at exactly 1000. - Key names are uninterpreted bytes (Conventions §5.1): TKeyList decodes them as UTF-8 with surrogateescape instead of ASCII, so a non-ASCII name no longer raises and every name maps back to exactly its bytes. Fixes nsmith-#77 Part of nsmith-#87 Assisted-by: claude-code:claude-opus-5-5
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #130 +/- ##
==========================================
+ Coverage 91.82% 91.94% +0.11%
==========================================
Files 37 37
Lines 2962 2966 +4
==========================================
+ Hits 2720 2727 +7
+ Misses 242 239 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #77, and the two items of #87 that touch keys.
#77, the compression test. A key payload is compressed iff
fObjLen > fNbytes − fKeyLen, the test at all of ROOT's read sites (root-io-spec Compression §1). This applies both inTKey.read_objectand inTKey_header.is_compressed(). A raw payload longer thanfObjLenhas slack, and its firstfObjLenbytes are the object (§1.1). The!=form sent that slack to the decompressor, which failed withUnknown compression algorithm.RBlob keys are refused. One change from #77's suggested test: root-io-spec NOTES 1 shows an
RBlobkey'sfObjLenis decorative, and one blob can hold several pages, each with a checksumfObjLendoesn't count. So even the correct test silently truncates an RBlob.read_objectnow raises aValueErrorfor classRBlob, pointing to the anchor's envelope locators and the page lists, the only valid way to find RNTuple bytes. The test uses an ordinary key with slack instead.#87, two items.
fVersion > 1000(TKey.cxx:660,:1269), sois_short()is<= 1000, andupdate_membersnow calls it. Differs only at exactly 1000. RNTuple's own mini-file parser uses>= 1000, but root-io-spec's Record erratum 9 says a reader should followTKey.TKeyList.__iter__/__getitem__decode names as UTF-8 withsurrogateescapeinstead of ASCII (Conventions §5.1: names are uninterpreted bytes). A non-ASCII name no longer raises, UTF-8 names read naturally, and every name maps back to exactly its bytes. Keys staystr, so no API change. Say if you'd rather haveMapping[bytes, TKey].The rest of #87 (
TDatime,VersionInfo,fUnits,compressed_size()) is left open.Tests:
tests/test_tkey.py, with synthetic keys. Covered: raw payload with slack, compressed detection, RBlob refused, the threshold at 4 / 1000 / 1004, and non-ASCII names (caféand a lone0xff). 4 of the 7 fail onmain. Full suite:361 passed, 53 skipped, 46 xfailed.Assisted-by: claude-code:claude-opus-5-5