Skip to content

Fix TKey's compression test (#77), refuse RBlob keys, and two key items of #87 - #130

Open
sathabbott wants to merge 1 commit into
nsmith-:mainfrom
sathabbott:fix/77-key-compression-test
Open

sathabbott wants to merge 1 commit into
nsmith-:mainfrom
sathabbott:fix/77-key-compression-test

Conversation

@sathabbott

Copy link
Copy Markdown
Collaborator

🤖 AI generated content

Fixes #77, and the two items of #87 that touch keys.

#77, the compression test. A key payload is compressed iff fObjLen > fNbytes − fKeyLen, the test at all of ROOT's read sites (root-io-spec Compression §1). This applies both in TKey.read_object and in TKey_header.is_compressed(). A raw payload longer than fObjLen has slack, and its first fObjLen bytes are the object (§1.1). The != form sent that slack to the decompressor, which failed with Unknown compression algorithm.

RBlob keys are refused. One change from #77's suggested test: root-io-spec NOTES 1 shows an RBlob key's fObjLen is decorative, and one blob can hold several pages, each with a checksum fObjLen doesn't count. So even the correct test silently truncates an RBlob. read_object now raises a ValueError for class RBlob, pointing to the anchor's envelope locators and the page lists, the only valid way to find RNTuple bytes. The test uses an ordinary key with slack instead.

#87, two items.

  • Large-key threshold: a key is large when fVersion > 1000 (TKey.cxx:660, :1269), so is_short() is <= 1000, and update_members now calls it. Differs only at exactly 1000. RNTuple's own mini-file parser uses >= 1000, but root-io-spec's Record erratum 9 says a reader should follow TKey.
  • Key names: TKeyList.__iter__ / __getitem__ decode names as UTF-8 with surrogateescape instead of ASCII (Conventions §5.1: names are uninterpreted bytes). A non-ASCII name no longer raises, UTF-8 names read naturally, and every name maps back to exactly its bytes. Keys stay str, so no API change. Say if you'd rather have Mapping[bytes, TKey].

The rest of #87 (TDatime, VersionInfo, fUnits, compressed_size()) is left open.

Tests: tests/test_tkey.py, with synthetic keys. Covered: raw payload with slack, compressed detection, RBlob refused, the threshold at 4 / 1000 / 1004, and non-ASCII names (café and a lone 0xff). 4 of the 7 fail on main. Full suite: 361 passed, 53 skipped, 46 xfailed.

Assisted-by: claude-code:claude-opus-5-5

- A key payload is compressed iff fObjLen > fNbytes - fKeyLen, the test at
  all of ROOT's read sites (root-io-spec Compression §1), in read_object
  and in TKey_header.is_compressed(). A raw payload longer than fObjLen has
  slack; its first fObjLen bytes are the object. The "!=" form read that
  slack as a compression block header.
- read_object refuses a key of class RBlob. Its fObjLen is decorative and
  one blob can hold several pages (root-io-spec RNTuple NOTES 1), so even
  the correct test would silently truncate it; RNTuple bytes are found
  through the anchor's envelope locators and the page lists.
- A key is large when fVersion > 1000, as in TKey.cxx, so is_short() is
  <= 1000 and update_members uses it (root-io-spec Record §2 and its
  erratum 9). Differs only at exactly 1000.
- Key names are uninterpreted bytes (Conventions §5.1): TKeyList decodes
  them as UTF-8 with surrogateescape instead of ASCII, so a non-ASCII name
  no longer raises and every name maps back to exactly its bytes.

Fixes nsmith-#77
Part of nsmith-#87

Assisted-by: claude-code:claude-opus-5-5
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.94%. Comparing base (3f43287) to head (437202a).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #130      +/-   ##
==========================================
+ Coverage   91.82%   91.94%   +0.11%     
==========================================
  Files          37       37              
  Lines        2962     2966       +4     
==========================================
+ Hits         2720     2727       +7     
+ Misses        242      239       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TKey: "is the payload compressed" must test fObjlen > fNbytes - fKeylen, not !=

2 participants