Skip to content

Readers that merge distinct on-disk values: kBool, negative char* lengths, long-form short TStrings #140

Open

Description

@nsmith-

🤖 AI generated content

Rule (AGENTS.md, #134): a value read as a Python builtin must decode injectively on every input the reader accepts. Inputs that would decode to the same value are rejected or kept, not merged; otherwise a writer can't put back the bytes that were read. Three readers on main (66b9290) merge distinct inputs. Each case is small; they're filed together because they are the same problem.

1. kBool read as bool. TStreamerInfo.py:277 maps kBool to ">?", and struct turns any nonzero byte into True, so a stored 0x99 writes back as 0x01.

  • Rejecting is not an option here. ROOT writes 0x99 for a persistent member that no code path assigns, and a reader must not validate the field (root-io-spec ElementTypes §2.5, TStorage.cxx:291-295). Fixture: classes/formula byte 665, TFormula::fAllParametersSetted.
  • So the byte has to be kept: e.g. the raw u8 with a truthiness property, or a bool-like type that remembers its byte.
  • numpy bool arrays (kArrayBool, kOffsetP + 18) need checking: frombuffer may keep the raw bytes, but that isn't guaranteed.

2. ROOTString("charstar") maps every n <= 0 to b"". structutil.py's ROOTString.read does length = max(length, 0). ROOT writes 0 for both a null pointer and an empty string (TBufferFile.cxx:306-314; root-io-spec Conventions §5.4), so 0 → b"" is fine. A negative length is never written, so it should raise, not read as b"".

3. ROOTString("TString") accepts the long form for short strings. 255 then a u32 n <= 254 reads as the same bytes as the short form, and would be written back short. ROOT uses the long form only for n > 254 (TBufferFile.cxx:215; root-io-spec Conventions §5.1: "a leading 0xFF never means 255 characters"). ROOT's reader accepts the long form either way, so rejecting it refuses a file ROOT would read. No known writer produces it, so I'd reject with a clear error, but keeping a flag is the alternative.

Tests: 1 against classes/formula byte 665. 2 and 3 with synthetic bytes beside tests/test_root_string.py's existing byte-level cases, since no writer produces them.

Related: #134 (the rule), #139 (where each value's ROOT type is recorded), #123 (TDatime, which has the same question for invalid date fields).

Assisted-by: claude-code:claude-opus-5-5

Activity

  1. sathabbott commented on Oct 1, 2026

    @sathabbott
    Collaborator

    🤖 AI generated content

    One more case of the same kind: framed reads check the stream header's byte count but drop its version. A std::string data member (ROOTString("TString", framed=True)) and the StdVector, StdSet and StdMap readers keep only the payload, so headers that differ only in the version word read the same. For collections that word is the writing ROOT's TStreamerInfo class version (8, 9 or 10, root-io-spec Collections §2), so it does vary between files. Does this belong here, or with #105, as keeping the stream header next to the value?

  2. nsmith- commented on Oct 2, 2026

    @nsmith-
    OwnerAuthor

    The std collections probably need the same holistic header treatment as discussed in #105, to be designed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions