Deprecate Python dataclass annotations as the serialization definition for streamer-generated dataclasses #67

🤖 AI generated content

Scope

For classes generated from a file's TStreamerInfo, stop using the generated Python dataclass (its base-class list and its Annotated[...] member annotations, consumed by @serializable) as the definition of how to deserialize. The dataclass stays as the deserialized representation; at runtime the streamer info itself — or a data model transformed from it — steers the deserialization into the dataclass.

Hand-written bootstrap classes are out of scope: they can keep defining their layout by annotation.

This issue absorbs #22 and the "Memberwise streaming not implemented for TObject base class" line item of #9 (which turned out to be an element-ordering bug, see below).

Planning: TODO. This issue only ties together the open items that motivate the change or that it would unblock.

Items this ties together

• On-disk element order cannot be expressed through Python inheritance (6 xfails in Checklist for remaining types in scikit-hep/testdata #9: uproot-issue-76, uproot-issue407, uproot-issue408, uproot-issue434, uproot-issue433-splitlevel2/4). JTRIGGER::JPMTSelector is class JPMTSelector : public std::vector<JPMTIdentifier_t>, public TObject; its info holds a TStreamerSTL named vector<JTRIGGER::JPMTIdentifier_t> followed by a TStreamerBase named TObject. We generate class JTRIGGER3a3aJPMTSelector(TObject) with the vector as a member, and _read_all_members (src/rootfilespec/bootstrap/streamedobject.py:248) reads Python bases before update_members, so the TObject read lands on the member-wise collection frame 40 00 00 3e 40 09 …. The xfail message blames member-wise streaming of the TObject base; the cause is ordering. Spec: Streamer-driven reading §3 "The element loop" ("the order on disk is the order in the array"), §4.3 "A base class that is an STL container" (names this class and these files), and §10 invariant 4. docs/design.md already notes that dataclass member order under multiple inheritance differs from the declared base order.
• Append version to all class names #22 — selecting a streamer info by version / checksum. Schema evolution §4: version word > 0 → match fClassVersion; ≤ 0 → checksum lookup (or no checksum if an entry has fClassVersion == 0); foreign classes are all version 1 and distinguished only by checksum. Today DynamicFileContext.type_by_name accepts expect_version and ignores it, and dynamic.py:23-32 keeps the last info per name. Decisions recorded in Append version to all class names #22: the generated model name carries the checksum (the class version goes in the class docstring), and exact duplicates can be dropped (they are identical modulo fBits, see the evidence in Append version to all class names #22).
• StreamHeader uses the fByteCount >= 6 heuristic to decide whether a checksum follows a version word of 0 #111 — whether a checksum follows a version word of 0 must be decided from the file's streamer infos at the dispatch site, which the current per-class read cannot do.
• A TStreamerBase with fType == -1 (suppressed TObject base) must consume no bytes #94 — fType == -1 (suppressed TObject base) is a property of the element, invisible once bases are Python inheritance.
• Implement member-wise reading of collections (e.g. uproot-issue-268.root) #70 — member-wise streaming traverses the same element list column-wise, and uses TStreamerBase::fBaseVersion because base version words are not on disk there. A per-class update_members cannot express a second traversal order; a runtime element model can.
• Represent string-like members (TString, std::string, char*) as plain bytes in deserialized dataclasses, as promised in docs/design.md #68 and Implement ElementType.kCharStar #20 — once annotations no longer define serialization, members can be the plain Python types promised in docs/design.md (e.g. bytes for TString/std::string/char*).
• Implement double32 type #41 — Double32_t/Float16_t decoding depends on the element's fTitle and its kHasRange bit, i.e. on streamer-element state rather than on the member's Python type.
• Remove hand-written assumed.TFormula (it shadows the file's own StreamerInfo) and fix TStreamerLoop of TString* #72 — TFormula ≤ 8 / ≥ 9 (and TF1 ≤ 7 / ≥ 8) are two unrelated C++ classes under one name (TFormula and TF1, intro and §1.1); checksum-keyed models keep them apart.
• Per-class table for user classes whose recorded StreamerInfo does not describe the bytes (e.g. KM3NETDAQ::JDAQ* in uproot-issue433-splitlevel*.root) #69, RooFit classes with hand-written streamers (RooRealVar, RooLinkedList, ...): implement per root-io-spec RooFit, replacing the framing-layer special cases #71 — per-class overrides for classes whose recorded info does not describe the bytes fit naturally as entries in a runtime model rather than as if cls.__name__ == … branches in the framing loop.
• Honour fArrayLength for pointer codes, TStreamerLoop and TStreamerSTL; keep multi-dimensional shapes #90, Handle version-dependent TStreamerElement / TStreamerBase layouts and the read-time fixups #95, ElementType lacks 85/86/87 (arrays of TString/TObject/TNamed), and an unknown code fails the whole file #91 — element-level features that currently each need a bespoke annotation type.
• TypedDict instead of dataclass? #6 (TypedDict instead of dataclass?) — decoupling the representation from the serialization definition makes the choice of representation cheap to revisit.
• docs/design.md describes read/update_members over the inheritance tree as the mechanism and will need a section for streamer-steered classes.

Assisted-by: claude-code:claude-fable-5-1