Skip to content

fix: keep TTree branch lookup indices valid when a generated counter collides - #1698

Open
ariostas wants to merge 3 commits into
scikit-hep:mainfrom
ariostas:fix-cascadetree-counter-lookup
Open

ariostas wants to merge 3 commits into
scikit-hep:mainfrom
ariostas:fix-cascadetree-counter-lookup

Conversation

@ariostas

@ariostas ariostas commented Aug 5, 2026 •

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Addresses finding 1 of "PR 5" in #1688.

Tree._branch_lookup maps a branch name to its index in Tree._branch_data. When a jagged branch's generated counter name collides with a branch that was already declared, the colliding datum was removed with del:

https://github.com/scikit-hep/uproot5/blob/main/src/uproot/writing/_cascadetree.py#L203-L207

which shifts every later datum down by one, while _branch_lookup keeps pointing at the old indices.

f.mktree("t", {"nx": np.dtype("int32"), "y": np.dtype("float64"), "x": <jagged>})
# _branch_lookup == {'nx': 1, 'y': 1, 'x': 2}

y aliases the generated counter, and extending the tree fails.

Changes, depending on what the counter collides with:

  • A scalar integer branch (the documented case of providing a counter explicitly): the datum is replaced in place instead of deleted. That also keeps the counter ahead of the jagged branch it counts, since a colliding name must have been declared before the branch that generates it.
  • Another generated counter (jagged branches sharing a counter, e.g. counter_name=lambda n: "nJet" for Jet_pt and Jet_eta): the existing counter datum is reused. Previously the earlier branch kept a reference to a counter that was never written, and writing the tree failed with a struct.error. extend also failed when comparing the shared counter, because awkward.to_numpy rejects its big-endian dtype; it now uses numpy.asarray.
  • Anything else (a float, a fixed-size array, a jagged branch, a record, or a record field): mktree now raises a ValueError. Previously the declared branch was silently replaced, leaving orphaned branches (the jagged branch's own counter, or the record's fields) that made extend fail with unrelated errors.
  • A record or record field declared after the counter: mktree also raises a ValueError. Previously the record or field was silently dropped, or, for a dict record, appended as a second branch with the counter's name.

Note this needs mktree to reproduce: since v5.7.0 dict assignment produces an RNTuple, which does not go through this code.

Tree._branch_lookup maps a branch name to its index in Tree._branch_data. When
a jagged branch's generated counter name matched a branch that had already been
declared, the colliding datum was removed with 'del self._branch_data[...]',
which shifts every later datum down by one while _branch_lookup keeps pointing
at the old indices.

mktree with scalar 'nx', scalar 'y' and jagged 'x' produced the lookup
{'nx': 1, 'y': 1, 'x': 2}: 'y' aliased the generated counter, and extending the
tree failed. Replace the datum in place instead, which also keeps the counter
ahead of the jagged branch it counts.

Assisted-by: claude-code:claude-opus-5[1m]
@codecov

codecov Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.91304% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.16%. Comparing base (e598b61) to head (d75479e).
⚠️ Report is 22 commits behind head on main.

Files with missing lines Patch % Lines
src/uproot/writing/_cascadetree.py 73.91% 3 Missing and 3 partials ⚠️

❌ Your patch check has failed because the patch coverage (73.91%) is below the target coverage (98.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
Files with missing lines Coverage Δ
src/uproot/writing/_cascadetree.py 85.10% <73.91%> (+0.99%) ⬆️

... and 20 files with indirect coverage changes

@TaiSakuma TaiSakuma added the type/fix PR title type: fix (set automatically) label Aug 14, 2026
Jagged branches that share a counter name now share the counter datum,
instead of the later one orphaning the earlier one's counter. extend no
longer fails comparing shared counters because of their big-endian dtype.
A generated counter only replaces a scalar integer branch; colliding with
any other branch raises a ValueError. Tests are condensed and renamed
after the PR number.

Assisted-by: Claude Code:claude-opus-5-5
@ariostas
ariostas marked this pull request as ready for review September 24, 2026 19:23
@ariostas
ariostas requested review from ianna and a lite review from Copilot September 24, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Record/counter collision cases can still be silently ignored or create duplicate/orphaned branches.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

Fixes TTree branch lookup corruption caused by generated counter-name collisions during mktree and extend.

Changes:

  • Replaces scalar counter collisions in place.
  • Reuses compatible generated counters.
  • Rejects incompatible collisions and adds regression tests.
File Summary
src/​uproot/​writing/​_cascadetree.py Updates counter collision and comparison handling.
tests/​test_1698_cascadetree_counter_collisions.py Adds collision regression tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/uproot/writing/_cascadetree.py
A record or record field whose name matches a counter generated by an
earlier jagged branch was silently dropped, or for dict records appended
as a second branch with the same name. Such collisions now raise a
ValueError, and dict record fields no longer duplicate existing names.

Assisted-by: Claude Code:claude-opus-5-5

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/fix PR title type: fix (set automatically)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants