feat: add hooks for working with graphed-org/graphed by lgray · Pull Request #721 · scikit-hep/hist

lgray · 2026-09-14T14:18:28Z

Add hooks to hist that allow it to be used with the delayed execution engine "graphed", it has stabilized enough to where it makes sense to support the code upstream and ease installation.

The graphed hooks ride hist's own CI: graphed, graphed-histogram and graphed-executors (all on PyPI at 0.0.2) are entries in the test dependency group behind python_version >= "3.11" (graphed requires 3.11+), so tests/test_graphed.py runs on the 3.11–3.14 checks legs and leaves the 3.10 checks/mypy/minimums jobs and the core-only 3.15 leg untouched; the >=0.0.2 floors keep "Check minimums" off the 0.0.1 releases. One test (test_uproot_ttree_fill_end_to_end) stays skipped upstream: it needs uproot.graphed, which arrives with scikit-hep/uproot5#1720, and it skips rather than errors on an installed uproot that predates it.

The pre-commit fixes are committed by hand because pre-commit.ci cannot push to a fork owned by an organization.

lgray · 2026-09-14T17:08:02Z

@henryiii this one is good to go now - please have a look

…freeze-HIST-1) src/hist/graphed mirrors src/hist/dask over graphed-histogram: Hist/NamedHist with QuickConstruct and named-axis fills recording into the graphed IR; compute() returns a real hist.Hist. Pinned: deferred == eager twins bit for bit over graphed-numpy, graphed-awkward (ragged fills flatten), and a real uproot TTree; weighted 2D NamedHist; multi-fill; ProcessExecutor plan == compute; the whole-dataset loader never runs. Full hist suite green alongside (200 passed). CI workflow 'graphed' runs the full suite + integration on the graphed-mvp branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…group test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

USER-DIRECTED (graphed-histogram freeze-M23-1): plan() + an R7 executor aggregates; hist.Hist(value)/hist.NamedHist(value) wrap results back in-memory (names/labels survive via the spec's axis-__dict__ handling); the unused _in_memory_type property is gone. Same pins; full hist suite green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

….write) The reference runner moved to the execution contract; graphed.write no longer exposes it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The 8 graphed-*-mvp prototype packages are now one pip-installable distribution `graphed`. Rewrite the merged import roots in the graphed integration test (graphed_core/graphed_numpy/graphed_awkward -> graphed.core/.numpy/.awkward); hist.graphed's own graphed_histogram / graphed_exec_local deps stay separate. CI graphed.yml installs the consolidated `graphed[awkward,numpy]` (one git URL in place of the CORE/DEBUG/FRONTEND/NUMPY/AWKWARD siblings) plus graphed-exec-local, graphed-histogram, and the uproot fork. Assisted-by: ClaudeCode:claude-opus-4.8

…ecation) graphed-exec-local deprecated ProcessExecutor on 2026-06-17; the warning fails under this suite's `filterwarnings = error`. Use the non-deprecated parent (same behaviour) in the process-executor integration test. Assisted-by: ClaudeCode:claude-opus-4.8

… rename CI now installs graphed-executors @ .../graphed-executors and graphed-histogram @ .../graphed-histogram; the executor importorskip targets graphed_executors.local. Assisted-by: ClaudeCode:claude-opus-4.8

@main

…ased) Both are published on PyPI, so switch their CI env vars from git+@main to the released dists. graphed stays git+@main (active development); the uproot/hist forks stay on their branches. Assisted-by: ClaudeCode:claude-opus-4.8

Upstream hist's `nox -s mypy` (the "Type check" CI job, absent from the older base this fork was cut from) type-checks all of `src` — including `src/hist/graphed/` — in an env that installs only hist's own `test`/`plot` groups, not `graphed_histogram`. So `import graphed_histogram.boost` raised `import-not-found` (4 errors, 2 files). Add `graphed_histogram.*` to the existing untyped-external `ignore_missing_imports` override, alongside scipy/matplotlib/mplhep/etc. Correct whether or not the package is installed (an ignore-missing-imports override is a no-op when the module is found), so it needs no per-line `# type: ignore` that strict mode would flag unused where the dep is present. Verified: `nox -s mypy` -> "Success: no issues found in 30 source files". Assisted-by: ClaudeCode:claude-opus-4.8

`hist.graphed.Hist`/`NamedHist` inherit `BaseHist.fill`, which resolves EVERY keyword through `_name_to_index` before delegating to `graphed_histogram.boost.Histogram.fill`. So graphed's two fill-MODE flags — `variation_axis=` (m52 axis mode) and `unweighted=` — were read as axis names and died: ValueError: The axis name variation_axis could not be found Add one `FillModeMixin`, mixed into both graphed hist classes ahead of the hist bases (the single point both ghb.Histogram subclasses route through). With neither flag set it delegates to `super().fill(...)` verbatim (sibling/default mode byte-identical to before); with one set it resolves the axis order itself via `_name_to_index` and calls `ghb.Histogram.fill` directly. Tests: `fill(..., variation_axis=True)` through hist.graphed now equals the low-level positional fill (fails on origin/graphed-mvp with the exact ValueError); a 2-D out-of-order kwarg fill matches the positional fill and NOT a swapped one (axis order); a no-flag missing-axis fill still raises hist's message (the flag path engages only for flags); and `unweighted=True` with a weight= factor is forwarded to graphed's refusal. All three properties verified discriminating by mutation. Assisted-by: ClaudeCode:claude-opus-5

The `graphed` integration workflow pinned graphed / graphed-executors / graphed-histogram from PyPI ("now released"), but the hist.graphed tests use m52 features that land on the repos' main branches ahead of any PyPI release: `graphed.vary`/`graphed.points` and the `variation_axis`/`unweighted` histogram fill modes. Post-merge, the released PyPI graphed had neither, so the graphed-mvp push CI failed: AttributeError: module 'graphed' has no attribute 'vary' TypeError: Histogram.fill() got an unexpected keyword argument 'unweighted' Point the three co-developed packages at git main (uproot already installs from git), matching what the local editable dev stack tests against. Repin to PyPI once a release carries m52. Assisted-by: ClaudeCode:claude-opus-4.8

The graphed.yml workflow and the .graphed/ tracking directory belong to the graphed-project development pipeline, not to hist; upstream CI covers the hist.graphed hooks once graphed is a test dependency. Assisted-by: ClaudeCode:claude-opus-5[1m]

tests/test_graphed.py is guarded by importorskip, so without this the hist.graphed hooks are never exercised in CI. graphed needs Python 3.11+, so the marker keeps the 3.10 jobs (checks, mypy, minimums) unchanged; the floors keep the lowest-direct "Check minimums" resolution off the 0.0.1 releases. Assisted-by: ClaudeCode:claude-opus-5[1m]

Satisfy the repo's ruff-check/ruff-format hooks on tests/test_graphed.py: unused PartitionedSource.read_partition arguments become _columns/_resources, two compound asserts are split, imports sorted, dead noqa dropped, reformatted. Assisted-by: ClaudeCode:claude-opus-5[1m]

The hist.graphed docstring and the test-module header carried graphed-project work-item and freeze-tag labels an upstream reader cannot resolve; they now state the same behaviour in plain words. Assisted-by: ClaudeCode:claude-opus-5

An installed uproot without the graphed reader (any release before scikit-hep/uproot5#1720) raised AttributeError instead of skipping. Assisted-by: ClaudeCode:claude-fable-5-1

lgray · 2026-09-15T13:36:58Z

@henryiii ping!

lgray · 2026-09-16T18:39:41Z

@henryiii ping!

lgray · 2026-09-17T13:52:19Z

@henryiii I am once again asking you for a review!

henryiii · 2026-09-17T15:28:28Z

Seems fine, little hard to review without more context into graphed. A few questions:

The names exactly match. Is that best? That is, Hist vs. GraphedHist, etc. Reprs won't show which one you have, for example.
Given there already is a graphed_histogram, it seems like the integration could live there? I think the only internal API you are using is _name_to_index? (I'm fine with it here, just checking)
from .. import hist then hist.Hist might be better than as HistInMemory
Maybe there should be a graphed extra?
FWIW I'll drop 3.10 support in a couple of weeks, probably right after the next release.

lgray · 2026-09-17T16:43:00Z

@henryiii The graphed subdir implementation mirrors exactly that of how it was done for dask-histogram. The reason I chose this was so that the semantics are the same and the namespace tells you what mode you're operating in.

i.e. hist.Hist and hist.graphed.Hist and hist.dask.Hist all get you a histogram you create by invoking Hist.new that all take the same arguments. To me, this makes switching between modes easier for code since you can just change the namespace and most of the code remains the name.

I'm also taking precedent here since hist.dask is implemented in the same way.

I would be fine with altering the reprs to show that these are dask or graphed histograms rather than in-memory histograms.

For where this lives - I think it makes the most sense for all the hist extensions to histograms to live in hist, rather than have a diaspora, especially when they are this thin.

I found calling it HistInMemory makes the distinction between the delayed object and the final render histogram a bit simpler, but that's just me.

Do we need a graphed extra when people will be using this typically only when they have graphed (or dask in the dask) case already installed? I think the current complaint on missing import is good enough. It makes people stop to think what they are doing.

lgray · 2026-09-17T17:04:43Z

As for graphed:

It is a task-graph engine similar to dask but with better scaling properties and a HEP-opinionated stance.

lgray force-pushed the graphed-mvp branch from 9bd6dde to b4cb743 Compare September 14, 2026 14:18

lgray marked this pull request as ready for review September 14, 2026 17:07

lgray requested a review from henryiii September 14, 2026 17:08

lgray and others added 18 commits September 14, 2026 10:38

ci: hist's test deps are a PEP 735 dependency group — install with --…

329f859

…group test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ci: graphed-histogram repo renamed to graphed-histogram-mvp

aace67f

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

M32: import SequentialRunner from graphed_core.execution (was graphed…

83c3b1e

….write) The reference runner moved to the execution contract; graphed.write no longer exposes it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

use graphed from pypi as well

3d27322

style: ruff

44f6358

Satisfy the repo's ruff-check/ruff-format hooks on tests/test_graphed.py: unused PartitionedSource.read_partition arguments become _columns/_resources, two compound asserts are split, imports sorted, dead noqa dropped, reformatted. Assisted-by: ClaudeCode:claude-opus-5[1m]

test: skip the uproot end-to-end fill when uproot lacks uproot.graphed

8bc3281

An installed uproot without the graphed reader (any release before scikit-hep/uproot5#1720) raised AttributeError instead of skipping. Assisted-by: ClaudeCode:claude-fable-5-1

lgray force-pushed the graphed-mvp branch from deaae37 to 8bc3281 Compare September 14, 2026 17:38

henryiii approved these changes Sep 17, 2026

View reviewed changes

henryiii merged commit 4d1e354 into scikit-hep:main Sep 17, 2026
13 checks passed

This was referenced Sep 19, 2026

feat: cluster a graphed array with ClusterSequence scikit-hep/fastjet#401

Merged

feat(nanoevents): NanoEventsFactory.from_root(mode="graphed") scikit-hep/coffea#1637

Open

Navigation Menu

feat: add hooks for working with graphed-org/graphed - #721

feat: add hooks for working with graphed-org/graphed#721
henryiii merged 18 commits into
scikit-hep:mainfrom
graphed-org:graphed-mvp

lgray commented Sep 14, 2026 •

edited

Loading

Uh oh!

lgray commented Sep 14, 2026

Uh oh!

lgray commented Sep 15, 2026

Uh oh!

lgray commented Sep 16, 2026

Uh oh!

lgray commented Sep 17, 2026

Uh oh!

henryiii commented Sep 17, 2026 •

edited

Loading

Uh oh!

lgray commented Sep 17, 2026

Uh oh!

lgray commented Sep 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lgray commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lgray commented Sep 14, 2026

Uh oh!

lgray commented Sep 15, 2026

Uh oh!

lgray commented Sep 16, 2026

Uh oh!

lgray commented Sep 17, 2026

Uh oh!

henryiii commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lgray commented Sep 17, 2026

Uh oh!

lgray commented Sep 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lgray commented Sep 14, 2026 •

edited

Loading

henryiii commented Sep 17, 2026 •

edited

Loading