Skip to content

pyarrow: lifting the <17 ceiling is a three-repo fixture re-vendor, not a dependency bump (C-72) #174

Description

@Polichinel

Register C-72. Surfaced by reviewing the 32 Dependabot alerts raised when main was brought current.

The CVE

GHSA        use-after-free reading IPC files with pre-buffering   HIGH
vulnerable  >= 15.0.0, < 23.0.1
patched     23.0.1
our pin     >= 16.1.0, <  17.0.0    ← inside the range; the ceiling excludes the fix

pyproject.toml:15. It is the only one of the 32 alerts against a dependency this repo declares — the other 31 are poetry.lock resolution from views-pipeline-core 2.3.0's tree (C-62).

This repo is not exploitable, and I want to be precise about why

The defect is on the read path. Production only writes arrow (contract/wire/shard.pyviews_frames.io.arrow). The only three vf_arrow.load calls in the repository are byte-parity tests reading fixtures we generated ourselves. No partner or producer input reaches the vulnerable code in this process.

views-faoapi does read our shards — but it pins pyarrow==23.0.1, the patched version. So nobody on the platform is currently exposed through this path.

The actual problem is not the CVE — it is what fixing it costs

Upgrading to 23.0.1 changes the bytes this repo emits. That is not speculation: the five byte-parity tests that fail locally in this repo fail because the local environment already runs 23.0.1 while CI runs the pinned 16.1.x, and the two produce different bytes for identical input.

And per ADR-013 §10:

All three implementing repos' test suites consume the same bytes … The other two repos vendor a copy and carry a pinned root-hash equality testA change to the fixture is a change to the contract.

Verified: views-faoapi vendors it at tests/forecast/golden/wire_contract/SHA256SUMS with tests/forecast/test_wire_golden_fixture.py.

So the upgrade is a coordinated three-repo event, not a poetry update.

What must happen, in order

  1. views-pipeline-core #280 lifts the platform pyarrow < 17 ceiling (the viewser → views-storage chain is the real constraint)
  2. This repo bumps the pin and regenerates tests/fixtures/wire_contract/ + SHA256SUMS
  3. The new root hash is recorded in the fixture README and ADR-013's post-adoption record
  4. views-faoapi and views-pipeline-core re-vendor and re-pin
  5. contract_version — decide whether this is a version bump. §10 says a fixture change is a contract change; the payload schema does not change, only the encoder's bytes. This is the open decision.

Worth knowing

views-faoapi already pins pyarrow==23.0.1 — it is past the ceiling that pipeline-core #280 exists to lift. So the "platform-wide <17 ceiling" is not uniform today, and whatever #280 concludes should account for one consumer having already moved.

Not urgent, and this issue says so deliberately

Nothing is exposed. This is filed so that whoever eventually runs a dependency update finds the coordination requirement before regenerating bytes three repos pin, rather than discovering it from a red CI in someone else's repository.

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-decisionRequires a human decision before proceeding

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions