Skip to content

L2 call graph is not byte-reproducible: Jedi overload resolution for open() varies run to run #146

Description

@rahlk

Problem

Two byte-identical invocations of the analyzer produce different call graphs. The
variance is small but real, and it is not the sharding defect from #145 — it
survives with --pycg-shard sequential, with --ray, and at any --pycg-max-iter.

Measured on test/fixtures/whole_applications/flask (82 files), -a 2 --no-venv --pycg-shard --pycg-shard-ceiling 25 --clear-cache, two runs per configuration:

config edges A / B differing edges
sequential, --pycg-max-iter 50 1263 / 1263 1
sequential, --pycg-max-iter 3 1263 / 1263 2
--ray, --pycg-max-iter 50 1168 / 1168 3
--ray, --pycg-max-iter 3 1168 / 1167 2

0.1–0.3% of edges. Ray adds no meaningful variance, so this is not a scheduling
artifact.

Root cause

Almost every differing edge involves file-IO types — _TextIOBase,
_BufferedIOBase, FileIO. The reproducible instance is
src/flask/cli.py:1004:

with open(startup) as f:
    eval(compile(f.read(), startup, "exec"), ctx)

open() is overloaded in typeshed (text mode → TextIOWrapper, binary →
BufferedReader). Jedi's overload resolution returns a different candidate set
on different runs
, so f.read() resolves to either:

can://python/flask/src/flask/cli.py/shell_command() -> …/@external/builtins._TextIOBase/read
can://python/flask/src/flask/cli.py/shell_command() -> …/@external/builtins._BufferedIOBase/read

SymbolTableBuilder._first_definition is already a deterministic picker —
min(definitions, key=lambda d: (d.full_name or "", d.name or "")). Given both
candidates it would always choose _BufferedIOBase (B < T). The instability
is in the candidate set Jedi hands us, not in our selection.

PYTHONHASHSEED=0 is already pinned for the driver (#99) and propagated into Ray
workers via runtime_env.env_vars (core.py:48), so hash randomisation is not
the cause.

Caveats and known risks

Definition of done

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions