Skip to content

fix(codegen): withhold in_place outputs from dependents - #401

Open
raphaelvigee wants to merge 1 commit into
raphaelvigee/fix-codegen-copy-xattrfrom
raphaelvigee/in-place-outputs-not-consumable
Open

fix(codegen): withhold in_place outputs from dependents#401
raphaelvigee wants to merge 1 commit into
raphaelvigee/fix-codegen-copy-xattrfrom
raphaelvigee/in-place-outputs-not-consumable

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Aug 16, 2026

Copy link
Copy Markdown
Member

Stacked on #398.

A codegen = "in_place" output is a tracked source file. It stays visible to glob()/file() — it must, since it is what the transform reads — and unlike a copy output there is no provenance stamp that could hide it, because hiding it would hide the source.

So when a dependent could also take those bytes as artifacts, the same file reached it by two paths, and those paths disagreed for as long as the transform had run and the write-back had not. heph's one-producer-per-file rule had nothing to arbitrate with: for copy the xattr decides, and here nothing can.

The change

A dependent of an in_place target now receives every output group except the in_place ones. Naming one explicitly is an error:

'//pkg:fmt' output 'src' is `codegen = "in_place"`: those are tracked source files,
not artifacts a dependent can consume. Read them from the tree with glob()/file();
keep the dep if you need the transform to run.

This costs the dependent nothing. After the write-back the tree holds exactly the transformed bytes, so reading them through glob() yields what the artifact would have — by the single path that is always right.

The edge keeps its other two jobs:

Groups

Transparent groups do not go through link — they inline their members' results directly — so they would have been a way around the rule: wrap the formatter in a group, depend on the group. They now ask consumable_outputs for the same answer.

Two cases deliberately keep OutputMatcher::All: a member with no in_place output (nothing to withhold, and it keeps the group path allocation-free), and a member that is itself a group (it applies the rule to its own members, and declares no outputs of its own to enumerate — enumerating would have starved nested groups, which a_group_still_passes_a_plain_members_outputs_through pins).

Tests

  • in_place_outputs_are_not_staged_into_a_dependentconfirmed to fail without the change, not just to pass with it: the consumer's test -z "$SRC_SRC" trips on the staged file.
  • naming_an_in_place_output_as_a_dep_fails — the explicit-group error and its wording.
  • a_group_does_not_leak_its_in_place_members_outputs — the group path.
  • a_group_still_passes_a_plain_members_outputs_through — a group of a group still surfaces the leaf's output.

Blast radius

Nothing in tree consumed an in_place output. example/fmt has no dependents, and the Go lint --fix driver — the only other producer — is depended on by nothing; its e2e suite passes (4/4).

Known wart, not addressed here

A dependent's @heph/fs glob and its codegen dep resolve concurrently in inputs_result_meta, and glob results are memoized per request_id — so whether the dependent hashes pre- or post-write-back bytes is an ordering race, self-correcting on the next run. deps = ["//pkg:fmt"] therefore cannot be relied on to sequence formatting before a read. The real fix is a phase boundary (codegen settles, then the graph hashes), not a dependency edge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W9pJTeJGLJva1b2ZCJGtgz

@raphaelvigee
raphaelvigee marked this pull request as ready for review August 16, 2026 20:52
An `in_place` output *is* a tracked source file. It stays visible to
`glob()`/`file()` — it must, since it is what the transform reads — and
unlike a `copy` output there is no provenance stamp that could hide it,
because hiding it would hide the source. So when a dependent could also
take those bytes as artifacts, the same file reached it by two paths that
disagree for as long as the transform had run and the write-back had not,
and heph's one-producer-per-file rule had nothing to arbitrate with.

Withhold them. A dependent of an in_place target now receives every group
except the in_place ones, and naming an in_place group explicitly
(`//pkg:fmt|src`) is an error that says where to read the files instead.

This costs the dependent nothing. After the write-back the tree holds
exactly the transformed bytes, so reading them through `glob()` yields
what the artifact would have — by the single path that is always right.
And the edge keeps its other two jobs: the target is still resolved (so
the transform runs, and lands in the tree), and its hashout still folds
into the dependent's `hashin`, because `inputs_result_meta` reads
hashouts with `OutputMatcher::None` and never consulted this list.

Transparent groups do not go through `link`, so they would have been a
way to smuggle the outputs past the rule — wrap the formatter in a group,
depend on the group. They now ask `consumable_outputs` for the same
answer. A member with no in_place output, and a member that is itself a
group, both keep `OutputMatcher::All`: the former has nothing to
withhold, the latter applies the rule to its own members and declares no
outputs of its own.

`in_place_outputs_are_not_staged_into_a_dependent` was confirmed to fail
without the change (the consumer's `test -z "$SRC_SRC"` trips on the
staged file), not just to pass with it.

Nothing in tree consumed an in_place output: `example/fmt` has no
dependents, and the Go `lint --fix` driver — the only other producer — is
depended on by nothing. Its e2e suite passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W9pJTeJGLJva1b2ZCJGtgz
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/in-place-outputs-not-consumable branch from 5ddff64 to 3a68efb Compare August 17, 2026 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant