Skip to content

rfc: branch operations v2 - #677

Closed
aaltshuler wants to merge 2 commits into
mainfrom
codex/branch-ops-v2
Closed

aaltshuler wants to merge 2 commits into
mainfrom
codex/branch-ops-v2

Conversation

@aaltshuler

Copy link
Copy Markdown
Contributor

What & why

Branch operations still reconstruct growing manifest history, and ancestry-eligible merges can replay source data into target-owned datasets. Propose RFC 0060 to replace current-state reconstruction with explicit immutable graph snapshots and conditional adoption commits, preserving the target's first-parent history.

The draft defines one current __manifest authority for logical refs, exact retained snapshot roots, independent writes after adoption, graph-wide immutable tags, historical reads/forks, merge preview, and safe logical deletion. Whole-state restore follows; rebase, cherry-pick, and selective revert require a later logical-change replay contract.

This PR changes only the RFC and registry. It remains draft / not-started and does not accept a new format or activate runtime behavior.

Backing issue / RFC

Related work: accepted RFC 0057 / #638, branch-context and retained-ancestry work in #662, and bounded merge preparation in #668. The RFC records scoped relationships to existing authority, retention, recovery, feed, and GQ decisions. No existing decision is superseded by this draft.

Notes for reviewers

  • The candidate uses strict native Lance Overwrite with engine-owned fresh retries. Cold latest-version discovery, central-CAS contention, and retained metadata amplification require qualification.
  • Adoption must preserve source isolation, first writes, tags, Blob dependencies, and deletion/cleanup safety. Initial destructive cleanup requires a quiesced graph; no new distributed fencing is claimed.
  • Existing-graph cutover must preserve selected contents and complete parent-record closure, or explicitly refuse/reset identities and feed baselines.
  • The completed AWS campaign is divergent-merge bottleneck evidence: 50 native-S3 measurements and eight NVMe controls. It does not measure promotion speedup. Small eligible fixtures, full first-use/reclamation lifecycles, and activation gates are specified separately.

Checklist

  • One focused RFC and registry update.
  • Tests for behavior changes: N/A; documentation only. Existing test owners and future qualification gates are specified.
  • Public supported-behavior docs: N/A; no new behavior ships.
  • Reviewed against architectural invariants and pinned Lance 11 surfaces; unresolved representation/protocol gates remain explicit.

Local verification

  • python3 scripts/check-docs.py — passed (132 Markdown files).
  • bash scripts/check-agents-md.sh — passed.
  • typos docs/rfcs/0060-branch-ops-v2.md docs/rfcs/README.md — passed.
  • git diff --cached --check — passed.
  • Independent design and benchmark-evidence review — completed.
  • Rust builds, tests, DST, and benchmarks — not rerun for this documentation-only PR.

@ragnorc

ragnorc commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

A current-format slice of the "Fork current/retained snapshot" row lands separately in #1016: branch create takes a published graph commit id in a typed from_commit field (--from-commit on the CLI, from commit "<id>" in GQ), exclusive with from. It stays within the current authority model rather than this RFC's new one, and follows the RFC's rules where they overlap:

  • Retained roots only, typed refusal otherwise. The fork clones the __manifest version that published the commit (Lance create_branch at that version on the writing incarnation's tree), so the first manifest names the commit's pins and lineage in the one native create. Before the create, every table version the commit pins must open with every object its manifest references; anything missing is refused (not_retained, HTTP 410), never approximated. Opening alone is not proof: the collector deletes a version's files before its manifest, so after an interrupted run a manifest can open over deleted objects. Pins the source head also holds are roots and skip the probe.
  • Same accepted schema. A commit whose schema contract is not the accepted one is refused (schema_changed), as §5 asks for forks until historical SchemaIR retention is qualified. A version only the legacy history serves is refused too (earlier_storage_format).
  • Retention after the fork. The collector already keeps a named branch's base version and its pins, so the forked commit stays retained while the branch lives. No new pin, tag or refcount.
  • Cleanup fencing. Fork and cleanup stay in the existing single-writer-process control envelope (cleanup holds every branch gate; the fork holds its source's). Nothing here claims the online/distributed GC fencing this RFC defers.
  • Bare commit ids are not data pins. As the RFC says, only the branch the fork creates retains the commit; before the fork, retention depends on cleanup policy.
  • Unpublished proposals and unattached snapshots are refused. Neither is branch history, and both are detached __manifest versions, which Lance cannot branch from with a head.

When this RFC's snapshot authority lands, forking from a retained root or a tag should replace this mechanism without changing the wire spelling: from names a branch, from_commit a commit, and a tag would get its own field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants