Skip to content

[sync] Deleted a file whose last change came from the source, however it first arrived. - #7

Merged
SgtSwagrid merged 2 commits into
mainfrom
feature_last_update_provenance
Oct 26, 2026
Merged

[sync] Deleted a file whose last change came from the source, however it first arrived.#7
SgtSwagrid merged 2 commits into
mainfrom
feature_last_update_provenance

Conversation

@SgtSwagrid

@SgtSwagrid SgtSwagrid commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

syncDeletions kept a file downstream unless the version first added to the target was one that the current source had once had. So a file that came from an earlier source, before its repository was moved beneath a new parent, was kept forever. That earlier copy was a version the new parent never had, even though every later version came from the new parent. Deleting .mcp.json from scala-config stopped at four such repositories (scala-library-template, scala-website-template, asset-loader and themis), which had to be cleaned up by hand.

Now only the file's last change counts. A file is deleted if it still matches a version that the source once had, and was last changed in the target before the source deleted it.

Changes

  • scripts/merge-files.sh:
    • Drops the "didn't originate from the source" check, and with it the walk through the target's history for the version that the file was last added with.
    • It now needs only when the file last changed in the target, which one git log -1 --first-parent gives. That dates a change by when it landed on the branch, as before, including a change that arrived through a merge commit.
    • The "modified" and "changed or re-added after the deletion" checks stay. The comment describing the rule is rewritten to match.
  • README.md: syncDeletions describes the new rule, and that a file the source overwrote, or another source first added, belongs to the source from then on.

Behaviour

  • Newly deleted: a file first added by an earlier source, and later updated by this one.
  • Newly deleted: a file the target had first, which this source then overwrote. Previously kept as "didn't originate from the source".
  • Unchanged: a file modified in the target, or changed or re-added after the deletion, is still kept. A file synced and never touched is still deleted.

Testing

No CI exercises the script, so I ran it before and after the change against a source and target repository built with known commit times, one file per case:

File Old rule New rule
synced, untouched deleted deleted
first added by an earlier source, then updated by this one kept deleted
target's own, then overwritten by this source kept deleted
synced, then modified in the target kept kept
deletion synced, then re-added by the target kept kept
created by the target after the source deleted it kept kept
synced on a branch, merged (merge commit) before the deletion deleted deleted
synced on a branch before the deletion, merged after it kept kept
still in the source copied copied

bash -n passes. git log -1 --first-parent dates a merged change by the merge, not by the branch's commit, which a plain git log -1 would give.

Notes

Every config runs sync.yml@main, so this applies to all syncs as soon as it merges. Files the old rule already kept (fairmap's, basis's and others' .mcp.json) are deleted by the next sync from their source, if they match.

🤖 Generated with Claude Code

SgtSwagrid and others added 2 commits October 26, 2026 15:11
@SgtSwagrid @claude
[sync] Deleted a file whose last change came from the source, however…
b0b5404 
… it first arrived.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SgtSwagrid @claude
[sync] Read when a file last changed in the target with one git log.
36d4d80 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SgtSwagrid
SgtSwagrid merged commit 7ab646f into main Oct 26, 2026
3 checks passed
@SgtSwagrid
SgtSwagrid deleted the feature_last_update_provenance branch October 26, 2026 14:23
@SgtSwagrid SgtSwagrid mentioned this pull request Oct 26, 2026
Merged
SgtSwagrid added a commit that referenced this pull request Oct 26, 2026
@SgtSwagrid @claude
[sync] Synced updates by the same rule as deletions, and removed sync…
3235862 
…Deletions. (#8)

## Summary

Deletions were synced only for a file whose last change in the target
came from the source (#7), but updates still overwrote whatever was
downstream. Now one rule decides every file, whether the source updated,
added or deleted it, and `syncDeletions` is gone.

The rule: every file the source has ever had is given its state in the
source (its latest version, or none if deleted). That happens only if
the file's state in the target is one the source had before the target
last changed it, and the source has changed it since. A file's absence
counts as a source state once the source has deleted it. A file the
target never had is added.

## Changes

- **`scripts/merge-files.sh`**: one pass over every file in the source's
history replaces the old two steps (copy everything, then a separate
deletion check).
- It records each file's current state, when the source last changed it,
and when the source first had each of its states.
- It applies a file's source state only under the rule above. Git's
all-zero blob for a deleted file stands for absence, so additions,
updates and deletions go through the same lines.
- It logs `Added`, `Updated`, `Deleted`, `Kept` (with the reason) or
`Skipped` (ignored); unchanged files are no longer logged as `Copied`.
- It now stops with an error on a shallow clone, as nothing can be
decided without full history (the workflow always fetches it).
- **`scripts/publish.sh`**: `replay()`, which re-applies queued sync
commits onto the latest target branch, uses the same idea. A change,
whatever its kind, is applied only where the file is still as it was
when the change was made. A change already present is skipped silently,
and one to a file the target changed since is kept. Before, updates were
always applied and only deletions were checked.
- **`syncDeletions`** is removed from `parse-config.sh`,
`load-config.sh` and the README. No graph in this account sets it.
- **`README.md`**: a new "Which files are synced" section explains the
rule. The `syncDeletions` section is gone. "Merge semantics" no longer
claims updates overwrite everything, that copies shouldn't be modified,
or that the most recent source wins; with this rule the first source to
sync a file keeps it. The replay remark no longer refers to deletions
alone.
- **`templates/pull-request-body.md`**: the default warning no longer
says files with matching names are overwritten.
- **`sync.yml`**: the comment on full history no longer mentions
deletions alone.

## Behaviour

| Target's file | Before | Now |
|---|---|---|
| synced, untouched; source updates or deletes it | updated / deleted |
updated / deleted |
| modified in the target; source updates it | **overwritten** | kept |
| deleted in the target; source updates it | **re-added** | kept |
| the target's own, from before the source added it | **overwritten** |
kept |
| re-added after its deletion was synced | kept | kept |
| deleted by a synced deletion; source re-adds it | added | added |
| never in the target; source adds it | added | added |
| first from an earlier source, then synced by this one | updated |
updated |
| ignored | untouched | untouched |

So a file changed downstream stops following the source altogether. To
make it follow again, give it the source's latest version.

## Testing

No CI exercises the scripts, so I ran them against repositories built
with known commit times:

- **`merge-files.sh`**: one file per case above, plus the earlier
scenarios (merge commits, a file re-added in the source after a synced
deletion). Every case came out as in the table. The previous script, on
the same histories, overwrote or restored the three files it now keeps.
- **`replay()`**: a sync commit, made on an old base, replayed onto a
newer base where the target had changed some files meanwhile. Its
update, addition and deletion applied; the target's changed file, a file
it changed that the sync deleted, and a file it added itself were kept;
a change already present was skipped. The previous `replay()` overwrote
two of those three files.
- `bash -n` passes for every script.

## Notes

- Every config runs `sync.yml@main`, so this applies to every sync as
soon as it merges.
- A file whose target version came from an earlier source, and is a
version the current source never had, now stays as it is, as if the
target had changed it. That is only possible where a repository was
moved beneath a new parent and its syncs from the new parent were never
merged. Such files show as `Kept: <file> (changed in the target)` in the
sync logs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@SgtSwagrid SgtSwagrid mentioned this pull request Oct 26, 2026
Merged
SgtSwagrid added a commit that referenced this pull request Oct 26, 2026
@SgtSwagrid @claude
[sync] Always overwrote synced files, and deleted each file once, sin…
28ba536 
…ce the last merged sync. (#9)

## Summary

Replaces the provenance rule of #7 and #8 with a simpler model:

- **Updates:** every sync copies every non-ignored file, overwriting
whatever the target has.
- **Deletions:** each file the source deletes is deleted from the target
once. A sync deletes the files the source has deleted since the source
commit of the last sync from it that the target merged.
- **Opting out:** only through `ignore`.

Measuring deletions from the last *merged* sync, rather than from the
push that triggered the run, means:
- a sync run that's skipped or fails is made up for by the next;
- a sync PR rebuilt by later pushes keeps every deletion;
- a file the target re-adds after a deletion was merged is left alone.

All the provenance and history comparison is gone.

## Changes

- **`scripts/last-sync.sh`** (new, its own workflow step): finds the
source commit of the last sync this target merged, and exports it as
`SOURCE_SYNCED`.
- It reads the new `GitHub-Graph-Source` trailer on this source
directory's commit in the most recently merged sync PR. GitHub keeps PR
commits after a merge, whether squashed or not and with the branch
deleted.
- It searches the latest 20 merged sync PRs; 50 exceeded GitHub's
GraphQL node limit.
- With no such PR (the first sync, or syncs made before this version),
it falls back to the commit before the triggering push, so that push's
own deletions still go through. Without that either, it syncs no
deletions.
- **`scripts/merge-files.sh`**: copies every non-ignored file in the
source root, as before #7. It then deletes the target's copy of each
file deleted between `SOURCE_SYNCED` and the source's head (`git diff
--diff-filter=D`), skipping ignored files. If that commit isn't in the
source's history (say, after a force push), it warns and skips
deletions. Over 90 lines are gone: the per-file history walks, the state
map and the `Kept` reasons.
- **`scripts/publish.sh`**:
- Each sync commit now also carries `GitHub-Graph-Source: <source
commit>`, after the key trailer. The PR body, which drops everything
from the key trailer on, is unaffected.
- `replay()` goes back to applying every change as it is. No
carry-forward is needed: this source's commit is rebuilt each time from
everything since the last merged sync.
  - The key calculation moved to `load-config.sh`.
- **`scripts/load-config.sh`**: exports the source directory's key
(`SYNC_KEY`) and the two trailer names, for `last-sync.sh` and
`publish.sh`.
- **`.github/workflows/sync.yml`**: a "Find last sync" step before
"Merge files", given the target's token and `github.event.before`.
- **`README.md`**: "Which files are synced" describes the model: copy
everything, delete once since the last merged sync, opt out with
`ignore`, and what a skipped run, a re-added file and a closed PR mean.
"Merge semantics" says again that files are overwritten, that downstream
changes are lost unless ignored, and that the most recent source wins.
- **`templates/pull-request-body.md`**: the default warning is back to
its original text, which is accurate again.

## Testing

No CI exercises the scripts, so I ran them locally:

- **`merge-files.sh`**, against a source and target with known
histories:
- from the last merged sync, it deleted the two files deleted by a
skipped run;
- it left alone a file deleted before that sync (which the target still
has);
  - it updated a file the source deleted and re-added within the range;
  - it skipped an ignored deleted file;
- it overwrote the target's edited and own copies, and kept a
target-only file.

With no `SOURCE_SYNCED` it copied everything and deleted nothing. With a
commit missing from the history, it warned and skipped deletions.
- **`last-sync.sh`**, with `gh` stubbed:
  - it read the trailer, ignoring a lookalike in the body text;
- it fell back to the commit before the push when the trailer was
missing, as with older syncs;
- it synced no deletions for an all-zero (branch-creating) before or
none at all.
- **The real `--jq` filter**, against SgtSwagrid/codeswitch's merged
sync PRs: it found the latest merged sync commit from
scala-browser-plugin-config by its key trailer. That commit predates the
new trailer, so a live run would use the fallback until the next merged
sync.
- **The commit message**: `%(trailers:key=…)` reads both trailers, and
the rendered PR body excludes them.
- **`replay()`**: an update, addition and deletion all apply onto a
newer base, including over the target's changes. Deleting a file the
target already deleted is a no-op.
- **Syntax**: `bash -n` passes for every script.

## Notes

- **Overwriting:** every config runs `sync.yml@main`, so once merged,
the next sync of every target overwrites any local edits to synced
files, as before #7.
- **Deletions in the meantime:** until each target merges its first sync
under this version, its deletions come only from the triggering push.
- **Repositories:** the template's and Codeswitch's version files, and
basis and abstract-nonsense, simply follow their configs again.
- **Changed downstream:** the files #8 had begun keeping are overwritten
at the next sync, if they still differ.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@SgtSwagrid