[mindsers]

Summary

Closes #313.

configfile deploys the configuration files (dotfiles) of a git repository. It keeps its own copy of the repository, a mirror in ~/.configfile/dotfiles, and links your files to it. configfile update syncs that mirror with the remote: it fetches, saves any local changes as a patch, then resets the mirror to the remote with git reset --hard.

To keep two processes from changing the same files at once, modules deploy and modules undeploy take a lock, ~/.configfile/lock (src/lock.ts). update and init did not, although they change what deployments read:

• an update run during a deployment (two terminals, or a setup script running both) could reset the mirror while the deployment read it: a link could be made to a file the reset removes a moment later, or a local copy made from a half-updated file;
• init could rewrite ~/.configfilerc while another command was reading it.

Command	Takes the lock
update	for the whole sync: fetch, saving local changes, reset
init	after its questions, while it clones or reuses the repository and saves the configuration
modules deploy, undeploy	unchanged: for the files of each module in turn
scripts run	no: a script can run for a long time and run configfile itself

A deployment and a sync now wait for each other, as two deployments already did: up to 10 seconds, then the command stops with an error naming the other process.

Other changes

• Commands say when they wait. Waiting used to be silent for up to 10 seconds; every command that takes the lock now prints Waiting for another configfile (process N) to finish… once each time it starts waiting.
• Two processes can no longer hold the lock at once. The lock used to be created empty, then the pid written into it, and two processes finding the lock of a process that no longer exists could both remove it: one of them the lock the other had just taken (reproduced with 8 processes started together). Now the pid is written to a draft file first, which is then hard-linked as the lock (this fails when a lock exists, like the previous exclusive create): the lock never exists empty. Where hard links are not supported, the lock is created in two steps, as before.
• A stale lock is taken over by one process at a time, holding lock.takeover. Another process doing it is waited for like a lock, with the same time limit and message. A guard left by a process that died is moved aside and checked before being removed. One rare case remains: a guard left behind, found by several processes at the very same moment.
• Calls of the same process take their turn. The lock file names a process, so it cannot tell two calls of one process apart; they are queued in memory.
• A lock that cannot be read, or that holds no pid, is never taken over. Its owner cannot be checked: an unreadable lock stops the command with Cannot take the lock …; a lock without pid (configfile 1.0 creates it empty for a moment) is waited for.
• The release is checked. A lock that cannot be removed, or that was replaced or removed while held, is reported with a warning, and no longer replaces the command's own result. A lock this process left behind is taken over at once by its next call. Drafts left by processes that died are removed.

Design

• The lock covers the fetch too. The fetch is the slow part (network), and a deployment waiting behind it gives up after 10 seconds. Taking the lock only around the reset was considered, but the whole sync keeps the rule simple: while update runs, nothing deploys.
• init takes the lock after its questions, so someone thinking at the URL prompt never blocks other commands. Once it holds the lock, it checks the folder and the configuration again: another configfile may have cloned the folder (it is then reused) or saved a configuration (it then stops rather than replace it without asking). A git prompt for credentials during the clone, or during update's fetch, does hold the lock.
• scripts run stays without the lock, as the issue proposes; the docs ask not to run scripts during an update.
• A known limit: since feat(modules): run scripts when a module is deployed or undeployed #318, modules deploy --all takes the lock module by module, so that scripts run by modules can run configfile themselves. An update can therefore still run between two modules of a deployment. Files are never deployed while the sync runs, but the module list and settings are read, without the lock, when the deployment starts: the modules after the sync are deployed with those settings. The safety page says so.

Other Informations

Compatibility

• update and init can now fail with Another configfile (process N) is changing files… when another command holds the lock for more than 10 seconds, or with Cannot take the lock … when ~/.configfile cannot be used. Before, they went ahead.
• With a home folder that cannot be written and --folder outside it, init now fails before cloning (its lock lives in ~/.configfile), instead of cloning and then failing to save the configuration.
• modules deploy and undeploy can now wait for an update or init, and fail the same way when the fetch or clone takes more than 10 seconds.
• A lock that cannot be read now stops a command at once (Cannot take the lock …), instead of after 10 seconds.
• A damaged lock, holding no pid, is no longer taken over: the command waits, then asks to delete it.
• New messages: the waiting message, a warning when the lock cannot be released as it should, and init stopping when another configfile saved a configuration while it waited (unless --force).
• Exit codes and the history are otherwise unchanged.

Review guide

1. src/lock.ts: acquire and createExclusive (the draft and the link), takeOver and removeStale (the lock.takeover guard), readOwner (an unreadable lock stops the command), release, the in-process queue in withLock, and withCommandLock, which every command uses to say it waits.
2. src/commands/update.ts and src/commands/init.ts: where the lock is taken.
3. src/commands/modules.ts: only switched to withCommandLock.
4. Docs: concepts/safety.md, reference/modules.md, reference/scripts.md, and the changelog.

Tests

pnpm check passes. New tests:

• update waits for another process holding the lock, says so once, then syncs;
• update holds the lock during the fetch and the reset: a git reference-transaction hook, run by both, records the lock's content each time;
• a modules deploy started during an update waits for it, and deploys the synced files;
• update fails before syncing when the lock cannot be taken, and the run is recorded in the history;
• init waits for the lock after its question, saying so, and fails before cloning when the lock cannot be taken; once it waited, it reuses a folder cloned meanwhile, and does not replace a configuration saved meanwhile;
• modules deploy and undeploy say they wait; a command whose lock cannot be removed warns and keeps its result; init --force replaces a configuration saved meanwhile;
• withLock:
  • 8 processes started together against the lock of a dead process never hold it at once, also with a takeover guard left behind (the first fails with the original version of this PR in 5 runs out of 6);
  • a takeover in progress is waited for and gives up in time; a guard left behind, by another process or this one, is taken over;
  • 600 calls of one process never run at once (fails every time without the queue);
  • a lock without pid is never taken over; a lock that cannot be read stops the command; only the lock is left in its folder; drafts of dead processes are removed;
  • a lock that cannot be removed keeps the task's result, is reported, and is taken over by the next call; a lock replaced or removed while held is reported and left alone;
  • onWait is called once with the other process;
  • without hard links (tests/lock-fallback.test.ts), the lock is still taken and waited for.
• the tests that wait for a command to wait release the other process once the command says it waits, not after a fixed delay; the two that use a git reference-transaction hook are skipped on git older than 2.28.

One existing init test, "explains how to recover when the configuration cannot be saved", made the whole home folder read-only; it now creates ~/.configfile first, so it still tests the configuration failing after the clone.

Manual test

From the root of a checkout of this branch (Node.js 24.11 or later, bash). It uses a throwaway home folder and takes about 25 seconds:

cli="$PWD/src/cli.ts"
export HOME="$(mktemp -d)"            # throwaway home: your own files are not touched
git init --quiet "$HOME/remote"
mkdir -p "$HOME/remote/files/zsh"
echo 'export EDITOR=vim' > "$HOME/remote/files/zsh/zshrc"
echo '{ "files": [{ "source_path": "zshrc", "target_path": "~/.zshrc", "deploy": "global" }] }' > "$HOME/remote/files/zsh/settings.json"
git -C "$HOME/remote" add . && git -C "$HOME/remote" -c user.name=me -c user.email=me@example.com commit --quiet -m first
cd "$HOME" && node "$cli" init --repo "$HOME/remote"

# 1. Another configfile holds the lock for 3 seconds: update waits, says so, then syncs.
sleep 3 & echo $! > "$HOME/.configfile/lock"
node "$cli" update

# 2. It holds it for longer than 10 seconds: update gives up, naming the process (exit 1).
sleep 30 & echo $! > "$HOME/.configfile/lock"
node "$cli" update; echo "exit $?"
node "$cli" modules deploy --all; echo "exit $?"   # deployments wait for the same lock
kill %%

# 3. The lock of a configfile that no longer exists is taken over.
sh -c 'echo $$ > "$HOME/.configfile/lock"'
node "$cli" update

node "$cli" history -n 4

Steps 1 and 3 end with Already up to date with origin/master; step 2 prints, for both commands (modules deploy first prints Deploying zsh…), with the full path of the lock:

 Info  Waiting for another configfile (process N) to finish…
 Error  Another configfile (process N) is changing files. Try again when it is done, or delete ~/.configfile/lock if no configfile is running.