feat(flows): add the local and ssh environment drivers - #79
Merged
Merged
Conversation
`ExecSession.signal` returned early once the process it started had exited, so a signal sent to end a session never reached what that process left in its group -- a `sleep &` holding stdout open -- and the session, and the command, went on running on the target. Signal the group while the process is alive, and after it has exited for as long as the session's output is still being pumped. Once the process is reaped its number is free, so the group is only signalled while no process holds that number again: a group is only ever made with its leader's number. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Implement `open_env`, `local_env` and a new `probe` in `hmz.runtime.flowing.environments`: one `MachineEnvDriver` written against a `Machine`, with a machine for this host and one for a host reached over ssh. Both serve every environment capability of the SPI. - exec: argv or bash script, in a process group of its own with nothing on stdin; stdout and stderr apart; a timeout, a cancellation or closing the driver kills the whole group and ends the wait, even where a daemon that left the group holds the output open; a non-zero status is returned, a death by signal N is 128 + N. - read/write: relative to the workdir, absolute or `~/...`; writes are atomic, make their parent directories and keep an existing file's mode. - derive_subdir; derive_worktree (`git worktree add --detach`, fresh directories under humanize's home); temporary copies, reflinked where the filesystem can, made beside their place and renamed in, with the copied repository made to stand alone (worktrees kept inside it repointed, gitfiles leading out cut loose); holders kept process-wide and, on this machine, an flock other processes honour; scratch directories; removal renames out of place first; everything named deterministically so a resumed run finds what it left. - Closing the root is final: it stops every command of its machine, the machine's own copying included and whatever was still starting, lets go of its holds and its connection. - ssh rides coganchor's transport: the serving half bootstrapped over `ssh` with one master connection per host, `/` exported, one probe command learning home, humanize's home, CPUs, memory and GPUs. Nothing touches the network until something is asked, or `probe`. Only the link failing is taken for a broken connection, never an errno the host reports or one slow answer. Tests: the driver contract over an in-memory machine and the pure logic (unit), the contract and the rest against real processes and git and against a stand-in `ssh` (integration), and the contract over a real `ssh` to `localhost` or to a throwaway loopback `sshd` (system). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
futrime
pushed a commit
that referenced
this pull request
Sep 25, 2026
feat(flows): add the local and ssh environment drivers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 1, unit 4: the environment drivers behind
hmz.runtime.flowing.spi.EnvDriver. This PR is based onfeat/new-flow-api.Public API (
hmz.runtime.flowing.environments)open_env(spec: EnvSpec) -> EnvDriver:local@/absorssh@[user@]host[:port]/absorssh@host/~/rel. It never touches the network. It raisesEnvUnavailablewhen a local workdir is missing or the ssh destination is invalid (for example, one starting with-).local_env(workdir: Path) -> EnvDriver: fills aLocalEnvrole.probe(driver) -> None(new): connects and learns the machine's resources off the loop, then checks that the workdir exists. It raisesEnvUnavailableorEnvConnectionError. The ways in should await it on every root env before checking resource requirements. Until it runs, an ssh env reportsavailable=Falseand cpu 1, memory 0, gpus 0. It is not in the lazy-export table, becauseruntime/flowing/__init__.pybelongs to unit 2.MachineEnvDriver, and each serves every item ofENV_CAPABILITIES.Semantics the engine relies on
close()kills the whole group and ends the wait. This holds even when asetsiddaemon keeps the output open.==, and they are tracked process-wide.flockalso makes other processes' holdsTempCloneBusy. Over ssh, exclusivity is per process only.destroy_*must be called before closing the root. Closing the root is final: every driver on that machine then raisesEnvError("... is closed")..gitincluded, and its repository stands alone:--sharedand keeps its index.On disk
Everything lives under humanize's home on that machine:
hmz.home()locally,${HUMANIZE_HOME:-~/.humanize}over ssh.envs/<name>-<digest(workdir)>/clones/<id>-<digest>/, plus a.lockfile beside each clone.envs/<name>-<digest(workdir)>/scratch/<id>-<digest>/.envs/<name>-<digest(workdir)>/worktrees/<ref>-<random>/.Names are deterministic, so a resumed run reattaches. A copy is built next to its final path and renamed into place; removal renames it away before deleting. The whole
envs/directory is safe to delete whenever no run is using it.SSH
transport.connect(ssh://…, ["/"])plusRemoteClient, with the zipapp bootstrapped oversshand ControlMaster giving one master connection per host.placement()returnsPlacement(SSH, provider, workdir, AnchoredConfig(AnchorConfig("ssh://provider", workspace=<abs workdir>))). For a~workdir it returns the absolute path once probed, andremote_path="~/…"before that.ExecSession.signalnow reaches the group after the leader has exited, so remote timeouts actually kill background children.Tests
tests/unit/flows/test_env_drivers.py): the driver contract over an in-memory machine, plus holds, closing, paths, error mapping, GPU and probe parsing, and building an ssh env without touching the network.tests/integration/flows/test_{local,ssh}_envs.py):check_env_driver(..., repo=True)against real processes and the version-control tool. The ssh path runs through a stand-insshwith coganchor's real serving half.tests/system/flows/test_ssh_envs.py): the contract over realssh. It useslocalhostif passwordless ssh works; otherwise it starts a throwaway loopbacksshdwith its own keys and leaves~/.sshuntouched. Here it ran against the throwaway sshd: 2 passed.uv run pre-commit run --all-filespasses, anduv run pytestpasses across all three tiers.🤖 Generated with Claude Code