Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,13 +178,23 @@ Because the wrapper runs `builtin cd` in your interactive shell, shadowenv activ

## dev runner — enroll a host as a self-hosted runner

Repos opt in by declaring a `runner:` block in `dev.yml` (labels required; `dir`/`name`/`version` optional), so every repo declares only its runner identity instead of vendoring a setup script. The command exists only where that block does. It is a machine-setup verb: rare, admin-flavored, human-run.
A machine-setup verb: rare, admin-flavored, human-run — and declared nowhere. Enrollment identity is a machine fact, not a repo one, so there is no `runner:` block in dev.yml (the key is retired and warns): scope and labels compose from flags, with derived defaults covering the common enrollments.

**`dev runner register`** converges, then enrolls. First the **label contracts**: a capability label is not just routing metadata — it names an obligation the host must satisfy, and register converges the (possibly empty) requirements of every label being advertised. A bare target-host label (e.g. a gamebox) converges nothing; the agent capability labels (`ai-build`, `ai-learn`) mark the box an **agent host** and carry the **agent host bootstrap** (below). Then the enrollment ceremony, per repo served: actions-runner download, registration-token mint via your `gh` auth, `config.sh --unattended --replace`, service install — idempotent, including across scopes (a repo-scoped runner re-registered `--org` deregisters at its old scope first). Flags: `--org` (one runner serving the whole org), `--repo`, and `--labels`/`--dir`/`--name` overrides for hosts that differ from the block's default; `--agent-user` overrides the `ai-agent` run-as default. `dev runner-setup` survives as an alias for `register`.
```bash
dev runner register # repo scope; label derived from the project name
dev runner register --labels ue-engine # repo scope, custom roles
dev runner register --org --ai-flow # org agent host: the full ai-flow label set
dev runner register --org --labels ai-build # org custom pool (e.g. a beefy build box)
dev runner status # THIS machine's enrollments + host facts
```

**Demand and supply.** Workflows demand labels (`runs-on: [self-hosted, <label>]`); enrollments supply them. GitHub routes a job to any online runner visible to the repo (its own repo-scoped runners plus the org's) whose labels cover the demand — first available wins, with **no repo-beats-org priority**. So an org agent host and a repo-scoped `ai-build` box sharing a label is capacity pooling; a strict partition is expressed through label sets, never through scope. Three vocabularies exist today: a repo's own CI label, **derived from its dev.yml `name:`** (normalized: `Cellbound3D` → `cellbound3d` — the manifest name is the package identity, the ecosystem norm, and the org is the registry giving it uniqueness); special fleets' explicit `--labels` (e.g. unreal-engine's `ue-engine` / `macos,ue-editor` boxes); and the **ai-flow vocabulary** (`ai-ask ai-edit ai-split ai-build ai-learn`) — defined by ai-flow's reusable workflow (one label per slash command), mirrored here as `LabelContracts::AI_FLOW_LABELS` behind `--ai-flow`.

**`dev runner register`** converges, then enrolls. First the **label contracts**: a capability label is not just routing metadata — it names an obligation the host must satisfy, and register converges the (possibly empty) requirements of every label being advertised. A bare target-host label (e.g. a gamebox) converges nothing; the agent capability labels (`ai-build`, `ai-learn`) mark the box an **agent host** and carry the **agent host bootstrap** (below). Then the enrollment: register looks for an existing enrollment at the target scope (every `~/actions-runner-*/.runner` on the host — dir names never matter) and **amends its labels in place on GitHub** instead of re-enrolling, so re-running is both the idempotent no-op and the drift repair; only a scope nothing serves gets the full ceremony (actions-runner download, registration-token mint via your `gh` auth, `config.sh --unattended --replace`, service install — itself idempotent, including across scopes: a repo-scoped runner re-registered `--org` deregisters at its old scope first). Enrollment state is **inspected, never recorded**: labels live on GitHub, the scope in the runner dir's own `.runner` record — nothing in dev.yml, Settings, or any inventory file. Flags: `--org`, `--repo`, `--ai-flow`, and `--labels`/`--dir`/`--name` identity overrides; `--agent-user` overrides the `ai-agent` run-as default. `--org` needs no project checkout; bare register does (the derivation source). `dev runner-setup` survives as an alias for `register`.

The **agent host bootstrap** is host-singular, idempotent, and admin-prompting (re-running register re-converges — that is also the drift repair; macOS-only today). Every fact is inspectable and none is recorded — there is no record file. It converges: the hidden non-admin agent user with its own home; the cooperative `ai` group with both identities enrolled; the one-way sudoers edge (runner user → agent, `NOPASSWD:SETENV`, staged and `visudo -c` validated before landing root-owned 0440); the shared data root with the one-off `~/.dev` migration (below); the agent's own container engine when the served repo declares `build.container` (colima via brew, the agent's `container_engine: colima` record, its VM sized from the repo's `resources:` hint); and after enrollment, the `_work` job-checkout tree as a cooperative group space plus the runner service env — `Umask` 002 and `AI_FLOW_AGENT_USER`, the single record of "jobs landing here execute as X".
The **agent host bootstrap** is host-singular, idempotent, and admin-prompting (macOS-only today). Every fact is inspectable and none is recorded — there is no record file. It converges: the hidden non-admin agent user with its own home; the cooperative `ai` group with both identities enrolled; the one-way sudoers edge (runner user → agent, `NOPASSWD:SETENV`, staged and `visudo -c` validated before landing root-owned 0440); the shared data root with the one-off `~/.dev` migration (below); the agent's own container engine when the enrolling checkout declares `build.container` (colima via brew, the agent's `container_engine: colima` record, its VM sized from the repo's `resources:` hint); and after enrollment, the `_work` job-checkout tree as a cooperative group space plus the runner service env — `Umask` 002 and `AI_FLOW_AGENT_USER`, the single record of "jobs landing here execute as X".

**`dev runner status`** is the inspect-only half: the checkout's `runner:` block vs this host's registration (read from the runner's own `.runner` record — works offline), plus the re-derived reality of each advertised label's contract (agent user, group memberships, sudoers edge, `_work` grant, shared root, agent engine where required) and host tooling (`brew bundle check` against the org Brewfile, self-skipping when none ships).
**`dev runner status`** is the inspect-only half, and it is the *machine's* view, not any repo's: every discovered enrollment (scope and name from its own `.runner` record — works offline), each one's labels read from GitHub (their single home; unknown when offline, flagged when the runner is gone server-side), the re-derived reality of each agent-labeled enrollment's contract (agent user, group memberships, sudoers edge, `_work` grant, shared root, agent engine where required), and host tooling (`brew bundle check` against the org Brewfile, self-skipping when none ships).

### The data root (shared on agent hosts)

Expand Down Expand Up @@ -433,7 +443,7 @@ Custom integrations implement `Dev::Deps::Integration` (with `install_all(pins,
- **`dev cd <repo>`** — jump to a checkout under `$DEV_CD_ROOT` (default `~/src`) by fuzzy name, with Tab completion (see [dev cd](#dev-cd--jump-between-checkouts)). Global: works without a `dev.yml`.
- **`dev clone [<org>/]<repo>`** — clone a GitHub repo via your `gh` auth into the canonical `$DEV_CD_ROOT/github.com/<org>/<repo>` path (org defaults to `d3mlabs`) and land there (see [dev clone](#dev-clone--clone-into-the-canonical-layout)). Clone-only — run `dev up` yourself. Global: works without a `dev.yml`.
- **`dev cache gc [--keep N]`** — reclaim host caches dev owns (see below).
- **`dev runner register|status`** — enroll this host as the repo's self-hosted GitHub Actions runner (converging the advertised labels' contracts first), or inspect the registration and host facts (see [dev runner](#dev-runner--enroll-a-host-as-a-self-hosted-runner)). Registered only when `dev.yml` declares a `runner:` block; `dev runner-setup` is an alias for `register`.
- **`dev runner register|status`** — enroll this host as a self-hosted GitHub Actions runner (repo-scoped with a label derived from the project name, or `--org` with `--ai-flow`/`--labels`; contracts converge first, existing enrollments are label-amended in place), or inspect this machine's enrollments and host facts (see [dev runner](#dev-runner--enroll-a-host-as-a-self-hosted-runner)). Ungated — exists everywhere; `dev runner-setup` is an alias for `register`.
- **`dev reset-container`** — remove the persistent build container (clears its incremental cache); registered only when `build.container.persist` is set.
- **`dev plan …`** — global (works without a `dev.yml`; the workspace is the nearest dev.yml or git root). Sync Cursor plans with GitHub issues (ai-flow): the issue is the canonical plan, the local `.cursor/plans/gh-<n>-<slug>.plan.md` is a transient working copy carrying an `<!-- ai-flow … -->` header. Subcommands: `new "<title>" [--blank] [--org]` (create issue + linked plan — templated by default with the tech-design document (brief sections + `## Tech design` skeleton), resolved from the target repo's committed `.github/ISSUE_TEMPLATE/plan.md` when present (with a staleness warning when that mirror lags dev's bundle) else dev's bundled `share/plan-templates/tech-design.md`; `--blank` scaffolds just the H1; `--org` scaffolds a `Target repos:` line), `link <n> [<file>]` / `link <file>` (attach a draft to an existing issue / create one from it), `pull <n> [--merge]` (fetch, 3-way merging when both sides changed — the merge base lives at `~/.local/state/ai-flow/`), `push [<file>|<n>]` (guarded body PATCH — refuses to clobber newer remote edits; a number resolves the linked plan like `pull`), `status` (clean / ahead / behind / diverged, per linked plan), and `init` (materialize/update the plan template mirror at `.github/ISSUE_TEMPLATE/plan.md` in the working tree — review with `git diff`, then commit; only mirrors still carrying dev's marker comment are ever overwritten, so a repo customizes its template by editing the file and dropping the marker). `Dev::Plan::Templates` is the canonical owner of the template and mirror layout. `--org` targets the org plans repo (`plans_repo:` in `~/.config/dev/config.yml`, or `DEV_PLANS_REPO`) instead of the current repo's origin. Every invocation also refreshes the user-global links for dev's shipped skills (`share/cursor-skills/*` → `~/.cursor/skills/`, so the Cursor agent knows these verbs) and the org learnings artifacts (see [Agent skills & org learnings](#agent-skills--org-learnings)). For auto-push, a participating repo adds a Cursor `afterFileEdit` hook to `.cursor/hooks.json` running `dev plan hook-after-edit` — it reads the hook payload from stdin and no-ops unless the edited file is a linked plan. What happens to a plan after it's canonical — `/ask`, `/edit`, `/split` (two-phase dry/apply), `/build` — is ai-flow's remote half: see [plan-lifecycle.md](https://github.com/d3mlabs/ai-flow/blob/HEAD/docs/plan-lifecycle.md) and [commands.md](https://github.com/d3mlabs/ai-flow/blob/HEAD/docs/commands.md).
- **`dev learnings sync|status|invariants|init`** — global (works without a `dev.yml`). `sync` refreshes the whole learnings read path now, blocking, errors bubbling: pull the machine cache of the knowledge repo, relink skills (shipped, org, and the project's gem skills), render the invariants rule and link it into the enclosing project. Outside a project the machine-global parts run and the project-scoped ones are skipped. `status` reports the configured knowledge repo, cache location and age, and what's rendered/linked per tier. `invariants` prints the Tier-0 prompt block (the invariants section extracted from the org index) — the seam prompt-building consumers like ai-flow shell out to instead of parsing the cache themselves. `init` scaffolds the canonical empty learnings layout at the enclosing repo's root: the repo-tier index (`.cursor/rules/learnings-index.mdc` with its `alwaysApply: true` front matter, capture/curation preamble, soft cap, and org-tier trailer — no entries), or with `--org` the knowledge-repo layout (`index.md` with the fixed `## Invariants (always-on)` / `## Knowledge (on-demand)` section structure `dev learnings sync` parses, plus the `skills/` corpus directory) for a new org adopting the loop. The scaffold is **write-once-committed**: an existing index is reported and left untouched (exit 0), so consumers such as ai-flow's `/learn` call `init` unconditionally before capturing into an unseeded repo. `Dev::Learnings::Layout` is the canonical owner of both tiers' paths and templates. See [Agent skills & org learnings](#agent-skills--org-learnings).
Expand Down
9 changes: 9 additions & 0 deletions lib/dev/label_contracts.rb
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,15 @@ class LabelContracts
# The capability labels whose contract is the agent host bootstrap.
AGENT_CAPABILITY_LABELS = T.let(%w[ai-build ai-learn].freeze, T::Array[String])

# The full ai-flow label vocabulary — one label per slash command,
# routed by ai-flow's reusable workflow (`runner="ai-${word}"`). ai-flow
# defines the vocabulary (its README "Adoption checklist" names this
# list as the contract); this is dev's documented mirror — a cross-repo
# literal duplicated knowingly, like the agent user/group names —
# behind `dev runner register --org --ai-flow` (the agent host enrolls
# with the full set; a partitioned topology passes --labels instead).
AI_FLOW_LABELS = T.let(%w[ai-ask ai-edit ai-split ai-build ai-learn].freeze, T::Array[String])

# The agent host obligation: the host-singular bootstrap, the agent's
# own container engine when the served repo builds in one, and the
# post-enrollment service/workdir setup.
Expand Down
84 changes: 84 additions & 0 deletions lib/dev/runner_discovery.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# typed: strict
# frozen_string_literal: true

require "json"

module Dev
# This machine's runner enrollments, inspected — never recorded
# (plans#26): the actions-runner install dirs under $HOME are the only
# local state, and each configured dir's .runner file (written by
# config.sh) names the enrollment's scope and runner name. Labels are
# deliberately NOT here — GitHub is their single home; Dev::RunnerRegistry
# reads them. Offline by construction, so register and status can find
# this host's enrollments without the network.
class RunnerDiscovery
extend T::Sig

# One local enrollment: where it lives and what its .runner records.
class Enrollment < T::Struct
# Absolute install dir. Its name is history (register defaults it from
# the first label at enrollment time) — never identity; the .runner
# record inside is what binds it to a scope.
const :dir, String

# "owner/repo" (repo scope) or "owner" (org scope).
const :scope, String

# The runner's GitHub-side name (.runner agentName).
const :name, String
end

# @param home [String] the home dir to scan (injectable for tests)
sig { params(home: String).void }
def initialize(home: Dir.home)
@home = home
end

# Every configured enrollment on this host, in dir order. Unconfigured
# dirs (downloaded but never registered, or garbage) are silently
# skipped — they are not enrollments.
#
# @return [Array<Enrollment>]
sig { returns(T::Array[Enrollment]) }
def enrollments
Dir.glob(File.join(@home, "actions-runner-*")).sort.filter_map { |dir| self.class.read(dir) }
end

# The enrollment serving a scope, when one exists. The lookup spans
# every runner dir because dir names drift from labels over a box's
# life (e.g. an org enrollment living in a repo-named dir from its
# pre-org history) — matching on the .runner record is what makes
# re-registration self-healing instead of a duplicate enrollment.
#
# @param scope [String] "owner/repo" or "owner"
# @return [Enrollment, nil]
sig { params(scope: String).returns(T.nilable(Enrollment)) }
def for_scope(scope)
enrollments.find { |enrollment| enrollment.scope == scope }
end

class << self
extend T::Sig

# Parse a runner dir's .runner record. config.sh writes the file with
# a UTF-8 BOM, so read with "bom|utf-8" or JSON.parse chokes on the
# first byte. nil when absent or unreadable — an unconfigured dir.
#
# @param dir [String] a runner install dir
# @return [Enrollment, nil]
sig { params(dir: String).returns(T.nilable(Enrollment)) }
def read(dir)
raw = File.read(File.join(dir, ".runner"), encoding: "bom|utf-8")
record = JSON.parse(raw)
url = record["gitHubUrl"].to_s
scope = url.sub(%r{\Ahttps://github\.com/}, "").chomp("/")
name = record["agentName"].to_s
return nil if scope.empty? || scope == url || name.empty?

Enrollment.new(dir: dir, scope: scope, name: name)
rescue JSON::ParserError, Errno::ENOENT
nil
end
end
end
end
Loading
Loading