From 669698f6c0d2843e7229e0bf4031c86852144013 Mon Sep 17 00:00:00 2001 From: Jean-Philippe Date: Sat, 12 Sep 2026 20:06:17 -0400 Subject: [PATCH 1/3] RunnerDiscovery + RunnerRegistry: enrollment state, inspected never recorded MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two read seams for runner enrollment state (plans#26): RunnerDiscovery enumerates this host's actions-runner dirs and parses each .runner record (scope + name, BOM-aware) — offline by construction, spanning every dir so an org enrollment living in a repo-named dir (its pre-org history) is still found. RunnerRegistry is the GitHub-side view over the gh Executor seam: find a runner by scope+name (custom labels only — read-only ones are GitHub's), amend its custom labels in place via PUT, loud QueryError vs nil so callers can tell offline from gone. Refs #161. Co-authored-by: Cursor --- lib/dev/runner_discovery.rb | 84 +++++++++++++++++++++ lib/dev/runner_registry.rb | 91 +++++++++++++++++++++++ test/dev/runner_discovery_test.rb | 73 ++++++++++++++++++ test/dev/runner_registry_test.rb | 118 ++++++++++++++++++++++++++++++ 4 files changed, 366 insertions(+) create mode 100644 lib/dev/runner_discovery.rb create mode 100644 lib/dev/runner_registry.rb create mode 100644 test/dev/runner_discovery_test.rb create mode 100644 test/dev/runner_registry_test.rb diff --git a/lib/dev/runner_discovery.rb b/lib/dev/runner_discovery.rb new file mode 100644 index 0000000..5719e46 --- /dev/null +++ b/lib/dev/runner_discovery.rb @@ -0,0 +1,84 @@ +# typed: strict +# frozen_string_literal: true + +require "json" + +module Dev + # This machine's runner enrollments, inspected — never recorded + # (plans#26): the actions-runner install dirs under $HOME are the only + # local state, and each configured dir's .runner file (written by + # config.sh) names the enrollment's scope and runner name. Labels are + # deliberately NOT here — GitHub is their single home; Dev::RunnerRegistry + # reads them. Offline by construction, so register and status can find + # this host's enrollments without the network. + class RunnerDiscovery + extend T::Sig + + # One local enrollment: where it lives and what its .runner records. + class Enrollment < T::Struct + # Absolute install dir. Its name is history (register defaults it from + # the first label at enrollment time) — never identity; the .runner + # record inside is what binds it to a scope. + const :dir, String + + # "owner/repo" (repo scope) or "owner" (org scope). + const :scope, String + + # The runner's GitHub-side name (.runner agentName). + const :name, String + end + + # @param home [String] the home dir to scan (injectable for tests) + sig { params(home: String).void } + def initialize(home: Dir.home) + @home = home + end + + # Every configured enrollment on this host, in dir order. Unconfigured + # dirs (downloaded but never registered, or garbage) are silently + # skipped — they are not enrollments. + # + # @return [Array] + sig { returns(T::Array[Enrollment]) } + def enrollments + Dir.glob(File.join(@home, "actions-runner-*")).sort.filter_map { |dir| self.class.read(dir) } + end + + # The enrollment serving a scope, when one exists. The lookup spans + # every runner dir because dir names drift from labels over a box's + # life (e.g. an org enrollment living in a repo-named dir from its + # pre-org history) — matching on the .runner record is what makes + # re-registration self-healing instead of a duplicate enrollment. + # + # @param scope [String] "owner/repo" or "owner" + # @return [Enrollment, nil] + sig { params(scope: String).returns(T.nilable(Enrollment)) } + def for_scope(scope) + enrollments.find { |enrollment| enrollment.scope == scope } + end + + class << self + extend T::Sig + + # Parse a runner dir's .runner record. config.sh writes the file with + # a UTF-8 BOM, so read with "bom|utf-8" or JSON.parse chokes on the + # first byte. nil when absent or unreadable — an unconfigured dir. + # + # @param dir [String] a runner install dir + # @return [Enrollment, nil] + sig { params(dir: String).returns(T.nilable(Enrollment)) } + def read(dir) + raw = File.read(File.join(dir, ".runner"), encoding: "bom|utf-8") + record = JSON.parse(raw) + url = record["gitHubUrl"].to_s + scope = url.sub(%r{\Ahttps://github\.com/}, "").chomp("/") + name = record["agentName"].to_s + return nil if scope.empty? || scope == url || name.empty? + + Enrollment.new(dir: dir, scope: scope, name: name) + rescue JSON::ParserError, Errno::ENOENT + nil + end + end + end +end diff --git a/lib/dev/runner_registry.rb b/lib/dev/runner_registry.rb new file mode 100644 index 0000000..916b4ee --- /dev/null +++ b/lib/dev/runner_registry.rb @@ -0,0 +1,91 @@ +# typed: strict +# frozen_string_literal: true + +require "json" + +require "dev/runner_setup" + +module Dev + # The GitHub-side view of the self-hosted runners enrolled at a scope: + # find one by name, amend its custom labels in place. Labels have exactly + # one home — GitHub — so this is both how register converges an existing + # enrollment's labels without re-enrolling and how status reports them + # (plans#26: inspected, never recorded). + # + # The gh CLI boundary rides the same Executor seam RunnerSetup uses, so + # tests exercise the orchestration without the network. + class RunnerRegistry + extend T::Sig + + # GitHub could not be queried (offline, unauthenticated) — distinct + # from a runner genuinely absent at the scope (find returns nil). + class QueryError < StandardError; end + + # The label amend was refused (permissions, deleted runner). + class AmendError < StandardError; end + + # One enrolled runner, as the amendable subset of GitHub's record: + # custom labels only — the read-only ones (self-hosted, OS, arch) are + # GitHub's, not ours to converge. + class Runner < T::Struct + const :id, Integer + const :custom_labels, T::Array[String] + end + + # @param executor [#capture] CLI boundary (injectable for tests) + sig { params(executor: T.untyped).void } + def initialize(executor: RunnerSetup::Executor.new) + @exec = executor + end + + # The runner enrolled under `name` at `scope`, or nil when none is. + # + # @param scope [String] "owner/repo" or "owner" + # @param name [String] the runner's GitHub-side name + # @return [Runner, nil] + # @raise [QueryError] when GitHub can't be queried — loud, never a + # silent not-found, so callers can tell "gone" from "unknown" + sig { params(scope: String, name: String).returns(T.nilable(Runner)) } + def find(scope:, name:) + out, err, ok = @exec.capture("gh", "api", "--paginate", "#{api_base(scope)}/actions/runners", "--jq", ".runners[]") + raise QueryError, "could not list the runners at #{scope}: #{err.strip}" unless ok + + record = out.each_line.map { |line| JSON.parse(line) }.find { |runner| runner["name"] == name } + return nil if record.nil? + + custom = Array(record["labels"]).select { |label| label["type"] == "custom" }.map { |label| label["name"].to_s } + Runner.new(id: Integer(record.fetch("id")), custom_labels: custom) + end + + # Replace the runner's custom labels with `labels` (GitHub's PUT + # semantics: the full custom set, read-only labels untouched). This is + # the no-re-enrollment amend path: the service, its name, and its dir + # all stay put. + # + # @param scope [String] "owner/repo" or "owner" + # @param runner_id [Integer] + # @param labels [Array] the desired custom label set + # @return [void] + # @raise [AmendError] when the PUT is refused + sig { params(scope: String, runner_id: Integer, labels: T::Array[String]).void } + def amend!(scope:, runner_id:, labels:) + argv = ["gh", "api", "-X", "PUT", "#{api_base(scope)}/actions/runners/#{runner_id}/labels"] + argv += labels.flat_map { |label| ["-f", "labels[]=#{label}"] } + _out, err, ok = @exec.capture(*argv) + raise AmendError, "could not amend the labels of runner #{runner_id} at #{scope}: #{err.strip}" unless ok + end + + private + + # The API path prefix for a scope, following its shape (repos/... for + # "owner/repo", orgs/... for a bare org) — same convention as + # RunnerSetup's token minting. + # + # @param scope [String] + # @return [String] + sig { params(scope: String).returns(String) } + def api_base(scope) + scope.include?("/") ? "repos/#{scope}" : "orgs/#{scope}" + end + end +end diff --git a/test/dev/runner_discovery_test.rb b/test/dev/runner_discovery_test.rb new file mode 100644 index 0000000..763ae60 --- /dev/null +++ b/test/dev/runner_discovery_test.rb @@ -0,0 +1,73 @@ +# typed: false +# frozen_string_literal: true + +require "test_helper" +require "dev/runner_discovery" +require "tmpdir" +require "fileutils" +require "json" + +transform!(RSpock::AST::Transformation) +class Dev::RunnerDiscoveryTest < Minitest::Test + test "enrollments reads every configured actions-runner dir under home" do + Given "a home with two configured runner dirs and one unconfigured" + home = Dir.mktmpdir + write_runner(home, "actions-runner-cellbound3d", scope: "d3mlabs", name: "JeanPhiippesMBP") + write_runner(home, "actions-runner-ue-engine", scope: "d3mlabs/unreal-engine", name: "gaming-box") + FileUtils.mkdir_p(File.join(home, "actions-runner-fresh")) # downloaded, never configured + + When "discovering" + enrollments = Dev::RunnerDiscovery.new(home: home).enrollments + + Then "each .runner record surfaces as an enrollment, dir-ordered; the unconfigured dir is silent" + enrollments.map { |e| [e.dir, e.scope, e.name] } == [ + [File.join(home, "actions-runner-cellbound3d"), "d3mlabs", "JeanPhiippesMBP"], + [File.join(home, "actions-runner-ue-engine"), "d3mlabs/unreal-engine", "gaming-box"], + ] + end + + test "for_scope finds the enrollment serving a scope regardless of its dir name" do + Given "an org enrollment living in a repo-named dir (its pre-org history)" + home = Dir.mktmpdir + write_runner(home, "actions-runner-cellbound3d", scope: "d3mlabs", name: "JeanPhiippesMBP") + + When "looking up the org scope" + enrollment = Dev::RunnerDiscovery.new(home: home).for_scope("d3mlabs") + + Then "the dir name does not matter — the .runner record does" + enrollment.dir == File.join(home, "actions-runner-cellbound3d") + enrollment.name == "JeanPhiippesMBP" + end + + test "for_scope is nil when no enrollment serves the scope" do + Given "a home with only a repo-scoped enrollment" + home = Dir.mktmpdir + write_runner(home, "actions-runner-cellbound3d", scope: "d3mlabs/cellbound-3d", name: "box") + + Expect + Dev::RunnerDiscovery.new(home: home).for_scope("d3mlabs").nil? + end + + test "read parses the BOM config.sh writes and nils out garbage" do + Given "a dir whose .runner is not JSON" + home = Dir.mktmpdir + dir = File.join(home, "actions-runner-broken") + FileUtils.mkdir_p(dir) + File.write(File.join(dir, ".runner"), "not json") + + Expect "garbage reads as unconfigured, never raises" + Dev::RunnerDiscovery.read(dir).nil? + Dev::RunnerDiscovery.new(home: home).enrollments == [] + end + + private + + # A .runner record the way config.sh writes it: UTF-8 BOM + JSON with + # gitHubUrl/agentName (scope "owner" or "owner/repo"). + def write_runner(home, dir_name, scope:, name:) + dir = File.join(home, dir_name) + FileUtils.mkdir_p(dir) + record = { "agentName" => name, "gitHubUrl" => "https://github.com/#{scope}", "workFolder" => "_work" } + File.write(File.join(dir, ".runner"), "\uFEFF#{JSON.pretty_generate(record)}") + end +end diff --git a/test/dev/runner_registry_test.rb b/test/dev/runner_registry_test.rb new file mode 100644 index 0000000..bba2d05 --- /dev/null +++ b/test/dev/runner_registry_test.rb @@ -0,0 +1,118 @@ +# typed: false +# frozen_string_literal: true + +require "test_helper" +require "dev/runner_registry" +require "json" + +transform!(RSpock::AST::Transformation) +class Dev::RunnerRegistryTest < Minitest::Test + # Answers capture calls from a responder and records each argv. + class RecordingExecutor + attr_reader :captures + + def initialize(&responder) + @responder = responder + @captures = [] + end + + def capture(*argv) + @captures << argv + @responder ? @responder.call(argv) : ["", "", true] + end + end + + test "find returns the named runner's id and custom labels at an org scope" do + Given "a gh answering with two runners, one line each" + lines = [ + runner_json(id: 7, name: "other-box", custom: ["ue-engine"]), + runner_json(id: 42, name: "JeanPhiippesMBP", custom: %w[ai-ask ai-build]), + ].join("\n") + exec = RecordingExecutor.new { [lines, "", true] } + registry = Dev::RunnerRegistry.new(executor: exec) + + When "finding by name" + runner = registry.find(scope: "d3mlabs", name: "JeanPhiippesMBP") + + Then "the org endpoint is paginated and the read-only labels are excluded" + exec.captures == [["gh", "api", "--paginate", "orgs/d3mlabs/actions/runners", "--jq", ".runners[]"]] + runner.id == 42 + runner.custom_labels == %w[ai-ask ai-build] + end + + test "find hits the repos endpoint for a repo scope" do + Given "a gh answering with one runner" + exec = RecordingExecutor.new { [runner_json(id: 1, name: "box", custom: ["cellbound3d"]), "", true] } + registry = Dev::RunnerRegistry.new(executor: exec) + + When "finding at a repo scope" + registry.find(scope: "d3mlabs/cellbound-3d", name: "box") + + Then + exec.captures.fetch(0).fetch(3) == "repos/d3mlabs/cellbound-3d/actions/runners" + end + + test "find is nil when no runner of that name is enrolled at the scope" do + Given "a gh answering with unrelated runners" + exec = RecordingExecutor.new { [runner_json(id: 7, name: "other-box", custom: []), "", true] } + registry = Dev::RunnerRegistry.new(executor: exec) + + Expect + registry.find(scope: "d3mlabs", name: "JeanPhiippesMBP").nil? + end + + test "find raises when GitHub cannot be queried" do + Given "a gh that fails (offline, auth)" + exec = RecordingExecutor.new { ["", "connect: network is unreachable", false] } + registry = Dev::RunnerRegistry.new(executor: exec) + + When "finding" + registry.find(scope: "d3mlabs", name: "box") + + Then "the failure is loud, never a silent not-found" + error = raises Dev::RunnerRegistry::QueryError + error.message.include?("network is unreachable") + end + + test "amend! replaces the runner's custom labels in place" do + Given "a recording gh" + exec = RecordingExecutor.new { ["", "", true] } + registry = Dev::RunnerRegistry.new(executor: exec) + + When "amending" + registry.amend!(scope: "d3mlabs", runner_id: 42, labels: %w[ai-ask ai-edit ai-build ai-split ai-learn]) + + Then "one PUT carries the full custom set" + exec.captures == [[ + "gh", "api", "-X", "PUT", "orgs/d3mlabs/actions/runners/42/labels", + "-f", "labels[]=ai-ask", "-f", "labels[]=ai-edit", "-f", "labels[]=ai-build", + "-f", "labels[]=ai-split", "-f", "labels[]=ai-learn" + ]] + end + + test "amend! raises when the PUT fails" do + Given "a gh refusing the amend" + # A lambda (not a plain block): blocks auto-splat their single array + # argument under RSpock's transformation. + responder = ->(argv) { argv.include?("PUT") ? ["", "HTTP 403", false] : ["", "", true] } + exec = RecordingExecutor.new(&responder) + registry = Dev::RunnerRegistry.new(executor: exec) + + When "amending" + registry.amend!(scope: "d3mlabs", runner_id: 42, labels: ["ai-ask"]) + + Then + error = raises Dev::RunnerRegistry::AmendError + error.message.include?("HTTP 403") + end + + private + + # One `--jq .runners[]` output line: GitHub's runner object with read-only + # labels (self-hosted, OS, arch) alongside the custom ones. + def runner_json(id:, name:, custom:) + labels = [{ "name" => "self-hosted", "type" => "read-only" }] + + custom.map { |l| { "name" => l, "type" => "custom" } } + JSON.generate({ "id" => id, "name" => name, "labels" => labels }) + end +end From 9aef370a91271db61b8fd8b6a8e409c8fe5ecaa3 Mon Sep 17 00:00:00 2001 From: Jean-Philippe Date: Sat, 12 Sep 2026 20:50:34 -0400 Subject: [PATCH 2/3] dev runner: scope x labels with derived defaults; retire the dev.yml runner block MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Enrollment identity is a machine fact, not a repo declaration (plans#26): - Bare `dev runner register` enrolls repo-scoped with the label derived from the manifest name (ProjectManifest#slug — the name is the package identity, the org the registry); --org composes with --ai-flow (the full ai-flow vocabulary, mirrored as LabelContracts::AI_FLOW_LABELS) or --labels. --org needs no checkout; runner is ungated (projectless catalog included) and runner-setup stays an alias. - Idempotent and self-healing: register discovers an existing enrollment at the target scope (RunnerDiscovery over every local runner dir) and amends its labels in place on GitHub (RunnerRegistry PUT) instead of re-enrolling; contracts converge every run; a stale local dir whose runner is gone server-side re-enrolls into the same dir. - status is discovery-based — the machine's view, never a repo's: scope/name from .runner records (offline-safe), labels from GitHub (unknown when offline, flagged when gone), agent facts per agent- labeled enrollment. - dev.yml `runner:` is retired (warns and is ignored); RUNNER_HOST_KEYS and the manifest/context runner fields are gone; RunnerSetupConfig is now flag-resolved only. ProjectContext carries the manifest name. Closes #161. Co-authored-by: Cursor --- README.md | 20 +- lib/dev/label_contracts.rb | 9 + lib/dev/runner_setup.rb | 24 +- lib/dev/runner_status.rb | 110 +++-- src/dev/builtins/runner_command.rb | 182 ++++++--- src/dev/execution_context.rb | 5 +- src/dev/project_manifest.rb | 37 +- src/dev/project_manifest_loader.rb | 55 +-- src/dev/runner.rb | 40 +- src/dev/runner_setup_config.rb | 23 +- test/dev/builtin_executor_test.rb | 2 +- test/dev/builtins/cache_command_test.rb | 1 + test/dev/builtins/cd_command_test.rb | 2 +- test/dev/builtins/check_command_test.rb | 2 +- test/dev/builtins/clone_command_test.rb | 2 +- test/dev/builtins/config_command_test.rb | 2 +- test/dev/builtins/cred_command_test.rb | 2 +- test/dev/builtins/deps_command_test.rb | 2 +- test/dev/builtins/help_command_test.rb | 2 +- .../dev/builtins/install_deps_command_test.rb | 2 +- test/dev/builtins/learnings_command_test.rb | 2 +- test/dev/builtins/plan_command_test.rb | 2 +- .../builtins/provide_image_command_test.rb | 1 + .../builtins/reset_container_command_test.rb | 1 + test/dev/builtins/runner_command_test.rb | 377 ++++++++++++------ test/dev/builtins/up_command_test.rb | 1 + test/dev/builtins/update_deps_command_test.rb | 2 +- test/dev/command_executor_test.rb | 2 +- test/dev/command_service_test.rb | 2 +- test/dev/execution_context_test.rb | 2 +- test/dev/overridden_executor_test.rb | 2 +- test/dev/project_manifest_loader_test.rb | 124 +----- test/dev/runner_status_test.rb | 124 ++++-- test/dev/runner_test.rb | 25 +- 34 files changed, 715 insertions(+), 476 deletions(-) diff --git a/README.md b/README.md index 68fa4b7..59a90ed 100644 --- a/README.md +++ b/README.md @@ -178,13 +178,23 @@ Because the wrapper runs `builtin cd` in your interactive shell, shadowenv activ ## dev runner — enroll a host as a self-hosted runner -Repos opt in by declaring a `runner:` block in `dev.yml` (labels required; `dir`/`name`/`version` optional), so every repo declares only its runner identity instead of vendoring a setup script. The command exists only where that block does. It is a machine-setup verb: rare, admin-flavored, human-run. +A machine-setup verb: rare, admin-flavored, human-run — and declared nowhere. Enrollment identity is a machine fact, not a repo one, so there is no `runner:` block in dev.yml (the key is retired and warns): scope and labels compose from flags, with derived defaults covering the common enrollments. -**`dev runner register`** converges, then enrolls. First the **label contracts**: a capability label is not just routing metadata — it names an obligation the host must satisfy, and register converges the (possibly empty) requirements of every label being advertised. A bare target-host label (e.g. a gamebox) converges nothing; the agent capability labels (`ai-build`, `ai-learn`) mark the box an **agent host** and carry the **agent host bootstrap** (below). Then the enrollment ceremony, per repo served: actions-runner download, registration-token mint via your `gh` auth, `config.sh --unattended --replace`, service install — idempotent, including across scopes (a repo-scoped runner re-registered `--org` deregisters at its old scope first). Flags: `--org` (one runner serving the whole org), `--repo`, and `--labels`/`--dir`/`--name` overrides for hosts that differ from the block's default; `--agent-user` overrides the `ai-agent` run-as default. `dev runner-setup` survives as an alias for `register`. +```bash +dev runner register # repo scope; label derived from the project name +dev runner register --labels ue-engine # repo scope, custom roles +dev runner register --org --ai-flow # org agent host: the full ai-flow label set +dev runner register --org --labels ai-build # org custom pool (e.g. a beefy build box) +dev runner status # THIS machine's enrollments + host facts +``` + +**Demand and supply.** Workflows demand labels (`runs-on: [self-hosted,