diff --git a/docs/content/community/what-is-hive-commons.md b/docs/content/community/what-is-hive-commons.md index 07510ed..f0355cf 100644 --- a/docs/content/community/what-is-hive-commons.md +++ b/docs/content/community/what-is-hive-commons.md @@ -62,7 +62,9 @@ Every repository in the organization is Apache 2.0 licensed, requires DCO sign-off on every commit, and follows the CNCF Code of Conduct. Hive is maintained by a Maintainer Committee spanning multiple organizations, with a documented contributor ladder and explicit governance for AI-agent -contributions. Hive Commons has applied for the +contributions. The Commons lets contributors subscribe to multiple hives, rank +where their compute should go, and route work by `ranked`, `spread`, or +`neediest` strategies. Hive Commons has applied for the [CNCF Sandbox](https://github.com/cncf/sandbox/issues/516). - [Governance](https://github.com/hivecommons/.github/blob/main/GOVERNANCE.md) · [Maintainers](https://github.com/hivecommons/.github/blob/main/MAINTAINERS.md) · [Code of Conduct](https://github.com/hivecommons/.github/blob/main/CODE_OF_CONDUCT.md) · [Security policy](https://github.com/hivecommons/.github/blob/main/SECURITY.md) diff --git a/docs/content/hive/integrations.md b/docs/content/hive/integrations.md index f06cad2..72e0385 100644 --- a/docs/content/hive/integrations.md +++ b/docs/content/hive/integrations.md @@ -155,3 +155,36 @@ Hive supports agent CLIs, inference engines, and OpenAI-compatible model gateway ![Akamai (Linode) infrastructure provider](/integrations/akamai.svg) ![Oracle Cloud (OKE) infrastructure provider](/integrations/oracle.svg) ![Cloudflare DNS edge and tunnels provider](/integrations/cloudflare.svg) ![GitHub Copilot AI inference supporter](/integrations/github-copilot.svg) ![Bluehost domain hosting provider](/integrations/bluehost.svg) Hive Commons thanks [Akamai (Linode)](https://www.linode.com/) and [Oracle Cloud (OKE)](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/) for Kubernetes infrastructure donated through the CNCF, [Cloudflare](https://www.cloudflare.com/) for DNS, edge and tunnels, [GitHub Copilot](https://github.com/features/copilot) for AI inference donated to CNCF projects through the CNCF, and [Bluehost](https://www.bluehost.com/) for domain hosting. + +## Source control, work sources, and sign-in + +The product landing page groups runtime integrations separately from the agent +CLI and inference list above. Current shipped surfaces are: + +- **Source control:** GitHub and GitHub Enterprise are the production GitHub App + path. The forge-neutral adapter layer includes GitLab and Gitea / Forgejo + issue and change-request flows as those paths graduate. +- **Work sources:** GitHub Issues (default), GitHub Projects, Linear, Jira + Cloud, Jira Data Center / Server (`deployment: datacenter`), Spektacular run + stages, and Wavefront migration graphs. +- **Sign-in:** Microsoft, GitHub, IBM, Google, and other OIDC providers by hub + configuration. + +Jira Data Center / Server support uses Jira REST API v2, preserves context-path +base URLs, and supports Personal Access Token bearer auth or basic auth fallback. +Custom CA and TLS trust settings are still in progress and are not listed as +shipped stable functionality. + +## Recent Hive platform additions + +- **Swarm mode:** per-repository swarms with idle-unlock, themes, Discord + announcements, and `/api/leaderboard/swarm`. +- **The Commons:** contributors can subscribe to multiple hives, rank them, and + choose `ranked`, `spread`, or `neediest` routing; `hivectl hives web` opens a + local management UI. +- **Teams and achievements:** distro / OS / agent team leaderboards are exposed + at `/api/leaderboard/teams`, and Achievements 2.0 adds Solo, Dual, Fireteam, + and Raid tiers plus the local-model track. +- **Edge-only v6:** admin MCP (`/api/admin/mcp`, `cmd/hive-admin-mcp`) and the + dashboard Extensions tab for Spektacular are on the v6 edge branch until that + line is promoted. diff --git a/docs/content/hive/work-sources.md b/docs/content/hive/work-sources.md new file mode 100644 index 0000000..0a845f7 --- /dev/null +++ b/docs/content/hive/work-sources.md @@ -0,0 +1,345 @@ +> **Synced from Hive.** This page is pulled from [hivecommons/hive@v5](https://github.com/hivecommons/hive/blob/v5/src/docs/work-sources.md) during the docs build. Edit the canonical source in the Hive repository. + +# Work sources (`governor.work_source`) + +`work_source` selects where a hive reads actionable work items from (Step 01 +of the governor loop — see [Architecture](/docs/hive/architecture)). It accepts four +primary `type` values: `github` (default), `github_projects`, `linear`, and +`jira`. A fifth item kind, `type: run`, is additive rather than a primary +adapter: enable it with `run_stages: true` to append pending long-running run +stages as work items without fabricating GitHub issues. A second additive +kind, `wavefront`, lists the ready nodes of an imported migration graph the +same way (see below). +Absent or `type: ""` behaves exactly like existing hives with no +`work_source` block — GitHub Issues on the configured `project.repos` +(`pkg/config/config.go:1336-1348`, `pkg/worksource/factory.go:15-89`). + +This page documents all four. Linear has its own deeper guide — +[Linear agent integration](https://github.com/hivecommons/hive/blob/v5/src/docs/linear-agent.md) — for the two-way agent-session +integration (webhooks, session acknowledgement, writing back to Linear); +this page covers only the read side (`work_source.linear`) for parity with +the other three. + +Hosted and dashboard-managed hives can configure the same block from **Settings → Work source**; self-hosted operators can also edit `governor.work_source` in `hive.yaml`. Custom CA bundles and TLS trust overrides for Jira Data Center are not documented here because they are still in progress and are not part of v5. + +## Run stages (`run_stages: true`) + +Run stages let the governor offer a pending run stage such as `spec`, `plan`, +or `implement` as a non-issue-shaped work item. The primary source remains the +configured adapter; run stages are appended only when the flag is enabled: + +```yaml +governor: + work_source: + type: github + run_stages: true +``` + +With the flag omitted or set to `false`, `ListIssues` output is unchanged. With +it enabled, pending stages use source type `run`, `Number: 0`, and an external +ID of `:`. Their stable key is therefore +`!:`, not a fabricated `repo#0`. Later stages carry +a dependency on the previous stage's run key, so `plan` records its dependency +on `spec` and `implement` records its dependency on `plan`. + +The dashboard's `RunStageAccessor` backs this source with the lease registry: +each stage lease is the run's pending stage, a stage some connection is +currently working is not offered again, and `implement` is listed only once the +run's imported plan is approved. The hive binary wires that accessor during +dashboard boot; if the accessor is unavailable, the additive source fails +closed by listing no run stages. The Spektacular (Spek) stage runner that advances the +lease is described in [spektacular.md](https://github.com/hivecommons/hive/blob/v5/src/docs/spektacular.md). + +## Wavefront migration graph (`wavefront.enabled: true`) + +The second additive item kind is a versioned code-migration graph such as +Crustify's C/C++ to Rust migration driven by Wavefront (#8362, the second +proving workload of #7620). Wavefront stays authoritative for its semantic +graph: it owns dependency closure, cycle handling, waves, and batching. Hive +reads the graph, lists each ready node as a run-stage work item, withholds +blocked nodes, and records a receipt when a node completes. Hive never +re-derives the graph with an LLM, never fabricates GitHub issues or pull +requests for nodes, and the adapter (`pkg/worksource/wavefront`) makes no +GitHub call at all. Publication of results is out of scope (#8353). + +```yaml +governor: + work_source: + type: github + wavefront: + enabled: true + path: /data/wavefront/graph.json # or url: https://wavefront.example/graph.json + repo: acme/crust # owner/name every node key is scoped to + receipts_dir: /data/wavefront/receipts +``` + +`enabled` defaults to `false`; with it off, `ListIssues` output is +byte-identical to a hive that has never heard of the block. Exactly one of +`path` or `url` is required when enabled, and `repo` is required. The graph +is re-read on every governor cycle, so a plan Wavefront republishes is picked +up without a restart. `receipts_dir` is optional; empty keeps receipts in +memory for the life of the process. + +**Graph document.** A JSON object with a `graph` name, a `revision`, and a +`nodes` list. Each node has `id`, `title`, an optional `kind`, an optional +`depends_on` list of node ids, and an optional `status` (`pending` (default), +`ready`, `done`, or `blocked`). Hive validates structure only: an unnamed +graph, a missing revision, a duplicate or malformed id, an edge to an +undefined node, or a cycle is refused as a whole; the graph is never repaired. +A small fixture lives at +`pkg/worksource/wavefront/testdata/wavefront-fixture/graph.json`. + +**Listing.** A node is ready when it is not `done` or `blocked`, holds no +completion receipt at the current revision, and every node it depends on is +either `done` in the graph or holds a completion receipt at the current +revision. Ready nodes are listed in wave order as run-stage items: source type +`run`, `Number: 0`, stage `implement`, external id `:`, stable +key `!:`, labels `hive-run`, `stage/implement`, +`wavefront`, `wavefront/`, and `graph-rev/`. Each +`depends_on` edge is carried as a `DependsOn` entry keyed +`!:`, so admission sees the same edges Wavefront +declared. Blocked nodes and everything downstream of them are withheld, not +listed as unresolved. + +**Stale-plan guard.** The `graph-rev/` label is the revision the +item was minted at. `Source.Verify` and `Source.Complete` take that revision +back and refuse it with `ErrStaleRevision` when the graph has since moved. +A refused item is never mapped onto the new graph; it is simply re-listed +from the current revision if it is still ready. A receipt recorded under an +older revision no longer satisfies anything, so a dependent whose upstream +changed is withheld again rather than silently continuing (the #8346 +provenance guard, expressed through the revision). + +**Receipts.** `Source.Complete(ctx, externalID, revision, artifacts, startedAt)` +writes one `stage-receipt/v1` record (#8295) per completed node under +`//.json`: work key, assignment id +`:`, stage `implement`, contract `wavefront-node/v1`, +`input_revision` `wavefront-graph@`, and the artifact-borne +progress anchors the stage produced (default `wavefront//`). +Completing an already-completed node at the same revision returns the existing +receipt unchanged, so a retry is idempotent. Completing a node whose +dependencies are not all satisfied is refused with `ErrNotReady`. Receipts are +the only state Hive keeps for the migration; there is no side database. + +**Plan import.** `wavefront.ImportPlan(store, epic, graph, expectedRevision, +opts)` admits the graph as an epic's child beads through +`planning.DecomposeFromOutput`: the graph is rendered as the already-structured +task list that function consumes (`N. [] (depends: ...) +[agent_suitable]`, in wave order) and materialized with no prompt and no +planner call. Node ids become each child's `plan_ref`; edges become bead +dependencies. A graph whose revision is not `expectedRevision` is refused +before anything is written. + +**Not wired yet.** The adapter lists work and records receipts. Handing a +listed node to an agent worktree, the mid-node crash reconciling to Unknown, +and the burndown counts on `/api/runs/{key}` are follow-ups on the #8290 +umbrella; the adapter's `Receipts()` and `Ready()` are the seams they read. + +## `type: github` (default) + +No config needed. Enumerates open issues on `project.repos` via the existing +GitHub client (`pkg/worksource/factory.go:17-18`). This is the only source +type most hives ever need. + +## `type: github_projects` — GitHub Projects (Projects v2) + +Reads items from a GitHub Projects v2 board via the GraphQL API +(`pkg/worksource/github_projects.go`). Use this when work is tracked on a +project board rather than as bare repo issues. + +```yaml +governor: + work_source: + type: github_projects + github_projects: + project_number: 7 # required — the number in the project URL + org: your-org # optional — falls back to project.org + states: [Todo, In Progress] # optional — Status field values to include; empty = all + priority_field: Priority # optional — name of a single-select priority field + iteration_field: Sprint # optional — name of an iteration field; only current-iteration items are returned when set + default_repo: your-org/repo # optional — used when an item's own repo can't be determined +``` + +Config fields (`GitHubProjectsSourceConfig`, `pkg/config/config.go:1696-1703`): + +| YAML key | Go field | Required | Notes | +|---|---|---|---| +| `project_number` | `ProjectNumber` | Yes | The project's number, visible in its URL (`.../projects/7`). | +| `org` | `Org` | No | Defaults to `project.org` if empty (`factory.go:23`). | +| `states` | `States` | No | Filters on the project's `Status` single-select field. Empty means no filtering. | +| `priority_field` | `PriorityField` | No | Name of a single-select field to map to normalized priority (`urgent`/`high`/`medium`/`low`/`none`; see `normalizePriority`, `github_projects.go:318-331`). Empty means every issue gets priority `none`. | +| `iteration_field` | `IterationField` | No | Name of an iteration field. When set, only items whose iteration window covers "now" are returned (`inCurrentIteration`, `github_projects.go:299-315`). | +| `default_repo` | `DefaultRepo` | No | `owner/repo` used when the project item's own repository can't be read from the GraphQL response. | + +**Credentials.** The adapter authenticates with the same GitHub token the +hive already uses for issues/PRs (`ghToken` passed into +`worksource.FromConfig`, `pkg/worksource/factory.go:15,23`) — there is no +separate `token` field in `github_projects:`. That token needs the +`read:project` scope in addition to whatever repo scopes it already carries; +a classic PAT with only `repo` cannot read a Projects v2 board and the +GraphQL call fails with a GraphQL-level authorization error surfaced as +`worksource/github_projects: graphql error: ...` +(`github_projects.go:219-221`). + +**Failure mode.** A wrong `project_number` or an org the token can't see +returns a GraphQL error on every governor cycle, logged but not fatal — the +hive keeps running with an empty work-item list from this source. + +## `type: linear` + +```yaml +governor: + work_source: + type: linear + linear: + api_key: ${LINEAR_API_KEY} + hold_labels: [hold] # additive; GitHub's hold defaults still apply + teams: + - key: ENG + repo: your-org/default-repo + states: [Todo, In Progress] + cycles: current # optional; only issues in the active cycle + projects: # optional; when present, only these projects enumerate + - name: Platform + repo: your-org/platform + assigned_only: true # optional; requires a connected Linear agent app + session_agent: scanner # which hive agent takes Linear agent sessions +``` + +`api_key` and at least one `teams[].key`/`teams[].repo` are required — +construction fails closed with an explicit error naming the missing field +otherwise (`factory.go:32-53`). `assigned_only: true` additionally requires +the Linear agent app to already be connected (`linearagent.StoredViewerID`); +without a connected app it is a startup error, not "enumerate everything" +(`factory.go:58-67`). + +See [Linear agent integration](https://github.com/hivecommons/hive/blob/v5/src/docs/linear-agent.md) for the OAuth app setup, +webhook wiring, `LINEAR_CLIENT_ID`/`LINEAR_CLIENT_SECRET`/ +`LINEAR_WEBHOOK_SECRET`, and how agents write back to Linear. + +**ACMM gap issues.** The dashboard's ACMM "Open Issue" / "Open All" buttons +file on GitHub by default even on a Linear-sourced hive. Set +`governor.acmm.issue_tracker: work_source` to file them on the Linear team +mapped to the criterion's repo instead (`teams[].repo` match, else the first +team) — see [ACMM policy matrix → Where gap issues are filed](/docs/hive/acmm-policy-matrix#where-acmm-gap-issues-are-filed). + +## `type: jira` — Jira Cloud or Jira Data Center + +Reads issues from Jira Cloud via REST API v3, or from Jira Data Center / Server +via REST API v2 when `deployment: datacenter` is set (`pkg/worksource/jira.go`). +The Jira Data Center REST reference documents API version `2` and URI layouts +that include the instance context path, such as +`http://host:port/context/rest/api-name/api-version/resource-name` +([Atlassian Jira Data Center REST API reference](https://docs.atlassian.com/software/jira/docs/api/REST/9.14.0/)). +Atlassian's server examples use `/rest/api/2/...` endpoints for issues and +searches ([Jira REST API examples](https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/)). +Data Center PATs are available in Jira Core/Software 8.14+ and are sent as +bearer tokens ([Using Personal Access Tokens](https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html)). +Jira Cloud rich text comments/descriptions use ADF JSON +([Atlassian Document Format](https://developer.atlassian.com/cloud/jira/platform/apis/document/structure/)); +Data Center accepts plain text / wiki-markup string bodies. + +```yaml +governor: + work_source: + type: jira + jira: + base_url: https://your-org.atlassian.net # required + email: bot@your-org.com # required — Atlassian account email + api_token: ${JIRA_API_TOKEN} # required + project_keys: [ENG, OPS] # used to build the default JQL + # jql: "project in (ENG) AND statusCategory != Done" # optional full override + repo: your-org/default-repo # required for agents to know what to clone + hold_labels: [hold, blocked] # optional — Jira labels that gate an issue +``` + +Jira Data Center / Server example: + +```yaml +governor: + work_source: + type: jira + jira: + deployment: datacenter + base_url: https://jira.example.com/jira # context paths are preserved + api_token: ${JIRA_DATACENTER_PAT} # preferred PAT bearer auth + # Or, for older instances without PATs: + # username: hive-bot + # password: ${JIRA_DATACENTER_PASSWORD} + project_keys: [ENG, OPS] + repo: your-org/default-repo + hold_labels: [hold, blocked] +``` + +Config fields (`JiraSourceConfig`, `pkg/config/config.go`): + +| YAML key | Go field | Required | Notes | +|---|---|---|---| +| `deployment` | `Deployment` | No | Empty/`cloud` keeps Jira Cloud REST API v3. `datacenter` or `server` uses REST API v2 and Data Center auth/body shapes. | +| `base_url` | `BaseURL` | Yes | Jira instance root, e.g. Cloud `https://myorg.atlassian.net` or Data Center `https://jira.example.com/jira`; context paths are preserved. | +| `email` | `Email` | Cloud yes | Atlassian account email; used for Cloud HTTP Basic auth alongside the API token. | +| `username` | `Username` | DC basic auth only | Jira Data Center username when using basic auth. | +| `api_token` | `APIToken` | Cloud yes; DC preferred | Cloud API token (Basic password) or Data Center Personal Access Token (Bearer). | +| `password` | `Password` | DC basic auth only | Jira Data Center password, used only when `api_token` is empty. Prefer PATs where supported. | +| `project_keys` | `ProjectKeys` | No¹ | Project keys to enumerate, e.g. `["ENG","OPS"]`. Used to build the default JQL when `jql` is empty. | +| `jql` | `JQL` | No | Full JQL override. When empty, the adapter builds `project in (<keys>) AND statusCategory != Done AND issuetype != Epic`. | +| `repo` | `Repo` | Yes in practice | GitHub `owner/name` repo agents clone to work these issues; every returned `Issue.Repo` is set to this single value — Jira source config maps to exactly one repo, unlike Linear's per-team repo map. | +| `hold_labels` | `HoldLabels` | No | Jira label values that gate an issue out of the work list, the Jira analogue of GitHub's `hold` label. | + +¹ `project_keys` is not enforced as required by the constructor, but if both +it and `jql` are empty the built JQL becomes `project in () AND ...`, which +Jira will reject — set one or the other. + +**Credentials.** Jira Cloud REST v3 uses HTTP Basic auth with the account +email as username and the **API token** (not the account password) as +password. Jira Data Center sends the PAT as bearer-token HTTP auth when +`api_token` is set; otherwise it falls back to HTTP Basic with +`username`/`password` (or `email` as the username when `username` is empty). +Supply secrets via `${JIRA_API_TOKEN}` / `${JIRA_DATACENTER_PAT}` / +`${JIRA_DATACENTER_PASSWORD}` environment-variable substitution in `hive.yaml` +— never inline literal credentials in committed config. + +**Data model differences.** Cloud users expose `accountId` and rich-text +description/comment bodies as ADF JSON. Data Center commonly exposes users by +`name` / `key`; the adapter prefers those identities when `deployment: +datacenter` is set. Data Center comments sent by Hive are plain strings so Jira +can render them as wiki markup/plain text; Cloud comments use a minimal ADF +document. + +**Priority mapping.** Jira priority names are normalized case-insensitively +(`normalizeJiraPriority`, `jira.go:126-141`): + +| Jira priority | Normalized | +|---|---| +| `highest`, `critical`, `blocker`, `p0` | `urgent` | +| `high`, `p1` | `high` | +| `medium`, `p2` | `medium` | +| `low`, `lowest`, `p3`, `p4` | `low` | +| anything else | `none` | + +**Hold-label gating.** `ListIssues` drops any issue whose Jira `labels` +field contains one of the configured `hold_labels` before it is returned to +the governor — the same shape as GitHub's `hold` label, but Jira has no +built-in "hold" concept so this is opt-in and additive +(`jira.go:143-175`). + +**Pagination and endpoints.** Cloud uses `/rest/api/3/search`; Data Center +uses `/rest/api/2/search`. Both are paged with `startAt`/`maxResults`, and the +adapter requests 100 issues at a time until it has walked the full result set. +Issue fetch, comment, and transition helper paths use the same API-version +switch (`/issue/{key}`, `/issue/{key}/comment`, and +`/issue/{key}/transitions`). + +**Failure mode.** A bad `base_url`, invalid credentials, or non-2xx Jira +response returns `worksource/jira: search returned <status>: <body>` on +every governor cycle, logged but not fatal — the hive keeps running with an +empty work-item list from this source until the misconfiguration is fixed. + +## Open questions + +- The issue that requested this page (#4926) says GitHub Projects requires + a "project node ID". The code instead takes a plain integer + `project_number` (the number visible in the project's URL) and resolves it + server-side via GraphQL (`organization(login).projectV2(number)`, + `github_projects.go:56-57`) — there is no node-ID field in + `GitHubProjectsSourceConfig`. This page documents the code as written. diff --git a/scripts/sync-hive-docs.ts b/scripts/sync-hive-docs.ts index 24a1d77..823865b 100644 --- a/scripts/sync-hive-docs.ts +++ b/scripts/sync-hive-docs.ts @@ -25,6 +25,7 @@ const files: Array<{ source: string; target?: string }> = [ { source: "agent-configuration.md" }, { source: "hivectl.md" }, { source: "manual-provisioning.md" }, + { source: "work-sources.md" }, { source: "getting-started.md" }, { source: "net-admin-requirement.md" }, { source: "release-channels.md" },