From 1092e15420331b9ecc4f5c82f5b6961bb4067792 Mon Sep 17 00:00:00 2001 From: "Vincent (Wen Yu) Ge" Date: Mon, 28 Sep 2026 23:42:46 -0400 Subject: [PATCH] docs: describe the new shape READMEs, AGENTS, skills, developer interfaces and the what-calls-what diagram. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589 --- .../skills/adding-framework-support/SKILL.md | 57 +- .claude/skills/adding-skill-program/SKILL.md | 184 ++-- .claude/skills/exploring-the-wizard/SKILL.md | 11 +- .claude/skills/ink-tui/SKILL.md | 68 +- .../skills/ink-tui/references/ARCHITECTURE.md | 122 ++- .claude/skills/ink-tui/references/PATTERNS.md | 20 +- .../skills/ink-tui/references/PRIMITIVES.md | 64 +- .../ink-tui/references/TERMINAL-COMPAT.md | 37 +- .claude/skills/wizard-development/SKILL.md | 77 +- .../references/ANTI-PATTERNS.md | 12 +- .../references/ARCHITECTURE.md | 296 ++++-- .../references/MAINTAINING-SKILLS.md | 5 +- .github/CODEOWNERS | 17 +- AGENTS.md | 176 ++-- README.md | 673 ++++++++------ docs/benchmarking.md | 192 ++-- docs/developer-interfaces.md | 272 ++++-- docs/error-catalog.md | 101 +- docs/images/wizard-run-paths.svg | 59 +- docs/local-dev.md | 228 +++-- docs/runbooks/agentic-monorepo-detection.md | 40 +- docs/runbooks/aio-capture.md | 55 +- docs/runbooks/warlock-kill-switch.md | 204 ++--- e2e-harness/ARCHITECTURE.md | 135 ++- scripts/README.md | 43 +- src/agent/README.md | 38 +- src/agent/runner/README.md | 24 +- src/agent/runner/sequence/README.md | 22 +- src/agent/tools/index.ts | 2 +- src/cli/README.md | 60 ++ src/cli/commands/mcp/tui-availability.ts | 2 +- src/headless/README.md | 45 + src/host/README.md | 51 ++ src/programs/README.md | 135 ++- src/programs/audit/events/constants.ts | 2 +- src/programs/posthog-integration/constants.ts | 2 +- .../posthog-integration/test/README.md | 13 +- src/programs/self-driving/ARCHITECTURE.md | 860 ++++++++---------- src/rules-stubs/universal.md | 37 - src/shared/README.md | 112 ++- src/shared/errors/wizard-error.ts | 4 +- src/shared/token-pricing.ts | 4 +- src/tools/README.md | 77 ++ src/tui/README.md | 102 +++ 44 files changed, 2850 insertions(+), 1890 deletions(-) create mode 100644 src/cli/README.md create mode 100644 src/headless/README.md create mode 100644 src/host/README.md delete mode 100644 src/rules-stubs/universal.md create mode 100644 src/tools/README.md create mode 100644 src/tui/README.md diff --git a/.claude/skills/adding-framework-support/SKILL.md b/.claude/skills/adding-framework-support/SKILL.md index 903ef23f0..5b11ea8ce 100644 --- a/.claude/skills/adding-framework-support/SKILL.md +++ b/.claude/skills/adding-framework-support/SKILL.md @@ -8,7 +8,7 @@ compatibility: Designed for coding agents working on the PostHog wizard codebase. metadata: author: posthog - version: '2.0' + version: '2.1' --- # Adding Framework Support @@ -20,8 +20,9 @@ creating its own runner or changing existing routing defaults. ## Extend the framework configuration -Start with [FrameworkConfig](../../../src/programs/framework-config.ts) and a nearby -example under [src/programs/frameworks](../../../src/programs/frameworks/). Framework-specific +Start with [FrameworkConfig](../../../src/programs/framework-config.ts) and a +nearby example under +[src/programs/frameworks](../../../src/programs/frameworks/). Framework-specific detection, context, environment conventions, and UI metadata belong here. Integration instructions and examples belong in context-mill. @@ -30,18 +31,23 @@ Integration instructions and examples belong in context-mill. frameworks before language fallbacks and generic Node last; preserve the overlap rules in the [detection checks](../../../src/programs/detection/__tests__/framework.test.ts). -2. Add the config under `src/programs/frameworks//-wizard-agent.ts`. Use a - `type` for framework context so it satisfies `Record`. +2. Add the config under `src/programs/frameworks//-wizard-agent.ts`. + Use a `type` for framework context so it satisfies `Record`. Export the config; the integration program already supplies execution. -3. Import the config into [FRAMEWORK_REGISTRY](../../../src/programs/frameworks/registry.ts). - The display label comes from `metadata.name`. +3. Import the config into + [FRAMEWORK_REGISTRY](../../../src/programs/frameworks/registry.ts). + `metadata.name` is the framework's label once detection or the picker settles + it. Read the current interface for the complete required fields. In particular, `detection.detectPackageManager` is required: reuse an adapter from -[package-manager detection](../../../src/programs/detection/package-manager.ts). Use -`metadata.setup.questions` for unresolved project variants; `gatherContext` -collects framework context. Optional notices and extra MCP servers also belong -in metadata. +[package-manager detection](../../../src/programs/detection/package-manager.ts). +Use `metadata.setup.questions` for unresolved project variants; +`metadata.gatherContext` collects framework context. When the gathered context +names a more specific variant, return its label from +`metadata.getDetectedFrameworkLabel(context)`; detection sets the label through +its context. Framework configs never use the UI. Optional notices and extra MCP +servers also belong in metadata. Use `usesPackageJson: false` for frameworks without a package.json dependency. Their required `getVersion` callback can return `undefined`. Minimum-version @@ -52,17 +58,17 @@ itself. ## Detection and examples -| Starting point | Pattern to reuse | -| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | -| [Next.js](../../../src/programs/frameworks/nextjs/) | `hasDeclaredDependency` from `utils/package-json`, `tryGetPackageJson` from `utils/setup-utils`, and router setup questions | -| [Django](../../../src/programs/frameworks/django/) | Python project files, context gathering, and Python package-manager detection | -| [Laravel](../../../src/programs/frameworks/laravel/) | Composer and framework-specific filesystem signals | -| [Rails](../../../src/programs/frameworks/rails/) | Gemfile detection and Ruby conventions | +| Starting point | Pattern to reuse | +| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| [Next.js](../../../src/programs/frameworks/nextjs/) | `hasDeclaredDependency` from `utils/package-json`, `tryGetPackageJson` from `utils/package-json`, and router setup questions | +| [Django](../../../src/programs/frameworks/django/) | Python project files, context gathering, and Python package-manager detection | +| [Laravel](../../../src/programs/frameworks/laravel/) | Composer and framework-specific filesystem signals | +| [Rails](../../../src/programs/frameworks/rails/) | Gemfile detection and Ruby conventions | -Use [bounded filesystem helpers](../../../src/shared/utils/bounded-fs.ts) for project -scans and reads. They bound traversal and skip dependency/build directories; add -framework-specific exclusions with `extraIgnore`. Keep complex parsers and -detectors beside the config so they can be checked independently. +Use [bounded filesystem helpers](../../../src/shared/utils/bounded-fs.ts) for +project scans and reads. They bound traversal and skip dependency/build +directories; add framework-specific exclusions with `extraIgnore`. Keep complex +parsers and detectors beside the config so they can be checked independently. ## Complete the content side @@ -84,15 +90,14 @@ enable a new gateway model. Check detection against the target framework and the closest overlapping framework/fallback. Reuse the existing detection checks; add a focused case only for behavior they do not cover. Confirm package-manager selection and matching -content-mill variants. For an end-to-end run, use a disposable test app and the +context-mill variants. For an end-to-end run, use a disposable test app and the [exploration guide](../exploring-the-wizard/SKILL.md). For prompt, environment-upload, or outro changes, inspect the current [integration program](../../../src/programs/posthog-integration/) and the -selected sequence. Some fields remain in the interface without a current -consumer: `getOutroNextSteps` is not used by the integration outro. Linear -post-run/outro hooks are not shared by the orchestrator; see the -[program guide](../adding-skill-program/SKILL.md). +selected sequence. `ui.getOutroNextSteps` and `analytics.getEventProperties` +have no consumer. Linear post-run/outro hooks are not shared by the +orchestrator; see the [program guide](../adding-skill-program/SKILL.md). Use the proportionate validation guidance in [wizard-development](../wizard-development/SKILL.md). Documentation-only changes diff --git a/.claude/skills/adding-skill-program/SKILL.md b/.claude/skills/adding-skill-program/SKILL.md index 3aedd8778..5b19a3f3c 100644 --- a/.claude/skills/adding-skill-program/SKILL.md +++ b/.claude/skills/adding-skill-program/SKILL.md @@ -8,7 +8,7 @@ compatibility: Designed for coding agents working on the PostHog wizard codebase. metadata: author: posthog - version: '3.0' + version: '4.0.1' --- # Adding a Skill-Based Program @@ -20,10 +20,9 @@ supported legacy fallback, deprecated as the default for new work; the shared guide owns the fallback criteria and gateway model/effort/system-prompt contract. -These are contribution defaults. Current runtime -[bindings](../../../src/agent/runner/switchboard/index.ts) still default -many programs to Anthropic plus linear; documentation changes do not migrate -them. +These are contribution defaults. A program whose config sets no `binding` runs +on [DEFAULT_BINDING](../../../src/agent/runner/switchboard/index.ts), Pi and +linear. ## Choose the contribution surface @@ -32,16 +31,16 @@ them. [Context-mill](https://github.com/PostHog/context-mill) owns skill content and `cliEntries`. A new skill-backed child of an existing family ships through context-mill; inspect - [family dispatch](../../../src/programs/dispatch-family.ts). Unpromoted - skills run through [the skill command](../../../src/commands/skill.ts). + [family dispatch](../../../src/cli/commands/dispatch-family.ts). Unpromoted + skills run through [the skill command](../../../src/cli/commands/skill.ts). - **Native program:** use a - [ProgramConfig](../../../src/programs/program-step.ts) when the wizard - needs its own flow, screens, detection, composition, or other native behavior. - Keep product instructions in context-mill. + [ProgramConfig](../../../src/programs/program-step.ts) when the wizard needs + its own flow, screens, detection, composition, or other native behavior. Keep + product instructions in context-mill. -Command names, program `id`, content-mill `agentFlow`, and `skillId` have +Command names, program `id`, context-mill `agentFlow`, and `skillId` have different roles. Use the full product name for public commands. The config field -is `id`, not the retired `flowKey`. +is `id`. ## Build a native orchestrator program @@ -50,29 +49,44 @@ example and read the [runner architecture](../wizard-development/references/ARCHITECTURE.md) when changing execution behavior. -1. Add the program config under `src/programs//`. Set `agentFlow` when - its content-mill flow differs from `id`; setting it explicitly also documents - the content dependency. Keep a `run` definition so the outer runner executes - agent work. +1. Add the program folder `src/programs//`. Its `index.ts` is the only + entry: it exports the `ProgramConfig` as `config`, and any detectors or keys + the TUI and CLI need. No other file in the folder is named `index.ts`. Copy a + sibling's `tsconfig.json` into the folder, unchanged, and add + `{ "path": "" }` to `references` in + [`src/programs/tsconfig.json`](../../../src/programs/tsconfig.json). The + folder is one project, its tests included, and `pnpm typecheck` (`tsc -b`) + builds it. Without the reference, the registry's import fails with TS6307. + Keep tests in `/__tests__`; they reach other layers through their entries + only. Shared program code goes in `src/programs/shared/`, `detection/`, + `warehouse-sources/`, `frameworks/` or `oauth/`, never in another program's + folder, and has no `index.ts`. A program imports its own files and the shared + program code by relative path, such as `../shared/skill-program`. Set + `agentFlow` when its context-mill flow differs from `id`; setting it + explicitly also documents the content dependency. Keep a `run` definition so + `runProgram` executes agent work. 2. Supply the flow's seed and task prompts in context-mill, including the task dependencies and applicable skill variants. The [orchestrator](../../../src/agent/runner/sequence/orchestrator/orchestrator-runner.ts) loads `agentFlow ?? id`, requires a seed prompt, and checks task-skill variants before running. `run.skillId` alone does not define this flow. -3. Register the config in - [PROGRAM_REGISTRY](../../../src/programs/program-registry.ts) and add its - Pi/orchestrator entry to - [PROGRAM_BINDINGS](../../../src/agent/runner/switchboard/index.ts). - [Existing binding checks](../../../src/agent/runner/__tests__/switchboard.test.ts) - enforce coverage; `ProgramId` currently widens to `string`. -4. For a standalone native command, create a command module with - [nativeCommandFactory](../../../src/commands/factories/native-command-factory.ts) - and register it in [bin.ts](../../../bin.ts). A native family child uses the - handlers in family dispatch. Program registration derives screen sequences - and store lookup, not the top-level CLI `.use()` chain. -5. Check [program OAuth scopes](../../../src/programs/oauth/program-scopes.ts) - against the tools the program needs; add scopes only when the base set is - insufficient. +3. Register the config: import it as + `import { config as } from '@programs/'` in + [program-registry.ts](../../../src/programs/program-registry.ts) and add it + to `PROGRAM_REGISTRY`. Set the config's `binding` to Pi and the orchestrator. + `ProgramId` is a `string`. +4. For a standalone native command, set `command` on the config. The CLI builds + the command with + [nativeCommandFactory](../../../src/cli/commands/factories/native-command-factory.ts) + and lists it at the program's place in the registry + ([`wizardCommands`](../../../src/cli/commands/index.ts)). A native family + child uses the handlers in family dispatch. +5. Check the program's OAuth scopes against the tools it needs. Add + `oauthScopeAdditions` to the config only when the base set is insufficient. + Keep a program's own set in its folder, as + [replay-vision](../../../src/programs/replay-vision/scopes.ts) does; sets + several programs share live in + [`program-scopes.ts`](../../../src/programs/oauth/program-scopes.ts). Model and effort selections in flow frontmatter must be supported by the wizard and gateway. Follow the cross-repo procedure in @@ -82,60 +96,88 @@ or changing gateway-required prompt material. ## Simple linear programs and existing flows For a very simple linear flow, use -[createSkillProgram](../../../src/programs/agent-skill/index.ts) to -configure installation of one skill. Register the native program as above with -an explicit Pi/linear binding; the factory does not select a sequence. Read -`SkillProgramOptions` for required fields; -[audit](../../../src/programs/audit/) demonstrates factory customization and -a dynamic `run(session)` that seeds a ledger. -[Revenue analytics](../../../src/programs/revenue-analytics/) builds its -config directly and adds prerequisite detection. +[createSkillProgram](../../../src/programs/shared/skill-program.ts) to configure +installation of one skill. Register the native program as above. The factory +sets no `binding`, so the program runs on `DEFAULT_BINDING` unless you add one +to the config it returns. Read `SkillProgramOptions` for required fields; +[audit](../../../src/programs/audit/) demonstrates factory customization and a +dynamic `run(session)` that seeds a ledger. +[Revenue analytics](../../../src/programs/revenue-analytics/) builds its config +directly and adds prerequisite detection. `ProgramRun.customPrompt`, `abortCases`, `postRun`, and `buildOutroData` are -consumed by the -[linear sequence](../../../src/agent/runner/sequence/linear.ts). `postRun` -runs after success; `buildOutroData` receives session and credentials, with host -information inside credentials. The orchestrator currently uses its own task -prompts, failure handling, and outro, and does not invoke those hooks. Check -this limitation before migrating a linear flow; setting an orchestrator binding -does not preserve these behaviors automatically. +consumed by the [linear sequence](../../../src/agent/runner/sequence/linear.ts). +`postRun` runs after success; `buildOutroData` receives session and credentials, +with host information inside credentials. The orchestrator currently uses its +own task prompts, failure handling, and outro, and does not invoke those hooks. +Check this limitation before migrating a linear flow; setting an orchestrator +binding does not preserve these behaviors automatically. ## Screens, prerequisites, and composition -Reuse [AGENT_SKILL_STEPS](../../../src/programs/agent-skill/steps.ts): -intro, health check, auth, run, outro, and keep-skills. Auth also applies the -shared [AI opt-in gate](../../../src/programs/ai-opt-in-gate.ts) for agent -programs. Override `screenId`, not `screen`, when adapting a step. New screens -need an entry in [ScreenId](../../../src/ui/tui/screen-sequences.ts), a -component, and registration in -[screen-registry](../../../src/ui/tui/screen-registry.tsx). Follow -[ink-tui](../ink-tui/SKILL.md) for rendering and store usage. - -Use a headless step's `onReady` for session-dependent detection, then render -structured `frameworkContext.detectError` data in the intro. `onInit` runs when -the TUI starts rendering with its initial session; `onReady` runs after the real -session is assigned. See [store hooks](../../../src/ui/tui/store.ts) and -[run-wizard](../../../src/lib/runners/run-wizard.ts). The -[noninteractive runner](../../../src/lib/runners/run-non-interactive.ts) also -walks `onReady` by default; set `ciPreRun` only when it needs a different -prerequisite strategy. +The program's UI lives in `src/tui/programs//`: `flow.ts`, `deck/`, +`screens/`, `screen-ids.ts`, and an `index.ts(x)` entry whose `TUI_PROGRAMS` +maps each program id it serves to a +[TuiProgram](../../../src/tui/programs/types.ts) (flow, deck, tips, screens, and +control `actions`/`setters`). Outside the folder, the +[TUI program registry](../../../src/tui/programs/index.ts) imports and spreads +the entry. A program with no entry runs +[AGENT_SKILL_STEPS](../../../src/tui/programs/shared/skill-flow.ts): intro, +health check, auth, run, outro, and keep-skills, with the skill deck. Use +`skillFlow(introScreenId)` to swap in your own intro screen. Auth also applies +the shared [AI opt-in gate](../../../src/tui/ai-opt-in-gate.ts), +`withAiOptInGate(config, flow)`, for agent programs. Set `healthCheck: false` on +the config when the run skips the PostHog readiness check, and leave the +health-check step out of its flow. Flow steps are +[FlowStep](../../../src/tui/flow.ts)s. Override `screenId` when adapting one. A +new screen needs an id in the folder's `screen-ids.ts`, a component, and an +entry in its entry's `screens`; the +[screen registry](../../../src/tui/screen-registry.tsx) mounts every program's +screens. Writes only the program makes, its screens' answers and any session +fields that go with them, go in the folder's `store-actions.ts` through +`store.updateTuiState`. The folder is its own tsconfig project: it may import +the TUI core, through `@tui/*` and never by relative path, `@programs`, +`@programs/types` and its own `@programs/`, never another program's folder +or the registry. Besides its line in `src/tui/programs/index.ts`, a new folder +needs a `tsconfig.json` copied from a sibling, unchanged, and +`{ "path": "programs/" }` in `references` in +[`src/tui/tsconfig.json`](../../../src/tui/tsconfig.json). Then run +`pnpm typecheck`. Without both, the registry's `@tui/programs/` import fails +with TS6307. + +Follow [ink-tui](../ink-tui/SKILL.md) for rendering and store usage. If the +program is team-owned, add both folders to `CODEOWNERS`. + +Use the config's `onReady(ctx)` for session-dependent detection, then render +structured `frameworkContext.detectError` data in the intro. A step's `onInit` +runs when the TUI starts rendering with its initial session. `onReady` runs +after the real session is assigned: `runTui` calls `runReadyHooks` in +[store.ts](../../../src/tui/store.ts) before the intro, and +[`detectProgram`](../../../src/programs/detect-program.ts) runs it as the +detection of a run whose store has none, as in headless. Set `ciPreRun` only +when a CI session needs a different prerequisite strategy: `detectProgram` runs +it in place of `onReady` for a CI session. Programs read a `ProgramSession` and +never use the UI: `run` gets a `RunnerContext`, `ciPreRun` gets a +`CiRunnerContext` and a run step's `onRunPrep` gets the runner's `log` +([runner-context.ts](../../../src/programs/runner-context.ts)). `requires` currently records metadata; it does not execute or enforce prior -programs. Compose real work through `ProgramStep.run`, with `onRunPrep` and -`targetDir` when needed. The -[integration run step](../../../src/programs/posthog-integration/index.ts) -and [self-driving](../../../src/programs/self-driving/) demonstrate this. -Composed sub-runs are structurally linear; orchestrators cannot nest. A host run -step without `run` can also set `targetDir` and `onRunPrep` to scope the -program's own agent to a picked project and keep its sequence, as -[error-tracking](../../../src/programs/error-tracking/) does. +programs. Compose real work through `runSteps`, keyed by the flow step id, with +`runProgramId`, `onRunPrep` and `targetDir` when needed. +[Self-driving](../../../src/programs/self-driving/index.ts) runs +`posthog-integration` composed at its `integrate-run` step. Composed sub-runs +are structurally linear; orchestrators cannot nest. A run step without +`runProgramId` can set `targetDir` and `onRunPrep` to scope the program's own +agent to a picked project and keep its sequence, as +[error-tracking](../../../src/programs/error-tracking/index.ts) does for its +`run` step. ## Validate the affected path Reuse the relevant registry, binding, detection, or routing checks. Add a focused behavioral test only for a meaningful gap; avoid tests that repeat configuration fields. Check that the selected CLI route resolves the intended -program and that its content-mill flow is available. Use the +program and that its context-mill flow is available. Use the [exploration guide](../exploring-the-wizard/SKILL.md) for a warranted end-to-end run against a disposable app. Follow [wizard-development](../wizard-development/SKILL.md) for proportionate checks; diff --git a/.claude/skills/exploring-the-wizard/SKILL.md b/.claude/skills/exploring-the-wizard/SKILL.md index 4ac32d73f..1286e09a5 100644 --- a/.claude/skills/exploring-the-wizard/SKILL.md +++ b/.claude/skills/exploring-the-wizard/SKILL.md @@ -9,7 +9,7 @@ compatibility: wizard-ci MCP server. metadata: author: posthog - version: '5.0' + version: '5.0.1' --- # Exploring the wizard as an agent @@ -80,15 +80,15 @@ own decisions through the same state and action contract. 2. Capture `render_screen` before each decision and during task or phase changes. Save numbered frames such as `/tmp/wz-explore-snaps/01-intro.txt`. 3. Commit only actions currently offered. Common choices are below; the - [action registry](../../../e2e-harness/action-registry.ts) defines the full + [TUI control actions](../../../src/tui/control/actions.ts) define the full set. 4. For a full run, confirm setup and call `run_agent` at `auth`. Continue reading state and handling overlays while it runs; polling alone cannot answer them. 5. Check `runPhase` (`idle`, `running`, `completed`, `error`), background status, and the rendered outro. On error, capture the frame and reason before - dismissing it. An error outro can wait for dismissal while `integration` - still says `running`; a host exit can instead surface as a socket error. + dismissing it. `integration` says `failed` once the run's phase is `error`; + a host exit can instead surface as a socket error. 6. After successful agent completion, finish the offered outro and follow-up actions. For the integration flow, `session.skillsComplete` marks the tail's completion. Other programs can have a terminal outro or exit screen. @@ -125,7 +125,8 @@ directories such as `vendor`, `venv`, `Pods`, `build`, and `dist`. Verify a failed copy before treating null detection as a regression; remove throwaway copies after recording their results. -The shared log is `/tmp/posthog-wizard.log`. Record its byte count before a run +The shared log is `/tmp/posthog-wizard.log` unless the server's environment sets +`POSTHOG_WIZARD_LOG_FILE`. Record its byte count before a run and read from that count plus one afterward. Run sweeps serially so their logs remain attributable. `read_state` omits `frameworkContext`; an empty `setupQuestions` list alone does not prove a router mode. When necessary, diff --git a/.claude/skills/ink-tui/SKILL.md b/.claude/skills/ink-tui/SKILL.md index d7a1f2247..cf2ca5745 100644 --- a/.claude/skills/ink-tui/SKILL.md +++ b/.claude/skills/ink-tui/SKILL.md @@ -7,7 +7,7 @@ description: > license: MIT metadata: author: posthog - version: '2.0' + version: '3.1' --- # Ink TUI @@ -34,42 +34,58 @@ conventions rather than copying upstream component manuals. ## Add a screen -1. Create a component in [screens](../../../src/ui/tui/screens/). -2. Add its `ScreenId` in - [screen-sequences.ts](../../../src/ui/tui/screen-sequences.ts). -3. Register the component in - [screen-registry.tsx](../../../src/ui/tui/screen-registry.tsx). -4. Reference it through `screenId` in the owning - [program's steps](../../../src/programs/), with the appropriate +A screen one program owns: + +1. Create the component in `src/tui/programs//screens/`. +2. Add its id to that folder's `screen-ids.ts`. +3. Map the id to the component in the `screens` of the folder's entry + (`index.ts(x)`), and add its control commits to `actions` (an empty list if + it has none). +4. Reference it through `screenId` in the program's flow (`flow.ts`, or the + `skillFlow()` a skill program uses), with the appropriate visibility, completion, and gate predicates. -Screen sequences derive from program steps. Do not hand-maintain a second -sequence array or add program navigation to the router. Additional state or -service wiring depends on the screen's needs; `App` remains the shared shell. +A tool's screens follow the same steps in `src/tui/tools//`, whose entry +exports `TUI_TOOLS`; see [src/tools](../../../src/tools/README.md). + +A screen several programs or the core use lives in +[screens](../../../src/tui/screens/), with its id in +[screen-ids.ts](../../../src/tui/screen-ids.ts) and its mount in +[screen-registry.tsx](../../../src/tui/screen-registry.tsx). Screen sequences +derive from each program's flow. Do not hand-maintain a second sequence array or +add program navigation to the router. Additional state or service wiring depends +on the screen's needs; `App` remains the shared shell. ## Preserve the UI boundary -Business logic calls [WizardUI](../../../src/ui/wizard-ui.ts) through -[getUI](../../../src/ui/index.ts). Screens use -[WizardStore](../../../src/ui/tui/store.ts) setters for reactive changes. The -router resolves program screens from session predicates; overlays interrupt that +Programs never use the UI. They get a runner context from +[runner-context.ts](../../../src/programs/runner-context.ts). TUI code, the +login included, takes the [WizardStore](../../../src/tui/store.ts) it reports +through as an argument: `pushStatus` for a line, `showOutroError` for an error +outro, and [abortOnScreens](../../../src/tui/abort.ts) for a decided failure. +There is no UI interface shared with headless or the CLI. Screens use store +setters for reactive changes. The router resolves program screens from step +predicates over the session and the TUI state; overlays interrupt that resolution. Local state is appropriate for presentation details such as tab selection, not wizard progression. -For new state, first decide whether it belongs in -[WizardSession](../../../src/lib/wizard-session.ts) or display-only store state. -Use an explicit setter that notifies subscribers. When business logic needs the -operation, extend `WizardUI`, [InkUI](../../../src/ui/tui/ink-ui.ts), and -[LoggingUI](../../../src/ui/logging-ui.ts) together. Reuse existing enums and -union types rather than introducing competing status vocabularies. +For new state, first decide whether it belongs in the shared +[WizardSession](../../../src/programs/session/wizard-session.ts), which extends +the [ProgramSession](../../../src/programs/program-session.ts) programs read, in +the TUI's own [TuiState](../../../src/tui/tui-state.ts) when only screens read +it, or in display-only store state. Use an explicit setter that notifies +subscribers. When TUI code needs a new operation to report through, add it to +the store; a run's progress reaches the store through +[agent-progress.ts](../../../src/tui/agent-progress.ts). Reuse existing enums +and union types rather than introducing competing status vocabularies. ## Reuse and check -Compose existing primitives and use [styles.ts](../../../src/ui/tui/styles.ts) -for shared colors, icons, and alignment. Export new public primitives from -[primitives/index.ts](../../../src/ui/tui/primitives/index.ts), add a realistic -[playground demo](../../../src/ui/tui/playground/demos/), and register it in -[PlaygroundApp.tsx](../../../src/ui/tui/playground/PlaygroundApp.tsx). +Compose existing primitives and use [styles.ts](../../../src/tui/styles.ts) for +shared colors, icons, and alignment. Export new public primitives from +[primitives/index.ts](../../../src/tui/primitives/index.ts), add a realistic +[playground demo](../../../src/tui/playground/demos/), and register it in +[PlaygroundApp.tsx](../../../src/tui/playground/PlaygroundApp.tsx). Run `pnpm try --playground` to inspect primitives. Use [exploring-the-wizard](../exploring-the-wizard/SKILL.md) when exercising actual diff --git a/.claude/skills/ink-tui/references/ARCHITECTURE.md b/.claude/skills/ink-tui/references/ARCHITECTURE.md index 9521fac74..4e5863079 100644 --- a/.claude/skills/ink-tui/references/ARCHITECTURE.md +++ b/.claude/skills/ink-tui/references/ARCHITECTURE.md @@ -2,29 +2,54 @@ ## Ownership -| Surface | Owns | -| ----------------------------------------------------------------- | ----------------------------------------------------------------------------- | -| [WizardSession](../../../../src/lib/wizard-session.ts) | Run configuration, decisions, credentials, and lifecycle state | -| [ProgramStep](../../../../src/programs/program-step.ts) | Screens, visibility/completion predicates, gates, and initialization hooks | -| [screen-sequences.ts](../../../../src/ui/tui/screen-sequences.ts) | `ScreenId`, `Screen`, `Sequence`, and the derived `PROGRAM_SEQUENCES` | -| [WizardRouter](../../../../src/ui/tui/router.ts) | Resolution and the `Overlay` stack | -| [WizardStore](../../../../src/ui/tui/store.ts) | Reactive state, gate promises, display observations, and pending interactions | -| [WizardUI](../../../../src/ui/wizard-ui.ts) | Typed operations available to business logic | +| Surface | Owns | +| ------------------------------------------------------------------- | ----------------------------------------------------------------------------- | +| [WizardSession](../../../../src/programs/session/wizard-session.ts) | Run configuration, decisions, credentials, and lifecycle state | +| [TuiState](../../../../src/tui/tui-state.ts) | Each screen's answers, what an overlay shows, and the TUI's launch choices | +| [FlowStep](../../../../src/tui/flow.ts) | Screens, visibility/completion predicates, gates, and `onInit` hooks | +| [TUI programs](../../../../src/tui/programs/index.ts) | Each program's flow, deck, tips, screens and control commits, by program id | +| [TUI tools](../../../../src/tui/tools/index.ts) | Each tool's flow, screens and control commits, by tool id | +| [screen-sequences.ts](../../../../src/tui/screen-sequences.ts) | `Screen`, `Sequence` and `programSequence(programId)`; re-exports `ScreenId` | +| [WizardRouter](../../../../src/tui/router.ts) | Resolution and the `Overlay` stack | +| [WizardStore](../../../../src/tui/store.ts) | Reactive state, gate promises, display observations, and pending interactions | ## Program screens -`createProgramSequence` projects program steps with a `screenId` into `Screen` -entries: `id`, `show`, and `isComplete`. Completion defaults to the step's -`gate` when no separate `isComplete` is provided. Steps without a screen are -omitted, and the exit screen is appended. The projection applies -[withAiOptInGate](../../../../src/programs/ai-opt-in-gate.ts) consistently -with store gate creation. - -`WizardRouter.resolve(session)` first checks the overlay stack, then returns the -first visible, incomplete screen. It also handles the failed-authentication -outro case. Follow this method and the -[router tests](../../../../src/ui/tui/__tests__/router.test.ts) for exact -behavior; there is no program cursor or `next()` API. +Each program's TUI lives in `src/tui/programs//`: its flow (`flow.ts`, or +`skillFlow()`), `screens/`, `screen-ids.ts` and an optional `deck/`. +Its entry (`index.ts(x)`) exports `TUI_PROGRAMS`, mapping each program id it +serves to a [TuiProgram](../../../../src/tui/programs/types.ts): flow, deck, +tips, the screens it mounts, and its control `actions` and `setters`. +`getTuiProgram(programId)`, `getFlow(programId)` and `listTuiPrograms()` look +them up, and a program with no entry gets the generic skill program +(`programs/shared/skill-program.tsx`). The screen registry, control and the +store read programs only through these lookups. Each program folder has its own +`tsconfig.json`, listed in `references` in +[`src/tui/tsconfig.json`](../../../../src/tui/tsconfig.json), so it can't import +another program's folder, and the core sees the registry only through its +signature in [`types/tui-programs.d.ts`](../../../../types/tui-programs.d.ts). + +`createProgramSequence` in [flow.ts](../../../../src/tui/flow.ts) projects flow +steps with a `screenId` into `Screen` entries: `id`, `show`, and `isComplete`. +Completion defaults to the step's `gate` when no separate `isComplete` is +provided. Steps without a screen are omitted, and the exit screen is appended. +`programSequence` applies +[withAiOptInGate](../../../../src/tui/ai-opt-in-gate.ts) before projecting, the +same wrapper the store uses for its gates. + +`WizardRouter.resolve(view)` takes the store as a `TuiView`, its `session` and +its TUI state, the same input every step predicate reads. It first routes a +failed agent run to the mint-failure handoff, then checks the overlay stack, +then returns the first visible, incomplete screen. It also handles the +failed-authentication outro case. Follow this method and the +[router tests](../../../../src/tui/__tests__/router.test.ts) for exact behavior. + +Each tool's screens live in `src/tui/tools//`, built the same way: its entry +exports `TUI_TOOLS`, `getTuiTool(id)` and `listTuiTools()` look them up, and +`runTuiTool` in [run-tool.ts](../../../../src/tui/run-tool.ts) runs one, with no +program run. [`flowOwner(id)`](../../../../src/tui/flow-owner.ts) finds the +owner of a flow id: the tool's TUI first, then the program's. Only a program's +flow gets the AI opt-in step. The screen sequence is a presentation projection of the program. It is distinct from the agent execution sequence selected by the runner; see @@ -32,74 +57,79 @@ from the agent execution sequence selected by the runner; see ## Gates and initialization -Read `ProgramStep` before adding asynchronous work: +Read `FlowStep` and the program's `ProgramConfig.onReady` before adding +asynchronous work: - `gate` supplies a blocking checkpoint, while `isComplete` determines when its screen is finished. Separate them when those conditions differ. - `onInit` runs when the TUI starts rendering, before the real session is assigned. Reserve it for work independent of that session. -- `onReady` runs after the real session is assigned and is awaited in order. +- `ProgramConfig.onReady` runs detection after the real session is assigned. The + store's `runReadyHooks` awaits it once. -[The store](../../../../src/ui/tui/store.ts) derives gate promises from the +[The store](../../../../src/tui/store.ts) derives gate promises from the program. `getGate(stepId)` resolves once: a predicate becoming false later does not close it again. A missing gate returns a resolved promise. `waitUntil(predicate)` evaluates live state at the await point; use that distinction when a decision can change after startup. -[startTUI](../../../../src/ui/tui/start-tui.ts) invokes `runInitHooks` after +[startTUI](../../../../src/tui/start-tui.ts) invokes `runInitHooks` after rendering starts. Merely constructing a store for a test or playground does not start those effects. ## Reactive mutations -The session is a nanostores map. Store setters update it and call -`emitChange()`, which increments the React snapshot version, checks gates, and -detects screen transitions. Consumers subscribe through -`subscribe`/`getSnapshot` and `useSyncExternalStore`. +The session is a nanostores map, and the TUI state an atom beside it. Store +setters update them and call `emitChange()` once, which increments the React +snapshot version, checks gates, and detects screen transitions. Consumers +subscribe through `subscribe`/`getSnapshot` and `useSyncExternalStore`. Display observations such as status messages, tasks, and event plans have separate store atoms. Reuse the existing setters (`pushStatus`, `syncTodos`, `setEventPlan`) for those updates. Do not mutate session fields behind the store after attaching the session. -[InkUI](../../../../src/ui/tui/ink-ui.ts) translates `getUI()` calls into store -operations. Business logic should use this interface rather than import the -store; screens can use the store directly. +TUI code reports through the store it is handed: the login and the scans a +screen runs push status lines, and +[agent-progress.ts](../../../../src/tui/agent-progress.ts) writes a run's +progress to the store (`displayProgress` for a `runProgram` run, `scanProgress` +for a scan). Programs never use the UI. They get a runner context. ## Interaction requests and overlays -Screens own input handling, while `WizardUI` exposes typed interaction requests. +Screens own input handling, while the store exposes typed interaction requests. `requestQuestion` opens the `WizardAsk` overlay and resolves with answers; `showTaskNotice` opens an optional-task notice and resolves with the decision. Their cancellation methods settle pending requests and dismiss the corresponding overlay. Use the established methods rather than pushing a question overlay without its pending state or promise. -Read [WizardAskScreen](../../../../src/ui/tui/screens/WizardAskScreen.tsx), -[TaskNoticeScreen](../../../../src/ui/tui/screens/TaskNoticeScreen.tsx), and the -matching store methods before changing their lifecycle. -[LoggingUI](../../../../src/ui/logging-ui.ts) rejects question requests, -declines optional task notices, and leaves the manual-auth-code promise pending; -it cannot collect terminal input. +Read [WizardAskScreen](../../../../src/tui/screens/WizardAskScreen.tsx), +[TaskNoticeScreen](../../../../src/tui/screens/TaskNoticeScreen.tsx), and the +matching store methods before changing their lifecycle. Headless has no screens: +an uncontrolled headless run passes `runProgram` no interaction, so the agent's +questions and optional task notices get no answer there. A controlled one passes +`storeInteraction`, and its parent answers over the control socket. -`Overlay` lives in the router; `ScreenName` is `ScreenId | Overlay`. Store -`pushOverlay`/`popOverlay` wrappers notify subscribers and preserve transition -direction. Their use for interrupts is separate from program progression. Choose -actual overlay values from the enum; health checks are program screens. +`Overlay` lives in the router; `ScreenName` is a string: a core `ScreenId`, an +`Overlay`, or a program's screen id. Store `pushOverlay`/`popOverlay` wrappers +notify subscribers and preserve transition direction. Their use for interrupts +is separate from program progression. Choose actual overlay values from the +enum; the health check is a core flow screen, not an overlay. ## Components and services -[App](../../../../src/ui/tui/App.tsx) creates the service bundle and screen +[App](../../../../src/tui/App.tsx) creates the service bundle and screen registry, then renders -[ScreenContainer](../../../../src/ui/tui/primitives/ScreenContainer.tsx). The +[ScreenContainer](../../../../src/tui/primitives/ScreenContainer.tsx). The registry injects services where needed, such as the -[MCP installer](../../../../src/ui/tui/services/mcp-installer.ts). Prefer this +[MCP installer](../../../../src/tui/services/mcp-installer.ts). Prefer this boundary when a screen needs external operations rather than coupling a new component to step internals. `ScreenContainer` owns transitions, shared keyboard hints, viewport handling, and -[ScreenErrorBoundary](../../../../src/ui/tui/primitives/ScreenErrorBoundary.tsx). +[ScreenErrorBoundary](../../../../src/tui/primitives/ScreenErrorBoundary.tsx). The boundary records an error outro and run phase on render failure. When changing completion predicates, verify that failure and dismissal remain reachable; the boundary does not replace those predicates. diff --git a/.claude/skills/ink-tui/references/PATTERNS.md b/.claude/skills/ink-tui/references/PATTERNS.md index 1db9a4c03..54f1b9195 100644 --- a/.claude/skills/ink-tui/references/PATTERNS.md +++ b/.claude/skills/ink-tui/references/PATTERNS.md @@ -5,12 +5,12 @@ codebase. | Task | Start here | | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | -| Compose a run screen with conditional event-plan tab | [RunScreen](../../../../src/ui/tui/screens/RunScreen.tsx) | -| Build a program intro | [IntroScreenLayout](../../../../src/ui/tui/screens/IntroScreenLayout.tsx) | -| Compose a learning deck | [LearnCard](../../../../src/ui/tui/components/LearnCard.tsx), [LearnDeckDemo](../../../../src/ui/tui/playground/demos/LearnDeckDemo.tsx) | -| Compare layout primitives | [LayoutDemo](../../../../src/ui/tui/playground/demos/LayoutDemo.tsx) | -| Exercise picker and confirmation controls | [InputDemo](../../../../src/ui/tui/playground/demos/InputDemo.tsx) | -| Add an interaction modal | [WizardAskScreen](../../../../src/ui/tui/screens/WizardAskScreen.tsx), [AskModalDemo](../../../../src/ui/tui/playground/demos/AskModalDemo.tsx) | +| Compose a run screen with conditional event-plan tab | [RunScreen](../../../../src/tui/screens/RunScreen.tsx) | +| Build a program intro | [IntroScreenLayout](../../../../src/tui/screens/IntroScreenLayout.tsx) | +| Compose a learning deck | [LearnCard](../../../../src/tui/components/LearnCard.tsx), [LearnDeckDemo](../../../../src/tui/playground/demos/LearnDeckDemo.tsx) | +| Compare layout primitives | [LayoutDemo](../../../../src/tui/playground/demos/LayoutDemo.tsx) | +| Exercise picker and confirmation controls | [InputDemo](../../../../src/tui/playground/demos/InputDemo.tsx) | +| Add an interaction modal | [WizardAskScreen](../../../../src/tui/screens/WizardAskScreen.tsx), [AskModalDemo](../../../../src/tui/playground/demos/AskModalDemo.tsx) | ## Keep progression in session state @@ -21,14 +21,14 @@ through its session predicate and explicit setter. Do not create a parallel ## Keyboard handling -Use [useKeyBindings](../../../../src/ui/tui/hooks/useKeyBindings.ts) for visible +Use [useKeyBindings](../../../../src/tui/hooks/useKeyBindings.ts) for visible shortcuts: it couples handlers to labels in the shared hints bar. -[KeyboardHintsDemo](../../../../src/ui/tui/playground/demos/KeyboardHintsDemo.tsx) +[KeyboardHintsDemo](../../../../src/tui/playground/demos/KeyboardHintsDemo.tsx) shows the pattern. Reuse its `KeyMatch` vocabulary and choose a distinct registration id for each mounted control. For an ordinary-key dismissal, use -[useDismissOnAnyKey](../../../../src/ui/tui/hooks/useDismissOnAnyKey.ts). It +[useDismissOnAnyKey](../../../../src/tui/hooks/useDismissOnAnyKey.ts). It ignores Ctrl/Meta combinations so the global Ctrl+T HUD shortcut does not also dismiss the screen. @@ -43,7 +43,7 @@ handlers will receive the same key. Visible text belongs inside Ink `Text`. Reuse the shared color/alignment constants and the relevant primitive's border treatment. Use -[useStdoutDimensions](../../../../src/ui/tui/hooks/useStdoutDimensions.ts) when +[useStdoutDimensions](../../../../src/tui/hooks/useStdoutDimensions.ts) when layout must update on terminal resize; reading `useStdout` alone does not subscribe. Account for the shell's title, hints, tabs, status expansion, and optional HUD rather than allocating the full terminal height to screen content. diff --git a/.claude/skills/ink-tui/references/PRIMITIVES.md b/.claude/skills/ink-tui/references/PRIMITIVES.md index 66c066cdc..57722ea96 100644 --- a/.claude/skills/ink-tui/references/PRIMITIVES.md +++ b/.claude/skills/ink-tui/references/PRIMITIVES.md @@ -1,35 +1,35 @@ # Wizard primitives -Use [primitives/index.ts](../../../../src/ui/tui/primitives/index.ts) for public +Use [primitives/index.ts](../../../../src/tui/primitives/index.ts) for public exports and each source file for its current props. Shared styling lives in -[styles.ts](../../../../src/ui/tui/styles.ts). +[styles.ts](../../../../src/tui/styles.ts). ## Choose an existing component -| Need | Component and behavior | -| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| App shell | [ScreenContainer](../../../../src/ui/tui/primitives/ScreenContainer.tsx): title, screen transitions, hints, token HUD, viewport guard, and error boundary | -| Small-terminal notice | [ViewportTooSmall](../../../../src/ui/tui/primitives/ViewportTooSmall.tsx): minimum dimensions and wrapping for the shared guard | -| Tabs and status | [TabContainer](../../../../src/ui/tui/primitives/TabContainer.tsx): local active tab, arrow navigation, optional expandable status | -| Single/multiple choice | [PickerMenu](../../../../src/ui/tui/primitives/PickerMenu.tsx): columns, paging, filtering, and confirm-button interaction | -| Categorized choices | [GroupedPickerMenu](../../../../src/ui/tui/primitives/GroupedPickerMenu.tsx): category headers, scrolling, and multi-select | -| Multi-select submission | [ConfirmButton](../../../../src/ui/tui/primitives/ConfirmButton.tsx): bordered submit row with focus and optional count | -| Continue/cancel | [ConfirmationInput](../../../../src/ui/tui/primitives/ConfirmationInput.tsx): text choices, left/right focus, enter activates, escape cancels | -| Prompt heading | [PromptLabel](../../../../src/ui/tui/primitives/PromptLabel.tsx) | -| Task progress | [ProgressList](../../../../src/ui/tui/primitives/ProgressList.tsx): active labels, completion count, and skipped-task handling | -| Loading state | [LoadingBox](../../../../src/ui/tui/primitives/LoadingBox.tsx) | -| Planned events | [EventPlanViewer](../../../../src/ui/tui/primitives/EventPlanViewer.tsx) | -| Two panes | [SplitView](../../../../src/ui/tui/primitives/SplitView.tsx): equal-width panes with a gap | -| Aligned content | [CardLayout](../../../../src/ui/tui/primitives/CardLayout.tsx): horizontal/vertical alignment | -| Divider | [Divider](../../../../src/ui/tui/primitives/Divider.tsx): measures width on mount | -| Modal layout | [ModalOverlay](../../../../src/ui/tui/primitives/ModalOverlay.tsx): presentation for interrupt screens | -| Terminal links | [LinkText](../../../../src/ui/tui/primitives/LinkText.tsx) and [link helpers](../../../../src/ui/tui/primitives/link-helpers.ts) | -| Log tail | [LogViewer](../../../../src/ui/tui/primitives/LogViewer.tsx): bounded tail reads and throttled file watching | -| HN feed | [HNViewer](../../../../src/ui/tui/primitives/HNViewer.tsx): fetches stories and handles its navigation | -| Progressive content | [ContentSequencer](../../../../src/ui/tui/primitives/ContentSequencer.tsx): renders blocks from [content-types.ts](../../../../src/ui/tui/primitives/content-types.ts) | -| Screen transition | [DissolveTransition](../../../../src/ui/tui/primitives/DissolveTransition.tsx): shared wipe animation | -| Shared hints | [KeyboardHintsBar](../../../../src/ui/tui/primitives/KeyboardHintsBar.tsx): renders registered bindings | -| Render failure | [ScreenErrorBoundary](../../../../src/ui/tui/primitives/ScreenErrorBoundary.tsx) | +| Need | Component and behavior | +| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| App shell | [ScreenContainer](../../../../src/tui/primitives/ScreenContainer.tsx): title, screen transitions, hints, token HUD, viewport guard, and error boundary | +| Small-terminal notice | [ViewportTooSmall](../../../../src/tui/primitives/ViewportTooSmall.tsx): minimum dimensions and wrapping for the shared guard | +| Tabs and status | [TabContainer](../../../../src/tui/primitives/TabContainer.tsx): local active tab, arrow navigation, optional expandable status | +| Single/multiple choice | [PickerMenu](../../../../src/tui/primitives/PickerMenu.tsx): columns, paging, filtering, and confirm-button interaction | +| Categorized choices | [GroupedPickerMenu](../../../../src/tui/primitives/GroupedPickerMenu.tsx): category headers, scrolling, and multi-select | +| Multi-select submission | [ConfirmButton](../../../../src/tui/primitives/ConfirmButton.tsx): flat submit row with a focus marker and optional count | +| Continue/cancel | [ConfirmationInput](../../../../src/tui/primitives/ConfirmationInput.tsx): text choices, left/right focus, enter activates, escape cancels | +| Prompt heading | [PromptLabel](../../../../src/tui/primitives/PromptLabel.tsx) | +| Task progress | [ProgressList](../../../../src/tui/primitives/ProgressList.tsx): active labels, completion count, and skipped-task handling | +| Loading state | [LoadingBox](../../../../src/tui/primitives/LoadingBox.tsx) | +| Planned events | [EventPlanViewer](../../../../src/tui/primitives/EventPlanViewer.tsx) | +| Two panes | [SplitView](../../../../src/tui/primitives/SplitView.tsx): equal-width panes with a gap | +| Aligned content | [CardLayout](../../../../src/tui/primitives/CardLayout.tsx): horizontal/vertical alignment | +| Divider | [Divider](../../../../src/tui/primitives/Divider.tsx): measures width on mount | +| Modal layout | [ModalOverlay](../../../../src/tui/primitives/ModalOverlay.tsx): presentation for interrupt screens | +| Terminal links | [LinkText](../../../../src/tui/primitives/LinkText.tsx) and [link helpers](../../../../src/tui/primitives/link-helpers.ts) | +| Log tail | [LogViewer](../../../../src/tui/primitives/LogViewer.tsx): bounded tail reads and throttled file watching | +| HN feed | [HNViewer](../../../../src/tui/primitives/HNViewer.tsx): fetches stories and handles its navigation | +| Progressive content | [ContentSequencer](../../../../src/tui/primitives/ContentSequencer.tsx): renders blocks from [content-types.ts](../../../../src/tui/primitives/content-types.ts) | +| Screen transition | [DissolveTransition](../../../../src/tui/primitives/DissolveTransition.tsx): shared wipe animation | +| Shared hints | [KeyboardHintsBar](../../../../src/tui/primitives/KeyboardHintsBar.tsx): renders registered bindings | +| Render failure | [ScreenErrorBoundary](../../../../src/tui/primitives/ScreenErrorBoundary.tsx) | The directory also contains internal block renderers and layout helpers. Prefer the public exports for composition; read the implementation before depending on @@ -40,14 +40,14 @@ an internal helper. Both picker primitives place a `ConfirmButton` after the options. Arrow keys navigate through options and onto that button. Enter toggles an option or submits when the button is focused. Grouped selection also supports `a` for all -options. Space aliases enter except when `PickerMenu` has an active filter, -where space is text input. +options. Space aliases enter except in a filterable `PickerMenu` (10 or more +options, or `filterable`), where space is filter text. Use these controls for new multi-select surfaces so submission behavior and keyboard hints stay consistent. See -[InputDemo](../../../../src/ui/tui/playground/demos/InputDemo.tsx) for realistic +[InputDemo](../../../../src/tui/playground/demos/InputDemo.tsx) for realistic props and the -[picker tests](../../../../src/ui/tui/__tests__/picker-filter.test.ts) for +[picker tests](../../../../src/tui/__tests__/picker-filter.test.ts) for filtering behavior. ## Extend the catalog @@ -57,6 +57,6 @@ infrastructure primitives may own state, subscriptions, or file/network I/O; that is not a reason to put program-specific behavior in a new primitive. Export a public component from the barrel, add a -[playground demo](../../../../src/ui/tui/playground/demos/), and register it in -[PlaygroundApp](../../../../src/ui/tui/playground/PlaygroundApp.tsx). Run +[playground demo](../../../../src/tui/playground/demos/), and register it in +[PlaygroundApp](../../../../src/tui/playground/PlaygroundApp.tsx). Run `pnpm try --playground` to review its actual terminal output. diff --git a/.claude/skills/ink-tui/references/TERMINAL-COMPAT.md b/.claude/skills/ink-tui/references/TERMINAL-COMPAT.md index 1b941103a..e6672404a 100644 --- a/.claude/skills/ink-tui/references/TERMINAL-COMPAT.md +++ b/.claude/skills/ink-tui/references/TERMINAL-COMPAT.md @@ -2,32 +2,32 @@ ## Startup and exit -[startTUI](../../../../src/ui/tui/start-tui.ts) creates the store, installs -`InkUI`, renders the app, and starts program initialization. -[terminal.ts](../../../../src/ui/tui/terminal.ts) owns the alternate screen -buffer and black background shared with the playground. Reuse this lifecycle. +[startTUI](../../../../src/tui/start-tui.ts) creates the store, renders the app, +and starts program initialization. +[terminal.ts](../../../../src/tui/terminal.ts) owns the alternate screen buffer +and black background shared with the playground. Reuse this lifecycle. Cleanup unmounts Ink, restores the terminal, and prints the -[exit line](../../../../src/ui/tui/exit-line.ts). Ink teardown on Ctrl+C is -followed by analytics shutdown and process exit so background handles do not -leave a process running without its interface. +[exit line](../../../../src/tui/exit-line.ts). Ink teardown on Ctrl+C calls the +host's `onInterrupt`, which shuts analytics down and ends the run 130, so +background handles do not leave a process running without its interface. ## Dimensions and resizing -[useStdoutDimensions](../../../../src/ui/tui/hooks/useStdoutDimensions.ts) +[useStdoutDimensions](../../../../src/tui/hooks/useStdoutDimensions.ts) subscribes to resize events and supplies fallback dimensions when a stream has none. -[ScreenContainer](../../../../src/ui/tui/primitives/ScreenContainer.tsx) centers +[ScreenContainer](../../../../src/tui/primitives/ScreenContainer.tsx) centers and caps content width. On a real TTY, it shows -[ViewportTooSmall](../../../../src/ui/tui/primitives/ViewportTooSmall.tsx) below +[ViewportTooSmall](../../../../src/tui/primitives/ViewportTooSmall.tsx) below the minimum dimensions declared there. The underlying screen stays mounted and is hidden from layout, preserving input state and avoiding repeated mount effects. Its input handlers still run; the guard does not pause the wizard. Piped stdout does not activate this resize notice. Use -[ViewportGuardDemo](../../../../src/ui/tui/playground/demos/ViewportGuardDemo.tsx) +[ViewportGuardDemo](../../../../src/tui/playground/demos/ViewportGuardDemo.tsx) to inspect the guard. For a layout change, inspect the affected screen near the minimum size and at a wider size, including expanded status or the HUD if those share its available height. @@ -35,7 +35,7 @@ share its available height. ## Noninteractive execution Mode selection belongs to command/runner entry points. The -[default integration command](../../../../src/commands/basic-integration/index.ts) +[default integration command](../../../../src/cli/commands/basic-integration/index.ts) routes explicit CI/headless requests separately and rejects a detected noninteractive environment for the ordinary interactive path. [Environment detection](../../../../src/shared/utils/environment.ts) checks @@ -43,12 +43,13 @@ stdout/stderr TTY state outside development mode; checking stdin alone does not describe this command's behavior. `--ci` is a development/test path. The separate experimental headless contract -is defined in [headless-mode.ts](../../../../src/lib/headless-mode.ts); keep its -hidden flag out of user-facing guidance. Other commands can have their own +is defined in [headless-mode.ts](../../../../src/shared/headless-mode.ts), and +its `HEADLESS_FLAG` constant lives in [env.ts](../../../../src/env.ts). Keep +that hidden flag out of user-facing guidance. Other commands can have their own fallback, such as -[MCP TUI availability](../../../../src/commands/mcp/tui-availability.ts). +[MCP TUI availability](../../../../src/cli/commands/mcp/tui-availability.ts). Do not add an automatic default-answer or Inquirer fallback in a screen. -Noninteractive behavior belongs in [LoggingUI](../../../../src/ui/logging-ui.ts) -and the command/runner contract, including explicit handling of interactions -that cannot be answered. +Noninteractive behavior belongs in the +[headless host](../../../../src/headless/run.ts) and the command/runner +contract, including explicit handling of interactions that cannot be answered. diff --git a/.claude/skills/wizard-development/SKILL.md b/.claude/skills/wizard-development/SKILL.md index 3fa776651..cd3a7629c 100644 --- a/.claude/skills/wizard-development/SKILL.md +++ b/.claude/skills/wizard-development/SKILL.md @@ -8,7 +8,7 @@ description: > compatibility: Coding agents working in the PostHog Wizard repository. metadata: author: posthog - version: '2.0' + version: '2.2' --- # Wizard development @@ -17,24 +17,27 @@ Ask who owns the concern and what they should need to understand to change it. Product knowledge belongs behind typed configuration boundaries; runner and UI infrastructure should consume those boundaries. -| Concern | Owner | -| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Framework detection, context, env conventions | [FrameworkConfig](../../../src/programs/framework-config.ts) and [framework configs](../../../src/programs/frameworks/) | -| Integration instructions and orchestrator flows/tasks | [context-mill](https://github.com/PostHog/context-mill) | -| Programs, steps, prerequisites and outcomes | [programs](../../../src/programs/) | -| Sequence, harness, model and effort selection | [switchboard](../../../src/agent/runner/switchboard/) | -| Local tool permissions and scanner adapters | [agent-interface](../../../src/agent/agent-interface.ts), [YARA hooks](../../../src/agent/yara-hooks.ts), [Pi security](../../../src/agent/runner/harness/pi/security.ts) | -| Scanner rules | [warlock](https://github.com/PostHog/warlock) | -| Token admission and budgets | [PostHog mint endpoint](https://github.com/PostHog/posthog/blob/master/posthog/llm/wizard_gateway_token.py) and [ai-gateway](https://github.com/PostHog/ai-gateway) | -| Screen resolution and rendering | [TUI](../../../src/ui/tui/) through [WizardUI](../../../src/ui/wizard-ui.ts) | +| Concern | Owner | +| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Framework detection, context, env conventions | [FrameworkConfig](../../../src/programs/framework-config.ts) and [framework configs](../../../src/programs/frameworks/) | +| Integration instructions and orchestrator flows/tasks | [context-mill](https://github.com/PostHog/context-mill) | +| Programs: detection, runs, prerequisites and outcomes | One folder per program in [programs](../../../src/programs/), entered through its `index.ts` | +| Tools: commands that run no agent | [tools](../../../src/tools/), entered through `@tools`, with their screens in [TUI tools](../../../src/tui/tools/) | +| Program screens, flows and decks | [TUI programs](../../../src/tui/programs/) | +| Sequence, harness, model and effort selection | [switchboard](../../../src/agent/runner/switchboard/) | +| Local tool permissions and scanner adapters | [agent-interface](../../../src/agent/agent-interface.ts), [YARA hooks](../../../src/agent/yara-hooks.ts), [Pi security](../../../src/agent/runner/harness/pi/security.ts) | +| Scanner rules | [warlock](https://github.com/PostHog/warlock) | +| Token admission and budgets | [PostHog mint endpoint](https://github.com/PostHog/posthog/blob/master/posthog/llm/wizard_gateway_token.py) and [ai-gateway](https://github.com/PostHog/ai-gateway) | +| Screen resolution and rendering | [TUI](../../../src/tui/) through its [store](../../../src/tui/store.ts) | +| Layer import boundaries | One tsconfig project per layer, checked by `pnpm typecheck`, and ESLint rules for the paths the compiler can't see; see [layer boundaries](references/ARCHITECTURE.md#layer-boundaries) | ## Execution policy and model admission - **Pi is the default choice for new work.** Harness choice and model provider are separate: Pi supports gateway-backed Anthropic and OpenAI transports. - **Prefer orchestration.** A seed agent plans work, then task agents execute - focused conversations. Start from - [metrics](../../../src/programs/metrics/) and its context-mill flow. + focused conversations. Start from [metrics](../../../src/programs/metrics/) + and its context-mill flow. - **Linear is for very simple tasks and legacy support.** Composed program sub-runs are also structurally clamped to linear; orchestration cannot nest through that seam. @@ -42,19 +45,18 @@ infrastructure should consume those boundaries. the default.** Retain it for major Pi vulnerabilities or missing support for new Anthropic models. -Existing routing has not all migrated: -[DEFAULT_BINDING](../../../src/agent/runner/switchboard/index.ts) selects -Pi + linear, with per-program and flag overrides. Set new -bindings explicitly. Migrating an existing program requires checking its flow, -tasks, and lifecycle hooks; changing the default constant alone is insufficient. -Both harnesses implement `run` and `runTask`. +[DEFAULT_BINDING](../../../src/agent/runner/switchboard/index.ts) selects Pi + +linear for a program whose config sets no `binding`; flags and development +overrides sit on top. Set new bindings explicitly. Migrating an existing program +requires checking its flow, tasks, and lifecycle hooks; changing the default +constant alone is insufficient. Both harnesses implement `run` and `runTask`. Adding a model or effort is a cross-repository change: 1. Define the Wizard model ID and capabilities in [constants](../../../src/shared/constants.ts) and - [models](../../../src/agent/runner/switchboard/models.ts); select it - through the switchboard or supported context-mill stage metadata. + [models](../../../src/agent/runner/switchboard/models.ts); select it through + the switchboard or supported context-mill stage metadata. 2. Check the PostHog mint's `WIZARD_MODEL_ALLOWLIST` and allowed efforts. The token's policy is enforced by the gateway; a local constant or CLI override cannot authorize a model. @@ -80,11 +82,17 @@ capabilities table. - For a capability, follow [adding-skill-program](../adding-skill-program/SKILL.md). Prefer a context-mill command within an existing family when that is sufficient. A - native command needs a command module and registration in `bin.ts`, as well as - program and binding registration. -- For screens or primitives, follow [ink-tui](../ink-tui/SKILL.md). Program - steps drive screen sequences; business logic calls `getUI()`, and session - mutations use store setters that emit changes. + native program with a `command` gets its CLI command from the program + registry; a command with its own shape goes in `CUSTOM_COMMANDS` in + `src/cli/commands/index.ts`. +- For a command that runs no agent, such as `mcp add` or `doctor`, follow + [Add a tool](../../../src/tools/README.md#add-a-tool). A tool never goes + through `runProgram`, and no program imports one. +- For screens or primitives, follow [ink-tui](../ink-tui/SKILL.md). A program's + flow in `src/tui/programs//` drives its screen sequence. Programs never + use the UI; they get a runner context. TUI code takes the store it reports + through as an argument, and session mutations use store setters that emit + changes. - For headless exploration, follow [exploring-the-wizard](../exploring-the-wizard/SKILL.md). Drive current legal actions and inspect error outros and pending questions throughout execution. @@ -94,13 +102,12 @@ only when it gives a real owner a smaller, reusable boundary. ## Lifecycle and security -[runner/index.ts](../../../src/agent/runner/index.ts) bootstraps, resolves a -binding, and dispatches to a sequence. `agent-runner.ts` is a compatibility -re-export, not the implementation. Sequences own their lifecycle; harnesses own -SDK calls. `ProgramRun.postRun`, `buildOutroData`, `customPrompt`, and -`abortCases` are consumed by the linear sequence, not by the orchestrator. Put -orchestrated work in its flow/tasks and inspect its completion path when -extending it. +[runner/index.ts](../../../src/agent/runner/index.ts) resolves the route from +`RunConfig.routing`, bootstraps, and dispatches to a sequence. Sequences own +their lifecycle; harnesses own SDK calls. `ProgramRun.postRun`, +`buildOutroData`, `customPrompt`, and `abortCases` are consumed by the linear +sequence, not by the orchestrator. Put orchestrated work in its flow/tasks and +inspect its completion path when extending it. Keep tool enforcement at the boundary: shared permissions, harness-specific adapters, and warlock scanning. Scanner failure must not silently allow an @@ -112,8 +119,8 @@ User-provided secrets should travel through [secret-vault](../../../src/shared/secret-vault.ts) references. Tool implementations resolve values host-side where they are used; they must not return the raw value to the model. Inspect the -[wizard-tools](../../../src/agent/tools/) implementations and the Pi -adapter when extending this shared tool surface. +[wizard-tools](../../../src/agent/tools/) implementations and the Pi adapter +when extending this shared tool surface. ## Verification and maintenance diff --git a/.claude/skills/wizard-development/references/ANTI-PATTERNS.md b/.claude/skills/wizard-development/references/ANTI-PATTERNS.md index 9eed4d84f..afc55f080 100644 --- a/.claude/skills/wizard-development/references/ANTI-PATTERNS.md +++ b/.claude/skills/wizard-development/references/ANTI-PATTERNS.md @@ -38,10 +38,18 @@ state that a step's `show`, `isComplete`, or `gate` predicate reads. Use the existing overlay mechanism for a real overlay. `requires` documents a prerequisite but does not run it. Detect prerequisites -explicitly or compose work through `ProgramStep.run`. Use store setters for -shared session data; do not rely on another program having populated it by +explicitly or compose work through `ProgramConfig.runSteps`. Use store setters +for shared session data; do not rely on another program having populated it by accident. Shared detection can be a function called by both `onReady` hooks. +## A program reaching past its folder + +A program that reaches the UI, renders, or imports another program's folder ties +its owner to code they don't own. Programs get a runner context and read a +`ProgramSession`. Their screens live in `src/tui/programs//`. Move code two +programs share into `src/programs/shared/` or `detection/`. Don't widen a +layer's tsconfig `paths` to make an import compile. + ## Bundling fast-changing integration knowledge Context-mill publishes integration skills and agent flows independently of the diff --git a/.claude/skills/wizard-development/references/ARCHITECTURE.md b/.claude/skills/wizard-development/references/ARCHITECTURE.md index 570d1e5e1..e2fe0c03a 100644 --- a/.claude/skills/wizard-development/references/ARCHITECTURE.md +++ b/.claude/skills/wizard-development/references/ARCHITECTURE.md @@ -8,27 +8,35 @@ description: ## Runner and lifecycle -[run-wizard.ts](../../../../src/lib/runners/run-wizard.ts) owns the program's -interactive lifecycle: create the session and TUI, assign the session, run -readiness hooks, and traverse steps and gates. -[store.ts](../../../../src/ui/tui/store.ts) runs `onInit` when the TUI starts; -`onReady` runs after the real session is assigned. Keep session-dependent -detection in `onReady`. Noninteractive execution has its own lifecycle in -[run-non-interactive.ts](../../../../src/lib/runners/run-non-interactive.ts). - -[runner/index.ts](../../../../src/agent/runner/index.ts) resolves a -program's `run` definition, calls shared bootstrap, selects a binding, -dispatches the sequence, and flushes the scanner report on cleanup. The old -[agent-runner.ts](../../../../src/agent/agent-runner.ts) is a compatibility -export. - -| Layer | Source and responsibility | -| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Bootstrap | [shared/bootstrap.ts](../../../../src/agent/runner/shared/bootstrap.ts): shared health/settings/auth/flag and MCP setup | -| Switchboard | [switchboard/index.ts](../../../../src/agent/runner/switchboard/index.ts): resolve sequence, harness, model and effort override | -| Linear sequence | [sequence/linear.ts](../../../../src/agent/runner/sequence/linear.ts): one conversation, skill/prompt assembly, post-run hooks and outro | -| Orchestrator | [orchestrator-runner.ts](../../../../src/agent/runner/sequence/orchestrator/orchestrator-runner.ts): seed plan, task queue, focused conversations, handoffs and completion | -| Harness | [harness/types.ts](../../../../src/agent/runner/harness/types.ts): SDK boundary, implemented by Pi and Anthropic | +The CLI picks a host and hands it the launch values. +[runTui](../../../../src/tui/run.ts) owns the interactive lifecycle: start the +TUI, build and assign the session, run the readiness hooks, settle the intro, +and call `runProgram` once, with the flow's screens answering its steps through +[tuiWorkflow](../../../../src/tui/workflow.ts). +[store.ts](../../../../src/tui/store.ts) runs each flow step's `onInit` when the +TUI starts. `runReadyHooks` awaits the program's `ProgramConfig.onReady` once, +after the real session is assigned, and marks detection complete. Keep +session-dependent detection in `onReady`. Noninteractive execution has its own +host, [runHeadless](../../../../src/headless/run.ts), which builds its own +session store and calls `runProgram` with no workflow; `runProgram` then runs +the same `onReady` as its detection, or `ciPreRun` when the program sets one. + +[runProgram](../../../../src/programs/run-program.ts) runs detection when the +store hasn't, resolves the program's `run` definition, applies its readiness, +login, approval and flag policy, and calls `runAgent` for each run step the host +confirms and then the program's own run, writing everything into the caller's +[SessionStore](../../../../src/programs/session/session-store.ts). +[runner/index.ts](../../../../src/agent/runner/index.ts) resolves the route from +`RunConfig.routing`, calls shared bootstrap, dispatches the sequence, and +flushes the scanner report on cleanup. + +| Layer | Source and responsibility | +| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Bootstrap | [shared/bootstrap.ts](../../../../src/agent/runner/shared/bootstrap.ts): logging targets, the gateway mint and the scan-triage classifier; health, settings, login and flags arrive resolved from `runProgram` | +| Switchboard | [switchboard/index.ts](../../../../src/agent/runner/switchboard/index.ts): resolve sequence, harness, model and effort override | +| Linear sequence | [sequence/linear.ts](../../../../src/agent/runner/sequence/linear.ts): one conversation, skill/prompt assembly, post-run hooks and outro | +| Orchestrator | [orchestrator-runner.ts](../../../../src/agent/runner/sequence/orchestrator/orchestrator-runner.ts): seed plan, task queue, focused conversations, handoffs and completion | +| Harness | [harness/types.ts](../../../../src/agent/runner/harness/types.ts): SDK boundary, implemented by Pi and Anthropic | The contribution policy is [Pi by default, orchestration preferred](../SKILL.md#execution-policy-and-model-admission). @@ -40,7 +48,8 @@ route. ### Configuration hooks -Read [ProgramRun](../../../../src/agent/runner/shared/types.ts) for exact +Read [ProgramRun](../../../../src/programs/program-run.ts) and +[AgentRunDefinition](../../../../src/agent/runner/shared/types.ts) for exact signatures. | Surface | Current consumer | @@ -48,15 +57,18 @@ signatures. | `customPrompt`, `abortCases` | Linear prompt assembly and abort handling | | `postRun(session, credentials)` | Linear success path, before outro | | `buildOutroData(session, credentials)` | Linear custom outro; otherwise defaults from run metadata | +| `buildOutroNextSteps(session, ...)` | Orchestrated outro bullets | | `agentFlow` and task skill variants | Orchestrator flow selection and task execution | -| `ProgramStep.run` | Explicit program composition in the outer lifecycle | +| `ProgramConfig.binding` | The program's route; `DEFAULT_BINDING` when unset | +| `ProgramConfig.runSteps` | Explicit program composition in the outer lifecycle | | `ProgramConfig.requires` | Dependency metadata; it does not execute prerequisite programs | Do not migrate a linear program merely by changing its binding if it depends on these hooks. Inspect the orchestrator's flow and completion path instead. [Metrics](../../../../src/programs/metrics/) is a current Pi/orchestrator -example. Native command modules still need registration in -[bin.ts](../../../../bin.ts); screen sequences derive from the program registry. +example. A program with a top-level `command` gets its CLI command from the +registry through [wizardCommands](../../../../src/cli/commands/index.ts), and +screen sequences derive from each program's TUI flow. ## Switchboard contract @@ -65,8 +77,8 @@ than copying their fields into another document. It receives the program, flag snapshot/payloads, composition state, and development overrides, returning the binding and stamping a trace of the selected precedence rungs. -- Harness/model: development CLI override, declared flag route, per-program - binding, default. +- Harness/model: development CLI override, declared flag route, the program's + binding (`DEFAULT_BINDING` when it declares none). - Sequence: composed linear clamp, development CLI override, `runTask` capability clamp, program flag route, sequence experiment, binding/default. @@ -78,10 +90,10 @@ resolution. Composed sub-runs remain linear even when a CLI override requests orchestration. Effort resolves in two stages: the binding supplies an override, then -[modelCapabilities](../../../../src/agent/runner/switchboard/models.ts) -applies capabilities and defaults. All selected models and efforts must also be -admitted by the minted token and gateway. Local routing cannot bypass that -external policy; see the +[modelCapabilities](../../../../src/agent/runner/switchboard/models.ts) applies +capabilities and defaults. All selected models and efforts must also be admitted +by the minted token and gateway. Local routing cannot bypass that external +policy; see the [model admission checklist](../SKILL.md#execution-policy-and-model-admission). Flags belong in @@ -89,8 +101,9 @@ Flags belong in Experiments declare their program scope; malformed payloads yield no experiment route. Reuse the [switchboard tests](../../../../src/agent/runner/__tests__/switchboard.test.ts) -and experiment tests to check full bindings and isolation of unrelated programs. -Do not add a second flag-reading path inside a harness or sequence. +and experiment tests to check full bindings and isolation of unrelated programs; +they run on synthetic programs, and each program pins its own binding in its +`__tests__`. Do not add a second flag-reading path inside a harness or sequence. ## Security boundaries @@ -99,10 +112,10 @@ Wizard's local tool boundary separately restricts operations on the user's project. Local commandments provide model guidance; they are not an enforcement mechanism. -- [agent-interface.ts](../../../../src/agent/agent-interface.ts) configures - the Anthropic SDK's tool permissions, sandbox, and gateway transport. -- [yara-hooks.ts](../../../../src/agent/yara-hooks.ts) adapts warlock scans to SDK - tool hooks. +- [agent-interface.ts](../../../../src/agent/agent-interface.ts) configures the + Anthropic SDK's tool permissions, sandbox, and gateway transport. +- [yara-hooks.ts](../../../../src/agent/yara-hooks.ts) adapts warlock scans to + SDK tool hooks. - [Pi security](../../../../src/agent/runner/harness/pi/security.ts) adapts shared permissions and scanning to Pi tool-call/result events, including blocking, violation latching, and tool-call limits. @@ -119,44 +132,58 @@ terminates the run. Check each adapter's state machine when changing rejection handling. Preserve useful rejection diagnostics without logging secret content. Both SDK paths use a scoped gateway token minted through -[gateway-session.ts](../../../../src/agent/gateway-session.ts), not the user's raw -OAuth credential as a model API key. Rejection and refresh behavior belong at -that seam; do not restore a legacy-gateway fallback to bypass admission. +[gateway-session.ts](../../../../src/agent/gateway-session.ts), not the user's +raw OAuth credential as a model API key. Rejection and refresh behavior belong +at that seam; do not restore a legacy-gateway fallback to bypass admission. ### Secret vault: keeping values out of the model [secret-vault.ts](../../../../src/shared/secret-vault.ts) stores user-provided values in memory and returns opaque references. Sensitive `wizard_ask` answers become `secret:` refs; `set_env_values` resolves the ref host-side when -writing. Follow [wizard-tools](../../../../src/agent/tools/) and the Pi -adapters when adding a secret-consuming tool. Return references and metadata to -the agent, never the raw value. References are session-scoped, not durable +writing. Follow [wizard-tools](../../../../src/agent/tools/) and the Pi adapters +when adding a secret-consuming tool. Return references and metadata to the +agent, never the raw value. References are session-scoped, not durable credentials. ## UI state and agent output -Business logic uses [WizardUI](../../../../src/ui/wizard-ui.ts) through -`getUI()`. [InkUI](../../../../src/ui/tui/ink-ui.ts) updates the TUI store; -[LoggingUI](../../../../src/ui/logging-ui.ts) is available for noninteractive -callers that select it. A missing TTY does not automatically mean an arbitrary -caller uses LoggingUI; snapshot CI drives Ink in a PTY. `requestQuestion` and -task notices are supported interactions, not console prompts to invent in -business logic. +Programs never use the UI. A program's `run` gets a `RunnerContext` and its +`ciPreRun` gets a `CiRunnerContext`, both from +[runner-context.ts](../../../../src/programs/runner-context.ts), and it reads a +`ProgramSession`. Each host reports through its own renderer, with no UI +interface shared between them: TUI code takes the +[WizardStore](../../../../src/tui/store.ts) it reports through as an argument, +headless prints through its +[LoggingUI](../../../../src/headless/renderers/logging-ui.ts), and CLI commands +through `consoleLog` (`@shared/console-log`). There are no process-global sinks. +An agent run's debug lines are info log progress, which the host's progress +handler shows. An abort takes its presenter as an argument: +[abortOnScreens](../../../../src/tui/abort.ts) in the TUI, `printAbortOutro` in +headless and before the TUI mounts. `runProgram` writes run state into the +session store; `displayProgress` updates the TUI's display state; `LoggingUI` +prints log lines for `--ci` and headless runs. A missing TTY does not +automatically mean an arbitrary caller prints log lines; snapshot CI drives Ink +in a PTY. `requestQuestion` and task notices are supported interactions, not +console prompts to invent in business logic. Harness adapters translate SDK messages, status markers, task updates and tool -activity into WizardUI calls. Anthropic message processing lives in -[agent-interface.ts](../../../../src/agent/agent-interface.ts); Pi uses its -own session event handlers. Orchestrated tasks also have queue and handoff -state. Do not assume all harness output passes through `handleSDKMessage`. +activity into `AgentProgress` events. The agent never renders: +[agent-progress.ts](../../../../src/tui/agent-progress.ts) writes each event to +the store. Anthropic message processing lives in +[agent-interface.ts](../../../../src/agent/agent-interface.ts); Pi uses its own +session event handlers. Orchestrated tasks also have queue and handoff state. Do +not assume all harness output passes through `handleSDKMessage`. Session changes go through explicit store setters. They emit updates, re-evaluate gates, detect transitions, and refresh rendering. The -[router](../../../../src/ui/tui/router.ts) resolves overlays first, then the -first visible incomplete screen from -[screen-sequences.ts](../../../../src/ui/tui/screen-sequences.ts). Those -sequences are projected from registered program steps. Change the -state/predicate that represents progress rather than adding imperative -navigation. +[router](../../../../src/tui/router.ts) routes a failed agent run to the +mint-failure handoff, then resolves overlays, then the first visible incomplete +screen from [screen-sequences.ts](../../../../src/tui/screen-sequences.ts). +Those sequences are projected from each registered program's or tool's flow, +which [`flowOwner`](../../../../src/tui/flow-owner.ts) looks up by id: the TUI +tool registry first, then the TUI program registry. Change the state/predicate +that represents progress rather than adding imperative navigation. ## MCP and instrumentation @@ -166,6 +193,149 @@ them differently; inspect the selected harness rather than assuming identical tool names or discovery. Context-mill supplies skills and flow/task prompts. [Middleware](../../../../src/agent/middleware/) provides opt-in message/phase -instrumentation. The linear sequence creates the benchmark pipeline; there is no -pipeline construction in the compatibility `agent-runner.ts`. Inspect the actual -consumer before extending instrumentation to another sequence or harness. +instrumentation. The linear sequence creates the benchmark pipeline. Inspect the +actual consumer before extending instrumentation to another sequence or harness. + +## Layer boundaries + +Each layer is a TypeScript project, and `pnpm typecheck` is `tsc -b` on the root +[tsconfig.json](../../../../tsconfig.json), which only lists them. The build +orders itself by `references` and emits declarations into `.tsbuild/`. A layer's +`tsconfig.json` extends [tsconfig.no-tui.json](../../../../tsconfig.no-tui.json) +(the TUI extends [tsconfig.base.json](../../../../tsconfig.base.json)), holds +its folder with its tests, and `references` only the layers it may import. +Importing a file of a project it doesn't reach fails with TS6307. A few ESLint +rules close the paths the compiler can't see. CI runs `pnpm typecheck` in +[build.yml](../../../../.github/workflows/build.yml). + +`tsconfig.base.json` holds the compiler options and the aliases, entries only: +`@env`, `@shared/*`, `@utils/*`, `@host/*`, `@agent`, `@agent/types`, +`@programs`, `@programs/types`, `@programs/`, `@tools`, `@tui`, `@headless` +and `@cli`. Three deep aliases stay for the layer that owns them: `@agent/*`, +`@programs/*` and `@cli/*` in tests and source, and `@tui/*` in the TUI. +`extends` copies `compilerOptions` but not `references`, and replaces `paths` +whole, so the projects that need other aliases list them again. + +| Layer | Project | References | +| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- | +| env | `src/tsconfig.json`: `env.ts` | Nothing | +| shared | `src/shared/tsconfig.json` | env | +| host | `src/host/tsconfig.json` | env, shared | +| agent | `src/agent/tsconfig.json` | env, shared | +| tools | `src/tools/tsconfig.json` | env, shared, agent | +| shared program code | `src/programs/tsconfig.core.json` | env, shared, agent | +| one program | `src/programs//tsconfig.json` | The shared program code | +| programs entry | `src/programs/tsconfig.json`: `index.ts`, `types.ts`, `program-registry.ts`, `run-program.ts`, `detect-program.ts`, `detect-map.ts`, `__tests__` | The shared program code and every program | +| headless | `src/headless/tsconfig.json` | env, shared, host, agent, programs entry | +| TUI core | `src/tui/tsconfig.core.json` | env, shared, host, agent, programs entry, tools | +| one TUI program | `src/tui/programs//tsconfig.json` | The TUI core | +| one TUI tool | `src/tui/tools//tsconfig.json` | The TUI core | +| TUI entry | `src/tui/tsconfig.json`: `index.ts`, `programs/index.ts`, `tools/index.ts`, `playground/` and the core's tests | The TUI core, every TUI program and tool | +| cli | `src/cli/tsconfig.json` | env, shared, host, agent, programs entry, tools, TUI entry, headless | +| bin | `tsconfig.bin.json`: `bin.ts`, `tsdown.config.ts`, `vitest.config.ts` | env, shared, cli | +| harness | `e2e-harness/tsconfig.json`: `e2e-harness/`, `scripts/` and `docs/examples/` | env, shared, host, agent, tools, programs entry, TUI entry, headless | +| mocks | `__mocks__/tsconfig.json` | Nothing | + +The host layer, `src/host`, is how the hosts end a run: `startHostExit`, +`wizardAbort`, `registerShutdown` and `withControlledAbort` in +`@host/wizard-abort`. A host resolves its run's exit code; `wizardAbort` shows a +decided failure's outro through the presenter its caller passes and hands the +code to it, and only the CLI calls `process.exit`. It also holds the control +API, `@host/control`, which each host attaches to its own store: the server +knows no program, so a host passes the program ids a request may name. Headless, +the TUI, the CLI and the harness reference it. The agent and the programs don't, +so an import of it there fails with TS6307. + +The TUI, headless and the CLI each have one entry, their `index.ts`. Each entry +function loads its module on first call, so importing an entry is as cheap as +importing its launch types: + +| Entry | Exports | Imported by | +| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | +| `@tui` | `runTui` with `TuiLaunch`, `TuiLaunchChoices` and `TuiControlAttach`, `runTuiTool` with `TuiToolLaunch`, `renderFamilyPicker` with `FamilyPickerOption`, `runPlayground`, `createTuiTarget` and `tuiProgramFlow` with `FlowStep`, `ScreenId` and `Overlay` | The CLI and the e2e harness | +| `@headless` | `runHeadless`, `HeadlessLaunch` and `HeadlessMode` | The CLI | +| `@cli` | `runCli` | `bin.ts` | + +### Splits and stubs + +The programs and the TUI each split in two so a folder can't import its sibling. +The core (`tsconfig.core.json`) holds the shared code, and each program, TUI +program and TUI tool folder is its own project that references it, so a +sibling's folder fails with TS6307. The entry project (`tsconfig.json` beside +the core) holds the registry and references the core and every folder, so a +registry imports the folders and nothing imports the registry back. The entry +project also holds the layer's tests, since a test may import any part. The TUI +core calls the registries through two signature stubs, `types/tui-programs.d.ts` +and `types/tui-tools.d.ts`, that its own `paths` map to `@tui/programs/index` +and `@tui/tools/index`; it reads a flow through `flowOwner`, so a tool's id +never falls back to the skill flow. + +A new program, TUI program or TUI tool folder needs its own `tsconfig.json`, +copied from a sibling, and a `references` line in its entry project. See +[adding-skill-program](../../adding-skill-program/SKILL.md). + +### Fences + +Outside the TUI, `ink`, `ink/*`, `react`, `react/*`, `@inkjs/ui` and +`ink-testing-library` map, in `tsconfig.no-tui.json` and `tsconfig.bin.json`, to +`types/tui-only.fence`. That resolves to `types/tui-only.d.fence.ts`, and +TypeScript rejects every import of a `.d.ts` file with an arbitrary extension +while `allowArbitraryExtensions` is off, so never turn it on. A named, default, +namespace or side-effect import, `import()`, `typeof import()` and `export *` +all fail with TS6263. A missing fence file falls back to the real package, so +keep it. Three more options close the rest: + +| Option | Effect | +| ------------------------------ | ------------------------------------------------------------------------------------------------------- | +| `noUncheckedSideEffectImports` | `import 'x'` resolves like any other import, so it meets the fences and fails when nothing resolves | +| `moduleDetection: "force"` | Every `.ts` file is a module, so `declare module 'ink'` in one augments the fence and fails with TS6263 | +| `resolveJsonModule: false` | A JSON import fails with TS2732. The MCP prompt copy is a TS data module, `mcp-role-prompts.copy.ts` | + +### What each import fails with + +A project reaches what it references, and what those reference. TypeScript +doesn't stop that second hop, and `paths` can't stop a relative path, so +`tsconfig.bin.json` maps only `@cli` for `bin.ts`, and ESLint closes the rest. + +| Import | Fails with | +| ----------------------------------------------------------------------------------------------------------------- | ----------------------------- | +| A layer's file from a layer that doesn't reference it: `@programs/audit` in the agent, `@headless` in the TUI | TS6307 | +| A sibling program or TUI program folder, or a registry from a program or TUI folder | TS6307 | +| An alias the project doesn't map: `@tui/run` in the CLI, `@tui`, `@headless` or `@agent` in `bin.ts` | TS2307 | +| `ink`, `react`, `@inkjs/ui` or `ink-testing-library` outside the TUI, in any form | TS6263 | +| A relative path that leaves the layer's folder: `../tui/run` in the CLI, `../src/programs/audit/x` in the harness | ESLint `no-restricted-syntax` | +| A deep alias outside its layer: `@programs/audit/x` or `@agent/runner/x` in the harness | ESLint `no-restricted-syntax` | + +TS6307 lands on the import line, and again at the imports of every file that +import pulls in. tsc sorts errors by path, so an import of another program or +the registry can print dozens of errors in other program folders before the one +in your file. + +### What ESLint checks + +TypeScript can't see these paths, so `pnpm lint` rejects each with a rule in +[.eslintrc.cjs](../../../../.eslintrc.cjs): + +| Rule | Rejects | Why the compiler can't | +| ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `no-restricted-imports`, patterns `.tsbuild` and `node_modules` | An import or re-export whose path has a `.tsbuild/` or `node_modules/` segment | `paths` never sees a relative path, so `../../node_modules/ink/build/index.js` compiles past the fence | +| `no-restricted-syntax` on `import()` and `typeof import()` | The same paths in `import('...')`, a template-literal `import()` and `typeof import('...')` | The same, and `no-restricted-imports` sees only import and export declarations | +| `no-restricted-imports`, paths `module` and `node:module`, and `no-restricted-syntax` on their `import()` and `typeof import()` | Any import, re-export, `import()` or `typeof import()` of `module` or `node:module` | Its loaders, such as `createRequire`, return `any` and load what no project resolves. Two uses carry a disable line with their reason: the SDK's CLI path in `src/agent/agent-interface.ts` and the `@xterm/headless` CJS entry in `e2e-harness/tui-capture.ts` | +| `no-restricted-globals`, `require` | A bare `require` | `@types/node` declares it, so TypeScript types a call as `any` and resolves no path. A local `require` from `createRequire` is not the global | +| `@typescript-eslint/triple-slash-reference`, `path: 'never'` | `/// ` | It loads a declaration file past the project's file list, and a `declare module` in that file beats `paths` | +| `no-restricted-syntax` on `vi.mock` and `vi.doMock` | A string-form mock, `vi.mock('')` | A string is not an import, so no project resolves it. `vi.mock(import(''))` resolves in the test's project, and its factory must match the module's exports | +| `no-restricted-syntax`, one override per layer folder and depth | A relative import that leaves its layer's folder, and a deep alias outside its own layer | A project reaches every file of a project it references, by any path. `@agent/types` and each `@programs/` stay allowed | + +Neither the compiler nor these rules stop an `import()` of a computed path, +which TypeScript types as `any`, or the string a test hands `vi.importActual`, +which no project resolves. ESLint ignores `scripts/`, so the rules don't reach +it; the harness project still compiles it against its references only. + +Don't widen a layer's `paths` or `references` to make an import compile, and +don't route an import through one of these gaps. Move the code, or add the name +to the entry it should come through. A test of another layer's internals belongs +in that layer's tests. A test mocks another layer through the entry it imports, +`vi.mock(import(''))`. The factory's value must match the module's +exports; a partial stub of an object or a function with a narrower return is +cast `as never`. A harness test reads a TUI program's flow through +`tuiProgramFlow` on `@tui`. diff --git a/.claude/skills/wizard-development/references/MAINTAINING-SKILLS.md b/.claude/skills/wizard-development/references/MAINTAINING-SKILLS.md index 92a1430db..5ba038a19 100644 --- a/.claude/skills/wizard-development/references/MAINTAINING-SKILLS.md +++ b/.claude/skills/wizard-development/references/MAINTAINING-SKILLS.md @@ -19,8 +19,9 @@ someone following a skill is also a reason to revisit it. - Open each linked source and check required fields, signatures, and actual consumers. -- Trace registration: native commands, program registry, binding registry, and - context-mill families have different responsibilities. +- Trace registration: the program registry, the CLI's custom commands, each + program's `binding`, and context-mill families have different + responsibilities. - Check sequence-specific behavior. A hook supported by linear execution need not run under orchestration. - Distinguish contribution policy from existing runtime defaults and deployed diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 7bceb017f..48af003c7 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -19,11 +19,18 @@ /src/programs/warehouse-source/ @PostHog/team-warehouse-sources /src/programs/web-analytics-doctor/ @PostHog/team-web-analytics -# Team-owned program decks -/src/ui/tui/decks/error-tracking-upload-source-maps/ @PostHog/team-error-tracking -/src/ui/tui/decks/revenue-analytics/ @PostHog/team-web-analytics -/src/ui/tui/decks/self-driving/ @PostHog/team-self-driving -/src/ui/tui/decks/warehouse-source/ @PostHog/team-warehouse-sources +# Team-owned program UI (flows, screens, decks) +/src/tui/programs/ai-observability/ @PostHog/team-ai-observability +/src/tui/programs/error-tracking-upload-source-maps/ @PostHog/team-error-tracking +/src/tui/programs/metrics/ @PostHog/apm +/src/tui/programs/posthog-integration/ @PostHog/team-wizard-docs +/src/tui/programs/revenue-analytics/ @PostHog/team-web-analytics +/src/tui/programs/self-driving/ @PostHog/team-self-driving +/src/tui/programs/warehouse-source/ @PostHog/team-warehouse-sources +/src/tui/programs/web-analytics-doctor/ @PostHog/team-web-analytics + +# Team-owned custom commands +/src/cli/commands/self-driving.ts @PostHog/team-self-driving # Agent runner (harness, model, and tools) /src/agent/ @PostHog/team-wizard-docs diff --git a/AGENTS.md b/AGENTS.md index 0ecf21c7c..a603e3b7f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,17 +27,68 @@ Each domain has a dedicated boundary: the scanner through SDK hooks and Pi tool events; see [security boundaries](.claude/skills/wizard-development/references/ARCHITECTURE.md#security-boundaries). Gateway admission and required safety prompts live in - [ai-gateway](https://github.com/PostHog/ai-gateway). To disable scanning in - the field without a release, see the kill-switch runbook: - `docs/runbooks/warlock-kill-switch.md`. ONLY USE THIS IF ABSOLUTELY NECESSARY. -- **Agent** → `src/agent/`, imported only through `@agent` (values) and - `@agent/types` (types); see [src/agent/README.md](src/agent/README.md) -- **Shared** → `src/shared/`, stateless library code with no upward imports; - see [src/shared/README.md](src/shared/README.md) -- **Programs** → program configs and `runProgram` in `src/programs/`; see - [src/programs/README.md](src/programs/README.md) and the + [ai-gateway](https://github.com/PostHog/ai-gateway). Scanning has no remote + off switch; the [Warlock runbook](docs/runbooks/warlock-kill-switch.md) covers + what to do when it misbehaves. +- **Agent** → `src/agent/`. Layer source outside it imports it only through + `@agent` (values) and `@agent/types` (types); see + [src/agent/README.md](src/agent/README.md) +- **Shared** → `src/shared/`, library code with no upward imports. It also holds + process-global state: the log file, the analytics client, the exit cleanup + list and the process's one OAuth login session (`@shared/oauth-session`); see + [src/shared/README.md](src/shared/README.md) +- **Host** → `src/host/` (`@host/*`), how the hosts end a run: `startHostExit`, + `wizardAbort` and `registerShutdown`, and the control API server each host + attaches (`@host/control`). Every `wizardAbort` call passes the presenter that + shows its outro. A host resolves an exit code and only the CLI calls + `process.exit`. It imports env and shared only. Headless, the TUI, the CLI and + the e2e harness may import it. The agent and the programs may not; see + [src/host/README.md](src/host/README.md) +- **Programs** → one folder per program in `src/programs//`, which other + layer source enters only through its `index.ts`, plus `runProgram`. A program + is an agent run; see [src/programs/README.md](src/programs/README.md) and the [developer interfaces](docs/developer-interfaces.md) -- **TUI** → screens, primitives and content decks in `src/ui/tui/` +- **Tools** → `src/tools/` (`@tools`, one entry), the commands that run no + agent: `mcp add`, `mcp remove`, `mcp tutorial`, `slack`, `doctor`, + `provision`, `cli add` and `skill list`. A tool may import env, shared and the + `@agent` entry (the MCP tutorial's prompt stream only), never the programs, + host, TUI, headless or CLI, and no program imports it. Its screens live in + `src/tui/tools//`, run by `runTuiTool`; every tool runner resolves an exit + code. See [src/tools/README.md](src/tools/README.md) +- **TUI** → the interactive host (`runTui`), screens, primitives and the screen + store in `src/tui/`, which other layers enter only through its one entry, + `@tui` (`src/tui/index.ts`); see [src/tui/README.md](src/tui/README.md). Each + program's flow, deck, screens and control commits live in + `src/tui/programs//`, and each tool's in `src/tui/tools//`, entered + through its `index` module; the TUI core reaches them only through the TUI + program and tool registries. A program's flow uses a step a tool also shows, + such as Connect Slack, by its core screen id +- **Headless** → the host with no screens (`runHeadless`), `LoggingUI` and + controlled headless's hooks in `src/headless/`, which other layers enter only + through its one entry, `@headless` (`src/headless/index.ts`); see + [src/headless/README.md](src/headless/README.md). The two hosts share only the + kind of session store they build, `SessionStore` in `@programs`, and the host + layer; neither loads the other's code, and each calls `runProgram`; see + [what calls what](README.md#what-calls-what) +- **CLI** → commands, runners and the choice of host in `src/cli/`, the only + layer that imports both the TUI and headless, each through its entry. `bin.ts` + imports the CLI through its entry, `@cli` (`src/cli/index.ts`); see + [src/cli/README.md](src/cli/README.md) +- **Layers** → one tsconfig project per layer folder, on TypeScript project + references, and `pnpm typecheck` is `tsc -b`. A project's `references` name + the layers it may import, so importing a file of any other fails with TS6307. + Outside the TUI, `ink`, `react`, `@inkjs/ui` and `ink-testing-library` resolve + to a fence that fails every import form. ESLint rules in `pnpm lint` close the + paths the compiler can't see: a relative import that leaves its layer's folder + and a deep alias outside its own layer; a `.tsbuild/` or `node_modules/` path + in an import, re-export, `import()` or `typeof import()`; any import of + `module` or `node:module`; a bare `require`; triple-slash `path` references; + and string-form `vi.mock`; see + [layer boundaries](.claude/skills/wizard-development/references/ARCHITECTURE.md#layer-boundaries) + and [import aliases](README.md#import-aliases). Code outside a layer imports + it only through its public entries. Each layer's tests sit in its project, so + a test deep-imports only its own layer and mocks another through its entry, + `vi.mock(import(''))` Adding a new concern means finding the narrowest existing surface, not adding logic to the runner. Keep changes local to the boundary that owns them. @@ -75,27 +126,18 @@ execution remains useful for very simple tasks and legacy support. The Anthropic Agent SDK is a supported legacy fallback, deprecated as the default; retain it for major Pi vulnerabilities or gaps in support for new Anthropic models. -This is the contribution policy, not a claim that every existing binding has -migrated: `DEFAULT_BINDING` is Pi + linear. Set new bindings -explicitly and check sequence-specific hooks before migrating existing flows. -See +This is the contribution policy; existing programs' bindings vary, and +`DEFAULT_BINDING` is Pi + linear. Set new bindings explicitly and check +sequence-specific hooks before migrating existing flows. See [execution policy and model admission](.claude/skills/wizard-development/SKILL.md#execution-policy-and-model-admission) for the gateway allowlists, required system prompt, and composition constraints. ## CLI command surface -The CLI was overhauled to a smaller, extensible command surface. **Use the new -command names.** Old names mostly no longer exist — only some are kept as -aliases. - -| Old command | New command | Status | -| -------------------------- | --------------------------- | ---------------------------------------------------------- | -| `wizard integrate` | `wizard` (default flow) | command removed | -| `wizard events-audit` | `wizard audit events` | moved into `audit` family | -| `wizard audit` (single) | `wizard audit ` | now a family — see [Audit subcommands](#audit-subcommands) | -| `wizard audit-3000` | _removed_ | retired | -| `wizard revenue` | `wizard revenue-analytics` | renamed (old `revenue` removed) | -| `wizard upload-sourcemaps` | `wizard upload-source-maps` | renamed; `upload-sourcemaps` kept as alias | +`wizard` with no command runs the default flow, the `posthog-integration` +program. `wizard --help` lists the other commands. `audit` is a family: see +[Audit subcommands](#audit-subcommands). `upload-sourcemaps` is an alias of +`upload-source-maps`. ### Audit subcommands @@ -103,8 +145,8 @@ aliases. | Subcommand | What it audits | | ----------------------------- | ---------------------------------------------------- | -| `wizard audit events` | event capture quality + cost | -| `wizard audit all` | comprehensive audit across every area (**default**) | +| `wizard audit events` | event capture quality + cost | +| `wizard audit all` | comprehensive audit across every area (**default**) | | `wizard audit autocapture` | autocapture setup + cost | | `wizard audit feature-flags` | feature flag usage + cost | | `wizard audit identify` | `$identify` implementation | @@ -116,7 +158,7 @@ aliases. A skill and a command are the **same machinery** — a context-mill skill becomes a command when its `cli:` block sets `role: command`. So `wizard audit events` _is_ the `audit-events` skill, just promoted. `wizard skill ` -([`skill.ts`](src/commands/skill.ts)) runs a skill that **wasn't** promoted. +([`skill.ts`](src/cli/commands/skill.ts)) runs a skill that **wasn't** promoted. Two surfaces, one mechanism. So `wizard audit ` is choosing an audit area — it is **not** asking for a skill name, despite `wizard audit --help` @@ -125,15 +167,23 @@ confuse it with the top-level `wizard skill` command. ### Where the surface is defined (source of truth) -- **Registration:** [`bin.ts`](bin.ts) — the `.use()` chain wires each command. -- **Command shape:** [`src/commands/command.ts`](src/commands/command.ts) — the - `Command` interface every command implements. +- **Registration:** [`src/cli/commands/index.ts`](src/cli/commands/index.ts) — + `wizardCommands()` lists every top-level command in `PROGRAM_REGISTRY` order, + and `runCli` in [`src/cli/index.ts`](src/cli/index.ts), which + [`bin.ts`](bin.ts) calls, registers that list. A program with a top-level + `command` gets its command built from its config; the custom commands are + listed there by hand: the default flow, the families (`audit`, `self-driving`, + `skill`) and the tools' commands (`mcp`, `cli`, `doctor`, `slack`). +- **Command shape:** + [`src/cli/commands/command.ts`](src/cli/commands/command.ts) — the `Command` + interface every command implements. - **Flat native commands** (e.g. `revenue-analytics`, `upload-source-maps`) are built with `nativeCommandFactory` - ([`src/commands/factories/native-command-factory.ts`](src/commands/factories/native-command-factory.ts)). + ([`src/cli/commands/factories/native-command-factory.ts`](src/cli/commands/factories/native-command-factory.ts)) + from the program's config. They have no command file. - **Family commands** (e.g. `audit`) resolve subcommands at runtime against the `cliEntries` in `skill-menu.json`. Logic lives in - [`src/programs/dispatch-family.ts`](src/programs/dispatch-family.ts). + [`src/cli/commands/dispatch-family.ts`](src/cli/commands/dispatch-family.ts). Adding a skill-backed subcommand is a **context-mill** release, not a wizard change. @@ -149,11 +199,13 @@ confuse it with the top-level `wizard skill` command. ### Adding a command alias (keep an old name working) -Give the `Command.name` an array of `[newName, ...legacyNames]`. yargs treats -the extra entries as aliases. See -[`src/commands/upload-sourcemaps.ts`](src/commands/upload-sourcemaps.ts). -Reserve aliases for names that external callers (users' scripts) may still use — -when the only caller is one we control, update the caller instead. +For a command built from a program config, list the old names in the config's +`commandAliases`, as +[`error-tracking-upload-source-maps`](src/programs/error-tracking-upload-source-maps/index.ts) +does. For a custom command, give `Command.name` an array of +`[newName, ...legacyNames]`. yargs treats the extra entries as aliases. Reserve +aliases for names that external callers (users' scripts) may still use — when +the only caller is one we control, update the caller instead. ## Commands @@ -165,6 +217,7 @@ pnpm test # Unit tests (builds first) pnpm test:watch # Unit tests in watch mode pnpm test:e2e # End-to-end tests pnpm lint # Prettier + ESLint checks +pnpm typecheck # tsc -b, one project per layer pnpm fix # Auto-fix lint issues pnpm dev # Build, link globally, watch for changes ``` @@ -176,8 +229,8 @@ nonmutating lint checks, and scope formatting fixes to edited files. Do not add tests for prose, compiler-enforced shapes, or duplicated implementation. Keep new code comments to one line; put longer explanations in linked docs. -Local `--ci`, smoke-test, and full headless runs require two separate secrets: -a PostHog personal API key and an already-issued gateway token supplied through +Local `--ci`, smoke-test, and full headless runs require two separate secrets: a +PostHog personal API key and an already-issued gateway token supplied through `WIZARD_CI_GATEWAY_TOKEN_FILE`, plus the target project ID. Follow the [credential setup](docs/local-dev.md#credentials-for-local-ci-and-headless-runs). @@ -186,8 +239,9 @@ a PostHog personal API key and an already-issued gateway token supplied through Four things can independently point at a local server — the wizard binary, context-mill (`:8765`), the MCP server (`:8787`), and PostHog (`:8010`). One flag per service (`--local-context-mill`, `--local-mcp`, `--local-posthog`), -plus `--local-dev` for all three. They're dev-build-only; published builds -reject them. +plus `--local-dev` for all three. Only non-production builds accept them, such +as `pnpm try` and `pnpm build:ci`; production builds, including the published +package and the `pnpm dev` link, reject them. Note `wizard mcp add --local` is **not** one of these — it writes a `posthog-local` entry into your editor's MCP config, and is unrelated to where a @@ -201,18 +255,30 @@ wizard run points. Full catalog: [`docs/local-dev.md`](docs/local-dev.md). - TypeScript everywhere. Use `type` (not `interface`) for framework context types so they satisfy `Record`. -- All UI calls go through `getUI()` (returns `WizardUI` interface). Never import - the store directly from business logic. A program's `run` and `ciPreRun` - use the runner context they receive, not `getUI()`. -- Shared helpers never call `getUI()`; they take a sink or return data. `debug()` - reaches the UI through the sink `src/ui/index.ts` installs. -- Outside `src/agent`, import the agent through `@agent` or `@agent/types`. Add - to those entry modules rather than deep-importing; lint and - `pnpm test:arch` reject `@agent/*` paths elsewhere. -- Session mutations go through explicit store setters that call `emitChange()`. - Never mutate `session` directly — nanostore holds a shallow copy. -- The router resolves the active screen from session state. No imperative - navigation (`goTo`, `navigate`, `push`) anywhere. +- Programs never use the UI. A program's `run` and `ciPreRun` get a runner + context, and `onReady` gets a ready context. TUI code reports through the + store it is handed, headless through its `LoggingUI`, and the CLI through + `consoleLog`; there is no UI interface shared between hosts, and nothing looks + a UI up. Never import the store directly from business logic. +- Shared and host helpers take a sink or return data; there are no + process-global sinks. An agent run's debug lines are info log progress, shown + by the host's progress handler. An abort takes its presenter as its first + argument: `abortOnScreens(store)` in the TUI, `printAbortOutro` in headless + and before the TUI mounts. +- Outside `src/agent`, import the agent through `@agent` or `@agent/types`, and + every other layer through its entries. Inside the agent and the programs, + import your own modules by relative path. An entry re-exports only from layers + its consumers also map, so no declaration turns to `any` in a consumer. Add to + an entry rather than deep-importing; `pnpm typecheck` and `pnpm lint` reject a + deeper path in layer source, tests, the e2e harness, scripts and docs examples. A test of + another layer's internals belongs in that layer's tests, and a test helper + beside the tests that use it. +- Session mutations go through explicit store setters, which notify the store's + subscribers. Never mutate `session` directly — nanostore holds a shallow copy. +- State only the TUI reads (a screen's answer, what an overlay shows) lives in + the TUI store (`TuiState`, read as `store.X`), never in the shared session. +- The router resolves the active screen from the session and the TUI state. No + imperative navigation (`goTo`, `navigate`, `push`) anywhere. - Never write secrets to source code or hardcode API keys. Use the `wizard-tools` MCP server (`check_env_keys` / `set_env_values`) for `.env` file operations. diff --git a/README.md b/README.md index f08942c45..9db87427b 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,8 @@ posthoglogo

- -> have any feedback, please drop an email to **[wizard@posthog.com](mailto:wizard@posthog.com)**. +> have any feedback, please drop an email to +> **[wizard@posthog.com](mailto:wizard@posthog.com)**.

PostHog wizard ✨

@@ -19,22 +19,36 @@ To use the wizard, you can run it directly using: npx @posthog/wizard@latest ``` -Currently the wizard can be used for over 16+ frameworks for frontend, backend, and mobile applications. If you have other integrations you would like the wizard to -support, please open a [GitHub issue](https://github.com/posthog/wizard/issues)! +Currently the wizard can be used for over 16+ frameworks for frontend, backend, +and mobile applications. If you have other integrations you would like the +wizard to support, please open a +[GitHub issue](https://github.com/posthog/wizard/issues)! -Visit our [docs](https://posthog.com/docs/ai-engineering/ai-wizard) to learn more. +Visit our [docs](https://posthog.com/docs/ai-engineering/ai-wizard) to learn +more. ## Privacy & data usage -The wizard uses **AI models from Anthropic or OpenAI**, routed through PostHog's AI gateway, to read your project's source files and integrate PostHog. A few things worth knowing up front: - -- **Source files** are sent to the selected model provider as part of the agent's context. -- **`.env*` files and secrets** stay on your machine. The wizard's security scanner blocks anything it identifies as a secret from being read by the agent. -- **Telemetry** (run metadata — phase, task list, planned events) is sent to PostHog by default. Pass `--no-telemetry` (or set `POSTHOG_WIZARD_NO_TELEMETRY=1`) to disable. -- **AI opt-in**: for existing organizations in interactive runs, the wizard checks `is_ai_data_processing_approved` and waits for approval before agent work. CI and signup runs bypass this interactive gate. -- **Prefer your own AI?** The wizard's integration knowledge ships as a context-mill skill you can download and run inside your own agent. - -The wizard's "Privacy & data usage" menu (intro screen) and the `[I]` shortcut on the auth screen surface the same information in-terminal. +The wizard uses **AI models from Anthropic or OpenAI**, routed through PostHog's +AI gateway, to read your project's source files and integrate PostHog. A few +things worth knowing up front: + +- **Source files** are sent to the selected model provider as part of the + agent's context. +- **`.env*` files and secrets** stay on your machine. The wizard's security + scanner blocks anything it identifies as a secret from being read by the + agent. +- **Telemetry** (run metadata — phase, task list, planned events) is sent to + PostHog by default. Pass `--no-telemetry` (or set + `POSTHOG_WIZARD_TELEMETRY=false`) to disable. +- **AI opt-in**: for existing organizations in interactive runs, the wizard + checks `is_ai_data_processing_approved` and waits for approval before agent + work. CI and signup runs bypass this interactive gate. +- **Prefer your own AI?** The wizard's integration knowledge ships as a + context-mill skill you can download and run inside your own agent. + +The wizard's "Privacy & data" menu (intro screen) and the `[I]` shortcut on the +auth screen surface the same information in-terminal. ## MCP Commands @@ -47,45 +61,62 @@ npx @posthog/wizard@latest mcp add # Remove PostHog MCP server from supported clients npx @posthog/wizard@latest mcp remove + +# Try the PostHog MCP with your agent, no install needed +npx @posthog/wizard@latest mcp tutorial ``` +These, `slack`, `doctor`, `provision`, `cli add` and `skill list` are tools: +commands that do their job without an agent run. See +[src/tools](src/tools/README.md). + ## Wizard programs -The wizard's commands are grouped into **programs** — self-contained agentic jobs that install, audit, or wire up a specific piece of PostHog. They're powered by skills from the [context mill](https://github.com/PostHog/context-mill). +The wizard's commands are grouped into **programs** — self-contained agentic +jobs that install, audit, or wire up a specific piece of PostHog. They're +powered by skills from the +[context mill](https://github.com/PostHog/context-mill). ### PostHog integration (default) -Running the wizard with no arguments installs PostHog into your project. It detects your framework, wires up initialization, instruments a starter set of events, and walks you through a first dashboard: +Running the wizard with no arguments installs PostHog into your project. It +detects your framework, wires up initialization, instruments a starter set of +events, and walks you through a first dashboard: ```bash npx @posthog/wizard@latest ``` -Powered by the `posthog-integration` program. Most other programs below build on it (they declare `requires: ['posthog-integration']`) and will offer to run it first if PostHog isn't already set up. +Powered by the `posthog-integration` program. Most other programs below build on +it and declare `requires: ['posthog-integration']`. ### Self-driving -Autonomously sets up PostHog self-driving end-to-end. It connects GitHub, enables Session Replay and Error Tracking, wires up signal sources, and configures a Signals scout troop that watches your project for you. +Autonomously sets up PostHog self-driving end-to-end. It connects GitHub, +enables Session Replay and Error Tracking, wires up signal sources, and +configures a Signals scout troop that watches your project for you. ```bash npx @posthog/wizard@latest self-driving ``` -If PostHog isn't already installed, the wizard runs the default integration first (composed run) before starting the self-driving setup. +If PostHog isn't already installed, the wizard runs the default integration +first (composed run) before starting the self-driving setup. ### Audit Audit an existing PostHog integration for correctness and best practices. The -`audit` command is a **family**. With no subcommand it runs the **events** -audit (the default); pass a subcommand to run a specific one: +`audit` command is a **family**. With no subcommand, an interactive terminal +runs the audit the skill menu marks as the default (today **all**); pass a +subcommand to run a specific one: ```bash -# Runs the events audit (the default) — no subcommand needed +# Runs the default audit — no subcommand needed npx @posthog/wizard@latest audit # Or run a specific audit directly -npx @posthog/wizard@latest audit events # event capture quality + cost (default) -npx @posthog/wizard@latest audit all # comprehensive audit across every area +npx @posthog/wizard@latest audit events # event capture quality + cost +npx @posthog/wizard@latest audit all # comprehensive audit across every area (default) npx @posthog/wizard@latest audit autocapture # autocapture setup + cost npx @posthog/wizard@latest audit feature-flags # feature flag usage + cost npx @posthog/wizard@latest audit identify # your $identify implementation @@ -96,12 +127,12 @@ npx @posthog/wizard@latest audit web-analytics # web analytics setup Most audit subcommands resolve at runtime from the published skill registry, so new audits appear without a wizard release (`web-analytics` is wizard-native). -> **`audit ` chooses an audit area — it does not take a skill name.** -> The audit subcommands above *are* context-mill skills promoted to commands (via -> a `cli: role: command` block); [`wizard skill `](#run-a-single-skill) -> runs a skill that hasn't been promoted. Same machinery, two surfaces. -> (`wizard audit --help` still labels the positional `[skill]` — read it as "pick -> a subcommand.") +> **`audit ` chooses an audit area — it does not take a skill +> name.** The audit subcommands above _are_ context-mill skills promoted to +> commands (via a `cli: role: command` block); +> [`wizard skill `](#run-skill) runs a skill that hasn't been +> promoted. Same machinery, two surfaces. (`wizard audit --help` labels the +> positional `[skill]` — read it as "pick a subcommand.") ### Revenue Analytics @@ -112,7 +143,7 @@ npx @posthog/wizard@latest revenue-analytics ``` Requires PostHog and Stripe SDKs already installed. Supports `--ci` with the -same flags as the main wizard. (Renamed from `revenue` in the CLI overhaul.) +same flags as the main wizard. ### Data Warehouse @@ -129,12 +160,15 @@ OAuth sources open the PostHog app's new-source flow in your browser. ### Upload source maps -Upload JavaScript source maps to PostHog error tracking so stack traces are symbolicated back to your original code: +Upload JavaScript source maps to PostHog error tracking so stack traces are +symbolicated back to your original code: ```bash npx @posthog/wizard@latest upload-source-maps ``` +`upload-sourcemaps` also works. + ### Run skill Run any context-mill skill directly by name, even if it isn't exposed as its own @@ -149,43 +183,50 @@ npx @posthog/wizard@latest skill # run one by name Reviews are auto-requested via [`.github/CODEOWNERS`](.github/CODEOWNERS) — the file is the source of truth; this table just mirrors it for readability. -`team-wizard-docs` is the default reviewer; the team-owned paths below -route review to their owning team instead. - -| Path | Owning team | -|---|---| -| `*` (everything else, including all other programs) | `@PostHog/team-wizard-docs` | -| `src/agent/` | `@PostHog/team-wizard-docs` | -| `src/programs/ai-observability/` | `@PostHog/team-ai-observability` | -| `src/programs/posthog-integration/` | `@PostHog/team-wizard-docs` | -| `src/programs/error-tracking-upload-source-maps/` | `@PostHog/team-error-tracking` | -| `src/programs/mcp-analytics/` | `@PostHog/team-mcp-analytics` | -| `src/programs/metrics/` | `@PostHog/apm` | -| `src/programs/replay-vision/` | `@PostHog/team-replay` | -| `src/programs/revenue-analytics/` | `@PostHog/team-web-analytics` | -| `src/programs/self-driving/` | `@PostHog/team-self-driving` | -| `src/programs/warehouse-source/` | `@PostHog/team-warehouse-sources` | -| `src/programs/web-analytics-doctor/` | `@PostHog/team-web-analytics` | -| `src/ui/tui/decks/error-tracking-upload-source-maps/` | `@PostHog/team-error-tracking` | -| `src/ui/tui/decks/revenue-analytics/` | `@PostHog/team-web-analytics` | -| `src/ui/tui/decks/self-driving/` | `@PostHog/team-self-driving` | -| `src/ui/tui/decks/warehouse-source/` | `@PostHog/team-warehouse-sources` | - -Ownership is by directory. Programs not listed above -(`agent-skill`, `audit`, `events-audit`, `mcp`, `migration`, `posthog-doctor`, -`shared`, `slack`) fall through the default and are owned by -`team-wizard-docs`. Today CODEOWNERS only auto-requests review — approval is -not a merge gate. +`team-wizard-docs` is the default reviewer; the team-owned paths below route +review to their owning team instead. + +| Path | Owning team | +| ----------------------------------------------------- | --------------------------------- | +| `*` (everything else, including all other programs) | `@PostHog/team-wizard-docs` | +| `src/agent/` | `@PostHog/team-wizard-docs` | +| `src/programs/ai-observability/` | `@PostHog/team-ai-observability` | +| `src/programs/posthog-integration/` | `@PostHog/team-wizard-docs` | +| `src/programs/error-tracking-upload-source-maps/` | `@PostHog/team-error-tracking` | +| `src/programs/mcp-analytics/` | `@PostHog/team-mcp-analytics` | +| `src/programs/metrics/` | `@PostHog/apm` | +| `src/programs/replay-vision/` | `@PostHog/team-replay` | +| `src/programs/revenue-analytics/` | `@PostHog/team-web-analytics` | +| `src/programs/self-driving/` | `@PostHog/team-self-driving` | +| `src/programs/warehouse-source/` | `@PostHog/team-warehouse-sources` | +| `src/programs/web-analytics-doctor/` | `@PostHog/team-web-analytics` | +| `src/tui/programs/ai-observability/` | `@PostHog/team-ai-observability` | +| `src/tui/programs/error-tracking-upload-source-maps/` | `@PostHog/team-error-tracking` | +| `src/tui/programs/metrics/` | `@PostHog/apm` | +| `src/tui/programs/posthog-integration/` | `@PostHog/team-wizard-docs` | +| `src/tui/programs/revenue-analytics/` | `@PostHog/team-web-analytics` | +| `src/tui/programs/self-driving/` | `@PostHog/team-self-driving` | +| `src/tui/programs/warehouse-source/` | `@PostHog/team-warehouse-sources` | +| `src/tui/programs/web-analytics-doctor/` | `@PostHog/team-web-analytics` | +| `src/cli/commands/self-driving.ts` | `@PostHog/team-self-driving` | + +Ownership is by directory, except the self-driving command file. Programs not +listed above (`agent-skill`, `audit`, `error-tracking`, `migration`), the shared +program code and the tools (`src/tools`, `src/tui/tools`) fall through the +default and are owned by `team-wizard-docs`. Today CODEOWNERS only auto-requests +review — approval is not a merge gate. ## Headless signup + install (agents / CI) -> ⚠️ `--ci` is **not currently supported in published builds** (see [CI Mode](#ci-mode)). -> This flow works in development builds only. +> ⚠️ `--ci` is **not currently supported in published builds** (see +> [CI Mode](#ci-mode)). This flow works in development builds only. -For a fully non-interactive first-run (no existing PostHog account, no TTY, -no browser), combine `--ci --signup --email`. The wizard provisions a new -account, uses the returned personal API key to run the normal CI install, -and wires PostHog into the project at `--install-dir`: +For a fully non-interactive first-run (no existing PostHog account, no TTY, no +browser), combine `--ci --signup --email`. The wizard provisions a new account, +uses the returned personal API key to run the normal CI install, and wires +PostHog into the project at `--install-dir`. Like every `--ci` run, it needs the +gateway token file from +[local credentials](docs/local-dev.md#credentials-for-local-ci-and-headless-runs): ```bash npx @posthog/wizard@latest --ci --signup \ @@ -193,8 +234,7 @@ npx @posthog/wizard@latest --ci --signup \ --install-dir . ``` -Optional flags: `--name "Your Name"`, `--region eu` (default `us`), -`--integration nextjs` (else auto-detected). +Optional flags: `--name "Your Name"`, `--region eu` (default `us`). ### Provision only @@ -211,38 +251,39 @@ npx @posthog/wizard@latest provision --email user@example.com --region eu --json ``` Success prints the full `ProvisioningResult` (`projectApiKey`, `host`, -`projectId`, `accountId`, `accessToken`, `refreshToken`, and -`personalApiKey` if present). Failure exits 1; in `--json` mode the error -is emitted to stderr as `{"error":"...","code":"..."}`, with `code` set to -`email_exists` when the address is already registered. +`projectId`, `accountId`, `accessToken`, `refreshToken`, and `personalApiKey` if +present). Failure exits 1; in `--json` mode the error is emitted to stderr as +`{"error":"...","code":"..."}`, with `code` set to `email_exists` when the +address is already registered. -> ⚠️ **Output contains live credentials.** Pipe it into a secrets store — -> do not let it be captured by shared CI logs. Mask the step output or -> redirect stdout to a file your job reads and discards. +> ⚠️ **Output contains live credentials.** Pipe it into a secrets store — do not +> let it be captured by shared CI logs. Mask the step output or redirect stdout +> to a file your job reads and discards. # Options The following CLI arguments are available: -| Option | Description | Type | Default | Choices | Environment Variable | -| ----------------- | ---------------------------------------------------------------- | ------- | ------- | ---------------------------------------------------- | ------------------------------ | -| `--help` | Show help | boolean | | | | -| `--version` | Show version number | boolean | | | | -| `--debug` | Enable verbose logging | boolean | `false` | | `POSTHOG_WIZARD_DEBUG` | -| `--signup` | Create a new PostHog account during setup | boolean | `false` | | `POSTHOG_WIZARD_SIGNUP` | -| `--install-dir` | Directory to install PostHog in | string | | | `POSTHOG_WIZARD_INSTALL_DIR` | -| `--ci` | Enable CI mode for non-interactive execution | boolean | `false` | | `POSTHOG_WIZARD_CI` | -| `--api-key` | PostHog personal API key (phx_xxx) for authentication | string | | | `POSTHOG_WIZARD_API_KEY` | -| `--no-telemetry` | Disable wizard run-state telemetry | boolean | `false` | | `POSTHOG_WIZARD_NO_TELEMETRY` | - +| Option | Description | Type | Default | Choices | Environment Variable | +| --------------- | ----------------------------------------------------------------------- | ------- | ------- | ------- | ---------------------------- | +| `--help` | Show help | boolean | | | | +| `--version` | Show version number | boolean | | | | +| `--debug` | Enable verbose logging | boolean | `false` | | `POSTHOG_WIZARD_DEBUG` | +| `--signup` | Create a new PostHog account during setup | boolean | `false` | | `POSTHOG_WIZARD_SIGNUP` | +| `--install-dir` | Directory to install PostHog in | string | | | `POSTHOG_WIZARD_INSTALL_DIR` | +| `--ci` | Enable CI mode for non-interactive execution (dev and test builds only) | boolean | `false` | | `POSTHOG_WIZARD_CI` | +| `--api-key` | PostHog personal API key (phx_xxx) for authentication | string | | | `POSTHOG_WIZARD_API_KEY` | +| `--project-id` | PostHog project ID to use | string | | | `POSTHOG_WIZARD_PROJECT_ID` | +| `--email` | Email address for signup (with `--signup`) | string | | | `POSTHOG_WIZARD_EMAIL` | +| `--telemetry` | Send wizard run state to PostHog; `--no-telemetry` disables it | boolean | `true` | | `POSTHOG_WIZARD_TELEMETRY` | # CI Mode **CI mode is available only in development/test builds.** Published builds reject `--ci`; use an interactive terminal for `npx @posthog/wizard@latest`. -Local CI runs require a PostHog personal API key **and a separate gateway -token file**, plus the target project ID. See +Local CI runs require a PostHog personal API key **and a separate gateway token +file**, plus the target project ID. See [local credentials](docs/local-dev.md#credentials-for-local-ci-and-headless-runs) for setup and the CI secret names. With both secrets configured: @@ -257,15 +298,16 @@ When running in CI mode (`--ci`): - Bypasses OAuth login flow (uses personal API key directly) - Auto-selects defaults for all prompts - Skips MCP server installation -- Auto-continues on git warnings (uncommitted/untracked files) - Auto-consents to AI usage The CLI args override environment variables in CI mode. ### Required Flags for CI Mode -- `--api-key`: Personal API key (`phx_xxx`) from your [PostHog settings](https://app.posthog.com/settings/user-api-keys) -- `--install-dir`: Directory to install PostHog in (e.g., `.` for current directory) +- `--api-key`: Personal API key (`phx_xxx`) from your + [PostHog settings](https://app.posthog.com/settings/user-api-keys) +- `--install-dir`: Directory to install PostHog in (e.g., `.` for current + directory) ### Required API Key Scopes @@ -278,42 +320,41 @@ health_issue:read wizard_session:read wizard_session:write wizard_run:write ``` The source of truth is `WIZARD_OAUTH_SCOPES` in `src/shared/constants.ts`, which -documents why each scope is needed — if this block drifts, trust the code. -Some programs request more on top (`PROGRAM_SCOPE_ADDITIONS` in -`src/programs/oauth/program-scopes.ts`); the default integration flow adds -`integration:read` and `external_data_source:read` / -`external_data_source:write`. - -The `wizard-run-sync` flag selects remote synchronization: `wizard-session` -(the default) uses WizardSession; `wizard-run` uses WizardRun. In the latter -mode, local executions synchronize tasks and terminal status. Cloud executions -require an explicit `POSTHOG_WIZARD_RUN_ID` assignment and leave terminal status -to their worker. See [WizardRun synchronization](docs/local-dev.md#wizardrun-synchronization) -for limits, shutdown behavior, migration compatibility, and deployment checks. +documents why each scope is needed — if this block drifts, trust the code. Some +programs request more on top (`oauthScopeAdditions` on the program's config); +the default integration flow adds `integration:read` and +`external_data_source:read` / `external_data_source:write`. + +The `wizard-run-sync` flag selects remote synchronization: `wizard-session` (the +default) uses WizardSession; `wizard-run` uses WizardRun. In the latter mode, +local executions synchronize tasks and terminal status. Cloud executions require +an explicit `POSTHOG_WIZARD_RUN_ID` assignment and leave terminal status to +their worker. See +[WizardRun synchronization](docs/local-dev.md#wizardrun-synchronization) for +limits, shutdown behavior, migration compatibility, and deployment checks. ### OAuth app scope ceiling Both the interactive and cloud Wizard OAuth apps must allow `wizard_run:write` -in every deployed region. Existing tokens need renewed authorization; refresh -does not add the grant. Run synchronization needs no read scope. +in every deployed region. Run synchronization needs no read scope. -The wizard's OAuth app on the PostHog side caps the scopes its tokens may -carry (`OAuthApplication.scopes`). Any scope requested in this repo (see -`src/programs/oauth/program-scopes.ts`) must be grantable under that ceiling, or -`/authorize` drops it and the call that needs it 403s. +The wizard's OAuth app on the PostHog side caps the scopes its tokens may carry +(`OAuthApplication.scopes`). Any scope requested in this repo (see the programs' +`oauthScopeAdditions`) must be grantable under that ceiling, or `/authorize` +drops it and the call that needs it 403s. **A granted token can be narrower than the request even with a correct ceiling.** The consent screen lets the user deselect any scope the app doesn't mark required (`OAuthApplication.required_scopes`), and out-of-ceiling scopes are clamped silently (`clamp_scopes_to_ceiling`) — neither path errors; -`/oauth/token` just returns a smaller `scope`. So never assume the token -carries what was requested: the token response's `scope` field is the truth. -The wizard diffs granted vs requested at login (`missingOAuthScopes` in -`src/shared/utils/oauth.ts`), warns the user which permissions are missing, and emits -`wizard: oauth grant narrowed` so narrowed runs are countable in analytics. -The diff also rides on the session (`credentials.missingScopes`), so when a -run does fail on a scope-gated step, the error names the missing permission -and the fix instead of the generic report-a-bug line. +`/oauth/token` just returns a smaller `scope`. So never assume the token carries +what was requested: the token response's `scope` field is the truth. The wizard +diffs granted vs requested at login (`missingOAuthScopes` in +`src/programs/oauth/tokens.ts`), warns the user which permissions are missing, +and emits `wizard: oauth grant narrowed` so narrowed runs are countable in +analytics. The diff also rides on the session (`credentials.missingScopes`), so +when a run does fail on a scope-gated step, the error names the missing +permission and the fix instead of the generic report-a-bug line. **To make scopes impossible to deselect, list them explicitly in the app's `scopes`.** `required_scopes` is not a separate field — it is derived @@ -323,8 +364,8 @@ consent POST 400s with `invalid_scope` if the grant misses one), while scopes covered only by `@default` stay deselectable and `optional_scopes` are declinable extras. That is why `llm_gateway:read` and `wizard_session:*` are already un-deselectable today, and everything else is not. To pin the base set -the wizard cannot run without, seed each region's app with `@default` plus -every scope in `WIZARD_OAUTH_SCOPES`: +the wizard cannot run without, seed each region's app with `@default` plus every +scope in `WIZARD_OAUTH_SCOPES`: ``` python manage.py seed_oauth_app_scopes --client-id --dry-run \ @@ -332,10 +373,10 @@ python manage.py seed_oauth_app_scopes --client-id --dry-run \ ``` then re-run without `--dry-run`. Keep `@default` in the list — dropping it -narrows the ceiling to only the explicit entries and strips the -program-specific additions. The wizard has no client-side lever for any of -this; the login diff and prompt-threaded degrade above handle a narrowed -grant, but only pinning prevents one. +narrows the ceiling to only the explicit entries and strips the program-specific +additions. The wizard has no client-side lever for any of this; the login diff +and prompt-threaded degrade above handle a narrowed grant, but only pinning +prevents one. **The live wizard apps use the `@default` sentinel, so most net-new scopes need no ceiling edit.** The prod US app's `scopes` is: @@ -365,35 +406,14 @@ So the rule for a net-new scope this repo starts requesting is: admin / the `seed_oauth_app_scopes` command), per region. This is the only case that needs a manual prod edit. -Client IDs are per-region DB rows, not committed here — the prod US app is -`c4Rdw8DIxgtQfA80IiSnGKlNX8QN00cFWF00QQhM`, the dev app (localhost:8010) is -`DC5uRLVbGI02YQ82grxgnK6Qn12SXWpCqdPb60oZ`; the prod EU app's ID lives in the EU -deployment (referenced via `WIZARD_CLOUD_RUN_OAUTH_CLIENT_ID`) and should be -seeded the same `@default,…` way. +The interactive apps' client IDs are `POSTHOG_US_CLIENT_ID`, +`POSTHOG_EU_CLIENT_ID` and `POSTHOG_DEV_CLIENT_ID` (the localhost:8010 dev app) +in `src/shared/constants.ts`. Seed every region's app the same `@default,…` way. If an existing Wizard authorization predates a newly required scope, reconnect the Wizard OAuth app. Refresh tokens retain their original grant and cannot be used to silently add permissions. -# Command changes (CLI overhaul) - -The CLI was overhauled to consolidate commands into a smaller, extensible -surface. If you used an older command, here's where it went: - -| Old command | New command | What changed | -|---|---|---| -| `wizard integrate` | `wizard` (default flow) | Command removed; the default flow runs the integration | -| `wizard events-audit` | `wizard audit events` | Now an `audit`-family subcommand | -| `wizard audit` (single audit) | `wizard audit ` | Now a family; see [Audit](#audit) for the subcommands | -| `wizard audit-3000` | *removed* | Retired | -| `wizard revenue` | `wizard revenue-analytics` | Renamed (old `revenue` removed) | -| `wizard upload-sourcemaps` | `wizard upload-source-maps` | Renamed; `upload-sourcemaps` still works as an alias | - -> **Commands vs. programs:** `integrate` was the *command*; the program behind it -> is `posthog-integration`, which still exists and now powers the default flow. -> Other commands depend on it via `requires: ['posthog-integration']`. The -> program id is internal — it was never a command you typed. - # Steal this code While the wizard works great on its own, we also find the approach used by this @@ -406,16 +426,108 @@ that conventional code implies. If you want to use this code as a starting place for your own project, here's a quick explainer on its structure. -## Entrypoint: `run.ts` +## What calls what + +```mermaid +%%{init: {"block": {"padding": 20}}}%% +block-beta + columns 18 + user["a user at a terminal"]:3 ci["CI or a sandbox"]:3 embedder["your program"]:3 space:3 toolUser["a user or a script"]:3 caller["your code or a test"]:3 + cliBand["src/cli picks a host, owns the signals and the exit"]:6 space:12 + runWizard["runWizard"]:3 runWizardCI["runWizardCI
runWizardHeadless"]:3 space:6 toolCmd["src/cli
a tool's command"]:3 space:3 + hostBand["src/tui · src/headless the two hosts"]:6 space:12 + runTui["runTui"]:3 runHeadless["runHeadless"]:3 space:6 runTool["runTuiTool
or an @tools runner"]:3 space:3 + wizardStore["WizardStore
display over the store"]:3 space:15 + programsBand["@programs"]:12 space:6 + tuiStore["SessionStore"]:3 headlessStore["SessionStore"]:3 ownStore["SessionStore"]:3 detect["detectProjectsWithAgent"]:3 space:6 + tuiRun["runProgram"]:3 headlessRun["runProgram"]:3 ownRun["runProgram"]:3 space:9 + agentBand["@agent"]:18 + tuiAgent["runAgent"]:3 headlessAgent["runAgent"]:3 ownAgent["runAgent"]:3 detectAgent["runAgent"]:3 mcpStream["streamMcpPrompt"]:3 callerAgent["runAgent"]:3 + + user --> runWizard + runWizard --> runTui + runTui --> wizardStore + wizardStore --> tuiStore + tuiStore --> tuiRun + tuiRun --> tuiAgent + + ci --> runWizardCI + runWizardCI --> runHeadless + runHeadless --> headlessStore + headlessStore --> headlessRun + headlessRun --> headlessAgent + + embedder -- "builds and owns the store" --> ownStore + ownStore --> ownRun + ownRun --> ownAgent + + detect --> detectAgent + + toolUser --> toolCmd + toolCmd --> runTool + runTool -- "the MCP tutorial only" --> mcpStream + + caller -- "runAgent(config, input)" --> callerAgent + + classDef code fill:#3b82f626,stroke:#3b82f6,stroke-width:2px + classDef ext fill:none,stroke:#9ca3af,stroke-dasharray:4 3 + classDef band fill:#3b82f640,stroke:none + class runWizard,runWizardCI,runTui,runHeadless,wizardStore,tuiStore,headlessStore,ownStore,tuiRun,headlessRun,ownRun,tuiAgent,headlessAgent,ownAgent,detect,detectAgent,toolCmd,runTool,mcpStream,callerAgent code + class user,ci,embedder,toolUser,caller ext + class cliBand,hostBand,programsBand,agentBand band +``` -The entrypoint for this tool is `run.ts`. Use this file to interpret arguments -and set up the general flow of the application. +Each column is one way in, read top to bottom. A box that repeats across columns +is the same code. The CLI calls one of two hosts, the TUI or headless. They +share only the kind of session store they build: each builds its own, calls +`runProgram` with it, and `runProgram` writes the run into it. The TUI draws its +screens from the `WizardStore` on top of its store, and headless prints and +streams the run. + +A tool's command runs no program. The CLI calls the TUI's `runTuiTool` for the +tool's screens, or a console runner from `@tools`. Only the MCP tutorial reaches +the agent, through `streamMcpPrompt`. The TUI's intro can also hand off to a +tool's screens, such as `doctor`, in the same process. See +[src/tools](src/tools/README.md). + +Detection calls `runAgent` directly. `detectProjectsWithAgent` builds its own +run config for a project scan. It runs from a program's `ciPreRun`, which +`runProgram` calls as a CI run's detection, and from the TUI's detect screens. + +Dashed boxes are callers that build against a checkout of this repository, not +the npm package. Your program can build a store and call +`runProgram(id, { store })` itself. Your code can call `runAgent` with no +program at all. + +| Box | What it is | Where | +| --------------------------------- | ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | +| `runWizard` | Builds the launch values from the arguments and starts the TUI host | [src/cli/runners](src/cli/runners/run-wizard.ts) | +| `runWizardCI · runWizardHeadless` | Checks the non-interactive arguments and starts the headless host | [src/cli/runners](src/cli/runners/run-non-interactive.ts) | +| `a tool's command` | Parses the tool's arguments and calls its runner | [src/cli/commands](src/cli/commands) | +| `runTui` | The full-screen wizard: OAuth login, the WizardAsk screen as the answerer, the flow's screens as the workflow | [src/tui/run.ts](src/tui/run.ts) | +| `runHeadless` | A run with no screens: API-key login, no answerer unless a control socket asks, log lines and the task stream | [src/headless/run.ts](src/headless/run.ts) | +| `runTuiTool` | Shows a tool's screens on their own store and resolves the code a screen exits with; it runs no program | [src/tui/run-tool.ts](src/tui/run-tool.ts) | +| `@tools` runner | A tool that runs on the console, such as `addMcpServer`, `runDoctorReport`, `runProvision` or `listSkills` | [src/tools](src/tools/README.md) | +| `WizardStore` | The TUI's own state (the screens' answers), router, gates, overlays and display state, over its session store | [src/tui/store.ts](src/tui/store.ts) | +| `SessionStore` | Launch values, detection, the login and the run's progress; each host builds its own | [src/programs/session](src/programs/session) | +| `runProgram` | Runs a registered program: detection, readiness, login, the host's steps, then each agent run | [src/programs/run-program.ts](src/programs/run-program.ts) | +| `detectProjectsWithAgent` | A project scan: one `runAgent` call per attempt, with its own route and deadline | [src/programs/detection/agentic.ts](src/programs/detection/agentic.ts) | +| `runAgent` | One agent run from a `RunConfig` and `RunInput`; complete on its own | [src/agent](src/agent/README.md) | +| `streamMcpPrompt` | The MCP tutorial's prompt stream: one prompt against the PostHog MCP server, with its own SDK call | [src/agent/mcp-prompt-streaming.ts](src/agent/mcp-prompt-streaming.ts) | + +## Entrypoint: `bin.ts` + +The entrypoint for this tool is `bin.ts`. It checks the Node version, restores +any Claude settings backup an interrupted run left behind, and calls `runCli()` +from `src/cli/index.ts`. `runCli` registers every command from +`wizardCommands()` on the yargs setup in `src/cli/wizard.ts` and runs the one +that `process.argv` names. ## Analytics Did you know you can capture PostHog events even for smaller, supporting -products like a command line tool? `src/shared/utils/analytics.ts` is a great example -of how to do it. +products like a command line tool? `src/shared/utils/analytics.ts` is a great +example of how to do it. This file wraps `posthog-node` with some convenience functions to set up an analytics session and log events. We can see the usage and outcomes of this @@ -423,20 +535,13 @@ wizard alongside all of our other PostHog product data, and this is very powerful. For example: we could show in-product surveys to people who have used the wizard to improve the experience. -With `wizard-run-sync=wizard-session`, the wizard streams live run state — current -phase, task list, planned events — to `POST /api/projects/{id}/wizard/sessions/` -so the PostHog web app can render real-time progress. Updates are debounced -(250ms) with phase changes flushed immediately; failures fall back silently to -the wizard's debug log without disturbing the TUI. Pass `--no-telemetry` (or -set `POSTHOG_WIZARD_NO_TELEMETRY=1`) to disable either remote transport. - -## Leave rules behind - -Supporting agent sessions after we leave is important. There are plenty of ways -to break or misconfigure PostHog, so guarding against this is key. - -`src/shared/utils/rules/add-editor-rules.ts` demonstrates how to dynamically construct -rules files and store them in the project's `.cursor/rules` directory. +With `wizard-run-sync=wizard-session`, the wizard streams live run state — +current phase, task list, planned events — to +`POST /api/projects/{id}/wizard/sessions/` so the PostHog web app can render +real-time progress. Updates are debounced (250ms) with phase changes flushed +immediately; failures fall back silently to the wizard's debug log without +disturbing the TUI. Pass `--no-telemetry` (or set +`POSTHOG_WIZARD_TELEMETRY=false`) to disable either remote transport. ## Prompts and LLM interactions @@ -447,14 +552,15 @@ existing code at scale and then modify it reliably. _If_ they are well prompted. -`src/lib/prompts.ts` demonstrates how to wrap a deterministic fence around a -chaotic process. Every wizard session gets the same prompt, tailored to the -specific files in the project. +`src/agent/agent-prompt.ts` demonstrates how to wrap a deterministic fence +around a chaotic process. Every wizard session gets the same prompt, tailored to +the specific files in the project. -These prompts are channeled using `src/shared/utils/query.ts` to an LLM interface we -host. This gives us more control: we can be certain of the model version and -provider which interpret the prompts and modify the files. This way, we can find -the right tools for the job and again, apply them consistently. +These prompts go to the PostHog LLM gateway, an LLM interface we host, with a +scoped token that `src/agent/gateway-session.ts` mints for each run. This gives +us more control: we can be certain of the model version and provider which +interpret the prompts and modify the files. This way, we can find the right +tools for the job and again, apply them consistently. This also allows us to pick up the bill on behalf of our customers. @@ -463,13 +569,12 @@ users of the wizard, no training delays or other ambiguity. ## Keep secrets out of the LLM -The wizard somtimes needs to move a secret. The agent -orchestrates that journey, but the raw value should _never_ enter the LLM -conversation, where it would be sent to the model provider, written to -transcripts, and captured in logs. +The wizard somtimes needs to move a secret. The agent orchestrates that journey, +but the raw value should _never_ enter the LLM conversation, where it would be +sent to the model provider, written to transcripts, and captured in logs. -`src/shared/secret-vault.ts` is a small, reusable pattern for exactly this. It's a -session-scoped, in-memory vault: a tool that handles a secret calls `put()` to +`src/shared/secret-vault.ts` is a small, reusable pattern for exactly this. It's +a session-scoped, in-memory vault: a tool that handles a secret calls `put()` to store the raw value and hands the agent an opaque `secret:` reference instead. The agent passes that ref between tools as if it were the value; the host resolves it back to the real secret only at the last moment, inside the @@ -489,54 +594,93 @@ drive the work end to end, but the only thing it ever sees is an opaque handle. ## Build system -Built with [tsdown](https://tsdown.dev/) (Rolldown). `pnpm build` bundles `bin.ts` into ESM chunks in `dist/`, inlining all local source and keeping npm dependencies external. +Built with [tsdown](https://tsdown.dev/) (Rolldown). `pnpm build` bundles +`bin.ts` into ESM chunks in `dist/`, inlining all local source and keeping npm +dependencies external. ### Environment variables -**Build-time (locked).** `NODE_ENV` is replaced with `"production"` at compile time. It cannot be overridden at runtime. All URLs, OAuth client IDs, and dev-mode code paths resolve to their production values unconditionally. - -To add a new build-time constant, add it to `env` in `tsdown.config.ts` and export it from `src/env.ts`. - -**Runtime (allowlisted).** Runtime env reads go through `runtimeEnv()` in `src/env.ts`, which only accepts keys in the `RuntimeEnvKey` union: - -| Variable | Purpose | -|---|---| -| `POSTHOG_WIZARD_BENCHMARK_CONFIG` | Path to benchmark config file | -| `POSTHOG_WIZARD_BENCHMARK_FILE` | Output path for benchmark results | -| `POSTHOG_WIZARD_LOG_DIR` | Log directory override | -| `POSTHOG_WIZARD_DEBUG` / `DEBUG` | Enable debug output | -| `MCP_URL` | Override MCP server URL | -| `POSTHOG_API_KEY` | API key for MCP subprocess auth | -| `TERM`, `TERM_PROGRAM`, `CI`, etc. | Terminal/platform detection | -| `APPDATA`, `XDG_CONFIG_HOME` | Platform path resolution | +**Build-time (locked).** `pnpm build` replaces `NODE_ENV` with `"production"` at +compile time. It cannot be overridden at runtime. All URLs, OAuth client IDs, +and dev-mode code paths resolve to their production values. `pnpm build:ci` +inlines `ci` instead, which keeps dev and test flags such as `--ci`. + +To add a new build-time constant, add it to `env` in `tsdown.config.ts` and +export it from `src/env.ts`. + +**Runtime (allowlisted).** Most runtime env reads go through `runtimeEnv()` in +`src/env.ts`, which only accepts keys in the `RuntimeEnvKey` union: + +| Variable | Purpose | +| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------- | +| `POSTHOG_WIZARD_BENCHMARK_CONFIG` | Path to benchmark config file | +| `POSTHOG_WIZARD_BENCHMARK_FILE` | Output path for benchmark results | +| `POSTHOG_WIZARD_RUN_ID` | Assigned cloud WizardRun id | +| `POSTHOG_TASK_RUN_ID`, `POSTHOG_TASK_ID`, `POSTHOG_HANDOFF_OUTPUT_PATH` | The PostHog task run that launched the wizard | +| `WIZARD_CI_GATEWAY_TOKEN_FILE`, `WIZARD_CI_GATEWAY_URL` | The gateway token and URL for `--ci` runs | +| `WIZARD_CI_FLAG_OVERRIDES`, `WIZARD_CI_EXCLUDE_TASKS` | Flag and task overrides for CI builds | +| `POSTHOG_WIZARD_WARLOCK_DISABLED` | Local Warlock override; see the [runbook](docs/runbooks/warlock-kill-switch.md) | +| `MCP_URL` | Override MCP server URL | +| `APPDATA`, `XDG_CONFIG_HOME`, `OPENCODE_CONFIG_DIR` | Platform path resolution | To add a new runtime env var, add its key to `RuntimeEnvKey` in `src/env.ts`. - -**Direct `process.env` access** is only used for subprocess environment writes (e.g. `agent-interface.ts` setting `ANTHROPIC_BASE_URL`), vendored code, and tests. +The CLI reads every `POSTHOG_WIZARD_*` variable as an option and rejects unknown +ones, so a CLI run with `POSTHOG_WIZARD_BENCHMARK_CONFIG`, +`POSTHOG_WIZARD_BENCHMARK_FILE` or `POSTHOG_WIZARD_WARLOCK_DISABLED` set exits +with `Unknown argument`. `POSTHOG_WIZARD_LOG_FILE` is the exception: it is the +env form of `--log-file`, a declared option that moves the debug log from its +default, `posthog-wizard.log` in the temp directory. + +**Direct `process.env` access** covers terminal detection (`TERM`, +`TERM_PROGRAM`, `CI`, …), subprocess environment writes (e.g. +`agent-interface.ts` setting `ANTHROPIC_BASE_URL`), a few launch-time reads, +vendored code, and tests. ### Import aliases -Path aliases defined in `tsconfig.build.json`, resolved by tsdown: - -| Alias | Maps to | -|---|---| -| `@env` | `src/env.ts` | -| `@lib/*` | `src/lib/*` | -| `@agent` | `src/agent/index.ts`, the agent's runtime entry; the only agent import outside `src/agent` besides types | -| `@agent/types` | `src/agent/types.ts`, type-only | -| `@agent/*` | `src/agent/*`, inside `src/agent` and its tests only | -| `@programs` | `src/programs/index.ts`, the programs runtime entry | -| `@programs/types` | `src/programs/types.ts`, type-only | -| `@programs/*` | `src/programs/*`, retained for internal imports and tracked migration edges | -| `@shared/*` | `src/shared/*` | -| `@utils/*` | `src/shared/utils/*` | -| `@ui/*` | `src/ui/*` | -| `@steps/*` | `src/steps/*` | +Path aliases defined in `tsconfig.base.json`, resolved by tsdown and `tsx`: + +| Alias | Maps to | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `@env` | `src/env.ts` | +| `@shared/*` | `src/shared/*` | +| `@utils/*` | `src/shared/utils/*` | +| `@host/*` | `src/host/*`, how a run ends (`startHostExit`, `wizardAbort`, `registerShutdown`) and the control API (`@host/control`), for headless, the TUI, the CLI and the e2e harness | +| `@agent` | `src/agent/index.ts`, the agent's runtime entry | +| `@agent/types` | `src/agent/types.ts`, type-only | +| `@agent/*` | `src/agent/*`, for the agent's tests. The agent imports its own modules by relative path, and no other layer maps it | +| `@programs` | `src/programs/index.ts`, the programs runtime entry | +| `@programs/types` | `src/programs/types.ts`, type-only | +| `@programs/` | `src/programs//index.ts`, one program's entry. Other layers reach nothing else in a program folder | +| `@programs/*` | `src/programs/*`. Other layers reach only the `@programs/` entries; the programs import their own code by relative path, and only their tests use this alias | +| `@tools` | `src/tools/index.ts`, the tools' one entry: the commands that run no agent. The TUI and the CLI import it; programs never do | +| `@tui` | `src/tui/index.ts`, the TUI's one entry, for the CLI and the e2e harness | +| `@tui/*` | `src/tui/*`. Only the TUI's own projects may use it | +| `@headless` | `src/headless/index.ts`, headless's one entry, for the CLI | +| `@cli` | `src/cli/index.ts`, the CLI's one entry, for `bin.ts` | +| `@cli/*` | `src/cli/*`. Only the CLI may use it | +| `@e2e-harness/*` | `e2e-harness/*`, for the harness, scripts and tests | + +Each layer's tsconfig project `references` only the layers it may import, so +`pnpm typecheck` rejects an import of any other. Outside the TUI, `ink`, +`react`, `@inkjs/ui` and `ink-testing-library` resolve to a fence that fails +every import form, and no layer imports JSON. `pnpm lint` rejects the paths the +compiler can't see: a relative import that leaves its layer's folder, a deep +alias outside its own layer, a `.tsbuild/` or `node_modules/` path, `module` +loaders and a bare `require`, triple-slash references, a string-form `vi.mock` +or `vi.importActual`, and the hosts importing the agent's runtime. See +[layer boundaries](.claude/skills/wizard-development/references/ARCHITECTURE.md#layer-boundaries) +for what each layer may import and why each rule exists. Code outside a layer +imports it only through the entries above. The e2e harness, scripts and +`docs/examples` are one layer project, `e2e-harness/tsconfig.json`. Tests follow +the same rule: each layer's tests sit in its project, and a test mocks another +layer through its entry. ## Running locally For `--ci`, smoke tests, and full headless runs, configure both the personal API -key and gateway token file first: [local credentials](docs/local-dev.md#credentials-for-local-ci-and-headless-runs). +key and gateway token file first: +[local credentials](docs/local-dev.md#credentials-for-local-ci-and-headless-runs). Interactive runs mint their gateway token after authentication. ### Quick test without linking @@ -551,19 +695,20 @@ pnpm try --install-dir=[a path] pnpm run dev ``` -This builds, links globally, and watches for changes. Leave it running - any `.ts` file changes will auto-rebuild. Then from any project: +This builds, links globally, and watches for changes. Leave it running - any +`.ts` file changes will auto-rebuild. Then run `wizard` from any project. -```bash -wizard --integration=nextjs +The linked build is a production build, so it rejects the dev-only flags. Run +those from source: +```bash # Point individual services at local dev servers: -wizard --integration=nextjs --local-context-mill # skills from localhost:8765 -wizard --integration=nextjs --local-mcp # MCP from localhost:8787 -wizard --integration=nextjs --local-dev # context-mill + MCP + PostHog +pnpm try --install-dir=[a path] --local-context-mill # skills from localhost:8765 +pnpm try --install-dir=[a path] --local-mcp # MCP from localhost:8787 +pnpm try --install-dir=[a path] --local-dev # context-mill + MCP + PostHog ``` See [`docs/local-dev.md`](docs/local-dev.md) for the full catalog. -`--local-mcp` selects the MCP server; `--local-context-mill` selects the skills server. ### Testing @@ -576,7 +721,7 @@ bin/test To run E2E tests run: ```bash -bin/test-e2e +pnpm test:e2e ``` E2E tests are a bit more complicated to create and adjust due to to their mocked @@ -588,8 +733,8 @@ You can hand the wizard to an AI agent and have it drive the real flow itself deciding each screen and snapshotting the TUI to see what happened. The agent drives through the `wizard-ci` MCP tools (`open_app` / `read_state` / `perform_action` / `render_screen` / `run_agent`), which are registered in this -repo's `.mcp.json` and bound in every session here — approve `wizard-ci` the first -time you're prompted. The how-to is the `exploring-the-wizard` skill +repo's `.mcp.json` and bound in every session here — approve `wizard-ci` the +first time you're prompted. The how-to is the `exploring-the-wizard` skill (`.claude/skills/exploring-the-wizard/SKILL.md`), which an agent discovers automatically. @@ -597,12 +742,12 @@ Example prompt — explore against [open-saas](https://github.com/wasp-lang/open-saas): > Explore the PostHog wizard against open-saas, following the -> `exploring-the-wizard` skill. Reuse my phx key file path, gateway token file path, and project id, -> asking only for missing inputs. Launch the MCP server with -> `WIZARD_CI_GATEWAY_TOKEN_FILE` set to the gateway token file path; -> then clone `https://github.com/wasp-lang/open-saas` into a throwaway `/tmp` -> copy. Drive the whole flow yourself through the `wizard-ci` MCP tools, deciding -> each screen: +> `exploring-the-wizard` skill. Reuse my phx key file path, gateway token file +> path, and project id, asking only for missing inputs. Launch the MCP server +> with `WIZARD_CI_GATEWAY_TOKEN_FILE` set to the gateway token file path; then +> clone `https://github.com/wasp-lang/open-saas` into a throwaway `/tmp` copy. +> Drive the whole flow yourself through the `wizard-ci` MCP tools, deciding each +> screen: > > 1. `open_app` on the `/tmp` copy, then `read_state` to see the screen and the > actions legal right now. @@ -613,7 +758,8 @@ Example prompt — explore against > for any setup question, then `run_agent` at auth. > 4. Poll `read_state` until `integration` is `done` (or `failed` — then report > `integrationError`), snapshotting as the run screen progresses. -> 5. Finish the tail: dismiss outro / mcp / slack, then `keep_skills`. +> 5. Finish the tail: `dismiss_outro`, `set_mcp_outcome`, `dismiss_slack`, then +> `keep_skills`. > > Then show me the saved snapshots in order, the screen path, whether `posthog` > landed in the app, and anything that broke. @@ -629,28 +775,29 @@ To make your version of a tool usable with a one-line `npx` command: # Health checks -`src/shared/health-checks/` checks skills download origins before the wizard runs. -The entry point is `evaluateWizardReadiness()`, which only blocks on skill downloads: +`src/shared/health-checks/` checks skills download origins before the wizard +runs. The entry point is `evaluateWizardReadiness()`, which only blocks on skill +downloads: -| Decision | Meaning | -| ------------------- | --------------------------------------------------------------- | -| `yes` | Skills are reachable — proceed without outage warnings. | -| `no` | Neither skills origin is reachable — do not run. | +| Decision | Meaning | +| -------- | --------------------------------------------------------------------------------------------------------------------------- | +| `yes` | Skills are reachable — proceed without outage warnings. | +| `no` | Neither skills origin is reachable. Interactive runs show the outage and stop; non-interactive runs report it and continue. | ### Module layout -| File | Responsibility | -| --- | --- | -| `types.ts` | Enums, interfaces (`ServiceHealthStatus`, `AllServicesHealth`, etc.) | +| File | Responsibility | +| -------------- | ----------------------------------------------------------------------- | +| `types.ts` | Enums, interfaces (`ServiceHealthStatus`, `AllServicesHealth`, etc.) | | `endpoints.ts` | Direct gateway (`/readyz`) and skills origin (`skill-menu.json`) checks | | `readiness.ts` | `checkAllExternalServices`, `evaluateWizardReadiness`, readiness config | -| `index.ts` | Barrel re-export | -| `testme.md` | Test running instructions and endpoint reference | +| `index.ts` | Barrel re-export | +| `testme.md` | Test running instructions and endpoint reference | ## What blocks a run -The `DEFAULT_WIZARD_READINESS_CONFIG` in `readiness.ts` controls this. It has -two arrays: +The `DEFAULT_WIZARD_READINESS_CONFIG` in `readiness.ts` controls this. Its type +has two arrays, and the default sets only `downBlocksRun`: - **`downBlocksRun`** — if any of these report status **Down**, readiness is **No**. @@ -667,29 +814,35 @@ The same policy applies during signup. Third-party status pages are not queried. After minting a token, `gateway-session.ts` checks `/readyz` on the returned gateway URL and reports an unavailable gateway through the existing error path. -`skillsOrigin` is one entry covering two origins: skills are published to -GitHub Releases and an AWS mirror under the same filenames, and downloads fail -over between them (`src/shared/fetch-retry.ts`). Both are probed in parallel, so -the key only reports **Down** when neither origin answers — a GitHub Releases -outage on its own doesn't block a run, including a 403 or 404, which is as -often about the origin (expired asset redirect, blocked region, a publish that -reached one origin and not the other) as about the asset. +`skillsOrigin` is one entry covering two origins: skills are published to GitHub +Releases and an AWS mirror under the same filenames, and downloads fail over +between them (`src/shared/fetch-retry.ts`). Both are probed in parallel, so the +key only reports **Down** when neither origin answers — a GitHub Releases outage +on its own doesn't block a run, including a 403 or 404, which is as often about +the origin (expired asset redirect, blocked region, a publish that reached one +origin and not the other) as about the asset. ## Smoke test helper (`scripts/smoke-test-ci.sh`) -This repo includes a helper script to run a full end‑to‑end smoke test of the wizard packaged in a tarball against a real app from [`posthog/wizard-workbench`](https://github.com/PostHog/wizard-workbench). This will catch certain packaging issues that might not be caught by other tests. +This repo includes a helper script to run a full end‑to‑end smoke test of the +wizard packaged in a tarball against a real app from +[`posthog/wizard-workbench`](https://github.com/PostHog/wizard-workbench). This +will catch certain packaging issues that might not be caught by other tests. **Prerequisites** - Point to a `wizard-workbench` checkout either by: - Setting `WIZARD_WORKBENCH_ROOT=/absolute/path/to/wizard-workbench`, or - - Cloning `wizard-workbench` next to this repo (so it lives at `../wizard-workbench`). -- Set `POSTHOG_PERSONAL_API_KEY` either in your shell or in `../wizard-workbench/.env`. + - Cloning `wizard-workbench` next to this repo (so it lives at + `../wizard-workbench`). +- Set `POSTHOG_PERSONAL_API_KEY` either in your shell or in + `../wizard-workbench/.env`. - Set `WIZARD_CI_GATEWAY_TOKEN_FILE` to an absolute path containing the separate - AI gateway token. See [local credentials](docs/local-dev.md#credentials-for-local-ci-and-headless-runs). + AI gateway token. See + [local credentials](docs/local-dev.md#credentials-for-local-ci-and-headless-runs). - Set `POSTHOG_WIZARD_PROJECT_ID` to the intended test project and - `POSTHOG_WIZARD_REGION` to `us` or `eu` (CI uses `us`). The helper also accepts - `POSTHOG_PROJECT_ID` and `POSTHOG_REGION` as fallback names. + `POSTHOG_WIZARD_REGION` to `us` or `eu` (CI uses `us`). The helper also + accepts `POSTHOG_PROJECT_ID` and `POSTHOG_REGION` as fallback names. **Usage** @@ -719,8 +872,10 @@ The script will: - Copy the selected app into a temp directory - Install dependencies for the app - Install the packed wizard tarball into an isolated temp project -- Run `wizard` in `--ci` mode against the copied app and perform basic post‑install checks +- Run `wizard` in `--ci` mode against the copied app and perform basic + post‑install checks ## Contributing -Start with [AGENTS.md](AGENTS.md) for the development skills and execution policy. +Start with [AGENTS.md](AGENTS.md) for the development skills and execution +policy. diff --git a/docs/benchmarking.md b/docs/benchmarking.md index e5fadf4f7..63596055e 100644 --- a/docs/benchmarking.md +++ b/docs/benchmarking.md @@ -29,29 +29,29 @@ Selection criteria, checked in this order: 1. **Real product, in production** — an open-source app people actually run (stars are a proxy; a hosted instance is better evidence). 2. **Single-app repo** — reject monorepos: fetch the repo's top-level listing - and reject on `pnpm-workspace.yaml`, `turbo.json`, `lerna.json`, or - top-level `apps/`/`packages/` directories. + and reject on `pnpm-workspace.yaml`, `turbo.json`, `lerna.json`, or top-level + `apps/`/`packages/` directories. 3. **Greenfield** — grep the repo for `posthog` (manifest and source). An app that already integrates PostHog measures augmentation discipline, not integration quality; keep at most one such app and exclude it from quality scoring. 4. **Framework coverage** — spread picks across the frameworks the wizard supports; results do not transfer between them. -5. **Locally installable** — its toolchain (node/python/php/ruby/gradle) - exists on the bench machine, or its runs will fail for reasons that are - yours, not the model's. - -Apps used in the 2026-07 benchmark, as worked examples of the spread: - -| app | upstream | stack | -|---|---|---| -| Maybe | `maybe-finance/maybe` | Rails | -| Outline | `outline/outline` | React + Koa / TS | -| WordPress-Android | `wordpress-mobile/WordPress-Android` | native Kotlin | -| healthchecks | `healthchecks/healthchecks` | Django | -| Firefly III | `firefly-iii/firefly-iii` | Laravel, server-rendered | -| Monica | `monicahq/monica` | Laravel + Inertia/Vue | -| Papermark | `mfts/papermark` | Next.js — already shipped posthog-js; kept as the augment-existing case, excluded from quality scoring | +5. **Locally installable** — its toolchain (node/python/php/ruby/gradle) exists + on the bench machine, or its runs will fail for reasons that are yours, not + the model's. + +Example apps that give that spread: + +| app | upstream | stack | +| ----------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------ | +| Maybe | `maybe-finance/maybe` | Rails | +| Outline | `outline/outline` | React + Koa / TS | +| WordPress-Android | `wordpress-mobile/WordPress-Android` | native Kotlin | +| healthchecks | `healthchecks/healthchecks` | Django | +| Firefly III | `firefly-iii/firefly-iii` | Laravel, server-rendered | +| Monica | `monicahq/monica` | Laravel + Inertia/Vue | +| Papermark | `mfts/papermark` | Next.js — already shipped posthog-js; kept as the augment-existing case, excluded from quality scoring | ## Setup (from a bare machine) @@ -90,21 +90,22 @@ Define each config as a `WIZARD_CI_FLAG_OVERRIDES` JSON, plus the baseline as Everything below ships in this repo (`wizard/`) and its workbench (`wizard-workbench/`); paths are from each repo's root. -- **Headless run (snapshotting CI harness):** `wizard/scripts/tui-snapshots.no-jest.ts` - spawns the real TUI (`wizard/scripts/tui-host.no-jest.ts`) in a PTY via +- **Headless run (snapshotting CI harness):** + `wizard/scripts/tui-snapshots.no-jest.ts` spawns the real TUI + (`wizard/scripts/tui-host.no-jest.ts`) in a PTY via `wizard/e2e-harness/tui-capture.ts`, self-drives the fixed e2e profile (`wizard/e2e-harness/wizard-ci-driver.ts`, `wizard/e2e-harness/profiles.ts`) through auth, the agent run, and the outro, and writes each screen as an `NN-.ans` frame. An `NN-outro.ans` frame is the flow-completion - signal. `tsx` runs source — no build step. Invocation: see the run-cell - recipe below. + signal. `tsx` runs source — no build step. Invocation: see the run-cell recipe + below. - **Config selection:** the flag axis is `wizard-orchestrator` (on → the - orchestrator on pi, per-task models from context-mill frontmatter; off → the - linear anthropic default). Per-stage variations ride - `wizard-orchestrator-override` payloads (`{stage: {model?, effort?}}`, - variant keys in `wizard/src/agent/runner/switchboard/flags/schemes.ts`). - The baseline is `{"wizard-orchestrator":"false"}` — never an empty override, - or live remote flags leak into the baseline. + orchestrator on pi, per-task models from context-mill frontmatter; off → + `DEFAULT_BINDING`, linear on pi). Per-stage variations ride + `wizard-orchestrator-override` payloads (`{stage: {model?, effort?}}`, variant + keys in `wizard/src/agent/runner/switchboard/flags/schemes.ts`). The baseline + is `{"wizard-orchestrator":"false"}` — never an empty override, or live remote + flags leak into the baseline. ## Running one cell @@ -122,12 +123,12 @@ BENCH=~/bench; WORK="$BENCH/$LABEL"; OUT="$BENCH/$LABEL-out"; CAP=600 rm -rf "$WORK" "$OUT"; mkdir -p "$WORK" "$OUT" rsync -a --exclude node_modules --exclude .git --exclude build --exclude dist \ --exclude .next --exclude .svelte-kit --exclude .turbo "$SRC/" "$WORK/" -git -C "$WORK" init -q && git -C "$WORK" add -A +git -C "$WORK" init -q -b main && git -C "$WORK" add -A git -C "$WORK" -c user.email=b@b -c user.name=b commit -qm base --no-verify git -C "$WORK" checkout -q -b integ cd "$WIZARD"; SECONDS=0 -SNAP_OUT="$OUT/frames" APP_DIR="$WORK" \ +SNAP_OUT="$OUT/frames" APP_DIR="$WORK" POSTHOG_WIZARD_LOG_FILE="$OUT/wizard.log" \ POSTHOG_KEY_FILE=~/wizard-bench-key.txt PROJECT_ID= \ WIZARD_CI_FLAG_OVERRIDES="$FLAGS" \ npx tsx scripts/tui-snapshots.no-jest.ts > "$OUT/stdout.txt" 2>&1 & @@ -147,81 +148,85 @@ files=$(git -C "$WORK" diff --name-only main integ | wc -l | tr -d ' ')" \ | tee "$OUT/result.txt" ``` -Both commits are `--no-verify` (see Traps). The diff, frames, stdout, and -result line are the cell's complete artifact set — everything else (the shared -debug log) is unreliable under parallelism. +Both commits are `--no-verify` (see Traps). The diff, frames, stdout, wizard +log, and result line are the cell's complete artifact set. +`POSTHOG_WIZARD_LOG_FILE` gives each cell its own debug log; without it every +run appends to the shared `/tmp/posthog-wizard.log`, which is unreliable under +parallelism. ## Running the matrix -- One app at a time; per app, launch its configs in parallel (≤4 on one - machine) and `wait`. Contention inflates absolute times roughly uniformly. -- Cost: anthropic-harness cells report `modelUsage.costUSD` in - `/tmp/posthog-wizard.log` — zero the log before each app's wave and slice - the block per baseline run. pi-harness cells do not persist token totals; - add a temporary hook in the pi harness success path that writes the session - token stats to a per-run file, and price them at list rates. -- Rerun any anomalous cell solo (zeroed log, no parallelism) before drawing a - conclusion from it. +- One app at a time; per app, launch its configs in parallel (≤4 on one machine) + and `wait`. Contention inflates absolute times roughly uniformly. +- Cost: pi-harness cells, the baseline included, write no cost. Orchestrated pi + tasks log `[pi-task] usage … in= out= cacheR= cacheW=` lines in the cell's + `wizard.log`. A linear pi run's token totals reach only the `agent completed` + analytics event, so add a temporary hook in the pi harness success path that + writes them to a per-run file. Price pi tokens at list rates. + Anthropic-harness cells report `modelUsage.costUSD` in the log. +- Rerun any anomalous cell solo (no parallelism) before drawing a conclusion + from it. ## Traps — each of these has produced a wrong conclusion - **Target-app git hooks.** Your `git commit` runs the app's husky/lint-staged hooks if a prior install activated them; a failing hook silently rolls the - tree back and the run measures as zero-diff. Always commit `--no-verify`. - On any zero-diff run, check `git stash list` before believing it. -- **Zero-diff has many causes.** Distinguish: the agent honestly declined - (read its setup report), the agent's tool calls failed, your harness ate the - work, or `.gitignore` hid it (env files never show in diffs). Attribute - before you blame the model. + tree back and the run measures as zero-diff. Always commit `--no-verify`. On + any zero-diff run, check `git stash list` before believing it. +- **Zero-diff has many causes.** Distinguish: the agent honestly declined (read + its setup report), the agent's tool calls failed, your harness ate the work, + or `.gitignore` hid it (env files never show in diffs). Attribute before you + blame the model. - **"Reached the outro" is not success.** The flow completes even when nothing was integrated. Treat completion as outro + a non-trivial diff. -- **Parallel runs interleave shared state.** The shared debug log cannot be - attributed per-run; capture everything per-run or run solo when attribution - matters. +- **Parallel runs interleave shared state.** The default debug log is one file + for every run and cannot be attributed per-run; give each cell its own with + `POSTHOG_WIZARD_LOG_FILE`, capture everything else per-run, or run solo when + attribution matters. - **Sandbox/allowlist gaps look like model failures.** If a config produces - empty or thin work, check whether a blocked command (package-manager - install, formatter) caused it, and whether other models worked around the - same block. File the gap; exclude the affected cells. -- **Repo-wide format scripts.** An agent running the app's `format`/`lint - --fix` buries its real diff under hundreds of churn files. Count "real - files" excluding scaffolding, lockfiles, env files — and read a sample of - the churn before scoring. -- **A stale credential fails silently mid-batch.** Read the key per run, not - per session. + empty or thin work, check whether a blocked command (package-manager install, + formatter) caused it, and whether other models worked around the same block. + File the gap; exclude the affected cells. +- **Repo-wide format scripts.** An agent running the app's `format`/`lint --fix` + buries its real diff under hundreds of churn files. Count "real files" + excluding scaffolding, lockfiles, env files — and read a sample of the churn + before scoring. +- **A stale credential fails silently mid-batch.** Read the key per run, not per + session. ## Judging Use the wizard-workbench PR evaluator's rubric — do not invent your own. It lives at `wizard-workbench/services/pr-evaluator/`: -- **Rubric criteria:** `wizard-workbench/services/pr-evaluator/prompts/evaluation.md` - — per-item YES/NO/N-A checks grouped into four dimensions. -- **Scoring math:** `wizard-workbench/services/pr-evaluator/evaluator.ts` — - each dimension scores `max(1, round(pass_rate × 5))` over its applicable - items; confidence = `min(app_sanity, round(mean of the four))`. +- **Rubric criteria:** + `wizard-workbench/services/pr-evaluator/prompts/evaluation.md` — per-item + YES/NO/N-A checks grouped into four dimensions. +- **Scoring math:** `wizard-workbench/services/pr-evaluator/evaluator.ts` — each + dimension scores `max(1, round(pass_rate × 5))` over its applicable items; + confidence = `min(app_sanity, round(mean of the four))`. - **Automated run:** from `wizard-workbench/`, `pnpm run evaluate --branch --base --test-run` (needs `POSTHOG_PERSONAL_API_KEY`; judge model via `EVALUATOR_MODEL`). Output lands in `wizard-workbench/test-evaluations//` as `rubric.json` + `scores.json`. -- **Manual run:** an agent applies the same rubric directly to each cell's - diff — faster for many cells, and what the 2026-07 benchmark did. Either - way, report the four dimensions under their full names, 1–5 each - (5 production-ready, 3 works with real issues, 1 broken or empty): +- **Manual run:** an agent applies the same rubric directly to each cell's diff + — faster for many cells. Either way, report the four dimensions under their + full names, 1–5 each (5 production-ready, 3 works with real issues, 1 broken + or empty): -- **Files** (`file_analysis`) — right files touched, nothing unrelated, - imports valid +- **Files** (`file_analysis`) — right files touched, nothing unrelated, imports + valid - **App** (`app_sanity`) — nothing broken: builds, existing code and configs preserved, changes minimal - **PostHog** (`posthog_implementation`) — SDK installed, initialized at the - right entry points, env-based keys, real distinct id, identify, error - tracking + right entry points, env-based keys, real distinct id, identify, error tracking - **Events** (`event_quality`) — real user actions, useful properties, no PII, consistent names -Verify claims against the diff (grep for `capture`/`identify` call sites, -check the init file, check the manifest), and build or typecheck where cheap. -Judge the same subset of apps for every config you compare. +Verify claims against the diff (grep for `capture`/`identify` call sites, check +the init file, check the manifest), and build or typecheck where cheap. Judge +the same subset of apps for every config you compare. ## Publishing evidence @@ -230,48 +235,55 @@ inspectable: 1. Fork each app to the operator's account. 2. Pin a `bench-base` branch at the exact commit the runs used. -3. Per cell: branch from `bench-base`, apply the **sanitized** patch, push, - open a draft PR against `bench-base`. +3. Per cell: branch from `bench-base`, apply the **sanitized** patch, push, open + a draft PR against `bench-base`. 4. Sanitize before anything touches a public fork: drop env files, wizard - scaffolding, and lockfiles from the patch; redact every token literal. - Verify zero secrets in the pushed diff before opening the PR. + scaffolding, and lockfiles from the patch; redact every token literal. Verify + zero secrets in the pushed diff before opening the PR. ## Report template ```markdown # — model benchmark - + ## Summary — configs that completed everywhere -| config | completed | median time | median cost | quality (judged on) | - ## Results - -| config | | | … | -