Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions docs/guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ checkout for those files.
These docs track the latest `main`. The current PyPI release is `0.2.0`.
The generic research `openai_compatible` backend, SkillOpt-Sleep handoff,
Sleep support for non-Azure OpenAI-compatible endpoints, the Sleep
`--preferences` flag, and Cursor source/backend/plugin support landed after
that release and require a source install from `main` until the next
release.
`--preferences` flag, Cursor source/backend/plugin support, and Pi
source/backend support landed after that release and require a source
install from `main` until the next release.

### Source checkout

Expand Down Expand Up @@ -63,6 +63,19 @@ Install extras for specific benchmarks or backends:
Claude Code CLI separately. The SDK extra is only needed when selecting an
SDK-backed Claude Code exec path.

=== "Pi coding-agent CLI (optional)"

```bash
npm install -g --ignore-scripts @earendil-works/pi-coding-agent
```

Install and authenticate the [Pi coding-agent CLI](https://github.com/earendil-works/pi)
only when using SkillOpt-Sleep with `--backend pi`. Harvesting local Pi
transcripts with `--source pi` does not require the CLI or provider
authentication. By default, the source reads below
`~/.pi/agent/sessions`; `--pi-home` selects the parent directory that
contains `agent/sessions`.

=== "Qwen (Local)"

```bash
Expand Down
54 changes: 50 additions & 4 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@
> include the generic research `openai_compatible` backend, Sleep handoff,
> Sleep support for non-Azure OpenAI-compatible endpoints, the Sleep
> `--preferences` flag, the research `cursor_exec` target harness, or Cursor
> source/backend/plugin support or VS Code Copilot transcript harvesting; use
> a source install from `main` for those features until the next release.
> source/backend/plugin support, Pi source/backend support, or VS Code Copilot
> transcript harvesting; use a source install from `main` for those features
> until the next release.

## Training

Expand Down Expand Up @@ -128,12 +129,14 @@ Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
|---|---|
| `--project PATH` | Project used for transcript scope, targets, state, and staging (default: current directory) |
| `--scope invoked\|all` | Harvest this project or all projects |
| `--source claude\|codex\|copilot\|cursor\|auto` | Transcript source; `auto` keeps Codex-then-Claude precedence and does not select Copilot or Cursor |
| `--backend mock\|claude\|codex\|copilot\|cursor\|handoff\|azure_openai` | Replay/optimizer backend |
| `--source claude\|codex\|copilot\|cursor\|pi\|auto` | Transcript source; `auto` keeps Codex-then-Claude precedence and does not select Copilot, Cursor, or Pi |
| `--backend mock\|claude\|codex\|copilot\|cursor\|pi\|handoff\|azure_openai` | Replay/optimizer backend |
| `--model NAME` | Backend-specific model override |
| `--cursor-home PATH` | Override `~/.cursor` for Cursor transcript harvesting |
| `--pi-home PATH` | Parent directory containing Pi's `agent/sessions` tree (default: `~/.pi`) |
| `--vscode-workspace-storage PATH` | Override VS Code's `User/workspaceStorage` root for Copilot transcript harvesting |
| `--cursor-path PATH` | Path to the installed Cursor Agent CLI |
| `--pi-path PATH` | Path to the installed Pi coding-agent CLI |
| `--preferences TEXT` | House rules supplied to reflection |
| `--lookback-hours N` | Initial transcript lookback; `0` scans all history |
| `--max-sessions N` / `--max-tasks N` | Bound the harvested workload |
Expand All @@ -143,6 +146,11 @@ Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
| `--progress` / `--json` | Progress or machine-readable output |
| `--auto-adopt` | Apply an accepted staged proposal automatically |

The `mock` and `handoff` backends make no network calls. A real backend sends
mining, replay, judging, and reflection prompts derived from harvested
transcripts and tasks to its selected provider. Review that provider's
data-retention and privacy policy before processing sensitive sessions.

### VS Code GitHub Copilot Chat source

`--source copilot` reads local VS Code GitHub Copilot Chat session logs from
Expand All @@ -167,6 +175,44 @@ The managed `schedule` command does not persist `--source` or
`"vscode_workspace_storage": "/absolute/path/to/workspaceStorage"` in
`~/.skillopt-sleep/config.json`.

### Pi source and backend

`--source pi` reads local session JSONL files below
`~/.pi/agent/sessions`; use `--pi-home PATH` to select the parent directory that
contains `agent/sessions`. This local source does not require the Pi CLI or
provider authentication. It retains user/assistant text, tool names, and lexical
feedback found in user text, while excluding thinking, tool arguments, tool
outputs, images, and unrelated metadata. The absolute project `cwd` from the
session header is retained for scope filtering and may appear in miner prompts
sent to a real backend and its provider. Known secret-shaped strings in retained
message text are redacted as defense in depth, not as a guarantee. Pi is an explicit source: `--source auto`
retains Codex-then-Claude precedence and does not select it.

Transcript source and model backend are independent. `--backend pi` launches a
locally installed, authenticated Pi CLI and makes real provider calls for
mining, replay, judging, and reflection. Use `--pi-path PATH` to select its
executable and `--model NAME` to override its configured model:

```bash
skillopt-sleep run --project "$(pwd)" \
--source pi --backend pi --pi-path /absolute/path/to/pi \
--model provider/model --max-sessions 5 --max-tasks 3 --progress
```

For these calls, SkillOpt disables Pi tools, skills, context files, extensions,
prompt templates, themes, and session writes. Pi authentication and model
configuration remain available. It also enables Pi's offline startup mode, so
configured npm/git packages are not installed or updated and model catalogs are
not refreshed; the selected model provider is still contacted for generation.
These controls should not be treated as permanent or complete isolation. The
provider selected in Pi receives the transcript-derived prompts.

The managed `schedule` command preserves the backend but not `--source`,
`--pi-home`, `--pi-path`, or `--model`. Before scheduling Pi, put
`transcript_source`, `pi_home`, `pi_path`, and `model` in
`~/.skillopt-sleep/config.json`; use an absolute `pi_path` and verify
authentication for the scheduled account.

### Cursor source and backend

`--source cursor` reads local Cursor JSONL transcripts from
Expand Down
59 changes: 55 additions & 4 deletions docs/sleep/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ normal agent requests.
One "night":

```
harvest Claude Code / Codex / VS Code Copilot / Cursor transcripts → mine recurring tasks → replay via the configured backend (isolation varies by backend; mock/handoff make no network calls)
harvest Claude Code / Codex / VS Code Copilot / Cursor / Pi transcripts → mine recurring tasks → replay via the configured backend (isolation varies by backend; mock/handoff make no network calls)
→ consolidate (reflect → bounded edit → GATE on real held-out tasks)
→ stage proposal → (you) adopt
```
Expand Down Expand Up @@ -49,6 +49,22 @@ experience → long-term competence).
> context, and account/model metadata. Known secret-shaped strings are
> redacted, but this remains defense in depth rather than a guarantee.
>
> The Pi source reads local sessions below `~/.pi/agent/sessions`, retaining
> user/assistant text, tool names, and lexical feedback found in user text. It
> excludes thinking, tool arguments, tool outputs, images, and unrelated
> metadata. The absolute project `cwd` from the session header is retained for
> scope filtering and may appear in miner prompts sent to a real backend and its
> provider. Known secret-shaped strings in retained message text are redacted
> only as defense in depth.
> The Pi backend uses the installed, authenticated Pi CLI to contact the user's
> selected model provider. Calls disable tools, skills, context files, extensions, prompt
> templates, themes, and session writes, while retaining Pi authentication and
> model configuration. Pi's offline startup mode also prevents configured
> npm/git package installation, package updates, and model-catalog refresh; it
> does not prevent the selected provider call. This is not a guarantee of
> permanent or complete isolation. Review the provider's retention and privacy
> policy before sending transcript-derived prompts from sensitive sessions.
>
> By default, each stateful night also writes a local `evidence.jsonl` under
> the project staging tree (beside the report when one is staged); dry-runs
> write evidence under the configured Sleep state directory. The log contains
Expand All @@ -73,9 +89,9 @@ skillopt-sleep schedule # install a nightly cron entry for this project

> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base
> commands above. Cursor source/backend/plugin support, VS Code Copilot
> transcript harvesting, Sleep handoff, non-Azure OpenAI-compatible endpoints,
> and `--preferences` landed later and require a source install from `main`
> until the next release.
> transcript harvesting, Pi source/backend support, Sleep handoff, non-Azure
> OpenAI-compatible endpoints, and `--preferences` landed later and require a
> source install from `main` until the next release.

The per-agent integrations below still come from the repo; the CLI above is the
standalone, pip-only way to run a cycle. Claude Code, Codex, Cursor, Copilot, and
Expand Down Expand Up @@ -119,6 +135,41 @@ The managed scheduler does not preserve `--source` or
`"vscode_workspace_storage": "/absolute/path/to/workspaceStorage"` in
`~/.skillopt-sleep/config.json`.

### Pi

Pi transcript harvesting and model execution are independent. Use `--source pi`
to read local session JSONL files below `~/.pi/agent/sessions`, or set
`--pi-home` to the parent directory that contains `agent/sessions` (the default
is `~/.pi`). The source alone does not require Pi CLI installation or provider
authentication. Pi is never selected implicitly:
`--source auto` retains Codex-then-Claude precedence.

`--backend pi` uses a locally installed and authenticated Pi CLI for real
model-provider calls during mining, replay, judging, and reflection. Select its
executable with `--pi-path` and override its configured model with `--model`:

```bash
skillopt-sleep run --project "$(pwd)" \
--source pi --backend pi --pi-path /absolute/path/to/pi \
--model provider/model --max-sessions 5 --max-tasks 3 --progress
```

These calls disable tools, skills, context files, extensions, prompt templates,
themes, and session writes, but still use the user's Pi authentication and model
configuration. They also enable Pi's offline startup mode to prevent configured
npm/git package installation, package updates, and model-catalog refresh; the
selected model provider is still contacted. Treat those controls as bounded
invocation setup, not permanent or complete isolation. A real Pi backend sends
transcript-derived prompts to the provider configured in Pi; inspect that
provider's retention and privacy policy first. The `mock` and `handoff` backends
make no network calls.

The managed scheduler records the backend but does not preserve `--source`,
`--pi-home`, `--pi-path`, or `--model`. Before scheduling Pi, set
`transcript_source`, `pi_home`, `pi_path`, and `model` in
`~/.skillopt-sleep/config.json`. Use an absolute `pi_path` and verify the
scheduled account's Pi authentication.

### Cursor

Cursor transcript harvesting and model execution are independent. Use
Expand Down
73 changes: 64 additions & 9 deletions plugins/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ an importable `skillopt_sleep` module. Install with `uv tool install skillopt` o

> **Version note.** This integration reference tracks `main`. PyPI 0.2.0
> supports the base Sleep CLI, while Cursor source/backend/plugin support,
> handoff, Sleep support for non-Azure OpenAI-compatible endpoints, and
> `--preferences` require a source checkout from `main` until the next release.
> Pi source/backend support, handoff, Sleep support for non-Azure
> OpenAI-compatible endpoints, and `--preferences` require a source checkout
> from `main` until the next release.

## One sleep cycle

Expand All @@ -64,10 +65,10 @@ optimization.

## Data boundary

- Harvesting is local and read-only. The `mock` backend has no model-provider
data path and no API spend.
- A real backend sends truncated transcript excerpts and derived task content to
the provider selected for mining, replay, judging, and reflection.
- Harvesting is local and read-only. The `mock` and `handoff` backends make no
network calls; handoff writes prompts for separate, user-controlled completion.
- A real backend sends mining, replay, judging, and reflection prompts derived
from truncated transcript excerpts and tasks to the selected provider.
- The Cursor source reads local user/assistant message text, explicit turn
errors, and tool names from `~/.cursor/projects/*/agent-transcripts`; it does
not retain tool arguments, tool outputs, or other record types. Known
Expand All @@ -79,6 +80,19 @@ optimization.
`tool_called` validation fail before Agent mode starts; use another backend for
those tasks. Cursor and the model provider selected by Cursor can receive the
resulting prompt content.
- The Pi backend sends prompts through the installed, authenticated Pi CLI to
the provider configured by the user. It disables tools, skills, context files,
extensions, prompt templates, themes, and session writes for these calls, but
retains the user's Pi authentication and model configuration. Pi's offline
startup mode prevents configured npm/git package installation, package
updates, and model-catalog refresh; it does not prevent the selected provider
call. These controls are not a guarantee of permanent or complete isolation.
- The Pi source retains user/assistant text, tool names, and lexical feedback
found in user text. It excludes thinking, tool arguments, tool outputs, images,
and unrelated metadata. The absolute project `cwd` from the session header is
retained for scope filtering and may appear in miner prompts sent to a real
backend and its provider. Known secret-shaped strings in retained message text
are redacted only as defense in depth.
- Outbound prompts are not currently guaranteed to be free of secrets. Do not
use a third-party provider on sensitive transcripts without reviewing the data
source and the provider's retention policy.
Expand Down Expand Up @@ -114,11 +128,13 @@ Common implemented flags include:

| Flag | Default | Purpose |
|---|---|---|
| `--backend mock\|claude\|codex\|cursor\|copilot\|handoff\|azure_openai` | `mock` | select who performs model calls |
| `--backend mock\|claude\|codex\|cursor\|copilot\|pi\|handoff\|azure_openai` | `mock` | select who performs model calls |
| `--model NAME` | backend default | select a backend-specific model |
| `--source claude\|codex\|cursor\|auto` | `claude` | select the transcript source; `auto` retains Codex-then-Claude precedence and does not select Cursor |
| `--source claude\|codex\|copilot\|cursor\|pi\|auto` | `claude` | select the transcript source; `auto` retains Codex-then-Claude precedence and does not select Copilot, Cursor, or Pi |
| `--cursor-home PATH` | `~/.cursor` | override the Cursor transcript home |
| `--cursor-path PATH` | auto-detect `cursor-agent` | select the Cursor Agent CLI executable |
| `--pi-home PATH` | `~/.pi` | select the parent directory containing `agent/sessions` |
| `--pi-path PATH` | auto-detect `pi` | select the Pi coding-agent CLI executable |
| `--project PATH` | current directory | select the project and invoked harvest scope |
| `--scope invoked\|all` | `invoked` | limit transcript harvesting |
| `--target-skill-path PATH` | managed skill | select a specific `SKILL.md` to stage/adopt |
Expand Down Expand Up @@ -153,6 +169,45 @@ python -m skillopt_sleep run --backend codex --project "$(pwd)" \

Preferences guide reflection but remain subject to the validation gate.

### Pi source and backend

Pi transcript harvesting is explicit: `--source pi` reads session JSONL files
below `~/.pi/agent/sessions`; use `--pi-home` to select the parent directory
that contains `agent/sessions`. This source does not require the Pi CLI or
provider authentication. It retains user/assistant text, tool names, and lexical
feedback found in user text, while excluding thinking, tool arguments, tool
outputs, images, and unrelated metadata. The absolute project `cwd` from the
session header is retained for scope filtering and may appear in miner prompts
sent to a real backend and its provider. Known secret-shaped strings in retained
message text are redacted as defense in depth, not as a guarantee. `--source auto` keeps Codex-then-Claude
precedence and does not select Pi.

The source and backend are independent. `--backend pi` uses a locally installed,
authenticated Pi CLI to make real model-provider calls for mining, replay,
judging, and reflection. Select another executable with `--pi-path` and a model
with `--model`:

```bash
python -m skillopt_sleep run --project "$(pwd)" \
--source pi --backend pi --pi-path /absolute/path/to/pi \
--model provider/model --max-sessions 5 --max-tasks 3 --progress
```

Pi calls disable tools, skills, context files, extensions, prompt templates,
themes, and session writes. They still use the user's Pi authentication and
model configuration. Pi's offline startup mode also prevents configured npm/git
package installation, package updates, and model-catalog refresh; it does not
prevent the selected provider call. This is bounded invocation setup rather
than permanent or complete isolation. Transcript-derived prompts reach the
provider configured in Pi; review that provider's data-retention and privacy
policy before using sensitive sessions.

The managed scheduler stores the selected backend but does not persist
`--source`, `--pi-home`, `--pi-path`, or `--model`. Before scheduling Pi, set
`transcript_source`, `pi_home`, `pi_path`, and `model` in
`~/.skillopt-sleep/config.json`; prefer an absolute `pi_path` and verify that the
scheduled account is authenticated.

### Cursor source and backend

Cursor transcript harvesting is explicit: use `--source cursor` rather than
Expand Down Expand Up @@ -221,7 +276,7 @@ the shipping CLI defaults.
## Safety summary

- Session harvesting is read-only.
- `mock` replay makes no provider calls.
- `mock` and `handoff` make no network calls.
- `run` stages proposals; `adopt` is the normal live-change boundary.
- Adoption backs up existing target files.
- `--max-sessions` and `--max-tasks` bound work, but the main CLI does not yet
Expand Down
15 changes: 14 additions & 1 deletion plugins/openclaw/run_sleep.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,27 @@
# Patch get_backend to know about our backend
_orig_get_backend = _b.get_backend

def get_backend(name, model="", codex_path="", cursor_path="", project_dir=""):
def get_backend(
name,
*,
model="",
claude_path="claude",
codex_path="",
pi_path="",
cursor_path="",
azure_endpoint="",
project_dir="",
):
if name == "openclaw-deepseek":
return OpenClawDeepSeekBackend(model=model or "deepseek-v4-pro")
return _orig_get_backend(
name,
model=model,
claude_path=claude_path,
codex_path=codex_path,
pi_path=pi_path,
cursor_path=cursor_path,
azure_endpoint=azure_endpoint,
project_dir=project_dir,
)

Expand Down
7 changes: 6 additions & 1 deletion plugins/openclaw/skillopt_sleep_openclaw.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,12 @@ def reflect(

rubric_text = ""
if failures:
rubric_text = f"\n\n## REFERENCE ANSWERS\n{chr(10).join(f'Q: {t.intent[:120]}\\nA: {t.reference}' for t, _ in failures[:3] if t.reference)}"
reference_answers = "\n".join(
f"Q: {t.intent[:120]}\nA: {t.reference}"
for t, _ in failures[:3]
if t.reference
)
rubric_text = f"\n\n## REFERENCE ANSWERS\n{reference_answers}"

sys = (
"You are SkillOpt-Sleep's bounded-edit optimizer. Your job is to propose 1-4 MINIMAL text edits to a skill or memory document "
Expand Down
Loading