feat(sleep): add Pi transcript source and CLI backend

[auspic7]

Summary

Adds explicit Pi integration to SkillOpt-Sleep:

• --source pi harvests local Pi JSONL sessions from ~/.pi/agent/sessions.
• --backend pi runs mining, replay, judging, and reflection through a locally installed, authenticated Pi CLI.
• Source and backend selection remain independent.

Pi is opt-in. --source auto keeps the existing Codex-then-Claude precedence and does not select Pi.

Original contribution

The original branch introduced:

• Pi transcript parsing and source routing.
• Extraction of user/assistant text, tool names, and lexical user feedback.
• Exclusion of thinking blocks, tool arguments, tool outputs, and images.
• Deliberate exclusion of transient toolResult.isError values from task-outcome feedback.
• A Pi CLI backend with model/path configuration and isolated print-mode execution.
• Launch, timeout, and non-zero-exit diagnostics.

The original PR reports:

• ruff check passing on the touched files.
• 54 passed, 1 skipped for the initial harvester and Sleep tests.
• A real local harvest of 12 Korean/English Pi sessions producing 11 tasks.
• A live Pi CLI call using zai/glm-5.2.

These are the original contributors' reported validations; maintainer validation is listed separately below.

Maintainer integration and hardening

Because the branch had become stale and conflicted with current main, maintainers integrated it without rewriting the original commits and added:

• Compatibility with the current backend factory, dual-backend wiring, OpenClaw wrappers, CLI, and plugin checks.
• Explicit-only Pi source routing; Pi does not change --source auto.
• Pi v1/v2/v3 session validation and active-branch traversal.
• Fail-closed handling for malformed JSONL, unsupported versions, duplicate IDs, and invalid parent chains.
• Stronger CLI isolation: no tools, skills, context files, extensions, prompt templates, themes, or session writes.
• Prompt delivery over stdin and redacted failure diagnostics.
• PI_OFFLINE=1 to disable Pi startup update/package/telemetry traffic while still allowing the configured model-provider request.
• Non-sticky caching for failed model calls.
• Privacy documentation covering retained message text and the project cwd boundary.
• Focused configuration, routing, integration, and compatibility tests.

Privacy boundary

Pi transcript harvesting is local and read-only. When a real backend is selected, transcript-derived prompts are sent to that backend's configured provider.

Known secret-shaped strings in retained message text are redacted on a best-effort basis. The absolute project cwd is retained for scope filtering and may appear in miner prompts; it is not covered by message-text redaction.

Validation

Maintainer-performed validation:

• 76 passed in 1.05s across the Pi backend, harvester, integration, model-warning, and plugin-sync tests.
• Real Pi CLI v0.83.0 against a local OpenAI-compatible test provider:
  • PI_OFFLINE=1 still reached /v1/chat/completions and returned the expected response.
  • An intentionally unavailable package plus a disabled npm command did not trigger package installation.
• Full suite: 664 passed, 6 skipped, 130 subtests passed.
• compileall, CLI help, and git diff --check passed.

[auspic7]

@microsoft-github-policy-service agree

[Yif-Yang]

Hi @auspic7 — thanks for this, it's a clean addition. This PR has been in draft for a while; is it ready to merge? I reviewed and tested it locally (pi harvester + backend tests pass, no regressions). One small thing worth addressing before/after merge: in skillopt_sleep/backend.py, PiCliBackend._call() swallows all launch/timeout exceptions and ignores non-zero exit codes, so a missing pi binary returns "" — indistinguishable from an empty model answer (the sibling CodexCliBackend records last_call_error + logs a warning). Also the section comment above the class still reads "Claude Code CLI backend". Happy to merge once you mark it ready — or I can help land it with those fixes if you'd like.

[Yif-Yang]

Hi @auspic7 — thanks, 70629dc2 addresses the launch/timeout/non-zero-exit handling and the stale section label from our last note. Is this PR now ready to leave draft? If so, please rebase onto current main and mark it Ready for review; we will re-run the focused tests and review it promptly.

[Yif-Yang]

Hi @auspic7 — friendly status check on this one. The CLA is all set, but the PR is still in draft and currently has merge conflicts. When you have a moment, could you rebase onto the latest main to resolve the conflicts and mark the PR as Ready for review? Once that is done, we will take another pass promptly. Thanks!

[Copilot]

Pull request overview

This PR adds an explicit Pi integration path to SkillOpt-Sleep, enabling both (1) harvesting Pi session transcripts from the local ~/.pi/agent/sessions tree and (2) running the Sleep cycle through a locally installed/authenticated Pi CLI backend, while keeping source selection (--source) and backend selection (--backend) independent and preserving existing --source auto precedence.

Changes:

• Added a Pi transcript harvester (--source pi) with schema validation, active-branch selection for v2/v3 sessions, and secret-shaped string redaction.
• Added a Pi CLI backend (--backend pi) that runs in print mode with strong isolation flags and offline startup env settings.
• Expanded CLI/config/routing, OpenClaw plugin wiring, and tests/docs to cover Pi paths end-to-end.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/test_plugin_sync.py	Extends plugin sync coverage to ensure OpenClaw wrapper matches updated shared backend factory signature (including Pi params).
tests/test_pi_integration.py	Adds CLI/config and harvest routing tests ensuring --source pi is explicit and --source auto does not select Pi.
tests/test_model_change_warning.py	Ensures model-key construction forwards pi_path into backend build for cache-key correctness.
tests/test_harvest_pi.py	Comprehensive unit coverage for Pi session digestion/validation, branch selection, and redaction behavior.
tests/test_backend_pi.py	Unit tests for Pi CLI backend invocation, isolation flags, error handling, caching semantics, and token accounting.
skillopt_sleep/harvest_sources.py	Adds pi as an explicit transcript source routed to the new harvester.
skillopt_sleep/harvest_pi.py	Implements Pi JSONL reading, schema checks, active-branch extraction, normalization into SessionDigest, and directory walking harvest.
skillopt_sleep/cycle.py	Includes pi_path in backend construction inputs for stable model-keying and runtime wiring.
skillopt_sleep/config.py	Adds pi_home, pi_path, and a derived pi_sessions_dir to configuration defaults/accessors.
skillopt_sleep/backend.py	Introduces PiCliBackend, adds Pi aliases to get_backend, and wires pi_path through build_backend (single + dual backend).
skillopt_sleep/main.py	Adds CLI flags --pi-home / --pi-path and extends --backend / --source choices to include pi.
plugins/README.md	Documents Pi source/backend behavior, privacy boundary, and CLI flags in the plugin integration README.
plugins/openclaw/skillopt_sleep_openclaw.py	Refactors reference-answer rubric formatting for readability/maintainability.
plugins/openclaw/run_sleep.py	Updates OpenClaw’s backend factory shim to accept/forward the expanded shared signature including Pi params.
docs/sleep/README.md	Adds Pi source/backend documentation and updates high-level pipeline description to include Pi.
docs/reference/cli.md	Updates CLI reference to include Pi flags and clarifies network/no-network backend boundary language.
docs/guide/installation.md	Adds optional Pi CLI installation guidance and clarifies that Pi harvesting does not require the CLI.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Yif-Yang]

Thanks @brandon and @auspic7 for bringing the Pi integration forward, including the follow-up CLI failure handling.

Since this PR remained in draft and had diverged from main, we used the allowed maintainer edits to integrate current main and complete the compatibility, isolation, privacy-documentation, and test hardening. The four original commits and their authorship have been preserved; the maintainer integration and hardening remain separate commits in the history.

One important final behavior is that Pi remains explicit-only: --source pi enables it, while --source auto keeps the existing Codex-then-Claude precedence.

Maintainer validation completed successfully:

• 76 focused Pi, routing, model-warning, and plugin compatibility tests passed.
• Full suite: 664 passed, 6 skipped, 130 subtests passed.
• Pi CLI v0.83.0 reached a local OpenAI-compatible test provider under PI_OFFLINE=1, while the package-install path remained disabled.
• Independent integration, security, and final code reviews found no merge blocker.

We are marking this ready and proceeding with a normal merge commit so the original contribution history remains intact.

[Yif-Yang]

Maintainer integration, full-suite validation, Pi CLI smoke testing, and independent security/correctness reviews are complete. No merge blocker remains.

[Yif-Yang]

Re-approved on the latest head (716197e) after the maintainer hardening, focused/full regression suites, local Pi-provider smoke test, and independent integration/security reviews. Ready to merge.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/test_harvest_pi.py:14

• In this helper you write JSONL records without specifying an encoding. Most other tests in this repo use encoding="utf-8" for text writes, and leaving it implicit can cause platform-dependent behavior (e.g., Windows default code pages) if Pi sessions contain non-ASCII text.

    with open(p, "w") as f: