- Quick Start
- What It Does
- Architecture
- API Key Setup (Soul Mutations)
- Per-User Authentication
- Installation
- Docker / GHCR
- Full Setup Guide
- Project Structure
- Dropbox Corpus Sync
- macOS launchd & Keychain
- Agentic Layer
- Filesystem, SQL & Passive Network Connectors
- NeMo Guardrails
- Agentic Harness Scaffold
- Coding Harness Console
- GitHub Agentic Coding Harness
- Telegram Channel
- OpenTweet Channel
- Security Model
- Remaining Work
- Archive & Roadmap
The fastest path to a running RAG server — macOS/Linux shown; only the torch pin and the activation command differ on Windows (see Installation for the full per-platform split):
git clone https://github.com/CGFixIT/CyClaw && cd CyClaw
python3.12 -m venv .venv && source .venv/bin/activate
pip install torch==2.13.0+cpu --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt -r requirements-test.txt -c constraints.txt --ignore-installed PyYAML
ollama pull qwen3.8:27b-mlx # Ollama must already be running on :11434
export CYCLAW_API_KEY="$(openssl rand -hex 20)" # needed for the /soul/* endpoints
python -m retrieval.indexer # builds the retrieval index, once
uvicorn gate:app --host 127.0.0.1 --port 8787 # → http://127.0.0.1:8787Confirm it's alive: curl http://127.0.0.1:8787/health.
On macOS (Apple Silicon, the primary platform): the +cpu torch wheel
above doesn't exist there — use plain torch==2.13.0 and strip the torch/
--extra-index-url lines from both manifests first, or just run
bash ./macos/setup-cyclaw.sh, which handles the torch difference, Ollama,
the index, and both servers for you.
Full step-by-step guide — exact per-platform commands, Docker, the
coding-harness console, and every REST endpoint with a copy-pasteable curl:
setup-guide.md.
CyClaw is a personal RAG (Retrieval-Augmented Generation) backend that:
- Answers questions exclusively from your local Markdown corpus — no internet by default.
- Enforces every safety invariant via LangGraph topology — not prompts, not config flags, not discipline.
- Maintains a persistent soul/personality layer (
soul.md) with SHA-256 drift detection, atomic evolution writes, and user-gated modification. - Falls back to an external LLM only with explicit user confirmation in hybrid mode — Grok (xAI) or Claude (Anthropic), selected per-query, each independently triple-gated at config, env, and per-query level.
- Exposes both a FastAPI HTTP gateway and an MCP server for Claude Desktop / Copilot Studio integration.
- Ships optional, out-of-band operator layers — Dropbox corpus sync (
sync/) and agentic GitHub context / governed local workflows (agentic/,.claude/) — never imported into the request path, drivable from the browser terminal via the governed Sync and Agentic consoles. - Extends the agentic layer to local data (v1.8+) with opt-in filesystem (
agentic/fsconnect/), read-only SQL (agentic/sqlconnect/), and passive LAN inventory (agentic/netconnect/) connectors — all disabled by default and out-of-band. - Adds an optional NeMo Guardrails content-safety layer (v1.8,
guardrails/) that soft-importsnemoguardrails, degrades to offline heuristic rails, and is defense-in-depth only — never a routing authority. Whenguardrails.enabledis the literaltrue,utils/guardrail_bridge.pywires the visibleguardrail_input/guardrail_outputnodes. Seeguardrails/README.md. - Scaffolds a governed harness-optimizer / Deep Agents layer (v1.9,
agentic/harness_optimizer/+agentic/deepagent_github/) — opt-in, disabled by default, out-of-band; phases 0–9 implemented and tested (PR #515, 2026-07-13). Superseded by item 11: the live coding pipeline is nowreal_repo_loop.py, whose draft-PR write path was armed on 2026-08-07 behind theagentic.enabledmaster switch that still shipsfalse. - Ships a local coding-harness console (v1.9,
harness/+powershell//macos/) — a grok-build-style slash-command console on127.0.0.1:8790chatting with the local model over the OpenAI-compatible endpoint, with per-session token tallies,/goal+ human-gated/loop,/skillsand/toolswiring diagrams, and allowlist-only/web(off by default). Home is%USERPROFILE%\.CyClaw(Windows) or~/.CyClaw(macOS/Linux); same I6 isolation as every other out-of-band layer. Seeharness/README.md. - Adds a real-repo GitHub agentic coding harness (v1.9,
agentic/real_repo_loop.py+agentic/executor/) — clone → plan → patch → verify → human decides → commit, with pushing aclaude/*branch and opening a draft PR as two further separate decisions. A diff-scope gate refuses candidates that rewrite the tests judging them, verification runs as sandboxed argv-list subprocesses, and the layer ships off:agentic.enabled: falseis the master switch (plus per-call reason/confirm) andallow_git_write_tools: falseholds push. - Adds an optional per-user authentication layer (
gate_auth.py+utils/authn*) — scrypt password hashes, session cookie + CSRF for browsers, bearer device tokens for programmatic clients, three roles (admin/operator/audit), and thecyclaw-userconsole script. Every/auth/*route exists regardless ofauth.enabledand returns 503 (not 404) when it is off, so route presence never discloses the feature state. Whenauth.enabledis true,POST /queryand the console require a session or named device token. The shipped default leaves/queryopen. See Per-User Authentication. - Adds an optional facts + episodes memory store (
gate_memory.py+memory/) — SQLite+FTS5, propose/apply governance (a non-empty humanreasonplus an injection scan on apply, parallel to soul's I5), and an optional retrieval-fusion hook. Everymemory:switch shipsfalse; mutating routes require the same BearerCYCLAW_API_KEYas the other admin endpoints. Seememory/README.mdand the plan indocs/memory/README.md— notdocs/memories/, the sandbox notes. - Ships an optional Telegram channel (v1.9,
telegram/, shippedenabled: false) — an out-of-band phone remote: outbound notify (mode: "notify") or allowlisted long-poll chat (the shippedmode: "chat"; T1-first is still the advised enable order). Inbound text only ever reaches the RAG pipeline through loopbackPOST /query. T3 hybrid-confirm (allow_hybrid_confirm, default off) is the only way chat text can setuser_confirmed_online— one-shot, via the exact private-chat command/online on <grok|claude>— and T4 media staging (media.enabled, default off) writes only through the existingagentic/fsconnectpath. Seedocs/channels/TELEGRAM_DESIGN.md. - Adds an offline slop-detection probe for the agentic coding loop (v1.9.x,
agentic/unslop_bridge.py+ vendored scanners underagentic/vendor/unslop/) — scansreal_repo_loop.py's model responses and proposed prose files (.md/.rst/.txt) for AI-writing tells, logs redacted findings (SHA-256 doc hash + counts, never raw text) tologs/unslop.jsonl, and surfaces a nudge back into the loop.unslop.enabledshipsfalse; the scanner runs fully offline with no network calls and never crosses the I6 boundary. - Projects the audit trail into a Numbat forensic stream (
utils/numbat_emitter.py,numbat:block — the one optional subsystem that shipsenabled: true) — a derived NDJSON stream atlogs/numbat-events.ndjsonlthat the pinned Numbat 0.2.0 CLI can score for patterns likesecrets.read_private_keyandexfil.curl_post_file, fed by the out-of-band action plane and the mainline plane (utils/logger.audit_logprojects every audit record).audit.jsonlstays authoritative; records are projected after SHA-256 query hashing and PII redaction, so raw query text reaches neither stream — but every event carries hostname/username/uid metadata, which makes the stream a second sensitive local log, not a privacy improvement (file sink only, no HTTP; disable withnumbat.enabled: false). Fail-soft end to end, at the terminalaudit_loggernode, so it can never turn a good response into a 500. Seedocs/security-philosophy/numbat_secondary_evaluator.md. - Ships an optional OpenTweet X channel (
opentweet/, shippedenabled: false) — an out-of-band weekly poster; generation only through loopbackPOST /querywithuser_confirmed_online: false, default write an OpenTweet draft, schedulers that generate and never load. See OpenTweet Channel. - Tracks what the paid providers cost (
utils/spend.py,logs/spend.jsonlvialogging.spend_file) — every billed Grok/Claude call appends token counts, provider/model, and asourcetag separating the/queryplane from the agentic plane. Tokens are the ground truth; dollars are derived at read time. A separate stream fromaudit.jsonlthat never persists query text, prompts, or credentials, and is not a policy point;cyclaw-metricsprints spend windows, flags rate staleness, and compares CyClaw's rate table against xAI's owncost_in_usd_ticksso a wrong rate surfaces instead of accumulating. Seedocs/spend/README.md.
gate.py (FastAPI on 127.0.0.1:8787) runs the TrustedHostMiddleware Host
allowlist, then the per-IP rate limiter (60 req/min — it runs first, before
the injection filter), then the config-driven injection filter, then soul
init, and hands a GraphState to the 12-node LangGraph state machine in
graph.py. Routing is graph edges only — retrieve is the unconditional
entry, and every path converges on audit_logger before END. The numbered
plain-text version of this flow is CLAUDE.md §2 "The Map"; the
invariants it encodes are in INVARIANTS.md.
flowchart TD
A(["🌐 Client\nHTTP POST /query\nor MCP tool call"])
A --> B
subgraph GATEWAY ["gate.py — FastAPI 127.0.0.1:8787"]
B["TrustedHostMiddleware\nHost header allowlist"]
B --> C["Rate Limiter\n60 req/min per IP"]
C --> D["Prompt Injection Filter\n40 patterns · config-driven · lru_cache"]
D --> E["Build GraphState\nquery + user_confirmed_online"]
end
E --> F
subgraph GRAPH ["graph.py — LangGraph 12-node State Machine"]
F(["① retrieve\nChroma + BM25 + RRF"])
F --> G["② route_by_score\ntop_score ≥ 0.028?"]
G -->|"YES — local context"| X["③ guardrail_input\noffline rail · opt-in\npass-through when disabled"]
X -->|"blocked"| L
X -->|"passed · high score"| H["④ local_llm\nOllama :11434\nqwen3.8:27b-mlx"]
G -->|"NO — vault miss"| I["⑤ user_gate\nneeds_confirm = true"]
I -->|"confirmed=true + hybrid\n+ grok.enabled + provider=grok"| PG["⑥ pre_action_hook_grok\nsync · disabled=pass-through\nexit 2 → deny"]
PG -->|"exit 0 → allow"| J["⑦ grok_fallback\nxAI grok-4.5\ntriple-gated · not railed"]
I -->|"confirmed=true + hybrid\n+ claude.enabled + provider=claude"| PC["⑧ pre_action_hook_claude\nsync · disabled=pass-through\nexit 2 → deny"]
PC -->|"exit 0 → allow"| W["⑨ claude_fallback\nAnthropic claude-sonnet-5\ntriple-gated · not railed"]
I -->|"confirmed=false\nor offline mode"| X
X -->|"passed · vault miss"| K["⑩ offline_best_effort\nlocal LLM · no RAG gate"]
I -->|"confirmed=None — PAUSE\nreturn needs_confirm to the client"| L
H --> Y["⑪ guardrail_output\noffline rail · opt-in\ngrounding check: local_llm only"]
J --> Y
W --> Y
K --> Y
Y --> L
PG -.->|"deny"| L
PC -.->|"deny"| L
L(["⑫ audit_logger\nSHA-256 hash · PII redact\n→ logs/audit.jsonl"])
end
L --> M(["📤 QueryResponse\nanswer · sources · model_used\nretrieval_mode · needs_confirm"])
subgraph RETRIEVAL ["retrieval/hybrid_search.py"]
N["ChromaDB\nsemantic · 384-dim cosine"]
O["BM25Okapi\nkeyword · Porter stemming"]
P["RRF fusion\nk=60 · equal weighting"]
N --> P
O --> P
end
F <-->|"hybrid search"| P
subgraph SOUL ["utils/personality.py"]
Q["soul.md\nSHA-256 drift detection"]
R["SQLite / Postgres\nversion history · TTL prune"]
Q <--> R
end
H <-->|"soul preamble\n≤ 8000 chars"| Q
K <-->|"soul preamble"| Q
subgraph OOB ["Out-of-band — never imported by gate/graph/MCP"]
S["agentic/cli.py\nGitHub read ops"]
T["agentic/fsconnect/\nscoped FS read/write"]
U["sync/cli.py\nDropbox corpus pull"]
V["guardrails/\nNeMo rails skeleton"]
end
style GATEWAY fill:#1a3a5c,color:#ffffff,stroke:#4a90d9
style GRAPH fill:#1a3a2a,color:#ffffff,stroke:#4a9d5a
style RETRIEVAL fill:#3a2a1a,color:#ffffff,stroke:#d9904a
style SOUL fill:#3a1a3a,color:#ffffff,stroke:#d94ad9
style OOB fill:#2a2a2a,color:#aaaaaa,stroke:#666666,stroke-dasharray:5 5
style J fill:#5c1a1a,color:#ffffff
style W fill:#5c1a1a,color:#ffffff
style L fill:#1a1a3a,color:#ffffff
What the diagram compresses: HybridRetriever (retrieval/hybrid_search.py)
fuses ChromaDB (semantic, all-MiniLM-L6-v2, 384-dim cosine, CPU-only
embeddings) with BM25Okapi (keyword, Porter stemming) by RRF (k=60, equal
weighting) and carries per-chunk provenance metadata in every result. The
telemetry kill block runs before any SDK import, and the MCP server and the
indexer apply the same block.
CyClaw's soul mutation endpoints (/soul/propose, /soul/apply, /soul/reload, /soul/restore) require a Bearer API key. Without it they return HTTP 401 immediately — intentional fail-closed behavior.
All
/soul/*endpoints — includingGET /soul— require a validAuthorization: Bearer <key>token. Only/health,/query,GET /index/status,GET /auth/setup-status,POST /auth/login(issues the session itself; 503 whenauth.enabledis false), and the console pages (GET /,/static/*) are unauthenticated.POST /index/buildandPOST /auth/bootstrap-passwordcarry no credential either, but neither is open: each is gated on a loopback socket peer plus a same-origin check and returns 403 off-box —/index/build409 while a build is already running,/auth/bootstrap-password409 once the first admin password is set.POST /query, though credential-free by default, additionally carries an unconditional same-origin check — a cross-site browser request is rejected 403CROSS_SITE_BLOCKEDregardless ofauth.enabled; requests carrying neitherOriginnorSec-Fetch-Site(curl, PowerShell, schedulers) are unaffected.
Opting out entirely:
config.yaml'ssecurity.api_key_optional(defaultfalse) removes theCYCLAW_API_KEYrequirement from every route above and the harness console's guarded routes (agent run/push/publish included), for both apps at once — but only for requests arriving from this machine. The bypass is granted on the socket peer, so a remote caller still needs the real key no matter how the process was launched. Entries insecurity.allowed_hostsdo not change that: that list filters requestHostheaders and opens no listening socket. What would matter is the bind itself —gate.pyrefuses to start with a non-loopbackapi.hostwhile the flag istrue, andconfig-guard's C13 warns on that pair. Note it also does nothing under Docker: NAT rewrites the source address, so the container sees the bridge gateway rather than loopback and the routes stay key-gated (setCYCLAW_API_KEYin the container instead).
Set it for the current shell. Generate a real value instead of typing one —
openssl ships with macOS and every Linux distribution:
export CYCLAW_API_KEY="$(openssl rand -hex 20)"
echo "$CYCLAW_API_KEY"
uvicorn gate:app --host 127.0.0.1 --port 8787Persist it in your shell profile. macOS has defaulted to zsh since
Catalina, so that means ~/.zshrc unless you switched — check with
echo $SHELL. On bash, append it to the first existing login file in this
order: ~/.bash_profile, ~/.bash_login, ~/.profile; create
~/.bash_profile only when none exists, because macOS bash login shells do not
read ~/.bashrc (Linux bash does).
echo 'export CYCLAW_API_KEY="your-strong-local-secret"' >> ~/.zshrc # or the bash file above
source ~/.zshrcFull macOS walkthrough — including launching the harness console beside the
gateway and exercising every REST endpoint with curl — is in
setup-guide.md.
(Windows is the fallback path; CyClaw is developed and verified on macOS
first. Everything below still works and is CI-covered on windows-latest.)
$env:CYCLAW_API_KEY = "your-strong-local-secret" # current session only
uvicorn gate:app --host 127.0.0.1 --port 8787Persist it for the current user (writes the user environment permanently);
verify with echo $env:CYCLAW_API_KEY before launching:
[System.Environment]::SetEnvironmentVariable("CYCLAW_API_KEY", "your-strong-local-secret", [System.EnvironmentVariableTarget]::User)Windows Server, system-wide (all users, requires admin): the same call with
[System.EnvironmentVariableTarget]::Machine, or System Properties →
Advanced → Environment Variables → System variables → New. From cmd.exe:
set CYCLAW_API_KEY=your-strong-local-secret for the session and
setx CYCLAW_API_KEY "your-strong-local-secret" to persist.
Create .env in the repo root:
# Keys live here, never in config.yaml — config.yaml only names which
# provider is enabled; the key itself is read from the environment.
CYCLAW_API_KEY=your-strong-local-secret
GROK_API_KEY=your-xai-key-or-dummy-when-offline
ANTHROPIC_API_KEY=your-anthropic-key
Then tighten it — a hand-created file inherits the shell's umask (usually
0644, i.e. world-readable), and macos/invoke-cyclaw.sh refuses to source a
dotenv that is not 600 or 400 rather than load secrets from a file other
local accounts can read:
chmod 600 .envOn Windows, a hand-created file often inherits BUILTIN\Users read. Tighten it
the same way powershell/Invoke-CyClaw.ps1 requires before it will source the
file (owner-only; refuse Everyone / Users / Authenticated Users):
icacls .env /inheritance:r /grant:r "${env:USERNAME}:(R,W)"macos/setup-cyclaw-keys.sh and the harness console's /api panel both already
write ~/.CyClaw/.env at 600; only a hand-made file needs this step.
The Claude variable is ANTHROPIC_API_KEY, not CLAUDE_API_KEY —
llm/client.py and agentic/config.py both read the former, and nothing in
the codebase reads the latter. Setting the wrong name is silent: Claude simply
reports unavailable and the query falls back to a local answer.
Load it before launching:
# Bash / Zsh
export $(grep -v '^#' .env | xargs)
uvicorn gate:app --host 127.0.0.1 --port 8787# PowerShell
Get-Content .env | ForEach-Object {
if ($_ -match '^([^#=][^=]*)=(.*)$') {
[System.Environment]::SetEnvironmentVariable($Matches[1].Trim(), $Matches[2].Trim())
}
}
uvicorn gate:app --host 127.0.0.1 --port 8787CyClaw is loopback-only (127.0.0.1:8787) — the key never crosses a network. Still:
- Use at least 20 random characters:
openssl rand -hex 20(Linux/macOS) or[System.Web.Security.Membership]::GeneratePassword(24,4)(PowerShell) - Do not reuse a password from elsewhere
- Do not commit the key to Git (
.envis already in.gitignore) - Don't forget to set the api key via terminal on Mac or env var in Windows or the web app will not recognize it.
CyClaw ships two independent credential systems, and confusing them is the most common setup mistake:
| System | Secret | Guards | Toggle |
|---|---|---|---|
| Operator API key | CYCLAW_API_KEY env var (Bearer) |
/soul/*, /ops/*, /memory/*, /audit/summary, and the harness console's guarded routes |
Always on (fail-closed when unset); security.api_key_optional is the one deliberate loopback-peer bypass |
| Per-user auth | Per-account scrypt password → session cookie, or a named device token | POST /query and the console's user surface |
auth.enabled in config.yaml — ships false |
The per-user layer is gate_auth.py + utils/authn*; the full design is
docs/AUTHENTICATION_DESIGN.md. With
auth.enabled: false (the shipped default) POST /query takes no credential
and every /auth/* route answers 503, not 404 — route presence never
discloses whether the feature is on.
-
Set
auth.enabled: trueinconfig.yamland restartgate.py. The store is SQLite atauth.db_path(data/auth/cyclaw_auth.db);CYCLAW_AUTH_DB_URL— its own env var, deliberately not the personality subsystem'sCYCLAW_DB_URL— switches it to apostgresql://DSN. -
Set the first admin password. The bootstrap account is username
admin, created with no password. Use the terminal console's first-boot box on loopback, or post directly:curl -s -X POST http://127.0.0.1:8787/auth/bootstrap-password \ -H 'Content-Type: application/json' \ -d '{"password":"<a long passphrase>"}'
That route is loopback-peer + same-origin only — 403 from off-box, 409 once a password is set, 503 when
auth.enabledis false — and carries no credential on purpose: on a genuine first run there is nothing to present yet.GET /auth/setup-status(no credential, but same-origin-checked and rate-limited) reports{enabled, needs_password, username}so a console can tell first-boot from logged-out. -
Add the accounts operators actually use with the local-only
cyclaw-userconsole script (no HTTP route reaches it). Subcommands:add,list,role,disable,enable,passwd,token create|list|revoke; new accounts default tooperator:cyclaw-user add alice --role operator # prompts for the password cyclaw-user token create alice laptop # prints the token ONCE
Three roles, checked server-side on every request:
| Capability | admin |
operator |
audit |
|---|---|---|---|
POST /query |
yes | yes | no — 403 AUTH_ROLE_DENIED |
List users (GET /auth/users) |
yes | yes | no |
| Create user / reset another user's password | yes | yes, but never on an admin account |
no |
| Set role, delete user | yes | no | no |
| Disable / enable | yes | non-admins only | no |
Change own password (POST /auth/password) |
yes | yes | yes |
GET /auth/audit/summary |
yes | no | yes |
The last enabled admin is protected — disable, delete, and role-change
all refuse it, so an operator cannot lock the deployment out of its own admin
surface. GET /auth/audit/summary is the reduced, session-gated audit view, not
the API-key-gated GET /audit/summary.
Browsers get a cyclaw_session cookie plus a CSRF token that every mutating
/auth/* route requires in the X-CyClaw-CSRF header; POST /query is
deliberately CSRF-exempt because it takes a session or a bearer device token
and mutates no auth state. Programmatic clients send a named device token as
Authorization: Bearer <token> — displayed once, stored only as a hash,
revoked by label. A session dies at whichever of auth.session's two limits
comes first: idle_timeout_sec: 43200 (12 h, rolling) or
absolute_timeout_sec: 604800 (7 d, never resets). Failed logins back off per
account — the first 5 consecutive failures are free, then the delay doubles
from 2 s to a 900 s ceiling; no admin action is needed to recover.
Passwords are scrypt-hashed (utils/authn.py), and no password, session id,
CSRF token, or device token is ever written to audit.jsonl. Design and
rationale: docs/AUTHENTICATION_DESIGN.md.
Enabling auth.enabled does not by itself make a non-loopback bind safe or
permitted. gate.py's _require_loopback_bind still refuses a non-loopback
api.host, and the auth+TLS route past it is refused outright while
security.api_key_optional is true — that flag removes the CYCLAW_API_KEY
gate from /soul/*, /ops/*, and /memory/*, which per-user auth does not
replace. Set security.api_key_optional back to false first, then generate a
certificate with the bundled openssl wrapper (no new runtime dependency):
cyclaw-gen-cert --hostname "$(hostname)" --days 825Flags: --certfile, --keyfile, --hostname (defaults to the machine
hostname), --days (default 825), --san (repeatable extra SAN entry,
e.g. IP:10.0.0.5 or DNS:box.local), and --force — required to
overwrite an existing cert/key pair; without it the command refuses rather
than clobbering one. Read
docs/THREAT_MODEL.md before exposing the port —
CyClaw's stated scope is trusted-operator (single by default, a small trusted
group behind auth.enabled), loopback-bound, and single-tenant.
| Requirement | Version | Notes |
|---|---|---|
| Python | 3.12 | Primary supported runtime |
| Ollama | Any | Must be running on localhost:11434 |
| Model pulled in Ollama | — | qwen3.8:27b-mlx (default), mistral:7b, or any chat model |
| macOS (primary) | 14 Sonoma+ | Apple Silicon only. An Intel Mac cannot install this repo's pinned torch at all — no x86_64 wheel is published at that pin |
| Windows / Linux (fallback) | — | Both fully supported and CI-covered; they share the +cpu torch path below |
Optional local-backend failover. Ollama is the primary local backend;
CyClaw can also fail over to LM Studio (or any OpenAI-compatible loopback
server) when Ollama isn't reachable. Off by default — set
models.local_llm.fallback.enabled: true and fill in fallback.model with
the LM Studio id (no Ollama-style name:tag colon). A short probe
(fallback.probe_timeout_sec, default 1.5s) tries Ollama first and LM Studio
second, LocalLLMClient and /health share the choice, and it re-probes if
neither answered the first time (llm/client.py's resolve_local_backend).
Prefer GHCR when you want a prebuilt linux/amd64 runtime without a local pip install.
Pull ghcr.io/cgfixit/cyclaw and run with the existing compose hardening (loopback publish,
read-only rootfs, seccomp builtin). Full operator guide: docs/DOCKER.md.
export CYCLAW_IMAGE_TAG=1.9.0
docker compose pull && docker compose up -d
curl -sS http://127.0.0.1:8787/healthNative install (below) remains the primary path for Apple Silicon and for the coding harness.
macOS is the primary platform and needs a different torch step than
Windows/Linux: the +cpu local-version wheel does not exist for macOS, and
both manifests hardcode that pin, so the generic block fails twice on a Mac.
git clone https://github.com/CGFixIT/CyClaw
cd CyClaw
python3.12 -m venv .venv
source .venv/bin/activate
# 1) torch FIRST, and PLAIN — no +cpu suffix, no --index-url override.
# Apple Silicon has one arm64 wheel; there is no CPU/CUDA build to pick between.
pip install "torch==2.13.0"
# 2) Everything else, from a requirements.txt copy with the torch and
# PyTorch-index lines stripped out, and a constraints.txt copy that keeps
# torch pinned minus the +cpu suffix (--ignore-installed reinstalls torch
# too, so an unconstrained copy floats it) — the same thing CI's
# macos-latest leg runs.
grep -v -e '^torch==' -e '^--extra-index-url https://download.pytorch.org' \
requirements.txt > /tmp/requirements-macos.txt
sed 's/^\(torch==[0-9][0-9.]*\)+cpu$/\1/' constraints.txt > /tmp/constraints-macos.txt
pip install -r /tmp/requirements-macos.txt -c /tmp/constraints-macos.txt \
--ignore-installed PyYAMLPrefer a script? bash ./macos/setup-cyclaw.sh is the single operator-facing
entry point (offers to clone, asks its few choices once, then runs
macos/setup-from-clone.sh: installer + Keychain keys + Ollama check +
retrieval index + both servers). bash ./macos/install-cyclaw.sh is the
installer alone — it handles the torch difference but skips the Ollama / index /
API-key steps, so the gateway stays degraded (503 on /query) until you do
them. Flags, privacy notes, and tradeoffs:
macos/README.md and
setup-guide.md.
git clone https://github.com/CGFixIT/CyClaw
cd CyClaw
python3.12 -m venv .venv
source .venv/bin/activate
# 1) CPU-only torch first (CVE-2025-32434 fixed in 2.6.0; 2.13.0 is within the patched range)
pip install torch==2.13.0+cpu --index-url https://download.pytorch.org/whl/cpu
# 2) The rest, pinned to the verified transitive tree. --ignore-installed PyYAML
# avoids a resolver conflict with a system PyYAML some platforms preinstall.
pip install -r requirements.txt -r requirements-test.txt -c constraints.txt --ignore-installed PyYAMLFor a from-scratch dev box or a full manual smoke test — Postgres/pgvector, NeMo Guardrails, dev/test tools, and both cloud providers — substitute step 2 with:
pip install -e ".[all]" -c constraints.txtmkdir -p index logs
export GROK_API_KEY=dummy^ # optional — gate.py/the retriever/logger self-create these on first run
data/personality/soul.md ships committed to git with CyClaw's real
personality already in place — do not recreate it from a placeholder on a
fresh clone. If it's ever deleted, PersonalityManager self-heals with a
generic default, but that's a recovery path, not the normal first-run state.
CyClaw ships two independent local web apps. Neither starts the other; run whichever you need, or both in separate terminal tabs.
# The RAG gateway — serves static/terminal.html at / plus the whole REST API
python -m retrieval.indexer # once, before the first /query
uvicorn gate:app --host 127.0.0.1 --port 8787 # → http://127.0.0.1:8787
# The coding-harness console — serves static/harness.html
python -m harness.server # → http://127.0.0.1:8790The cyclaw-* short names need a self-install. cyclaw-server,
cyclaw-harness, cyclaw-index, cyclaw-mcp, cyclaw-metrics,
cyclaw-clear-cache, cyclaw-user, and cyclaw-gen-cert are
[project.scripts] shims that pip writes only when the project itself is
installed; requirements.txt has no self-install line, so add
pip install -e . -c constraints.txt if you want them. The python -m … forms
always work and are what both shipped launchers use.
Override the harness port with CYCLAW_HARNESS_PORT=8795 python -m harness.server, not a CLI flag; it refuses to bind a non-loopback address.
uvicorn harness.server:app also works (the module-level app is built
lazily, so importing never touches ~/.CyClaw), but prefer -m: the uvicorn
form bypasses the bind-address guard, so --host 0.0.0.0 opens a public socket
that python -m harness.server would have refused — TrustedHostMiddleware
still rejects non-loopback Host headers, but that is one layer fewer.
Open / for the terminal UI and /health for readiness. The terminal exposes five operator consoles — Soul, Sync, Agentic, Filesystem, and SQL — the latter four calling POST /ops/sync, /ops/agentic, /ops/fsconnect, and /ops/sqlconnect (API-key gated, rate-limited, audited).
Every gateway route with a copy-pasteable curl invocation, plus what each
status code means, is in
setup-guide.md.
CyClaw/
├── gate.py
├── gate_ops.py # /ops/* endpoints (sync/agentic/fsconnect/sqlconnect subprocess shims)
├── gate_auth.py # /auth/* endpoints — session cookie + CSRF, bearer device tokens
├── gate_memory.py # /memory/* + /query/export/html — optional, default-off memory admin surface
├── graph.py
├── metrics.py # audit.jsonl analyzer + spend.jsonl Spend section (cyclaw-metrics)
├── config.yaml # single source of truth
├── README.md
├── mcp_hybrid_server.py # retrieval-only MCP server
├── memory/ # optional facts + episodes store (default-off)
│ ├── README.md # package pointer (not docs/memories/)
│ ├── store.py # SQLite + FTS5 backend for facts/episodes
│ ├── policy.py # propose/apply governance (reason required, injection scan)
│ ├── retrieval_adapter.py # optional fusion hook into hybrid retrieval
│ ├── mirror.py # episode staging (lazy, non-fatal)
│ ├── consolidation.py # stub — stay false in v1
│ └── models.py # typed request/response shapes
├── agentic/ # out-of-band GitHub context + governed registry (see README.md)
│ ├── cli.py
│ ├── context.py
│ ├── gh_client.py
│ ├── registry.py
│ ├── writer.py # gh pr create --draft; armed flag, held by agentic.enabled
│ ├── real_repo_loop.py # (v1.9 P10) clone → plan → patch → verify → human decides → commit
│ ├── unslop_bridge.py # (v1.9.x) offline slop-detection probe for real_repo_loop; default off
│ ├── vendor/unslop/ # vendored offline AI-writing-tell scanners (suggest.py); no network calls
│ ├── executor/ # sandboxed argv-list check runner; required fail-closed hard sandbox (hard_sandbox.py)
│ ├── fsconnect/ # (v1.8) local/SMB filesystem connector
│ │ ├── cli.py
│ │ ├── client.py # scoped reads (fs_list/stat/read/grep)
│ │ ├── pathsafe.py # held-handle containment core (POSIX + Windows reads)
│ │ ├── writer.py # gated, atomic writes (default-disabled)
│ │ └── indexer.py # toggleable RAG-corpus indexing of the share
│ ├── sqlconnect/ # (v1.8) read-only SQL scaffold (Postgres/MSSQL)
│ │ ├── cli.py
│ │ └── client.py # SELECT-only query guard, env-only DSN
│ ├── harness_optimizer/ # (v1.9) governed better-harness-style optimizer scaffold
│ │ ├── core.py # Experiment/Surface/RunReport/CandidateDecision models
│ │ ├── proposer.py # scoped train/holdout workspace builder
│ │ ├── mcp/tools.py # audited, symlink-hardened proposer workspace tools
│ │ └── governance.py # visible-case-hardcoding + governance-finding gates
│ └── deepagent_github/ # (v1.9) workspace tools + cloud planner; DeepAgents subgraph retired
│ ├── repo_workspace.py # live: jailed workspace tools (clone/read/write/commit/push) used by real_repo_loop
│ ├── chat_client.py # live: cloud-provider planner adapter (Grok/Claude)
│ ├── builder.py # retired DeepAgents subgraph (2026-07-31) — kept, not deleted
│ ├── permissions.py # phase-5 no-write policy refusal
│ └── subagents.py # validated SubAgent specs, no bare-string tools
├── guardrails/ # (v1.8) opt-in rails; graph nodes via guardrail_bridge
│ ├── README.md
│ ├── cli.py
│ ├── config.py
│ ├── integration.py # soft-imports nemoguardrails; degrades gracefully
│ ├── rails.py # offline heuristic rails (injection/soul/grounding)
│ ├── metrics.py # separate logs/guardrails.jsonl stream (hashes only)
│ └── config/ # NeMo config.yml + rails.co (Colang flows)
├── harness/ # (v1.9) coding console on 127.0.0.1:8790 (see harness/README.md)
│ ├── README.md # slash-command usage (/goal /loop /skills /tools /web)
│ ├── server.py # FastAPI control plane (cyclaw-harness)
│ ├── sessions.py # JSON session store with per-session token tallies + /goal
│ ├── ollama.py # loopback-only OpenAI-compatible /v1 chat client
│ ├── config.py # ~/.CyClaw (or %USERPROFILE%\.CyClaw) home layout
│ ├── prompts.py # ponytail + karpathy (+ optional soul, /goal, /web extract)
│ ├── registry_view.py # merged catalog (AST-parses MCP tools; I6)
│ ├── tools_view.py # /tools wiring diagram (live routes vs MCP catalog)
│ ├── skills_view.py # /skills wiring diagram (prompt + agent-check vs catalog)
│ ├── web_search.py # allowlist-only GET; off by default; no search engine
│ ├── agent_policy.py # check-profile allowlist — console sends profile names, never argv
│ ├── agent_routes.py # the seven /api/agent/* routes (6 of the 29 guarded)
│ ├── auth_routes.py # the /api/auth/* console auth surface
│ ├── env_keys.py # allowlisted dotenv secret store ($CYCLAW_HOME/.env; file-only)
│ ├── memory_notes.py # console /memory notes store
│ └── schemas.py # request models
├── telegram/ # (v1.9) optional Telegram channel (out-of-band), shipped enabled: false
│ ├── cli.py
│ ├── client.py # Bot API client — outbound notify + long-poll inbound chat
│ ├── config.py # loads config.yaml's `telegram:` block
│ ├── runner.py # long-poll loop; answers via loopback POST /query only
│ ├── state.py # T3 hybrid-confirm consent state (default off)
│ ├── media.py # T4 attachment staging via agentic/fsconnect (default off)
│ └── ratelimit.py
├── opentweet/ # optional X channel (out-of-band), shipped enabled: false
│ ├── cli.py # status / test / post / schedule-plist / schedule-task
│ ├── client.py # loopback /query + OpenTweet REST; trust_env=False
│ ├── config.py # loads config.yaml's `opentweet:` block
│ ├── runner.py # topic → query → validate → draft/schedule
│ └── selftest.py
├── powershell/ # Windows installer/launcher for the harness
│ ├── Install-CyClaw.ps1 # home + venv + PATH shim + profile function
│ ├── Invoke-CyClaw.ps1
│ └── Uninstall-CyClaw.ps1
├── macos/ # macOS/Linux installer/launchd glue (see macos/README.md)
│ ├── setup-cyclaw.sh # single entry point: clone? + one-shot + optional start/browser/autofill
│ ├── setup-from-clone.sh # one-shot after git clone (Apple Silicon)
│ ├── install-cyclaw.sh
│ ├── uninstall-cyclaw.sh
│ ├── invoke-cyclaw.sh # gate :8787 + harness :8790
│ ├── setup-cyclaw-keys.sh # Keychain + ~/.CyClaw/.env (never config.yaml)
│ ├── setup-fsconnect.sh # confined ~/CyClaw-FS list/stat/read
│ ├── cyclaw-keychain-*.sh # Keychain inject/store for launchd jobs
│ ├── generate_service_plist.py # supervised gate/harness LaunchAgent — requires --confirm + --reason, never loads
│ └── LaunchAgents/ # templates only — never auto-loaded
├── .claude/ # local operator workflows and prompts
│ ├── commands/
│ ├── hooks/
│ ├── memory/
│ ├── patterns/
│ ├── rules/
│ ├── skills/
│ ├── tools/
│ └── utility-prompts/
├── retrieval/
│ ├── indexer.py
│ ├── hybrid_search.py
│ ├── embeddings.py
│ ├── stemmer.py
│ ├── vector_store.py # pluggable: embedded ChromaDB (default) or pgvector; sole Chroma chokepoint
│ └── clear_cache.py # dry-run-by-default embedding-cache cleaner (cyclaw-clear-cache)
├── llm/
│ └── client.py
├── sync/ # optional Dropbox corpus sync
│ ├── cli.py
│ ├── runner.py
│ └── scheduler.py
├── utils/
│ ├── sanitizer.py
│ ├── logger.py
│ ├── personality.py
│ ├── health.py
│ ├── ratelimit.py
│ ├── launchd_plist.py # stdlib-only plist builder shared by the telegram / fsconnect / opentweet / generate_service_plist generators (sync.scheduler builds its own)
│ ├── guardrail_bridge.py # only bridge from graph.py to guardrails/ (never a direct import)
│ ├── numbat_emitter.py # derived Numbat NDJSON stream: action-plane emits + mainline audit projection
│ ├── spend.py # append-only Grok/Claude token ledger (logs/spend.jsonl); dollars derived at read time
│ ├── sequence_detect.py # offline forensic join of audit.jsonl + spend.jsonl on query_hash (CLI only)
│ ├── authn.py # per-user auth primitives: scrypt hashing, lockout arithmetic, id generation
│ ├── authn_store.py # users/sessions/device_tokens backend (CYCLAW_AUTH_DB_URL)
│ ├── authn_manager.py # AuthManager — ties authn.py + authn_store.py together; no HTTP awareness
│ ├── authn_cli.py # cyclaw-user console script (local-only by construction)
│ ├── gen_cert.py # cyclaw-gen-cert — self-signed cert + key with hostname/LAN SAN
│ └── telemetry_kill.py # shared kill block — applied by gate.py, mcp_hybrid_server.py, retrieval/vector_store.py
├── schemas/ # Pydantic API models (api.py; extra='forbid', strict)
├── scripts/ # install-githooks.sh, check-pr-template.sh, measure_local_llm_throughput.py
├── deploy/ # apparmor/ falco/ seccomp/ container-hardening profiles (all opt-in)
├── tests/
├── docs/
├── static/
├── data/
│ ├── corpus/
│ ├── personality/
│ └── agentic/ # skills_registry.json — governed store, ships empty
└── .github/workflows/
Every top-level package and directory above now carries its own README.md
(map + traps + links to its authoritative doc); the tree omits most of them
for brevity.
An optional, out-of-band rclone-backed pull sync mirrors a Dropbox corpus
into data/corpus/ without touching gate.py, graph.py, or the MCP request
path. It carries safety fuses (max_delete, max_transfer), an OS-backed
single-instance lock (fcntl.flock / msvcrt.locking) that keeps a scheduled
run and a manual run from racing and releases even if the process dies, audit
logging of changed corpus files, an optional reindex trigger, and scheduler glue
for cron / Windows Task Scheduler plus an opt-in Darwin-only launchd backend
(sync.scheduler_backend: "launchd", schedule_frequency daily/weekly/monthly)
that generates the plist and prints the launchctl bootstrap command — never
loading it itself.
python -m sync.cli setup # first-run bootstrap; then: test | sync --dry-run | sync | status | schedule | unscheduleThe Sync Console panel drives the same actions via POST /ops/sync
(loopback-only, API-key gated, audited). Full setup and scheduling:
docs/! How-To-Guides/Dropbox_Sync_Guide.md;
module internals (lock lifecycle, exit codes, error taxonomy):
docs/SYNC_README.md.
CyClaw's scheduled and supervised jobs on macOS run through generated launchd
LaunchAgents with one uniform posture: every generator writes a plist from
real resolved install paths and prints the exact launchctl bootstrap command
— none of them ever loads the agent itself. Loading a background job is
always a separate, explicit operator action.
- Secrets never land in a plist. Token-bearing jobs chain
macos/cyclaw-keychain-env.sh, which fetches the secret from the Keychain at process start, exports it, andexecs the real command — failing closed (nothing launched) if the item is missing or empty. Store secrets first withmacos/cyclaw-keychain-set.sh, a no-echo prompt driven bysecurityitself so the secret never appears in any argv; the item is trust-pinned with-T /usr/bin/security. - Scheduled jobs — Dropbox sync (see Dropbox Corpus Sync),
Telegram poll/health, fsconnect trash emptying, and OpenTweet — each a
generate-only
*-plistsubcommand (commands below). - Supervised services (highest risk) —
macos/generate_service_plist.pywrites a KeepAlive LaunchAgent forgate.pyor the harness console. Because that turns a loopback server into an always-on listener that survives reboot, it refuses to write without--confirmand a non-empty--reason(the reason-required idiom soul mutations use). Restart-on-crash only; a cleanlaunchctl stopstays stopped. - Uninstall symmetry —
macos/uninstall-cyclaw.shunschedules any registered sync job and boots out + removes landed CyClaw LaunchAgents by label (telegram-poll,telegram-health,fsconnect-trash,gate,harness,keys-rotate,opentweet), so no background job outlives the install. Sync's own launchd job is owned bysync.cli unschedule.
Core commands
bash macos/cyclaw-keychain-set.sh com.cgfixit.cyclaw.telegram-bot-token # store a secret (TTY prompt)
python -m telegram.cli poll-plist # Darwin-only; generates, never loads
python -m telegram.cli health-plist
python -m agentic.fsconnect.cli trash-empty-plist
python -m opentweet.cli schedule-plist # Darwin-only; generates, never loads
python macos/generate_service_plist.py --service gate \
--reason "keep the RAG server up across reboots" --confirm
python macos/generate_service_plist.py --service harness \
--reason "keep the coding console up across reboots" --confirmScript-by-script reference: macos/README.md. Design and
phase ledger: docs/work/MACOS_LAUNCHD_INTEGRATION_PLAN.md.
CyClaw now includes a concise, governed agentic layer for local operator workflows. It is opt-in, disabled by default, and fully out-of-band: it is never imported by gate.py, graph.py, or mcp_hybrid_server.py. The coding console (harness/) is a sibling package, not a subpackage; data/agentic/skills_registry.json is a governed store that ships empty (the harness reads it, apply-skill writes it). Package guide: agentic/README.md.
What it adds: read-only GitHub context through the gh CLI (invoked as an
argv list, never via a shell; no GitHub token is stored or forwarded by
CyClaw), a governed local skills registry with explicit human gating, and the
operator workflows under .claude/ (.claude/README.md).
All agentic reads, refusals, and registry changes are audit logged.
The GitHub write path (gh pr create --draft) is implemented and its
code-level gate EXECUTION_ENABLED ships True (operator-signed 2026-08-07),
as do mode: "write" and writes_enabled. The layer master switch
agentic.enabled still ships false, so a default checkout cannot open a PR —
that switch, plus a per-call reason and confirm, is what refuses. Procedure
and the CYCLAW_AGENTIC_WRITE_DISABLE rollback:
docs/agentic/GITHUB_WRITE_ENABLEMENT.md.
agentic:
enabled: true # the one edit this block asks you to make
repo: "cgfixit/CyClaw"
mode: "write" # ships open since 2026-08-07
writes_enabled: true # ships open since 2026-08-07
gh_min_version: "2.40.0"
registry_path: "data/agentic/skills_registry.json"python -m agentic.cli status
python -m agentic.cli context --repo
python -m agentic.cli context --pr 123
python -m agentic.cli context --issue 45
python -m agentic.cli test
python -m agentic.cli propose-skill --name deploy --desc "..." --body-file s.md --reason "draft"
python -m agentic.cli apply-skill --name deploy --desc "..." --body-file s.md --reason "add deploy runbook" --confirmThe Agentic Console panel drives these from the terminal UI via
POST /ops/agentic; skill-Apply is refused under shipped defaults by
agentic.enabled: false (the CLI no-ops while it is off), and a per-call
reason + --confirm remain mandatory once it is on.
v1.8 extends the agentic layer beyond GitHub to local data, for the regulated or security conscious use case where AI use is compliance heavy. All three connectors are opt-in, disabled by default, and fully out-of-band — never imported by gate.py, graph.py, or mcp_hybrid_server.py, so the six security invariants hold by construction. While disabled, their CLIs are a pure no-op (exit 0).
Scoped reads and separately-gated writes over a local or SMB share,
sharing one held-handle security core (pathsafe.py): POSIX descends with
openat / O_NOFOLLOW from a held root fd; Windows read/list/stat locks the
canonical root ancestry against rename, opens once with CreateFileW, verifies
GetFinalPathNameByHandleW containment, and consumes that same handle —
Windows writes remain hard-refused. UNC, NTFS alternate data streams
(file::$DATA), \\?\ / \\.\ device paths, .. traversal, and symlink /
reparse traversal are denied; segment-aware containment closes
CVE-2025-53110 (sibling-prefix), and held-handle authority prevents
name-reopen junction swaps.
- Reads (
fs_list/fs_stat/fs_read/fs_grep/fs_glob/fs_largest) are confined toallowed_roots, audited, capped at 5 MiB, and content-scanned (OWASP ∪banned_patterns, advisory). - Writes (
fs_write/fs_append/fs_mkdir/fs_move) shipwrites_enabled: false; confined to a separatewritable_rootslist; gated by a humanreason(+--confirmfor destructive ops); atomic (tmp+os.replace); content-agnostic (never calls the LLM). A code-levelFS_WRITE_HARD_DISABLEkill switch forces dry-run regardless of config. - Toggleable RAG-corpus indexing of the share (
index_enabled, dry-run default) stages eligible files into the corpus and triggers a reindex subprocess — a generate → write → index loop without importing retrieval.
python -m agentic.fsconnect.cli status
python -m agentic.fsconnect.cli read --path "<path>" # scoped read
python -m agentic.fsconnect.cli grep --path "<path>" --pattern "<pattern>"
python -m agentic.fsconnect.cli largest --path "<dir>" --top 20 --min-bytes 1048576
python -m agentic.fsconnect.cli write --path "<path>" --reason "..." # dry-run unless writes_enabled
python -m agentic.fsconnect.cli index --apply # stage share → corpus
python -m agentic.fsconnect.cli test # pre-flight self-testEnable in config.yaml:
fsconnect:
enabled: true
allowed_roots: ["/srv/share"] # REQUIRED when enabled; existing dirs
max_file_bytes: 5242880 # 5 MiB read cap
largest_max_entries: 100000 # truthful traversal ceiling per largest command
writes_enabled: false # master write switch (dry-run plans while false)
writable_roots: [null] # null => ~/CyClaw-FS (macOS) | /var/lib/cyclaw-fs (Linux) | C:\CyClaw-FS
max_write_bytes: 10485760 # 10 MiB write cap
index_enabled: false # toggle RAG-corpus indexing of the shareRead-only on-prem SQL (Postgres / MSSQL), enforced three ways: a
SELECT/WITH-only query guard (rejects DDL/DML, stacked statements, and
comment-hidden keywords by scanning a quote-stripped copy), a session-level
read-only transaction, and a hard allow_write: false. The DSN comes from an
environment variable only (CYCLAW_SQL_DSN); drivers (psycopg / pyodbc)
import lazily. The quote-stripping scan is a single left-to-right pass that
gives '...', "...", [...], and Postgres $tag$...$tag$ quoting the same
precedence the database does — an earlier regex-alternation version could be
fooled by a quote nested inside a different quoting form (e.g. $$'$$) into
treating a stacked DROP as part of one SELECT; the other two layers were
never affected.
python -m agentic.sqlconnect.cli status
python -m agentic.sqlconnect.cli schema # list table schemas (read-only)
python -m agentic.sqlconnect.cli query --table public.users # bounded preview
python -m agentic.sqlconnect.cli testsqlconnect:
enabled: false
driver: "postgres" # "postgres" | "mssql"
dsn_env: "CYCLAW_SQL_DSN" # DSN from this env var only
statement_timeout_ms: 5000
max_rows: 1000
allow_write: false # reserved; v0.1 cannot write regardlessReports best-effort local host metadata and reads the OS's existing ARP/neighbor cache. No ping, port probe, subnet sweep, packet send, scheduling, or request-path integration; every returned IPv4 address is filtered through operator-supplied CIDRs that must be subnets of RFC1918 or loopback space.
python -m agentic.netconnect.cli status
python -m agentic.netconnect.cli self
python -m agentic.netconnect.cli arp
python -m agentic.netconnect.cli testnetconnect:
enabled: false
allowed_cidrs: ["192.168.1.0/24"]
allowed_net_ops: [self, arp]
command_timeout_sec: 5
max_neighbors: 512An opt-in content-safety layer in guardrails/ (package README). Absence of the guardrails: block, or enabled: false (the shipped default), is a pure no-op. When enabled, utils/guardrail_bridge.py wires two visible graph.py nodes — guardrail_input (after route_by_score) and guardrail_output (after generation; grounding check on the local_llm path only) — still defense-in-depth only, never a routing authority: the graph's own edges decide where a blocked query goes. gate.py / graph.py / mcp_hybrid_server.py never import guardrails directly (I6). Status table: docs/NeMo/README.md.
nemoguardrails is an optional dependency: the layer soft-imports it and,
when absent, degrades to offline heuristic rails that need no second LLM call —
an input rail (prompt-injection marker scan + soul/identity-mutation intent
detection, the content-layer arm of the Soul-Governance invariant) and an
output rail (token-overlap grounding check against the retrieved
context, flagging likely-hallucinated answers below hallucination_threshold).
When nemoguardrails is installed, the same Python checks back the live
NeMo actions via the Colang flows in guardrails/config/rails.co, so offline
heuristics and live rails never drift. Decisions go to a separate metrics
stream (logs/guardrails.jsonl) that stores only SHA-256 hashes.
python -m guardrails.cli status | check "your query here" | metrics | testConfig keys (guardrails.enabled, engine, model, hallucination_threshold,
metrics_path) and their shipped values live in config.yaml; the status
table, phased history, and rail semantics are in
docs/NeMo/README.md.
A governed, opt-in, disabled-by-default, out-of-band scaffold for two
related capabilities — agentic/harness_optimizer/ (a better-harness-style
optimizer with train/holdout scoring and a hard acceptance gate) and
agentic/deepagent_github/ (a LangChain Deep Agents-backed local GitHub coding
harness, lazily importing deepagents only when enabled). Phases 0–9 are
implemented and tested (tests/test_agentic_harness_*.py; phases 6–9 landed in
PR #515, 2026-07-13, documented in
docs/work/DEEP_AGENT_HARNESS_PHASES_6_9.md; full plan and phase ledger in
docs/work/GITHUB_DEEP_AGENT_HARNESS_OPTIMIZER_PLAN.md).
Superseded 2026-08-01. Phase 9's security gate was subsequently satisfied and P10 landed, so "not authorization to add an executor" no longer describes the current tree: a sandboxed verification executor (
agentic/executor/) and a draft-PR write path (agentic/writer.py::execute_write) both exist, and the live real-repo coding pipeline isagentic/real_repo_loop.py— not thedeepagents-backed graph this section describes. Both new capabilities still ship disarmed. See GitHub Agentic Coding Harness below for what is actually wired today.
Every gate below the master agentic.enabled switch that arms a run
(deepagent_github.enabled, allow_deepagents_dependency,
allow_filesystem_write_tools, allow_shell_execution, allow_github_writes,
harness_optimizer.enabled) ships false in config.yaml; the three
cloud-provider switches ship armed but unreachable behind those masters (see the
next-but-one section). While disabled, nothing under either package is reachable
from agentic.cli, and no deepagents/langchain optional dependency is imported.
A grok-build-style local coding console, shipped as the strictly out-of-band
harness/ package (Windows merged 2026-07-22; macOS/Linux port merged
2026-08-01). Like agentic/, sync/, and guardrails/, it is never imported
by gate.py, graph.py, or mcp_hybrid_server.py and never imports them
(invariant I6).
harness/ itself is pure Python with no OS branch in its request-handling
path — same routes, same slash commands, same security posture everywhere.
Only the install/launch glue is platform-coupled, which is why there are two
sibling script trees (powershell/, macos/) rather than one abstraction.
- Launch:
python -m harness.serverserves the console athttp://127.0.0.1:8790(static/harness.html); loopback-only bind, non-loopback hosts refused.gate.pykeeps:8787. (cyclaw-harnessis the same entry point afterpip install -e .— see Installation.) - Install: Windows —
powershell -ExecutionPolicy Bypass -File .\powershell\Install-CyClaw.ps1(home, venv,cyclaw.cmdPATH shim, profile function). macOS / Linux —bash ./macos/setup-cyclaw.sh(single entry point) orbash ./macos/install-cyclaw.sh(installer alone; branches onuname -sto install the correct plain torch on macOS, the step a hand-install most often gets wrong; bash 3.2 and zsh, BSD userland, no Homebrew dependency). Uninstall withbash ./macos/uninstall-cyclaw.sh(--remove-home,--remove-fsconnect); it also removes every landed CyClaw LaunchAgent (see macOS launchd & Keychain). - Chat: talks to the local model through the OpenAI-compatible
/v1endpoint fromconfig.yaml'smodels.local_llm.base_url— no keys, no login, offline. Every reply shows token counts; sessions persist as human-inspectable JSON with atomic writes. - Reuse, not duplication: GitHub actions go through the same
utils.ops_runnersubprocess shim as/ops/agentic(read mode by default);/skillsand/toolsare wiring diagrams (what this console actually injects, runs, or has registered) — not a dump of every file on disk. MCPhybrid_searchappears under/tools allas catalog-only; the console does not invoke it./goalis session data in the system prompt;/loopis a human-gated sequence of/api/chatturns toward that goal and never starts/api/agent/*./webis allowlist-only GET, off by default, no search engine. The governeddata/agentic/skills_registry.jsoncatalog is still merged intoGET /api/registry(read-only here). The system prompt is composed from the repo's ownponytail+karpathy-guidelinesskills, with the governed soul appended read-only when enabled, plus optional/goaland/web injectextracts.
Full setup, slash-command reference, home layout, and security posture:
harness/README.md,
docs/HARNESS_POWERSHELL.md (Windows), and
docs/HARNESS_MACOS.md (macOS/Linux — only what
genuinely differs: install glue, the torch build, git-credential-osxkeychain,
and the note that pathsafe.ScopedRoots' POSIX openat/O_NOFOLLOW
containment is the stronger branch).
The real-repo coding pipeline: clone → plan → patch → verify → human decides →
commit, with pushing and opening a draft PR as two further, separate decisions.
Driven by agentic/real_repo_loop.py, which fuses three previously-independent
pieces — the planner's model call, a jailed real clone
(agentic/deepagent_github/repo_workspace.py), and the sandboxed verification
executor (agentic/executor/). Out-of-band like every other agentic feature:
never imported by gate.py, graph.py, or mcp_hybrid_server.py (invariant I6).
It ships held, not disarmed. The three switches that gate whether a run
happens at all — agentic.enabled, deepagent_github.enabled,
allow_git_write_tools — ship false. The write-path constant
agentic/writer.py::EXECUTION_ENABLED and the cloud switches (mode: "write",
writes_enabled, allow_cloud_providers, both providers) ship open since
the signed enablement of 2026-08-07, so on a default checkout it is the master
switches plus a per-call reason/confirm that refuse (see "already armed,
waiting on the master switches" below).
real-repo-run-plan(optional, two-stage) asks a model for a plan and prints it — clones, writes, and commits nothing. You read and edit the plan and feed it back with--plan-file, so one model plans, a human approves, and another model codes against the approved text. The plan is injection-scanned on load, truncated at 6,000 chars, and its SHA-256 is recorded on the run.real-repo-runclones the configured repo into a jailed workspace, asks the planner for whole-file replacements, writes them, runs the selected checks, and stops before committing (status: pending_decision; a run that never passes reportsexhausted).real-repo-run-decide --decision approvecommits locally;rejectdiscards. Neither pushes.real-repo-run-pushputs theclaude/*branch on origin.real-repo-run-publishopens a draft PR (gh pr create --draft).real-repo-run-discardreclaims a decided or orphaned run's clone (rejectandexhaustedfree theirs immediately; only an approved run keeps its clone, since push and publish still need it).
Each escalation is its own command and its own decision, deliberately not
folded into approve.
- Diff-scope gate. A candidate that writes into any of
config.yaml'sagentic.deepagent_github.protected_write_paths(the tests, CI, lint, and config files that judge the candidate's own acceptance — the classic reward-hacking failure of a make-the-checks-pass loop) is refused outright, and writes are budget-capped (max_write_budget_bytes). - Two scanners, two questions, on the same bytes. Proposed content gets an
injection scan (is this trying to talk to a model?) and a code-shape
scan (
inspect_code_shape;scan_code_shapeshipstrue) that matches combinations — a secret path plus network egress, a decode plus dynamic exec, a socket plus fd-dup or a shell path, a pipe-to-shell — because a working key-exfiltration payload contains no injection phrase at all. Every hit is CRITICAL and refuses the candidate. - Verification runs as argv-list subprocesses, never a shell:
cwdpinned to the clone, a scrubbed env allowlist (PATH,LANG,LC_ALL,PYTHONPATH,VIRTUAL_ENV,PYTHONIOENCODING) plus a disposableHOME/USERPROFILE, forcedNO_PROXY=*/PIP_NO_INDEX=1, and a 120s per-check timeout. Every non-empty check list runs inside a required, fail-closed hard sandbox (hard_sandbox.py: Windows Job Object withKILL_ON_JOB_CLOSE, Darwinsandbox-execdenying network and off-cwd writes, Linuxunshare --net) — a missing binary or failed capability probe raisesHardSandboxUnavailable, with no silent fallback. Residual limits (no microVM; Windows is a process-tree kill, so sockets keep working there) are indocs/THREAT_MODEL.md's executor amendments. - The console sends check-profile names, never argv —
harness/agent_policy.pyresolves them against a fixed allow-list (pytest,ruff,invariant-guard,config-guard); a body that could carry argv would make an authenticated route a remote shell. push_branchpasses no credential. Its env allowlist deliberately excludesGH_TOKEN/GITHUB_TOKENbecause that environment is shared with the executor; it authenticates only via a HOME-resident credential helper (gh auth setup-git). Branch names are forced into the PR-template vendor namespaces (utils/agent_identity.py) andrun_idis validated as 32-char lowercase hex before it can become an argv element.- Optional offline slop-detection nudge (
unslop.enabled, shipsfalse): hits become feedback appended to the next planning prompt, never a gate.
The block below is what config.yaml ships. The three switches that gate
whether a run happens at all — agentic.enabled, deepagent_github.enabled,
allow_git_write_tools — ship false; the three cloud switches ship true,
armed alongside models.grok / models.claude on 2026-08-07
(docs/THREAT_MODEL.md's eighth amendment). Read it as "already armed, waiting
on the master switches," not as "off": reaching a cloud provider still needs
the two masters, the provider's API-key env var, and a per-run --confirm-online.
agentic:
enabled: false # master switch -- ships closed
deepagent_github:
enabled: false # ships closed
allow_git_write_tools: false # gates every write/commit/push in the clone
model: "qwen3.8:27b-mlx" # local planner model; cite models.local_llm.model
workspace_root: "data/agentic/workspaces"
max_write_budget_bytes: 100000
max_handoff_chars: 200000 # outbound-prompt cap for cloud egress
planner_max_tokens: 3072 # real-repo completion cap; keep it within Ollama num_ctx
allow_cloud_providers: true # gate 3 of the cloud chain -- ARMED
providers:
grok: { enabled: true, model: "grok-4.5" } # ARMED
claude: { enabled: true, model: "claude-sonnet-5" } # ARMEDSetting a provider enabled: true while allow_cloud_providers is false is a
config error, not a silent no-op — the three move together. Opening a PR needs
agentic.mode: "write", writes_enabled: true, and agentic/writer.py's
EXECUTION_ENABLED (all ship open) and agentic.enabled: true plus a
per-call reason and confirm; the arming checklist and the
CYCLAW_AGENTIC_WRITE_DISABLE rollback are in
docs/agentic/GITHUB_WRITE_ENABLEMENT.md.
# Optional stage 0: plan, review by hand, then hand the approved text to the coder.
python -m agentic.cli real-repo-run-plan \
--pr 123 --instruction "fix the off-by-one in the parser" --out plan.md
python -m agentic.cli real-repo-run \
--pr 123 --instruction "fix the off-by-one in the parser" \
--read-file src/parser.py --checks-file checks.json \
--plan-file plan.md \
--branch claude/parser-fix --commit-message "fix: off-by-one" \
--reason "triage issue 123" --confirm
python -m agentic.cli real-repo-run-status --run-id "<32-hex>"
python -m agentic.cli real-repo-run-decide --run-id "<32-hex>" --decision approve
python -m agentic.cli real-repo-run-push --run-id "<32-hex>"
python -m agentic.cli real-repo-run-publish --run-id "<32-hex>" --reason "..." --confirm
python -m agentic.cli real-repo-run-discard --run-id "<32-hex>"Exit codes are an API: 0 ok · 2 failed · 3 env/config · 4 write refused.
real-repo-run exits 0 whether or not a candidate was accepted — the record's
status field carries that.
Seven routes on 127.0.0.1:8790. GET /api/agent/checks is open (a hardcoded
allow-list, spawns nothing); the other six — POST /api/agent/run,
GET /api/agent/runs/{id}, POST /api/agent/runs/{id}/{decision,push,publish,discard}
— require a Bearer CYCLAW_API_KEY plus an Origin/Sec-Fetch-Site
cross-site check. POST /api/agent/run is deliberately synchronous (the run
record and its run_id first exist in the response), and its wall-clock budget
is derived from the request — iterations × planner_timeout + iterations × checks × 120s + 300s, capped at 3600s — because a flat budget was a real bug:
subprocess.run(timeout=) sends an uncatchable SIGKILL, so a request whose own
planner budget exceeded it left a leaked clone and a permanently running
record. Console equivalents: /agent run|confirm|status|approve|reject|push|publish|discard.
The loop is local-only by default, and the local path (no --provider flag)
needs nothing beyond the base install — LocalProposerClient is a plain
httpx call and nothing on that path imports deepagents or langchain.
--provider grok|claude --confirm-online drives the loop with a cloud model
behind a six-condition chain: agentic.enabled →
deepagent_github.enabled → allow_cloud_providers → providers.<name>.enabled
→ the provider's API-key env var (GROK_API_KEY / ANTHROPIC_API_KEY, presence
only, never a network probe) → per-run --confirm-online. Every outbound prompt
is injection-scanned, redacted, hashed, and audited as egress before it leaves
the process.
Cloud SDKs are opt-in extras, deliberately absent from the default install,
requirements.txt, and the Docker image:
pip install -e ".[agentic-deepagents]" -c constraints.txt # Claude only
pip install -e ".[agentic-deepagents-cloud]" -c constraints.txt # Grok only — just langchain-xai
pip install -e ".[agentic-deepagents,agentic-deepagents-cloud]" -c constraints.txt # bothfull (what CI and dev boxes get) pulls agentic-deepagents but deliberately
not agentic-deepagents-cloud, so a machine that never touches Grok never
carries langchain-xai; [all] is the only extra that installs both. The
published Docker image installs requirements.txt only, so running this
feature in a container means installing on top (pip install -e . for local
mode, or one of the commands above for cloud).
CyClaw includes an optional, out-of-band Telegram channel (telegram/,
shipped enabled: false) that gives the single trusted operator a
phone-reachable remote — outbound notifications and, when configured,
allowlisted two-way chat — without touching gate.py, graph.py, or the MCP
request path (invariant I6). Inbound chat text only ever reaches the RAG
pipeline via loopback POST /query, never a direct call into graph.py.
Outbound notify (mode: "notify") or long-poll two-way chat (mode: "chat",
the shipped YAML, still enabled: false; long-poll only, no public webhook
listener beside the loopback server; T1-first remains the recommended enable
order). allowed_chat_ids is required non-empty when enabled, and the bot
token comes only from the env var named by bot_token_env
(TELEGRAM_BOT_TOKEN), never from YAML. T3 hybrid-confirm consent
(allow_hybrid_confirm: false by default): the exact private-chat command
/online on <grok|claude> is the only way chat text can set
user_confirmed_online, for one next message only (hybrid_confirm_ttl_sec,
hard-capped at 300s) — core's triple gate remains the final authority. T4 media
staging (media.enabled: false) accepts private-chat attachments captioned
/save --confirm <reason> only through the existing agentic/fsconnect write
path. The poll-plist / health-plist generators never load, and their
secrets are injected at process start by the Keychain wrapper (see
macOS launchd & Keychain).
Core commands
python -m telegram.cli status
python -m telegram.cli test
python -m telegram.cli send --chat-id "<id>" --text "..." # T1; add --dry-run to preview
python -m telegram.cli poll # T2; requires telegram.mode: chat
python -m telegram.cli poll-plist # Darwin-only; generates, never loads
python -m telegram.cli health-plist # Darwin-only; generates, never loadsSee docs/channels/TELEGRAM_DESIGN.md for
architecture, the T0–T4 phase ledger, and the threat-model obligations
(docs/THREAT_MODEL.md's seventh amendment), and
telegram/README.md for package internals.
Optional out-of-band X poster (opentweet/, shipped enabled: false).
gate.py, graph.py, and the MCP server never import it (invariant I6).
Generation is loopback POST /query with user_confirmed_online: false.
The default write is an OpenTweet draft; scheduled_date is opt-in via
opentweet.schedule_enabled. Schedulers never send publish_now.
Core commands
python -m opentweet.cli status
python -m opentweet.cli test
python -m opentweet.cli post --topic "..." # add --dry-run to preview
python -m opentweet.cli schedule-plist # Darwin; generates, never loads
python -m opentweet.cli schedule-task # Windows; generates, never registersSee docs/channels/OPENTWEET_DESIGN.md
and opentweet/README.md. Keychain/CredMan wrappers
are in macos/README.md and
powershell/README.md.
| Layer | Mechanism |
|---|---|
| Network | Binds 127.0.0.1:8787 — no external exposure by design |
| Input | Config-driven injection filter (policy.prompt_filter) |
| Rate limit | 60 req/min per IP |
| Proxy bypass | All httpx clients set trust_env=False — ambient HTTP(S)_PROXY/.netrc cannot reroute local traffic, see the path-embedded Telegram bot token, or carry GROK_API_KEY / ANTHROPIC_API_KEY on a confirmed hybrid call (utils/health.py, llm/client.py local + Grok + Claude, harness/ollama.py, telegram/client.py, opentweet/client.py). This reverses the old “operator proxy governs paid egress” exception. |
| Telemetry | Canonical kill maps (utils/telemetry_kill.py: telemetry + a visibly-separate update-check map, plus a removed-outright scrub set incl. the declarative-OTel config names) applied before any SDK import by every maintained Python chokepoint (invariant-guard G1 pins 15 orderings) AND delivered as literal environment before the interpreter starts at every process boundary — Docker ENV, the shipped launchers, generated launchd plists / Windows tasks / cron lines, and verifier/gh children via build_telemetry_safe_env; ONNX Runtime additionally gets the post-import disable_telemetry_events() call at its load seams (utils/onnx_telemetry.py). HF Hub network calls are also cut off once the embedding model is confirmed cached (retrieval/embeddings.py). Not a network kill switch: intentional policy-gated egress is classified separately in SECURITY.md |
| Audit | All paths log SHA-256 query hash + PII-redacted metadata |
| Grok gating | Triple gate: mode=hybrid AND grok.enabled=true AND user_confirmed_online=true |
| Claude gating | Same triple gate, independently: mode=hybrid AND claude.enabled=true AND user_confirmed_online=true |
| Soul writes | Explicit human reason string + enforced write-boundary scan + atomic write |
| Agentic writes | pr_create implemented; the source constant and two config gates ship open since 2026-08-07, so agentic.enabled (ships false) plus per-call reason/confirm is what refuses. pr_comment/issue_comment remain plan-only. Git-level writes (real-repo commit/push and draft-PR publish) are additionally gated on deepagent_github.allow_git_write_tools, which ships false |
| Filesystem connector | Reads scoped to allowed_roots (5 MiB cap) with POSIX held-fd descent and Windows same-handle containment; writes default-OFF and hard-refused on Windows, otherwise confined to separate writable_roots, gated by human reason + --confirm, and atomic; UNC/ADS/device-path/../symlink escapes are denied |
| SQL connector | Read-only: SELECT/WITH-only query guard + session read-only + hard allow_write: false; DSN from env var only; disabled scaffold by default |
| Network connector | Passive only and disabled by default; explicit RFC1918/loopback CIDRs; self plus existing OS neighbor-cache reads; no ping, sweep, port probe, packet send, scheduler, or /ops route |
| Guardrails | Out-of-band, opt-in defense-in-depth; degrades to offline heuristic rails without nemoguardrails; never a routing authority; separate hash-only metrics stream |
| Telegram channel | Out-of-band, ships enabled: false; non-empty allowed_chat_ids allowlist required to arm; inbound chat reaches the pipeline only via loopback POST /query (never a direct graph.py call); T3 hybrid-confirm consent (allow_hybrid_confirm) ships off — only an explicit `/online on <grok |
| OpenTweet channel | Out-of-band, ships enabled: false; answers only via loopback POST /query with user_confirmed_online: false; default write is a draft; schedulers generate-don't-load and never send publish_now; API key from env / Keychain / CredMan, never YAML or a plist EnvironmentVariables dict |
| launchd secrets (macOS) | Generated plists never embed tokens — macos/cyclaw-keychain-env.sh injects secrets from the macOS Keychain at exec time and fails closed when the item is missing; cyclaw-keychain-set.sh stores them via a no-echo security prompt so the secret never appears in argv or the plist; the gate/harness supervised-agent generator additionally requires --confirm + a non-empty --reason |
/ops/* routes |
Loopback-only, require_api_key gated, rate-limited (60/min), every call audited (ops_sync_executed / ops_agentic_executed / ops_fsconnect_executed / ops_sqlconnect_executed); shells out via subprocess.run([...]) — never imports sync/ or agentic/ |
/auth/* routes |
Per-user auth design (gate_auth.py, docs/AUTHENTICATION_DESIGN.md); first-boot GET /auth/setup-status (no credential; same-origin-checked) and loopback-only POST /auth/bootstrap-password; session cookie + CSRF for browsers, bearer device tokens for programmatic clients; three roles (admin/operator/audit) gate the /auth/users* admin surface, with the last enabled admin protected from disable/delete/role-change; every /auth/* handler checks auth.enabled first and returns 503 (not 404) so route presence never discloses whether the feature is on. When auth.enabled is true, POST /query requires a session or named device token |
/memory/* + /query/export/html routes |
Optional, default-off memory admin surface (gate_memory.py); every memory: switch ships false; require_api_key gated, rate-limited; mutating routes (propose/apply/reject) require a non-empty reason string, with an injection scan on apply |
| Container | Non-root, no-new-privileges, cap_drop: ALL, read-only rootfs, seccomp, resource limits; optional eBPF/Falco detection (deploy/falco/, off by default) |
Docker / GHCR: published runtime image
ghcr.io/cgfixit/cyclaw(tag-triggered). Operator guide, pull/run commands, Falco opt-in notes, and explicit non-goals (no microVM):docs/DOCKER.md. Host publish remains127.0.0.1only.
Scope: CyClaw is a trusted-operator, loopback-bound local server — one operator by default, a small set of mutually trusted operators with their own accounts and roles once
auth.enabledis on, and single-tenant either way (everyone reaches the same corpus, soul, and model). LAN or WAN exposure is possible only through the threat model's documented bind exceptions (auth + TLS), never by default. The full threat model — what the sandbox does and does not cover (no microVM by design) and why — is documented indocs/THREAT_MODEL.md. The underlying design philosophy (telemetry kill, offline-first posture) lives indocs/security-philosophy/.
designed and built by Chris Grady, with AI tooling used under human review / CI / invariant-guard
source-available - all rights reserved
