Skip to content

Latest commit

 

History

3,647 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Local (portable) AI you can Trust.

Python 3.12 FastAPI LangGraph CyClaw CI/CD testing

Screenshots: local AI

Table of Contents


Quick Start

The fastest path to a running RAG server — macOS/Linux shown; only the torch pin and the activation command differ on Windows (see Installation for the full per-platform split):

git clone https://github.com/CGFixIT/CyClaw && cd CyClaw
python3.12 -m venv .venv && source .venv/bin/activate
pip install torch==2.13.0+cpu --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt -r requirements-test.txt -c constraints.txt --ignore-installed PyYAML
ollama pull qwen3.8:27b-mlx                      # Ollama must already be running on :11434
export CYCLAW_API_KEY="$(openssl rand -hex 20)"  # needed for the /soul/* endpoints
python -m retrieval.indexer                      # builds the retrieval index, once
uvicorn gate:app --host 127.0.0.1 --port 8787    # → http://127.0.0.1:8787

Confirm it's alive: curl http://127.0.0.1:8787/health.

On macOS (Apple Silicon, the primary platform): the +cpu torch wheel above doesn't exist there — use plain torch==2.13.0 and strip the torch/ --extra-index-url lines from both manifests first, or just run bash ./macos/setup-cyclaw.sh, which handles the torch difference, Ollama, the index, and both servers for you.

Full step-by-step guide — exact per-platform commands, Docker, the coding-harness console, and every REST endpoint with a copy-pasteable curl: setup-guide.md.


What It Does

CyClaw is a personal RAG (Retrieval-Augmented Generation) backend that:

  1. Answers questions exclusively from your local Markdown corpus — no internet by default.
  2. Enforces every safety invariant via LangGraph topology — not prompts, not config flags, not discipline.
  3. Maintains a persistent soul/personality layer (soul.md) with SHA-256 drift detection, atomic evolution writes, and user-gated modification.
  4. Falls back to an external LLM only with explicit user confirmation in hybrid mode — Grok (xAI) or Claude (Anthropic), selected per-query, each independently triple-gated at config, env, and per-query level.
  5. Exposes both a FastAPI HTTP gateway and an MCP server for Claude Desktop / Copilot Studio integration.
  6. Ships optional, out-of-band operator layers — Dropbox corpus sync (sync/) and agentic GitHub context / governed local workflows (agentic/, .claude/) — never imported into the request path, drivable from the browser terminal via the governed Sync and Agentic consoles.
  7. Extends the agentic layer to local data (v1.8+) with opt-in filesystem (agentic/fsconnect/), read-only SQL (agentic/sqlconnect/), and passive LAN inventory (agentic/netconnect/) connectors — all disabled by default and out-of-band.
  8. Adds an optional NeMo Guardrails content-safety layer (v1.8, guardrails/) that soft-imports nemoguardrails, degrades to offline heuristic rails, and is defense-in-depth only — never a routing authority. When guardrails.enabled is the literal true, utils/guardrail_bridge.py wires the visible guardrail_input / guardrail_output nodes. See guardrails/README.md.
  9. Scaffolds a governed harness-optimizer / Deep Agents layer (v1.9, agentic/harness_optimizer/ + agentic/deepagent_github/) — opt-in, disabled by default, out-of-band; phases 0–9 implemented and tested (PR #515, 2026-07-13). Superseded by item 11: the live coding pipeline is now real_repo_loop.py, whose draft-PR write path was armed on 2026-08-07 behind the agentic.enabled master switch that still ships false.
  10. Ships a local coding-harness console (v1.9, harness/ + powershell/ / macos/) — a grok-build-style slash-command console on 127.0.0.1:8790 chatting with the local model over the OpenAI-compatible endpoint, with per-session token tallies, /goal + human-gated /loop, /skills and /tools wiring diagrams, and allowlist-only /web (off by default). Home is %USERPROFILE%\.CyClaw (Windows) or ~/.CyClaw (macOS/Linux); same I6 isolation as every other out-of-band layer. See harness/README.md.
  11. Adds a real-repo GitHub agentic coding harness (v1.9, agentic/real_repo_loop.py + agentic/executor/) — clone → plan → patch → verify → human decides → commit, with pushing a claude/* branch and opening a draft PR as two further separate decisions. A diff-scope gate refuses candidates that rewrite the tests judging them, verification runs as sandboxed argv-list subprocesses, and the layer ships off: agentic.enabled: false is the master switch (plus per-call reason/confirm) and allow_git_write_tools: false holds push.
  12. Adds an optional per-user authentication layer (gate_auth.py + utils/authn*) — scrypt password hashes, session cookie + CSRF for browsers, bearer device tokens for programmatic clients, three roles (admin/operator/audit), and the cyclaw-user console script. Every /auth/* route exists regardless of auth.enabled and returns 503 (not 404) when it is off, so route presence never discloses the feature state. When auth.enabled is true, POST /query and the console require a session or named device token. The shipped default leaves /query open. See Per-User Authentication.
  13. Adds an optional facts + episodes memory store (gate_memory.py + memory/) — SQLite+FTS5, propose/apply governance (a non-empty human reason plus an injection scan on apply, parallel to soul's I5), and an optional retrieval-fusion hook. Every memory: switch ships false; mutating routes require the same Bearer CYCLAW_API_KEY as the other admin endpoints. See memory/README.md and the plan in docs/memory/README.md — not docs/memories/, the sandbox notes.
  14. Ships an optional Telegram channel (v1.9, telegram/, shipped enabled: false) — an out-of-band phone remote: outbound notify (mode: "notify") or allowlisted long-poll chat (the shipped mode: "chat"; T1-first is still the advised enable order). Inbound text only ever reaches the RAG pipeline through loopback POST /query. T3 hybrid-confirm (allow_hybrid_confirm, default off) is the only way chat text can set user_confirmed_online — one-shot, via the exact private-chat command /online on <grok|claude> — and T4 media staging (media.enabled, default off) writes only through the existing agentic/fsconnect path. See docs/channels/TELEGRAM_DESIGN.md.
  15. Adds an offline slop-detection probe for the agentic coding loop (v1.9.x, agentic/unslop_bridge.py + vendored scanners under agentic/vendor/unslop/) — scans real_repo_loop.py's model responses and proposed prose files (.md/.rst/.txt) for AI-writing tells, logs redacted findings (SHA-256 doc hash + counts, never raw text) to logs/unslop.jsonl, and surfaces a nudge back into the loop. unslop.enabled ships false; the scanner runs fully offline with no network calls and never crosses the I6 boundary.
  16. Projects the audit trail into a Numbat forensic stream (utils/numbat_emitter.py, numbat: block — the one optional subsystem that ships enabled: true) — a derived NDJSON stream at logs/numbat-events.ndjsonl that the pinned Numbat 0.2.0 CLI can score for patterns like secrets.read_private_key and exfil.curl_post_file, fed by the out-of-band action plane and the mainline plane (utils/logger.audit_log projects every audit record). audit.jsonl stays authoritative; records are projected after SHA-256 query hashing and PII redaction, so raw query text reaches neither stream — but every event carries hostname/username/uid metadata, which makes the stream a second sensitive local log, not a privacy improvement (file sink only, no HTTP; disable with numbat.enabled: false). Fail-soft end to end, at the terminal audit_logger node, so it can never turn a good response into a 500. See docs/security-philosophy/numbat_secondary_evaluator.md.
  17. Ships an optional OpenTweet X channel (opentweet/, shipped enabled: false) — an out-of-band weekly poster; generation only through loopback POST /query with user_confirmed_online: false, default write an OpenTweet draft, schedulers that generate and never load. See OpenTweet Channel.
  18. Tracks what the paid providers cost (utils/spend.py, logs/spend.jsonl via logging.spend_file) — every billed Grok/Claude call appends token counts, provider/model, and a source tag separating the /query plane from the agentic plane. Tokens are the ground truth; dollars are derived at read time. A separate stream from audit.jsonl that never persists query text, prompts, or credentials, and is not a policy point; cyclaw-metrics prints spend windows, flags rate staleness, and compares CyClaw's rate table against xAI's own cost_in_usd_ticks so a wrong rate surfaces instead of accumulating. See docs/spend/README.md.

Architecture

gate.py (FastAPI on 127.0.0.1:8787) runs the TrustedHostMiddleware Host allowlist, then the per-IP rate limiter (60 req/min — it runs first, before the injection filter), then the config-driven injection filter, then soul init, and hands a GraphState to the 12-node LangGraph state machine in graph.py. Routing is graph edges only — retrieve is the unconditional entry, and every path converges on audit_logger before END. The numbered plain-text version of this flow is CLAUDE.md §2 "The Map"; the invariants it encodes are in INVARIANTS.md.

LangGraph Topology (rendered)

flowchart TD
    A(["🌐 Client\nHTTP POST /query\nor MCP tool call"])
    A --> B

    subgraph GATEWAY ["gate.py — FastAPI 127.0.0.1:8787"]
        B["TrustedHostMiddleware\nHost header allowlist"]
        B --> C["Rate Limiter\n60 req/min per IP"]
        C --> D["Prompt Injection Filter\n40 patterns · config-driven · lru_cache"]
        D --> E["Build GraphState\nquery + user_confirmed_online"]
    end

    E --> F

    subgraph GRAPH ["graph.py — LangGraph 12-node State Machine"]
        F(["① retrieve\nChroma + BM25 + RRF"])
        F --> G["② route_by_score\ntop_score ≥ 0.028?"]
        G -->|"YES — local context"| X["③ guardrail_input\noffline rail · opt-in\npass-through when disabled"]
        X -->|"blocked"| L
        X -->|"passed · high score"| H["④ local_llm\nOllama :11434\nqwen3.8:27b-mlx"]
        G -->|"NO — vault miss"| I["⑤ user_gate\nneeds_confirm = true"]
        I -->|"confirmed=true + hybrid\n+ grok.enabled + provider=grok"| PG["⑥ pre_action_hook_grok\nsync · disabled=pass-through\nexit 2 → deny"]
        PG -->|"exit 0 → allow"| J["⑦ grok_fallback\nxAI grok-4.5\ntriple-gated · not railed"]
        I -->|"confirmed=true + hybrid\n+ claude.enabled + provider=claude"| PC["⑧ pre_action_hook_claude\nsync · disabled=pass-through\nexit 2 → deny"]
        PC -->|"exit 0 → allow"| W["⑨ claude_fallback\nAnthropic claude-sonnet-5\ntriple-gated · not railed"]
        I -->|"confirmed=false\nor offline mode"| X
        X -->|"passed · vault miss"| K["⑩ offline_best_effort\nlocal LLM · no RAG gate"]
        I -->|"confirmed=None — PAUSE\nreturn needs_confirm to the client"| L
        H --> Y["⑪ guardrail_output\noffline rail · opt-in\ngrounding check: local_llm only"]
        J --> Y
        W --> Y
        K --> Y
        Y --> L
        PG -.->|"deny"| L
        PC -.->|"deny"| L
        L(["⑫ audit_logger\nSHA-256 hash · PII redact\n→ logs/audit.jsonl"])
    end

    L --> M(["📤 QueryResponse\nanswer · sources · model_used\nretrieval_mode · needs_confirm"])

    subgraph RETRIEVAL ["retrieval/hybrid_search.py"]
        N["ChromaDB\nsemantic · 384-dim cosine"]
        O["BM25Okapi\nkeyword · Porter stemming"]
        P["RRF fusion\nk=60 · equal weighting"]
        N --> P
        O --> P
    end

    F <-->|"hybrid search"| P

    subgraph SOUL ["utils/personality.py"]
        Q["soul.md\nSHA-256 drift detection"]
        R["SQLite / Postgres\nversion history · TTL prune"]
        Q <--> R
    end

    H <-->|"soul preamble\n≤ 8000 chars"| Q
    K <-->|"soul preamble"| Q

    subgraph OOB ["Out-of-band — never imported by gate/graph/MCP"]
        S["agentic/cli.py\nGitHub read ops"]
        T["agentic/fsconnect/\nscoped FS read/write"]
        U["sync/cli.py\nDropbox corpus pull"]
        V["guardrails/\nNeMo rails skeleton"]
    end

    style GATEWAY fill:#1a3a5c,color:#ffffff,stroke:#4a90d9
    style GRAPH fill:#1a3a2a,color:#ffffff,stroke:#4a9d5a
    style RETRIEVAL fill:#3a2a1a,color:#ffffff,stroke:#d9904a
    style SOUL fill:#3a1a3a,color:#ffffff,stroke:#d94ad9
    style OOB fill:#2a2a2a,color:#aaaaaa,stroke:#666666,stroke-dasharray:5 5
    style J fill:#5c1a1a,color:#ffffff
    style W fill:#5c1a1a,color:#ffffff
    style L fill:#1a1a3a,color:#ffffff
Loading

What the diagram compresses: HybridRetriever (retrieval/hybrid_search.py) fuses ChromaDB (semantic, all-MiniLM-L6-v2, 384-dim cosine, CPU-only embeddings) with BM25Okapi (keyword, Porter stemming) by RRF (k=60, equal weighting) and carries per-chunk provenance metadata in every result. The telemetry kill block runs before any SDK import, and the MCP server and the indexer apply the same block.


API Key Setup (Soul Mutations)

CyClaw's soul mutation endpoints (/soul/propose, /soul/apply, /soul/reload, /soul/restore) require a Bearer API key. Without it they return HTTP 401 immediately — intentional fail-closed behavior.

All /soul/* endpoints — including GET /soul — require a valid Authorization: Bearer <key> token. Only /health, /query, GET /index/status, GET /auth/setup-status, POST /auth/login (issues the session itself; 503 when auth.enabled is false), and the console pages (GET /, /static/*) are unauthenticated. POST /index/build and POST /auth/bootstrap-password carry no credential either, but neither is open: each is gated on a loopback socket peer plus a same-origin check and returns 403 off-box — /index/build 409 while a build is already running, /auth/bootstrap-password 409 once the first admin password is set. POST /query, though credential-free by default, additionally carries an unconditional same-origin check — a cross-site browser request is rejected 403 CROSS_SITE_BLOCKED regardless of auth.enabled; requests carrying neither Origin nor Sec-Fetch-Site (curl, PowerShell, schedulers) are unaffected.

Opting out entirely: config.yaml's security.api_key_optional (default false) removes the CYCLAW_API_KEY requirement from every route above and the harness console's guarded routes (agent run/push/publish included), for both apps at once — but only for requests arriving from this machine. The bypass is granted on the socket peer, so a remote caller still needs the real key no matter how the process was launched. Entries in security.allowed_hosts do not change that: that list filters request Host headers and opens no listening socket. What would matter is the bind itself — gate.py refuses to start with a non-loopback api.host while the flag is true, and config-guard's C13 warns on that pair. Note it also does nothing under Docker: NAT rewrites the source address, so the container sees the bridge gateway rather than loopback and the routes stay key-gated (set CYCLAW_API_KEY in the container instead).

macOS / Linux — zsh or bash

Set it for the current shell. Generate a real value instead of typing one — openssl ships with macOS and every Linux distribution:

export CYCLAW_API_KEY="$(openssl rand -hex 20)"
echo "$CYCLAW_API_KEY"
uvicorn gate:app --host 127.0.0.1 --port 8787

Persist it in your shell profile. macOS has defaulted to zsh since Catalina, so that means ~/.zshrc unless you switched — check with echo $SHELL. On bash, append it to the first existing login file in this order: ~/.bash_profile, ~/.bash_login, ~/.profile; create ~/.bash_profile only when none exists, because macOS bash login shells do not read ~/.bashrc (Linux bash does).

echo 'export CYCLAW_API_KEY="your-strong-local-secret"' >> ~/.zshrc   # or the bash file above
source ~/.zshrc

Full macOS walkthrough — including launching the harness console beside the gateway and exercising every REST endpoint with curl — is in setup-guide.md.

Windows — PowerShell / cmd.exe

(Windows is the fallback path; CyClaw is developed and verified on macOS first. Everything below still works and is CI-covered on windows-latest.)

$env:CYCLAW_API_KEY = "your-strong-local-secret"      # current session only
uvicorn gate:app --host 127.0.0.1 --port 8787

Persist it for the current user (writes the user environment permanently); verify with echo $env:CYCLAW_API_KEY before launching:

[System.Environment]::SetEnvironmentVariable("CYCLAW_API_KEY", "your-strong-local-secret", [System.EnvironmentVariableTarget]::User)

Windows Server, system-wide (all users, requires admin): the same call with [System.EnvironmentVariableTarget]::Machine, or System Properties → Advanced → Environment Variables → System variables → New. From cmd.exe: set CYCLAW_API_KEY=your-strong-local-secret for the session and setx CYCLAW_API_KEY "your-strong-local-secret" to persist.

All platforms — .env file (already in .gitignore)

Create .env in the repo root:

# Keys live here, never in config.yaml — config.yaml only names which
# provider is enabled; the key itself is read from the environment.
CYCLAW_API_KEY=your-strong-local-secret
GROK_API_KEY=your-xai-key-or-dummy-when-offline
ANTHROPIC_API_KEY=your-anthropic-key

Then tighten it — a hand-created file inherits the shell's umask (usually 0644, i.e. world-readable), and macos/invoke-cyclaw.sh refuses to source a dotenv that is not 600 or 400 rather than load secrets from a file other local accounts can read:

chmod 600 .env

On Windows, a hand-created file often inherits BUILTIN\Users read. Tighten it the same way powershell/Invoke-CyClaw.ps1 requires before it will source the file (owner-only; refuse Everyone / Users / Authenticated Users):

icacls .env /inheritance:r /grant:r "${env:USERNAME}:(R,W)"

macos/setup-cyclaw-keys.sh and the harness console's /api panel both already write ~/.CyClaw/.env at 600; only a hand-made file needs this step.

The Claude variable is ANTHROPIC_API_KEY, not CLAUDE_API_KEYllm/client.py and agentic/config.py both read the former, and nothing in the codebase reads the latter. Setting the wrong name is silent: Claude simply reports unavailable and the query falls back to a local answer.

Load it before launching:

# Bash / Zsh
export $(grep -v '^#' .env | xargs)
uvicorn gate:app --host 127.0.0.1 --port 8787
# PowerShell
Get-Content .env | ForEach-Object {
    if ($_ -match '^([^#=][^=]*)=(.*)$') {
        [System.Environment]::SetEnvironmentVariable($Matches[1].Trim(), $Matches[2].Trim())
    }
}
uvicorn gate:app --host 127.0.0.1 --port 8787

Choosing an API key value

CyClaw is loopback-only (127.0.0.1:8787) — the key never crosses a network. Still:

  • Use at least 20 random characters: openssl rand -hex 20 (Linux/macOS) or [System.Web.Security.Membership]::GeneratePassword(24,4) (PowerShell)
  • Do not reuse a password from elsewhere
  • Do not commit the key to Git (.env is already in .gitignore)
  • Don't forget to set the api key via terminal on Mac or env var in Windows or the web app will not recognize it.

Per-User Authentication (v1.9)

CyClaw ships two independent credential systems, and confusing them is the most common setup mistake:

System Secret Guards Toggle
Operator API key CYCLAW_API_KEY env var (Bearer) /soul/*, /ops/*, /memory/*, /audit/summary, and the harness console's guarded routes Always on (fail-closed when unset); security.api_key_optional is the one deliberate loopback-peer bypass
Per-user auth Per-account scrypt password → session cookie, or a named device token POST /query and the console's user surface auth.enabled in config.yaml — ships false

The per-user layer is gate_auth.py + utils/authn*; the full design is docs/AUTHENTICATION_DESIGN.md. With auth.enabled: false (the shipped default) POST /query takes no credential and every /auth/* route answers 503, not 404 — route presence never discloses whether the feature is on.

Turning it on

  1. Set auth.enabled: true in config.yaml and restart gate.py. The store is SQLite at auth.db_path (data/auth/cyclaw_auth.db); CYCLAW_AUTH_DB_URL — its own env var, deliberately not the personality subsystem's CYCLAW_DB_URL — switches it to a postgresql:// DSN.

  2. Set the first admin password. The bootstrap account is username admin, created with no password. Use the terminal console's first-boot box on loopback, or post directly:

    curl -s -X POST http://127.0.0.1:8787/auth/bootstrap-password \
      -H 'Content-Type: application/json' \
      -d '{"password":"<a long passphrase>"}'

    That route is loopback-peer + same-origin only — 403 from off-box, 409 once a password is set, 503 when auth.enabled is false — and carries no credential on purpose: on a genuine first run there is nothing to present yet. GET /auth/setup-status (no credential, but same-origin-checked and rate-limited) reports {enabled, needs_password, username} so a console can tell first-boot from logged-out.

  3. Add the accounts operators actually use with the local-only cyclaw-user console script (no HTTP route reaches it). Subcommands: add, list, role, disable, enable, passwd, token create|list|revoke; new accounts default to operator:

    cyclaw-user add alice --role operator     # prompts for the password
    cyclaw-user token create alice laptop     # prints the token ONCE

Roles, sessions, lockout

Three roles, checked server-side on every request:

Capability admin operator audit
POST /query yes yes no — 403 AUTH_ROLE_DENIED
List users (GET /auth/users) yes yes no
Create user / reset another user's password yes yes, but never on an admin account no
Set role, delete user yes no no
Disable / enable yes non-admins only no
Change own password (POST /auth/password) yes yes yes
GET /auth/audit/summary yes no yes

The last enabled admin is protected — disable, delete, and role-change all refuse it, so an operator cannot lock the deployment out of its own admin surface. GET /auth/audit/summary is the reduced, session-gated audit view, not the API-key-gated GET /audit/summary.

Browsers get a cyclaw_session cookie plus a CSRF token that every mutating /auth/* route requires in the X-CyClaw-CSRF header; POST /query is deliberately CSRF-exempt because it takes a session or a bearer device token and mutates no auth state. Programmatic clients send a named device token as Authorization: Bearer <token> — displayed once, stored only as a hash, revoked by label. A session dies at whichever of auth.session's two limits comes first: idle_timeout_sec: 43200 (12 h, rolling) or absolute_timeout_sec: 604800 (7 d, never resets). Failed logins back off per account — the first 5 consecutive failures are free, then the delay doubles from 2 s to a 900 s ceiling; no admin action is needed to recover. Passwords are scrypt-hashed (utils/authn.py), and no password, session id, CSRF token, or device token is ever written to audit.jsonl. Design and rationale: docs/AUTHENTICATION_DESIGN.md.

Serving it beyond loopback

Enabling auth.enabled does not by itself make a non-loopback bind safe or permitted. gate.py's _require_loopback_bind still refuses a non-loopback api.host, and the auth+TLS route past it is refused outright while security.api_key_optional is true — that flag removes the CYCLAW_API_KEY gate from /soul/*, /ops/*, and /memory/*, which per-user auth does not replace. Set security.api_key_optional back to false first, then generate a certificate with the bundled openssl wrapper (no new runtime dependency):

cyclaw-gen-cert --hostname "$(hostname)" --days 825

Flags: --certfile, --keyfile, --hostname (defaults to the machine hostname), --days (default 825), --san (repeatable extra SAN entry, e.g. IP:10.0.0.5 or DNS:box.local), and --forcerequired to overwrite an existing cert/key pair; without it the command refuses rather than clobbering one. Read docs/THREAT_MODEL.md before exposing the port — CyClaw's stated scope is trusted-operator (single by default, a small trusted group behind auth.enabled), loopback-bound, and single-tenant.

Installation

Prerequisites

Requirement Version Notes
Python 3.12 Primary supported runtime
Ollama Any Must be running on localhost:11434
Model pulled in Ollama qwen3.8:27b-mlx (default), mistral:7b, or any chat model
macOS (primary) 14 Sonoma+ Apple Silicon only. An Intel Mac cannot install this repo's pinned torch at all — no x86_64 wheel is published at that pin
Windows / Linux (fallback) Both fully supported and CI-covered; they share the +cpu torch path below

Optional local-backend failover. Ollama is the primary local backend; CyClaw can also fail over to LM Studio (or any OpenAI-compatible loopback server) when Ollama isn't reachable. Off by default — set models.local_llm.fallback.enabled: true and fill in fallback.model with the LM Studio id (no Ollama-style name:tag colon). A short probe (fallback.probe_timeout_sec, default 1.5s) tries Ollama first and LM Studio second, LocalLLMClient and /health share the choice, and it re-probes if neither answered the first time (llm/client.py's resolve_local_backend).

Docker (optional runtime image)

Prefer GHCR when you want a prebuilt linux/amd64 runtime without a local pip install. Pull ghcr.io/cgfixit/cyclaw and run with the existing compose hardening (loopback publish, read-only rootfs, seccomp builtin). Full operator guide: docs/DOCKER.md.

export CYCLAW_IMAGE_TAG=1.9.0
docker compose pull && docker compose up -d
curl -sS http://127.0.0.1:8787/health

Native install (below) remains the primary path for Apple Silicon and for the coding harness.

Install — macOS (Apple Silicon)

macOS is the primary platform and needs a different torch step than Windows/Linux: the +cpu local-version wheel does not exist for macOS, and both manifests hardcode that pin, so the generic block fails twice on a Mac.

git clone https://github.com/CGFixIT/CyClaw
cd CyClaw
python3.12 -m venv .venv
source .venv/bin/activate
# 1) torch FIRST, and PLAIN — no +cpu suffix, no --index-url override.
#    Apple Silicon has one arm64 wheel; there is no CPU/CUDA build to pick between.
pip install "torch==2.13.0"
# 2) Everything else, from a requirements.txt copy with the torch and
#    PyTorch-index lines stripped out, and a constraints.txt copy that keeps
#    torch pinned minus the +cpu suffix (--ignore-installed reinstalls torch
#    too, so an unconstrained copy floats it) — the same thing CI's
#    macos-latest leg runs.
grep -v -e '^torch==' -e '^--extra-index-url https://download.pytorch.org' \
    requirements.txt > /tmp/requirements-macos.txt
sed 's/^\(torch==[0-9][0-9.]*\)+cpu$/\1/' constraints.txt > /tmp/constraints-macos.txt
pip install -r /tmp/requirements-macos.txt -c /tmp/constraints-macos.txt \
    --ignore-installed PyYAML

Prefer a script? bash ./macos/setup-cyclaw.sh is the single operator-facing entry point (offers to clone, asks its few choices once, then runs macos/setup-from-clone.sh: installer + Keychain keys + Ollama check + retrieval index + both servers). bash ./macos/install-cyclaw.sh is the installer alone — it handles the torch difference but skips the Ollama / index / API-key steps, so the gateway stays degraded (503 on /query) until you do them. Flags, privacy notes, and tradeoffs: macos/README.md and setup-guide.md.

Install — Windows / Linux (fallback)

git clone https://github.com/CGFixIT/CyClaw
cd CyClaw
python3.12 -m venv .venv
source .venv/bin/activate
# 1) CPU-only torch first (CVE-2025-32434 fixed in 2.6.0; 2.13.0 is within the patched range)
pip install torch==2.13.0+cpu --index-url https://download.pytorch.org/whl/cpu
# 2) The rest, pinned to the verified transitive tree. --ignore-installed PyYAML
#    avoids a resolver conflict with a system PyYAML some platforms preinstall.
pip install -r requirements.txt -r requirements-test.txt -c constraints.txt --ignore-installed PyYAML

Every optional feature in one environment (any platform)

For a from-scratch dev box or a full manual smoke test — Postgres/pgvector, NeMo Guardrails, dev/test tools, and both cloud providers — substitute step 2 with:

pip install -e ".[all]" -c constraints.txt

Required local prep

mkdir -p index logs   
export GROK_API_KEY=dummy

^ # optional — gate.py/the retriever/logger self-create these on first run

data/personality/soul.md ships committed to git with CyClaw's real personality already in place — do not recreate it from a placeholder on a fresh clone. If it's ever deleted, PersonalityManager self-heals with a generic default, but that's a recovery path, not the normal first-run state.

Run

CyClaw ships two independent local web apps. Neither starts the other; run whichever you need, or both in separate terminal tabs.

# The RAG gateway — serves static/terminal.html at / plus the whole REST API
python -m retrieval.indexer                          # once, before the first /query
uvicorn gate:app --host 127.0.0.1 --port 8787        # → http://127.0.0.1:8787

# The coding-harness console — serves static/harness.html
python -m harness.server                             # → http://127.0.0.1:8790

The cyclaw-* short names need a self-install. cyclaw-server, cyclaw-harness, cyclaw-index, cyclaw-mcp, cyclaw-metrics, cyclaw-clear-cache, cyclaw-user, and cyclaw-gen-cert are [project.scripts] shims that pip writes only when the project itself is installed; requirements.txt has no self-install line, so add pip install -e . -c constraints.txt if you want them. The python -m … forms always work and are what both shipped launchers use.

Override the harness port with CYCLAW_HARNESS_PORT=8795 python -m harness.server, not a CLI flag; it refuses to bind a non-loopback address. uvicorn harness.server:app also works (the module-level app is built lazily, so importing never touches ~/.CyClaw), but prefer -m: the uvicorn form bypasses the bind-address guard, so --host 0.0.0.0 opens a public socket that python -m harness.server would have refused — TrustedHostMiddleware still rejects non-loopback Host headers, but that is one layer fewer.

Open / for the terminal UI and /health for readiness. The terminal exposes five operator consoles — Soul, Sync, Agentic, Filesystem, and SQL — the latter four calling POST /ops/sync, /ops/agentic, /ops/fsconnect, and /ops/sqlconnect (API-key gated, rate-limited, audited).

Every gateway route with a copy-pasteable curl invocation, plus what each status code means, is in setup-guide.md.


Project Structure

CyClaw/
├── gate.py
├── gate_ops.py                 # /ops/* endpoints (sync/agentic/fsconnect/sqlconnect subprocess shims)
├── gate_auth.py                # /auth/* endpoints — session cookie + CSRF, bearer device tokens
├── gate_memory.py              # /memory/* + /query/export/html — optional, default-off memory admin surface
├── graph.py
├── metrics.py                  # audit.jsonl analyzer + spend.jsonl Spend section (cyclaw-metrics)
├── config.yaml                 # single source of truth
├── README.md
├── mcp_hybrid_server.py        # retrieval-only MCP server
├── memory/                     # optional facts + episodes store (default-off)
│   ├── README.md               # package pointer (not docs/memories/)
│   ├── store.py                # SQLite + FTS5 backend for facts/episodes
│   ├── policy.py               # propose/apply governance (reason required, injection scan)
│   ├── retrieval_adapter.py    # optional fusion hook into hybrid retrieval
│   ├── mirror.py               # episode staging (lazy, non-fatal)
│   ├── consolidation.py        # stub — stay false in v1
│   └── models.py               # typed request/response shapes
├── agentic/                    # out-of-band GitHub context + governed registry (see README.md)
│   ├── cli.py
│   ├── context.py
│   ├── gh_client.py
│   ├── registry.py
│   ├── writer.py               # gh pr create --draft; armed flag, held by agentic.enabled
│   ├── real_repo_loop.py       # (v1.9 P10) clone → plan → patch → verify → human decides → commit
│   ├── unslop_bridge.py        # (v1.9.x) offline slop-detection probe for real_repo_loop; default off
│   ├── vendor/unslop/          # vendored offline AI-writing-tell scanners (suggest.py); no network calls
│   ├── executor/               # sandboxed argv-list check runner; required fail-closed hard sandbox (hard_sandbox.py)
│   ├── fsconnect/              # (v1.8) local/SMB filesystem connector
│   │   ├── cli.py
│   │   ├── client.py           # scoped reads (fs_list/stat/read/grep)
│   │   ├── pathsafe.py         # held-handle containment core (POSIX + Windows reads)
│   │   ├── writer.py           # gated, atomic writes (default-disabled)
│   │   └── indexer.py          # toggleable RAG-corpus indexing of the share
│   ├── sqlconnect/             # (v1.8) read-only SQL scaffold (Postgres/MSSQL)
│   │   ├── cli.py
│   │   └── client.py           # SELECT-only query guard, env-only DSN
│   ├── harness_optimizer/      # (v1.9) governed better-harness-style optimizer scaffold
│   │   ├── core.py             # Experiment/Surface/RunReport/CandidateDecision models
│   │   ├── proposer.py         # scoped train/holdout workspace builder
│   │   ├── mcp/tools.py        # audited, symlink-hardened proposer workspace tools
│   │   └── governance.py       # visible-case-hardcoding + governance-finding gates
│   └── deepagent_github/       # (v1.9) workspace tools + cloud planner; DeepAgents subgraph retired
│       ├── repo_workspace.py   # live: jailed workspace tools (clone/read/write/commit/push) used by real_repo_loop
│       ├── chat_client.py      # live: cloud-provider planner adapter (Grok/Claude)
│       ├── builder.py          # retired DeepAgents subgraph (2026-07-31) — kept, not deleted
│       ├── permissions.py      # phase-5 no-write policy refusal
│       └── subagents.py        # validated SubAgent specs, no bare-string tools
├── guardrails/                 # (v1.8) opt-in rails; graph nodes via guardrail_bridge
│   ├── README.md
│   ├── cli.py
│   ├── config.py
│   ├── integration.py          # soft-imports nemoguardrails; degrades gracefully
│   ├── rails.py                # offline heuristic rails (injection/soul/grounding)
│   ├── metrics.py              # separate logs/guardrails.jsonl stream (hashes only)
│   └── config/                 # NeMo config.yml + rails.co (Colang flows)
├── harness/                    # (v1.9) coding console on 127.0.0.1:8790 (see harness/README.md)
│   ├── README.md               # slash-command usage (/goal /loop /skills /tools /web)
│   ├── server.py               # FastAPI control plane (cyclaw-harness)
│   ├── sessions.py             # JSON session store with per-session token tallies + /goal
│   ├── ollama.py               # loopback-only OpenAI-compatible /v1 chat client
│   ├── config.py               # ~/.CyClaw (or %USERPROFILE%\.CyClaw) home layout
│   ├── prompts.py              # ponytail + karpathy (+ optional soul, /goal, /web extract)
│   ├── registry_view.py        # merged catalog (AST-parses MCP tools; I6)
│   ├── tools_view.py           # /tools wiring diagram (live routes vs MCP catalog)
│   ├── skills_view.py          # /skills wiring diagram (prompt + agent-check vs catalog)
│   ├── web_search.py           # allowlist-only GET; off by default; no search engine
│   ├── agent_policy.py         # check-profile allowlist — console sends profile names, never argv
│   ├── agent_routes.py         # the seven /api/agent/* routes (6 of the 29 guarded)
│   ├── auth_routes.py          # the /api/auth/* console auth surface
│   ├── env_keys.py             # allowlisted dotenv secret store ($CYCLAW_HOME/.env; file-only)
│   ├── memory_notes.py         # console /memory notes store
│   └── schemas.py              # request models
├── telegram/                   # (v1.9) optional Telegram channel (out-of-band), shipped enabled: false
│   ├── cli.py
│   ├── client.py               # Bot API client — outbound notify + long-poll inbound chat
│   ├── config.py               # loads config.yaml's `telegram:` block
│   ├── runner.py                # long-poll loop; answers via loopback POST /query only
│   ├── state.py                 # T3 hybrid-confirm consent state (default off)
│   ├── media.py                 # T4 attachment staging via agentic/fsconnect (default off)
│   └── ratelimit.py
├── opentweet/                  # optional X channel (out-of-band), shipped enabled: false
│   ├── cli.py                  # status / test / post / schedule-plist / schedule-task
│   ├── client.py               # loopback /query + OpenTweet REST; trust_env=False
│   ├── config.py               # loads config.yaml's `opentweet:` block
│   ├── runner.py               # topic → query → validate → draft/schedule
│   └── selftest.py
├── powershell/                 # Windows installer/launcher for the harness
│   ├── Install-CyClaw.ps1      # home + venv + PATH shim + profile function
│   ├── Invoke-CyClaw.ps1
│   └── Uninstall-CyClaw.ps1
├── macos/                      # macOS/Linux installer/launchd glue (see macos/README.md)
│   ├── setup-cyclaw.sh         # single entry point: clone? + one-shot + optional start/browser/autofill
│   ├── setup-from-clone.sh     # one-shot after git clone (Apple Silicon)
│   ├── install-cyclaw.sh
│   ├── uninstall-cyclaw.sh
│   ├── invoke-cyclaw.sh        # gate :8787 + harness :8790
│   ├── setup-cyclaw-keys.sh    # Keychain + ~/.CyClaw/.env (never config.yaml)
│   ├── setup-fsconnect.sh      # confined ~/CyClaw-FS list/stat/read
│   ├── cyclaw-keychain-*.sh    # Keychain inject/store for launchd jobs
│   ├── generate_service_plist.py  # supervised gate/harness LaunchAgent — requires --confirm + --reason, never loads
│   └── LaunchAgents/           # templates only — never auto-loaded
├── .claude/                    # local operator workflows and prompts
│   ├── commands/
│   ├── hooks/
│   ├── memory/
│   ├── patterns/
│   ├── rules/
│   ├── skills/
│   ├── tools/
│   └── utility-prompts/
├── retrieval/
│   ├── indexer.py
│   ├── hybrid_search.py
│   ├── embeddings.py
│   ├── stemmer.py
│   ├── vector_store.py         # pluggable: embedded ChromaDB (default) or pgvector; sole Chroma chokepoint
│   └── clear_cache.py          # dry-run-by-default embedding-cache cleaner (cyclaw-clear-cache)
├── llm/
│   └── client.py
├── sync/                       # optional Dropbox corpus sync
│   ├── cli.py
│   ├── runner.py
│   └── scheduler.py
├── utils/
│   ├── sanitizer.py
│   ├── logger.py
│   ├── personality.py
│   ├── health.py
│   ├── ratelimit.py
│   ├── launchd_plist.py        # stdlib-only plist builder shared by the telegram / fsconnect / opentweet / generate_service_plist generators (sync.scheduler builds its own)
│   ├── guardrail_bridge.py     # only bridge from graph.py to guardrails/ (never a direct import)
│   ├── numbat_emitter.py       # derived Numbat NDJSON stream: action-plane emits + mainline audit projection
│   ├── spend.py                # append-only Grok/Claude token ledger (logs/spend.jsonl); dollars derived at read time
│   ├── sequence_detect.py      # offline forensic join of audit.jsonl + spend.jsonl on query_hash (CLI only)
│   ├── authn.py                # per-user auth primitives: scrypt hashing, lockout arithmetic, id generation
│   ├── authn_store.py          # users/sessions/device_tokens backend (CYCLAW_AUTH_DB_URL)
│   ├── authn_manager.py        # AuthManager — ties authn.py + authn_store.py together; no HTTP awareness
│   ├── authn_cli.py            # cyclaw-user console script (local-only by construction)
│   ├── gen_cert.py             # cyclaw-gen-cert — self-signed cert + key with hostname/LAN SAN
│   └── telemetry_kill.py       # shared kill block — applied by gate.py, mcp_hybrid_server.py, retrieval/vector_store.py
├── schemas/                    # Pydantic API models (api.py; extra='forbid', strict)
├── scripts/                    # install-githooks.sh, check-pr-template.sh, measure_local_llm_throughput.py
├── deploy/                     # apparmor/ falco/ seccomp/ container-hardening profiles (all opt-in)
├── tests/
├── docs/
├── static/
├── data/
│   ├── corpus/
│   ├── personality/
│   └── agentic/                # skills_registry.json — governed store, ships empty
└── .github/workflows/

Every top-level package and directory above now carries its own README.md (map + traps + links to its authoritative doc); the tree omits most of them for brevity.


Dropbox Corpus Sync

An optional, out-of-band rclone-backed pull sync mirrors a Dropbox corpus into data/corpus/ without touching gate.py, graph.py, or the MCP request path. It carries safety fuses (max_delete, max_transfer), an OS-backed single-instance lock (fcntl.flock / msvcrt.locking) that keeps a scheduled run and a manual run from racing and releases even if the process dies, audit logging of changed corpus files, an optional reindex trigger, and scheduler glue for cron / Windows Task Scheduler plus an opt-in Darwin-only launchd backend (sync.scheduler_backend: "launchd", schedule_frequency daily/weekly/monthly) that generates the plist and prints the launchctl bootstrap command — never loading it itself.

python -m sync.cli setup          # first-run bootstrap; then: test | sync --dry-run | sync | status | schedule | unschedule

The Sync Console panel drives the same actions via POST /ops/sync (loopback-only, API-key gated, audited). Full setup and scheduling: docs/! How-To-Guides/Dropbox_Sync_Guide.md; module internals (lock lifecycle, exit codes, error taxonomy): docs/SYNC_README.md.


macOS launchd & Keychain (v1.9)

CyClaw's scheduled and supervised jobs on macOS run through generated launchd LaunchAgents with one uniform posture: every generator writes a plist from real resolved install paths and prints the exact launchctl bootstrap command — none of them ever loads the agent itself. Loading a background job is always a separate, explicit operator action.

  • Secrets never land in a plist. Token-bearing jobs chain macos/cyclaw-keychain-env.sh, which fetches the secret from the Keychain at process start, exports it, and execs the real command — failing closed (nothing launched) if the item is missing or empty. Store secrets first with macos/cyclaw-keychain-set.sh, a no-echo prompt driven by security itself so the secret never appears in any argv; the item is trust-pinned with -T /usr/bin/security.
  • Scheduled jobs — Dropbox sync (see Dropbox Corpus Sync), Telegram poll/health, fsconnect trash emptying, and OpenTweet — each a generate-only *-plist subcommand (commands below).
  • Supervised services (highest risk) — macos/generate_service_plist.py writes a KeepAlive LaunchAgent for gate.py or the harness console. Because that turns a loopback server into an always-on listener that survives reboot, it refuses to write without --confirm and a non-empty --reason (the reason-required idiom soul mutations use). Restart-on-crash only; a clean launchctl stop stays stopped.
  • Uninstall symmetrymacos/uninstall-cyclaw.sh unschedules any registered sync job and boots out + removes landed CyClaw LaunchAgents by label (telegram-poll, telegram-health, fsconnect-trash, gate, harness, keys-rotate, opentweet), so no background job outlives the install. Sync's own launchd job is owned by sync.cli unschedule.

Core commands

bash macos/cyclaw-keychain-set.sh com.cgfixit.cyclaw.telegram-bot-token   # store a secret (TTY prompt)
python -m telegram.cli poll-plist                                        # Darwin-only; generates, never loads
python -m telegram.cli health-plist
python -m agentic.fsconnect.cli trash-empty-plist
python -m opentweet.cli schedule-plist                                   # Darwin-only; generates, never loads
python macos/generate_service_plist.py --service gate \
    --reason "keep the RAG server up across reboots" --confirm
python macos/generate_service_plist.py --service harness \
    --reason "keep the coding console up across reboots" --confirm

Script-by-script reference: macos/README.md. Design and phase ledger: docs/work/MACOS_LAUNCHD_INTEGRATION_PLAN.md.


Agentic Layer (v1.6.0)

CyClaw now includes a concise, governed agentic layer for local operator workflows. It is opt-in, disabled by default, and fully out-of-band: it is never imported by gate.py, graph.py, or mcp_hybrid_server.py. The coding console (harness/) is a sibling package, not a subpackage; data/agentic/skills_registry.json is a governed store that ships empty (the harness reads it, apply-skill writes it). Package guide: agentic/README.md.

What it adds: read-only GitHub context through the gh CLI (invoked as an argv list, never via a shell; no GitHub token is stored or forwarded by CyClaw), a governed local skills registry with explicit human gating, and the operator workflows under .claude/ (.claude/README.md). All agentic reads, refusals, and registry changes are audit logged.

The GitHub write path (gh pr create --draft) is implemented and its code-level gate EXECUTION_ENABLED ships True (operator-signed 2026-08-07), as do mode: "write" and writes_enabled. The layer master switch agentic.enabled still ships false, so a default checkout cannot open a PR — that switch, plus a per-call reason and confirm, is what refuses. Procedure and the CYCLAW_AGENTIC_WRITE_DISABLE rollback: docs/agentic/GITHUB_WRITE_ENABLEMENT.md.

Enable it

agentic:
  enabled: true                  # the one edit this block asks you to make
  repo: "cgfixit/CyClaw"
  mode: "write"                  # ships open since 2026-08-07
  writes_enabled: true           # ships open since 2026-08-07
  gh_min_version: "2.40.0"
  registry_path: "data/agentic/skills_registry.json"

Main agentic commands

python -m agentic.cli status
python -m agentic.cli context --repo
python -m agentic.cli context --pr 123
python -m agentic.cli context --issue 45
python -m agentic.cli test
python -m agentic.cli propose-skill --name deploy --desc "..." --body-file s.md --reason "draft"
python -m agentic.cli apply-skill --name deploy --desc "..." --body-file s.md --reason "add deploy runbook" --confirm

The Agentic Console panel drives these from the terminal UI via POST /ops/agentic; skill-Apply is refused under shipped defaults by agentic.enabled: false (the CLI no-ops while it is off), and a per-call reason + --confirm remain mandatory once it is on.


Filesystem, SQL & Passive Network Connectors (v1.8+)

v1.8 extends the agentic layer beyond GitHub to local data, for the regulated or security conscious use case where AI use is compliance heavy. All three connectors are opt-in, disabled by default, and fully out-of-band — never imported by gate.py, graph.py, or mcp_hybrid_server.py, so the six security invariants hold by construction. While disabled, their CLIs are a pure no-op (exit 0).

agentic/fsconnect/ — local / SMB filesystem connector

Scoped reads and separately-gated writes over a local or SMB share, sharing one held-handle security core (pathsafe.py): POSIX descends with openat / O_NOFOLLOW from a held root fd; Windows read/list/stat locks the canonical root ancestry against rename, opens once with CreateFileW, verifies GetFinalPathNameByHandleW containment, and consumes that same handle — Windows writes remain hard-refused. UNC, NTFS alternate data streams (file::$DATA), \\?\ / \\.\ device paths, .. traversal, and symlink / reparse traversal are denied; segment-aware containment closes CVE-2025-53110 (sibling-prefix), and held-handle authority prevents name-reopen junction swaps.

  • Reads (fs_list / fs_stat / fs_read / fs_grep / fs_glob / fs_largest) are confined to allowed_roots, audited, capped at 5 MiB, and content-scanned (OWASP ∪ banned_patterns, advisory).
  • Writes (fs_write / fs_append / fs_mkdir / fs_move) ship writes_enabled: false; confined to a separate writable_roots list; gated by a human reason (+ --confirm for destructive ops); atomic (tmp + os.replace); content-agnostic (never calls the LLM). A code-level FS_WRITE_HARD_DISABLE kill switch forces dry-run regardless of config.
  • Toggleable RAG-corpus indexing of the share (index_enabled, dry-run default) stages eligible files into the corpus and triggers a reindex subprocess — a generate → write → index loop without importing retrieval.
python -m agentic.fsconnect.cli status
python -m agentic.fsconnect.cli read  --path "<path>"                     # scoped read
python -m agentic.fsconnect.cli grep  --path "<path>" --pattern "<pattern>"
python -m agentic.fsconnect.cli largest --path "<dir>" --top 20 --min-bytes 1048576
python -m agentic.fsconnect.cli write --path "<path>" --reason "..."      # dry-run unless writes_enabled
python -m agentic.fsconnect.cli index --apply           # stage share → corpus
python -m agentic.fsconnect.cli test                    # pre-flight self-test

Enable in config.yaml:

fsconnect:
  enabled: true
  allowed_roots: ["/srv/share"]   # REQUIRED when enabled; existing dirs
  max_file_bytes: 5242880         # 5 MiB read cap
  largest_max_entries: 100000     # truthful traversal ceiling per largest command
  writes_enabled: false           # master write switch (dry-run plans while false)
  writable_roots: [null]          # null => ~/CyClaw-FS (macOS) | /var/lib/cyclaw-fs (Linux) | C:\CyClaw-FS
  max_write_bytes: 10485760       # 10 MiB write cap
  index_enabled: false            # toggle RAG-corpus indexing of the share

agentic/sqlconnect/ — read-only SQL connector (v0.1 scaffold)

Read-only on-prem SQL (Postgres / MSSQL), enforced three ways: a SELECT/WITH-only query guard (rejects DDL/DML, stacked statements, and comment-hidden keywords by scanning a quote-stripped copy), a session-level read-only transaction, and a hard allow_write: false. The DSN comes from an environment variable only (CYCLAW_SQL_DSN); drivers (psycopg / pyodbc) import lazily. The quote-stripping scan is a single left-to-right pass that gives '...', "...", [...], and Postgres $tag$...$tag$ quoting the same precedence the database does — an earlier regex-alternation version could be fooled by a quote nested inside a different quoting form (e.g. $$'$$) into treating a stacked DROP as part of one SELECT; the other two layers were never affected.

python -m agentic.sqlconnect.cli status
python -m agentic.sqlconnect.cli schema                 # list table schemas (read-only)
python -m agentic.sqlconnect.cli query --table public.users   # bounded preview
python -m agentic.sqlconnect.cli test
sqlconnect:
  enabled: false
  driver: "postgres"             # "postgres" | "mssql"
  dsn_env: "CYCLAW_SQL_DSN"      # DSN from this env var only
  statement_timeout_ms: 5000
  max_rows: 1000
  allow_write: false             # reserved; v0.1 cannot write regardless

agentic/netconnect/ — passive LAN inventory (v0.1 scaffold)

Reports best-effort local host metadata and reads the OS's existing ARP/neighbor cache. No ping, port probe, subnet sweep, packet send, scheduling, or request-path integration; every returned IPv4 address is filtered through operator-supplied CIDRs that must be subnets of RFC1918 or loopback space.

python -m agentic.netconnect.cli status
python -m agentic.netconnect.cli self
python -m agentic.netconnect.cli arp
python -m agentic.netconnect.cli test
netconnect:
  enabled: false
  allowed_cidrs: ["192.168.1.0/24"]
  allowed_net_ops: [self, arp]
  command_timeout_sec: 5
  max_neighbors: 512

NeMo Guardrails (v1.8)

An opt-in content-safety layer in guardrails/ (package README). Absence of the guardrails: block, or enabled: false (the shipped default), is a pure no-op. When enabled, utils/guardrail_bridge.py wires two visible graph.py nodes — guardrail_input (after route_by_score) and guardrail_output (after generation; grounding check on the local_llm path only) — still defense-in-depth only, never a routing authority: the graph's own edges decide where a blocked query goes. gate.py / graph.py / mcp_hybrid_server.py never import guardrails directly (I6). Status table: docs/NeMo/README.md.

nemoguardrails is an optional dependency: the layer soft-imports it and, when absent, degrades to offline heuristic rails that need no second LLM call — an input rail (prompt-injection marker scan + soul/identity-mutation intent detection, the content-layer arm of the Soul-Governance invariant) and an output rail (token-overlap grounding check against the retrieved context, flagging likely-hallucinated answers below hallucination_threshold). When nemoguardrails is installed, the same Python checks back the live NeMo actions via the Colang flows in guardrails/config/rails.co, so offline heuristics and live rails never drift. Decisions go to a separate metrics stream (logs/guardrails.jsonl) that stores only SHA-256 hashes.

python -m guardrails.cli status | check "your query here" | metrics | test

Config keys (guardrails.enabled, engine, model, hallucination_threshold, metrics_path) and their shipped values live in config.yaml; the status table, phased history, and rail semantics are in docs/NeMo/README.md.


Agentic Harness Scaffold (v1.9)

A governed, opt-in, disabled-by-default, out-of-band scaffold for two related capabilities — agentic/harness_optimizer/ (a better-harness-style optimizer with train/holdout scoring and a hard acceptance gate) and agentic/deepagent_github/ (a LangChain Deep Agents-backed local GitHub coding harness, lazily importing deepagents only when enabled). Phases 0–9 are implemented and tested (tests/test_agentic_harness_*.py; phases 6–9 landed in PR #515, 2026-07-13, documented in docs/work/DEEP_AGENT_HARNESS_PHASES_6_9.md; full plan and phase ledger in docs/work/GITHUB_DEEP_AGENT_HARNESS_OPTIMIZER_PLAN.md).

Superseded 2026-08-01. Phase 9's security gate was subsequently satisfied and P10 landed, so "not authorization to add an executor" no longer describes the current tree: a sandboxed verification executor (agentic/executor/) and a draft-PR write path (agentic/writer.py::execute_write) both exist, and the live real-repo coding pipeline is agentic/real_repo_loop.pynot the deepagents-backed graph this section describes. Both new capabilities still ship disarmed. See GitHub Agentic Coding Harness below for what is actually wired today.

Every gate below the master agentic.enabled switch that arms a run (deepagent_github.enabled, allow_deepagents_dependency, allow_filesystem_write_tools, allow_shell_execution, allow_github_writes, harness_optimizer.enabled) ships false in config.yaml; the three cloud-provider switches ship armed but unreachable behind those masters (see the next-but-one section). While disabled, nothing under either package is reachable from agentic.cli, and no deepagents/langchain optional dependency is imported.


Coding Harness Console (v1.9)

A grok-build-style local coding console, shipped as the strictly out-of-band harness/ package (Windows merged 2026-07-22; macOS/Linux port merged 2026-08-01). Like agentic/, sync/, and guardrails/, it is never imported by gate.py, graph.py, or mcp_hybrid_server.py and never imports them (invariant I6).

harness/ itself is pure Python with no OS branch in its request-handling path — same routes, same slash commands, same security posture everywhere. Only the install/launch glue is platform-coupled, which is why there are two sibling script trees (powershell/, macos/) rather than one abstraction.

  • Launch: python -m harness.server serves the console at http://127.0.0.1:8790 (static/harness.html); loopback-only bind, non-loopback hosts refused. gate.py keeps :8787. (cyclaw-harness is the same entry point after pip install -e . — see Installation.)
  • Install: Windows — powershell -ExecutionPolicy Bypass -File .\powershell\Install-CyClaw.ps1 (home, venv, cyclaw.cmd PATH shim, profile function). macOS / Linux — bash ./macos/setup-cyclaw.sh (single entry point) or bash ./macos/install-cyclaw.sh (installer alone; branches on uname -s to install the correct plain torch on macOS, the step a hand-install most often gets wrong; bash 3.2 and zsh, BSD userland, no Homebrew dependency). Uninstall with bash ./macos/uninstall-cyclaw.sh (--remove-home, --remove-fsconnect); it also removes every landed CyClaw LaunchAgent (see macOS launchd & Keychain).
  • Chat: talks to the local model through the OpenAI-compatible /v1 endpoint from config.yaml's models.local_llm.base_url — no keys, no login, offline. Every reply shows token counts; sessions persist as human-inspectable JSON with atomic writes.
  • Reuse, not duplication: GitHub actions go through the same utils.ops_runner subprocess shim as /ops/agentic (read mode by default); /skills and /tools are wiring diagrams (what this console actually injects, runs, or has registered) — not a dump of every file on disk. MCP hybrid_search appears under /tools all as catalog-only; the console does not invoke it. /goal is session data in the system prompt; /loop is a human-gated sequence of /api/chat turns toward that goal and never starts /api/agent/*. /web is allowlist-only GET, off by default, no search engine. The governed data/agentic/skills_registry.json catalog is still merged into GET /api/registry (read-only here). The system prompt is composed from the repo's own ponytail + karpathy-guidelines skills, with the governed soul appended read-only when enabled, plus optional /goal and /web inject extracts.

Full setup, slash-command reference, home layout, and security posture: harness/README.md, docs/HARNESS_POWERSHELL.md (Windows), and docs/HARNESS_MACOS.md (macOS/Linux — only what genuinely differs: install glue, the torch build, git-credential-osxkeychain, and the note that pathsafe.ScopedRoots' POSIX openat/O_NOFOLLOW containment is the stronger branch).


GitHub Agentic Coding Harness (v1.9)

The real-repo coding pipeline: clone → plan → patch → verify → human decides → commit, with pushing and opening a draft PR as two further, separate decisions. Driven by agentic/real_repo_loop.py, which fuses three previously-independent pieces — the planner's model call, a jailed real clone (agentic/deepagent_github/repo_workspace.py), and the sandboxed verification executor (agentic/executor/). Out-of-band like every other agentic feature: never imported by gate.py, graph.py, or mcp_hybrid_server.py (invariant I6).

It ships held, not disarmed. The three switches that gate whether a run happens at all — agentic.enabled, deepagent_github.enabled, allow_git_write_tools — ship false. The write-path constant agentic/writer.py::EXECUTION_ENABLED and the cloud switches (mode: "write", writes_enabled, allow_cloud_providers, both providers) ship open since the signed enablement of 2026-08-07, so on a default checkout it is the master switches plus a per-call reason/confirm that refuse (see "already armed, waiting on the master switches" below).

How a run works

  1. real-repo-run-plan (optional, two-stage) asks a model for a plan and prints it — clones, writes, and commits nothing. You read and edit the plan and feed it back with --plan-file, so one model plans, a human approves, and another model codes against the approved text. The plan is injection-scanned on load, truncated at 6,000 chars, and its SHA-256 is recorded on the run.
  2. real-repo-run clones the configured repo into a jailed workspace, asks the planner for whole-file replacements, writes them, runs the selected checks, and stops before committing (status: pending_decision; a run that never passes reports exhausted).
  3. real-repo-run-decide --decision approve commits locally; reject discards. Neither pushes.
  4. real-repo-run-push puts the claude/* branch on origin.
  5. real-repo-run-publish opens a draft PR (gh pr create --draft).
  6. real-repo-run-discard reclaims a decided or orphaned run's clone (reject and exhausted free theirs immediately; only an approved run keeps its clone, since push and publish still need it).

Each escalation is its own command and its own decision, deliberately not folded into approve.

Security posture

  • Diff-scope gate. A candidate that writes into any of config.yaml's agentic.deepagent_github.protected_write_paths (the tests, CI, lint, and config files that judge the candidate's own acceptance — the classic reward-hacking failure of a make-the-checks-pass loop) is refused outright, and writes are budget-capped (max_write_budget_bytes).
  • Two scanners, two questions, on the same bytes. Proposed content gets an injection scan (is this trying to talk to a model?) and a code-shape scan (inspect_code_shape; scan_code_shape ships true) that matches combinations — a secret path plus network egress, a decode plus dynamic exec, a socket plus fd-dup or a shell path, a pipe-to-shell — because a working key-exfiltration payload contains no injection phrase at all. Every hit is CRITICAL and refuses the candidate.
  • Verification runs as argv-list subprocesses, never a shell: cwd pinned to the clone, a scrubbed env allowlist (PATH, LANG, LC_ALL, PYTHONPATH, VIRTUAL_ENV, PYTHONIOENCODING) plus a disposable HOME/USERPROFILE, forced NO_PROXY=* / PIP_NO_INDEX=1, and a 120s per-check timeout. Every non-empty check list runs inside a required, fail-closed hard sandbox (hard_sandbox.py: Windows Job Object with KILL_ON_JOB_CLOSE, Darwin sandbox-exec denying network and off-cwd writes, Linux unshare --net) — a missing binary or failed capability probe raises HardSandboxUnavailable, with no silent fallback. Residual limits (no microVM; Windows is a process-tree kill, so sockets keep working there) are in docs/THREAT_MODEL.md's executor amendments.
  • The console sends check-profile names, never argvharness/agent_policy.py resolves them against a fixed allow-list (pytest, ruff, invariant-guard, config-guard); a body that could carry argv would make an authenticated route a remote shell.
  • push_branch passes no credential. Its env allowlist deliberately excludes GH_TOKEN/GITHUB_TOKEN because that environment is shared with the executor; it authenticates only via a HOME-resident credential helper (gh auth setup-git). Branch names are forced into the PR-template vendor namespaces (utils/agent_identity.py) and run_id is validated as 32-char lowercase hex before it can become an argv element.
  • Optional offline slop-detection nudge (unslop.enabled, ships false): hits become feedback appended to the next planning prompt, never a gate.

Enable it

The block below is what config.yaml ships. The three switches that gate whether a run happens at all — agentic.enabled, deepagent_github.enabled, allow_git_write_tools — ship false; the three cloud switches ship true, armed alongside models.grok / models.claude on 2026-08-07 (docs/THREAT_MODEL.md's eighth amendment). Read it as "already armed, waiting on the master switches," not as "off": reaching a cloud provider still needs the two masters, the provider's API-key env var, and a per-run --confirm-online.

agentic:
  enabled: false                        # master switch -- ships closed
  deepagent_github:
    enabled: false                      # ships closed
    allow_git_write_tools: false        # gates every write/commit/push in the clone
    model: "qwen3.8:27b-mlx"                # local planner model; cite models.local_llm.model
    workspace_root: "data/agentic/workspaces"
    max_write_budget_bytes: 100000
    max_handoff_chars: 200000           # outbound-prompt cap for cloud egress
    planner_max_tokens: 3072             # real-repo completion cap; keep it within Ollama num_ctx
    allow_cloud_providers: true         # gate 3 of the cloud chain -- ARMED
    providers:
      grok:   { enabled: true, model: "grok-4.5" }        # ARMED
      claude: { enabled: true, model: "claude-sonnet-5" } # ARMED

Setting a provider enabled: true while allow_cloud_providers is false is a config error, not a silent no-op — the three move together. Opening a PR needs agentic.mode: "write", writes_enabled: true, and agentic/writer.py's EXECUTION_ENABLED (all ship open) and agentic.enabled: true plus a per-call reason and confirm; the arming checklist and the CYCLAW_AGENTIC_WRITE_DISABLE rollback are in docs/agentic/GITHUB_WRITE_ENABLEMENT.md.

Commands

# Optional stage 0: plan, review by hand, then hand the approved text to the coder.
python -m agentic.cli real-repo-run-plan \
  --pr 123 --instruction "fix the off-by-one in the parser" --out plan.md

python -m agentic.cli real-repo-run \
  --pr 123 --instruction "fix the off-by-one in the parser" \
  --read-file src/parser.py --checks-file checks.json \
  --plan-file plan.md \
  --branch claude/parser-fix --commit-message "fix: off-by-one" \
  --reason "triage issue 123" --confirm

python -m agentic.cli real-repo-run-status  --run-id "<32-hex>"
python -m agentic.cli real-repo-run-decide  --run-id "<32-hex>" --decision approve
python -m agentic.cli real-repo-run-push    --run-id "<32-hex>"
python -m agentic.cli real-repo-run-publish --run-id "<32-hex>" --reason "..." --confirm
python -m agentic.cli real-repo-run-discard --run-id "<32-hex>"

Exit codes are an API: 0 ok · 2 failed · 3 env/config · 4 write refused. real-repo-run exits 0 whether or not a candidate was accepted — the record's status field carries that.

From the harness console

Seven routes on 127.0.0.1:8790. GET /api/agent/checks is open (a hardcoded allow-list, spawns nothing); the other six — POST /api/agent/run, GET /api/agent/runs/{id}, POST /api/agent/runs/{id}/{decision,push,publish,discard} — require a Bearer CYCLAW_API_KEY plus an Origin/Sec-Fetch-Site cross-site check. POST /api/agent/run is deliberately synchronous (the run record and its run_id first exist in the response), and its wall-clock budget is derived from the request — iterations × planner_timeout + iterations × checks × 120s + 300s, capped at 3600s — because a flat budget was a real bug: subprocess.run(timeout=) sends an uncatchable SIGKILL, so a request whose own planner budget exceeded it left a leaked clone and a permanently running record. Console equivalents: /agent run|confirm|status|approve|reject|push|publish|discard.

Optional cloud planner (Grok / Claude)

The loop is local-only by default, and the local path (no --provider flag) needs nothing beyond the base install — LocalProposerClient is a plain httpx call and nothing on that path imports deepagents or langchain. --provider grok|claude --confirm-online drives the loop with a cloud model behind a six-condition chain: agentic.enableddeepagent_github.enabledallow_cloud_providersproviders.<name>.enabled → the provider's API-key env var (GROK_API_KEY / ANTHROPIC_API_KEY, presence only, never a network probe) → per-run --confirm-online. Every outbound prompt is injection-scanned, redacted, hashed, and audited as egress before it leaves the process.

Cloud SDKs are opt-in extras, deliberately absent from the default install, requirements.txt, and the Docker image:

pip install -e ".[agentic-deepagents]"                          -c constraints.txt   # Claude only
pip install -e ".[agentic-deepagents-cloud]"                    -c constraints.txt   # Grok only — just langchain-xai
pip install -e ".[agentic-deepagents,agentic-deepagents-cloud]" -c constraints.txt   # both

full (what CI and dev boxes get) pulls agentic-deepagents but deliberately not agentic-deepagents-cloud, so a machine that never touches Grok never carries langchain-xai; [all] is the only extra that installs both. The published Docker image installs requirements.txt only, so running this feature in a container means installing on top (pip install -e . for local mode, or one of the commands above for cloud).


Telegram Channel (v1.9)

CyClaw includes an optional, out-of-band Telegram channel (telegram/, shipped enabled: false) that gives the single trusted operator a phone-reachable remote — outbound notifications and, when configured, allowlisted two-way chat — without touching gate.py, graph.py, or the MCP request path (invariant I6). Inbound chat text only ever reaches the RAG pipeline via loopback POST /query, never a direct call into graph.py.

Outbound notify (mode: "notify") or long-poll two-way chat (mode: "chat", the shipped YAML, still enabled: false; long-poll only, no public webhook listener beside the loopback server; T1-first remains the recommended enable order). allowed_chat_ids is required non-empty when enabled, and the bot token comes only from the env var named by bot_token_env (TELEGRAM_BOT_TOKEN), never from YAML. T3 hybrid-confirm consent (allow_hybrid_confirm: false by default): the exact private-chat command /online on <grok|claude> is the only way chat text can set user_confirmed_online, for one next message only (hybrid_confirm_ttl_sec, hard-capped at 300s) — core's triple gate remains the final authority. T4 media staging (media.enabled: false) accepts private-chat attachments captioned /save --confirm <reason> only through the existing agentic/fsconnect write path. The poll-plist / health-plist generators never load, and their secrets are injected at process start by the Keychain wrapper (see macOS launchd & Keychain).

Core commands

python -m telegram.cli status
python -m telegram.cli test
python -m telegram.cli send --chat-id "<id>" --text "..."   # T1; add --dry-run to preview
python -m telegram.cli poll                                # T2; requires telegram.mode: chat
python -m telegram.cli poll-plist                          # Darwin-only; generates, never loads
python -m telegram.cli health-plist                        # Darwin-only; generates, never loads

See docs/channels/TELEGRAM_DESIGN.md for architecture, the T0–T4 phase ledger, and the threat-model obligations (docs/THREAT_MODEL.md's seventh amendment), and telegram/README.md for package internals.


OpenTweet Channel

Optional out-of-band X poster (opentweet/, shipped enabled: false). gate.py, graph.py, and the MCP server never import it (invariant I6). Generation is loopback POST /query with user_confirmed_online: false. The default write is an OpenTweet draft; scheduled_date is opt-in via opentweet.schedule_enabled. Schedulers never send publish_now.

Core commands

python -m opentweet.cli status
python -m opentweet.cli test
python -m opentweet.cli post --topic "..."          # add --dry-run to preview
python -m opentweet.cli schedule-plist              # Darwin; generates, never loads
python -m opentweet.cli schedule-task               # Windows; generates, never registers

See docs/channels/OPENTWEET_DESIGN.md and opentweet/README.md. Keychain/CredMan wrappers are in macos/README.md and powershell/README.md.


Security Model

Layer Mechanism
Network Binds 127.0.0.1:8787 — no external exposure by design
Input Config-driven injection filter (policy.prompt_filter)
Rate limit 60 req/min per IP
Proxy bypass All httpx clients set trust_env=False — ambient HTTP(S)_PROXY/.netrc cannot reroute local traffic, see the path-embedded Telegram bot token, or carry GROK_API_KEY / ANTHROPIC_API_KEY on a confirmed hybrid call (utils/health.py, llm/client.py local + Grok + Claude, harness/ollama.py, telegram/client.py, opentweet/client.py). This reverses the old “operator proxy governs paid egress” exception.
Telemetry Canonical kill maps (utils/telemetry_kill.py: telemetry + a visibly-separate update-check map, plus a removed-outright scrub set incl. the declarative-OTel config names) applied before any SDK import by every maintained Python chokepoint (invariant-guard G1 pins 15 orderings) AND delivered as literal environment before the interpreter starts at every process boundary — Docker ENV, the shipped launchers, generated launchd plists / Windows tasks / cron lines, and verifier/gh children via build_telemetry_safe_env; ONNX Runtime additionally gets the post-import disable_telemetry_events() call at its load seams (utils/onnx_telemetry.py). HF Hub network calls are also cut off once the embedding model is confirmed cached (retrieval/embeddings.py). Not a network kill switch: intentional policy-gated egress is classified separately in SECURITY.md
Audit All paths log SHA-256 query hash + PII-redacted metadata
Grok gating Triple gate: mode=hybrid AND grok.enabled=true AND user_confirmed_online=true
Claude gating Same triple gate, independently: mode=hybrid AND claude.enabled=true AND user_confirmed_online=true
Soul writes Explicit human reason string + enforced write-boundary scan + atomic write
Agentic writes pr_create implemented; the source constant and two config gates ship open since 2026-08-07, so agentic.enabled (ships false) plus per-call reason/confirm is what refuses. pr_comment/issue_comment remain plan-only. Git-level writes (real-repo commit/push and draft-PR publish) are additionally gated on deepagent_github.allow_git_write_tools, which ships false
Filesystem connector Reads scoped to allowed_roots (5 MiB cap) with POSIX held-fd descent and Windows same-handle containment; writes default-OFF and hard-refused on Windows, otherwise confined to separate writable_roots, gated by human reason + --confirm, and atomic; UNC/ADS/device-path/../symlink escapes are denied
SQL connector Read-only: SELECT/WITH-only query guard + session read-only + hard allow_write: false; DSN from env var only; disabled scaffold by default
Network connector Passive only and disabled by default; explicit RFC1918/loopback CIDRs; self plus existing OS neighbor-cache reads; no ping, sweep, port probe, packet send, scheduler, or /ops route
Guardrails Out-of-band, opt-in defense-in-depth; degrades to offline heuristic rails without nemoguardrails; never a routing authority; separate hash-only metrics stream
Telegram channel Out-of-band, ships enabled: false; non-empty allowed_chat_ids allowlist required to arm; inbound chat reaches the pipeline only via loopback POST /query (never a direct graph.py call); T3 hybrid-confirm consent (allow_hybrid_confirm) ships off — only an explicit `/online on <grok
OpenTweet channel Out-of-band, ships enabled: false; answers only via loopback POST /query with user_confirmed_online: false; default write is a draft; schedulers generate-don't-load and never send publish_now; API key from env / Keychain / CredMan, never YAML or a plist EnvironmentVariables dict
launchd secrets (macOS) Generated plists never embed tokens — macos/cyclaw-keychain-env.sh injects secrets from the macOS Keychain at exec time and fails closed when the item is missing; cyclaw-keychain-set.sh stores them via a no-echo security prompt so the secret never appears in argv or the plist; the gate/harness supervised-agent generator additionally requires --confirm + a non-empty --reason
/ops/* routes Loopback-only, require_api_key gated, rate-limited (60/min), every call audited (ops_sync_executed / ops_agentic_executed / ops_fsconnect_executed / ops_sqlconnect_executed); shells out via subprocess.run([...]) — never imports sync/ or agentic/
/auth/* routes Per-user auth design (gate_auth.py, docs/AUTHENTICATION_DESIGN.md); first-boot GET /auth/setup-status (no credential; same-origin-checked) and loopback-only POST /auth/bootstrap-password; session cookie + CSRF for browsers, bearer device tokens for programmatic clients; three roles (admin/operator/audit) gate the /auth/users* admin surface, with the last enabled admin protected from disable/delete/role-change; every /auth/* handler checks auth.enabled first and returns 503 (not 404) so route presence never discloses whether the feature is on. When auth.enabled is true, POST /query requires a session or named device token
/memory/* + /query/export/html routes Optional, default-off memory admin surface (gate_memory.py); every memory: switch ships false; require_api_key gated, rate-limited; mutating routes (propose/apply/reject) require a non-empty reason string, with an injection scan on apply
Container Non-root, no-new-privileges, cap_drop: ALL, read-only rootfs, seccomp, resource limits; optional eBPF/Falco detection (deploy/falco/, off by default)

Docker / GHCR: published runtime image ghcr.io/cgfixit/cyclaw (tag-triggered). Operator guide, pull/run commands, Falco opt-in notes, and explicit non-goals (no microVM): docs/DOCKER.md. Host publish remains 127.0.0.1 only.

Scope: CyClaw is a trusted-operator, loopback-bound local server — one operator by default, a small set of mutually trusted operators with their own accounts and roles once auth.enabled is on, and single-tenant either way (everyone reaches the same corpus, soul, and model). LAN or WAN exposure is possible only through the threat model's documented bind exceptions (auth + TLS), never by default. The full threat model — what the sandbox does and does not cover (no microVM by design) and why — is documented in docs/THREAT_MODEL.md. The underlying design philosophy (telemetry kill, offline-first posture) lives in docs/security-philosophy/.


designed and built by Chris Grady, with AI tooling used under human review / CI / invariant-guard

source-available - all rights reserved

About

CyClaw is a secure, offline-first local AI agent built on three invariants: RAG-first retrieval, LangGraph topology as security policy - https://bit.ly/CyClaw-Demo

Topics

Resources

Security policy

Stars

6 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages