From cf06bdca67133f6768a16b8df450adb20e2919ca Mon Sep 17 00:00:00 2001 From: Prasenjit Sarkar Date: Tue, 7 Jul 2026 20:24:49 +0100 Subject: [PATCH] docs(readme): rewrite Semantic Memory section for v2 The README still described the pre-v2 memory system (wrong table names, the dead 'prepend to message' enrichment, no namespaces/decay/pinning, stale consolidation). Rewrite it to match what shipped: namespace scoping, decay/reinforcement/pinning, the 250ms-budgeted auto-recall with a recalled-context chip, LLM-backed consolidation with concat fallback, versioned storage, and the new overview/pin endpoints. --- .release-please-manifest.json | 2 +- CHANGELOG.md | 6 ++++++ README.md | 29 +++++++++++++++++++---------- package.json | 2 +- web/package.json | 2 +- 5 files changed, 28 insertions(+), 13 deletions(-) diff --git a/.release-please-manifest.json b/.release-please-manifest.json index 816df2d..0477999 100644 --- a/.release-please-manifest.json +++ b/.release-please-manifest.json @@ -1,3 +1,3 @@ { - ".": "0.3.1" + ".": "0.3.2" } diff --git a/CHANGELOG.md b/CHANGELOG.md index dfee548..7f323d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ > Campfire began as a fork of [the-companion](https://github.com/The-Vibe-Company/companion) and diverged into a separate product. Pre-fork history (versions up to 0.42.0) lives in the upstream repository; Campfire's own releases start at 0.1.0. +## 0.3.2 (2026-07-07) + +### Documentation + +* **readme:** rewrite the Semantic Memory section for v2 — namespaces (global/repo/session/agent), decay + reinforcement + pinning, the auto-recall enrichment and recalled-context chip, the JUDGE→DISTILL→CONSOLIDATE pipeline, versioned `fragments_v2`/`consolidated_v2` storage, and the new overview/pin endpoints. The previous copy described the pre-v2 (and partly dead) behavior + ## 0.3.1 (2026-07-07) ### Fixes diff --git a/README.md b/README.md index e252317..1603952 100644 --- a/README.md +++ b/README.md @@ -1203,17 +1203,17 @@ It operates as a non-blocking observer: no agent message is ever delayed by it, | Layer | What it does | |-------|-------------| -| **Semantic Memory** | Stores observations, decisions, and patterns from each agent session as vector embeddings in a local LanceDB database. Any session can query this shared knowledge base for relevant context before starting a task. | +| **Semantic Memory** | Stores observations, decisions, and patterns from each session in a local LanceDB database, scoped by namespace (global / repo / session / agent). Memories decay over time unless reused, and relevant ones are auto-recalled into a session's next prompt. See [Semantic Memory](#semantic-memory-details) below. | | **Deliberation Engine** | Proposes structured decisions across sessions (e.g. "which approach should we use?"). Connected viewers and agents can respond; the engine aggregates votes with role-weighted majority and resolves to `approved`, `rejected`, or `synthesized`. | | **Capability Discovery** | Each session self-reports its strengths, available tools, and context usage. When routing a task, the engine scores all connected sessions and picks the best fit. Confidence probes can be sent to agents in real time to verify self-reported capabilities. | | **Shared Context Stream** | A live think-aloud stream where agents can inject thoughts and observations. The engine detects semantic links (agrees, disagrees, builds on, contradicts) between fragments and tracks consensus scores across the session group. | **How it works end-to-end:** -1. When an agent produces output, the CI layer silently extracts observations and stores them as `MemoryFragment` records (with vector embeddings if an embedding provider is configured). -2. When a user sends a message, the CI layer queries the memory store for relevant context and prepends it to the message — giving agents access to knowledge from past sessions. +1. When an agent produces output, the CI layer extracts observations, decisions, and patterns and stores them as `MemoryFragment` records in the right namespace (with vector embeddings if an embedding provider is configured). Thinking-block content is scrubbed before anything is stored. +2. When a user sends a message, the CI layer recalls the most relevant memories from the repo, agent, and global namespaces (under a 250 ms budget so chat is never blocked) and prepends them to the message. The recalled items are shown to the user as a collapsible "recalled context" chip on that message. 3. Browser clients can send `memory_query`, `memory_store`, `deliberation_respond`, `route_task`, and `inject_thought` messages over WebSocket. The server handles them and broadcasts results back to all connected viewers. -4. Sessions consolidate their episodic memories into distilled `ConsolidatedKnowledge` entries when they end. +4. Episodic fragments are consolidated into distilled `ConsolidatedKnowledge` — triggered on turn boundaries, idle, session end, or manually — via a JUDGE → DISTILL → CONSOLIDATE pipeline (see below). **Embedding providers:** @@ -1225,16 +1225,25 @@ Vector search requires an embedding provider. Configure one in **Settings**: | `ollama` | `nomic-embed-text` (default) | 768 | Requires local Ollama instance | | `none` | — | — | Fragments stored without embeddings; metadata-only search | -Without an embedding provider, memory still works — queries fall back to a full scan filtered by session, repo root, tags, and type. +Without an embedding provider, memory still works — recall falls back to a recency- and confidence-ranked scan over the relevant namespaces (so recent repo decisions still surface, just without semantic similarity). + + + +**How Semantic Memory works:** + +- **Namespaces** — every memory lives in a scope: `global` (cross-repo conventions), `repo:` (one repository), `session:` (one session's episodic notes), and `agent:` (backend-specific quirks). Recall for a session draws from repo, agent, and global scopes with per-namespace depth limits. +- **Decay & reinforcement** — memories lose weight over time on a per-namespace half-life; being recalled reinforces a memory and extends its life (capped, so nothing becomes immortal). Low-weight, consolidated memories are eventually evicted. You can **pin** a memory so it never decays. +- **Consolidation (JUDGE → DISTILL → CONSOLIDATE)** — raw fragments are filtered and clustered locally, then distilled into durable knowledge by an LLM (via your configured OpenRouter key). If no key is set, it falls back to a simple concatenation, badged as such in the UI — consolidation is never blocked on configuration. +- **UI** — the Memory panel shows per-namespace counts and decayed-weight bars with pin/unpin controls; **Settings → Memory** exposes per-namespace decay half-lives, reinforce multipliers, and recall depths. **Storage:** -All CI data is stored locally under `~/.campfire/memory/lancedb/` — no external service required. +All CI data is stored locally under `~/.campfire/memory/` — no external service required. Schema is versioned via `meta.json` and migrated automatically from earlier versions. | Table | Purpose | |-------|---------| -| `fragments.lance` | Episodic memory fragments with embeddings | -| `consolidated.lance` | Distilled knowledge synthesized from sessions | +| `fragments_v2` | Episodic memory fragments (namespace-scoped, with embeddings when available) | +| `consolidated_v2` | Distilled knowledge synthesized from fragments | Capability data is stored as JSON in `~/.campfire/capabilities/` and learning history is appended to `~/.campfire/capability-learning.jsonl`. @@ -1260,12 +1269,12 @@ curl -X POST http://localhost:4567/api/sessions/route-task \ | Group | Endpoints | |-------|-----------| -| Memory | `GET/POST /sessions/:id/memory`, `GET /sessions/:id/memory/query`, `POST /sessions/:id/memory/consolidate`, `GET /memory/global` | +| Memory | `GET/POST /sessions/:id/memory`, `GET /sessions/:id/memory/query`, `GET /sessions/:id/memory/overview`, `POST /sessions/:id/memory/consolidate`, `POST /memory/pin`, `GET /memory/global` | | Deliberation | `GET /sessions/:id/deliberations`, `GET /sessions/:id/deliberations/:proposalId`, `POST .../respond`, `POST .../resolve` | | Capabilities | `POST /sessions/route-task`, `GET /capabilities`, `GET /capabilities/history`, `POST /capabilities/feedback` | | Shared Context | `GET /sessions/:id/context/stream`, `GET /sessions/:id/context/consensus`, `GET /sessions/:id/context/thread/:fragmentId` | -Architecture details are documented in [`CLAUDE.md`](CLAUDE.md). A design study for the next iteration of the memory layer lives at [`docs/design/semantic-memory-v2.md`](docs/design/semantic-memory-v2.md). +Architecture details are documented in [`CLAUDE.md`](CLAUDE.md). The full semantic-memory design (namespaces, decay, retrieval scoring, and the consolidation pipeline) is written up in [`docs/design/semantic-memory-v2.md`](docs/design/semantic-memory-v2.md). --- diff --git a/package.json b/package.json index 6e77f89..3431f70 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "the-campfire-workspace", - "version": "0.3.1", + "version": "0.3.2", "private": true, "description": "Workspace root for Campfire \u2014 the collaborative web platform for AI coding agents. The published package lives in web/.", "scripts": { diff --git a/web/package.json b/web/package.json index 597fc43..f0e679a 100644 --- a/web/package.json +++ b/web/package.json @@ -1,6 +1,6 @@ { "name": "the-campfire", - "version": "0.3.1", + "version": "0.3.2", "type": "module", "description": "Campfire \u2014 collaborative web platform for AI coding agents. Run Claude Code, Codex, Goose, Aider, and more from one browser UI with real-time collaboration, permission voting, and automation.", "license": "MIT",