diff --git a/TRACKER.md b/TRACKER.md index 342eaff..a9dfc23 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -1,36 +1,32 @@ # AgentContextOS — Build Tracker -> **How to use:** Update `Status` and `PR` columns when a step lands on `main`. -> In a new session, read this file first to instantly know where we are. +> **How to use:** Read this first in every session to know exactly where the build stands. +> When a step lands on `main`: flip its **Status** + **PR**, add a row to [PR & Branch History](#pr--branch-history), and refresh the [Status](#status) block. -> **Standing requirement (all steps):** Every deliverable — scripts, tooling, CI, dev-stack commands — -> must work on **Windows, macOS, and Linux** without WSL. -> Use `Taskfile.yml` for task targets (cross-platform), `Makefile` for Unix convenience. -> CI matrix always includes `ubuntu-22.04`, `macos-14`, and `windows-latest`. +> **Standing requirement (all steps):** Every deliverable — scripts, tooling, CI, dev-stack — must work on +> **Windows, macOS, and Linux** without WSL. Task targets via `Taskfile.yml` (cross-platform) + `Makefile` +> (Unix convenience). CI matrix always includes `ubuntu-22.04`, `macos-14`, and `windows-latest`. -**Last updated:** 2026-06-05 (**Step 5.6 ✅ complete** (#137–#142); **Step 5.7 underway** — slice **5.7a** A/B analyzer + tracker + dashboard in review; **Phase 5 6/7**) -**Current phase:** Phase 5 — Eval & Observability (**6 of 7 steps complete**). Phases 0–4 complete + Steps 5.1–5.6 → **63/84** steps. -**Next action:** **Phase 5 Step 5.7 — A/B testing & shadow mode** (the final Phase-5 step), delivered in vertical slices (5.7a–d). **5.7a (analyzer + tracker + dashboard) is in review** on `build/phase-5/step-5.7a-ab-analyzer`: the pure **`analyze_ab_experiment`** (`rag_config.eval`) computes lift + a normal-approximation Welch confidence interval + significance using stdlib `statistics.NormalDist` (no numpy/scipy — the drift-PSI / cost-z-score spirit) → **`ABAnalysisResult`** (`rag_core.eval`, additive, not in `dist/schemas`); **`ABExperimentTracker`** (`rag-observability`, beside `CostTracker`) is a pure per-`(experiment, variant)` sample *holder* decoupled from the analyzer (so observability keeps no `rag_config` dep), and the gateway composes the two at **`GET /v1/status/experiments`**. `cfg.experiments` is **opt-in** (A/B routing can change responses), wired inert by default in `build_app_from_config`; `ragctl experiments` is the smoke. **Observe-only** — no query-path change yet; shadow fan-out (5.7b) + variant routing (5.7c) feed the tracker next. 17 tests (analyzer + tracker + gateway + config) + ragctl smoke; mypy --strict (119) / ruff / RAG001 + the gateway/observability/config/eval suites green; `rag.schema` + `dist/openapi` regen. See [ADR-0032](docs/adr/ADR-0032-ab-testing-shadow-mode.md), [reference/experiments.md](docs/reference/experiments.md). **Next slice 5.7b** — shadow mode (fan out N% of live queries to a candidate retriever, observe-only, feed the tracker). Earlier, **Step 5.6 — Status & Metrics GUI (full build) ✅ complete** (#137–#142): Drift/Feedback + Cost cards, the Query Trace viewer, the regression bisector, drift+cost Grafana dashboards, and observability cross-links. Earlier, Step 5.5 ✅ delivered **drift monitors** — five monitors that catch retrieval degradation before users complain. A new **`rag-drift`** package (deps: `rag-core` + `rag-observability`) holds **`population_stability_index`** (a pure, binned, dependency-free **PSI** — `< 0.1` no shift / `0.1–0.25` moderate / `> 0.25` significant), **`DriftMonitor`** (bounded reference + current windows, **PSI** *or* **mean-drop** statistic, `ok`/`drifted`/`insufficient_data` verdict, `rebaseline`), and **`DriftMonitorRegistry`** (the five monitors — **query distribution / embedding PSI / retrieval score** via PSI over a scalar feature, **citation-clickthrough / faithfulness** via mean-drop; `observe` / `seed_reference` / `evaluate`). **Two statistics over one scalar-window abstraction**: each monitor observes a single float, so one PSI engine serves all three distribution monitors (embedding PSI uses the embedding **L2 norm**; per-dimension PSI is a production extension). **Infra-scoped** (one registry per deployment, like circuit breakers), held on `app.state`, **fed via `observe`** from the signals the gateway already computes: `/v1/query` appends query length + top retrieval score + the HyDE-embedding norm + the guard's grounded-claim fraction (`1 − blocked/claims` = faithfulness); `/v1/feedback` appends citation-clickthrough (1.0 per `citation_click`, else 0.0). Each observe is a cheap O(1) deque append behind a `None` check — **observe-only**, never alters a request — and sparse-by-default signals report `insufficient_data` until fed. **Detection happens on the dashboard poll**: **`GET /v1/status/drift`** calls `registry.evaluate()`, computing the **`DriftReport`** and emitting **`drift.detected`** *once per transition into drift* (a recovery is silent, then re-alerts — the `breaker.opened` transition-edge semantics) as a PII-free structured event **plus** a **`drift.detected` webhook** (the `WebhookEventType` reserved in Step 3.9 "for the future drift detectors") when `cfg.drift.alert_tenant` + a dispatcher are configured. **`POST /v1/status/drift/{metric}/rebaseline`** is the operator "accept the new normal" (404 disabled / 400 unknown metric); drift folds into `/v1/status/health` (any monitor over threshold → `degraded`). New frozen core types **`DriftSnapshot` / `DriftReport`** + `DriftMetric` / `DriftMethod` / `DriftStatus` StrEnums (`dist/schemas` + `dist/openapi` regenerated). **Inert by default** in `build_app`; `build_app_from_config` builds it from **`cfg.drift`** (`enabled` / `window_size` / `min_samples` / `psi_bins` / `psi_threshold` (0.2) / `mean_drop_threshold` (0.1) / `alert_tenant`; `dist/rag.schema.{json,yaml}` regenerated) and attaches the webhook dispatcher once it exists. Adds no governed SPI call so the PolicyEngine coverage linter passes with no new entry. `ragctl drift` seeds a baseline, feeds a shifted current, and prints the report. ~34 new tests (PSI units, monitor PSI/mean-drop/insufficient/rebaseline/bounding, registry observe/report/evaluate transition + webhook + recovery-realert, gateway dashboard/hook-drives-drift/rebaseline/404/400/disabled/health, config, ragctl); all gates green (ruff, mypy --strict 288 files, RAG001, log-schema/event-registry + schema/openapi-drift gates). Per-tenant drift, per-dimension embedding PSI, a background detection scheduler, and the admin-console drift card (Step 5.6 GUI) stay deferred. See row 5.5 below + [ADR-0030](docs/adr/ADR-0030-drift-monitors.md), [reference/drift.md](docs/reference/drift.md), [architecture/drift-monitors.md](docs/architecture/drift-monitors.md). Earlier, Step 5.4 ✅ delivered **online metrics & feedback** — closing the loop from the offline eval to the *online* signal. A new **`rag-feedback`** package (deps: `rag-core` + `rag-observability`, mirroring `rag-provenance`) whose **`FeedbackRecorder`** takes a feedback submission tied to a query (`request_id`), normalises the signal to a **`[-1, 1]`** satisfaction score, **PII-redacts** any free-text comment via an injected `PIIDetector`, persists a `FeedbackRecord` through the tenant-scoped **`FeedbackStore`** SPI (`NoopFeedbackStore`, in-memory FIFO ring), and emits a PII-free **`feedback.recorded`** event — all **degrade-open** (a store/redaction failure logs `feedback.record_degraded` and the endpoint still acks `stored=false`). **`POST /v1/feedback`** is **one polymorphic endpoint** for explicit (`thumb_up`/`thumb_down`/`rating`/`comment`) **and** implicit (`citation_click`/`answer_copied`/`regenerated`/`accepted`/`dwell`) signals — a single **`FeedbackSignal`** enum, `kind` inferred — with **body identity** like `/v1/query` (`request_id` ties to the query); an invalid signal / out-of-range rating → 422. Unlike provenance (which *hashes* text), feedback **redacts-and-keeps** the comment because its value is human-readable — the default gateway wires a pass-through **`NoopPIIDetector`** (the seam where production injects a real one), `comment_redacted` flags whether a redactor ran, and the event never carries the text. The pure **`aggregate_feedback`** folds a tenant's records into **`FeedbackStats`** (total / explicit-vs-implicit / sentiment buckets / mean score / by-signal) for the per-tenant **`GET /v1/status/feedback`** dashboard (operational surface alongside `/v1/status/quotas`, tenant via query-param / `X-Tenant-Id` header), with a presence-based `feedback` component in `/v1/status/health`. **Event-only observability** (no per-submission OTel span — the quota/breaker pattern — so `telemetry_attrs` is untouched). New frozen core types **`FeedbackRecord` / `FeedbackStats`** + `FeedbackKind` / `FeedbackSignal` StrEnums + gateway wire types **`FeedbackRequest` / `FeedbackAck`** (`dist/schemas` + `dist/openapi` regenerated). **Inert by default** in `build_app` (recorder/store `None` → the broad gateway suite untouched, the endpoint acks `stored=false`); `build_app_from_config` builds them from **`cfg.feedback`** (`enabled` (default true) / `redact_comments` / `max_comment_chars` / `max_records`; `dist/rag.schema.{json,yaml}` regenerated). Adds no governed SPI call (the recorder writes feedback + redacts text, neither a policed read) so the PolicyEngine coverage linter passes with no new entry. `ragctl feedback ` smoke command drives the full record (explicit + implicit) → aggregate flow against noop SPIs, creds-free. ~45 new tests (store conformance/isolation/bounding + recorder score/kind/redaction/degrade-open + aggregator + 12 gateway POST/dashboard/422/disabled/isolation + 4 config + 2 ragctl + event-registry); all gates green (ruff, mypy --strict 284 files, RAG001, log-schema/event-registry + schema/openapi-drift gates). A durable (Postgres) `FeedbackStore` for cross-worker aggregation, a real PII detector by default, and the admin-console feedback card (Step 5.6 GUI) stay deferred. See row 5.4 below + [ADR-0029](docs/adr/ADR-0029-online-feedback.md), [reference/feedback.md](docs/reference/feedback.md), [architecture/online-feedback.md](docs/architecture/online-feedback.md). Earlier, Step 5.3 ✅ delivered the **CI eval gate** — turning the Step 5.2 harness's deterministic numbers into a PR-blocking regression gate. A new standalone **`.github/workflows/eval-gate.yml`** runs the golden-set harness on every PR, compares the fresh metrics against a committed **`tests/eval/baselines/main.json`**, and posts a baseline-vs-current **diff table** as one sticky PR comment — failing the build when a gated metric breaches its floor *or* regresses beyond tolerance. The baseline is a new compact **`EvalBaseline`** (`rag_core.eval`): the five headline means + per-domain/per-difficulty breakdowns, **without** the volatile `run_id`/`created_at`/500-sample body — so re-capturing identical metrics yields a **byte-identical** file and the committed baseline only changes in review when the numbers actually move (the 11 k-line `report.json` would have churned every refresh). The gate runs **two config-driven checks** per gated metric, both from `thresholds.yaml` `hard_gates`: an **absolute floor** (`current ≥ min`, holds on the first run, shared with the harness `--check`) and a **regression delta** (`baseline − current ≤ max_regression_delta`, fails even above the floor so quality can't erode run-over-run). Only the three metrics this harness measures — **recall@10 / mrr / faithfulness** — gate; nDCG + citation precision are shown but never gate, and the `acl_violation_rate`/`pii_egress_rate` hard gates (enforced by the red-team suites) are silently skipped. The harness `load_thresholds` now **delegates** to the gate's `load_gate_thresholds`, so the floor the harness gates on and the floor CI gates on can never diverge. The pure comparison + threshold parsing + Markdown renderer join the metric library in **`rag_config.eval`** (`compare_to_baseline` / `render_gate_comment` / `GateThresholds` — no new package edge); the frozen result types **`EvalBaseline` / `MetricDelta` / `EvalComparison`** join `rag_core.eval` (additive, defaulted, and — like the other eval types — **not** in `dist/schemas`, so no schema-drift impact); the CI runner **`python -m eval.golden_set_v0.gate`** (`--comment-out` / `--update-baseline` / `--json-out`) lives beside the harness. The workflow is **deliberately standalone** (not folded into `ci.yml`) so it alone holds `pull-requests: write` (least-privilege — the cross-OS test matrix stays read-only) and **intentionally always-run** (no path filter) so "Eval gate" is a clean required status check (a path-filtered required check can hang "pending"; `ci.yml` solves that with its `ci-pass` aggregator, which a separate workflow can't `needs:`). The comment is posted **even on failure** (the gate step records pass/fail as a step output, a later step enforces) via a stable `` marker that `actions/github-script` finds-and-updates, and is **fork-PR tolerant** (`continue-on-error` so a read-only token never masks the verdict). The default full run passes the gate at recall@10 ≈ 0.906 / mrr ≈ 0.884 / faithfulness ≈ 0.929, exactly matching the committed baseline. ~16 new gate tests (threshold parse + baseline projection + floor/regression/improvement/None-skip comparison + pass/fail/sample-count comment rendering + committed-baseline integrity + committed-report↔baseline consistency + full-harness gate-pass) on top of the 5.2 suite; all gates green (ruff, mypy --strict 278 files, RAG001, log-schema/event-registry + schema-drift gates). Per-domain gating, a real-backend gate, and online/feedback signals stay deferred. See row 5.3 below + [ADR-0028](docs/adr/ADR-0028-ci-eval-gate.md), [reference/eval-harness.md](docs/reference/eval-harness.md), [architecture/ci-eval-gate.md](docs/architecture/ci-eval-gate.md). Earlier, Step 5.2 ✅ delivered the **offline golden-set eval harness** — completing the Step 0.8 skeleton (whose retrieval was a stub and faithfulness RAGAS-only) into a deterministic, offline, CI-stable eval. A new **`eval/golden_set_v0/`** harness (mirroring `agent_loop_v0` / `gateway_load_v0`; run `python -m eval.golden_set_v0.harness`) drives the **real `HybridRetriever`** over a synthetic **5-domain** corpus and reports the five metrics — **Recall@k, MRR, nDCG@k, Faithfulness, Citation Precision** — overall + **per-domain** + **per-difficulty**, writing `report.json` / `report.md` / `report.html`. The headline insight: the **noop backends give a real signal** — `NoopKeywordStore` already ranks by query-token overlap, and a deterministic **`HashingEmbedder`** (bag-of-words hash; `NoopEmbedder` returns zero vectors) feeds the dense path so cosine tracks token overlap, the two fusing via RRF exactly as in production — all seedless-deterministic, so the numbers are a stable Step-5.3 baseline. The **golden set is committed data**: `corpus.py` defines 100 concepts (20/domain) and `generate.py` expands each into 5 phrasings (easy/medium/hard by *retrieval* difficulty) → **500 queries / 5 domains / 100 each** under `tests/eval/golden/.jsonl`, **drift-gated** (`check_golden_set()`) like the schema gate; each concept is **3 passages** (`chunk_id == source_uri`, no side table) so Citation Precision + nDCG aren't degenerate. Metrics: **`ndcg_at_k`** joins the pure `rag_config.eval` library; faithfulness gets a **dependency-free `lexical_faithfulness`** default (per-claim token coverage, the lexical spirit of `rag_guard.LexicalNLIScorer`) so CI stays ML-free, with **RAGAS** the optional heavy upgrade. The pure metric library + **`render_html_report`** stay in `rag_config` (no new package edge); the runner lives in `eval/` where importing `rag_retrieval` is fine; **`run_eval`'s defaults are unchanged** (stub retrieval, faithfulness `None` without RAGAS) so the Step 0.8 contract + tests hold. `EvalMetrics` / `EvalReport` gain only **additive, defaulted** fields (`ndcg_at_k` / `domain` / `difficulty`; `ndcg_at_k_mean` / `by_domain` / `by_difficulty` via a new `GroupEvalMetrics`) — eval types aren't in `dist/schemas`, so no schema-drift impact. **`render_html_report`** emits a self-contained HTML page (inline CSS, no JS, no template engine) with threshold-coloured metric cards + per-domain/difficulty tables + a worst-samples table. `ragctl eval run` gains **`--html`** + an nDCG/per-domain output line; `eval show` shows nDCG + per-domain. The harness **`--check`** enforces the `thresholds.yaml` hard-gate floor now (recall@10 ≥ 0.70 / mrr ≥ 0.60 / faithfulness ≥ 0.80); the regression-delta + PR comment + committed `baselines/main.json` are Step 5.3. The default full run scores **recall@10 ≈ 0.906 / mrr ≈ 0.884 / nDCG ≈ 0.859 / faithfulness ≈ 0.929 / citation precision ≈ 0.272**, with real per-domain (0.82–0.98) + per-difficulty (easy > medium > hard) spread so regressions are detectable. `pythonpath` gains `"."` so `eval/` is importable in tests (no collision — `tests/` is a real package). ~40 new tests (nDCG + faithfulness metric units, 500-sample golden-set integrity + generator-drift gate, harness real-retrieval + threshold-check + HTML structure, `ragctl eval --html`/nDCG, schema-default coverage); all gates green (ruff, mypy --strict, RAG001, log-schema / event-registry + schema-drift gates). Real-backend eval, RAGAS-as-default, and the Step-5.3 regression gate stay out of scope. See row 5.2 below + [ADR-0027](docs/adr/ADR-0027-offline-eval-harness.md), [reference/eval-harness.md](docs/reference/eval-harness.md), [architecture/golden-set-eval.md](docs/architecture/golden-set-eval.md). Earlier, Step 5.1 ✅ delivered **per-query tracing & provenance** — the first Phase-5 step, making one query *explainable + trustworthy after the fact* via three composing parts. **(1) Stable span attributes.** Rather than mechanically rewrite all 21 pipeline-span call sites, the stability is stamped at the **one shared choke point**: `span_from_trace_context` now sets **`rag.schema_version`** on *every* span (provably total — every span uses the helper), and a new **`rag_core.telemetry_attrs`** registry names the base attribute keys (`rag.trace_id` / `rag.span_id` / `rag.sampled` / `rag.schema_version`) + a `SPAN_*` constant per pipeline span + `PIPELINE_SPANS`; the base set is the runtime contract, enforced by `packages/core/tests/test_telemetry_attrs.py` (and a registry-completeness drift test). The gateway entry spans (`gateway.query` / `gateway.retrieve`) are migrated to the constants and additionally stamp **`rag.request_id`** — the reference implementation; the remaining 19 spans' domain attributes stay documented literals (cosmetic migration deferred). Bump `SPAN_SCHEMA_VERSION` only on a breaking attribute change; additive keys don't. **(2) HMAC-signed provenance.** A new core-only **`rag-provenance`** package (deps: `rag-core` + `rag-observability`) holds `ProvenanceSigner` (HMAC-SHA256 over `f"{timestamp}." + record.model_dump_json()`, **modeled exactly on the webhook signer** so the platform has one signing model) and `ProvenanceRecorder` (build → sign → store → emit). New frozen core types `ProvenanceRecord` / `ProvenanceCitation` / `ProvenanceSignature` / `SignedProvenanceRecord` / `ProvenanceVerification` + `SpanRecord` (`dist/schemas` regenerated) and a `ProvenanceStore` SPI (ctx-first `put`/`get`, tenant-scoped + bounded) with `NoopProvenanceStore`; top-level `ProvenanceError` / `ProvenanceNotFoundError` (→ HTTP 404). rag-core 0.26 → 0.27. **Privacy by default** — the query + answer are persisted as **SHA-256 hashes**; raw text is opt-in (`capture_query_text` / `capture_answer_text`); citations are *locators* (chunk/doc id + source uri + score), never excerpt text. **Degrade-open** (the Phase-4 reliability invariant carried into Phase 5) — `capture()` traps every exception, logs **`provenance.record_degraded`**, and returns `None`; the response is already built and is returned regardless, so recording can never break a query. **Signing is optional but the record is not** — with no `signing_secret`, records are still captured + retrievable, just `signature = None`. Observability is the canonical **`provenance.record`** span (`rag.provenance.{signed,citations_n,degraded,elapsed_ms}`) + the pre-registered PII-free **`provenance.recorded`** event (ids + counts only). **(3) `GET /v1/query/{id}/trace`.** A new **`TraceCollector`** (an OTel `SpanProcessor` in `rag-observability`, the read-side sibling of Step 3.11's `MetricsCollector` / `LogTail`) buckets finished spans by the **`rag.trace_id`** already on every span — so capture needs **zero** call-site changes; the endpoint resolves that id from the signed record (looked up by `request_id`, **tenant-scoped via header identity**), verifies the signature fresh, and returns `QueryTraceResponse` { provenance, verification, spans }. A miss (unknown id / other tenant / disabled) is a 404; no auth is 401. **Wiring: inert by default, on via config.** The plain `build_app` leaves provenance unset (recorder/store/collector `None`) so the broad existing gateway suite + the `uvicorn` demo app are untouched (no new per-query log events, no global span-processor accumulation); `build_app_from_config` builds the three from `cfg.provenance` (enabled by default) and installs the collector. `cfg.provenance` (`enabled` / `signing_secret` (env-interpolated) / `capture_query_text` / `capture_answer_text` / `capture_spans` / `max_records` / `max_traces` / `max_spans_per_trace`) drives it from `rag.yaml` (`dist/rag.schema.{json,yaml}` regenerated); `ragctl provenance ` smoke command runs the full build → sign → store → verify → tamper-detect flow against noop SPIs, creds-free. Adds no governed SPI call (the recorder writes provenance, not a policed read) so the PolicyEngine coverage linter passes with no new allowlist entries. 47 new tests (signer sign/verify/tamper/wrong-secret/unsigned + recorder build/capture/degrade-open/capture-text/tenant-scope + store conformance/isolation/bounding + `TraceCollector` capture/group/bound/install + span-attribute contract + registry drift + 7 gateway trace-endpoint incl. cross-tenant/no-auth/disabled + 4 config + 4 `ragctl provenance`); all gates green (ruff, mypy --strict, RAG001, log-schema/event-registry + schema-drift gates). A Redis-backed `ProvenanceStore` (cross-worker retrieval, mirrors `RedisQuotaStore`), migrating the remaining 19 spans' domain attributes to registry constants, an admin-console trace viewer (Step 5.6 GUI), and signing-key rotation beyond the `key_id` label stay deferred. See row 5.1 below + [ADR-0026](docs/adr/ADR-0026-per-query-tracing-provenance.md), [reference/provenance.md](docs/reference/provenance.md), [architecture/per-query-tracing.md](docs/architecture/per-query-tracing.md). Earlier, **Phase 4 — Reliability completed (6/6).** Step 4.6 ✅ delivered **latency tuning & load test** — the final Phase-4 step, turning the gateway's latency into a measured, CI-enforced contract. The headline is a **gateway-overhead p99 ≤ 30 ms gate**: with the default noop backends a request exercises only the gateway's own work (FastAPI routing, request-context + metrics middleware, query understanding, routing + RRF fusion, retrieval-cache lookup, Pydantic (de)serialisation), so per-request wall time *is* the overhead — deliberately distinct from the backend-bound **end-to-end** target (≤ 250 ms SaaS / ≤ 500 ms dev, kept as a documented operational number in [performance.md](docs/architecture/performance.md), not a unit gate). The gate signal is the **server-side `gateway.request_duration_ms`** histogram the Step 3.11 metrics middleware already records (the same number `GET /v1/status/metrics` exposes), read back from the app's `MetricsCollector` after driving load — so a green gate and a healthy dashboard agree. Load is driven **in-process** via httpx `ASGITransport` (no socket/uvicorn/port — deterministic + infra-free), **sequentially** (concurrency 1 measures per-request overhead; one event loop only *queues* under concurrency, which is latency-under-load, a different question), and the assertion is **retry-tolerant** (accept the first clean run, fail only if every attempt exceeds — a real regression spikes them all, a lone GC/scheduling blip doesn't). Because a latency budget needs stable timing, the `perf`-marked gate is **excluded from the cross-OS `lint-test` sweep** (`-m "not perf"`) and enforced by a dedicated single-runner **`perf-gate`** CI job on ubuntu (added to `ci-pass`); observed overhead p99 is ~5–10 ms, comfortable under 30 ms. One reusable engine, **`rag_gateway.perf`** (`measure_gateway_overhead` + `profile_gateway`, with `LatencyReport` / `RouteLatency` / `ProfileEntry` / `Scenario`), backs three surfaces: the pytest gate (`tests/perf/`), the `eval/gateway_load_v0` harness, and the `ragctl perf` smoke command. The long-promised **`tests/contract/budgets.py`** registry is now real — the per-SPI p99 table from performance.md as structured data plus the gateway overhead + end-to-end budgets; the overhead number's single runtime source is `rag_gateway.perf.DEFAULT_GATEWAY_OVERHEAD_P99_MS`, mirrored in the registry and kept equal by a drift test. The **load test** is two-pronged: an in-process concurrent harness (`python -m eval.gateway_load_v0.harness`, default concurrency 1 = the overhead snapshot, `--concurrency N` stresses one loop with a queueing caveat, writes `report.json`/`report.md`, `--check` exits non-zero over budget) and a **Locust** `locustfile.py` for real over-the-wire load against a running gateway (an ad-hoc `uv pip install locust`, deliberately *not* a locked dependency to keep `uv.lock` + the CI env lean). The **async pipeline profiling** deliverable is `profile_gateway`: a cProfile breakdown sorted by **own** CPU time (so the asyncio event loop doesn't dominate the top) with **warmup run outside the profiler** (so one-time import path-scanning doesn't mask steady-state frames) — surfacing Pydantic validation / routing / middleware as the real hot frames. An empirical finding: disabling GC during measurement made the tail *worse* (allocations pile up), so GC stays on and retry-tolerance absorbs the occasional pause. Adds no new domain types / schemas / structured-log events / routes, so it touches none of the schema/event/OpenAPI drift gates. ~14 new tests (4 gate + 6 budgets-registry + 4 `ragctl perf`); all gates green (ruff, mypy --strict, RAG001, the new perf gate). Full-pipeline profiling with a seeded corpus, the import-finder micro-overhead (~0.1 ms/req), and a real-backend end-to-end latency gate stay deferred (Phase 7 hardening). See row 4.6 below + [ADR-0025](docs/adr/ADR-0025-latency-load-testing.md), [reference/perf.md](docs/reference/perf.md), [architecture/latency.md](docs/architecture/latency.md). Earlier, Step 4.5 ✅ delivered **per-tenant quotas & rate limiting** — the Phase-4 noisy-neighbour defence, enforced *through* the PolicyEngine PDP ([ADR-0005](docs/adr/ADR-0005-policy-engine.md)) rather than beside it. A new **`rag-quota`** package (deps: `rag-core` + `rag-observability` + `rag-policy`) splits the concern into a *mechanism* and a *policy*: the new **`QuotaStore` SPI** (in `rag-core`; sliding-window `check_and_consume` / `usage` + cumulative `adjust_gauge` / `gauge` + `reset`) with a behaviour-neutral `NoopQuotaStore`, a real per-process **`InMemoryQuotaStore`** (Cloudflare-style weighted two-slot sliding-window counter + storage gauge, injected clock), and a cross-worker **`RedisQuotaStore`** (in `rag-backends`, the *same* algorithm via one atomic Lua script per check, injected when the cache points at Redis); the **`QuotaEnforcer`** (dispatch over five dimensions — **QPS** rate limit + monthly **tokens** / **cost** / **queries** accruals + **storage** gauge; per-tenant limit resolution with per-field default-merge; cost in integer micro-dollars derived from a `dollars_per_1k_tokens` price; uncapped short-circuit; `quota.exceeded` event; snapshot/reset); and the **`QuotaPolicyEngine`** PDP adapter that decorates an inner engine, answering `rate_limit` / `quota_check` from the enforcer and delegating ACL / PII / egress / `filter_pushdown` to the inner engine. New frozen core types `QuotaVerdict` / `QuotaSnapshot` + `QuotaDimension` StrEnum (`dist/schemas` regenerated) + `QuotaExceededError(RateLimitError)` (so the existing 429 mapping covers it). Wired into every request-bearing surface via one injectable `quota_enforcer` on `build_app` (which wraps the PolicyEngine in `QuotaPolicyEngine` when present): REST `/v1/query` + `/v1/retrieve`, OpenAI `/v1/chat/completions` (sync + stream) + `/v1/embeddings`, and the `/v1/ingest/document` storage gauge — a shared `quota_guard` helper runs the QPS rate-limit + monthly-quota pre-check at entry (raising `RateLimitError` / `QuotaExceededError` with a `Retry-After` header) and records actual LLM tokens (+derived cost) post-response. **Disabled by default** (`quotas.enabled: false`) since an enabled quota can *reject* requests (unlike the behaviour-neutral breaker); **fail-open by default** (`fail_open: true`) — a metering-store outage degrades to admit + log `quota.store_degraded`, never raises (the Phase-4 reliability invariant). **Hot-path discipline**: a disabled quota early-returns before any store call, and no per-call OTel span — observability is the pre-registered PII-free **`quota.exceeded`** event (typed `QuotaEvent`, once per denial, carrying only dimension / scope / counters) + the pull-able `QuotaSnapshot`. Operator surface: `GET /v1/status/quotas` (per-tenant per-dimension usage), `POST /v1/status/quotas/{tenant}/reset` (one-click headroom; 400 unknown dimension, 404 disabled), and a presence-based `quotas` component in `/v1/status/health` (quotas are per-tenant, so usage does *not* fold into the infra roll-up the way breakers do); the admin console's **Live Status** page gains a per-tenant **Quotas** card with usage bars (amber ≥ 75% / red ≥ 90%) + a one-click *Reset usage* (live-vs-seed via `useLive`). `cfg.quotas` (`enabled` / `fail_open` / `qps_window_seconds` / `monthly_window_seconds` / `dollars_per_1k_tokens` / `default`) + an extended `TenantQuota` (`tokens_per_month` / `dollars_per_month` / `storage_bytes` alongside `qps` / `queries_per_month`) drive it from `rag.yaml` (`dist/rag.schema.{json,yaml}` regenerated); `ragctl quota` smoke command drives the full QPS-throttle → window-roll → token / storage-cap flow against a fake clock. The PolicyEngine coverage linter passes with **no new allowlist entries** (the `quota_guard` helper *is* a `policy_engine.evaluate` call site; the store methods aren't governed reads). ~82 new tests (store sliding-window / gauge / isolation + enforcer dimension / fail-open / event / cost-derivation + QuotaPolicyEngine delegate / deny + limit-merge + 13 gateway 429 / Retry-After / status + 5 cross-tenant red-team + 9 config + 4 ragctl + 6 Redis integration + `QuotaEvent` schema) + 2 admin-ui vitest; all gates green (mypy --strict, ruff, RAG001, log-schema / event-registry + schema-drift gates; admin-ui tsc / lint / build + vitest), browser-verified. Delete-path storage decrement, exact token reservation, per-API-key / per-corpus limits, and cross-region aggregation stay deferred to Phase 6. See row 4.5 below + [ADR-0024](docs/adr/ADR-0024-quotas-rate-limiting.md), [reference/quota.md](docs/reference/quota.md), [architecture/quotas.md](docs/architecture/quotas.md). Earlier, Step 4.4 ✅ delivered **per-backend circuit breakers** — the backend-isolation half of Phase-4 reliability, formalising the Step 2.10 `BackendHealthTracker`'s deferred half-open state. A new core-only **`rag-breaker`** package (deps: `rag-core` + `rag-observability`) holds `CircuitBreaker`: a three-state machine (**closed** → count consecutive failures → **open** at `failure_threshold` → after `reset_timeout_s` admit a single trial → **half-open** → `success_threshold` successes close, any failure re-opens) with an injected monotonic clock for deterministic timeout tests + one `asyncio.Lock`. Integration is **by wrapping the backend SPI, not editing the fan-out**: `Breaker{Vector,Keyword,Graph}RetrievalBackend` *are* the retrieval SPIs and route the retrieval call through a breaker; an **open** breaker raises the new `CircuitOpenError`, which `HybridRetriever`'s existing `asyncio.gather(return_exceptions=True)` already drops — so one open backend silently leaves the fusion (and a total outage degrades through the Step 4.2 fallback chain) with **zero changes** to `HybridRetriever` or the router. A `BreakerRegistry` mints one breaker per backend label (shared config + clock, infra-scoped not tenant-scoped), held on `app.state`; `build_default_retrieval_router(breaker_registry=…)` wraps the (noop) backends and `build_app_from_config` threads the *same* registry through both places when `cfg.breakers.enabled`. New frozen core type `BreakerSnapshot` + `CircuitState` StrEnum (`dist/schemas` regenerated) + top-level `CircuitOpenError`; rag-core 0.25 → 0.26. **On by default** (`breakers.enabled: true`) — a closed breaker is pure pass-through and failing fast on a down backend beats hanging (like the behaviour-neutral fallback chain, unlike the content-affecting guard). **Hot-path discipline**: the breaker sits inside the retrieval fan-out so it opens **no OTel span per call** — observability is the pre-registered **`breaker.opened`** event (typed `BreakerEvent`, once per trip into open, carrying only counts / threshold / from-state / tenant — never query/answer text) + the pull-able `BreakerSnapshot`. Operator surface: `GET /v1/status/breakers` lists state, `POST /v1/status/breakers/{name}/force-close` is the **one-click force-close** (404 unknown), and breaker state folds into `/v1/status/health` (`open → down`, `half_open → degraded`); the admin console's **Live Status** page gains a "Circuit breakers" card with a per-breaker Force-close button (live-vs-seed via `useLive`). `cfg.breakers` (`enabled` / `failure_threshold` / `reset_timeout_s` / `half_open_max_calls` / `success_threshold`) drives it from `rag.yaml` (`dist/rag.schema.{json,yaml}` regenerated); `ragctl breaker [-f N] [--no-recover]` smoke command drives the full closed→open→half-open→closed flow against a fake clock. The PolicyEngine boundary is unchanged — the wrappers forward the caller's already-policy-merged call (canonical `read_chunk` consumer stays `HybridRetriever`), allowlisted in `tests/policy/coverage.py` with justification like GraphRAG/agent. ~74 new tests (28 breaker state-machine + 6 registry + 4 SPI-wrapper incl. the HybridRetriever-isolation integration + 6 gateway + 3 ragctl + 7 config + `BreakerEvent` schema) + 2 admin-ui vitest (render + one-click force-close); all gates green (mypy --strict 260 files, ruff, RAG001, log-schema/event-registry + schema-drift gates, **2145 passed / 61 skipped** workspace-wide; admin-ui tsc/lint/build + 23 vitest). Timeout-driven tripping, per-`PlanNode` scoping, and cross-worker shared state stay deferred (Steps 4.6 / 6.x). See row 4.4 below + [ADR-0023](docs/adr/ADR-0023-circuit-breakers.md), [reference/breaker.md](docs/reference/breaker.md), [architecture/circuit-breaker.md](docs/architecture/circuit-breaker.md). Earlier, Step 4.3 ✅ delivered the **hallucination guard** — a Phase-4 answer-side reliability gate verifying a generated answer's faithfulness to the retrieved context, per claim. A new core-only **`rag-guard`** package (deps: `rag-core` + `rag-observability`) holds `HallucinationGuard`: it splits the answer into claims (sentence segmentation behind a `ClaimExtractor` Protocol), scores each claim against the evidence via the new **`NLIScorer` SPI** (in `rag-core.spi`; shape mirrors `Reranker` — atomic `score` + a default-loops `score_batch`, so the *guard* owns the best-entailment-across-evidence aggregation and a backend answers the narrow NLI question), thresholds each claim against a **per-tenant** floor (`per_tenant_thresholds[tenant] → threshold`), and applies a `GuardAction` (**`annotate`** observe-only default / **`redact`** drop unsupported claims / **`block`** withhold the answer). The shipped default scorer `LexicalNLIScorer` (token-set overlap, dependency-free, deterministic — same spirit as the Step 4.1 cache Jaccard / Step 2.9 set-overlap) flags ungrounded claims out of the box; a real cross-encoder NLI model is a plugin implementing the same SPI (`NoopNLIScorer` — always-entails — is the behaviour-neutral test default). New frozen core types `NLILabel` / `NLIScore` / `GuardVerdict` / `GuardAction` / `ClaimVerdict` / `GuardResult` (`dist/schemas` regenerated) + a top-level `GuardError`. The guard runs **after** `egress_text` (so it checks the policy-approved evidence) and is wired into every chunk-bearing answer surface via one injectable `hallucination_guard` + `guard_enabled` on `build_app`: REST `/v1/query` (`answer.guard`; `answer.text` reflects the action), OpenAI `/v1/chat/completions` (sync — `rag.guard` + a blocked verdict ⇒ `finish_reason=content_filter`; stream — **advisory-only**, verdict on the terminal chunk since deltas are already sent), and the MCP `query` tool. The agent runtime is deferred (its event stream surfaces the answer but not chunk provenance). **Disabled by default** (`guard.enabled: false`) since it can flag/withhold content — unlike the always-on fallback chain; `annotate`-first lets operators measure faithfulness in production before enforcing. **Reliability invariant** (this is Phase 4): a scorer failure degrades to a pass-through (answer returned unmodified, logged `guard.scorer_degraded`), never raises. Canonical `guard.check` OTel span (`rag.guard.{threshold,action,scorer,claims_n,blocked_n,verdict,min_entailment,degraded,elapsed_ms}`) + the pre-registered **`guard.claim_blocked`** structured event (typed `GuardEvent`, emitted once per check when `blocked_n > 0`) carrying only counts / threshold / verdict / action / scorer — **never** the answer or claim text (log PII gate). `cfg.guard` (`enabled` / `threshold` / `action` / `per_tenant_thresholds` / `min_claim_chars` / `max_claims`) drives it from `rag.yaml` (`dist/rag.schema.{json,yaml}` regenerated); `ragctl guard -c --threshold --action` smoke command runs the full flow against `LexicalNLIScorer`. Proto `Answer` is allow-listed Pydantic-only (the gRPC surface predates the guard). ~45 new tests (claims + lexical scorer + orchestrator pass/flag/redact/block + per-tenant threshold + scorer-degradation + event + span + GuardConfig validation + NoopNLIScorer conformance + 6 gateway + 7 ragctl + 8 config + GuardEvent); all gates green (mypy --strict 256 files, ruff, RAG001, log-schema/event-registry + schema-drift gates, **2097 passed / 61 skipped** workspace-wide). See row 4.3 below + [ADR-0022](docs/adr/ADR-0022-hallucination-guard.md), [reference/guard.md](docs/reference/guard.md), [architecture/hallucination-guard.md](docs/architecture/hallucination-guard.md). Earlier, Step 4.2 ✅ delivered the **fallback chain** — a reliability degradation ladder (`hybrid → BM25-only → keyword → "no answer"`) wrapping the Step 2.10 `RetrievalRouter` as a new `rag_retrieval.fallback.FallbackChain`. Per [ADR-0008](docs/adr/ADR-0008-cost-aware-planner.md) it has two triggers: **proactive budget** (an injected `CostEstimator` picks the richest tier that fits `RequestContext.budget` *before* dispatch — an uncapped budget never downgrades, so the wrap is behaviour-neutral by default) and **reactive** advance on `RetrievalError` (backend down/partition) or an empty / below-`min_results` result ("retrieval returned nothing useful"). The terminal `no_answer` returns a **graceful empty result (HTTP 200 + `decision.reason="fallback:no_answer"`)** instead of a 502 — toggle via `no_answer_on_exhaustion` (error-exhaustion only; an all-empty exhaustion is always a graceful 200). The `keyword` rung relaxes the caller's *metadata* filters to widen recall; the tenant/ACL pushdown applied inside `HybridRetriever` is **never** relaxed (recall can't widen the security boundary). New frozen core types `FallbackTier` / `FallbackTrigger` / `FallbackResult` (rag-core 0.24 → 0.25; `dist/schemas` regenerated); a `SupportsRoute` Protocol lets the chain drop in anywhere a router is expected, so `CorpusRouter` (relaxed to accept the Protocol) + the gateway's direct / OpenAI / agent paths all gain fallback by wrapping one injected `retrieval_router`. Gateway `build_app` wraps real routers by default — guarded by `isinstance(RetrievalRouter)` so the route-only test stubs (and the raw-502 path) stay untouched, and `fallback_enabled=False` / `cfg.retrieval.fallback.enabled: false` opts out; `cfg.retrieval.fallback` (`enabled` / `min_results` / `advance_on_empty` / `relax_filters_on_keyword` / `no_answer_on_exhaustion`) drives it from `rag.yaml` (`dist/rag.schema.{json,yaml}` regenerated). Canonical `retrieve.fallback` OTel span (per-tier `retrieve.route` spans nested) + the pre-registered `fallback.engaged` structured event (no raw query text). `ragctl fallback ` smoke command (`--simulate-failure` / `--simulate-empty` / `--budget-tokens` / `--min-results` / `--no-keyword-relax` / `--strict`). 39 new tests (24 chain + 6 ragctl + 4 gateway incl. the graceful-200 path + 5 config); ruff + mypy --strict + RAG001 + log-schema/event-registry + schema-drift gates green. Full per-`PlanNode` plan mutation + timeout-driven triggering stay deferred (ADR-0008 §Step 4.2 backlink; Steps 2.6 / 4.4 / 4.6). See row 4.2 below + [architecture/fallback-chain.md](docs/architecture/fallback-chain.md), [reference/fallback.md](docs/reference/fallback.md). Earlier, Step 4.1 ✅ delivered the **tiered semantic cache** — the production implementation behind the `RetrievalCache` + `AnswerCache` SPIs (split out in 1.1e; this is what [caching.md](docs/architecture/caching.md) called "4.1 production implementation"). A new core-only **`rag-cache`** package (deps: `rag-core` + `rag-observability`) holds the **L0** tier (`InMemory{Retrieval,Answer,Embedding}Cache` over a generic `TtlLruCache` — LRU + per-entry TTL), the **L2** tier (`Semantic{Retrieval,Answer}Cache` + `QuerySimilarityIndex` — token-set Jaccard over recent queries, generalising the Step 2.11 [ADR-0010](docs/adr/ADR-0010-semantic-embedding-cache.md) embedding cache), and the **`Tiered{Retrieval,Answer}Cache`** composition: read-through with promotion (an L1/L2 hit warms L0), write-through to every tier, `cache.hit`/`cache.miss`/`cache.invalidated` events + a `cache.lookup` OTel span + a pull-able `TierStats` snapshot. The **L1** tier (`Redis{Retrieval,Answer}Cache`, exact + tag-set invalidation) lives in `rag-backends` (where the redis client already is) and is **injected** as `l1=`, so the default gateway pulls no infrastructure. **Reliability invariant** (this is Phase 4): every L1 (network) call is wrapped — a Redis failure degrades to a miss/skip, logged `cache.l1_degraded`, never raised; `health()` is the L0 floor. The L2 match holds a **`scope`** token (a hash of the non-text params — top_k/filters/corpus/ACL) constant so a paraphrase hit can never serve a result built with a different top_k. Cross-tenant isolation is enforced in every impl + asserted by a new red-team suite (`tests/redteam/test_cross_tenant_cache.py`, fulfilling the caching.md promise). Gateway: `/v1/retrieve` + `/v1/query` consult the tiered `RetrievalCache` before understanding+routing (a hit returns cached refs with a truthful `reason="served_from_cache"` decision + `rag.gateway.cache_hit` span attr); the ingest path calls `invalidate_corpus`; `corpus_version` is derived best-effort from `Corpus.updated_at` (the answer-cache consultation + corpus-version-bump-on-ingest are Phase 6 governance work — the answer-cache impl is built + injected + tested). `CacheConfig` gains `tiered`/`l0_max_entries`/`{retrieval,answer}_ttl_seconds`/`semantic.{enabled,threshold,max_entries_per_tenant}` (schemas regenerated). New `rag-observability` events (`CacheEvent` + 3 constants). All gates green (mypy --strict 249 files, ruff, RAG001, log-schema/event-registry gates, 100+ new tests incl. 11 Redis integration tests verified against a live server). See row 4.1 below + [ADR-0021](docs/adr/ADR-0021-tiered-cache.md), [reference/cache.md](docs/reference/cache.md), [architecture/tiered-cache.md](docs/architecture/tiered-cache.md). - -Earlier, Step 3.11 ✅ delivered the **Status & Metrics GUI (MVP)** — the gateway's live-status surface plus the operator console's Observability pages. A new `rag-observability` read-side (`MetricsCollector`, a pull-able counter/gauge/histogram snapshot that *complements* OTel's push-only instruments, + `LogTail`, a bounded log ring buffer polled by `seq` cursor) backs a new `rag_gateway.status` surface: REST `/v1/status/health` (overall + per-component roll-up, where `surface` health is derived from real per-route error rates, not synthetic probes), `/v1/status/metrics`, `/v1/status/logs`, `/v1/connectors/status`, an **SSE** `/v1/status/logs/stream` (reusing the chat/agent `data:` framing), and a **WebSocket** `/v1/status/ws` (health+metrics push) — the two transports matched to data shape per [ADR-0020](docs/adr/ADR-0020-status-transports.md). A request-timing middleware feeds the metrics (status surface excluded so the SSE stream can't skew it); CORS is installed for the console origin. The console (`apps/admin-ui`) gains an **Observability** nav group with **Live Status** / **Metrics** / **Logs** pages (WS + SSE, seed-fallback hybrid) plus a Dashboard system-status banner for ≤3-click drilldown. `rag-gateway` 0.7.0 → 0.8.0. All gates green (mypy --strict 242 files, ruff, RAG001, 45 new tests; `tsc`/`next lint`/`next build`/`vitest`), browser-verified. See row 3.11 below + [reference/status-api.md](docs/reference/status-api.md), [architecture/status-metrics-gui.md](docs/architecture/status-metrics-gui.md), [guides/operator-console-live.md](docs/guides/operator-console-live.md). The *full* 12-page GUI build (Grafana, drift alerts, regression bisector) is Step 5.6. - -Earlier in Phase 3, Step 3.10 ✅ delivered the **Admin UI v0**: a Next.js 14 (App Router) + TypeScript + Tailwind operator console at `apps/admin-ui`, built to a high-fidelity design handoff (`apps/admin-ui/design/`) targeting shadcn/ui. Primitives (`Button`/`Card`/`Badge`/`Dialog`/`Sheet`/`Dropdown`/`Switch`/`Toast`/`Table`/`ChipInput`/…) are **hand-rolled in TS** (Radix-free, no shadcn CLI) against the handoff's shadcn CSS-variable tokens (light+dark) + Geist fonts + lucide-react, for deterministic offline builds. Nine pages behind one app shell (collapsible sidebar + top bar with tenant switcher / theme toggle / review-state control): **Dashboard**, **Corpora**, **Connectors**, **Glossary**, **Webhooks**, **Audit Log**, **API Keys** (Preview), **Tenants** (Preview), **Config Viewer**. **Live-vs-seed hybrid** (`lib/use-live.ts`): Corpora (`GET /v1/corpora`) + Webhooks (`/v1/webhooks/subscriptions` CRUD + `/test`) read/write the real gateway when `NEXT_PUBLIC_GATEWAY_URL` is set, falling back to typed seed data with a "Demo data" badge so the console always renders; the other pages are seed-only (API Keys/Tenants marked Preview — backends land in Phase 6). Header-driven identity (active tenant → `X-Tenant-Id`). Webhooks page has the one-time signing-secret reveal + send-test (signed headers + attempts); Audit has filters + pagination + metadata drawer; Config is a read-only `rag.yaml` viewer with section nav + YAML highlight. 14 vitest+RTL tests (shell nav, Corpora seed path + new-corpus dialog, webhook secret-reveal flow, audit filtering); `tsc`/`next lint`/`next build` all clean; dedicated CI `admin-ui` job; visual fidelity confirmed against the handoff screenshots. See row 3.10 below + [ADR-0019](docs/adr/ADR-0019-admin-ui.md), [reference/admin-ui.md](docs/reference/admin-ui.md), [architecture/admin-ui.md](docs/architecture/admin-ui.md), [guides/admin-ui-quickstart.md](docs/guides/admin-ui-quickstart.md). Phase 3 status: 10/11. - -Earlier in Phase 3, Step 3.4 ✅ delivered the OpenAI-compatible surface: `rag_gateway.openai_compat` exposes `POST /v1/embeddings`, `POST /v1/chat/completions` (with **retrieval pre-fetch**), and `GET /v1/models` so any OpenAI client SDK can use the gateway as a drop-in `base_url`. Chat runs the Phase 2 pipeline (understand → route → optional rerank/pack) over the conversation's last user turn (or `rag.query` override) and injects the retrieved context as a system message immediately before that turn; retrieval is on by default (`rag.enabled`), so a vanilla client gets RAG for free, while a RAG-aware client tunes it via the non-standard `rag` request field (`RagOptions`, mirroring `QueryRequest`'s retrieval knobs + defaults, pinned by a conformance gate) and reads citations + routing decision back under the `rag` response field (`RagAnnotations`). `stream=true` emits OpenAI-faithful SSE `chat.completion.chunk` events (role-only open → content deltas → terminal chunk with `finish_reason`/`usage`/`rag` → `data: [DONE]`). Embeddings delegate to a new top-level `Embedder` injectable (`build_default_embedder`), supporting `encoding_format: base64` (little-endian float32). Identity is header-driven (`Authorization: Bearer` / `X-Tenant-Id`) with a dev-only anon fallback gated on the wired `Auth` being `NoopAuth` (creds-free demo; real `Auth` → `401`). Errors use the OpenAI `{"error": {message,type,param,code}}` envelope. Governance: chat consults `PolicyEngine.egress_text` before every LLM call, embeddings consults `quota_check` before embed — both inside the coverage linter. New `rag-core` 0.20 → 0.21 wire types in `rag_core.openai_types` (`EmbeddingsRequest/Response`, `ChatCompletionRequest/Response/Chunk`, `RagOptions`, `RagAnnotations`, `ModelList`, `OpenAIError`). `ragctl chat` (+ `--stream`/`--no-rag`) + `ragctl embeddings` smoke commands. 36 new tests (18 gateway + 6 conformance + 6 ragctl + ...); schemas regenerated. See row 3.4 below + [ADR-0013](docs/adr/ADR-0013-openai-compatibility.md), [reference/openai-compat.md](docs/reference/openai-compat.md), [architecture/openai-compat.md](docs/architecture/openai-compat.md), [guides/openai-quickstart.md](docs/guides/openai-quickstart.md). Phase 3 status: 4/11. +--- -Step 2.11 + all five pre-3.1 must-fix items are landed: +## Status -- **G-02** ([PR #101](https://github.com/officialCodeWork/AgentContextOS/pull/101)) — `RetrievalRouter.route` split into public `decide` + `execute`; sticky-mode agent loops stop paying the decide cost per iteration. -- **G-03** ([PR #102](https://github.com/officialCodeWork/AgentContextOS/pull/102)) — `Budget` rejects `0` / negative caps at construction (`mode="before"` validator); `None` = uncapped, positive = capped; `spend()` uses `model_construct` so the post-spend exhaustion floor stays valid. -- **G-06** ([PR #103](https://github.com/officialCodeWork/AgentContextOS/pull/103)) — `HyDEGenerator` takes optional `embedding_cache`; `LLMQueryRewriter` takes optional generic `Cache` with `prompt_version`-keyed entries. -- **G-01** ([PR #104](https://github.com/officialCodeWork/AgentContextOS/pull/104), [ADR-0010](docs/adr/ADR-0010-semantic-embedding-cache.md)) — `SemanticEmbeddingCache` wrapper above any `EmbeddingCache` with token-set-Jaccard similarity lookup. Harness measured **overall hit rate 28.2% → 53.7%**, **decomposable bucket 0% → 43.3%**. -- **G-04** ([PR #105](https://github.com/officialCodeWork/AgentContextOS/pull/105), [ADR-0008 backlink](docs/adr/ADR-0008-cost-aware-planner.md#step-211-g-04-backlink--agent-loop-interim-cost-attribution)) — `CostEstimator` Protocol + `DefaultCostEstimator` for per-stage budget attribution (`understand` / `embed` / `retrieve` / `rerank`). Full per-SPI `Cost`-return shape is deferred to Phase 3.x. +| | | +|---|---| +| **Last updated** | 2026-06-07 | +| **Current phase** | Phase 5 — Eval & Observability (**6 / 7 steps**) | +| **Overall** | **63 / 84 steps** — Phases 0–4 complete | +| **Next action** | **Step 5.7b — Shadow mode** — fan out N% of live queries to a candidate retriever (observe-only) and feed the A/B tracker. Final Phase-5 step, delivered in slices 5.7a–d. | -1542 tests pass workspace-wide. Step 2.11 shipped `AgentLoopV0` (in `rag-retrieval` 0.2 → 0.3 as `agent_loop.py`) — thin iterative-retrieve orchestrator with budget enforcement (`Budget.spend` on every iter via `ctx.model_copy`), sticky corpus routing (`AgentLoopConfig.allow_route_switch=False` pins iter-0 decision; `True` re-decides every iter), plan-reuse measurement (`_decisions_match` ignores `reason` + `degraded_backends`), shared `EmbeddingCache` integration (HyDE-vector preferred → text-hash cache hit → `Embedder.embed` + write-back → `None`). Cross-package isolation preserved via structural `QueryUnderstander` + `UnderstoodQueryLike` Protocols so `rag-retrieval` still depends only on `rag-core` (mirrors `GraphAdapter` Protocol pattern). Stop conditions (in evaluation order): budget exhaustion on `tokens`/`dollars`/`wall_ms`/`iter` (checked top-of-iter); `should_continue=False` (end-of-iter); diminishing returns when `new_chunks_n < min_new_chunks_per_iter` and `iter_idx > 0`; `refine_query` returning `None`; `max_iterations` (loop completion). `Budget.spend` rounds wall_ms up via `math.ceil` so sub-ms iters can't dodge a tight wall budget. Sub-query fan-out (deferred from 2.10) wired by consuming `UnderstoodQueryLike.sub_queries` in order after iter 0 unless `refine_query` callback overrides. New `rag-core` 0.18 → 0.19 types: `StopReason` StrEnum (7 values), `BudgetSpend` (frozen Pydantic, 4 dimensions), `CacheStats` (frozen Pydantic with `embedding_hit_rate` property), `AgentLoopError(RetrievalError)`. Canonical `agent_loop.run` OTel span with `rag.agent_loop.{max_iterations,allow_route_switch,iterations,stop_reason,budget_tokens_used,budget_wall_ms_used,cache_hits_embedding,cache_misses_embedding,plan_reuse_rate,sticky_route_rate,final_chunks_n,elapsed_ms}` mirroring Step 2.10's `retrieve.route` schema; structured-log event `agent_loop.iteration_complete` per iter. `ragctl agent-loop ` smoke command drives the full flow against noop SPIs with `--max-iter`/`--budget-tokens`/`--budget-wall-ms`/`--budget-iter`/`--min-new-chunks`/`--allow-route-switch`/`--sub-query` (repeatable) flags — no LLM credentials. 50-query harness in `eval/agent_loop_v0/` (queries.jsonl + harness.py + toy 50-doc corpus across 5 topic clusters) runs all 50 queries deterministically + writes `report.json` + `report.md`. Buckets: follow_up (10) / refinement (10) / decomposable (10) / shape_shifter (10) / budget_stressor (10). Latest run (2026-05-27): plan-reuse 58.9%, sticky 58.9%, embedding-cache hit 28.2% overall (**0.0% on decomposable** — the headline finding driving G-01). 66 new tests: pure-helper units (stable-hash 5 / decisions-match 6 / budget-stop 7 / diff-budget 3 / dedupe 1 / rate 4 / next-query 5) + orchestrator integration (happy path 2 / budget-exhaustion 4 / diminishing-returns 3 / caller-stop 3 / sticky-route 2 / cache 4 / sub-query consumption 1 / error propagation 3 / input validation 4 / structured output 2 / telemetry 1) + 5 `ragctl agent-loop` CLI; 1468 tests pass workspace-wide. PolicyEngine boundary: `agent_loop.py` makes no governed SPI calls itself — coverage linter passes naturally (`router.route` is the entry point; `HybridRetriever.retrieve` is the canonical `read_chunk` PDP consumer). Cross-platform: pure-Python throughout; deps only rag-core + pydantic + opentelemetry-api (already in graph). Schemas regenerated (`BudgetSpend.json`, `CacheStats.json`). New docs: [reference/agent-loop.md](docs/reference/agent-loop.md), [architecture/agent-loop-v0.md](docs/architecture/agent-loop-v0.md), [spikes/agent-loop-v0-gaps.md](docs/spikes/agent-loop-v0-gaps.md); updated [retrieval-router.md](docs/architecture/retrieval-router.md) (resolves sub-query fan-out + sticky-routing deferrals), [request-context.md](docs/architecture/request-context.md) (adds agent-loop budget-derivation pattern + zero-vs-None gotcha), [caching.md](docs/architecture/caching.md) (adds agent-loop reuse semantics + 0%-hit-rate decomposable finding); new `docs/spikes/` section added to [docs/README.md](docs/README.md). Deliberately deferred (matches spike contract — Phase 3.x territory): LLM-as-controller (`rag-agent` package, Phase 3.x), real per-SPI cost reporting (ADR-0008 + Phase 3.1 fix list), semantic embedding cache (G-01 fix), 500+-query harness expansion (Phase 5 eval), `probe_health_on_decide=True` harness mode (Phase 3.x), cross-tenant cache analysis (Phase 4.x circuit breaker). Step 2.10 shipped a new retrieval router layered on the Step 2.5 `HybridRetriever` (no new package — `rag-retrieval` v0.1 → v0.2). `RetrievalRouter` async orchestrator + pure `classify_shape` query-shape classifier (`KEYWORD_HEAVY` / `SEMANTIC` / `ENTITY_RICH` / `MIXED`; deterministic ≤3-token + question-word + glossary-expansion + capitalised-multi-word heuristics with HyDE-vector promotion to `SEMANTIC`) + in-memory `BackendHealthTracker` with rolling-window failure counter (default `failure_threshold=3` / `window_seconds=60` / `degrade_seconds=30`; explicit `mark_degraded()` for synchronous `health()` probe failures; injected clock for deterministic tests). `RoutingDecision` Pydantic frozen output + `QueryShape` StrEnum live in `rag-core` (0.17 → 0.18) so downstream packages can reference them without importing `rag-retrieval`; new `RouterError(RetrievalError)` rounds out the error hierarchy. Shape-bias weight table (multiplicative on `RouterConfig.base_weights`): `KEYWORD_HEAVY` → (0.3,1.5,0.5), `SEMANTIC` → (1.5,1.0,0.8), `ENTITY_RICH` → (1.0,1.2,1.8), `MIXED` → (1.0,1.0,1.0). Per-call vector input resolution priority: caller-supplied vector > `UnderstoodQuery.hyde_vector` > `Embedder.embed(text)` > skip. BM25-only fallback flag fires (and is surfaced both on the `RoutingDecision` and as an OTel attribute) when both vector + graph paths are unavailable; the router records `health.record_failure` per selected label on every `RetrievalError` from `HybridRetriever.retrieve` and `health.record_success` on the happy path so degradation state converges within a window without any external feedback loop. Canonical `retrieve.route` OTel span with `rag.tenant_id` + `rag.route.{shape,use_vector,use_keyword,use_graph,bm25_fallback,degraded_n,degraded,weight_vector,weight_keyword,weight_graph,results_n,elapsed_ms,error}` mirroring Step 2.7's `rerank` + Step 2.8's `pack` + Step 2.9's `graphrag.*` schema; structured-log event `retrieval.route_decision` mirrors the same data for log-only deployments. `ragctl route ` smoke command runs the full decide → execute flow against the noop SPIs with `--simulate-failure {vector,keyword,graph}` (repeatable) + `--no-graph` flags so BM25-only fallback is visible without external services. 42 new tests: 21 shape classifier (every shape × every trigger + determinism + empty-input + glossary-vs-capitalisation priority) + 13 health tracker (config validation + failure-window pruning + record_success clears + explicit mark_degraded expiry + per-label isolation + snapshot) + 23 router integration (per-shape routing + skip-unwired + skip-no-input + HyDE-vector preference + caller-vector-wins-over-embedder + degradation gating + BM25-only fallback + retrieval-error failure-feedback + happy-path success-clears-failures + corpus_ids passthrough + `probe_health_on_decide=True` flow + tenant attribute + real `NoopVectorStore` round-trip + failure_threshold=1 immediate degrade) + 5 `ragctl route` CLI (default run + keyword-heavy detection + degradation simulation + BM25-fallback scenario + bad-label rejection); 1600 tests passing across the workspace. PolicyEngine boundary: router does NOT call PDP — every retrieval call goes through `HybridRetriever.retrieve` (the canonical `read_chunk` call site); coverage linter passes naturally because `router.py` makes no governed SPI calls itself. Cross-platform: pure-Python throughout; only deps are rag-core + opentelemetry-api (already in the dep graph). Schemas regenerated (`RoutingDecision.json`). New docs: [reference/router.md](docs/reference/router.md) (public API + ragctl examples + extension points + OTel schema), [architecture/retrieval-router.md](docs/architecture/retrieval-router.md) (design rationale + in-memory-only choice + BM25-fallback semantics + deferred concerns + reviewer checklist); [architecture/hybrid-fusion.md](docs/architecture/hybrid-fusion.md) updated to resolve the Step 2.10 deferral. Deliberately deferred (matches scope question answered before kickoff): sub-query fan-out via `UnderstoodQuery.sub_queries` (Step 2.11 agent loop), cross-worker degradation state (Step 4.x circuit breaker), per-corpus weight overrides (Step 3.5 corpus router), sticky multi-turn routing (Step 2.11 agent loop), cost-aware backend skip (ADR-0008 planner + Step 3.5), learned shape classifier (Step 5.7 A/B + Step 5.4 labels). Step 2.9 shipped a new `rag-graphrag` package (v0.1.0): `LouvainDetector` (default, pure-Python NetworkX) + `LeidenDetector` (optional `[leiden]` extra) for community detection over the existing `GraphRetrievalBackend` SPI; `CommunitySummarizer` (LLM-backed per-community natural-language summaries with strict JSON parsing + code-fence tolerance + NoopLLM fallback for smoke flows); `InMemoryCommunityStore` (tenant-scoped storage for `Community` + `CommunitySummary` with cross-tenant raise); `graphrag_adapter` (entity-node → `ChunkRef` via `node.properties["chunk_ids"]`); `GraphRAGRetriever` async orchestrator that scores query against community summaries (set-overlap with 2× weight on `key_entities` v1; embedder-cosine path deferred to Phase 3), collects key-entity seeds across top-K communities (deduped), fans out via `GraphRetrievalBackend.expand()`, runs results through the adapter, and returns `list[ChunkRef]` ready for RRF fusion via `HybridRetriever`. New `Community` + `CommunitySummary` Pydantic frozen models in `rag_core.types` + `GraphRAGError(RetrievalError)`; `rag-core` 0.16 → 0.17. Three canonical OTel spans (`graphrag.detect` / `summarize` / `retrieve`) with `rag.graphrag.*` attributes mirroring Step 2.7/2.8 schema. `ragctl graphrag` smoke command runs the full detect → summarise → retrieve flow against a 10-node toy "people + companies" graph with `NoopLLM` + `NoopGraphStore` — no infra or LLM credentials required. PolicyEngine boundary: package itself does not call PDP (detection runs over already-policed sub-graph; summarisation is post-policy text transform — added to `tests/policy/coverage.py` allowlist with justifying comment; retriever returns `ChunkRef` to upstream `HybridRetriever` which IS the canonical `read_chunk` PDP consumer). 54 new tests: 13 detector + 10 store + 7 adapter + 12 summariser + 10 retriever + 5 `ragctl graphrag` CLI; all pure-Python + no external deps. New docs: [reference/graphrag.md](docs/reference/graphrag.md), [architecture/graphrag.md](docs/architecture/graphrag.md); [architecture/hybrid-fusion.md](docs/architecture/hybrid-fusion.md) updated to point at `rag-graphrag` for the "richer graph adapter" call-out. Step 2.8 shipped a new `rag-packer` package (v0.1.0): `ContextPacker` orchestrator runs tenant-safety → sort-by-score → hash dedup (always) → opt-in 5-shingle Jaccard dedup → greedy token-budget fit (uses Step 1.5's `TokenCounter` Protocol with `Chunk.token_count` cache fast-path) → lost-in-the-middle reorder (Liu et al. 2024 default; `descending`/`ascending` alternatives) → heuristic conflict scan (`trust_mismatch` via 3-shingle Jaccard ≥ 0.30 between different `TrustLevel`s + `numeric_disagreement` via regex extraction of `(context, number, unit)` triples). Output is `PackedContext` (added to `rag-core` 0.15 → 0.16 alongside `ConflictAnnotation` + `PackerError(RetrievalError)`). Strict failure mode (`PackerError` on budget≤0 / mixed-tenant / content_ref-only without cached token_count); conflict scan is the one degrade-and-log path (advisory metadata, never gating). Canonical `pack` OTel span with `rag.tenant_id`, `rag.pack.{chunks_in,chunks_out,tokens_used,tokens_budget,dropped_dedup,dropped_budget,conflicts_n,tokenizer,order,elapsed_ms}` mirroring Step 2.7's `rerank` schema. `ragctl pack` smoke command runs the noop-reranker → packer flow against a 5-doc toy corpus with mixed trust levels so conflict annotations are visible. +**Recently shipped** -**Phase 1 complete — CLI demo milestone delivered.** `ragctl ingest ` runs a folder of mixed-format documents through the full pipeline (connector → parse → chunk → enrich → PII → embed → store) and `POST /v1/ingest/document` exposes the same pipeline over HTTP. Next milestone on the demo ladder is **🥈 Curl-able RAG** at Step 3.1 (gateway query surface), which requires all of Phase 2 first. +- **5.7a** ✅ A/B analyzer + experiment tracker + `GET /v1/status/experiments` dashboard — [#143](https://github.com/officialCodeWork/AgentContextOS/pull/143) +- **5.6** ✅ Status & Metrics GUI (full build) — drift/feedback/cost cards, query-trace viewer, regression bisector, Grafana dashboards, cross-links — [#137–#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) +- **5.5** ✅ Drift monitors — `rag-drift`, five PSI / mean-drop monitors — [#136](https://github.com/officialCodeWork/AgentContextOS/pull/136) +- **5.4** ✅ Online metrics & feedback — `rag-feedback` — [#135](https://github.com/officialCodeWork/AgentContextOS/pull/135) +- **5.3** ✅ CI eval gate — [#134](https://github.com/officialCodeWork/AgentContextOS/pull/134) · **5.2** ✅ Offline golden-set eval harness — [#133](https://github.com/officialCodeWork/AgentContextOS/pull/133) · **5.1** ✅ Per-query tracing & provenance — [#132](https://github.com/officialCodeWork/AgentContextOS/pull/132) -> **Refactor window (Steps 1.1a–1.1f):** Before resuming the connectors framework (1.2), we insert a six-step refactor that locks in architecture + optimization decisions which are very expensive to retrofit later (PolicyEngine PDP, RequestContext-threaded SPIs, split Retrieval/Index backends, bulk + streaming + ID-only methods, Pipeline + Batcher primitives, three-way cache split, hot-path discipline). See [docs/adr/ADR-0005…0009] and [docs/architecture/policy-engine.md], [request-context.md], [caching.md], [performance.md]. +> Full per-step detail is in the phase sections below; each step links its ADR / reference / architecture docs. --- @@ -61,227 +57,729 @@ Step 2.11 + all five pre-3.1 must-fix items are landed: --- -## Phase 0 — Foundation (Weeks 0–4) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 0.1 | Monorepo & build system | ✅ | `build/phase-0/step-0.1-monorepo` | [#3](https://github.com/officialCodeWork/AgentContextOS/pull/3) | pyproject.toml, Makefile, .pre-commit-config, CI workflows, docker-compose, ADR-0001, lockfiles | -| 0.2 | Core domain types & errors | ✅ | `build/phase-0/step-0.2-core-types` | [#4](https://github.com/officialCodeWork/AgentContextOS/pull/4) | `types.py` (11 Pydantic v2 models), `errors.py` (14-type exception hierarchy), `gen_schemas.py` (JSON Schema export), `proto/core.proto` | -| 0.2c | Cross-platform tooling | ✅ | `build/phase-0/step-0.2c-cross-platform` | [#6](https://github.com/officialCodeWork/AgentContextOS/pull/6) | `Taskfile.yml` (Windows/Mac/Linux task runner), `scripts/bootstrap.ps1`, `scripts/bootstrap.sh` updated, CI matrix expanded to `windows-latest` | -| 0.3 | Plugin SPI | ✅ | `build/phase-0/step-0.3-plugin-spi` | [#7](https://github.com/officialCodeWork/AgentContextOS/pull/7) | SPI interfaces in `packages/core/src/rag_core/spi/`: VectorStore, KeywordStore, GraphStore, Embedder, Reranker, LLM, Cache, Queue, Storage, Secrets, Auth, Telemetry, Connector, Parser, OCR, PIIDetector; noop implementations in `spi/noop/`; 79 conformance tests in `tests/contract/` | -| 0.4 | `rag.yaml` schema & loader | ✅ | `build/phase-0/step-0.4-rag-yaml-schema-loader` | [#8](https://github.com/officialCodeWork/AgentContextOS/pull/8) | JSON Schema for rag.yaml (`dist/rag.schema.json`), Pydantic v2 schema (`packages/config/`), env-var interpolation, `ragctl config validate`, polling hot-reload, 91 fixture tests, 5 example configs | -| 0.5 | CI/CD pipeline hardening | ✅ | `build/phase-0/step-0.5-ci-cd-hardening` | [#9](https://github.com/officialCodeWork/AgentContextOS/pull/9) | Path-filtered jobs, dependency caching, Docker layer cache, SBOM generation, cosign image signing, Dependabot | -| 0.6 | Local dev stack (Docker Compose) | ✅ | `build/phase-0/step-0.6-local-dev-stack` | [#26](https://github.com/officialCodeWork/AgentContextOS/pull/26) | Health-check scripts, seed data loader, `make dev-reset`, pgvector schema migration, Qdrant collection init | -| 0.7 | Tracing & metrics foundation | ✅ | `build/phase-0/step-0.7-tracing-metrics` | [#27](https://github.com/officialCodeWork/AgentContextOS/pull/27) | OTel SDK wired, `TraceContext` propagation, `SpiMetrics` counter+histogram, Grafana provisioned dashboards, `ragctl traces` | -| 0.7b | Structured logging foundation | ✅ | `build/phase-0/step-0.7b-structured-logging` | [#29](https://github.com/officialCodeWork/AgentContextOS/pull/29), [#30](https://github.com/officialCodeWork/AgentContextOS/pull/30) | Shared logger package (`packages/core/src/rag_core/logging.py`), event registry (`events.py`), RAG001 pre-commit hook + ruff T201 rule, schema+PII CI gates in `tests/logs/` (50 tests; log-gates job now blocking). **Follow-up (0.7b-observability-package):** extracted `packages/observability/` (`rag-observability`) with full 7-field JSON schema (ts, level, service, module, msg, env, version), `set_log_context()` contextvar-based context manager, `_context.py`; `rag_core.logging` and `rag_core.events` reduced to backwards-compat shims (391 tests pass). | -| 0.7c | Audit log skeleton | ✅ | `build/phase-0/step-0.7c-audit-log-skeleton` | [#31](https://github.com/officialCodeWork/AgentContextOS/pull/31) | `AuditStore` SPI (append/events/verify_chain), `NoopAuditStore` (SHA-256 hash chain), `AuditWriter` facade (store + structured log), 14 conformance tests | -| 0.8 | Eval skeleton | ✅ | `build/phase-0/step-0.8-eval-skeleton` | [#33](https://github.com/officialCodeWork/AgentContextOS/pull/33) | `rag_core.eval` domain types (GoldenSample, EvalMetrics, EvalReport), `rag_config.eval` metric functions (recall@k, MRR, citation_precision), RagasAdapter spike, `ragctl eval run/show`, 5-sample golden JSONL fixture, `tests/eval/` harness (39 tests) | -| 0.9 | IaC foundation | ✅ | `build/phase-0/step-0.9-iac-foundation` | [#34](https://github.com/officialCodeWork/AgentContextOS/pull/34) | Terraform modules for Postgres/pgvector, Redis, Qdrant, Elasticsearch (Kubernetes-native, Helm provider); `rag-platform` Helm chart (Deployment, Service, ConfigMap, ServiceAccount, HPA, PDB, Ingress); dev + prod environments; `task infra:*` + `task helm:*` targets; ADR-0003 | -| 0.10 | `ragctl` CLI scaffold | ✅ | `build/phase-0/step-0.10-ragctl-cli-scaffold` | [#36](https://github.com/officialCodeWork/AgentContextOS/pull/36) | `packages/ragctl/` package (Typer 0.12+), root `ragctl` entry point, working `config`/`eval`/`traces`/`version` groups, scaffolded `ingest`/`query`/`logs`/`tenant`/`plugin`/`secret` groups (announce target step + exit 0), shell completion via `--install-completion`/`--show-completion`; 19 ragctl tests; docs/reference/ragctl.md + docs/guides/ragctl-quickstart.md; py.typed markers added to rag-config + rag-observability | +## Phase 0 — Foundation (Weeks 0–4) ✅ ---- +| Step | Title | Status | PR | +|------|-------|:------:|----| +| 0.1 | Monorepo & build system | ✅ | [#3](https://github.com/officialCodeWork/AgentContextOS/pull/3) | +| 0.2 | Core domain types & errors | ✅ | [#4](https://github.com/officialCodeWork/AgentContextOS/pull/4) | +| 0.2c | Cross-platform tooling | ✅ | [#6](https://github.com/officialCodeWork/AgentContextOS/pull/6) | +| 0.3 | Plugin SPI | ✅ | [#7](https://github.com/officialCodeWork/AgentContextOS/pull/7) | +| 0.4 | `rag.yaml` schema & loader | ✅ | [#8](https://github.com/officialCodeWork/AgentContextOS/pull/8) | +| 0.5 | CI/CD pipeline hardening | ✅ | [#9](https://github.com/officialCodeWork/AgentContextOS/pull/9) | +| 0.6 | Local dev stack (Docker Compose) | ✅ | [#26](https://github.com/officialCodeWork/AgentContextOS/pull/26) | +| 0.7 | Tracing & metrics foundation | ✅ | [#27](https://github.com/officialCodeWork/AgentContextOS/pull/27) | +| 0.7b | Structured logging foundation | ✅ | [#29](https://github.com/officialCodeWork/AgentContextOS/pull/29), [#30](https://github.com/officialCodeWork/AgentContextOS/pull/30) | +| 0.7c | Audit log skeleton | ✅ | [#31](https://github.com/officialCodeWork/AgentContextOS/pull/31) | +| 0.8 | Eval skeleton | ✅ | [#33](https://github.com/officialCodeWork/AgentContextOS/pull/33) | +| 0.9 | IaC foundation | ✅ | [#34](https://github.com/officialCodeWork/AgentContextOS/pull/34) | +| 0.10 | `ragctl` CLI scaffold | ✅ | [#36](https://github.com/officialCodeWork/AgentContextOS/pull/36) | -## Phase 1 — Ingestion + Knowledge Store (Weeks 4–8) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 1.1 | Storage backends | ✅ | `build/phase-1/step-1.1-storage-backends` | [#40](https://github.com/officialCodeWork/AgentContextOS/pull/40) | `rag-backends` package: `PgVectorStore` (asyncpg + pgvector, ivfflat), `QdrantVectorStore` (query_points API), `RedisCache`, `S3Storage` (aioboto3, MinIO-compatible), `LocalFileStorage`; integration tests (skip-if-no-service); MinIO added to dev stack; `task test-integration` + `task test-backends`; ADR-0004 | -| 1.1a | Core type & SPI refactor | ✅ | `build/phase-1/step-1.1a-core-type-spi-refactor` | [#44](https://github.com/officialCodeWork/AgentContextOS/pull/44) | `RequestContext` frozen model threaded through every SPI; `tenant_id` + `acl_labels` typed required on `Chunk`/`Embedding` (not metadata dict); `trust_level` on `Chunk` for prompt-injection defense; `dtype` on `Embedding` (float32/int8/binary); `BlobRef` for lazy chunk text; `QueryPlan` + `ChunkRef` + `Cost` + `PlanNode` types; typed `StageEvent`. `tests/contract/spi_signature.py` linter (RequestContext-first); rag-backends (`PgVectorStore`, `QdrantVectorStore`, `RedisCache`, `S3Storage`, `LocalFileStorage`) migrated; conformance + integration tests updated; `Budget.spend()` for agent-loop sub-turn budgets; schemas regenerated. ADR-0005 / ADR-0007 / ADR-0008 / ADR-0009 referenced. | -| 1.1b | SPI split — Retrieval/Index, bulk + streaming + ID-only | ✅ | `build/phase-1/step-1.1b-spi-split-retrieval-index` | [#45](https://github.com/officialCodeWork/AgentContextOS/pull/45) | Split `VectorStore`/`KeywordStore`/`GraphStore` into `*RetrievalBackend` (read) + `*IndexBackend` (write) composite ABCs. `retrieve_ids` returns `list[ChunkRef]`; `hydrate` lives on the retrieval side (keyword full-Chunk; vector pass-through). Bulk: `bulk_index`/`bulk_delete` (+ graph bulk node/edge variants); streaming: `stream_index` async-iterator default that batches into `bulk_index`. `Embedder` split into single `embed` + canonical `bulk_embed`. New `IndexHint` + `WriteVolume` types passed to writes (per ADR-0009). Noop impls, `PgVectorStore`, `QdrantVectorStore` migrated; conformance + integration tests updated; `spi_signature.py` extended to enforce the split. Schemas regenerated (`IndexHint.json`). | -| 1.1c | PolicyEngine package | ✅ | `build/phase-1/step-1.1c-policy-engine-package` | [#46](https://github.com/officialCodeWork/AgentContextOS/pull/46) | New `packages/policy/` (`rag-policy` v0.1.0): `PolicyEngine` SPI + `NoopPolicyEngine` (always-ALLOW with tenant-scoped `filter_pushdown`); `PolicyDecision` enum (read_chunk / ingest_doc / egress_text / quota_check / rate_limit / execute_plan); `PolicyResult` (allow/deny/transform) with predicate helpers; `QuotaSubject` / `RateLimitSubject`; `FilterExpr` mini-language (Eq / AnyIn / And / Or / Not / TrueExpr) returned by `filter_pushdown`; `PolicyWriter` facade mirroring `AuditWriter` and emitting `policy.decision` structured logs via `rag-observability`. Coverage linter `tests/policy/coverage.py` greps for governance-relevant SPI calls without adjacent `PolicyEngine`/`PolicyWriter` consultation, with file allowlist that consumers shrink as they wire the PDP in. Workspace + pytest pythonpath updated. 20 conformance tests added. ADR-0005 finalized. | -| 1.1d | Pipeline + Batcher primitives | ✅ | `build/phase-1/step-1.1d-pipeline-batcher-primitives` | [#50](https://github.com/officialCodeWork/AgentContextOS/pull/50) | `Pipeline` primitive in `rag-core` (`rag_core.pipeline`): async DAG with bounded queues, per-stage worker counts, backpressure, joiner-based shutdown handling per-stage worker-count differences, `on_error="fail"`/`"skip"` policies, optional `error_handler`. `Batcher[Req, Resp]` middleware (`rag_core.batcher`, DataLoader pattern) coalescing concurrent `load(req)` calls into batched `batch_fn(reqs)` invocations; size + time triggers (`max_batch_size`, `max_wait_ms`); per-request `asyncio.Future` for isolated error propagation; `flush()` for deterministic shutdown. PEP-695-typed generics. 22 unit tests (8 Batcher + 14 Pipeline). `docs/architecture/pipeline-batcher.md` + `rag-core` reference + `performance.md` backlinks. `rag-core` 0.5.0 → 0.6.0. | -| 1.1e | Cache SPI split + perf discipline + async telemetry | ✅ | `build/phase-1/step-1.1e-cache-split-perf-telemetry` | [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) | `EmbeddingCache` (`rag_core.spi.embedding_cache`, key tenant+model_id+model_version+text_hash, `invalidate_model`), `RetrievalCache` (`rag_core.spi.retrieval_cache`, key tenant+plan_hash+corpus_version, `invalidate_corpus`), `AnswerCache` (`rag_core.spi.answer_cache`, key tenant+plan_hash+corpus_version+policy_version, `invalidate_policy`). All ctx-first; signature linter extended. Noop in-memory impls + 18 conformance tests covering hit/miss/version-partition/invalidate/tenant-isolation. `AsyncTelemetrySink` in `rag-observability` — bounded buffer (default 10_000), non-blocking `submit()`, per-kind drop + exporter-error counters, idempotent start/stop, drain-timeout-cancel; 9 unit tests. Hot-path discipline doc in `performance.md` expanded with concrete hot/cold call-site table. New `docs/reference/rag-observability.md`. `caching.md` SPI shapes updated with ctx-first signatures. `rag-core` 0.6.0 → 0.7.0. | -| 1.1f | ADRs 0005–0009 + reviewer checklist | ✅ | `build/phase-1/step-1.1f-adrs-reviewer-checklist` | [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) | ADR-0005 (PolicyEngine PDP), ADR-0006 (two-stage reranker default), ADR-0007 (tiered storage with BlobRef), ADR-0008 (cost-aware planner replacing reactive fallback), ADR-0009 (vector index strategy + quantization) all promoted to Accepted with implementation-path backlinks. Reviewer checklist in `docs/architecture/performance.md` extended with explicit items for plan/budget awareness, IndexHint+dtype handling, two-stage rerank shape, and BlobRef-safe consumers. ADR backlinks added to `rag_core.spi.reranker` and `rag_core.spi.storage` (the two missing module-level pointers). | -| 1.2 | Connectors framework | ✅ | `build/phase-1/step-1.2-connectors-framework` | [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) | `Connector` SPI evolved to `(Document, ConnectorState)` async-iterator with resumable watermark (`ConnectorState` frozen model: connector_id, tenant_id, cursor, last_seen_at, extra). Internal `Crawler[Raw]` base class in `rag_backends.connectors._base` (iteration scaffold, error policy, monotonic watermark). Built-in connectors: `FilesystemConnector` (cross-platform pathlib, sha256 hashing, sorted resume cursor), `S3Connector` (aioboto3, ListObjectsV2 continuation token, MinIO-compatible), `GCSConnector` (gcloud-aio-storage, REST nextPageToken, `[gcs]` extra). Policy boundary documented — connectors are upstream of `PolicyEngine`; the ingest pipeline (Step 1.10) is the enforcement point. 9 contract tests (NoopConnector resume + watermark + tenant mismatch), 9 unit tests (FilesystemConnector), MinIO + GCS integration tests skip-if-no-service. `ConnectorState` schema added to `dist/schemas/`. New docs: [reference/connectors.md](docs/reference/connectors.md), [architecture/connectors.md](docs/architecture/connectors.md). | -| 1.3 | Document parsers | ✅ | `build/phase-1/step-1.3-document-parsers` | [#57](https://github.com/officialCodeWork/AgentContextOS/pull/57) | New `rag-parsers` package (`packages/parsers/`, v0.1.0) shipping `PlainTextParser`, `MarkdownParser` (markdown-it-py), `HtmlParser` (BeautifulSoup + stdlib `html.parser`), `JsonParser`, `CsvParser`, `YamlParser` (PyYAML `safe_load`), `PdfParser` (pypdf), `DocxParser` (python-docx), `PptxParser` (python-pptx), `XlsxParser` (openpyxl). `Parser` SPI evolved: `parse(...)` now returns `ParsedDocument` (`Document` + extracted text + structural `Block` list with offsets + heading levels + parser-specific `attributes`). New types in `rag_core`: `Block`, `BlockType` (8-value enum), `ParsedDocument`. `rag_parsers.mime.detect_mime()` (puremagic + OOXML ZIP central-directory disambiguation + extension shortcut + hint fallback). `ParserRegistry` + `default_registry()` (first-match-wins; specific MIMEs before generics). `ragctl parse ` smoke-test command (MIME detect → parser select → print summary + first-N blocks). 35 parser unit tests + 6 ragctl tests + extended contract tests (148 contract tests total pass). Schemas regenerated (`Block.json`, `ParsedDocument.json`). `rag-core` 0.7.0 → 0.8.0. Cross-platform: every parser library is pure-Python or wheel-distributed; no `libmagic` / `lxml` system deps. New docs: [reference/parsers.md](docs/reference/parsers.md), [architecture/parsers.md](docs/architecture/parsers.md). | -| 1.4 | OCR pipeline | ✅ | `build/phase-1/step-1.4-ocr-pipeline` | [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) | New `rag-ocr` package (`packages/ocr/`, v0.1.0): `TesseractOCR` (per-word regions, ISO-639-3 lang codes, `pytesseract` wrapper) + `PaddleOCRBackend` (per-line regions, quadrilateral → axis-aligned bbox, lazy engine init); both behind optional extras `[tesseract]`/`[paddle]` so default install stays pure-Python. OCR SPI evolved: `BoundingBox(x0,y0,x1,y1)` + `OCRRegion(text,confidence,bbox,page_number)` + `OCRResult(text,confidence,regions,page_number,engine)` Pydantic v2 frozen models in `rag_core.types`; aggregate confidence is text-length-weighted mean (1.0 on empty regions so "no text" doesn't read as failure); `NoopOCR` migrated; 5 contract tests updated. `ragctl ocr ` smoke command (`--engine tesseract\|paddleocr`, `--lang`, `--regions N`) — 5 CLI tests. 22 unit tests under `tests/ocr/` stub both engines at the `sys.modules` boundary so CI runs without Tesseract binary or PaddlePaddle wheels. Schemas regenerated (`BoundingBox.json`, `OCRRegion.json`, `OCRResult.json`). `rag-core` 0.8.0 → 0.9.0. New docs: [reference/ocr.md](docs/reference/ocr.md), [architecture/ocr.md](docs/architecture/ocr.md). | -| 1.5 | Structure-aware chunker | ✅ | `build/phase-1/step-1.5-structure-aware-chunker` | [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) | New `rag-chunker` package (`packages/chunker/`, v0.1.0): `HeadingAwareChunker` walks `ParsedDocument.blocks`, maintains a heading-level stack for `parent_id` semantics (h2 under h1 → parent=h1; new h2 sibling reparents under the same h1), accumulates body blocks under each heading up to a token budget (default 512 tokens, 64 overlap), splits oversized blocks on sentence boundaries (pure-Python regex), hard-splits sentences that still exceed the budget. Within-section overlap only; never bleeds across heading boundaries. Chunker SPI added: `rag_core.spi.chunker.Chunker` ABC (`async chunk(ctx, parsed) -> list[Chunk]`); `NoopChunker` one-chunk-per-block fallback; 7 contract tests + signature linter extension enforcing monotonic `position`, parent-refs-earlier-only, tenant/document propagation. `TokenCounter` Protocol + `TiktokenCounter` (default `cl100k_base`, swap encodings via constructor); tiktoken pulled in as default dep with wheels for all OS+arch combos. `ocr_result_to_parsed_document()` adapter turns Step 1.4 `OCRResult` into a synthetic `ParsedDocument` so OCR'd image pages flow through the same chunker (each `OCRRegion` → paragraph `Block` with `engine`/`ocr_confidence`/`bbox`/`page` in attributes). `ragctl chunk ` smoke command (`--max-tokens`, `--overlap-tokens`, `--mime`, `--chunks N`) — 5 CLI tests. 33 unit tests under `tests/chunker/` (sentence splitter + tokenizer + heading-aware + OCR adapter). `rag-core` 0.9.0 → 0.10.0. Cross-platform: pure-Python except `tiktoken` extension; unit tests use a `_WordCounter` stand-in so token-boundary assertions don't depend on BPE specifics. New docs: [reference/chunker.md](docs/reference/chunker.md), [architecture/chunker.md](docs/architecture/chunker.md). | -| 1.6 | Metadata enricher | ✅ | `build/phase-1/step-1.6-metadata-enricher` | [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) | New `rag-enricher` package (`packages/enricher/`, v0.1.0): `DefaultEnricher` tags each chunk's metadata with `language` (langdetect, deterministic seed; skipped < 20 chars), `doc_type` (from `Document.mime_type` via `doc_type_from_mime()` — markdown/pdf/docx/pptx/xlsx/json/csv/yaml/html/text/ocr or first-of-mime fallback), `created_at`/`modified_at` (parser-extracted dates from `Document.metadata` override `Document.created_at`/`updated_at`), `author` and `title` when present, `source_uri`, `document_id`, `section_path` (root-first heading-text list from walking `parent_id` chain via the new `section_path()` helper; O(d) via per-doc lookup table; cycle-guarded), and `reading_level` (Flesch-Kincaid grade via textstat — gated on body chunks + English + ≥80 chars so we skip rather than emit a misleading "grade 0"). Enricher SPI added: `rag_core.spi.enricher.Enricher` ABC (`async enrich(ctx, chunks, parsed) -> list[Chunk]`); `NoopEnricher` passthrough; 4 contract tests + signature linter extension enforce length/order/identifying-fields preserved (only `metadata` differs). `LanguageDetector` Protocol + `LangdetectDetector` default (seeded `DetectorFactory.seed=0` so cache keys derived from chunk metadata are stable across runs); swappable via constructor. `ragctl enrich ` smoke command pipes parse → chunk → enrich and prints tag-coverage summary + first-N enriched chunks with language / reading level / section path — 4 CLI tests. 30 unit tests under `tests/enricher/` (doc_type, language, section_path, default_enricher integration). `rag-core` 0.10.0 → 0.11.0. Cross-platform: langdetect + textstat + pyphen all ship pure-Python wheels — no system binaries, no native extensions. New docs: [reference/enricher.md](docs/reference/enricher.md), [architecture/enricher.md](docs/architecture/enricher.md). | -| 1.7 | PII detection | ✅ | `build/phase-1/step-1.7-pii-detection` | [#66](https://github.com/officialCodeWork/AgentContextOS/pull/66) | New `rag-pii` package (`packages/pii/`, v0.1.0): `RegexPIIDetector` (pure-Python, always available — EMAIL/PHONE/SSN/CREDIT_CARD/IP_ADDRESS with type-specific confidence scores and Luhn-validated credit cards; regex patterns mirror `rag_observability.events.check_pii` so the Step 0.7b log-leak CI gate and ingest detection share one definition of PII) + `PresidioPIIDetector` (optional `[presidio]` extra; lazy `AnalyzerEngine` init; overlap resolution matches the regex path). `PiiProcessor` applies `ctx.pii_policy.action` across the chunk list — **redact** (default, `` markers), **mask** (`*` of equal length, preserves layout), **block** (chunk dropped) — pre-filters spans by `policy.entities` + `policy.min_score`, and returns `(filtered_chunks, outcomes)` so callers see exactly what happened per chunk without re-running the detector. Pure helpers `redact_spans()` / `mask_spans()` exported for custom flows. Event protocol: one `pii.detected` `PiiEvent` per entity type per affected chunk, carrying only metadata (`field_name=chunk:`, `pii_type`, `action_taken`, plus `chunk_id`/`document_id`/`principal_id`/`span_count` in `metadata`) — never raw matched text, so the existing log-leak CI gate keeps passing. SPI evolution: `PIISpan` promoted from dataclass to Pydantic v2 frozen model in `rag_core.types` (validates `end >= start` + 0–1 score); schema exported (`PIISpan.json`). `ragctl pii ` smoke command (`--detector regex\|presidio`, `--action redact\|mask\|block`, `--chunks N`) — 5 CLI tests. 35 unit tests under `tests/pii/` (rewriters, regex incl. parametrised Luhn, processor, Presidio stubbed at `sys.modules`). 8 contract tests cover SPI + PIISpan validation. `rag-core` 0.11.0 → 0.12.0. Cross-platform: core pure-Python; `[presidio]` extra needs `spacy download en_core_web_sm` separately. New docs: [reference/pii.md](docs/reference/pii.md), [architecture/pii.md](docs/architecture/pii.md). | -| 1.8 | Embedder pipeline | ✅ | `build/phase-1/step-1.8-embedder-pipeline` | [#68](https://github.com/officialCodeWork/AgentContextOS/pull/68) | New `rag-embedders` package (`packages/embedders/`, v0.1.0): **3 plugin classes covering 4 model families** behind the existing `Embedder` SPI. `OpenAIEmbedder` (`[openai]` extra) covers `text-embedding-3-{small,large}` + `ada-002` with server-side dim reduction for v3 models via the `dimensions` request parameter; `CohereEmbedder` (`[cohere]` extra) covers `embed-{english,multilingual}-v3.0` with SDK v4/v5 response-shape probing and `search_document` default `input_type`; `SentenceTransformersEmbedder` (`[sentence-transformers]` extra) covers any HuggingFace model, with `bge_large_en()` and `e5_large_v2()` factory helpers — E5 passage prefix auto-applied via `_PASSAGE_PREFIXES`; encoding runs in `asyncio.to_thread` so the sync API doesn't block the loop. All three subclass `BatchingEmbedder` for shared sub-batch splitting (OpenAI 2048 / Cohere 96 / local 32), exponential-back-off retry via `RetryPolicy` (`retry_on` tuple declares which provider exceptions are transient), and dimension normalization (truncate + L2 renorm). `normalize_dimension()` exposed as a pure helper. `ragctl embed ` smoke command — default `noop` backend exercises the full parse → chunk → enrich → PII → embed pipeline without provider credentials; `--backend openai\|cohere\|bge\|e5` selects real backends. 41 unit tests under `tests/embedders/` (dim + retry + base + 3 backend test files with each provider stubbed at `sys.modules`); 5 CLI tests. PolicyEngine coverage linter allowlist extended to cover `ragctl/main.py` (operator CLI, not production policy boundary). Cross-platform: core pure-Python; `[sentence-transformers]` extra pulls in PyTorch (~600 MB) but ships wheels for ubuntu/macos/windows. New docs: [reference/embedders.md](docs/reference/embedders.md), [architecture/embedders.md](docs/architecture/embedders.md). | -| 1.9 | CDC connectors (incremental sync) | ✅ | `build/phase-1/step-1.9-cdc-connectors` | [#70](https://github.com/officialCodeWork/AgentContextOS/pull/70) | Three change-data-capture connectors + a fingerprint-based dedup wrapper, all satisfying the existing `Connector` SPI so the Step 1.10 ingest pipeline drives them uniformly with bulk crawlers. `PostgresCDCConnector` (`[postgres-cdc]` extra — `psycopg[binary]`) uses the built-in `pgoutput` logical-replication plugin; WAL LSN as cursor; INSERT/UPDATE for the same primary key collapse to `Document.id="pg::"` so the latest value wins; DELETE shares the id with `metadata["cdc_op"]=="D"` so the ingest pipeline can treat it as a tombstone. `S3EventsConnector` (uses existing aioboto3) long-polls SQS for S3 event notifications, parses both direct-to-SQS and SNS-wrapped envelopes, filters cross-bucket records; tombstones (`ObjectRemoved`) get a synthesised content hash since eTag is absent for deletes; SQS messages are deleted only after their records are emitted (at-least-once delivery, dedup helper covers the exactly-once gap). `WebhookReceiverConnector` (pure stdlib) is push-style — wraps an `asyncio.Queue` that the gateway's webhook route fills via `publish(document, content)`; three backpressure modes (`wait`/`drop`/`raise`) for different operator preferences. `DedupFilter` wraps any `Connector` + `Cache` and drops emissions whose `Document.fingerprint` (== `Document.content_hash`) is already cached; tenant-namespaced cache key; TTL bounds memory + defines dedup horizon; itself satisfies the `Connector` SPI so it composes anywhere; exposes `DedupStats(seen, deduped)` for ingest-pipeline reporting. `Document.fingerprint` property added to `rag-core` (returns `content_hash`, exposed so storage layout is hideable). 35 unit tests under `tests/backends/` covering dedup/webhook/S3-events/postgres-cdc — all SDKs stubbed at `sys.modules` so CI runs without real Postgres / AWS / event sources. Cross-platform: webhook is pure-stdlib; S3 events reuses default aioboto3; `[postgres-cdc]` ships pre-built psycopg wheels for ubuntu/macos/windows (no system libpq). New docs sections in [reference/connectors.md](docs/reference/connectors.md) (CDC section + DedupFilter section + extended error table) and [architecture/connectors.md](docs/architecture/connectors.md) (pgoutput choice rationale, S3-event transport shapes, push-style webhook design, fingerprint dedup semantics). | -| 1.10 | Write path & ingest API | ✅ | `build/phase-1/step-1.10-write-path-ingest-api` | [#72](https://github.com/officialCodeWork/AgentContextOS/pull/72) | **CLI demo milestone — Phase 1 complete.** New `rag-ingest` package (`packages/ingest/`, v0.1.0): `IngestPipeline` orchestrator stitches every Phase 1 stage into one async call — `ingest_document(ctx, document, content, *, mime_type=None) → IngestResult` and `ingest_connector(ctx, connector, *, on_result=None) → IngestRunSummary`. Stage order parse → chunk → enrich → PII → embed → index (PII runs before embed so vectors only ever encode policy-compliant text — vector similarity attacks can't reconstruct PII). Optional `PolicyEngine.evaluate(ingest_doc, …)` short-circuits before parse (deny → `status=blocked_by_policy` without paying for parse/chunk/embed). Per-document errors caught and surfaced as `status=failed` so one bad doc never aborts a run; `KeyboardInterrupt`/`SystemExit` deliberately propagate. Package depends only on `rag-core`; `rag_policy.types` imported lazily inside `_policy_check` so rag-ingest itself stays optional-policy. SPI evolution: `IngestStatus` enum + `IngestResult` + `IngestRunSummary` Pydantic frozen models in `rag_core.types`; schema exported (`IngestResult.json`, `IngestRunSummary.json`); `rag-core` 0.12.0 → 0.13.0. FastAPI gateway 0.2.0 → 0.3.0: `POST /v1/ingest/document` multipart upload (tenant_id/corpus_id/principal_id/source_uri form fields + file part) returning `IngestResult` JSON; `GET /healthz`; `GET /v1/info`; `build_app(pipeline=None)` factory for DI; module-level `app` for `uvicorn rag_gateway.app:app`; `python-multipart` dep added. `ragctl ingest ` wired for real (replaces Step 0.10 scaffold) — directory crawl with summary, `--per-doc` for streaming, single-file mode, `--tenant`/`--corpus` overrides; uses `FilesystemConnector` + noop embedder/vector store so it runs anywhere. 38 new tests: 16 IngestPipeline unit (happy path / unsupported MIME / MIME override / PII metadata / embeddings indexed / blocked_by_policy short-circuit / skipped_empty / PII block drops chunks / error wrapping / MIME fallback) + 6 gateway FastAPI TestClient + 6 ragctl ingest CLI + scaffold-test cleanup. Cross-platform: pure-Python throughout (python-multipart is a pure wheel); no infra required for the CLI demo. Policy boundary: IngestPipeline IS the canonical `ingest_doc` PolicyEngine consumer (call site, not a bypasser); coverage linter still passes because the consultation lives in `_policy_check` with the PolicyDecision marker. Deliberately deferred: rag.yaml → backend wiring (Step 3.5), per-stage observability spans (Step 5.1), AuditWriter emissions for ingest events (Step 6.6), outbound webhooks (Step 3.9). New docs: [reference/ingest.md](docs/reference/ingest.md), [architecture/ingest.md](docs/architecture/ingest.md). | +### 0.1 — Monorepo & build system ✅ [#3](https://github.com/officialCodeWork/AgentContextOS/pull/3) ---- +- `pyproject.toml`, `Makefile`, `.pre-commit-config`, CI workflows +- `docker-compose`, lockfiles +- ADR-0001 -## Phase 2 — Retrieval Engine (Weeks 8–12) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 2.1 | Knowledge store read layer | ✅ | `build/phase-2/step-2.1-knowledge-store-read-layer` | [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | `FilterExpr` AST + in-memory `evaluate()` reference semantics relocated from `rag-policy` into `rag_core.filter` so SPI signatures can take it directly without inverting the dependency graph (`rag_policy.filter` keeps re-exporting every symbol for back-compat). `VectorRetrievalBackend.retrieve_ids` switched from `dict[str, Any]` to typed `FilterExpr \| None`; `KeywordRetrievalBackend.retrieve_ids` gains the same parameter. New SPI signature gate `test_retrieve_ids_accepts_filter_expr` enforces the shape on every retrieval ABC. Noop stores (`NoopVectorStore`, `NoopKeywordStore`) apply the predicate via `evaluate()` against a per-chunk attribute view — the canonical oracle every translator must agree with. `PgVectorStore`: new `_filter_sql.translate()` produces parameterised `WHERE` fragments, `acl_labels TEXT[]` column + GIN index added (idempotent `ADD COLUMN IF NOT EXISTS` migration on `initialize()`), `bulk_index` persists ACL labels from `Embedding.acl_labels`. `QdrantVectorStore`: new `_filter_qdrant.translate()` builds `Filter(must / must_not / should)` merged with existing tenant/corpus `must` clauses, payload gains `acl_labels` on every write. Unsupported fields raise `RetrievalError` (no silent push-down loss). 75 new tests (16 `evaluate` units, 16 SQL translator units, 11 Qdrant translator units, 5 vector contract push-down, 3 keyword contract push-down, 1 signature gate, schema regen). New docs: [architecture/retrieval-read-layer.md](docs/architecture/retrieval-read-layer.md); `reference/rag-core.md` + `reference/rag-policy.md` updated. `rag-core` 0.13.0 → 0.14.0. Schemas regenerated (`Eq.json`, `AnyIn.json`, `And.json`, `Or.json`, `Not.json`, `TrueExpr.json`). | -| 2.2 | Vector retrieval backends | ✅ | `build/phase-2/step-2.2-vector-retrieval-backends` | [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | Three new vector backends behind their own optional extras: `WeaviateVectorStore` (`[weaviate]`, weaviate-client v4 async; single `RagEmbeddings` collection w/ `tenant_id` filter + UUID5 `(tenant, chunk)` IDs; `text[]` `acl_labels` property pushed down via `_filter_weaviate.translate()` that lazy-imports the SDK and composes `Filter.by_property(...).equal/contains_any` with `&`/`\|`/`~`), `PineconeVectorStore` (`[pinecone]`, PineconeAsyncio v5+; namespace=tenant primary isolation primitive plus belt-and-braces `tenant_id` filter; metadata: `tenant_id`/`chunk_id`/`corpus_id`/`model`/`acl_labels`; `_filter_pinecone.translate()` emits MongoDB-style `$eq`/`$in`/`$ne`/`$nin`/`$and`/`$or` and rewrites `Not` via De Morgan since Pinecone has no `$not`), `ElasticsearchDenseVectorStore` (`[elasticsearch]`, AsyncElasticsearch 8.x w/ `dense_vector` + native kNN; single index, `tenant_id` keyword filter, composite `_id="::"`; `_filter_elasticsearch.translate()` emits query-DSL `bool.filter`/`terms`/`match_all`/`match_none`). All five backends now consume `IndexHint` via the new shared `_index_hint.select_index_variant()` helper that centralises the ADR-0009 §1 size/recall → variant table — pgvector picks ivfflat vs HNSW DDL at `initialize()` time (with a new `_index_ddl_for` helper; PQ tier falls back to HNSW w/ larger `m` since pgvector has no PQ), Qdrant configures `HnswConfigDiff` + scalar `ScalarQuantization(INT8)` for the PQ tier (new `_qdrant_index_config` helper), Weaviate picks `Configure.VectorIndex.{flat,hnsw,hnsw+pq}` via a `_vector_index_config` helper, ES picks `index=false` (flat) vs HNSW `index_options.{m,ef_construction}` per tier, Pinecone records the advisory variant in logs (managed indexes). Each SDK is imported lazily inside `__init__` so importing the module without the extra still succeeds (constructing the class raises a clear `ImportError` pointing at the extra). 53 new tests: 11 `_index_hint` parametrised tier tests, 12 `_filter_pinecone` translator units, 11 `_filter_elasticsearch` translator units, 13 `_filter_weaviate` translator units (SDK stubbed at `sys.modules` so they run without the extra) + 6 integration suites per backend gated on `pytest.importorskip` + a service-reachability fixture (Weaviate via `/v1/.well-known/ready`, ES via `_cluster/health`, Pinecone via `TEST_PINECONE_API_KEY`). Policy-engine coverage allowlist extended for the three new files (the call site, not the engine, consults `PolicyEngine`). New extras + workspace wiring in `packages/backends/pyproject.toml` (`weaviate`/`pinecone`/`elasticsearch`) and mypy overrides in root `pyproject.toml` so CI without the extras stays green. Cross-platform: all three SDKs ship pure-Python or wheel distributions for ubuntu/macos/windows. Deliberately deferred: `ragctl ann tune` (Step 2.6 — query understanding owns the planner-side tuning surface) and per-corpus `dtype: int8` knob in `rag.yaml` (Step 3.5 — corpus router lands rag.yaml → backend wiring). New docs: [reference/backends.md](docs/reference/backends.md) (Weaviate / Pinecone / ES sections + IndexHint mapping tables per backend), [architecture/vector-backends.md](docs/architecture/vector-backends.md) (IndexHint → variant table, FilterExpr translator contracts, tenant-isolation patterns, extension points). | -| 2.3 | BM25 / keyword retrieval | ✅ | `build/phase-2/step-2.3-bm25-keyword-retrieval` | [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) | Two new `KeywordStore` backends behind their own optional extras, on top of the Step 2.1 read-layer contract — no new SPI surface, per the Step 2.3 constraint. `ElasticsearchKeywordStore` (`[elasticsearch]`, AsyncElasticsearch 8.x; single `rag-keyword` index for all tenants w/ `tenant_id` keyword filter + composite `_id="::"`; BM25 over operator-selected analyzer on `body`/`title`/`section_path`; `acl_labels`/`trust_level`/`document_id`/`corpus_id` keyword-filterable; `metadata` stored as `object` w/ `enabled: false` so the mapping doesn't explode; new `rag_backends.keyword._filter_es.translate()` emits ES query-DSL `bool.filter`/`terms`/`match_all`/`match_none` with the broader keyword-side field set vs. the vector translator). `TantivyKeywordStore` (`[tantivy]`, tantivy-py 0.22+; RAM or persistent index; sync API wrapped in `asyncio.to_thread`; composite `_doc_id` field enables idempotent re-indexing via `delete_documents` before `add_document`; `_filter_tantivy.translate()` composes `boolean_query` w/ `term_query` + pairs `MustNot` with `all_query` so `Not(x)` composes stand-alone; metadata stashed as JSON blob for hydrate round-trip). **Per-field boosting is a backend configuration knob, not a per-call SPI parameter** (operator passes `field_boosts={"title": 3.0, "section_path": 2.0, "body": 1.0}` at construction time; ES applies via `multi_match.fields` with `type=best_fields`, tantivy applies via parser's `field_boosts` arg). Same indexed-field schema across both backends so swapping is a config change. SDKs imported lazily so `import rag_backends.keyword.{elasticsearch,tantivy}` without the extra still succeeds. 47 new tests: 13 ES filter-translator units + 14 tantivy filter-translator units (tantivy module stubbed at `sys.modules`) + 9 ES backend units (full SDK stubbed — initialize/bulk_index/search/mget/delete) + 11 tantivy backend units (full SDK stubbed — including idempotency + boost ranking). Policy-engine coverage allowlist extended for the two new files (call site, not the backend, consults `PolicyEngine`). `tantivy>=0.22` added to root `pyproject.toml` mypy overrides. Cross-platform: tantivy-py ships pre-built wheels for ubuntu/macos/windows (no Rust toolchain required at install time); ES SDK pure-Python. Deliberately deferred: rag.yaml → backend wiring (Step 3.5 corpus router), real-time index lag metrics (Step 5.1), custom highlighter / snippet generation (Step 2.8 context packer owns snippet extraction). New docs: [reference/backends.md](docs/reference/backends.md) (ES BM25 + tantivy sections), [architecture/keyword-backends.md](docs/architecture/keyword-backends.md) (per-field boosting as a backend config knob, indexed-field schema, tenant isolation pattern, FilterExpr translator contracts, tantivy `Not` quirk, when to pick which). | -| 2.4 | Graph retrieval | ✅ | `build/phase-2/step-2.4-graph-retrieval` | [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) | Three new `GraphStore` backends + a structured multi-hop traversal primitive layered onto the existing SPI surface. New `NeighborResult` Pydantic frozen model in `rag_core.types` (node_id / labels / properties / distance / path). New `expand(ctx, seed_ids, *, hops, rel_types, direction, node_filter, edge_filter, limit) → list[NeighborResult]` method on `GraphRetrievalBackend` — added as a **non-abstract default method** that raises `NotImplementedError`, so the abstract SPI surface is unchanged and existing ABC consumers (NoopGraphStore aside, which we explicitly upgraded) keep compiling. `NoopGraphStore` upgraded with a reference BFS implementation supporting `FilterExpr` node/edge filters — the conformance oracle every concrete backend must agree with. **`FilterExpr` push-down on edges + nodes uses the existing AST — no new types, no new parameter shape.** `edge_filter` applies to **every relationship** along the traversal path (the Step 2.4 headline ACL use case); `node_filter` applies to the **final neighbor only**, mirroring Cypher `WHERE neighbor.foo = X` placed at the path endpoint rather than every intermediate node (intermediate-node filtering would silently drop reachability — almost always wrong). Three backends: `Neo4jGraphStore` (`[neo4j]`, neo4j async driver via Bolt + Cypher), `MemgraphGraphStore` (`[memgraph]`, same neo4j driver — Memgraph speaks the same protocol), `NetworkXGraphStore` (`[networkx]`, pure-Python in-process MultiDiGraph). Neo4j + Memgraph share `BoltCypherStore` so the Cypher composition + parameter handling + tenant isolation + relationship-type validation are written once. Schema convention shared across all three: nodes MERGE'd by `(id, tenant_id)`, labels stored as a property (avoids APOC / dynamic labels), every relationship carries `tenant_id` so multi-hop traversal blocks cross-tenant leaks at every edge, relationship types validated against `^[A-Za-z_][A-Za-z0-9_]*$` + backtick-quoted (Cypher has no relationship-type parameter — interpolation is unavoidable, but injection is not). New `rag_backends.graph._filter_cypher.translate(expr, *, variable, array_fields, param_prefix)` emits parameterised Cypher `WHERE` fragments + a parameter dict; `param_prefix` lets a caller compose multiple translator outputs into one Cypher statement without parameter collisions (used by `expand()` to combine edge + node filters). 65+ new tests: 13 conformance tests for `expand()` on `NoopGraphStore` (the oracle — one-hop / two-hop / seed exclusion / path recording / rel_types / direction / edge & node filters / limit / tenant isolation / hops≤0 / missing seed), 16 Cypher translator units, 21 Neo4j + Memgraph unit tests (neo4j driver stubbed at `sys.modules` — every Cypher statement captured + asserted on for shape + parameter scoping + injection guard), 14 NetworkX backend tests (real networkx pulled via the extra; results compared element-wise against the Noop reference oracle for portability). Policy-engine coverage allowlist extended for the four new backend files (call site, not the backend, consults `PolicyEngine`). `neo4j>=5.20` / `networkx>=3.2` added to root `pyproject.toml` mypy overrides. Schemas regenerated (`NeighborResult.json`). `query()` on `NetworkXGraphStore` deliberately raises `RetrievalError` — NetworkX has no native query language, callers should use `expand()` or reach into the exposed `.graph` MultiDiGraph property for direct NetworkX algorithm calls. Cross-platform: neo4j driver is pure-Python; networkx is pure-Python (~2 MB, no system deps). Deliberately deferred: GraphRAG / community detection (Step 2.9), per-edge weight scoring (Step 2.9), `rag.yaml` → backend wiring (Step 3.5 corpus router), per-tenant separate databases (Step 6.2 physical tenancy). New docs: [reference/backends.md](docs/reference/backends.md) (Neo4j / Memgraph / NetworkX sections including `expand()` Cypher composition + schema convention + extension points), [architecture/graph-backends.md](docs/architecture/graph-backends.md) (why `expand()` is the canonical primitive, tenant-isolation pattern, FilterExpr push-down on edges vs. node, NoopGraphStore as the conformance oracle, when to pick which backend). | -| 2.5 | Hybrid RRF fusion | ✅ | `build/phase-2/step-2.5-hybrid-rrf-fusion` | [#86](https://github.com/officialCodeWork/AgentContextOS/pull/86) | New `rag-retrieval` package (`packages/retrieval/`, v0.1.0). Pure `rrf_fuse(rankings, *, weights, k=60, top_k=None) → list[ChunkRef]` primitive — score-free fusion using only 1-indexed source ranks via `Σ w_i / (k + rank_i)`; preserves earliest-source metadata; raises `RetrievalError` on cross-tenant inputs (defence-in-depth against upstream wiring bugs). `HybridRetriever` async orchestrator fans out to vector + keyword + graph backends in parallel via `asyncio.gather(return_exceptions=True)`; per-call source routing (a wired backend whose per-call input is `None` is silently skipped — leaves query-shape routing to Step 2.10); per-backend top-k defaults to `max(top_k * 3, 30)` so RRF sees the long tail; partial-failure tolerance (one degraded backend logs `hybrid_retriever.source_failed` + drops out of fusion; only when *all* sources fail does the orchestrator raise `RetrievalError`); canonical `read_chunk` policy call site via `policy_engine.filter_pushdown` `And`-merged with caller-supplied `FilterExpr`. `HybridWeights(vector, keyword, graph)` frozen dataclass (negative-weight validator). `GraphAdapter` async protocol + `default_graph_adapter` treating `node_id == chunk_id` for the chunk-graph projection — `NeighborResult` lists arrive sorted by ascending distance per the Step 2.4 SPI contract, so RRF's rank-based fusion gets the right ordering without re-sorting; richer adapter lands with GraphRAG (Step 2.9). `ragctl hybrid ` smoke command — seeds a 5-document toy corpus + tiny graph, runs all three paths against the noop SPIs, prints fused top-k with per-chunk RRF scores; flags for `--top-k`, `--w-vector` / `--w-keyword` / `--w-graph`, `--rrf-k`. 43 new tests under `tests/retrieval/` (19 pure-fusion units: formula correctness incl. multi-list contribution sum, weight scaling, weight-zero source disabling, k smoothing curve; edge cases: empty input, top_k truncate/zero, deterministic chunk_id tie-break, metadata-from-first-source; tenant safety: mixed-tenant raise; argument validation: invalid k/top_k/weights — plus 5 graph-adapter units and 19 orchestrator integration tests covering single-source happy paths, per-call source skip, fan-out fusion, weighting effects, parallel fan-out wall-time check, partial-source failure, caller-filter pass-through, end-to-end with real `NoopVectorStore` + `NoopKeywordStore` + `NoopGraphStore`) + 5 `ragctl hybrid` CLI tests. PolicyEngine coverage allowlist *not* extended — `HybridRetriever` IS the canonical `read_chunk` call site, not a bypasser. New docs: [reference/retrieval.md](docs/reference/retrieval.md) (rrf_fuse formula + HybridRetriever API + extension points), [architecture/hybrid-fusion.md](docs/architecture/hybrid-fusion.md) (why RRF over score normalisation or learned fusion, `k = 60` justification, graph adapter rationale, what's deferred to Steps 2.6–2.10, reviewer checklist). | -| 2.6 | Query understanding | ✅ | `build/phase-2/step-2.6-query-understanding` | [#90](https://github.com/officialCodeWork/AgentContextOS/pull/90) | New `rag-query` package (`packages/query/`, v0.1.0): four query-understanding components + a parallel orchestrator. `LLMQueryRewriter` + `NoopQueryRewriter` (rewriter SPI) — paraphrase via LLM with robust one-per-line parsing (numbering / bullets / case-only dupes / original-echo filtered out). `LLMQueryDecomposer` + `HeuristicQueryDecomposer` + `NoopQueryDecomposer` (decomposer SPI) — heuristic splits on conjunctions (`and`/`then`/`but also`) + sentence terminators with `min_chars=30` floor + "any part < 3 chars → abort" guard; `[]` is the canonical "no decomposition applies" sentinel (avoids ambiguity with successful single-question decomposition); LLM variant collapses the original-echo case. `HyDEGenerator` (Gao et al. 2022) — LLM hypothetical-answer paragraph + optional `Embedder` for dense vector; component-wise averaging when `n > 1` per §2.3 of the paper; embedder-less mode supported for cache-hit / per-tenant flows. `InMemoryGlossary` + `NoopGlossaryExpander` (glossary SPI) — whole-word case-insensitive lookup, multi-word entries sorted by descending key length (operator pre-wins single-word prefixes), returns `{matched_term_lowercase: [synonym, ...]}` so the retriever picks the fold-in policy. `UnderstoodQuery` Pydantic frozen output: `original_text`, `rewrites`, `sub_queries`, `expansion_terms`, `hyde_documents`, `hyde_vector`, `metadata`; `keyword_queries` convenience property dedupes `[original_text, *rewrites]` case-insensitively for keyword fan-out. `QueryUnderstandingPipeline` async orchestrator fans out via `asyncio.gather(return_exceptions=True)` (KeyboardInterrupt/SystemExit propagate); per-component exceptions get logged as `query.understand.component_failed` and degrade to that component's identity output (`metadata["partial_failures"]` lists labels). Canonical `query.understand` OTel span via existing `span_from_trace_context(ctx.trace)` with attributes `rag.tenant_id` / `rag.query.{length_chars,rewrites_n,sub_queries_n,expansions_n,hyde_docs_n,hyde_vector_present,partial_failures_n,failed_components,elapsed_ms}`. `ragctl understand ` smoke command runs the full pipeline against `NoopLLM` + `NoopEmbedder` (no credentials required) with `--rewrites N` / `--hyde/--no-hyde` / `--decompose/--no-decompose` / `--glossary-term` / `--glossary-synonyms` flags. 55 unit tests under `tests/query/` (6 rewriter / 12 decomposer / 9 HyDE / 12 glossary / 11 pipeline integration incl. parallel fan-out wall-time check + partial-failure semantics + happy path with all four components) + 5 CLI tests. PolicyEngine coverage linter allowlist extended for the three LLM/Embedder-consuming component files (`rewriter.py`, `decomposer.py`, `hyde.py`) with justifying comment — `QueryUnderstandingPipeline` itself does NOT call PolicyEngine because query understanding is pre-retrieval text transformation; the downstream `HybridRetriever` is the canonical `read_chunk` PDP consumer and Phase 3 gateway will be the LLM-call PDP consumer. Cross-platform: pure-Python throughout; no infra required for the CLI smoke. Deliberately deferred: tenant-scoped glossary loaders (Phase 6), per-sub-query retrieval fan-out (Step 2.10 router), plan-driven component selection (Step 2.10 + 2.11), PolicyEngine consultation for LLM calls (Phase 3 gateway). New docs: [reference/query.md](docs/reference/query.md), [architecture/query-understanding.md](docs/architecture/query-understanding.md). | -| 2.7 | Cross-encoder reranker | ✅ | `build/phase-2/step-2.7-cross-encoder-reranker` | [#92](https://github.com/officialCodeWork/AgentContextOS/pull/92) | New `rag-reranker` package (`packages/reranker/`, v0.1.0) implementing the ADR-0006 two-stage cascade. SPI evolution: `Reranker.rerank()` placeholder → three-method shape (`fast_rerank` returning `list[ChunkRef]` with default pass-through truncation, `precise_rerank` returning `list[Chunk]` carrying score on `Chunk.score`, `should_early_exit` as a pure ctx-less predicate exempt in the signature linter); `rag-core` 0.14 → 0.15 added `Chunk.score: float \| None` + new `RerankerError` (RetrievalError subclass). Pure `mmr_rerank()` Carbonell-Goldstein primitive (cosine-sim cache; first-pick = argmax relevance so the algorithm is stable at λ=0; deterministic index-order tie-break). `RerankPipeline` async orchestrator: fast_rerank → should_early_exit? → hydrate via caller-supplied `Hydrator` callable (decouples from any specific backend package) → precise_rerank → optional MMR (embedding source resolution: `Chunk.metadata['embedding']` first, `Embedder.bulk_embed` fallback, else raises). Strict failure mode (propagates `RerankerError(stage=...)` rather than silently degrading — gateway decides the policy). Three backends behind optional extras: `SentenceTransformersCrossEncoder` (`[sentence-transformers]` extra; `bge_reranker_v2_m3()` ADR-default + `minilm_l6()` fast factories; sync model wrapped in `asyncio.to_thread`; sigmoid normalises raw logits into [0,1]); `CohereReranker` (`[cohere]` extra; v5+ async client; accepts pre-built client for tests; defensive [0,1] score clamp); `JinaReranker` (`[jina]` extra; `aiohttp` POST to `api.jina.ai/v1/rerank`; ownership-aware session lifecycle so caller-supplied sessions aren't closed). All SDKs imported lazily so the modules import cleanly without their extras. Canonical `rerank` OTel span via `span_from_trace_context` with `rag.tenant_id`, `rag.rerank.{backend,top_k_in,top_k_out,stage1_n,stage2_n,early_exit,mmr_applied,mmr_lambda,elapsed_ms}` (schema mirrors Step 2.6's `query.understand` for dashboard consistency). `ragctl rerank ` smoke command (default `--backend noop` runs creds-free; `--mmr/--no-mmr`, `--mmr-lambda`, `--stage1-top-n`, `--stage2-top-n`, `--top-k`, `--model`, `--backend`). 92 tests added: 9 contract reranker tests + 23 MMR units + 17 pipeline integration (in-memory OTel exporter asserts every span attribute; failure-mode coverage for all four stages; MMR metadata vs. embedder-fallback paths) + 15 sentence_transformers + 15 Cohere + 13 Jina (every SDK stubbed at `sys.modules` so CI runs without the extras) + 5 `ragctl rerank` CLI. Schemas regenerated (`Chunk.json` gains `score` property). Cross-platform: pure-Python pipeline + MMR + CLI; `[sentence-transformers]` ships PyTorch wheels for ubuntu/macos/windows. ADR-0006 promoted from "Step 2.7 will land" to "Landed in Step 2.7" with backlink table; `docs/architecture/reranking.md` + `docs/reference/reranker.md` cover two-stage rationale, MMR algorithm + embedding-source choice, composition contract with `HybridRetriever` (deliberately separable per the corpus-router argument in Step 2.10), strict failure-mode rationale, OTel span schema, and reviewer checklist. PolicyEngine boundary documented: reranker is post-retrieval over already-policed ChunkRefs; no PDP consult inside the pipeline (the coverage linter passes because `_hydrator(ctx, refs)` doesn't match `\.hydrate\(`). | -| 2.8 | Context packer | ✅ | `build/phase-2/step-2.8-context-packer` | [#94](https://github.com/officialCodeWork/AgentContextOS/pull/94) | New `rag-packer` package (`packages/packer/`, v0.1.0). `ContextPacker` async orchestrator: tenant-safety check → sort by descending `Chunk.score` → SHA-256 hash dedup over normalised content (lowercase + strip punctuation + collapse whitespace) → opt-in 5-shingle Jaccard dedup (`PackerConfig.jaccard_threshold`) → greedy token-budget fit (uses Step 1.5's `TokenCounter` Protocol; `TiktokenCounter("cl100k_base")` default; cached `Chunk.token_count` fast-path skips re-encoding) → reorder (default `lost_in_middle` per Liu et al. 2024 — strongest at both edges, weakest in middle; `descending` / `ascending` alternatives configurable per call) → heuristic conflict scan (`detect_trust_mismatch` flags chunks at different `TrustLevel`s with 3-shingle Jaccard overlap ≥ 0.30; `detect_numeric_disagreement` regex-extracts `(context, number, unit)` triples with currency-prefix + comma-separator support and flags when same leading-word context emits different normalised values across chunks). Output is frozen `PackedContext` Pydantic model (added to `rag_core.types` alongside `ConflictAnnotation` — `rag-core` 0.15 → 0.16) carrying ordered chunks + token usage + drop counters + conflict annotations + tokenizer audit string + monotonic elapsed_ms. Strict failure via `PackerError(RetrievalError)` on `budget <= 0`, mixed-tenant input (defence-in-depth mirroring `rrf_fuse`), and `content_ref`-only chunks without a cached `token_count` (ADR-0007: packer is not in the storage business). One narrow degrade-and-log path: conflict-scan exceptions log `packer.conflicts_failed` and return `conflicts=[]` — conflicts are advisory metadata, never gating. Canonical `pack` OTel span via `span_from_trace_context` carrying `rag.tenant_id`, `rag.pack.{chunks_in,chunks_out,tokens_used,tokens_budget,dropped_dedup,dropped_budget,conflicts_n,tokenizer,order,elapsed_ms}` (schema mirrors Step 2.7's `rerank`). `ragctl pack ` smoke command runs the noop-reranker → packer flow against a 5-doc toy corpus with mixed `TrustLevel`s so conflict annotations are visible without external services; flags for `--budget`, `--top-k`, `--order`, `--jaccard-threshold`, `--conflicts/--no-conflicts`. 78 new tests: 13 dedup units (hash + jaccard, normalisation invariants, tie-break, content_ref edge cases) + 15 reorder units (all three strategies × edge cases for n=0,1,2,3,4,5, lost-in-middle math match against docstring example, strongest-at-both-edges invariant) + 10 budget units (skip-and-continue, cached vs uncached token_count, content_ref-only with/without cached count, zero/negative budget) + 13 conflict units (both detectors × overlap thresholds + currency-prefix + bare-number-no-context skip) + 17 pipeline integration (in-memory OTel exporter asserts every span attribute, mixed-tenant raise, jaccard opt-in, conflict-scan-failure degrade via monkeypatch, None-score → 0.0 treatment) + 5 `ragctl pack` CLI + 5 ragctl ones from the rerank suite preserved. PolicyEngine boundary: no PDP consult inside packer (chunks arriving here already passed `read_chunk` PDP at `HybridRetriever`; the packer only reads already-policed text); coverage linter passes naturally because the packer touches none of the governed SPI methods. Cross-platform: pure-Python throughout (only deps are rag-core + rag-chunker). Schemas regenerated (`PackedContext.json`, `ConflictAnnotation.json`). Deliberately deferred: LLM-based contradiction detection (Step 5.x), semantic embedding-cosine / MinHash dedup, multi-section interleaving + per-section budget allocation, citation graph dedup (Step 2.9 GraphRAG), content-truncation / mid-chunk cuts, `content_ref`-only hydration inside the packer (ADR-0007 follow-up), per-trust-level XML isolation tags (HLD places those in the LLM adapter, not the packer), `pack` policy selection inside the corpus router (Step 2.10). | -| 2.9 | GraphRAG | ✅ | `build/phase-2/step-2.9-graphrag` | [#96](https://github.com/officialCodeWork/AgentContextOS/pull/96) | New `rag-graphrag` package (`packages/graphrag/`, v0.1.0): `LouvainDetector` (default, pure-Python NetworkX) + `LeidenDetector` (optional `[leiden]` extra) for community detection over the existing `GraphRetrievalBackend` SPI — no new SPIs introduced. `CommunitySummarizer` (LLM-backed per-community summaries via existing `rag_core.spi.llm.LLM` SPI with strict-JSON prompt template, code-fence tolerance, NoopLLM-friendly fallback path so smoke flows work without a real LLM). `InMemoryCommunityStore` (tenant-scoped storage with cross-tenant raise mirroring `rrf_fuse` + `ContextPacker` patterns; reverse `node_id → community_id` index rebuilt on every `upsert_community`). `graphrag_adapter` (entity-node → `ChunkRef` via `node.properties["chunk_ids"]`; emits one ref per chunk per entity; `1.0/(distance*rank_in_node)` score; isolated entity nodes silently skipped). `GraphRAGRetriever` async orchestrator: set-overlap community scoring with 2× weight on `key_entities` (embedder-cosine scorer is the Phase 3 follow-up alongside per-tenant embedder config), top-K community selection, dedup-across-communities key-entity seed collection, `GraphRetrievalBackend.expand()` fan-out, adapter unwrap; returns `list[ChunkRef]` ready for `HybridRetriever`'s `graph` source. New `Community` + `CommunitySummary` Pydantic frozen models in `rag_core.types` + new `GraphRAGError(RetrievalError)`; `rag-core` 0.16 → 0.17. Three canonical OTel spans (`graphrag.detect` / `graphrag.summarize` / `graphrag.retrieve`) with `rag.graphrag.*` attribute schema mirroring Step 2.7's `rerank` + Step 2.8's `pack` for dashboard consistency. `ragctl graphrag ` smoke command runs the full detect → summarise → retrieve flow against a 10-node "people + companies" toy graph (two clearly-separated 4-cliques joined by one bridge) with `NoopLLM` + `NoopGraphStore` — no infra or LLM credentials required. 54 new tests: 13 detector unit (Louvain happy path / tenant_id propagation / internal-edge recording / singleton drop / empty graph / directed-graph collapse / min_size validation / determinism / Leiden-without-extra raise / `tenant_subgraph` filter) + 10 `InMemoryCommunityStore` (upsert + read + cross-tenant raise + tenant-isolation on read + list scoping + `community_for_node` + reverse-index update on re-upsert) + 12 summariser (JSON path / code-fence strip / NoopLLM fallback / invalid-JSON-object raise / missing-summary raise / non-list-entities raise / LLM failure wrap / cross-tenant raise / max_key_entities truncate / node_properties leak-prevention / empty-list passthrough) + 7 adapter (chunk-id unwrap / score decay with rank / isolated-entity skip / non-list-chunk_ids skip / empty input / source metadata / distance-zero clamp) + 12 retriever integration (basic retrieve / empty store / empty query / no-match query / community_top_k cap / score descending / graph-failure wrap / hops propagation / seed dedup across communities / tenant isolation) + 5 `ragctl graphrag` CLI. PolicyEngine boundary: package itself does NOT call PDP (detection runs over already-policed sub-graph; summarisation is post-policy text transform — added to `tests/policy/coverage.py` allowlist with justifying comment; `GraphRAGRetriever` returns `ChunkRef` to upstream `HybridRetriever` which IS the canonical `read_chunk` PDP consumer). Cross-platform: pure-Python throughout; default deps are `rag-core` + `networkx>=3.2`; `[leiden]` extra pulls `python-igraph` + `leidenalg` C extensions only on demand. Schemas regenerated (`Community.json`, `CommunitySummary.json`). Deliberately deferred: persistent community store (Step 3.5 corpus router), hierarchical Leiden output (Phase 3 summary-of-summaries), embedder-cosine community scoring (Phase 3), edge-weight-aware Louvain (one-line change once indexer emits weights), `/v1/communities` REST endpoint + admin UI (Phase 3), LLM contradiction reconciliation across communities (Step 5.x), drift monitoring for community quality (Step 5.5). New docs: [reference/graphrag.md](docs/reference/graphrag.md), [architecture/graphrag.md](docs/architecture/graphrag.md); [architecture/hybrid-fusion.md](docs/architecture/hybrid-fusion.md) updated to point at `rag-graphrag` for the "richer graph adapter" call-out. | -| 2.10 | Retrieval router | ✅ | `build/phase-2/step-2.10-retrieval-router` | [#98](https://github.com/officialCodeWork/AgentContextOS/pull/98) | New retrieval router layered on Step 2.5's `HybridRetriever` (no new package — `rag-retrieval` 0.1 → 0.2). `RetrievalRouter` async orchestrator + pure `classify_shape` query-shape classifier (`KEYWORD_HEAVY` / `SEMANTIC` / `ENTITY_RICH` / `MIXED`; deterministic ≤3-token + question-word + glossary-expansion + capitalised-multi-word heuristics with HyDE-vector promotion to `SEMANTIC`). In-memory `BackendHealthTracker` with rolling-window failure counter (default 3 failures in 60s → degraded for 30s; explicit `mark_degraded()` for synchronous `health()` probe failures; injected clock for deterministic tests). New `RoutingDecision` Pydantic frozen model + `QueryShape` StrEnum in `rag-core` (0.17 → 0.18) + `RouterError(RetrievalError)`. Shape-bias weight table (multiplicative on `RouterConfig.base_weights`): `KEYWORD_HEAVY` → (0.3,1.5,0.5), `SEMANTIC` → (1.5,1.0,0.8), `ENTITY_RICH` → (1.0,1.2,1.8), `MIXED` → (1.0,1.0,1.0). Per-call vector input priority: caller-supplied > `UnderstoodQuery.hyde_vector` > on-the-fly `Embedder.embed()` > skip. **BM25-only fallback** flag fires when both vector + graph paths are unavailable; surfaced both on the `RoutingDecision` and as an OTel attribute. Failure-feedback loop: router calls `health.record_failure` per selected label on every `RetrievalError` from `HybridRetriever.retrieve` and `health.record_success` on the happy path so degradation state converges within a window without an external loop. Canonical `retrieve.route` OTel span with `rag.tenant_id` + `rag.route.{shape,use_vector,use_keyword,use_graph,bm25_fallback,degraded_n,degraded,weight_vector,weight_keyword,weight_graph,results_n,elapsed_ms,error}` mirroring Steps 2.7-2.9; structured log `retrieval.route_decision` mirrors the same data. `ragctl route ` smoke command runs the full decide → execute flow against the noop SPIs with `--simulate-failure {vector,keyword,graph}` (repeatable) + `--no-graph` flags so BM25-only fallback is visible without external services. 42 new tests: 21 shape classifier + 13 health tracker + 23 router integration (including in-memory OTel span attribute assertions + real `NoopVectorStore` round-trip + `probe_health_on_decide=True` path + failure_threshold=1 immediate-degrade) + 5 `ragctl route` CLI; 1600 tests passing workspace-wide. PolicyEngine boundary: router does NOT call PDP — every retrieval call goes through `HybridRetriever.retrieve` (the canonical `read_chunk` call site); coverage linter passes naturally because `router.py` makes no governed SPI calls. Cross-platform: pure-Python throughout. Schemas regenerated (`RoutingDecision.json`). Deliberately deferred (confirmed with user before kickoff): sub-query fan-out via `UnderstoodQuery.sub_queries` (Step 2.11 agent loop), cross-worker degradation state (Step 4.x circuit breaker), per-corpus weight overrides (Step 3.5 corpus router), sticky multi-turn routing (Step 2.11 agent loop), cost-aware backend skip (ADR-0008 planner + Step 3.5), learned shape classifier (Step 5.7 A/B + Step 5.4 labels). New docs: [reference/router.md](docs/reference/router.md), [architecture/retrieval-router.md](docs/architecture/retrieval-router.md); [architecture/hybrid-fusion.md](docs/architecture/hybrid-fusion.md) updated to resolve the Step 2.10 deferral. | -| 2.11 | Agent-loop validation spike | ✅ | `build/phase-2/step-2.11-agent-loop-validation-spike` | PR pending | `AgentLoopV0` in `rag-retrieval` 0.2 → 0.3 (`agent_loop.py`): thin iterative-retrieve orchestrator composing the Step 2.5/2.10 hybrid + router with budget enforcement (`Budget.spend` per iter via `ctx.model_copy`; rounds wall_ms up via `math.ceil`), sticky corpus routing (`AgentLoopConfig.allow_route_switch=False` pins iter-0 decision), plan-reuse measurement (`_decisions_match` ignoring `reason` + `degraded_backends`), shared `EmbeddingCache` integration (HyDE-vector > text-hash cache > `Embedder.embed` + write-back > skip), sub-query fan-out via `UnderstoodQueryLike.sub_queries`. Cross-package isolation via `QueryUnderstander` + `UnderstoodQueryLike` Protocols (rag-retrieval still rag-core-only dependency). New `rag-core` 0.18 → 0.19 types: `StopReason` (7-value StrEnum), `BudgetSpend`, `CacheStats` (with computed `embedding_hit_rate`), `AgentLoopError(RetrievalError)`. Canonical `agent_loop.run` OTel span + `agent_loop.iteration_complete` structured-log event. `ragctl agent-loop` smoke command (no LLM creds). 50-query harness in `eval/agent_loop_v0/` over toy 50-doc corpus, 5 buckets × 10 queries (follow_up / refinement / decomposable / shape_shifter / budget_stressor); deterministic against noop SPIs; writes `report.json` + `report.md`. **Headline finding:** 0% embedding-cache hit rate on the decomposable bucket — drives G-01 must-fix-before-3.1. 66 new tests; 1468 pass workspace-wide. New docs: [reference/agent-loop.md](docs/reference/agent-loop.md), [architecture/agent-loop-v0.md](docs/architecture/agent-loop-v0.md), [spikes/agent-loop-v0-gaps.md](docs/spikes/agent-loop-v0-gaps.md); updated retrieval-router.md (resolves sub-query fan-out + sticky-routing deferrals), request-context.md (budget-derivation pattern), caching.md (decomposable 0%-hit finding). Schemas regenerated: `BudgetSpend.json`, `CacheStats.json`. Cross-platform: pure-Python throughout. | +### 0.2 — Core domain types & errors ✅ [#4](https://github.com/officialCodeWork/AgentContextOS/pull/4) ---- +- `types.py` — 11 Pydantic v2 models · `errors.py` — 14-type exception hierarchy +- `gen_schemas.py` — JSON Schema export · `proto/core.proto` -## Phase 3 — Gateway & Agent Runtime (Weeks 12–18) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 3.1 | Gateway service (REST + OpenAPI) | ✅ | `build/phase-3/step-3.1-gateway-query-surface` | [#107](https://github.com/officialCodeWork/AgentContextOS/pull/107) | **🥈 Curl-able RAG milestone delivered.** `rag-gateway` 0.3 → 0.4 ships the full Phase 3 entry surface: `POST /v1/query` (full RAG: understanding → router → optional rerank / pack / LLM generate; returns `QueryResponse` with chunks, chunk_refs, citations, packed context, optional Answer, per-stage timings), `POST /v1/retrieve` (retrieval-only — returns `RetrieveResponse` with router chunk_refs + decision), `GET /v1/corpora` + `GET /v1/corpora/{id}` (read-only tenant-scoped corpus discovery via new `CorpusStore` SPI), preserved `POST /v1/ingest/document` from Step 1.10. New `rag-core` 0.19 → 0.20 wire types in `rag_core.gateway_types`: `Corpus`, `QueryRequest`, `QueryResponse`, `RetrieveRequest`, `RetrieveResponse`, `Answer`, `CorpusList`, `StageTimings`, `GatewayError`. New `CorpusStore` SPI in `rag_core.spi.corpus_store` + `NoopCorpusStore` (tenant-scoped in-memory; production Postgres-backed impl deferred to Step 3.5). FastAPI middleware (`apps/gateway/src/rag_gateway/middleware.py`) reads `X-Request-Id` / W3C `traceparent` / `Authorization: Bearer` headers and builds the per-request `RequestContext`; route handlers consume it from `request.state.gateway` (header-driven routes) or build their own from the request body identity (body-driven `/v1/query`, `/v1/retrieve`). Domain errors mapped to HTTP via FastAPI exception handlers: `ACLDeniedError` → 403, `AuthError` → 401, `RateLimitError` → 429, `RetrievalError` → 502, `ValueError` → 400; all 4xx/5xx bodies use the `GatewayError` envelope (`{"error": {code, message, request_id, trace_id, context}}`). PolicyEngine consult: `PolicyEngine.evaluate(egress_text, packed_chunks)` runs before every LLM `complete` call so operators plug in PII redaction / trust-level gating / per-tenant content rules; `deny` short-circuits to `answer=null` (chunks still returned), `transform` substitutes the subject before generation. OTel spans: `gateway.query` + `gateway.retrieve` wrap the per-stage child spans (`query.understand`, `retrieve.route`, `rerank`, `pack`). Structured-log events: `gateway.query_complete` (per call), `gateway.answer.failed` (LLM failure under generate=true; non-fatal — chunks still returned), `gateway.answer.policy_denied` (PDP egress deny). `build_app(...)` factory accepts every pipeline as an injectable keyword (understanding / retrieval_router / reranker / packer / llm / corpus_store / auth / policy_engine / default_tenant_id); defaults are noop SPIs throughout so the gateway boots and serves every route with zero infrastructure. Default cache wires the G-01 `SemanticEmbeddingCache` wrapper. OpenAPI 3.1 auto-generated via FastAPI 0.115+ with tags (`health` / `ingest` / `queries` / `corpora`), response-model declarations on every route, and explicit `responses={401, 403, 404, 422, 429, 500, 502}` tables; Swagger UI at `/docs`, ReDoc at `/redoc`, OpenAPI JSON at `/openapi.json`. `ragctl query` smoke command drives the in-process app via `TestClient` (no live HTTP listener needed); flags `--top-k`, `--rerank/--no-rerank`, `--pack/--no-pack`, `--generate/--no-generate`, `--corpus` (repeatable), `--tenant`, `--principal`. 43 new tests: 32 gateway (`test_query.py` — happy path 6, error mapping 4, retrieve 2, corpora 5, OpenAPI 3, middleware 5), 8 conformance (`test_corpus_store.py` — health / list / get / cross-tenant / register / unregister), 5 ragctl `query` CLI; 1553 pass workspace-wide. PolicyEngine coverage linter passes naturally (the egress-text consult inside `_generate_answer` keeps `llm.complete` covered). Cross-platform: pure-Python throughout; FastAPI 0.115+ + Uvicorn already in deps. Schemas regenerated (`Corpus.json`, `CorpusList.json`, `QueryRequest.json`, `QueryResponse.json`, `RetrieveRequest.json`, `RetrieveResponse.json`, `StageTimings.json`, `Answer.json`, `GatewayError.json`). New docs: [reference/gateway.md](docs/reference/gateway.md) (route catalogue + request/response shapes + error envelope + OpenAPI + auth + production wiring), [architecture/gateway.md](docs/architecture/gateway.md) (thin-gateway rationale + composition diagram + body-vs-header identity + two-chunk-field design + answer-generation policy + middleware + error mapping + reviewer checklist), [guides/curl-quickstart.md](docs/guides/curl-quickstart.md) (5-minute walkthrough from curl to gateway response). Deliberately deferred (matches scope confirmation before kickoff): SSE streaming (Step 3.2 gRPC streaming is the natural home), real `Auth` impl (Phase 6 governance), `rag.yaml`-driven production wiring (Step 3.5 corpus router), corpus CRUD beyond read (Step 3.10 admin UI), `POST /v1/ingest/document` improvements (Step 3.x as needed). | -| 3.2 | gRPC service | ✅ | `build/phase-3/step-3.2-grpc-service` | [#109](https://github.com/officialCodeWork/AgentContextOS/pull/109) | **gRPC surface for the gateway.** `proto/rag.proto` defines package `rag.gateway.v1` with `RagService` carrying server-streaming `Query` (emits `StageStarted` / `StageCompleted` / `AnswerDelta` / terminal `QueryResponse` on a `QueryStreamEvent` oneof) + unary `Retrieve` / `IngestDocument` / `ListCorpora` / `GetCorpus`; standard `grpc.health.v1.Health` + `grpc.reflection.v1alpha.ServerReflection` co-registered so `grpcurl` works without `.proto` files. Python stubs generated via `grpcio-tools` + `mypy-protobuf` (pure-Python install, cross-platform) into `apps/gateway/src/rag_gateway/_grpc_gen/`, committed for the drift gate; ruff + mypy exclude the generated dir. `task proto:lint` (buf, CI only — `bufbuild/buf-setup-action`) + `task proto:gen` + `task proto:check-drift` (runs on ubuntu-22.04 / macos-14 / windows-latest to catch OS-specific codegen drift). `GrpcRagService` reuses the same Phase 2 composition as the REST surface: accepts `understanding`/`retrieval_router`/`reranker`/`packer`/`llm`/`corpus_store`/`policy_engine`/`ingest_pipeline` injectables via `build_grpc_server(...)` (mirrors `build_app(...)`); defaults pull from the shared `build_default_*` helpers in `rag_gateway.app` so one process serves REST + gRPC with one wiring. Domain-error → gRPC StatusCode mapping mirrors REST: `ACLDeniedError` → `PERMISSION_DENIED`, `AuthError` → `UNAUTHENTICATED`, `RateLimitError` → `RESOURCE_EXHAUSTED`, `RetrievalError` → `UNAVAILABLE`, `ValueError` → `INVALID_ARGUMENT`, `GetCorpus` miss → `NOT_FOUND`, fall-through → `INTERNAL`; `GatewayError` envelope packed binary-proto on trailing metadata under `rag-gateway-error-bin` so SDKs recover the same `{code, message, request_id, trace_id, context}` shape REST returns as JSON. PolicyEngine `egress_text` consult preserved before every LLM call (deny → answer omitted, transform → substituted subject); OTel spans `grpc.query` / `grpc.retrieve` mirror REST attribute schema; structured-log events `grpc.query_complete` / `grpc.ingest_complete` / `grpc.answer.failed` / `grpc.answer.policy_denied`. Body-driven identity for `Query` / `Retrieve` / `IngestDocument` (tenant + principal on the request message); metadata-driven for `ListCorpora` / `GetCorpus` (`x-tenant-id` / `x-principal-id` / `traceparent` / `x-request-id` headers — W3C Trace Context parsed into per-request `TraceContext`). `ragctl grpc-query ` smoke command spins up an in-process `grpc.aio.server` on an ephemeral loopback port and prints the streamed events + terminal envelope; flags mirror `ragctl query` (`--top-k` / `--rerank` / `--pack` / `--generate` / `--corpus` / `--tenant` / `--principal` / `--show-events`). 48 new tests: 32 gateway (`apps/gateway/tests/test_grpc_query_gateway.py` — happy path 4, error mapping 5, retrieve/corpora 6, health/reflection 1, request-context propagation 3) + 11 conformance (`tests/contract/test_grpc_proto_compat.py` — proto ↔ Pydantic field parity for QueryRequest / QueryResponse / RetrieveRequest / RetrieveResponse / Answer / StageTimings / Corpus / CorpusList / GatewayError; enum parity for QueryShape / IngestStatus / TrustLevel; service contract method list + streaming semantics) + 5 ragctl `grpc-query` CLI; 1643 tests pass workspace-wide (up from 1553). Cross-platform: pure-Python codegen + grpc.aio asyncio server work on Windows / macOS / Linux without WSL; CI proto-drift job runs on the full matrix. Pre-existing mypy duplicate-module issue (test_query.py exists in both `apps/gateway/tests/` + `packages/ragctl/tests/`) resolved by excluding `*/tests/*` + `_grpc_gen` from mypy's module sweep; pre-existing schema drift on `dist/schemas/QueryResponse.json` regenerated. New docs: [reference/grpc.md](docs/reference/grpc.md), [architecture/grpc-service.md](docs/architecture/grpc-service.md), [guides/grpcurl-quickstart.md](docs/guides/grpcurl-quickstart.md), [ADR-0011](docs/adr/ADR-0011-grpc-wire-protocol.md); [docs/README.md](docs/README.md) updated with all four. Deliberately deferred (matches scope confirmation before kickoff): bidirectional streaming `Converse` RPC (Step 3.6 agent loop's natural home), token-by-token LLM streaming (wire shape `AnswerDelta` ready; needs backend support), gRPC-Web / Connect (Step 3.7 SDK work), TLS-by-default (deployments override `add_insecure_port`), `core.proto` reconciliation (ADR-0011 — separate step), chunked-upload `IngestDocument` (multi-MB payloads stay on REST multipart for now). | -| 3.3 | MCP server | ✅ | `build/phase-3/step-3.3-mcp-server` | [#111](https://github.com/officialCodeWork/AgentContextOS/pull/111) | **MCP surface for the gateway.** New `rag_gateway.mcp` subpackage (`apps/gateway/src/rag_gateway/mcp/`) builds a `FastMCP` server over the official `mcp` Python SDK (`mcp>=1.2`), transported over stdio, exposing three tools — `query` (end-to-end RAG: understand → route → optional rerank/pack/generate), `retrieve` (retrieval-only fused chunk refs + routing decision), `ingest` (single-document text ingest through the full pipeline). Tools are thin: build a per-call `RequestContext` from identity args (fresh `request_id`; `tenant_id` required → `AuthError` on empty), delegate to a `ToolBackend`, and project the rich result onto lean agent-facing models (`McpQueryResult`/`McpRetrieveResult`/`McpIngestResult` in `mcp/types.py`). **`ToolBackend` Protocol** (`mcp/backend.py`) is the topology seam: `InProcessBackend` composes the Phase 2 pipeline directly (same injectable deps + `build_default_*` helpers as `build_app`/`build_grpc_server`, reusing `rag_gateway.query`'s `_generate_answer`/`_chunks_to_citations`/`_Timings`), so REST + gRPC + MCP share one process wiring; a future `GatewayClientBackend` (HTTP client to a running gateway) slots in at the seam without touching the tool layer (per ADR-0012). `build_mcp_server(*, backend=None, …)` accepts the same composition kwargs as `build_app` or a ready `ToolBackend`; `serve(**kwargs)` runs the blocking stdio loop; `python -m rag_gateway.mcp` (`mcp/__main__.py`) is the entry point. Domain errors (`ACLDeniedError`/`AuthError`/`RateLimitError`/`RetrievalError`/`RagError`/`ValueError`) map to MCP `ToolError(f"{code}: {message}")`, mirroring the REST `GatewayError` body + gRPC trailing-metadata envelope. PolicyEngine `egress_text` consult preserved before every LLM call (governed inside the shared `_generate_answer`). Structured-log events `mcp.server_built` / `mcp.server_listening` / `mcp.query_complete` / `mcp.ingest_complete`. **`@ragplatform/mcp` npm launcher** (`apps/gateway/mcp-launcher/`) — a thin pure-Node `bin` that shells out to `python -m rag_gateway.mcp` with inherited stdio (one implementation, in Python; not a second TS server); cross-platform interpreter discovery (`$RAGPLATFORM_MCP_PYTHON` → `python3`/`python`/`py`), `spawn(shell:false)` (no injection surface), `ENOENT` fallback chain, signal forwarding, exit-code mirroring. `ragctl mcp-query ` (in-memory client/server session — no stdio process; flags mirror `query`/`grpc-query`) + `ragctl mcp-serve` smoke commands. Tool args kept field-for-field in sync with `QueryRequest`/`RetrieveRequest` minus deferred `filters`/`request_id`, enforced by conformance gate. 21 new tests: 13 gateway (`apps/gateway/tests/test_mcp_server.py` — tool discovery/schemas, three happy paths, error mapping, tenant propagation, answer+citation projection, corpus_ids) + 4 conformance (`tests/contract/test_mcp_tool_compat.py` — tool↔request-model field parity + tool set) + 4 ragctl (`packages/ragctl/tests/test_mcp.py`); 1705 pass workspace-wide. `mypy --strict` clean; RAG001 + log-schema gates pass. Cross-platform: pure-Python server + pure-Node launcher, no infra. New docs: [reference/mcp.md](docs/reference/mcp.md), [architecture/mcp-server.md](docs/architecture/mcp-server.md), [guides/mcp-quickstart.md](docs/guides/mcp-quickstart.md), [ADR-0012](docs/adr/ADR-0012-mcp-server-topology.md), [mcp-launcher/README.md](apps/gateway/mcp-launcher/README.md); [docs/README.md](docs/README.md) updated with all four. Deliberately deferred: `GatewayClientBackend` (seam ready, impl not built), `filters` tool arg, real auth (Phase 6; `principal_id` trusted for now), MCP resources/prompts (tools only), token-by-token streaming, binary/multi-format ingest (REST multipart for now). | -| 3.4 | OpenAI-compatible endpoints | ✅ | `build/phase-3/step-3.4-openai-compatible-endpoints` | [#113](https://github.com/officialCodeWork/AgentContextOS/pull/113) | **OpenAI-compatible surface for drop-in client adoption.** New `rag_gateway.openai_compat` (`apps/gateway/src/rag_gateway/openai_compat.py`) ships `POST /v1/embeddings`, `POST /v1/chat/completions`, and `GET /v1/models` matching the OpenAI REST dialect, so the official OpenAI SDKs / LangChain `ChatOpenAI` / LlamaIndex / curl point a `base_url` at the gateway and work unchanged. **Retrieval pre-fetch** is the value-add: chat runs the Phase 2 pipeline (`understand → route → optional rerank → optional pack`) over the conversation's last `user` turn (or `rag.query` override), runs the `PolicyEngine.egress_text` consult over the retrieved chunks, and injects them as a single numbered-context **system** message inserted immediately before the last user turn (preserving the caller's own system prompt + history) before calling `LLM.complete`. Retrieval is on by default (`rag.enabled=true`); `rag.enabled=false` is a plain LLM proxy. The non-standard `rag` request field (`RagOptions`) mirrors `QueryRequest`'s retrieval knobs — `corpus_ids`/`top_k`/`rerank`/`pack`/`pack_budget` (+ `query` override) — with matching defaults, pinned by a CI conformance gate (`tests/contract/test_openai_compat.py`); citations + query-shape + bm25-fallback come back under the `rag` response field (`RagAnnotations`). `stream=true` returns an SSE `StreamingResponse` of OpenAI-faithful `chat.completion.chunk` events (role-only opening chunk → per-token content deltas → terminal chunk carrying `finish_reason`/`usage`/`rag` → `data: [DONE]`); pre-fetch + egress policy run synchronously before any byte streams (errors convert to the OpenAI envelope first), and a mid-stream LLM failure emits a `finish_reason:"error"` chunk rather than tearing the connection. Embeddings delegate to a new top-level `Embedder` injectable on `app.state.embedder` (`build_default_embedder()` → `NoopEmbedder(dim=8)`), accept `str`/`list[str]` input (token-id arrays → 400), and support `encoding_format:"base64"` (little-endian float32 via `struct.pack`); a `PolicyEngine.quota_check` consult precedes the embed (deny → 429). `/v1/models` advertises the wired LLM + embedder ids plus a `rag-gateway` alias. Identity is header-driven (`Authorization: Bearer` → `Principal` via `Auth` SPI; `X-Tenant-Id` → tenant) like `/v1/corpora`, with a **dev-only anonymous fallback gated on the wired `Auth` being `NoopAuth`** (so the demo is curl-able creds-free like `/v1/query`, while a real `Auth` backend rejects unauthenticated requests with 401); `app.state.default_tenant_id` stores the effective default tenant for the fallback. Errors render the OpenAI `{"error":{message,type,param,code}}` envelope (distinct from the native `GatewayError`), mapped via the shared `_http_status_for` + a status→`error.type` table (`authentication_error`/`permission_error`/`rate_limit_error`/`invalid_request_error`/`api_error`). Both governed call sites (`LLM.complete`/`stream`, `Embedder.bulk_embed`) sit beside their PDP consult so the PolicyEngine coverage linter passes with no allowlist entry. OTel spans `gateway.openai.embeddings` / `gateway.openai.chat`; structured-log events `openai.embeddings_complete` / `openai.chat_complete` / `openai.chat.policy_denied` / `openai.chat.stream_failed`. New `rag-core` 0.20 → 0.21 wire types in `rag_core.openai_types`: `EmbeddingsRequest`, `EmbeddingObject`, `EmbeddingsUsage`, `EmbeddingsResponse`, `ChatMessage`, `RagOptions`, `ChatCompletionRequest`, `ChatCompletionResponseMessage`, `ChatCompletionChoice`, `ChatCompletionUsage`, `RagAnnotations`, `ChatCompletionResponse`, `ChatCompletionDelta`, `ChatCompletionChunkChoice`, `ChatCompletionChunk`, `ModelObject`, `ModelList`, `OpenAIErrorDetail`, `OpenAIError` (exported from `rag_core` + `gen_schemas`). `ragctl chat ` (`--rag/--no-rag`, `--stream`, `--corpus`/`--top-k`/`--rerank`/`--pack`/`--tenant`) + `ragctl embeddings ` (`--encoding-format`) smoke commands drive the in-process app via `TestClient`. Token usage best-effort via the packer's `TiktokenCounter` (whitespace fallback) for embeddings + streaming; non-streaming chat reports the LLM's own counts. 36 new tests: 18 gateway (`apps/gateway/tests/test_openai_compat.py` — embeddings 6, chat non-stream 10, streaming 2 + models/auth/openapi), 6 conformance (`tests/contract/test_openai_compat.py` — `RagOptions`↔`QueryRequest` knob + default parity, OpenAI wire-shape discriminators), 6 ragctl (`packages/ragctl/tests/test_openai.py`). `mypy --strict` clean; RAG001 + log-schema + policy-coverage gates pass; schemas regenerated (`EmbeddingsRequest/Response.json`, `ChatCompletionRequest/Response/Chunk.json`, `ModelList.json`, `OpenAIError.json`). Cross-platform: pure-Python; FastAPI `StreamingResponse` for SSE. gateway 0.6 → 0.7 (FastAPI app `0.5.0`). New docs: [reference/openai-compat.md](docs/reference/openai-compat.md), [architecture/openai-compat.md](docs/architecture/openai-compat.md), [guides/openai-quickstart.md](docs/guides/openai-quickstart.md), [ADR-0013](docs/adr/ADR-0013-openai-compatibility.md); [docs/README.md](docs/README.md) updated with all four. Deliberately deferred: multi-part message content + tool/function calling (string content only), `n>1`/`logprobs`/`top_p` enforcement (accepted-and-ignored or rejected), real per-provider token accounting, `dimensions` enforcement (wired embedder controls dimension), remote `GatewayClientBackend`-style topology for these routes (in-process like the rest of the gateway). | -| 3.5 | Corpus router | ✅ | `build/phase-3/step-3.5-corpus-router` | [#115](https://github.com/officialCodeWork/AgentContextOS/pull/115) | **Per-query corpus selection + federated fan-out.** New `CorpusRouter` (`packages/retrieval/src/rag_retrieval/corpus_router.py`) sits *above* the Step 2.10 `RetrievalRouter`: it answers *which corpora a query should touch* (the router below answers *which backends* for one corpus). `decide(ctx, *, text, corpus_ids=None)` selects corpora in fixed precedence — **explicit** caller pin (intersected with tenant visibility) → **static rules** (config-driven keyword/regex boost/exclude, exclude wins over boost) → **learned** classifier (per-corpus term-weight softmax) → **all** visible — and auto-detects the *effective* strategy from what actually fired (a rule beats a learned signal beats `all`). When a tenant has no visible corpora the router emits `UNCONSTRAINED` and runs a single `corpus_ids=None` retrieval, preserving pre-3.5 behaviour (so it is a drop-in over the noop demo); `UNCONSTRAINED` is runtime-only and intentionally absent from the config enum. `route(...)` runs retrieval across the selection: with `fan_out=true` (default) one `RetrievalRouter.route` per corpus, fused via Step 2.5 `rrf_fuse` weighted by corpus relevance score (floored at `_SIGNAL_EPSILON` so a selected-but-zero corpus still contributes); with `fan_out=false` a single retrieval scoped to all selected corpora. `StaticRuleClassifier` (`CorpusRule`: `corpus_ids`/`match_any`/`match_regex`/`boost`/`exclude`/`name`) is deterministic + auditable; `LearnedCorpusClassifier` is a **dependency-free** multinomial-logistic scorer (offline term-weight profiles read off each corpus's `metadata["term_weights"]` × query bag-of-words → temperature-scaled softmax; uniform softmax = "no signal" → fall through) — genuinely learned in shape, pure-Python, no model runtime; pluggable behind the runtime-checkable `CorpusClassifier` Protocol. `CorpusRouterConfig` knobs (`min_corpora`/`max_corpora`/`score_threshold`/`fan_out`/`rrf_k`) validated at construction. New `rag-core` wire types in `rag_core.types`: `CorpusRoutingStrategy` StrEnum (EXPLICIT/STATIC_RULES/LEARNED/ALL/UNCONSTRAINED), `CorpusScore` + `CorpusRoutingDecision` (both frozen) + `CorpusRouterError(RetrievalError)`; `corpus_decision` surfaced on `QueryResponse`/`RetrieveResponse` (`rag_core.gateway_types`). Postgres-backed `PgCorpusStore` (`packages/backends/src/rag_backends/corpus/pg_corpus_store.py`) persists corpora + `routing_profile` JSONB (term_weights), imported lazily so the default gateway install needs no DB drivers. `rag-config` adds the `backends.corpus_store` block (`provider: none|in_memory|postgres`), the `corpora` registry (`CorpusDefn`, with `term_weights`), and `retrieval.routing` (`CorpusRoutingConfig` — `strategy`/`rules`/`min_corpora`/`max_corpora`/`score_threshold`/`fan_out`/`rrf_k`/`learned_temperature`; `max_corpora >= min_corpora` validator; duplicate-corpus-key rejection); config `CorpusRoutingStrategy` omits `unconstrained` (runtime-only). Config-driven wiring in `apps/gateway/src/rag_gateway/wiring.py`: `build_corpus_store_from_config` (none/in_memory → `NoopCorpusStore` seeded from `cfg.corpora`; postgres → lazy `PgCorpusStore`, requires `config.dsn`), `seed_corpus_store` (async upsert; no-op for in_memory), `build_corpus_router_from_config` (`static_rules` → rule classifier + learned fallback; `learned` → learned only; `all`/`explicit` → neither), `build_app_from_config` (boots the full app with config-driven store + router, forwarding remaining deps to `build_app` defaults — drop-in over bare `build_app()`); kept in the gateway (not `rag-config`) so the `config → core` boundary holds (rag-config never imports a backend). `ragctl corpus list`/`seed`/`route` (`-f rag.yaml`, `-t tenant`, `route` adds positional query + repeatable `-c corpus-id` → EXPLICIT) drive it config-only — `route` prints `configured` vs `effective` strategy, fan_out, candidate/selected counts, per-corpus scores + reasons; no DB needed with in_memory. PolicyEngine boundary: `corpus_router.py` + `pg_corpus_store.py` make **no** governed SPI calls (CorpusStore is a read-only tenant-scoped registry; all retrieval delegates to `RetrievalRouter` → `HybridRetriever`, the canonical `read_chunk` PDP site) → coverage linter passes with no allowlist entry (same posture as `router.py`). Trace + audit: canonical `corpus.route` OTel span (`rag.corpus_route.{strategy,candidate_n,selected_n,fan_out,selected,results_n,elapsed_ms}`) + `corpus.route_decision` structured-log event + synchronous `corpus.route` audit event (`AuditWriter.write`, `outcome=allowed`, resource = selected ids or `*`). 35 new tests: 17 router (`tests/retrieval/test_corpus_router.py` — 5 strategies, classifiers in isolation, config validation, fan-out fusion, single-pass scoping, unconstrained, top_k<=0, audit event), 6 wiring (`apps/gateway/tests/test_wiring.py` — per-strategy classifier wiring, postgres-without-dsn raises, `/v1/query` surfaces `corpus_decision`), 6 ragctl (`packages/ragctl/tests/test_corpus.py`), 4 config (`tests/config/test_schema.py` — routing block parse, max= max_steps` → `max_iterations`) so a budget exhausted by step N is observed before N+1 works. `max_steps` (config ceiling, defence-in-depth) and `budget_iter` (operator per-request cap) are deliberately distinct, each with its own `StopReason`. **Controller** (`controller.py`, `decide(ctx, *, snapshot, tools) -> CallTool | Finish`): `ScriptedController` (deterministic test backbone), `HeuristicController` (credential-free default — retrieve once for the goal, then finalize; makes the surface curl-able with zero creds), `LLMController` (production — JSON-action prompt, code-fence-tolerant parse degrading to `Finish(raw_text)`). **Tools** (`tools.py`): `Tool`/`ToolRegistry`/`ToolSpec`; built-in `RetrieveTool` (`default_top_k=8`, `max_top_k=50`) wraps a structural `Retriever` Protocol — a backend `RetrievalError` is **recoverable** (`ok=False` so the controller can retry/finalize), only non-recoverable faults raise `AgentLoopError → failed`. **Resumability**: whole run state is one frozen serialisable `AgentSnapshot` (`state.py`) checkpointed every step via `CheckpointStore`/`InMemoryCheckpointStore` (`checkpoint.py`, last-write-wins per `run_id`, keyed `(tenant_id, run_id)` for tenant isolation); resume = "load latest snapshot, keep stepping". Budget stored as **raw nullable caps** (not nested `Budget`) so a spent (zero) budget round-trips on resume without re-tripping the G-03 zero-cap validator — rebuilt via `Budget.model_construct`. New `rag-core` wire types in `rag_core.agent_types` (`AgentRequest`/`AgentEvent`/`AgentRunResult`/`AgentStep`/`AgentToolCall`/`AgentToolResult`, `AgentPhase`/`AgentEventKind` StrEnums) + `StopReason.final_answer` + `AgentLoopError`; `AgentEvent` is a single **flat** model (kind discriminator + optional payload fields), not a discriminated union. **Surfaces**: REST `POST /v1/agent` (`apps/gateway/src/rag_gateway/agent.py`, FastAPI `StreamingResponse`, one `AgentEvent` per `data:` line, `data: [DONE]` sentinel; `make_agent_router` + `build_default_agent_loop` + `_GatewayRetriever` adapting the policed understanding→router path) and gRPC `RagService/Converse` (`proto/rag.proto` adds `Converse(ConverseRequest) returns (stream ConverseStreamEvent)` + `AgentPhase`/`AgentEventKind`/`StopReason` enums + `BudgetSpend`/`AgentToolCall`/`AgentToolResult`/`AgentStep`/`ConverseRequest`/`AgentRunResult`/`ConverseStreamEvent` messages; `grpc_service.py` `Converse` handler + encoders, `grpc_server.py` wiring). Both surfaces serialise the same flat event frame-for-frame and reuse the **same egress helpers** (`_apply_egress_policy`/`_redact_result`/`_failed_event`). **Governance**: the loop makes **no** governed SPI calls — retrieval governed inside the wired `Retriever` (gateway's `HybridRetriever` `read_chunk` PDP site, same as `/v1/query`); final-answer egress runs `PolicyEngine.egress_text` **once** at the boundary (before the `answer_delta` frame, cached + replayed onto `run_completed`) so denied text never hits the wire; `controller.py` allowlisted in `tests/policy/coverage.py` (LLMController calls `llm.complete`), `loop.py` passes naturally. Resume-of-unknown-run surfaced **in-band** as a `run_failed` frame on both surfaces, not a transport abort. `ragctl agent ""` (`--tenant`/`--principal`/`--corpus`/`--top-k`/`--max-steps`/`--budget-tokens`/`--budget-iter`) drives the SSE stream in-process via `TestClient`. Observability: `agent.run` span (`rag.agent.{run_id,max_steps,steps,stop_reason,final_chunks_n,budget_tokens_used}`) + `grpc.converse` span; structured-log events `agent.step_complete`/`agent.run_failed`/`gateway.agent_complete`/`grpc.converse_complete`/`gateway.agent.policy_denied`. Tests: full `rag-agent` unit suite (state/loop/tools/controller) + 8 gRPC Converse gateway tests (`apps/gateway/tests/test_grpc_converse_gateway.py`) + extended proto-compat conformance (`tests/contract/test_grpc_proto_compat.py` — 7 new `_PAIRS`, StopReason/AgentPhase/AgentEventKind enum-coverage, `Converse` in service contract, server-streaming assertion). `mypy --strict` clean; RAG001 + log-schema + policy-coverage + SPI-signature + proto-drift + schema-drift gates pass; regenerated `_grpc_gen` stubs + `Agent*.json` schemas committed. Cross-platform: pure-Python throughout. New docs: [reference/agent.md](docs/reference/agent.md), [architecture/agent-loop.md](docs/architecture/agent-loop.md), [guides/agent-quickstart.md](docs/guides/agent-quickstart.md), [ADR-0015](docs/adr/ADR-0015-agent-runtime.md); [reference/grpc.md](docs/reference/grpc.md) + [docs/README.md](docs/README.md) updated. Deliberately deferred: durable (Postgres/Redis) checkpoint store, measured token/dollar accounting (flat estimates for now), more tools (web fetch/calculator/sub-agents behind the `Tool` seam), MCP / OpenAI agent-mode surfaces, bidirectional-streaming `Converse` (per ADR-0011). | -| 3.7 | Official SDKs | ✅ | `build/phase-3/step-3.7-official-sdks` | [#121](https://github.com/officialCodeWork/AgentContextOS/pull/121) | **Five-language client SDKs from two contracts.** Committed, drift-gated OpenAPI 3.1 spec (`dist/openapi.json` + `.yaml`) exported from the live FastAPI app by `scripts/export_openapi.py`; `scripts/check_openapi_drift.py` + CI job `openapi-drift` (mirrors `schema-drift` / `proto-drift`). Hand-written, fully-tested flagships: **Python** `agentcontextos` (`sdks/python`, sync `Client` + async `AsyncClient`, all REST routes incl. SSE `stream_chat`/`stream_agent`, reuses `rag-core` wire models, `ApiError` hierarchy, identity body-vs-header resolution; 32 tests vs a live in-process uvicorn gateway; mypy-strict) and **TypeScript** `@agentcontextos/sdk` (`sdks/typescript`, zero-runtime-dep `fetch`/`ReadableStream`, Node 18+ & browser, async-iterator streaming; 10 vitest tests + `tsc` typecheck). Generated on demand: **Go / Java / .NET** (`sdks/{go,java,dotnet}`) via `task sdk:gen` (`scripts/gen_sdks.py` → openapi-generator-cli REST clients + `buf generate` gRPC stubs; generator pinned in `openapitools.json`); generated source is a gitignored build artifact, only configs + scaffold READMEs committed. `buf.gen.yaml` repurposed to emit Go/Java/.NET proto stubs only (Python stays on `grpc_tools.protoc` so the existing committed-stub drift gate stays canonical). New Task/Make targets (`openapi:gen`/`check-drift`, `sdk:gen`/`gen-proto`/`gen-openapi`); CI `ts-sdk` job (typecheck+test+build) + `openapi-drift` job; `sdks/python` added to uv workspace + pytest + mypy, `sdks/typescript` to pnpm workspace. Gateway app version synced 0.5.0 → 0.7.0. [ADR-0016](docs/adr/ADR-0016-sdk-generation.md), [reference/sdks.md](docs/reference/sdks.md), [architecture/sdk-generation.md](docs/architecture/sdk-generation.md), [guides/sdk-quickstart.md](docs/guides/sdk-quickstart.md). | -| 3.8 | Framework adapters | ✅ | `build/phase-3/step-3.8-framework-adapters` | [#122](https://github.com/officialCodeWork/AgentContextOS/pull/122) | **Eight agent/RAG framework integrations wrapping the SDK `Client`.** New `agentcontextos.integrations` subpackage (in `sdks/python`): **LangChain** `AgentContextOSRetriever(BaseRetriever)` (sync `_get_relevant_documents` + async `_aget_relevant_documents`), **LlamaIndex** `AgentContextOSRetriever(BaseRetriever)` (`_retrieve`/`_aretrieve` → `NodeWithScore`), **Haystack** `@component` `AgentContextOSRetriever` (`run(query)` → `documents`), **DSPy** `AgentContextOSRM` (callable RM whose passages expose `.long_text` + `forward()` → `dspy.Prediction`), **LangGraph** `make_retrieval_tool` (`StructuredTool`) + `make_retrieval_node` (StateGraph node), **CrewAI** `AgentContextOSSearchTool(BaseTool)`, **AutoGen** `make_search_function` (plain callable) + `make_function_tool` (`autogen_core.FunctionTool`), **Semantic Kernel** `AgentContextOSPlugin` (`@kernel_function search`). Shared `integrations/_common.py` resolves an injected-or-built client, runs `/v1/query` with `pack=False`/`generate=False`, and maps `Chunk → (text, score, canonical metadata)` (`chunk_id`/`document_id`/`tenant_id`/`corpus_id`/`score`/`trust_level`) — retriever adapters return framework docs, tool adapters return a numbered `format_passages` string. Each submodule lazy-imports its framework behind a per-framework optional extra (`langchain`/`llama-index`/`haystack`/`dspy`/`langgraph`/`crewai`/`autogen`/`semantic-kernel`) with a friendly `missing_dependency` error; importing `agentcontextos.integrations` pulls in no framework. 13 tests (all 8 adapters verified locally against real framework base classes via a fake SDK client + `importorskip`; `format_passages` / AutoGen search-fn / `missing_dependency` covered framework-free); dedicated `integrations` CI job installs the lighter six extras and runs them. mypy overrides added for the eight framework module roots; CI mypy scope unchanged (modules subclass `Any` when extras absent). [ADR-0017](docs/adr/ADR-0017-framework-adapters.md), [reference/integrations.md](docs/reference/integrations.md), [architecture/framework-adapters.md](docs/architecture/framework-adapters.md), [guides/integrations-quickstart.md](docs/guides/integrations-quickstart.md). | -| 3.9 | Webhooks | ✅ | `build/phase-3/step-3.9-webhooks` | [#123](https://github.com/officialCodeWork/AgentContextOS/pull/123) | **Signed outbound webhook delivery.** New `rag-webhooks` package (depends only on `rag-core` + `httpx` + `opentelemetry-api`): `WebhookDispatcher` (tenant-scoped fan-out + per-subscription HMAC signing + exponential-backoff retries + `webhook.publish` OTel span + `webhook.delivery` log; `publish` / `deliver_one` / fire-and-forget `schedule`), `HttpWebhookSender` (httpx, captures transport errors instead of raising), `RetryPolicy`, `signing.py` (HMAC-SHA256 over `.` → `X-AgentContextOS-Signature: t=,v1=` + `verify()` with replay tolerance + `build_headers`), `events.py` payload builders. Wire types + SPIs in `rag-core` (0.23→0.24): `webhook_types.py` (`WebhookEvent` / `WebhookSubscription` (`wants()`/`redacted()`) / `WebhookDelivery` / `WebhookDeliveryAttempt` / `WebhookSendOutcome` / `DeliveryStatus` / `WebhookEventType` + `CreateSubscriptionRequest` / `SubscriptionList` / `WebhookTestResult` + structural `WebhookPublisher` Protocol), `SubscriptionStore` + `WebhookSender` SPIs (`list_all`/`list_for_event`/CRUD; ctx-first) + `NoopSubscriptionStore` (tenant-scoped in-memory) / `NoopWebhookSender` (recording, `fail_times`). Emission via the `WebhookPublisher` seam (no emitter depends on rag-webhooks): gateway ingest route `schedule`s `ingest.completed` (only on `IngestStatus.ingested`); `PolicyWriter` emits `audit.policy_violation` on `is_deny()` when a publisher is wired (fire-and-forget, never breaks the decision). `drift.detected` / `eval.regression` types defined; emitters deferred to Phase 5. Gateway: `make_webhooks_router()` (`POST`/`GET`/`GET {id}`/`DELETE {id}` + `POST {id}/test`), `build_app(subscription_store=, webhook_dispatcher=)` injectables (default in-memory store + HTTP dispatcher), secret returned once then `whsec_••••` masked, tenant-isolated 404s. Config: `WebhooksConfig` + `WebhookSubscriptionDefn` in `rag.yaml` (rag-config 0.2→0.3); `dist/rag.schema.*`, `dist/schemas/`, `dist/openapi.*` regenerated. `ragctl webhooks demo` (in-process sign+deliver, shows signature + retries). ~50 tests: 11 SPI contract + 23 rag-webhooks unit (signing/retry/sender via httpx MockTransport/dispatcher) + 8 gateway + 3 PolicyWriter-emission + 4 ragctl. [ADR-0018](docs/adr/ADR-0018-outbound-webhooks.md), [reference/webhooks.md](docs/reference/webhooks.md), [architecture/webhooks.md](docs/architecture/webhooks.md), [guides/webhooks-quickstart.md](docs/guides/webhooks-quickstart.md). | -| 3.10 | Admin UI v0 | ✅ | `build/phase-3/step-3.10-admin-ui` | [#124](https://github.com/officialCodeWork/AgentContextOS/pull/124) | **Next.js 14 operator console (`apps/admin-ui`).** App Router + TypeScript + Tailwind, built to a committed high-fidelity design handoff (`apps/admin-ui/design/`, shadcn/ui target). **Hand-rolled shadcn-style primitives** (Radix-free, no CLI) in `components/ui/` (primitives/overlays/toaster) against the handoff's shadcn CSS-variable tokens (light+dark in `globals.css`) + Geist fonts (`geist` pkg) + lucide-react — deterministic offline builds. App shell: collapsible grouped sidebar + sticky top bar (tenant switcher with env badge, cosmetic search, review-state toggle, theme toggle via `next-themes`, principal menu). **9 pages**: Dashboard (stat cards + recent activity + connector health), Corpora (table + detail sheet + new/delete dialogs), Connectors (card grid + multi-step add wizard + detail sheet w/ sync history), Glossary (two-pane chip editor), Webhooks (table + create + **one-time secret reveal** + send-test w/ signed headers + attempts), Audit Log (action/outcome/principal/text filters + pagination + metadata drawer), API Keys (Preview), Tenants (Preview), Config Viewer (read-only `rag.yaml` section nav + YAML highlight). **Live-vs-seed hybrid** (`lib/use-live.ts` + `lib/api.ts`): Corpora + Webhooks fetch the gateway when `NEXT_PUBLIC_GATEWAY_URL` set (header identity `X-Tenant-Id`), else seed data with a "Demo data" badge; others seed-only. Global providers (`tenant`/`sidebarCollapsed`/`viewState`/toasts). `DataPanel` renders skeleton/empty/error/ready. 14 vitest+RTL tests; `tsc --noEmit` + `next lint` + `next build` (static export) clean; CI `admin-ui` job (typecheck+lint+test+build); `apps/**`→`apps/gateway/**` Python path filter narrowed; pnpm workspace. Visual fidelity confirmed vs handoff screenshots. [ADR-0019](docs/adr/ADR-0019-admin-ui.md), [reference/admin-ui.md](docs/reference/admin-ui.md), [architecture/admin-ui.md](docs/architecture/admin-ui.md), [guides/admin-ui-quickstart.md](docs/guides/admin-ui-quickstart.md). | -| 3.11 | Status & Metrics GUI (MVP) | ✅ | `build/phase-3/step-3.11-status-metrics-gui` | [#125](https://github.com/officialCodeWork/AgentContextOS/pull/125) | **Operator live-status surface + console pages.** New `rag-observability` read-side: `MetricsCollector` (thread-safe in-memory counter/gauge/histogram — exact `count`/`sum`/`min`/`max` + bounded-reservoir `p50`/`p95`/`p99`, JSON-ready `snapshot()`; complements OTel's push-only instruments) and `LogTail` + `RingBufferLogHandler` (bounded ring buffer a logging handler mirrors `rag.*` records into; streamers poll by monotonic `seq` cursor to avoid cross-thread asyncio). New gateway surface `rag_gateway.status` (`make_status_router`): REST `GET /v1/status/health` (overall + per-component roll-up — `gateway` / wired-SPI `component` / per-route-error-rate `surface` health, no synthetic probes), `/v1/status/metrics` (snapshot + process info), `/v1/status/logs` (level/event/tenant filters), `/v1/connectors/status` (injectable in-memory `ConnectorStatusStore`, empty by default → console seed-fallback); **SSE** `GET /v1/status/logs/stream` (backlog replay + live tail, reusing the chat/agent `data:` framing); **WebSocket** `/v1/status/ws` (health+metrics snapshot push, client-tunable `interval_ms`/`channels`). Request-timing middleware records `gateway.requests_total` / `request_duration_ms` / `requests_in_flight` / `errors_total` (status surface excluded so the long-lived SSE stream can't skew the histogram/in-flight gauge); CORS installed for the console origin (`RAG_GATEWAY_CORS_ORIGINS`, default `:3100`). `build_app` gains injectable `metrics_collector` / `log_tail` / `connector_status_store` + `enable_cors`; per-app collector (test isolation) + process-global tail (idempotent re-pointing `attach_log_tail`); `rag-gateway` 0.7.0 → 0.8.0 (now declares the `rag-observability` dep, aligning the graph). Admin console (`apps/admin-ui`) gains an **Observability** nav group + 3 pages — **Live Status** (`/status`, WS), **Metrics** (`/metrics`, WS), **Logs** (`/logs`, SSE) — plus a Dashboard system-status banner for ≤3-click drilldown; new `useStatusStream` (WS) / `useLogStream` (SSE) hooks + REST fetchers (`fetchHealth`/`fetchMetrics`/`fetchLogs`/`fetchConnectorStatus`), a dependency-free SVG `Sparkline`/`Bar`, a `SourceBadge` (Demo data ↔ Live), all following the `useLive` seed-fallback hybrid (stream hooks no-op to seed without `NEXT_PUBLIC_GATEWAY_URL`, so jsdom tests never touch WS/EventSource; connectors use `fallbackOnEmpty`). PolicyEngine boundary: `status.py` makes **no** governed SPI calls (process state + post-policy log text) → coverage linter passes with no allowlist entry. 45 new tests (15 gateway status REST/SSE-generator/WS/middleware/CORS/connectors + 23 observability metrics/logbuffer + 7 admin-ui vitest status/metrics/logs); `mypy --strict` (242 files) + ruff + RAG001 (logbuffer + 2 test files allowlisted) green; `tsc` / `next lint` / `next build` (15 routes) + `vitest` (21) green; browser-verified all 3 pages + the Dashboard banner. New docs: [reference/status-api.md](docs/reference/status-api.md), [architecture/status-metrics-gui.md](docs/architecture/status-metrics-gui.md), [guides/operator-console-live.md](docs/guides/operator-console-live.md), [ADR-0020](docs/adr/ADR-0020-status-transports.md); updated [reference/rag-observability.md](docs/reference/rag-observability.md) + [reference/admin-ui.md](docs/reference/admin-ui.md) + [docs/README.md](docs/README.md). Deliberately deferred: Prometheus/OpenMetrics exposition, push (non-poll) log streaming + `Last-Event-ID` resume, deep per-backend liveness probes (Phase 4 reliability), status-surface auth, a durable connector-status runtime, and the full 12-page GUI build (Step 5.6). | +### 0.2c — Cross-platform tooling ✅ [#6](https://github.com/officialCodeWork/AgentContextOS/pull/6) ---- +- `Taskfile.yml` — Windows / macOS / Linux task runner +- `scripts/bootstrap.ps1` + `bootstrap.sh`; CI matrix expanded to `windows-latest` -## Phase 4 — Reliability (Weeks 18–22) +### 0.3 — Plugin SPI ✅ [#7](https://github.com/officialCodeWork/AgentContextOS/pull/7) -**Phase complete.** 6 of 6 steps complete (4.1 ✅, 4.2 ✅, 4.3 ✅, 4.4 ✅, 4.5 ✅, 4.6 ✅). +- 16 SPI ABCs in `rag_core/spi/`: VectorStore, KeywordStore, GraphStore, Embedder, Reranker, LLM, Cache, Queue, Storage, Secrets, Auth, Telemetry, Connector, Parser, OCR, PIIDetector +- Noop in-memory impls in `spi/noop/`; **79 conformance tests** in `tests/contract/` -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 4.1 | Semantic cache (L0/L1/L2) | ✅ | `build/phase-4/step-4.1-semantic-cache` | [#126](https://github.com/officialCodeWork/AgentContextOS/pull/126) | New `rag-cache` pkg: `TtlLruCache` + in-memory L0 (`InMemory{Retrieval,Answer,Embedding}Cache`), L2 similarity (`Semantic{Retrieval,Answer}Cache` / `QuerySimilarityIndex`), `Tiered{Retrieval,Answer}Cache` (read-through+promotion, write-through, `cache.*` events + `TierStats`); L1 `Redis{Retrieval,Answer}Cache` in `rag-backends` (tag-set invalidation), injected. Reliability: L1 failures degrade never raise. Gateway `/v1/{query,retrieve}` consult the cache (`served_from_cache`); `CacheConfig` tiered knobs; cross-tenant red-team suite; [ADR-0021](docs/adr/ADR-0021-tiered-cache.md) | -| 4.2 | Fallback chain | ✅ | `build/phase-4/step-4.2-fallback-chain` | [#127](https://github.com/officialCodeWork/AgentContextOS/pull/127) | `rag_retrieval.fallback.FallbackChain` wrapping `RetrievalRouter`: degradation ladder hybrid → BM25-only → keyword → "no answer"; triggers = proactive budget (CostEstimator start-tier, ADR-0008) + reactive `RetrievalError` / empty-below-`min_results`; graceful `no_answer` (HTTP 200, toggle `no_answer_on_exhaustion`); keyword rung relaxes caller filters only (ACL pushdown preserved); `SupportsRoute` drop-in (CorpusRouter relaxed) + `build_app` default wrap; `FallbackTier`/`FallbackTrigger`/`FallbackResult` core types; `cfg.retrieval.fallback`; `retrieve.fallback` span + `fallback.engaged` event; `ragctl fallback`; 39 tests; [ADR-0008](docs/adr/ADR-0008-cost-aware-planner.md), [fallback-chain.md](docs/architecture/fallback-chain.md) | -| 4.3 | Hallucination guard | ✅ | `build/phase-4/step-4.3-hallucination-guard` | [#128](https://github.com/officialCodeWork/AgentContextOS/pull/128) | New core-only `rag-guard` pkg: `HallucinationGuard` (claim split → per-claim NLI vs. evidence → per-tenant threshold → annotate/redact/block); `NLIScorer` SPI (mirrors `Reranker`) + `NoopNLIScorer` + dependency-free `LexicalNLIScorer` default; frozen core types `NLILabel`/`NLIScore`/`GuardVerdict`/`GuardAction`/`ClaimVerdict`/`GuardResult` + `GuardError`. Runs after `egress_text` on `/v1/query` + OpenAI chat (sync block ⇒ `content_filter`; stream advisory) + MCP `query` (agent deferred); `Answer.guard` + `RagAnnotations.guard`. Disabled by default; reliability degrade-never-raise (`guard.scorer_degraded`). `guard.check` span + PII-safe `guard.claim_blocked` event (`GuardEvent`); `cfg.guard`; `ragctl guard`; [ADR-0022](docs/adr/ADR-0022-hallucination-guard.md), [reference/guard.md](docs/reference/guard.md), [architecture/hallucination-guard.md](docs/architecture/hallucination-guard.md) | -| 4.4 | Circuit breakers | ✅ | `build/phase-4/step-4.4-circuit-breakers` | [#129](https://github.com/officialCodeWork/AgentContextOS/pull/129) | New core-only `rag-breaker` pkg: `CircuitBreaker` (3-state closed/open/half-open, consecutive-failure trip + timed half-open probe, injected clock), `BreakerRegistry` (per-backend, force-close), `Breaker{Vector,Keyword,Graph}RetrievalBackend` SPI wrappers — open breaker raises `CircuitOpenError` which `HybridRetriever`'s `gather(return_exceptions=True)` drops (zero fan-out changes). Generalises the Step 2.10 health tracker. `CircuitState`/`BreakerSnapshot` core types + `CircuitOpenError`; pre-registered `breaker.opened` event (`BreakerEvent`, PII-free); no per-call span (hot path). On by default. Gateway `GET /v1/status/breakers` + `POST .../{name}/force-close` + health roll-up; admin-ui Live Status breakers card w/ one-click force-close; `cfg.breakers`; `ragctl breaker`; [ADR-0023](docs/adr/ADR-0023-circuit-breakers.md), [reference/breaker.md](docs/reference/breaker.md), [architecture/circuit-breaker.md](docs/architecture/circuit-breaker.md) | -| 4.5 | Quotas & rate limiting | ✅ | `build/phase-4/step-4.5-quotas-rate-limiting` | [#130](https://github.com/officialCodeWork/AgentContextOS/pull/130) | New `rag-quota` pkg enforcing per-tenant quotas **through the PolicyEngine PDP** (ADR-0005): `QuotaPolicyEngine` (decorates an inner engine, answers `rate_limit` / `quota_check`) over a `QuotaEnforcer` (5 dimensions — QPS + monthly tokens / cost / queries + storage gauge; per-field default-merge; micro-dollar cost from `dollars_per_1k_tokens`; uncapped short-circuit; snapshot/reset) backed by a `QuotaStore` SPI — weighted two-slot sliding-window `InMemoryQuotaStore` + atomic-Lua `RedisQuotaStore` (in `rag-backends`, injected) + always-allow `NoopQuotaStore`. `QuotaVerdict` / `QuotaSnapshot` / `QuotaDimension` core types + `QuotaExceededError(RateLimitError)`. Wired on every surface via `build_app(quota_enforcer=…)` + a `quota_guard` helper: REST `/v1/query` + `/v1/retrieve`, OpenAI chat (sync + stream) + embeddings, ingest storage; entry pre-check raises 429 + `Retry-After`, post-response token/cost metering. **Disabled by default** (can reject); **fail-open by default** (store outage → admit + `quota.store_degraded`, never raises). PII-free `quota.exceeded` event (`QuotaEvent`); no per-call span. `GET /v1/status/quotas` + `POST .../{tenant}/reset` + health presence component; admin-ui Live Status Quotas card (usage bars + reset). `cfg.quotas` + extended `TenantQuota`; `ragctl quota`; coverage linter passes with no new allowlist; ~82 tests + 2 vitest. [ADR-0024](docs/adr/ADR-0024-quotas-rate-limiting.md), [reference/quota.md](docs/reference/quota.md), [architecture/quotas.md](docs/architecture/quotas.md) | -| 4.6 | Latency tuning & load test | ✅ | `build/phase-4/step-4.6-latency-load-test` | [#131](https://github.com/officialCodeWork/AgentContextOS/pull/131) | Gateway **overhead** p99 ≤ 30 ms gate (default noop backends, so per-request wall time *is* the gateway's own cost — distinct from the backend-bound end-to-end target). Gate signal is the server-side `gateway.request_duration_ms` the Step 3.11 `MetricsCollector` records (what `/v1/status/metrics` shows), driven **in-process** via httpx `ASGITransport`, **sequentially**, **retry-tolerant**; runs as a dedicated single-runner **`perf-gate`** CI job (excluded from the cross-OS sweep via `-m "not perf"`). One reusable engine `rag_gateway.perf` (`measure_gateway_overhead` + `profile_gateway`; `LatencyReport`/`RouteLatency`/`ProfileEntry`/`Scenario`) powers the gate + harness + `ragctl perf`. `tests/contract/budgets.py` made real (per-SPI table + gateway budgets; drift-guarded). Load test = in-process `eval/gateway_load_v0` harness (`report.json`/`report.md`, `--check`) + Locust `locustfile.py` (ad-hoc install, not a locked dep). Profiler sorts by own CPU time, warmup outside the profiler. `task perf` / `load-test`; `ragctl perf`; `perf` marker. ~14 tests. [ADR-0025](docs/adr/ADR-0025-latency-load-testing.md), [reference/perf.md](docs/reference/perf.md), [architecture/latency.md](docs/architecture/latency.md) | +### 0.4 — `rag.yaml` schema & loader ✅ [#8](https://github.com/officialCodeWork/AgentContextOS/pull/8) ---- +- JSON Schema (`dist/rag.schema.json`) + Pydantic v2 schema (`packages/config/`) +- Env-var interpolation, `ragctl config validate`, polling hot-reload +- 91 fixture tests; 5 example configs + +### 0.5 — CI/CD pipeline hardening ✅ [#9](https://github.com/officialCodeWork/AgentContextOS/pull/9) + +- Path-filtered jobs, dependency caching, Docker layer cache +- SBOM generation, cosign image signing, Dependabot + +### 0.6 — Local dev stack (Docker Compose) ✅ [#26](https://github.com/officialCodeWork/AgentContextOS/pull/26) + +- Health-check scripts, seed-data loader, `make dev-reset` +- pgvector schema migration, Qdrant collection init + +### 0.7 — Tracing & metrics foundation ✅ [#27](https://github.com/officialCodeWork/AgentContextOS/pull/27) + +- OTel SDK wired, `TraceContext` propagation +- `SpiMetrics` counter + histogram, Grafana provisioned dashboards, `ragctl traces` + +### 0.7b — Structured logging foundation ✅ [#29](https://github.com/officialCodeWork/AgentContextOS/pull/29), [#30](https://github.com/officialCodeWork/AgentContextOS/pull/30) + +- Shared logger package + event registry (`events.py`); RAG001 pre-commit hook + ruff `T201` +- Schema + PII CI gates in `tests/logs/` (50 tests; log-gates job blocking) +- **Follow-up:** extracted `packages/observability/` (`rag-observability`) — 7-field JSON schema (ts, level, service, module, msg, env, version), `set_log_context()` contextvar manager; `rag_core` reduced to back-compat shims (391 tests) + +### 0.7c — Audit log skeleton ✅ [#31](https://github.com/officialCodeWork/AgentContextOS/pull/31) + +- `AuditStore` SPI (append / events / verify_chain) + `NoopAuditStore` (SHA-256 hash chain) +- `AuditWriter` facade (store + structured log); 14 conformance tests -## Phase 5 — Eval & Observability (Weeks 22–28) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 5.1 | Per-query tracing & provenance | ✅ | `build/phase-5/step-5.1-tracing-provenance` | [#132](https://github.com/officialCodeWork/AgentContextOS/pull/132) | All pipeline spans with stable attributes (`rag.schema_version` stamped at the `span_from_trace_context` choke point + `telemetry_attrs` registry/contract); HMAC-signed `ProvenanceRecord` (new `rag-provenance` package, privacy-by-default hashes, degrade-open); `GET /v1/query/{id}/trace` (signed record + `TraceCollector`-captured span tree, tenant-scoped) | -| 5.2 | Offline eval harness (golden set) | ✅ | `build/phase-5/step-5.2-offline-eval-harness` | — | 500 queries / 5 domains (committed `tests/eval/golden/`, drift-gated) run through the real `HybridRetriever` over a synthetic corpus (noop SPIs + `HashingEmbedder`, deterministic); Recall@k / MRR / **nDCG@k** / **Faithfulness** (dependency-free `lexical_faithfulness`, RAGAS optional) / Citation Precision, overall + per-domain + per-difficulty; `render_html_report` HTML + `report.json`/`report.md`; `eval/golden_set_v0/` harness + `--check` threshold floor; `ragctl eval run --html` | -| 5.3 | CI eval gate | ✅ | `build/phase-5/step-5.3-ci-eval-gate` | [#134](https://github.com/officialCodeWork/AgentContextOS/pull/134) | Standalone always-run `eval-gate.yml` runs the harness, compares vs committed `tests/eval/baselines/main.json` (compact byte-stable `EvalBaseline`), enforces `thresholds.yaml` floors **+ regression deltas** (recall@10 / mrr / faithfulness gate; acl/pii are red-team's), posts a sticky marker PR comment with the diff table (even on failure, fork-PR tolerant). Comparison in `rag_config.eval` (`compare_to_baseline` / `render_gate_comment`), result types in `rag_core.eval`, runner `python -m eval.golden_set_v0.gate` | -| 5.4 | Online metrics & feedback | ✅ | `build/phase-5/step-5.4-online-metrics-feedback` | [#135](https://github.com/officialCodeWork/AgentContextOS/pull/135) | New `rag-feedback` package: `FeedbackRecorder` (normalise signal → `[-1,1]` score → PII-redact comment via injected `PIIDetector` → tenant-scoped `FeedbackStore.put` → `feedback.recorded`; degrade-open) + pure `aggregate_feedback` → `FeedbackStats`. One polymorphic `POST /v1/feedback` (explicit + implicit `FeedbackSignal`, body identity, redact-don't-hash, acks `stored=false` when inert/degraded) + per-tenant `GET /v1/status/feedback` dashboard. `FeedbackRecord`/`FeedbackStats` core types + `FeedbackRequest`/`FeedbackAck` wire types; `cfg.feedback`; `ragctl feedback`; admin-UI card deferred to 5.6 | -| 5.5 | Drift monitors | ✅ | `build/phase-5/step-5.5-drift-monitors` | [#136](https://github.com/officialCodeWork/AgentContextOS/pull/136) | New `rag-drift` package: `population_stability_index` (pure binned PSI) + `DriftMonitor` (ref/current windows, PSI or mean-drop, tri-state verdict, rebaseline) + `DriftMonitorRegistry` (5 monitors: query distribution / embedding PSI / retrieval score via PSI, citation-clickthrough / faithfulness via mean-drop). Infra-scoped, fed via `observe` from the query + feedback paths; `evaluate` emits `drift.detected` (event + Step 3.9 webhook) on the transition into drift. `DriftSnapshot`/`DriftReport` core types; `cfg.drift`; `GET /v1/status/drift` + `POST /v1/status/drift/{metric}/rebaseline`; `ragctl drift`; admin-UI card deferred to 5.6 | -| 5.6 | Status & Metrics GUI (full build) | ✅ | `build/phase-5/step-5.6*` | #137–#142 | Delivered in six vertical slices (5.6a–f): Drift/Feedback + Cost cards on Live Status, the Query Trace viewer page, the regression bisector, drift+cost Grafana dashboards, and observability cross-links. Deferred (documented): feedback/breaker/quota Grafana export, a Loki-events dashboard, per-domain eval gating | -| 5.6a | — Drift & Feedback console cards | ✅ | `build/phase-5/step-5.6a-drift-feedback-cards` | [#137](https://github.com/officialCodeWork/AgentContextOS/pull/137) | Surface the shipped `GET /v1/status/drift` (per-monitor verdict + one-click `POST …/rebaseline`) and `GET /v1/status/feedback` (per-tenant satisfaction: mean score, sentiment split, signal counts) as two new **Live Status** cards (`useLive` live-vs-seed, mirroring the 4.4 breaker / 4.5 quota cards). New TS types + `fetchDrift`/`rebaselineDrift`/`fetchFeedback` + `seedDrift`/`seedFeedback`; 3 vitest specs; admin-ui tsc/lint/build/vitest green, browser-verified | -| 5.6b | — Query Trace viewer | ✅ | `build/phase-5/step-5.6b-trace-viewer` | [#138](https://github.com/officialCodeWork/AgentContextOS/pull/138) | New **Query Trace** page (`/trace`, Observability nav): paste a query `request_id` → `GET /v1/query/{id}/trace` renders the signed provenance record (routing, evidence locators, privacy-hashed query/answer), an HMAC **verification pill**, the evidence citations, and the captured span tree as an offset/duration **waterfall** (click a span for its `rag.*` attributes). Input-driven fetch (not `useLive`) with seed fallback + a 404→not-found path. New TS types (`QueryTrace`/`SpanRecord`/`Provenance*`) + `fetchTrace` + `seedTrace`/`DEMO_REQUEST_ID`; 3 vitest specs; admin-ui tsc/lint/build/vitest (31) green, browser-verified | -| 5.6c | — Cost anomaly | ✅ | `build/phase-5/step-5.6c-cost-anomaly` | [#139](https://github.com/officialCodeWork/AgentContextOS/pull/139) | New **`CostTracker`** (in `rag-observability`, beside metrics/log/trace) — bounded per-tenant rolling window of recent per-request token costs, two-gate (ratio + z-score, z relaxed on flat baseline) tri-state verdict; **scale-free on tokens** so detection is decoupled from quota pricing; fed O(1) from `record_request_usage` **independent of quotas**; pull-based `GET /v1/status/cost` (`CostStatusResponse`, no per-request span/event); `cfg.cost`; `CostSnapshot` a `dataclass` so **no `rag-core`/`dist/schemas` churn** (only `rag.schema` + `dist/openapi` regen). Admin **Cost anomaly** card on Live Status. 16 Python tests (8 tracker + 4 gateway + 4 config) + 1 vitest; mypy --strict / ruff / RAG001 + gateway/observability/config suites green; browser-verified. Push alert (`cost.anomaly_detected` event/webhook) + per-model pricing deferred. [ADR-0031](docs/adr/ADR-0031-cost-anomaly.md), [reference/cost.md](docs/reference/cost.md), [architecture/cost-anomaly.md](docs/architecture/cost-anomaly.md) | -| 5.6d | — Regression bisector | ✅ | `build/phase-5/step-5.6d-regression-bisector` | [#140](https://github.com/officialCodeWork/AgentContextOS/pull/140) | `python -m eval.golden_set_v0.bisect --good --bad HEAD --metric recall_at_k_mean --threshold ` — when the eval gate goes red, **binary-search `good..bad`** (only O(log n) harness runs) to find the first commit that dropped a metric. Each candidate runs in a **throwaway `git worktree`** (never touches the working tree) with that commit's `packages/*/src` on `PYTHONPATH` so the *commit's* code runs. Pure search `bisect_commits` → `BisectResult`/`BisectStep` (in `rag_config.eval`/`rag_core.eval`, additive, not in `dist/schemas`); orchestration (`commits_between`/`score_commit_via_worktree`/`run_bisect`, injectable for tests) in `eval/golden_set_v0/bisect.py`. 7 unit tests (search logic, no git) + live worktree smoke (3-of-6 commits, recall 0.906, clean cleanup); mypy --strict (config+core) / ruff / RAG001 + full eval suite green. [reference/eval-harness.md](docs/reference/eval-harness.md#regression-bisector-step-56d) | -| 5.6e | — Grafana dashboards | ✅ | `build/phase-5/step-5.6e-grafana-dashboards` | [#141](https://github.com/officialCodeWork/AgentContextOS/pull/141) | Make the Phase-5 quality/cost signals **real Prometheus metrics**, then dashboard them (they were pull-only `/v1/status/*` + log events, invisible to Grafana). New `rag_observability.register_platform_metrics` registers **OTel observable gauges** — `rag.drift.{statistic,threshold,drifted}` (from the registry's side-effect-free `report()`) + `rag.cost.{ratio,elevated,recent_mean_tokens}` (from `CostTracker.snapshot_all`) — on the existing OTel→collector→Prometheus pipeline; registered once per process by `build_app_from_config` (NoOp until telemetry on). Provisioned **`rag-quality-cost`** Grafana dashboard (`infra/grafana/dashboards/`) graphs them. **Proven** by an OTel `InMemoryMetricReader` test (exact exported values) + a dashboard-validation gate (`tests/infra/test_grafana_dashboards.py`: JSON valid, datasource UIDs provisioned, PromQL references only exported metrics). No `rag-core`/schema/config change. **Feedback Prometheus export deferred** (async store needs a refresher; GUI card covers it); breaker/quota gauges + a Loki-events dashboard (dev-stack logs are OTLP-only) deferred. [guides/grafana-dashboards.md](docs/guides/grafana-dashboards.md) | -| 5.6f | — Cross-links + close-out | ✅ | `build/phase-5/step-5.6f-cross-links` | [#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) | Wire the observability surfaces together: the **Query Trace** viewer's *Logs* button → `/logs?trace=` (scopes the live tail to one query's lines via a clearable violet trace-chip, read from the URL param client-side), a log record's detail drawer → *View this trace's logs*, and a **Trace** quick-link on Live Status. Admin-ui only (logs/trace/status pages + 1 vitest cross-link spec). tsc/lint/build/vitest (33) green, browser-verified. Closes Step 5.6 | -| 5.7 | A/B testing & shadow mode | 🚧 | `build/phase-5/step-5.7*` | — | Delivered in vertical slices (5.7a–d): the analyzer + tracker + dashboard (5.7a), shadow mode (5.7b), A/B routing (5.7c), console + close-out (5.7d). Shadow N% of live queries; A/B routing; statistical analyzer with lift + CI | -| 5.7a | — A/B analyzer + tracker + dashboard | 🚧 | `build/phase-5/step-5.7a-ab-analyzer` | [#143](https://github.com/officialCodeWork/AgentContextOS/pull/143) | Pure **`analyze_ab_experiment`** (`rag_config.eval`) — lift + normal-approx Welch CI + significance via stdlib `statistics.NormalDist` (no numpy/scipy) → **`ABAnalysisResult`** (`rag_core.eval`, additive, not in `dist/schemas`). **`ABExperimentTracker`** (`rag-observability`, beside `CostTracker`) is a pure per-`(experiment, variant)` sample *holder* — decoupled from the analyzer so observability keeps no `rag_config` dep; the gateway composes the two at **`GET /v1/status/experiments`**. `cfg.experiments` (opt-in, since A/B routing can change responses); wired inert by default in `build_app_from_config`; `ragctl experiments`. Observe-only — no query-path change yet (shadow/routing are 5.7b/c). 17 tests (analyzer + tracker + gateway + config) + ragctl smoke; mypy --strict (119) / ruff / RAG001 + gateway/observability/config/eval suites green. [ADR-0032](docs/adr/ADR-0032-ab-testing-shadow-mode.md), [reference/experiments.md](docs/reference/experiments.md) | +### 0.8 — Eval skeleton ✅ [#33](https://github.com/officialCodeWork/AgentContextOS/pull/33) + +- `rag_core.eval` types (GoldenSample, EvalMetrics, EvalReport) + `rag_config.eval` metrics (recall@k, MRR, citation_precision) +- RagasAdapter spike, `ragctl eval run/show`, 5-sample golden JSONL, `tests/eval/` harness (39 tests) + +### 0.9 — IaC foundation ✅ [#34](https://github.com/officialCodeWork/AgentContextOS/pull/34) + +- Terraform modules for Postgres/pgvector, Redis, Qdrant, Elasticsearch (Kubernetes-native, Helm provider) +- `rag-platform` Helm chart (Deployment, Service, ConfigMap, ServiceAccount, HPA, PDB, Ingress); dev + prod envs +- `task infra:*` + `task helm:*`; ADR-0003 + +### 0.10 — `ragctl` CLI scaffold ✅ [#36](https://github.com/officialCodeWork/AgentContextOS/pull/36) + +- `packages/ragctl/` (Typer 0.12+), root `ragctl` entry point +- Working `config` / `eval` / `traces` / `version` groups; scaffolded `ingest` / `query` / `logs` / `tenant` / `plugin` / `secret` +- Shell completion; 19 tests; [reference/ragctl.md](docs/reference/ragctl.md) + [guides/ragctl-quickstart.md](docs/guides/ragctl-quickstart.md) --- -## Phase 6 — Governance & Tenancy (Weeks 28–34) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 6.1 | Logical multi-tenancy | ⏳ | — | — | Namespace isolation; per-tenant config, quotas, ACLs in `rag.yaml` | -| 6.2 | Physical tenancy (dedicated index) | ⏳ | — | — | Dedicated vector index per tenant; cross-tenant probe gate | -| 6.3 | ACL push-down at retrieval | ⏳ | — | — | ACL filter injected into every vector/BM25/graph query; `acl.egress_denied` event | -| 6.4 | ACL egress verifier | ⏳ | — | — | Post-retrieval re-check; defense-in-depth; zero ACL violation rate gate | -| 6.5 | PII policies | ⏳ | — | — | Per-tenant PII enforcement: block/redact/allow; egress redaction; `pii.egress_blocked` event | -| 6.6 | Immutable audit log | ⏳ | — | — | Hash-chain audit log; WORM export; tamper-evident verification; `GET /v1/audit` | -| 6.7 | BYOK (Bring Your Own Key) | ⏳ | — | — | KMS integration (AWS KMS, GCP KMS, HashiCorp Vault); envelope encryption for embeddings | -| 6.8 | SSO / SCIM | ⏳ | — | — | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config | -| 6.9 | Air-gapped install bundle | ⏳ | — | — | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification | -| 6.10 | Compliance posture | ⏳ | — | — | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies | +## Phase 1 — Ingestion + Knowledge Store (Weeks 4–8) ✅ + +> **Refactor window (1.1a–1.1f):** before resuming connectors, a six-step refactor locked in architecture decisions +> expensive to retrofit — PolicyEngine PDP, RequestContext-threaded SPIs, split Retrieval/Index backends, +> bulk + streaming + ID-only methods, Pipeline + Batcher primitives, three-way cache split, hot-path discipline. + +| Step | Title | Status | PR | +|------|-------|:------:|----| +| 1.1 | Storage backends | ✅ | [#40](https://github.com/officialCodeWork/AgentContextOS/pull/40) | +| 1.1a | Core type & SPI refactor | ✅ | [#44](https://github.com/officialCodeWork/AgentContextOS/pull/44) | +| 1.1b | SPI split — Retrieval/Index, bulk + streaming + ID-only | ✅ | [#45](https://github.com/officialCodeWork/AgentContextOS/pull/45) | +| 1.1c | PolicyEngine package | ✅ | [#46](https://github.com/officialCodeWork/AgentContextOS/pull/46) | +| 1.1d | Pipeline + Batcher primitives | ✅ | [#50](https://github.com/officialCodeWork/AgentContextOS/pull/50) | +| 1.1e | Cache SPI split + perf discipline + async telemetry | ✅ | [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) | +| 1.1f | ADRs 0005–0009 + reviewer checklist | ✅ | [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) | +| 1.2 | Connectors framework | ✅ | [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) | +| 1.3 | Document parsers | ✅ | [#57](https://github.com/officialCodeWork/AgentContextOS/pull/57) | +| 1.4 | OCR pipeline | ✅ | [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) | +| 1.5 | Structure-aware chunker | ✅ | [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) | +| 1.6 | Metadata enricher | ✅ | [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) | +| 1.7 | PII detection | ✅ | [#66](https://github.com/officialCodeWork/AgentContextOS/pull/66) | +| 1.8 | Embedder pipeline | ✅ | [#68](https://github.com/officialCodeWork/AgentContextOS/pull/68) | +| 1.9 | CDC connectors (incremental sync) | ✅ | [#70](https://github.com/officialCodeWork/AgentContextOS/pull/70) | +| 1.10 | Write path & ingest API | ✅ | [#72](https://github.com/officialCodeWork/AgentContextOS/pull/72) | + +### 1.1 — Storage backends ✅ [#40](https://github.com/officialCodeWork/AgentContextOS/pull/40) + +- `rag-backends`: `PgVectorStore` (asyncpg + pgvector, ivfflat), `QdrantVectorStore` (query_points), `RedisCache`, `S3Storage` (aioboto3, MinIO-compatible), `LocalFileStorage` +- Integration tests (skip-if-no-service); MinIO added to dev stack; `task test-integration` + `test-backends`; ADR-0004 + +### 1.1a — Core type & SPI refactor ✅ [#44](https://github.com/officialCodeWork/AgentContextOS/pull/44) + +- `RequestContext` frozen model threaded through every SPI (always first arg) +- Typed `tenant_id` + `acl_labels` on Chunk/Embedding (not metadata dict); `trust_level` on Chunk (injection defense); `dtype` on Embedding +- `BlobRef` (lazy chunk text), `QueryPlan`, `ChunkRef`, `Cost`, `PlanNode`, typed `StageEvent` +- `spi_signature.py` linter (ctx-first); backends migrated; ADR-0005 / 0007 / 0008 / 0009 + +### 1.1b — SPI split — Retrieval/Index, bulk + streaming + ID-only ✅ [#45](https://github.com/officialCodeWork/AgentContextOS/pull/45) + +- Split Vector/Keyword/Graph into `*RetrievalBackend` (read) + `*IndexBackend` (write) composites +- `retrieve_ids` → `list[ChunkRef]`; `hydrate` on the retrieval side +- Bulk `bulk_index` / `bulk_delete` (+ graph variants); `stream_index` async-iterator default; `Embedder` split single `embed` + `bulk_embed` +- `IndexHint` + `WriteVolume` passed to writes (ADR-0009) + +### 1.1c — PolicyEngine package ✅ [#46](https://github.com/officialCodeWork/AgentContextOS/pull/46) + +- `rag-policy` v0.1.0: `PolicyEngine` SPI + `NoopPolicyEngine` (always-ALLOW + tenant-scoped `filter_pushdown`) +- `PolicyDecision` enum (read_chunk / ingest_doc / egress_text / quota_check / rate_limit / execute_plan); `PolicyResult` (allow/deny/transform) +- `FilterExpr` mini-language (Eq / AnyIn / And / Or / Not / TrueExpr); `PolicyWriter` facade → `policy.decision` logs +- Coverage linter `tests/policy/coverage.py` (greps governance SPI calls w/o PDP consultation); 20 tests; ADR-0005 finalized + +### 1.1d — Pipeline + Batcher primitives ✅ [#50](https://github.com/officialCodeWork/AgentContextOS/pull/50) + +- `Pipeline` (`rag_core.pipeline`) — async DAG, bounded queues, per-stage workers, backpressure, `on_error` fail/skip +- `Batcher[Req, Resp]` (`rag_core.batcher`) — DataLoader coalescing, size + time triggers, per-request Future isolation, `flush()` +- PEP-695 generics; 22 unit tests + +### 1.1e — Cache SPI split + perf discipline + async telemetry ✅ [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) + +- `EmbeddingCache` / `RetrievalCache` / `AnswerCache` SPIs (ctx-first, version-partitioned, invalidation hooks) + noop impls (18 tests) +- `AsyncTelemetrySink` (`rag-observability`) — bounded buffer, non-blocking `submit()`, drop + error counters (9 tests) +- Hot-path discipline doc in `performance.md` (hot/cold call-site table) + +### 1.1f — ADRs 0005–0009 + reviewer checklist ✅ [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) + +- ADR-0005 (PolicyEngine PDP), 0006 (two-stage reranker), 0007 (tiered storage + BlobRef), 0008 (cost-aware planner), 0009 (vector index + quantization) → Accepted +- Reviewer checklist extended; module-level ADR backlinks on `reranker` + `storage` + +### 1.2 — Connectors framework ✅ [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) + +- `Connector` SPI → `(Document, ConnectorState)` async-iterator with resumable watermark +- Internal `Crawler[Raw]` base; built-in `FilesystemConnector`, `S3Connector`, `GCSConnector` (`[gcs]` extra) +- Connectors upstream of PolicyEngine (ingest pipeline is the enforcement point); 18 tests + +### 1.3 — Document parsers ✅ [#57](https://github.com/officialCodeWork/AgentContextOS/pull/57) + +- `rag-parsers`: 10 parsers — text, markdown, HTML, JSON, CSV, YAML, PDF, DOCX, PPTX, XLSX +- `Parser` SPI → `ParsedDocument` (text + structural `Block` list + `BlockType`); `detect_mime()`; `ParserRegistry` +- `ragctl parse`; 148 contract tests total + +### 1.4 — OCR pipeline ✅ [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) + +- `rag-ocr`: `TesseractOCR` (per-word) + `PaddleOCRBackend` (per-line) behind `[tesseract]` / `[paddle]` extras +- `BoundingBox` / `OCRRegion` / `OCRResult` frozen models (length-weighted confidence) +- `ragctl ocr`; 27 tests (engines stubbed at `sys.modules`) + +### 1.5 — Structure-aware chunker ✅ [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) + +- `rag-chunker`: `HeadingAwareChunker` — heading-level stack for `parent_id`, token budget (512/64 overlap), sentence-boundary splitting, never bleeds across headings +- `Chunker` SPI + `NoopChunker`; `TokenCounter` Protocol + `TiktokenCounter`; OCR→ParsedDocument adapter +- `ragctl chunk`; 40 tests + +### 1.6 — Metadata enricher ✅ [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) + +- `rag-enricher`: `DefaultEnricher` tags language, doc_type, created/modified, author, title, section_path, reading_level (Flesch-Kincaid) +- `Enricher` SPI; `LanguageDetector` Protocol + seeded `LangdetectDetector` +- `ragctl enrich`; 34 tests + +### 1.7 — PII detection ✅ [#66](https://github.com/officialCodeWork/AgentContextOS/pull/66) + +- `rag-pii`: `RegexPIIDetector` (EMAIL/PHONE/SSN/CREDIT_CARD/IP, Luhn-validated; shares definition with the log-leak gate) + `PresidioPIIDetector` (`[presidio]`) +- `PiiProcessor` — redact (default) / mask / block; `pii.detected` event carries metadata only (never raw text) +- `PIISpan` promoted to frozen model; `ragctl pii`; 48 tests + +### 1.8 — Embedder pipeline ✅ [#68](https://github.com/officialCodeWork/AgentContextOS/pull/68) + +- `rag-embedders`: `OpenAIEmbedder`, `CohereEmbedder`, `SentenceTransformersEmbedder` (BGE + E5) — 4 model families behind `Embedder` SPI +- `BatchingEmbedder` shared sub-batch split + retry (`RetryPolicy`) + dimension normalize +- `ragctl embed` (noop default, no creds); 46 tests + +### 1.9 — CDC connectors (incremental sync) ✅ [#70](https://github.com/officialCodeWork/AgentContextOS/pull/70) + +- `PostgresCDCConnector` (pgoutput WAL LSN cursor, tombstones), `S3EventsConnector` (SQS long-poll, SNS-wrapped), `WebhookReceiverConnector` (push, backpressure modes) +- `DedupFilter` (wraps any Connector + Cache, drops by `Document.fingerprint`); `DedupStats` +- 35 tests (SDKs stubbed) + +### 1.10 — Write path & ingest API ✅ [#72](https://github.com/officialCodeWork/AgentContextOS/pull/72) + +- 🥈 **CLI demo milestone — Phase 1 complete.** +- `rag-ingest`: `IngestPipeline` — parse → chunk → enrich → PII → embed → index (PII before embed so vectors never encode PII); `PolicyEngine.evaluate(ingest_doc)` short-circuit; per-doc error isolation +- `POST /v1/ingest/document` (multipart) + `build_app(pipeline=)` DI; `ragctl ingest` wired for real +- IngestPipeline is the canonical `ingest_doc` PDP consumer; 38 tests + +## Phase 2 — Retrieval Engine (Weeks 8–12) ✅ + +| Step | Title | Status | PR | +|------|-------|:------:|----| +| 2.1 | Knowledge store read layer | ✅ | [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | +| 2.2 | Vector retrieval backends | ✅ | [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | +| 2.3 | BM25 / keyword retrieval | ✅ | [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) | +| 2.4 | Graph retrieval | ✅ | [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) | +| 2.5 | Hybrid RRF fusion | ✅ | [#86](https://github.com/officialCodeWork/AgentContextOS/pull/86) | +| 2.6 | Query understanding | ✅ | [#90](https://github.com/officialCodeWork/AgentContextOS/pull/90) | +| 2.7 | Cross-encoder reranker | ✅ | [#92](https://github.com/officialCodeWork/AgentContextOS/pull/92) | +| 2.8 | Context packer | ✅ | [#94](https://github.com/officialCodeWork/AgentContextOS/pull/94) | +| 2.9 | GraphRAG | ✅ | [#96](https://github.com/officialCodeWork/AgentContextOS/pull/96) | +| 2.10 | Retrieval router | ✅ | [#98](https://github.com/officialCodeWork/AgentContextOS/pull/98) | +| 2.11 | Agent-loop validation spike | ✅ | [#100](https://github.com/officialCodeWork/AgentContextOS/pull/100) | + +### 2.1 — Knowledge store read layer ✅ [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) + +- `FilterExpr` AST + in-memory `evaluate()` relocated to `rag_core.filter` so SPI signatures take it directly (`rag_policy.filter` re-exports for back-compat) +- `retrieve_ids` switched to typed `FilterExpr | None` on Vector + Keyword backends; new signature gate +- Noop stores apply the predicate via `evaluate()` (conformance oracle); pgvector `_filter_sql.translate()` + `acl_labels TEXT[]` GIN column; Qdrant `_filter_qdrant.translate()`; unsupported fields raise (no silent push-down loss) +- 75 tests; rag-core 0.13 → 0.14; [architecture/retrieval-read-layer.md](docs/architecture/retrieval-read-layer.md) + +### 2.2 — Vector retrieval backends ✅ [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) + +- `WeaviateVectorStore` (`[weaviate]`), `PineconeVectorStore` (`[pinecone]`), `ElasticsearchDenseVectorStore` (`[elasticsearch]`) — each with its own `FilterExpr` translator + tenant isolation + `acl_labels` push-down +- All five vector backends consume `IndexHint` via shared `select_index_variant()` (ADR-0009 size/recall → ivfflat / HNSW / PQ) +- Lazy SDK imports; 53 tests; [reference/backends.md](docs/reference/backends.md), [architecture/vector-backends.md](docs/architecture/vector-backends.md) + +### 2.3 — BM25 / keyword retrieval ✅ [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) + +- `ElasticsearchKeywordStore` (`[elasticsearch]`) + `TantivyKeywordStore` (`[tantivy]`) on the Step 2.1 read-layer contract (no new SPI) +- Per-field boosting is a backend config knob (`field_boosts`), not a per-call param; identical indexed-field schema across both (swap = config change) +- `_filter_es` / `_filter_tantivy` translators (tantivy `Not` paired with `all_query`); 47 tests; [architecture/keyword-backends.md](docs/architecture/keyword-backends.md) + +### 2.4 — Graph retrieval ✅ [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) + +- `Neo4jGraphStore` (`[neo4j]`), `MemgraphGraphStore` (`[memgraph]`), `NetworkXGraphStore` (`[networkx]`); Neo4j + Memgraph share `BoltCypherStore` +- New `expand(ctx, seed_ids, hops, rel_types, direction, node/edge_filter, limit) → list[NeighborResult]` (non-abstract default; `NoopGraphStore` BFS is the oracle) +- `FilterExpr` push-down on edges (every hop — the ACL case) + final node; `_filter_cypher.translate()` with `param_prefix`; rel-type injection guard; tenant isolation on every edge +- 65+ tests; `NeighborResult` schema; [architecture/graph-backends.md](docs/architecture/graph-backends.md) + +### 2.5 — Hybrid RRF fusion ✅ [#86](https://github.com/officialCodeWork/AgentContextOS/pull/86) + +- New `rag-retrieval` package. Pure `rrf_fuse(rankings, weights, k=60, top_k)` — rank-only `Σ w/(k+rank)`, earliest-source metadata, cross-tenant raise +- `HybridRetriever` — parallel vector + keyword + graph fan-out (`asyncio.gather(return_exceptions=True)`), per-call source skip, partial-failure tolerance (only all-fail raises) +- **Canonical `read_chunk` PDP call site** via `policy_engine.filter_pushdown` And-merged with the caller filter; `HybridWeights`; `GraphAdapter` protocol +- `ragctl hybrid`; 43 tests; [reference/retrieval.md](docs/reference/retrieval.md), [architecture/hybrid-fusion.md](docs/architecture/hybrid-fusion.md) + +### 2.6 — Query understanding ✅ [#90](https://github.com/officialCodeWork/AgentContextOS/pull/90) + +- New `rag-query` package — four components + a parallel orchestrator +- `LLMQueryRewriter`; `LLM` / `Heuristic` / `Noop` `QueryDecomposer` (conjunction + sentence splits, `[]` sentinel); `HyDEGenerator` (Gao 2022, optional embedder); `InMemoryGlossary` + expander +- `UnderstoodQuery` frozen output (`rewrites` / `sub_queries` / `expansion_terms` / `hyde_*`); `QueryUnderstandingPipeline` fan-out, per-component degrade to identity +- `query.understand` span; `ragctl understand`; 60 tests; [reference/query.md](docs/reference/query.md), [architecture/query-understanding.md](docs/architecture/query-understanding.md) + +### 2.7 — Cross-encoder reranker ✅ [#92](https://github.com/officialCodeWork/AgentContextOS/pull/92) + +- New `rag-reranker` package — ADR-0006 two-stage cascade. `Reranker` SPI → `fast_rerank` / `precise_rerank` / `should_early_exit`; `Chunk.score` added (rag-core 0.14 → 0.15) +- Pure `mmr_rerank()` (Carbonell-Goldstein, cosine cache, stable at λ=0); `RerankPipeline` — fast → early-exit? → hydrate → precise → optional MMR; strict failure (`RerankerError(stage=)`) +- Backends: `SentenceTransformersCrossEncoder` (BGE v2-m3 / MiniLM), `CohereReranker`, `JinaReranker` (lazy extras) +- `rerank` span; `ragctl rerank`; 92 tests; [architecture/reranking.md](docs/architecture/reranking.md), [reference/reranker.md](docs/reference/reranker.md) + +### 2.8 — Context packer ✅ [#94](https://github.com/officialCodeWork/AgentContextOS/pull/94) + +- New `rag-packer` package. `ContextPacker` — tenant check → sort by score → hash dedup → opt-in Jaccard dedup → greedy token-budget fit (`TokenCounter` / `TiktokenCounter`, `Chunk.token_count` fast-path) → reorder (`lost_in_middle` default, Liu 2024) → heuristic conflict scan (trust-mismatch + numeric-disagreement) +- `PackedContext` + `ConflictAnnotation` frozen models (rag-core 0.15 → 0.16); strict `PackerError` (budget ≤ 0 / mixed-tenant / `content_ref` w/o `token_count`); conflict scan is the one degrade-and-log path +- `pack` span; `ragctl pack`; 78 tests + +### 2.9 — GraphRAG ✅ [#96](https://github.com/officialCodeWork/AgentContextOS/pull/96) + +- New `rag-graphrag` package: `LouvainDetector` (default) + `LeidenDetector` (`[leiden]`) community detection over `GraphRetrievalBackend` (no new SPI) +- `CommunitySummarizer` (LLM strict-JSON, NoopLLM fallback), `InMemoryCommunityStore` (tenant-scoped), `graphrag_adapter` (entity-node → `ChunkRef`) +- `GraphRAGRetriever` — set-overlap community scoring (2× on `key_entities`) → seed collection → `expand()` fan-out → returns `ChunkRef` for `HybridRetriever` +- `Community` / `CommunitySummary` (rag-core 0.16 → 0.17); 3 `graphrag.*` spans; `ragctl graphrag`; 54 tests; [reference/graphrag.md](docs/reference/graphrag.md), [architecture/graphrag.md](docs/architecture/graphrag.md) + +### 2.10 — Retrieval router ✅ [#98](https://github.com/officialCodeWork/AgentContextOS/pull/98) + +- `RetrievalRouter` + pure `classify_shape` (KEYWORD_HEAVY / SEMANTIC / ENTITY_RICH / MIXED) layered on `HybridRetriever` (rag-retrieval 0.1 → 0.2) +- In-memory `BackendHealthTracker` (rolling-window, injected clock); shape-bias weight table; vector-input priority (caller > HyDE > embed > skip) +- **BM25-only fallback** when vector + graph are down; failure-feedback loop (`record_failure` / `record_success`) converges degradation without an external loop +- `RoutingDecision` + `QueryShape` (rag-core 0.17 → 0.18); `retrieve.route` span; `ragctl route`; 42 tests; [reference/router.md](docs/reference/router.md), [architecture/retrieval-router.md](docs/architecture/retrieval-router.md) + +### 2.11 — Agent-loop validation spike ✅ [#100](https://github.com/officialCodeWork/AgentContextOS/pull/100) + +- `AgentLoopV0` (`rag-retrieval` 0.2 → 0.3) — thin iterative-retrieve orchestrator: per-iter budget enforcement, sticky routing, plan-reuse measurement, shared `EmbeddingCache`, sub-query fan-out; cross-package isolation via `QueryUnderstander` / `UnderstoodQueryLike` Protocols +- `StopReason` / `BudgetSpend` / `CacheStats` / `AgentLoopError` (rag-core 0.18 → 0.19); `agent_loop.run` span +- 50-query harness in `eval/agent_loop_v0/` (5 buckets). **Headline finding:** 0% embedding-cache hit on the decomposable bucket → drove the G-01 must-fix +- `ragctl agent-loop`; 66 tests; [reference/agent-loop.md](docs/reference/agent-loop.md), [spikes/agent-loop-v0-gaps.md](docs/spikes/agent-loop-v0-gaps.md) +- **Pre-3.1 must-fix window** ([#101–#105](https://github.com/officialCodeWork/AgentContextOS/pull/105)): G-02 split route → decide + execute · G-03 reject zero budgets · G-06 cache HyDE + rewriter · G-01 `SemanticEmbeddingCache` (hit rate 28% → 54%, ADR-0010) · G-04 per-stage `CostEstimator` (ADR-0008) + +## Phase 3 — Gateway & Agent Runtime (Weeks 12–18) ✅ + +| Step | Title | Status | PR | +|------|-------|:------:|----| +| 3.1 | Gateway service (REST + OpenAPI) | ✅ | [#107](https://github.com/officialCodeWork/AgentContextOS/pull/107) | +| 3.2 | gRPC service | ✅ | [#109](https://github.com/officialCodeWork/AgentContextOS/pull/109) | +| 3.3 | MCP server | ✅ | [#111](https://github.com/officialCodeWork/AgentContextOS/pull/111) | +| 3.4 | OpenAI-compatible endpoints | ✅ | [#113](https://github.com/officialCodeWork/AgentContextOS/pull/113) | +| 3.5 | Corpus router | ✅ | [#115](https://github.com/officialCodeWork/AgentContextOS/pull/115) | +| 3.6 | Agent loop | ✅ | [#120](https://github.com/officialCodeWork/AgentContextOS/pull/120) | +| 3.7 | Official SDKs | ✅ | [#121](https://github.com/officialCodeWork/AgentContextOS/pull/121) | +| 3.8 | Framework adapters | ✅ | [#122](https://github.com/officialCodeWork/AgentContextOS/pull/122) | +| 3.9 | Webhooks | ✅ | [#123](https://github.com/officialCodeWork/AgentContextOS/pull/123) | +| 3.10 | Admin UI v0 | ✅ | [#124](https://github.com/officialCodeWork/AgentContextOS/pull/124) | +| 3.11 | Status & Metrics GUI (MVP) | ✅ | [#125](https://github.com/officialCodeWork/AgentContextOS/pull/125) | + +### 3.1 — Gateway service (REST + OpenAPI) ✅ [#107](https://github.com/officialCodeWork/AgentContextOS/pull/107) + +- 🥈 **Curl-able RAG milestone.** `rag-gateway` 0.3 → 0.4: `POST /v1/query` (full RAG → `QueryResponse`), `POST /v1/retrieve`, `GET /v1/corpora` + `/{id}` (new `CorpusStore` SPI + `NoopCorpusStore`) +- `rag-core` 0.19 → 0.20 wire types in `rag_core.gateway_types` (`Corpus`, `QueryRequest/Response`, `RetrieveRequest/Response`, `Answer`, `StageTimings`, `GatewayError`) +- Middleware builds per-request `RequestContext` (X-Request-Id / traceparent / Bearer); domain errors → HTTP (403 / 401 / 429 / 502 / 400) via `GatewayError` envelope; `PolicyEngine.egress_text` before every LLM call +- `build_app(...)` injectable factory (noop defaults, zero infra); OpenAPI 3.1 + Swagger / ReDoc; `ragctl query`; 43 tests; [reference/gateway.md](docs/reference/gateway.md), [guides/curl-quickstart.md](docs/guides/curl-quickstart.md) + +### 3.2 — gRPC service ✅ [#109](https://github.com/officialCodeWork/AgentContextOS/pull/109) + +- `proto/rag.proto` — `rag.gateway.v1.RagService`: server-streaming `Query` (`QueryStreamEvent` oneof) + unary `Retrieve` / `IngestDocument` / `ListCorpora` / `GetCorpus`; health + reflection co-registered (grpcurl works) +- Python stubs via `grpcio-tools` + `mypy-protobuf` (committed, drift-gated); `task proto:lint` / `gen` / `check-drift` on full OS matrix +- `GrpcRagService` reuses the same Phase 2 composition as REST (`build_grpc_server(...)`); error → StatusCode mapping mirrors REST; `GatewayError` packed on trailing metadata +- `ragctl grpc-query`; 48 tests; [ADR-0011](docs/adr/ADR-0011-grpc-wire-protocol.md), [reference/grpc.md](docs/reference/grpc.md) + +### 3.3 — MCP server ✅ [#111](https://github.com/officialCodeWork/AgentContextOS/pull/111) + +- New `rag_gateway.mcp` — `FastMCP` over the official `mcp` SDK (stdio); three tools `query` / `retrieve` / `ingest` +- `ToolBackend` Protocol is the topology seam: `InProcessBackend` shares one process wiring with REST + gRPC; a future `GatewayClientBackend` slots in (ADR-0012) +- `@ragplatform/mcp` npm launcher (pure-Node `bin` → `python -m rag_gateway.mcp`, cross-platform interpreter discovery, no injection surface) +- `ragctl mcp-query` / `mcp-serve`; 21 tests; [ADR-0012](docs/adr/ADR-0012-mcp-server-topology.md), [reference/mcp.md](docs/reference/mcp.md) + +### 3.4 — OpenAI-compatible endpoints ✅ [#113](https://github.com/officialCodeWork/AgentContextOS/pull/113) + +- `rag_gateway.openai_compat`: `POST /v1/embeddings`, `POST /v1/chat/completions`, `GET /v1/models` — drop-in for OpenAI SDKs / LangChain / LlamaIndex +- **Retrieval pre-fetch:** chat runs the Phase 2 pipeline over the last user turn, egress-policed, injected as a system message; on by default (`rag.enabled`); `RagOptions` request field + `RagAnnotations` response field +- `stream=true` → OpenAI-faithful `chat.completion.chunk` SSE; base64 embeddings; header identity w/ NoopAuth dev-anon fallback; OpenAI `{error:{...}}` envelope +- `rag-core` 0.20 → 0.21 `openai_types`; `ragctl chat` / `embeddings`; 36 tests; [ADR-0013](docs/adr/ADR-0013-openai-compatibility.md), [guides/openai-quickstart.md](docs/guides/openai-quickstart.md) + +### 3.5 — Corpus router ✅ [#115](https://github.com/officialCodeWork/AgentContextOS/pull/115) + +- `CorpusRouter` sits *above* the Step 2.10 `RetrievalRouter` — it answers *which corpora* a query touches. `decide()` precedence: explicit pin → static rules → learned classifier → all (auto-detects effective strategy); `UNCONSTRAINED` preserves pre-3.5 behaviour +- `route()` fan-out (default): one `RetrievalRouter.route` per corpus, `rrf_fuse` weighted by corpus score; `StaticRuleClassifier` (auditable) + dependency-free `LearnedCorpusClassifier` (softmax over `term_weights`) behind `CorpusClassifier` Protocol +- `CorpusRoutingStrategy` / `CorpusScore` / `CorpusRoutingDecision` (rag-core); `corpus_decision` on Query/Retrieve responses; Postgres `PgCorpusStore` (lazy); config-driven `build_app_from_config` in gateway wiring (keeps the `config → core` boundary) +- `corpus.route` span + audit event; `ragctl corpus list/seed/route`; 35 tests; [ADR-0014](docs/adr/ADR-0014-corpus-routing.md), [reference/corpus-router.md](docs/reference/corpus-router.md) + +### 3.6 — Agent loop ✅ [#120](https://github.com/officialCodeWork/AgentContextOS/pull/120) + +- New `rag-agent` package (rag-core only) graduates the 2.11 spike. `AgentLoop` is an explicit state machine (`planning → acting → observing` / `finalizing`) driven by a pluggable `Controller`; `stream` / `run` / `resume` +- Four nullable budget dims spent per step (stop checked top-of-step); `ScriptedController` / `HeuristicController` (creds-free) / `LLMController`; `RetrieveTool` (recoverable `RetrievalError`); resumable via frozen `AgentSnapshot` + `CheckpointStore` +- `rag_core.agent_types`; surfaces: REST `POST /v1/agent` (SSE) + gRPC `RagService/Converse` (streaming) sharing one flat event frame + egress helpers; final-answer egress-policed once at the boundary +- `agent.run` / `grpc.converse` spans; `ragctl agent`; full unit + gRPC + proto-compat suites; [ADR-0015](docs/adr/ADR-0015-agent-runtime.md), [reference/agent.md](docs/reference/agent.md) + +### 3.7 — Official SDKs ✅ [#121](https://github.com/officialCodeWork/AgentContextOS/pull/121) + +- Committed, drift-gated OpenAPI 3.1 (`dist/openapi.{json,yaml}` via `export_openapi.py`; `openapi-drift` CI job) +- Hand-written flagships: **Python** `agentcontextos` (sync + async, SSE, reuses `rag-core` models; 32 tests vs a live gateway) + **TypeScript** `@agentcontextos/sdk` (zero-dep fetch / ReadableStream, Node + browser; 10 vitest) +- Generated on demand: **Go / Java / .NET** via `task sdk:gen` (openapi-generator + `buf generate`); generated source gitignored, configs committed +- `ts-sdk` CI job; gateway 0.5 → 0.7; [ADR-0016](docs/adr/ADR-0016-sdk-generation.md), [reference/sdks.md](docs/reference/sdks.md) + +### 3.8 — Framework adapters ✅ [#122](https://github.com/officialCodeWork/AgentContextOS/pull/122) + +- `agentcontextos.integrations` (in `sdks/python`) — **8 integrations** wrapping the SDK `Client`: LangChain, LlamaIndex, Haystack, DSPy, LangGraph, CrewAI, AutoGen, Semantic Kernel +- Shared `_common.py` runs `/v1/query` (pack / generate off) → maps `Chunk` to framework docs / tool strings; each lazy-imports its framework behind an optional extra with a friendly `missing_dependency` +- 13 tests vs real framework base classes; `integrations` CI job; [ADR-0017](docs/adr/ADR-0017-framework-adapters.md), [reference/integrations.md](docs/reference/integrations.md) + +### 3.9 — Webhooks ✅ [#123](https://github.com/officialCodeWork/AgentContextOS/pull/123) + +- New `rag-webhooks` package (rag-core + httpx): `WebhookDispatcher` (tenant fan-out + HMAC signing + backoff retries + span / log), `HttpWebhookSender`, `signing.py` (HMAC-SHA256 over `.`, `verify()` with replay tolerance) +- `rag-core` 0.23 → 0.24 `webhook_types` + `SubscriptionStore` / `WebhookSender` SPIs + noops; emission via the `WebhookPublisher` seam (no emitter depends on rag-webhooks); ingest `ingest.completed`, `PolicyWriter` `audit.policy_violation`; `drift.detected` / `eval.regression` types reserved for Phase 5 +- Gateway `make_webhooks_router()` (CRUD + `/test`), secret returned once then masked; `WebhooksConfig` in rag.yaml; `ragctl webhooks demo`; ~50 tests; [ADR-0018](docs/adr/ADR-0018-outbound-webhooks.md), [reference/webhooks.md](docs/reference/webhooks.md) + +### 3.10 — Admin UI v0 ✅ [#124](https://github.com/officialCodeWork/AgentContextOS/pull/124) + +- Next.js 14 (App Router) + TS + Tailwind operator console (`apps/admin-ui`) to a committed design handoff; hand-rolled shadcn-style primitives (Radix-free, no CLI) on shadcn CSS-var tokens + Geist + lucide — deterministic offline builds +- App shell (collapsible sidebar, tenant switcher, theme toggle) + **9 pages**: Dashboard, Corpora, Connectors, Glossary, Webhooks (one-time secret reveal + send-test), Audit Log (filters + pagination), API Keys (Preview), Tenants (Preview), Config Viewer +- **Live-vs-seed hybrid** (`lib/use-live.ts`): Corpora + Webhooks hit the gateway when `NEXT_PUBLIC_GATEWAY_URL` is set, else seed data + a "Demo data" badge +- 14 vitest + RTL; tsc / lint / build clean; `admin-ui` CI job; [ADR-0019](docs/adr/ADR-0019-admin-ui.md), [reference/admin-ui.md](docs/reference/admin-ui.md) + +### 3.11 — Status & Metrics GUI (MVP) ✅ [#125](https://github.com/officialCodeWork/AgentContextOS/pull/125) + +- `rag-observability` read-side: `MetricsCollector` (counter / gauge / histogram, reservoir p50/p95/p99, `snapshot()`) + `LogTail` + `RingBufferLogHandler` (poll by `seq` cursor) +- `rag_gateway.status` (`make_status_router`): REST `/v1/status/health` (per-component roll-up from real error rates) / `metrics` / `logs` / `connectors/status`; **SSE** logs stream; **WebSocket** health + metrics push (ADR-0020) +- Request-timing middleware feeds the metrics (status surface excluded); CORS for the console; `build_app` injectables; gateway 0.7 → 0.8 +- Admin **Observability** nav: Live Status (WS) / Metrics (WS) / Logs (SSE) + Dashboard banner; 45 tests; [ADR-0020](docs/adr/ADR-0020-status-transports.md), [reference/status-api.md](docs/reference/status-api.md) · *full 12-page GUI is Step 5.6* + +## Phase 4 — Reliability (Weeks 18–22) ✅ + +| Step | Title | Status | PR | +|------|-------|:------:|----| +| 4.1 | Semantic cache (L0/L1/L2) | ✅ | [#126](https://github.com/officialCodeWork/AgentContextOS/pull/126) | +| 4.2 | Fallback chain | ✅ | [#127](https://github.com/officialCodeWork/AgentContextOS/pull/127) | +| 4.3 | Hallucination guard | ✅ | [#128](https://github.com/officialCodeWork/AgentContextOS/pull/128) | +| 4.4 | Circuit breakers | ✅ | [#129](https://github.com/officialCodeWork/AgentContextOS/pull/129) | +| 4.5 | Quotas & rate limiting | ✅ | [#130](https://github.com/officialCodeWork/AgentContextOS/pull/130) | +| 4.6 | Latency tuning & load test | ✅ | [#131](https://github.com/officialCodeWork/AgentContextOS/pull/131) | + +### 4.1 — Semantic cache (L0/L1/L2) ✅ [#126](https://github.com/officialCodeWork/AgentContextOS/pull/126) + +- New `rag-cache` package: `TtlLruCache` + L0 (`InMemory{Retrieval,Answer,Embedding}Cache`), L2 similarity (`Semantic{Retrieval,Answer}Cache` + `QuerySimilarityIndex`), `Tiered{Retrieval,Answer}Cache` (read-through + promotion, write-through, `cache.*` events + `TierStats`) +- L1 `Redis{Retrieval,Answer}Cache` in `rag-backends` (tag-set invalidation), injected; **reliability:** L1 failures degrade, never raise +- Gateway `/v1/{query,retrieve}` consult the cache (`served_from_cache`); `CacheConfig` tiered knobs; cross-tenant red-team suite +- [ADR-0021](docs/adr/ADR-0021-tiered-cache.md), [reference/cache.md](docs/reference/cache.md) + +### 4.2 — Fallback chain ✅ [#127](https://github.com/officialCodeWork/AgentContextOS/pull/127) + +- `rag_retrieval.fallback.FallbackChain` wrapping `RetrievalRouter` — degradation ladder hybrid → BM25-only → keyword → "no answer" +- Triggers: proactive budget (`CostEstimator` start-tier, ADR-0008) + reactive `RetrievalError` / empty-below-`min_results`; graceful `no_answer` (HTTP 200, `no_answer_on_exhaustion` toggle); keyword rung relaxes caller filters only (ACL pushdown preserved) +- `SupportsRoute` drop-in (CorpusRouter relaxed) + `build_app` default wrap; `FallbackTier` / `FallbackTrigger` / `FallbackResult`; `retrieve.fallback` span + `fallback.engaged` event +- `ragctl fallback`; 39 tests; [architecture/fallback-chain.md](docs/architecture/fallback-chain.md), [reference/fallback.md](docs/reference/fallback.md) + +### 4.3 — Hallucination guard ✅ [#128](https://github.com/officialCodeWork/AgentContextOS/pull/128) + +- New core-only `rag-guard` package: `HallucinationGuard` (claim split → per-claim NLI vs. evidence → per-tenant threshold → annotate / redact / block); `NLIScorer` SPI (mirrors `Reranker`) + `NoopNLIScorer` + dependency-free `LexicalNLIScorer` default +- Frozen types `NLILabel` / `NLIScore` / `GuardVerdict` / `GuardAction` / `ClaimVerdict` / `GuardResult` + `GuardError`; runs after `egress_text` on `/v1/query` + OpenAI chat (sync block ⇒ `content_filter`; stream advisory) + MCP `query`; `Answer.guard` + `RagAnnotations.guard` +- **Disabled by default;** reliability degrade-never-raise (`guard.scorer_degraded`); `guard.check` span + PII-safe `guard.claim_blocked` event +- `cfg.guard`; `ragctl guard`; [ADR-0022](docs/adr/ADR-0022-hallucination-guard.md), [reference/guard.md](docs/reference/guard.md) + +### 4.4 — Circuit breakers ✅ [#129](https://github.com/officialCodeWork/AgentContextOS/pull/129) + +- New core-only `rag-breaker` package: `CircuitBreaker` (3-state closed / open / half-open, consecutive-failure trip + timed half-open probe, injected clock), `BreakerRegistry` (per-backend, force-close) +- `Breaker{Vector,Keyword,Graph}RetrievalBackend` SPI wrappers — an open breaker raises `CircuitOpenError`, which `HybridRetriever`'s `gather(return_exceptions=True)` already drops (zero fan-out changes); generalises the 2.10 health tracker +- `CircuitState` / `BreakerSnapshot` + `CircuitOpenError`; PII-free `breaker.opened` event; no per-call span; **on by default** +- Gateway `GET /v1/status/breakers` + `POST .../{name}/force-close` + health roll-up; admin-ui Live Status breakers card; `cfg.breakers`; `ragctl breaker`; [ADR-0023](docs/adr/ADR-0023-circuit-breakers.md), [reference/breaker.md](docs/reference/breaker.md) + +### 4.5 — Quotas & rate limiting ✅ [#130](https://github.com/officialCodeWork/AgentContextOS/pull/130) + +- New `rag-quota` package enforcing per-tenant quotas **through the PolicyEngine PDP** (ADR-0005): `QuotaPolicyEngine` (answers `rate_limit` / `quota_check`) over a `QuotaEnforcer` (5 dims — QPS + monthly tokens / cost / queries + storage gauge; micro-dollar cost; snapshot / reset) +- `QuotaStore` SPI — weighted two-slot sliding-window `InMemoryQuotaStore` + atomic-Lua `RedisQuotaStore` (rag-backends, injected) + `NoopQuotaStore`; `QuotaVerdict` / `QuotaSnapshot` / `QuotaDimension` + `QuotaExceededError(RateLimitError)` +- Wired on every surface via `build_app(quota_enforcer=)` + `quota_guard` (REST query / retrieve, OpenAI chat + embeddings, ingest storage); 429 + `Retry-After` pre-check, post-response metering. **Disabled by default** (can reject); **fail-open by default** (`quota.store_degraded`) +- `GET /v1/status/quotas` + reset + health component; admin-ui Quotas card; `ragctl quota`; ~82 tests + 2 vitest; [ADR-0024](docs/adr/ADR-0024-quotas-rate-limiting.md), [reference/quota.md](docs/reference/quota.md) + +### 4.6 — Latency tuning & load test ✅ [#131](https://github.com/officialCodeWork/AgentContextOS/pull/131) + +- Gateway **overhead p99 ≤ 30 ms gate** (default noop backends → per-request wall time *is* the gateway's own cost; distinct from the backend-bound end-to-end target); gate signal is the server-side `gateway.request_duration_ms` from the 3.11 `MetricsCollector` +- Driven in-process via httpx `ASGITransport`, sequentially, retry-tolerant; dedicated single-runner **`perf-gate`** CI job (excluded from the cross-OS sweep via `-m "not perf"`) +- One reusable engine `rag_gateway.perf` (`measure_gateway_overhead` + `profile_gateway`) powers gate + harness + `ragctl perf`; `tests/contract/budgets.py` made real (drift-guarded); load test = in-process `eval/gateway_load_v0` + Locust `locustfile.py` +- ~14 tests; [ADR-0025](docs/adr/ADR-0025-latency-load-testing.md), [reference/perf.md](docs/reference/perf.md), [architecture/latency.md](docs/architecture/latency.md) + +## Phase 5 — Eval & Observability (Weeks 22–28) 🚧 (6 / 7) + +| Step | Title | Status | PR | +|------|-------|:------:|----| +| 5.1 | Per-query tracing & provenance | ✅ | [#132](https://github.com/officialCodeWork/AgentContextOS/pull/132) | +| 5.2 | Offline eval harness (golden set) | ✅ | [#133](https://github.com/officialCodeWork/AgentContextOS/pull/133) | +| 5.3 | CI eval gate | ✅ | [#134](https://github.com/officialCodeWork/AgentContextOS/pull/134) | +| 5.4 | Online metrics & feedback | ✅ | [#135](https://github.com/officialCodeWork/AgentContextOS/pull/135) | +| 5.5 | Drift monitors | ✅ | [#136](https://github.com/officialCodeWork/AgentContextOS/pull/136) | +| 5.6 | Status & Metrics GUI (full build) | ✅ | [#137–#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) | +| 5.6a | — Drift & Feedback console cards | ✅ | [#137](https://github.com/officialCodeWork/AgentContextOS/pull/137) | +| 5.6b | — Query Trace viewer | ✅ | [#138](https://github.com/officialCodeWork/AgentContextOS/pull/138) | +| 5.6c | — Cost anomaly | ✅ | [#139](https://github.com/officialCodeWork/AgentContextOS/pull/139) | +| 5.6d | — Regression bisector | ✅ | [#140](https://github.com/officialCodeWork/AgentContextOS/pull/140) | +| 5.6e | — Grafana dashboards | ✅ | [#141](https://github.com/officialCodeWork/AgentContextOS/pull/141) | +| 5.6f | — Cross-links + close-out | ✅ | [#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) | +| 5.7 | A/B testing & shadow mode | 🚧 | — | +| 5.7a | — A/B analyzer + tracker + dashboard | ✅ | [#143](https://github.com/officialCodeWork/AgentContextOS/pull/143) | +| 5.7b | — Shadow mode | ⏳ | — | +| 5.7c | — A/B routing | ⏳ | — | +| 5.7d | — Console + close-out | ⏳ | — | + +### 5.1 — Per-query tracing & provenance ✅ [#132](https://github.com/officialCodeWork/AgentContextOS/pull/132) + +- Stable span attributes: `rag.schema_version` stamped at the `span_from_trace_context` choke point + a `telemetry_attrs` registry/contract +- HMAC-signed `ProvenanceRecord` — new `rag-provenance` package, privacy-by-default hashes, degrade-open +- `GET /v1/query/{id}/trace` — signed record + `TraceCollector`-captured span tree, tenant-scoped +- [ADR-0026](docs/adr/ADR-0026-per-query-tracing-provenance.md), [reference/provenance.md](docs/reference/provenance.md), [architecture/per-query-tracing.md](docs/architecture/per-query-tracing.md) + +### 5.2 — Offline eval harness (golden set) ✅ [#133](https://github.com/officialCodeWork/AgentContextOS/pull/133) + +- 500 queries / 5 domains (committed `tests/eval/golden/`, drift-gated) run through the real `HybridRetriever` over a synthetic corpus (noop SPIs + `HashingEmbedder`, deterministic) +- Metrics: Recall@k / MRR / **nDCG@k** / **Faithfulness** (dependency-free `lexical_faithfulness`, RAGAS optional) / Citation Precision — overall + per-domain + per-difficulty +- `render_html_report` HTML + `report.json` / `report.md`; `eval/golden_set_v0/` harness + `--check` threshold floor; `ragctl eval run --html` +- [ADR-0027](docs/adr/ADR-0027-offline-eval-harness.md), [reference/eval-harness.md](docs/reference/eval-harness.md), [architecture/golden-set-eval.md](docs/architecture/golden-set-eval.md) + +### 5.3 — CI eval gate ✅ [#134](https://github.com/officialCodeWork/AgentContextOS/pull/134) + +- Standalone always-run `eval-gate.yml` runs the harness, compares vs committed `tests/eval/baselines/main.json` (compact byte-stable `EvalBaseline`) +- Enforces `thresholds.yaml` floors **+ regression deltas** (recall@10 / mrr / faithfulness gate; acl / pii are the red-team suites'); sticky-marker PR comment with a diff table (even on failure, fork-PR tolerant) +- Comparison in `rag_config.eval` (`compare_to_baseline` / `render_gate_comment`); result types in `rag_core.eval`; runner `python -m eval.golden_set_v0.gate` +- [ADR-0028](docs/adr/ADR-0028-ci-eval-gate.md), [architecture/ci-eval-gate.md](docs/architecture/ci-eval-gate.md) + +### 5.4 — Online metrics & feedback ✅ [#135](https://github.com/officialCodeWork/AgentContextOS/pull/135) + +- New `rag-feedback` package: `FeedbackRecorder` (normalise signal → `[-1,1]` score → PII-redact comment via an injected `PIIDetector` → tenant-scoped `FeedbackStore.put` → `feedback.recorded`; degrade-open) + pure `aggregate_feedback` → `FeedbackStats` +- One polymorphic `POST /v1/feedback` (explicit + implicit `FeedbackSignal`, body identity, redact-don't-hash, acks `stored=false` when inert / degraded) + per-tenant `GET /v1/status/feedback` dashboard +- `FeedbackRecord` / `FeedbackStats` + `FeedbackRequest` / `FeedbackAck`; `cfg.feedback`; `ragctl feedback` +- [ADR-0029](docs/adr/ADR-0029-online-feedback.md), [reference/feedback.md](docs/reference/feedback.md), [architecture/online-feedback.md](docs/architecture/online-feedback.md) + +### 5.5 — Drift monitors ✅ [#136](https://github.com/officialCodeWork/AgentContextOS/pull/136) + +- New `rag-drift` package: `population_stability_index` (pure binned PSI) + `DriftMonitor` (ref / current windows, PSI or mean-drop, tri-state verdict, rebaseline) + `DriftMonitorRegistry` (five monitors) +- Monitors: query distribution / embedding PSI / retrieval score (via PSI) + citation-clickthrough / faithfulness (via mean-drop); infra-scoped, fed via `observe` from the query + feedback paths +- `evaluate` emits `drift.detected` (event + Step 3.9 webhook) on the transition into drift; `DriftSnapshot` / `DriftReport`; `cfg.drift`; `GET /v1/status/drift` + `POST .../{metric}/rebaseline`; `ragctl drift` +- [ADR-0030](docs/adr/ADR-0030-drift-monitors.md), [reference/drift.md](docs/reference/drift.md), [architecture/drift-monitors.md](docs/architecture/drift-monitors.md) + +### 5.6 — Status & Metrics GUI (full build) ✅ [#137–#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) + +- Delivered in **six vertical slices (5.6a–f)**, detailed below: Drift / Feedback + Cost cards on Live Status, the Query Trace viewer page, the regression bisector, drift + cost Grafana dashboards, observability cross-links +- Deferred (documented): feedback / breaker / quota Grafana export, a Loki-events dashboard, per-domain eval gating + +#### 5.6a — Drift & Feedback console cards ✅ [#137](https://github.com/officialCodeWork/AgentContextOS/pull/137) + +- Surface `GET /v1/status/drift` (per-monitor verdict + one-click rebaseline) and `GET /v1/status/feedback` (mean score, sentiment split, signal counts) as two **Live Status** cards (`useLive` live-vs-seed) +- New TS types + `fetchDrift` / `rebaselineDrift` / `fetchFeedback` + seeds; 3 vitest; browser-verified + +#### 5.6b — Query Trace viewer ✅ [#138](https://github.com/officialCodeWork/AgentContextOS/pull/138) + +- New **Query Trace** page (`/trace`): paste a `request_id` → `GET /v1/query/{id}/trace` renders the signed provenance record, an HMAC **verification pill**, evidence citations, and the captured span tree as an offset / duration **waterfall** +- Input-driven fetch + seed fallback + 404 path; `QueryTrace` / `SpanRecord` / `Provenance*` types + `fetchTrace`; 3 vitest + +#### 5.6c — Cost anomaly ✅ [#139](https://github.com/officialCodeWork/AgentContextOS/pull/139) + +- New `CostTracker` (`rag-observability`) — bounded per-tenant rolling window of recent per-request token costs; two-gate (ratio + z-score) tri-state verdict; **scale-free on tokens** (decoupled from quota pricing); fed O(1) from `record_request_usage`, independent of quotas +- Pull-based `GET /v1/status/cost` (no per-request span / event); `CostSnapshot` a `dataclass` → no `rag-core` / `dist/schemas` churn; admin Cost-anomaly card; 16 tests + 1 vitest +- [ADR-0031](docs/adr/ADR-0031-cost-anomaly.md), [reference/cost.md](docs/reference/cost.md), [architecture/cost-anomaly.md](docs/architecture/cost-anomaly.md) + +#### 5.6d — Regression bisector ✅ [#140](https://github.com/officialCodeWork/AgentContextOS/pull/140) + +- `python -m eval.golden_set_v0.bisect --good --bad HEAD --metric --threshold ` — **binary-search `good..bad`** (O(log n) harness runs) for the first commit that dropped a metric +- Each candidate runs in a throwaway `git worktree` with that commit's `packages/*/src` on `PYTHONPATH`; pure `bisect_commits` → `BisectResult` / `BisectStep` (additive); 7 unit tests + live worktree smoke +- [reference/eval-harness.md](docs/reference/eval-harness.md#regression-bisector-step-56d) + +#### 5.6e — Grafana dashboards ✅ [#141](https://github.com/officialCodeWork/AgentContextOS/pull/141) + +- `rag_observability.register_platform_metrics` registers OTel observable gauges — `rag.drift.{statistic,threshold,drifted}` (from `report()`) + `rag.cost.{ratio,elevated,recent_mean_tokens}` (from `CostTracker.snapshot_all`) — on the existing OTel → collector → Prometheus pipeline; registered once by `build_app_from_config` +- Provisioned `rag-quality-cost` Grafana dashboard; proven by an OTel `InMemoryMetricReader` test + a dashboard-validation gate (`tests/infra/test_grafana_dashboards.py`) +- [guides/grafana-dashboards.md](docs/guides/grafana-dashboards.md) + +#### 5.6f — Cross-links + close-out ✅ [#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) + +- Wires the observability surfaces together: the Trace viewer's *Logs* button → `/logs?trace=` (clearable violet trace-chip), a log detail drawer → *View this trace's logs*, a **Trace** quick-link on Live Status +- Admin-ui only; 1 vitest cross-link spec; closes Step 5.6 + +### 5.7 — A/B testing & shadow mode 🚧 + +- Final Phase-5 step, delivered in vertical slices **5.7a–d**: analyzer + tracker + dashboard (5.7a ✅), shadow mode (5.7b), A/B routing (5.7c), console + close-out (5.7d) +- Shadow N% of live queries; A/B variant routing; statistical analyzer with lift + confidence interval + +#### 5.7a — A/B analyzer + tracker + dashboard ✅ [#143](https://github.com/officialCodeWork/AgentContextOS/pull/143) + +- Pure `analyze_ab_experiment` (`rag_config.eval`) — lift + normal-approx Welch CI + significance via stdlib `statistics.NormalDist` (no numpy / scipy) → `ABAnalysisResult` (`rag_core.eval`, additive, not in `dist/schemas`) +- `ABExperimentTracker` (`rag-observability`, beside `CostTracker`) — a pure per-`(experiment, variant)` sample holder, decoupled from the analyzer (observability keeps no `rag_config` dep); the gateway composes the two at `GET /v1/status/experiments` +- `cfg.experiments` (opt-in, since A/B routing can change responses); wired inert by default; **observe-only** — no query-path change yet (shadow / routing are 5.7b/c); `ragctl experiments`; 17 tests +- [ADR-0032](docs/adr/ADR-0032-ab-testing-shadow-mode.md), [reference/experiments.md](docs/reference/experiments.md) + +#### 5.7b — Shadow mode ⏳ + +- **Next up.** Fan out N% of live queries to a candidate retriever (observe-only) and feed the A/B tracker + +## Phase 6 — Governance & Tenancy (Weeks 28–34) ⏳ + +| Step | Title | Status | Planned deliverables | +|------|-------|:------:|----------------------| +| 6.1 | Logical multi-tenancy | ⏳ | Namespace isolation; per-tenant config, quotas, ACLs in `rag.yaml` | +| 6.2 | Physical tenancy (dedicated index) | ⏳ | Dedicated vector index per tenant; cross-tenant probe gate | +| 6.3 | ACL push-down at retrieval | ⏳ | ACL filter injected into every vector / BM25 / graph query; `acl.egress_denied` event | +| 6.4 | ACL egress verifier | ⏳ | Post-retrieval re-check; defense-in-depth; zero-ACL-violation-rate gate | +| 6.5 | PII policies | ⏳ | Per-tenant PII enforcement (block / redact / allow); egress redaction; `pii.egress_blocked` event | +| 6.6 | Immutable audit log | ⏳ | Hash-chain audit log; WORM export; tamper-evident verification; `GET /v1/audit` | +| 6.7 | BYOK (Bring Your Own Key) | ⏳ | KMS integration (AWS KMS, GCP KMS, HashiCorp Vault); envelope encryption for embeddings | +| 6.8 | SSO / SCIM | ⏳ | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config | +| 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification | +| 6.10 | Compliance posture | ⏳ | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies | --- -## Phase 7 — Pilot, Harden, GA (Weeks 34–40) - -| Step | Title | Status | Branch | PR | Key Deliverables | -|------|-------|--------|--------|----|-----------------| -| 7.1 | Load testing | ⏳ | — | — | Locust suite; 1000 QPS sustained; p99 < 500 ms; chaos under load | -| 7.2 | Chaos engineering | ⏳ | — | — | Chaos Monkey / LitmusChaos; kill each backend; verify fallback chain holds | -| 7.3 | Red-team | ⏳ | — | — | Prompt injection, ACL bypass, PII egress, tenant escape probes | -| 7.4 | Design partner onboarding | ⏳ | — | — | 2–3 design partners; feedback incorporated; case study documented | -| 7.5 | Documentation site | ⏳ | — | — | Docusaurus/MkDocs site; API reference generated from OpenAPI; quickstart guides | -| 7.6 | Marketplace listings | ⏳ | — | — | AWS Marketplace, Azure Marketplace, GCP Marketplace AMI/Helm listings | -| 7.7 | Packaging & distribution | ⏳ | — | — | PyPI publish (`rag-platform`), npm publish (`@ragplatform/sdk`), Docker Hub images | -| 7.8 | Support & SLA | ⏳ | — | — | Support tiers defined; SLA dashboards; PagerDuty integration; runbooks | -| 7.9 | Billing integration | ⏳ | — | — | Stripe metered billing; usage export API; invoice generation | -| 7.10 | GA cutover | ⏳ | — | — | `main` tag `v1.0.0`; release notes; all Phase 7 exit gates passed | +## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳ + +| Step | Title | Status | Planned deliverables | +|------|-------|:------:|----------------------| +| 7.1 | Load testing | ⏳ | Locust suite; 1000 QPS sustained; p99 < 500 ms; chaos under load | +| 7.2 | Chaos engineering | ⏳ | Chaos Monkey / LitmusChaos; kill each backend; verify the fallback chain holds | +| 7.3 | Red-team | ⏳ | Prompt injection, ACL bypass, PII egress, tenant-escape probes | +| 7.4 | Design partner onboarding | ⏳ | 2–3 design partners; feedback incorporated; case study documented | +| 7.5 | Documentation site | ⏳ | Docusaurus / MkDocs site; API reference generated from OpenAPI; quickstart guides | +| 7.6 | Marketplace listings | ⏳ | AWS / Azure / GCP Marketplace AMI / Helm listings | +| 7.7 | Packaging & distribution | ⏳ | PyPI publish (`rag-platform`), npm publish (`@ragplatform/sdk`), Docker Hub images | +| 7.8 | Support & SLA | ⏳ | Support tiers defined; SLA dashboards; PagerDuty integration; runbooks | +| 7.9 | Billing integration | ⏳ | Stripe metered billing; usage export API; invoice generation | +| 7.10 | GA cutover | ⏳ | `main` tag `v1.0.0`; release notes; all Phase 7 exit gates passed | --- ## PR & Branch History -| PR | Title | Branch | Status | Merged | -|----|-------|--------|--------|--------| -| [#1](https://github.com/officialCodeWork/AgentContextOS/pull/1) | Problems catalog | `planning/V1/problems-1` | ✅ Merged | 2026-05-22 | -| [#2](https://github.com/officialCodeWork/AgentContextOS/pull/2) | V1 execution plan + logging standard + GUI spec + doc restructure | `planning/V1/execution-plan` | ✅ Merged | 2026-05-22 | -| [#3](https://github.com/officialCodeWork/AgentContextOS/pull/3) | Phase 0 Step 0.1 — monorepo and build system | `build/phase-0/step-0.1-monorepo` | ✅ Merged | 2026-05-22 | -| [#4](https://github.com/officialCodeWork/AgentContextOS/pull/4) | Phase 0 Step 0.2 — core domain types & errors | `build/phase-0/step-0.2-core-types` | ✅ Merged | 2026-05-22 | -| [#5](https://github.com/officialCodeWork/AgentContextOS/pull/5) | Backwards no-op (duplicate of #4) | `main` | ❌ Closed | 2026-05-22 | -| [#6](https://github.com/officialCodeWork/AgentContextOS/pull/6) | Phase 0 Step 0.2c — cross-platform support | `build/phase-0/step-0.2c-cross-platform` | ✅ Merged | 2026-05-22 | -| [#7](https://github.com/officialCodeWork/AgentContextOS/pull/7) | Phase 0 Step 0.3 — Plugin SPI interfaces | `build/phase-0/step-0.3-plugin-spi` | ✅ Merged | 2026-05-22 | -| [#8](https://github.com/officialCodeWork/AgentContextOS/pull/8) | Phase 0 Step 0.4 — rag.yaml schema & loader | `build/phase-0/step-0.4-rag-yaml-schema-loader` | ✅ Merged | 2026-05-22 | -| [#9](https://github.com/officialCodeWork/AgentContextOS/pull/9) | Phase 0 Step 0.5 — CI/CD pipeline hardening | `build/phase-0/step-0.5-ci-cd-hardening` | ✅ Merged | 2026-05-22 | -| #10–#24 | Dependabot dependency bumps (GH Actions, Docker, npm, Python dev deps) | various | ✅ Merged | 2026-05-23 | -| [#25](https://github.com/officialCodeWork/AgentContextOS/pull/25) | chore: release main (Release Please auto-PR) | `release-please--branches--main` | ✅ Merged | 2026-05-23 | -| [#26](https://github.com/officialCodeWork/AgentContextOS/pull/26) | Phase 0 Step 0.6 — local dev stack | `build/phase-0/step-0.6-local-dev-stack` | ✅ Merged | 2026-05-23 | -| [#27](https://github.com/officialCodeWork/AgentContextOS/pull/27) | feat(core): OTel tracing + SPI metrics foundation (Step 0.7) | `build/phase-0/step-0.7-tracing-metrics` | ✅ Merged | 2026-05-23 | -| [#28](https://github.com/officialCodeWork/AgentContextOS/pull/28) | chore: release main (Release Please auto-PR) | `release-please--branches--main` | 🟡 Open | — | -| [#29](https://github.com/officialCodeWork/AgentContextOS/pull/29) | feat(core): structured logging foundation (Step 0.7b) | `build/phase-0/step-0.7b-structured-logging` | ✅ Merged | 2026-05-23 | -| [#30](https://github.com/officialCodeWork/AgentContextOS/pull/30) | refactor(observability): extract packages/observability with full log schema (Step 0.7b follow-up) | `build/phase-0/step-0.7b-observability-package` | ✅ Merged | 2026-05-23 | -| [#31](https://github.com/officialCodeWork/AgentContextOS/pull/31) | feat(core): audit log skeleton — AuditStore SPI + hash-chain noop + AuditWriter (Step 0.7c) | `build/phase-0/step-0.7c-audit-log-skeleton` | ✅ Merged | 2026-05-23 | -| [#32](https://github.com/officialCodeWork/AgentContextOS/pull/32) | docs: add CLAUDE.md with project context and per-PR documentation requirement | `build/phase-0/step-0.7c-audit-log-skeleton` | ✅ Merged | 2026-05-23 | -| [#33](https://github.com/officialCodeWork/AgentContextOS/pull/33) | feat(eval): eval skeleton — golden-set schema, metrics, RAGAS spike, ragctl eval (Step 0.8) | `build/phase-0/step-0.8-eval-skeleton` | ✅ Merged | 2026-05-23 | -| [#34](https://github.com/officialCodeWork/AgentContextOS/pull/34) | feat(infra): IaC foundation — Terraform modules + Helm chart skeleton (Step 0.9) | `build/phase-0/step-0.9-iac-foundation` | ✅ Merged | 2026-05-23 | -| [#35](https://github.com/officialCodeWork/AgentContextOS/pull/35) | chore(tracker): sync PR links for steps 0.7b–0.9 and history #28–#34 | `chore/tracker-sync-pr28-34` | ✅ Merged | 2026-05-23 | -| [#36](https://github.com/officialCodeWork/AgentContextOS/pull/36) | feat(ragctl): consolidated control-plane CLI scaffold (Step 0.10) | `build/phase-0/step-0.10-ragctl-cli-scaffold` | ✅ Merged | 2026-05-23 | -| [#38](https://github.com/officialCodeWork/AgentContextOS/pull/38) | ci: wire RAG001 logging check into ci.yml on all OSes | `fix/ci-rag001-gate` | ✅ Merged | 2026-05-23 | -| [#40](https://github.com/officialCodeWork/AgentContextOS/pull/40) | feat(backends): storage backend plugins — pgvector, Qdrant, Redis, S3 (Step 1.1) | `build/phase-1/step-1.1-storage-backends` | ✅ Merged | 2026-05-24 | -| [#41](https://github.com/officialCodeWork/AgentContextOS/pull/41) | docs(planning): Phase 1 architecture-refactor window (Steps 1.1a–1.1f) + ADRs 0005–0009 | `planning/phase-1-architecture-refactor` | ✅ Merged | 2026-05-24 | -| [#44](https://github.com/officialCodeWork/AgentContextOS/pull/44) | refactor(core): RequestContext + ctx-threaded SPI (Step 1.1a) | `build/phase-1/step-1.1a-core-type-spi-refactor` | ✅ Merged | 2026-05-24 | -| [#45](https://github.com/officialCodeWork/AgentContextOS/pull/45) | refactor(core): SPI split — RetrievalBackend / IndexBackend (Step 1.1b) | `build/phase-1/step-1.1b-spi-split-retrieval-index` | ✅ Merged | 2026-05-24 | -| [#46](https://github.com/officialCodeWork/AgentContextOS/pull/46) | feat(policy): rag-policy package — PolicyEngine PDP + coverage linter (Step 1.1c) | `build/phase-1/step-1.1c-policy-engine-package` | ✅ Merged | 2026-05-24 | -| [#47](https://github.com/officialCodeWork/AgentContextOS/pull/47) | chore(schemas): track generated JSON Schemas + add drift CI gate | `chore/track-generated-schemas-drift-gate` | ✅ Merged | 2026-05-24 | -| [#48](https://github.com/officialCodeWork/AgentContextOS/pull/48) | chore(ci): lint scripts/ + ignore S603/S607 in scripts/** | `chore/lint-ignore-subprocess-in-scripts` | ✅ Merged | 2026-05-24 | -| [#49](https://github.com/officialCodeWork/AgentContextOS/pull/49) | chore(tracker): sync PR links for steps 1.1a/1.1c + log #46–#48 | `chore/tracker-sync-pr44-48` | ✅ Merged | 2026-05-24 | -| [#50](https://github.com/officialCodeWork/AgentContextOS/pull/50) | feat(core): Pipeline + Batcher primitives (Step 1.1d) | `build/phase-1/step-1.1d-pipeline-batcher-primitives` | ✅ Merged | 2026-05-24 | -| [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) | feat(core,observability): Cache SPI split + async telemetry sink (Step 1.1e) | `build/phase-1/step-1.1e-cache-split-perf-telemetry` | ✅ Merged | 2026-05-24 | -| [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) | docs(adr,architecture): finalize ADRs 0005–0009 + extend reviewer checklist (Step 1.1f) | `build/phase-1/step-1.1f-adrs-reviewer-checklist` | ✅ Merged | 2026-05-24 | -| [#53](https://github.com/officialCodeWork/AgentContextOS/pull/53) | chore(tracker): sync Step 1.1f → ✅ and log PR #52 | `chore/tracker-sync-pr52` | ✅ Merged | 2026-05-24 | -| [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) | feat(core,backends): connectors framework — SPI evolution + filesystem/S3/GCS (Step 1.2) | `build/phase-1/step-1.2-connectors-framework` | ✅ Merged | 2026-05-24 | -| [#56](https://github.com/officialCodeWork/AgentContextOS/pull/56) | chore(tracker): sync Step 1.2 → ✅ and log PR #55 | `chore/tracker-sync-pr55` | ✅ Merged | 2026-05-24 | -| [#57](https://github.com/officialCodeWork/AgentContextOS/pull/57) | feat(core,parsers): document parsers — PDF/DOCX/PPTX/XLSX/HTML/MD/text/JSON/CSV/YAML (Step 1.3) | `build/phase-1/step-1.3-document-parsers` | ✅ Merged | 2026-05-24 | -| [#58](https://github.com/officialCodeWork/AgentContextOS/pull/58) | chore(tracker): sync Step 1.3 → ✅ and log PRs #56, #57 | `chore/tracker-sync-pr57` | ✅ Merged | 2026-05-25 | -| [#59](https://github.com/officialCodeWork/AgentContextOS/pull/59) | fix(parsers): normalize CRLF/CR to LF in text & markdown parsers | `fix/parsers-windows-crlf` | ✅ Merged | 2026-05-25 | -| [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) | feat(core,ocr): OCR pipeline — region-aware SPI + Tesseract + PaddleOCR (Step 1.4) | `build/phase-1/step-1.4-ocr-pipeline` | ✅ Merged | 2026-05-25 | -| [#61](https://github.com/officialCodeWork/AgentContextOS/pull/61) | chore(tracker): sync Step 1.4 → ✅ and log PRs #58, #59, #60 | `chore/tracker-sync-pr60` | ✅ Merged | 2026-05-25 | -| [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) | feat(core,chunker): structure-aware chunker (Step 1.5) | `build/phase-1/step-1.5-structure-aware-chunker` | ✅ Merged | 2026-05-25 | -| [#63](https://github.com/officialCodeWork/AgentContextOS/pull/63) | chore(tracker): sync Step 1.5 → ✅ and log PRs #61, #62 | `chore/tracker-sync-pr62` | ✅ Merged | 2026-05-25 | -| [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) | feat(core,enricher): metadata enricher (Step 1.6) | `build/phase-1/step-1.6-metadata-enricher` | ✅ Merged | 2026-05-25 | -| [#65](https://github.com/officialCodeWork/AgentContextOS/pull/65) | chore(tracker): sync Step 1.6 → ✅ and log PRs #63, #64 | `chore/tracker-sync-pr64` | ✅ Merged | 2026-05-25 | -| [#66](https://github.com/officialCodeWork/AgentContextOS/pull/66) | feat(core,pii): PII detection + per-tenant policy enforcement (Step 1.7) | `build/phase-1/step-1.7-pii-detection` | ✅ Merged | 2026-05-25 | -| [#67](https://github.com/officialCodeWork/AgentContextOS/pull/67) | chore(tracker): sync Step 1.7 → ✅ and log PRs #65, #66 | `chore/tracker-sync-pr66` | ✅ Merged | 2026-05-25 | -| [#68](https://github.com/officialCodeWork/AgentContextOS/pull/68) | feat(embedders): embedder pipeline — OpenAI / Cohere / BGE / E5 (Step 1.8) | `build/phase-1/step-1.8-embedder-pipeline` | ✅ Merged | 2026-05-25 | -| [#69](https://github.com/officialCodeWork/AgentContextOS/pull/69) | chore(tracker): sync Step 1.8 → ✅ and log PRs #67, #68; advance to 1.10 | `chore/tracker-sync-pr68` | ✅ Merged | 2026-05-25 | -| [#70](https://github.com/officialCodeWork/AgentContextOS/pull/70) | feat(backends): CDC connectors + Document.fingerprint dedup (Step 1.9) | `build/phase-1/step-1.9-cdc-connectors` | ✅ Merged | 2026-05-25 | -| [#71](https://github.com/officialCodeWork/AgentContextOS/pull/71) | chore(tracker): sync Step 1.9 → ✅ and log PRs #69, #70 | `chore/tracker-sync-pr70` | ✅ Merged | 2026-05-25 | -| [#72](https://github.com/officialCodeWork/AgentContextOS/pull/72) | feat(ingest,gateway): write path + POST /v1/ingest (Step 1.10 — CLI demo) | `build/phase-1/step-1.10-write-path-ingest-api` | ✅ Merged | 2026-05-25 | -| [#73](https://github.com/officialCodeWork/AgentContextOS/pull/73) | chore(tracker): mark Phase 1 complete (Step 1.10 → ✅; log PRs #71, #72) | `chore/tracker-sync-pr72` | ✅ Merged | 2026-05-25 | -| [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | feat(core,backends): knowledge store read layer + FilterExpr push-down (Step 2.1) | `build/phase-2/step-2.1-knowledge-store-read-layer` | ✅ Merged | 2026-05-25 | -| [#75](https://github.com/officialCodeWork/AgentContextOS/pull/75) | chore(tracker): sync Step 2.1 → ✅ and log PRs #73, #74 | `chore/tracker-sync-pr74` | ✅ Merged | 2026-05-26 | -| [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | feat(backends): vector retrieval backends — Weaviate / Pinecone / Elasticsearch + IndexHint wiring (Step 2.2) | `build/phase-2/step-2.2-vector-retrieval-backends` | ✅ Merged | 2026-05-26 | -| [#77](https://github.com/officialCodeWork/AgentContextOS/pull/77) | chore(tracker): sync Step 2.2 → log PR #76 + history rows #75, #76 | `chore/tracker-sync-pr76` | ✅ Merged | 2026-05-26 | -| [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) | feat(backends): BM25 keyword retrieval — Elasticsearch + tantivy (Step 2.3) | `build/phase-2/step-2.3-bm25-keyword-retrieval` | ✅ Merged | 2026-05-26 | -| [#83](https://github.com/officialCodeWork/AgentContextOS/pull/83) | chore(tracker): sync Step 2.3 → log PR #82 + history rows #77, #82 | `chore/tracker-sync-pr82` | ✅ Merged | 2026-05-26 | -| [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) | feat(core,backends): graph retrieval — Neo4j + Memgraph + NetworkX + expand() (Step 2.4) | `build/phase-2/step-2.4-graph-retrieval` | ✅ Merged | 2026-05-26 | -| [#85](https://github.com/officialCodeWork/AgentContextOS/pull/85) | chore(tracker): sync Step 2.4 → log PR #84 + history rows #83, #84 | `chore/tracker-sync-pr84` | ✅ Merged | 2026-05-26 | -| [#86](https://github.com/officialCodeWork/AgentContextOS/pull/86) | feat(retrieval): hybrid RRF fusion — rrf_fuse + HybridRetriever (Step 2.5) | `build/phase-2/step-2.5-hybrid-rrf-fusion` | ✅ Merged | 2026-05-26 | -| [#87](https://github.com/officialCodeWork/AgentContextOS/pull/87) | fix(tooling): unblock mypy by removing duplicate package-local tests/__init__.py | `fix/mypy-duplicate-tests-module` | ✅ Merged | 2026-05-26 | -| [#88](https://github.com/officialCodeWork/AgentContextOS/pull/88) | fix(retrieval): cast policy_filter to restore FilterExpr across lazy boundary | `fix/mypy-no-any-return-hybrid-policy-filter` | ✅ Merged | 2026-05-26 | -| [#94](https://github.com/officialCodeWork/AgentContextOS/pull/94) | feat(packer): rag-packer — context packer with dedup, reorder, budget, conflicts (Step 2.8) | `build/phase-2/step-2.8-context-packer` | ✅ Merged | 2026-05-26 | -| [#95](https://github.com/officialCodeWork/AgentContextOS/pull/95) | chore(tracker): sync Step 2.8 → ✅ and log PR #94 | `chore/tracker-sync-pr94` | ✅ Merged | 2026-05-26 | -| [#96](https://github.com/officialCodeWork/AgentContextOS/pull/96) | feat(graphrag): rag-graphrag — community detection + summarisation + graph-aware retrieval (Step 2.9) | `build/phase-2/step-2.9-graphrag` | ✅ Merged | 2026-05-26 | +Complete log of every PR. Routine Dependabot bumps are grouped; everything else is listed individually. + +| PR | Merged | Title | +|----|--------|-------| +| [#1](https://github.com/officialCodeWork/AgentContextOS/pull/1) | 2026-05-21 | Problems To Start With | +| [#2](https://github.com/officialCodeWork/AgentContextOS/pull/2) | 2026-05-21 | Add V1 execution plan, logging standard, GUI spec, and doc restructure | +| [#3](https://github.com/officialCodeWork/AgentContextOS/pull/3) | 2026-05-21 | feat: Phase 0 Step 0.1 — monorepo and build system | +| [#4](https://github.com/officialCodeWork/AgentContextOS/pull/4) | 2026-05-22 | Phase 0 Step 0.2 — Core domain types & errors | +| [#5](https://github.com/officialCodeWork/AgentContextOS/pull/5) | Closed | Duplicate merge of #4 (closed, no-op) | +| [#6](https://github.com/officialCodeWork/AgentContextOS/pull/6) | 2026-05-22 | feat(tooling): Phase 0 Step 0.2c — cross-platform support | +| [#7](https://github.com/officialCodeWork/AgentContextOS/pull/7) | 2026-05-22 | feat(core): Phase 0 Step 0.3 — Plugin SPI interfaces | +| [#8](https://github.com/officialCodeWork/AgentContextOS/pull/8) | 2026-05-22 | feat(config): Phase 0 Step 0.4 — rag.yaml schema & loader | +| [#9](https://github.com/officialCodeWork/AgentContextOS/pull/9) | 2026-05-22 | feat(ci): Phase 0 Step 0.5 — CI/CD pipeline hardening | +| #10–#24 | 2026-05-22 | Dependabot dependency bumps — GitHub Actions / npm / pip (15 PRs) | +| [#25](https://github.com/officialCodeWork/AgentContextOS/pull/25) | 2026-05-23 | chore: release main | +| [#26](https://github.com/officialCodeWork/AgentContextOS/pull/26) | 2026-05-23 | feat(dev): Phase 0 Step 0.6 — local dev stack | +| [#27](https://github.com/officialCodeWork/AgentContextOS/pull/27) | 2026-05-23 | feat(core): OTel tracing + SPI metrics foundation (Step 0.7) | +| [#28](https://github.com/officialCodeWork/AgentContextOS/pull/28) | Open | chore: release main | +| [#29](https://github.com/officialCodeWork/AgentContextOS/pull/29) | 2026-05-23 | feat(core): structured logging foundation (Step 0.7b) | +| [#30](https://github.com/officialCodeWork/AgentContextOS/pull/30) | 2026-05-23 | refactor(observability): extract packages/observability with full log schema (Step 0.7b follow-up) | +| [#31](https://github.com/officialCodeWork/AgentContextOS/pull/31) | 2026-05-23 | feat(core): audit log skeleton — AuditStore SPI + hash-chain noop + AuditWriter (Step 0.7c) | +| [#32](https://github.com/officialCodeWork/AgentContextOS/pull/32) | 2026-05-23 | docs: add CLAUDE.md with project context and per-PR documentation requirement | +| [#33](https://github.com/officialCodeWork/AgentContextOS/pull/33) | 2026-05-23 | feat(eval): eval skeleton — golden-set schema, metrics, RAGAS spike, ragctl eval (Step 0.8) | +| [#34](https://github.com/officialCodeWork/AgentContextOS/pull/34) | 2026-05-23 | feat(infra): IaC foundation — Terraform modules + Helm chart skeleton (Step 0.9) | +| [#35](https://github.com/officialCodeWork/AgentContextOS/pull/35) | 2026-05-23 | chore(tracker): sync PR links for steps 0.7b–0.9 and history #28–#34 | +| [#36](https://github.com/officialCodeWork/AgentContextOS/pull/36) | 2026-05-23 | feat(ragctl): consolidated control-plane CLI scaffold (Step 0.10) | +| [#37](https://github.com/officialCodeWork/AgentContextOS/pull/37) | 2026-05-23 | fix(logging): allowlist rag_core/audit.py for RAG001 | +| [#38](https://github.com/officialCodeWork/AgentContextOS/pull/38) | 2026-05-23 | ci: wire RAG001 logging check into ci.yml on all OSes | +| [#39](https://github.com/officialCodeWork/AgentContextOS/pull/39) | 2026-05-23 | chore(tracker): backfill PR links for #35, #36, #38 | +| [#40](https://github.com/officialCodeWork/AgentContextOS/pull/40) | 2026-05-23 | feat(backends): storage backend plugins — pgvector, Qdrant, Redis, S3 (Step 1.1) | +| [#41](https://github.com/officialCodeWork/AgentContextOS/pull/41) | 2026-05-23 | docs(planning): Phase 1 architecture-refactor window (Steps 1.1a–1.1f) + ADRs 0005–0009 | +| [#42](https://github.com/officialCodeWork/AgentContextOS/pull/42) | 2026-05-23 | chore(tracker): log PR #41 in history table | +| [#43](https://github.com/officialCodeWork/AgentContextOS/pull/43) | 2026-05-23 | chore(claude): add /nxt slash command to report next tracker item | +| [#44](https://github.com/officialCodeWork/AgentContextOS/pull/44) | 2026-05-23 | feat(core): RequestContext + ctx-threaded SPI (Step 1.1a) | +| [#45](https://github.com/officialCodeWork/AgentContextOS/pull/45) | 2026-05-24 | feat(core): SPI split — RetrievalBackend / IndexBackend (Step 1.1b) | +| [#46](https://github.com/officialCodeWork/AgentContextOS/pull/46) | 2026-05-24 | feat(policy): rag-policy package — PolicyEngine PDP + coverage linter (Step 1.1c) | +| [#47](https://github.com/officialCodeWork/AgentContextOS/pull/47) | 2026-05-24 | chore(schemas): track generated JSON Schemas + add drift CI gate | +| [#48](https://github.com/officialCodeWork/AgentContextOS/pull/48) | 2026-05-24 | chore(ci): lint scripts/ + ignore S603/S607 in scripts/** | +| [#49](https://github.com/officialCodeWork/AgentContextOS/pull/49) | 2026-05-24 | chore(tracker): sync PR links for steps 1.1a/1.1c + log #46–#48 | +| [#50](https://github.com/officialCodeWork/AgentContextOS/pull/50) | 2026-05-24 | feat(core): Pipeline + Batcher primitives (Step 1.1d) | +| [#51](https://github.com/officialCodeWork/AgentContextOS/pull/51) | 2026-05-24 | feat(core,observability): Cache SPI split + async telemetry (Step 1.1e) | +| [#52](https://github.com/officialCodeWork/AgentContextOS/pull/52) | 2026-05-24 | docs(adr,architecture): finalize ADRs 0005–0009 + extend reviewer checklist (Step 1.1f) | +| [#53](https://github.com/officialCodeWork/AgentContextOS/pull/53) | 2026-05-24 | chore(tracker): sync Step 1.1f → ✅ and log PR #52 | +| [#54](https://github.com/officialCodeWork/AgentContextOS/pull/54) | 2026-05-24 | chore(tracker): log PR #53 in history | +| [#55](https://github.com/officialCodeWork/AgentContextOS/pull/55) | 2026-05-24 | feat(core,backends): connectors framework — SPI evolution + filesystem/S3/GCS (Step 1.2) | +| [#56](https://github.com/officialCodeWork/AgentContextOS/pull/56) | 2026-05-24 | chore(tracker): sync Step 1.2 → ✅ and log PR #55 | +| [#57](https://github.com/officialCodeWork/AgentContextOS/pull/57) | 2026-05-24 | feat(core,parsers): document parsers — PDF/DOCX/PPTX/XLSX/HTML/MD/text/JSON/CSV/YAML (Step 1.3) | +| [#58](https://github.com/officialCodeWork/AgentContextOS/pull/58) | 2026-05-25 | chore(tracker): sync Step 1.3 → ✅ and log PRs #56, #57 | +| [#59](https://github.com/officialCodeWork/AgentContextOS/pull/59) | 2026-05-25 | fix(parsers): normalize CRLF/CR to LF in text & markdown parsers | +| [#60](https://github.com/officialCodeWork/AgentContextOS/pull/60) | 2026-05-25 | feat(core,ocr): OCR pipeline — region-aware SPI + Tesseract + PaddleOCR (Step 1.4) | +| [#61](https://github.com/officialCodeWork/AgentContextOS/pull/61) | 2026-05-25 | chore(tracker): sync Step 1.4 → ✅ and log PRs #58, #59, #60 | +| [#62](https://github.com/officialCodeWork/AgentContextOS/pull/62) | 2026-05-25 | feat(core,chunker): structure-aware chunker (Step 1.5) | +| [#63](https://github.com/officialCodeWork/AgentContextOS/pull/63) | 2026-05-25 | chore(tracker): sync Step 1.5 → ✅ and log PRs #61, #62 | +| [#64](https://github.com/officialCodeWork/AgentContextOS/pull/64) | 2026-05-25 | feat(core,enricher): metadata enricher (Step 1.6) | +| [#65](https://github.com/officialCodeWork/AgentContextOS/pull/65) | 2026-05-25 | chore(tracker): sync Step 1.6 → ✅ and log PRs #63, #64 | +| [#66](https://github.com/officialCodeWork/AgentContextOS/pull/66) | 2026-05-25 | feat(core,pii): PII detection + per-tenant policy enforcement (Step 1.7) | +| [#67](https://github.com/officialCodeWork/AgentContextOS/pull/67) | 2026-05-25 | chore(tracker): sync Step 1.7 → ✅ and log PRs #65, #66 | +| [#68](https://github.com/officialCodeWork/AgentContextOS/pull/68) | 2026-05-25 | feat(embedders): embedder pipeline — OpenAI / Cohere / BGE / E5 (Step 1.8) | +| [#69](https://github.com/officialCodeWork/AgentContextOS/pull/69) | 2026-05-25 | chore(tracker): sync Step 1.8 → ✅ and log PRs #67, #68; advance to 1.10 | +| [#70](https://github.com/officialCodeWork/AgentContextOS/pull/70) | 2026-05-25 | feat(backends): CDC connectors + Document.fingerprint dedup (Step 1.9) | +| [#71](https://github.com/officialCodeWork/AgentContextOS/pull/71) | 2026-05-25 | chore(tracker): sync Step 1.9 → ✅ and log PRs #69, #70 | +| [#72](https://github.com/officialCodeWork/AgentContextOS/pull/72) | 2026-05-25 | feat(ingest,gateway): write path + POST /v1/ingest (Step 1.10 — CLI demo milestone) | +| [#73](https://github.com/officialCodeWork/AgentContextOS/pull/73) | 2026-05-25 | chore(tracker): mark Phase 1 complete (Step 1.10 → ✅; log PRs #71, #72) | +| [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | 2026-05-25 | feat(core,backends): knowledge store read layer + FilterExpr push-down (Step 2.1) | +| [#75](https://github.com/officialCodeWork/AgentContextOS/pull/75) | 2026-05-25 | chore(tracker): sync Step 2.1 → ✅ and log PRs #73, #74 | +| [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | 2026-05-25 | feat(backends): Weaviate, Pinecone, Elasticsearch vector backends + IndexHint wiring (Step 2.2) | +| [#77](https://github.com/officialCodeWork/AgentContextOS/pull/77) | 2026-05-26 | chore(tracker): sync Step 2.2 → log PR #76 + history rows #75, #76 | +| [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) | 2026-05-26 | feat(backends): BM25 keyword retrieval — Elasticsearch + tantivy (Step 2.3) | +| [#83](https://github.com/officialCodeWork/AgentContextOS/pull/83) | 2026-05-26 | chore(tracker): sync Step 2.3 → log PR #82 + history rows #77, #82 | +| [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) | 2026-05-26 | feat(core,backends): graph retrieval — Neo4j + Memgraph + NetworkX + expand() (Step 2.4) | +| [#85](https://github.com/officialCodeWork/AgentContextOS/pull/85) | 2026-05-26 | chore(tracker): sync Step 2.4 → log PR #84 + history rows #83, #84 | +| [#86](https://github.com/officialCodeWork/AgentContextOS/pull/86) | 2026-05-26 | feat(retrieval): hybrid RRF fusion — rrf_fuse + HybridRetriever (Step 2.5) | +| [#87](https://github.com/officialCodeWork/AgentContextOS/pull/87) | 2026-05-26 | fix(tooling): unblock mypy by removing duplicate package-local tests/__init__.py | +| [#88](https://github.com/officialCodeWork/AgentContextOS/pull/88) | 2026-05-26 | fix(retrieval): cast policy_filter to restore FilterExpr across lazy boundary | +| [#89](https://github.com/officialCodeWork/AgentContextOS/pull/89) | 2026-05-26 | chore(tracker): sync Step 2.5 → ✅ and log PRs #85–#88 | +| [#90](https://github.com/officialCodeWork/AgentContextOS/pull/90) | 2026-05-26 | feat(query): rag-query — query understanding (rewriter, decomposer, HyDE, glossary) | +| [#91](https://github.com/officialCodeWork/AgentContextOS/pull/91) | 2026-05-26 | chore(tracker): sync Step 2.6 → ✅ and log PR #90 | +| [#92](https://github.com/officialCodeWork/AgentContextOS/pull/92) | 2026-05-26 | feat(reranker): rag-reranker — two-stage cross-encoder + MMR (Step 2.7) | +| [#93](https://github.com/officialCodeWork/AgentContextOS/pull/93) | 2026-05-26 | chore(tracker): sync Step 2.7 → ✅ and log PR #92 | +| [#94](https://github.com/officialCodeWork/AgentContextOS/pull/94) | 2026-05-26 | feat(packer): rag-packer — context packer (dedup + reorder + budget + conflicts) (Step 2.8) | +| [#95](https://github.com/officialCodeWork/AgentContextOS/pull/95) | 2026-05-26 | chore(tracker): sync Step 2.8 → ✅ and log PR #94 | +| [#96](https://github.com/officialCodeWork/AgentContextOS/pull/96) | 2026-05-26 | feat(graphrag): rag-graphrag — community detection + summarisation + graph-aware retrieval (Step 2.9) | +| [#97](https://github.com/officialCodeWork/AgentContextOS/pull/97) | 2026-05-26 | chore(tracker): sync Step 2.9 → ✅ and log PR #96 | +| [#98](https://github.com/officialCodeWork/AgentContextOS/pull/98) | 2026-05-26 | feat(retrieval): retrieval router — query-shape classifier + health tracker + BM25-only fallback (Step 2.10) | +| [#99](https://github.com/officialCodeWork/AgentContextOS/pull/99) | 2026-05-26 | chore(tracker): sync Step 2.10 → ✅ and log PR #98 | +| [#100](https://github.com/officialCodeWork/AgentContextOS/pull/100) | 2026-05-26 | feat(retrieval): agent-loop validation spike — AgentLoopV0 + 50-query harness + gap list (Step 2.11) | +| [#101](https://github.com/officialCodeWork/AgentContextOS/pull/101) | 2026-05-26 | refactor(retrieval): split RetrievalRouter.route into decide + execute (G-02) | +| [#102](https://github.com/officialCodeWork/AgentContextOS/pull/102) | 2026-05-26 | fix(core): reject zero Budget caps at construction (G-03) | +| [#103](https://github.com/officialCodeWork/AgentContextOS/pull/103) | 2026-05-26 | feat(query): cache HyDE embeddings + rewriter outputs (G-06) | +| [#104](https://github.com/officialCodeWork/AgentContextOS/pull/104) | 2026-05-26 | feat(retrieval): semantic embedding cache + ADR-0010 (G-01) | +| [#105](https://github.com/officialCodeWork/AgentContextOS/pull/105) | 2026-05-26 | feat(retrieval): per-stage cost estimator + ADR-0008 backlink (G-04) | +| [#106](https://github.com/officialCodeWork/AgentContextOS/pull/106) | 2026-05-26 | chore(tracker): sync Phase 2 + pre-3.1 fixes complete; log PRs #101–#105 | +| [#107](https://github.com/officialCodeWork/AgentContextOS/pull/107) | 2026-05-27 | feat(gateway): Curl-able RAG — /v1/query + /v1/retrieve + /v1/corpora (Step 3.1) | +| [#108](https://github.com/officialCodeWork/AgentContextOS/pull/108) | 2026-05-27 | chore(tracker): sync PR #107 link for Step 3.1 | +| [#109](https://github.com/officialCodeWork/AgentContextOS/pull/109) | 2026-05-27 | feat(gateway): gRPC service — RagService over rag.gateway.v1 (Step 3.2) | +| [#110](https://github.com/officialCodeWork/AgentContextOS/pull/110) | 2026-05-27 | chore(tracker): sync PR #109 link for Step 3.2 | +| [#111](https://github.com/officialCodeWork/AgentContextOS/pull/111) | 2026-06-01 | feat(gateway): MCP server — query/retrieve/ingest tools (Step 3.3) | +| [#112](https://github.com/officialCodeWork/AgentContextOS/pull/112) | 2026-06-01 | chore(tracker): sync PR #111 link for Step 3.3 | +| [#113](https://github.com/officialCodeWork/AgentContextOS/pull/113) | 2026-06-01 | feat(gateway): OpenAI-compatible endpoints with retrieval pre-fetch (Step 3.4) | +| [#114](https://github.com/officialCodeWork/AgentContextOS/pull/114) | 2026-06-01 | chore(tracker): sync PR #113 link for Step 3.4 | +| [#115](https://github.com/officialCodeWork/AgentContextOS/pull/115) | 2026-06-03 | feat(retrieval): corpus router with static rules, learned classifier & federated fan-out (Step 3.5) | +| [#119](https://github.com/officialCodeWork/AgentContextOS/pull/119) | 2026-06-03 | chore(tracker): sync PR #115 link for Step 3.5 | +| [#120](https://github.com/officialCodeWork/AgentContextOS/pull/120) | 2026-06-04 | feat(agent): production agent runtime over REST SSE + gRPC Converse (Step 3.6) | +| [#121](https://github.com/officialCodeWork/AgentContextOS/pull/121) | 2026-06-04 | feat(sdks): official Python/TypeScript/Go/Java/.NET SDKs (Step 3.7) | +| [#122](https://github.com/officialCodeWork/AgentContextOS/pull/122) | 2026-06-04 | feat(integrations): framework adapters for 8 agent/RAG frameworks (Step 3.8) | +| [#123](https://github.com/officialCodeWork/AgentContextOS/pull/123) | 2026-06-04 | feat(webhooks): signed outbound webhook delivery (Step 3.9) | +| [#124](https://github.com/officialCodeWork/AgentContextOS/pull/124) | 2026-06-04 | feat(admin-ui): Next.js 14 operator console (Step 3.10) | +| [#125](https://github.com/officialCodeWork/AgentContextOS/pull/125) | 2026-06-04 | feat(gateway): status & metrics surface + operator console pages (Step 3.11) | +| [#126](https://github.com/officialCodeWork/AgentContextOS/pull/126) | 2026-06-04 | feat(cache): tiered semantic cache L0/L1/L2 (Step 4.1) | +| [#127](https://github.com/officialCodeWork/AgentContextOS/pull/127) | 2026-06-04 | feat(retrieval): fallback chain (Step 4.2) | +| [#128](https://github.com/officialCodeWork/AgentContextOS/pull/128) | 2026-06-04 | feat(guard): hallucination guard (Step 4.3) | +| [#129](https://github.com/officialCodeWork/AgentContextOS/pull/129) | 2026-06-04 | feat(breaker): per-backend circuit breakers (Step 4.4) | +| [#130](https://github.com/officialCodeWork/AgentContextOS/pull/130) | 2026-06-04 | feat(quota): per-tenant quotas & rate limiting (Step 4.5) | +| [#131](https://github.com/officialCodeWork/AgentContextOS/pull/131) | 2026-06-04 | feat(perf): gateway latency gate, load test & profiling (Step 4.6) | +| [#132](https://github.com/officialCodeWork/AgentContextOS/pull/132) | 2026-06-04 | feat(provenance): per-query tracing & HMAC-signed provenance (Step 5.1) | +| [#133](https://github.com/officialCodeWork/AgentContextOS/pull/133) | 2026-06-05 | feat(eval): offline golden-set eval harness (Step 5.2) | +| [#134](https://github.com/officialCodeWork/AgentContextOS/pull/134) | 2026-06-05 | feat(eval): CI eval gate (Step 5.3) | +| [#135](https://github.com/officialCodeWork/AgentContextOS/pull/135) | 2026-06-05 | feat(feedback): online metrics & feedback (Step 5.4) | +| [#136](https://github.com/officialCodeWork/AgentContextOS/pull/136) | 2026-06-05 | feat(drift): drift monitors (Step 5.5) | +| [#137](https://github.com/officialCodeWork/AgentContextOS/pull/137) | 2026-06-05 | feat(admin-ui): drift & feedback Live Status cards (Step 5.6a) | +| [#138](https://github.com/officialCodeWork/AgentContextOS/pull/138) | 2026-06-05 | feat(admin-ui): query trace & provenance viewer (Step 5.6b) | +| [#139](https://github.com/officialCodeWork/AgentContextOS/pull/139) | 2026-06-05 | feat(cost): per-tenant cost-anomaly detection + console card (Step 5.6c) | +| [#140](https://github.com/officialCodeWork/AgentContextOS/pull/140) | 2026-06-05 | feat(eval): regression bisector over git history (Step 5.6d) | +| [#141](https://github.com/officialCodeWork/AgentContextOS/pull/141) | 2026-06-05 | feat(observability): drift + cost Prometheus metrics + Grafana dashboard (Step 5.6e) | +| [#142](https://github.com/officialCodeWork/AgentContextOS/pull/142) | 2026-06-05 | feat(admin-ui): observability cross-links + Step 5.6 close-out (5.6f) | +| [#143](https://github.com/officialCodeWork/AgentContextOS/pull/143) | 2026-06-05 | feat(experiments): A/B analyzer + tracker + dashboard (Step 5.7a) | +| #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge | +| #81 | Closed | Dependabot bump — superseded | ---