From a8c2c40eb9f48ef1fd742162f715e45fe3e976d8 Mon Sep 17 00:00:00 2001 From: Deep Kumar Singh Kushwah Date: Wed, 10 Jun 2026 15:00:55 +0530 Subject: [PATCH 1/2] =?UTF-8?q?docs:=20GA=20polish=20=E2=80=94=20site=20pu?= =?UTF-8?q?blishing=20scope,=20HLD=20v1.2,=20ragctl=20reference,=20port=20?= =?UTF-8?q?+=20link=20fixes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The seven highest-impact fixes from the GA documentation review: 1. Docs site publishing scope: exclude internal areas (ga/, pilots/) from the public Docusaurus site via an explicit, commented exclude list; links into them from published pages now point at GitHub. 2. HLD refreshed to v1.2: Released status (was "Draft for review"), GA status lines, a "Shipped as" column mapping all 25 problems to their ADR / architecture doc, OpenAI-compat surface corrected, ACL egress verifier recorded as shipped defense-in-depth. 3. docs/reference/ragctl.md rewritten from the real CLI: 37 commands + 13 groups catalogued (was a stale Step 0.10 scaffold doc listing 4); stale module-docstring table in main.py replaced with a doc pointer. 4. Gateway port standardized to 8000 across all guides + reference docs (curl-quickstart, operator-console-live, gateway.md, status-api.md were on 8080). 5. Root README rewritten: GA positioning, a 5-minute first-query path, corrected repo tree, duplicate "Quick start" + stray tree fragment removed. 6. Generated REST reference cross-links: gen_api_reference.py now emits real Markdown links (resolved from docs/reference/, anchors verified against target headings) instead of stripping them; 3 broken ADR links fixed (renamed ADR filenames). 7. CLAUDE.md: stale "Phase 5" status block replaced with GA status + TRACKER.md pointer, stale package versions dropped, repo tree and docs-pipeline note brought current. Verification: pytest tests/docs + packages/ragctl (191 passed), ruff + mypy on touched Python, 1207 relative doc links checked (0 broken), Docusaurus build green with ga/ + pilots/ absent from the output and runbooks/ + adr/ still published. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 63 ++++--- README.md | 113 ++++++------ docs/README.md | 18 +- docs/adr/ADR-0041-airgap-bundle.md | 2 +- docs/adr/ADR-0046-design-partner-pilots.md | 2 +- docs/adr/ADR-0052-ga-cutover.md | 4 +- docs/architecture/RAG-Platform-HLD.md | 74 ++++---- docs/architecture/agent-loop-v0.md | 4 +- docs/guides/curl-quickstart.md | 22 +-- docs/guides/design-partner-pilots.md | 18 +- docs/guides/operator-console-live.md | 16 +- docs/reference/gateway.md | 6 +- docs/reference/pilot.md | 2 +- docs/reference/ragctl.md | 194 +++++++++++++++------ docs/reference/rest-api.md | 10 +- docs/reference/status-api.md | 10 +- docs/release-notes/v1.0.0.md | 3 +- packages/ragctl/src/ragctl/main.py | 35 +--- scripts/gen_api_reference.py | 68 +++++++- website/docusaurus.config.ts | 12 ++ website/sidebars.ts | 9 +- 21 files changed, 423 insertions(+), 262 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 89869f6..def505d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,30 +12,45 @@ so every backend (vector store, embedder, LLM, etc.) is swappable without touchi what is in progress, and what is next. The memory index at `~/.claude/projects/.../memory/MEMORY.md` carries supporting facts between sessions. -Current phase: **Phase 5 — Eval & Observability** (5 of 7 steps complete as of 2026-06-05). Phases 0–4 complete + Steps 5.1–5.5 ✅ delivered (62 of 84 steps total). -Next step: **5.6 — Status & Metrics GUI (full build)** (all 12 GUI pages; Grafana dashboards; regression bisector; drift alerts; cost anomaly; cross-links). Step 5.5 ✅ delivered **drift monitors** — five monitors that catch retrieval degradation before users complain. A new **`rag-drift`** package (deps: rag-core + rag-observability) holds `population_stability_index` (a pure, binned, dependency-free **PSI**), `DriftMonitor` (bounded reference + current windows, **PSI** or **mean-drop** statistic, `ok`/`drifted`/`insufficient_data` verdict), and `DriftMonitorRegistry` (the five monitors — **query distribution / embedding PSI / retrieval score** via PSI over a scalar feature, **citation-clickthrough / faithfulness** via mean-drop). **Infra-scoped** (one registry per deployment, like circuit breakers), held on `app.state`, **fed via `observe`** from signals the gateway already computes: the query path appends query length + top retrieval score + the HyDE-embedding L2 norm + the guard's grounded-claim fraction (`1 − blocked/claims` = faithfulness); the feedback path appends citation-clickthrough (1.0 per `citation_click`, else 0.0). Each observe is a cheap O(1) deque append behind a `None` check — **observe-only**, never alters a request — and sparse-by-default signals report `insufficient_data` until fed. **Detection happens on the dashboard poll**: `GET /v1/status/drift` calls `registry.evaluate()`, computing the `DriftReport` and emitting **`drift.detected`** *once per transition into drift* (a recovery is silent, then re-alerts — `breaker.opened` semantics) as a PII-free structured event **plus** a `drift.detected` **webhook** (the type reserved in Step 3.9) when `cfg.drift.alert_tenant` + a dispatcher are configured. `POST /v1/status/drift/{metric}/rebaseline` is the operator "accept the new normal"; drift folds into `/v1/status/health` (any monitor over threshold → `degraded`). New frozen core types **`DriftSnapshot` / `DriftReport`** + `DriftMetric` / `DriftMethod` / `DriftStatus` StrEnums (`dist/schemas` + `dist/openapi` regenerated); **inert by default** in `build_app`, built from **`cfg.drift`** (`enabled`/`window_size`/`min_samples`/`psi_bins`/`psi_threshold` (0.2)/`mean_drop_threshold` (0.1)/`alert_tenant`) by `build_app_from_config`. Adds no governed SPI call so the PolicyEngine coverage linter passes with no new entry. `ragctl drift` seeds a baseline, feeds a shifted current, and prints the report. ~34 new tests; all gates green (ruff, mypy --strict 288 files, RAG001, log-schema/event-registry + schema/openapi-drift gates). Per-tenant drift, per-dimension embedding PSI, and the admin-console drift card (Step 5.6 GUI) stay deferred. See [`docs/reference/drift.md`](docs/reference/drift.md), [`docs/architecture/drift-monitors.md`](docs/architecture/drift-monitors.md), and [ADR-0030](docs/adr/ADR-0030-drift-monitors.md). Earlier, Step 5.4 ✅ delivered **online metrics & feedback** — a new **`rag-feedback`** package (deps: rag-core + rag-observability, mirroring `rag-provenance`) whose `FeedbackRecorder` takes a feedback submission tied to a query (`request_id`), normalises the signal to a `[-1,1]` score, **PII-redacts** any free-text comment via an injected `PIIDetector`, persists a `FeedbackRecord` through the tenant-scoped `FeedbackStore` SPI (`NoopFeedbackStore`), and emits a PII-free `feedback.recorded` event — all **degrade-open** (a store/redaction failure logs `feedback.record_degraded` and acks `stored=false`). `POST /v1/feedback` is **one polymorphic endpoint** for explicit (`thumb_up`/`thumb_down`/`rating`/`comment`) **and** implicit (`citation_click`/`answer_copied`/`regenerated`/`accepted`/`dwell`) signals — a single `FeedbackSignal` enum, `kind` inferred — with **body identity** like `/v1/query`; the pure `aggregate_feedback` powers the per-tenant `GET /v1/status/feedback` dashboard (`FeedbackStats`). Unlike provenance (which *hashes* text), feedback **redacts-and-keeps** the comment (the default gateway wires a pass-through `NoopPIIDetector` seam; production injects a real one; `comment_redacted` flags whether a redactor ran; the event never carries the text). Inert by default in `build_app` (the broad gateway suite is untouched; the endpoint acks `stored=false`); `build_app_from_config` builds it from `cfg.feedback` (`enabled`/`redact_comments`/`max_comment_chars`/`max_records`). Core types (`FeedbackRecord`/`FeedbackStats`) + wire types (`FeedbackRequest`/`FeedbackAck`) regenerate `dist/schemas`/`dist/openapi`; `ragctl feedback` drives the full record→aggregate flow against noop SPIs; the admin-console feedback card is deferred to Step 5.6. ~45 new tests; all gates green (ruff, mypy --strict 284 files, RAG001, log-schema/event-registry + schema/openapi-drift gates). See [`docs/reference/feedback.md`](docs/reference/feedback.md), [`docs/architecture/online-feedback.md`](docs/architecture/online-feedback.md), and [ADR-0029](docs/adr/ADR-0029-online-feedback.md). Earlier, Step 5.3 ✅ delivered the **CI eval gate** — a standalone, always-run `.github/workflows/eval-gate.yml` runs the Step 5.2 harness on every PR, compares the fresh metrics against the committed compact **`EvalBaseline`** in `tests/eval/baselines/main.json`, enforces the `thresholds.yaml` floors **+ regression deltas** (only **recall@10 / mrr / faithfulness** gate; `acl_violation_rate`/`pii_egress_rate` are the red-team suites'), and posts a sticky-marker PR comment with a baseline-vs-current diff table (posted even on failure, fork-PR tolerant) — failing the build on a floor breach *or* a regression beyond tolerance. The baseline is byte-stable (headline means + per-domain/per-difficulty, **no** `run_id`/`created_at`/`samples`), so it only churns in review when numbers move; the pure comparison + threshold parsing + Markdown renderer (`compare_to_baseline` / `render_gate_comment` / `load_gate_thresholds` / `GateThresholds`) join the metric library in `rag_config.eval`, the result types (`EvalBaseline` / `MetricDelta` / `EvalComparison`) are additive in `rag_core.eval` (not in `dist/schemas`), and the CI runner is `python -m eval.golden_set_v0.gate` (`--comment-out` / `--update-baseline` to refresh the baseline, maintainer-reviewed). Step 5.2 earlier delivered the **offline golden-set eval harness** — `python -m eval.golden_set_v0.harness` runs the real `HybridRetriever` over the committed **500-query / 5-domain** golden set (`tests/eval/golden/`) and reports **Recall@k / MRR / nDCG@k / Faithfulness / Citation Precision** (overall + per-domain + per-difficulty) as `report.json` plus a self-contained HTML report; the noop backends give a real, deterministic signal (`NoopKeywordStore` token overlap + a `HashingEmbedder` for the zero-vector `NoopEmbedder`, fused via RRF), so the numbers are a stable CI baseline, with the pure metric library + `render_html_report` in `rag_config.eval`, the runner in `eval/golden_set_v0/`, and faithfulness defaulting to a dependency-free `lexical_faithfulness` (RAGAS optional). See [`docs/reference/eval-harness.md`](docs/reference/eval-harness.md), [`docs/architecture/ci-eval-gate.md`](docs/architecture/ci-eval-gate.md), [`docs/architecture/golden-set-eval.md`](docs/architecture/golden-set-eval.md), [ADR-0028](docs/adr/ADR-0028-ci-eval-gate.md), and [ADR-0027](docs/adr/ADR-0027-offline-eval-harness.md). +**Status: v1.0.0 GA (2026-06-10) — all 8 phases, 84/84 steps complete.** Engineering scope is +done; the remaining items are external/process deliverables (external pentest, SOC 2 audit, +registry first-publish, on-call staffing, launch assets) tracked in +[docs/ga/ga-readiness-checklist.md](docs/ga/ga-readiness-checklist.md). Post-GA work comes from +the V1.1 backlog — take the current step from `TRACKER.md`'s **Status** block, and do not +duplicate per-step detail into this file: it drifts (this section sat at "Phase 5" while the +repo shipped Phase 7). TRACKER.md is the single source of truth for build status. ## Repo layout ``` AgentContextOS/ -├── packages/ +├── packages/ # 30 focused packages — the foundational ones: │ ├── core/ # rag-core — domain types, errors, SPI interfaces, noop impls, AuditWriter, Pipeline, Batcher │ ├── config/ # rag-config — rag.yaml schema (Pydantic v2), loader, ragctl config CLI │ ├── observability/ # rag-observability — structured JSON logger, set_log_context(), event registry, async exporter │ ├── policy/ # rag-policy — PolicyEngine (central PDP for ACL/PII/quotas/redaction); added in Step 1.1c │ ├── backends/ # rag-backends — PgVectorStore, QdrantVectorStore, RedisCache, S3Storage, LocalFileStorage -│ └── graphrag/ # rag-graphrag — community detection + sub-graph summarisation + graph-aware retrieval (Step 2.9) +│ ├── ragctl/ # operator CLI — 37 commands + 13 sub-command groups (see docs/reference/ragctl.md) +│ └── …/ # per-stage + governance packages: parsers, chunker, embedders, enricher, ingest, +│ # ocr, pii, retrieval, query, reranker, packer, cache, breaker, graphrag, agent, +│ # guard, quota, provenance, feedback, drift, injection, sso, webhooks, compliance ├── apps/ -│ └── gateway/ # rag-gateway (FastAPI, future home of the REST + gRPC service) +│ ├── gateway/ # rag-gateway — REST + gRPC + MCP + OpenAI-compatible service +│ └── admin-ui/ # Next.js operator console +├── sdks/ # Generated Python / TypeScript / Go / Java / .NET clients +├── website/ # Docusaurus docs site — sources docs/ directly (internal dirs excluded in its config) +├── eval/ # Golden-set eval harness + CI eval gate runner ├── tests/ │ ├── contract/ # SPI conformance suites (pytest mark: contract) │ ├── logs/ # Logging schema + PII + event-registry gates +│ ├── docs/ # Doc tests — quickstart ragctl commands + /v1 paths must be real │ └── config/ # rag.yaml loader tests + invalid YAML fixtures -├── scripts/ # check_logging.py (RAG001), bootstrap.sh/.ps1, dev-wait/seed -├── infra/ # Terraform + Helm (future) +├── scripts/ # check_logging.py (RAG001), bootstrap.sh/.ps1, export_openapi, gen_api_reference +├── infra/ # Helm chart, Terraform, OTel/Prometheus configs +├── packaging/ # Docker image + air-gapped bundle build +├── marketplace/ # Cloud marketplace listings + pricing.yaml ├── proto/ # core.proto — canonical cross-language schema -├── dist/ # Generated: rag.schema.json, rag.schema.yaml (committed, CI drift gate) +├── dist/ # Generated + committed + drift-gated: OpenAPI, JSON schemas ├── Taskfile.yml # Cross-platform task runner (Windows + macOS + Linux) ├── Makefile # Unix convenience wrapper (same targets as Taskfile) ├── TRACKER.md # Build tracker — read this first every session @@ -46,19 +61,19 @@ AgentContextOS/ | Package | Import root | Purpose | |---------|-------------|---------| -| `rag-core` v0.2.0 | `rag_core` | Domain types (Pydantic v2 frozen models), 14-type error hierarchy, plugin SPI ABCs + noop impls, `AuditWriter`, `Pipeline` + `Batcher` primitives (added in Step 1.1d) | -| `rag-config` v0.1.0 | `rag_config` | `rag.yaml` schema, env-var interpolation loader, `ConfigWatcher` hot-reload, `ragctl config` CLI | -| `rag-observability` v0.1.0 | `rag_observability` | 7-field JSON structured logger, `set_log_context()` contextvar manager, event registry, async exporter with drop-on-overflow | -| `rag-policy` v0.1.0 (Step 1.1c) | `rag_policy` | `PolicyEngine` SPI + noop impl; central PDP for ACL / PII / quotas / redaction; `PolicyWriter` facade | -| `rag-backends` v0.1.0 | `rag_backends` | Real backend implementations: PgVectorStore, QdrantVectorStore, RedisCache, S3Storage, LocalFileStorage | -| `rag-graphrag` v0.1.0 (Step 2.9) | `rag_graphrag` | GraphRAG: `LouvainDetector` / `LeidenDetector` for community detection, `CommunitySummarizer` for LLM-backed sub-graph summaries, `InMemoryCommunityStore`, `GraphRAGRetriever` + `graphrag_adapter` for use as the `graph` source in `HybridRetriever` | -| `rag-guard` v0.1.0 (Step 4.3) | `rag_guard` | Hallucination guard: `HallucinationGuard` (per-claim NLI faithfulness of a generated answer vs. retrieved context → per-tenant threshold → annotate/redact/block), `LexicalNLIScorer` (dependency-free default behind the `rag_core.spi.nli.NLIScorer` SPI), `SentenceClaimExtractor`; emits `guard.claim_blocked` | -| `rag-quota` v0.1.0 (Step 4.5) | `rag_quota` | Per-tenant quotas & rate limiting *through* the PolicyEngine PDP: `QuotaPolicyEngine` (decorates an inner engine, answers `rate_limit` / `quota_check`), `QuotaEnforcer` (QPS / token / cost / queries / storage dimensions; fail-open degrade; `quota.exceeded` event), `QuotaStore` SPI behind a weighted sliding-window `InMemoryQuotaStore` (+ injected `RedisQuotaStore` in `rag-backends`); `QuotaVerdict` / `QuotaSnapshot` / `QuotaDimension` core types + `QuotaExceededError` | -| `rag-provenance` v0.1.0 (Step 5.1) | `rag_provenance` | HMAC-signed per-query provenance: `ProvenanceSigner` (HMAC-SHA256 over the canonical record JSON, modeled on the webhook signer), `ProvenanceRecorder` (build → sign → store → emit; **degrade-open**, privacy-by-default hashes). Backed by the `ProvenanceStore` SPI + `NoopProvenanceStore` and the `ProvenanceRecord` / `ProvenanceCitation` / `ProvenanceSignature` / `SignedProvenanceRecord` / `ProvenanceVerification` / `SpanRecord` core types. The read-side `TraceCollector` (a `rag-observability` `SpanProcessor`) captures per-query spans by `rag.trace_id` for `GET /v1/query/{id}/trace` | -| `rag-feedback` v0.1.0 (Step 5.4) | `rag_feedback` | Online feedback & implicit signals: `FeedbackRecorder` (normalise signal → `[-1,1]` score → **PII-redact** comment via an injected `PIIDetector` → tenant-scoped `FeedbackStore.put` → `feedback.recorded` event; **degrade-open**) + pure `aggregate_feedback` → `FeedbackStats`. Backed by the `FeedbackStore` SPI + `NoopFeedbackStore` and the `FeedbackRecord` / `FeedbackStats` / `FeedbackKind` / `FeedbackSignal` core types; `FeedbackRequest` / `FeedbackAck` wire types. Drives `POST /v1/feedback` (explicit + implicit, one `signal` enum) + the `GET /v1/status/feedback` per-tenant dashboard | -| `rag-drift` v0.1.0 (Step 5.5) | `rag_drift` | Drift monitors: `population_stability_index` (pure binned PSI) + `DriftMonitor` (bounded reference/current windows, PSI or mean-drop, tri-state verdict) + `DriftMonitorRegistry` (five monitors — query distribution / embedding PSI / retrieval score / citation-clickthrough / faithfulness; `observe` / `evaluate` with transition-edge `drift.detected` event + webhook). Backed by the `DriftSnapshot` / `DriftReport` / `DriftMetric` / `DriftMethod` / `DriftStatus` core types. Infra-scoped (like breakers), fed from the query + feedback paths; drives `GET /v1/status/drift` + `POST /v1/status/drift/{metric}/rebaseline` | -| `rag-injection` v0.1.0 (Step 7.3) | `rag_injection` | Prompt-injection guard: `PromptInjectionGuard.inspect(ctx, chunks)` drops chunks that try to hijack the model before the LLM (pluggable `InjectionDetector` + dependency-free `HeuristicInjectionDetector` — attack-grammar regexes), paired with `INJECTION_RESISTANT_SYSTEM_PROMPT` + `build_user_message` so untrusted context is fenced *data* in the user turn (never a system-trust position). `InjectionConfig` / `InjectionResult` / `InjectionAction` / `InjectionVerdict` / `InjectionMatch`; `injection.blocked` event; degrade-open. Wired on every answer surface (`/v1/query`, OpenAI chat, MCP), off by default (`cfg.injection`) | -| `rag-gateway` v0.10.0 | `rag_gateway` | Gateway service over four surfaces: REST (`/v1/query`, `/v1/retrieve`, `/v1/corpora`, `/v1/ingest/document`, `/v1/query/{id}/trace`), gRPC (`RagService`), MCP (`query`/`retrieve`/`ingest` tools), and OpenAI-compatible (`/v1/embeddings`, `/v1/chat/completions`, `/v1/models`) — all sharing one in-process Phase 2 wiring | +| `rag-core` | `rag_core` | Domain types (Pydantic v2 frozen models), 14-type error hierarchy, plugin SPI ABCs + noop impls, `AuditWriter`, `Pipeline` + `Batcher` primitives (added in Step 1.1d) | +| `rag-config` | `rag_config` | `rag.yaml` schema, env-var interpolation loader, `ConfigWatcher` hot-reload, `ragctl config` CLI | +| `rag-observability` | `rag_observability` | 7-field JSON structured logger, `set_log_context()` contextvar manager, event registry, async exporter with drop-on-overflow | +| `rag-policy` (Step 1.1c) | `rag_policy` | `PolicyEngine` SPI + noop impl; central PDP for ACL / PII / quotas / redaction; `PolicyWriter` facade | +| `rag-backends` | `rag_backends` | Real backend implementations: PgVectorStore, QdrantVectorStore, RedisCache, S3Storage, LocalFileStorage | +| `rag-graphrag` (Step 2.9) | `rag_graphrag` | GraphRAG: `LouvainDetector` / `LeidenDetector` for community detection, `CommunitySummarizer` for LLM-backed sub-graph summaries, `InMemoryCommunityStore`, `GraphRAGRetriever` + `graphrag_adapter` for use as the `graph` source in `HybridRetriever` | +| `rag-guard` (Step 4.3) | `rag_guard` | Hallucination guard: `HallucinationGuard` (per-claim NLI faithfulness of a generated answer vs. retrieved context → per-tenant threshold → annotate/redact/block), `LexicalNLIScorer` (dependency-free default behind the `rag_core.spi.nli.NLIScorer` SPI), `SentenceClaimExtractor`; emits `guard.claim_blocked` | +| `rag-quota` (Step 4.5) | `rag_quota` | Per-tenant quotas & rate limiting *through* the PolicyEngine PDP: `QuotaPolicyEngine` (decorates an inner engine, answers `rate_limit` / `quota_check`), `QuotaEnforcer` (QPS / token / cost / queries / storage dimensions; fail-open degrade; `quota.exceeded` event), `QuotaStore` SPI behind a weighted sliding-window `InMemoryQuotaStore` (+ injected `RedisQuotaStore` in `rag-backends`); `QuotaVerdict` / `QuotaSnapshot` / `QuotaDimension` core types + `QuotaExceededError` | +| `rag-provenance` (Step 5.1) | `rag_provenance` | HMAC-signed per-query provenance: `ProvenanceSigner` (HMAC-SHA256 over the canonical record JSON, modeled on the webhook signer), `ProvenanceRecorder` (build → sign → store → emit; **degrade-open**, privacy-by-default hashes). Backed by the `ProvenanceStore` SPI + `NoopProvenanceStore` and the `ProvenanceRecord` / `ProvenanceCitation` / `ProvenanceSignature` / `SignedProvenanceRecord` / `ProvenanceVerification` / `SpanRecord` core types. The read-side `TraceCollector` (a `rag-observability` `SpanProcessor`) captures per-query spans by `rag.trace_id` for `GET /v1/query/{id}/trace` | +| `rag-feedback` (Step 5.4) | `rag_feedback` | Online feedback & implicit signals: `FeedbackRecorder` (normalise signal → `[-1,1]` score → **PII-redact** comment via an injected `PIIDetector` → tenant-scoped `FeedbackStore.put` → `feedback.recorded` event; **degrade-open**) + pure `aggregate_feedback` → `FeedbackStats`. Backed by the `FeedbackStore` SPI + `NoopFeedbackStore` and the `FeedbackRecord` / `FeedbackStats` / `FeedbackKind` / `FeedbackSignal` core types; `FeedbackRequest` / `FeedbackAck` wire types. Drives `POST /v1/feedback` (explicit + implicit, one `signal` enum) + the `GET /v1/status/feedback` per-tenant dashboard | +| `rag-drift` (Step 5.5) | `rag_drift` | Drift monitors: `population_stability_index` (pure binned PSI) + `DriftMonitor` (bounded reference/current windows, PSI or mean-drop, tri-state verdict) + `DriftMonitorRegistry` (five monitors — query distribution / embedding PSI / retrieval score / citation-clickthrough / faithfulness; `observe` / `evaluate` with transition-edge `drift.detected` event + webhook). Backed by the `DriftSnapshot` / `DriftReport` / `DriftMetric` / `DriftMethod` / `DriftStatus` core types. Infra-scoped (like breakers), fed from the query + feedback paths; drives `GET /v1/status/drift` + `POST /v1/status/drift/{metric}/rebaseline` | +| `rag-injection` (Step 7.3) | `rag_injection` | Prompt-injection guard: `PromptInjectionGuard.inspect(ctx, chunks)` drops chunks that try to hijack the model before the LLM (pluggable `InjectionDetector` + dependency-free `HeuristicInjectionDetector` — attack-grammar regexes), paired with `INJECTION_RESISTANT_SYSTEM_PROMPT` + `build_user_message` so untrusted context is fenced *data* in the user turn (never a system-trust position). `InjectionConfig` / `InjectionResult` / `InjectionAction` / `InjectionVerdict` / `InjectionMatch`; `injection.blocked` event; degrade-open. Wired on every answer surface (`/v1/query`, OpenAI chat, MCP), off by default (`cfg.injection`) | +| `rag-gateway` | `rag_gateway` | Gateway service over four surfaces: REST (`/v1/query`, `/v1/retrieve`, `/v1/corpora`, `/v1/ingest/document`, `/v1/query/{id}/trace`), gRPC (`RagService`), MCP (`query`/`retrieve`/`ingest` tools), and OpenAI-compatible (`/v1/embeddings`, `/v1/chat/completions`, `/v1/models`) — all sharing one in-process Phase 2 wiring | ## Key architecture patterns @@ -198,8 +213,10 @@ Examples: `build/phase-0/step-0.8-eval-skeleton`, `build/phase-1/step-1.1-storag ## Documentation requirement (every PR) -**No PR is complete without a `docs/` update.** Documentation written at PR time is used later -to generate public docs (Step 7.5 — Docusaurus/MkDocs site) and technical docs for design partners. +**No PR is complete without a `docs/` update.** Documentation written at PR time is published +on the public docs site (`website/`, Docusaurus — it sources `docs/` directly; internal areas +like `docs/ga/` and `docs/pilots/` are excluded in `website/docusaurus.config.ts`) and serves +design partners. Treat every `docs/` file as public unless its directory is in that exclude list. ### Where each type of doc goes diff --git a/README.md b/README.md index 88df613..41cc6ea 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,30 @@ # AgentContextOS -**One-stop RAG platform for enterprise AI** — pluggable retrieval, multi-tenancy, agent loops, full observability. +**Production-grade, multi-tenant RAG platform** — pluggable retrieval, enterprise governance, agent loops, and full observability behind one API. **v1.0.0 GA.** -## Quick start +Four API surfaces — REST, gRPC, MCP, and OpenAI-compatible — over one engine: hybrid retrieval (dense + sparse + graph) with reranking, hallucination guarding, signed per-query provenance, tenant isolation with ACL/PII enforcement, quotas, drift monitoring, eval gates, and billing. Every backend (vector store, embedder, LLM, …) is a swappable plugin behind an SPI, and `rag.yaml` is the portability contract — the same image runs on a laptop, a VM, and Kubernetes. + +## Run a query in 5 minutes + +No credentials, no infrastructure: + +```bash +git clone https://github.com/officialCodeWork/AgentContextOS.git +cd AgentContextOS && uv sync + +uv run uvicorn rag_gateway.app:app --port 8000 # terminal 1 +curl -s http://localhost:8000/healthz # terminal 2 → {"status":"ok"} +``` + +Follow the **[curl quickstart](docs/guides/curl-quickstart.md)** for ingest + query end-to-end, then pick your surface: +[SDKs](docs/guides/sdk-quickstart.md) · +[OpenAI-compatible](docs/guides/openai-quickstart.md) · +[MCP](docs/guides/mcp-quickstart.md) · +[gRPC](docs/guides/grpcurl-quickstart.md) · +[`ragctl` CLI](docs/guides/ragctl-quickstart.md) · +[admin console](docs/guides/admin-ui-quickstart.md) + +## Development setup ```bash # Prerequisites: uv, pnpm, docker @@ -11,7 +33,7 @@ make dev # start postgres + redis + qdrant make dev-full # + otel-collector, jaeger, prometheus, grafana, loki -make lint # ruff + mypy +make lint # ruff + mypy --strict make test # full test suite make help # all targets ``` @@ -20,68 +42,35 @@ make help # all targets ``` AgentContextOS/ -├── packages/ -│ └── core/ # Domain types, errors, plugin SPIs (Step 0.2–0.3) +├── packages/ # 30 Python packages: rag-core (types, SPIs), rag-config, +│ # rag-observability, rag-policy, rag-backends, per-stage +│ # pipeline packages (parsers, chunker, embedders, …), +│ # governance (guard, quota, pii, sso, …), and ragctl ├── apps/ -│ ├── gateway/ # FastAPI HTTP/gRPC gateway (Phase 3) -│ └── admin-ui/ # Next.js operator GUI (Phase 3) -├── tests/ -│ ├── contract/ # SPI conformance suites (Step 0.3) -│ ├── eval/ # Golden set, baselines, thresholds (Phase 5) -│ └── logs/ # Log schema + PII gates (Step 0.7b) -├── proto/ # Protobuf definitions (Step 0.2) -├── infra/ -│ ├── otel/ # OTel collector config -│ └── prometheus/ # Prometheus scrape config -├── docs/ -│ ├── adr/ # Architecture Decision Records -│ ├── architecture/ # HLD + architecture diagram -│ └── research/ # RAG problems catalog -├── scripts/ -│ └── bootstrap.sh # One-shot setup for a fresh clone +│ ├── gateway/ # rag-gateway — REST + gRPC + MCP + OpenAI-compatible service +│ └── admin-ui/ # Next.js operator console +├── sdks/ # Generated Python / TypeScript / Go / Java / .NET clients +├── website/ # Docusaurus documentation site (sources docs/ directly) +├── eval/ # Golden-set eval harness + CI eval gate runner +├── tests/ # Cross-package suites: contract, logs, policy, docs, eval +├── proto/ # core.proto — canonical cross-language schema +├── dist/ # Generated + committed + drift-gated: OpenAPI, JSON schemas +├── infra/ # Helm chart, Terraform, OTel / Prometheus configs +├── packaging/ # Docker image + air-gapped bundle build +├── marketplace/ # Cloud marketplace listing artifacts + pricing model +├── planning/ # The 8-phase execution plan the build followed +├── docs/ # ADRs, architecture, guides, reference, runbooks, compliance ├── docker-compose.yml # Local dev stack (core + observability profiles) -├── rag.yaml # Platform config contract (Step 0.4) -├── pyproject.toml # Python uv workspace root -├── pnpm-workspace.yaml # JS/TS pnpm workspace -└── Makefile # All developer commands +├── rag.yaml # Platform config contract +├── Taskfile.yml # Cross-platform task runner (Makefile wraps it on Unix) +└── TRACKER.md # The 84-step build record (all complete) ``` ## Documentation -- [High-Level Design](docs/architecture/RAG-Platform-HLD.md) — architecture, components, KPIs -- [Execution Plan](planning/README.md) — 8-phase build plan with testable steps -- [Logging Standard](planning/LOGGING-STANDARD.md) — mandatory structured-logging contract -- [GUI Specification](planning/GUI-SPECIFICATION.md) — operator dashboard spec -- [ADR-0001](docs/adr/ADR-0001-monorepo-and-tech-stack.md) — monorepo and tech stack decisions -├── docs/ -│ ├── architecture/ # Design documents -│ │ ├── RAG-Platform-HLD.md — High-Level Design (architecture, components, KPIs) -│ │ └── high-level-architecture.svg — Layered architecture diagram -│ └── research/ # Problem analysis and research -│ ├── enterprise-rag-problems.md — 22+ catalogued RAG problems with severity ratings -│ ├── enterprise-rag-problems.pdf — PDF export of the problem catalog -│ └── rag-problems-world.html — Interactive HTML problem explorer -└── planning/ # V1 execution plan - ├── README.md — Planning index and how-to-use guide - ├── EXECUTION-PLAN.md — Master build order, principles, DoD - ├── TESTING-STRATEGY.md — Test pyramid, eval gates, red-team layers - ├── LOGGING-STANDARD.md — Mandatory structured-logging contract - ├── GUI-SPECIFICATION.md — Operator Status & Metrics GUI spec (12 pages) - ├── PROBLEM-TRACEABILITY.md — RAG problem → primary step + gate test - ├── RISK-REGISTER.md — Tracked risks, owners, mitigations - └── phases/ - ├── phase-0-foundation.md — Monorepo, plugin SPI, CI/CD, observability - ├── phase-1-ingestion.md — Connectors, parsers, chunker, embedder - ├── phase-2-retrieval.md — Vector, BM25, graph, hybrid RRF, reranker - ├── phase-3-gateway-agent.md — REST/gRPC/MCP, agent loop, SDKs, GUI MVP - ├── phase-4-reliability.md — Semantic cache, fallback chain, guard, breakers - ├── phase-5-eval-observability.md — Eval harness, CI gate, drift monitors, full GUI - ├── phase-6-governance.md — Multi-tenancy, ACL, audit log, BYOK, SSO - └── phase-7-pilot-ga.md — Load testing, chaos, partners, GA cutover -``` - -## Quick start - -1. Read [docs/architecture/RAG-Platform-HLD.md](docs/architecture/RAG-Platform-HLD.md) for the full architecture. -2. Read [docs/research/enterprise-rag-problems.md](docs/research/enterprise-rag-problems.md) for the problem space. -3. Open [planning/README.md](planning/README.md) to navigate the execution plan. +- [docs/README.md](docs/README.md) — full documentation index (published as the docs site) +- [High-Level Design](docs/architecture/RAG-Platform-HLD.md) — architecture, components, problem → capability map +- [REST API reference](docs/reference/rest-api.md) — generated from the drift-gated OpenAPI contract +- [v1.0.0 release notes](docs/release-notes/v1.0.0.md) — what shipped at GA +- [Problem catalog](docs/research/enterprise-rag-problems.md) — the 22+ enterprise RAG failure modes this platform addresses +- [Execution plan](planning/README.md) — the 8-phase build plan, now complete ([TRACKER.md](TRACKER.md) is the step-by-step record) diff --git a/docs/README.md b/docs/README.md index 851cdde..830beb9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -230,17 +230,19 @@ broken, and what to fix before committing to the next phase. Design-partner pilot program (Step 7.4) — the operator runbook is [guides/design-partner-pilots.md](guides/design-partner-pilots.md); this area holds the fill-in templates, per-vertical kits, and published case studies. +Internal program material — kept on GitHub only, not published on the docs site +(links below point at GitHub). | File | Description | |------|-------------| -| [README.md](pilots/README.md) | Pilots index — layout, the five templates, the per-vertical kit registry, the active/graduated-pilot table | -| [templates/onboarding-checklist.md](pilots/templates/onboarding-checklist.md) | Per-partner onboarding: tenant + governance + corpus + surfaces + measurement + sign-off (scripted by `ragctl pilot onboard`) | -| [templates/success-criteria.md](pilots/templates/success-criteria.md) | Signed-before-kickoff criteria across quality / latency / integration / security, each with a threshold + the platform signal that proves it | -| [templates/weekly-kpi.md](pilots/templates/weekly-kpi.md) | One-per-week KPI dashboard (volume / quality / satisfaction / drift / latency / cost / reliability) filled by `ragctl pilot report` | -| [templates/feedback-log.md](pilots/templates/feedback-log.md) | The intake → triage → incorporate → close loop + the roadmap fold-back for the lessons-learned doc | -| [templates/case-study.md](pilots/templates/case-study.md) | The referenceable customer story — challenge / solution / reference architecture / results (auditable KPI numbers) / quote | -| [customer-support/README.md](pilots/customer-support/README.md) | Customer-support / internal-KB pilot kit (Step 7.4b) — sample corpus (PII handbook + product FAQ + a planted injection probe), domain-calibrated success criteria (deflection), the `ragctl pilot` seed-and-demo flow, the PII + injection security demonstration | -| [customer-support/case-study.md](pilots/customer-support/case-study.md) | Worked case study (Step 7.4d) — the framework run end-to-end on the kit; real `ragctl pilot report` KPIs (satisfaction +0.733 · 0/5 drift · cost ok → PASS) + the PII + injection security demonstration | +| [README.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/README.md) | Pilots index — layout, the five templates, the per-vertical kit registry, the active/graduated-pilot table | +| [templates/onboarding-checklist.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/onboarding-checklist.md) | Per-partner onboarding: tenant + governance + corpus + surfaces + measurement + sign-off (scripted by `ragctl pilot onboard`) | +| [templates/success-criteria.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/success-criteria.md) | Signed-before-kickoff criteria across quality / latency / integration / security, each with a threshold + the platform signal that proves it | +| [templates/weekly-kpi.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/weekly-kpi.md) | One-per-week KPI dashboard (volume / quality / satisfaction / drift / latency / cost / reliability) filled by `ragctl pilot report` | +| [templates/feedback-log.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/feedback-log.md) | The intake → triage → incorporate → close loop + the roadmap fold-back for the lessons-learned doc | +| [templates/case-study.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/case-study.md) | The referenceable customer story — challenge / solution / reference architecture / results (auditable KPI numbers) / quote | +| [customer-support/README.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/customer-support/README.md) | Customer-support / internal-KB pilot kit (Step 7.4b) — sample corpus (PII handbook + product FAQ + a planted injection probe), domain-calibrated success criteria (deflection), the `ragctl pilot` seed-and-demo flow, the PII + injection security demonstration | +| [customer-support/case-study.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/customer-support/case-study.md) | Worked case study (Step 7.4d) — the framework run end-to-end on the kit; real `ragctl pilot report` KPIs (satisfaction +0.733 · 0/5 drift · cost ok → PASS) + the PII + injection security demonstration | ## runbooks/ diff --git a/docs/adr/ADR-0041-airgap-bundle.md b/docs/adr/ADR-0041-airgap-bundle.md index c936c50..127ae6d 100644 --- a/docs/adr/ADR-0041-airgap-bundle.md +++ b/docs/adr/ADR-0041-airgap-bundle.md @@ -3,7 +3,7 @@ **Status:** Accepted **Date:** 2026-06-08 **Step:** 6.9 — Air-gapped install bundle (Phase 6 — Governance & Tenancy) -**Related:** [ADR-0038](ADR-0038-immutable-audit-log.md) (content-hash + optional-signature pattern), [ADR-0003](ADR-0003-iac-foundation.md) (Helm chart), [architecture/airgap-bundle.md](../architecture/airgap-bundle.md), [reference/airgap.md](../reference/airgap.md), [guides/airgap-install.md](../guides/airgap-install.md) +**Related:** [ADR-0038](ADR-0038-immutable-audit-log.md) (content-hash + optional-signature pattern), [ADR-0003](ADR-0003-iac-kubernetes-native.md) (Helm chart), [architecture/airgap-bundle.md](../architecture/airgap-bundle.md), [reference/airgap.md](../reference/airgap.md), [guides/airgap-install.md](../guides/airgap-install.md) ## Context diff --git a/docs/adr/ADR-0046-design-partner-pilots.md b/docs/adr/ADR-0046-design-partner-pilots.md index 6ca7758..10ff20b 100644 --- a/docs/adr/ADR-0046-design-partner-pilots.md +++ b/docs/adr/ADR-0046-design-partner-pilots.md @@ -3,7 +3,7 @@ **Status:** Accepted **Date:** 2026-06-10 **Step:** 7.4 — Design-partner pilots (Phase 7 — Pilot, Harden, GA) -**Related:** [guides/design-partner-pilots.md](../guides/design-partner-pilots.md), [docs/pilots/](../pilots/README.md), [5.4 online feedback](ADR-0029-online-feedback.md), [5.5 drift monitors](ADR-0030-drift-monitors.md), [5.6c cost anomaly](ADR-0031-cost-anomaly.md), [5.2–5.3 eval harness + gate](ADR-0027-offline-eval-harness.md), [6.10 compliance posture](ADR-0042-compliance-posture.md), [7.3 red-team / external pentest](ADR-0045-red-team-security.md), [planning/phases/phase-7-pilot-ga.md](../../planning/phases/phase-7-pilot-ga.md) +**Related:** [guides/design-partner-pilots.md](../guides/design-partner-pilots.md), [docs/pilots/](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/README.md), [5.4 online feedback](ADR-0029-online-feedback.md), [5.5 drift monitors](ADR-0030-drift-monitors.md), [5.6c cost anomaly](ADR-0031-cost-anomaly.md), [5.2–5.3 eval harness + gate](ADR-0027-offline-eval-harness.md), [6.10 compliance posture](ADR-0042-compliance-posture.md), [7.3 red-team / external pentest](ADR-0045-red-team-security.md), [planning/phases/phase-7-pilot-ga.md](../../planning/phases/phase-7-pilot-ga.md) ## Context diff --git a/docs/adr/ADR-0052-ga-cutover.md b/docs/adr/ADR-0052-ga-cutover.md index 883c026..6f327bd 100644 --- a/docs/adr/ADR-0052-ga-cutover.md +++ b/docs/adr/ADR-0052-ga-cutover.md @@ -3,7 +3,7 @@ **Status:** Accepted **Date:** 2026-06-10 **Step:** 7.10 — GA cutover (Phase 7 — Pilot, Harden, GA) -**Related:** [release-notes/v1.0.0.md](../release-notes/v1.0.0.md), [ga/ga-readiness-checklist.md](../ga/ga-readiness-checklist.md), [7.7 packaging](ADR-0049-packaging-distribution.md), [planning/phases/phase-7-pilot-ga.md](../../planning/phases/phase-7-pilot-ga.md) +**Related:** [release-notes/v1.0.0.md](../release-notes/v1.0.0.md), [ga/ga-readiness-checklist.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/ga/ga-readiness-checklist.md), [7.7 packaging](ADR-0049-packaging-distribution.md), [planning/phases/phase-7-pilot-ga.md](../../planning/phases/phase-7-pilot-ga.md) ## Context @@ -27,7 +27,7 @@ before removal in the next MAJOR. The committed OpenAPI/proto/schema artifacts m deprecations visible in review. **3. The GA bar is a checklist of gates, not a declaration.** The -[GA readiness checklist](../ga/ga-readiness-checklist.md) maps every Phase 7 exit +[GA readiness checklist](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/ga/ga-readiness-checklist.md) maps every Phase 7 exit gate to a CI gate or an in-repo artifact; inherently-external items (external pentest, SOC 2 audit, signed referenceable customers, registry-publish accounts, on-call staffing, launch assets) are marked **(external)** and ride on top — the diff --git a/docs/architecture/RAG-Platform-HLD.md b/docs/architecture/RAG-Platform-HLD.md index c6ad731..5e1f619 100644 --- a/docs/architecture/RAG-Platform-HLD.md +++ b/docs/architecture/RAG-Platform-HLD.md @@ -1,10 +1,12 @@ # Enterprise RAG Platform — High-Level Design (HLD) -**Document version:** 1.1 -**Date:** 24 May 2026 (revision) -**Status:** Draft for review +**Document version:** 1.2 +**Date:** 10 June 2026 (GA revision) +**Status:** Released — describes the shipped v1.0.0 GA system **Owner:** Platform Engineering +**v1.2 changes (GA):** Updated for the v1.0.0 GA cutover — all eight phases (84 steps) are complete and shipped. §5 now maps every problem to the architecture doc / ADR where its capability shipped; §8.1 reflects the shipped OpenAI-compatible surface (`/v1/embeddings`, `/v1/chat/completions`, `/v1/models`); §10 records that the ACL egress verifier shipped as a defense-in-depth layer ([ADR-0036](../adr/ADR-0036-acl-egress-verifier.md)) rather than being subsumed by the PDP as v1.1 expected; §12 marks the roadmap complete. See the [v1.0.0 release notes](../release-notes/v1.0.0.md) and [ADR-0052 — GA cutover](../adr/ADR-0052-ga-cutover.md). + **v1.1 changes:** Promoted `RequestContext` to a first-class architectural primitive threaded through every SPI; added a central `PolicyEngine` (PDP) component that subsumes ACL push-down + ACL egress verifier + PII gates + quota gates; split the "Knowledge Store" SPI into separate Retrieval and Index surfaces with bulk + streaming + ID-only methods; added `Pipeline` + `Batcher` runtime primitives; split the semantic cache into three caches with distinct invalidation rules (`EmbeddingCache`, `RetrievalCache`, `AnswerCache`); typed `trust_level` on chunks as the foundation for prompt-injection defense; introduced `BlobRef` for lazy chunk text. See ADRs 0005–0009 and the Phase 1 refactor window (Steps 1.1a–1.1f) in [TRACKER.md](../../TRACKER.md). --- @@ -13,6 +15,8 @@ The Enterprise RAG Platform is a **portable, pluggable, multi-tenant Retrieval-Augmented Generation engine** that solves the recurring RAG failures observed across enterprise deployments — retrieval quality, ingestion chaos, hallucination, latency, governance, multi-tenancy, and agentic-loop support. +**Status (10 June 2026):** **v1.0.0 GA shipped** — all eight phases (84 steps) complete; the engineering scope of this document is built and released. Remaining items are external/process deliverables (external pentest, SOC 2 audit, registry first-publish, on-call staffing) tracked in the [GA readiness checklist](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/ga/ga-readiness-checklist.md). + It is delivered as **Open-Core software** (Apache 2.0) plus a **Managed SaaS** and an **Enterprise self-hosted edition**, designed to be deployed anywhere (laptop → on-prem → cloud) and integrated with any existing agentic / RAG framework (LangChain, LlamaIndex, Semantic Kernel, MCP, custom). ### Vision @@ -133,33 +137,37 @@ This platform addresses all of these as first-class capabilities. ## 5. Core Capabilities (Problem → Capability Mapping) -| Problem | Capability | -|---|---| -| Vector-only misses exact matches | Hybrid retrieval (BM25 + dense + graph) with RRF fusion | -| Wrong chunks at Top-K | Cross-encoder reranker + MMR | -| Multi-hop queries fail | Query decomposition + GraphRAG + agent loop | -| Vocabulary mismatch | Query rewriter, HyDE, per-tenant synonym/acronym glossary | -| Context window poisoning | Conflict detector + context packer | -| Duplicate chunks | Near-duplicate detection (MinHash/embeddings) | -| Lost-in-the-middle | Relevance-anchored reordering | -| Chunking destroys structure | Layout-aware chunker with parent-child links | -| Format chaos | Multi-parser ingestion + OCR | -| Metadata poverty | Auto-enrichment (NER, summary, classification) | -| No incremental sync | CDC connectors + delta indexing + soft deletes | -| Embedding mismatch | Pluggable / domain-tuned embedders | -| Latency | Caching, parallel retrievers, ANN tuning, early-exit | -| No fallback | Tiered fallback chain (rerank → BM25 → keyword → "no answer") | -| No semantic cache | Embedding-similarity cache, tenant-scoped TTL | -| No retrieval metric | Eval harness: recall@k, MRR, nDCG, faithfulness | -| Hallucination blind | Per-stage tracing + faithfulness scoring | -| No drift detection | Embedding/query/score distribution monitors | -| Cross-tenant leakage | Namespace + row-level isolation; physical isolation tier | -| No doc-level ACLs | ACL push-down at retrieval + egress verification | -| PII leakage | Detect/redact at ingest + egress scanner | -| No audit trail | Immutable provenance log per query | -| Not agentic | Iterative `/agent-loop` API with budget controls | -| Framework lock-in | OpenAPI + MCP + adapters for major frameworks | -| No corpus routing | Router/planner selects corpus + retriever | +Every capability below is shipped in v1.0.0. The **Shipped as** column links the +architecture doc and/or decision record where it landed, so this table doubles +as the index for "how does the platform handle X?" questions. + +| Problem | Capability | Shipped as | +|---|---|---| +| Vector-only misses exact matches | Hybrid retrieval (BM25 + dense + graph) with RRF fusion | [hybrid-fusion.md](hybrid-fusion.md) | +| Wrong chunks at Top-K | Cross-encoder reranker + MMR | [reranking.md](reranking.md) · [ADR-0006](../adr/ADR-0006-two-stage-rerank.md) | +| Multi-hop queries fail | Query decomposition + GraphRAG + agent loop | [graphrag.md](graphrag.md) · [agent-loop.md](agent-loop.md) · [ADR-0015](../adr/ADR-0015-agent-runtime.md) | +| Vocabulary mismatch | Query rewriter, HyDE, per-tenant synonym/acronym glossary | [query-understanding.md](query-understanding.md) | +| Context window poisoning | Conflict detector + context packer | [context-packing.md](context-packing.md) | +| Duplicate chunks | Near-duplicate detection (MinHash/embeddings) | [context-packing.md](context-packing.md) | +| Lost-in-the-middle | Relevance-anchored reordering | [context-packing.md](context-packing.md) | +| Chunking destroys structure | Layout-aware chunker with parent-child links | [chunker.md](chunker.md) | +| Format chaos | Multi-parser ingestion + OCR | [parsers.md](parsers.md) · [ocr.md](ocr.md) | +| Metadata poverty | Auto-enrichment (NER, summary, classification) | [enricher.md](enricher.md) | +| No incremental sync | CDC connectors + delta indexing + soft deletes | [connectors.md](connectors.md) | +| Embedding mismatch | Pluggable / domain-tuned embedders | [embedders.md](embedders.md) | +| Latency | Caching, parallel retrievers, ANN tuning, early-exit | [latency.md](latency.md) · [tiered-cache.md](tiered-cache.md) · [ADR-0021](../adr/ADR-0021-tiered-cache.md) | +| No fallback | Tiered fallback chain (rerank → BM25 → keyword → "no answer") | [fallback-chain.md](fallback-chain.md) | +| No semantic cache | Embedding-similarity cache, tenant-scoped TTL | [caching.md](caching.md) · [ADR-0010](../adr/ADR-0010-semantic-embedding-cache.md) | +| No retrieval metric | Eval harness: recall@k, MRR, nDCG, faithfulness | [golden-set-eval.md](golden-set-eval.md) · [ADR-0027](../adr/ADR-0027-offline-eval-harness.md) · [ADR-0028](../adr/ADR-0028-ci-eval-gate.md) | +| Hallucination blind | Per-stage tracing + faithfulness scoring | [hallucination-guard.md](hallucination-guard.md) · [per-query-tracing.md](per-query-tracing.md) · [ADR-0022](../adr/ADR-0022-hallucination-guard.md) | +| No drift detection | Embedding/query/score distribution monitors | [drift-monitors.md](drift-monitors.md) · [ADR-0030](../adr/ADR-0030-drift-monitors.md) | +| Cross-tenant leakage | Namespace + row-level isolation; physical isolation tier | [multi-tenancy.md](multi-tenancy.md) · [ADR-0033](../adr/ADR-0033-logical-multi-tenancy.md) · [ADR-0034](../adr/ADR-0034-physical-multi-tenancy.md) | +| No doc-level ACLs | ACL push-down at retrieval + egress verification | [ADR-0035](../adr/ADR-0035-acl-pushdown.md) · [ADR-0036](../adr/ADR-0036-acl-egress-verifier.md) | +| PII leakage | Detect/redact at ingest + egress scanner | [pii.md](pii.md) · [ADR-0037](../adr/ADR-0037-pii-egress-policies.md) | +| No audit trail | Immutable provenance log per query | [audit-log.md](audit-log.md) · [ADR-0038](../adr/ADR-0038-immutable-audit-log.md) · [ADR-0026](../adr/ADR-0026-per-query-tracing-provenance.md) | +| Not agentic | Iterative `/agent-loop` API with budget controls | [agent-loop.md](agent-loop.md) · [ADR-0015](../adr/ADR-0015-agent-runtime.md) | +| Framework lock-in | OpenAPI + MCP + adapters for major frameworks | [openai-compat.md](openai-compat.md) · [mcp-server.md](mcp-server.md) · [framework-adapters.md](framework-adapters.md) · [ADR-0017](../adr/ADR-0017-framework-adapters.md) | +| No corpus routing | Router/planner selects corpus + retriever | [corpus-router.md](corpus-router.md) · [ADR-0014](../adr/ADR-0014-corpus-routing.md) | --- @@ -215,7 +223,7 @@ A single config file is the portability contract. ### 8.1 Protocols - REST (OpenAPI) + gRPC + Streaming - **MCP server** (Claude Desktop, Cursor, VS Code, any MCP agent) -- OpenAI-compatible `/v1/embeddings` and `/v1/retrieval` +- OpenAI-compatible `/v1/embeddings`, `/v1/chat/completions`, and `/v1/models` - Webhooks for ingest / drift / feedback ### 8.2 SDKs @@ -260,7 +268,7 @@ retriever = RagPlatformRetriever(endpoint="https://rag.mycorp.com", api_key="... All security and governance decisions are routed through a single **PolicyEngine** (PDP) component — see ADR-0005 and [docs/architecture/policy-engine.md](policy-engine.md). Scattered checks across retrieval, ingest, and egress are replaced by `PolicyEngine.evaluate(ctx, decision, subject)`. - **Tenancy:** logical (namespace + row-level via typed `Chunk.tenant_id`) at SMB tier; physical (dedicated index/keys, per-tenant connection pools) at Enterprise tier. `tenant_id` is a typed required field on `Chunk` and `Embedding`, indexed at every backend — no JSON-filter cost. -- **ACL-aware retrieval:** chunks tagged with typed `acl_labels: tuple[str, ...]`; PolicyEngine returns a `FilterExpr` pushed into every retrieval call. Egress verification is *unnecessary* because no path bypasses the PDP — what was Step 6.4 (egress verifier) is therefore redundant. +- **ACL-aware retrieval:** chunks tagged with typed `acl_labels: tuple[str, ...]`; PolicyEngine returns a `FilterExpr` pushed into every retrieval call ([ADR-0035](../adr/ADR-0035-acl-pushdown.md)). An independent egress verifier re-checks every outbound chunk as **defense-in-depth** ([ADR-0036](../adr/ADR-0036-acl-egress-verifier.md)) — v1.1 expected the PDP to make it redundant, but it shipped as a second, independently testable layer so a push-down bug cannot become a leak. - **Prompt-injection defense:** every `Chunk` carries a typed `trust_level: Literal["system", "tenant_curated", "tenant_user", "external"]`. The context packer + LLM adapter use this to choose isolation strategy (XML-tagged section, separate turn, refusal). Not a runtime ad-hoc check. - **PII:** Presidio / Azure AI Language at ingest; egress redaction policy enforced through PolicyEngine (`TRANSFORM` decisions); per-tenant `block / redact / mask / encrypt / tag-only`. - **Audit:** immutable hash-chained log (`AuditWriter` facade over `AuditStore`); records `RequestContext`, query, `QueryPlan`, retrieved IDs, scores, policy decisions, model, answer, citations. @@ -284,6 +292,8 @@ All security and governance decisions are routed through a single **PolicyEngine ## 12. Phased Execution Roadmap +**All eight phases are complete** — v1.0.0 GA was cut on 2026-06-10 (84/84 steps; see [TRACKER.md](../../TRACKER.md) for the step-by-step record and the [v1.0.0 release notes](../release-notes/v1.0.0.md) for what shipped). + | Phase | Window | Deliverables | |---|---|---| | 0 — Foundation | Wk 0–4 | Plugin SPI, `rag.yaml`, monorepo, CI/CD, IaC, eval skeleton | diff --git a/docs/architecture/agent-loop-v0.md b/docs/architecture/agent-loop-v0.md index 1bdd139..0d87798 100644 --- a/docs/architecture/agent-loop-v0.md +++ b/docs/architecture/agent-loop-v0.md @@ -176,9 +176,9 @@ That's a Phase 2 design hole the gap-list memo prioritises as ## Related ADRs and docs -- [ADR-0005 — PolicyEngine as central PDP](../adr/ADR-0005-policy-engine-pdp.md) +- [ADR-0005 — PolicyEngine as central PDP](../adr/ADR-0005-policy-engine.md) - [ADR-0008 — Cost-aware planner](../adr/ADR-0008-cost-aware-planner.md) -- [ADR-0009 — Vector index strategy + quantization](../adr/ADR-0009-vector-index-strategy-quantization.md) +- [ADR-0009 — Vector index strategy + quantization](../adr/ADR-0009-vector-index-strategy.md) - [request-context.md](request-context.md) — the per-request envelope - [caching.md](caching.md) — the three-cache split (Embedding / Retrieval / Answer) diff --git a/docs/guides/curl-quickstart.md b/docs/guides/curl-quickstart.md index 285cf83..6877c8d 100644 --- a/docs/guides/curl-quickstart.md +++ b/docs/guides/curl-quickstart.md @@ -20,20 +20,20 @@ uv sync ## 1) Start the gateway ```bash -uv run uvicorn rag_gateway.app:app --host 0.0.0.0 --port 8080 --reload +uv run uvicorn rag_gateway.app:app --port 8000 ``` -You should see FastAPI start on port 8080. In another terminal, verify: +You should see FastAPI start on port 8000. In another terminal, verify: ```bash -curl -s http://localhost:8080/healthz +curl -s http://localhost:8000/healthz # {"status":"ok"} -curl -s http://localhost:8080/v1/info | jq +curl -s http://localhost:8000/v1/info | jq # Lists all available endpoints. ``` -The interactive Swagger UI is live at `http://localhost:8080/docs`. +The interactive Swagger UI is live at `http://localhost:8000/docs`. ## 2) Ingest a document @@ -53,7 +53,7 @@ the original query. BM25 keyword search and graph traversal are common complements to dense vector search. EOF -curl -s -X POST http://localhost:8080/v1/ingest/document \ +curl -s -X POST http://localhost:8000/v1/ingest/document \ -F 'tenant_id=demo' \ -F 'corpus_id=docs' \ -F 'principal_id=curl' \ @@ -75,7 +75,7 @@ Response: ## 3) Query the gateway ```bash -curl -s http://localhost:8080/v1/query \ +curl -s http://localhost:8000/v1/query \ -H 'content-type: application/json' \ -d '{ "tenant_id": "demo", @@ -136,7 +136,7 @@ This runs the in-process FastAPI gateway against the noop wiring and exercises t ## Asking for an LLM answer ```bash -curl -s http://localhost:8080/v1/query \ +curl -s http://localhost:8000/v1/query \ -H 'content-type: application/json' \ -d '{ "tenant_id": "demo", @@ -154,7 +154,7 @@ The dev wiring wires `NoopLLM` so you get a synthetic echo response. Production If you want raw retrieval refs (e.g. you're using your own reranker): ```bash -curl -s http://localhost:8080/v1/query \ +curl -s http://localhost:8000/v1/query \ -H 'content-type: application/json' \ -d '{ "tenant_id": "demo", @@ -170,7 +170,7 @@ The `chunk_refs` field carries the router output unconditionally; `chunks` (the ## Listing corpora ```bash -curl -s http://localhost:8080/v1/corpora \ +curl -s http://localhost:8000/v1/corpora \ -H 'x-tenant-id: demo' \ -H 'x-principal-id: curl' | jq ``` @@ -180,7 +180,7 @@ By default the in-memory `NoopCorpusStore` is empty. Production wires `Postgres ## Reading the OpenAPI spec ```bash -curl -s http://localhost:8080/openapi.json | jq '.paths | keys' +curl -s http://localhost:8000/openapi.json | jq '.paths | keys' # ["/healthz", "/v1/info", "/v1/ingest/document", "/v1/query", # "/v1/retrieve", "/v1/corpora", "/v1/corpora/{corpus_id}"] ``` diff --git a/docs/guides/design-partner-pilots.md b/docs/guides/design-partner-pilots.md index 7d79023..2a8ffab 100644 --- a/docs/guides/design-partner-pilots.md +++ b/docs/guides/design-partner-pilots.md @@ -31,10 +31,10 @@ a **checkable machine** behind it. | Stage | Goal | Exit gate | Artifact | |-------|------|-----------|----------| | **Qualify** | Confirm fit: distinct vertical, real corpus, exec sponsor, a definition of success | Go / no-go | — | -| **Onboard** | Provision the tenant, seed the corpus, configure governance, agree the criteria | Success criteria **signed** | [onboarding-checklist](../pilots/templates/onboarding-checklist.md), [success-criteria](../pilots/templates/success-criteria.md) | -| **Run** | Weekly cadence — ship, measure, triage feedback | Criteria met **≥ 3 consecutive weeks** | [weekly-kpi](../pilots/templates/weekly-kpi.md), [feedback-log](../pilots/templates/feedback-log.md) | +| **Onboard** | Provision the tenant, seed the corpus, configure governance, agree the criteria | Success criteria **signed** | [onboarding-checklist](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/onboarding-checklist.md), [success-criteria](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/success-criteria.md) | +| **Run** | Weekly cadence — ship, measure, triage feedback | Criteria met **≥ 3 consecutive weeks** | [weekly-kpi](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/weekly-kpi.md), [feedback-log](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/feedback-log.md) | | **Review** | Confirm acceptance, capture lessons | Acceptance sign-off | — | -| **Graduate** | Publish the case study, fold lessons into the roadmap, convert | Case study **published** | [case-study](../pilots/templates/case-study.md) | +| **Graduate** | Publish the case study, fold lessons into the roadmap, convert | Case study **published** | [case-study](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/case-study.md) | The "≥ 3 consecutive weeks" run gate is the plan's pilot-acceptance test: a single good week can be luck; three is a trend. @@ -53,7 +53,7 @@ good week can be luck; three is a trend. ## Onboarding Onboarding is a configured deployment, not a build. The mechanical path (see the -[onboarding checklist](../pilots/templates/onboarding-checklist.md) and, once +[onboarding checklist](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/onboarding-checklist.md) and, once Step 7.4c lands, `ragctl pilot onboard`): 1. **Provision the tenant** — a namespace + per-tenant `rag.yaml` (Step 6.1): @@ -80,7 +80,7 @@ signal, so acceptance is measured, not argued: | **Integration scope** | named surfaces live · N connectors · M corpora | `rag.yaml` + `GET /v1/corpora` | | **Security review** | ACL + PII + audit + injection guard on · red-team gate green · residency satisfied | `GET /v1/status/compliance` + `GET /v1/audit` | -Capture them in [success-criteria.md](../pilots/templates/success-criteria.md), +Capture them in [success-criteria.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/success-criteria.md), one row per criterion with the agreed threshold and the signal that proves it. ## Weekly cadence + KPIs @@ -100,14 +100,14 @@ from signals the platform already emits: | **Reliability** — error rate, breaker trips | `GET /v1/status/health`, `GET /v1/status/breakers` | `ragctl pilot report` (Step 7.4c) assembles these into the -[weekly-kpi](../pilots/templates/weekly-kpi.md) table for a tenant + week so the +[weekly-kpi](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/weekly-kpi.md) table for a tenant + week so the report is generated, not hand-tallied. ## Feedback loop — intake → triage → incorporate → close | Step | What happens | Where | |------|--------------|-------| -| **Intake** | End-user signal (explicit thumb/rating/comment, or implicit click/copy/regenerate) and partner-reported issues | `POST /v1/feedback` + [feedback-log](../pilots/templates/feedback-log.md) | +| **Intake** | End-user signal (explicit thumb/rating/comment, or implicit click/copy/regenerate) and partner-reported issues | `POST /v1/feedback` + [feedback-log](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/feedback-log.md) | | **Triage** | Categorise (bug / quality / feature / docs / infra), set severity, assign an owner | feedback-log | | **Incorporate** | Link to the issue/PR; quality items become per-pilot golden-set cases or detector/config changes | feedback-log → backlog | | **Close** | Resolved + the roadmap entry recorded | feedback-log status = `closed` | @@ -124,7 +124,7 @@ On acceptance, publish two artifacts under `docs/pilots//`: connectors, and governance configuration. - **Customer story** — challenge → solution → results, with the *real* KPI numbers from the weekly reports and a sponsor quote. Use - [case-study.md](../pilots/templates/case-study.md). + [case-study.md](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/templates/case-study.md). ## In-repo vs external @@ -142,7 +142,7 @@ pentest. ## See also -- [docs/pilots/](../pilots/README.md) — templates, per-vertical kits, case studies +- [docs/pilots/](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/README.md) — templates, per-vertical kits, case studies - [reference/feedback.md](../reference/feedback.md), [reference/drift.md](../reference/drift.md), [reference/cost.md](../reference/cost.md), [reference/eval-harness.md](../reference/eval-harness.md), [reference/status-api.md](../reference/status-api.md) - [ADR-0046](../adr/ADR-0046-design-partner-pilots.md) — the pilot-program decision - [planning/phases/phase-7-pilot-ga.md](../../planning/phases/phase-7-pilot-ga.md) — Step 7.4 deliverables + acceptance diff --git a/docs/guides/operator-console-live.md b/docs/guides/operator-console-live.md index 731d193..077713e 100644 --- a/docs/guides/operator-console-live.md +++ b/docs/guides/operator-console-live.md @@ -11,22 +11,22 @@ console deps installed (`pnpm install` in `apps/admin-ui`, done by bootstrap). The default gateway is credential-free and fully observable with no infrastructure: ```bash -uv run uvicorn rag_gateway.app:app --port 8080 +uv run uvicorn rag_gateway.app:app --port 8000 ``` ## 2. Curl the status surface ```bash # Overall + per-component health (worst component wins) -curl -s localhost:8080/v1/status/health | jq '{status, components: (.components | length)}' +curl -s localhost:8000/v1/status/health | jq '{status, components: (.components | length)}' # Generate a little traffic, then look at the metrics it produced -for i in $(seq 1 5); do curl -s localhost:8080/healthz >/dev/null; done -curl -s localhost:8080/v1/status/metrics \ +for i in $(seq 1 5); do curl -s localhost:8000/healthz >/dev/null; done +curl -s localhost:8000/v1/status/metrics \ | jq '.counters[] | select(.name=="gateway.requests_total")' # Recent structured logs (filter to warnings+) -curl -s "localhost:8080/v1/status/logs?level=WARN&limit=20" | jq '.returned' +curl -s "localhost:8000/v1/status/logs?level=WARN&limit=20" | jq '.returned' ``` Health is unauthenticated (it carries no tenant data, like `/healthz`). After the @@ -38,7 +38,7 @@ roll-up — per-route health is derived from real traffic, not a synthetic probe **Log tail over SSE** (one-way, append-only — the browser's `EventSource` shape): ```bash -curl -N "localhost:8080/v1/status/logs/stream?backlog=10" +curl -N "localhost:8000/v1/status/logs/stream?backlog=10" # : open # data: {"seq": 41, "level": "INFO", "event": "gateway.query_complete", ...} # : keep-alive @@ -48,7 +48,7 @@ curl -N "localhost:8080/v1/status/logs/stream?backlog=10" ```bash # any ws client; e.g. websocat -websocat "ws://localhost:8080/v1/status/ws?channels=health,metrics&interval_ms=2000" +websocat "ws://localhost:8000/v1/status/ws?channels=health,metrics&interval_ms=2000" # {"kind":"snapshot","ts":"...","health":{...},"metrics":{...}} ``` @@ -71,7 +71,7 @@ Set the gateway URL and the same pages light up with real data: ```bash cd apps/admin-ui -NEXT_PUBLIC_GATEWAY_URL=http://localhost:8080 pnpm dev +NEXT_PUBLIC_GATEWAY_URL=http://localhost:8000 pnpm dev ``` - **Live Status** opens a WebSocket and shows live health (the "Demo data" badge diff --git a/docs/reference/gateway.md b/docs/reference/gateway.md index 4d05806..8fd9654 100644 --- a/docs/reference/gateway.md +++ b/docs/reference/gateway.md @@ -21,10 +21,10 @@ ```bash # Start the gateway against the dev / noop wiring. -uv run uvicorn rag_gateway.app:app --host 0.0.0.0 --port 8080 +uv run uvicorn rag_gateway.app:app --host 0.0.0.0 --port 8000 # Query it. -curl -s http://localhost:8080/v1/query \ +curl -s http://localhost:8000/v1/query \ -H 'content-type: application/json' \ -d '{ "tenant_id": "demo", @@ -119,7 +119,7 @@ Retrieval-only. Same identifying triple + `corpus_ids` / `top_k` / `filters` as List corpora visible to the calling tenant. Tenant is read from `X-Tenant-Id` header in dev mode or extracted from the bearer token in production. ```bash -curl -s http://localhost:8080/v1/corpora \ +curl -s http://localhost:8000/v1/corpora \ -H 'x-tenant-id: demo' \ -H 'x-principal-id: curl' | jq ``` diff --git a/docs/reference/pilot.md b/docs/reference/pilot.md index 17bbd8d..9f221d4 100644 --- a/docs/reference/pilot.md +++ b/docs/reference/pilot.md @@ -102,6 +102,6 @@ latency carry their own gates (`ragctl eval`, `ragctl perf`). ## See also - [guides/design-partner-pilots.md](../guides/design-partner-pilots.md) — the runbook -- [pilots/](../pilots/README.md) — templates + the customer-support kit +- [pilots/](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/pilots/README.md) — templates + the customer-support kit - [reference/feedback.md](feedback.md), [reference/drift.md](drift.md), [reference/cost.md](cost.md) - [ADR-0046](../adr/ADR-0046-design-partner-pilots.md) diff --git a/docs/reference/ragctl.md b/docs/reference/ragctl.md index 4252218..e06366f 100644 --- a/docs/reference/ragctl.md +++ b/docs/reference/ragctl.md @@ -3,13 +3,18 @@ ## Overview `ragctl` is the operator-facing control-plane CLI for AgentContextOS. It bundles -configuration, retrieval, evaluation, observability, and tenant-management -commands into a single binary that talks to local files, the gateway service, -and the supporting infrastructure (Jaeger, etc.). +configuration, ingestion, retrieval, gateway, reliability, observability, and +governance commands into a single binary that talks to local files, an +in-process gateway, and the supporting infrastructure (Jaeger, etc.). -Step 0.10 delivers the consolidated CLI scaffold. Each sub-command group is -wired into the root `ragctl` app so the surface area is discoverable today, -even where the underlying functionality has yet to land. +Most commands are **self-contained, credential-free demos**: they drive the +real pipeline (or a real in-process FastAPI/gRPC/MCP gateway) against the noop +SPI wiring, so every platform capability can be exercised on a laptop with no +infrastructure. That makes `ragctl` both the operator tool and the fastest way +to see any subsystem work end-to-end. + +The surface today: **37 commands + 13 sub-command groups**, all working except +three scaffolds (`plugin`, `logs`, `secret`) that print a notice and exit `0`. ## Usage @@ -21,23 +26,104 @@ ragctl --help ragctl version ``` -## Command groups - -| Group | Step | Status | What it does | -|-----------|---------|----------|--------------| -| `config` | 0.4 | working | Validate and diff `rag.yaml` files. | -| `traces` | 0.7 | working | Query distributed traces from Jaeger. | -| `eval` | 0.8 | working | Run and inspect golden-set evaluations. | -| `version` | 0.10 | working | Print the installed `ragctl` version. | -| `plugin` | 1.1 | scaffold | Manage SPI plugin registration. | -| `ingest` | 1.10 | scaffold | Trigger ingestion pipelines via the gateway. | -| `query` | 3.1 | scaffold | Run a query against the gateway. | -| `logs` | 5.6 | scaffold | Tail structured logs from the platform. | -| `tenant` | 6.1 | scaffold | Manage tenants. | -| `secret` | 6.7 | scaffold | Manage tenant secrets. | - -Scaffold commands exit `0` and print a one-line "delivered in Step X.Y" -notice so operators can probe the planned interface without surprise. +Run via the workspace until the standalone PyPI package lands: +`uv run ragctl `. + +## Command catalog + +### Ingestion pipeline (Phase 1) + +| Command | What it does | +|---------|--------------| +| `parse` | Detect MIME, parse a file, print a structural summary (Step 1.3) | +| `ocr` | Run an OCR plugin on an image, print region-aware results (Step 1.4) | +| `chunk` | Parse, then chunk via `HeadingAwareChunker` (Step 1.5) | +| `enrich` | parse → chunk → enrich; print enriched metadata (Step 1.6) | +| `pii` | parse → chunk → enrich → PII; print detection outcomes (Step 1.7) | +| `embed` | The pipeline end-to-end through embedding (Step 1.8) | +| `ingest` | Full ingest pipeline against a folder or file (Step 1.10) | + +### Retrieval & query understanding (Phase 2) + +| Command | What it does | +|---------|--------------| +| `hybrid` | Hybrid RRF retrieval over an in-memory toy corpus (Step 2.5) | +| `understand` | `QueryUnderstandingPipeline` over one query (Step 2.6) | +| `rerank` | `RerankPipeline` over a tiny in-memory corpus (Step 2.7) | +| `pack` | `ContextPacker` against a toy corpus (Step 2.8) | +| `graphrag` | GraphRAG detect → summarize → retrieve over a toy graph (Step 2.9) | +| `route` | Retrieval router classifying + dispatching a query (Step 2.10) | +| `agent-loop` | `AgentLoopV0` over a single query end-to-end (Step 2.11) | + +### Gateway surfaces — in-process (Phase 3) + +| Command | What it does | +|---------|--------------| +| `query` | Drive `POST /v1/query` against an in-process gateway (Step 3.1) | +| `grpc-query` | Drive `RagService.Query` against an in-process gRPC server (Step 3.2) | +| `mcp-serve` | Run the MCP server over stdio (Step 3.3) | +| `mcp-query` | Drive the MCP `query` tool against an in-memory server (Step 3.3) | +| `chat` | Drive `POST /v1/chat/completions` (OpenAI-compatible, Step 3.4) | +| `embeddings` | Drive `POST /v1/embeddings` (OpenAI-compatible, Step 3.4) | +| `agent` | Drive `POST /v1/agent` — agent loop over SSE (Step 3.6) | +| `corpus` *(group)* | Inspect the corpus store + exercise the corpus router (Step 3.5) | +| `webhooks` *(group)* | Exercise outbound webhook signing + delivery (Step 3.9) | + +### Reliability (Phase 4) + +| Command | What it does | +|---------|--------------| +| `fallback` | Fallback chain degrading a query gracefully (Step 4.2) | +| `guard` | Hallucination guard checking an answer's faithfulness (Step 4.3) | +| `breaker` | Circuit breaker isolating a failing backend (Step 4.4) | +| `quota` | Quota enforcer admitting + throttling a tenant (Step 4.5) | +| `perf` | Probe the in-process gateway's latency overhead (Step 4.6) | + +### Observability & eval (Phases 0 + 5) + +| Command | What it does | +|---------|--------------| +| `config` *(group)* | Validate and diff `rag.yaml` files (Step 0.4) | +| `traces` *(group)* | Query distributed traces from Jaeger (Step 0.7) | +| `eval` *(group)* | Run and inspect golden-set evaluations (Step 0.8 / 5.2) | +| `provenance` | Per-query provenance: build → sign → store → verify (Step 5.1) | +| `feedback` | Record explicit + implicit signals → aggregate (Step 5.4) | +| `drift` | Seed a reference, feed a drifted current, report (Step 5.5) | +| `experiments` | A/B analyzer: control + candidate samples → lift + CI (Step 5.7) | +| `shadow` | Shadow mode end-to-end against in-process stubs (Step 5.7b) | +| `ab` | A/B routing end-to-end against in-process stubs (Step 5.7c) | + +### Governance & enterprise (Phase 6) + +| Command | What it does | +|---------|--------------| +| `tenant` *(group)* | Inspect per-tenant config + resolve logical tenancy (Step 6.1) | +| `audit` | Signed WORM audit export: seed → export → verify (Step 6.6b) | +| `kms` | BYOK envelope encryption with an in-process key manager (Step 6.7) | +| `scim` | SCIM 2.0 provisioning against an in-process directory (Step 6.8) | +| `sso` *(group)* | Per-tenant SSO config + OIDC federation demo (Step 6.8) | +| `airgap` *(group)* | Build / verify / install the air-gapped bundle (Step 6.9) | +| `compliance` *(group)* | Compliance posture + data-retention / GDPR erasure demo (Step 6.10) | + +### Commercial & pilots (Phase 7) + +| Command | What it does | +|---------|--------------| +| `pilot` *(group)* | Onboard a design-partner pilot + report weekly KPIs (Step 7.4) | +| `billing` | Metering → invoice flow: meter usage, then bill it (Step 7.9) | + +### Scaffolds + +| Group | Planned step | What it will do | +|-------|--------------|-----------------| +| `plugin` | 1.1 | Manage SPI plugin registration | +| `logs` | 5.6 | Tail structured logs from the platform | +| `secret` | 6.7 | Manage tenant secrets | + +Scaffold commands exit `0` and print a one-line notice so operators can probe +the planned interface without surprise. + +## Worked examples ### `ragctl config` @@ -47,6 +133,16 @@ ragctl config validate path/to/rag.yaml --json # machine-readable result ragctl config diff path/a.yaml path/b.yaml # show backend differences ``` +### `ragctl query` + +```bash +uv run ragctl query "what is RAG?" --top-k 3 +``` + +Spins up the in-process FastAPI gateway against the noop wiring and runs the +full understand → route → rerank → pack pipeline — the quickest end-to-end +smoke test there is. + ### `ragctl eval` ```bash @@ -75,6 +171,9 @@ ragctl traces --url http://jaeger.observability.svc:16686 # remote Jaeger ragctl version # → ragctl 0.1.0 ``` +Every other command follows the same shape — run it with `--help` for its +flags; each prints a structured, human-readable walkthrough of its subsystem. + ## Shell completion `ragctl` ships with Typer-powered completion for bash, zsh, fish, and @@ -96,56 +195,49 @@ via `[project.scripts]` in `pyproject.toml`. ``` packages/ragctl/ -├── pyproject.toml # rag-ragctl, depends on rag-core/config/observability +├── pyproject.toml # rag-ragctl, depends on rag-core/config/observability + domain packages ├── README.md # short package overview └── src/ragctl/ ├── __init__.py # exports app, main, __version__ - ├── main.py # root Typer app + all sub-command groups + ├── main.py # root Typer app + all commands and sub-command groups └── py.typed # type marker for downstream consumers ``` -### Dependencies - -`rag-ragctl` depends on: -- `rag-core` — domain types and errors -- `rag-config` — `rag.yaml` loader and eval metrics (powers `config` + `eval` groups) -- `rag-observability` — pulled in transitively for log context -- `typer>=0.12` — CLI framework - -The CLI keeps **no business logic** of its own. Each command is a thin -adapter over a function exported from the corresponding domain package. +### Design -### Scaffold sub-apps +The CLI keeps **no business logic** of its own. Each command is a thin adapter +over a function exported from the corresponding domain package, and the +gateway-surface commands build the same in-process app the test suite uses +(`build_app` with noop SPIs) — so a `ragctl` run exercises exactly the code +that ships. -Sub-apps that aren't implemented yet are constructed by the local helper -`_scaffold_app(group, step, help_text)`. The helper creates a Typer sub-app -whose default callback prints a "delivered in Step X.Y" notice and exits 0. -This keeps the published command shape stable from day one — operators can -script against `ragctl ingest ...` today and the same script will keep -working when Step 1.10 lands the real implementation. +The three remaining scaffold groups are constructed by the local helper +`_scaffold_app(group, step, help_text)`, which prints a "delivered in +Step X.Y" notice and exits 0. This keeps the published command shape stable — +operators can script against `ragctl logs ...` today and the same script keeps +working when the real implementation lands. ## Extension points -To add a new sub-command group: +To add a new command or sub-command group: 1. Create or import the domain logic in the appropriate package - (`rag-core`, `rag-config`, future `rag-retrieval`, etc.). -2. In `ragctl/main.py`, build a new `typer.Typer()` sub-app, register - commands on it, and attach it to the root `app` with `app.add_typer(...)`. -3. Update the **Command groups** table in this file and the README. + (`rag-core`, `rag-config`, the relevant `packages/`, etc.). +2. In `ragctl/main.py`, register an `@app.command()` (or build a + `typer.Typer()` sub-app and attach it with `app.add_typer(...)`). +3. Update the **Command catalog** table in this file and the package README. 4. Add tests under `packages/ragctl/tests/`. To replace a scaffold with a real implementation: 1. Remove the `_scaffold_app(...)` call for that group from `main.py`. 2. Build the real sub-app the same way you would for a new group. -3. Update the **Command groups** table — change Status from `scaffold` to - `working` and link to the deeper reference page. +3. Move the row out of the **Scaffolds** table in this file. ## See also - [Quickstart](../guides/ragctl-quickstart.md) — getting started in 5 minutes - [Eval framework architecture](../architecture/eval-skeleton.md) — what drives `ragctl eval` -- [IaC overview](../architecture/iac.md) — infra surfaces the CLI will - manage in later steps +- [Gateway reference](gateway.md) — the surfaces `query` / `chat` / + `embeddings` / `agent` drive in-process diff --git a/docs/reference/rest-api.md b/docs/reference/rest-api.md index 1024546..b836725 100644 --- a/docs/reference/rest-api.md +++ b/docs/reference/rest-api.md @@ -19,7 +19,7 @@ Run an agent loop over a goal, streaming events as SSE Drive the agent loop for ``body.goal`` and stream its events. -See ``docs/reference/agent.md`` (``#post-v1agent``). +See [agent.md](agent.md). **Request body** (`application/json`): `AgentRequest` (required) @@ -118,7 +118,7 @@ List corpora visible to the calling tenant Return every corpus the calling principal can read. -See docs/reference/gateway.md +See [docs/reference/gateway.md](gateway.md#get-v1corpora) for the response shape + pagination semantics (paging is a Phase 3.x add — v0 returns everything visible). @@ -215,7 +215,7 @@ Create a chat completion with retrieval pre-fetch (OpenAI-compatible) RAG chat completion: retrieve → inject context → generate. -See ``docs/reference/openai-compat.md`` (``#post-v1chatcompletions``). +See [openai-compat.md](openai-compat.md#post-v1chatcompletions). **Request body** (`application/json`): `ChatCompletionRequest` (required) @@ -237,7 +237,7 @@ Create embeddings (OpenAI-compatible) Embed text via the wired ``Embedder`` SPI. -See ``docs/reference/openai-compat.md`` (``#post-v1embeddings``). +See [openai-compat.md](openai-compat.md#post-v1embeddings). **Request body** (`application/json`): `EmbeddingsRequest` (required) @@ -278,7 +278,7 @@ End-to-end RAG query. Runs the Phase 2 pipeline in sequence — understanding, router, optional rerank, optional pack, optional LLM — and returns the combined evidence envelope. See -docs/reference/gateway.md. +[docs/reference/gateway.md](gateway.md#post-v1query). **Request body** (`application/json`): `QueryRequest` (required) diff --git a/docs/reference/status-api.md b/docs/reference/status-api.md index 189e187..11c8525 100644 --- a/docs/reference/status-api.md +++ b/docs/reference/status-api.md @@ -38,7 +38,7 @@ different origin can call them; origins come from `RAG_GATEWAY_CORS_ORIGINS` ### Health — `GET /v1/status/health` ```bash -curl -s localhost:8080/v1/status/health | jq +curl -s localhost:8000/v1/status/health | jq ``` ```json @@ -65,7 +65,7 @@ curl -s localhost:8080/v1/status/health | jq ### Metrics — `GET /v1/status/metrics` ```bash -curl -s localhost:8080/v1/status/metrics | jq '.histograms[0]' +curl -s localhost:8000/v1/status/metrics | jq '.histograms[0]' ``` ```json @@ -98,7 +98,7 @@ Query params: `limit` (1–2000, default 200), `level` (minimum severity), `even in chronological order: ```bash -curl -s "localhost:8080/v1/status/logs?level=WARN&limit=50" | jq '.records[-1]' +curl -s "localhost:8000/v1/status/logs?level=WARN&limit=50" | jq '.records[-1]' ``` ```json @@ -111,7 +111,7 @@ curl -s "localhost:8080/v1/status/logs?level=WARN&limit=50" | jq '.records[-1]' ### Log stream — `GET /v1/status/logs/stream` (SSE) ```bash -curl -N "localhost:8080/v1/status/logs/stream?level=INFO&backlog=20" +curl -N "localhost:8000/v1/status/logs/stream?level=INFO&backlog=20" ``` ``` @@ -129,7 +129,7 @@ Query params: `level`, `event`, `tenant`, `interval_ms` (250–10000, default 10 ### Live push — `WS /v1/status/ws` (WebSocket) ```js -const ws = new WebSocket("ws://localhost:8080/v1/status/ws?channels=health,metrics&interval_ms=2000"); +const ws = new WebSocket("ws://localhost:8000/v1/status/ws?channels=health,metrics&interval_ms=2000"); ws.onmessage = (e) => { const frame = JSON.parse(e.data); // {kind:"snapshot", ts, health?, metrics?} }; diff --git a/docs/release-notes/v1.0.0.md b/docs/release-notes/v1.0.0.md index cdf2916..6e51056 100644 --- a/docs/release-notes/v1.0.0.md +++ b/docs/release-notes/v1.0.0.md @@ -64,4 +64,5 @@ the deprecation policy. Support tiers + SLAs: [support-sla.md](../guides/support ## GA readiness Every Phase 7 exit gate is recorded in the -[GA readiness checklist](../ga/ga-readiness-checklist.md). +[GA readiness checklist](https://github.com/officialCodeWork/AgentContextOS/blob/main/docs/ga/ga-readiness-checklist.md) +(internal tracking — kept on GitHub, not published on the docs site). diff --git a/packages/ragctl/src/ragctl/main.py b/packages/ragctl/src/ragctl/main.py index 4c56e3e..f82afb1 100644 --- a/packages/ragctl/src/ragctl/main.py +++ b/packages/ragctl/src/ragctl/main.py @@ -1,33 +1,12 @@ """ragctl — AgentContextOS control-plane CLI. -Root entry point for the consolidated CLI. Sub-command groups land step by step: - -| Group | Step delivered | Status | -|-----------|-----------------|----------| -| config | 0.4 | working | -| traces | 0.7 | working | -| eval | 0.8 | working | -| version | 0.10 | working | -| parse | 1.3 | working | -| ocr | 1.4 | working | -| chunk | 1.5 | working | -| enrich | 1.6 | working | -| pii | 1.7 | working | -| embed | 1.8 | working | -| ingest | 1.10 | working | -| hybrid | 2.5 | working | -| understand| 2.6 | working | -| rerank | 2.7 | working | -| pack | 2.8 | working | -| corpus | 3.5 | working | -| tenant | 6.1 | working | -| plugin | 1.1 | scaffold | -| query | 3.1 | scaffold | -| logs | 5.6 | scaffold | -| secret | 6.7 | scaffold | - -Scaffold commands print a one-line "delivered in step X.Y" notice and exit 0 -so operators can discover the planned surface area today. +Root entry point for the consolidated CLI. The full command catalog lives in +``docs/reference/ragctl.md`` — keep the status there, not here (a per-command +table in this docstring drifted as commands landed). + +Three groups remain scaffolds that print a one-line "delivered in step X.Y" +notice and exit 0 so operators can discover the planned surface area today: +``plugin`` (1.1), ``logs`` (5.6), ``secret`` (6.7). """ from __future__ import annotations diff --git a/scripts/gen_api_reference.py b/scripts/gen_api_reference.py index 89da1c5..02aa616 100644 --- a/scripts/gen_api_reference.py +++ b/scripts/gen_api_reference.py @@ -22,14 +22,72 @@ DEFAULT_OUT = ROOT / "docs" / "reference" / "rest-api.md" _METHOD_ORDER = ["get", "post", "put", "patch", "delete"] -_MD_LINK = re.compile(r"\[([^\]]+)\]\([^)]+\)") +_MD_LINK = re.compile(r"\[([^\]]+)\]\(([^)]+)\)") +_RST_DOC_REF = re.compile( + r"``docs/(?P[a-z-]+)/(?P[A-Za-z0-9._-]+\.md)``" + r"(?:\s*\(``(?P#[A-Za-z0-9_-]+)``\))?" +) +_DOC_PATH = re.compile(r"docs/(?P[a-z-]+)/(?P[A-Za-z0-9._-]+\.md)$") +_HEADING = re.compile(r"^#{1,6}\s+(.+?)\s*$", re.MULTILINE) +_NON_SLUG = re.compile(r"[^\w\s-]") +_ANCHOR_CACHE: dict[Path, frozenset[str]] = {} + + +def _slug(heading: str) -> str: + """GitHub-style heading slug: lowercase, drop punctuation, spaces → '-'.""" + cleaned = _NON_SLUG.sub("", heading.strip().lower()) + return re.sub(r"\s", "-", cleaned) + + +def _anchors(path: Path) -> frozenset[str]: + if path not in _ANCHOR_CACHE: + text = path.read_text(encoding="utf-8") + _ANCHOR_CACHE[path] = frozenset(_slug(m.group(1)) for m in _HEADING.finditer(text)) + return _ANCHOR_CACHE[path] + + +def _doc_href(sub: str, file: str, anchor: str) -> str: + """Build a link target that resolves from ``docs/reference/`` — dropping + the fragment when the target file has no matching heading (docstring + anchor hints go stale; a file-level link beats a broken one).""" + rel = file if sub == "reference" else f"../{sub}/{file}" + if anchor and anchor.lstrip("#") not in _anchors(ROOT / "docs" / sub / file): + anchor = "" + return f"{rel}{anchor}" def _plain(text: str | None) -> str: - """Flatten Markdown links to their label (drop relative URLs that wouldn't - resolve from ``docs/reference/``) and trim — descriptions come from source - docstrings authored relative to other files.""" - return _MD_LINK.sub(r"\1", text or "").strip() + """Normalise description text into Markdown that works from + ``docs/reference/rest-api.md``. + + Source docstrings reference other docs two ways: reST literals + (````docs/reference/agent.md`` (``#anchor``)``) and Markdown links whose + relative URLs are authored against the *source file* location. Both are + rewritten into links that resolve from ``docs/reference/`` (verified + against the working tree); absolute http(s) links pass through; anything + unresolvable keeps only its label.""" + + def _rst(m: re.Match[str]) -> str: + sub, file = m.group("sub"), m.group("file") + if not (ROOT / "docs" / sub / file).exists(): + return f"`docs/{sub}/{file}`" + return f"[{file}]({_doc_href(sub, file, m.group('anchor') or '')})" + + def _link(m: re.Match[str]) -> str: + label, url = m.group(1), m.group(2) + if url.startswith(("http://", "https://", "#")): + return m.group(0) + path, _, frag = url.partition("#") + anchor = f"#{frag}" if frag else "" + if (ROOT / "docs" / "reference" / path).resolve().exists(): + return m.group(0) # already resolves from docs/reference/ + dm = _DOC_PATH.search(path) + if dm and (ROOT / "docs" / dm.group("sub") / dm.group("file")).exists(): + return f"[{label}]({_doc_href(dm.group('sub'), dm.group('file'), anchor)})" + return label + + text = _RST_DOC_REF.sub(_rst, text or "") + return _MD_LINK.sub(_link, text).strip() def _params_table(params: list[dict[str, Any]]) -> list[str]: diff --git a/website/docusaurus.config.ts b/website/docusaurus.config.ts index 315bddb..09d2dff 100644 --- a/website/docusaurus.config.ts +++ b/website/docusaurus.config.ts @@ -32,6 +32,18 @@ const config: Config = { path: "../docs", routeBasePath: "/", sidebarPath: "./sidebars.ts", + // Internal-only areas stay on GitHub and are NOT published on the + // public site. The first four globs restore the Docusaurus defaults + // (setting `exclude` replaces them); the rest are deliberate + // publishing decisions — flip a line to change one. + exclude: [ + "**/_*.{js,jsx,ts,tsx,md,mdx}", + "**/_*/**", + "**/*.test.{js,jsx,ts,tsx}", + "**/__tests__/**", + "ga/**", // internal GA-readiness process tracking + "pilots/**", // partner-program templates + kits (GTM internal) + ], editUrl: "https://github.com/officialCodeWork/AgentContextOS/tree/main/", }, diff --git a/website/sidebars.ts b/website/sidebars.ts index 982fe98..e8d2adc 100644 --- a/website/sidebars.ts +++ b/website/sidebars.ts @@ -1,9 +1,10 @@ import type { SidebarsConfig } from "@docusaurus/plugin-content-docs"; -// Auto-generated from the ../docs directory tree. Each subfolder (architecture, -// reference, guides, adr, pilots, compliance, research, spikes) becomes a -// category; ordering falls back to alphabetical unless a doc sets -// `sidebar_position` frontmatter. +// Auto-generated from the ../docs directory tree. Each published subfolder +// (adr, architecture, compliance, guides, reference, release-notes, research, +// runbooks, spikes) becomes a category; internal areas (ga/, pilots/) are +// excluded in docusaurus.config.ts. Ordering falls back to alphabetical +// unless a doc sets `sidebar_position` frontmatter. const sidebars: SidebarsConfig = { docsSidebar: [{ type: "autogenerated", dirName: "." }], }; From e26f637b29b0e511052863f35b7d81232319b79b Mon Sep 17 00:00:00 2001 From: Deep Kumar Singh Kushwah Date: Wed, 10 Jun 2026 15:12:16 +0530 Subject: [PATCH 2/2] =?UTF-8?q?docs(readme):=20proper=20"how=20to=20run"?= =?UTF-8?q?=20README=20=E2=80=94=20demo,=20real=20backends,=20console,=20p?= =?UTF-8?q?roduction?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restructures the root README around running the product, with every instruction verified against the repo: - "What's running once it's up" — gateway surfaces/ports, admin console, ragctl, so a new user knows what success looks like. - Demo mode: clone → uv sync → uvicorn :8000 → curl ingest + query → ragctl one-liner, with the honest two-process noop-store caveat. - Real backends: make dev stack, a working rag.yaml provider snippet, the examples/configs/ profiles, ragctl config validate, and a tested serve.py (rag_config.load + build_app_from_config — builds 58 routes). - Admin console: pnpm filter command + gateway URL env. - Production channels: container / Helm / PyPI / air-gap install commands, with an explicit known-gap note — the v1.0.0 image default CMD does not start the server, so the serve command must be passed (Helm inherits the gap); config-driven entrypoint tracked for v1.0.1. - Surface guides table (REST/SDK/OpenAI/MCP/gRPC/adapters/webhooks/agent). Co-Authored-By: Claude Fable 5 --- README.md | 147 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 126 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 41cc6ea..45772e9 100644 --- a/README.md +++ b/README.md @@ -2,40 +2,144 @@ **Production-grade, multi-tenant RAG platform** — pluggable retrieval, enterprise governance, agent loops, and full observability behind one API. **v1.0.0 GA.** -Four API surfaces — REST, gRPC, MCP, and OpenAI-compatible — over one engine: hybrid retrieval (dense + sparse + graph) with reranking, hallucination guarding, signed per-query provenance, tenant isolation with ACL/PII enforcement, quotas, drift monitoring, eval gates, and billing. Every backend (vector store, embedder, LLM, …) is a swappable plugin behind an SPI, and `rag.yaml` is the portability contract — the same image runs on a laptop, a VM, and Kubernetes. +Four API surfaces — REST, gRPC, MCP, and OpenAI-compatible — over one engine: hybrid retrieval (dense + sparse + graph) with reranking, hallucination guarding, signed per-query provenance, tenant isolation with ACL/PII enforcement, quotas, drift monitoring, eval gates, and billing. Every backend (vector store, embedder, LLM, …) is a swappable plugin behind an SPI, and `rag.yaml` is the portability contract — the same code runs on a laptop, a VM, and Kubernetes. -## Run a query in 5 minutes +## What's running once it's up -No credentials, no infrastructure: +| Component | Where | What you get | +|-----------|-------|--------------| +| **Gateway** | `http://localhost:8000` | REST (`/v1/query`, `/v1/retrieve`, `/v1/ingest/document`, `/v1/feedback`, …), OpenAI-compatible (`/v1/chat/completions`, `/v1/embeddings`, `/v1/models`), gRPC, and MCP — plus `/v1/status/*` (health / metrics / logs) and Swagger UI at `/docs` | +| **Admin console** | `http://localhost:3100` | Operator GUI: live status, metrics, log tail, tenants, corpora, connectors | +| **`ragctl`** | your terminal | Control-plane CLI — 37 commands + 13 groups (config, eval, drift, pilots, billing, …); see the [command reference](docs/reference/ragctl.md) | + +## Prerequisites + +- [`uv`](https://docs.astral.sh/uv/) — manages Python 3.12+ and all server dependencies (**required**) +- Docker — only needed to run real backends locally (the demo needs none) +- Node 20 + `pnpm` — only for the admin console and the docs site + +## 1. Try it in 5 minutes — demo mode + +Zero credentials, zero infrastructure. The gateway starts against an in-memory noop wiring that still exercises the full real pipeline (understand → route → retrieve → rerank → pack): ```bash git clone https://github.com/officialCodeWork/AgentContextOS.git cd AgentContextOS && uv sync -uv run uvicorn rag_gateway.app:app --port 8000 # terminal 1 -curl -s http://localhost:8000/healthz # terminal 2 → {"status":"ok"} +# Terminal 1 — start the gateway +uv run uvicorn rag_gateway.app:app --port 8000 + +# Terminal 2 — check it's up +curl -s http://localhost:8000/healthz # → {"status":"ok"} +``` + +Ingest a document, then query: + +```bash +curl -s -X POST http://localhost:8000/v1/ingest/document \ + -F 'tenant_id=demo' -F 'corpus_id=docs' -F 'principal_id=me' \ + -F 'file=@README.md' | jq '{document_id, status, chunk_count}' + +curl -s http://localhost:8000/v1/query \ + -H 'content-type: application/json' \ + -d '{"tenant_id":"demo","principal_id":"me","query":"what is AgentContextOS?","top_k":3}' \ + | jq '{chunks: (.chunks|length), citations: (.citations|length), timings}' +``` + +Or run the whole ingest-and-query loop in one process with the CLI: + +```bash +uv run ragctl query "what is RAG?" --top-k 3 +``` + +Heads-up: in demo mode the two terminals don't share the in-memory store, so cross-process queries come back empty — use `ragctl query` for a populated demo, or wire real backends (next section). Full walkthrough with expected outputs: **[curl quickstart](docs/guides/curl-quickstart.md)**. + +## 2. Run with real backends + +**Start the backing services** (Postgres + pgvector, Qdrant, Redis, Elasticsearch, MinIO): + +```bash +make dev # or: task dev +make dev-full # + OTel collector, Jaeger, Prometheus, Grafana, Loki +``` + +**Configure `rag.yaml`** — change providers here, nothing else changes: + +```yaml +backends: + vector_store: { provider: qdrant } + keyword_store: { provider: elasticsearch } + embedder: { provider: openai, model: text-embedding-3-large } + llm: { provider: anthropic, model: claude-sonnet-4-6 } + cache: { provider: redis } +``` + +Ready-made profiles for each deployment shape live in [`examples/configs/`](examples/configs/) — `laptop.yaml`, `single-vm.yaml`, `k8s-onprem.yaml`, `saas.yaml`, `airgap.yaml`. Validate any config with: + +```bash +uv run ragctl config validate rag.yaml +``` + +**Serve the configured app.** The gateway builds its wiring from config via `build_app_from_config`: + +```python +# serve.py +from rag_config import load +from rag_gateway.wiring import build_app_from_config + +app = build_app_from_config(load("rag.yaml")) +``` + +```bash +uv run uvicorn serve:app --host 0.0.0.0 --port 8000 ``` -Follow the **[curl quickstart](docs/guides/curl-quickstart.md)** for ingest + query end-to-end, then pick your surface: -[SDKs](docs/guides/sdk-quickstart.md) · -[OpenAI-compatible](docs/guides/openai-quickstart.md) · -[MCP](docs/guides/mcp-quickstart.md) · -[gRPC](docs/guides/grpcurl-quickstart.md) · -[`ragctl` CLI](docs/guides/ragctl-quickstart.md) · -[admin console](docs/guides/admin-ui-quickstart.md) +For fully programmatic wiring — injecting backend instances directly instead of via config — see [production wiring](docs/reference/gateway.md#production-wiring). -## Development setup +## 3. Open the admin console ```bash -# Prerequisites: uv, pnpm, docker -./scripts/bootstrap.sh # or: make bootstrap +pnpm install +NEXT_PUBLIC_GATEWAY_URL=http://localhost:8000 pnpm --filter @ragplatform/admin-ui dev +# → http://localhost:3100 +``` + +Without a gateway URL it renders seed data (every page carries a "Demo data" badge), so you can preview the console standalone. Guide: [admin-ui quickstart](docs/guides/admin-ui-quickstart.md) · live-data tour: [operator console](docs/guides/operator-console-live.md). + +## 4. Deploy to production + +| Channel | Install | Notes | +|---------|---------|-------| +| **Container** | `docker run -p 8000:8000 ghcr.io/officialcodework/agentcontextos/rag-gateway:v1.0.0 uvicorn rag_gateway.app:app --host 0.0.0.0 --port 8000` | cosign-signed, SBOM-attested | +| **Helm** | `helm install rag oci://ghcr.io/officialcodework/agentcontextos/charts/rag-platform` | chart source in [`infra/helm/rag-platform/`](infra/helm/rag-platform/); wire backends via `values.yaml` | +| **PyPI** | `pip install rag-platform` | meta-package pins all components to a reproducible server stack | +| **Air-gapped** | signed offline bundle + `install.sh` | [air-gap install guide](docs/guides/airgap-install.md) | + +> **Known gap at v1.0.0:** the published image's *default* CMD prints the version instead of starting the server — pass the `uvicorn` serve command explicitly as shown above. The Helm chart inherits the same gap (its pods rely on the image default), so review the chart against your serve command before first install. A config-driven serve entrypoint is tracked for v1.0.1. -make dev # start postgres + redis + qdrant -make dev-full # + otel-collector, jaeger, prometheus, grafana, loki +One `vX.Y.Z` tag publishes every channel — signed (cosign keyless) with SBOMs. Details: [packaging & distribution](docs/guides/packaging-distribution.md). + +## Talk to it from your stack + +| Surface | Guide | +|---------|-------| +| REST + curl | [curl-quickstart](docs/guides/curl-quickstart.md) | +| Python / TypeScript SDKs | [sdk-quickstart](docs/guides/sdk-quickstart.md) | +| OpenAI-compatible (drop-in `chat/completions`) | [openai-quickstart](docs/guides/openai-quickstart.md) | +| MCP — Claude Desktop, Cursor, VS Code | [mcp-quickstart](docs/guides/mcp-quickstart.md) | +| gRPC | [grpcurl-quickstart](docs/guides/grpcurl-quickstart.md) | +| LangChain / LlamaIndex / Semantic Kernel adapters | [integrations-quickstart](docs/guides/integrations-quickstart.md) | +| Webhooks (ingest / drift / feedback events) | [webhooks-quickstart](docs/guides/webhooks-quickstart.md) | +| Agent loop (`POST /v1/agent`) | [agent-quickstart](docs/guides/agent-quickstart.md) | + +## Developing on the platform + +```bash +./scripts/bootstrap.sh # uv sync + pnpm install + pre-commit hooks (or: make bootstrap) -make lint # ruff + mypy --strict -make test # full test suite -make help # all targets +make lint # ruff + mypy --strict +make test # full test suite +make help # all targets ``` ## Repository layout @@ -55,7 +159,7 @@ AgentContextOS/ ├── tests/ # Cross-package suites: contract, logs, policy, docs, eval ├── proto/ # core.proto — canonical cross-language schema ├── dist/ # Generated + committed + drift-gated: OpenAPI, JSON schemas -├── infra/ # Helm chart, Terraform, OTel / Prometheus configs +├── infra/ # Helm chart, Terraform, OTel / Prometheus / Grafana configs ├── packaging/ # Docker image + air-gapped bundle build ├── marketplace/ # Cloud marketplace listing artifacts + pricing model ├── planning/ # The 8-phase execution plan the build followed @@ -72,5 +176,6 @@ AgentContextOS/ - [High-Level Design](docs/architecture/RAG-Platform-HLD.md) — architecture, components, problem → capability map - [REST API reference](docs/reference/rest-api.md) — generated from the drift-gated OpenAPI contract - [v1.0.0 release notes](docs/release-notes/v1.0.0.md) — what shipped at GA +- [Support & SLAs](docs/guides/support-sla.md) — support tiers, SLA targets, runbooks - [Problem catalog](docs/research/enterprise-rag-problems.md) — the 22+ enterprise RAG failure modes this platform addresses - [Execution plan](planning/README.md) — the 8-phase build plan, now complete ([TRACKER.md](TRACKER.md) is the step-by-step record)