🧬 Also the LLM plane of ml-platform — and still standalone

This repository's core, use-case, eval harness and twelve ADRs were vendored with their git history into the ml-platform monorepo, where the agent core became libs/llm-core and the store assistant became projects/store-assistant.

This repository was not archived, and stays live — as a distribution. ml-platform's libs/llm-core is authoritative for the agent core (platform-ADR-010), and core/ here is a one-way export of it: pinned to a source commit in core/EXPORTED_FROM.json, with a hash of every file. CI re-runs the export at that commit and fails on any difference, so a hand edit cannot pass by updating the hashes too. This repository is how to take the agent core without the platform around it.

So changes to core/ go to ml-platform, and arrive here on the next export. Everything outside core/ — the tienda use-case, the app, the evaluation harness — is this repository's own.

Why the code was consolidated. This repository carried an explicit cross-repository contract with the sibling template — two CI configurations, two changelogs, two ADR sets, two coherence systems, and a shared plan living in one repository while governing another. A third participant would have made that coordination cost superlinear. Consolidation removes the coordination rather than optimising it. The reasoning, and a dated Correction reversing the original decision to archive this repository, are both in ml-platform ADR-002.

The commits that closed this repository's first phase are worth reading on their own: ADR-011 and ADR-012 re-derived the tier topology from measured hardware, and ADR-012 carries a same-day Correction documenting two wrong claims — a budget taken from a single reading of a fluctuating quantity, and a model rejected on a benchmark that had been run under the assumption it was cited to justify. Both are preserved rather than edited away. That episode is the founding evidence for ml-platform's verification rules.

---

agent-local — Reusable Local LLM Agent Platform

A business-agnostic, multi-tier local LLM agent core that teams can adopt across domains. The valuable logic — grammar-constrained routing, an adaptive reasoning loop, objective escalation and a deterministic policy gate — lives in a reusable core/. A new domain is a thin usecases/<name>/ folder, never a fork of the core (see ADR-001).

The shipped example use-case, tienda, is a WhatsApp store assistant.

🧬 Part of a lineage — standalone, and exported from a platform

This repository is the LLM plane of a deliberately connected line of work:

1. ML-MLOps Portfolio — three production ML services; the lessons were paid for here.
2. ml-service-template — those lessons encoded as a governed scaffold for classical ML on Kubernetes (scikit-learn, XGBoost, LightGBM; single team, 1–5 models).
3. agent-local (this repo) — the same governance philosophy (AUTO/CONSULT/STOP, eval-gated autonomy, policy-as-data, no fine-tuning yet) generalized to a new domain: local LLM agents.
4. ml-platform — the multi-project substrate above the template's boundary, which vendored this repository's core in as libs/llm-core and its use-case as projects/store-assistant — and which is now authoritative for that core.

The cross-repository contract described here previously was dissolved, not optimised. ml-platform platform-ADR-002 consolidated the code precisely so that no plan document in one repository governs another. What remains is one-way, and it runs from the platform: the core is developed there and exported here. It stays business-agnostic because the platform enforces it — its library may not import a use-case, so the exported core resolves none by name, and build_agent(load_usecase(root), registry) is wired by the caller. See ADR-001 for why that agnosticism is the property worth protecting.

Status: Phase 1 (read-only, fixtures). Routing quality gate PASSED (20/20) on the Tier-0 router — an initial 19/20 was re-scored after bench/RESULTS.md found the router right and the gold label wrong. Code is structured for the full multi-tier stack.

---

Why this exists

Most "LLM agent" code couples the loop, prompts and business rules into one app. That doesn't scale to multiple use-cases: the safety-critical logic diverges across copies. Here, that logic is centralized and consumed by configuration:

core/                 # business-agnostic engine — single source of truth
  config.py           #   UsecaseConfig loader
  schemas.py          #   typed Pydantic contracts
  router.py           #   Tier-0 router (GBNF-constrained JSON)
  tiers.py            #   tier clients (endpoints injected from config)
  tools.py            #   ToolRegistry (per-use-case namespaces)
  retrieval.py        #   BM25 index + semantic_retrieval factory
  policy.py           #   deterministic policy gate (rules are data)
  agent.py            #   the 7-station loop
  __init__.py         #   load_agent(name)
usecases/
  tienda/             # example use-case (config + tools + data + prompts + evals)
    config.yaml       #   endpoints, allowed_intents, policy rules, prompt templates
    tools.py          #   build_registry(config) -> ToolRegistry
    prompts/ grammars/ data/ policies/ budgets.yaml evals/sets/
app/
  main.py             # FastAPI surface; loads a use-case via AGENT_USECASE

---

Architecture: the loop

Customer ─▶ FastAPI ─▶ Agent.handle()
                          │
   1. route    (Tier 0, GBNF)        → intent / tier / risk / confidence
   2. plan     (Tier N)              → list of tool calls
   3. tools    (APP executes)        → observations
   4. reflect  (conditional)         → only on tool-failure or risk ≥ medium
   5. generate (Tier N)              → draft answer
   6. critic   (Tier N/N+1)          → verify against observations (risk ≥ medium)
   7. policy   (deterministic)       → MANDATORY gate; no response bypasses it
   8. finalize                       → answer + metrics

Adaptive depth: simple smalltalk goes plan → tools → policy → final without paying for reflection/critique.

Objective escalation (in code, never in the prompt): confidence < 0.70 bumps a tier; a critic rejection bumps once; Tier-3 requires explicit budget permission.

---

Quickstart

Prerequisites

• Python 3.11+
• For the default profile: a llama.cpp llama-server build and a GGUF router model (Tier 0). Roughly 2 GB of RAM — one model, not four.

Install

python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"     # or: pip install -r requirements-dev.txt

Pick a tier topology (ADR-011)

Resident memory — not token cost — is the binding constraint on a workstation, so Tier 0 (small, hot, grammar-constrained) stays local while the large, rarely-called tiers go remote. One committed config serves all three profiles; the environment picks which is active.

Profile	How to select it	Resident RAM
local-only	export nothing — higher tiers are dropped and every escalation resolves down to Tier 0	~2 GB
hybrid	export AGENT_TIER{1,2,3}_URL / _MODEL / _API_KEY	~2 GB
all-remote	additionally set AGENT_TIER0_URL + AGENT_TIER0_KIND=remote	~0 GB

Credentials are named, never inlined: the config references a variable name, so it can be committed. See .env.example for every variable.

all-remote loses GBNF grammar constraints on routing (a llama.cpp capability) and falls back to response_format: json_object. It is meant for CI, not for the quality-gated path — the routing accuracy gate is only meaningful under local-only or hybrid.

Run the Tier-0 router

llama-server -m /path/to/router-model.gguf --port 8091 -ngl 99 -c 8192 --host 127.0.0.1

Use it

# Tests (no model required)
pytest

# Routing eval (gate: >= 18/20 intent accuracy)
python evals/run.py 01_intent.jsonl --usecase tienda

# Dev API
AGENT_USECASE=tienda python -m app.main
curl -X POST http://localhost:8000/dev/message \
  -H "Content-Type: application/json" \
  -d '{"text": "tienen coca de 600 fria?"}'

Docker (app + llama.cpp Tier-0)

cp .env.example .env       # set MODELS_DIR to your host model directory
docker compose up --build

Models are mounted as a read-only volume — never baked into the image.

---

Add your own use-case

usecases/<name>/
├── __init__.py        # from .tools import build_registry
├── config.yaml        # endpoints, allowed_intents, policy rules, prompts
├── prompts/router.md
├── grammars/route.gbnf
├── tools.py           # build_registry(config) -> ToolRegistry
├── data/              # fixtures (Phase 1) or API clients (Phase 2)
├── policies/*.md      # BM25-indexed docs
├── budgets.yaml
└── evals/sets/*.jsonl

Then: AGENT_USECASE=<name> python -m app.main. See the full authoring guide docs/usecases.md (contract, consumption modes, bring-your-own-models) and CONTRIBUTING.md.

---

Acceptance gates

Phase	Gate	Status
F0	Tier-0 router speed ≥ 25 tok/s	✅ (see bench/RESULTS.md)
F1	Routing intent accuracy ≥ 18/20	✅ 20/20
F1	All tools read-only (order_create dry-run)	✅
F1	Deterministic policy gate enforced	✅
F2.0	ExecutiveController + per-tier circuit breaker	✅
F2.0	Tier-client retry/backoff (transient blips ≠ tier failure)	✅
F1.6	Latency-budget enforced (safe degrade past deadline)	✅
F2.0	Resident-memory invariant enforced at config load (ADR-011)	✅
F2.0	Suite runs with no model and no credentials (local-only)	✅ 156 passed

---

Non-negotiable principles

1. No fine-tuning at this stage — routing + prompts + retrieval.
2. The model never mutates critical state without the policy gate — enforced structurally by the fail-closed tool capability contract (ADR-006).
3. Every lane needs an eval harness before increasing autonomy.
4. The simplest loop that works.
5. Inventory/price/stock are never held in model memory — always live tools.
6. Local-first; cloud only as explicit, budgeted overflow. "Budgeted" is literal — remote tiers are opt-in per tier, capped by budgets.yaml, and the number of models allowed to occupy RAM is an enforced config invariant (ADR-011).

---

Roadmap

• Phase 1 — Skeleton ✅ (this): core + use-case, routing gate, policy gate, Docker.
• Phase 2 — executive controller, versioned YAML policies, verifier pass, 10 eval sets, SQLite queue + sagas for multi-day flows.
• Phase 3 — telemetry (PII-redacted), shadow mode, retrieval growth loop.
• Phase 4 — QLoRA (strategic gate; requires ≥4 weeks of logs + a new ADR).

---

Documentation

• ADR-001 — reusable platform, not a copy template
• ADR-002 — calibrated infrastructure
• ADR-003 — policy rules as versioned data
• ADR-004 — cross-tier verification
• ADR-005 — decision telemetry as a contract
• ADR-006 — fail-closed tool capability contract
• ADR-007 — structured tool-calling contract
• ADR-008 — retrieval/tier surface is caller-isolated, not server-isolated
• ADR-009 — reflection output is a notes channel, never an observation
• ADR-010 — MCP / A2A interoperability: Rejected (with revisit triggers)
• ADR-011 — hybrid tier topology: resident memory is the binding constraint
• ADR-012 — "local" is two budgets, not one: device-aware memory invariant
• docs/SECURITY_MODEL.md — OWASP Top 10 for LLM Applications control-by-control map
• docs/workstation-memory-budget.md — running the platform on a memory-constrained machine (ADR-011 in practice)
• CHANGELOG.md — version history
• CONTRIBUTING.md — dev setup, adding use-cases, quality gates
• SECURITY.md — vulnerability reporting process (see docs/SECURITY_MODEL.md for the OWASP threat-model mapping)
• bench/RESULTS.md — benchmark + routing gate evidence

---

License

Apache-2.0.