diff --git a/Makefile b/Makefile index 5ec595c..ec198fb 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: help setup lint format type test test-all coverage eval demo serve worker bench-load bench-recovery docker-build clean all changelog cut-release +.PHONY: help setup lint format type test test-all coverage eval demo serve worker bench-load bench-recovery docker-build clean all changelog cut-release diagram help: ## Show this help @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \ @@ -59,3 +59,8 @@ changelog: ## Regenerate CHANGELOG.md from Conventional Commits (git-cliff) cut-release: ## Cut a release locally: version bump, changelog, tag, GitHub release (RELEASE_VERSION=vX.Y.Z optional) scripts/release.sh $(RELEASE_VERSION) + +diagram: ## Render every diagram in docs/architecture to SVG (house theme, white background) + @for f in docs/architecture/*.mmd; do \ + npx -y @mermaid-js/mermaid-cli -q -i "$$f" -o "$${f%.mmd}.svg" -c mermaid.config.json -b white || exit 1; \ + done diff --git a/docs/architecture/architecture.svg b/docs/architecture/architecture.svg index c5dc9f5..3c771fc 100644 --- a/docs/architecture/architecture.svg +++ b/docs/architecture/architecture.svg @@ -1 +1 @@ -ObservabilityState (SQLite, WAL)Agent runtimeDurable jobsInterfacesasksubmit jobresearch stepsclaim, checkpointsearch, spansspans, countersApps, TS client, chatMCP serverstdio, 5 toolsHTTP APIask, SSE stream, jobsKeyStorehashed API keysRate limiterper tenantJobWorkerlease, heartbeatOrchestratorplanner, worker, criticAgentRuntime.askReAct loopTenant toolssearch, calculatorllm-gatewayscreen, redact, cacheFailover chainretry, backoffProvidersoffline, OpenAI, AnthropicConstrained JSONschema automatonDocuments + indexBM25, TF-IDF, HNSWJobs and leasesstep ledgerSpans and keysper tenantMemory per tenantagent-memory, sessionsTracerrequest to tool spansPrometheus/metricsEval suitevia public API \ No newline at end of file +ObservabilityState (SQLite, WAL)Agent runtimeDurable jobsInterfacesasksubmit jobresearch stepsclaim, checkpointsearch, spansspans, countersApps, TS client, chatMCP serverstdio, 5 toolsHTTP APIask, SSE stream, jobsKeyStorehashed API keysRate limiterper tenantJobWorkerlease, heartbeatOrchestratorplanner, worker, criticAgentRuntime.askReAct loopTenant toolssearch, calculatorllm-gatewayscreen, redact, cacheFailover chainretry, backoffProvidersoffline, OpenAI, AnthropicConstrained JSONschema automatonDocuments + indexBM25, TF-IDF, HNSWJobs and leasesstep ledgerSpans and keysper tenantMemory per tenantagent-memory, sessionsTracerrequest to tool spansPrometheus/metricsEval suitevia public API \ No newline at end of file diff --git a/docs/architecture/job-recovery.svg b/docs/architecture/job-recovery.svg index 732dd74..cbca0b2 100644 --- a/docs/architecture/job-recovery.svg +++ b/docs/architecture/job-recovery.svg @@ -1 +1 @@ -Worker BWorker ASQLiteAPIloop[each step]SIGKILL.Heartbeats stop.INSERT job (queued)claim: token=1,expires=now+Lsave plan (blackboardsnapshot)record attempt (q1)Orchestrator.attempt_su-btasksave spansledger row q1 + board (onlyif owner and token=1)claim fails until expires < nowclaim: token=2Blackboard.restore(boa-rd)record attempt (next incomplete step)ledger rows, then status=succeeded \ No newline at end of file +Worker BWorker ASQLiteAPIloop[each step]SIGKILL.Heartbeats stop.INSERT job (queued)claim: token=1,expires=now+Lsave plan (blackboardsnapshot)record attempt (q1)Orchestrator.attempt_su-btasksave spansledger row q1 + board (onlyif owner and token=1)claim fails until expires < nowclaim: token=2Blackboard.restore(boa-rd)record attempt (next incomplete step)ledger rows, then status=succeeded \ No newline at end of file diff --git a/docs/architecture/job-states.svg b/docs/architecture/job-states.svg index 3dcde91..00223c1 100644 --- a/docs/architecture/job-states.svg +++ b/docs/architecture/job-states.svg @@ -1 +1 @@ -POST /v1/jobsclaim, lease_token + 1heartbeat, fenced step commit,reclaim after lease expiryreport checkpointedrejected or out of attemptsqueuedrunningsucceededfailed \ No newline at end of file +POST /v1/jobsclaim, lease_token + 1heartbeat, fenced step commit,reclaim after lease expiryreport checkpointedrejected or out of attemptsqueuedrunningsucceededfailed \ No newline at end of file diff --git a/docs/architecture/request-path.svg b/docs/architecture/request-path.svg index 549f475..f44cd34 100644 --- a/docs/architecture/request-path.svg +++ b/docs/architecture/request-path.svg @@ -1 +1 @@ -StateModel, tools and retrievalAgent runtimeHTTP edge (api.py)Bearer keytenant from key rowallowed requestmodel calls, tools, searchspansClientapi.pyThreadingHTTPServerauth.py: HMAC key lookup401 if missing or revokedper-tenant token bucket429 when emptyTracer spanhttp.requestAgentRuntime.askReActAgent loopanswer, citations,grounded, trace_idllm_gateway.Gateway422 when blockedGroundedPolicyProviderCalculatorToolconstrained_generationoptional schemaKnowledgeBasetenant index onlyHybridRetrieverBM25 + TF-IDF + HNSWann_vector_searchHNSWspans table \ No newline at end of file +StateModel, tools and retrievalAgent runtimeHTTP edge (api.py)Bearer keytenant from key rowallowed requestmodel calls, tools, searchspansClientapi.pyThreadingHTTPServerauth.py: HMAC key lookup401 if missing or revokedper-tenant token bucket429 when emptyTracer spanhttp.requestAgentRuntime.askReActAgent loopanswer, citations,grounded, trace_idllm_gateway.Gateway422 when blockedGroundedPolicyProviderCalculatorToolconstrained_generationoptional schemaKnowledgeBasetenant index onlyHybridRetrieverBM25 + TF-IDF + HNSWann_vector_searchHNSWspans table \ No newline at end of file diff --git a/mermaid.config.json b/mermaid.config.json new file mode 100644 index 0000000..b852e61 --- /dev/null +++ b/mermaid.config.json @@ -0,0 +1,64 @@ +{ + "theme": "base", + "themeVariables": { + "fontFamily": "Inter, Helvetica, Arial, sans-serif", + "fontSize": "15px", + "primaryColor": "#dbeafe", + "primaryBorderColor": "#1d4ed8", + "primaryTextColor": "#0b1b3a", + "lineColor": "#475569", + "secondaryColor": "#dcfce7", + "tertiaryColor": "#f8fafc", + "actorBkg": "#6366F1", + "actorBorder": "#1E293B", + "actorTextColor": "#ffffff", + "actorLineColor": "#94A3B8", + "signalColor": "#334155", + "signalTextColor": "#0b1b3a", + "sequenceNumberColor": "#ffffff", + "labelBoxBkgColor": "#F59E0B", + "labelBoxBorderColor": "#1E293B", + "labelTextColor": "#1E293B", + "loopTextColor": "#1E293B", + "noteBkgColor": "#CBD5E1", + "noteBorderColor": "#1E293B", + "noteTextColor": "#1E293B", + "activationBkgColor": "#0EA5E9", + "activationBorderColor": "#1E293B", + "pie1": "#6366F1", + "pie2": "#10B981", + "pie3": "#F59E0B", + "pie4": "#0EA5E9", + "pie5": "#CBD5E1", + "pie6": "#8B5CF6", + "pie7": "#14B8A6", + "pie8": "#F97316", + "pieStrokeColor": "#1E293B", + "pieStrokeWidth": "1px", + "pieOuterStrokeColor": "#1E293B", + "pieTitleTextColor": "#0b1b3a", + "pieSectionTextColor": "#0b1b3a", + "pieLegendTextColor": "#0b1b3a", + "pieOpacity": "1" + }, + "flowchart": { + "curve": "basis", + "padding": 12, + "nodeSpacing": 40, + "rankSpacing": 50 + }, + "themeCSS": "rect.actor, .actor-man circle, rect.note, rect.labelBox, rect.activation0, rect.activation1, rect.activation2, g.stateGroup rect, .statediagram-state rect, .statediagram-state path, .node rect, .node polygon, g.er.entityBox, .er.entityBox, path.pieCircle, .entityBox { filter: drop-shadow(1px 2px 2px rgba(185,185,185,1)); } .node:has(.label.name) .outer-path path:not([fill='none']) { fill: #10B981; } .node:has(.label.name) .outer-path path { stroke: #1E293B; } .row-rect-odd path:not([fill='none']) { fill: #ECFDF5; } .row-rect-even path:not([fill='none']) { fill: #ffffff; } .node:has(.label.name) .divider path { stroke: #1E293B; } .node:has(.label.name) .label.name p, .node:has(.label.name) .label.name span { color: #ffffff; }", + "sequence": { + "wrap": true, + "width": 150, + "actorMargin": 24, + "boxMargin": 8, + "noteMargin": 8, + "messageMargin": 32, + "mirrorActors": false, + "showSequenceNumbers": false + }, + "state": { + "padding": 12 + } +}
Observability
State (SQLite, WAL)
Agent runtime
Durable jobs
Interfaces
ask
submit job
research steps
claim, checkpoint
search, spans
spans, counters
Apps, TS client, chat
MCP serverstdio, 5 tools
HTTP APIask, SSE stream, jobs
KeyStorehashed API keys
Rate limiterper tenant
JobWorkerlease, heartbeat
Orchestratorplanner, worker, critic
AgentRuntime.askReAct loop
Tenant toolssearch, calculator
llm-gatewayscreen, redact, cache
Failover chainretry, backoff
Providersoffline, OpenAI, Anthropic
Constrained JSONschema automaton
Documents + indexBM25, TF-IDF, HNSW
Jobs and leasesstep ledger
Spans and keysper tenant
Memory per tenantagent-memory, sessions
Tracerrequest to tool spans
Prometheus/metrics
Eval suitevia public API
POST /v1/jobs
claim, lease_token + 1
heartbeat, fenced step commit,reclaim after lease expiry
report checkpointed
rejected or out of attempts
queued
running
succeeded
failed
State
Model, tools and retrieval
HTTP edge (api.py)
Bearer key
tenant from key row
allowed request
model calls, tools, search
spans
Client
api.pyThreadingHTTPServer
auth.py: HMAC key lookup401 if missing or revoked
per-tenant token bucket429 when empty
Tracer spanhttp.request
AgentRuntime.askReActAgent loop
answer, citations,grounded, trace_id
llm_gateway.Gateway422 when blocked
GroundedPolicyProvider
CalculatorTool
constrained_generationoptional schema
KnowledgeBasetenant index only
HybridRetrieverBM25 + TF-IDF + HNSW
ann_vector_searchHNSW
spans table