A production-grade, multi-step AI agent that ingests a support ticket, enriches it with NLP, retrieves relevant knowledge-base articles, drafts a grounded reply with an LLM, routes it through a human-review gate, posts it back to the origin system, and emits a structured audit event — orchestrated across Azure, AWS, and GCP, with every cloud resource provisioned by Terraform.
Azure → NLP enrichment (AI Language + Container App) + KB vector search (AI Search)
AWS → LLM inference (Bedrock) + persistence (S3 + DynamoDB)
GCP → orchestration (Vertex AI Pipelines) + review + delivery + Cloud Trace
The entire agent runs locally on deterministic, dependency-free backends, so you can see it work before touching a cloud account.
# from the repo root — no install, no network
python local_sim/run_local.py --allYou'll see, per ticket: extracted entities, intent, sentiment, P1–P4 priority, the top-3 KB articles, the drafted reply (with citations and a ≤300-word cap), the review decision, delivery, and end-to-end latency.
Try a single ticket, or turn on the human-review gate:
python local_sim/run_local.py --subject "Locked out" --body "Lost my 2FA phone, urgent demo!"
python local_sim/run_local.py --all --reviewRun the tests, the quality evaluation, and the latency benchmark:
pip install pytest flask # the library itself has zero runtime deps
pytest -q # 147 unit + integration tests
python evals/run_eval.py # golden-set quality gate (retrieval + grounding)
python loadtest/bench_local.py --iterations 2000 # p50/p95/p99 → loadtest/report.md| Req | Where |
|---|---|
| FR-1 ingest ticket JSON | event bridge → Pub/Sub → trigger → pipeline; Ticket contract |
| FR-2 entities/intent/sentiment/priority | agent_core/nlp.py (Azure or local) |
| FR-3 top-3 KB by similarity | agent_core/retrieval.py (Azure AI Search or local TF-IDF) |
| FR-4 grounded ≤300-word cited reply | agent_core/llm.py + agent_core/guardrails.py |
| FR-5 human-review gate | services/gcp_review_gate + orchestrator.stage_review |
| FR-6 post reply to origin | agent_core/delivery.py (Zendesk/Jira/dry-run) |
| FR-7 structured audit event | agent_core/audit.py → Pub/Sub → BigQuery / Cloud Logging |
| NFR-1 p95 < 8s | loadtest/; alert on a latency distribution metric |
| NFR-2 apply < 20 min | docs/RUNBOOK.md destroy/re-apply drill |
| NFR-3 no static creds | env/secret-injected credentials; WIF/OIDC/SP in CI; Google→AWS web identity |
| NFR-4 per-cloud deploy/destroy | -target=module.<cloud> |
| NFR-5 $50/day budgets | budget + notification channels in each module |
services/common/agent_core/ the shared, dependency-free agent brain
contracts.py Ticket, AgentState, AuditEvent, enums (JSON round-trip)
nlp.py retrieval.py llm.py pluggable backends: local default + cloud
guardrails.py post-draft citation/safety checks (FR-4 safety net)
delivery.py audit.py pluggable delivery + audit sinks
privacy.py PII masking for data at rest
awsauth.py signed GCP→AWS calls (web identity → SigV4)
telemetry.py W3C traceparent propagation + OTel fallback
orchestrator.py the DAG: ingest→nlp→retrieval→draft→review→delivery→audit
services/azure_nlp/ Flask NLP service (Azure Container Apps)
services/aws_inference/ Lambda: draft (Bedrock) + persist (S3/DynamoDB)
services/gcp_event_bridge/ Cloud Run: authenticated Azure Event Grid → Pub/Sub
services/gcp_pipeline_trigger/Cloud Run: Pub/Sub push → Vertex PipelineJob
services/gcp_review_gate/ Cloud Run: human-review UI + resume-on-approve
pipelines/dsl/pipeline.py Vertex AI Pipeline (KFP v2) — one container per step
infra/ Terraform: root + bootstrap + aws/ azure/ gcp/ modules
local_sim/ end-to-end offline simulator + sample tickets
evals/ golden-set quality gate (runs in CI)
loadtest/ Locust file + offline percentile benchmark
observability/ OpenTelemetry Collector config → Cloud Trace
docs/ RUNBOOK, COST, ADRs, architecture diagram
data/kb/ 8-article sample knowledge base
tests/ unit + integration (run with pytest or unittest)
.github/workflows/ ci, index-kb, terraform-plan, terraform-apply, terraform-destroy
Everything is controlled by environment variables, with safe local defaults:
| Variable | Local default | Cloud value |
|---|---|---|
NLP_BACKEND |
local |
azure |
RETRIEVAL_BACKEND |
local |
azure |
LLM_BACKEND |
local |
bedrock |
DELIVERY_BACKEND |
dryrun |
zendesk / jira |
AUDIT_SINK |
stdout |
pubsub |
REVIEW_GATE_ENABLED |
unset → gate external tickets | true |
So the same code path you just ran locally is what runs in the cloud — only the backends change. See docs/RUNBOOK.md for full cloud provisioning, and the ADRs for why each choice was made.
The agent writes text that gets sent to customers, from tickets that anyone can write. Three things stand between those two facts:
- Ticket text is data, never instructions. It is fenced and labelled untrusted in the prompt, and it cannot close its own fence.
- Every draft is checked after generation (guardrails.py): citations must be articles that were actually retrieved, a reply may not solicit a secret, and it may not link anywhere the knowledge base doesn't.
- Nothing flagged auto-delivers. With
REVIEW_GATE_ENABLEDunset, externally-supplied tickets go to a human by default; a flagged draft always does, even when the gate is off. See ADR-0006.
See docs/RUNBOOK.md. In short: bootstrap the state bucket
→ fill terraform.tfvars → package_lambda.sh → terraform apply →
build_images.sh → re-apply with the pushed tag → index_kb.py → compile and
publish the pipeline spec.
Images are tagged with the git SHA, never latest: a deploy is a new tag and a
rollback is the previous one.
See CONTRIBUTING.md for the conventions this codebase
follows (the backend pattern, the zero-dependency rule for agent_core, and
what CI enforces).
This build upgrades the original spec — all documented in the ADRs and
summarized at the top of docs/adr/README.md. The headline changes: a single
dependency-free core library instead of duplicated per-cloud logic
(ADR-0005); a single GCS state backend instead of the impossible
three-backend design (ADR-0004); fully offline-runnable local backends so
the system is testable with zero cloud accounts; container-per-step Vertex
components for real per-step retries/DLQ (ADR-0003); review-by-default with
post-draft guardrails (ADR-0006); and keyless GCP→AWS calls via Google
web identity (ADR-0007).