FastAPI backend for routing user queries across skill/domain classifiers, model selection, optional hybrid RAG, parallel SLM calls, and response fusion.
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000Open:
- API docs: http://127.0.0.1:8000/docs
- Health: http://127.0.0.1:8000/health
Invoke-RestMethod -Method Post http://127.0.0.1:8000/query `
-ContentType "application/json" `
-Body '{"query":"Summarize the finance policy for refund reserves","conversation_id":"demo"}'Invoke-RestMethod -Method Post http://127.0.0.1:8000/ingest `
-ContentType "application/json" `
-Body '{"documents":[{"text":"Refund reserves are reviewed quarterly by finance.","metadata":{"domain":"finance","skill":"summarization","source":"policy"}}]}'- config/routing.yaml: maps
skill+domainrouting keys to model IDs, RAG usage, and fusion strategy. - config/models.yaml: declares available models, endpoints, tags, latency, cost, and weights.
The default SLM client is a deterministic mock adapter for local development. Replace MockModelClient in app/services/model_client.py or add an adapter for vLLM, Ollama, or hosted APIs.
pytest