Skip to content

Latest commit

 

History

History
46 lines (32 loc) · 1.45 KB

README.md

File metadata and controls

46 lines (32 loc) · 1.45 KB

Adaptive Multi-Model Query Routing System

FastAPI backend for routing user queries across skill/domain classifiers, model selection, optional hybrid RAG, parallel SLM calls, and response fusion.

Quick Start

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000

Open:

Query

Invoke-RestMethod -Method Post http://127.0.0.1:8000/query `
  -ContentType "application/json" `
  -Body '{"query":"Summarize the finance policy for refund reserves","conversation_id":"demo"}'

Ingest Documents

Invoke-RestMethod -Method Post http://127.0.0.1:8000/ingest `
  -ContentType "application/json" `
  -Body '{"documents":[{"text":"Refund reserves are reviewed quarterly by finance.","metadata":{"domain":"finance","skill":"summarization","source":"policy"}}]}'

Configuration

  • config/routing.yaml: maps skill+domain routing keys to model IDs, RAG usage, and fusion strategy.
  • config/models.yaml: declares available models, endpoints, tags, latency, cost, and weights.

The default SLM client is a deterministic mock adapter for local development. Replace MockModelClient in app/services/model_client.py or add an adapter for vLLM, Ollama, or hosted APIs.

Tests

pytest