Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ pixi run -e mojo mojo run opengateway/mojo/main.mojo

OpenGateway ships **two HTTP servers** that implement the same OpenAI-compatible contract:

1. **FastAPI server** (`opengateway/main.py`) — default, Python, asyncio.
2. **Mojo + flare server** (`opengateway/mojo/main.mojo`) — opt-in, Mojo, static binary.
1. **Mojo + flare server** (`opengateway/mojo/main.mojo`), default, Mojo, static binary.
2. **FastAPI server** (`opengateway/main.py`), opt-in dev path, Python, asyncio.

Both delegate provider calls to the same Python adapters
(`opengateway/providers/`). The Mojo server uses the PythonObject
Expand All @@ -64,6 +64,41 @@ See [docs/architecture.md](docs/architecture.md) and
[ADR-002](adr/002-mojo-api-surface.md) for the full layering
rationale.

### Repository layout

```
opengateway/
├── opengateway/
│ ├── main.py # FastAPI server (opt-in dev path)
│ ├── auth.py # Virtual key + root key auth
│ ├── config.py # Settings via pydantic-settings (RouteRule, ROUTES_JSON)
│ ├── keys.py # API key generator (sk-og-{token})
│ ├── router.py # Model-to-provider routing (Python)
│ ├── providers/ # Provider adapters (called by the bridge)
│ │ ├── base.py
│ │ ├── openai.py
│ │ └── anthropic.py # stub, falls back from OpenAI when configured
│ ├── mojo/ # Mojo server on flare, default deployment
│ │ ├── main.mojo # flare HTTP server + middleware stack
│ │ ├── router.mojo # Model to provider module routing
│ │ ├── test_router.mojo
│ │ └── __init__.mojo
│ └── mojo_bridge/ # Python side of the Mojo bridge
│ ├── auth.py
│ ├── chat.py
│ ├── stream.py
│ ├── db.py # Postgres virtual key store
│ ├── ratelimit.py # Redis sliding-window rate limiter
│ └── routing.py # ROUTES_JSON resolution + fallback orchestration
├── tests/ # pytest suite + mock_upstream.py
├── docs/ # architecture.md, configuration.md, release-process.md
├── adr/ # Architecture Decision Records
├── pyproject.toml # Python package config
├── pixi.toml # Mojo environment config
├── Dockerfile # Container build
└── docker-compose.yml # Local dev stack (Postgres + Redis + gateway)
```

## Pull Request Process

1. Ensure your code passes all tests and linting (`ruff`, `mypy`,
Expand Down
310 changes: 23 additions & 287 deletions README.md

Large diffs are not rendered by default.

117 changes: 26 additions & 91 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,38 +10,14 @@ them.

## High-level request flow

```
┌─────────────────────┐
OpenAI-compatible client │ POST /v1/chat/... │
(curl, openai-python, │ Bearer <virtual │
SDK, internal app) ────►│ key> │
└─────────┬───────────┘
│
┌─────────────────────┴─────────────────────┐
│ │
┌───────▼────────┐ ┌───────▼────────┐
│ FastAPI app │ │ flare server │
│ (opengateway │ │ (opengateway │
│ .main) │ │ .mojo.main) │
│ │ │ │
│ Python, │ │ Mojo, static │
│ uvicorn, │ │ binary or │
│ asyncio │ │ pixi run │
└───────┬────────┘ └───────┬────────┘
│ │
│ ┌────────────────────────────────┐ │
└─────►│ opengateway.providers.* │◄──┘
│ (Python adapters, async) │
│ - openai.py (existing) │
│ - anthropic.py (planned) │
│ - bedrock.py (planned) │
└────────────────┬───────────────┘
│
┌─────────▼──────────┐
│ Upstream LLM API │
│ (OpenAI / Anthrop │
│ ic / Bedrock...) │
└────────────────────┘
```mermaid
flowchart TD
client["OpenAI-compatible client<br/>curl, openai-python, SDK, internal app"]
client -->|"POST /v1/chat/completions<br/>Bearer sk-og-token"| fastapi["FastAPI app<br/>opengateway.main<br/>Python, uvicorn, asyncio"]
client -->|"POST /v1/chat/completions<br/>Bearer sk-og-token"| flare["flare server<br/>opengateway.mojo.main<br/>Mojo, static binary or pixi run"]
fastapi --> providers["opengateway.providers.*<br/>Python adapters, async<br/>openai.py shipped<br/>anthropic.py, bedrock.py planned"]
flare --> providers
providers --> upstream["Upstream LLM API<br/>OpenAI / Anthropic / Bedrock"]
```

Both servers implement the same endpoint contract, share the same
Expand Down Expand Up @@ -109,25 +85,18 @@ Python (`bridge_module.handle_chat(payload, auth_header, provider_module)`).
It returns an envelope dict `{"status": <int>, "body": <json str>}`
so the Mojo handler never has to catch Python exceptions itself.

```
Mojo handler Python bridge
───────────── ──────────────
chat_completions(req) handle_chat(body, auth, module)
│ │
│ req.text() │ authenticate_authorization()
│ req.headers.get("authorization") │ _validate_request(body)
│ │ _enforce_model_access(...)
│ │ _enforce_budget(...)
│ │ asyncio.run(_run_completion(...))
│ │ └─► provider.chat(...)
│ │
│ json_loads(body_text) │
│ bridge.handle_chat(payload, ...) │
│ │
│ ◄──────────── envelope dict ────────│
│
▼
ok_json(body_json)
```mermaid
sequenceDiagram
participant M as Mojo handler
participant P as Python bridge
M->>M: req.text() and headers.get("authorization")
M->>P: handle_chat(payload, auth_header, provider_module)
Note over P: authenticate_authorization()
Note over P: _validate_request(body)
Note over P: _enforce_model_access(...) and _enforce_budget(...)
Note over P: asyncio.run(_run_completion(...)) calls provider.chat(...)
P-->>M: envelope dict with status and body
M->>M: json_loads(body_text), ok_json(body_json)
```

The Mojo layer never calls `httpx`, `asyncpg`, or any async Python
Expand Down Expand Up @@ -185,46 +154,12 @@ two servers.

## Deployment shapes

```
┌────────────────────────────┐
│ Local dev (Python default) │
│ │
│ uv run opengateway │
│ (uses opengateway.main │
│ FastAPI app on :8080) │
└────────────────────────────┘

┌────────────────────────────┐
│ Edge / serverless │
│ │
│ pixi run -e mojo mojo │
│ build main.mojo -O3 │
│ ./main │
│ (static binary, no venv, │
│ 15–80 MB) │
└────────────────────────────┘

┌────────────────────────────┐
│ Docker (default) │
│ │
│ docker compose up │
│ (FastAPI + Postgres + │
│ Redis via Compose) │
└────────────────────────────┘

┌────────────────────────────┐
│ Docker (Mojo variant) │
│ │
│ pixi run -e mojo mojo │
│ build main.mojo │
│ docker build -f │
│ Dockerfile.mojo . │
│ (multi-stage: │
│ modular image → static │
│ binary → distroless │
│ final image, ~30 MB) │
└────────────────────────────┘
```
| Shape | Command | Notes |
|---|---|---|
| **Local dev (Python default)** | `uv run opengateway` | FastAPI app on :8080 |
| **Edge / serverless** | `pixi run -e mojo mojo build main.mojo -O3` then `./main` | Static binary, no venv, 15-80 MB |
| **Docker (default)** | `docker compose up` | FastAPI + Postgres + Redis via Compose |
| **Docker (Mojo variant)** | `pixi run -e mojo mojo build main.mojo` then `docker build -f Dockerfile.mojo .` | Multi-stage: modular image to static binary to distroless, ~30 MB |

See `docs/release-process.md` for the full release flow and
`docs/mojo-python-ai-gateway.md` for the design rationale.
Binary file added docs/assets/banner.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/banner.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/assets/logo-explore/banner.anim.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading