diff --git a/README.md b/README.md index ab92e93..7da89c6 100644 --- a/README.md +++ b/README.md @@ -1,46 +1,38 @@ # Virtuous -Virtuous is an **agent-first, typed RPC API framework for Go** with **self-generating docs and clients**. +**An agent-first framework for writing Go services that enforces sensible constraints.** [![Release](https://img.shields.io/github/v/tag/swetjen/virtuous)](https://github.com/swetjen/virtuous/tags) [![Build Status](https://github.com/swetjen/virtuous/actions/workflows/ci.yaml/badge.svg)](https://github.com/swetjen/virtuous/actions/workflows/ci.yaml) [![Go Version](https://img.shields.io/github/go-mod/go-version/swetjen/virtuous)](go.mod) [![License](https://img.shields.io/github/license/swetjen/virtuous)](https://github.com/swetjen/virtuous/blob/main/LICENSE) -**RPC-first:** APIs are plain Go functions with typed inputs and outputs, served over HTTP. Routes, schemas, docs, and JS/TS/Python clients are generated at runtime from those functions. +Virtuous gives you two libraries and a strong opinion about which to use: -**Compatibility:** `httpapi` wraps existing `net/http` handlers when you must preserve a legacy shape or migrate gradually. New work should start with RPC. +- **`rpc`** — the default. APIs are plain Go functions with typed inputs and + outputs. Routes, schemas, docs, and JS/TS/Python clients are generated at + runtime from those functions. +- **`httpapi`** — the HTTP-native library. Use it to migrate existing + `net/http` handlers and for routes that need raw HTTP control, while still + getting OpenAPI and generated clients. -## Table of contents +It's a strong migration target from **swaggo, gin, echo, chi, and vanilla +`net/http`** — keep your handlers, wrap them in `httpapi`, and move to `rpc` at +your own pace. -- [RPC](#rpc) -- [HTTP API (httpapi)](#http-api-httpapi) -- [Combined (migration demo)](#combined-migration-demo) -- [Why RPC?](#why-rpc) -- [Docs](#docs) -- [Requirements](#requirements) +## The constraints (and why each exists) -## Why RPC (default) +The constraints are the product. They are what keep a Virtuous service small, +predictable, and safe for an agent to extend without drifting. -Virtuous treats APIs as **typed functions** instead of loosely defined HTTP resources. That keeps the surface area small, predictable, and agent-friendly. +| Constraint | Why it exists | +| --- | --- | +| **Types are the contract** | Request/response structs *are* the API. There is no separate schema to sync, so OpenAPI and SDKs can't drift from the code. | +| **Routes are inferred** | RPC paths derive from package + function names. No manual path design to maintain or argue about. | +| **A narrow status model** | RPC handlers return `200` / `422` / `500` (plus `401` from guards). Error handling stays explicit and uniform. | +| **Docs and clients are runtime truth** | They're emitted from the running server, not hand-written, so they always match what's deployed. | -What this means in practice: - -- Inputs/outputs are Go structs; they *are* the contract and generate OpenAPI + SDKs automatically. -- Routes derive from package + function names, so naming stays consistent without manual path design. -- A minimal handler status model (200/422/500) keeps error handling explicit and uniform. -- Docs and clients are emitted from the running server, so they cannot drift from the code. - -`httpapi` stays in the toolbox for teams migrating existing handlers or preserving exact legacy responses. - -## RPC - -RPC uses plain Go functions with typed requests and responses. -Routes, schemas, and clients are inferred from package and function names. - -This model minimizes surface area, avoids configuration drift, and produces predictable client code. - -### Quick start (cut, paste, run) +## Quick start (cut, paste, run) ```bash mkdir virtuous-demo @@ -88,9 +80,9 @@ func GetState(_ context.Context, req GetStateRequest) (StateResponse, int) { } func main() { -router := rpc.NewRouter(rpc.WithPrefix("/rpc")) -router.HandleRPC(GetState) -router.ServeAllDocs() + router := rpc.NewRouter(rpc.WithPrefix("/rpc")) + router.HandleRPC(GetState) + router.ServeAllDocs() server := &http.Server{Addr: ":8000", Handler: router} fmt.Println("Listening on :8000") @@ -98,264 +90,61 @@ router.ServeAllDocs() } ``` -![Virtuous Basic API Docs](docs/example.png) - Run it: ```bash go run . ``` -### Advanced patterns - -#### 1) One guard for a collection of routes (group-level) - -Use a dedicated router for the guarded group, then mount both routers on one mux. - -```go -admin := rpc.NewRouter( - rpc.WithPrefix("/rpc/admin"), - rpc.WithGuards(sessionGuard{}), // applies to every admin route -) -admin.HandleRPC(adminusers.GetMany) -admin.HandleRPC(adminusers.Disable) - -public := rpc.NewRouter(rpc.WithPrefix("/rpc/public")) -public.HandleRPC(publichealth.Check) - -mux := http.NewServeMux() -mux.Handle("/rpc/admin/", admin) -mux.Handle("/rpc/public/", public) -``` - -If you wrap a mux subtree with middleware directly, that works for transport security, but `rpc.WithGuards(...)` is the docs/client-aware path because it emits OpenAPI security metadata. - -#### 2) Multiple documentation sets (Public Service, Secret Service) - -Today, one router emits one OpenAPI document for all routes on that router. -For separate docs, split routes across routers. - -```go -publicAPI := rpc.NewRouter(rpc.WithPrefix("/rpc/public")) -publicAPI.HandleRPC(publicsvc.GetCatalog) -publicAPI.ServeAllDocs( - rpc.WithDocsOptions( - rpc.WithDocsPath("/rpc/public/docs"), - rpc.WithOpenAPIPath("/rpc/public/openapi.json"), - ), - rpc.WithClientJSPath("/rpc/public/client.gen.js"), - rpc.WithClientTSPath("/rpc/public/client.gen.ts"), - rpc.WithClientPYPath("/rpc/public/client.gen.py"), -) - -secretAPI := rpc.NewRouter( - rpc.WithPrefix("/rpc/secret"), - rpc.WithGuards(internalTokenGuard{}), -) -secretAPI.HandleRPC(secretsvc.RotateKeys) -secretAPI.ServeAllDocs( - rpc.WithDocsOptions( - rpc.WithDocsPath("/rpc/secret/docs"), - rpc.WithOpenAPIPath("/rpc/secret/openapi.json"), - ), - rpc.WithClientJSPath("/rpc/secret/client.gen.js"), - rpc.WithClientTSPath("/rpc/secret/client.gen.ts"), - rpc.WithClientPYPath("/rpc/secret/client.gen.py"), -) - -mux := http.NewServeMux() -mux.Handle("/rpc/public/", publicAPI) -mux.Handle("/rpc/secret/", secretAPI) -``` - -#### 3) Basic auth on the docs route - -Use a mountable docs handler so you can protect docs independently from API routes. - -```go -func docsBasicAuth(user, pass string, next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - u, p, ok := r.BasicAuth() - if !ok || u != user || p != pass { - // Sends a Basic Auth challenge so browsers show a username/password prompt. - w.Header().Set("WWW-Authenticate", `Basic realm="Virtuous Docs"`) - http.Error(w, "unauthorized", http.StatusUnauthorized) - return - } - next.ServeHTTP(w, r) - }) -} - -router := rpc.NewRouter(rpc.WithPrefix("/rpc")) -router.HandleRPC(states.GetMany) - -// Keep generated clients on default routes. -router.ServeAllDocs(rpc.WithoutDocs()) - -// Mount docs separately, with only selected modules enabled. -docs := router.DocsHandler( - rpc.WithModules(rpc.ModuleAPI, rpc.ModuleObservability), -) -admin := router.AdminHandler( - rpc.WithModules(rpc.ModuleObservability), - rpc.WithPublicAdmin(), // protected by docsBasicAuth below -) - -mux := http.NewServeMux() -mux.Handle("/rpc/", router) // API routes -mux.Handle( - "/admin/docs/", - http.StripPrefix("/admin/docs", docsBasicAuth("docs", "secret", docs)), -) -mux.Handle( - "GET /admin/docs/_admin/", - http.StripPrefix("/admin/docs/_admin", docsBasicAuth("docs", "secret", admin)), -) -``` - -This mounts docs at `/admin/docs/`, with OpenAPI at `/admin/docs/openapi.json`. - -#### 4) OR auth semantics (accept either of two schemes) - -For RPC routes that accept either credential type, express that logic in one composite guard and attach it once. For legacy `httpapi` routes, use `httpapi.AuthAny(...)`. - -```go -type bearerOrAPIKeyGuard struct { - bearer bearerGuard - apiKey apiKeyGuard -} +Then open **`http://localhost:8000/rpc/docs`** for the Scalar API reference. The +OpenAPI spec is at `/rpc/openapi.json` and generated clients at +`/rpc/client.gen.{js,ts,py}`. -func (g bearerOrAPIKeyGuard) Spec() guard.Spec { - return guard.Spec{ - Name: "BearerOrApiKey", - In: "header", - Param: "Authorization", - } -} - -func (g bearerOrAPIKeyGuard) Middleware() func(http.Handler) http.Handler { - return func(next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if g.bearer.authenticate(r) || g.apiKey.authenticate(r) { - next.ServeHTTP(w, r) - return - } - http.Error(w, "unauthorized", http.StatusUnauthorized) - }) - } -} -``` +![Virtuous Basic API Docs](docs/example.png) -#### 5) Minimal docs modules +## Why RPC is the default -By default docs show the API and Observability modules. Restrict the visible modules with `WithModules(...)`. +Virtuous treats APIs as **typed functions**, not as collections of loosely +related HTTP resources. That keeps the surface area small and the intent +explicit — which matters most when APIs are consumed by agents. -```go -router := rpc.NewRouter(rpc.WithPrefix("/rpc")) -router.HandleRPC(states.GetMany) -router.ServeDocs( - rpc.WithModules( - rpc.ModuleAPI, - rpc.ModuleObservability, - ), -) -``` +- **Clarity over convention** — function names express intent directly, without + guessing paths or schemas. +- **Types as the contract** — request and response structs *are* the API; no + separate schema to sync. +- **Predictable code generation** — small, explicit signatures produce reliable + client SDKs. +- **Fewer invalid states** — avoids ambiguous partial updates, nested resources, + and overloaded semantics. +- **Runtime truth** — routes, schemas, docs, and clients all derive from the same + runtime definitions. -### Handler signature +RPC still runs on HTTP and uses HTTP status codes intentionally. What changes is +the *mental model*: from "resources and verbs" to "operations with inputs and +outputs." RPC is the default because it's **harder to misuse and easier to +automate**. -RPC handlers must follow one of these forms: +### RPC handler signature and status model ```go func(context.Context, Req) (Resp, int) func(context.Context) (Resp, int) ``` -### Status model - -RPC handlers return an HTTP status code directly. - -Supported statuses: - -- `200` — success -- `422` — invalid input -- `500` — server error - -Guarded routes may also return `401` when middleware rejects the request. - -Docs and SDKs are served at runtime. - -Default `ServeAllDocs()` paths: - -- `/rpc/docs` -- `/rpc/openapi.json` -- `/rpc/client.gen.*` -- Observability redirect: `/rpc/_virtuous/observability` -- Metrics JSON: `/rpc/_virtuous/metrics` -- Responses should include a canonical `error` field (string or struct) when errors occur. - -If you need custom placement or docs-only middleware, use `router.DocsHandler(...)` and mount it on your mux. Mount `router.AdminHandler(...)` separately when exposing observability admin endpoints. Admin endpoints require `WithAdminGuards(...)` or explicit `WithPublicAdmin()` when protected by external middleware. - -### Observability - -Basic per-RPC request metrics are tracked in memory by default. Advanced error grouping, guard metrics, and sampled traces are opt-in. - -```go -router := rpc.NewRouter( - rpc.WithPrefix("/rpc"), - rpc.WithAdvancedObservability( - rpc.WithObservabilitySampling(0.25), - ), -) - -router.HandleRPC(states.GetMany, auth.BearerGuard{}) -router.ServeAllDocs() -``` - -This enables: - -- `/rpc/_virtuous/metrics` for JSON metrics -- `/rpc/_virtuous/observability` as a redirect to the docs page - -Live route/event logging is opt-in at mux boundary: - -```go -handler := router.AttachLogger(mux) // attach once at top-level -``` - -If logger attachment is missing, the docs `Observability` view shows a zero-data setup snippet. - -For local request tracing, enable the debug console on the router. It prints one compact request line with an `ok`/`warn`/`err` status badge, method, path, duration, client IP, route pattern, and response bytes. The default stderr logger colors the badge, status, and method when the destination is a terminal; captured writers stay plain text. - -```go -router := rpc.NewRouter( - rpc.WithPrefix("/rpc"), - rpc.WithDebugConsole(), -) -``` - -```text -[virtuous] warn 422 POST /rpc/users/user-login 1.6ms ip=203.0.113.8 route=/rpc/users/user-login bytes=44 -``` - -## HTTP API (httpapi) - -`httpapi` wraps classic `net/http` handlers and preserves existing request/response shapes. It also emits OpenAPI 3.0 specs for typed handlers. - -Use this when: -- Migrating an existing API to Virtuous -- Developing rich HTTP APIs -- Maintaining compatibility with established OpenAPI contracts +Handlers return an HTTP status directly: `200` (success), `422` (invalid input), +or `500` (server error). Guarded routes may also return `401`. Responses should +include a canonical `error` field when something goes wrong. -### Quick start +More: **[RPC patterns cookbook](docs/rpc/patterns.md)** covers group guards, +multiple docs sets, protected docs, OR auth, and observability. -Method-prefixed patterns (`GET /path`) are required for docs and client generation. -Typed `httpapi` routes default to JSON, while explicit metadata covers compatibility needs such as typed path/query params, form request bodies, custom response media types, multiple statuses, and OR auth. +## When to use httpapi -For `httpapi`, keep the verb in the route string and use one of the blessed typed-handler patterns: - -- `httpapi.WrapFunc(...)` for quick adapters around existing handler functions. -- `httpapi.TypedHandlerFunc` for compact typed handlers. -- Structs implementing `httpapi.TypedHandler` when route documentation needs richer metadata. +`httpapi` wraps classic `net/http` handlers, preserves existing +request/response shapes, and emits OpenAPI 3.0 for typed handlers. Reach for it +when you're **migrating an existing API**, **need raw HTTP control** (custom +media types, multi-status routes, form/multipart bodies), or must **preserve an +established OpenAPI contract**. ```go router := httpapi.NewRouter() @@ -369,296 +158,13 @@ router.HandleTyped( router.ServeAllDocs() ``` -The legacy HTTP router supports the same opt-in debug console: - -```go -router := httpapi.NewRouter(httpapi.WithDebugConsole()) -``` - -For larger routes, move the contract onto the handler implementation so router files remain easy to scan: - -```go -type StateByCodeHandler struct { - Store StateStore -} - -var _ httpapi.TypedHandler = StateByCodeHandler{} - -func (h StateByCodeHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { - // runtime handler -} - -func (h StateByCodeHandler) RequestType() any { - return GetStateRequest{} -} +More: **[httpapi patterns cookbook](docs/http-legacy/patterns.md)** covers typed +handlers, guards, OR auth, typed path/query params, form bodies, and explicit +response specs. -func (h StateByCodeHandler) ResponseType() any { - return StateResponse{} -} +## Migrating to Virtuous -func (h StateByCodeHandler) Metadata() httpapi.HandlerMeta { - return httpapi.HandlerMeta{ - Service: "States", - Method: "GetByCode", - Responses: []httpapi.ResponseSpec{ - {Status: 200, Body: StateResponse{}}, - {Status: 404, Body: ErrorResponse{}}, - }, - } -} - -router.HandleTyped("GET /api/v1/lookup/states/{code}", StateByCodeHandler{Store: store}) -``` - -Docs modules can be toggled the same way: - -```go -router.ServeDocs( - httpapi.WithModules(httpapi.ModuleAPI), -) -``` - -### Advanced patterns - -#### 1) One guard for a collection of routes (group-level intent) - -`httpapi` does not have a router-wide `WithGuards(...)` option today. -Use a shared guard slice and pass it to each route in the collection. - -```go -adminGuards := []httpapi.Guard{sessionGuard{}} - -router := httpapi.NewRouter() -router.HandleTyped( - "GET /api/admin/users", - httpapi.WrapFunc(AdminUsersGetMany, nil, UsersResponse{}, httpapi.HandlerMeta{ - Service: "AdminUsers", - Method: "GetMany", - }), - adminGuards..., -) -router.HandleTyped( - "POST /api/admin/users/disable", - httpapi.WrapFunc(AdminUsersDisable, nil, DisableUserResponse{}, httpapi.HandlerMeta{ - Service: "AdminUsers", - Method: "Disable", - }), - adminGuards..., -) -``` - -If you apply middleware only at mux level, requests are still protected, but auth metadata is not emitted in OpenAPI unless guards are attached to typed routes. - -#### 2) Multiple documentation sets (Public Service, Secret Service) - -Use separate routers, each with its own docs/OpenAPI/client paths. - -```go -publicAPI := httpapi.NewRouter() -publicAPI.HandleTyped( - "GET /public/health", - httpapi.WrapFunc(PublicHealth, nil, HealthResponse{}, httpapi.HandlerMeta{ - Service: "PublicService", - Method: "Health", - }), -) -publicAPI.ServeAllDocs( - httpapi.WithDocsOptions( - httpapi.WithDocsPath("/public/docs"), - httpapi.WithOpenAPIPath("/public/openapi.json"), - ), - httpapi.WithClientJSPath("/public/client.gen.js"), - httpapi.WithClientTSPath("/public/client.gen.ts"), - httpapi.WithClientPYPath("/public/client.gen.py"), -) - -secretGuards := []httpapi.Guard{internalTokenGuard{}} -secretAPI := httpapi.NewRouter() -secretAPI.HandleTyped( - "POST /secret/rotate-keys", - httpapi.WrapFunc(RotateKeys, nil, RotateKeysResponse{}, httpapi.HandlerMeta{ - Service: "SecretService", - Method: "RotateKeys", - }), - secretGuards..., -) -secretAPI.ServeAllDocs( - httpapi.WithDocsOptions( - httpapi.WithDocsPath("/secret/docs"), - httpapi.WithOpenAPIPath("/secret/openapi.json"), - ), - httpapi.WithClientJSPath("/secret/client.gen.js"), - httpapi.WithClientTSPath("/secret/client.gen.ts"), - httpapi.WithClientPYPath("/secret/client.gen.py"), -) - -mux := http.NewServeMux() -mux.Handle("/public/", publicAPI) -mux.Handle("/secret/", secretAPI) -``` - -#### 3) Basic auth on the docs route - -Use a mountable docs handler so you can protect docs separately from API routes. - -```go -func docsBasicAuth(user, pass string, next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - u, p, ok := r.BasicAuth() - if !ok || u != user || p != pass { - // Sends a Basic Auth challenge so browsers show a username/password prompt. - w.Header().Set("WWW-Authenticate", `Basic realm="Virtuous Docs"`) - http.Error(w, "unauthorized", http.StatusUnauthorized) - return - } - next.ServeHTTP(w, r) - }) -} - -router := httpapi.NewRouter() -router.HandleTyped( - "GET /api/v1/lookup/states/{code}", - httpapi.WrapFunc(StateByCode, nil, StateResponse{}, httpapi.HandlerMeta{ - Service: "States", - Method: "GetByCode", - }), -) - -router.ServeAllDocs(httpapi.WithoutDocs()) // keep generated clients -docs := router.DocsHandler( - httpapi.WithModules(httpapi.ModuleAPI, httpapi.ModuleObservability), -) -admin := router.AdminHandler( - httpapi.WithModules(httpapi.ModuleObservability), - httpapi.WithPublicAdmin(), // protected by docsBasicAuth below -) - -mux := http.NewServeMux() -mux.Handle("/", router) // API routes -mux.Handle( - "/admin/docs/", - http.StripPrefix("/admin/docs", docsBasicAuth("docs", "secret", docs)), -) -mux.Handle( - "GET /admin/docs/_admin/", - http.StripPrefix("/admin/docs/_admin", docsBasicAuth("docs", "secret", admin)), -) -``` - -This mounts docs at `/admin/docs/`, with OpenAPI at `/admin/docs/openapi.json`. - -#### 4) OR auth semantics (accept either of two schemes) - -Normal guard lists mean every guard runs, so they model AND auth. When a route should accept either credential type, wrap the guards with `httpapi.AuthAny(...)`. - -```go -router.HandleTyped( - "GET /api/v1/secure/report", - httpapi.WrapFunc(GetSecureReport, nil, ReportResponse{}, httpapi.HandlerMeta{ - Service: "Reports", - Method: "GetSecure", - }), - httpapi.AuthAny(bearerGuard{}, apiKeyGuard{}), -) -``` - -#### 5) Typed path/query params - -Use `path` and `query` tags on request structs to preserve scalar parameter types in OpenAPI and generated clients. Handlers still parse runtime values from `*http.Request`. - -```go -type GetStateRequest struct { - ID int64 `path:"id" doc:"Numeric state ID."` - Verbose bool `query:"verbose,omitempty" doc:"Include extra fields."` -} - -router.HandleTyped( - "GET /api/v1/states/{id}", - httpapi.WrapFunc(StateByID, GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ - Service: "States", - Method: "GetByID", - }), -) -``` - -Use `enum:"..."` when a scalar path/query/body field has constrained values: - -```go -type ReportListRequest struct { - SortBy string `query:"sort_by,omitempty" enum:"created_at,name"` - SortOrder string `query:"sort_order,omitempty" enum:"asc,desc"` -} -``` - -If the route is already mounted on another mux, use `Describe` to add only the generated docs/client contract: - -```go -router.Describe("GET /api/v1/states/{id}", GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ - Service: "States", - Method: "GetByID", -}) -``` - -#### 6) Form request body contract - -Use `HandlerMeta.RequestBody` when the request media type is not JSON. `httpapi.FormBody(...)` emits `application/x-www-form-urlencoded`; `httpapi.MultipartBody(...)` emits `multipart/form-data` and maps `httpapi.File` fields to binary file parts. Generated clients encode `form` tag wire names. - -```go -type FacebookComplianceRequest struct { - Mode string `json:"mode" form:"hub.mode"` - VerifyToken string `json:"verifyToken" form:"hub.verify_token"` -} - -router.HandleTyped( - "POST /facebook/compliance", - httpapi.WrapFunc(FacebookCompliance, nil, httpapi.NoResponse200{}, httpapi.HandlerMeta{ - Service: "Callbacks", - Method: "FacebookCompliance", - RequestBody: httpapi.FormBody(FacebookComplianceRequest{}), - }), -) -``` - -#### 7) Optional request body contract - -Request bodies are required by default when you pass a typed request. -Use `httpapi.Optional` when a route should accept either no body or a JSON body. - -```go -router := httpapi.NewRouter() -router.HandleTyped( - "POST /api/v1/search", - httpapi.WrapFunc(Search, httpapi.Optional[SearchRequest](), SearchResponse{}, httpapi.HandlerMeta{ - Service: "Search", - Method: "Run", - }), -) -``` - -#### 8) Explicit response specs - -Use `HandlerMeta.Responses` when a route needs multiple statuses or a custom response media type. - -```go -router := httpapi.NewRouter() -router.HandleTyped( - "GET /api/v1/assets/{id}/preview.png", - httpapi.WrapFunc(ServePreviewPNG, nil, nil, httpapi.HandlerMeta{ - Service: "Assets", - Method: "GetPreview", - Responses: []httpapi.ResponseSpec{ - {Status: 200, Body: []byte{}, MediaType: "image/png"}, - {Status: 404, Body: ErrorResponse{}}, - }, - }), -) -``` - -## Combined (migration demo) - -Both routers can be mounted in the same server to support incremental migration. - -This layout is intended for transition periods, not as a long-term structure. +You don't have to rewrite. Mount both routers on one mux during the transition: ```go httpRouter := httpstates.BuildRouter() @@ -672,34 +178,19 @@ mux.Handle("/rpc/", rpcRouter) mux.Handle("/", httpRouter) ``` -## Why RPC? - -Virtuous uses an RPC-style API model because it produces **simpler, more reliable systems**—especially when APIs are consumed by agents. - -RPC treats APIs as **typed functions**, not as collections of loosely related HTTP resources. This keeps the surface area small and the intent explicit. - -### What RPC optimizes for - -- **Clarity over convention** — function names express intent directly, without guessing paths or schemas. -- **Types as the contract** — request and response structs *are* the API; no separate schema to sync. -- **Predictable code generation** — small, explicit signatures produce reliable client SDKs. -- **Fewer invalid states** — avoids ambiguous partial updates, nested resources, and overloaded semantics. -- **Runtime truth** — routes, schemas, docs, and clients all derive from the same runtime definitions. - -### HTTP still matters - -Virtuous RPC runs on HTTP and uses HTTP status codes intentionally. -What changes is the *mental model*: from “resources and verbs” to “operations with inputs and outputs.” - -For teams migrating existing APIs or preserving established contracts, Virtuous also supports classic `net/http` handlers via `httpapi`. - -RPC is the default because it’s **harder to misuse and easier to automate**. - +- **From swaggo:** the [Swaggo migration guide](docs/tutorials/migrate-swaggo.md) + has annotation-mapping rules, route-by-route `rpc` vs `httpapi` decisions, and a + copy-paste agent prompt. +- **From gin, echo, chi, fiber, or vanilla `net/http`:** see + [Coming from gin/echo/chi/fiber/net-http](docs/tutorials/coming-from-routers.md) + for a concept map and before/after recipes — keep your handlers, wrap them in + `httpapi`, migrate to `rpc` incrementally. ## Docs -- `docs/overview.md` — primary documentation (RPC-first) -- `docs/agent_quickstart.md` — agent-oriented usage guide +- [`docs/overview.md`](docs/overview.md) — primary documentation (RPC-first) +- [`docs/agent_quickstart.md`](docs/agent_quickstart.md) — agent-oriented usage guide +- [`docs/doc-spec.md`](docs/doc-spec.md) — the documentation contract these docs follow - `example/byodb/docs/STYLEGUIDES.md` — byodb styleguide index and canonical flow ## Requirements diff --git a/docs/doc-spec.md b/docs/doc-spec.md index d01d28e..38d0a72 100644 --- a/docs/doc-spec.md +++ b/docs/doc-spec.md @@ -1,301 +1,172 @@ -Virtuous Documentation System Plan -Authoring, Rendering, and Agent-First Strategy +--- +title: Documentation Contract +description: The authoring rules every Virtuous doc follows — frontmatter, headings, code blocks, and callouts. +section: Specs +audience: both +status: stable +--- -OVERVIEW +# Documentation Contract -Virtuous documentation should be treated as a first-class system, not a collection of pages. +This page is the contract for Virtuous documentation. Every doc in this tree +follows it, including this one. If a rule here is not being followed, the doc is +wrong — not the rule. -The goals are: +The contract is deliberately small. It only requires things that are enforceable +today on GitHub-rendered Markdown and useful to both humans and agents. It does +not mandate tooling (static site generator, build-time JSON index, versioned URL +trees) that the project has not built. Those ideas live in +[Not yet — deferred ideas](#not-yet--deferred-ideas) so the contract never claims +capabilities that do not exist. -Fast human onboarding +## Why this exists -Deterministic agent consumption +Virtuous treats code, schemas, and clients as a single runtime truth that cannot +drift. Documentation has historically been the exception — easy to let rot. A +short, mechanical contract keeps docs diff-friendly, predictably structured for +agents, and honest about what is stable versus experimental. -Zero ambiguity about contracts and guarantees +> [!NOTE] +> "Agent" below means an LLM coding agent reading these docs to write or migrate +> Virtuous code. Predictable structure and explicit stability signals are what +> let an agent consume docs deterministically. -Tight coupling between code, specs, and docs +## Frontmatter (required) -Long-term versionability without rewrites - -Documentation should feel like: -A readable specification with executable examples. - -AUTHORING FORMAT - -Primary authoring format: Markdown -But not “loose” Markdown — disciplined, structured Markdown with strict conventions. - -Markdown is chosen because: - -Humans already know it - -Agents can parse it reliably - -Git-native and diff-friendly - -Supported by all modern static site generators - -Plain Markdown alone is insufficient. Every document must follow a contract. - -FRONTMATTER (REQUIRED FOR ALL DOCS) - -Each document begins with structured frontmatter metadata. +Every doc begins with YAML frontmatter. GitHub renders it as a table, so it is +visible to humans and parseable by agents. Required fields: -title: Human-readable page title - -description: One-sentence summary - -section: Top-level nav category - -audience: human, agent, or both - -stability: stable | experimental | internal - -Optional but recommended: - -since: version introduced - -related: list of related doc paths - -deprecated: true | false - -Purpose of frontmatter: - -Navigation - -Search - -Version filtering - -Agent discovery - -Stability signaling - -Docs are data, not prose. - -HEADING RULES (STRICT) - -Exactly one top-level title per page - -Headings must not skip levels - -Heading hierarchy must be consistent across the site - -Recommended semantic sections: - -Overview - -Why This Exists - -How It Works - -Example - -Guarantees - -Anti-Patterns - -Notes for Agents - -Agents rely on predictable section names. - -CODE BLOCK RULES - -Every code block must be language-tagged - -No untyped code blocks allowed - -Code examples must compile or be clearly labeled as illustrative - -Code is not decoration. It is executable documentation. - -CALLOUTS AND SEMANTIC BLOCKS - -Avoid ad-hoc prose like “Note:” or “Warning:”. - -Use semantic callouts that can be rendered consistently: - -Warning - -Info - -Agent - -Anti-Pattern - -Guarantee - -These should be parsed at render time, not improvised in content. - -RENDERING STRATEGY - -Documentation should be rendered using a static site generator (SSG). - -Requirements: - -Static output - -Fast navigation - -Versioning support - -Search indexing - -Clean URLs - -CI-friendly builds - -Recommended options: - -Astro with Markdown and custom components (preferred) - -Docusaurus (excellent if versioning is prioritized early) - -Hugo (acceptable but less flexible for agent metadata) - -The renderer must not obscure content structure. - -DOCS AS DATA (AGENT-FIRST REQUIREMENT) - -At build time, documentation should emit: - -HTML for humans - -A structured JSON index for agents - -Each page should be representable as: - -Path - -Title - -Audience - -Stability - -Section list - -Version constraints - -This allows: - -Agent discovery of capabilities - -Deterministic navigation - -Filtering unstable or experimental content - -Docs should be queryable, not just readable. - -VERSIONING STRATEGY - -Documentation must be versioned alongside the framework. - -Recommended URL structure: - -/docs/latest - -/docs/v0.1 - -/docs/v0.2 - -Docs should not silently change meaning. - -Each page should declare: - -When it was introduced - -Whether it is stable or experimental - -Agents must be able to reason about compatibility. - -SITE STRUCTURE (CANONICAL) - -Top-level documentation sections: - -Getting Started - -Concepts - -Tutorials - -RPC - -HTTP (Legacy) - -Agents - -Reference - -Examples - -Specs - -Python Loader - -Internals - -This structure should map directly to folders on disk. - -No hidden magic. - -UX PRINCIPLES - -For humans: - -Minimal navigation depth - -Fast search - -Copyable code - -Clear warnings and guarantees - -Strong opinionation - -For agents: - -Predictable structure - -Explicit guarantees - -No prose ambiguity - -Stable anchors - -Machine-readable metadata - -WHY THIS FITS VIRTUOUS - -This documentation system: - -Mirrors Virtuous’ typed, explicit design philosophy - -Treats docs as runtime truth - -Makes agent support intentional, not accidental - -Avoids long-term documentation drift - -Scales cleanly as the framework grows - -Virtuous documentation should feel like: -A contract you can read and trust. - -RECOMMENDED NEXT STEPS - -Commit this document as the docs system contract - -Lock the frontmatter schema - -Choose the static site generator - -Draft “Getting Started” using these rules - -Build the agent docs index alongside HTML - -Once the contract is locked, implementation is mechanical. - -END OF DOCUMENT +| Field | Meaning | +| --- | --- | +| `title` | Human-readable page title. Matches the H1. | +| `description` | One sentence. What the page is for. | +| `section` | Top-level nav category (see [Sections](#sections)). | +| `audience` | `human`, `agent`, or `both`. | +| `status` | `stable`, `experimental`, or `internal`. | + +Optional fields: + +| Field | Meaning | +| --- | --- | +| `since` | Version the documented behavior was introduced (e.g. `0.0.55`). | +| `related` | List of related doc paths, relative to `docs/`. | + +```yaml +--- +title: RPC Router +description: How rpc.NewRouter wires handlers, prefixes, and guards. +section: RPC +audience: both +status: stable +related: + - rpc/handlers.md + - rpc/guards.md +--- +``` + +> [!IMPORTANT] +> `status: internal` marks docs that ship in the tree but are not part of the +> public story (design notes, audits, trackers). Keep them under a clearly +> internal path and never link them from human onboarding pages. + +## Headings + +- Exactly one H1 (`#`) per page. It matches `title`. +- Do not skip heading levels (no H2 → H4). +- Use the canonical section names below when a page needs that kind of content. + +## Sections + +Use the subset of these sections that the page needs, in this order. Not every +page needs all of them — a reference page may be only `Overview` plus tables; a +concept page may add `Why this exists` and `Guarantees`. The point is that when a +section *is* present, it uses the canonical name and order, so agents can find it. + +1. **Overview** — what this is, in two or three sentences. +2. **Why this exists** — the constraint or problem it solves. Optional for pure reference. +3. **How it works** — mechanics, signatures, flow. +4. **Example** — at least one runnable block. See [Code blocks](#code-blocks). +5. **Guarantees** — what callers can rely on. Use a `> [!IMPORTANT]` callout. +6. **Anti-patterns** — what not to do, and why. +7. **Notes for agents** — constraints, footguns, and verification steps specific + to an agent generating code. Optional but encouraged on RPC/httpapi pages. + +## Code blocks + +- Every fenced block is language-tagged. No bare ``` fences. +- Go examples must be `gofmt`-clean (tabs, not spaces) and compile against the + documented API. If a block is intentionally partial, prefix it with a sentence + that says so ("Illustrative — omits imports"). +- After a runnable block, say what success looks like: the URL to hit, the + expected output, or the file that gets generated. + +```go +router := rpc.NewRouter(rpc.WithPrefix("/rpc")) +router.HandleRPC(states.GetByCode) +router.ServeAllDocs() +// Docs now live at /rpc/docs; OpenAPI at /rpc/openapi.json. +``` + +## Callouts + +Use GitHub alert syntax so callouts render natively on github.com. Do not write +ad-hoc `**Note:**` prose. + +| Intent | Syntax | +| --- | --- | +| General note | `> [!NOTE]` | +| Recommended path / tip | `> [!TIP]` | +| Guarantee or required behavior | `> [!IMPORTANT]` | +| Footgun / easy to misuse | `> [!WARNING]` | +| Data loss / hard to reverse | `> [!CAUTION]` | + +> [!WARNING] +> Map an anti-pattern to `> [!WARNING]`, not `> [!CAUTION]`. Reserve `CAUTION` +> for things that lose data or are hard to undo. + +## Sections (canonical nav) + +These map directly to folders under `docs/`. No hidden categories. + +- Getting Started — `getting-started/` +- Concepts — `concepts/` +- Tutorials — `tutorials/` +- RPC — `rpc/` +- HTTP (httpapi) — `http-legacy/` +- Agents — `agents/` +- Reference — `reference/` +- Examples — `examples/` +- Specs — `specs/` (and this contract) +- Python Loader — `python-loader/` +- Internals — `internals/` + +## Guarantees + +> [!IMPORTANT] +> A doc that carries valid frontmatter, a single H1, language-tagged code blocks, +> and canonical section names where applicable is contract-compliant. Reviewers +> may reject docs that violate these four rules. Everything else in this page is +> guidance, not a gate. + +## Anti-patterns + +- Frontmatter that lies about `status` (marking experimental APIs `stable`). +- Untagged code fences, or Go blocks that have not been `gofmt`-ed. +- Inventing callout types outside the [table above](#callouts). +- Linking `internal` docs from onboarding or overview pages. +- Re-documenting the same cookbook in two places. Link to one canonical page. + +## Not yet — deferred ideas + +These were in the original plan and may return when the project is ready to build +and maintain them. They are explicitly **not** part of the current contract, so no +doc is judged against them: + +- A static site generator with clean URLs and search indexing. +- A build-time machine-readable JSON index of all pages for agent discovery. +- Versioned doc trees (`/docs/v0.1`, `/docs/v0.2`) with per-page version + constraints. + +When one of these ships, promote it out of this section and into the body. diff --git a/docs/http-legacy/patterns.md b/docs/http-legacy/patterns.md new file mode 100644 index 0000000..5705420 --- /dev/null +++ b/docs/http-legacy/patterns.md @@ -0,0 +1,341 @@ +--- +title: httpapi Patterns (Cookbook) +description: Copy-paste recipes for typed handlers, guards, multiple docs sets, OR auth, typed params, form bodies, and explicit response specs with the httpapi router. +section: HTTP (httpapi) +audience: both +status: stable +related: + - http-legacy/overview.md + - http-legacy/typed-handlers.md + - http-legacy/query-params.md +--- + +# httpapi Patterns (Cookbook) + +## Overview + +Recipes for the `httpapi` router — the HTTP-native library for migrating existing +`net/http` handlers and for routes that need raw HTTP control while still emitting +OpenAPI and generated clients. For new typed services, prefer the +[RPC router](../rpc/overview.md). + +## How it works + +Method-prefixed patterns (`GET /path`) are required for docs and client +generation. Typed `httpapi` routes default to JSON; explicit metadata covers +compatibility needs such as typed path/query params, form request bodies, custom +response media types, multiple statuses, and OR auth. + +Keep the verb in the route string and use one of the blessed typed-handler forms: + +- `httpapi.WrapFunc(...)` — quick adapters around existing handler functions. +- `httpapi.TypedHandlerFunc` — compact typed handlers. +- Structs implementing `httpapi.TypedHandler` — when route documentation needs + richer metadata. + +```go +router := httpapi.NewRouter() +router.HandleTyped( + "GET /api/v1/lookup/states/{code}", + httpapi.WrapFunc(StateByCode, GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ + Service: "States", + Method: "GetByCode", + }), +) +router.ServeAllDocs() +``` + +The legacy HTTP router supports the same opt-in debug console as RPC: + +```go +router := httpapi.NewRouter(httpapi.WithDebugConsole()) +``` + +For larger routes, move the contract onto the handler implementation so router +files stay easy to scan: + +```go +type StateByCodeHandler struct { + Store StateStore +} + +var _ httpapi.TypedHandler = StateByCodeHandler{} + +func (h StateByCodeHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { + // runtime handler +} + +func (h StateByCodeHandler) RequestType() any { + return GetStateRequest{} +} + +func (h StateByCodeHandler) ResponseType() any { + return StateResponse{} +} + +func (h StateByCodeHandler) Metadata() httpapi.HandlerMeta { + return httpapi.HandlerMeta{ + Service: "States", + Method: "GetByCode", + Responses: []httpapi.ResponseSpec{ + {Status: 200, Body: StateResponse{}}, + {Status: 404, Body: ErrorResponse{}}, + }, + } +} + +router.HandleTyped("GET /api/v1/lookup/states/{code}", StateByCodeHandler{Store: store}) +``` + +Docs modules can be toggled the same way as RPC: + +```go +router.ServeDocs( + httpapi.WithModules(httpapi.ModuleAPI), +) +``` + +## One guard for a collection of routes (group-level intent) + +`httpapi` does not have a router-wide `WithGuards(...)` option today. Use a shared +guard slice and pass it to each route in the collection. + +```go +adminGuards := []httpapi.Guard{sessionGuard{}} + +router := httpapi.NewRouter() +router.HandleTyped( + "GET /api/admin/users", + httpapi.WrapFunc(AdminUsersGetMany, nil, UsersResponse{}, httpapi.HandlerMeta{ + Service: "AdminUsers", + Method: "GetMany", + }), + adminGuards..., +) +router.HandleTyped( + "POST /api/admin/users/disable", + httpapi.WrapFunc(AdminUsersDisable, nil, DisableUserResponse{}, httpapi.HandlerMeta{ + Service: "AdminUsers", + Method: "Disable", + }), + adminGuards..., +) +``` + +> [!WARNING] +> Applying middleware only at the mux level protects requests but does **not** +> emit auth metadata in OpenAPI. Attach guards to typed routes so generated +> clients know the route is protected. + +## Multiple documentation sets (Public Service, Secret Service) + +Use separate routers, each with its own docs/OpenAPI/client paths. + +```go +publicAPI := httpapi.NewRouter() +publicAPI.HandleTyped( + "GET /public/health", + httpapi.WrapFunc(PublicHealth, nil, HealthResponse{}, httpapi.HandlerMeta{ + Service: "PublicService", + Method: "Health", + }), +) +publicAPI.ServeAllDocs( + httpapi.WithDocsOptions( + httpapi.WithDocsPath("/public/docs"), + httpapi.WithOpenAPIPath("/public/openapi.json"), + ), + httpapi.WithClientJSPath("/public/client.gen.js"), + httpapi.WithClientTSPath("/public/client.gen.ts"), + httpapi.WithClientPYPath("/public/client.gen.py"), +) + +secretGuards := []httpapi.Guard{internalTokenGuard{}} +secretAPI := httpapi.NewRouter() +secretAPI.HandleTyped( + "POST /secret/rotate-keys", + httpapi.WrapFunc(RotateKeys, nil, RotateKeysResponse{}, httpapi.HandlerMeta{ + Service: "SecretService", + Method: "RotateKeys", + }), + secretGuards..., +) +secretAPI.ServeAllDocs( + httpapi.WithDocsOptions( + httpapi.WithDocsPath("/secret/docs"), + httpapi.WithOpenAPIPath("/secret/openapi.json"), + ), + httpapi.WithClientJSPath("/secret/client.gen.js"), + httpapi.WithClientTSPath("/secret/client.gen.ts"), + httpapi.WithClientPYPath("/secret/client.gen.py"), +) + +mux := http.NewServeMux() +mux.Handle("/public/", publicAPI) +mux.Handle("/secret/", secretAPI) +``` + +## Basic auth on the docs route + +Use a mountable docs handler so you can protect docs separately from API routes. + +```go +func docsBasicAuth(user, pass string, next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + u, p, ok := r.BasicAuth() + if !ok || u != user || p != pass { + // Sends a Basic Auth challenge so browsers show a username/password prompt. + w.Header().Set("WWW-Authenticate", `Basic realm="Virtuous Docs"`) + http.Error(w, "unauthorized", http.StatusUnauthorized) + return + } + next.ServeHTTP(w, r) + }) +} + +router := httpapi.NewRouter() +router.HandleTyped( + "GET /api/v1/lookup/states/{code}", + httpapi.WrapFunc(StateByCode, nil, StateResponse{}, httpapi.HandlerMeta{ + Service: "States", + Method: "GetByCode", + }), +) + +router.ServeAllDocs(httpapi.WithoutDocs()) // keep generated clients +docs := router.DocsHandler( + httpapi.WithModules(httpapi.ModuleAPI, httpapi.ModuleObservability), +) +admin := router.AdminHandler( + httpapi.WithModules(httpapi.ModuleObservability), + httpapi.WithPublicAdmin(), // protected by docsBasicAuth below +) + +mux := http.NewServeMux() +mux.Handle("/", router) // API routes +mux.Handle( + "/admin/docs/", + http.StripPrefix("/admin/docs", docsBasicAuth("docs", "secret", docs)), +) +mux.Handle( + "GET /admin/docs/_admin/", + http.StripPrefix("/admin/docs/_admin", docsBasicAuth("docs", "secret", admin)), +) +``` + +This mounts docs at `/admin/docs/`, with OpenAPI at `/admin/docs/openapi.json`. + +## OR auth semantics (accept either of two schemes) + +Normal guard lists mean every guard runs, so they model AND auth. When a route +should accept either credential type, wrap the guards with `httpapi.AuthAny(...)`. + +```go +router.HandleTyped( + "GET /api/v1/secure/report", + httpapi.WrapFunc(GetSecureReport, nil, ReportResponse{}, httpapi.HandlerMeta{ + Service: "Reports", + Method: "GetSecure", + }), + httpapi.AuthAny(bearerGuard{}, apiKeyGuard{}), +) +``` + +## Typed path/query params + +Use `path` and `query` tags on request structs to preserve scalar parameter types +in OpenAPI and generated clients. Handlers still parse runtime values from +`*http.Request`. + +```go +type GetStateRequest struct { + ID int64 `path:"id" doc:"Numeric state ID."` + Verbose bool `query:"verbose,omitempty" doc:"Include extra fields."` +} + +router.HandleTyped( + "GET /api/v1/states/{id}", + httpapi.WrapFunc(StateByID, GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ + Service: "States", + Method: "GetByID", + }), +) +``` + +Use `enum:"..."` when a scalar path/query/body field has constrained values: + +```go +type ReportListRequest struct { + SortBy string `query:"sort_by,omitempty" enum:"created_at,name"` + SortOrder string `query:"sort_order,omitempty" enum:"asc,desc"` +} +``` + +If the route is already mounted on another mux, use `Describe` to add only the +generated docs/client contract: + +```go +router.Describe("GET /api/v1/states/{id}", GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ + Service: "States", + Method: "GetByID", +}) +``` + +## Form request body contract + +Use `HandlerMeta.RequestBody` when the request media type is not JSON. +`httpapi.FormBody(...)` emits `application/x-www-form-urlencoded`; +`httpapi.MultipartBody(...)` emits `multipart/form-data` and maps `httpapi.File` +fields to binary file parts. Generated clients encode `form` tag wire names. + +```go +type FacebookComplianceRequest struct { + Mode string `json:"mode" form:"hub.mode"` + VerifyToken string `json:"verifyToken" form:"hub.verify_token"` +} + +router.HandleTyped( + "POST /facebook/compliance", + httpapi.WrapFunc(FacebookCompliance, nil, httpapi.NoResponse200{}, httpapi.HandlerMeta{ + Service: "Callbacks", + Method: "FacebookCompliance", + RequestBody: httpapi.FormBody(FacebookComplianceRequest{}), + }), +) +``` + +## Optional request body contract + +Request bodies are required by default when you pass a typed request. Use +`httpapi.Optional` when a route should accept either no body or a JSON body. + +```go +router := httpapi.NewRouter() +router.HandleTyped( + "POST /api/v1/search", + httpapi.WrapFunc(Search, httpapi.Optional[SearchRequest](), SearchResponse{}, httpapi.HandlerMeta{ + Service: "Search", + Method: "Run", + }), +) +``` + +## Explicit response specs + +Use `HandlerMeta.Responses` when a route needs multiple statuses or a custom +response media type. + +```go +router := httpapi.NewRouter() +router.HandleTyped( + "GET /api/v1/assets/{id}/preview.png", + httpapi.WrapFunc(ServePreviewPNG, nil, nil, httpapi.HandlerMeta{ + Service: "Assets", + Method: "GetPreview", + Responses: []httpapi.ResponseSpec{ + {Status: 200, Body: []byte{}, MediaType: "image/png"}, + {Status: 404, Body: ErrorResponse{}}, + }, + }), +) +``` diff --git a/docs/overview.md b/docs/overview.md index ea5b4d0..2ce3545 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -240,13 +240,6 @@ These routers are useful for low-level HTTP control, but they do not provide nat If you have existing handlers built on these routers, keep them and wrap them via `httpapi` while you plan an RPC migration. -Agent prompt (porting legacy handlers): +See the full guide for a concept map, before/after recipes per framework, and an agent port prompt: -```text -Port legacy handlers into Virtuous. -- Target Virtuous version: read `VERSION` in the repo and pin it in the output. -- For each handler, decide: RPC (new) or httpapi (legacy). -- For legacy: wrap http.HandlerFunc with httpapi.WrapFunc and register a method-prefixed route. -- For new: create an RPC handler and register with router.HandleRPC. -- Keep documentation served from Virtuous routers only. -``` +- `docs/tutorials/coming-from-routers.md` diff --git a/docs/rpc/patterns.md b/docs/rpc/patterns.md new file mode 100644 index 0000000..58dc646 --- /dev/null +++ b/docs/rpc/patterns.md @@ -0,0 +1,232 @@ +--- +title: RPC Patterns (Cookbook) +description: Copy-paste recipes for guards, multiple docs sets, protected docs, OR auth, and observability with the RPC router. +section: RPC +audience: both +status: stable +related: + - rpc/router.md + - rpc/guards.md + - rpc/docs-and-clients.md +--- + +# RPC Patterns (Cookbook) + +## Overview + +Recipes for the `rpc` router beyond the basics. Each one is self-contained — copy +it, swap in your handlers, run it. For the minimal first service, start with the +[quickstart](../getting-started/quickstart.md). + +## One guard for a collection of routes (group-level) + +Use a dedicated router for the guarded group, then mount both routers on one mux. + +```go +admin := rpc.NewRouter( + rpc.WithPrefix("/rpc/admin"), + rpc.WithGuards(sessionGuard{}), // applies to every admin route +) +admin.HandleRPC(adminusers.GetMany) +admin.HandleRPC(adminusers.Disable) + +public := rpc.NewRouter(rpc.WithPrefix("/rpc/public")) +public.HandleRPC(publichealth.Check) + +mux := http.NewServeMux() +mux.Handle("/rpc/admin/", admin) +mux.Handle("/rpc/public/", public) +``` + +> [!TIP] +> Wrapping a mux subtree with middleware works for transport security, but +> `rpc.WithGuards(...)` is the docs/client-aware path: it emits OpenAPI security +> metadata so generated clients know the route is protected. + +## Multiple documentation sets (Public Service, Secret Service) + +One router emits one OpenAPI document for all routes on that router. For separate +docs, split routes across routers. + +```go +publicAPI := rpc.NewRouter(rpc.WithPrefix("/rpc/public")) +publicAPI.HandleRPC(publicsvc.GetCatalog) +publicAPI.ServeAllDocs( + rpc.WithDocsOptions( + rpc.WithDocsPath("/rpc/public/docs"), + rpc.WithOpenAPIPath("/rpc/public/openapi.json"), + ), + rpc.WithClientJSPath("/rpc/public/client.gen.js"), + rpc.WithClientTSPath("/rpc/public/client.gen.ts"), + rpc.WithClientPYPath("/rpc/public/client.gen.py"), +) + +secretAPI := rpc.NewRouter( + rpc.WithPrefix("/rpc/secret"), + rpc.WithGuards(internalTokenGuard{}), +) +secretAPI.HandleRPC(secretsvc.RotateKeys) +secretAPI.ServeAllDocs( + rpc.WithDocsOptions( + rpc.WithDocsPath("/rpc/secret/docs"), + rpc.WithOpenAPIPath("/rpc/secret/openapi.json"), + ), + rpc.WithClientJSPath("/rpc/secret/client.gen.js"), + rpc.WithClientTSPath("/rpc/secret/client.gen.ts"), + rpc.WithClientPYPath("/rpc/secret/client.gen.py"), +) + +mux := http.NewServeMux() +mux.Handle("/rpc/public/", publicAPI) +mux.Handle("/rpc/secret/", secretAPI) +``` + +## Basic auth on the docs route + +Use a mountable docs handler so you can protect docs independently from API routes. + +```go +func docsBasicAuth(user, pass string, next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + u, p, ok := r.BasicAuth() + if !ok || u != user || p != pass { + // Sends a Basic Auth challenge so browsers show a username/password prompt. + w.Header().Set("WWW-Authenticate", `Basic realm="Virtuous Docs"`) + http.Error(w, "unauthorized", http.StatusUnauthorized) + return + } + next.ServeHTTP(w, r) + }) +} + +router := rpc.NewRouter(rpc.WithPrefix("/rpc")) +router.HandleRPC(states.GetMany) + +// Keep generated clients on default routes. +router.ServeAllDocs(rpc.WithoutDocs()) + +// Mount docs separately, with only selected modules enabled. +docs := router.DocsHandler( + rpc.WithModules(rpc.ModuleAPI, rpc.ModuleObservability), +) +admin := router.AdminHandler( + rpc.WithModules(rpc.ModuleObservability), + rpc.WithPublicAdmin(), // protected by docsBasicAuth below +) + +mux := http.NewServeMux() +mux.Handle("/rpc/", router) // API routes +mux.Handle( + "/admin/docs/", + http.StripPrefix("/admin/docs", docsBasicAuth("docs", "secret", docs)), +) +mux.Handle( + "GET /admin/docs/_admin/", + http.StripPrefix("/admin/docs/_admin", docsBasicAuth("docs", "secret", admin)), +) +``` + +This mounts docs at `/admin/docs/`, with OpenAPI at `/admin/docs/openapi.json`. + +> [!NOTE] +> `http.StripPrefix` is required because the handler from `DocsHandler(...)` +> serves routes relative to its own root. Without it, the handler sees +> `/admin/docs/openapi.json` instead of `/openapi.json` and 404s. + +## OR auth semantics (accept either of two schemes) + +For RPC routes that accept either credential type, express that logic in one +composite guard and attach it once. (For legacy `httpapi` routes, use +`httpapi.AuthAny(...)` — see the [httpapi patterns](../http-legacy/patterns.md).) + +```go +type bearerOrAPIKeyGuard struct { + bearer bearerGuard + apiKey apiKeyGuard +} + +func (g bearerOrAPIKeyGuard) Spec() guard.Spec { + return guard.Spec{ + Name: "BearerOrApiKey", + In: "header", + Param: "Authorization", + } +} + +func (g bearerOrAPIKeyGuard) Middleware() func(http.Handler) http.Handler { + return func(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if g.bearer.authenticate(r) || g.apiKey.authenticate(r) { + next.ServeHTTP(w, r) + return + } + http.Error(w, "unauthorized", http.StatusUnauthorized) + }) + } +} +``` + +## Minimal docs modules + +By default docs show the API and Observability modules. Restrict the visible +modules with `WithModules(...)`. + +```go +router := rpc.NewRouter(rpc.WithPrefix("/rpc")) +router.HandleRPC(states.GetMany) +router.ServeDocs( + rpc.WithModules( + rpc.ModuleAPI, + rpc.ModuleObservability, + ), +) +``` + +## Observability + +Basic per-RPC request metrics are tracked in memory by default. Advanced error +grouping, guard metrics, and sampled traces are opt-in. + +```go +router := rpc.NewRouter( + rpc.WithPrefix("/rpc"), + rpc.WithAdvancedObservability( + rpc.WithObservabilitySampling(0.25), + ), +) + +router.HandleRPC(states.GetMany, auth.BearerGuard{}) +router.ServeAllDocs() +``` + +This enables: + +- `/rpc/_virtuous/metrics` for JSON metrics +- `/rpc/_virtuous/observability` as a redirect to the docs page + +Live route/event logging is opt-in at the mux boundary: + +```go +handler := router.AttachLogger(mux) // attach once at top-level +``` + +> [!NOTE] +> If logger attachment is missing, the docs `Observability` view shows a +> zero-data setup snippet rather than failing. + +For local request tracing, enable the debug console on the router. It prints one +compact request line with an `ok`/`warn`/`err` status badge, method, path, +duration, client IP, route pattern, and response bytes. The default stderr logger +colors the badge, status, and method when the destination is a terminal; captured +writers stay plain text. + +```go +router := rpc.NewRouter( + rpc.WithPrefix("/rpc"), + rpc.WithDebugConsole(), +) +``` + +```text +[virtuous] warn 422 POST /rpc/users/user-login 1.6ms ip=203.0.113.8 route=/rpc/users/user-login bytes=44 +``` diff --git a/docs/tutorials/coming-from-routers.md b/docs/tutorials/coming-from-routers.md new file mode 100644 index 0000000..906a8a2 --- /dev/null +++ b/docs/tutorials/coming-from-routers.md @@ -0,0 +1,231 @@ +--- +title: Coming from gin, echo, chi, fiber, or net/http +description: A concept map and before/after recipes for migrating an existing Go router to Virtuous httpapi, then optionally to RPC. +section: Tutorials +audience: both +status: stable +related: + - tutorials/migrate-swaggo.md + - concepts/rpc-vs-httpapi.md + - http-legacy/patterns.md +--- + +# Coming from gin, echo, chi, fiber, or net/http + +## Overview + +If you already have a Go service on gin, echo, chi, fiber, or vanilla +`net/http`, you do not rewrite it to adopt Virtuous. You wrap your existing +handlers in [`httpapi`](../http-legacy/overview.md) — preserving every route and +status — and immediately gain runtime OpenAPI plus generated JS/TS/Python +clients. Move routes to [`rpc`](../rpc/overview.md) later, at your own pace. + +> [!TIP] +> Migrating from **swaggo**? That has its own dedicated guide with annotation +> mapping rules: [Migrate from Swaggo](migrate-swaggo.md). This page is for the +> router itself. + +## Why this exists + +gin, echo, chi, and fiber are good at low-level HTTP control, but none of them +generate runtime OpenAPI and typed clients from the handlers you actually ship. +That's the gap Virtuous fills. The migration is mechanical because Virtuous meets +your code where it already is: at the `net/http` boundary. + +## The decision: wrap or rewrite + +| Your situation | Do this | Effort | +| --- | --- | --- | +| You want docs/clients fast, with zero behavior change | Wrap handlers in `httpapi` | Low | +| The handler is already `http.HandlerFunc` (chi, net/http) | Wrap as-is | Lowest | +| The handler uses a framework context (gin, echo, fiber) | Convert the signature to `net/http`, then wrap | Low–medium | +| The route is new, or you're ready to make it typed | Write it as an `rpc` function | Medium | + +The honest split: **chi and vanilla `net/http` handlers wrap with no logic +change** (they're already `http.Handler`). **gin, echo, and fiber handlers need a +signature rewrite** to `net/http`, but the rewrite is mechanical — swap the +context accessors (see the [accessor map](#accessor-map) below). + +## Concept map + +How each framework's primitives map to Virtuous `httpapi`: + +| Concept | gin | echo | chi | fiber | net/http (1.22+) | Virtuous httpapi | +| --- | --- | --- | --- | --- | --- | --- | +| New router | `gin.Default()` | `echo.New()` | `chi.NewRouter()` | `fiber.New()` | `http.NewServeMux()` | `httpapi.NewRouter()` | +| Register route | `r.GET("/x/:id", h)` | `e.GET("/x/:id", h)` | `r.Get("/x/{id}", h)` | `app.Get("/x/:id", h)` | `mux.HandleFunc("GET /x/{id}", h)` | `router.HandleTyped("GET /x/{id}", httpapi.WrapFunc(h, ...))` | +| Handler shape | `func(*gin.Context)` | `func(echo.Context) error` | `http.HandlerFunc` | `func(*fiber.Ctx) error` | `http.HandlerFunc` | `http.HandlerFunc` (wrapped) | +| Path param | `c.Param("id")` | `c.Param("id")` | `chi.URLParam(r, "id")` | `c.Params("id")` | `r.PathValue("id")` | `r.PathValue("id")` + `path:"id"` tag | +| Query param | `c.Query("q")` | `c.QueryParam("q")` | `r.URL.Query().Get("q")` | `c.Query("q")` | `r.URL.Query().Get("q")` | same + `query:"q"` tag | +| Write JSON | `c.JSON(200, v)` | `c.JSON(200, v)` | encode to `w` | `c.JSON(v)` | encode to `w` | encode to `w` | +| Middleware / auth | `r.Use(mw)` | `e.Use(mw)` | `r.Use(mw)` | `app.Use(mw)` | wrap handler | `guard.Guard` (emits OpenAPI security) | +| OpenAPI spec | manual / swaggo | manual / swaggo | manual | manual | manual | **generated at runtime** | +| Typed clients | none | none | none | none | none | **generated JS/TS/PY** | + +> [!NOTE] +> The `path:"id"` / `query:"q"` struct tags are optional. Your handler still +> parses runtime values from `*http.Request`; the tags only add scalar type +> fidelity to the generated OpenAPI and clients. See +> [typed path/query params](../http-legacy/patterns.md#typed-pathquery-params). + +## chi and net/http: wrap as-is + +chi and vanilla `net/http` handlers are already `http.HandlerFunc`, so the only +change is registering them on an `httpapi` router with typed request/response +metadata. + +### Before (chi) + +```go +func StateByCode(w http.ResponseWriter, r *http.Request) { + code := chi.URLParam(r, "code") + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(StateResponse{State: State{Code: code}}) +} + +r := chi.NewRouter() +r.Get("/api/v1/states/{code}", StateByCode) +``` + +### After (Virtuous httpapi) + +```go +func StateByCode(w http.ResponseWriter, r *http.Request) { + code := r.PathValue("code") // was chi.URLParam(r, "code") + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(StateResponse{State: State{Code: code}}) +} + +router := httpapi.NewRouter() +router.HandleTyped( + "GET /api/v1/states/{code}", + httpapi.WrapFunc(StateByCode, GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ + Service: "States", + Method: "GetByCode", + }), +) +router.ServeAllDocs() +``` + +The route shape and status codes are unchanged. The only edit inside the handler +is swapping `chi.URLParam(r, "code")` for the standard `r.PathValue("code")`. Now +the route appears at `/api/v1/states/{code}`, with docs at `/docs`, the spec at +`/openapi.json`, and clients at `/client.gen.{js,ts,py}`. + +> [!TIP] +> Already mounting the route on another mux and just want the docs/client +> contract without re-installing the handler? Use +> [`router.Describe(...)`](../http-legacy/patterns.md#typed-pathquery-params). + +## gin, echo, and fiber: rewrite the signature, then wrap + +These frameworks use their own request context, so migrating a handler means +converting it to the `net/http` signature. The body logic is the same — only the +context accessors change. + +### Before (gin) + +```go +func StateByCode(c *gin.Context) { + code := c.Param("code") + c.JSON(http.StatusOK, StateResponse{State: State{Code: code}}) +} + +r := gin.Default() +r.GET("/api/v1/states/:code", StateByCode) +``` + +### After (Virtuous httpapi) + +```go +func StateByCode(w http.ResponseWriter, r *http.Request) { + code := r.PathValue("code") + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(StateResponse{State: State{Code: code}}) +} + +router := httpapi.NewRouter() +router.HandleTyped( + "GET /api/v1/states/{code}", + httpapi.WrapFunc(StateByCode, GetStateRequest{}, StateResponse{}, httpapi.HandlerMeta{ + Service: "States", + Method: "GetByCode", + }), +) +router.ServeAllDocs() +``` + +Note the route string changes from gin's `:code` to the standard `{code}` +placeholder. + +### Accessor map + +echo and fiber follow the same rewrite. Swap each framework accessor for its +`net/http` equivalent: + +| What you need | gin | echo | fiber | net/http (after) | +| --- | --- | --- | --- | --- | +| Path param | `c.Param("code")` | `c.Param("code")` | `c.Params("code")` | `r.PathValue("code")` | +| Query param | `c.Query("q")` | `c.QueryParam("q")` | `c.Query("q")` | `r.URL.Query().Get("q")` | +| Bind JSON body | `c.BindJSON(&v)` | `c.Bind(&v)` | `c.BodyParser(&v)` | `json.NewDecoder(r.Body).Decode(&v)` | +| Write JSON | `c.JSON(200, v)` | `c.JSON(200, v)` | `c.JSON(v)` | `json.NewEncoder(w).Encode(v)` | +| Status code | `c.Status(422)` | `return c.NoContent(422)` | `c.SendStatus(422)` | `w.WriteHeader(422)` | + +> [!WARNING] +> Framework middleware (`r.Use(...)`) does not carry over automatically. Re-attach +> auth as a [`guard.Guard`](../rpc/guards.md) so it both runs at request time +> **and** emits OpenAPI security metadata for generated clients. Plain mux-level +> middleware protects requests but is invisible to the docs. + +## Going further: convert to RPC + +Once a route no longer needs to preserve its REST shape, rewrite it as a typed +RPC function and let Virtuous infer the path: + +```go +type GetByCodeRequest struct { + Code string `json:"code" doc:"Two-letter state code."` +} + +type GetByCodeResponse struct { + State State `json:"state"` + Error string `json:"error,omitempty"` +} + +func GetByCode(_ context.Context, req GetByCodeRequest) (GetByCodeResponse, int) { + if req.Code == "" { + return GetByCodeResponse{Error: "code is required"}, http.StatusUnprocessableEntity + } + return GetByCodeResponse{State: State{Code: req.Code}}, http.StatusOK +} + +router := rpc.NewRouter(rpc.WithPrefix("/rpc")) +router.HandleRPC(GetByCode) +router.ServeAllDocs() +``` + +`states.GetByCode` is now served at `/rpc/states/get-by-code` — no route string to +maintain. See [RPC vs httpapi](../concepts/rpc-vs-httpapi.md) for when to make the +jump, and run both routers side by side during the transition (the +[combined demo](../overview.md#combined-demo-only)). + +## Notes for agents + +```text +Port a Go API from gin/echo/chi/fiber/net-http into Virtuous. + +- Read the target Virtuous version from `VERSION` and report it explicitly. +- For each existing route, decide: httpapi (preserve REST shape) or rpc (new/typed). +- httpapi path: register with router.HandleTyped("METHOD /path", httpapi.WrapFunc(...)) + using method-prefixed patterns and {param} placeholders (not :param). +- If the handler uses a framework context (gin/echo/fiber), rewrite it to + func(http.ResponseWriter, *http.Request) and swap context accessors: + path -> r.PathValue, query -> r.URL.Query().Get, JSON -> encoding/json. +- chi and net/http handlers wrap with no logic change beyond r.PathValue. +- Re-attach framework middleware/auth as guard.Guard so OpenAPI security is emitted; + mux-only middleware is not documented. +- Expose docs and clients via ServeAllDocs(). +- Add path:"..."/query:"..." tags only for scalar type fidelity in docs/clients. +- rpc handlers use func(context.Context, Req) (Resp, int) returning 200, 422, or 500; + do not handcraft rpc route strings (paths are inferred). +``` diff --git a/docs/tutorials/overview.md b/docs/tutorials/overview.md index 6dab909..6328a29 100644 --- a/docs/tutorials/overview.md +++ b/docs/tutorials/overview.md @@ -6,4 +6,5 @@ Short, task-focused walkthroughs for common workflows. ## Start here -- `migrate-swaggo.md` +- `migrate-swaggo.md` — migrate from Swaggo annotations to typed Virtuous docs. +- `coming-from-routers.md` — port a gin, echo, chi, fiber, or `net/http` service.