Skip to content

Refuse unknown model ids before reserving - #144

Merged
skyfallwastaken merged 1 commit into
neofrom
unknown-models
Sep 26, 2026
Merged

skyfallwastaken merged 1 commit into
neofrom
unknown-models

Conversation

@skyfallwastaken

Copy link
Copy Markdown
Member

Requests for a model id OpenRouter doesn't list used to go upstream anyway. OpenRouter rejected them with a 400, but the gateway still reserved, recorded and settled each one, so every typo left a "phantom" error row in Activity.

Now the proxy refuses an id the model listing doesn't know with a 400, before anything is reserved or recorded. The message is the same one OpenRouter uses: <id> is not a valid model ID.

An id resolves if it matches, in order:

  1. a listed id (this includes ~ aliases and listed variants such as :free and :batch);
  2. a listed canonical_slug (dated ids);
  3. the base id of a routing variant (openai/gpt-4o-mini:nitro, :floor, …). As a bonus, these now reserve at the base model's price instead of the flat unknown-model hold.

If the listing itself can't be fetched, requests still go through and OpenRouter decides, as before.

Screenshots

The same two typo'd requests from a fresh account, then that account's Activity:

# neo
POST /proxy/v1/chat/completions {"model":"openai/gpt-5-nanoo"}                → 400 (forwarded to OpenRouter)
POST /proxy/v1/chat/completions {"model":"notreal/this-model-does-not-exist"} → 400 (forwarded to OpenRouter)

# this branch
POST /proxy/v1/chat/completions {"model":"openai/gpt-5-nanoo"}                → 400 {"error":"openai/gpt-5-nanoo is not a valid model ID"}
POST /proxy/v1/chat/completions {"model":"notreal/this-model-does-not-exist"} → 400 {"error":"notreal/this-model-does-not-exist is not a valid model ID"}
POST /proxy/v1/chat/completions {"model":"openai/gpt-4o-mini:nitro"}          → 200
Before (neo): two phantom error rows After: only the real request
before after

(The :nitro row still says "nitro" on this branch. The Activity PR fixes that display.)

Testing

  • proxy.integration.test.ts:
    • An unlisted model gets a 400, makes no upstream call, and leaves no billing rows.
    • A :nitro variant is forwarded unchanged, finalized, and reserved at the base model's price.
  • The proxy.test.ts fixture now lists the requested model with no fixed price, so the unknown-model-hold tests still exercise the hold.
  • bun run typecheck: 0 errors. bun test: 0 fail.

🤖 Generated with Claude Code

@skyfallwastaken
skyfallwastaken merged commit 63d30a2 into neo Sep 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant