Skip to content

fix(backend): persist token revocation, typed provider errors, strict retry, safer defaults - #155

Merged
Eslam93 merged 18 commits into
mainfrom
fix/backend-hardening
Nov 7, 2026
Merged

Eslam93 merged 18 commits into
mainfrom
fix/backend-hardening

Conversation

@Eslam93

@Eslam93 Eslam93 commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Why

These are the verified backend defects from the post-interview audit:

  • Refresh-token reuse detection revoked the user's tokens, then rolled the revocation back, so a stolen chain stayed valid.
  • 422 responses echoed the submitted value, including a rejected password.
  • Unhandled 500s had no request id and no security or CORS headers.
  • Provider failures on the text path became generic 500s, while the image path put OpenAI's error text in its 502 body.
  • The corrective retry dropped strict mode and told the model its JSON was invalid when the JSON was usually fine.
  • Model calls held an idle transaction open.
  • Prompts asked the model for stats the brief never gave.
  • Compose exposed Postgres (password app) on every interface.
  • The UI had no way to know the backend runs in mock mode.

What changed

Commit Change
fix(auth) The reuse revocation is committed before the 401, so the request's rollback can't undo it. The integration harness now runs the real get_db_session: the old bare-yield override is why CI never caught this. REQUIRE_DB=1 makes a missing Postgres fail CI instead of skipping the suite.
fix(api) 422 Validation errors keep only loc, msg and type, so Pydantic's input, ctx and url are gone. The envelope shape and the WEAK_PASSWORD remap are unchanged.
fix(api) 500 A pure-ASGI catch-all, innermost in the stack, renders INTERNAL_ERROR inside RequestID, SecurityHeaders and CORS. The 500 now carries request_id, X-Request-ID and the security headers. It reports to Sentry explicitly, because the exception no longer reaches Sentry's ASGI hook. The old Exception handler stays as a last resort.
fix(providers) One ProviderError family with a typed kind, mapped by a single handler to a fixed status, code and message (table below). Upstream text goes to the log only, never into an exception message or a response. insufficient_quota is no longer retried. AI_RATE_LIMITED forwards Retry-After as a header and as details.retry_after_seconds.
fix(content) strict retry The corrective retry keeps the same strict json_schema and names each failing field with Pydantic's message, never the value. This applies to content generation and both improver stages.
fix(db) end_read_transaction() commits the read transaction before every text-path model call, so a 5–30 s call no longer holds a pooled connection idle in transaction.
test(prompts) Golden files pin the rendered system and user prompts for the four content types and both improver stages. UPDATE_GOLDENS=1 regenerates them.
fix(prompts) Removes the asks for invented stats (LinkedIn hook, ad copy, persuasive improver). The email sign-off ends in a [Your name] placeholder instead of "— The Magna team". A test asserts that no rendered prompt mentions "Magna". The four changed prompt modules move to v2.
fix(dev) Compose binds Postgres and the backend to 127.0.0.1. The nginx frontend placeholder and the Redis that nothing read are gone, along with the USE_REDIS / REDIS_URL settings. The frontend runs with pnpm dev, as the README now says.
feat(api) GET /api/v1/meta is public and returns {"provider_mode": "openai" | "mock" | "bedrock", "app_version": "..."}. It exposes nothing else.
fix(content) cut-off retry A strict retry that comes back cut off or filtered degrades to stage 3 with attempt 1's output, as before, rather than failing. Its tokens and cost are still counted.
fix(prompts) / ci blog_post moves to v2 for its new retry wording. The backend-ci comment now says the job fails without Postgres.

New error codes:

Code Status When
AI_PROVIDER_AUTH_FAILED 502 OpenAI rejected the server's key (401/403)
AI_RATE_LIMITED 429 Upstream rate limit after retries; carries Retry-After when OpenAI sent one
AI_UNAVAILABLE 503 5xx or a connection failure after retries, or out of quota
AI_TIMEOUT 504 Timed out after retries
AI_OUTPUT_TRUNCATED 502 A strict-schema reply hit the length limit
AI_CONTENT_REJECTED 422 OpenAI's safety system blocked the request
AI_PROVIDER_MISCONFIGURED 503 The server's AI configuration can't work, e.g. a missing API key or an unpriced model
PROVIDER_ERROR 502 Any other provider failure

Behaviour changes to know about

  • Two tabs refreshing at once now sign the user out on every device. This was always the documented policy, but the rollback bug kept it from taking effect. The frontend's single-flight refresh only covers one tab (follow-up below).
  • Provider failures return the codes above instead of a 500 (text path) or a 502 carrying upstream text (image path).
  • Prompt versions move to *.v2, so rows written from now on record the new prompts.
  • /health drops its never-probed redis key. Every in-repo caller checks only the status code.
  • A stale .env with USE_REDIS / REDIS_URL still boots; a test covers this.
  • docker compose up now starts only Postgres and the backend, reachable only from this machine.
  • python -m app.scripts.seed now commits part-way through, because the services commit before model calls. The seed is idempotent, so a re-run picks up where it stopped.

Verification

  • ruff check, ruff format --check, mypy app (strict) and alembic upgrade head pass.
  • pytest with REQUIRE_DB=1: 597 passed (466 on the base branch). Coverage is 90.68%, up from 89.05%. Every commit passes all checks on its own.
  • Each regression test was shown to fail on the pre-fix code. For example, with the new commit removed, the refresh-reuse test sees the rotated token still unrevoked.
  • Provider error mapping is tested end to end through both real OpenAI providers with httpx.MockTransport scripted responses. A sentinel string checks that no upstream text reaches a response body or an exception message.
  • Three independent review passes (security, correctness, tests/CI/docs) found 10 issues, all low severity. 8 are fixed in this PR; the other 2 are listed below.

Not verified: no live OpenAI call was made, because no API key was used. Error classification follows OpenAI's documented error codes and the SDK's exception types.

Follow-ups noticed but not fixed here

  • The image path still holds SELECT … FOR UPDATE NOWAIT across the prompt call, the image call and the upload. Fixing it needs a schema change (README section 12, item 7: job-table dedupe for image regeneration).
  • With FastAPI 0.136's default request scope, get_db_session commits after the response is sent. The auth routes don't commit explicitly first, so a failed commit there would follow a 200.
  • The frontend could dedupe refreshes across tabs (Web Locks or BroadcastChannel) to avoid the sign-out-everywhere effect.
  • infra/lib/compute-stack.ts still injects USE_REDIS=false, now dead config, and a few infra comments mention it. Changing it needs CDK snapshot updates.
  • The mock LLM's canned email still signs off as "The MagnaCMS team", and the seed's demo brand voice is called "Magna in-house". Both belong with the rename.
  • The *_parse_failed_* log events could include the failure note, which names fields but no values.

Stack

PR 2 of 3, stacked on #154 (model catalog and pricing). Merge that first; GitHub retargets this PR once it merges. The frontend PR is stacked on this one.

🤖 Generated with Claude Code

Eslam93 and others added 17 commits November 6, 2026 20:25
OpenAI bills gpt-5.4-mini at $0.75 input, $0.075 cached input and $4.50
output per 1M tokens; the old $0.15/$0.60 table recorded text costs 5-7.5x
too low. Prices now live in a dated, sourced catalog, cached prompt tokens
bill at the cached rate, and an unpriced response snapshot falls back to
the configured model instead of raising after the call was paid for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OpenAI shut gpt-image-1 down on 2026-10-23, so image generation with the
shipped default failed. gpt-image-2.5-flare is OpenAI's named replacement.
GPT Image models bill per token, so cost now comes from response.usage at
catalog rates, falling back to a labelled per-image estimate only when
usage is missing; PNG output is requested to match the stored extension.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unknown or unpriced model was only discovered after the paid API call,
and nothing flagged a configured model past its shutdown date. Settings
now checks both OpenAI models against the catalog in openai mode: a
shut-down model fails in every environment, and an unknown, wrong-kind or
unpriced one fails outside local. Errors name the env var and replacement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gpt-image-1's shutdown went unnoticed until image generation broke.
check_model_lifecycle fails if a configured model is unknown, unpriced,
retired or within --days of a catalogued shutdown, or if OpenAI's
deprecations page lists it as deprecated (substitute-only mentions don't
count; an unreadable page fails as "could not verify"). Runs weekly in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaying a rotated refresh token revoked all of the user's refresh tokens,
but the 401 that followed made get_db_session roll that back, so a stolen
chain stayed valid for 30 days. Commit it first; two tabs refreshing at once
now do sign the user out on every device, as documented. Tests run the real
get_db_session (a bare-yield override hid this); a missing Postgres fails CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The validation handler returned Pydantic's raw error dicts, whose `input`
is the rejected value itself, so a weak or over-long password came straight
back in the 422 body, along with Pydantic's `ctx` and `url`. Each error now
keeps only `loc`, `msg` and `type`. The envelope shape and the WEAK_PASSWORD
remap are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Starlette runs an Exception handler in ServerErrorMiddleware, outside every
user middleware, so unhandled 500s had a null meta.request_id and no
X-Request-ID, security or CORS headers. An innermost pure-ASGI catch-all now
logs them, reports them to Sentry (they no longer reach its ASGI hook) and
sends the INTERNAL_ERROR envelope. The old handler stays as a last resort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two unrelated ProviderError classes meant the text path never translated
provider failures, so timeouts, 429s and truncated output became generic
500s, while the image path wrapped OpenAI's error text into its 502 body.
One ProviderError family now carries a typed kind that a single handler
maps to a fixed status, code and message; insufficient_quota isn't retried.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ailing field

Stage 2 dropped strict json_schema and told the model its JSON was invalid,
though the usual first-pass failure is valid JSON breaking a Pydantic limit
(meta_description's 160 characters), so the paid retry often failed again.
Content and both improver stages now retry under the same strict schema and
name each failing field with Pydantic's message, never the value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The current-user and brand-voice reads autobegin a transaction that stayed
open, idle in transaction on a pooled connection, through each 5-30 s model
call, so about 20 concurrent generations exhausted the 10+10 pool. Content
generation and both improver stages now commit it before every provider call.
The image path keeps its FOR UPDATE NOWAIT lock; that needs a schema change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing tested the prompt wording, so a changed instruction or default could
ship unseen and without a PROMPT_VERSION bump. Golden files now pin the
rendered system and user prompts of the four content types and both improver
stages, from fixed inputs; UPDATE_GOLDENS=1 regenerates them. They match the
current prompts: this commit changes no prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The LinkedIn hook asked for a "specific stat", ad copy for "concrete numbers
when honest" and the persuasive improver for "concrete numbers and proof", so
the model made up figures the brief never gave; the email example signed off
as "The Magna team". Those three now use only facts stated in the brief, the
sign-off ends in a [Your name] placeholder, and all four prompts move to v2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and Redis

Compose published Postgres (password app) and the backend on every
interface, and started an nginx stub and a Redis that nothing reads.
/health drops its never-probed `redis: "unknown"`: in-repo consumers only
check the status code, the frontend types `dependencies` as a string map,
and the top-level keys are unchanged. Stale .env Redis keys are ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The frontend can't tell that a backend runs AI_PROVIDER_MODE=mock, so
canned content can pass for real model output. A public route reports the
mode for a persistent banner. It needs no auth, so it returns only
provider_mode and app_version: no model ids, keys or other settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under the strict schema, a corrective retry that stops at the length limit
or is filtered raised AI_OUTPUT_TRUNCATED or AI_CONTENT_REJECTED: the request
failed after two or three billed calls and saved nothing, where it used to
degrade to stage 3. The provider's error now carries the billed reply, so
content and the improver degrade with attempt 1's output and count every call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The corrective-retry instruction no longer claims the reply was invalid
JSON, and the retry now runs under the strict schema with a note naming the
failing field, yet blog_post stayed at v1 while the other prompt modules
moved to v2. A row's prompt_version is the only record of which retry
wording produced it; the snapshots keep only the first attempt's prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The services comment still said the integration tests skip when
Postgres is unreachable, and gave a stale test count. With REQUIRE_DB=1
the job fails instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#154 landed on main as one squash commit (57b57d8) whose tree is this
branch's old base, so this merge keeps this branch's tree unchanged
(-s ours) and makes main its merge base. The PR diff is again only this
branch's own changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Eslam93
Eslam93 merged commit 3e1b333 into main Nov 7, 2026
2 checks passed
Eslam93 added a commit that referenced this pull request Nov 7, 2026
#155 landed on main as one squash commit (3e1b333) whose tree is this
branch's old base, so this merge keeps this branch's tree unchanged
(-s ours) and makes main its merge base. The PR diff is again only this
branch's own changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Eslam93
Eslam93 deleted the fix/backend-hardening branch November 7, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant