Roadmap Phase 3: native structured outputs for all LLM call sites

[Ndevu12]

Summary

Implements the structured-outputs item from Phase 3 of the improvement roadmap. Every LLM call site now attaches its pydantic schema to the agent (output_type) so pydantic-ai enforces the schema natively — validating responses and retrying with validation feedback on violations — instead of asking for JSON in prose and repairing whatever comes back. Callers receive typed objects or fall back to the existing heuristics.

Changes

• New src/models/structured.py — create_structured_agent, run_structured, try_run_structured, plus slim structured-mode role prompts: with the schema enforced by the framework, the "respond with ONLY JSON …" scaffolding in the legacy prompts is unnecessary noise (the legacy prompts remain for the legacy path).
• Converted call sites — per-paper extraction, collective synthesis, gap analysis, and LLM query expansion (which previously streamed prose and ran manual JSON extraction).
• Hallucination guard — extraction pins identity fields (paper ID, title) from retrieval metadata, so model output can never relabel a paper; evidence is backfilled from grounded full-text passages when the model omits it.
• Same safety net — the extraction circuit breaker and every heuristic fallback behave identically on the structured path.
• Config-gated — llm.structured_outputs (default true). Disabling routes through the legacy prose-JSON repair path unchanged, kept for backends that cannot honor response schemas. Once the structured path is proven against local Ollama models, the roadmap's follow-up is retiring the ~2,900-line repair stack in src/utils/.

Documentation

.env.example, the environment-variables reference, and the heuristic-vs-LLM page document RA_LLM__STRUCTURED_OUTPUTS.

Verification

• pipenv run pytest tests/ — 426 passed (9 new tests: run_structured against pydantic-ai's TestModel, failure handling against a FunctionModel that raises, identity pinning, circuit-breaker behavior on structured failures, structured synthesis/gap/expansion paths — all offline), 1 skipped
• Legacy-path tests now pin structured_outputs=false explicitly, so both paths stay covered
• ruff check src tests setups — clean; scripts/check_docs_policy.py — passes
• Real-model behavior (Ollama JSON-schema support quality on 3B/8B models) could not be exercised in this environment — recommend one local run with RA_SYNTHESIS__LLM_ENABLED=true before flipping any defaults that depend on it