[SagiEv]

Summary

Fixes 500 Internal Server Error when generating networking messages in the Quick Message Sender (POST /api/messages/generate) across both Gemini and Groq providers.

Root Causes Identified

1. AI Service Startup Blocking (connect ECONNREFUSED 127.0.0.1:8001):
   • embed_router.py loaded SentenceTransformer('all-MiniLM-L6-v2') synchronously at module import time, delaying Uvicorn port binding on 127.0.0.1:8001 by ~4 minutes on service restart while POST /api/settings/test-ai succeeded (since token validation calls provider APIs directly from Node.js).
2. Gemini Runtime 503 UNAVAILABLE & Missing Fallback on .invoke():
   • Primary Gemini models (gemini-3.8-flash, gemini-3.7-flash) frequently return 503 UNAVAILABLE under high demand or hold connections open without a timeout. Previously, LLMRouter only caught errors during lazy model instantiation rather than runtime .invoke() / .with_structured_output() execution, and message_creator/service.py bypassed fallback routing.
3. pdf-parse v2 API Mismatch & Prompt Vector Pollution:
   • backend/package.json uses pdf-parse@^2.4.5 (which exports { PDFParse } instead of a v1 function).
   • skillsRepository.findAll and experienceRepository.findAllProjects returned 384-dim embedding float arrays (SELECT *), causing the 1000/2000-character truncation in message_creator/service.py to be consumed by float vectors instead of user skills/projects.

Changes Made

• backend/ai_service/embed_router.py: Replaced import-time SentenceTransformer loading with _LazySentenceTransformer so uvicorn binds to port 8001 immediately on startup.
• backend/ai_service/router/llm_router.py & backend/ai_service/providers/gemini_provider.py:
  • Added _FallbackChatModel wrapper in LLMRouter.get_model() to automatically step through MODEL_REGISTRY[provider]["fallbacks"] on runtime 503, 429, 404, model-unavailable, or timeout errors for both .invoke() and .with_structured_output().
  • Added timeout=12 to ChatGoogleGenerativeAI and capped max_retries=1 on models that have fallbacks behind them so overloaded primary models fail fast.
• backend/ai_service/router/rate_limiter.py: Added socket_connect_timeout=1 and socket_timeout=1 and broadened Redis fail-open handling to redis.RedisError.
• backend/services/messages.service.js & backend/services/tailor.service.js:
  • Added support for pdf-parse v2 (new PDFParse({ data }).getText()) alongside v1 function exports.
  • Stripped embedding vectors from skills_pool and projects_pool before sending context to the AI service.
  • Improved mapAiError for ECONNREFUSED / ECONNRESET.
• frontend/src/pages/DashboardPage.jsx: Updated Quick Message Sender error banner to read err.response?.data?.message returned by the Node backend error middleware.

Test Plan

• pytest backend/ai_service -v (73 passed, including new test_runtime_fallback_on_503_unavailable)
• npm --prefix backend run test:unit -- tests/controllers/messages.controller.test.js (3 passed)
• Verified live end-to-end networking message generation for both Groq and Gemini (confirming automatic fallback from gemini-3.8-flash -> gemini-3.7-flash -> gemini-3.5-flash-lite on 503 UNAVAILABLE)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
job-pilot	Ready	Preview	Nov 3, 2026 4:13pm UTC