way to many fixes to mention

[heitorrosa]

No description provided.

[Copilot]

🟡 Changes recommended

Critical and moderate findings remain unresolved across database configuration, Prometheus history and memory handling, threaded database access, scraper startup, and stream reliability.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Broad maintenance update spanning stock APIs, Prometheus streaming and memory, authentication, logging, deployment, and tests.

Changes:

• Refactors stock caching, querying, compression, and quota handling.
• Adds Prometheus SSE replay, memory/vector tooling, compaction, sandbox, and MCP updates.
• Updates sessions, logging, scraper integration, dependencies, Docker configuration, and test coverage.

File summaries

File	Summary
TODO.md	Tracks scraper and metrics follow-ups.
tests/test_vector_utils.py	Adds vector utility coverage.
tests/test_vector_normalize.py	Tests vector normalization.
tests/test_thread_safety.py	Expands concurrency coverage. Nit (3 votes): boolean-only assertions do not verify distinct thread-local sessions.
tests/test_sync_cache.py	Tests synchronous cache behavior.
tests/test_streammessage_no_chatsession.py	Updates stream history assertions.
tests/test_stream_bus.py	Tests asynchronous stream buffering and replay.
tests/test_stocks_util.py	Tests stock utility behavior.
tests/test_stocks_presorted.py	Tests cache ordering.
tests/test_stocks_mcp_middleware.py	Tests stock MCP middleware.
tests/test_stocks_cache_zstd.py	Tests compressed cache handling.
tests/test_status_endpoint.py	Tests application status loading.
tests/test_sso_state_cookie.py	Tests SSO state cookies.
tests/test_sessions.py	Tests session lifecycle changes.
tests/test_sandbox.py	Tests sandbox behavior.
tests/test_sandbox_workspace.py	Tests workspace operations.
tests/test_sandbox_tools.py	Tests sandbox tools.
tests/test_sandbox_persistence.py	Tests sandbox persistence.
tests/test_sandbox_auth_gating.py	Tests sandbox authentication gating.
tests/test_roles.py	Tests permission roles.
tests/test_roles_coverage.py	Expands role coverage.
tests/test_relevance_score.py	Tests relevance scoring.
tests/test_query_snapshot.py	Tests cache snapshots.
tests/test_query_search_dates.py	Tests date-aware stock queries.
tests/test_prometheus_tools.py	Tests Prometheus tools.
tests/test_prometheus_resume.py	Tests stream resume behavior.
tests/test_prometheus_agent_context.py	Tests agent context construction.
tests/test_pagination.py	Tests pagination behavior.
tests/test_oauth_cookie_security.py	Tests OAuth cookie security.
tests/test_models.py	Updates model tests.
tests/test_memory_tools_wiring.py	Tests memory tool registration.
tests/test_memory_prompt_guidance.py	Tests memory prompt guidance.
tests/test_memory_manager.py	Tests memory management.
tests/test_memory_maintenance.py	Tests memory maintenance.
tests/test_memory_extraction.py	Tests memory extraction.
tests/test_memory_dedup.py	Tests memory deduplication.
tests/test_memory_async_loop.py	Tests asynchronous memory operations.
tests/test_mcp_pool.py	Tests MCP client pooling.
tests/test_matrix_cache.py	Tests embedding matrix caching.
tests/test_logout_cookie_domain.py	Tests logout cookie domains.
tests/test_logging_config_coverage.py	Tests logging configuration.
tests/test_controllers_coverage.py	Expands controller coverage.
tests/test_config.py	Tests configuration changes.
tests/test_compact.py	Tests Prometheus compaction.
tests/test_auth_util.py	Tests authentication utilities.
tests/test_api_key_quota_atomic.py	Tests atomic API-key quotas.
tests/test_agent_sandbox_integration.py	Tests agent-sandbox integration.
tests/conftest.py	Updates shared test fixtures.
run.py	Updates startup and status reporting.
requirements.txt	Updates dependencies.
pyproject.toml	Configures asynchronous test execution.
main/utils/models/loader.py	Adds embedding model loading.
main/utils/migrator.py	Updates migration execution.
main/utils/logging_config.py	Refactors Discord delivery and queued logging. Moderate (2 votes): failed webhook messages are suppressed before successful delivery; moderate (1 vote): queued exception data loses traceback details.
main/utils/connectivity.py	Uses thread-local HTTP sessions.
main/controller/user_controller.py	Updates session API responses. Moderate (2 votes): browser and operating-system metadata are omitted from serialization.
main/controller/stocksapi_controller.py	Adds cached stock endpoints.
main/controller/prometheus_controller.py	Adds streaming and workspace endpoints.
main/controller/authentication_controller.py	Updates authentication cookies. Moderate (1 vote): Domain=localhost cookies are rejected for IP-based loopback requests.
main/app/user/user.py	Integrates session validation.
main/app/stocks_api/util.py	Adds stock field and date utilities.
main/app/stocks_api/sync_cache.py	Adds synchronous cache wrapping.
main/app/stocks_api/key.py	Adds atomic API-key quota updates.
main/app/stocks_api/compress.py	Adds compact response encoding. Moderate (1 vote): nested fields use the unmapped key and are not compacted or URL-filtered.
main/app/stocks_api/cache.py	Adds synchronized cache snapshots.
main/app/scraper_b3/scraper.py	Updates scraper scoring integration. Critical (3 votes): the absolute xango import fails during normal startup.
main/app/prometheus/vector.py	Adds vector normalization and codecs.
main/app/prometheus/tools.py	Adds threaded memory tools. Critical (2 votes): a request-owned SQLAlchemy session is passed to a worker thread; moderate (1 vote): embedding inference blocks the event loop.
main/app/prometheus/stream_bus.py	Adds SSE stream buffering and replay. Moderate (2 votes): completed resumes can wait 30 seconds; moderate (1 vote): replay cursors become incorrect after buffer truncation; moderate (2 votes): finished channels remain retained indefinitely.
main/app/prometheus/sandbox.py	Adds sandbox lifecycle and workspace handling.
main/app/prometheus/memory.py	Adds cached vector search and memory extraction. Moderate (1 vote): invalidation can race with stale cache insertion; critical (1 vote): extraction lacks a separate USE_PROMETHEUS authorization check.
main/app/prometheus/mcp.py	Adds MCP client pooling.
main/app/prometheus/compact.py	Adds dynamic field extraction and compaction. Moderate (2 votes): failed field discovery can permanently retain a fallback regex; moderate (1 vote): extraction reads content instead of metadata.result.
main/app/prometheus/chat.py	Updates conversation history handling.
main/app/prometheus/agent.py	Integrates memory, tools, and streaming. Critical (2 votes): an ISO timestamp is passed where history handling requires a datetime after compaction.
main/app/authentication/util.py	Updates JWT and password utilities.
main/app/authentication/session.py	Adds user-agent session metadata.
main/app/authentication/constants.py	Centralizes authentication constants.
graphify-out/.graphify_labels.json	Updates generated graph labels.
Dockerfile	Updates the Python image and dependency installation.
docker-compose.yml	Updates service, database, and cache configuration. Critical (1 vote): bare environment forwarding leaves database URLs unset; moderate (1 vote): removing the cache volume makes generated cache files non-persistent and pollutes the checkout.
config.py	Updates environment-backed settings.
ci.ps1	Updates local CI checks.
AGENTS.md	Documents repository conventions and workflows.

Review details

Suppressed comments (10)

docker-compose.yml:66

• Removing the named cache volume while retaining the /app bind mount moves /app/cache into the source checkout. The generated feather files and lock file can pollute the checkout and disappear on a fresh deployment, forcing a full cache rebuild; keep a dedicated /app/cache volume or mount the cache to a persistent deployment path.

      - .:/app
      - workspaces:/data/workspaces

main/app/prometheus/compact.py:145

• The new snapshot extraction is fed the same loop-event dictionaries that agent.persistEvent() writes, where the tool payload is under metadata.result; the implementation reads a top-level content value instead. Therefore every real tool-result event contributes an empty string and episode summaries never include the key values this extractor is intended to capture. Read and unwrap the metadata result in the extraction loop.

def extractSnapshots(toolResults: list[dict]) -> list[str]:

main/app/prometheus/memory.py:46

• An invalidation can occur after loader() reads the old rows but before this cache write. The old matrix is then inserted after invalidateUser() has run and, because this entry has no expiry, remains stale indefinitely. Use a generation/versioned key or synchronize loading with invalidation.

    asyncio.run(matrixCache.set(cacheKey, (freshIds, freshMatrix), tags=("matrix", f"matrix-user:{uid}")))

main/app/prometheus/memory.py:464

• This method still accepts userRoles and the existing tests expect extraction to stop for a user without USE_PROMETHEUS, but this code only uses the roles to choose the extended-memory cap. Any internal caller can therefore spend the extraction Gemini quota and write memories for a non-Prometheus user. Keep a separate USE_PROMETHEUS check before calculating the cap.

            cap = (
                MEMORY_EXTRACT_PREMIUM_CAP
                if Roles.checkAccess(userRoles, Permission.PROMETHEUS_EXTENDED_MEMORIES)
                else MEMORY_EXTRACT_FREE_CAP
            )

main/app/prometheus/stream_bus.py:11

• cursor is an absolute replay position (the tests use it to count prior events), but this buffer drops entries from the front once it reaches 5,000 events. subscribe() then treats the truncated list as if it still started at zero, so long-stream resumes skip events or replay the wrong ones. Track the retained list's starting cursor and translate or reject cursors outside the retained window.

MAX_BUFFER_EVENTS = 5000  # ponytail: hard cap; older events dropped, clients reconcile from history

main/app/prometheus/tools.py:73

• The same caller-owned SQLAlchemy Session is passed into upsertMemory from asyncio.to_thread. Because dispatchToolCall supplies the stream's db, saving a memory crosses the session into another thread and can cause unsafe concurrent use or connection/thread errors; the worker must own its session or the database call must stay on the owning thread.

        result = await asyncio.to_thread(
            PrometheusMemory.upsertMemory,
            db,  # type: ignore[arg-type]

main/app/prometheus/tools.py:70

• Embedding inference runs synchronously on the event-loop thread before the database operation is offloaded. Loading/encoding with the sentence-transformer model can block all concurrent Prometheus requests; offload this computation as well.

        embedding = embed([value])[0]

main/app/stocks_api/compress.py:69

• Because f maps original JSON column names to abbreviated keys here, the later nested-field loop checks the wrong key: nests still uses the original name while out contains the abbreviation. Nested JSON fields such as NOTICIAS are therefore never compacted or URL-filtered. Use the mapped key when applying each nested spec.

    m, h, f = abbrs["meta"], abbrs["historical"], abbrs["fundamental"]
    out = {}

main/controller/authentication_controller.py:52

• This new cookie setup uses resolveCookieDomain() for every host. That helper returns localhost for 127.0.0.1, but a cookie with Domain=localhost does not domain-match the IP host and is rejected by browsers, so loopback login requests lose their token. Return a host-only cookie domain for localhost/IP requests.
  main/utils/logging_config.py:73
• The listener receives records through QueueHandler, whose standard prepare() clears exc_info before enqueueing. That makes the handler's traceback branch ineffective for all queued exception logs; preserve exception data in the queue handler or format it before this attachment.

• Files reviewed: 88/91 changed files
• Comments generated: 10
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.