[dev-app-source]

Summary

This PR adds a comprehensive design document for the market data backend subsystem (backend/app/market/) and implements 9 targeted fixes and enhancements to the existing implementation. The document serves as the authoritative specification for the unified data model, price cache, GBM simulator, Massive API client, and SSE streaming infrastructure.

Key Changes

Documentation:

• Added planning/MARKET_DATA_DESIGN.md (1748 lines): complete, drop-in reference design with code examples, architecture diagrams, wire formats, testing strategies, and edge-case handling for all market data components.

Code Fixes & Enhancements (marked Δ in design):

1. Fix Massive snapshot parsing (massive_client.py):

   • Read last_trade.sip_timestamp (nanoseconds) instead of non-existent timestamp attribute
   • Implement price fallback chain: last_trade.price → min.close → day.close → prev_day.close
   • Resolves production bug where every real snapshot raised AttributeError and was skipped

2. Daily change metrics (models.py, cache.py):

   • Add reference_price field to PriceUpdate
   • Add day_change and day_change_percent properties for watchlist display
   • Reference is session open (simulator) or previous close (Massive)

3. Ticker normalization (models.py):

   • Introduce normalize_ticker() function: canonical uppercase form with validation
   • Prevents "aapl" and "AAPL" from being tracked as separate tickers

4. Cache robustness (cache.py):

   • remove() now bumps version counter (fixes removed tickers staying on screen)
   • version property read under lock
   • Respect timestamp=0 in update logic

5. SSE stream improvements (stream.py):

   • Fresh APIRouter per create_stream_router() call (eliminates module-level router footgun)
   • Add heartbeat comments to keep idle connections alive through proxies
   • Empty cache sends {} instead of failing

6. Seedable simulator RNG (simulator.py):

   • Add seed= parameter to GBMSimulator and SimulatorDataSource
   • Use private numpy.Generator and random.Random for reproducible tests

7. Massive robustness (massive_client.py):

   • Set retries=0 on RESTClient (prevent burning free tier on 429 responses)
   • Copy ticker list before passing to worker thread (thread-safety)
   • Drop results for tickers removed mid-poll
   • Track and log consecutive failures

8. Configurable poll interval (factory.py):

   • Add optional MASSIVE_POLL_INTERVAL environment variable
   • Default 15s (free tier), configurable for paid tiers (2–5s)

9. Ticker tracking helper (tracking.py - new):

   • Add sync_tracked_tickers() function: unified rule for watchlist ∪ open positions
   • Enables held positions to keep live prices after being un-watched

Implementation Details

• All code blocks in the design document are complete, drop-in modules tested against the installed massive package
• Maintains backward compatibility: existing public API unchanged, only additions and fixes
• No new package dependencies required
• Comprehensive test strategy documented (§13) with examples for each module
• Edge cases and failure modes explicitly addressed (§14)
• Configuration reference and implementation checklist provided (§15–16)

Design Principles

The architecture enforces:

• Producers push, consumers pull: data sources write to cache on their own schedule; downstream code reads from cache
• Single active source per process: chosen at startup by factory based on MASSIVE_API_KEY
• PriceUpdate is the only type leaving the market layer: serialized with to_dict() for JSON/SSE
• Thread-safe immutable value objects: PriceUpdate is frozen; `

https://claude.ai/code/session_01WJ9YY6aYQUo32DNgC5GLJw

[claude]

This design contradicts the contract in planning/PLAN.md (Section 6, "Daily Change" and "SSE Streaming", included in the root CLAUDE.md). PLAN.md requires PriceUpdate to carry session_open and to_dict() to emit session_open and daily_change_percent. The SSE example shows those exact keys, and the frontend watchlist column reads daily_change_percent.

This doc instead uses reference_price, day_change and day_change_percent (here, and in the to_dict and SSE examples further down). Implementing it as written would send keys the PLAN-driven frontend doesn't read, so the watchlist daily-change column would break.

PLAN.md also lists an is_supported(ticker) -> bool method on the market data source (Ticker Validation) and a fixed 50-ticker simulator universe. Neither appears in this design.

Please rename the fields to the PLAN.md names, or update PLAN.md first so there is a single source of truth.

PLAN.md reference: https://github.com/dev-app-source/finally/blob/8818dc3f3d33b28900fef28f8905dc934e4b6715/planning/PLAN.md