Repository navigation
Implement first database/storage layer #38
Description
Goal
Implement the first DuckDB-based storage layer for ARGUS based on the storage research.
Why
ARGUS needs persistent local storage to move beyond temporary API requests and in-memory analytics. A first storage layer makes it possible to save normalized historical market data, read it back for analytics and prepare the project for later storage-first workflows.
This ticket focuses only on the database/storage foundation. Workflow orchestration, API fallback and service refactoring are handled in separate sub-issues.
Scope
-
Add DuckDB as the first storage dependency
-
Create the first minimal DuckDB schema based on the storage research:
data_sourcesinstrumentsprice_bars
-
Store normalized internal model data, not raw API response structures
-
Add a small storage/repository module for:
- initializing the database
- inserting/upserting data sources
- inserting/upserting instruments
- saving price bars
- reading price bars by instrument, source and date range
-
Keep storage logic separate from API clients, UI code and metric functions
-
Add tests for schema creation, saving and reading historical data
-
Document basic setup and usage
Out of scope
- Storage-first/read-through workflow
- API fallback when data is missing
- Refactoring
TimeSeriesServiceintoMarketDataService - Refactoring existing services
- GUI integration
- Analytics/reporting integration
- Cache strategy
Acceptance criteria
- DuckDB is added as the selected first storage technology
- A minimal DuckDB schema exists for
data_sources,instrumentsandprice_bars - The schema follows the storage research in
docs/research-databases-and-storage.md - Historical FX or market data can be saved using the internal data model
- Stored price bars can be read back by instrument, source and date range
- FX rates are stored as
closevalues inprice_bars - Storage code does not depend directly on raw API response structures
- Storage logic is separated from API clients, UI code and metric functions
- Basic tests cover database initialization, saving data and reading data
- README or docs explain how to use the storage layer
Note
Priority: Should
Activity
feat(#38): create db foundation
feat(#38): add initt_db func
feat(#38): add id sequence
feat(#38): insert logic for source
feat(#38): insert logic - instrument
feat(#38): insert logic - price bar
feat(#38): add id search for bar
feat(#38): improve param select
refactor(#38): upsert instead insert
refactor(#38): get_create instead
style(#38): fix func names
feat(#38): add read func
feat(#38): try for read func
test(#38): test for init
test(#38): add more storage tests
test(#38): add read tests
docs(#38): doc for storage
docs(#38): update roadmap
docs(#38): add docstrings