Module storage Roadmap

Navigation: Home > Modules

Storage Module Roadmap

Current Status

Production-capable storage runtime exists for durable persistence, MVCC/WAL lifecycle behavior, backup/PITR flows, blob/tiering behavior, and storage audit/integrity surfaces. Source revalidation on 2026-09-03 confirms fail-closed guards are now active for missing storage dependency prerequisites in production (StorageEngine default encryption/key-provider/index-manager and null index-manager), and ggml_tensor_bridge.cpp now fail-closes in production when allocator, prefetch, or TT type-registration backends are unavailable while preserving explicit test-only fallbacks under THEMIS_UNIT_TEST.

In Progress

• [~] hardening failure-path behavior under sustained write/load and maintenance overlap (Target: Q3 2026)
• [~] improving diagnostics consistency across storage, replay, and recovery stages (Target: Q3 2026)
• [~] stabilizing benchmark-backed release guardrails for storage hot paths (Target: Q3 2026)
• remove implicit SecuritySignatureManager(nullptr) memory-store fallback from production paths; null-backend construction now fails closed unless the caller explicitly enables the test-only fallback option (Target: Q3 2026)
• enforce StorageEngine production fail-closed dependency prerequisites: default/no-op encryption, key-provider, index-manager, and null index-manager construction are rejected in production mode (Target: Q4 2026)
• BLOCK 3: Storage Module Integration with AccessCoordinator (Target: Q4 2026) ✅ COMPLETE
  • Added PromotionListener support to TieredStorageManager
  • Added setPromotionListener() method in header and implementation
  • Emit onStorageAccess() signals when detecting hot tiers
    • emitPromotionEvent() helper method
    • Hot pattern detection in get() for WARM/COLD tiers
    • Hot pattern detection in runMigrationCycle()
  • Implement predictive promotion callbacks with access window tracking

Planned Features

Short-term (3-6 months)

• integrate with AccessCoordinator for unified cache-storage tier management (Target: Q4 2026)
  • Add PromotionListener callbacks to emit hot-access signals to coordinator
  • Implement coordinator-guided promotion paths (cold→warm→L3)
  • Extend TieredStorageManager with coordinator hooks
  • See: src/access_model/ROADMAP.md Phase 4
• make BackupManager::decompressPath() fail closed or perform real decompression when zstd/lz4 are unavailable; unsupported restore builds now reject the operation instead of copying raw bytes verbatim (Target: Q4 2026)
• make BackupManager::decryptFile() fail closed or perform real decryption when OpenSSL is unavailable; unsupported restore builds now reject the operation instead of copying ciphertext verbatim (Target: Q4 2026)
• [~] wire the remaining ggml_tensor_bridge.cpp production injection seams (GgmlAllocFn, PrefetchFn, TypeRegistrationFn) at server initialization (Target: Q4 2026)
  • map(ctx, ...) now forwards the live ggml_context* into GgmlAllocFn and still uses that context for real ggml allocation/copy when THEMIS_HAS_GGML is enabled, reducing reliance on the allocator stub path
  • EmbeddedLLM startup now calls registerGgmlTypeTT() once under THEMIS_ENABLE_GGML_BRIDGE, closing the uninitialized TT-type registration path for ggml-backed inference startup
  • fail-closed production guards added: map() invalidates when no allocator path exists, prefetch() throws when no prefetch backend is available, and registerGgmlTypeTT() throws when no registration backend exists
  • startup wiring of concrete allocator/prefetch/type-registration providers in server bootstrap remains open
• tighten deterministic behavior under heavy WAL replay and compaction pressure (Target: Q4 2026)
  • Addressed by ST-SOAK-02 (WALReplayStability) and ST-STRESS-02 (ConcurrentCompactionStress)
• expand stress coverage for blob/tiering and PITR edge scenarios (Target: Q4 2026)
  • Addressed by ST-STRESS-03 (BlobTieringEdgeCases) in tests/storage/test_storage_highcardinality_stress.cpp
• improve operator-facing diagnostics for recovery and maintenance incidents (Target: Q4 2026)
  • Addressed by docs/operability/RUNBOOK_STORAGE_ENGINE.md (5 scenarios with structured log patterns)
• finish remote cloud backup transport wiring in backup_manager.cpp for S3/GCS/Azure so cloud restore now reconstructs backups from a manifest plus payload blobs instead of depending on local-mirror-only behavior (Target: Q4 2026)

Mid-term (6-12 months)

• [~] re-baseline p95/p99 envelopes for write/replay/recovery-sensitive paths (Target: Q1 2027)
• [~] broaden benchmark depth for mount-latency and storage allocator edge paths (Target: Q1 2027)
• [~] harden long-run reliability under sustained mixed read/write pressure (Target: Q1 2027)

Distributed Maturity Phase 3 — Track 2 Items (Q3–Q4 2026)

These items are part of the next-phase Track 2: Distributed Systems Maturity — 3.4 Storage plan (see ROADMAP.md §Track 2). Hard gate per item: deterministic under-load benchmark + release_critical CI green.

• Tiered storage (hot/warm/cold) with automatic data migration: implement age- and access-frequency-based tier migration; data that has not been accessed within a configurable window automatically demoted from hot (NVMe) → warm (HDD) → cold (object storage) (Target: Q4 2026)
  • Inputs: tier policy config (storage.tier.hot_max_age_s, storage.tier.warm_max_age_s, storage.tier.cold_backend); access-frequency tracker
  • Acceptance: migration runs without blocking foreground I/O; cold-to-hot promotion completes within 100 ms for objects ≤ 1 MB; storage_tier_migration_total counter wired; release_critical green
• Cloud-native S3/GCS/Azure backend hardening: harden the existing cloud backend adapter for sustained production use: retry with exponential backoff, integrity checksums (MD5/CRC32C), multipart upload for objects > 100 MB, SDK version pinning (Target: Q4 2026)
  • Acceptance: 72-hour soak at 1 000 object writes/s with zero data-integrity errors; multipart upload tested for 1 GB object; SDK versions pinned in vcpkg manifest

Implementation Phases

Phase 1: Design / API Contract

• Freeze storage module API contract — WAL durability, MVCC snapshot isolation, crash-recovery, backup/PITR, tiering transparency, error taxonomy (include/storage/storage_api_contract.h) (Target: Q3 2026)
• Define explicit StorageErrorCode taxonomy (WAL_WRITE_FAILED, CHECKPOINT_FAILED, RECOVERY_INCOMPLETE, PITR_INVALID_TIMESTAMP, COMPACTION_ABORTED, STORAGE_EXHAUSTED, …) (Target: Q3 2026)

Phase 2: Core Implementation

• [~] complete hardening for WAL/MVCC and backup/PITR internals (Target: Q4 2026)
  • SecuritySignatureManager now rejects null-backend production use instead of silently downgrading integrity persistence to an in-memory map
  • remote cloud backup archive transport now uses provider blob backends plus a manifest-driven restore contract for S3/GCS/Azure
• [~] align tiered/blob/redundancy behavior to bounded runtime contracts (Target: Q4 2026)

Phase 3: Error Handling and Edge Cases

• standardize fail-safe behavior for replay faults, storage pressure, and recovery errors (Target: Q4 2026) ✅ COMPLETE (2026-08-03)
  • Created storage_error_diagnostics.h/cpp with unified error classification
  • Implemented storage_recovery_fault_handler.h/cpp for recovery fault handling
  • Implemented storage_pressure_manager.h/cpp for capacity management
  • Created comprehensive Phase 3 focused tests (24 test cases)
• unify diagnostics across persistence, maintenance, and recovery incident classes (Target: Q4 2026) ✅ COMPLETE (2026-08-03)
  • StorageErrorContext struct for detailed error information
  • emitDiagnosticEvent() for structured event emission
  • Incident type classification (RECOVERY_FAULT, STORAGE_PRESSURE, etc.)
  • Error severity levels (INFO, LOW, MEDIUM, HIGH, CRITICAL)

Phase 4: Tests

• Contract-hardening focused tests STR-01..STR-16 covering WAL durability, MVCC isolation, crash-recovery, and PITR invariants (tests/storage/test_storage_contract_hardening_focused.cpp) (Target: Q4 2026)
• Expand focused regressions for replay, PITR, and tiered/blob edge scenarios (Target: Q4 2026)
• Extend deterministic stress fixtures for mixed write + maintenance workloads (Target: Q4 2026)

Phase 5: Performance and Hardening

• Lock benchmark-backed release gates for storage hot paths: SGRG-01..SGRG-06 in benchmarks/storage/bench_storage_release_gates.cpp (WAL throughput ≥100k ops/s, MVCC read p99≤100µs, MVCC write p99≤500µs, checkpoint p99≤10ms, tiering decision p99≤50µs, compaction check p99≤100µs) (Target: Q4 2026)
• Validate p95/p99 and throughput behavior against release baselines (Target: Q4 2026)

Phase 6: Documentation and Acceptance

• core storage module docs aligned to source-verifiable behavior
• roadmap/future planning separated from historical changelog entries
• Contract header frozen: include/storage/storage_api_contract.h
• Contract-hardening tests delivered: tests/storage/test_storage_contract_hardening_focused.cpp (STR-01..STR-16)
• Release-gate benchmarks delivered: benchmarks/storage/bench_storage_release_gates.cpp (SGRG-01..SGRG-06)

Production Readiness Checklist

• core storage surfaces documented and source-verified
• module-level security and failure behavior documented
• benchmark mapping documented in performance expectations
• Contract header frozen: include/storage/storage_api_contract.h (Phase 1)
• Contract-hardening tests: tests/storage/test_storage_contract_hardening_focused.cpp (Phase 4, STR-01..STR-16)
• Release-gate benchmarks: benchmarks/storage/bench_storage_release_gates.cpp (Phase 5, SGRG-01..SGRG-06)
• Benchmark CMakeLists registered: benchmarks/storage/CMakeLists.txt
• Error diagnostics system: include/storage/storage_error_diagnostics.h (Phase 3)
• Recovery fault handler: include/storage/storage_recovery_fault_handler.h (Phase 3)
• Storage pressure manager: include/storage/storage_pressure_manager.h (Phase 3)
• Phase 3 focused tests: tests/storage/test_storage_phase3_error_handling_focused.cpp (24 test cases)
• [~] remaining hardening tasks closed for durability/recovery edge paths
• release benchmark stabilization complete
  • Wave D dedicated gates ST-BM-01..04 delivered in benchmarks/storage/bench_storage_dedicated_gates.cpp

Known Issues and Limitations

• runtime behavior depends on storage configuration, backend profile, and workload shape.
• selected replay/recovery/tiering edge scenarios need continued hardening.
• benchmark depth should continue expanding for advanced storage workloads.
• provider-native remote transport now depends on the linked blob backend plus credentials/runtime environment; multipart/retry soak hardening remains follow-up work.

Breaking Changes

No breaking storage contract planned. Any contract-breaking change requires migration notes and changelog entry before merge.

Program Execution Model — Wave Context

This module is a contributing module in the program-level Wave A → B → C → D execution model. It does not own a primary wave deliverable but must remain release_critical-green throughout all waves and must deliver Wave D operability improvements in Q1 2027. See ../../ROADMAP.md for the full wave model and exit criteria.

Wave D Contribution for storage

• Deliver or validate distributed tracing, high-cardinality stress coverage, exporter reliability, and operator remediation hints as applicable to this module (Target: Q1 2027)
  • Delivered: tests/storage/test_storage_highcardinality_stress.cpp (ST-STRESS-01..03: 100k-key write, concurrent compaction, blob tiering edge cases)
  • Delivered: benchmarks/storage/bench_storage_dedicated_gates.cpp (ST-BM-01..04: write p95, read p95, WAL-replay throughput, compaction trigger p95)
• Contribute to or validate long-duration soak test coverage for this module's primary paths (Target: Q1 2027)
  • Delivered: tests/integration/test_storage_engine_soak.cpp (ST-SOAK-01..03: write throughput ≥10k w/s, WAL replay stability, compaction reliability)
• Ensure runbook coverage for operator-critical scenarios in this module (Target: Q1 2027)
  • Delivered: docs/operability/RUNBOOK_STORAGE_ENGINE.md (5 scenarios: WAL corruption, compaction stall, blob tier mount failure, PITR restore failure, write stall)

Cross-Wave Requirements

• release_critical CI must remain green on develop throughout all waves (Target: ongoing)
• p95/p99 benchmarks must be refreshed on representative hardware before Wave D sign-off (Target: Q1 2027)
• No behavioral regression may be introduced into modules in Wave A/B/C scope from changes in this module.

Program-Level Success Criteria (contribution)

• [~] This module's distributed/acceleration paths fail closed (Target: Q1 2027)
• Benchmark-backed p95/p99 baselines exist on representative hardware (Target: Q1 2027)
  • Delivered: benchmarks/storage/bench_storage_dedicated_gates.cpp ST-BM-01..04
• Operator-critical paths have diagnostics, alerts, and runbooks (Target: Q1 2027)
  • Delivered: docs/operability/RUNBOOK_STORAGE_ENGINE.md

---

ThemisDB 1.9.0-beta · Home · Module-Index · GitHub · Issues