Module ai Roadmap

Navigation: Home > Modules

AI Module Roadmap

Current Status

Production runtime exists for prompt validation, endpoint invocation, JSON mapping, and structured fail-closed error handling.

Wave Summary (2026-09-16):

Wave	Scope	Status	Evidence
A	Runtime reliability: validation, retry, fail-closed error handling, output-field validation	✅ COMPLETE	WAVE_A_CLOSURE_EVIDENCE_BUNDLE.md
B	ML enhancements: Self-RAG (B1), RotatE KGC (B2), Multi-Task LoRA (B3)	✅ SIGNED OFF 2026-09-09 (#6286)	WAVE_B_CLOSURE_EVIDENCE_BUNDLE.md
C	AI safety: CAI safety module (C1), federated learning (C2), graph phase gate (C3)	✅ COMPLETE	WAVE_C_CLOSURE_EVIDENCE_BUNDLE.md
D	Operability: runbook, observability, soak tests, distributed tracing, hardware baselines	🟡 IN PROGRESS — D1✅ D3~	WAVE_D_ROADMAP.md

Module Documentation Enhancements (2026-07-19)

• Added comprehensive Doxygen documentation to all implementation functions
• Documented AIPluginGenerator::validatePrompt() with detailed validation rules
• Documented AIPluginGenerator::generatePlugin() with complete execution pipeline
• Documented all CAI ethics integration functions with semantics and contracts
• Added inline documentation for error handling, thread-safety, and retry policies
• Lines added: ai_plugin_generator.cpp +133, cai_ethics_integration.cpp +130
• Total module documentation lines: ~1,473 (up from 1,210)

In Progress

• Validation hardening for non-description prompt fields (Target: Q3 2026)
• Endpoint safety hardening (allow-list, response-size limits) (Target: Q3 2026)
• Performance gate consolidation for AI generation proxy benchmarks (Target: Q3 2026)

Planned Features

Short-term (3-6 months)

• Enforce schema-level validation for all generated payload fields (Target: Q4 2026)
• Introduce deterministic retry/backoff policy for transient endpoint failures (Target: Q4 2026)
• Add explicit redaction policy for diagnostic output fields (Target: Q4 2026)

Mid-term (6-12 months)

• Integrate optional sandbox verification gate for generated code artifacts (artifact materialization + optional callback verification enforced in generator path) (Target: Q1 2027)
• Add dedicated benchmark target for AI plugin generation path (Target: Q1 2027)
• Expand observability counters for error classes and endpoint quality signals (Target: Q1 2027)

Long-term (Q3 2027+)

• Wave C C1: Constitutional AI (CAI) safety module with 21 built-in principles, critic-revision loop, EthicsEvaluator integration, and CAI-01..15 + CAI-BENCH-01 coverage — include/ai/cai_ethics_integration.h, tests/test_cai_safety_module.cpp
• Wave C C2: Federated learning coordinator with secure aggregation, Byzantine-robust averaging, DP tuning, and FEDERATED-01..15 + FEDERATED-BENCH-01 coverage — tests/test_federated_privacy_training.cpp
• Wave C C3: Graph phase gate orchestrator — DAG-based ML pipeline phase tracking with gate criteria, gap reporting, acyclic registration enforcement, and GRAPH_PHASE_GATE-01..15 + GRAPH_PHASE_GATE-BENCH-01 coverage — include/graph/graph_phase_gate_orchestrator.h, tests/graph/test_graph_phase_gate_orchestration.cpp (issue #6287)
• Human safety benchmark program for C1 (500 samples, 3 annotators) and convergence benchmark for C2 (10-node setup) — tests/test_cai_safety_module.cpp (CAI-BENCH-01), tests/test_federated_privacy_training.cpp (FEDERATED-BENCH-01)
• Integrate optional sandbox verification gate for generated code artifacts (Target: Q1 2027)
• Add dedicated benchmark target for AI plugin generation path (Target: Q1 2027)
• Expand observability counters for error classes and endpoint quality signals (Target: Q1 2027)
• Wave B B1: Self-RAG design/implementation/benchmark package (Target: Q1–Q2 2027) — core impl + IEE integration + ALCE acceptance-gate coverage done
• Wave B B2: RotatE knowledge-graph completion integration package (Target: Q1–Q2 2027) — core impl + KGC-01..15 tests + TransE-baseline acceptance coverage done
• Wave B B3: Multi-task LoRA fine-tuning package (Target: Q1–Q2 2027) — core impl + ablation/benchmark + acceptance-gate coverage done

Implementation Phases

Phase 1: Design / API Contract

• Stable API for prompt/config/result types in public header
• Validation-first behavior contract defined and implemented

Phase 2: Core Implementation

• Endpoint invocation path implemented with configurable transport
• JSON response mapping to GeneratedPlugin implemented

Phase 3: Error Handling and Edge Cases

• Non-2xx, transport, and parse failures normalized to structured errors
• Extended validation for capability/dependency fields (Target: Q3 2026)

Phase 4: Tests

• Focused unit coverage for constructor, validation, and endpoint/error paths
• Integration suite with deterministic endpoint fixtures (Target: Q3 2026)

Phase 5: Performance and Hardening

• Add module-specific benchmark instead of proxy-only tracking (Target: Q1 2027)
• Enforce endpoint allow-list and payload size bounds (Target: Q4 2026)

Phase 6: Documentation and Acceptance

• Core module docs aligned with source-verifiable behavior
• Completed work tracked in changelog; roadmap remains forward-looking

Production Readiness Checklist

• Validation-first execution path documented and verified
• Structured error handling for endpoint and parse failures verified
• Proxy benchmark mapping documented in performance expectations
• Dedicated benchmark target registered
• Hardening follow-ups closed for endpoint safety controls

Known Issues and Limitations

• Wave C C1/C2 production-runtime integration now covers AIPluginGenerator and LLMAQLHandler (executeInfer, executeInferStreaming, executeRAG, executeChat) via opt-in safety-gate and telemetry hooks.
• Schema-level validation for all LLM output fields is enforced: code fields ≤ 1 MiB, security_report ≤ 64 KiB, version ≤ 64 chars (defaults to 0.1.0), manifest.description truncated at 8192 chars, oversized build_dependencies entries silently dropped.

Wave C Timeline and Dependencies

• Start: Q3 2027 (early July)
• Target: End Q4 2027 (mid-December)
• Estimated Effort: 16–24 weeks total (depending on federated security requirements)

Dependencies

• Wave A + Wave B stability checks tracked in focused regression suites and release-gate docs (CTEST.md, Wave A #5038, Wave B #5039)
• Constitutional AI principles formalized in ethics framework (src/ai/cai_ethics_integration.cpp, tests/test_cai_safety_module.cpp)
• Multi-node federated benchmark infra/security review tracking established (FEDERATED-BENCH-01 coverage + issue traceability #5040/#5039)
• Graph phase gate orchestrator implemented and all exit criteria confirmed (GRAPH_PHASE_GATE-01..15 + GRAPH_PHASE_GATE-BENCH-01 — issue #6287)

Wave C Exit Criteria ✅ COMPLETE

Closure Date: 2026-09-16 | Evidence: src/ai/WAVE_C_CLOSURE_EVIDENCE_BUNDLE.md

• C1 CAI Safety: CAI-01..15 + CAI-BENCH-01 all passing (tests/test_cai_safety_module.cpp)
• C1 CAI Safety: safety alignment ≥ 0.80, latency ≤ 2.0 s, FPR ≤ 10%
• C2 Federated Learning: FEDERATED-01..15 + FEDERATED-BENCH-01 all passing (tests/test_federated_privacy_training.cpp)
• C2 Federated Learning: convergence ≥ 95% of centralized baseline, overhead ≤ 2.0 s, DP budget configurable
• C3 Graph Phase Gate: GRAPH_PHASE_GATE-01..15 + BENCH-01 all passing (tests/graph/test_graph_phase_gate_orchestration.cpp)
• C3 Graph Phase Gate: BENCH-01 ≤ 500 ms wall-clock
• Production-runtime hook integration: C1 safety gate + C2 telemetry in LLMAQLHandler paths (executeInfer, executeInferStreaming, executeRAG, executeChat)

References

• src/ai/FUTURE_ENHANCEMENTS.md#wave-c--strategic-ml-enhancements-q3-2027
• docs/research/ml_enhancements_bibliography.md
• #5040 (Wave C Issue)
• #5038 (Wave A Issue)
• #5039 (Wave B Issue)

Wave C Research Publication Opportunity

• Joint paper: ThemisDB Integration of Research-Backed ML Features
• Target venue window: MLSys 2028 / FAccT 2028
• Dedicated benchmark coverage exists via benchmarks/bench_ai_plugin_generator.cpp / benchmarks/ai/bench_ai_plugin_generator.cpp.
• Advanced field-level prompt validation remains incomplete.
• Sandbox artifact materialization and optional callback verification are enforced when enable_sandbox_gate is enabled; external sandbox engines remain deployment-specific.
• Wave B ML enhancement implementation and acceptance-gate coverage are complete; production promotion remains gated on Wave A deployment completion and latency prerequisites.

Wave A — Runtime Reliability ✅ COMPLETE

Closure Date: 2026-09-16 | Evidence: src/ai/WAVE_A_CLOSURE_EVIDENCE_BUNDLE.md

A1: Prompt Validation Hardening

• Description length enforced (max 8 192 chars)
• Token-list size limits (required_capabilities, dependencies)
• Duplicate token detection and rejection
• ASCII control-character stripping on all string fields
• Format validation for capability/dependency tokens

A2: Endpoint Safety Hardening

• Configurable endpoint allow-list enforced before outbound calls
• Request payload ceiling (256 KiB) enforced fail-closed
• Response payload ceiling (8 MiB) enforced before parse
• Non-2xx HTTP responses normalized to structured error
• Transport-level failures normalized to structured error

A3: Retry / Backoff Policy

• 3-attempt retry for transient endpoint failures
• Exponential backoff: 100 ms / 200 ms / 400 ms
• Non-retryable failures fail immediately
• Retry attempts logged at WARN level

A4: Output-Field Validation

• Generated code ≤ 1 MiB per field
• security_report ≤ 64 KiB
• version ≤ 64 chars (default 0.1.0)
• manifest.description truncated at 8 192 chars
• Oversized build_dependencies entries dropped

A5: Fail-Closed Error Handling

• All error branches return structured Error result
• Stats struct: 7 observable counters
• Log redaction: sensitive fields bounded at 120 chars
• Thread-safety contract documented (not thread-safe for concurrent generatePlugin)

Wave A Exit Criteria

• Validation-first execution path verified
• Endpoint safety controls verified
• Retry policy documented and source-verified
• Output-field validation complete
• All HIGH-severity MODULE_GAPS.md findings resolved (2026-07-19)
• Focused test coverage: tests/ai/test_ai_plugin_generator.cpp, tests/ai/test_ai_decision_auditor.cpp
• Benchmark target registered: benchmarks/ai/bench_ai_plugin_generator.cpp

---

Wave B (Q1–Q2 2027) Tracking

B1: Self-RAG (Self-Retrieving, Auto-Critique)

• Retrieval controller (binary classify: retrieve now?)
• Critic model (Relevant/Partial/Irrelevant)
• Iterative refinement loop (max 3 rounds)
• Unit tests SELF_RAG-01..12
• InferenceEngineEnhanced callback integration
• ALCE benchmark vs vanilla RAG

B2: Knowledge Graph Completion (RotatE)

• RotatE embedding model (relation-as-rotation)
• Triple loss with negative sampling
• Link-prediction head
• Unit tests KGC-01..15
• TransE baseline benchmark
• KnowledgeGraphReasoner integration

B3: Multi-Task LoRA Fine-Tuning

• Shared LoRA base + task-specific projections
• Domain-gating mechanism
• Joint loss with configurable task weighting
• Unit tests MTL-01..10
• Shared-vs-single-task baseline ablation
• 3-task benchmark evaluation

Acceptance Gates

• Hallucination rate reduction ≥ 20% vs standard RAG
• Self-RAG latency increase ≤ 1.5× vs baseline
• Precision@K retrieval ≥ 0.85 on golden-doc tests
• RotatE MRR ≥ 0.35 and Hits@10 ≥ 0.55 on deterministic acceptance fixture
• RotatE inference latency ≤ 50 ms for top-20 predictions
• Multi-task LoRA average task performance ≥ +8% vs single-task
• Multi-task LoRA training time increase ≤ 15%

Wave B Exit Criteria ✅ SIGNED OFF 2026-09-09

Sign-off: Issue #6286 ✅ SIGNED OFF 2026-09-09 | Evidence: src/ai/WAVE_B_CLOSURE_EVIDENCE_BUNDLE.md

• B1 Self-RAG: ALCE-01..05 acceptance tests passing (tests/rag/test_self_rag_alce.cpp)
• B2 RotatE KGC: KGC-01..15 unit tests passing (tests/test_rotate_completion.cpp)
• B3 Multi-Task LoRA: MTL acceptance tests passing (tests/training/test_multitask_lora_acceptance_gates.cpp)
• All seven acceptance-gate criteria met (hallucination, latency, precision, MRR, Hits@10, LoRA perf, LoRA overhead)
• CI evidence archival complete on develop
• [~] Production promotion: gated on Wave A hardware baselines (pending Q4 2026)

Dependencies

• [~] Wave A deployment complete (Speculative Decoding, DPR, Fairness) — in progress Q4 2026
• LLM inference P95 latency < 200 ms
• KnowledgeGraphReasoner stable + benchmark suite passing

References

• Research bibliography: ../../docs/research/ml_enhancements_bibliography.md
• Future enhancements detail: FUTURE_ENHANCEMENTS.md
• Issue scope: https://github.com/makr-code/ThemisDB/issues/5039

---

Wave D — Operability Hardening 🟡 IN PROGRESS Q1 2027

Detail: src/ai/WAVE_D_ROADMAP.md | Dependency: Wave A–C exit criteria confirmed

D1: AI Generation Operator Runbook

• Draft runbook: incident taxonomy (validation/endpoint/transport/parse/sandbox failures) (Target: 2026-11)
• Decision tree for endpoint-timeout + retry-storm scenarios (Target: 2026-11)
• Stats-counter operational thresholds and alert rules (Target: 2026-12)
• CAI safety gate latency incident section (Target: 2026-12)
• Output: docs/operability/RUNBOOK_AI_GENERATION.md

D2: Observability Expansion

• Extend AIPluginGenerator::Stats with per-error-class counters (Target: 2026-11)
• CAI safety gate per-call latency histogram (p50/p95/p99) (Target: 2026-11)
• Federated aggregation round-latency counter (Target: 2026-12)
• OpenTelemetry span propagation in generatePlugin (Target: Q1 2027)
• Span propagation in LLMAQLHandler CAI/federated gate paths (Target: Q1 2027)

D3: Soak Tests (Sustained Load)

• tests/ai/test_ai_generation_soak_60min.cpp — 60-min endpoint-stress + retry-budget-exhaustion (Target: 2026-12)
• CAI 10-min sustained evaluation test — 500 evaluations/min (Target: 2026-12)
• Federated 30-min round-stability test (Target: Q1 2027)
• Execute full soak suite on representative hardware (Target: Q1 2027)

D4: Representative-Hardware Baselines

• generatePlugin + validatePrompt p95/p99 hardware baselines (Target: Q1 2027)
• CAI safety gate p95/p99 overhead hardware baselines (Target: Q1 2027)
• Federated aggregation round p95 hardware baseline (Target: Q1 2027)

Wave D Exit Criteria

• AI generation runbook published (docs/operability/RUNBOOK_AI_GENERATION.md)
• Observability expansion complete (extended Stats + CAI/federated latency histograms)
• OpenTelemetry span propagation in generatePlugin and LLMAQLHandler AI paths
• [~] 60-min soak test created and green on representative hardware (test created 2026-09-16; full run hardware-blocked)
• Representative-hardware p95/p99 baselines captured for AI generation + CAI gate
• Wave D sign-off: human approval at docs/operability/WAVE_D_SIGN_OFF.md

Module Validation Evidence (2026-07-19)

Documentation Status

• @file Doxygen Headers: 100% coverage (4/4 files)

  • include/ai/ai_plugin_generator.h — hardened implementation metadata
  • include/ai/cai_ethics_integration.h — hardened implementation metadata
  • src/ai/ai_plugin_generator.cpp — hardened implementation metadata
  • src/ai/cai_ethics_integration.cpp — hardened implementation metadata

• Function/Method Documentation: Enhanced 2026-07-19

  • Added comprehensive Doxygen comments to all public methods
  • Added parameter/return/error documentation to key functions
  • Documented thread-safety contracts and error handling semantics
  • Implementation files received the primary Doxygen expansion (+263 lines across 2 .cpp files)
  • Header files received follow-up contract clarifications for transport, validation, and latency semantics

Code Quality Metrics

• Lines of Code:

  • Source: 946 lines (ai_plugin_generator.cpp: 634, cai_ethics_integration.cpp: 312)
  • Headers: 476 lines (ai_plugin_generator.h: 283, cai_ethics_integration.h: 193)
  • Total: 1,422 lines (module core)

• API Stability:

  • Public API stable (AIPluginGenerator, CAIEthicsIntegration)
  • Config structures fully documented with field semantics
  • Callback types fully documented with signatures and contract details
  • Zero breaking changes in current implementation

Feature Coverage

• ✅ Phase 1: Stable API for prompt/config/result types
• ✅ Phase 2: Endpoint invocation with configurable transport
• ✅ Phase 3: Structured error handling and edge cases
• ✅ Phase 4: Focused unit test coverage (2 test files: test_ai_decision_auditor.cpp, test_ai_plugin_generator.cpp)
• ✅ Phase 5: Observable counters implemented (Stats struct with 7 counters)
• ✅ Phase 6: Documentation aligned with implementation

Hardening Completeness

• ✅ Prompt validation (description length, token list sizes, format validation)
• ✅ Request sanitization (ASCII control character stripping)
• ✅ Endpoint allow-list enforcement
• ✅ Request/response size limits (256 KiB / 8 MiB)
• ✅ Retryable endpoint invocation (3 attempts, exponential backoff)
• ✅ Response parsing with malformed input rejection
• ✅ Output field validation (code size, manifest fields)
• ✅ Optional C1 CAI safety gate (Wave C feature)
• ✅ Optional sandbox artifact materialization + callback verification enforced in the generator path
• ✅ Optional C2 federated telemetry (Wave C feature)
• ✅ Observability counters for error classes

Production Readiness Status

• Maturity: 🟡 Hardened implementation (all HIGH-severity gaps reviewed; full production validation still pending environment-complete build/test)
• Configuration: Validatable via CMakeLists.txt, CMakePresets.json
• Testing: Focused test targets auto-discovered (module_ai_*_focused.exe)
• Error Handling: Fail-closed with structured Error result types
• Logging: Redaction-aware with configurable truncation (120 chars max)
• Thread Safety: Document-specified (not thread-safe for concurrent generatePlugin)
• Memory Safety: RAII compliance, smart pointers for ownership
• Security: Input validation, output bounds checking, endpoint allow-listing

Known Limitations (MODULE_GAPS.md reference)

• HIGH-severity scanner findings were reviewed and either remediated or reclassified with source-backed explanations (2026-07-19)
• MEDIUM-priority scope/documentation gaps remain tracked in MODULE_GAPS.md
• Dedicated benchmark target for the AI plugin generator path is registered and tracked in benchmark docs
• Sandbox verification now includes built-in artifact materialization, read-back verification, and optional callback enforcement

Acceptance Criteria Status

• Validation-first execution path implemented and verified
• Structured error handling for all failure points
• Hardening follow-ups complete for endpoint safety, payload validation, observability counters, benchmark coverage, and sandbox artifact verification
• API documentation complete and comprehensive
• Implementation semantics documented at function level
• Roadmap and Future Enhancements synchronized (2026-07-19)
• All HIGH-severity gaps reviewed with either code remediation or source-backed disposition notes (2026-07-19)

Breaking Changes

No breaking API change planned. Any signature/semantic contract change requires explicit migration notes and changelog entry.

---

ThemisDB 1.9.0-beta · Home · Module-Index · GitHub · Issues