Due-diligence report + WP-014/015/016: safety net, state hygiene, risk-gate slice - #1
Open
jinujon007 wants to merge 13 commits into
Open
Due-diligence report + WP-014/015/016: safety net, state hygiene, risk-gate slice#1jinujon007 wants to merge 13 commits into
jinujon007 wants to merge 13 commits into
Conversation
…ce-tagged) Full repository-first audit at WP-013: verified state (233 tests green, paper live with first real rebalance 07-17, shadow books matching), architecture/engineering review, product maturity scorecard, and web-verified market + OSS research (2026-07-21). Findings new to the registers, for follow-up: - E-2: equal-weight sizing always drops the 10th buy on full redeployment (observed: SYRMA.NS, 9893 < 10011) - bundle with TD-014 into a Momentum v1.1 ADR - R-001 checklist is Angel-One-era; SEBI Apr-2026 regime's <=10 orders/sec personal-API carve-out makes the Zerodha path simpler - ADR-023 (MLflow) and ADR-022 (empyrical) stale against verified 2026 upstream state - revisit before Phase 5 - yfinance: silent history revisions documented upstream; bhavcopy primary + Kite historical recommended at live time INVENTORY.md regenerated for the new tracked file (CI freshness gate). Operational state files (paper_state.json, universe_pit.db, equity_curve.csv) deliberately left uncommitted - TD-016 decision pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-039) The validation record's evidence files (paper_trades.csv untracked; risk.db, daily_run.log, shadow/ gitignored; paper_state.json single-copy) had zero off-repo protection, and a silent dead scheduler already cost a month of data once (PRD section 2). Shell-level net, ahead of Phase 7's AlertSink by the ADR-034 slice precedent: - tools/send_alert.ps1: one webhook POST per non-clean daily run (QUANTOS_ALERT_URL; unset = SKIPPED, never a failure; status-only messages) - tools/backup_state.ps1: dated daily backup of all non-regenerable state to QUANTOS_BACKUP_DIR (default D:\QuantOS_Backups), newest 30 kept; failure = FAILED step -> DEGRADED tile + alert - tools/daily_watchdog.ps1 (+ register_watchdog_task.ps1, task registered on this machine): weekdays 16:30, alerts when today has no "daily run start" entry; 3-minute recheck guards the StartWhenAvailable boot race - daily_run.ps1: PsStep helper, problem accumulation, backup step, conditional alert dispatch last Self-review caught and fixed: Copy-Item nested shadow\shadow on same-day re-runs (reproduced live, now copies contents into a pre-created folder, double-run verified idempotent); watchdog boot-race false positive. Freeze-safe: reads state/logs only, writes nothing any signal path consumes (CONTEXT.md permitted class). Log lines keep api/collectors.py parser semantics (verified end-to-end). Registers: R-005 -> Low, R-006 -> Low-Medium; TD-017 (10th-buy sizing, audit E-2) and TD-018 (legacy _fetch_close fail-open, audit E-1) recorded. PROJECT_STATE.yaml refreshed (stale WP-010/178-test entries). Gates: 233 passed / ruff clean / format clean / mypy --strict clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
data/paper_state.json was git-tracked while the scheduled task mutated it daily: permanent dirty tree, and any `git checkout/restore -- data/` would silently roll the live validation account back in time. Decision deliberately deferred until a real backup existed - WP-014 provided it. - git rm --cached data/paper_state.json; gitignore it + data/paper_trades.csv + AI-session scaffolding (.claude/, LOOP.md, STATE.md, loop-*.md). History/durability = ADR-039 dated backups. - universe_pit.db and pinned results CSVs stay tracked (evidence, weekly/deliberate cadence, git history valuable) - this commit also records the 2026-07-17 Friday PIT snapshot (504 tickers) and syncs equity_curve.csv (content verified byte-identical to the pinned baseline sha e3d29859..., drift was rewrite-only). - Rollback foot-gun inverts into protection: checking out a commit where the file was tracked now refuses instead of clobbering. TD-016 marked Resolved. INVENTORY regenerated (645 files). Gates: 233 passed / ruff clean / format clean / mypy --strict clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
First real Part V control behind the WP-008 seam: - risk/limits.py: check_position_limit (pure; exactly-at-limit allowed, ambiguity blocks), PositionLimitGate (BUYs capped at limit_pct of NAV; SELLs never blocked - an exit reduces the exposure the limit bounds; unreadable book = typed fail-closed block), CompositeGate (first breach blocks; empty stack refused at construction). - BookView/OrderLike/Checkable structural protocols keep risk's ADR-032 cell clean (no portfolio/brokers imports) - same pattern as gate.py's untyped order. - Thresholds injected at the composition root (Part V default 0.15) until WP-006 layered config exists; changes still gated by ADR-025. - demo_pipeline: engine now behind CompositeGate([KillSwitchGate, PositionLimitGate(PaperBook, 0.15)]) with a live BookView over the paper broker; new 7b drill proves an oversized (~20% NAV) order is BLOCKED and journaled. - 16 new tests: math boundaries, SELL pass-through, fail-closed broken book, composite ordering + empty-stack refusal, engine integration (BLOCKED journaled, broker never sees the order). Paper daily cycle deliberately un-gated: it does not route through the engine until Phase 6 cutover, and its semantics are frozen with the validation clock. Gates cover every engine-mediated order - the only path live capital will use (ADR-010). Gates: 249 passed / ruff clean / format clean / mypy --strict clean / determinism 3x byte-identical (sha e3d29859... pinned). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… pinned sha 6192c9d6) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jinujon007
changed the title
…rd in-repo) Dated governance program log (2026-07-14 convention) capturing the audit + WP-014/015/016 loop, register deltas, operator queue, next WP. CURRENT_TASK.md was stale at WP-010/011. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every completed paper run appends date,total_value,cash,positions,degraded to data/paper_equity_history.csv (true append, last-write-wins per date on --force reruns; row written only after save_state). tools/paper_metrics.py computes total return, annualized Sharpe and max drawdown from it — the Sept-9 gate's 'paper Sharpe > 1.0' is computable for the first time. Freeze-safe: capture is pure logging inside the frozen script (CONTEXT.md permitted class; validation clock intact). CSV gitignored per ADR-040 (daily-mutated state) and added to the WP-014 backup set. 10 new tests (259 total); ruff/format/mypy-strict/determinism 3x all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Old venv carried ~205 packages vs the 59-line locked closure (langchain*, openai, vibe-trading-ai dead weight from a pre-QuantOS era). Deleted and rebuilt at the canonical venv path: pip install -r requirements-lock.txt + pip install --no-deps -e . under Python 3.13.7. Proof the lockfile reproduces the running environment (the debt's exact claim): 259 tests, ruff, format --check, mypy --strict (44 files), paper_trader --selftest, determinism 10x byte-identical with pinned shas e3d29859/6192c9d6 - all green under the rebuilt env. Register + state synced; no source changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both Constitution-era ADRs were flagged 'verified stale - revisit before Phase 5' by the 2026-07-21 due diligence (SS9.1): empyrical/pyfolio is in maintenance mode (last release 2025-12), so metric formulas are ported natively with the source cited at the definition site (paper_metrics.py gains the citation docstring; quantstats stays report-sidecar-only). MLflow 3.x pivoted to GenAI tracing - Phase 5 tracking becomes a ~50-line native run-manifest, built with Phase 5, not before. Retroactively governance-cleans WP-017's native Sharpe; closes the DD action item. No behavior change (docstring only); gates green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… stubs generate_inventory.py bucketed everything under the module-skeleton prefixes as 'Module Scaffold (empty)', mislabeling 33 real quantos_core files (WP-001..017 code) as zero-implementation placeholders - recurring since WP-002. Classification is now per-file: __init__.py/.gitkeep stay scaffold; everything else lands in a new 'Platform Code' bucket. INVENTORY.md regenerated (33 platform / 23 placeholder); pinned by tests/test_inventory_classifier.py; PROJECT_STATE scaffold count corrected 15 -> 4 (analytics, live, monitoring, validation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… drift CI caught INVENTORY.md regenerated before tests/test_inventory_classifier.py was git-tracked (the generator walks git ls-files) - regenerated with all files staged; the inventory-match gate did exactly its job. ADR-044 files the DD M-6 recommendation as PROPOSED: NSE bhavcopy primary (vendored fetch logic), yfinance quarantined cross-check, Kite historical at live, exchange-calendars XNSE for the Clock port. Deliberately not self-approved - commits external data surfaces and a future paid plan; Phase 2 fetch-adapter implementation blocks on operator approval (loop stop condition 4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADR-044 operator-approved same day. quantos_core/data/bhavcopy.py: UDiFF CM parser golden-pinned against the real published file (2026-07-20, RELIANCE close 1323.10), fail-closed on format change, mixed trade dates, duplicate symbols, and non-positive closes; fetch_bhavcopy_zip is a thin shell (404 = typed holiday/not-published failure). quantos_core/utils/trading_calendar.py wraps exchange-calendars XBOM (ADR-044 said XNSE - no such calendar exists; correction recorded in the ADR's acceptance note). tools/fetch_bhavcopy.py archives each session's zip as-published into data/bhavcopy/ (immutable, idempotent, gitignored) - live-verified: 2026-07-21 session archived, 2,685 equity rows parsed. yfinance is now quarantined for all new code (cross-check only); the frozen daily loop keeps its path until cutover per the validation freeze. Lockfile extended by freezing the verified canonical venv (previous transitive pins unchanged; from-scratch re-resolve rejected because it floated versions the gates never ran against) and re-verified by a clean install running all 282 tests. 20 new tests (282 total); ruff/format/mypy --strict (46 files)/ determinism 10x byte-identical all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…havcopy provider (ADR-045) Corporate-action adjustment from NSE's official Bc records (PR bundle, same cookie-free archives host as the bhavcopy): BONUS/FVSPLT/FVCONS parsed to exact factors incl. 8 live wording variants; dividends price-neutral; unparseable computable families, paise denominations, ambiguous re-worded duplicates and out-of-band factors all halt typed. Fail-closed validate_close_frame quality gate (XBOM calendar coverage, dense positive closes, tunable +/-35% band). BhavcopyPriceProvider serves adjusted validated closes behind the frozen PriceProvider port with a 45-session Bc lookback (NSE drops records pre-ex-date: 262/6530 live, max gap 21 sessions). Two-file range-mode backfill; archives cover 2025-06-02..present (280 sessions, verified 100% clean). The first detection design (UDiFF prev-close ratio) was disproven against the real archive (HDFCBANK 1:1 bonus ex 2025-08-26 published raw) and replaced same session; the quality gate caught it on first live run. Adversarial review: 7 findings fixed and test-pinned. Live-verified: HDFCBANK pre-ex close back-adjusts to exactly 982.05 = 1964.10 x 0.5; RELIANCE 2026-07-20 matches the golden value. 341 offline tests; ruff, mypy --strict, determinism 3x byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
Adds
docs/01_audits/(AI) QuantOS Institutional Due Diligence - 2026-07-21.md— a full 12-phase, evidence-tagged audit (repo state verified at WP-0138bc915e: 233 tests green, paper live with the first real rebalance, shadow books matching; plus web-verified market and OSS research).INVENTORY.mdregenerated for the new tracked file so the CI freshness gate passes.Reviewer notes:
data/paper_state.json,data/universe_pit.db,data/results/equity_curve.csv) are deliberately not in this PR — the scheduler mutates them daily and TD-016 (commit-daily vs gitignore) is an open decision needing its own one-line ADR.Solo-operator cooling-off checklist
CONTEXT.md)? No — documentation only; the freeze list is untouched.tests/golden/)? No.INVENTORY.mdregeneration.Verification
pytestpasses locally (offline subset): 233 passed, 1 deselected, 24s.ruff check ./ruff format --check .: not applicable to.mdfiles; no Python touched. CI runs both regardless.momentum_backtest.py/paper_trader.pyuntouched — determinism check not required (CI runs it 3× regardless).🤖 Generated with Claude Code