Fix/credibility and isolation - #1
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR pivots the repo toward an “honest” detection-timing backtest credibility artifact (SVB / Wirecard / FTX) and away from the deprecated causal-simulation/orchestrator framing, while adding supporting ingestion, reporting, and compliance gates needed to generate and validate client-facing outputs.
Changes:
- Adds three pinned historical backtest events (fixtures + validation summaries/metrics) and updates docs/README positioning around detection track record.
- Introduces GDELT DOC 2.0 ingestion + fixture builder tooling and adds offline tests for the backtest harness and deck-building CLI.
- Tightens delivery/review compliance gates (disclaimer + banned language + evidence requirements) and removes deprecated orchestrator surface area (moved under
archive/).
Reviewed changes
Copilot reviewed 67 out of 92 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| simulations/validation/wirecard-2020/summary.md | Adds Wirecard validation summary |
| simulations/validation/wirecard-2020/metrics.json | Adds Wirecard validation metrics |
| simulations/validation/svb-2023/summary.md | Adds SVB validation summary |
| simulations/validation/svb-2023/metrics.json | Adds SVB validation metrics |
| simulations/validation/ftx-2022/summary.md | Adds FTX validation summary |
| simulations/validation/ftx-2022/metrics.json | Adds FTX validation metrics |
| simulations/validation/README.md | Documents per-event validation format |
| simulations/README.md | Reframes simulations as backtests + artifacts |
| simulations/backtest/grade_live_signals.py | Adds live feedback-to-ledger script |
| simulations/backtest/events/wirecard-2020/watchlist.json | Adds Wirecard watchlist fixture |
| simulations/backtest/events/wirecard-2020/observations.jsonl | Adds Wirecard observation corpus |
| simulations/backtest/events/wirecard-2020/ground_truth.json | Adds Wirecard ground truth |
| simulations/backtest/events/svb-2023/watchlist.json | Adds SVB watchlist fixture |
| simulations/backtest/events/svb-2023/ground_truth.json | Adds SVB ground truth |
| simulations/backtest/events/ftx-2022/watchlist.json | Adds FTX watchlist fixture |
| simulations/backtest/events/ftx-2022/ground_truth.json | Adds FTX ground truth |
| simulations/backtest/build_fixture.py | Adds fixture rebuild tool (GDELT + anchors) |
| scripts/render_working_paper.py | Adds Markdown→HTML paper renderer |
| scripts/build_deck.py | Adds deck builder CLI (fixture/live modes) |
| RELEASE_CHECKLIST.md | Updates pre-public checklist for backtests |
| README.md | Rewrites top-level positioning + track record table |
| master_orchestrator.py | Removes deprecated orchestrator entrypoint |
| landing_page/og.svg | Adds OG image asset |
| landing_page/favicon.svg | Adds favicon asset |
| landing_page/DEPLOY.md | Adds Cloudflare Pages deploy guide |
| docs/working_paper/README.md | Adds working paper build/repro guide |
| docs/benchmark_spec.md | Removes causal-world-model benchmark track |
| CONTRIBUTING.md | Updates contrib guidance to new scope |
| CODEOWNERS | Updates ownership boundaries |
| business_research/README.md | Adds business research index |
| business_research/01_code_audit.md | Adds “honest audit” document |
| business_research/02_reorg_octus_comparable.md | Adds comparable analysis |
| business_research/03_market_research.md | Adds market research notes |
| business_research/04_recommendation.md | Adds monetization recommendation |
| archive/validation-2026-04-15/summary.md | Archives old validation summary |
| archive/validation-2026-04-15/signoff.md | Archives old signoff |
| archive/validation-2026-04-15/completions_usage_2026-04-14_2026-04-15.json | Archives old provider usage evidence |
| archive/validation_summary.template.md | Archives old template |
| archive/svb-2023_observations_hand_curated.jsonl | Archives old hand-curated SVB corpus |
| archive/sensory_core.py | Archives deprecated sensory core |
| archive/sample_risk_decision_log.md | Archives deprecated sample log |
| archive/run_replay.py | Archives deprecated replay runner |
| archive/replay_vix_shock.md | Archives deprecated replay doc |
| archive/replay_rate_hike_2026.md | Archives deprecated replay doc |
| archive/replay_credit_spread_blowout.md | Archives deprecated replay doc |
| archive/README.md | Documents archived/deprecated scope |
| archive/README_replay_case_studies.md | Archives old replay template |
| archive/orchestrator.py | Archives deprecated orchestrator |
| archive/mathematical_foundations.md | Archives causal-model writeup |
| archive/deployment_guide.md | Archives old deployment guide |
| archive/core/sensory_forager.py | Archives deprecated forager |
| archive/core/response_engine.py | Archives deprecated response engine |
| archive/core/optimizer.py | Archives deprecated optimizer |
| archive/core/model_adapter.py | Archives deprecated model adapter |
| archive/core/contracts.py | Archives deprecated contracts |
| archive/core/causal_brain.py | Archives deprecated causal brain |
| archive/core/init.py | Archives deprecated core init |
| archive/completion_validation_runbook.md | Archives deprecated runbook |
| aetherius/tests/test_gdelt_adapter.py | Adds unit tests for GDELT adapter |
| aetherius/tests/test_build_deck_cli.py | Adds offline E2E deck build tests |
| aetherius/tests/test_backtest_harness.py | Adds offline backtest harness tests |
| aetherius/README.md | Updates backend README scope pointers |
| aetherius/app/templates/pdf/daily_brief.html | Adds disclaimer to PDF template |
| aetherius/app/templates/email/daily_brief.html | Adds disclaimer to email template |
| aetherius/app/services/signals/service.py | Enforces evidence scoring for high severity |
| aetherius/app/services/review/service.py | Prevents resend from rewinding status |
| aetherius/app/services/ingestion/service.py | Makes freshness tolerant of naive datetimes |
| aetherius/app/services/ingestion/gdelt_adapter.py | Adds GDELT DOC 2.0 ingestion adapter |
| aetherius/app/services/ingestion/crawl_adapter.py | Removes deprecated crawl adapter |
| aetherius/app/services/entity_mapping/service.py | Extracts pure matcher + idempotency guard |
| aetherius/app/services/delivery/service.py | Adds disclaimer + banned-language gates |
| aetherius/app/api/routers/review.py | Admin-gates review actions |
| aetherius/app/api/routers/reporting.py | Adds client-scope dependency checks |
| .gitignore | Ignores fixture GDELT cache directory |
| .github/pull_request_template.md | Updates PR checklist to backtest/tests |
| .github/ISSUE_TEMPLATE/feature_request.md | Updates scope options |
| .github/ISSUE_TEMPLATE/bug_report.md | Updates run-mode options |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
143
to
162
| for item in watch_items: | ||
| rel = None | ||
| score = 0.0 | ||
| ticker_up = item.ticker.upper() | ||
| item_aliases = aliases_by_ticker.get(ticker_up, []) | ||
|
|
||
| direct_hit = ( | ||
| _contains_phrase(text_upper, ticker_up) | ||
| or _contains_phrase(text_upper, item.company_name) | ||
| or any(_contains_phrase(text_upper, alias) for alias in item_aliases) | ||
| item_aliases = aliases_by_ticker.get(item.ticker.upper(), []) | ||
| links = db.query(EntityRelationships).filter(EntityRelationships.watchlist_item_id == item.id).all() | ||
| relationships = [ | ||
| { | ||
| "related_entity_name": link.related_entity_name, | ||
| "related_ticker": link.related_ticker, | ||
| "relationship_type": link.relationship_type, | ||
| "strength": link.strength, | ||
| } | ||
| for link in links | ||
| ] | ||
| result = match_item( | ||
| text_upper, | ||
| ticker=item.ticker, | ||
| company_name=item.company_name, | ||
| sector=item.sector, | ||
| aliases=item_aliases, | ||
| relationships=relationships, | ||
| ) |
Comment on lines
+69
to
+86
| query = ( | ||
| db.query(ClientFeedback, BriefingRuns.run_type) | ||
| .outerjoin(BriefingRuns, BriefingRuns.id == ClientFeedback.briefing_run_id) | ||
| .order_by(ClientFeedback.created_at.asc()) | ||
| ) | ||
| for fb, run_type in query.all(): | ||
| rows.append( | ||
| { | ||
| "feedback_id": str(fb.id), | ||
| "recorded_at": fb.created_at.isoformat() if fb.created_at else "", | ||
| "client_id": str(fb.client_id), | ||
| "briefing_run_id": str(fb.briefing_run_id) if fb.briefing_run_id else "", | ||
| "run_type": run_type or "", | ||
| "feedback_type": fb.feedback_type or "", | ||
| "feedback_text": (fb.feedback_text or "").replace("\n", " ").strip(), | ||
| "recorded_by": fb.recorded_by or "", | ||
| } | ||
| ) |
Comment on lines
+17
to
+20
| from pathlib import Path | ||
|
|
||
| import markdown | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
python simulations/run_replay.py --shock-id shock-rate-50passespytest -q aetherius/tests)ruff check aetherius)Scope
core/,orchestrator.py,simulations/)aetherius/app/, workers, templates)Safety Checklist
.env, API keys, tokens)simulations/artifacts/, runtime logs, caches)Notes for Reviewers