From c717ca8e476924b50a368c64dd557dc36ab18553 Mon Sep 17 00:00:00 2001 From: Badry Darkoush Date: Thu, 3 Sep 2026 16:27:34 +0000 Subject: [PATCH 1/2] docs: add IntakeGateway/QueryGateway consolidation assessment Compares the two projects' runtimes, dependencies, modules, quality gates and container topologies; identifies the shared surface, ranks the merge conflicts by cost, records defects found during review, and recommends a staged convergence plan with costed alternatives. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L6oCRzNahV9djmgUtpNe9q --- docs/tech-stack-comparison.md | 391 ++++++++++++++++++++++++++++++++++ 1 file changed, 391 insertions(+) create mode 100644 docs/tech-stack-comparison.md diff --git a/docs/tech-stack-comparison.md b/docs/tech-stack-comparison.md new file mode 100644 index 0000000..43a38ae --- /dev/null +++ b/docs/tech-stack-comparison.md @@ -0,0 +1,391 @@ +# IntakeGateway + QueryGateway — Tech Stack Comparison and Consolidation Assessment + +**Date:** 2026-09-03 +**Scope:** `badry-dev/IntakeGateway` @ `main` (125 commits), `badry-dev/QueryGateway` @ `main` (169 commits) +**Question:** Should these two projects be combined under one product? + +--- + +## 1. Verdict + +**Yes — combine them, but not by merging the two codebases as they stand.** + +The two systems are the inbound and outbound halves of the same product: IntakeGateway pulls data +from HTTP APIs into databases; QueryGateway publishes database queries as HTTP APIs. Same operator, +same credentials, same scheduling problem, same observability problem, same admin console problem. +Keeping them separate means maintaining two connection managers, two schedulers, two auth models, +two credential vaults, and two admin SPAs forever. + +The catch: they share almost no *implementation*. Every foundational choice diverged — +sync vs. async SQLAlchemy, SQLite vs. PostgreSQL, loguru vs. structlog, Ant Design vs. +Tailwind/shadcn, static shared token vs. JWT, Python 3.11 vs. 3.14. There is roughly +**~16,700 backend LOC and ~14,200 frontend LOC** across both, with near-zero directly reusable +overlap despite ~60% conceptual overlap. + +**Recommended path: converge onto QueryGateway's platform, port IntakeGateway's domain logic onto +it, ship as a monorepo with two deployables and one admin console.** Estimated 16–30 developer-weeks +to full convergence. See §7 for the staged plan and §8 for the cheaper alternatives. + +--- + +## 2. What each project actually is + +| | **IntakeGateway** | **QueryGateway** | +|---|---|---| +| **Direction** | HTTP API → database (ingestion) | Database → HTTP API (publication) | +| **Core artifact** | A *Task*: source endpoint + auth + field mapping + upsert rules + cron | An *Endpoint*: parameterized Oracle SQL + auth policy + live/snapshot strategy | +| **Destinations / sources** | Oracle, PostgreSQL, MySQL (claimed) | Oracle only | +| **Execution model** | Async, queued, long-running batch jobs | Synchronous request/response + scheduled snapshot refresh | +| **Key differentiator** | Connector/normalizer/validator/mapper/runner pipeline, OAuth2 token cache, cursor-based incremental fetch, SSRF guard, rate-limit handling | Bind-parameter-only SQL execution, schedule-owned parameter bindings, snapshot coverage/filter semantics, per-endpoint auth policies | +| **Backend LOC** | 8,649 (38 modules) | 8,055 (64 modules) | +| **Frontend LOC** | 5,797 (35 files) | 8,452 (71 files) | +| **Backend test LOC** | 7,368 (30 files) | 7,741 (28 files) | +| **First / last commit** | 2025-09-24 / 2026-09-02 | 2026-03-03 / 2026-09-02 | + +Both are actively developed and were last touched the same day. Neither is abandoned. + +--- + +## 3. Stack comparison + +### 3.1 Backend runtime + +| Concern | IntakeGateway | QueryGateway | Compatible? | +|---|---|---|---| +| Python | **3.11** (`target-version = "py311"`) | **3.14** (`requires-python`, mypy `python_version = "3.14"`) | ⚠️ One-way | +| Web framework | FastAPI 0.141.1 | FastAPI 0.141.1 | ✅ Identical | +| ASGI server | uvicorn[standard] 0.32.1 | uvicorn[standard] 0.52.3 | ✅ Bump | +| Validation | pydantic 2.10.6 / pydantic-settings 2.7.1 | pydantic-settings 2.15.0 | ✅ Bump | +| ORM | SQLAlchemy 2.0.36, **sync** (`create_engine`, `Session`) | SQLAlchemy 2.0.52, **async** (`create_async_engine`, `AsyncSession`) | ❌ Structural | +| App-state DB | **SQLite** (`sqlite:///./intakegateway_app.db`) | **PostgreSQL** (asyncpg + psycopg2-binary) | ❌ Structural | +| Schema mgmt | Alembic (4 revisions) **and** `Base.metadata.create_all()` at startup | Alembic only (5 revisions), enforced one-shot `migrate` container | ⚠️ IG is inconsistent | +| Oracle driver | `oracledb==3.4.2`, imported eagerly | `oracledb>=4.0.2,<5`, isolated in `requirements-oracle.txt`, imported lazily | ⚠️ Major version gap | +| Async job execution | **Celery 5.4.0 + Redis 5.0.8** (separate worker process) | None — in-process only | ❌ Different topology | +| Scheduler | APScheduler 3.10.4, **separate process** (`python app/services/scheduler.py`) | APScheduler 3.11.3, **in-process** via FastAPI lifespan | ⚠️ Reconcilable | +| Cron parsing | `croniter==2.0.7` | APScheduler `CronTrigger` + custom builder | ⚠️ Reconcilable | +| Logging | **loguru 0.7.2** (f-string style) | **structlog 26.1.0** (JSON, mandated correlation fields) | ❌ Structural | +| Crypto | `cryptography==50.0.1`, Fernet, **str in / str out** | `cryptography==50.0.0`, Fernet, **str in / bytes out** | ⚠️ Serialization differs | +| HTTP client | httpx 0.27.2 (core dependency — it *is* the product) | httpx 0.28.1 (health checks only) | ✅ Bump | +| JSON path | `jsonpath-ng==1.6.1` | — | ✅ IG-only | +| Auth libs | — (no JWT, no bcrypt) | `PyJWT==2.13.0`, `bcrypt==5.0.0` | ❌ IG has no identity layer | + +### 3.2 Frontend + +| Concern | IntakeGateway | QueryGateway | Compatible? | +|---|---|---|---| +| React | 18.2 | 18.3 | ✅ | +| Build | **Vite 8.2** | **Vite 6.4** | ⚠️ Two majors apart | +| Router | react-router-dom **6.20** | react-router-dom **7.18** | ⚠️ One major apart | +| Data fetching | @tanstack/react-query 5 + axios | @tanstack/react-query 5 + axios | ✅ Identical pattern | +| Client state | **zustand 4.4** | React context (`AuthProvider`) only | ⚠️ | +| **UI system** | **Ant Design 6.3 + @ant-design/icons** | **Tailwind 3.4 + shadcn/ui (Radix) + lucide-react** | ❌ **Highest-cost conflict** | +| Styling | antd tokens, inline styles, `theme.ts` | Tailwind + `class-variance-authority` + `tailwind-merge` | ❌ | +| Editor | — | CodeMirror 6 (`@uiw/react-codemirror`, `@codemirror/lang-sql`) | ✅ QG-only | +| Dates | date-fns 4 + dayjs 1.11 (both) | — | ⚠️ IG carries two date libs | +| Lint | ESLint **8** (legacy `.eslintrc`, `--ext` flags) | ESLint **9** (flat config, `typescript-eslint`) | ⚠️ | +| Format | **none** | Prettier 3.4 + `prettier-plugin-tailwindcss`, `prettier:check` in CI | ❌ IG has no formatter gate | +| Test | vitest 4 + Testing Library + jsdom 29 | vitest 4 + Testing Library + jsdom 25 + axios-mock-adapter | ✅ | + +### 3.3 Quality gates and engineering discipline + +| Gate | IntakeGateway | QueryGateway | +|---|---|---| +| Ruff rule set | `E, F, W, I, UP` | `E, W, F, I, B, C4, UP, **S** (bandit), **T20**` | +| Type checking | **none** | **mypy `strict = true`** + pydantic plugin | +| Coverage gate | `fail_under = 50` | coverage reported, no hard floor | +| CI workflows | **1** (`ci.yml`) | **6** (backend, frontend, docker, security-scan, dependency-review, actions-lint) | +| Dependabot | ❌ | ✅ | +| CODEOWNERS | ❌ | ✅ | +| SHA-pinned actions | ✅ | ✅ | +| `docs/` files | 1 | 15 | +| Root governance docs | README, SECURITY.md (27KB), AGENTS.md, AI_WORKFLOW.md, SECURITY_AI.md, CLAUDE.md | README, per-area `.github/instructions/*`, CLAUDE.md | + +**QueryGateway enforces a materially higher bar.** Not by a little — strict mypy plus bandit rules +plus six gated workflows versus one workflow with no type checking is a different class of rigor. + +### 3.4 Deployment topology + +**IntakeGateway** (`docker-compose.yml`) — 4 services: `redis`, `api`, `worker`, `scheduler`. +No database container (SQLite on a bind mount). Backend bind-mounted with `--reload`. Redis +password-gated and unpublished. This is a **development** compose file with production-ish +hardening bolted on; there is a separate `compose.production.yml`. + +**QueryGateway** (`docker-compose.yml`) — 5 services: `db` (postgres:16), `migrate` (one-shot), +`api`, `web` (nginx), optional `oracle` (XE, profile-gated). **Two segmented networks**: `edge` +(nginx↔api, fixed `10.31.0.0/24` so `FORWARDED_ALLOW_IPS` can be scoped to exactly that hop) and +`backend` (api↔data tier). API is not published to the host. Postgres and Oracle bound to +`127.0.0.1` only. + +QueryGateway's compose is the more defensible production topology by a wide margin. + +--- + +## 4. Where the two genuinely overlap + +These are the modules that would be written once instead of twice: + +| Capability | IntakeGateway today | QueryGateway today | Consolidation value | +|---|---|---|---| +| **DB connection registry** | `connection_storage.py` (encrypted `connections.enc` file, fcntl/msvcrt locking) + `connection_pool.py` (engine cache, Oracle/PG/MySQL URL builder) | `models/connection.py` + `repositories/connection.py` + `services/connection.py` (Postgres rows, Fernet `LargeBinary`) | **High.** Same entity, two incompatible stores. | +| **Credential encryption** | `core/encryption.py` — Fernet, singleton, str→str, dev-mode key generation | `crypto.py` — Fernet, str→bytes, no dev fallback | **High.** ~identical, 5-minute unification. | +| **Scheduling** | APScheduler + croniter, separate process, cron string only, auto-pause after N failures | APScheduler in-lifespan, friendly cron builder + IANA timezones + schedule-owned parameter bindings + next-3-run preview | **High.** QG's model is strictly richer. | +| **Run / job history** | `task_run`, `task_run_log`, `task_log` tables + Runs/RunDetail pages | `job_run`, `access_log` tables + health dashboard | **High.** One observability model. | +| **Admin identity** | `API_TOKEN` shared secret, no UI login | JWT admin login, `RequireAuth`, bcrypt, per-endpoint auth methods | **High.** IG has no real identity story. | +| **Settings & health** | `/health` returning `{status, env}` | Settings table + full health dashboard (API/Postgres/Oracle/scheduler/recent jobs/stale snapshots) | **Medium.** QG's is production-grade. | +| **Oracle metadata introspection** | `oracle_metadata.py` (`USER_TAB_COLUMNS` for mapping targets) | `sql/executor.py` (`cursor.description` for output schema) | **Medium.** Complementary. | +| **Admin SPA shell** | antd Layout/Sider/Menu + react-query | shadcn Layout + RequireAuth + react-query | **High**, but expensive (§5.4). | +| **API client layer** | `api/client.ts` class wrapping axios | `lib/api.ts` module + `useResourceMutations` | **Medium.** Same pattern, different shape. | + +Roughly **40–50% of each backend is platform plumbing that would be shared**. The remaining +50–60% is genuinely distinct domain logic worth keeping separate regardless. + +--- + +## 5. Conflicts, ranked by cost to resolve + +### 5.1 ❌ Python 3.11 vs. 3.14 — with a Celery blocker underneath +QueryGateway *requires* 3.14 (asyncpg ≥0.31 CPython 3.14 wheels). IntakeGateway targets 3.11 and +depends on **Celery 5.4.0**, whose supported-interpreter matrix does not extend to 3.14. +**This must be validated before any shared-runtime decision.** If Celery cannot run on 3.14, the +choices are: (a) upgrade Celery and validate, (b) replace the queue with an async-native +alternative (`arq`, `dramatiq`), or (c) keep the intake worker on a separate 3.11/3.12 image. +Option (b) aligns better with an async codebase; option (c) is the low-risk fallback. + +### 5.2 ❌ Sync vs. async SQLAlchemy +IntakeGateway's routes, services, and runner are synchronous end to end (`Session`, `def` handlers, +Celery workers). QueryGateway is async end to end (`AsyncSession`, `async def`, `anyio.to_thread` +for the blocking Oracle driver). These do not compose in one process. Porting IntakeGateway to +async touches every route, every service, and the entire runner pipeline — **this is the single +largest backend line item.** + +*Mitigation:* QueryGateway already demonstrates the correct pattern for blocking drivers +(`anyio.to_thread.run_sync` around `oracledb`). The intake runner can keep its blocking write path +inside a thread while the surrounding API becomes async. That reduces the port to the route/service +boundary rather than the whole pipeline. + +### 5.3 ❌ SQLite + `create_all` vs. PostgreSQL + Alembic +IntakeGateway calls `Base.metadata.create_all()` in its lifespan **while also shipping four Alembic +revisions** — schema is managed twice, by two mechanisms, and SQLite needs WAL pragmas and a +`JSONText` shim (`db/types.py`) to emulate what Postgres does natively. QueryGateway is +Alembic-only and gates API startup on `migrate` completing. + +Merging means: one Postgres instance, one Alembic revision graph (the two graphs must be merged or +branched — they cannot both target one `alembic_version` table as-is), delete `create_all`, delete +the `JSONText` shim in favor of `JSONB`. + +### 5.4 ❌ Ant Design vs. Tailwind + shadcn/ui +~5,800 LOC of IntakeGateway frontend is coupled to antd components, antd theme tokens, and inline +styles. QueryGateway is Tailwind + Radix primitives with its own `components/ui/*` kit and a +Prettier/Tailwind class-ordering gate. + +There is no automated migration. Shipping both in one bundle means two CSS systems, two icon sets, +and ~800KB of avoidable payload. **This is the most expensive single item and the one most safely +deferred** — it can be staged behind a route boundary and paid down page by page. + +### 5.5 ❌ Incompatible authentication models +IntakeGateway: one static `API_TOKEN` compared with `hmac.compare_digest`, checked as a router +dependency, fail-closed in production, **no UI login and no user concept.** +QueryGateway: env-seeded single admin, bcrypt hash, JWT with `sub`/`exp`/`iat`, `HTTPBearer` +dependency, `RequireAuth` route guard, plus a separate per-endpoint auth-method system +(Bearer/Basic/API key) for the data plane. + +QueryGateway's model is the only one that survives a merge. IntakeGateway's management routes +would move behind `get_current_admin`. Note that **neither project has multi-user support** — +both are single-admin. If the combined product needs teams or RBAC, that is net-new work in both +scenarios and is a reason to build it once, in the merged product. + +### 5.6 ⚠️ Configuration collision +IntakeGateway: `case_sensitive=True`, `UPPER_SNAKE` names, `.env`. +QueryGateway: `case_sensitive=False`, `lower_snake` names, `.env`. +Both consume `ENCRYPTION_KEY` and both define an environment flag (`APP_ENV`) with different +allowed values and different production-gating behavior. Both read the same `.env` filename. +In one process these collide. One `Settings` class, one naming convention, one `.env` schema. + +### 5.7 ⚠️ Route namespace divergence +IntakeGateway: `/api/v1/tasks`, `/api/v1/runs`, `/api/v1/schedules`, `/api/v1/connections`. +QueryGateway: strict `/api/v1/admin/*` (console) and `/api/v1/data/*` (consumers), documented as a +hard convention with versioned breaking changes. + +QueryGateway's split is the better contract and is already load-bearing for its security model +(every `/data/*` request is authenticated, no exceptions). Intake routes move under +`/api/v1/admin/intake/*`. + +### 5.8 ⚠️ Logging: loguru vs. structlog +QueryGateway mandates structured JSON with `request_id`, `user`, `endpoint`, `status`, +`duration_ms`, `method`, `client_ip`, `event` — and scheduler jobs add `job_id`, `run_id`, +`row_count`, `success`. IntakeGateway uses loguru f-strings with ad-hoc redaction helpers. +Mechanical to port, but it touches every log call site (~hundreds). + +### 5.9 ⚠️ Credential storage: encrypted file vs. database +`connections.enc` with cross-process advisory locking (fcntl/msvcrt) exists specifically because +IntakeGateway's API, worker, and scheduler are three processes with no shared transactional store. +Once there is a shared PostgreSQL, that file — and its locking code, and its +"refuse to run on platforms without a locking primitive" branch — is deleted outright. **This is a +concrete, immediate win from consolidation.** + +--- + +## 6. Defects and debt surfaced during review + +These are worth fixing regardless of the merge decision. + +| # | Repo | Finding | +|---|---|---| +| 1 | IntakeGateway | **`backend/app/services/connection_pool.py:36` hardcodes `C:\oracle\instantclient_23_0`** as the Oracle thick-mode client path. Non-configurable and Windows-only; thick mode is silently unavailable in the Linux containers the project ships. QueryGateway does this correctly via `ORACLE_CLIENT_LIB_DIR`. | +| 2 | IntakeGateway | **PostgreSQL and MySQL destinations are advertised but undeliverable.** `connection_pool.py` builds `postgresql+psycopg2://` and `mysql+pymysql://` URLs, but **neither `psycopg2` nor `PyMySQL` appears in any requirements file or the Dockerfile.** Both destination types fail at engine creation with `ModuleNotFoundError`. The README claims all three. | +| 3 | IntakeGateway | Two stale requirements files — `requirements-minimal.txt` (FastAPI 0.104.0) and `requirements-simple.txt` (FastAPI 0.100.0) — pin releases four years out of date next to the real `requirements.txt` (0.141.1). Delete or document. | +| 4 | IntakeGateway | Schema managed twice: `create_all()` in lifespan **and** four Alembic revisions. Divergence is a matter of time. | +| 5 | IntakeGateway | No type checking and no formatter gate. QueryGateway runs strict mypy + Prettier in CI. | +| 6 | IntakeGateway | Duplicated comment line in `core/config.py` (`# Auto-pause a schedule after...` appears twice). Cosmetic. | +| 7 | IntakeGateway | Ships both `date-fns` and `dayjs`; antd already depends on dayjs. Drop one. | +| 8 | Both | `oracledb` major-version split (3.4.2 vs. ≥4.0.2). One version in a merged runtime; note that oracledb 4.x thick mode requires Oracle Client 19+, which QueryGateway's image already bundles (Instant Client 19.32). | +| 9 | Both | Single hardcoded admin identity, no multi-user, no RBAC, no audit of *who* changed configuration. Acceptable for a self-hosted single-operator tool; a blocker for team or multi-tenant use. | + +--- + +## 7. Recommended plan — converge onto QueryGateway's platform + +### 7.1 Direction of travel, and why + +**Adopt QueryGateway's foundations. Port IntakeGateway's domain logic onto them.** + +| Dimension | Winner | Why | +|---|---|---| +| Runtime | QG (async, 3.14) | Async is required for the request-serving data plane; sync is not required for anything IG does that a thread pool cannot cover. | +| Data tier | QG (PostgreSQL) | SQLite forces the encrypted-file connection store, the `JSONText` shim, and WAL workarounds. All three vanish. | +| Migrations | QG (Alembic-only + gated `migrate`) | IG's dual mechanism is a latent production incident. | +| Logging | QG (structlog) | Mandated correlation fields; already documented as a project convention. | +| Identity | QG (JWT + bcrypt + per-endpoint policies) | IG has no UI login at all. | +| UI | QG (Tailwind + shadcn) | Prettier-gated, no runtime theme provider, smaller payload, already the larger surface (8.4k vs. 5.8k LOC). | +| CI / typing | QG (6 workflows, strict mypy, bandit) | Strictly higher bar; adopting it raises IG, not the reverse. | +| Container topology | QG (segmented networks, nginx, one-shot migrate) | Defensible in production. | +| **Async job execution** | **IG (Celery + Redis worker)** | **QG has none.** Long-running, retryable, parallelizable imports need a real queue. Snapshot refresh should move onto it too. | +| **Domain pipeline** | **IG (connector/normalizer/validator/mapper/runner)** | Genuinely valuable and fully portable — no framework coupling beyond `Session` and loguru. | +| **SSRF guard, OAuth2 token cache, rate-limit handling, cursor watermarks** | **IG** | Net-new capability QG does not have. Keep all of it. | + +The asymmetry is clear: QueryGateway wins on *platform*, IntakeGateway wins on *domain logic and +execution topology*. That is exactly the split that makes a merge worthwhile — you keep the best +half of each. + +### 7.2 Target architecture + +``` + ┌──────────────────────────────┐ + │ Admin SPA (one console) │ + │ React 18 · Vite · TS │ + │ Tailwind + shadcn/ui │ + │ react-query + axios │ + │ JWT session (RequireAuth) │ + └──────────────┬───────────────┘ + │ /api/v1/admin/* + ┌──────────────▼───────────────┐ + │ nginx (edge network) │ + └──────────────┬───────────────┘ + │ + ┌────────────────────────┴────────────────────────┐ + │ │ + ┌─────────▼──────────┐ ┌────────────▼─────────┐ + │ Query plane (API) │ │ Intake plane (API) │ + │ FastAPI · async │ │ FastAPI · async │ + │ /api/v1/data/* │ │ /admin/intake/* │ + │ /admin/endpoints │ │ /admin/tasks │ + └─────────┬──────────┘ └────────────┬─────────┘ + │ │ + └──────────────┬───────────────────────────────────┘ + │ shared packages + ┌───────────────────▼────────────────────┐ + │ core: config · crypto · logging(JSON) │ + │ auth: JWT admin + endpoint policies │ + │ connections: registry + engine pool │ + │ scheduling: APScheduler + bindings │ + │ runs: job_run · access_log · health │ + └───────────────────┬────────────────────┘ + │ + ┌───────────────────▼────────────────────┐ + │ PostgreSQL 16 (state, snapshots, │ + │ encrypted credentials, run history) │ + │ Redis (queue broker) │ + │ Worker (imports + snapshot refresh) │ + │ Oracle / PostgreSQL / MySQL (external)│ + └────────────────────────────────────────┘ +``` + +One console, one identity, one Postgres, one credential vault, one scheduler, one run-history model. +Two backend deployables, because the scaling profiles genuinely differ: the query plane is +request-latency-bound; the intake plane is worker-throughput-bound. + +### 7.3 Staged plan + +| Phase | Work | Effort | Exit criterion | +|---|---|---|---| +| **0. Monorepo, no code change** | Merge both repos into one with `apps/query`, `apps/intake`, `packages/`. Adopt QG's CI matrix over both. Turn on mypy for IG in non-blocking mode; add Prettier. Freeze new antd surface area. | 1–2 wks | Both apps build, test, and deploy exactly as before, from one repo, under one CI. | +| **1. Identity and shell** | One `Settings` class and `.env` schema. One JWT admin auth; IG management routes move behind `get_current_admin`. One SPA shell with a login; IG pages mounted inside it (antd temporarily allowed behind a route boundary). Route namespaces normalized to `/api/v1/admin/*`. | 3–5 wks | One login, one nav, both feature sets reachable. | +| **2. Data tier** | IG app state SQLite → PostgreSQL. Merge the two Alembic graphs. Delete `create_all()` and `db/types.JSONText`. Migrate `connections.enc` → `connections` table; **generalize QG's `OracleConnection` into a `Connection` with a `db_type` discriminator** (Oracle/PostgreSQL/MySQL) and ship the missing `psycopg2`/`PyMySQL` drivers (fixes §6 #2). Retire the file-locking module. | 4–8 wks | One database, one migration graph, one credential store, three destination types actually working. | +| **3. Execution** | One scheduler service. Decide the queue (validate Celery on 3.14, else `arq`/`dramatiq`, else pin the worker to its own image). Move snapshot refresh onto the queue. Unify `task_run`/`job_run` into one run model + one health dashboard. Port loguru → structlog. | 4–8 wks | One scheduler, one queue, one run-history table, one JSON log stream. | +| **4. UI convergence** | Port IG's 8 pages from antd to Tailwind/shadcn, page by page. Drop antd, `@ant-design/icons`, and one of date-fns/dayjs. Align Vite 6↔8 and react-router 6↔7. | 4–8 wks | One design system; bundle size down materially. | +| **Ongoing** | Fix §6 findings; decide multi-user/RBAC. | — | — | + +**Total: 16–31 developer-weeks (≈4–7 months at one focused engineer).** +Phases 0–1 alone (4–7 weeks) deliver the single most visible win: one product, one login, one console. + +### 7.4 Sequencing rules + +1. **Phase 0 must not change behavior.** Repo topology and CI only. Anything else invites a + multi-month rewrite with no shippable intermediate. +2. **Resolve the Python/Celery question before Phase 3, not during it.** It determines whether the + queue is a port or a replacement — a several-week swing. +3. **Defer the UI (Phase 4) to last.** It is the most expensive and the least risky to postpone; + two design systems behind one login is ugly but shippable. +4. **Do not merge the two Alembic graphs by hand under time pressure.** Two revision trees against + one `alembic_version` table is a data-loss shape. Branch labels or a squashed baseline, decided + deliberately. + +--- + +## 8. Alternatives, honestly costed + +| Option | What it is | Cost | Value | Verdict | +|---|---|---|---|---| +| **A. Full single application** | One FastAPI app, one deployable, one SPA | 20–35 dev-wks | Maximum coherence | **Over-merged.** Request-latency and worker-throughput planes want independent scaling and deploy cadence. | +| **B. Monorepo + shared packages, two deployables, one console** ⭐ | §7.2 | **16–31 dev-wks** | ~85% of A's value | **Recommended.** Kills all duplication that matters; keeps the planes independently scalable. | +| **C. Monorepo, shared CI/conventions only** | Same repo, no shared code | 2–4 dev-wks | Low | Cheap coherence, zero compounding benefit. Connections/auth/scheduler stay duplicated forever. | +| **D. Status quo — two repos** | Nothing changes | 0 | 0 | Every capability gets built twice. IG's missing identity layer and PG/MySQL drivers never get QG's rigor applied to them. | + +**If budget is the constraint: do C now, and A/B's Phase 0 + Phase 1 next quarter.** Even the +identity unification alone (Phase 1) is worth its cost — IntakeGateway currently has no admin +login, which is the most significant single gap between the two projects. + +--- + +## 9. Open decisions requiring your input + +1. **Queue technology on Python 3.14.** Validate Celery 5.4→latest on 3.14. If it fails: `arq` + (async-native, Redis, small) is the natural fit for an async codebase; `dramatiq` if you want + Celery-like semantics. Fallback: pin the intake worker to its own 3.12 image. +2. **Multi-user / RBAC.** Both projects are single-admin. If the combined product needs teams, + build it once during Phase 1 — retrofitting after Phase 4 costs several times more. +3. **Product name and namespace.** "Gateway" with Intake and Query planes is the obvious framing, + but this is a positioning decision, not a technical one. +4. **Destination breadth.** Does the query plane also need PostgreSQL/MySQL sources (matching + intake's advertised breadth), or does Oracle-only remain the query-plane contract? This changes + the shape of the shared `Connection` entity in Phase 2. +5. **Snapshot storage.** Query-plane snapshots are PostgreSQL JSONB today. If intake volumes push + the shared Postgres hard, snapshots are the first thing to move to object storage. + +--- + +## 10. Summary + +- The two projects are **complementary halves of one product**, not competitors. Combining is + strategically correct. +- They share **~60% of concepts and near-0% of implementation**. Every foundational choice diverged. +- **QueryGateway has the better platform** (async, PostgreSQL, structlog, JWT, strict mypy, bandit, + 6 CI workflows, segmented container networks). **IntakeGateway has the better execution topology + and the more valuable domain logic** (Celery worker, connector/normalizer/mapper/runner pipeline, + SSRF guard, OAuth2 token cache, cursor watermarks). +- **Converge onto QueryGateway's platform; port IntakeGateway's domain logic onto it.** + Monorepo, shared packages, one admin console, two backend deployables. +- **16–31 developer-weeks** for full convergence; **4–7 weeks** for the visible win (one product, + one login, one console). +- Two IntakeGateway defects should be fixed regardless: the **hardcoded Windows Oracle client path** + and the **PostgreSQL/MySQL destinations that ship without their drivers**. From 3ce48fe91f37e2ab4d61853e9018bc031cf78e1f Mon Sep 17 00:00:00 2001 From: Badry Darkoush Date: Thu, 3 Sep 2026 17:04:07 +0000 Subject: [PATCH 2/2] docs: add module-by-module consolidation plan Records which implementation wins per module across both codebases, with the reasoning, and recalibrates the phase estimates for AI-assisted development where review throughput rather than code production is the binding constraint. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01L6oCRzNahV9djmgUtpNe9q --- docs/consolidation-plan.md | 202 +++++++++++++++++++++++++++++++++++++ 1 file changed, 202 insertions(+) create mode 100644 docs/consolidation-plan.md diff --git a/docs/consolidation-plan.md b/docs/consolidation-plan.md new file mode 100644 index 0000000..e3de8ae --- /dev/null +++ b/docs/consolidation-plan.md @@ -0,0 +1,202 @@ +# Consolidation Plan — Module-by-Module Adoption + +**Companion to** `docs/tech-stack-comparison.md`. That document answers *whether* to combine. +This one answers *what gets adopted from where*, at implementation level, and *how long it takes +under AI-assisted development*. + +**Rule applied throughout:** take the better implementation regardless of which repo it lives in. +The verdict is per module, not per project. Neither codebase wins across the board. + +--- + +## 1. Backend — module-by-module + +### 1.1 Foundations (QueryGateway wins outright) + +| Module | Adopt | Why | Action for IntakeGateway | +|---|---|---|---| +| **Declarative base / PK strategy** | **QG** `models/base.py` — `UUIDPrimaryKeyMixin`, `TimestampMixin` with `server_default=func.now()` + `onupdate` | IG uses integer autoincrement plus an `ID_TYPE` shim that exists only to paper over SQLite. UUIDs are correct when rows are minted by API, worker, and scheduler processes independently. | Migrate all IG tables to UUID PKs. Delete `db/types.py`. | +| **Repository layer** | **QG** `repositories/base.py` — `BaseCrudRepository[ModelT]` (PEP 695 generics), `__init_subclass__` validation that fails at class-definition time, mapper-introspection allow-list on `update()`, `_IMMUTABLE_AUDIT_FIELDS` | Genuinely good code. The `inspect(self.model).column_attrs` allow-list is materially safer than a `hasattr` check, and the comment explains why. IG has **no repository layer** — routes touch `Session` directly. | Build IG's entities on this base. ~6 new repositories. | +| **Engine / session** | **QG** `database.py` — async engine, `async_sessionmaker`, `expire_on_commit=False` | — | Port IG's sync `SessionLocal` usage. | +| **Global error handling** | **QG** `exceptions.py` — three handlers; `jsonable_encoder` on 422 so a `ValueError` in a validator can't turn a 422 into a 500; generic 500 body with detail only in logs | IG has **no global handlers**. Unhandled exceptions leak framework detail. | Register QG's three handlers on the intake app. | +| **Request correlation** | **QG** `middleware.py` — pure-ASGI (not `BaseHTTPMiddleware`), request-ID allow-list regex, 64-char cap matching the DB column | The docstring records why `BaseHTTPMiddleware` was rejected: it wraps `call_next` in a new task, which breaks task-bound asyncpg connections. That is hard-won knowledge; do not re-derive it. | Adopt as-is. | +| **Migrations** | **QG** — Alembic only, gated by the one-shot `migrate` compose service | IG runs `create_all()` in lifespan **and** ships four Alembic revisions. | Delete `init_app_database()`. Merge revision graphs (see §3.3). | +| **Container topology** | **QG** — `edge`/`backend` network split, fixed `10.31.0.0/24` so `FORWARDED_ALLOW_IPS` scopes to exactly the nginx hop, API unpublished, data tier on loopback | IG's compose bind-mounts source with `--reload`. | Adopt QG's; add `worker` + `redis` services to it. | + +### 1.2 Configuration and secrets (split verdict) + +| Module | Adopt | Detail | +|---|---|---| +| **Settings class** | **QG's shape, IG's content** | Take QG's `SettingsConfigDict`, lower_snake naming, and — importantly — its **fail-fast validators that actually probe**: `_validate_bcrypt_hash` runs `bcrypt.checkpw(b"probe", value)` at startup so a malformed hash is a boot failure rather than a 500 on every login; `_validate_fernet_key` constructs a `Fernet` for the same reason; `_reject_wildcard_origin` refuses to boot with `CORS_ORIGINS=*` under `allow_credentials=True`. Then carry over IG's operational settings: `HTTP_TIMEOUT_SECONDS`, `HTTP_MAX_RESPONSE_MB`, `OAUTH_TOKEN_REFRESH_SKEW_SECONDS`, `HTTP_RETRY_AFTER_MAX_SECONDS`, `BACKFILL_MAX_WINDOW_DAYS`, `ALLOWED_SOURCE_HOSTS`, `SCHEDULE_MAX_CONSECUTIVE_FAILURES`. | +| **Crypto** | **QG's `crypto.py` + IG's `rotate_key`** | QG's is 40 lines, no singleton, no dev fallback. IG's `EncryptionService` **generates a temporary key when `APP_ENV=dev-only`** — a footgun that silently produces undecryptable data. Drop it. But IG has `rotate_key(old, new, ciphertext)` and QG has **no key rotation at all**; a merged product storing credentials for three database types needs it. Port that one method onto QG's module. | +| **Log redaction** | **IG** | QG has nothing comparable. Keep IG's `mask_headers`, `_is_secret_header` (substring matching, so *user-configured* API-key header names get masked too), `_redact_url_for_log` (parser-based, drops query and userinfo), and `_redact_cursor` (length + SHA-256 prefix rather than the value). Re-implement on structlog processors. | + +### 1.3 Authentication (QueryGateway wins; keep one IG piece) + +| Module | Adopt | Detail | +|---|---|---| +| **Admin identity** | **QG** | JWT + bcrypt + `HTTPBearer(auto_error=False)`, router-level `dependencies=[Depends(get_current_admin)]`, constant-time username comparison (bcrypt runs even on username mismatch), and `verify_password` handling bcrypt 5.0's *raise* on >72-byte input in both directions. IG has no UI login and no user concept. | +| **Machine-to-machine token** | **IG** (keep, demoted) | IG's `hmac.compare_digest` static-token path is genuinely useful for CI and automation calling the intake API. Keep it as an **optional secondary credential** alongside JWT — not as the primary auth. | +| **Data-plane auth policies** | **QG** | `auth_method` per endpoint (Bearer/Basic/API key), `generate_api_key` with ~192 bits entropy, rotation endpoints. Extends naturally to intake webhooks later. | + +### 1.4 Database connectivity (merge required — neither is complete) + +| Concern | Adopt | Detail | +|---|---|---| +| **Where connections live** | **QG** (Postgres rows, Fernet `LargeBinary`, repository + service) | This **deletes `connection_storage.py` entirely** — the encrypted file, the fcntl/msvcrt dual-platform locking, the `_file_lock` context manager, and the "refuse to run on platforms without a locking primitive" branch. All of it exists solely because IG's three processes share no transactional store. Best single code-deletion win in the merge. | +| **What a connection can be** | **IG** (`db_type` discriminator) | Generalize QG's `OracleConnection` → `Connection` with `db_type ∈ {oracle, postgresql, mysql}`, keeping QG's `pool_min/max/timeout`, `query_timeout`, `mode`, `is_active`, and the service-name/SID exclusivity rule. **Ship `psycopg2-binary` and `PyMySQL`** — IG builds those URLs today but neither driver is in any requirements file (see companion doc §6 #2). | +| **Engine caching** | **IG** (`connection_pool.py` engine/session-factory cache) | QG opens a fresh `oracledb.connect` per query. IG's per-connection engine cache is the better pattern for a system that will now serve both planes. Rebuild it on the DB-backed registry. | +| **Oracle client init** | **QG** | `ORACLE_CLIENT_LIB_DIR`, called once in lifespan, logs a warning rather than crashing on failure. IG hardcodes `C:\oracle\instantclient_23_0` at `connection_pool.py:36` — Windows-only, so thick mode is dead in the Linux containers it ships. Straight replacement. | +| **Driver version** | **QG** (`oracledb>=4.0.2,<5`, lazy import, isolated `requirements-oracle.txt`) | IG pins 3.4.2 and imports eagerly. QG's image already bundles Instant Client 19.32, which satisfies 4.x thick mode. | + +### 1.5 SQL execution (clean split — read vs write) + +| Direction | Adopt | Detail | +|---|---|---| +| **Read path** | **QG** `sql/executor.py` + `sql/param_models.py` | Bind-parameters-only, `conn.call_timeout`, `anyio.to_thread.run_sync` so the blocking driver never touches the event loop, and `build_param_model()` constructing a **dynamic Pydantic model per endpoint** so request params are typed and coerced before they reach the driver. IG has nothing comparable. | +| **Write path** | **IG** `runner.py` | `_build_insert_statement`, `_quote_column_name`, `_clean_identifier` (whitelist regex on identifiers), batched binds via `_rows_for_bind_aliases`, and `process_rows_with_upsert`'s bulk SELECT → bulk INSERT/UPDATE. QG is read-only; this is IG's core value. **Keep it synchronous inside a thread** using QG's own `anyio.to_thread` pattern — the driver blocks either way, and rewriting it to async buys nothing while risking the batching logic. | +| **Schema introspection** | **IG** `oracle_metadata.py` | `USER_TAB_COLUMNS` queries for mapping targets. Generalize per `db_type`. | + +### 1.6 Ingestion pipeline (IntakeGateway — unique, port with upgrades) + +| Module | Verdict | +|---|---| +| `api_connector.py` | **Keep.** Auth methods, fetch, header masking, Retry-After parsing. | +| `normalizer.py` | **Keep.** `jsonpath-ng` record selection + nested flattening. | +| `validator.py` | **Keep,** but the hand-rolled `ValidationError` class and `type_validators` dict duplicate what Pydantic does. **Consider rebuilding on QG's `build_param_model` approach** — one dynamic model per mapping instead of a validator registry. | +| `mapper.py` | **Keep.** Transform registry. | +| `transform_suggester.py` | **Keep.** Type-aware transform suggestions — good UX, no equivalent. | +| `runner.py` | **Keep** the pipeline and batching. Port logging to structlog. | +| `url_guard.py` | **Keep — and this is a gap in QG.** Scheme + resolved-IP validation, metadata-service (169.254.169.254) blocking, `ALLOWED_SOURCE_HOSTS` escape hatch, and an honest docstring about the residual DNS-rebinding window. The merged product needs this the moment it fetches any user-supplied URL. | +| `oauth_token_service.py` | **Keep, with a required fix.** Grant types, expiry skew, encrypted token cache. **But `_get_lock` returns an `asyncio.Lock` keyed by task_id, which serializes refreshes only within one process.** With multiple workers, concurrent refreshes race. Replace with a Postgres advisory lock or a Redis lock **during the port**, not after — the merge is when this becomes exploitable. | +| Rate-limit / cursor logic | **Keep.** Retry-After parsing, backoff caps, and watermarks that advance only on non-backfill non-replay success. Subtle and correct. | + +**Port condition:** these modules are the weakest-typed code in either repo — `def trim(x)`, bare `dict` returns, no annotations. They go into a **strict-mypy** codebase. Annotate on the way through; mypy strict will surface real bugs here. + +### 1.7 Scheduling and execution (split verdict — this is the interesting one) + +| Concern | Adopt | Detail | +|---|---|---| +| **Scheduler placement** | **QG** (in-lifespan APScheduler) | IG's separate `python app/services/scheduler.py` process exists because SQLite can't coordinate. Once there's a shared Postgres, it collapses into the API lifespan. | +| **Schedule semantics** | **QG** — decisively | `schedule_bindings.resolve_schedule_parameters` (fixed / SQL NULL / logical run date / relative date / calendar-window boundaries), IANA timezones, `preview_schedule_runs` showing the next three resolved runs, and a `_binding_hash` for change detection. IG is a raw cron string plus `croniter`. **Drop croniter** — APScheduler's `CronTrigger` covers it. | +| **Failure safety valve** | **IG** | Auto-pause after `SCHEDULE_MAX_CONSECUTIVE_FAILURES`, with the pause persisted *before* the job is removed (a fix visible in IG's history). QG has no equivalent. Real operational value — port it. | +| **Job queue** | **IG** — QG has none | Celery with `autoretry_for`, `retry_backoff`, `retry_backoff_max=600`, `retry_jitter`, and `on_failure`/`on_success`/`on_retry` hooks. **Snapshot refresh should move onto this queue too** — QG currently runs Oracle queries inside the API process, which is fine at low volume and not fine later. | +| **Queue technology** | **Decide empirically in Phase 0** | Celery 5.4 does not support 3.14. Options: upgrade Celery and validate; swap to `arq` (async-native, Redis, small — the natural fit for an async codebase); `dramatiq` for Celery-like semantics; or pin the worker to its own 3.12 image. **Test this on day one** (§3.1). | + +### 1.8 Observability (merge — neither is complete) + +| Concern | Adopt | +|---|---| +| **Log format** | **QG** — structlog JSON, contextvar binding, mandated `request_id`/`user`/`endpoint`/`status`/`duration_ms`/`method`/`client_ip`/`event` plus `job_id`/`run_id`/`row_count`/`success` on jobs. Port IG's loguru call sites; keep IG's redaction (§1.2). | +| **Run history** | **Merge.** QG's `JobRun` shape (UUID, status enum, structured, survives schedule deletion) + IG's counters (`rows_fetched/inserted/updated/skipped`, `error_count`, `warning_count`, `cursor_start/end`, `is_backfill`, `is_replay`, `replay_of_run_id`). One `Run` table, `kind ∈ {import, snapshot_refresh}`, union of columns. | +| **Row-level errors** | **IG** — `task_run_log` with bounded staging (`ROW_ERROR_FLUSH_SIZE = 1000`). Unique; the bound matters on large imports. | +| **Access audit** | **QG** — `access_logs`, deliberately no FK so rows survive endpoint deletion, written by middleware. Extend to the intake admin plane. | +| **Health** | **QG** — component probes (API/Postgres/Oracle/scheduler), recent job outcomes, stale-snapshot detection. IG returns `{status, env}`. Add queue depth and worker liveness. | + +### 1.9 Snapshot caching — QueryGateway only, keep entirely + +`compile_snapshot_filters`, `snapshot_covers_request`, `validate_snapshot_parameter_ranges`, +`filter_snapshot_rows`, `validate_snapshot_rows_match_resolved_parameters`. Coverage is checked +against **persisted job-run parameters** before cached rows are filtered — the correct design, and +subtle enough that it should not be touched during the merge. + +--- + +## 2. Frontend — module-by-module + +| Module | Adopt | Detail | +|---|---|---| +| **Design system** | **QG** — Tailwind + shadcn/Radix, `components/ui/*` | IG's ~5,800 LOC of antd gets rewritten. No automated path. | +| **Shell / routing / auth** | **QG** — `Layout`, `RequireAuth`, `AuthProvider`, `tokenStorage`, react-router 7 | IG has no login. | +| **API client** | **QG** — module-style `lib/api.ts` (290 LOC) + per-domain `types/*.ts` + `getApiError` | IG has a 302-LOC monolithic `ApiClient` class and a 442-LOC single `types/index.ts`. | +| **CRUD page pattern** | **QG** — `useResourceMutations` (107 LOC, replaces ~40 LOC of boilerplate per page) | Directly replaces IG's hand-rolled 484-LOC `hooks/api.ts`. | +| **Wizard architecture** | **QG's structure, IG's content** | QG: `EndpointWizard` 266 LOC + five step components + **pure logic extracted and unit-tested separately** (`bindParams.ts`, `parameterDefaults.ts`, `cronSchedule.ts`, `scheduleBindings.ts`). IG: **one 692-LOC `TaskWizard.tsx`**. Restructure IG's six steps into QG's step-component shape during the port — this is the single highest-value frontend refactor. | +| **Domain editors** | **IG** — `ColumnMappingEditor` (442 LOC), `UpsertConfigEditor`, transform-suggestion UI | Unique functionality. Rewrite presentation, preserve logic. | +| **SQL editor** | **QG** — CodeMirror 6 via `@uiw/react-codemirror` | — | +| **Build tool** | **IG — Vite 8** | IG is two majors ahead of QG's Vite 6. Take the newer one. | +| **Lint / format** | **QG** — ESLint 9 flat config, `typescript-eslint`, Prettier + `prettier-plugin-tailwindcss` | IG has ESLint 8 legacy config and **no formatter**. | +| **Coverage floor** | **IG** — `fail_under = 50` in `pyproject.toml` | QG reports coverage but sets no floor. Take IG's gate, then raise it. | +| **Client state** | **Neither** — drop `zustand` | react-query + context covers everything both apps do. | +| **Date libraries** | **Drop both** where possible | IG ships `date-fns` *and* `dayjs`; removing antd removes the dayjs requirement. | + +**Net frontend direction:** QueryGateway's architecture and tooling, IntakeGateway's domain +features and build tool. + +--- + +## 3. Revised timeline for AI-assisted development + +The companion document's 16–31 developer-weeks assumed hand-written code. That is the wrong unit +here. Under AI-assisted development, code production stops being the constraint and three other +things become it: + +1. **Review throughput.** Every diff still has to be read. This merge touches roughly 18–20k of the + ~30k combined LOC. At a sustainable 1,500–2,500 reviewed LOC/day that is **8–13 days on its own**, + and it does not compress. +2. **Validation wall-clock.** CI runs, integration tests against real Oracle and Postgres, a data + migration verified row-for-row, soak time on a new queue. Bounded by machines and by how long + you are willing to watch, not by typing speed. +3. **Irreducible empirical unknowns.** Does the queue run on 3.14? Does the merged Alembic baseline + apply cleanly to a populated database? Did the antd→shadcn port preserve every interaction? AI + drafts the answer in minutes; only running it settles it. + +### 3.1 Compression by work category + +| Category | Examples in this merge | Speedup | +|---|---|---| +| **Mechanical / pattern-following** | loguru→structlog call sites, route renames, config unification, sync→async signature churn, antd→shadcn page ports, repository scaffolding, type annotations on IG's pipeline, test generation | **8–15×** | +| **Design-carrying** | merged `Connection` entity, unified `Run` model, Alembic baseline strategy, queue selection, distributed OAuth lock | **2–3×** | +| **Empirical / wall-clock** | queue-on-3.14 validation, data-migration verification, visual QA, security review of merged auth, load testing | **~1×** | + +### 3.2 Revised estimate + +| Phase | Hand-written | **AI-assisted** | What actually gates it | +|---|---|---|---| +| **0. Monorepo + CI + queue spike** | 1–2 wks | **0.5–1 day** | Config only, no behavior change | +| **1. Identity + shell + config** | 3–5 wks | **2–3 days** | Review, plus a security pass on merged auth | +| **2. Data tier** | 4–8 wks | **3–5 days** | **Migration verification, not the code** | +| **3. Execution** | 4–8 wks | **3–6 days** | The queue answer from Phase 0 | +| **4. UI convergence** | 4–8 wks | **3–5 days** | Per-page visual QA — not skippable | +| **Total** | **16–31 wks** | **12–20 working days**, over **3–6 calendar weeks** | | + +**Phases 0–1 — one product, one login, one console — is 3–4 days.** + +### 3.3 What changes about the *approach*, not just the estimate + +**Resolve the queue question on day one, not in Phase 3.** It costs about an hour to settle +empirically: build a 3.14 image, install the candidate, run a trivial task. It determines whether +Phase 3 is a port or a rewrite — the largest single variance in the estimate. Under a multi-month +hand-written plan you could defer it; under a 12–20-day plan it goes first. + +**Write characterization tests before each port — this is what got cheap.** IG's pipeline has +7.4k LOC of tests but no type annotations, and it is about to be moved to async + structlog + +strict mypy. Generating a behavioral test suite against the *current* implementation before +touching it is now half a day's work, and it converts the riskiest phase into a mechanical one. +Previously this was the first thing cut for time. Don't cut it. + +**The phasing matters more with AI assistance, not less.** AI makes a big-bang merge feel +affordable. It isn't, for exactly the reason above: the gate is review, and a 20k-line +unreviewable diff closes it. Keep every phase independently shippable and independently revertible. + +**Plan for the Alembic merge as a design task, not a code task.** Two revision trees against one +`alembic_version` table is a data-loss shape. Decide deliberately between branch labels and a +squashed baseline, and verify the chosen path against a *populated* copy of the database. AI writes +the migration in minutes; the verification is the work. + +**Budget review capacity explicitly.** At 12–20 days of production, review is the binding +constraint on all but one phase. If review time is limited, the correct response is to lengthen the +calendar, not to shrink the phases. + +--- + +## 4. What to do first + +1. **Queue spike** (1 hr) — settle 3.14 compatibility empirically. +2. **Phase 0** (0.5–1 day) — monorepo, QG's CI matrix over both apps, mypy on IG in non-blocking + mode, Prettier added, no behavior change. +3. **Characterization tests** for IG's pipeline (0.5 day) — before anything is ported. +4. **Phase 1** (2–3 days) — one `Settings`, one JWT auth, one SPA shell, routes under + `/api/v1/admin/*`. + +That is roughly one week to a single product with one login, one console, and both feature sets +reachable — with the two highest-variance unknowns already retired.