-
Notifications
You must be signed in to change notification settings - Fork 0
v1.2.0 — Hardening & Performance (roadmap phases 0–5) #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
6619660
Phase 0.1/0.2: upgrade dependency pins, split out dev dependencies
claude 38646b7
Phase 0.3/0.4: add CI workflow and ruff/pytest configuration
claude 2dc846d
Phase 0.5: apply ruff lint fixes and formatting
claude 3fc5730
Phase 0.6: correct the license references to GPL-3.0
claude 9070be2
Phase 0.7: gate Render auto-deploy on passing CI checks
claude 4e37551
Phase 0.8: remove dead find_candidate_arrays and sync the stale docs
claude e493e29
Phase 1.1: neutralize spreadsheet formula injection on all four expor…
claude 1fd8a25
Phase 1.2/1.3: stop logging API-fetch URLs; malformed API JSONL is a 400
claude 4c8a49a
Phase 1.4: replace the outbound header rule with a real allowlist
claude c5291b2
Phase 1.5: add the missing hardening headers and tighten CSP
claude 773e886
Phase 1.6: fail fast on the dev SECRET_KEY in production; validate in…
claude b03c5a9
Phase 1.7: bound recursion depth in extract_table_data and handle Rec…
claude 0651651
Phase 1.8: run DNS on a shared bounded pool with admission control
claude ed60332
Phase 1.9: restrict API fetch to an allowlist of ports
claude a36b688
Phase 1.10: opt-in proxy-aware rate limiting
claude 7614f58
Phase 1.11-1.15: JSON error handlers, no-store, upload checks, cookie…
claude 4e285bc
Phase 2.1: gzip large text and JSON responses
claude dac58c6
Phase 2.2: cache static assets for a day behind versioned URLs
claude 0a9515f
Phase 2.3-2.10: exports, preview projection, client caps, topology guard
claude 8e1773b
Phase 2.3: record the confirming measurement for MAX_EXPORT_CELLS
claude b1f6a23
Phase 3.1-3.5: split process_json, add annotations, return preview_limit
claude 213da85
Phase 4.1-4.5, 4.7, 4.8: client features and the /health split
claude 20b3ef4
Phase 5.1-5.5: sync docs and memory, add DX tooling, bump to 1.2.0
claude b47ed46
Phase 1.11 follow-up: return JSON for CSRF rejections too
claude 0bc406c
Address Codex review: port allowlist, readiness check, tree cap, JSON…
claude a737165
Address CodeRabbit review: response leak, theme vars, a11y, CI hardening
claude 1ea1036
Fix import cycle and port-allowlist regression from review round 3
claude f5d0728
CI: use ref_name so the push and pull_request groups actually match
claude 4141121
CI: key concurrency on the source repo; fix MD022 in MEMORY.md
claude c92563c
fix: reject non-dict export rows before the CSV stream opens
claude eb1ec15
fix: honor Accept-Encoding quality values before gzipping
claude ce9b716
fix: enforce the API_FETCH_TIMEOUT / gunicorn --timeout invariant at …
claude 53b928f
fix: escape '<' in Markdown export cells
claude bc2a841
fix: move focus into the About modal and hand it back on close
claude 28ebba6
docs: correct three statements that contradicted the code, drop a dea…
claude 5dc42b2
ci: cover each commit once instead of cancelling a duplicate run
claude File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Copy to .env and adjust. Every value below is the built-in default, so an | ||
| # unset variable behaves exactly like the line shown. | ||
| # | ||
| # cp .env.example .env | ||
| # | ||
| # .env is gitignored. Never commit real secrets. | ||
|
|
||
| # --- Required in production -------------------------------------------------- | ||
|
|
||
| # Flask/CSRF signing key. The built-in development default is publicly known, so | ||
| # the app REFUSES TO START with it when APP_ENV=production. | ||
| # python -c "import secrets; print(secrets.token_hex(32))" | ||
| SECRET_KEY=dev-secret-key-change-in-production | ||
|
|
||
| # The single canonical production signal. Setting it to "production" enables the | ||
| # SECRET_KEY fail-fast, the Secure session cookie, and the rate-limit topology | ||
| # guard. No other spelling is accepted -- two names would let a deployment | ||
| # satisfy one gate and silently miss another. | ||
| # APP_ENV=production | ||
|
|
||
| # --- Deployment topology (see README "Deployment topology and rate limiting") -- | ||
|
|
||
| # Worker count. Single source of truth: gunicorn reads it natively and every | ||
| # documented start command passes --workers "$WEB_CONCURRENCY". Required under | ||
| # APP_ENV=production. | ||
| WEB_CONCURRENCY=1 | ||
|
|
||
| # Replica count. Invisible from inside the process, so it is declared here and | ||
| # must mirror render.yaml's numInstances. Required under APP_ENV=production. | ||
| APP_REPLICAS=1 | ||
|
|
||
| # Rate-limit counter storage. memory:// counters are process-local, so the | ||
| # effective limit is multiplied by workers x replicas. Anything above 1x1 | ||
| # requires a shared backend (and requirements-redis.txt installed). | ||
| RATELIMIT_STORAGE_URI=memory:// | ||
|
|
||
| # Trust X-Forwarded-* from exactly one proxy hop. Enable ONLY when the app sits | ||
| # behind a proxy you control; otherwise clients can forge their rate-limit bucket. | ||
| TRUST_PROXY=0 | ||
|
|
||
| # --- Limits ------------------------------------------------------------------ | ||
|
|
||
| MAX_UPLOAD_SIZE=10485760 | ||
| PREVIEW_ROW_LIMIT=25 | ||
| FLATTEN_MAX_DEPTH=10 | ||
|
|
||
| # XLSX-only export budget, in cells (rows x columns). Derived from the | ||
| # measurement in docs/export-budget-v1.2.md. CSV/TSV are streamed and uncapped. | ||
| # 0 disables the guard. | ||
| MAX_EXPORT_CELLS=250000 | ||
|
|
||
| # --- API fetch --------------------------------------------------------------- | ||
|
|
||
| API_FETCH_TIMEOUT=30 | ||
| API_FETCH_MAX_RESPONSE=10485760 | ||
|
|
||
| # Ports the API-fetch feature may connect to. Empty disables the check. | ||
| API_ALLOWED_PORTS=80,443,8443 | ||
|
|
||
| # DNS admission control. API_DNS_TIMEOUT bounds how long a REQUEST waits, not how | ||
| # long the lookup runs -- getaddrinfo exposes no timeout and cannot be cancelled. | ||
| # API_DNS_MAX_WORKERS bounds concurrency, which is the actual starvation fix. | ||
| API_DNS_TIMEOUT=3 | ||
| API_DNS_MAX_WORKERS=4 | ||
| API_DNS_ADMISSION_TIMEOUT=1 | ||
|
|
||
| # --- Rate limits ------------------------------------------------------------- | ||
|
|
||
| RATE_LIMIT_DEFAULT=120/minute | ||
| RATE_LIMIT_PROCESS=30/minute | ||
| RATE_LIMIT_EXPORT=60/minute | ||
|
|
||
| # --- Misc -------------------------------------------------------------------- | ||
|
|
||
| # Cache lifetime for static assets. Safe because asset URLs carry ?v=APP_VERSION. | ||
| STATIC_MAX_AGE=86400 | ||
|
|
||
| # Responses below this size are not worth compressing. | ||
| GZIP_MIN_SIZE=1024 | ||
|
|
||
| # Set to 0 to omit the version from /health. | ||
| HEALTH_REVEAL_VERSION=1 | ||
|
|
||
| # Flask debug mode. Never enable in production. | ||
| FLASK_DEBUG=0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| name: CI | ||
|
|
||
| # Every commit is covered exactly once: a branch commit by its pull_request run, | ||
| # a main commit by its push run. | ||
| # | ||
| # `push: ["**"]` alongside `pull_request` ran BOTH on every push to a PR branch. | ||
| # The concurrency group below collapsed that pair, but collapsing means one of | ||
| # the two is cancelled -- and GitHub attaches a cancelled check run to the head | ||
| # commit, where it is not a success. So the PR reported `mergeable_state: | ||
| # unstable` on every push even with CI fully green, which is indistinguishable | ||
| # at a glance from a real failure. Not generating the duplicate beats cancelling | ||
| # it. | ||
| # | ||
| # The trade-off: a branch with no pull request open gets no CI. That is the | ||
| # intended shape -- the checks exist to gate the merge, and opening the PR is | ||
| # what starts them. | ||
| on: | ||
| push: | ||
| branches: ["main"] | ||
| pull_request: | ||
|
|
||
| # Nothing here touches the GitHub API beyond checkout, so the token needs no | ||
| # write scope; an explicit block also stops it inheriting wider repo defaults. | ||
| permissions: | ||
| contents: read | ||
|
|
||
| # Supersede a branch's own earlier run when a new commit lands on it: pushing | ||
| # three times in a minute should not leave three runs competing for runners to | ||
| # report on commits nobody is waiting for any more. | ||
| # | ||
| # This cancels only runs for SUPERSEDED commits, never the current head's -- | ||
| # the `on:` block above is what guarantees one run per commit, so there is no | ||
| # same-SHA pair left to collapse. A cancelled check run lands on the old commit, | ||
| # which is why this no longer costs the head commit its green state. | ||
| # | ||
| # The branch name alone is not unique across repositories: two pull requests | ||
| # opened from different forks on a branch both named `feature` would share a | ||
| # group, and cancel-in-progress would have them kill each other's checks. Keying | ||
| # on the source repo keeps forks apart. head_ref is the source branch on a | ||
| # pull_request and empty on a push; ref_name is the branch without the | ||
| # refs/heads/ prefix (github.ref would keep it). | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.head.repo.full_name || github.repository }}-${{ github.head_ref || github.ref_name }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
|
badry-dev marked this conversation as resolved.
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| with: | ||
| # Nothing after this step talks to GitHub, so leaving the token in | ||
| # .git/config only widens what a compromised dependency can reach. | ||
| persist-credentials: false | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.11" | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install -r requirements-dev.txt | ||
|
|
||
| - name: Lint | ||
| run: ruff check . | ||
|
|
||
| - name: Format check | ||
| run: ruff format --check . | ||
|
|
||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: "22" | ||
|
|
||
| - name: Tests | ||
| run: python -m pytest tests/ -v | ||
|
|
||
| - name: Client-side export assertions (F1) | ||
| run: node tests/js/test_export_sanitize.mjs | ||
|
|
||
| - name: Client-side render/cap assertions (P4/P5/P13) | ||
| run: node tests/js/test_render_caps.mjs | ||
|
|
||
| - name: Client-side feature assertions (Phase 4) | ||
| run: node tests/js/test_features.mjs | ||
|
|
||
| - name: Audit runtime dependencies | ||
| run: pip-audit -r requirements.txt | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.