Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ PREVIEW_ROW_LIMIT=25
FLATTEN_MAX_DEPTH=10

# XLSX-only export budget, in cells (rows x columns). Derived from the
# measurement in docs/export-budget-v1.2.md. CSV/TSV are streamed and uncapped.
# measurement in docs/plan-status-v1.2.md, Appendix A. CSV/TSV are streamed and
# uncapped.
# 0 disables the guard.
MAX_EXPORT_CELLS=250000

Expand Down
9 changes: 5 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

## [1.2.0] - 2026-08-21 — "Hardening & Performance"

Implements `docs/roadmap-v1.2.md`, closing every finding in
`docs/security-review-v1.2.md` (F1–F17) and `docs/performance-review-v1.2.md`
(P1–P13).
Implements the v1.2 roadmap, closing every security finding (F1–F17) and
performance finding (P1–P13). Those four planning documents were later
consolidated into `docs/plan-status-v1.2.md`; their full text remains in git
history at `065883f`.

No response key changed name, type or meaning; `/process` only gained keys.

Expand Down Expand Up @@ -68,7 +69,7 @@ No response key changed name, type or meaning; `/process` only gained keys.
- **Diskless, memory-bounded exports (P3).** CSV/TSV are generator-streamed and
uncapped. XLSX keeps a normal-mode workbook (no OS temp files) plus
`MAX_EXPORT_CELLS`, measured rather than guessed — see
`docs/export-budget-v1.2.md`.
`docs/plan-status-v1.2.md`, Appendix A.
- **Lazy tree picker (P4)** and **client render caps (P5)**: children build on
first toggle; nested objects stop at 20 keys, primitive arrays at 20 items,
strings at 500 characters.
Expand Down
16 changes: 7 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,9 @@ json-table-tool/
│ ├── test_render_caps.mjs
│ └── test_features.mjs
├── docs/
│ ├── code-health-final.md
│ ├── security-review-v1.2.md
│ ├── performance-review-v1.2.md
│ ├── roadmap-v1.2.md
│ └── export-budget-v1.2.md # How MAX_EXPORT_CELLS was measured
│ └── plan-status-v1.2.md # The planning doc: what shipped, what is pending,
│ # the export-budget measurement (App. A) and the
│ # perf budget + RSS protocol (App. B)
├── .github/workflows/ci.yml # lint, format, tests, JS assertions, pip-audit
├── pyproject.toml # ruff + pytest configuration
├── requirements.txt # Runtime dependencies (exact-pinned)
Expand Down Expand Up @@ -317,7 +315,7 @@ convention is a dedicated bump commit, not a bump ridden along with a feature.

### Re-deriving the Excel export budget
`MAX_EXPORT_CELLS` is a measured number, not a chosen one. Follow the method in
`docs/export-budget-v1.2.md` (fresh process per measured request, `ru_maxrss`
converted for the platform, delta across two runs), re-fit against the narrowest
aspect ratio you care about, and re-run a confirming point at the value you
intend to ship. Record the new data in that file.
`docs/plan-status-v1.2.md` Appendix A (fresh process per measured request,
`ru_maxrss` converted for the platform, delta across two runs), re-fit against
the narrowest aspect ratio you care about, and re-run a confirming point at the
value you intend to ship. Record the new data in that appendix.
2 changes: 1 addition & 1 deletion MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ Keep entries short — if it grows past ~10 lines, it probably belongs in `READM
### 2026-08-21 — The XLSX export budget is measured, in cells, and on by default (area: performance)

**What:** `MAX_EXPORT_CELLS` (default 250,000) caps Excel exports only. CSV/TSV are generator-streamed and stay uncapped.
**Why:** openpyxl memory tracks `rows × columns`, not rows — at equal cell counts a narrow, tall sheet costs *more* (84.3 MiB at 50k×3 vs 73.6 at 15k×10), so a row limit says almost nothing about the footprint. The default comes from the measurement in `docs/export-budget-v1.2.md`, not from feel. An unlimited default would leave a High finding unmitigated; uncapped CSV/TSV is what keeps the export contract as wide as the input contract.
**Why:** openpyxl memory tracks `rows × columns`, not rows — at equal cell counts a narrow, tall sheet costs *more* (84.3 MiB at 50k×3 vs 73.6 at 15k×10), so a row limit says almost nothing about the footprint. The default comes from the measurement in `docs/plan-status-v1.2.md` (Appendix A), not from feel. An unlimited default would leave a High finding unmitigated; uncapped CSV/TSV is what keeps the export contract as wide as the input contract.
**How to apply:** Re-derive the number whenever the measurement is re-run — do not round it to something tidy. Never truncate an oversized export: `/process` advertises `total_cells`/`max_export_cells` so the client greys Excel out beforehand, and `/export-xlsx` returns 400 for direct callers. And keep exports diskless: openpyxl's `write_only` mode writes worksheet parts to OS temp files, and `SpooledTemporaryFile` is either pointless (its default `max_size=0` never rolls over) or disk-backed.

**Owner / source:** performance review P3, decision D6.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -559,7 +559,7 @@ need no server round trip and nothing is persisted — responses over
`GZIP_MIN_SIZE` are gzipped to keep that affordable. The preview rows are a
truncated *copy*, so exports keep full fidelity. Excel exports are bounded by
`MAX_EXPORT_CELLS`, measured rather than guessed
(`docs/export-budget-v1.2.md`); CSV and TSV stream and stay uncapped. The JSON
(`docs/plan-status-v1.2.md`, Appendix A); CSV and TSV stream and stay uncapped. The JSON
tree picker builds children only when a node is opened.

---
Expand Down
10 changes: 5 additions & 5 deletions config.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
# Publicly known, and therefore only ever acceptable outside production.
DEV_SECRET_KEY = 'dev-secret-key-change-in-production'

# See MAX_EXPORT_CELLS below and docs/export-budget-v1.2.md for how this number
# was measured.
# See MAX_EXPORT_CELLS below and docs/plan-status-v1.2.md (Appendix A) for how
# this number was measured.
DEFAULT_MAX_EXPORT_CELLS = 250_000


Expand Down Expand Up @@ -164,9 +164,9 @@ class Config:
# 500 columns have wildly different footprints at the same row count.
#
# Enabled by default: an unlimited default would leave P3 (High) unmitigated.
# The value is derived from the Performance Review section 4 measurement (see
# docs/export-budget-v1.2.md), not chosen by feel -- re-derive it whenever that
# measurement is re-run. 0 disables the guard for operators who knowingly opt
# The value is derived from the measurement in docs/plan-status-v1.2.md
# (Appendix A), not chosen by feel -- re-derive it whenever that measurement
# is re-run. 0 disables the guard for operators who knowingly opt
# out. CSV/TSV stay uncapped and streamed, so every dataset /process accepts
# remains exportable by some route.
MAX_EXPORT_CELLS = env_int('MAX_EXPORT_CELLS', DEFAULT_MAX_EXPORT_CELLS)
Expand Down
Loading
Loading