Skip to content

Establish paired editor performance baseline without runtime changes #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
itlezy merged 2 commits into from
Nov 9, 2026
Merged

Establish paired editor performance baseline without runtime changes #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
97ed460
perf: add repeatable paired editor benchmark tooling
itlezy Nov 9, 2026
cf24515
perf: record published-build baseline and long-line timing concern
itlezy Nov 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions ROADMAP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,36 @@ packages portable x86/x64 ZIPs with source metadata and checksums. Version tags
publish releases after required checks pass; RC tags publish prereleases.
Legacy installer/signing scripts remain outside the portable release workflow.

The focused fixes and line clipboard shortcuts are published in
[v4.2.25.1040-rc.1](https://github.com/itlezy/notepad2-mod/releases/tag/v4.2.25.1040-rc.1).
Hosted CI passes all four configurations with pytest/pywinauto; both downloaded
release executables passed the full 94-test suite after checksum verification.

## Performance reference

The manual paired benchmark uses generated documents and isolated settings,
with no application instrumentation. See [#14](https://github.com/itlezy/notepad2-mod/issues/14),
the README reproduction command, and
[raw results](tests/baselines/published-1029-vs-1040.json).

Published x64 releases 1029 and 1040 were compared using seven measured runs
after warmup, rotated mode order, and a fixed viewport. Median loading times in
milliseconds (previous Original / current Original / current Dark):

| Plain-text workload | Previous Original | Current Original | Current Dark |
| --- | --- | --- | --- |
| 128 KiB | 58.5 | 59.0 | 59.8 |
| 25 MiB | 350.0 | 349.0 | 349.4 |
| 1 MiB single line | 121.3 | 121.5 | 121.8 |

Private memory is close across releases and sampled idle CPU medians are zero.
The extreme single-line case remains expensive: 100 edit/undo pairs took roughly
10.9 / 11.2 / 11.2 seconds. Its 40-move scroll/repaint median increased from
276.1 ms to 349.5 ms in current Original and 357.1 ms in Dark; this is an open
investigation, not an established root cause or approval for engine changes.
The data includes desktop/automation overhead and is not a cold-start,
syntax-heavy, physical input latency, or universal performance guarantee.

## Easy light/dark switching

Track implementation in [#6: native Light/Dark editor themes](https://github.com/itlezy/notepad2-mod/issues/6).
Expand Down
30 changes: 30 additions & 0 deletions Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,3 +181,33 @@ Use `--configuration Debug` for the corresponding native helper runtime. A
pre-fix binary can be checked with `-m pytest tests/test_save_prompts.py
--legacy-save-behavior --executable <path>` to reproduce the nine loss paths.
These tests do not replace visual review or establish large-file performance.

## Performance baseline

Run the manual Python benchmark separately from normal tests. It uses pywinauto
and psutil as test-only tools; no instrumentation runs inside the editor.

```powershell
.\.venv\Scripts\python.exe -m pip install --require-hashes -r tests/benchmark-requirements.txt
.\.venv\Scripts\python.exe tests/benchmark_editor.py --baseline <previous-release-exe> --candidate <candidate-exe> --output bin/performance/session/results.json
```

Use published x64 executables on the same machine and avoid other benchmarks or
desktop interaction while it runs. It displays generated plain-text documents
in a fixed viewport, with isolated settings and no clipboard access. Large-file
confirmation is disabled only in those temporary settings. Existing installations,
user documents, preferences and associations are untouched.

The default seven runs follow a discarded warmup for each mode/workload and rotate
the order of baseline Original, candidate Original, and candidate Dark. Workloads
are 128 KiB text, 25 MiB text, and a 1 MiB single line. Results include raw samples,
medians, spread, executable/workload hashes and machine details. Metrics cover
loading, initial repaint, literal search, 100 edit/undo pairs, 40 scroll/repaint
moves, process memory and a one-second idle CPU sample. The single-line workload
can take several minutes. Use a fresh output directory; generated fixtures remain
beside the report as build artifacts.

This measures warm-cache command/paint timings including automation overhead.
It does not establish cold-start, syntax-heavy or physical typing latency.
Occlusion, display scaling, scanner activity and machine load affect results;
there is deliberately no automatic performance pass/fail threshold.
Loading