diff --git a/CHANGELOG.md b/CHANGELOG.md index d2c1574..6dcf5a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,11 +68,11 @@ - Before a link session starts, the game's save is written and a copy of it is kept as `.sav.link1`; the two copies before it move to `.link2` and `.link3`. If the save cannot be written or copied, the session does - not start and the Link menu says why. -- Link > Restore pre-link save puts one of those copies back, chosen by its - date and time. The current save is kept as `.sav.bak`, and the game - restarts on the restored save. The item is not available during a - session. + not start and the Link Cable menu says why. +- Link Cable > Restore pre-link save puts one of those copies back, chosen + by its date and time. The current save is kept as `.sav.bak`, and + the game restarts on the restored save. The item is not available during + a session. - The runtime log records how every `.sav` was read (its size, the save chip chosen and whether they disagree), how long every save write took, whether it worked and how many tries it needed, how long the save copy diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4bee561..f59a25e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -84,10 +84,10 @@ The optional desktop launcher has its own `psp-build`, `package` and `artifact-smoke` use the release profile, the EBOOT players get, unless given `PROFILE=diagnostic`. The diagnostic profile adds the features that `source/src/build_profile.h` switches on (today the input -trace, Diagnostics > Pin a test toast, the `diag-autorun.txt` file that -`source/src/autorun_config.h` describes, the simulated network that its -`net_profile=` key puts on the ad-hoc link, and the rendering-off and -audio-off arms of Diagnostics > Unbound benchmark (docs/PERFORMANCE.md); +trace, Diagnostics > Pin a test toast, the Unbound benchmark rows with +rendering off and with audio off (docs/PERFORMANCE.md), the +`diag-autorun.txt` file that `source/src/autorun_config.h` describes, and the +simulated network that its `net_profile=` key puts on the ad-hoc link; `DIAGNOSTIC_CORE_PROFILE=1` adds the New core's profile and `DIAGNOSTIC_HW_PROBE=1` the memory system probe), is titled diff --git a/README.md b/README.md index b8d0373..7d34fac 100644 --- a/README.md +++ b/README.md @@ -252,9 +252,9 @@ in `source/`: the corresponding source ZIP, ### Performance comparison -On PSP, use **Diagnostics > Start measurement**, play a repeatable section, -stop measurement and save the report. Collect three reports per build, then -run: +On PSP, use **Diagnostics > Start recording**, play a repeatable section, +then **Stop recording** and **Save report**. Collect three reports per +build, then run: ```sh python3 scripts/compare-benchmarks.py \ diff --git a/benchmarks/README.md b/benchmarks/README.md index 51b1ab5..79d0749 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -14,9 +14,10 @@ benchmarks//-r//.txt - `-r` is the report's `game-code` and `rom-revision`, for example `BPRE-r0`. - `` is the report's `version`, for example `1.4.3`. -- `.txt` is a report saved from Diagnostics > Start measurement, - unchanged. Every run in one directory must share the settings - `scripts/compare-benchmarks.py` checks (`MATCH_FIELDS`). +- `.txt` is a report saved with Diagnostics > Start recording, Stop + recording and Save report, unchanged. Every run in one directory must + share the settings `scripts/compare-benchmarks.py` checks + (`MATCH_FIELDS`). Rules: diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 2347677..d8026bb 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -4,9 +4,10 @@ How GBA Rush PSP speed is measured, and the baselines measured so far. ## Comparing two builds -On the PSP, use Diagnostics > Start measurement, play a repeatable section, -stop and save the report. Capture at least three reports per build with the -same ROM, save or save state, renderer, clock and settings, then compare: +On the PSP, use Diagnostics > Start recording, play a repeatable section, +then Stop recording and Save report. Capture at least three reports per +build with the same ROM, save or save state, renderer, clock and settings, +then compare: ```sh python3 scripts/compare-benchmarks.py \ @@ -62,8 +63,9 @@ for the time per frame: - `phase-main-update-gba-us`: update_gba's own time (scheduler, input, presenting the frame, DMA), without the two phases below and without the - waits the work time leaves out (the VBlank waits and the GE wait). During - a lockstep link it also holds the link waits. + waits the work time leaves out (the VBlank waits, the sound sync's wait + for the audio thread and the GE wait). During a lockstep link it also + holds the link waits. - `phase-main-scanline-us`: update_scanline. - `phase-main-gbc-sound-us`: the PSG channels (update_gbc_sound). Direct sound, hundreds of calls a frame, is not timed. @@ -117,9 +119,11 @@ a capture is not counted. Diagnostics > Unbound benchmark loads the current save state slot as the slot menu does (a numbered slot backs up the progress it replaces to the auto slot first, and the load asks before replacing the game), then runs -3600 frames as fast as the PSP can: every frame is drawn, nothing waits -for the VBlank, VSync is off, and the sound is mixed but dropped, so the -emulation never waits for audio. The game gets no input. The capture then +3600 frames as fast as the PSP can: every frame is drawn (with Exact on +the Media Engine, a frame the Media Engine has not finished in time shows +the previous one again and counts as late), nothing waits for the VBlank, +VSync is off, and the sound is mixed but dropped, so the emulation never +waits for audio. The game gets no input. The capture then stops by itself and its report is saved to `benchmarks/`; the Diagnostics status row shows its name, and the game returns to its normal speed. A link refuses the benchmark. Opening the menu, a load or save state hotkey, diff --git a/docs/hardware-tests.md b/docs/hardware-tests.md index 1c8523b..dd4dab8 100644 --- a/docs/hardware-tests.md +++ b/docs/hardware-tests.md @@ -509,7 +509,7 @@ Each test has four fields: - Pass: `scripts/compare-benchmarks.py` reports no failure for any of the 5 games on either model. - Steps: for each of 5 games, take 3 measurements with Diagnostics > Start - measurement on the previous release and 3 on the candidate, alternating + recording on the previous release and 3 on the candidate, alternating the two builds, with the same save or state, renderer, clocks and settings. Compare each game's two sets of reports with `scripts/compare-benchmarks.py`. diff --git a/docs/wip/perf-method.md b/docs/wip/perf-method.md index ca9983a..999dafe 100644 --- a/docs/wip/perf-method.md +++ b/docs/wip/perf-method.md @@ -3,14 +3,14 @@ - **Branch:** `features/state-of-the-art-development/wip/perf-method`, staccato da `features/state-of-the-art-development/main`. - **Ticket:** PRF-1, PRF-2, PRF-3, PRF-8.6, PRF-10.1 (19 storie, circa 30 gg). - **File principali:** `scripts/compare-benchmarks.py`, `source/src/perf_*.c`, `tools/`. -- **Dipende da:** `wip/release-and-ci` (ENG-1.6, ENG-1.2, ENG-1.3, ENG-2.2), in main da 8ace097; restano `wip/platform` (PRF-4.1) e `wip/open-bios` (CMP-1.4) per PRF-1.5, `wip/replay-oracle` (ENG-3.2) per PRF-1.7. +- **Dipende da:** `wip/release-and-ci` (ENG-1.6, ENG-1.2, ENG-1.3, ENG-2.2), in main da 8ace097; `wip/platform` (PRF-4.1) in main da cf5ef4f; resta `wip/open-bios` (CMP-1.4) per PRF-1.5; per PRF-1.7 ENG-3.2 (`wip/replay-oracle`) è in main da 2ff83ab ma aspetta la prova su PSP. - **Nota:** PRF-8.6 e PRF-10.1 sono misure che servono a entrambi i core. - **Regole:** quelle di «Branch di lavoro dell'epica» in `docs/BACKLOG.md`. Il testo completo di ogni ticket (cosa, file, test, accettazione, licenza) è in `docs/BACKLOG.md` sotto il suo ID. Si spunta un ticket quando la sua accettazione è verificata; questo file si cancella nella PR che chiude il branch. - **Legenda:** «subito» = nessuna dipendenza, si può iniziare oggi; «aspetta» = dipendenze in altri branch, si parte quando sono in main. ## 1.5 -- [x] **PRF-2.1** — Tempo di lavoro per frame, frame in ritardo e oltre budget (report v7) · S, 2 gg · dipende da: ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_metrics e test_perf_report in make check e make test-sanitize, test_compare_benchmarks.py e check-performance.sh in make check, `psp_video.c` compilato nei due profili PSP_ +- [x] **PRF-2.1** — Tempo di lavoro per frame, frame in ritardo e oltre budget (report v7) · S, 2 gg · dipende da: ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_metrics e test_perf_report in make check e make test-sanitize, test_compare_benchmarks.py e check-performance.sh in make check, `psp_video.c` compilato nei due profili PSP; revisione del 2026-11-16 (25cd5a1): il sonno del sound sync in attesa del thread audio (frameskip manuale o spento) non conta più come lavoro, con una lettura di `sceKernelGetSystemTimeWide` in più in `sound.c`, solo a cattura attiva e nei frame che aspettano il thread audio: il proprietario conferma che la «Cosa» («usa solo tempi già misurati») lo ammette; la build PSP di `sound.c` cambiato si vede nella CI della PR di revisione_ - [ ] **PRF-8.6** — Suite SMC e di riferimento misurata sulla 1.4.3 · M, 3 gg · dipende da: — · PSP: sì · **subito** ## 1.6 @@ -19,19 +19,19 @@ - [x] **PRF-1.2** — Gate sugli estremi: p99, massimo robusto, work p95, frame oltre 1,5× budget al minuto · S, 1 gg · dipende da: PRF-1.1, PRF-2.1, ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_compare_benchmarks.py in make check_ - [x] **PRF-1.3** — Ordine alternato verificato, ora di cattura e affermazione dichiarata prima · S, 1 gg · dipende da: PRF-1.1, PRF-2.1, ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_report in make check e make test-sanitize, test_compare_benchmarks.py in make check, `main.c` compilato nei due profili PSP_ - [x] **PRF-1.4** — Archivio delle baseline e tabella per modello generata · M, 2 gg · dipende da: PRF-1.1 · PSP: no -- [ ] **PRF-1.5** — Sessione hardware: baseline 1.6 per PSP-1000 e uno slim · M, 2,5 gg · dipende da: PRF-1.3, PRF-1.4, PRF-2.2, PRF-2.3, PRF-3.1, PRF-4.1, CMP-1.4, ENG-1.6 · PSP: sì · aspetta: PRF-4.1 (`wip/platform`), CMP-1.4 (`wip/open-bios`) · _anche PRF-2.3, ferma per la decisione (m)_ +- [ ] **PRF-1.5** — Sessione hardware: baseline 1.6 per PSP-1000 e uno slim · M, 2,5 gg · dipende da: PRF-1.3, PRF-1.4, PRF-2.2, PRF-2.3, PRF-3.1, PRF-4.1, CMP-1.4, ENG-1.6 · PSP: sì · aspetta: CMP-1.4 (`wip/open-bios`); PRF-4.1 è in main (cf5ef4f) · _anche PRF-2.3, ferma per la decisione (m)_ - [ ] **PRF-1.6** — Controllo delle prestazioni nel gate di release · S, 1 gg · dipende da: PRF-1.4, F-139, F-118 · PSP: no · _in parte: CI verde su 8d7bd38 (run 43664592279); i report si legano all'rc col `build-id` del manifest, non con lo sha256 di EBOOT.PBP (i report non lo scrivono), e il controllo vale solo per le release che richiedono `perf-baseline-5`: il proprietario decide se basta_ - [ ] **PRF-2.2** — Timer di fase su frame campionati: update_gba, update_scanline, PSG audio · M, 2,5 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: no · _in parte: CI verde su 8d7bd38 (run 43664592279); a cattura spenta, oltre al test di flag per scanline, update_gba ne fa due per chiamata (ingresso e uscita) e il PSG uno per aggiornamento: il proprietario decide se l'accettazione («un solo test di flag per scanline») lo ammette_ - [ ] **PRF-2.3** — Contatori di dispatch per regione e di SMC · S, 1 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: no · _ferma per la decisione (m): i contatori stanno nel core Classic (`cpu.c`); il proprietario sceglie se riportarla al core New (`arm_mips`) o congelarla fino a SOA-10.1_ - [x] **PRF-2.4** — Prelazioni del thread di emulazione, divise tra lavoro e attesa · S, 1 gg · dipende da: PRF-2.1 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_metrics e test_perf_report in make check e make test-sanitize, `main.c` compilato nei due profili PSP_ - [ ] **PRF-2.5** — Sonda del costo del lookup (solo build diagnostica) · S, 1 gg · dipende da: PRF-2.3, F-137 · PSP: sì · _ferma per la decisione (m): cronometra il lookup del core Classic per il gate di PRF-7, congelato; il proprietario sceglie se congelarla fino a SOA-10.1 o riportarla al dispatch del core New_ - [ ] **PRF-2.6** — Validazione su hardware della strumentazione · S, 1,5 gg · dipende da: PRF-2.2, PRF-2.4, PRF-3.2 · PSP: sì -- [ ] **PRF-3.1** — Benchmark senza limite da uno slot di stato · M, 2 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: sì · _codice pronto (e108fdd), manca la prova su PSP_ -- [ ] **PRF-3.2** — Bracci render-off e audio-off (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _codice pronto (1a411a1), manca la prova su PSP_ -- [ ] **PRF-3.3** — Replay di traccia nel bench (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _codice pronto (c4098c0), manca la prova su PSP; lo stesso stato si verifica con l'hash di F-120 (ENG-3, `wip/replay-oracle`), non ancora in main_ +- [ ] **PRF-3.1** — Benchmark senza limite da uno slot di stato · M, 2 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: sì · _codice pronto (e108fdd), manca la prova su PSP; nella prova anche un bench con Exact sul Media Engine, su un titolo leggero e su uno pesante: il bench non aspetta mai, e se il Media Engine resta indietro 300 frame di fila Exact passa al processore principale fino al riavvio dell'EBOOT, mentre il report scrive solo `renderer`; si vede avviando l'EBOOT, aprendo il gioco e lanciando subito il bench (che carica lo slot da sé), poi uscendo dall'EBOOT subito dopo il bench, nel runtime log (`python3 tools/decode_runtime_log.py`): `exact-renderer=cpu-too-slow` invece di `media-engine`, e i frame in ritardo nel secondo valore di `me-frames/frames-late/me-busy-us-per-frame` (gruppi 11 e 9 di LINK_COUNTERS, scritti all'uscita e contati dall'avvio dell'EBOOT: così il gruppo 9 tiene quasi solo i frame del bench)_ +- [ ] **PRF-3.2** — Bracci render-off e audio-off (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _in parte: i bracci sono pronti (1a411a1), ma in main il bench non calcola nessun hash dello stato (`input_diagnostic_trace_begin_bench` in `input.c` scarta quelli del replay), quindi nessuna build controlla ancora lo stato a fine bench; il codice degli hash di F-120 (ENG-3.3, 91bc1ef) è in main da 2ff83ab, ma ENG-3.3 aspetta la prova su PSP; l'hash a fine bench (decisione del 2026-11-16: uno all'ultimo frame del bench con traccia, senza la regione audio, nel perf report come `bench-state-hash`) è scritto in `wip/replay-oracle` (bcf3657) e arriva in main con la sua PR di revisione; poi i bracci si confrontano con la stessa traccia e lo stesso slot, audio-off compreso perché l'hash lascia fuori i campioni, e si fa la prova su PSP_ +- [ ] **PRF-3.3** — Replay di traccia nel bench (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _in parte: il replay della traccia è pronto (c4098c0), ma il bench non calcola nessun hash dello stato, quindi «lo stesso stato (hash F-120)» non si controlla ancora in main; il codice degli hash di F-120 (ENG-3.3, 91bc1ef) è in main da 2ff83ab, ma ENG-3.3 aspetta la prova su PSP; l'hash a fine bench (decisione del 2026-11-16: uno all'ultimo frame del bench con traccia, nel perf report come `bench-state-hash`) è scritto in `wip/replay-oracle` (bcf3657) e arriva in main con la sua PR di revisione; poi la prova su PSP_ - [ ] **PRF-3.4** — compare-benchmarks per i bench e rumore misurato su hardware · S, 1 gg · dipende da: PRF-3.1, PRF-1.1 · PSP: sì · _codice pronto (8ebfcb6), manca la prova su PSP: i 3+3 run sul PSP-1000 e il rumore in `docs/PERFORMANCE.md`_ -- [ ] **PRF-10.1** — Esperimento di padding (il gate) · S, 1,5 gg · dipende da: PRF-3.1, PRF-1.1, F-137 · PSP: sì · _codice pronto (3018116), manca la prova su PSP: 12 run per modello e la decisione in `docs/PERFORMANCE.md`; la build PSP con `PADDING_PROBE_BYTES` non è mai stata fatta, nemmeno in CI_ +- [ ] **PRF-10.1** — Esperimento di padding (il gate) · S, 1,5 gg · dipende da: PRF-3.1, PRF-1.1, F-137 · PSP: sì · _codice pronto (3018116), manca la prova su PSP: 12 run per modello e la decisione in `docs/PERFORMANCE.md`; la build PSP con `PADDING_PROBE_BYTES` non è mai stata fatta, nemmeno in CI; il confronto per cartelle salta i `-alloc.txt` delle build diagnostiche da e421190_ ## 2.0 -- [ ] **PRF-1.7** — Suite pubblica di almeno 40 scene per modello · M, 3 gg · dipende da: PRF-1.4, PRF-1.5, PRF-3.1, ENG-3.2 · PSP: sì · aspetta: ENG-3.2 (`wip/replay-oracle`) +- [ ] **PRF-1.7** — Suite pubblica di almeno 40 scene per modello · M, 3 gg · dipende da: PRF-1.4, PRF-1.5, PRF-3.1, ENG-3.2 · PSP: sì · aspetta: ENG-3.2 (`wip/replay-oracle`), in main da 2ff83ab ma non ancora provata su PSP diff --git a/scripts/compare-benchmarks.py b/scripts/compare-benchmarks.py index cdb2c3f..74061a5 100755 --- a/scripts/compare-benchmarks.py +++ b/scripts/compare-benchmarks.py @@ -656,6 +656,11 @@ def format_result(result): # The report fields that make a group (title x model) in directory mode. GROUP_FIELDS = ("psp-model", "game-code", "rom-revision") +# The files beside the reports that are no report: notes, the -core.txt +# profiles of a DIAGNOSTIC_CORE_PROFILE build and the -alloc.txt traces +# every diagnostic build writes. +NOT_REPORTS = (".md", "-core.txt", "-alloc.txt") + def group_name(key): """A group's name as the output prints it, such as 0/BPRE-r0.""" @@ -665,8 +670,7 @@ def group_name(key): def collect_groups(*directories): """The reports anywhere under the directories, keyed by GROUP_FIELDS. - Every file but the .md notes and the -core.txt profiles of a - DIAGNOSTIC_CORE_PROFILE build must be a stopped .txt report, every + Every file but NOT_REPORTS must be a stopped .txt report, every directory must hold at least one, and the runs of one group must all come from the same version. """ @@ -675,7 +679,7 @@ def collect_groups(*directories): for directory in map(pathlib.Path, directories): count = 0 for path in sorted(directory.rglob("*")): - if not path.is_file() or path.name.endswith((".md", "-core.txt")): + if not path.is_file() or path.name.endswith(NOT_REPORTS): continue name = path.as_posix() if path.suffix != ".txt": diff --git a/scripts/perf-table.py b/scripts/perf-table.py index 7f713b4..bf2b780 100755 --- a/scripts/perf-table.py +++ b/scripts/perf-table.py @@ -124,9 +124,10 @@ def summarize(reports): def collect(root): """The summarized groups of the archive, keyed (model, game, version). - Every file but the .md notes must be a .txt report three directories - down, and a group's reports must match the model, game and version - directories they are in. + Every file but the .md notes and the diagnostic files that + compare-benchmarks skips too (NOT_REPORTS) must be a .txt report + three directories down, and a group's reports must match the model, + game and version directories they are in. """ root = pathlib.Path(root) if not root.is_dir(): @@ -135,7 +136,7 @@ def collect(root): for path in root.rglob("*") if path.is_file()) groups = {} for name, path in files: - if name.endswith(".md"): + if name.endswith(compare_benchmarks.NOT_REPORTS): continue parts = name.split("/") if len(parts) != 4 or not name.endswith(".txt"): diff --git a/source/src/perf_metrics.c b/source/src/perf_metrics.c index 2c53e58..3d47c37 100644 --- a/source/src/perf_metrics.c +++ b/source/src/perf_metrics.c @@ -396,7 +396,8 @@ void perf_metrics_observe_memory_available(PerfMetrics *metrics, } /* Adds one VBlank wait to the count and total of its site and to the waits - of the current frame; an unknown site is ignored. */ + of the current frame; the audio drain, no VBlank wait, goes only to the + frame's waits, and an unknown site is ignored. */ void perf_metrics_record_wait(PerfMetrics *metrics, PerfMetricsWaitSite site, uint32_t wait_us) { @@ -420,6 +421,10 @@ void perf_metrics_record_wait(PerfMetrics *metrics, count = &metrics->vblank_audio_count; total = &metrics->vblank_audio_total_us; break; + case PERF_METRICS_WAIT_AUDIO_DRAIN: + metrics->frame_wait_us = perf_metrics_saturating_add( + metrics->frame_wait_us, wait_us); + return; default: return; } diff --git a/source/src/perf_metrics.h b/source/src/perf_metrics.h index 4f3810e..8a9c4d2 100644 --- a/source/src/perf_metrics.h +++ b/source/src/perf_metrics.h @@ -74,12 +74,15 @@ typedef struct } PerfMetricsHistogram; /* Where the emulator thread waited for the PSP VBlank: frame pacing in - synchronize(), before presenting a frame, or in the sound sync. */ + synchronize(), before presenting a frame, or in the sound sync. The + last site is no VBlank wait: the sound sync sleeping until the audio + thread takes a block, which only leaves the frame's work time. */ typedef enum { PERF_METRICS_WAIT_BACKPRESSURE, PERF_METRICS_WAIT_PRESENT, - PERF_METRICS_WAIT_AUDIO + PERF_METRICS_WAIT_AUDIO, + PERF_METRICS_WAIT_AUDIO_DRAIN } PerfMetricsWaitSite; /* What a phase timer measured. UPDATE_GBA is the whole call; the scanline diff --git a/source/src/sound.c b/source/src/sound.c index 1acf590..21afff3 100644 --- a/source/src/sound.c +++ b/source/src/sound.c @@ -1098,6 +1098,7 @@ void synchronize_sound(void) u64 started; u32 factor; u32 limit; + int drained = 0; __builtin_allegrex_sync(); if(sound_perf_underrun_pending) @@ -1153,8 +1154,13 @@ void synchronize_sound(void) perf_monitor_record_audio_recovery(); break; } + drained = 1; sceKernelDelayThread(100); } + // The loop slept until the audio thread took a block: not work time. + if(drained && perf_monitor_active()) + perf_monitor_record_wait(PERF_METRICS_WAIT_AUDIO_DRAIN, + (u32)(sceKernelGetSystemTimeWide() - started)); } } } diff --git a/tests/test_compare_benchmarks.py b/tests/test_compare_benchmarks.py index c96a14c..ec59cda 100644 --- a/tests/test_compare_benchmarks.py +++ b/tests/test_compare_benchmarks.py @@ -803,10 +803,15 @@ def test_group_mode_rejects_bad_input(self): baseline)[0], 2) self.assertIn("group 1/BPRE-r0: at least 3 reports", errors.getvalue()) - # The core profile a DIAGNOSTIC_CORE_PROFILE build writes next - # to each report is not a report. + # The core profile a DIAGNOSTIC_CORE_PROFILE build and the + # allocation trace a diagnostic build write next to each + # report are no reports. (pathlib.Path(baseline) / "BPRE-r0" / "1.4.3" / "run-0-core.txt").write_text("profile\n") + (pathlib.Path(baseline) / "BPRE-r0" / "1.4.3" / + "run-0-alloc.txt").write_text( + "alloc-trace entries=1 thread=emulator memory-hold-kib=0 " + "held-kib=0\nmalloc 64 caller=0x08804000 at=0x09000000\n") self.assertEqual(len(MODULE.collect_groups(baseline)[ ("1", "BPRE", "0")]), 3) diff --git a/tests/test_perf_metrics.c b/tests/test_perf_metrics.c index 954607b..6cbaf91 100644 --- a/tests/test_perf_metrics.c +++ b/tests/test_perf_metrics.c @@ -332,6 +332,28 @@ static void test_work_time_subtracts_waits(void) assert(result.frame_time_total_us == 26700); } +/* The sound sync's sleep until the audio thread takes a block leaves the + frame's work time like a wait, but it is no VBlank wait: no site counts + it, and the frame still counts as late. */ +static void test_audio_drain_leaves_work_only(void) +{ + PerfMetrics metrics; + PerfMetricsSnapshot result; + + perf_metrics_init(&metrics); + assert(perf_metrics_start(&metrics)); + perf_metrics_record_wait(&metrics, PERF_METRICS_WAIT_AUDIO, 500); + perf_metrics_record_wait(&metrics, PERF_METRICS_WAIT_AUDIO_DRAIN, 8000); + perf_metrics_record_frame(&metrics, 16700, 200, 0, 0); + result = snapshot(&metrics); + assert(result.work_time_total_us == 16700 - 500 - 8000 - 200); + assert(result.vblank_audio_count == 1); + assert(result.vblank_audio_total_us == 500); + assert(result.vblank_backpressure_count == 0); + assert(result.vblank_present_count == 0); + assert(result.frames_late == 1); +} + /* Work percentiles come from the work histogram in 250 us buckets: 18 frames of 8100 us work, one of 12100 and one of 20000 give p50 8250, p95 12250 and p99 20000, while the frame-time p95 is 16750. */ @@ -1050,6 +1072,7 @@ int main(void) test_saturating_counters(); test_wait_and_frame_probes(); test_work_time_subtracts_waits(); + test_audio_drain_leaves_work_only(); test_work_time_percentiles(); test_work_time_clamps_at_zero(); test_waits_reset_each_frame(); diff --git a/tests/test_perf_table.py b/tests/test_perf_table.py index b359c69..b5d5321 100644 --- a/tests/test_perf_table.py +++ b/tests/test_perf_table.py @@ -186,10 +186,13 @@ def test_misplaced_reports_rejected(self): MODULE.collect(archive(pathlib.Path(root), runs)) def test_empty_archive(self): - # benchmarks/README.md allows .md notes anywhere in the archive. + # benchmarks/README.md allows .md notes anywhere in the archive; + # the files a diagnostic build writes beside a report are skipped + # as compare-benchmarks skips them. notes = self.directory / "psp-1000" / "BPRE-r0" / "1.4.3" notes.mkdir(parents=True) - for path in (self.directory / "README.md", notes / "notes.md"): + for path in (self.directory / "README.md", notes / "notes.md", + notes / "run-1-core.txt", notes / "run-1-alloc.txt"): path.write_text("notes", encoding="utf-8") self.assertEqual(MODULE.render(MODULE.collect(self.directory)), diff --git a/tests/test_perf_video_sentinel.c b/tests/test_perf_video_sentinel.c index f03acb8..d82bf65 100644 --- a/tests/test_perf_video_sentinel.c +++ b/tests/test_perf_video_sentinel.c @@ -108,7 +108,7 @@ int main(void) "C frame_time_total_us==50000"); perf_monitor_stop(50000); - /* D: Diagnostics > Start measurement runs with the menu open, so the + /* D: Diagnostics > Start recording runs with the menu open, so the * backdrop's presents must stay out of the capture. */ video_power_resume(); /* picks blit_gu: option_screen_scale is 0 */ begin_session();