Skip to content

[DRAFT] Optimize quality metrics performance#4715

Open
alejoe91 wants to merge 8 commits into
SpikeInterface:mainfrom
alejoe91:reduce-quality-metrics-memory
Open

[DRAFT] Optimize quality metrics performance#4715
alejoe91 wants to merge 8 commits into
SpikeInterface:mainfrom
alejoe91:reduce-quality-metrics-memory

Conversation

@alejoe91

Copy link
Copy Markdown
Member

Depends on #4713

Optimizes speed and RAM usage (especially with lazy mode) for metrics computation by pre-extracting and sharing spike-vector arrays across metrics that need them using the tmp_data. This is especsially important in the context of lazy computation, as arrays need to be decompressed and loaded in memory.

Here is a benchmark with 20M spikes, amplitude_cutoff + amplitude_median + noise_cutoff computed together:

Backend Lazy Old compute time New compute time Speedup Old peak RSS Δ New peak RSS Δ
zarr True 11.28 s 5.46 s 2.07x 1118 MB 1118 MB
zarr False 2.63 s 2.31 s 1.14x 1197 MB 1198 MB
binary_folder True 2.91 s 2.62 s 1.11x 1116 MB 1114 MB
binary_folder False 2.62 s 2.14 s 1.22x 1193 MB 1194 MB

TODO:

  • change unit/metric loop in compute for metrics that loop through unit to gather unit-level data
  • test with fully lazy spike vector

@alejoe91 alejoe91 added performance Performance issues/improvements metrics Related to metrics module labels Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

metrics Related to metrics module performance Performance issues/improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant