diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e4f56c..5e6c0e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,309 @@ This changelog starts at version 1.4.0. For earlier versions see the ## Unreleased +### `SelectionGUI` is now the automatic feature selection interface + +There is one point-selection window again. `SelectionGUI` names the interface +built on `pyidi.selection`, described in the next section; the window it named +in 1.3 is `SelectionGUIOld`, deprecated and removed in 1.5. + +Two windows offering the same five tools was never the plan — the new one was +built alongside the old one so the old one would keep working while it was +written. It turned out to be a superset rather than a companion: every +selection method has a counterpart, and both filters are evaluators. + +| `SelectionGUIOld` | `SelectionGUI` | +| --- | --- | +| Grid | Polygon with role `points`, or the `lattice` selector | +| Manual | Points tool | +| Along the line | Line tool | +| Brush | Brush tool | +| Remove point | Remove point tool | +| Shi-Tomasi filter | `shi_tomasi` evaluator | +| Gradient in direction | `gradient_direction` evaluator | + +**Most scripts need no edit.** The constructor takes the same arguments and +`get_points()` returns the same `(n_points, 2)` array in `(row, col)` order, so +`SelectionGUI(video, subset_size=21)` followed by `set_points(gui)` keeps +working and opens the better window. + +What does not carry over, for anything that reached deeper: + +- **`get_filtered_points()` and `get_selected_points()` are gone.** There is one + `get_points()`. Filtering is no longer a second pass over a selection that + already exists — it is the selection. +- **The internal attributes have no counterpart** — `selections`, + `subset_size_spinbox`, `candidate_points` and the rest. +- **`Grid` is not a mode.** Draw a polygon and set its row to the `points` role, + or keep it a mask and choose the `lattice` selector. +- **Scores near the image border differ slightly**, because gradients are taken + over real neighbours instead of ones reflected at the subset edge. The new + value is the correct one. + +`SelectionGUIOld` prints a `DeprecationWarning` on construction and is otherwise +unchanged. `FeatureSelectionGUI`, the working name used while this was being +built, never appeared in a release; the name raises a `RuntimeError` saying +where it went. + +### The active tab and tool are visible + +Both groups of buttons -- the two tabs across the top, and the region tools -- +were left to the platform theme to mark, and a default theme separates a +checked `QPushButton` from an unchecked one by a shade or two. That is not a +difference you can find across a panel, and this interface asks the question +twice: which tab am I on, and which tool is active. A checked button is now +filled and bold, in the blue the selections list already highlights with. + +Only the checked state is styled. Everything else stays whatever the platform +theme makes it, so the window does not have to carry a theme of its own to have +a legible one -- which is what the older window did, and why it looks dated on +a modern desktop. + +### Erasing is a tool, not a mode the brush is in + +`Remove w/ brush` joins the mask tools, next to `Remove point`. It replaces the +`Deselect painted area` toggle that used to sit inside the brush controls, +which made painting a mode within a mode: the same brush added or subtracted +depending on a checkable button several rows below it, and which one it was +about to do was not visible where the work happened. + +The two tools that take away are now side by side, at the two scales they +work at -- one point, or everything a stroke covers. `Remove w/ brush` is the +brush in reverse and shares its radius, so a stroke erases exactly as wide as +it paints. What it does is unchanged: it subtracts only the part actually +painted over, a region keeps whatever the stroke missed, and it disappears +only once nothing of it is left. + +Two controls on that tab also stop being shown when nothing can act on them. +`Brush radius` appears for the tools that paint, and `Point spacing` for a row +that lays points out along or inside its shape -- a polygon, line or brush +stroke in the `points` role. Neither is read otherwise: a `mask` row has its +points chosen by the selection, so their distance is the separation on the +other tab, and a `points`-tool row is the coordinates that were clicked. This +follows what the selector's own settings already did, appearing and +disappearing with the selector rather than greying out. + +### Automatic feature selection + +The `pyidi.selection` package, and the interface over it, add automatic feature +selection: score the whole image, then pick the best-separated features inside +the region you drew, rather than placing subsets on a grid and discarding the +poor ones. On a random speckle pattern or an intricate structure this is the +difference between sampling where the features are and sampling where the grid +happens to fall. Implements the workflow discussed in +[issue #51](https://github.com/ladisk/pyidi/issues/51), using the +mask/evaluate/select vocabulary agreed there. + +The blocker was never the idea, it was the cost. `SelectionGUIOld` scores one +subset at a time — a Sobel and a 2x2 eigendecomposition per point — which is +fine for a few hundred grid points and takes minutes at one megapixel, so the +spacing control was really a compute budget. Shi-Tomasi is now computed as a +whole-image Sobel plus three box sums and a closed-form minimum eigenvalue: +the same quantity, a few separable passes, tens of milliseconds per megapixel. +Dense candidates therefore cost nothing and a region goes back to meaning +"where I want points". + +The three steps are separated, and only the middle one is expensive: + +- **mask** — regions define an *area*. Each row in the selections list carries + a role: `mask` rows contribute their area, `points` rows contribute + coordinates that bypass scoring entirely, and the role is switchable per row. + Hand-picked points always survive whatever their score, and no automatic + point is placed within the separation of one. +- **evaluate** — Shi-Tomasi and gradient-in-direction, computed over the whole + image, with `NaN` marking the border where the subset window would leave it. + Scores are named and cached on (evaluator, parameters, subset size), so + several coexist and switching between them is free the second time. + Evaluators are a registry: a new one is a function plus parameter + descriptors, and it appears in the menu with no GUI change. +- **select** — a threshold plus a **separation**: the distance no two points + may come closer than, which is the one control for how many you get. A bare + threshold on a dense score image returns a solid blob of adjacent pixels on + every corner, so the separation is what makes the result a set of features. + A `lattice` selector reproduces regular grid sampling inside the same + pipeline, for full-field work where even coverage matters more than feature + strength. + +The threshold is a **quality**: a fraction of the best feature in the region, +so 0.01 means "at least a hundredth as good as the best one here", on a +logarithmic slider because the useful settings span three decades. A percentile +ranks *pixels*, and on a dense score image the pixels are overwhelmingly +background — on a typical frame its 90th percentile is under a five-hundredth +of the best feature, so nine tenths of a percentile slider's travel sits inside +the featureless area and lowering it floods the frame with background rather +than admitting weaker features. The reference is the 99.9th percentile of the +scores rather than their maximum, so one specular highlight cannot drag every +useful setting into the floor of the slider. `percentile of scores` remains +available, and is the right rule for the `lattice` selector, whose candidates +are already spaced out. A third rule, a fraction of the literal maximum, was +offered and then dropped: it is quality with a reference one bright pixel can +move, so on any usable frame it is indistinguishable and on a bad one worse. + +**Decimation** thins the points that were already selected — every n-th, +survivors left exactly where they are — for when the selection is right and +only the count is too high for the computation about to run. Widening the +separation re-selects and moves every point, which is a different thing, so the +two are separate controls. Hand-placed points are never decimated, and what a +region selected is recorded as occupied before it is thinned, so decimating one +region leaves gaps rather than inviting another to fill them. + +Decimation is deliberately not the density control, because thinning the pixels +above the threshold that way does not work. On a 1024×1024 frame with 357 000 +of them, thinned to twenty thousand points, keeping every n-th in score order +leaves 78 % of the subsets within three pixels of another one and keeping every +n-th in scan order 92 %; the separation leaves none. Score order fails because +consecutive ranks are neighbours on the same feature, scan order because the +stride aliases against the row length. + +The **point cap** now says when it is what stopped the selection. It had no +other symptom — it simply stopped adding points, and since it keeps the +highest-scoring ones, a selection that hit it looked like a tight cluster on +the strongest features and read as though the threshold or the spacing had +caused it. + +Selection is fast enough to drive from a slider. The exact greedy walk is +linear in the candidates, and a loose threshold leaves hundreds of thousands +of them — 40 ms to 300 ms depending on the separation, which nothing can drag. +So the candidates are reduced first, to the best pixel in each cell of a grid +half the separation across. That costs yield and not the guarantee: at a +separation of 11 it finds 1708 points where the exact walk finds 2193, in 9 ms +instead of 39, and the walk still runs so the separation still holds exactly. + +The rest of the redraw was Python loops over the points rather than the +selection itself. Vectorised — deduplication through one `unique` over folded +coordinates, per-entry crediting through one mask lookup each, occupancy +through one indexed assignment — and with each entry's rasterisation cached +against a fingerprint of its geometry, the pipeline half of a redraw at 20 000 +points on a megapixel frame went from 46 ms to 21 ms. Earlier in the same work +a redraw stopped running the selection three times over. + +What is left is drawing, so redraws are **coalesced**: while one costs less +than a frame it still happens immediately, and above that the requests collapse +into a single deferred redraw carrying the latest values. A fast drag therefore +repaints as often as it can rather than queueing every position on the way, and +lands on the value the control stopped at. + +Three things then made every redraw more expensive than it had to be. A masked +selection ran over the whole frame, though nothing outside the mask was ever +eligible: it now runs inside the mask's bounding box, snapped back to a whole +reduction cell so the block grid falls where it would have and the answer is +identical. Every selected point was stamped into a full-frame occupancy array +so that a later group could not fill its gaps -- a Python loop over all of them, +84 ms at seventeen thousand points, and there is usually no later group, so it +is now skipped unless one is coming. And the seeded whole-image row was +re-rasterising each region from scratch to decide whether to stand down, +bypassing the cache that already had the answer. + +What is left of a redraw is drawing, and the two big point layers -- the +selected points and the dim blue candidates -- are now one stroked path each +rather than a `ScatterPlotItem`. The item keeps a record per spot and rebuilds +a symbol atlas, 17 ms for seventeen thousand points on every redraw; a path of +very short segments stroked with a round-cap pen draws the same dots from one +vectorised call, in 2 ms. Layers with a per-point colour, a symbol or a hover +behaviour still use the scatter item. + +Together, on a 2560x1600 frame at separation 6: placing a polygon corner over a +drawn region went from 115 ms to 30 ms, and a redraw with the whole frame +selected from 342 ms to 158 ms. + +A brush stroke updates nothing but itself. It used to cost a mouse move what a +full redraw costs -- a whole-frame RGBA overlay rebuilt and re-uploaded per dab, +and the entire point cloud handed back to the scatter item to take the covered +points out of it -- which is 24 ms a move at 17 000 points on a four-megapixel +frame, paid while the mouse is moving. The stroke is now a path of overlapping +discs rather than a raster, and the crossing-out is drawn *over* the red points +instead of replacing them, so a move costs the points it reached and nothing +else: 0.1 ms. The selection itself is re-run once, when the stroke lands. + +Because a mask or threshold edit only re-derives from a cached score, the +interface updates while a slider is still moving; only a subset-size or +evaluator change recomputes. + +The window presents this as **two** tabs, not three, and deliberately does not +number them. Evaluation does not depend on the mask, so mask and evaluate are +siblings feeding select rather than a sequence. The tabs are named for the +steps they hold: `Evaluate + select` holds the evaluator and the selection +controls together, since changing one changes what the other means; `Mask` is +where the candidates get trimmed. The selections list is on the `Mask` tab and +only there — every row in it, and every button under it, acts on something +drawn there — while the subset size stays on both, because both steps read it. The selections +list starts with a `Whole image` row, so points are on screen the moment the +window opens and masking is editing rather than a precondition. That row is +ordinary — uncheck it, paint it away, or delete it, and deleting it selects +nothing rather than reverting to the whole frame. Since mask rows combine as a +union, drawing your own region unchecks it, so the region restricts the +selection instead of being absorbed into a union that changes nothing. +`Clear all` starts over rather than clearing to nothing: it seeds that row +again, so the whole frame is selected, as when the window opened. + +While masking, the points are shown in three tiers, because "no point here" +otherwise means two different things: red for a point being taken, dim blue for +a feature the mask is leaving out, and a magenta ring for the points the +selected row accounts for. The dim tier is the whole-frame selection, so it +does not move while a mask is edited — painting a region turns points from blue +to red where it lands rather than re-selecting underneath you — and it is +cached across mask edits, which is what keeps it affordable: about a fifth +added to a redraw on that tab. + +The subset size takes odd values only, by stepping and by typing: a subset is +centred on its point, so an even extent has no centre to be, and the pipeline +reads one as the odd size below it in any case — a subset size of 10 scores +through an 11-pixel window and draws an 11-pixel rectangle — so the even values +were a second spelling of the odd ones. It sits below the tabs rather than on +one of them: the scoring window follows it, which is what makes it one of the few settings that stales +the cached score, and it is also the size of the rectangle drawn round each +point while masking. `Show score overlay` is on both tabs and the two controls +stay in step. Control groups are flat rather than nested, settings the current +selector does not read are hidden rather than greyed out, and the descriptive +paragraphs are tooltips -- between them they were most of a panel that was +narrow enough to clip its own values. A direction +can be dragged out on the image, as in `SelectionGUIOld`, as well as typed or +taken from the `X`/`Y` presets, and a line shows the direction in force. +Points under a deselect stroke are crossed out while the stroke is being +painted, rather than only disappearing once the mouse comes up. + +The pipeline imports without Qt and is usable from a script through +`pyidi.selection.select_points` (one call) or `SelectionPipeline` (keeps the +score cache alive across parameter changes). + +`Remove point` takes the point's whole reserved disc — its separation — out of +the mask, not the single pixel under the click. A selected point is re-derived +from the score on every run, so erasing one pixel just promotes its neighbour +and the point reappears a couple of pixels along. Removing one is undoable, as +every other edit is. Hand-placed points are still deleted outright, so clicking +the same pixel again puts one back. + +A click outside the image adds nothing. The view is always larger than the +frame — the aspect is locked, so one axis has a margin, and zooming out adds +more — and a subset centred off the frame cannot be tracked; any coordinate that +reaches the pipeline from elsewhere is dropped there too. + +Score images are kept in a bounded cache, eight deep. Each one is a full-frame +`float32`, 16 MB at 2560x1600, and every distinct set of evaluator parameters is +a different array, so a direction spin box dragged through sixty values would +otherwise hold sixty of them. The score overlay is redrawn as part of the +refresh, so it follows a change of subset size or of evaluator instead of going +stale. + +A deselect stroke touches only the regions it actually covers, rather than +giving every region on the list a frame-sized erasure array, and an undo +snapshot holds those arrays by reference rather than copying them into each of +the fifty slots. + +Ctrl is read off the mouse event rather than tracked by a key filter on the +window, which a panel widget with focus can swallow; letting go of Ctrl +part-way through a stroke finishes the stroke rather than abandoning it. + +Changing an evaluator parameter goes through the same coalescing as every other +control. It is the one control that can make a redraw expensive, since a +parameter not scored before is a whole-frame evaluation. + +**Scores differ slightly from the old per-subset filter near strong edges.** +`SelectionGUIOld` ran Sobel on the isolated subset, so gradients at the subset +border used values reflected from inside it; the whole-image version sees the +real neighbours. The new value is the correct one. This affects the new module +only — `SelectionGUIOld` is untouched and behaves exactly as before. + ### Example datasets `pyidi.datasets` downloads example recordings from Zenodo on first use and @@ -31,11 +334,287 @@ published this way has in common — a Zenodo record holding a Photron `cihx` header next to an uncompressed `mraw` file of fixed-size frames, so that a window can be addressed by byte offset. -### Fixes +### Documentation overhaul + +The documentation was restructured around the work done since 1.3.3. New +pages: Eulerian video magnification, reading a video (all supported formats, +including `.cine`, and the frame-rate caveats), results and reproducibility +(where analyses are saved, `load_analysis`, resuming, and what a `NaN` in the +result means), and an upgrading guide covering `SubsetSelection` -> +`SelectionGUI`, `use_numba` -> `use_compiled_kernel`, the stricter +`set_points()` contract, and the pre-1.0 `pyIDI` class. + +The methods page gained a "choosing a method" comparison, a parameter table +per method, and a section on prescribed rigid-body motion in +`DirectionalLucasKanade`. The mode-shape magnification and fiducial-marker +pages, previously stubs reading "more documentation is coming soon", now +document the actual API. `CHANGELOG.md` is rendered into the documentation. + +Sphinx gained `sphinx-design` (landing-page cards), `myst-parser` (Markdown), +`napoleon` (the Google-style docstrings in `fiducial.py` render correctly now) +and `intersphinx` (links into the numpy, scipy and Python documentation). The +build is warning-free. + +Two source-level fixes fell out of writing this: `ResultViewer` documented its +`displacements` argument as `(n_frames, n_points, 2)` when it indexes it as +`(n_points, n_frames, 2)` — the shape `get_displacements()` actually returns — +and `VideoReader.get_frame`'s docstring had a mis-indented field that broke +its rendering. The old documentation also claimed `load_analysis()` returns +two values; it returns three (`video, idi, settings`). + +### Eulerian video magnification + +New `EulerianMagnifier` class in `pyidi.postprocessing` (also available as a +functional `eulerian_magnification()` wrapper) adds linear Eulerian Video +Magnification (Wu et al., SIGGRAPH 2012) as a pre-test visualization tool: a +Laplacian pyramid decomposition, a temporal band-pass filter applied per +pyramid level, and linear amplification of the band-passed signal added back +onto the original. It reveals subtle, often sub-pixel motion directly in the +raw recording, before any displacement identification is run, so it is useful +for checking whether (and where) a structure is moving and for picking +regions of interest or seed points ahead of a full analysis. **This is +qualitative visualization only, not a measurement** - the amplification +distorts motion amplitudes non-linearly and must not be read as displacement. + +Configure with `freq_band=(low, high)` in Hz to isolate a suspected mode, +`amplification` for the gain, and `levels` for the pyramid depth. The +temporal filter is `filter_type="ideal"` (FFT brick-wall, default) or +`"butter"` (Butterworth). An optional 2D `mask` restricts amplification to a +region of interest, leaving the rest of the frame as recorded. `save()` +writes the result to mp4/avi/mov/gif, mapping the intensity range to 8-bit +for playback. + +The optional `lambda_c` spatial-wavelength attenuation, meant to damp +amplification of fine, noisy detail while leaving broad structural motion at +full gain, initially ramped in the wrong direction: the finest pyramid level +got the strongest amplification and the coarsest the weakest, the reverse of +what Wu et al. specify. This is now fixed to ramp from the coarsest band down +to the finest. A warning is also now raised if `lambda_c` ends up attenuating +every level to zero (the output would then equal the input unchanged), and +`save()` raises rather than silently defaulting to 30 fps when no frame rate +is available. The test suite was substantially hardened alongside these +fixes, including mutation-verified tests that would have caught a disabled +band-pass, a sign-inverted amplification, or a dropped pyramid band. + +### Rigid body motion in `DirectionalLucasKanade` + +`DirectionalLucasKanade` gained `set_rigid_body_motion(rbm_ij)`: a per-frame +`(n_time_points, 2)` array giving a known, prescribed rigid-body translation. +The tracking window for every point now follows this prescribed motion, and +the motion is subtracted back out of the result, so `self.displacements` +reports the local motion relative to the rigid body motion rather than each +point's absolute pixel position. Only the component of the rigid body motion +aligned with each point's tracking direction (`dij`) is currently supported. +If `set_rigid_body_motion` is never called, it defaults to zero and existing +analyses are unaffected. + +The same change also fixes the NumPy-path convergence check: `compute_delta` +(aliased as `compute_delta_numba`) returned a signed error, but the +optimizer's stopping test (`error < tol`) assumes a non-negative error, so +iterations could stop early on a spuriously negative error or fail to +converge. The error is now returned as its absolute value. + +### Fixed + +- **`import pyidi` failed outright when PyQt6 was installed without napari.** + `pyidi.GUIs` gated every class on PyQt6 alone and then imported the napari + `GUI` unconditionally, so `pip install pyqt6 pyqtgraph` without the `[qt]` + extra produced a `ModuleNotFoundError` from inside a submodule and took the + whole package down with it. Each class now checks its own dependencies: + `SelectionGUI`, `SelectionGUIOld`, `ResultViewer` and `Viewer` need PyQt6 and + pyqtgraph, the napari `GUI` needs napari and magicgui, and whichever are + unavailable become stubs that import cleanly and raise `RuntimeError` on + construction naming the missing packages. `Viewer` had no such stub at all + and raised `NameError`. +- **Asymmetric `pad` in `DirectionalLucasKanade` crashed every point.** + `_interpolate_reference` and `_warm_up_kernels` paired the `(pad_y, pad_x)` + axes in the opposite order to `_padded_slice`, so a non-square `pad` (e.g. + `configure(pad=(2, 5))`) built the reference spline over a grid of the + wrong shape. All three now use the same axis pairing. +- **A point already lost before a checkpoint could come back with garbage + displacements after resuming.** `failed_points` is rebuilt from scratch on + resume and is not itself checkpointed, so a resumed analysis had no record + that a point was already `NaN`. `np.round(NaN).astype(int)` is undefined + (e.g. `INT64_MIN` on x86, `0` on arm64) rather than raising, so such a point + could silently restart tracking from a finite but meaningless position. + `LucasKanade` and `DirectionalLucasKanade` now check the previous + displacement for NaN/inf before rounding it and keep the point marked + failed if so, matching what the compiled kernel already did. +- **A single untracked point (`NaN`) could break the displacement-vector + display.** The napari `GUI` and the Qt `result_viewer` scaled vectors by + `np.max`/`np.max(np.abs(...))`, both of which propagate to `NaN` if any + point in the result failed to track. They now use `np.nanmax`, with a + fallback when every point failed. +- With `processes` greater than one, warnings about failed points raised + inside a worker used worker-local point indices and, under the + `forkserver`/`spawn` start methods, might not reach the console at all. The + parent process now re-summarises failed points with global indices once + the worker results are merged, for both `LucasKanade` and + `DirectionalLucasKanade`. +- **`compute_inverse_numba` and `compute_delta_numba` are importable from + `LucasKanade` again.** The 1.4.0 numba rewrite renamed them to + `compute_inverse` and `compute_delta`; code importing the 1.3.3 names broke + as soon as it hit that import. Both old names are restored as aliases. +- **Removed points in `SelectionGUIOld` no longer reappear after a recompute.** + The `Remove point` tool used to delete from a selection's *derived* points, + which were regenerated from the source geometry whenever the subset size or + spacing changed, so a removed point could silently come back. Removals are + now recorded per selection and re-applied after every recompute. +- **`SelectionGUIOld`'s brush had its row/column spacing swapped for anisotropic + subsets.** For a non-square `subset_size=(height, width)`, the brush laid + its grid out with the axes transposed — columns stepped by the height and + rows stepped by the width. Square subsets were unaffected. Now fixed. +- **`Deselect painted area` no longer throws away a whole brush stroke.** + Deselecting over any part of a painted region discarded the entire stroke, + so nibbling a corner off a large brush selection wiped all of it. The + deselect stroke is now subtracted from the painted mask, so only the + overlapping area is lost and the rest of the stroke stays; the selection is + removed only once nothing is left painted. Because the mask itself is + edited rather than its derived points, the deselection also survives a + subset-size or spacing change. +- **`configure(show_pbar=False)` was ignored by `LucasKanade` and `DIC`** when + running in a single process; the progress bar was always shown. + `DirectionalLucasKanade` already honoured the setting. + +### Point selection consolidated on one window + +> The window these entries describe is the one now called `SelectionGUIOld`. +> See "`SelectionGUI` is now the automatic feature selection interface" above. + +pyidi had accumulated five separate point-selection implementations. Three were +dead code, one was documented but no longer developed, and the one under active +development was not reachable from the documented workflow. There is now one. + +- **`SubsetSelection` has been removed.** The tkinter widget in + `pyidi/GUIs/selection.py` is gone, and `SelectionGUI` replaces it. The name is + still importable, but instantiating it raises a `RuntimeError` naming the + replacement, so existing scripts fail with an actionable message rather than an + `ImportError`. Replace `SubsetSelection(video, roi_size=(21, 21), noverlap=0)` + with `SelectionGUI(video, subset_size=21, subset_overlap=0)`. +- **It became the documented interface.** Five selection methods (grid in a + polygon, manual points, along a polyline, brush, and remove-point) plus + automatic filtering by Shi-Tomasi corner strength or gradient direction. It + requires the Qt extras: `pip install pyidi[qt]`. +- **Note for `LucasKanade` users:** it can select anisotropic + subsets again. `subset_size` accepts a scalar or a `(height, width)` pair, + in the same `(vertical, horizontal)` convention as + `LucasKanade.configure(roi_size=...)`. In the UI, a `Square subsets` + checkbox (checked by default) keeps the previous square-only behaviour; + unchecking it frees the height and width spinboxes/sliders to be set + independently. +- Removed the dead selection code: `tools.ManualROI`, `tools.GridOfROI` (both + read a `video.reader.mraw` attribute that no longer exists), the unreachable + `PickPoints` class in `_simplified_optical_flow.py`, and the stray + `load_analysis copy.py`. + +### Point validation + +`set_points()` now validates its input instead of accepting almost anything. + +- Empty input, non-2-D input, a wrong column count, and coordinates outside the + image now raise `ValueError` with a message that says what was wrong. Empty and + 1-D input previously raised `IndexError: tuple index out of range`; out-of-range + and negative coordinates were previously accepted silently, which since 1.4.0 + surfaced only as a `NaN` result much later. +- **Sub-pixel points are now rounded to the nearest pixel, with a warning.** + Previously the same float input crashed in `SimplifiedOpticalFlow` (used + directly as an array index) but was silently truncated *toward zero* in + `LucasKanade`, `DirectionalLucasKanade`, and `DIC`. All four now agree, and + round rather than truncate. +- `set_points()` accepts any object exposing a `.points` attribute, so a + selection GUI instance can be passed directly. Previously only `SubsetSelection` + was recognised, and passing the Qt GUI failed with an opaque error. +- The napari `GUI` now routes its selections through `set_points()` as well, so + points picked in the UI get the same checks as programmatic ones. + +### `SelectionGUIOld` editing + +- **The four separate selection stores are now one ordered list.** Grids, + lines, brush strokes, and manually-clicked points all live in a single + always-visible `selections` list in the right-hand panel, replacing the + two mode-specific lists (Grid, Along-the-line) and their two delete + buttons. Every grid, every drawn line, and every brush stroke gets its own + row (`Grid 1`, `Line 1`, `Brush 1`, …); every manually-clicked point + is collected into one shared `Manual` row. Each row shows a live point + count, e.g. `Grid 1 — 142 pts`. The list is visible in every selection + mode, so everything built so far stays in view regardless of which tool is + active. +- **Clicking a row** makes it the active selection, switches the tool to that + row's type so its vertices are immediately draggable, and highlights its + points in the image with a magenta ring. The highlight is a Select-mode cue + and is hidden in Filter mode, where the Select-mode points are not drawn. + **Each row has a checkbox** that excludes its points from the result without + deleting the row — useful for trying a region in and out. +- **Any selection can now be deleted, including a brush stroke or the + `Manual` row.** One `Delete selected` button replaces the previous + `Delete selected grid` / `Delete selected polygon` pair, and works for + every kind. Deleting the last remaining selection now simply empties the + list, rather than re-seeding an empty placeholder entry as it briefly did. +- **Row labels are no longer reused.** Deleting `Grid 2` and then creating + another grid now gives `Grid 4`, not a second `Grid 3`. +- **Polygon and grid vertices can be dragged.** A left-drag starting within ~10 + screen pixels of an existing vertex moves it; a drag anywhere else still pans, + and the grab radius is constant in screen pixels at any zoom. Clicking exactly + on an existing vertex is now a no-op rather than adding a duplicate on top of + it. The derived subset points are recomputed once when the drag finishes, not + on every mouse-move. +- **Undo (Ctrl+Z)** now reverses deleting ANY selection — a grid, a line, a + brush stroke, or the `Manual` row — in addition to adding and moving a + vertex. It previously covered only grid/polyline deletion; brush strokes + and the manual row could not be undone at all. A restored selection comes + back at its original row with its original label. Filter results are still + not undoable. +- The "Start new line" button now reads "Start new grid" in Grid mode. The + status-bar hint said "Click 'Start new line' to begin a new grid" and now + matches the button. +- **Filter candidates now follow the selection.** The Filter-mode filters score + the subsets placed in Select mode, but their result was never revisited when + those subsets changed. Deselecting an area with the brush (or removing a + point, or deleting/unchecking a row) left its candidates on screen and — since + `get_points()` returns the candidates once a filter has been run — in the + returned points. Candidates outside the current selection are now dropped, and + the threshold sliders can no longer bring them back. The per-subset scores are + kept rather than discarded, so this is reversible: re-checking a row, or + undoing its deletion, restores its candidates without re-running the filter. +- **Subset rectangles now have hairline borders.** They used to be painted + entirely into one RGBA image the size of the frame, so a border could not be + thinner than one *image* pixel — which grows into a thick band as soon as you + zoom in, and on an 11 px subset already ate a fifth of its width. The + translucent interior is still drawn that way (one upload however many subsets + there are), but the borders are now a single vector path stroked with a + cosmetic pen, whose width is in *screen* pixels, so they stay one pixel thin + at any zoom. Building the interior no longer loops over the points in Python + either, which makes the redraw after a subset-size or spacing change several + times faster on large selections. +- **The order of `gui.points`/`gui.get_points()` is now creation order** + across all selection types combined, rather than grouped by type (all + manual points, then all line points, then all grid points, then all brush + points). No supported use depends on point order. + +### Fixed + +- **Mouse drags were offset from the cursor by 9 pixels.** The drag handlers read + `ev.pos()`/`ev.buttonDownPos()`, which are local to the ViewBox, and passed them + to `mapSceneToView()` and `sceneBoundingRect().contains()`, which expect scene + coordinates. The click handlers already used `scenePos()` and were correct, so + clicking and dragging disagreed. Most visibly this meant the **brush painted + about 9 px away from the cursor**, and its bounds check was wrong by the same + amount. All drag paths now use `scenePos()`/`buttonDownScenePos()`. + +### Other -`configure(show_pbar=False)` was ignored by `LucasKanade` and `DIC` when running -in a single process; the progress bar was always shown. `DirectionalLucasKanade` -already honoured the setting. +- New `pyidi/selection_geometry.py` holds the ROI-grid geometry as pure numpy, + with no GUI-toolkit dependency, shared by the napari `GUI` and the selection + windows. + Its functions do not share one coordinate convention - each docstring states + which one it uses, and the tests pin the difference deliberately. +- `SelectionGUIOld` accepts a numpy array as documented. A 2-D or 3-D array + previously raised `AttributeError` because the frame was only set for a + `VideoReader`; anything unusable now raises `TypeError`. +- First tests for the GUI package: `tests/test_selection_geometry.py` and + `tests/test_set_points_validation.py` (24 tests). +- Fixed `README.md`, which told users to call `video.set_points(...)`. + `VideoReader` has no such method - points are set on the method object. ## 1.4.0 diff --git a/README.md b/README.md index 8c84c1e..49d6e1e 100644 --- a/README.md +++ b/README.md @@ -1,109 +1,86 @@ [![Documentation Status](https://readthedocs.org/projects/pyidi/badge/?version=latest)](https://pyidi.readthedocs.io/en/latest/?badge=latest) ![example workflow](https://github.com/ladisk/pyidi/actions/workflows/python_package_testing.yaml/badge.svg) -# pyidi -Image-based Displacement Identification (IDI) implementation in python. +# pyIDI -See the [documentation](https://pyidi.readthedocs.io/en/latest/index.html) for `pyIDI`. +**Image-based Displacement Identification (IDI)** from high-speed video, in Python. -## Now version 1.0! +pyIDI reads a recording, tracks the points you select, and returns their sub-pixel +displacement history — ready for modal analysis. -In version 1.0, **we overhauled the package API**. With growing usage in IDEs other than -jupyter notebooks, we have made the package more user-friendly. The new API allows the -autocompletion and documentation of the package to be more accessible in IDEs like -VSCode, Cursor, PyCharm, etc. +📖 [**Documentation**](https://pyidi.readthedocs.io/en/latest/index.html) -To install the new version, use the following command: +## Installation ```bash -pip install pyidi -``` -or to upgrade (if already installed): -```bash -pip install -U pyidi +pip install pyidi # identification +pip install pyidi[qt] # + the point-selection and result-viewing GUIs ``` -### Whats different? +Python >= 3.10. -For the user, the main difference is that instead of calling the `pyIDI` class where the -method is set, first, the `VideoReader` class is called. Then, this instance is passed -to the specific method class. Here is an example: +## Quick start ```python -from pyidi import VideoReader, SimplifiedOpticalFlow +from pyidi import VideoReader, LucasKanade -# Read the video -video = VideoReader('video.cih') +video = VideoReader('measurement.cih') -# Pass the video to the selected method class -sof = SimplifiedOpticalFlow(video) +lk = LucasKanade(video) +lk.set_points(points=[[150, 200], [150, 260], [150, 320]]) # (row, column) +lk.configure(roi_size=(21, 21)) -sof.set_points(points=[[0, 1], [1, 1], [2, 1]]) -sof.configure(...) -displacements = sof.get_displacements() +displacements = lk.get_displacements() # (n_points, n_frames, 2), in pixels ``` -The methods themselves have not changed, only the way they are called. Unfortunately, this -breaks the backward compatibility with the previous version. We apologize for any -inconvenience this may cause. To keep using the old version, please install the package -with the following command: +`VideoReader` handles Photron `.cih`/`.cihx`, Phantom `.cine`, Pharsighted `.SLOW`, +image sequences, ordinary video files (MP4, AVI, MOV, ...), and `numpy.ndarray` +stacks of shape `(n_time_points, image_height, image_width)`. -```bash -pip install pyidi==0.30.2 -``` -or when using .cine videos: -```bash -pip install pyidi[cine] -``` -or use the legacy `pyIDI` class: +Points are set on the **method** object, not on the `VideoReader`. + +### Selecting points interactively ```python -from pyidi import pyIDI +from pyidi import SelectionGUI + +gui = SelectionGUI(video, subset_size=21) +lk.set_points(gui) ``` -Note that the legacy `pyIDI` class does not necessarily offer the full functionality of the new version. -The legacy `pyIDI` class is only kept for compatibility with the old version and will not be updated. +`SelectionGUI` scores every position in the frame and picks the +best-separated features inside the region you draw, so it finds the points +rather than filtering a grid you placed. Draw with a polygon, a brush, a +polyline or single clicks; set a region's role to `points` and it lays them +out without scoring. Vertex dragging and undo throughout. See the +[documentation](https://pyidi.readthedocs.io/en/latest/quick_start/feature_selection.html). +The window `SelectionGUI` named in 1.3 is now `SelectionGUIOld` — deprecated, +and removed in 1.5. It takes the same arguments and returns the same points, +so scripts carry over unchanged. -# Use Napari UI for quick displacement identification: - + +### Or drive everything from the napari UI -# BASIC USAGE: -Run GUI by instantiating GUI class (input is VideoReader object): ```python from pyidi import VideoReader, GUI -# Read the video video = VideoReader('data/data_synthetic.cih') - -# Run GUI gui = GUI(video) -``` - -Method class (e.g. `SimplifiedOpticalFlow`) is instantiated during the use of GUI. It is accessible in `gui.method`. To get displacements: -```python -method = gui.method -displacements = method.displacements +displacements = gui.method.displacements ``` -The `pyIDI` method works with various formats: `.cih`, `.cihx`, `.png`, `.avi` etc. Additionally, it can also work with `numpy.ndarray` as input. -If an array is passed, it must have a shape of: ``(n time points, image height, image width)``. - -Set the points where displacements will be determined: -``` -p = np.array([[0, 1], [1, 1], [2, 1]]) # example of points -video.set_points(points=p) -``` -Or use point selection UI to set individual points or grid inside selected area. For more information about UI see [documentation](https://pyidi.readthedocs.io/en/quick_start/napari.html). Launch viewer with: + +## Example dataset -# EXAMPLE DATASET: -A high-speed video of a vibrating music-box comb is published on Zenodo -([10.5281/zenodo.22105821](https://doi.org/10.5281/zenodo.22105821), CC BY 4.0) and can be -loaded directly from `pyidi`. Only the requested frames are downloaded and they are cached -in `~/.pyidi/datasets` (or in `PYIDI_DATA_DIR`), so the first call is the only slow one: +No recording of your own yet? A high-speed video of a vibrating music-box comb is +published on Zenodo ([10.5281/zenodo.22105821](https://doi.org/10.5281/zenodo.22105821), +CC BY 4.0) and loads directly from `pyidi`. Only the frames you ask for are downloaded, +and they are cached in `~/.pyidi/datasets` (or in `PYIDI_DATA_DIR`), so only the first +call is slow: ```python import pyidi @@ -117,36 +94,82 @@ lk.configure(roi_size=(21, 51)) # a region one tooth tall displacements = lk.get_displacements() ``` -The comb was recorded with a Photron FASTCAM SA-Z at 7500 fps. Its teeth are cantilevers of -graduated length, so each rings at its own natural frequencies, with sub-pixel amplitudes on -a naturally speckled surface — a convenient benchmark for displacement identification. The -identified frequencies land within a few cents of equal-tempered pitches across nearly two -octaves: +The comb was recorded with a Photron FASTCAM SA-Z at 7500 fps. Its teeth are cantilevers +of graduated length, so each rings at its own natural frequencies, with sub-pixel +amplitudes on a naturally speckled surface — a convenient benchmark for displacement +identification. The identified frequencies land within a few cents of equal-tempered +pitches across nearly two octaves: -Datasets are a registry, so this one is loaded like any other: `pyidi.datasets.list_datasets()` -says what is available, `pyidi.datasets.load_dataset('music_box')` loads it, and -`pyidi.datasets.register_dataset()` accepts a recording of your own published the same way -(a Zenodo record with a Photron `cihx` header next to an uncompressed `mraw` file). +Datasets are a registry, so this one is loaded like any other: +`pyidi.datasets.list_datasets()` says what is available, +`pyidi.datasets.load_dataset('music_box')` loads it, and +`pyidi.datasets.register_dataset()` accepts a recording of your own published the same +way — a Zenodo record with a Photron `cihx` header next to an uncompressed `mraw` file. The full example is in [`examples/Showcase_music_box.ipynb`](examples/Showcase_music_box.ipynb): -from the raw video to the notes of the comb and to the operating deflection shape of a single -tooth. If you use the dataset, please cite it: +from the raw video to the notes of the comb and to the operating deflection shape of a +single tooth. If you use the dataset, please cite it: - Stanovnik, G., & Slavič, J. (2026). **High-speed video of a vibrating music-box comb - (Photron FASTCAM SA-Z, 7500 fps, 640x552 px)** [Data set]. Zenodo. https://doi.org/10.5281/zenodo.22105821 - -# DEVELOPER GUIDELINES: -* Add _name_of_method.py with class that inherits after `IDIMethods` -* This class must have methods: - * `calculate_displacements` with attribute `displacements` - * `get_points` (static method - sets attribute video.points) -* In `pyIDI` add a new method of identification in `avaliable_methods` dictionary. - -# Citing -If you are using the `pyIDI` package for your research, consider citing our articles: -- Masmeijer, T., Habtour, E., Zaletelj, K., & Slavič, J. (2024). **Directional DIC method with automatic feature selection**. Mechanical Systems and Signal Processing, 224 . https://doi.org/10.1016/j.ymssp.2024.112080 + (Photron FASTCAM SA-Z, 7500 fps, 640x552 px)** [Data set]. Zenodo. + https://doi.org/10.5281/zenodo.22105821 + +## Methods + +| Method | Solves for | Use it when | +| --- | --- | --- | +| `SimplifiedOpticalFlow` | 2 translations, from the image gradient | a fast first look, motion well below a pixel | +| `LucasKanade` | 2 translations, iteratively | the default choice | +| `DirectionalLucasKanade` | 1 translation along a known direction | motion along a known axis; edge-like features | +| `DIC` | 6 (affine) or 3 (rigid) warp parameters | strain and in-plane rotation, not just translation | + +The Lucas-Kanade inner loop is compiled with `numba` and parallelized over points — +one to two orders of magnitude faster than the NumPy implementation. + +## Pre-test motion visualization + +Eulerian video magnification amplifies subtle, sub-pixel motion directly in the raw +recording, before any identification is run — useful for checking whether and where +a structure moves, and for isolating a single mode: + +```python +from pyidi.postprocessing import EulerianMagnifier + +evm = EulerianMagnifier(video) +evm.configure(freq_band=(45.0, 55.0), amplification=25) +evm.save('mode_50Hz', output_format='mp4') +``` + +This is qualitative visualization, **not** a measurement. + +## Upgrading + +Version 1.0 replaced the monolithic `pyIDI` class with a `VideoReader` plus a +separate method class, so that autocompletion and inline documentation work +properly in VSCode, PyCharm and similar editors. Later releases removed the old +`SubsetSelection` widget and changed how untrackable points are reported. + +See the [upgrading guide](https://pyidi.readthedocs.io/en/latest/migration.html) +for what to change. The legacy class is still importable +(`from pyidi import pyIDI`) for compatibility, but is not being developed. + +## Developer guidelines + +* Add `pyidi/methods/_name_of_method.py` with a class that inherits from `IDIMethod`. +* The class must implement: + * `configure()` — every parameter stored as a class attribute of the same name + (this is what makes settings reproducible, picklable and exportable to JSON); + * `calculate_displacements()` — sets `self.displacements`, of shape + `(n_points, n_frames, 2)`. +* Export the new class in `pyidi/methods/__init__.py`. + +## Citing + +If you are using `pyIDI` for your research, consider citing our articles: + +- Masmeijer, T., Habtour, E., Zaletelj, K., & Slavič, J. (2024). **Directional DIC method with automatic feature selection**. Mechanical Systems and Signal Processing, 224. https://doi.org/10.1016/j.ymssp.2024.112080 - Čufar, K., Slavič, J., & Boltežar, M. (2024). **Mode-shape magnification in high-speed camera measurements**. Mechanical Systems and Signal Processing, 213, 111336. https://doi.org/10.1016/J.YMSSP.2024.111336 - Zaletelj, K., Gorjup, D., Slavič, J., & Boltežar, M. (2023). **Multi-level curvature-based parametrization and model updating using a 3D full-field response**. Mechanical Systems and Signal Processing, 187, 109927. https://doi.org/10.1016/j.ymssp.2022.109927 - Zaletelj, K., Slavič, J., & Boltežar, M. (2022). **Full-field DIC-based model updating for localized parameter identification**. Mechanical Systems and Signal Processing, 164. https://doi.org/10.1016/j.ymssp.2021.108287 diff --git a/docs/requirements.txt b/docs/requirements.txt index ebf64ab..ef88c5b 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,3 +1,5 @@ pyidi sphinx-book-theme -sphinx-copybutton \ No newline at end of file +sphinx-copybutton +sphinx-design +myst-parser diff --git a/docs/source/_static/custom.css b/docs/source/_static/custom.css new file mode 100644 index 0000000..93b4ea5 --- /dev/null +++ b/docs/source/_static/custom.css @@ -0,0 +1,50 @@ +/* pyIDI documentation - small refinements on top of sphinx-book-theme. */ + +:root { + --pyidi-accent: #0b6e99; +} + +/* Landing-page cards: keep the grid calm and make the whole card feel clickable. */ +.sd-card { + border-radius: 0.5rem; + transition: transform 0.12s ease, box-shadow 0.12s ease; +} + +.sd-card:hover { + transform: translateY(-2px); + box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.12); +} + +.sd-card-title { + font-weight: 600; +} + +/* Signatures of documented classes and functions are easier to scan with a + little more separation from the description that follows. */ +dl.py.class > dt, +dl.py.function > dt, +dl.py.method > dt { + border-left: 3px solid var(--pyidi-accent); + padding-left: 0.6rem; + background: var(--pst-color-surface); +} + +/* Wide tables (the method comparison, the benchmark numbers) should scroll + rather than overflow the content column. */ +table.docutils { + display: block; + overflow-x: auto; + width: fit-content; + max-width: 100%; +} + +/* GIFs and screenshots in the tutorials. */ +img { + max-width: 100%; + height: auto; +} + +.pyidi-screenshot img { + border: 1px solid var(--pst-color-border); + border-radius: 0.4rem; +} diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst new file mode 100644 index 0000000..6682dab --- /dev/null +++ b/docs/source/changelog.rst @@ -0,0 +1,9 @@ +Changelog +========= + +The full changelog is kept in ``CHANGELOG.md`` at the root of the repository +and is reproduced here. It starts at version 1.4.0; for earlier versions see +the `commit history `_. + +.. include:: ../../CHANGELOG.md + :parser: myst_parser.sphinx_ diff --git a/docs/source/code/modules.rst b/docs/source/code/modules.rst index c6dd396..6b5f638 100644 --- a/docs/source/code/modules.rst +++ b/docs/source/code/modules.rst @@ -1,7 +1,42 @@ -pyIDI source code -================= +API reference +============= -Video Reader +Everything below is generated from the docstrings in the source. For a +task-oriented introduction, start from the :doc:`tutorial +<../quick_start/basic_usage>` instead. + +Top-level namespace +------------------- + +These names are importable directly from ``pyidi``: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Name + - Purpose + * - :class:`~pyidi.video_reader.VideoReader` + - Read a recording of any supported format. + * - ``SimplifiedOpticalFlow``, ``LucasKanade``, ``DirectionalLucasKanade``, + ``DIC`` + - The displacement identification methods. + * - ``SelectionGUI`` + - Interactive point selection (requires the ``[qt]`` extra). + * - ``SelectionGUIOld`` + - The 1.3 point-selection window, deprecated and removed in 1.5. + * - ``GUI``, ``ResultViewer``, ``Viewer`` + - napari and Qt viewers (require the ``[qt]`` extra). + * - ``load_analysis`` + - Reload a saved analysis from disk. + * - ``Fiducial`` + - Fiducial-marker tracking and rigid-body compensation. + * - ``postprocessing`` + - Eulerian video magnification and mode-shape magnification. + * - ``pyIDI`` + - The legacy pre-1.0 class, kept for compatibility only. + +Video reader ------------ .. automodule:: pyidi.video_reader @@ -13,38 +48,56 @@ Example datasets .. automodule:: pyidi.datasets :members: +Identification methods +---------------------- + IDIMethod base class -------------------- +^^^^^^^^^^^^^^^^^^^^ + +Every method inherits from ``IDIMethod``, which provides the shared +configuration handling, multiprocessing, checkpointing and result +persistence. .. automodule:: pyidi.methods.idi_method :members: Simplified optical flow ------------------------ +^^^^^^^^^^^^^^^^^^^^^^^ .. automodule:: pyidi.methods._simplified_optical_flow :members: -The Lucas-Kanade algorithm for translations -------------------------------------------- +Lucas-Kanade +^^^^^^^^^^^^ .. automodule:: pyidi.methods._lucas_kanade :members: Directional Lucas-Kanade ------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ .. automodule:: pyidi.methods._directional_lucas_kanade :members: Digital Image Correlation (DIC) -------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. automodule:: pyidi.methods._dic :members: -Postprocessing --------------- +Post-processing +--------------- + +.. _api-eulerian: + +Eulerian video magnification +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: pyidi.postprocessing._eulerian_magnification + :members: + +Mode-shape magnification +^^^^^^^^^^^^^^^^^^^^^^^^ .. automodule:: pyidi.postprocessing._motion_magnification :members: @@ -55,8 +108,50 @@ Fiducial markers .. automodule:: pyidi.fiducial :members: -pyIDI base class ----------------- +Point selection geometry +------------------------ + +Pure-numpy ROI-grid geometry, shared by the napari ``GUI``, ``SelectionGUI`` +and ``SelectionGUIOld``. No GUI toolkit is needed to import or use it. + +.. warning:: + + These functions do not all share one coordinate convention — each + docstring states which one it uses. + +.. automodule:: pyidi.selection_geometry + :members: + +Feature selection pipeline +-------------------------- + +The mask -> evaluate -> select pipeline behind +:doc:`../quick_start/feature_selection`. Importable without Qt, so the whole +selection can be scripted. + +.. automodule:: pyidi.selection.masks + :members: + +.. automodule:: pyidi.selection.evaluate + :members: + +.. automodule:: pyidi.selection.scores + :members: + +.. automodule:: pyidi.selection.select + :members: + +.. automodule:: pyidi.selection.pipeline + :members: + +Saved analyses +-------------- + +.. automodule:: pyidi.load_analysis + :members: + +Legacy pyIDI class +------------------ .. autoclass:: pyidi.pyidi_legacy.pyIDI :members: diff --git a/docs/source/conf.py b/docs/source/conf.py index 220888c..750ee75 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -40,24 +40,48 @@ # ones. extensions = [ 'sphinx.ext.autodoc', + 'sphinx.ext.autosummary', + 'sphinx.ext.napoleon', + 'sphinx.ext.intersphinx', 'sphinx.ext.mathjax', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode', 'sphinx.ext.githubpages', 'sphinx_copybutton', + 'sphinx_design', + 'myst_parser', ] +# MyST is enabled so that Markdown files kept at the repository root (the +# changelog) can be included in the build without being converted to rst. +myst_enable_extensions = ['colon_fence', 'deflist'] +myst_heading_anchors = 3 + +intersphinx_mapping = { + 'python': ('https://docs.python.org/3', None), + 'numpy': ('https://numpy.org/doc/stable/', None), + 'scipy': ('https://docs.scipy.org/doc/scipy/', None), + 'matplotlib': ('https://matplotlib.org/stable/', None), + 'napari': ('https://napari.org/stable/', None), +} +# Do not fail the build when an inventory cannot be downloaded (offline builds). +intersphinx_disabled_reftypes = ['*.std:doc'] + # Defined here: https://sphinx-copybutton.readthedocs.io/en/latest/use.html#using-regexp-prompt-identifiers (the >>> are not copied) copybutton_prompt_text = r">>> |\.\.\. |\$ |In \[\d*\]: | {2,5}\.\.\.: | {5,8}: " copybutton_prompt_is_regexp = True autodoc_default_options = { 'members': True, - 'private-members': True, 'special-members': '__init__', 'member-order': 'bysource', 'show-inheritance': None, } +autodoc_member_order = 'bysource' +autodoc_typehints = 'description' + +napoleon_google_docstring = True +napoleon_numpy_docstring = True # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -66,7 +90,10 @@ # You can specify multiple suffix as a list of string: # # source_suffix = ['.rst', '.md'] -source_suffix = '.rst' +source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', +} # The master toctree document. master_doc = 'index' @@ -92,19 +119,29 @@ # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # -# html_theme = 'sphinx_rtd_theme' html_theme = 'sphinx_book_theme' -# Theme options are theme-specific and customize the look and feel of a theme -# further. For a list of options available for each theme, see the -# documentation. -# -# html_theme_options = {} +html_title = f'pyIDI {release}' + +html_theme_options = { + 'repository_url': 'https://github.com/ladisk/pyidi', + 'repository_branch': 'master', + 'path_to_docs': 'docs/source', + 'use_repository_button': True, + 'use_issues_button': True, + 'use_edit_page_button': True, + 'use_download_button': False, + 'home_page_in_toc': True, + 'show_toc_level': 2, + 'navigation_with_keys': False, + 'article_header_start': [], +} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ["_static"] +html_static_path = ['_static'] +html_css_files = ['custom.css'] # Custom sidebar templates, must be a dictionary that maps document names # to template names. diff --git a/docs/source/contributing/documenting.rst b/docs/source/contributing/documenting.rst index c301234..2c507b5 100644 --- a/docs/source/contributing/documenting.rst +++ b/docs/source/contributing/documenting.rst @@ -1,103 +1,148 @@ .. _documenting-label: -Documenting the code -==================== +Contributing +============ -Requirements -^^^^^^^^^^^^ +Development install +------------------- -* *Sphinx* :: +.. code:: bash - pip install sphinx + git clone https://github.com/ladisk/pyidi.git + cd pyidi + pip install -e ".[dev,qt]" + pytest # tests + flake8 . --max-line-length=127 --max-complexity=10 # lint, as in CI -Automatic code documentation with autodoc ------------------------------------------ +Adding a displacement identification method +------------------------------------------- + +1. Create ``pyidi/methods/_name_of_method.py``. +2. Inherit from :class:`~pyidi.methods.idi_method.IDIMethod`. +3. Implement ``configure()`` and ``calculate_displacements()``. +4. Export the class in ``pyidi/methods/__init__.py``. +5. Add an ``automodule`` entry in ``docs/source/code/modules.rst`` and a + section in ``docs/source/quick_start/disp_id_methods.rst``. + +.. important:: + + **Every parameter of ``configure()`` must be stored as an attribute of the + same name.** + + .. code:: python + + def configure(self, param1=None, param2=None): + if param1 is not None: + self.param1 = param1 + if param2 is not None: + self.param2 = param2 + + This is not a style preference. The settings dictionary, the JSON export, + the checkpoint comparison that decides whether an interrupted analysis can + be resumed, and ``load_analysis()`` all work by reading the attributes + named after the ``configure()`` signature. A parameter stored under a + different name silently drops out of all four. + +``calculate_displacements()`` must set ``self.displacements`` with shape +``(n_points, n_frames, 2)``, in ``(row, column)`` order. + +Building the documentation +-------------------------- + +.. code:: bash + + cd docs + make html -The Sphinx autodoc_ extension automatically includes our Python modules documentation in the generated Sphinx documentation. +The result is in ``docs/build/html/index.html``. The build should be free of +warnings; a broken cross-reference is a warning, so this is worth checking +before opening a pull request. -.. _autodoc: http://www.sphinx-doc.org/en/stable/ext/autodoc.html +The documentation is built by Sphinx with the ``sphinx-book-theme``, +``sphinx-copybutton``, ``sphinx-design`` (the cards on the landing page) and +``myst-parser`` (which lets ``CHANGELOG.md`` be included directly). Read the +Docs builds it from ``docs/requirements.txt``, so a new extension has to be +added there as well as to the ``dev`` extra in ``pyproject.toml``. +Automatic code documentation with autodoc +----------------------------------------- + +The Sphinx autodoc_ extension pulls the docstrings out of the source. Only +modules listed in ``docs/source/code/modules.rst`` are included — there is no +point documenting every internal module, so the list covers the public API and +the classes developers extend. -Addind a new module documentaiton to the build -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. _autodoc: https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html -It is most likely not practical to iclude all of our apps Python modules in the build. Only the modules and classes with the most extensive documentation (docstrings), and the ones where most developer updates are expected should be included. +To add a module: -To add another module's documentation to the build, add a new entry to the ``doc/code/modules.rst`` file, with the correct relative Python path to the module. For example, the ``views.py`` documentation is included by:: +.. code:: rst - Tools - ----- - .. automodule:: pyidi.tools - :members: + Tools + ----- -For more information, see the autodoc_ documentation. + .. automodule:: pyidi.tools + :members: +Docstring style +--------------- -Docstring style: reStructuredText -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +pyIDI uses **reStructuredText (Sphinx) style** docstrings. Some modules use +NumPy or Google style; ``napoleon`` is enabled so those render correctly too, +but new code should match the surrounding style, which for most of the package +is reStructuredText. -* default docstring style in PyCharm_ -* ``"autoDocstring.docstringFormat": "sphinx"`` in `VSCode Python Docstring extenion`_ +* It is the default docstring style in PyCharm_. +* In VSCode, set ``"autoDocstring.docstringFormat": "sphinx"`` in the + `VSCode Python Docstring extension`_. .. _PyCharm: https://www.jetbrains.com/help/pycharm/python-integrated-tools.html -.. _`VSCode Python Docstring extenion`: https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring +.. _`VSCode Python Docstring extension`: https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring -Example_: +The order of the fields is: parameters (``:param :``), their types +(``:type :``), the return value (``:return:``, ``:rtype:``), then any +``.. note::``, ``.. warning::`` or ``.. seealso::`` directives. .. code-block:: python - def function1(self, arg1, arg2, arg3): - """returns (arg1 / arg2) + arg3 - - This is a longer explanation, which may include math with latex syntax - :math:`\\alpha`. - Then, you need to provide optional subsection in this order (just to be - consistent and have a uniform documentation. Nothing prevent you to - switch the order): - - - parameters using ``:param : `` - - type of the parameters ``:type : `` - - returns using ``:returns: `` - - examples (doctest) - - seealso using ``.. seealso:: text`` - - notes using ``.. note:: text`` - - warning using ``.. warning:: text`` - - todo ``.. todo:: text`` - - **Advantages**: - - Uses sphinx markups, which will certainly be improved in future - version - - Nice HTML output with the See Also, Note, Warnings directives - - - **Drawbacks**: - - Just looking at the docstring, the parameter, type and return - sections do not appear nicely - - :param arg1: the first value - :param arg2: the first value - :param arg3: the first value - :type arg1: int, float,... - :type arg2: int, float,... - :type arg3: int, float,... - :returns: arg1/arg2 +arg3 - :rtype: int, float - - :Example: - - >>> import template - >>> a = template.MainClass1() - >>> a.function1(1,1,1) - 2 - - .. note:: can be useful to emphasize - important feature - .. seealso:: :class:`MainClass2` - .. warning:: arg2 must be non-zero. - .. todo:: check that arg2 is non zero. - """ - - return arg1/arg2 + arg3 - -.. _Example: https://thomas-cokelaer.info/tutorials/sphinx/docstring_python.html \ No newline at end of file + def function1(self, arg1, arg2, arg3): + """Return ``(arg1 / arg2) + arg3``. + + A longer explanation, which may include maths in latex syntax + :math:`\\alpha`. + + :param arg1: the first value + :type arg1: int or float + :param arg2: the second value, must be non-zero + :type arg2: int or float + :param arg3: the third value + :type arg3: int or float + :return: ``arg1 / arg2 + arg3`` + :rtype: float + + .. warning:: ``arg2`` must be non-zero. + """ + return arg1 / arg2 + arg3 + +Two things trip up the build regularly: + +* a continuation line of a field must be indented further than the ``:param:`` + it belongs to, otherwise docutils reports *"Field list ends without a blank + line"*; +* a directive (``.. list-table::``, ``.. code::``) swallows every following + line that is indented at least as far as its content, so a block quote after + a table needs an unindented line between them. + +Releasing +--------- + +.. code:: bash + + python sync_version.py --bump patch # or minor / major + git tag vX.Y.Z && git push origin master --tags + +``sync_version.py`` keeps the version consistent across ``pyproject.toml``, +``pyidi/__init__.py`` and ``docs/source/conf.py``. CI publishes to PyPI on a +tag push. Record user-visible changes in ``CHANGELOG.md`` — it is rendered +into the documentation as the :doc:`../changelog` page. diff --git a/docs/source/fiducial_marker.rst b/docs/source/fiducial_marker.rst index d345f38..da43b7e 100644 --- a/docs/source/fiducial_marker.rst +++ b/docs/source/fiducial_marker.rst @@ -1,21 +1,140 @@ -Fiducial Marker-Based Motion Tracking and Compensation -====================================================== +.. _fiducial-markers: -This `Showcase `_ demonstrates the fiducial marker-based motion tracking and compensation capabilities of **pyIDI**. +Fiducial markers and motion compensation +======================================== -The example dataset originates from infrared-spectrum measurements. For convenience and to meet GitHub storage limits, the original data was undersampled to create a 25-frame video. The package can also process visible-range acquisitions. +When the camera and the structure move relative to each other for reasons that +have nothing to do with the vibration you are measuring — a shaking floor, a +drifting tripod, a specimen on a moving stage — that global motion contaminates +every identified displacement. -Contact Information -------------------- +The ``Fiducial`` class handles this by tracking markers of known geometry +(ArUco and related types) in the frame, computing the frame-to-frame +transformation they imply, and inverting it. Either the frames themselves are +warped back into the reference frame's coordinate system, or the known +rigid-body motion is fed to +:ref:`DirectionalLucasKanade ` as a prescribed motion. -For further details, please contact Dr. Janko Slavič (`janko.slavic@fs.uni-lj.si `_) or Dr. Lorenzo Capponi (`lorenzo.capponi@fs.uni-lj.si `_) +.. note:: + ``Fiducial`` takes a **numpy array** of frames, not a ``VideoReader``: + shape ``(n_frames, height, width)``, or ``(n_frames, height, width, 3)`` + for RGB, which is converted to grayscale automatically. -Acknowledgments ---------------- + .. code:: python -This work was conducted as part of the **ARTEMIDE** project, funded by the European Research Agency (ERA) under the Marie Skłodowska-Curie Actions (MSCA), Grant Agreement No. **101180595**. + frames = video.get_frames() + fid = Fiducial(frames) -.. note:: +Workflow +-------- + +.. code:: python + + from pyidi import VideoReader, Fiducial + + video = VideoReader('measurement.cih') + fid = Fiducial(video.get_frames()) + + # 1. optional: make the markers easier to find + fid.pre_process(clahe=True, apply_blur=True) + + # 2. detect the markers in every frame + id_coords = fid.detect_markers(marker_type='aruco') + + # 3. the transformation from each frame to the reference frame + transformations = fid.compute_transformations(id_coords, reference_index=0, + transform_type='euclidean') + + # 4a. warp the frames back into the reference coordinate system + aligned = fid.revert_frames(transformations, transform_type='euclidean') + + # 4b. ...or just the marker coordinates, to check the quality of the fit + transformed = fid.revert_fiducial(id_coords, transformations) + stats = fid.uncertainty_analysis(id_coords, transformed, plot=True) + +The aligned frames can be handed straight back to a +:class:`~pyidi.video_reader.VideoReader` as an array, and the identification +run on them as usual. + +Detection +--------- + +``detect_markers(video=None, marker_type='aruco', fiducial_dictionary=None, +known_ids=None)`` supports ``"aruco"``, ``"apriltag"``, ``"charuco"`` and +``"artoolkit"``. ``known_ids`` restricts detection to specific marker IDs, +which is worth setting when other marker-like patterns are in the field of +view. + +If detection is unreliable, ``pre_process()`` offers clipping and +normalisation, global histogram equalisation, CLAHE (adaptive histogram +equalisation), Gaussian blur, adaptive thresholding, and morphological +opening/closing. Detection quality usually improves more from fixing the +lighting than from any of these. + +Transformation types +-------------------- + +``compute_transformations`` and ``revert_frames`` take the same +``transform_type``: + +.. list-table:: + :header-rows: 1 + :widths: 20 20 60 + + * - Type + - Degrees of freedom + - Use when + * - ``'euclidean'`` + - 3 + - The camera-structure relation is a rigid translation plus rotation. + This is the default and the right choice for most vibration setups. + * - ``'affine'`` + - 6 + - Scale and shear are also present, for example if the object distance + changes. + * - ``'homography'`` + - 8 + - Full projective mapping, for out-of-plane camera motion looking at a + planar target. + +Prefer the least flexible transformation that fits. A more flexible model +absorbs real structural motion into the "compensation" and quietly removes the +thing you are measuring. + +If a transformation cannot be computed for a frame — too few common markers — +its entry is ``None``, and that frame passes through unchanged. + +Checking the compensation +------------------------- + +``uncertainty_analysis(id_coords, transformed_fiducial, plot=True)`` returns +per-frame and overall Euclidean error statistics between the transformed and +reference marker positions. The residual is the floor on what the compensation +can deliver: displacements smaller than it are not trustworthy after +compensation. + +Worked example +-------------- + +This `showcase notebook +`_ +demonstrates the full workflow. The example dataset originates from +infrared-spectrum measurements; to meet GitHub storage limits the original data +was undersampled to a 25-frame video. The package works equally on +visible-range acquisitions. + +Contact +------- + +For further details, please contact Dr. Janko Slavič +(`janko.slavic@fs.uni-lj.si `_) or +Dr. Lorenzo Capponi (`lorenzo.capponi@fs.uni-lj.si +`_). + +Acknowledgments +--------------- - More documentation is coming soon! +This work was conducted as part of the **ARTEMIDE** project, funded by the +European Research Agency (ERA) under the Marie Skłodowska-Curie Actions (MSCA), +Grant Agreement No. **101180595**. diff --git a/docs/source/index.rst b/docs/source/index.rst index 4d85b10..8f01f48 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,29 +1,170 @@ -.. pyIDI documentation master file, created by - sphinx-quickstart on Thu Jul 4 15:06:37 2019. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. +:og:description: Image-based Displacement Identification from high-speed video in Python. -Welcome to pyIDI's documentation! -================================= +pyIDI +===== + +**Image-based Displacement Identification (IDI)** from high-speed video, in Python. + +pyIDI reads a recording, tracks the points you select, and returns their +sub-pixel displacement history — an array of shape +``(n_points, n_frames, 2)`` you can feed straight into a modal analysis. + +.. code:: python + + from pyidi import VideoReader, LucasKanade + + video = VideoReader('measurement.cih') + + lk = LucasKanade(video) + lk.set_points(points=[[150, 200], [150, 260], [150, 320]]) + lk.configure(roi_size=(21, 21)) + + displacements = lk.get_displacements() # (n_points, n_frames, 2), in pixels + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item-card:: :octicon:`rocket` Getting started + :link: quick_start/basic_usage + :link-type: doc + + Install pyIDI, load a video, select points and run your first + identification. + + .. grid-item-card:: :octicon:`graph` Displacement methods + :link: quick_start/disp_id_methods + :link-type: doc + + Simplified Optical Flow, Lucas-Kanade, Directional DIC and full-field + DIC — what each one is for and how to configure it. + + .. grid-item-card:: :octicon:`eye` Selecting points + :link: quick_start/feature_selection + :link-type: doc + + The ``SelectionGUI``: draw a region, score the whole image, and let + the selection find the best-separated features inside it. + + .. grid-item-card:: :octicon:`beaker` Post-processing + :link: postprocessing/eulerian_magnification + :link-type: doc + + Eulerian video magnification for pre-test motion visualization, and + mode-shape magnification of identified displacements. + + .. grid-item-card:: :octicon:`code` API reference + :link: code/modules + :link-type: doc + + Every public class and function, generated from the source. + + .. grid-item-card:: :octicon:`arrow-right` Upgrading + :link: migration + :link-type: doc + + Coming from 1.3.3 or from the pre-1.0 ``pyIDI`` class? Start here. + +What pyIDI does +--------------- + +.. grid:: 1 1 3 3 + :gutter: 2 + + .. grid-item-card:: Reads what your camera wrote + + Photron ``.cih``/``.cihx``, Phantom ``.cine``, Pharsighted ``.SLOW``, + image sequences, ordinary video files, and plain + :class:`numpy.ndarray` stacks — behind one + :class:`~pyidi.video_reader.VideoReader` interface. + + .. grid-item-card:: Tracks to sub-pixel accuracy + + Four identification methods, from a fast whole-field gradient + estimate to an iterative full-field DIC solve, sharing one + configuration, checkpointing and result-saving framework. + + .. grid-item-card:: Scales to long recordings + + The Lucas-Kanade inner loop is compiled with ``numba`` and runs + across threads or processes, with crash-resistant checkpointing for + long analyses. .. toctree:: :maxdepth: 2 - :caption: Contents: - + :hidden: + :caption: Getting started + installation quick_start/basic_usage + quick_start/video_reader datasets - quick_start/points_selection + quick_start/feature_selection quick_start/napari + quick_start/points_selection + +.. toctree:: + :maxdepth: 2 + :hidden: + :caption: Identification + quick_start/disp_id_methods + quick_start/results + +.. toctree:: + :maxdepth: 2 + :hidden: + :caption: Post-processing + + postprocessing/eulerian_magnification mode_shape_magnification + fiducial_marker + +.. toctree:: + :maxdepth: 2 + :hidden: + :caption: Reference + code/modules + migration + changelog contributing/documenting - fiducial_marker +Citing pyIDI +------------ + +If you use pyIDI in your research, please cite the article behind the method +you used: + + Masmeijer, T., Habtour, E., Zaletelj, K., & Slavič, J. (2024). + **Directional DIC method with automatic feature selection**. + *Mechanical Systems and Signal Processing*, 224. + https://doi.org/10.1016/j.ymssp.2024.112080 + + Čufar, K., Slavič, J., & Boltežar, M. (2024). + **Mode-shape magnification in high-speed camera measurements**. + *Mechanical Systems and Signal Processing*, 213, 111336. + https://doi.org/10.1016/J.YMSSP.2024.111336 + + Zaletelj, K., Gorjup, D., Slavič, J., & Boltežar, M. (2023). + **Multi-level curvature-based parametrization and model updating using a + 3D full-field response**. *Mechanical Systems and Signal Processing*, + 187, 109927. https://doi.org/10.1016/j.ymssp.2022.109927 + + Zaletelj, K., Slavič, J., & Boltežar, M. (2022). + **Full-field DIC-based model updating for localized parameter + identification**. *Mechanical Systems and Signal Processing*, 164. + https://doi.org/10.1016/j.ymssp.2021.108287 + + Gorjup, D., Slavič, J., & Boltežar, M. (2019). + **Frequency domain triangulation for full-field 3D operating-deflection-shape + identification**. *Mechanical Systems and Signal Processing*, 133. + https://doi.org/10.1016/j.ymssp.2019.106287 + +The package itself is archived on Zenodo: +https://doi.org/10.5281/zenodo.4017153 Indices and tables -================== +------------------ * :ref:`genindex` * :ref:`modindex` diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 3b79fed..5fa99fa 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -3,67 +3,115 @@ Installation ============ -Requirements ------------- +.. code:: bash -pyIDI requires **Python >= 3.10**. + pip install pyidi -Basic install -------------- +That is enough to read every supported video format and run every +identification method. The interactive point-selection and result-viewing +tools need one extra: .. code:: bash - pip install pyidi + pip install pyidi[qt] + +Requirements +------------ + +pyIDI requires **Python >= 3.10**. + +Everything needed for identification is installed automatically, including +``numba`` (the compiled Lucas-Kanade kernel), ``imageio[pyav]`` (video files), +``pyMRAW`` (Photron), ``cine-handler`` (Phantom ``.cine``) and +``opencv-contrib-python`` (fiducial markers). Optional extras --------------- -``[qt]`` -^^^^^^^^ +``[qt]`` — the graphical tools +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Installs napari, PyQt6, pyqtgraph, and magicgui — required for any GUI usage -(``SelectionGUI``, ``ResultViewer``, ``GUI``, ``SubsetSelection``). -Without this extra, importing those classes raises a ``RuntimeError``. +Installs napari, PyQt6, pyqtgraph and magicgui. Required for +:ref:`SelectionGUI `, ``ResultViewer`` and the napari +:ref:`GUI `. Without it those classes can still be imported, but +instantiating one raises a ``RuntimeError`` telling you to install the extra. .. code:: bash pip install pyidi[qt] -``[dev]`` -^^^^^^^^^ +``[dev]`` — building the docs and running the tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Installs development and testing dependencies: sphinx, pytest, ipykernel, -ipywidgets, and related packages. +Sphinx and its extensions, pytest, and the notebook tooling. .. code:: bash pip install pyidi[dev] -Combining extras -^^^^^^^^^^^^^^^^ +Combining extras: .. code:: bash pip install pyidi[qt,dev] -ArUco / fiducial detection --------------------------- +Upgrading +--------- + +.. code:: bash + + pip install -U pyidi + +If you are coming from an older release, read :doc:`migration` — several +releases have changed things that will not go unnoticed. + +OpenCV conflicts +---------------- pyIDI relies on ``opencv-contrib-python`` (not the base ``opencv-python`` -package) for ArUco marker detection. This dependency was updated in a recent -release. If you have ``opencv-python`` already installed, uninstall it first -to avoid conflicts: +package) for ArUco marker detection. The two packages install into the same +namespace and conflict. If you already have ``opencv-python``, remove it +first: .. code:: bash pip uninstall opencv-python pip install pyidi -Editable / development install -------------------------------- - -To install in editable mode with all optional dependencies: +Development install +------------------- .. code:: bash + git clone https://github.com/ladisk/pyidi.git + cd pyidi pip install -e ".[dev,qt]" + +Running the tests: + +.. code:: bash + + pytest + +Building the documentation: + +.. code:: bash + + cd docs + make html + +The result is in ``docs/build/html/index.html``. + +Verifying the installation +-------------------------- + +.. code:: python + + import pyidi + + print(pyidi.__version__) + +The first Lucas-Kanade run in a fresh environment spends a few extra seconds +compiling the numba kernel. The compiled result is cached on disk, so later +runs skip it. If pyIDI is installed somewhere the cache cannot be written, set +``NUMBA_CACHE_DIR`` to a writable directory. diff --git a/docs/source/migration.rst b/docs/source/migration.rst new file mode 100644 index 0000000..7e0d097 --- /dev/null +++ b/docs/source/migration.rst @@ -0,0 +1,177 @@ +.. _migration: + +Upgrading +========= + +What breaks between versions, and what to write instead. For the full list of +changes see the :doc:`changelog`. + +From 1.4.0 +---------- + +``SelectionGUI`` now names a different window +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +``SelectionGUI`` is the interface built on :mod:`pyidi.selection`, documented +in :doc:`quick_start/feature_selection`. The window it named in 1.3 is +``SelectionGUIOld``: still importable, deprecated, removed in 1.5. + +Nothing changes for a script that opens the window and reads its points. The +constructor takes the same arguments and ``get_points()`` returns the same +``(n_points, 2)`` array in ``(row, col)`` order: + +.. code:: python + + gui = SelectionGUI(video, subset_size=21, subset_overlap=0) + lk.set_points(gui) + +To stay on the old window for now, rename it and expect the warning: + +.. code:: python + + from pyidi import SelectionGUIOld + + gui = SelectionGUIOld(video, subset_size=21, subset_overlap=0) + +Three things do not carry over. ``get_filtered_points()`` and +``get_selected_points()`` are gone -- there is one ``get_points()``, because +filtering is no longer a pass over an existing selection but the selection +itself. The internal attributes (``selections``, ``subset_size_spinbox``, +``candidate_points``) have no counterpart. And ``Grid`` is not a mode: draw a +polygon and set its row to the ``points`` role, or keep it a mask and pick the +``lattice`` selector. + +``SubsetSelection`` has been removed +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +pyIDI had accumulated five separate point-selection implementations. There is +now one: :ref:`SelectionGUI `. The tkinter ``SubsetSelection`` +widget is gone. The name is still importable, but instantiating it raises a +``RuntimeError`` naming the replacement, so old scripts fail with an +actionable message rather than an ``ImportError``. + +.. code:: python + + # before + selection = SubsetSelection(video, roi_size=(21, 21), noverlap=0) + + # now + selection = SelectionGUI(video, subset_size=21, subset_overlap=0) + +Note the sign convention: ``noverlap`` counted overlapping pixels, while +``subset_overlap`` is added to ``subset_size`` to give the step between subset +centres. A positive ``subset_overlap`` therefore spreads subsets *apart*; pass +a negative value to overlap them. + +``subset_size`` accepts a scalar or a ``(height, width)`` pair, in the same +``(vertical, horizontal)`` convention as +``LucasKanade.configure(roi_size=...)``. + +Also removed, all of it dead code: ``tools.ManualROI``, ``tools.GridOfROI``, +the unreachable ``PickPoints`` class in ``_simplified_optical_flow.py``, and +the ``pyidi.py`` module. + +``set_points()`` now validates its input +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Input that was previously accepted silently — or that failed much later with +an opaque error — is now rejected up front. See :ref:`point-conventions`. + +.. list-table:: + :header-rows: 1 + :widths: 34 33 33 + + * - Input + - Before + - Now + * - Empty, or 1-D + - ``IndexError: tuple index out of range`` + - ``ValueError`` naming the problem + * - Wrong number of columns + - accepted, failed later + - ``ValueError`` + * - Coordinates outside the image + - accepted silently, surfaced as ``NaN`` results + - ``ValueError`` + * - Sub-pixel (float) coordinates + - crash in ``SimplifiedOpticalFlow``, silent truncation *toward zero* + elsewhere + - rounded to the nearest pixel, with a warning + +``set_points()`` also accepts any object exposing a ``.points`` attribute, so +a selection GUI instance can be passed straight through: +``lk.set_points(gui)``. + +From 1.3.3 +---------- + +``use_numba`` is now ``use_compiled_kernel`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +``DirectionalLucasKanade.configure()`` accepted a ``use_numba`` argument that +never did anything. The compiled path is now real, is on by default, and is +selected by ``use_compiled_kernel``: + +.. code:: python + + # before (accepted, but had no effect) + lk1d.configure(roi_size=(9, 9), dij=(1, 0), use_numba=True) + + # now (this is the default; pass False for the NumPy implementation) + lk1d.configure(roi_size=(9, 9), dij=(1, 0), use_compiled_kernel=True) + +Untrackable points return ``NaN`` instead of aborting +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A point on a uniform region, or on a single straight edge with no gradient +along it, used to abort the whole analysis. It now goes to ``NaN`` from the +frame at which it was lost, every other point is computed normally, and the +detail is recorded in ``failed_points``. See :ref:`failed-points`. + +The practical consequence for downstream code: **results can contain +``NaN``**. Use ``np.nanmax``, ``np.nanmean`` and friends, or drop the failed +points explicitly. + +``compute_inverse_numba`` and ``compute_delta_numba`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The 1.4.0 numba rewrite renamed these two helpers to ``compute_inverse`` and +``compute_delta``, which broke code importing the 1.3.3 names. Both old names +are restored as aliases, so no change is needed. + +From 0.x — the pre-1.0 ``pyIDI`` class +-------------------------------------- + +Version 1.0 replaced the monolithic ``pyIDI`` class with a +:class:`~pyidi.video_reader.VideoReader` plus a separate method class. This +makes autocompletion and inline documentation work properly in VSCode, +PyCharm and similar editors, at the cost of backward compatibility. + +.. code:: python + + # before + from pyidi import pyIDI + + video = pyIDI('video.cih') + video.set_method('sof') + video.set_points(points) + displacements = video.get_displacements() + + # now + from pyidi import VideoReader, SimplifiedOpticalFlow + + video = VideoReader('video.cih') + + sof = SimplifiedOpticalFlow(video) + sof.set_points(points) + displacements = sof.get_displacements() + +The methods themselves are unchanged — only the way they are called. + +The legacy class is still importable as ``from pyidi import pyIDI``, kept for +compatibility only. It does not offer the full functionality of the current +API and is not being updated. To stay on the old version entirely: + +.. code:: bash + + pip install pyidi==0.30.2 diff --git a/docs/source/mode_shape_magnification.rst b/docs/source/mode_shape_magnification.rst index 721977d..5afa696 100644 --- a/docs/source/mode_shape_magnification.rst +++ b/docs/source/mode_shape_magnification.rst @@ -1,12 +1,109 @@ -Mode shape magnification +.. _mode-shape-magnification: + +Mode-shape magnification ======================== -For example of mode shape magnification, see this `showcase `_. +Mode-shape magnification takes an *identified* mode shape and warps the +reference image by it, scaled up by a chosen factor. Where +:doc:`Eulerian magnification ` works on +the raw video before any identification, this works on the result: the +displacements are already known, and the magnification is a faithful geometric +scaling of them. + +The implementation follows Čufar et al. [1]_. + +Two functions are provided: + +* ``mode_shape_magnification()`` — a single magnified image; +* ``animate()`` — an animation of the mode shape over one or more periods, + written to a file. + +A still image +------------- + +.. code:: python + + from pyidi.postprocessing import mode_shape_magnification + + magnified = mode_shape_magnification( + displacements=mode_shape, # (n_points, 2), the identified mode shape + magnification_factor=20, + idi=lk, # the method instance the shape came from + ) + +Passing the method instance as ``idi`` is the shortcut: the reference image and +the point coordinates are taken from it. Both can be given explicitly instead, +which is what you do when the shape comes from somewhere else: + +.. code:: python + + magnified = mode_shape_magnification( + displacements=mode_shape, + magnification_factor=20, + image=reference_image, # (height, width) + points=points, # (n_points, 2), row/column + ) -The mode shape magnification implementation is based on the following paper: +Other arguments: - [1] Čufar, K., Slavič, J., & Boltežar, M. (2024). Mode-shape magnification in high-speed camera measurements. Mechanical Systems and Signal Processing, 213, 111336. https://doi.org/10.1016/J.YMSSP.2024.111336 +.. list-table:: + :header-rows: 1 + :widths: 26 14 60 + + * - Parameter + - Default + - Meaning + * - ``background_brightness`` + - ``0.3`` + - Brightness of the background, in ``[0, 1]``. + * - ``show_undeformed`` + - ``False`` + - Draw the reference image underneath the magnified shape, so the + deformation can be read against the original geometry. + +An animation +------------ + +.. code:: python + + from pyidi.postprocessing import animate + + animate( + displacements=mode_shape, + magnification_factor=20, + idi=lk, + fps=30, + n_periods=3, + filename='mode_1', + output_format='gif', + ) + +The mode shape is animated through ``n_periods`` full periods at ``fps`` +frames per second and written to ``filename.output_format``. .. note:: - More documentation is comming soon! \ No newline at end of file + ``displacements`` here is a *mode shape*: one real displacement vector per + point, shape ``(n_points, 2)``. It is **not** the ``(n_points, n_frames, 2)`` + time history returned by ``get_displacements()`` — a 2-D array is required + and anything else raises ``TypeError``. Extract the shape first, for + example from an FRF-based modal identification of the displacement + histories. + + ``animate()`` scales that one shape harmonically through + ``n_periods`` periods; it does not replay the measured time history. + +Worked example +-------------- + +See the `showcase notebook +`_ +for a full example, from identification through modal analysis to the +magnified animation. + +Reference +--------- + +.. [1] Čufar, K., Slavič, J., & Boltežar, M. (2024). Mode-shape magnification + in high-speed camera measurements. *Mechanical Systems and Signal + Processing*, 213, 111336. https://doi.org/10.1016/J.YMSSP.2024.111336 diff --git a/docs/source/postprocessing/eulerian_magnification.rst b/docs/source/postprocessing/eulerian_magnification.rst new file mode 100644 index 0000000..f28fb6b --- /dev/null +++ b/docs/source/postprocessing/eulerian_magnification.rst @@ -0,0 +1,213 @@ +.. _eulerian-magnification: + +Eulerian video magnification +============================ + +Eulerian Video Magnification (EVM) amplifies subtle, often sub-pixel motion +directly in a raw recording. It answers the question you have *before* running +an identification: **is this structure moving at all, where, and at which +frequency?** + +.. warning:: + + **This is qualitative visualization, not a measurement.** Eulerian + magnification distorts motion amplitudes non-linearly. Never read + displacement off a magnified video — use + :doc:`../quick_start/disp_id_methods` for that. + +The implementation follows the linear EVM of Wu et al. [1]_: a spatial +Laplacian pyramid, a temporal band-pass filter applied per pyramid level, and +linear amplification of the band-passed signal added back onto the original. + +What it is good for +------------------- + +* **A pre-test sanity check.** Confirm the excitation is reaching the + structure and the camera is seeing it, before committing to a long analysis. +* **Finding where to put points.** The magnified video shows which parts of + the frame move, which is where the tracked points belong. +* **Isolating one mode.** Band-pass around a suspected natural frequency and + everything else — rigid-body drift, higher modes, flicker — is suppressed, + so the deflection shape of that one mode becomes visible. + +Quick start +----------- + +.. code:: python + + from pyidi import VideoReader + from pyidi.postprocessing import EulerianMagnifier + + video = VideoReader('measurement.cih') + + evm = EulerianMagnifier(video) + evm.configure( + freq_band=(45.0, 55.0), # Hz, around the mode of interest + amplification=25, + levels=4, + ) + + magnified = evm.get_magnified_video() # (n_frames, height, width) + evm.save('mode_50Hz', output_format='mp4') + +There is also a one-shot functional form, if you only want the array: + +.. code:: python + + from pyidi.postprocessing import eulerian_magnification + + magnified = eulerian_magnification(video, freq_band=(45.0, 55.0), amplification=25) + +Parameters +---------- + +Settings are stored on the object by ``configure()``. Only arguments that are +not ``None`` overwrite the current value, so you can adjust one knob at a time +and re-run. + +.. list-table:: + :header-rows: 1 + :widths: 18 12 70 + + * - Parameter + - Default + - Meaning + * - ``freq_band`` + - *required* + - Temporal pass-band ``(low, high)`` in Hz. Must satisfy + ``low < high``, and ``high`` must be below Nyquist (``0.5 * fps``). + * - ``amplification`` + - ``10.0`` + - The gain :math:`\alpha` applied to the band-passed signal. + * - ``levels`` + - ``4`` + - Laplacian pyramid depth. More levels means coarser spatial detail is + amplified too. Reduced automatically (with a warning) if the frames are + too small. + * - ``filter_type`` + - ``"ideal"`` + - ``"ideal"`` is an FFT brick-wall filter — the sharpest band, and the + right choice for isolating a single mode. ``"butter"`` is a Butterworth + filter and needs a strictly positive lower band edge. + * - ``lambda_c`` + - ``None`` + - Spatial-wavelength cutoff in pixels (experimental). See + :ref:`evm-lambda-c`. + * - ``fps`` + - from the video + - Sampling rate in Hz. Taken from the + :class:`~pyidi.video_reader.VideoReader` if it knows it. + * - ``mask`` + - ``None`` + - 2-D region-of-interest array matching the frame size. See + :ref:`evm-mask`. + * - ``show_progress`` + - ``True`` + - Progress bars while processing. + +Choosing a band and a gain +-------------------------- + +The band is the important setting; the gain is the one you tune afterwards. + +**Band.** Pick it around a frequency you already suspect — from a +preliminary identification, an accelerometer, or an FE model. A narrow band +around a single peak gives the cleanest result. A band that is too wide lets +several modes through at once and the video becomes hard to read. + +**Gain.** Start around 10-20 and increase until motion is visible. Too much +gain produces halos and ringing at strong edges: that is the linear +approximation breaking down, not a real feature of the structure. + +**Frame count.** The temporal filter needs enough periods of the frequency you +are isolating to resolve it. A 0.4 Hz mode at 60 fps needs on the order of a +thousand frames; a 500 Hz mode at 10 kHz needs far fewer. Rule of thumb: aim +for at least five to ten periods inside the analysed range. + +**Memory.** The whole range is held in memory as ``float32`` and a pyramid is +built on top of it. Bound this with ``frame_range``: + +.. code:: python + + magnified = evm.get_magnified_video(frame_range=(0, 1200)) + +``frame_range`` is passed through to +:meth:`VideoReader.get_frames `: +``None`` means all frames, an ``int`` means frames ``0..int``, and a +``(start, stop)`` tuple means that slice. For long or high-resolution +recordings, spatially downscaling the video before magnifying is the other +lever. + +.. _evm-mask: + +Restricting to a region of interest +----------------------------------- + +A ``mask`` limits amplification to part of the frame. Everything outside stays +exactly as recorded, which keeps a busy background from being amplified into +noise and makes the moving component easier to see. + +.. code:: python + + import numpy as np + + mask = np.zeros((video.image_height, video.image_width)) + mask[200:600, 300:900] = 1.0 + + evm.configure(mask=mask) + +The mask may be boolean or float. Float values in ``[0, 1]`` are honoured as a +soft blend, so feathering the edge (for example with a Gaussian blur) avoids a +visible seam at the mask boundary. + +.. _evm-lambda-c: + +``lambda_c`` — spatial attenuation +---------------------------------- + +``lambda_c`` is the spatial-wavelength cutoff from Wu et al.: pyramid levels +whose spatial wavelength is below the cutoff get progressively less +amplification. The intent is to damp the amplification of fine, noisy detail +while leaving broad structural motion at full gain. + +It is experimental and off by default (``None`` applies a constant +amplification to every band-pass level, and none to the low-pass residual). +Reach for it when a magnified video is dominated by high-frequency speckle +rather than by the motion you are after. + +.. note:: + + If ``lambda_c`` is small enough to attenuate every level to zero, the + output equals the input and a warning is raised — the video is not broken, + the cutoff is simply too aggressive. + +Saving the result +----------------- + +.. code:: python + + evm.save('mode_50Hz', output_format='mp4', fps=120) + +* ``output_format`` is one of ``"mp4"``, ``"avi"``, ``"mov"``, ``"gif"``. The + extension is appended to ``filename``. +* ``fps`` is the *playback* rate. It defaults to the sampling rate. For a + high-speed recording, playing back at the capture rate is unwatchable — pass + a slower rate to get the slow-motion effect. +* ``save()`` computes the video first if it has not been computed yet, or if a + ``frame_range`` is given. +* The intensity range is mapped to 8-bit using the range of the *source* + frames, so amplification overshoot does not wash out the whole video. + +A full worked script, including streaming a long recording in and downscaling +it, is in `examples/eulerian_magnification_varcila.py +`_. + +Reference +--------- + +.. [1] Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., & Freeman, W. + (2012). Eulerian Video Magnification for Revealing Subtle Changes in the + World. *ACM Transactions on Graphics (Proc. SIGGRAPH 2012)*, 31(4). + https://doi.org/10.1145/2185520.2185561 + +See :ref:`the API reference ` for the full signatures. diff --git a/docs/source/quick_start/basic_usage.rst b/docs/source/quick_start/basic_usage.rst index 0ae24cb..eccf720 100644 --- a/docs/source/quick_start/basic_usage.rst +++ b/docs/source/quick_start/basic_usage.rst @@ -3,15 +3,27 @@ Tutorial ======== -``pyidi`` is a python package for displacement identification from raw video. +pyIDI identifies displacements from a raw video. Every analysis has the same +four steps: -``VideoReader`` supports Photron ``.cih``/``.cihx`` files, image sequences (PNG, TIFF, BMP, JPEG, GIF), -standard video files (MP4, AVI, MKV, MOV, and others), ``.SLOW`` files (Pharsighted camera), and -``numpy.ndarray`` objects of shape ``(n time points, image height, image width)``. +.. code:: python + + from pyidi import VideoReader, LucasKanade + + video = VideoReader('measurement.cih') # 1. read the recording + lk = LucasKanade(video) # 2. pick a method + lk.set_points(points) # 3. say where to look + lk.configure(roi_size=(21, 21)) # and how + displacements = lk.get_displacements() # 4. run it + +The result is a :class:`numpy.ndarray` of shape ``(n_points, n_frames, 2)``, +in **pixels**, relative to the reference frame. The last axis is +``(row, column)`` — see :ref:`point-conventions`. + +The rest of this page walks through each step. -Loading the video ------------------ -First create the :py:class:`VideoReader ` object: +1. Loading the video +-------------------- .. code:: python @@ -19,114 +31,191 @@ First create the :py:class:`VideoReader ` object video = VideoReader('filename.cih') -Setting the method ------------------- -The video object must be passed to the :py:class:`IDIMethod ` class. -Available methods are: +``VideoReader`` accepts Photron ``.cih``/``.cihx``, Phantom ``.cine``, +Pharsighted ``.SLOW``, image sequences (PNG, TIFF, BMP, JPEG, GIF), ordinary +video files (MP4, AVI, MKV, MOV and others), and :class:`numpy.ndarray` +objects of shape ``(n_time_points, image_height, image_width)``. + +Check that the frame rate is what you think it is — a wrong ``fps`` silently +puts every identified frequency in the wrong place: + +.. code:: python -* :py:class:`SimplifiedOpticalFlow ` + print(video.N, video.image_height, video.image_width, video.fps) -* :py:class:`LucasKanade ` + video.configure(fps=10000) # if the file does not carry it, or carries it wrong -* :py:class:`DirectionalLucasKanade ` +See :doc:`video_reader` for the details of each format. -* :py:class:`DIC ` +2. Choosing a method +-------------------- -To use the Simplified Optical Flow method, the object must be instantiated: +The video object is passed to one of the +:class:`IDIMethod ` subclasses: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Method + - Use it when + * - :class:`SimplifiedOpticalFlow ` + - You want a fast first look at a whole field of points, with small + displacements (well below a pixel). + * - :class:`LucasKanade ` + - This is the default choice: iterative sub-pixel translation of a + subset, accurate over larger displacements. + * - :class:`DirectionalLucasKanade ` + - The motion is (or is assumed to be) along one known direction per + point — this buys accuracy on features that are only well defined + across that direction, such as a single edge. + * - :class:`DIC ` + - You need more than translation: strain and in-plane rotation of each + subset. .. code:: python from pyidi import SimplifiedOpticalFlow - + sof = SimplifiedOpticalFlow(video) -After the method object is instantiated, the points can be set and the arguments can be configured. +:doc:`disp_id_methods` covers what each method does and how to configure it. -For more details on the available methods, see the currently implemented :ref:`implemented_disp_id_methods`. +3. Setting the points +--------------------- -Setting the points ------------------- -Displacements are computed for certain points or certain regions of interest that are represented by a point. +.. _point-conventions: -Points must be of shape ``n_points x 2``: +Point conventions +^^^^^^^^^^^^^^^^^ -.. code:: python +Displacements are computed at points, each of which stands for the subset +(region of interest) drawn around it. Points are an array of shape +``(n_points, 2)`` in **image coordinates: row first, column second**. - points = [[1, 2], - [1, 5], - [2, 10]] +.. code:: python -where the first column indicates indices along **axis 0**, and the second column indices along **axis 1**. + points = [[1, 2], # row 1, column 2 + [1, 5], # row 1, column 5 + [2, 10]] # row 2, column 10 -The points must be passed to ``method`` object: +The first column is the **row** (``y``, axis 0) and the second is the +**column** (``x``, axis 1). The same convention applies to the result array +and to ``roi_size=(vertical, horizontal)``. .. code:: python sof.set_points(points=points) -If the points are not known, a :ref:`point-selection` or newer :ref:`napari` can be used to select the points. +``set_points()`` validates its input rather than accepting anything: + +* empty input, input that is not 2-D, or a wrong number of columns raises + ``ValueError``; +* coordinates outside the image raise ``ValueError``, listing the offending + points; +* sub-pixel (float) coordinates are rounded to the nearest pixel, with a + warning saying how many were changed. + +Selecting points interactively +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Configuring the method ----------------------- -The method can be configured using: +If you do not already know where the points go, pick them in the +:ref:`SelectionGUI ` or the :ref:`napari viewer `. +The GUI instance can be passed straight to ``set_points()``: .. code:: python - - sof.configure(...) + + from pyidi import SelectionGUI + + gui = SelectionGUI(video, subset_size=21) + sof.set_points(gui) # or: sof.set_points(gui.points) + +4. Configuring +-------------- + +Every method exposes its parameters through ``configure()``: + +.. code:: python + + lk.configure(roi_size=(21, 21), max_nfev=20, tol=1e-8, processes=1) + +Each argument is stored as an attribute of the same name, which is what makes +an analysis reproducible: the settings are written to ``settings.json`` next +to the results, and can be reloaded later. .. note:: - Some of the methods enable the multiprocessing option. By setting the number of processes, the - points are divided into groups and each group is processed in a separate process. + Most methods can spread the work over several processes. With + ``processes`` greater than one, the points are split into groups and each + group is handled by a separate process. + + In a script (as opposed to a Jupyter notebook), multiprocessing code must + be guarded: - A caveat is that when using the multiprocessing option in a shell (not jupyter notebook), the - code must be run in a ``if __name__ == '__main__':`` block. + .. code:: python + if __name__ == '__main__': + displacements = lk.get_displacements() -Get displacement ----------------- -Finally, displacements can be identified: +5. Getting the displacements +---------------------------- .. code:: python displacements = sof.get_displacements() -Saved analysis --------------- +``get_displacements()`` also accepts configuration keyword arguments, so the +configure step can be folded in: -The settings of the analysis and the identified displacements are saved in a directory next -to the loaded ``cih_file``. +.. code:: python -Directory content before the analysis: + displacements = lk.get_displacements(roi_size=(21, 21), processes=4) -- video_to_analyze.cih +The result has shape ``(n_points, n_frames, 2)``. To get the displacement +history of point ``i`` in the row direction: -Directory content after the analysis: +.. code:: python -* video_to_analyze.cih -* video_to_analyze_pyidi_analysis + u_row = displacements[i, :, 0] + u_col = displacements[i, :, 1] - * analysis_001 - - * points.pkl - * results.pkl - * settings.txt +.. warning:: -Loading saved analysis ----------------------- + Results may contain ``NaN``. A point that cannot be tracked no longer + aborts the analysis — it goes to ``NaN`` from the frame at which it was + lost, and the rest of the points are computed normally. Use ``np.nanmax`` + and friends, and check ``lk.failed_points``. See :ref:`failed-points`. -The saved analysis can be loaded using the ``load_analysis`` function: +A complete example +------------------ .. code:: python - from pyidi import load_analysis + import numpy as np + import matplotlib.pyplot as plt + from pyidi import VideoReader, LucasKanade - analysis_path = 'video_to_analyze_pyidi_analysis/analysis_001' + video = VideoReader('data/data_synthetic.cih') - video_loaded, info_dict = load_analysis(analysis_path) + lk = LucasKanade(video) + lk.set_points(points=np.array([[31, 35], [31, 215]])) + lk.configure(roi_size=(11, 11), int_order=3) -Now we can access the ``video_loaded`` attributes, e.g.: + displacements = lk.get_displacements() -.. code:: python + t = np.arange(video.N) / video.fps + plt.plot(t, displacements[0, :, 0], label='point 0, row') + plt.plot(t, displacements[1, :, 0], label='point 1, row') + plt.xlabel('time [s]') + plt.ylabel('displacement [pixel]') + plt.legend() + plt.show() + +Next +---- - video_loaded.displacements +* :doc:`results` — where the results are saved, how to reload them, and how to + view them. +* :doc:`disp_id_methods` — the methods in detail. +* :doc:`../postprocessing/eulerian_magnification` — see the motion before you + measure it. diff --git a/docs/source/quick_start/disp_id_methods.rst b/docs/source/quick_start/disp_id_methods.rst index c0601c9..d9fa27a 100644 --- a/docs/source/quick_start/disp_id_methods.rst +++ b/docs/source/quick_start/disp_id_methods.rst @@ -1,79 +1,329 @@ .. _implemented_disp_id_methods: Displacement identification methods -=============================================== +=================================== + +Four methods are implemented. They share the same interface — construct with a +:class:`~pyidi.video_reader.VideoReader`, ``set_points()``, ``configure()``, +``get_displacements()`` — and differ in what they solve for and at what cost. + +Choosing a method +----------------- + +.. list-table:: + :header-rows: 1 + :widths: 22 16 20 20 22 + + * - Method + - Solves for + - Displacement range + - Speed + - Typical use + * - :ref:`Simplified Optical Flow ` + - 2 translations, from the image gradient + - well below one pixel + - fastest + - a first look at a dense field of points + * - :ref:`Lucas-Kanade ` + - 2 translations, iteratively + - several pixels + - fast (compiled) + - the default choice + * - :ref:`Directional Lucas-Kanade ` + - 1 translation along a known direction + - several pixels + - fast (compiled) + - motion along a known axis; features defined in one direction only + * - :ref:`DIC ` + - 6 (affine) or 3 (rigid) warp parameters + - several pixels + - slowest + - strain and in-plane rotation, not just translation + +If you are unsure, start with ``LucasKanade``. + +.. _method-sof: Simplified Optical Flow (SOF) ----------------------------- +SOF estimates displacement directly from the image gradient and the intensity +change relative to a reference image. There is no iteration, which makes it +very fast, but it is a linearisation: it is only valid while the motion stays +well below one pixel. + +.. code:: python + + from pyidi import VideoReader, SimplifiedOpticalFlow + + video = VideoReader('measurement.cih') + + sof = SimplifiedOpticalFlow(video) + sof.set_points(points) + sof.configure(subset_size=3, reference_range=(0, 100)) + + displacements = sof.get_displacements() + +.. list-table:: + :header-rows: 1 + :widths: 26 14 60 + + * - Parameter + - Default + - Meaning + * - ``subset_size`` + - ``3`` + - Size of the averaging subset around each point. + * - ``reference_range`` + - ``(0, 100)`` + - Frames averaged into the reference image. Averaging suppresses sensor + noise in the reference. + * - ``pixel_shift`` + - ``False`` + - Track the integer part of the displacement by shifting the subset, + extending the usable range beyond a fraction of a pixel. + * - ``convert_from_px`` + - ``1.`` + - Distance unit per pixel, if you want the result in physical units. + * - ``mean_n_neighbours`` + - ``0`` + - Average the result over this many neighbouring points, to trade spatial + resolution for noise. + * - ``zero_shift`` + - ``False`` + - Shift each signal so its mean is zero. + * - ``frame_range`` + - ``'all'`` + - Part of the recording to process. + +Reference: + [1] Javh, J., Slavič, J., & Boltežar, M. (2017). The subpixel resolution of optical-flow-based modal analysis. Mechanical Systems and Signal Processing, 88, 89–99. https://doi.org/10.1016/j.ymssp.2016.11.009 +.. _method-lk: + Lucas-Kanade (LK) ----------------- +The Lucas-Kanade method iteratively solves for the translation of each subset +between the reference image and the current frame, to sub-pixel accuracy. This +is the workhorse method. + +.. code:: python + + from pyidi import VideoReader, LucasKanade + + video = VideoReader('measurement.cih') + + lk = LucasKanade(video) + lk.set_points(points) + lk.configure(roi_size=(21, 21), max_nfev=20, tol=1e-8) + + displacements = lk.get_displacements() + +.. list-table:: + :header-rows: 1 + :widths: 26 14 60 + + * - Parameter + - Default + - Meaning + * - ``roi_size`` + - ``(9, 9)`` + - Subset size in pixels, ``(vertical, horizontal)``. Larger is more + robust and less local; it must be large enough to contain distinctive + texture. + * - ``pad`` + - ``2`` + - Padding around the subset, so the interpolation has data to work with + at the edges. + * - ``max_nfev`` + - ``20`` + - Maximum iterations per point per frame. + * - ``tol`` + - ``1e-8`` + - Convergence threshold on the displacement increment. + * - ``int_order`` + - ``3`` + - Interpolation spline order. Only ``3`` runs on the compiled kernel. + * - ``reference_image`` + - ``0`` + - Frame index, a ``(start, stop)`` tuple to average over, or an array. + * - ``frame_range`` + - ``'full'`` + - Part of the recording to process. + * - ``processes`` + - ``1`` + - Number of worker processes. + * - ``resume_analysis`` + - ``False`` + - Continue an interrupted run from its last checkpoint. See + :doc:`results`. + * - ``use_compiled_kernel`` + - ``True`` + - Use the compiled numba kernel. See below. + +Reference: + [2] Lucas, B. D., & Kanade, T. (1981). An Iterative Image Registration Technique with an Application to Stereo Vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2 (pp. 674–679). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. Retrieved from http://dl.acm.org/citation.cfm?id=1623264.1623280 +.. _lk-performance: + Performance -~~~~~~~~~~~ +^^^^^^^^^^^ The inner optimization loop is compiled with ``numba`` and parallelized over -points. This is on by default (``use_compiled_kernel=True``) and is typically one to two -orders of magnitude faster than the pure NumPy implementation:: +points. This is on by default (``use_compiled_kernel=True``) and is typically +one to two orders of magnitude faster than the pure NumPy implementation:: lk.configure(roi_size=(9, 9), use_compiled_kernel=True) +Measured against 1.3.3 on the same machine, with identical results: + +.. list-table:: + :header-rows: 1 + :widths: 52 16 16 16 + + * - Case + - 1.3.3 + - 1.4.0 + - Speed-up + * - ``data_synthetic.cih``, 200 points, 101 frames + - 7.94 s + - 0.10 s + - 77x + * - synthetic 512x512, 400 points, 150 frames + - 21.04 s + - 0.24 s + - 89x + * - ``data_synthetic.mp4``, 60 points, 10 frames + - 2.16 s + - 0.06 s + - 36x + Notes: -* The compiled kernel supports cubic interpolation only. With ``int_order`` set - to anything other than ``3`` it falls back to the NumPy implementation and - warns once. -* The kernel is compiled the first time it runs, which takes a few seconds. The - result is cached on disk, so later runs in a fresh session skip it. Set - ``NUMBA_CACHE_DIR`` if pyidi is installed somewhere the cache cannot be +* The compiled kernel supports cubic interpolation only. With ``int_order`` + set to anything other than ``3`` it falls back to the NumPy implementation + and warns once. +* The kernel is compiled the first time it runs, which takes a few seconds. + The result is cached on disk, so later runs in a fresh session skip it. Set + ``NUMBA_CACHE_DIR`` if pyIDI is installed somewhere the cache cannot be written. -* ``use_compiled_kernel=False`` selects the NumPy implementation. Results agree - with the compiled kernel to floating-point round-off, so the switch affects - speed only. If numba is not importable the same fallback happens - automatically, with a warning. +* ``use_compiled_kernel=False`` selects the NumPy implementation. Results + agree with the compiled kernel to floating-point round-off, so the switch + affects speed only. If numba is not importable the same fallback happens + automatically, with a warning. The NumPy path is itself faster than 1.3.3, + because the frame is no longer re-read for every point. * Points are parallelized with threads when ``processes=1`` (the default), and with processes when ``processes`` is greater than one. The two are never combined, so they cannot oversubscribe the CPU. +* On Linux, pyIDI requests numba's fork-safe threading layer at import time, + and switches the worker pool away from ``fork`` if a GNU OpenMP runtime is + already loaded — forking after libgomp has been used crashes the children. + This is automatic; you only need to know about it if you set + ``NUMBA_THREADING_LAYER`` yourself. + +.. _failed-points-lk: Points that cannot be tracked -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -A point placed on a uniform region, or on a single straight edge with no gradient -along it, cannot be tracked. Rather than aborting the analysis, such a point is -set to ``NaN`` from the frame at which it was lost, every other point is computed -normally, and a warning is issued. Which points failed, and why, is recorded in -``failed_points``:: +A point placed on a uniform region, or on a single straight edge with no +gradient along it, cannot be tracked. Rather than aborting the analysis, such +a point is set to ``NaN`` from the frame at which it was lost, every other +point is computed normally, and a warning is issued. Which points failed, and +why, is recorded in ``failed_points`` — see :ref:`failed-points`. - displacements = lk.get_displacements() - lost = lk.failed_points # {point_index: {'frame': ..., 'status': ...}} +.. _method-dlk: -.. warning:: +Directional Lucas-Kanade +------------------------ - The detection is best effort. It catches points whose displacement becomes - non-finite or larger than the image, but a point can return physically - implausible values well below that bound. A result without ``NaN`` is not - proof that every point tracked correctly. Check the displacements against - what the structure can plausibly do. +The directional method solves for a *single* translation along a prescribed +direction per point, instead of two independent components. Constraining the +solve this way makes it possible to track features that plain Lucas-Kanade +cannot — most usefully a single straight edge, which carries no information +along its own length but is sharply defined across it. +.. code:: python -Directional DIC ------------------------- + from pyidi import VideoReader, DirectionalLucasKanade + + video = VideoReader('measurement.cih') + + lk1d = DirectionalLucasKanade(video) + lk1d.set_points(points) + lk1d.configure(roi_size=(9, 9), dij=(1, 0), pad=(2, 2)) + + displacements = lk1d.get_displacements() + +``dij`` is the assumed motion direction as ``(di, dj)`` — row and column +components, in the convention *negative is down, positive is right*. It is +normalised automatically. A single ``(2,)`` vector applies to every point; an +``(n_points, 2)`` array gives each point its own direction: + +.. code:: python + + lk1d.set_directions(dij) # (2,) or (n_points, 2) + +Per-point directions are what the automatic feature selection in [3]_ produces, +and they are saved alongside the results (``directions.pkl``) so a reloaded +analysis keeps them. + +.. note:: + + ``pad`` here is a ``(pad_y, pad_x)`` pair, unlike ``LucasKanade.pad`` which + is a scalar. A bare integer is accepted and broadcast to both axes. + +.. _rigid-body-motion: + +Prescribed rigid-body motion +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When the whole structure translates — a machine on soft mounts, a specimen on +a shaker, a camera that drifts — the interesting local motion sits on top of a +large common motion. If the rigid-body translation is known independently (for +example from a fiducial marker, see :doc:`../fiducial_marker`), it can be +prescribed: + +.. code:: python + + lk1d.configure(roi_size=(9, 9), dij=(1, 0)) + lk1d.set_rigid_body_motion(rbm_ij) # (n_time_points, 2), in pixels - [3] Masmeijer T., Habtour E., Zaletelj K. & Slavič J. (2025). Directional DIC method with automatic feature selection. Mechanical Systems and Signal Processing, 224. https://doi.org/10.1016/j.ymssp.2024.112080 + displacements = lk1d.get_displacements() + +Two things then happen. The tracking window of every point *follows* the +prescribed motion, so the feature stays inside the subset even when the +rigid-body translation is many pixels — the local solve never has to chase it. +And the prescribed motion is subtracted back out of the result, so +``displacements`` reports the local motion **relative to the rigid body**, +not each point's absolute position in the frame. + +.. code:: python + + lk1d.set_rigid_body_motion(None) # back to zero (requires configure() first) + +Limitations, as currently implemented: + +* Only the component of the rigid-body motion aligned with each point's + tracking direction ``dij`` is used. A rigid-body translation perpendicular + to ``dij`` is not compensated. +* The shape of ``rbm_ij`` is not validated at runtime; it must be + ``(n_time_points, 2)`` for the frame range being processed. +* If ``set_rigid_body_motion`` is never called, it defaults to zero and the + analysis behaves exactly as before. Performance -~~~~~~~~~~~ +^^^^^^^^^^^ ``DirectionalLucasKanade`` uses the same compiled kernel machinery as Lucas-Kanade, with the two-parameter translation solve replaced by the -one-parameter solve along the prescribed direction. Everything in the -`Lucas-Kanade performance notes <#performance>`_ above applies here too: -``use_compiled_kernel=True`` by default, cubic interpolation only, threads when -``processes=1`` and processes otherwise:: +one-parameter solve along the prescribed direction. Everything in +:ref:`lk-performance` applies here too: ``use_compiled_kernel=True`` by +default, cubic interpolation only, threads when ``processes=1`` and processes +otherwise:: lk1d.configure(roi_size=(9, 9), dij=(1, 0), use_compiled_kernel=True) @@ -84,51 +334,73 @@ one-parameter solve along the prescribed direction. Everything in the updating; the behaviour they asked for is now the default. Points that cannot be tracked -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The directional method only ever sees the image gradient projected onto the search direction, so it is easier to lose a point here than with plain Lucas-Kanade: a feature with a strong gradient across the direction but none along it is untrackable, however well defined it looks. Such a point is set to -``NaN`` from the frame at which it was lost, the rest of the analysis continues, -and the detail is recorded in ``failed_points``, exactly as for Lucas-Kanade. -The same best-effort warning applies. +``NaN`` from the frame at which it was lost, the rest of the analysis +continues, and the detail is recorded in ``failed_points``, exactly as for +Lucas-Kanade. + +.. [3] Masmeijer T., Habtour E., Zaletelj K. & Slavič J. (2025). Directional DIC method with automatic feature selection. Mechanical Systems and Signal Processing, 224. https://doi.org/10.1016/j.ymssp.2024.112080 + +.. _method-dic: Digital Image Correlation (DIC) ------------------------------- -Full-field 2D Digital Image Correlation method using Inverse Compositional Gauss-Newton -(IC-GN) optimization with the Zero Normalized Sum of Squared Differences (ZNSSD) criterion. +Full-field 2D Digital Image Correlation using Inverse Compositional +Gauss-Newton (IC-GN) optimization with the Zero Normalized Sum of Squared +Differences (ZNSSD) criterion. Unlike the methods above, DIC solves for the +full deformation of each subset, not just its translation. -This method is a port of the **pyDIC** library by the LADISK research group -(University of Ljubljana, Faculty of Mechanical Engineering) into the pyidi -multi-point ``IDIMethod`` framework. The original implementation is available at -https://github.com/ladisk/pyDIC and provides the algorithmic basis for this -class (gradient kernel, Jacobians, steepest-descent images, Hessian, -inverse-compositional warp update, ZNSSD error image). Please cite both the -underlying algorithm and the pyDIC repository when using this method. +.. code:: python + + from pyidi import VideoReader, DIC + + video = VideoReader('measurement.cih') + + dic = DIC(video) + dic.set_points(points) + dic.configure(roi_size=(21, 21), warp='affine', max_nfev=100, tol=1e-6) + + displacements = dic.get_displacements() Two warp models are supported: -* ``warp='affine'`` (default, 6 parameters): full first-order shape function with - translation, normal strains, shear and rotation. Parameter vector +* ``warp='affine'`` (default, 6 parameters): full first-order shape function + with translation, normal strains, shear and rotation. Parameter vector ``[du/dx, du/dy, u, dv/dx, dv/dy, v]``. -* ``warp='rigid'`` (3 parameters): translation and in-plane rotation. Parameter vector - ``[u, v, phi]``. +* ``warp='rigid'`` (3 parameters): translation and in-plane rotation. + Parameter vector ``[u, v, phi]``. + +In addition to the standard ``displacements`` array of shape +``(n_points, n_frames, 2)``, the method exposes the full converged warp +parameters as ``self.warp_params`` of shape ``(n_points, n_frames, n_param)``. +From the affine parameters one can directly recover in-plane strains and +rotation:: -In addition to the standard ``displacements`` array of shape ``(n_points, n_frames, 2)``, -the method exposes the full converged warp parameters as ``self.warp_params`` of shape -``(n_points, n_frames, n_param)``. From the affine parameters one can directly recover -in-plane strains and rotation, e.g.:: + eps_xx = dic.warp_params[..., 0] + eps_yy = dic.warp_params[..., 4] + shear_xy = 0.5 * (dic.warp_params[..., 1] + dic.warp_params[..., 3]) + rotation = 0.5 * (dic.warp_params[..., 3] - dic.warp_params[..., 1]) # rad - eps_xx = idi.warp_params[..., 0] - eps_yy = idi.warp_params[..., 4] - shear_xy = 0.5 * (idi.warp_params[..., 1] + idi.warp_params[..., 3]) - rotation = 0.5 * (idi.warp_params[..., 3] - idi.warp_params[..., 1]) # rad +``prefilter_gauss=True`` (the default) uses the Gauss-prefiltered finite +difference kernel ``[-0.446, 0, 0.446]`` for the reference gradient, instead +of ``[-0.5, 0, 0.5]``. + +This method is a port of the **pyDIC** library by the LADISK research group +(University of Ljubljana, Faculty of Mechanical Engineering) into the pyIDI +multi-point ``IDIMethod`` framework. The original implementation is available +at https://github.com/ladisk/pyDIC and provides the algorithmic basis for this +class (gradient kernel, Jacobians, steepest-descent images, Hessian, +inverse-compositional warp update, ZNSSD error image). Please cite both the +underlying algorithm and the pyDIC repository when using this method. -The implementation is a port of the pyDIC algorithm (https://github.com/ladisk/pyDIC) -into the pyidi multi-point method framework. +References: [4] Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 Years On: A Unifying Framework. International Journal of Computer Vision, 56(3), 221-255. https://doi.org/10.1023/B:VISI.0000011205.11775.fd - [5] Pan, B., Qian, K., Xie, H., & Asundi, A. (2009). Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review. Measurement Science and Technology, 20(6), 062001. https://doi.org/10.1088/0957-0233/20/6/062001 \ No newline at end of file + [5] Pan, B., Qian, K., Xie, H., & Asundi, A. (2009). Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review. Measurement Science and Technology, 20(6), 062001. https://doi.org/10.1088/0957-0233/20/6/062001 diff --git a/docs/source/quick_start/feature_selection.gif b/docs/source/quick_start/feature_selection.gif new file mode 100644 index 0000000..21ca93a Binary files /dev/null and b/docs/source/quick_start/feature_selection.gif differ diff --git a/docs/source/quick_start/feature_selection.rst b/docs/source/quick_start/feature_selection.rst new file mode 100644 index 0000000..7fe3aa5 --- /dev/null +++ b/docs/source/quick_start/feature_selection.rst @@ -0,0 +1,328 @@ +.. _point-selection: +.. _feature-selection: + +Point selection +=============== + +``SelectionGUI`` finds the points for you. Instead of placing subsets on a grid +and then discarding the poor ones, it scores *every* pixel of the image and +picks the best-separated maxima inside the region you drew. On a random speckle +pattern or an intricate structure that is the difference between sampling where +the features happen to be and sampling where the grid happens to fall. + +Placing the subsets yourself is not a separate mode: draw a region and set its +row to the ``points`` role and it lays them out on a grid without scoring +anything, alongside regions that *are* scored. See `Mask`_. + +.. versionchanged:: 1.4 + + ``SelectionGUI`` names this window as of 1.4. The one it replaced is + :doc:`SelectionGUIOld `, deprecated and removed in 1.5. + It takes the same arguments and returns the same ``(row, col)`` array, so a + script that opens the window and reads its points needs no edit; that page + lists what does not carry over. + +It needs the ``[qt]`` extra: + +.. code:: bash + + pip install pyidi[qt] + +Find, then trim +--------------- + +Three things happen, and the window shows them as two tabs: + +**Evaluate + select** — *evaluate* scores every subset position in the image at +once, and *select* turns that score into points with a threshold, a separation +and a cap. The two are tuned against each other, since changing the evaluator +changes what a threshold means, so they share one panel. + +**Mask** — regions say where points are allowed. The window opens with a +``Whole image`` row already in the selections list, so there are candidates over +the whole frame from the start and this tab is where you trim them: paint away +the clamp, drop the background, keep the part you care about. + +The selections list sits on the **Mask** tab and only there: every row in it, +and every button under it, acts on a region drawn there. The subset size, by +contrast, sits below the tabs rather than on either of them, because both +steps read it: the scoring window follows it, which is what makes it one of the +few settings that stales the score, and it is also the size of the rectangle +drawn round each point while you mask. It takes **odd values only** — a subset +is centred on its point, so an even extent has no centre to be, and the +pipeline reads one as the odd size below it anyway. ``Show score overlay`` is on +both tabs, and the two controls stay in step. + +The tabs are deliberately unnumbered and can be used in any order. Evaluation +does not depend on the mask — the score is always computed for the whole frame +and cached — so masking and scoring are not a sequence. Only the frame, the +evaluator, its parameters and the subset size feed the score; everything else +re-derives the points from the cached array, which is why editing a mask or +dragging a threshold updates while you are still moving the control. + +.. code:: python + + from pyidi import VideoReader, SelectionGUI + + video = VideoReader(input_file) + gui = SelectionGUI(video, subset_size=11) + + points = gui.points # or: gui.get_points() + +The window is modal: the call blocks until you close it. ``points`` is an +``(n_points, 2)`` integer array in **row/column** order, ready for +``set_points``. + +.. image:: feature_selection.gif + :alt: The selection window opening with points already spread over the whole + frame, a polygon then drawn corner by corner so the points outside it drop + away, and finally the separation swept to show it deciding how many points + there are. + :width: 700 + +The order in the animation is the one the interface is built around: the frame +is scored first and there are points on it before anything is drawn, the region +trims them, and the separation sets how many survive. + +Mask +---- + +The ``Whole image`` row that the window starts with is an ordinary row: uncheck +it, paint it away with ``Remove w/ brush``, or delete it. Deleting it selects +nothing — it does not silently revert to the whole frame. + +Mask rows combine as a **union**, so drawing a region while the whole frame is +still selected would change nothing at all. The first drawn region therefore +unchecks the ``Whole image`` row, with a note in the status bar; tick it again +to bring the whole frame back, or press Ctrl+Z. + +``Clear all`` starts over, which means the state the window opens in: every +selection dropped and the ``Whole image`` row seeded again. It is undoable. + +Six tools, on the right. The first four add, the last two take away: + +- **Polygon** — click to place corners; the enclosed area becomes a mask. +- **Brush** — hold Ctrl and drag to paint an area. +- **Line** — click to place vertices; points are spaced along the segments. +- **Points** — click to place individual points. +- **Remove point** — click near a point to remove it. A hand-placed point is + simply deleted, so clicking that pixel again puts it back. A *selected* point + is not stored anywhere — it is re-derived from the score every time the + selection runs — so removing one takes the ground it stands on out of the + mask: the disc it was reserving, its separation, so that nothing lands in its + place. +- **Remove w/ brush** — hold Ctrl and drag to take away everything the stroke + covers. It is the brush in reverse and shares its radius, so a stroke erases + exactly as wide as it paints, and it subtracts only the part you actually + paint over: a region keeps whatever the stroke missed, and disappears only + once nothing of it is left. + +``Remove point`` and ``Remove w/ brush`` are the same idea at two scales, +which is why they sit together. Erasing used to be a toggle inside the brush +controls, so the same tool added or subtracted depending on a button several +rows below it; now the tool *is* the answer to which one it does. + +Points under a deselect stroke are crossed out while you paint, so you can see +what the stroke is about to take before you let go. Nothing else moves until +you let go: the selection is re-run once, when the stroke lands. + +While masking, the points come in three tiers, because an empty patch otherwise +means two different things — nothing to track there, or something you have +masked away: + +- **red** — a point the selection is taking; +- **dim blue** — a feature the mask is leaving out; +- **ringed in magenta** — the points the row selected in the list accounts for. + +The dim tier is what the current settings would select over the whole frame, so +it does not move while you edit a mask: painting a region turns points from blue +to red where it lands rather than re-selecting underneath you. One consequence +is worth knowing — a blue point right at the edge of a mask need not coincide +exactly with a red one, because a selection inside a region starts its +separation afresh. + +Every region becomes a row in the ``Selections`` list, and **each row has a +role**: + +``mask`` + the row contributes its area, and the points inside it are chosen by the + selection step; +``points`` + the row contributes its coordinates directly, bypassing scoring entirely. + +Polygons and brush strokes start as ``mask``; lines and clicked points start as +``points``. ``Use as points`` / ``Use as mask`` switches a row over without +redrawing it. That is how a filtered region and a hand-placed line of points +coexist in one session: hand-picked points always survive, whatever their score, +and no automatic point is placed within the separation of one. + +Unchecking a row excludes it without deleting it. Ctrl+Z undoes adding a vertex, +moving a vertex, painting a stroke, deleting a row, removing a point, and a +deselection. + +Evaluate +-------- + +Two evaluators are built in, chosen at the top of the **Evaluate + select** +tab: + +- **Shi-Tomasi** — corner strength: high where the subset is constrained in + both directions, so a subset on a plain edge scores low (it can slide along + the edge) and one on a corner scores high. +- **Gradient in direction** — gradient strength along one chosen direction, for + when only one component of the motion matters. The direction is a + ``(row, col)`` pair, with ``X`` and ``Y`` presets and a ``Draw`` button that + lets you drag the direction out on the image. Whichever way you set it, a red + line shows the direction currently in force. + +Each evaluator describes itself in the tooltip of the ``Score`` menu. +``Show score overlay`` draws the score as a heatmap, so you can see where the +features are before committing to any points. The border where the subset window +would leave the image is drawn fully transparent — it is not scored, rather than +scored badly. + +The scoring window follows the subset size, so the score always answers the +question "how well would *this* subset track". Scores are cached per evaluator, +per parameter set and per subset size, so switching between two evaluators and +back costs nothing the second time. + +Select +------ + +Below the evaluator, on the same tab: + +- **Threshold** — by default a **quality**: a fraction of the best feature in + the region, so ``0.01`` means "at least a hundredth as good as the best one + here". The slider is logarithmic, because the useful settings span three + decades — featureless background scores around ``0.001`` of the best feature + and a strong corner around ``1``. +- **Separation** — the distance no two points may come closer than, and so the + control for how many you get: lower it for more. This is the one knob that + decides density. +- **Maximum points** — a safety valve. When it stops the selection the panel + says so, because a cap has no other symptom: it simply stops adding points, + and the result reads as though the threshold or the separation did it. +- **Keep every n-th** — decimation. It thins the points that were *already* + selected, leaving the survivors exactly where they are. Use it when the + selection is right and only the count is too high for the computation you are + about to run. Widening the separation instead re-selects and moves every + point, which is a different thing; hand-placed points are never decimated. + +Why decimation is not the density control +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The obvious way to thin a dense selection is to keep every n-th of the pixels +above the threshold, and it does not work. Measured on a 1024×1024 frame with +357 000 pixels above the threshold, thinned to twenty thousand points: + +=============================== ========== ================== +rule median gap pairs under 3 px +=============================== ========== ================== +every n-th, best first 2.0 px 78 % +every n-th, in scan order 1.0 px 92 % +separation *n* ≥ *n* px 0 % +=============================== ========== ================== + +Keeping every n-th by score fails because consecutive ranks are neighbours on +the same feature. Keeping every n-th in scan order fails because the stride +aliases against the row length and lands in columns. Either way most of the +subsets end up on top of another one, which is what the selection step exists to +prevent — so decimation stays what it is good at, thinning a selection that is +already well spread. + +The separation is enforced by a greedy walk from best to worst, accepting a +candidate only if nothing already accepted is within the separation of it. That +walk is exact but linear in the *candidates*, and a loose threshold leaves +hundreds of thousands of them — 40 ms to 300 ms, which no slider can drag. So +the candidates are reduced first, to the best pixel in each cell of a grid half +the separation across. What that approximation costs is yield: at a separation +of 11 it finds 1708 points where the exact walk finds 2193, in 9 ms instead of +39. What it does not cost is the guarantee — the walk still runs, so the +separation still holds exactly — and a point count is what the separation +control is for adjusting anyway. + +A masked selection works inside the mask's bounding box rather than over the +whole frame, since nothing outside the mask was ever eligible. The box is +snapped back to a whole reduction cell so the block grid falls exactly where it +would have on the frame, which is what makes the answer identical rather than +merely similar. A region drawn on a large frame therefore costs the region: +placing a polygon corner over a 600 × 600 region on a 2560 × 1600 frame takes +30 ms. + +Why quality and not a percentile +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A percentile ranks *pixels*, and on a dense score image the pixels are +overwhelmingly background. On a typical frame the 90th percentile of the score +is under a five-hundredth of the best feature, so nine tenths of a percentile +slider's travel is spent inside the featureless area: lowering it does not relax +the quality bar, it floods the frame with background. The separation then +spreads that background out evenly, which makes it look as though the spacing is +at fault. + +Quality is measured against the best feature instead, so the whole slider stays +inside the range that actually distinguishes features. The reference is the +99.9th percentile of the scores rather than their literal maximum, so one dust +mote or specular highlight cannot drag every useful setting into the floor of +the slider. + +``percentile of scores`` is still available in the ``Threshold`` menu. It is the +right rule for the ``lattice`` selector, where the candidates have already been +spaced out and there is no background flood to worry about. + +A third rule, a fraction of the literal maximum, was offered and then dropped: +it is the same rule as quality with a reference that one bright pixel can move, +so on any frame worth using it is indistinguishable and on a bad one it is +worse. + +The ``lattice`` selector places points on a regular grid of a given pitch +instead of at local maxima, optionally dropping cells that score too low. The +settings a selector has no use for are hidden rather than greyed out, so +switching to it swaps ``Separation`` for ``Grid pitch``. Use it +when you want even coverage rather than the best features — full-field work, +typically. It is a choice of selector, not a different mode. + +Without the GUI +--------------- + +The pipeline is a plain module and imports without Qt, so the same selection can +be scripted: + +.. code:: python + + from pyidi.selection import Entry, select_points + + region = Entry('polygon', [(20, 20), (20, 200), (180, 200), (180, 20)]) + points = select_points(image, [region], subset_size=11, + evaluator='shi_tomasi', separation=15, threshold=0.01) + +``Entry`` geometry is in ``(row, col)``, as is the returned array. For repeated +work on one frame, ``SelectionPipeline`` keeps the score cache alive across +parameter changes: + +.. code:: python + + from pyidi.selection import SelectionPipeline + + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('polygon', [(20, 20), (20, 200), (180, 200), (180, 20)]) + + for threshold in (0.2, 0.05, 0.01): + pipeline.selector_params['threshold'] = threshold + print(threshold, len(pipeline.points)) # scored once, not three times + +Scoring can be extended without touching any GUI code — register a function that +turns an image and a window into a score array, together with descriptors for +its parameters, and it appears in the evaluator menu: + +.. code:: python + + from pyidi.selection import Evaluator, Parameter, register_evaluator + + register_evaluator(Evaluator( + name='variance', + display_name='Local variance', + function=my_variance_score, # f(image, window) -> ndarray + parameters=(), + description='Contrast inside the subset.', + )) diff --git a/docs/source/quick_start/make_feature_selection_animation.py b/docs/source/quick_start/make_feature_selection_animation.py new file mode 100644 index 0000000..b2afb87 --- /dev/null +++ b/docs/source/quick_start/make_feature_selection_animation.py @@ -0,0 +1,160 @@ +"""Generate ``feature_selection.gif`` for the point-selection docs page. + +Produces an animated GIF of ``SelectionGUI`` (``pyidi/GUIs/feature_selection.py``) +working on the ``data/data_synthetic.cih`` demo video, in the order the interface +is meant to be used: + +1. the window as it opens -- already scored, with points over the whole frame, + because the ``Whole image`` mask row is seeded on startup; +2. the ``Mask`` tab, with a polygon drawn corner by corner; the points outside it + drop to the dim tier as soon as the polygon closes and becomes a real mask; +3. back on ``Evaluate + select``, the separation swept down and up, which is the + control that decides how many points there are. + +That order is the pitch: the score comes first and the region trims it, rather +than a grid being placed and then filtered. ``make_selection_animation.py`` +does the same job for the deprecated ``SelectionGUIOld``. + +Why the headless setup is needed +-------------------------------- +``SelectionGUI`` is a full Qt application whose constructor calls ``show()`` and +then enters the event loop, ending in ``sys.exit(app.exec())`` unless ``sys.ps1`` +is set. To build the window, drive it and grab pixels from it in a plain script: + +* ``QT_QPA_PLATFORM=offscreen`` must be set *before* Qt is imported, so Qt + renders into its software framebuffer instead of opening a display; +* ``sys.ps1`` is set before construction, so the constructor takes the + interactive branch rather than ``sys.exit(...)``; +* ``QtWidgets.QApplication.exec`` is monkeypatched to a no-op, because the + interactive branch still calls ``app.exec()``, which would block with nothing + driving it. + +The window is then driven through the same calls a real click makes -- see +``on_mouse_click``: ``add_vertex``, ``_retire_whole_image``, ``refresh``. + +Run with: + + QT_QPA_PLATFORM=offscreen python docs/source/quick_start/make_feature_selection_animation.py + +This (re)writes ``docs/source/quick_start/feature_selection.gif`` in place. +""" +import os +import sys + +os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") + +import imageio.v3 as iio # noqa: E402 +import numpy as np # noqa: E402 +from PyQt6 import QtGui, QtWidgets # noqa: E402 + +from pyidi.GUIs.feature_selection import STEP_FIND, STEP_MASK, SelectionGUI # noqa: E402 +from pyidi.video_reader import VideoReader # noqa: E402 + +HERE = os.path.dirname(os.path.abspath(__file__)) +DATA_PATH = os.path.join(HERE, "..", "..", "..", "data", "data_synthetic.cih") +OUT_PATH = os.path.join(HERE, "feature_selection.gif") + +#: Polygon corners as ``(row, col)`` -- the convention this GUI uses throughout, +#: unlike ``SelectionGUIOld``, which stores ``(x, y)``. Chosen to sit inside the +#: 128x256 demo frame with margin on all sides. +POLYGON_VERTICES = [ + (100, 40), + (35, 30), + (15, 110), + (40, 215), + (105, 200), +] + +#: Separation values swept on the ``Evaluate + select`` tab, to show that this +#: is the density control. Starts at the default so the first step is visible. +SEPARATIONS = [11, 7, 5, 8, 14] + +#: Per-frame display time (ms). The encoder deduplicates identical consecutive +#: frames, so the hold at each end is a long duration rather than repeated frames. +STEP_DURATION_MS = 550 +HOLD_DURATION_MS = 1800 + +#: Frames are rendered large and downscaled, which is cheaper than a small +#: window and keeps the text legible. +OUTPUT_WIDTH = 700 + + +def grab_frame(window): + """Process pending Qt events and grab the window as an (H, W, 4) uint8 array.""" + QtWidgets.QApplication.processEvents() + QtWidgets.QApplication.processEvents() + image = window.grab().toImage().convertToFormat(QtGui.QImage.Format.Format_RGBA8888) + width, height = image.width(), image.height() + ptr = image.bits() + ptr.setsize(height * width * 4) + return np.frombuffer(ptr, dtype=np.uint8).reshape((height, width, 4)).copy() + + +def downscale(frame, target_width): + """Nearest-neighbour downscale of an (H, W, C) array to ``target_width`` columns.""" + h, w = frame.shape[:2] + if w <= target_width: + return frame + scale = target_width / w + target_height = max(1, int(round(h * scale))) + col_idx = (np.arange(target_width) / scale).astype(int).clip(0, w - 1) + row_idx = (np.arange(target_height) / scale).astype(int).clip(0, h - 1) + return frame[row_idx][:, col_idx] + + +def main(): + sys.ps1 = ">>> " # Make SelectionGUI think it's running interactively. + QtWidgets.QApplication.exec = lambda self=None: 0 # Neutralise the blocking event loop. + + video = VideoReader(DATA_PATH) + window = SelectionGUI(video, subset_size=15) + # Fixed rather than merely resized: the status bar and the select-tab note + # change length as the window is driven, and a resize between two grabs + # would give the encoder frames of different shapes. + window.setFixedSize(1250, 820) + + frames = [] + + # 1. As opened: the whole frame is masked, so there are points to look at + # before anything has been drawn. + window.select_step(STEP_FIND) + window.refresh() + frames.append(grab_frame(window)) + + # 2. The Mask tab, then the polygon corner by corner. This is exactly what + # on_mouse_click does for a left click with the polygon tool active. + window.select_step(STEP_MASK) + window.select_tool('polygon') + frames.append(grab_frame(window)) + + for vertex in POLYGON_VERTICES: + window.add_vertex(vertex) + window._retire_whole_image() + window.refresh() + frames.append(grab_frame(window)) + + # 3. Back to Evaluate + select, sweeping the separation. + window.select_step(STEP_FIND) + window.refresh() + frames.append(grab_frame(window)) + + for separation in SEPARATIONS: + window.separation_spin.setValue(separation) + window.refresh() + print(f"separation {separation:>3} px -> {len(window.get_points()):>4} points") + frames.append(grab_frame(window)) + + # Belt and braces: the offscreen platform plugin does not implement + # propagateSizeHints(), so trim to the common extent before encoding. + height = min(f.shape[0] for f in frames) + width = min(f.shape[1] for f in frames) + frames = [downscale(f[:height, :width, :3], OUTPUT_WIDTH) for f in frames] + durations = ([HOLD_DURATION_MS] + + [STEP_DURATION_MS] * (len(frames) - 2) + + [HOLD_DURATION_MS]) + iio.imwrite(OUT_PATH, frames, duration=durations, loop=0) + print(f"Wrote {OUT_PATH} ({len(frames)} frames)") + + +if __name__ == "__main__": + main() diff --git a/docs/source/quick_start/make_selection_animation.py b/docs/source/quick_start/make_selection_animation.py new file mode 100644 index 0000000..2bd1170 --- /dev/null +++ b/docs/source/quick_start/make_selection_animation.py @@ -0,0 +1,141 @@ +"""Generate ``selection.gif`` for the points-selection docs page. + +Produces an animated GIF of the PyQt6-based ``SelectionGUIOld`` +(``pyidi/GUIs/subset_selection.py``) building up a Grid selection on the +``data/data_synthetic.cih`` demo video: an empty frame, the polygon +vertices of the selection region appearing one at a time, the subset +grid filling in as soon as the polygon closes, and the finished +selection held for a few extra frames. + +Why the headless setup is needed +--------------------------------- +``SelectionGUIOld`` is a full Qt application (``QtWidgets.QMainWindow``) +built for interactive use: its constructor calls ``self.show()`` and +then starts the Qt event loop, ending with +``sys.exit(app.exec())`` whenever ``sys.ps1`` is not set (i.e. a plain +script run, as opposed to an interactive interpreter). That is fine +when a person is using the GUI, but fatal for a script that wants to +construct the window, poke at its state, and grab pixels from it: + +* ``QT_QPA_PLATFORM=offscreen`` must be set *before* Qt is imported, so + Qt renders to its software framebuffer instead of trying to open a + real display. +* ``sys.ps1`` is set before constructing ``SelectionGUIOld``, so its + constructor takes the "interactive" branch instead of + ``sys.exit(...)``. +* ``sys.ps1`` alone is not enough: the "interactive" branch still calls + ``app.exec()``, which blocks in the Qt event loop with nothing driving + it. ``QtWidgets.QApplication.exec`` is therefore also monkeypatched to + a no-op returning 0, so construction returns immediately with a fully + built, shown window that this script can then drive by hand (appending + vertices directly to a selection entry's ``geometry`` and calling the + GUI's own display/recompute methods, the same calls ``handle_grid_drawing`` + makes on a real click). + +Run with: + + QT_QPA_PLATFORM=offscreen python docs/source/quick_start/make_selection_animation.py + +This (re)writes ``docs/source/quick_start/selection.gif`` in place. +""" +import os +import sys + +os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") + +import imageio.v3 as iio # noqa: E402 +import numpy as np # noqa: E402 +from PyQt6 import QtGui, QtWidgets # noqa: E402 + +from pyidi.GUIs.subset_selection import SelectionGUIOld # noqa: E402 +from pyidi.video_reader import VideoReader # noqa: E402 + +HERE = os.path.dirname(os.path.abspath(__file__)) +DATA_PATH = os.path.join(HERE, "..", "..", "..", "data", "data_synthetic.cih") +OUT_PATH = os.path.join(HERE, "selection.gif") + +#: Vertices (x, y) of the polygon used for the Grid selection, chosen to +#: sit comfortably inside the 256x128 demo frame with margin on all sides. +POLYGON_VERTICES = [ + (40, 100), + (30, 35), + (110, 15), + (215, 40), + (200, 105), +] + +#: Per-frame display time (milliseconds). The GIF encoder deduplicates +#: identical consecutive frames, so the "hold" at the end is done by +#: giving the final frame a long duration rather than by repeating it. +STEP_DURATION_MS = 500 +HOLD_DURATION_MS = 2500 + +#: Output size (window is built larger for a crisper render, then frames +#: are downscaled to this width to keep the GIF file size small). +OUTPUT_WIDTH = 700 + + +def grab_frame(window): + """Process pending Qt events and grab the window as an (H, W, 4) uint8 array.""" + QtWidgets.QApplication.processEvents() + QtWidgets.QApplication.processEvents() + pixmap = window.grab() + image = pixmap.toImage().convertToFormat(QtGui.QImage.Format.Format_RGBA8888) + width, height = image.width(), image.height() + ptr = image.bits() + ptr.setsize(height * width * 4) + arr = np.frombuffer(ptr, dtype=np.uint8).reshape((height, width, 4)).copy() + return arr + + +def downscale(frame, target_width): + """Nearest-neighbour downscale of an (H, W, C) array to ``target_width`` columns.""" + h, w = frame.shape[:2] + if w <= target_width: + return frame + scale = target_width / w + target_height = max(1, int(round(h * scale))) + col_idx = (np.arange(target_width) / scale).astype(int).clip(0, w - 1) + row_idx = (np.arange(target_height) / scale).astype(int).clip(0, h - 1) + return frame[row_idx][:, col_idx] + + +def main(): + sys.ps1 = ">>> " # Make SelectionGUIOld think it's running interactively. + QtWidgets.QApplication.exec = lambda self=None: 0 # Neutralise the blocking event loop. + + video = VideoReader(DATA_PATH) + window = SelectionGUIOld(video, subset_size=15, subset_overlap=3) + window.resize(1200, 800) + + frames = [] + + # 1. Empty frame: nothing selected yet. + frames.append(grab_frame(window)) + + # 2. Grid selection, matching what a real click does in handle_grid_drawing: + # register the "Grid 1" list entry, then add vertices one at a time, + # recomputing the ROI points (and therefore the filled grid) as soon as + # the polygon has at least 3 vertices. + grid = window.add_selection("grid") + + subset_size = window.subset_size_spinbox.value() + spacing = window.distance_spinbox.value() + + for vertex in POLYGON_VERTICES: + grid["geometry"].append(vertex) + window.recompute_entry(grid, subset_size, spacing) + window.update_geometry_display() + window.update_selected_points() + frames.append(grab_frame(window)) + + print(f"Selected subsets in final frame: {len(window.selected_points)}") + + frames = [downscale(f[:, :, :3], OUTPUT_WIDTH) for f in frames] + durations = [STEP_DURATION_MS] * (len(frames) - 1) + [HOLD_DURATION_MS] + iio.imwrite(OUT_PATH, frames, duration=durations, loop=0) + print(f"Wrote {OUT_PATH} ({len(frames)} frames)") + + +if __name__ == "__main__": + main() diff --git a/docs/source/quick_start/napari.rst b/docs/source/quick_start/napari.rst index e5fceee..733c4e9 100644 --- a/docs/source/quick_start/napari.rst +++ b/docs/source/quick_start/napari.rst @@ -5,6 +5,11 @@ Napari image viewer Interactive image viewer **napari** is implemented and can be used for viewing video and selecting points. More information about napari can be obtained `here `_. +The napari ``GUI`` covers the whole workflow — selecting points, configuring +the method and running the identification — in one window. For point selection +on its own, with regions, a brush and feature scoring, use +:ref:`SelectionGUI ` instead. + .. note:: The ``GUI`` class requires the Qt/napari dependencies. Install pyIDI with the ``[qt]`` extras before using this module:: diff --git a/docs/source/quick_start/points_selection.rst b/docs/source/quick_start/points_selection.rst index e1b2da3..726578b 100644 --- a/docs/source/quick_start/points_selection.rst +++ b/docs/source/quick_start/points_selection.rst @@ -1,46 +1,219 @@ -.. _point-selection: +.. _point-selection-old: -Point selection UI -================== +Point selection UI (deprecated) +=============================== -A convinient UI is available to make the point selection easier. +.. deprecated:: 1.4 -To use the UI, a ``VideoReader`` object must first be created: + This is the window ``SelectionGUI`` named in 1.3. It is now + ``SelectionGUIOld``, it is frozen, and it is removed in 1.5. + :doc:`feature_selection` documents the interface that ``SelectionGUI`` + names today, and constructing this one prints a ``DeprecationWarning``. + + The replacement does everything described on this page. The constructor + signature is identical and ``get_points()`` returns the same ``(row, col)`` + array, so a script that opens the window and reads its points only needs + the name changed -- or nothing at all, if it says ``SelectionGUI``. + +This page is kept for the 1.4 cycle, so that a script written against the old +window can be read alongside the code it drives. + +What does not carry over +------------------------ + +- ``get_filtered_points()`` and ``get_selected_points()``. The replacement has + one ``get_points()``: filtering is no longer a second pass over an existing + selection, it *is* the selection. +- The internal attributes -- ``selections``, ``subset_size_spinbox``, + ``candidate_points`` and the rest. Nothing in the replacement corresponds to + them. +- ``Grid`` as a mode. Draw a polygon and set its row to the ``points`` role, or + keep it a mask and choose the ``lattice`` selector. + +Scores near the image border also differ slightly, because the replacement +takes gradients over real neighbours instead of ones reflected at the subset +edge. The new value is the correct one. + +Everything below describes ``SelectionGUIOld`` as it behaves today. + +Using it +-------- + +It is a PyQt6-based tool, so the ``[qt]`` extra must be installed first: + +.. code:: bash + + pip install pyidi[qt] + +Without this extra, ``SelectionGUIOld`` can still be imported, but instantiating +it raises a ``RuntimeError``. + +To use the UI, a ``VideoReader`` object must first be created (a plain +``numpy.ndarray`` image also works): .. code:: python - from pyidi import VideoReader, SubsetSelection + from pyidi import VideoReader, SimplifiedOpticalFlow, SelectionGUIOld video = VideoReader(input_file) where ``input_file`` can be a Photron ``.cih``/``.cihx`` path, an image, a video file, a numpy array, or a ``.SLOW`` file. -A ``SubsetSelection`` object can then be created: +A ``SelectionGUIOld`` window can then be opened: .. code:: python - Points = SubsetSelection(video, roi_size=(21, 21), noverlap=0) + gui = SelectionGUIOld(video, subset_size=11, subset_overlap=0) + +The window is modal: the call blocks until you close it, and execution +continues on the next line with the selection available on the object. + +Here, ``subset_size`` is the side length (in pixels) of the +Region-Of-Interest/subset drawn around each point, and ``subset_overlap`` +sets the spacing between neighbouring subsets (the step between subset +centers is ``subset_size + subset_overlap``, so a positive value spreads the +subsets further apart and a negative value overlaps them). + +``subset_size`` can be a single int for a square subset, or a ``(height, +width)`` pair for an anisotropic one -- the same ``(vertical, horizontal)`` +convention as ``LucasKanade.configure(roi_size=(vertical, horizontal))``. In +the UI, the ``Subset Configuration`` group has a ``Square subsets`` checkbox +(checked by default) alongside the height/width spinboxes and sliders: while +checked, the width tracks the height and only one size can be set; unchecking +it frees the width spinbox/slider to be set independently. + +Selection mode +-------------- + +The window opens in **Select** mode. Five selection methods are available as +buttons on the right: + +- **Grid**: click to place the corners of a polygon; once at least three + corners are placed, a regular grid of subsets is generated inside the + polygon. ``Start new grid`` starts another grid; each one becomes its own + row in the selections list, described below. +- **Manual**: click on the image to add individual points one at a time. All + manually clicked points are collected into a single ``Manual`` row in the + list. +- **Along the line**: click to place points defining a polyline; subsets are + placed at regular intervals along its segments. ``Start new line`` starts + another line; as with ``Grid``, each one becomes its own row in the list. +- **Brush**: hold Ctrl and drag over the image to paint a region; subsets are + placed on a regular grid inside the painted area. Each stroke becomes its + own row in the list. The brush radius is set with a slider, and the + ``Deselect painted area`` toggle switches the brush to remove + already-selected subsets instead of adding new ones. Deselecting erases only + the area actually painted over: a stroke keeps whatever part of it was not + covered, and its row disappears only once nothing is left painted. +- **Remove point**: click near an existing point to remove it. The point + stays removed even if the subset size or spacing is changed afterwards. + +The ``Subset Configuration`` group lets you adjust the subset size, toggle +the subset rectangle overlay (``Show subsets``), and clear the current +selection (``Clear selections``). For ``Grid``, ``Along the line``, and +``Brush``, a ``Distance between subsets`` control sets the spacing described +above. + +The selections list +-------------------- + +Every selection made in any of the modes above — every grid, every line, +every brush stroke, and the single ``Manual`` row — is listed in the +right-hand panel as one always-visible ``selections`` list, regardless of +which mode is currently active. Each row shows the selection's label and its +current point count, e.g. ``Grid 1 — 142 pts``, updated live as the selection +changes. + +- **Clicking a row** makes it the active selection, switches the tool to + match its type so its vertices are immediately draggable, and rings its + points in the image in magenta. The ring is a Select-mode cue and is hidden + in Filter mode. +- **Each row has a checkbox.** Unchecking it excludes that selection's points + from the result without deleting it, so a region can be tried in and out + without redrawing it. +- **``Delete selected``** deletes the currently selected row, of any type — + including a single brush stroke or the ``Manual`` row. +- Labels are never reused: deleting ``Grid 2`` and then creating another grid + gives ``Grid 4``, not a second ``Grid 3``. + +Editing a selection +------------------- + +Grids and polylines stay editable after they are drawn. Clicking their row +in the selections list also switches to the matching tool, so a grid or line +can be edited without first re-selecting the corresponding button. + +**Moving a vertex.** A left-drag that starts within about 10 screen pixels of +an existing vertex moves that vertex; a drag anywhere else pans the view. The +grab radius is constant in screen pixels, so it behaves the same at any zoom. +The subsets are recomputed once, when the drag finishes. Clicking exactly on +an existing vertex does nothing, rather than stacking a duplicate on top of +it. + +**Undo (Ctrl+Z)** reverses adding a vertex, moving a vertex, and deleting a +selection — a grid, a polyline, a brush stroke, or the ``Manual`` row. A +restored selection comes back at its original row in the list, with its +original label. Filter results are *not* undoable. + +Filter mode +----------- + +Switching to **Filter** mode (top toolbar) applies automatic filtering on top +of the subsets placed in Select mode, to keep only the ones on +strongly-textured image content. Two filter methods are available: + +- **Shi-Tomasi**: ranks each subset by the corner strength of the image + content inside it (the smaller eigenvalue of the local gradient structure + tensor, in the style of the Shi-Tomasi corner criterion). A threshold + slider keeps only the subsets above a fraction of the strongest one. +- **Gradient in direction**: ranks each subset by the strength of the image + gradient projected onto a chosen direction. The direction is set either by + clicking ``Set direction on image`` and dragging across the image, or with + the ``X Direction``/``Y Direction`` preset buttons. A threshold slider + then keeps only the subsets with a strong-enough gradient in that + direction. + +Filtered (candidate) points are shown in green; ``Clear candidates`` resets +the filter back to the full selection from Select mode. + +The filter result follows the selection: going back to Select mode and +removing subsets -- with the brush in deselect mode, with ``Remove point``, or +by deleting or unchecking a row -- drops their candidates as well. Nothing is +recomputed, so putting the subsets back (re-checking the row, or undoing the +deletion) brings their candidates back too. Subsets *added* after a filter has +run are not scored until the filter is run again. + +Retrieving the points +---------------------- + +Once the selection is complete, the points can be retrieved through the +``.points`` property or the ``.get_points()`` method. If a filter has been +applied in Filter mode, the filtered (candidate) points are returned; +otherwise, the points from Select mode are returned. -where ``roi_size`` is the size of a single Region-Of-Interest/subset in ``y`` and -``x`` direction respsectivly. The ``noverlap`` argument prescribes the overlap of the -neighbouring ROIs. The density of the grid can be adjusted using ``noverlap``. +.. code:: python -The UI enables multiple modes of point selection. Currently, the following are -supported: + points = gui.points # or: gui.get_points() -- ``ROI grid``: A regular grid of ROIs is created based on the selected polygon. -- ``Deselect ROI polygon``: After defining the polygon and getting the points, this method can - be used to define a polygon within which the points are not selected. -- ``Only polygon``: Same as ROI grid but the points are not computed. Only polygon points - are available. -- ``Manual ROI select``: Manually select the ROIs at desired locations. +The returned array has shape ``(n_points, 2)``, with points given in +**row/column** (``y``/``x``) image coordinates: ``points[:, 0]`` is the row +(``y``) coordinate and ``points[:, 1]`` is the column (``x``) coordinate. The +points are returned in the order the underlying selections were created +(across grids, lines, brush strokes and manual clicks combined) — no +supported use depends on this order. -Once the selection in the UI is complete, the points can be retrieved: +The points can be passed directly to a method object, either as the GUI +instance itself (``set_points`` duck-types on a ``.points`` attribute) or as +the extracted array: .. code:: python - points = Points.points + sof = SimplifiedOpticalFlow(video) + sof.set_points(gui) # or: sof.set_points(points) .. image:: selection.gif + :alt: Animated demo of the SelectionGUIOld Grid method: a polygon is drawn + vertex by vertex over the video frame, filling in with the subset + grid it encloses. diff --git a/docs/source/quick_start/results.rst b/docs/source/quick_start/results.rst new file mode 100644 index 0000000..9703e87 --- /dev/null +++ b/docs/source/quick_start/results.rst @@ -0,0 +1,162 @@ +.. _results: + +Results, saving and reloading +============================= + +The displacement array +---------------------- + +``get_displacements()`` returns, and stores on the method object as +``.displacements``, an array of shape ``(n_points, n_frames, 2)`` in pixels, +relative to the reference frame: + +.. code:: python + + displacements = lk.get_displacements() + + displacements[i, :, 0] # displacement history of point i, row (y) direction + displacements[i, :, 1] # displacement history of point i, column (x) direction + +``DIC`` additionally exposes ``.warp_params`` of shape +``(n_points, n_frames, n_param)``, from which strain and rotation are read +directly — see :doc:`disp_id_methods`. + +.. _failed-points: + +Points that could not be tracked +-------------------------------- + +A point placed on a uniform region, or on a single straight edge with no +gradient along it, cannot be tracked. Since 1.4.0 this no longer aborts the +analysis: the point is set to ``NaN`` from the frame at which it was lost, +every other point is computed normally, and a warning is issued. + +Which points failed, and why, is recorded in ``failed_points``: + +.. code:: python + + displacements = lk.get_displacements() + + lk.failed_points # {point_index: {'frame': ..., 'status': ...}} + +``status`` distinguishes a singular gradient matrix (a flat or edge-only +subset — the point never had enough information to track) from a diverged +iteration (the optimizer ran away to a non-finite value). + +The practical consequence downstream is that results may contain ``NaN``: + +.. code:: python + + import numpy as np + + ok = ~np.isnan(displacements).any(axis=(1, 2)) # points that tracked all the way + amplitude = np.nanmax(np.abs(displacements), axis=1) + +.. warning:: + + The detection is best effort. It catches points whose displacement becomes + non-finite or larger than the image, but a point can return physically + implausible values well below that bound. **A result without ``NaN`` is + not proof that every point tracked correctly.** Check the displacements + against what the structure can plausibly do. + +Where results are saved +----------------------- + +``get_displacements()`` saves automatically (pass ``autosave=False`` to +suppress it). Results go into a directory next to the recording, one +sub-directory per run: + +.. code:: text + + measurement.cih + measurement_pyidi_analysis/ + analysis_001/ + points.pkl # the points that were tracked + results.pkl # the displacements + directions.pkl # only for DirectionalLucasKanade + warp_params.pkl # only for DIC + settings.json # every configure() argument, plus source and date + analysis_002/ + ... + +``settings.json`` is what makes a run reproducible: it records the input file, +the method, the creation date, the video dimensions, and the full +configuration. This is why every ``configure()`` parameter must be stored as +an attribute of the same name. + +Reloading a saved analysis +-------------------------- + +.. code:: python + + from pyidi import load_analysis + + video, idi, settings = load_analysis('measurement_pyidi_analysis/analysis_001') + + displacements = idi.displacements + points = idi.points + +``load_analysis`` returns three things: a fresh +:class:`~pyidi.video_reader.VideoReader`, the reconstructed method object with +its points, directions and results restored, and the settings dictionary. + +.. code:: python + + # the recording has moved since the analysis was run + video, idi, settings = load_analysis(path, input_file='new/location/measurement.cih') + + # only the points and settings, without reading the results back + video, idi, settings = load_analysis(path, load_results=False) + +Resuming an interrupted analysis +-------------------------------- + +Long analyses checkpoint as they go, into a ``temp_file`` directory beside the +recording. If a run is interrupted — a crash, a full disk, a closed laptop — +it can pick up from the last completed time point instead of starting over: + +.. code:: python + + lk.configure(resume_analysis=True) + displacements = lk.get_displacements() + +The checkpoint is only reused if the settings match the interrupted run; if +they do not, the analysis restarts from the beginning. The temporary files are +removed on successful completion. + +.. note:: + + ``failed_points`` is rebuilt from scratch on resume rather than being + restored from the checkpoint. A point that was already lost before the + interruption stays lost — the previous displacement is checked for + ``NaN``/``inf`` before it is used — but the recorded frame number refers to + the resumed run. + +Viewing the results +------------------- + +``ResultViewer`` animates the identified displacements over the recording +(requires the ``[qt]`` extra): + +.. code:: python + + from pyidi import ResultViewer + + viewer = ResultViewer( + video, + displacements=displacements, + points=points, + fps=30, # playback rate + magnification=10, # visual amplification of the displacement + point_size=10, + colormap='cool', + ) + +The window opens on construction and blocks until it is closed. ``points`` +uses the same ``(row, column)`` convention as everywhere else, and +``displacements`` is the array as returned by ``get_displacements()``. A +2-D ``(n_points, 2)`` array is also accepted and animated as a mode shape. + +If the analysis was run through the napari :doc:`GUI `, the method +object is at ``gui.method`` and the results at ``gui.method.displacements``. diff --git a/docs/source/quick_start/selection.gif b/docs/source/quick_start/selection.gif index 98db0cc..1af22cf 100644 Binary files a/docs/source/quick_start/selection.gif and b/docs/source/quick_start/selection.gif differ diff --git a/docs/source/quick_start/video_reader.rst b/docs/source/quick_start/video_reader.rst new file mode 100644 index 0000000..ab45cf6 --- /dev/null +++ b/docs/source/quick_start/video_reader.rst @@ -0,0 +1,167 @@ +.. _video-reader: + +Reading a video +=============== + +:class:`~pyidi.video_reader.VideoReader` is the single entry point for every +supported recording format. It hides the difference between a Photron header +plus raw file, a proprietary camera container, a folder of TIFFs and an MP4 — +downstream, every identification method sees the same interface. + +.. code:: python + + from pyidi import VideoReader + + video = VideoReader('measurement.cih') + +Supported formats +----------------- + +.. list-table:: + :header-rows: 1 + :widths: 24 26 50 + + * - Source + - Extensions + - Notes + * - Photron + - ``.cih``, ``.cihx`` + - Point at the header file; the ``.mraw`` next to it is memory-mapped. + Frame rate and bit depth are read from the header. + * - Phantom + - ``.cine`` + - Read through the ``cine-handler`` package, which is installed with + pyIDI. Frame rate comes from the file's setup block. + * - Pharsighted + - ``.SLOW`` + - Read by the bundled ``slow_reader``. + * - Image sequence + - ``.png``, ``.tif``, ``.tiff``, ``.bmp``, ``.jpg``, ``.jpeg``, ``.gif`` + - See :ref:`image-sequences` below. + * - Video file + - ``.avi``, ``.mkv``, ``.mp4``, ``.mov``, ``.m4v``, ``.wmv``, ``.webm``, + ``.flv``, ``.ogg``, ``.ogv`` + - Decoded with PyAV. Currently 8-bit only. + * - In memory + - :class:`numpy.ndarray` + - Shape ``(n_time_points, image_height, image_width)``. A ``root`` + directory must be given, because that is where results get written. + +.. code:: python + + # Photron + video = VideoReader('data/data_synthetic.cih') + + # Phantom + video = VideoReader('data/data_small_cine.cine') + + # an image sequence: point at any image in the folder + video = VideoReader('frames/im_0000.png') + + # a numpy array already in memory + video = VideoReader(array, root='analysis_output') + +.. _image-sequences: + +Image sequences +^^^^^^^^^^^^^^^ + +Give the path to *any* image in the sequence. All images must be in the same +directory and named so that a plain sort puts them in the right order — pad +the numbers: ``im_0000.png ... im_9999.png``, not ``im_0.png ... im_9999.png``. +Multi-image containers (``.gif``, multi-page ``.tif``) are read as a sequence +from the single file. + +Frame rate +---------- + +The frame rate is what turns a displacement history into a frequency, so it is +worth checking rather than assuming. + +pyIDI reads it from the file where the format carries it (Photron headers, +``.cine`` setup blocks, and video-container metadata). Where it does not — a +numpy array, most image sequences, and any container whose metadata is missing +or wrong — set it yourself: + +.. code:: python + + video = VideoReader('frames/im_0000.png', fps=10000) + # or later + video.configure(fps=10000) + +.. warning:: + + Ordinary video containers frequently report a *playback* rate rather than + the capture rate — a 6000 fps recording exported to MP4 will often claim + 30 fps. If you did not record the file yourself, verify the rate against + the acquisition settings. + +The value is available as ``video.fps``, and format-specific metadata is in +``video.info``. + +Colour and bit depth +-------------------- + +Frames come back as a 2-D :class:`numpy.ndarray` of shape +``(image_height, image_width)``, ``uint8`` or ``uint16`` depending on the +source. Colour material is converted to grayscale (the luma channel) by +default. To use a single channel or your own weights instead: + +.. code:: python + + # one channel + video.configure(video_format='rgb24', channel='R') + + # custom weights + video.configure(video_format='rgb24', channel_weights=[0.299, 0.587, 0.114]) + +``channel`` and ``channel_weights`` only take effect if ``video_format`` is set +to an RGB format matching the source bit depth (``rgb24``, ``rgb48le``, +``rgb48be``); the default formats (``gray``, ``gray16be``, ``gray16le``) +already deliver a monochrome frame. + +Reading frames +-------------- + +.. code:: python + + frame = video.get_frame(0) # a single frame, (height, width) + + frames = video.get_frames() # every frame, (n, height, width) + frames = video.get_frames(500) # frames 0..500 + frames = video.get_frames((100, 400)) # frames 100..400 + +Useful attributes: + +.. list-table:: + :widths: 30 70 + + * - ``video.N`` + - number of frames + * - ``video.image_width``, ``video.image_height`` + - frame size in pixels + * - ``video.fps`` + - frame rate in Hz + * - ``video.info`` + - metadata dictionary from the source file + * - ``video.root`` + - directory where the analysis results will be written + +Call ``video.close()`` when you are done with a ``.cine`` or memory-mapped +source, or let the object go out of scope. + +Viewing the recording +--------------------- + +.. code:: python + + video.gui() + +opens the napari viewer on the recording (requires the ``[qt]`` extra — see +:doc:`napari`). + +Next +---- + +* :doc:`points_selection` — choosing where to track. +* :doc:`disp_id_methods` — choosing how to track. diff --git a/examples/feature_selection_demo.py b/examples/feature_selection_demo.py new file mode 100644 index 0000000..041cdb0 --- /dev/null +++ b/examples/feature_selection_demo.py @@ -0,0 +1,244 @@ +"""Manual test-drive of the automatic feature selection. + +Run this CELL BY CELL in an interactive session (molten.nvim, IPython, VS Code), +not with ``python examples/feature_selection_demo.py``. + +Why: ``SelectionGUI.__init__`` ends with ``sys.exit(app.exec())`` whenever +``sys.ps1`` is absent, which is the case for a plain script run. The window would +open, and closing it would terminate the interpreter before any of the checks +below could run. In an interactive session it calls a bare ``app.exec()`` instead, +which blocks until you close the window and then hands control back. + +Either way the GUI is modal: execution stops at the ``SelectionGUI(...)`` +line until you close the window. + +Requires the Qt extras: pip install pyidi[qt] +""" + +# %% + +import numpy as np +import pyidi + +print(f'pyidi {pyidi.__version__}') + +VIDEO = 'data/data_synthetic.cih' # 128 x 256, 101 frames - general purpose +# VIDEO = "/media/klemenzaletelj/My Passport/ETRA_2026/20260618/CTC_02/CTC_02.cihx" +# VIDEO = 'data/data_showcase.cih' # 40 x 640 beam - good for a single region + +video = pyidi.VideoReader(VIDEO) +print(f'{video.N} frames, {video.image_height} x {video.image_width} (height x width)') + +# %% +# --------------------------------------------------------------------------- +# 1. Find, then trim. +# +# This is what SelectionGUI names as of 1.4. The window it replaced -- now +# SelectionGUIOld, deprecated -- placed subsets and then filtered them; this one +# scores the whole image and lets the selection find the features, which you +# then trim. +# +# The window OPENS WITH POINTS ALREADY ON IT. The selections list starts with a +# 'Whole image' mask row, so the candidates are there to look at before you have +# drawn anything. +# +# Two tabs, deliberately unnumbered - evaluation does not depend on the mask, so +# there is no step 1. +# +# --- EVALUATE + SELECT ----------------------------------------------------- +# * TURN ON 'Show score overlay'. This is the whole image scored at once. It +# is worth looking at before you pick anything - you can see where the +# trackable content actually is. +# * SWITCH TO 'Gradient in direction' and hit the X / Y preset buttons. The +# heatmap should change character completely. Then hit 'Draw' and DRAG A +# LINE on the image - the direction follows the drag, and a red line shows +# the direction in force. +# * SWITCH BACK to Shi-Tomasi. It should be instant: the array is cached. +# * DRAG THE THRESHOLD. The points update as you drag, with no recomputation. +# Same for 'Separation' and 'Maximum points'. The score overlay is on +# the same panel, so you can see whether a thin patch is a tight threshold +# or an area with nothing to offer. +# * RAISE 'Separation'. It is the one density control: no two points come +# closer than it, so lowering it is how you ask for more. Points thin out +# but stay on the strongest spots - unlike a grid, which thins out wherever +# the grid happens to land, and unlike 'Keep every n-th', which would leave +# most of the survivors back-to-back on the same feature. +# * DROP 'Maximum points' TO SOMETHING SMALL and lower the separation until +# the panel says the cap stopped it. A cap has no other symptom - it just +# stops adding points, which reads as though the threshold did it. +# * SWITCH THE SELECTOR to 'lattice'. That is the old regular-grid behaviour, +# reproduced inside the same pipeline. +# * NOTE THE PANEL. The selections list is not here: every row in it belongs +# to the Mask tab. The subset size is, because both tabs read it. +# +# --- MASK ------------------------------------------------------------------ +# Polygon - click corners; the enclosed AREA becomes a mask +# Brush - hold Ctrl and drag to paint an area +# Line - click vertices; points spaced along the segments +# Points - click to drop individual points +# Remove point - click near a point to delete it +# Remove w/ brush - hold Ctrl and drag to take away what the stroke covers +# +# * TRIM THE 'Whole image' ROW. Pick 'Remove w/ brush' and paint over +# the parts you do not want. That is the workflow this ordering exists for: +# find the candidates, then edit them. +# * OR DELETE IT and draw your own region. Deleting it selects NOTHING - it +# does not silently fall back to the whole frame. +# * A DRAWN REGION STANDS THE 'Whole image' ROW DOWN. Mask rows combine as a +# union, so a polygon on top of it would otherwise change nothing at all. +# Drawing one unchecks it and says so in the status bar; tick it again, or +# Ctrl+Z, to bring the whole frame back. +# * 'Show score overlay' is HERE TOO, and ganged to the one on the other tab. +# * EVERY ROW HAS A ROLE, shown in the list: 'mask' or 'points'. Polygons and +# brush strokes start as 'mask'; lines and clicked points start as 'points'. +# 'Use as points' / 'Use as mask' switches a row over WITHOUT redrawing it. +# * DESELECTION SURVIVES A PARAMETER CHANGE. Paint part of a region away, +# then change the subset size: it stays gone. +# * THREE TIERS OF POINT, and only on this tab. Red is being taken, dim blue +# is a feature the mask is leaving out, and a magenta ring marks what the +# row selected in the list accounts for. Draw a small polygon and watch the +# rest of the frame go blue: that is the difference between "nothing there" +# and "you masked it away". The blue ones do not move as you paint - they +# are the whole-frame selection, so a stroke turns them red where it lands. +# * WATCH THE POINTS AS YOU DESELECT. The ones under the stroke are crossed +# out while you are still painting, and gone when you let go. Nothing else +# moves until then: the stroke costs the points it has reached, and the +# selection is re-run once, when you let go. +# * 'Clear all' STARTS OVER rather than clearing to nothing: the whole frame +# comes back, as when the window opened. Deleting the 'Whole image' row on +# its own still selects nothing - a different act. +# * Ctrl+Z undoes a vertex add, a vertex move, a stroke, a deletion, a point +# removal, and a deselection. +# * CHANGE THE SUBSET SIZE (below the tabs, so it is there on both - the +# scoring window follows it, and so does the rectangle drawn round each +# point). This is the one interaction that DOES recompute the score. +# +# CLOSE THE WINDOW to continue. +# --------------------------------------------------------------------------- + +gui = pyidi.SelectionGUI(video, subset_size=11) + +# %% + +points = np.asarray(gui.points) + +print(f'selected {len(points)} points') +print(f'dtype {points.dtype}, shape {points.shape}') +if len(points): + print(f'row (y) range: {points[:, 0].min()} .. {points[:, 0].max()} (image height {video.image_height})') + print(f'col (x) range: {points[:, 1].min()} .. {points[:, 1].max()} (image width {video.image_width})') + +# EXPECT an (N, 2) integer array in (row, col) order, with the row values bounded +# by the image HEIGHT and the column values by the image WIDTH. + +# %% +# --------------------------------------------------------------------------- +# 2. Hand the points to a method. Nothing special is needed - the pipeline +# returns exactly what set_points() wants. +# --------------------------------------------------------------------------- + +lk = pyidi.LucasKanade(video) +lk.set_points(points) +print(f'{len(lk.points)} points, dtype {lk.points.dtype}') + +# %% +# --------------------------------------------------------------------------- +# 3. The same thing without a GUI. This imports without Qt. +# --------------------------------------------------------------------------- + +from pyidi.selection import Entry, select_points # noqa: E402 + +frame = video.get_frame(0) +h, w = frame.shape + +region = Entry('polygon', [(10, 10), (10, w - 10), (h - 10, w - 10), (h - 10, 10)]) +scripted = select_points(frame, [region], subset_size=11, separation=15, threshold=0.02) + +print(f'{len(scripted)} points from the whole frame') + +# %% +# --------------------------------------------------------------------------- +# 4. Sweeping a parameter. SelectionPipeline keeps the score cache alive, so +# only the FIRST of these actually evaluates anything. +# --------------------------------------------------------------------------- + +from pyidi.selection import SelectionPipeline # noqa: E402 + +pipeline = SelectionPipeline(frame, subset_size=11) +pipeline.add_entry('polygon', region.geometry) + +for threshold in (0.3, 0.1, 0.03, 0.01, 0.003): + pipeline.selector_params['threshold'] = threshold + print(f'quality {threshold:>5} -> {len(pipeline.points):>5} points') + +print(f'\nevaluator runs: {pipeline.store.n_evaluations}') +# EXPECT 1. If this printed 5, the score cache is not doing its job. + +# %% +# --------------------------------------------------------------------------- +# 5. Mixing automatic and hand-picked points. The literal points win: they +# survive any threshold, and nothing automatic is placed next to them. +# --------------------------------------------------------------------------- + +pipeline.selector_params.update({'threshold': 0.02, 'separation': 12}) +automatic = len(pipeline.points) + +pipeline.add_entry('points', [(h // 2, w // 2)]) +mixed = pipeline.points + +print(f'{automatic} automatic -> {len(mixed)} with one hand-picked point added') +assert (mixed == np.array([h // 2, w // 2])).all(axis=1).any(), 'the hand-picked point was dropped!' + +distances = np.hypot(mixed[:, 0] - h // 2, mixed[:, 1] - w // 2) +print(f'nearest automatic point is {np.sort(distances)[1]:.1f} px away (separation is 12)') + +# %% +# --------------------------------------------------------------------------- +# 6. Score images are named and cached per (evaluator, parameters, subset size), +# so two criteria can be live at once without either discarding the other. +# --------------------------------------------------------------------------- + +pipeline.define_score('sideways', 'gradient_direction', direction=(0, 1)) +pipeline.define_score('upright', 'gradient_direction', direction=(1, 0)) + +for name in (pipeline.default_score, 'sideways', 'upright'): + array = pipeline.store.get(name) + print(f'{name:>10}: max {np.nanmax(array):.3g}') + +print(f'\nevaluator runs: {pipeline.store.n_evaluations}') + +# %% +# --------------------------------------------------------------------------- +# 7. Adding an evaluator. No GUI code is involved - register a function and +# its parameter descriptors, and it turns up in the Evaluate menu. +# --------------------------------------------------------------------------- + +from scipy.ndimage import uniform_filter # noqa: E402 + +from pyidi.selection import Evaluator, register_evaluator # noqa: E402 + + +def local_variance(image, window): + """Variance of the image inside the subset window.""" + img = np.asarray(image, dtype=np.float64) + mean = uniform_filter(img, size=window, mode='constant') + mean_of_squares = uniform_filter(img * img, size=window, mode='constant') + return mean_of_squares - mean * mean + + +register_evaluator(Evaluator( + name='variance', + display_name='Local variance', + function=local_variance, + parameters=(), + description='Contrast inside the subset.', +)) + +variance_points = select_points(frame, [region], subset_size=11, + evaluator='variance', separation=15) +print(f'{len(variance_points)} points from the new evaluator') + +# %% +# Open the GUI again - 'Local variance' should now be in the evaluator menu. + +gui2 = pyidi.SelectionGUI(video, subset_size=11) diff --git a/examples/point_selection_demo.py b/examples/point_selection_demo.py new file mode 100644 index 0000000..dae21d2 --- /dev/null +++ b/examples/point_selection_demo.py @@ -0,0 +1,245 @@ +"""Manual test-drive of ``SelectionGUIOld``, the deprecated selection window. + +Kept so the 1.3 interface can still be exercised while it is around. For the +interface ``SelectionGUI`` names today, see ``feature_selection_demo.py``. + +Run this CELL BY CELL in an interactive session (molten.nvim, IPython, VS Code), +not with ``python examples/point_selection_demo.py``. + +Why: ``SelectionGUIOld.__init__`` ends with ``sys.exit(app.exec())`` whenever +``sys.ps1`` is absent, which is the case for a plain script run. The window would +open, and closing it would terminate the interpreter before any of the checks +below could run. In an interactive session it calls a bare ``app.exec()`` instead, +which blocks until you close the window and then hands control back. + +Either way the GUI is modal: execution stops at the ``SelectionGUIOld(...)`` line +until you close the window. Select your points, then close it to continue. + +Requires the Qt extras: pip install pyidi[qt] +""" + +# %% + +import numpy as np +import pyidi + +print(f'pyidi {pyidi.__version__}') + +VIDEO = 'data/data_synthetic.cih' # 128 x 256, 101 frames - general purpose +# VIDEO = 'data/data_showcase.cih' # 40 x 640 beam - good for 'Along the line' + +video = pyidi.VideoReader(VIDEO) +print(f'{video.N} frames, {video.image_height} x {video.image_width} (height x width)') + +# %% +# --------------------------------------------------------------------------- +# 1. Open the GUI and select some points. +# +# Try each of the five selection methods on the right-hand panel: +# Grid - click >=3 polygon corners; a grid fills the polygon +# Manual - click to drop individual points +# Along the line - click polyline vertices; points spaced along the segments +# Brush - hold Ctrl and drag to paint; toggle 'Deselect painted area' +# Remove point - click near a point to delete it +# +# Then switch to Filter mode (top toolbar) and try Shi-Tomasi / gradient +# direction filtering on top of the selection. +# +# --- things to check specifically ------------------------------------------ +# +# The right-hand panel now shows ONE 'selections' list, visible in every mode, +# with one row per grid, per line, per brush stroke, and a single 'Manual' row +# collecting every individually-clicked point. +# +# * CLICK A ROW. It should become the active selection, the tool should +# switch to match its type (e.g. clicking a Grid row switches to Grid +# mode), and its points should highlight in the image. +# * TOGGLE A ROW'S CHECKBOX. Unchecking it should remove its points from +# gui.points immediately, without deleting the row. Re-checking it should +# bring them back. +# * DELETE A BRUSH STROKE. Select a Brush row and hit 'Delete selected' - +# it should go. Previously a brush stroke could only be removed via +# 'Clear selections'. +# * DELETE THE MANUAL ROW. Same as above - previously not possible either. +# * Ctrl+Z AFTER DELETING A BRUSH STROKE. It should come back at its +# ORIGINAL row with its ORIGINAL label. Undo now covers deleting any row +# (grid, line, brush stroke, or the Manual row), not just grid/line as +# before. +# +# In Grid or 'Along the line' mode: +# * DRAG A VERTEX. Press within ~10 px of a corner you already placed and +# drag - it should follow the cursor, and the subsets should re-fill the +# new shape when you release. Dragging from empty space still pans. +# * CLICK EXACTLY ON A VERTEX. Nothing should happen. It used to drop a +# duplicate vertex on top of the existing one. +# * Ctrl+Z. Still undoes a vertex add and a vertex move, one step at a time. +# * DELETE THE ONLY GRID/LINE. With exactly one entry in the list, select it +# and hit delete - it should go. Previously this silently did nothing and +# you had to create a second one first. +# * The button above the list should read 'Start new grid' in Grid mode and +# 'Start new line' in 'Along the line' mode. +# +# In Brush mode: +# * The painted area should now be centred ON the cursor. It used to land +# about 9 px up and to the left, because the drag handlers were reading +# ViewBox-local coordinates instead of scene coordinates. +# * REMOVED POINTS SURVIVE A SPACING CHANGE. Paint a brush stroke (or draw a +# grid/line), use 'Remove point' to delete a couple of its points, then +# change 'Distance between subsets' or the subset size. The removed +# points should stay gone rather than reappearing - previously they were +# regenerated from the source geometry and silently came back. +# +# CLOSE THE WINDOW to continue. +# --------------------------------------------------------------------------- + +gui = pyidi.SelectionGUIOld(video, subset_size=11, subset_overlap=0) + +# %% + +points = gui.points # equivalently: gui.get_points() +points = np.asarray(points) + +print(f'selected {len(points)} points') +print(f'dtype {points.dtype}, shape {points.shape}') +if len(points): + print(f'row (y) range: {points[:, 0].min()} .. {points[:, 0].max()} (image height {video.image_height})') + print(f'col (x) range: {points[:, 1].min()} .. {points[:, 1].max()} (image width {video.image_width})') + print('\nfirst few points (row, col):') + print(points[:5]) + +# EXPECT: an (N, 2) array in (y, x) = (row, column) order, with the row values +# bounded by the image HEIGHT and the column values by the image WIDTH. If those +# two look swapped, the GUI's internal x/y reversal is wrong. + +# %% +# --------------------------------------------------------------------------- +# 2. Hand the points to a method. Both forms should work now. +# --------------------------------------------------------------------------- + +lk = pyidi.LucasKanade(video) + +# (a) pass the GUI object itself - this is the duck-typing fix; it used to fail +# with an opaque error because only the old SubsetSelection was recognised. +lk.set_points(gui) +print(f'(a) passing the GUI object -> {len(lk.points)} points, dtype {lk.points.dtype}') + +# (b) pass the array +lk.set_points(points) +print(f'(b) passing the array -> {len(lk.points)} points, dtype {lk.points.dtype}') + +assert np.array_equal(lk.points, np.asarray(gui.points)), 'the two forms disagree!' +print('\nboth forms agree.') + +# %% +# --------------------------------------------------------------------------- +# 3. Validation. Every one of these used to be accepted silently or to fail +# with an unhelpful IndexError. They should now raise a clear ValueError. +# --------------------------------------------------------------------------- + +sof = pyidi.SimplifiedOpticalFlow(video) + +bad_inputs = { + 'empty': [], + '1-D array': np.array([1, 2]), + 'three columns': np.array([[1, 2, 3]]), + 'outside the image': np.array([[9999, 9999]]), + 'negative coordinates': np.array([[-4, -4]]), +} + +for name, value in bad_inputs.items(): + try: + sof.set_points(value) + print(f' {name:22s} -> NOT REJECTED <-- unexpected') + except ValueError as e: + print(f' {name:22s} -> ValueError: {e}') + +# %% +# --------------------------------------------------------------------------- +# 4. Sub-pixel points are rounded to nearest, with a warning. +# Previously: SimplifiedOpticalFlow crashed on these, while LucasKanade and +# DirectionalLucasKanade silently truncated TOWARD ZERO (1.7 -> 1). +# --------------------------------------------------------------------------- + +import warnings + +with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter('always') + sof.set_points(np.array([[1.7, 2.3], [10.5, 20.5]])) + for w in caught: + print(f'warning: {w.message}') + +print(f'stored: {sof.points.tolist()} dtype {sof.points.dtype}') +# EXPECT [[2, 2], [10, 20]] as an integer dtype. The point of the check is that +# 1.7 -> 2, NOT 1: the old LucasKanade path truncated toward zero. The 10.5 -> 10 +# is numpy's round-half-to-even (np.rint), which is expected, not a bug. + +# %% +# --------------------------------------------------------------------------- +# 5. SelectionGUIOld now accepts a plain numpy array, as its docstring always +# claimed. This raised AttributeError before. +# CLOSE THE WINDOW to continue. +# --------------------------------------------------------------------------- + +frames = video.get_frames() # (n_frames, height, width) +print(f'passing a 3-D array {frames.shape}') + +gui_arr = pyidi.SelectionGUIOld(frames, subset_size=9) +print(f'frame used: {gui_arr.frame.shape} (should be 2-D: {frames.shape[1:]})') + +# %% +# A single 2-D image should work too. CLOSE THE WINDOW to continue. + +gui_img = pyidi.SelectionGUIOld(frames[0], subset_size=9) +print(f'frame used: {gui_img.frame.shape}') + +# %% +# And an unusable input should now say so clearly, rather than dying on a +# missing attribute deep in the constructor. + +try: + pyidi.SelectionGUIOld('not a video') +except TypeError as e: + print(f'TypeError: {e}') + +# %% +# --------------------------------------------------------------------------- +# 6. The retired widget. Scripts written against SubsetSelection should fail +# with a message that names the replacement. +# --------------------------------------------------------------------------- + +try: + pyidi.SubsetSelection(video, roi_size=(21, 21), noverlap=0) +except RuntimeError as e: + print(f'RuntimeError: {e}') + +# %% +# --------------------------------------------------------------------------- +# 7. Optional: run a short analysis on the selected points, to confirm the +# whole chain works end to end. Uses whatever you picked in step 1. +# +# NOTE: points whose ROI reaches past the image edge are clipped with a warning, +# and a point that cannot be tracked comes back as NaN rather than raising +# (behaviour introduced in 1.4.0) - so check for NaN, do not assume success. +# --------------------------------------------------------------------------- + +lk.set_points(points) +lk.configure(roi_size=(11, 11), int_order=3, processes=1) +disp = lk.get_displacements() + +print(f'displacements shape {disp.shape} (n_points, n_frames, 2)') +n_failed = int(np.isnan(disp).any(axis=(1, 2)).sum()) +print(f'{n_failed} of {len(points)} points failed to track') +if getattr(lk, 'failed_points', None): # a dict, empty when everything tracked + print(f'failed_points: {lk.failed_points}') + +# %% +# --------------------------------------------------------------------------- +# 8. Optional: the napari GUI, a separate interface again. +# Its point selection now routes through set_points too, so an out-of-bounds +# pick warns instead of being accepted silently. +# +# Pick a method, then 'Set points', then 'Configure', then 'Calculate'. +# --------------------------------------------------------------------------- + +# napari_gui = pyidi.GUI(video) +# print(napari_gui.method.points) diff --git a/pyidi/GUIs/__init__.py b/pyidi/GUIs/__init__.py index 1760baa..bac406c 100644 --- a/pyidi/GUIs/__init__.py +++ b/pyidi/GUIs/__init__.py @@ -1,28 +1,83 @@ +"""Graphical interfaces, each available only if its own toolkit is installed. + +Every class here needs the ``[qt]`` extra, but not the same part of it. The +selection windows and ``ResultViewer`` are PyQt6 and pyqtgraph; the napari +``GUI`` needs napari and magicgui instead. The requirement is therefore +checked per class rather than once for the package -- checking once means a +partial install, PyQt6 without napari being the likely one, turns ``import +pyidi`` into an ``ImportError`` from deep inside a submodule rather than a +message saying what to install. + +A class whose dependencies are missing is replaced by a stub that imports +cleanly and raises ``RuntimeError`` when constructed, naming what is absent. +""" + +import importlib.util import typing -try: - import PyQt6 +#: What each name needs beyond the base dependencies. Checked, not imported. +_REQUIREMENTS = { + 'SelectionGUI': ('PyQt6', 'pyqtgraph'), + 'SelectionGUIOld': ('PyQt6', 'pyqtgraph'), + 'ResultViewer': ('PyQt6', 'pyqtgraph'), + 'Viewer': ('PyQt6', 'pyqtgraph'), + 'GUI': ('napari', 'magicgui'), +} + + +def _installed(*modules): + """Whether every named top-level module can be found, without importing it.""" + return all(importlib.util.find_spec(name) is not None for name in modules) + + +def _unavailable(name): + """A stand-in for ``name`` that raises only when someone constructs it.""" + missing = ', '.join(m for m in _REQUIREMENTS[name] if not _installed(m)) + + class Unavailable: + def __init__(self, *args, **kwargs): + raise RuntimeError( + f"{name} requires the qt extras: pip install pyidi[qt] " + f"(missing: {missing})." + ) - HAS_PYQT6 = True -except ImportError: - HAS_PYQT6 = False + Unavailable.__name__ = name + Unavailable.__qualname__ = name + return Unavailable + + +HAS_PYQT6 = _installed('PyQt6', 'pyqtgraph') +HAS_NAPARI = _installed('napari', 'magicgui') if HAS_PYQT6 or typing.TYPE_CHECKING: - from .subset_selection import SelectionGUI + from .feature_selection import SelectionGUI + from .subset_selection import SelectionGUIOld from .result_viewer import ResultViewer from .result_viewer import Viewer +else: + SelectionGUI = _unavailable('SelectionGUI') + SelectionGUIOld = _unavailable('SelectionGUIOld') + ResultViewer = _unavailable('ResultViewer') + Viewer = _unavailable('Viewer') + +if HAS_NAPARI or typing.TYPE_CHECKING: from .gui import GUI else: - class SelectionGUI: - def __init__(self, *args, **kwargs): - raise RuntimeError("SelectionGUI requires the qt extras: pip install pyidi[qt]") + GUI = _unavailable('GUI') - class ResultViewer: - def __init__(self, *args, **kwargs): - raise RuntimeError("ResultViewer requires the qt extras: pip install pyidi[qt]") - class GUI: - def __init__(self, *args, **kwargs): - raise RuntimeError("GUI requires the qt extras: pip install pyidi[qt]") +class FeatureSelectionGUI: + def __init__(self, *args, **kwargs): + raise RuntimeError( + "FeatureSelectionGUI was a working name that never shipped. " + "It is now called SelectionGUI." + ) + -from .selection import SubsetSelection +class SubsetSelection: + def __init__(self, *args, **kwargs): + raise RuntimeError( + "SubsetSelection was removed in favour of SelectionGUI. " + "Replace SubsetSelection(...) with " + "SelectionGUI(video, subset_size=..., subset_overlap=...)." + ) diff --git a/pyidi/GUIs/feature_selection.py b/pyidi/GUIs/feature_selection.py new file mode 100644 index 0000000..a2e6d5b --- /dev/null +++ b/pyidi/GUIs/feature_selection.py @@ -0,0 +1,1909 @@ +"""Interactive front end for the mask -> evaluate -> select pipeline. + +``SelectionGUI`` drives :mod:`pyidi.selection`, whose three steps -- +mask, evaluate, select, in the vocabulary settled in issue #51 -- are presented +as *two* tabs. + +That is not a simplification of the pipeline but a truer picture of it. +Evaluation does not depend on the mask at all: the score store computes the +whole frame and deliberately never crops to a region, because a cropped score +would have to be discarded the moment the region grew. Mask and evaluate are +therefore siblings feeding select, not a sequence, and numbering them 1-2-3 +would imply an order the code does not have. + +So the interface leads with **Evaluate + select** -- named for the two steps +it holds, and holding both because changing the evaluator changes what +threshold makes sense, so the two are tuned against each other -- and follows +with **Mask**, which is where the candidates get trimmed. The selections list starts with a "Whole image" row so +there is something to trim from the moment the window opens. + +Only evaluation is expensive, and it depends on nothing but the frame, the +evaluator and the subset size. Everything else -- painting a mask, dragging a +polygon vertex, moving the threshold slider, changing the separation -- +re-derives the points from a cached score image, so it updates while the +control is still moving. Changing the subset size or the evaluator is the only +thing that pays for a recomputation. + +Coordinate convention: ``(row, col)`` everywhere, matching the pipeline and +numpy. The image item is set to ``axisOrder='row-major'`` so pyqtgraph's view +coordinates are ``x = column, y = row``, and the only conversion in the whole +module is that swap at the mouse-event boundary. ``SelectionGUIOld`` instead +transposes the frame and carries ``(x, y)`` internally, which is where most of +its axis-order bugs came from. + +This is the interface ``SelectionGUI`` names as of 1.4. The window it replaced +is still importable as :class:`~pyidi.SelectionGUIOld`, deprecated and frozen +until 1.5 -- it offered the same five tools and the same two filters, one +subset at a time. +""" + +import sys +import time + +import numpy as np +import pyqtgraph as pg +from PyQt6 import QtCore, QtGui, QtWidgets +from pyqtgraph import GraphicsLayoutWidget, ImageItem, ScatterPlotItem + +from ..selection import ( + DEFAULT_MAX_POINTS, + DEFAULT_THRESHOLD, + SELECTORS, + SelectionPipeline, + available_evaluators, +) +from ..selection_geometry import _as_size_pair + +#: Grab radius, in screen pixels, within which a drag hits an existing vertex. +#: Screen rather than image pixels so hit-testing feels the same at any zoom. +VERTEX_GRAB_RADIUS_PX = 10 + +#: The two tabs, in order, named for the pipeline steps each one holds. Also +#: the toolbar button labels and the keys of ``step_pages``. Deliberately +#: unnumbered: mask and evaluate do not depend on each other, so there is no +#: step 1. A ``+`` rather than an ``&`` because Qt reads an ampersand in a +#: button label as a mnemonic and swallows it. +STEP_FIND = 'Evaluate + select' +STEP_MASK = 'Mask' +STEPS = (STEP_FIND, STEP_MASK) + +#: What the status bar says on arriving at each tab. The window opens with the +#: whole frame selected, so neither of them is an instruction to draw anything. +STEP_HINTS = { + STEP_FIND: 'Score the frame, then turn the score into points. ' + 'Lower the separation for more points, raise the threshold for better ones.', + STEP_MASK: 'Red points are selected; grey ones are features the mask leaves out. ' + 'The selected row\'s points are ringed.', +} + +#: How a checked tab or tool is marked, applied to the step toolbar and the +#: region-tool grid. +#: +#: Qt's default themes separate a checked ``QPushButton`` from an unchecked one +#: by a shade or two, which is not a difference you can find across a panel -- +#: and this interface asks the question twice, once for which tab you are on and +#: once for which tool is active. Only the checked state is styled, so unchecked +#: buttons stay whatever the platform theme makes them and the window does not +#: have to carry a theme of its own to have a legible one. Bold as well as +#: coloured, so the cue does not rest on colour alone. +CHECKED_BUTTON_STYLE = """ +QPushButton:checked { + background-color: #0078d7; + color: white; + font-weight: bold; +} +""" + +#: Label of the mask row seeded on startup, covering the whole frame. +WHOLE_IMAGE_LABEL = 'Whole image' + +#: How long a redraw may take before the interface stops doing one per control +#: change and starts coalescing them. Roughly a frame: below this the display +#: can follow a dragged slider exactly, above it the requests have to be +#: collapsed or they queue up behind a redraw that is already too slow. +REDRAW_BUDGET_MS = 12.0 + +#: Region tools available in the mask step, as ``(button label, entry kind)``. +#: ``remove`` and ``erase`` are not entry kinds -- they take away rather than +#: making anything, and are the last pair so the two of them read as a group. +TOOLS = ( + ('Polygon', 'polygon'), + ('Brush', 'brush'), + ('Line', 'polyline'), + ('Points', 'points'), + ('Remove point', 'remove'), + ('Remove w/ brush', 'erase'), +) + +#: Kinds whose geometry is a list of draggable vertices. +VERTEX_KINDS = ('polygon', 'polyline') + +#: Tools that paint. Both lay a stroke down the same way and share the radius; +#: they differ only in what the finished stroke does, which is why the erase +#: tool is a tool rather than a mode the brush can be in. +BRUSH_TOOLS = ('brush', 'erase') + +#: Threshold rules offered, as ``(menu label, mode, slider decades or None)``. +#: +#: ``quality`` is logarithmic because the useful settings span three decades: +#: featureless background sits around 0.001 of the best feature and a strong +#: corner at 1, so a linear slider would spend most of its travel in a range +#: where nothing changes. `percentile` is linear over its natural range. +THRESHOLD_RULES = ( + ('quality of the best', 'quality', (1e-3, 1.0)), + ('percentile of scores', 'percentile', None), +) + +#: Where each rule's slider starts, and the value that position means. +THRESHOLD_DEFAULTS = {'quality': DEFAULT_THRESHOLD, 'percentile': 90.0} + + +def odd(value): + """``value`` rounded up to an odd number. + + :param value: a subset extent, in pixels + :type value: int + :rtype: int + """ + value = int(value) + return value if value % 2 else value + 1 + + +class OddSpinBox(QtWidgets.QSpinBox): + """A spin box that holds odd numbers only, by stepping and by typing. + + A subset is centred on the pixel it belongs to, so an even extent has no + centre to be. The pipeline already reads one as the odd size below it -- + a subset size of 10 scores through an 11-pixel window, and draws an + 11-pixel rectangle -- so the even values are a second spelling of the odd + ones and offering them only invites the question of what they do. + """ + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.setSingleStep(2) + + def validate(self, text, position): + state, text, position = super().validate(text, position) + if state == QtGui.QValidator.State.Acceptable and int(text) % 2 == 0: + # Intermediate rather than Invalid, so that typing "12" on the way + # to "121" is not rejected keystroke by keystroke. Anything still + # even when the box loses focus goes through fixup(). + return QtGui.QValidator.State.Intermediate, text, position + return state, text, position + + def fixup(self, text): + try: + return str(odd(text)) + except ValueError: + return super().fixup(text) + + def setValue(self, value): + """Set the value, rounded up to odd. Programmatic writes come here too.""" + super().setValue(odd(value)) + + +#: Length of the segment a dot is drawn as, in image pixels. Only has to be +#: non-zero: Qt draws nothing for a degenerate subpath. +DOT_LENGTH = 1e-3 + + +class DotCloud(QtWidgets.QGraphicsPathItem): + """Uniform round dots, drawn as one stroked path. + + A ``ScatterPlotItem`` is the obvious way to draw these and the wrong one at + this scale: it keeps a record per spot and rebuilds a symbol atlas, which is + 17 ms for seventeen thousand points and is paid on every redraw. A path of + zero-length segments stroked with a round-cap pen draws the same dots -- the + cap *is* the dot -- and is built by one vectorised call: 2 ms. + + The pen is cosmetic, so the dots keep their size in screen pixels at any + zoom, which is what the scatter item did too. + + Only the layers that are genuinely uniform use this. Anything with a per-point + colour, a symbol or a hover behaviour still wants the scatter item. + """ + + def __init__(self, colour, size): + super().__init__() + pen = pg.mkPen(*colour, width=size) + pen.setCapStyle(QtCore.Qt.PenCapStyle.RoundCap) + pen.setCosmetic(True) + self.setPen(pen) + self._pos = np.empty((0, 2)) + + def setData(self, pos): + """Draw a dot at each ``(x, y)`` row of ``pos``.""" + pos = np.asarray(pos, dtype=float) + self._pos = pos + if not len(pos): + self.setPath(QtGui.QPainterPath()) + return + # Each point twice, joined in pairs: one very short segment per point, + # whose round cap is the dot. Short rather than zero-length, because Qt + # drops a degenerate subpath and draws nothing at all; a thousandth of a + # pixel is far below the width the cap draws at any zoom. + x = np.repeat(pos[:, 0], 2) + x[1::2] += DOT_LENGTH + self.setPath(pg.arrayToQPath(x, np.repeat(pos[:, 1], 2), connect='pairs')) + + def clear(self): + self.setData(np.empty((0, 2))) + + def getData(self): + """``(x, y)``, matching :meth:`ScatterPlotItem.getData`.""" + return self._pos[:, 0], self._pos[:, 1] + + +class CanvasViewBox(pg.ViewBox): + """The image view, with brush painting and vertex dragging layered on panning. + + :param parent_gui: the window this view belongs to + :type parent_gui: SelectionGUI + """ + + def __init__(self, parent_gui, *args, **kwargs): + super().__init__(*args, **kwargs) + self.setMouseMode(self.PanMode) + self.parent_gui = parent_gui + self._drag = None + self._direction_start = None + + def _scene_to_rc(self, pos): + """A scene position as ``(row, col)`` floats, or ``None`` if off-image.""" + if not self.sceneBoundingRect().contains(pos): + return None + point = self.mapSceneToView(pos) + return point.y(), point.x() + + def _start_vertex_drag(self, ev): + """Grab a vertex if the drag began near one; otherwise let the view pan.""" + gui = self.parent_gui + if gui.step != STEP_MASK or gui.tool not in VERTEX_KINDS: + return False + position = self._scene_to_rc(ev.buttonDownScenePos()) + if position is None: + return False + entry, index = gui.vertex_at(position) + if entry is None: + return False + self._drag = {'entry': entry, 'index': index, 'original': entry.geometry[index]} + ev.accept() + return True + + def _continue_vertex_drag(self, ev): + """Move the grabbed vertex, committing undo and refreshing on release.""" + if self._drag is None: + return False + position = self._scene_to_rc(ev.scenePos()) + if position is not None: + self._drag['entry'].geometry[self._drag['index']] = position + self.parent_gui.draw_geometry() + if ev.isFinish(): + self.parent_gui.push_undo({ + 'type': 'vertex_move', + 'entry': self._drag['entry'], + 'index': self._drag['index'], + 'original': self._drag['original'], + }) + self.parent_gui.refresh() + self._drag = None + ev.accept() + return True + + def _handle_direction_drag(self, ev): + """Drag out the gradient direction while the ``Draw`` button is armed. + + A direction is a thing you point at. Typing two components and checking + the heatmap afterwards is a slower way of saying the same thing, so this + reproduces the drag ``SelectionGUIOld`` offered. + """ + gui = self.parent_gui + if not gui.drawing_direction: + return False + ev.accept() + if ev.isStart(): + self._direction_start = self._scene_to_rc(ev.buttonDownScenePos()) + return True + position = self._scene_to_rc(ev.scenePos()) + if self._direction_start is None or position is None: + return True + gui.show_direction(self._direction_start, position) + if ev.isFinish(): + gui.set_direction_from_drag(self._direction_start, position) + self._direction_start = None + return True + + @staticmethod + def _ctrl(ev): + """Whether Ctrl was down when this event was delivered. + + Read off the event rather than tracked by a key filter on the window: a + panel widget with focus can swallow the key press, and a Ctrl released + while the window is not focused is never seen at all, either of which + leaves a tracked flag stuck at the wrong value. + """ + return bool(ev.modifiers() & QtCore.Qt.KeyboardModifier.ControlModifier) + + def _handle_brush_drag(self, ev): + """Paint while Ctrl is held and the brush tool is active. + + A stroke already under way keeps the drag whether Ctrl is still down or + not, so letting go of the key mid-stroke finishes the stroke instead of + abandoning it half-painted. + """ + gui = self.parent_gui + if not (gui.step == STEP_MASK and gui.tool in BRUSH_TOOLS + and (self._ctrl(ev) or gui.painting)): + return False + ev.accept() + if ev.isStart(): + gui.brush_start() + elif ev.isFinish(): + gui.brush_move(self._scene_to_rc(ev.scenePos())) + gui.brush_end() + else: + gui.brush_move(self._scene_to_rc(ev.scenePos())) + return True + + def mouseClickEvent(self, ev): + gui = self.parent_gui + if gui.step == STEP_MASK and gui.tool in BRUSH_TOOLS: + if self._ctrl(ev): + ev.accept() + gui.brush_start() + gui.brush_move(self._scene_to_rc(ev.scenePos())) + gui.brush_end() + else: + ev.ignore() + return + super().mouseClickEvent(ev) + + def mouseDragEvent(self, ev, axis=None): + if self._handle_direction_drag(ev): + return + if self._handle_brush_drag(ev): + return + if ev.isStart(): + if self._start_vertex_drag(ev): + return + elif self._continue_vertex_drag(ev): + return + super().mouseDragEvent(ev, axis) + + +class SelectionGUI(QtWidgets.QMainWindow): + """Pick tracking points by masking, scoring and selecting. + + The window is modal: the constructor blocks until it is closed, and the + points are then available through :attr:`points` or :meth:`get_points`. + + .. versionchanged:: 1.4 + ``SelectionGUI`` now names this interface. The window it replaced is + :class:`~pyidi.SelectionGUIOld`, deprecated and removed in 1.5. The + constructor signature is unchanged and ``get_points()`` still returns + ``(row, col)``, so a script that only constructs the window and reads + its points needs no edit at all. + + Everything the old window did is here, under different names. Its five + selection methods are the mask tools, a row's *role* deciding whether a + region bounds the search or *is* the answer: + + ===================== ============================================================== + ``SelectionGUIOld`` here + ===================== ============================================================== + Grid Polygon tool with role ``points``, or the ``lattice`` selector + Manual Points tool + Along the line Line tool + Brush Brush tool + Remove point Remove point tool + Shi-Tomasi filter ``shi_tomasi`` evaluator + Gradient in direction ``gradient_direction`` evaluator + ===================== ============================================================== + + The difference is what happens underneath. The old window placed subsets + and then scored the ones it had placed; this one scores every position in + the frame once, caches that, and re-derives the points from it. So the + threshold and separation follow a dragged slider, the evaluators are a + registry rather than two hard-coded branches (see + :func:`~pyidi.selection.register_evaluator`), and the whole pipeline runs + without Qt -- :mod:`pyidi.selection` is importable on its own. + + :param video: a ``VideoReader``, a 2-D ``(height, width)`` image, or a 3-D + ``(n_frames, height, width)`` stack whose first frame is used + :type video: VideoReader or numpy.ndarray + :param subset_size: side length of the subset, as a scalar or a + ``(height, width)`` pair. The scoring window follows it, so the score + always answers the question "how well would *this* subset track". + :type subset_size: int or tuple + :param subset_overlap: extra spacing between the points a ``points``-role + entry lays out; positive spreads them apart, negative overlaps them + :type subset_overlap: int + :raises TypeError: if ``video`` is none of the accepted types + """ + + def __init__(self, video, subset_size=11, subset_overlap=0): + app = QtWidgets.QApplication.instance() or QtWidgets.QApplication([]) + super().__init__() + self.setWindowTitle('Feature Selection') + self.resize(1250, 820) + + self.frame = self._frame_from(video) + # Rounded up to odd here rather than only in the spin box, so that the + # pipeline and the control that shows it never disagree. + height, width = _as_size_pair(subset_size) + self.pipeline = SelectionPipeline(self.frame, (odd(height), odd(width)), subset_overlap) + self.pipeline.define_score('score', 'shi_tomasi') + + self.step = STEP_FIND + self.tool = 'polygon' + self._whole_image = None + self.drawing_direction = False + self.direction_spins = [] + self.direction_button = None + self.score_toggles = [] + self._paint = None + self._stroke_path = None + self._syncing = False + self._last_refresh_ms = 0.0 + self._refresh_timer = QtCore.QTimer(self) + self._refresh_timer.setSingleShot(True) + self._refresh_timer.timeout.connect(self.refresh) + self.active_index = None + self.undo_stack = [] + self.undo_limit = 50 + + QtGui.QShortcut(QtGui.QKeySequence.StandardKey.Undo, self).activated.connect(self.undo) + + self._build_ui() + self.image_item.setImage(self.frame) + self.add_whole_image_mask() + self.select_step(STEP_FIND) + self.refresh() + + self.show() + if not hasattr(sys, 'ps1'): + sys.exit(app.exec()) + else: + app.exec() + + # -- construction ------------------------------------------------------ + + @staticmethod + def _frame_from(video): + """The single 2-D frame to work on, whatever form the video came in.""" + from ..video_reader import VideoReader + + if isinstance(video, VideoReader): + return video.get_frame(0) + if isinstance(video, np.ndarray) and video.ndim == 3: + return video[0] + if isinstance(video, np.ndarray) and video.ndim == 2: + return video + raise TypeError( + f'`video` must be a VideoReader, or a 2-D (height, width) or 3-D ' + f'(n_frames, height, width) np.ndarray, got {type(video).__name__!r}.' + ) + + def _build_ui(self): + central = QtWidgets.QWidget() + self.setCentralWidget(central) + layout = QtWidgets.QVBoxLayout(central) + layout.setContentsMargins(0, 0, 0, 0) + layout.setSpacing(0) + + layout.addWidget(self._build_step_toolbar()) + self.splitter = QtWidgets.QSplitter(QtCore.Qt.Orientation.Horizontal) + layout.addWidget(self.splitter, stretch=1) + + self._build_canvas() + self._build_panel() + + self.status = self.statusBar() + + def _build_step_toolbar(self): + bar = QtWidgets.QWidget() + row = QtWidgets.QHBoxLayout(bar) + row.setContentsMargins(5, 4, 5, 4) + self.step_buttons = {} + group = QtWidgets.QButtonGroup(self) + group.setExclusive(True) + for name in STEPS: + button = QtWidgets.QPushButton(name) + button.setCheckable(True) + button.setMinimumWidth(120) + group.addButton(button) + row.addWidget(button) + button.clicked.connect(lambda _, n=name: self.select_step(n)) + self.step_buttons[name] = button + row.addStretch(1) + bar.setStyleSheet(CHECKED_BUTTON_STYLE) + bar.setSizePolicy(QtWidgets.QSizePolicy.Policy.Expanding, QtWidgets.QSizePolicy.Policy.Fixed) + return bar + + def _build_canvas(self): + self.pg_widget = GraphicsLayoutWidget() + self.view = CanvasViewBox(parent_gui=self, lockAspect=True, invertY=True) + self.pg_widget.addItem(self.view) + + # row-major throughout: view x is the column index and y the row index, + # so nothing in this module has to transpose anything but a mouse event. + self.image_item = ImageItem(axisOrder='row-major') + self.score_overlay = ImageItem(axisOrder='row-major') + self.roi_overlay = ImageItem(axisOrder='row-major') + + # The subset borders live apart from the translucent fill so they can be + # stroked with a *cosmetic* pen, whose width is in screen pixels: they stay + # a hairline at any zoom, where a raster border cannot go below one image + # pixel and becomes a thick band as soon as you zoom in. + self.roi_outline = QtWidgets.QGraphicsPathItem() + pen = pg.mkPen(0, 255, 0, 150) + pen.setCosmetic(True) + self.roi_outline.setPen(pen) + self.roi_outline.setBrush(pg.mkBrush(None)) + + # The stroke being painted is a path of overlapping discs rather than a + # raster overlay. A raster one has to be rebuilt and re-uploaded whole on + # every mouse move -- eight milliseconds a move on a four-megapixel frame, + # paid while the mouse is moving, which is exactly when it is felt. + self.brush_overlay = QtWidgets.QGraphicsPathItem() + self.brush_overlay.setPen(pg.mkPen(None)) + + self.geometry_line = pg.PlotDataItem(pen=pg.mkPen('y', width=2)) + self.geometry_vertices = ScatterPlotItem(pen=pg.mkPen(None), brush=pg.mkBrush(255, 255, 0, 200), size=7) + self.point_scatter = DotCloud((255, 100, 100, 220), 7) + # The features the mask is leaving out, shown while masking so that an + # empty patch says which of the two things it is: nothing to track + # there, or something you have masked away. Smaller and greyer than a + # selected point, and drawn under it, so the two never read alike. + self.candidate_scatter = DotCloud((90, 170, 255, 170), 5) + # Points the deselect brush is about to take away, shown while the stroke + # is still being painted rather than only after the mouse comes up. White, + # and drawn over the stroke's red wash rather than under it, which is the + # only place it is ever seen. + self.doomed_scatter = ScatterPlotItem( + pen=pg.mkPen(255, 255, 255, 240, width=1.5), brush=pg.mkBrush(None), size=9, symbol='x') + self.highlight_scatter = ScatterPlotItem( + pen=pg.mkPen(255, 0, 255, 230, width=2), brush=pg.mkBrush(None), size=13) + self.direction_line = pg.PlotDataItem(pen=pg.mkPen('r', width=2)) + + for item, z in ((self.image_item, 0), (self.score_overlay, 0.5), (self.roi_overlay, 1), + (self.roi_outline, 1), (self.geometry_line, 2), (self.geometry_vertices, 2), + (self.candidate_scatter, 2.5), (self.point_scatter, 3), + (self.highlight_scatter, 3), (self.direction_line, 3.5), + (self.brush_overlay, 4), (self.doomed_scatter, 4.5)): + item.setZValue(z) + self.view.addItem(item) + self.score_overlay.setVisible(False) + + self.pg_widget.scene().sigMouseClicked.connect(self.on_mouse_click) + self.splitter.addWidget(self.pg_widget) + + def _build_panel(self): + panel = QtWidgets.QWidget() + column = QtWidgets.QVBoxLayout(panel) + + self.step_stack = QtWidgets.QStackedLayout() + self.step_pages = {} + for name, builder in ((STEP_FIND, self._build_find_page), + (STEP_MASK, self._build_mask_page)): + page = QtWidgets.QWidget() + builder(QtWidgets.QVBoxLayout(page)) + self.step_stack.addWidget(page) + self.step_pages[name] = page + column.addLayout(self.step_stack, stretch=1) + + # Outside the stack, so it shows on both tabs. It is neither tab's + # setting: the scoring window follows it, and so does the rectangle + # drawn round every point. + # The subset size is on both tabs because both read it; the selections + # list is not, because every row in it and every button under it belongs + # to the mask step. Nothing on the other tab acts on a row. + column.addWidget(self._build_subset_group()) + self.selection_box = self._build_selection_list() + column.addWidget(self.selection_box) + + self.count_label = QtWidgets.QLabel('0 points') + font = self.count_label.font() + font.setBold(True) + self.count_label.setFont(font) + column.addWidget(self.count_label) + + panel.setMinimumWidth(320) + panel.setMaximumWidth(600) + self.splitter.addWidget(panel) + self.splitter.setStretchFactor(0, 5) + self.splitter.setStretchFactor(1, 0) + self.splitter.setSizes([920, 340]) + + def _build_mask_page(self, layout): + tools = QtWidgets.QGroupBox('Region tool') + grid = QtWidgets.QGridLayout(tools) + self.tool_buttons = {} + group = QtWidgets.QButtonGroup(self) + group.setExclusive(True) + # Two columns: five full-width buttons stacked was most of the panel's + # height for something you click once. + for index, (label, kind) in enumerate(TOOLS): + button = QtWidgets.QPushButton(label) + button.setCheckable(True) + group.addButton(button) + grid.addWidget(button, index // 2, index % 2) + button.clicked.connect(lambda _, k=kind: self.select_tool(k)) + self.tool_buttons[kind] = button + self.tool_buttons['polygon'].setChecked(True) + tools.setStyleSheet(CHECKED_BUTTON_STYLE) + layout.addWidget(tools) + + self.new_entry_button = QtWidgets.QPushButton('Start new polygon') + self.new_entry_button.setToolTip( + 'Begin a second polygon or line instead of adding vertices to the one ' + 'already selected in the list.') + self.new_entry_button.clicked.connect(self.start_new_entry) + layout.addWidget(self.new_entry_button) + + self.brush_radius = QtWidgets.QSlider(QtCore.Qt.Orientation.Horizontal) + self.brush_radius.setRange(1, 100) + self.brush_radius.setValue(12) + self.brush_radius.setToolTip( + 'Radius of the painted dab, in pixels. Shared by both brush tools, so ' + 'a stroke erases exactly as wide as it paints.') + self.brush_form = self._form() + self.brush_form.addRow('Brush radius', self.brush_radius) + layout.addLayout(self.brush_form) + # Painting is gated on a brush tool being active (see + # ``_handle_brush_drag``), so with any other tool this does nothing. + self._update_brush_row() + + self.spacing_spin = QtWidgets.QSpinBox() + self.spacing_spin.setRange(-500, 500) + self.spacing_spin.setValue(self.pipeline.spacing) + self.spacing_spin.valueChanged.connect(self._on_spacing_changed) + self.spacing_spin.setToolTip( + 'Extra spacing between the points a "points" row lays out. Rows that ' + 'act as a mask are unaffected: how far apart their points end up is ' + 'the separation, on the other tab.') + self.spacing_form = self._form() + self.spacing_form.addRow('Point spacing', self.spacing_spin) + layout.addLayout(self.spacing_form) + self._update_spacing_row() + + # The overlay is as useful here as on the other tab: it is what tells you + # whether the area you are about to keep has anything worth tracking in it. + self.show_score_mask = self._make_score_toggle() + layout.addWidget(self.show_score_mask) + + clear = QtWidgets.QPushButton('Clear all') + clear.setToolTip('Drop every selection and go back to the whole frame.') + clear.clicked.connect(self.clear_all) + layout.addWidget(clear) + layout.addStretch(1) + + def _build_subset_group(self): + """The subset size, shown on every tab because it belongs to none of them. + + It drives *evaluate* -- the scoring window is the subset size, so this is + one of only three things that make the score image stale -- and it is + also what the rectangle drawn round each point measures while you mask. + Putting it on one tab would make it look like that tab's setting and hide + it from the other, so it sits below the tabs instead. + """ + box = QtWidgets.QGroupBox('Subset') + grid = QtWidgets.QGridLayout(box) + height, width = self.pipeline.subset_size + + self.square_check = QtWidgets.QCheckBox('Square subsets') + self.square_check.setToolTip('Untick to set the height and the width separately.') + self.square_check.setChecked(height == width) + self.square_check.toggled.connect(self._on_square_toggled) + grid.addWidget(self.square_check, 0, 0, 1, 2) + + self.height_spin = OddSpinBox() + self.width_spin = OddSpinBox() + for spin, value in ((self.height_spin, height), (self.width_spin, width)): + spin.setToolTip( + 'The size of the subset each point stands for, in pixels. Odd only: ' + 'the subset is centred on its point, so an even extent has no centre ' + 'to be. The scoring window follows it, so this is one of the few ' + 'settings that makes the score stale and pays for a fresh evaluation.') + spin.setRange(3, 501) + spin.setValue(value) + spin.valueChanged.connect(self._on_subset_size_changed) + self.width_spin.setEnabled(height != width) + grid.addWidget(QtWidgets.QLabel('Height'), 1, 0) + grid.addWidget(self.height_spin, 1, 1) + grid.addWidget(QtWidgets.QLabel('Width'), 2, 0) + grid.addWidget(self.width_spin, 2, 1) + + self.show_subsets = QtWidgets.QCheckBox('Show subsets') + self.show_subsets.setToolTip( + 'Draw each point as the subset it stands for, so overlapping subsets are ' + 'visible. It changes nothing about the selection.') + self.show_subsets.setChecked(True) + self.show_subsets.toggled.connect(lambda _: self.draw_points()) + grid.addWidget(self.show_subsets, 3, 0, 1, 2) + return box + + def _build_find_page(self, layout): + """Evaluate and select, in one panel. + + They are tuned against each other -- switching the evaluator changes + what threshold means -- so splitting them across tabs would only buy a + tab switch after every change. + """ + layout.addWidget(self._build_evaluate_group()) + layout.addWidget(self._build_select_group()) + + # The only prose left on the panel, and it is empty unless something is + # actually wrong. Everything the labels used to say -- what an evaluator + # measures, what gets recomputed -- is a tooltip now: it was permanent + # screen furniture that you read once. + self.select_note = QtWidgets.QLabel('') + self.select_note.setWordWrap(True) + layout.addWidget(self.select_note) + layout.addStretch(1) + self._update_selector_rows() + + def _build_evaluate_group(self): + """The evaluator and its parameters, in one flat form. + + The parameters used to sit in a group box inside this one. Two nested + frames cost two sets of margins out of a panel that is already narrow, + which is what was clipping the values off the right-hand side, and the + inner title said nothing the rows did not. + """ + box = QtWidgets.QGroupBox('Evaluate') + layout = QtWidgets.QVBoxLayout(box) + self.param_layout = self._form() + layout.addLayout(self.param_layout) + + self.evaluator_combo = QtWidgets.QComboBox() + self.evaluators = available_evaluators() + for name, spec in sorted(self.evaluators.items()): + self.evaluator_combo.addItem(spec.display_name, name) + self.evaluator_combo.setCurrentIndex(self.evaluator_combo.findData('shi_tomasi')) + self.evaluator_combo.currentIndexChanged.connect(self._on_evaluator_changed) + self.param_layout.addRow('Score', self.evaluator_combo) + + self.param_widgets = {} + self._rebuild_param_widgets() + + self.show_score = self._make_score_toggle() + layout.addWidget(self.show_score) + return box + + @staticmethod + def _form(): + """A form layout whose fields take the width they are given. + + :rtype: PyQt6.QtWidgets.QFormLayout + """ + form = QtWidgets.QFormLayout() + form.setContentsMargins(0, 0, 0, 0) + form.setFieldGrowthPolicy(QtWidgets.QFormLayout.FieldGrowthPolicy.AllNonFixedFieldsGrow) + return form + + def _make_score_toggle(self): + """A ``Show score overlay`` checkbox, ganged to every other one. + + Each tab gets its own, because the overlay answers a different question + on each -- "is this threshold too tight or is there nothing there" on + one, "does the area I am keeping have anything in it" on the other -- + and having to switch tabs to turn it on would defeat both. + + :rtype: PyQt6.QtWidgets.QCheckBox + """ + box = QtWidgets.QCheckBox('Show score overlay') + box.setToolTip( + 'Draw the score itself as a heatmap, so you can see where the trackable ' + 'content is before committing to any points. The border the subset window ' + 'cannot reach is left transparent: it is unscored, not scored badly.') + box.toggled.connect(self._on_show_score_toggled) + self.score_toggles.append(box) + return box + + def _build_select_group(self): + """The selector and its settings, in one flat form. + + Rows that the current selector ignores are hidden rather than greyed + out: a disabled ``Grid pitch`` is a line of panel spent saying that this + line does not apply. + """ + box = QtWidgets.QGroupBox('Select') + layout = QtWidgets.QVBoxLayout(box) + self.select_form = self._form() + layout.addLayout(self.select_form) + + self.selector_combo = QtWidgets.QComboBox() + for name in sorted(SELECTORS): + self.selector_combo.addItem(name, name) + self.selector_combo.setCurrentIndex(self.selector_combo.findData('peaks')) + self.selector_combo.setToolTip( + '"peaks" puts a point on each local maximum of the score, which is how ' + 'the points end up on the features. "lattice" puts them on a regular ' + 'grid instead, for even coverage rather than the best features.') + self.selector_combo.currentIndexChanged.connect(self._on_selector_changed) + self.select_form.addRow('Points', self.selector_combo) + + self.threshold_mode = QtWidgets.QComboBox() + for label, mode, _ in THRESHOLD_RULES: + self.threshold_mode.addItem(label, mode) + self.threshold_mode.setToolTip( + 'Quality is a fraction of the best feature in the region, so 0.01 means ' + '"at least a hundredth as good as the best". Percentile ranks pixels ' + 'instead, and on a dense score image the pixels are overwhelmingly ' + 'background, so it is only really useful with the lattice selector.') + self.threshold_mode.currentIndexChanged.connect(self._on_threshold_mode_changed) + self.select_form.addRow('Threshold', self.threshold_mode) + + self.threshold_slider = QtWidgets.QSlider(QtCore.Qt.Orientation.Horizontal) + self.threshold_slider.setRange(0, 1000) + self.threshold_slider.setValue(self._slider_position('quality', DEFAULT_THRESHOLD)) + self.threshold_slider.setToolTip( + 'How good a subset has to be to be worth tracking. The quality scale is ' + 'logarithmic: featureless background sits near 0.001 of the best feature ' + 'and a strong corner near 1.') + self.threshold_slider.valueChanged.connect(self._on_threshold_changed) + self.threshold_label = QtWidgets.QLabel(f'{DEFAULT_THRESHOLD:.3g}') + self.threshold_label.setMinimumWidth(40) + self.threshold_label.setAlignment(QtCore.Qt.AlignmentFlag.AlignRight + | QtCore.Qt.AlignmentFlag.AlignVCenter) + slider_row = QtWidgets.QWidget() + slider_layout = QtWidgets.QHBoxLayout(slider_row) + slider_layout.setContentsMargins(0, 0, 0, 0) + slider_layout.addWidget(self.threshold_slider, stretch=1) + slider_layout.addWidget(self.threshold_label) + self.select_form.addRow('', slider_row) + + self.separation_spin = QtWidgets.QSpinBox() + self.separation_spin.setRange(1, 500) + self.separation_spin.setSuffix(' px') + self.separation_spin.setValue(self.pipeline.selector_params['separation']) + self.separation_spin.setToolTip( + 'No two points end up closer together than this, so it is the control for ' + 'how many you get: lower it for more. Thinning the pixels above the ' + 'threshold any other way puts most of the subsets back-to-back on the ' + 'same feature, which is why this is not a "keep every n-th".') + self.separation_spin.valueChanged.connect(self._on_separation_changed) + self.select_form.addRow('Separation', self.separation_spin) + + self.pitch_spin = QtWidgets.QSpinBox() + self.pitch_spin.setRange(1, 500) + self.pitch_spin.setValue(12) + self.pitch_spin.setToolTip('Distance between grid positions, for the lattice selector.') + self.pitch_spin.valueChanged.connect(self._on_pitch_changed) + self.select_form.addRow('Grid pitch', self.pitch_spin) + + self.max_points_spin = QtWidgets.QSpinBox() + self.max_points_spin.setRange(1, 200000) + self.max_points_spin.setValue(DEFAULT_MAX_POINTS) + self.max_points_spin.setToolTip( + 'A safety valve, not a target: the selection stops here however many ' + 'points the threshold and the separation would have given. It says so ' + 'below when it is what stopped it.') + self.max_points_spin.valueChanged.connect(self._on_max_points_changed) + self.select_form.addRow('Maximum points', self.max_points_spin) + + self.decimation_spin = QtWidgets.QSpinBox() + self.decimation_spin.setRange(1, 100) + self.decimation_spin.setValue(1) + self.decimation_spin.setToolTip( + 'Keep every n-th of the points already selected. Unlike a wider ' + 'separation, which re-selects and moves every point, this leaves the ' + 'survivors exactly where they are -- for when the selection is right and ' + 'only the count is too high for the computation you are about to run.') + self.decimation_spin.valueChanged.connect(self._on_decimation_changed) + self.select_form.addRow('Keep every n-th', self.decimation_spin) + return box + + @staticmethod + def _slider_position(mode, value): + """Where a threshold value sits on the 0..1000 slider, for ``mode``.""" + if mode == 'quality': + low, high = dict((m, r) for _, m, r in THRESHOLD_RULES)['quality'] + span = np.log10(high) - np.log10(low) + return int(round(1000 * (np.log10(max(value, low)) - np.log10(low)) / span)) + return int(round(value * (10 if mode == 'percentile' else 1000))) + + @staticmethod + def _slider_value(mode, position): + """The threshold a slider position means, and how to print it.""" + if mode == 'quality': + low, high = dict((m, r) for _, m, r in THRESHOLD_RULES)['quality'] + span = np.log10(high) - np.log10(low) + value = 10 ** (np.log10(low) + span * position / 1000.0) + return value, f'{value:.3g}' + if mode == 'percentile': + return position / 10.0, f'{position / 10.0:.1f}' + return position / 1000.0, f'{position / 1000.0:.3f}' + + def _build_selection_list(self): + box = QtWidgets.QGroupBox('Selections') + column = QtWidgets.QVBoxLayout(box) + self.entry_list = QtWidgets.QListWidget() + self.entry_list.currentRowChanged.connect(self._on_row_changed) + self.entry_list.itemChanged.connect(self._on_item_changed) + column.addWidget(self.entry_list) + + buttons = QtWidgets.QHBoxLayout() + self.role_button = QtWidgets.QPushButton('Use as points') + self.role_button.setToolTip( + 'A "mask" row says where points may go and lets the selection choose ' + 'them; a "points" row contributes its own coordinates directly, whatever ' + 'they score. Switching does not redraw the region.') + self.role_button.clicked.connect(self.toggle_role) + buttons.addWidget(self.role_button) + delete = QtWidgets.QPushButton('Delete') + delete.clicked.connect(self.delete_active) + buttons.addWidget(delete) + column.addLayout(buttons) + return box + + # -- steps and tools --------------------------------------------------- + + def select_step(self, name): + """Switch tab. + + :param name: one of :data:`STEPS` + :type name: str + """ + self.step = name + self.step_buttons[name].setChecked(True) + self.step_stack.setCurrentWidget(self.step_pages[name]) + self.status.showMessage(STEP_HINTS[name]) + # The magenta ring and the vertex handles mark what is being edited, so + # they belong to the Mask tab; elsewhere they would decorate geometry + # nobody is touching. + self.selection_box.setVisible(name == STEP_MASK) + self.highlight_scatter.setVisible(name == STEP_MASK) + self.geometry_line.setVisible(name == STEP_MASK) + self.geometry_vertices.setVisible(name == STEP_MASK) + self.refresh() + + def select_tool(self, kind): + """Make a region tool active. + + :param kind: an entry kind, or ``'remove'`` + :type kind: str + """ + self.tool = kind + self.tool_buttons[kind].setChecked(True) + self._update_brush_row() + self.new_entry_button.setEnabled(kind in VERTEX_KINDS) + if kind in VERTEX_KINDS: + self.new_entry_button.setText(f'Start new {"polygon" if kind == "polygon" else "line"}') + self.status.showMessage({ + 'polygon': 'Click to place polygon corners. The enclosed area becomes a mask.', + 'brush': 'Hold Ctrl and drag to paint a mask.', + 'polyline': 'Click to place line vertices. Points are spaced along the segments.', + 'points': 'Click to place individual points. They bypass scoring.', + 'remove': 'Click near a point to remove it.', + 'erase': 'Hold Ctrl and drag to take away the area you paint over. ' + 'Points about to go are crossed out while you paint.', + }[kind]) + + def start_new_entry(self): + """Begin a fresh polygon or polyline instead of extending the active one.""" + if self.tool in VERTEX_KINDS: + entry = self.pipeline.add_entry(self.tool, []) + self.active_index = len(self.pipeline.entries) - 1 + self.push_undo({'type': 'add', 'entry': entry}) + self.refresh() + + # -- entries ----------------------------------------------------------- + + def active_entry(self, kind=None): + """The entry the selections list has selected, if it is of ``kind``. + + :param kind: required entry kind, or ``None`` for any + :type kind: str or None + :rtype: Entry or None + """ + if self.active_index is None or not (0 <= self.active_index < len(self.pipeline.entries)): + return None + entry = self.pipeline.entries[self.active_index] + return entry if kind is None or entry.kind == kind else None + + def _entry_for_tool(self, kind): + """The entry a click should extend, creating one when there is none.""" + entry = self.active_entry(kind) + if entry is not None: + return entry + for index in range(len(self.pipeline.entries) - 1, -1, -1): + if self.pipeline.entries[index].kind == kind: + self.active_index = index + return self.pipeline.entries[index] + entry = self.pipeline.add_entry(kind, []) + self.active_index = len(self.pipeline.entries) - 1 + return entry + + def add_whole_image_mask(self): + """Seed the selections list with a mask covering the whole frame. + + Without it the window opens showing nothing, and the Mask tab would have + to be visited before anything happened -- which is the workflow this + ordering exists to avoid. With it, the candidates are there to look at + immediately and masking becomes what it should be: trimming them. + + It is a row like any other, so it can be unchecked, painted away with + the deselect brush, or deleted outright. Deleting it selects nothing, + which is the same rule as for every other mask row. The geometry is a + brush mask rather than a four-corner polygon because rasterising it is + then a copy rather than a point-in-polygon test over every pixel of the + frame, on every redraw. + + :return: the new entry + :rtype: Entry + """ + entry = self.pipeline.add_entry( + 'brush', np.ones(self.pipeline.shape, dtype=bool), label=WHOLE_IMAGE_LABEL) + self._whole_image = entry + self.active_index = len(self.pipeline.entries) - 1 + return entry + + def _retire_whole_image(self): + """Uncheck the seeded whole-image row once a drawn region covers something. + + Mask rows combine as a *union*, so a region drawn while the whole frame + is still selected changes nothing at all -- you draw a polygon and the + points do not move, which reads as the drawing being broken. Standing the + seeded row down as soon as another mask has area makes the drawing do what + it looks like it does. + + The row is unchecked rather than deleted, so ticking it again in the list + brings the whole frame back, and the undo stack records the change. + """ + seeded, others = None, False + for entry in self.pipeline.entries: + # By identity, not by label: the label is what the row is called, + # which is not the same as which row this is. + if entry is self._whole_image: + seeded = entry + elif entry.role == 'mask' and entry.visible and self.pipeline.area(entry).any(): + others = True + if seeded is None or not seeded.visible or not others: + return + seeded.visible = False + self.push_undo({'type': 'visible', 'entry': seeded, 'value': True}) + self.status.showMessage( + f'Unchecked "{WHOLE_IMAGE_LABEL}" so the region you drew takes effect. ' + 'Tick it again in the list to bring the whole frame back.') + + def toggle_role(self): + """Flip the active row between contributing an area and contributing points.""" + entry = self.active_entry() + if entry is None: + return + entry.role = 'points' if entry.role == 'mask' else 'mask' + self.refresh() + + def delete_active(self): + """Delete the active row.""" + entry = self.active_entry() + if entry is None: + return + index = self.active_index + self.push_undo({'type': 'delete', 'entry': entry, 'index': index}) + self.pipeline.entries.pop(index) + self.active_index = min(index, len(self.pipeline.entries) - 1) + if self.active_index < 0: + self.active_index = None + self.refresh() + + def clear_all(self): + """Start over: drop every selection and seed the whole-image row again. + + "Start over" means the state the window opens in, which has the whole + frame selected -- not an empty canvas. Clearing back to nothing would + leave you looking at a blank frame and needing to know that a mask is + what brings the points back. + + Deleting the whole-image row on its own still selects nothing. That is a + different act: it says "not this area", where this one says "forget what + I have done so far". + """ + self.push_undo({'type': 'restore', 'entries': list(self.pipeline.entries)}) + self.pipeline.entries = [] + self.add_whole_image_mask() + self.refresh() + + # -- mouse ------------------------------------------------------------- + + def on_mouse_click(self, event): + """Route a click on the image to whichever tool is active.""" + if self.step != STEP_MASK or event.button() != QtCore.Qt.MouseButton.LeftButton: + return + if not self.view.sceneBoundingRect().contains(event.scenePos()): + return + point = self.view.mapSceneToView(event.scenePos()) + position = (point.y(), point.x()) # view y is the row, x the column + + if self.tool == 'remove': + self.remove_nearest_point(position) + elif self.tool in VERTEX_KINDS: + self.add_vertex(position) + elif self.tool == 'points': + self.add_point(position) + else: + return + self._retire_whole_image() + self.refresh() + + def add_vertex(self, position): + """Append a vertex to the active polygon or polyline.""" + entry = self._entry_for_tool(self.tool) + rounded = (float(position[0]), float(position[1])) + if any(np.hypot(v[0] - rounded[0], v[1] - rounded[1]) < 1e-6 for v in entry.geometry): + return # clicking exactly on a vertex must not stack a duplicate + entry.geometry.append(rounded) + self.push_undo({'type': 'vertex_add', 'entry': entry}) + + def add_point(self, position): + """Append a hand-picked coordinate, if the click landed on the image. + + The view is larger than the frame -- the aspect is locked, so one axis + always has a margin, and zooming out adds more -- and a subset centred + off the frame is not something that can be tracked. So a click outside + it is ignored rather than recorded as a point nothing downstream can use. + """ + coordinate = (int(round(position[0])), int(round(position[1]))) + height, width = self.pipeline.shape + if not (0 <= coordinate[0] < height and 0 <= coordinate[1] < width): + self.status.showMessage('That click was outside the image, so no point was added.') + return + entry = self._entry_for_tool('points') + if coordinate in entry.geometry: + return + entry.geometry.append(coordinate) + entry.removed.discard(coordinate) + self.push_undo({'type': 'vertex_add', 'entry': entry}) + + def remove_nearest_point(self, position): + """Remove whichever displayed point is nearest the click, if any is close.""" + credited = getattr(self, '_credited', None) or self.pipeline.points_by_entry() + best, best_entry, best_distance = None, None, np.inf + for entry, points in zip(self.pipeline.entries, credited): + if not len(points): + continue + distances = np.hypot(points[:, 0] - position[0], points[:, 1] - position[1]) + nearest = int(distances.argmin()) + if distances[nearest] < best_distance: + best_distance = float(distances[nearest]) + best_entry = entry + best = (int(points[nearest, 0]), int(points[nearest, 1])) + if best is None or best_distance > max(self.pipeline.subset_size): + return + self.push_undo(self._snapshot()) + self.pipeline.remove_point(best_entry, best) + + def vertex_at(self, position): + """The vertex within the grab radius of ``position``, if any. + + The radius is constant in *screen* pixels, so grabbing a vertex feels the + same however far the view is zoomed. + + :param position: ``(row, col)`` in image coordinates + :type position: tuple + :return: ``(entry, vertex index)``, or ``(None, None)`` + :rtype: tuple + """ + scale = self.view.viewPixelSize()[0] or 1.0 + radius = VERTEX_GRAB_RADIUS_PX * scale + for entry in self.pipeline.entries: + if entry.kind != self.tool or not entry.visible: + continue + for index, vertex in enumerate(entry.geometry): + if np.hypot(vertex[0] - position[0], vertex[1] - position[1]) <= radius: + return entry, index + return None, None + + # -- brush ------------------------------------------------------------- + + @property + def painting(self): + """Whether a brush stroke is currently being laid down. + + :rtype: bool + """ + return self._paint is not None + + def brush_start(self): + """Begin a stroke.""" + self._paint = np.zeros(self.pipeline.shape, dtype=bool) + self._stroke_path = QtGui.QPainterPath() + self._stroke_path.setFillRule(QtCore.Qt.FillRule.WindingFill) + + def brush_move(self, position): + """Add a dab at ``position``, given as ``(row, col)`` or ``None``.""" + if self._paint is None or position is None: + return + row, col = int(round(position[0])), int(round(position[1])) + radius = self.brush_radius.value() + height, width = self._paint.shape + rows, cols = np.ogrid[max(0, row - radius):min(height, row + radius + 1), + max(0, col - radius):min(width, col + radius + 1)] + dab = (rows - row) ** 2 + (cols - col) ** 2 <= radius ** 2 + self._paint[max(0, row - radius):min(height, row + radius + 1), + max(0, col - radius):min(width, col + radius + 1)][dab] = True + # x is the column and y the row, and the ellipse is inscribed in the + # square the raster dab fills. + self._stroke_path.addEllipse( + QtCore.QRectF(col - radius, row - radius, 2 * radius + 1, 2 * radius + 1)) + self.draw_brush() + if self.deselect_mode: + # Cheap: it re-reads the stroke against points already computed, + # rather than re-running the pipeline on every mouse move. + self.draw_doomed() + + def brush_end(self): + """Commit the stroke, either as a new region or as a deselection.""" + if self._paint is None: + return + stroke = self._paint + self._paint = None + self._stroke_path = QtGui.QPainterPath() + if not stroke.any(): + self.draw_brush() + return + + if self.deselect_mode: + self.push_undo(self._snapshot()) + self.pipeline.deselect(stroke) + else: + entry = self.pipeline.add_entry('brush', stroke) + self.active_index = len(self.pipeline.entries) - 1 + self.push_undo({'type': 'add', 'entry': entry}) + self._retire_whole_image() + self.draw_brush() + self.refresh() + + @property + def deselect_mode(self): + """Whether a finished stroke subtracts rather than adds. + + Derived from the tool rather than stored, so there is no way for the + two to disagree. It used to be a checkable button inside the brush + controls, which made painting a mode within a mode: the same button + could add or take away depending on a toggle several rows below it. + ``Remove w/ brush`` is now a tool of its own, next to ``Remove point`` -- + the two things that take away, side by side. + + :rtype: bool + """ + return self.tool == 'erase' + + def _update_brush_row(self): + """Show the radius only while a tool that paints is active.""" + self.brush_form.setRowVisible(self.brush_radius, self.tool in BRUSH_TOOLS) + + # -- undo -------------------------------------------------------------- + + def _snapshot(self): + """A restorable copy of every entry's mutable state. + + Deselection touches an unpredictable set of entries at once -- erasing + part of some, emptying others -- so it is undone by restoring the whole + list rather than by trying to invert each edit. + + The ``erased`` array is held by reference, not copied: it is always + replaced wholesale rather than written into, so the array a snapshot + points at still holds what it held when the snapshot was taken. Copying + it would put a frame's worth of booleans per region into every one of + the fifty undo slots. The vertex list and the removed set *are* appended + to in place, so those are copied. + """ + return { + 'type': 'restore', + 'entries': list(self.pipeline.entries), + 'state': [(entry, + entry.erased, + list(entry.geometry) if isinstance(entry.geometry, list) else entry.geometry, + set(entry.removed)) + for entry in self.pipeline.entries], + } + + def push_undo(self, action): + """Record an undoable action. + + :param action: the action record + :type action: dict + """ + self.undo_stack.append(action) + del self.undo_stack[:-self.undo_limit] + + def undo(self): + """Reverse the last undoable action.""" + if not self.undo_stack: + return + action = self.undo_stack.pop() + self._REVERSALS[action['type']](self, action) + self.active_index = min(self.active_index or 0, len(self.pipeline.entries) - 1) + if self.active_index < 0: + self.active_index = None + self.refresh() + + def _undo_restore(self, action): + """Put the whole entry list, and every entry's state, back as it was.""" + self.pipeline.entries = list(action['entries']) + for entry, erased, geometry, removed in action.get('state', []): + entry.erased, entry.geometry, entry.removed = erased, geometry, removed + + #: How each recorded action is reversed. A table rather than a chain of + #: ``elif``s so adding an undoable action is one entry, not one more branch. + _REVERSALS = { + 'vertex_add': lambda self, a: a['entry'].geometry and a['entry'].geometry.pop(), + 'vertex_move': lambda self, a: a['entry'].geometry.__setitem__(a['index'], a['original']), + 'add': lambda self, a: self.pipeline.remove_entry(a['entry']), + 'delete': lambda self, a: self.pipeline.entries.insert(a['index'], a['entry']), + 'visible': lambda self, a: setattr(a['entry'], 'visible', a['value']), + 'restore': lambda self, a: self._undo_restore(a), + } + + # -- settings callbacks ------------------------------------------------ + + def _on_square_toggled(self, checked): + self.width_spin.setEnabled(not checked) + if checked: + self.width_spin.setValue(self.height_spin.value()) + self._on_subset_size_changed() + + def _on_subset_size_changed(self, *_): + if self.square_check.isChecked(): + self._syncing, previous = True, self._syncing + self.width_spin.setValue(self.height_spin.value()) + self._syncing = previous + self.pipeline.set_subset_size((self.height_spin.value(), self.width_spin.value())) + self.request_refresh() + + def _on_spacing_changed(self, value): + self.pipeline.spacing = value + self.request_refresh() + + def _on_evaluator_changed(self, *_): + self._rebuild_param_widgets() + self._redefine_score() + + def _on_show_score_toggled(self, checked): + for box in self.score_toggles: + if box.isChecked() != checked: + box.blockSignals(True) + box.setChecked(checked) + box.blockSignals(False) + self.score_overlay.setVisible(checked) + if checked: + self.draw_score() + + def _on_selector_changed(self, *_): + self.pipeline.selector = self.selector_combo.currentData() + self._update_selector_rows() + self.refresh() + + def _on_threshold_mode_changed(self, *_): + """Switch rule, and put the slider where that rule's default lives. + + Carrying the position across would be meaningless: the same position is + a percentile of 90 under one rule and a quality of 0.5 under another. + """ + mode = self.threshold_mode.currentData() + self.pipeline.selector_params['threshold_mode'] = mode + position = self._slider_position(mode, THRESHOLD_DEFAULTS[mode]) + if self.threshold_slider.value() == position: + self._on_threshold_changed(position) # setValue would not signal + else: + self.threshold_slider.setValue(position) + + def _on_threshold_changed(self, value): + threshold, text = self._slider_value(self.threshold_mode.currentData(), value) + self.threshold_label.setText(text) + self.pipeline.selector_params['threshold'] = threshold + self.request_refresh() + + def _on_decimation_changed(self, value): + self.pipeline.selector_params['decimation'] = value + self.request_refresh() + + def _on_separation_changed(self, value): + self.pipeline.selector_params['separation'] = value + self.request_refresh() + + def _on_pitch_changed(self, value): + self.pipeline.selector_params['pitch'] = value + self.request_refresh() + + def _on_max_points_changed(self, value): + self.pipeline.selector_params['max_points'] = value + self.request_refresh() + + def _update_spacing_row(self): + """Show ``Point spacing`` only while a row exists that it can move. + + It reaches exactly one place, :func:`~pyidi.selection.masks.literal_points`, + and only for a row that *lays points out*: a polygon, line or brush + stroke in the ``points`` role, whose coordinates are spaced along or + inside the shape. It cannot change the answer for the other two kinds of + row. A ``mask`` row has its points chosen by the selection instead, so + what sets their distance is the separation on the other tab; a + ``points``-*tool* row is the coordinates you clicked, which spacing has + no say over. + """ + spaced = any(entry.role == 'points' and entry.kind != 'points' + for entry in self.pipeline.entries) + self.spacing_form.setRowVisible(self.spacing_spin, spaced) + + def _update_selector_rows(self): + """Show only the rows the current selector actually reads.""" + peaks = self.selector_combo.currentData() == 'peaks' + self.select_form.setRowVisible(self.separation_spin, peaks) + self.select_form.setRowVisible(self.pitch_spin, not peaks) + + def _rebuild_param_widgets(self): + """Build the evaluator's parameter controls from its descriptors. + + Nothing here knows what a Shi-Tomasi or a gradient-direction parameter + is: the registry says a parameter is a float, an int or a direction, and + that is enough to make a widget for it. Adding an evaluator therefore + needs no change to this module. + """ + while self.param_layout.rowCount() > 1: # row 0 is the evaluator itself + self.param_layout.removeRow(1) + self.param_widgets = {} + self.direction_spins = [] + self.direction_button = None + # The line describes a parameter that no longer exists once the evaluator + # has changed, so it goes with the widget that owned it. + self.drawing_direction = False + self.direction_line.clear() + + spec = self.evaluators[self.evaluator_combo.currentData()] + for parameter in spec.parameters: + if parameter.kind == 'direction': + widget, getter = self._direction_widget(parameter) + else: + widget, getter = self._number_widget(parameter) + widget.setToolTip(parameter.description) + self.param_layout.addRow(parameter.name.replace('_', ' ').capitalize(), widget) + self.param_widgets[parameter.name] = getter + self.evaluator_combo.setToolTip(spec.description) + if self.direction_spins: + self._syncing = True # the line only, not a fresh evaluation + self.set_direction(*(spin.value() for spin in self.direction_spins)) + self._syncing = False + + def _number_widget(self, parameter): + spin = QtWidgets.QDoubleSpinBox() if parameter.kind == 'float' else QtWidgets.QSpinBox() + spin.setRange(parameter.minimum if parameter.minimum is not None else -1e9, + parameter.maximum if parameter.maximum is not None else 1e9) + spin.setValue(parameter.default) + spin.valueChanged.connect(lambda _: self._redefine_score()) + return spin, spin.value + + def _direction_widget(self, parameter): + """The two components, with the presets and the drag button beneath them. + + On one line the five controls squeeze the form's label column until + "Direction" elides to "Direct", so the buttons get a line of their own. + """ + widget = QtWidgets.QWidget() + column = QtWidgets.QVBoxLayout(widget) + column.setContentsMargins(0, 0, 0, 0) + column.setSpacing(2) + row = QtWidgets.QHBoxLayout() + row.setContentsMargins(0, 0, 0, 0) + column.addLayout(row) + spins = [] + for value in parameter.default: + spin = QtWidgets.QDoubleSpinBox() + spin.setRange(-1e6, 1e6) + # Three decimals rather than Qt's two: a normalised component wants + # better than the ~0.6 degrees two would round a dragged vector to. + spin.setDecimals(3) + spin.setSingleStep(0.05) + spin.setMaximumWidth(80) + spin.setValue(float(value)) + spin.valueChanged.connect(lambda _: self._redefine_score()) + row.addWidget(spin) + spins.append(spin) + row.addStretch(1) + self.direction_spins = spins + + buttons = QtWidgets.QHBoxLayout() + buttons.setContentsMargins(0, 0, 0, 0) + column.addLayout(buttons) + for label, vector in (('X', (0.0, 1.0)), ('Y', (1.0, 0.0))): + button = QtWidgets.QPushButton(label) + button.setMaximumWidth(52) + button.setToolTip(f'Along the image {label.lower()} axis.') + buttons.addWidget(button) + button.clicked.connect(lambda _, v=vector: self.set_direction(*v)) + self.direction_button = QtWidgets.QPushButton('Draw') + self.direction_button.setCheckable(True) + self.direction_button.setToolTip('Drag on the image to point the direction out.') + self.direction_button.setMaximumWidth(52) + self.direction_button.toggled.connect(self._set_direction_drawing) + buttons.addWidget(self.direction_button) + buttons.addStretch(1) + return widget, (lambda s=spins: (s[0].value(), s[1].value())) + + # -- gradient direction ------------------------------------------------ + + def _set_direction_drawing(self, enabled): + """Arm or disarm dragging the direction out on the image.""" + self.drawing_direction = enabled + if enabled: + self.status.showMessage('Drag on the image to set the gradient direction.') + else: + self.status.showMessage('') + + def show_direction(self, start, end): + """Draw the direction as a line between two ``(row, col)`` points.""" + self.direction_line.setData([start[1], end[1]], [start[0], end[0]]) + + def set_direction_from_drag(self, start, end): + """Adopt a dragged line as the gradient direction. + + The line stays where it was drawn rather than snapping to the middle of + the frame, because where you dragged it is usually the feature you were + pointing at. + """ + self.direction_button.setChecked(False) # one drag sets it once + self.set_direction(end[0] - start[0], end[1] - start[1], line=(start, end)) + + def set_direction(self, drow, dcol, line=None): + """Set the gradient direction to a ``(row, col)`` vector, normalised. + + :param drow: row component + :type drow: float + :param dcol: column component + :type dcol: float + :param line: the two ``(row, col)`` endpoints to draw, or ``None`` to + draw the vector through the middle of the frame + :type line: tuple or None + """ + norm = float(np.hypot(drow, dcol)) + if norm < 1e-9 or len(self.direction_spins) != 2: + return + unit = (drow / norm, dcol / norm) + + # Both components are written before the score is redefined, so a + # direction costs one evaluation rather than one per component. + previous, self._syncing = self._syncing, True + for spin, value in zip(self.direction_spins, unit): + spin.setValue(value) + self._syncing = previous + + if line is None: + rows, cols = self.pipeline.shape + span = min(rows, cols) / 4.0 + centre = ((rows - 1) / 2.0, (cols - 1) / 2.0) + line = ((centre[0] - unit[0] * span, centre[1] - unit[1] * span), + (centre[0] + unit[0] * span, centre[1] + unit[1] * span)) + self.show_direction(*line) + self._redefine_score() + + def _redefine_score(self): + """Re-declare the score from the current evaluator and parameters. + + Declaring costs nothing -- the store computes on request -- so this goes + through the same coalescing every other control uses. That matters more + here than anywhere else: this is the one control that can make a redraw + expensive, since a parameter it has not scored before is a whole-frame + evaluation, and a spin box dragged through sixty values would otherwise + queue sixty of them. + """ + if self._syncing: + return + params = {name: getter() for name, getter in self.param_widgets.items()} + self.pipeline.define_score('score', self.evaluator_combo.currentData(), **params) + self.request_refresh() + + # -- selections list --------------------------------------------------- + + def _on_row_changed(self, row): + if self._syncing: + return + self.active_index = row if row >= 0 else None + entry = self.active_entry() + if entry is not None and entry.kind != 'brush': + self.select_tool(entry.kind) + self.draw_geometry() + self.draw_highlight() + self._update_role_button() + + def _on_item_changed(self, item): + if self._syncing: + return + row = self.entry_list.row(item) + if 0 <= row < len(self.pipeline.entries): + self.pipeline.entries[row].visible = item.checkState() == QtCore.Qt.CheckState.Checked + self.refresh() + + def _update_role_button(self): + entry = self.active_entry() + self.role_button.setEnabled(entry is not None) + if entry is not None: + self.role_button.setText('Use as mask' if entry.role == 'points' else 'Use as points') + + def _refresh_list(self, credited): + self._syncing = True + self.entry_list.clear() + for entry, points in zip(self.pipeline.entries, credited): + item = QtWidgets.QListWidgetItem(f'{entry.label} — {entry.role} — {len(points)} pts') + item.setFlags(item.flags() | QtCore.Qt.ItemFlag.ItemIsUserCheckable) + item.setCheckState(QtCore.Qt.CheckState.Checked if entry.visible + else QtCore.Qt.CheckState.Unchecked) + self.entry_list.addItem(item) + if self.active_index is not None and 0 <= self.active_index < self.entry_list.count(): + self.entry_list.setCurrentRow(self.active_index) + self._syncing = False + self._update_role_button() + + # -- drawing ----------------------------------------------------------- + + def request_refresh(self): + """Redraw, collapsing the flood of requests a dragged control produces. + + A slider emits a change per pixel of travel, and a redraw is not always + cheap enough to keep up. Two behaviours, chosen by measurement rather + than by a fixed delay: + + While a redraw costs less than a frame, it happens immediately, so the + display tracks the control exactly -- which is the whole appeal of a + live slider and is worth nothing to defer. + + Once it costs more, requests are *coalesced*: the first one schedules a + redraw for the moment the event queue next drains, and every request + arriving before then is absorbed into it. Nothing queues up, so a fast + drag does not repaint every position on the way -- it repaints as often + as it can and always with the value the control is on *now*, which is + where it stops. A slow drag drains the queue between steps and still + redraws at every one. + """ + if self._last_refresh_ms <= REDRAW_BUDGET_MS: + self.refresh() + elif not self._refresh_timer.isActive(): + self._refresh_timer.start(0) + + def flush_refresh(self): + """Run a coalesced redraw now, if one is pending.""" + if self._refresh_timer.isActive(): + self.refresh() + + def refresh(self): + """Re-run the pipeline once and redraw everything that depends on it. + + Once, not three times: the total, the per-row counts and the highlight + all come from the same pass, and at twenty thousand points each pass is + tens of milliseconds you would otherwise pay three times over on every + step of a slider drag. + """ + self._refresh_timer.stop() + started = time.perf_counter() + self._points, self._credited = self.pipeline.points_and_credits() + self._refresh_list(self._credited) + self._update_spacing_row() + self.count_label.setText(f'{len(self._points)} points') + self.draw_points() + self.draw_candidates() + self.draw_geometry() + self.draw_highlight() + self._refresh_select_note() + # Part of the redraw, and timed with it: the overlay is drawn from the + # score, so a change of evaluator has to reach it too -- and its cost is + # exactly the sort the coalescing above exists to notice. + if self.show_score.isChecked(): + self.draw_score() + self._last_refresh_ms = (time.perf_counter() - started) * 1000.0 + + def draw_points(self): + """Draw the selected points and, optionally, their subset rectangles.""" + points = getattr(self, '_points', None) + if points is None or not len(points): + self.point_scatter.clear() + self.doomed_scatter.clear() + self.clear_subset_rectangles() + return + # +0.5 puts the marker at the pixel centre rather than its top-left corner. + self.point_scatter.setData(pos=points[:, ::-1] + 0.5) + self.draw_doomed() + if self.show_subsets.isChecked(): + height, width = self.pipeline.subset_size + self.draw_subset_rectangles(points, height // 2, width // 2) + else: + self.clear_subset_rectangles() + + def draw_doomed(self): + """Cross out the points the deselect stroke has covered so far. + + Drawn *over* the red points rather than swapped for them, so a stroke in + progress costs only the handful of points it has reached: replacing the + cloud would mean handing every one of tens of thousands of positions back + to the scatter item on every mouse move, which is what made a long stroke + drag behind the cursor. + """ + points = getattr(self, '_points', None) + if points is None or not len(points) or self._paint is None or not self.deselect_mode: + self.doomed_scatter.clear() + return + doomed = self._paint[points[:, 0], points[:, 1]] + if not doomed.any(): + self.doomed_scatter.clear() + return + self.doomed_scatter.setData(pos=points[doomed][:, ::-1] + 0.5) + + def draw_candidates(self): + """Show what the mask is leaving out, on the mask step. + + Three tiers while masking, because "no point here" is otherwise + ambiguous: dim grey for a feature the mask excludes, the ordinary red + for a point that is being taken, and the magenta ring + (:meth:`draw_highlight`) for the ones the selected row accounts for. + + The candidates are the whole-frame selection, so they do not move while + a mask is edited -- painting a region turns points from grey to red + where it lands rather than re-selecting underneath you. The consequence + is that a grey point near the edge of a mask need not coincide exactly + with a red one, since a selection inside a region starts its separation + afresh. + """ + if self.step != STEP_MASK: + self.candidate_scatter.clear() + return + candidates = self.pipeline.candidate_points() + outside = (~self.pipeline.mask[candidates[:, 0], candidates[:, 1]] + if len(candidates) else np.zeros(0, dtype=bool)) + if not outside.any(): + self.candidate_scatter.clear() + return + self.candidate_scatter.setData(pos=candidates[outside][:, ::-1] + 0.5) + + def clear_subset_rectangles(self): + """Remove both halves of the subset-rectangle display.""" + self.roi_overlay.clear() + self.roi_outline.setPath(QtGui.QPainterPath()) + + def draw_subset_rectangles(self, points, half_h, half_w): + """Draw each subset as a translucent fill plus a hairline border. + + The two halves are drawn by different means because each is cheap in a + different way. The fill goes into one RGBA image, which costs a single + upload however many subsets there are. The borders go into one + ``QPainterPath`` stroked with a cosmetic pen, whose width is in screen + pixels -- that is what keeps them a hairline at any zoom, where a raster + border is pinned to one image pixel and becomes a band when zoomed in. + + Both are built with whole-array numpy rather than a loop over the points, + which is what keeps the redraw quick for tens of thousands of subsets. + + :param points: subset centres, as an ``(n, 2)`` array of ``(row, col)`` + :type points: numpy.ndarray + :param half_h: half the subset height, in pixels + :type half_h: int + :param half_w: half the subset width, in pixels + :type half_w: int + """ + n_rows, n_cols = self.pipeline.shape + span_r, span_c = 2 * half_h + 1, 2 * half_w + 1 + + r0 = points[:, 0].astype(int) - half_h + c0 = points[:, 1].astype(int) - half_w + inside = (r0 >= 0) & (c0 >= 0) & (r0 + span_r <= n_rows) & (c0 + span_c <= n_cols) + r0, c0 = r0[inside], c0[inside] + if not len(r0): + self.clear_subset_rectangles() + return + + # Mark every covered pixel at once by broadcasting the per-subset index + # ranges against each other, giving an (n, span_r, span_c) fancy index. + covered = np.zeros((n_rows, n_cols), dtype=bool) + covered[(r0[:, None] + np.arange(span_r))[:, :, None], + (c0[:, None] + np.arange(span_c))[:, None, :]] = True + overlay = np.zeros((n_rows, n_cols, 4), dtype=np.uint8) + overlay[..., 1] = covered * np.uint8(180) + overlay[..., 3] = covered * np.uint8(40) + self.roi_overlay.setImage(overlay, autoLevels=False) + + # Five corners per rectangle (the first repeated to close it) separated by + # a nan, which is how arrayToQPath is told to start a new sub-path. + top, left = r0.astype(float), c0.astype(float) + bottom, right = top + span_r, left + span_c + xs = np.empty((len(left), 6)) + ys = np.empty((len(top), 6)) + xs[:, 0] = xs[:, 3] = xs[:, 4] = left + xs[:, 1] = xs[:, 2] = right + ys[:, 0] = ys[:, 1] = ys[:, 4] = top + ys[:, 2] = ys[:, 3] = bottom + xs[:, 5] = ys[:, 5] = np.nan + self.roi_outline.setPath(pg.arrayToQPath(xs.ravel(), ys.ravel(), connect='finite')) + + def draw_geometry(self): + """Outline the active vertex-based entry and show its vertices.""" + entry = self.active_entry() + if self.step != STEP_MASK or entry is None or entry.kind not in VERTEX_KINDS or not entry.geometry: + self.geometry_line.clear() + self.geometry_vertices.clear() + return + vertices = np.asarray(entry.geometry, dtype=float) + closed = np.vstack([vertices, vertices[:1]]) if entry.kind == 'polygon' else vertices + self.geometry_line.setData(closed[:, 1], closed[:, 0]) + self.geometry_vertices.setData(pos=vertices[:, ::-1]) + + def draw_highlight(self): + """Ring the points belonging to the active row.""" + if self.step != STEP_MASK or self.active_index is None: + self.highlight_scatter.clear() + return + credited = getattr(self, '_credited', None) or self.pipeline.points_by_entry() + points = credited[self.active_index] if self.active_index < len(credited) else [] + if not len(points): + self.highlight_scatter.clear() + return + array = np.asarray(points, dtype=float) + self.highlight_scatter.setData(pos=array[:, ::-1] + 0.5) + + def draw_brush(self): + """Show the stroke currently being painted. + + The path is filled with the winding rule, so the dabs read as one + translucent stroke rather than as a chain of discs darkening where they + overlap. + """ + if self._paint is None: + self.brush_overlay.setPath(QtGui.QPainterPath()) + return + colour = (255, 0, 0, 80) if self.deselect_mode else (0, 200, 255, 80) + self.brush_overlay.setBrush(pg.mkBrush(*colour)) + self.brush_overlay.setPath(self._stroke_path) + + def draw_score(self): + """Show the current score image as a heatmap, transparent where invalid.""" + score = self.pipeline.store.get(self.pipeline.ensure_default_score()) + finite = np.isfinite(score) + rgba = np.zeros((*score.shape, 4), dtype=np.uint8) + if finite.any(): + values = score[finite] + low, high = float(values.min()), float(values.max()) + normalised = np.zeros(score.shape, dtype=float) + if high > low: + normalised[finite] = (values - low) / (high - low) + colours = pg.colormap.get('viridis').map(normalised, mode='byte') + rgba[finite] = colours[finite] + # The NaN border keeps alpha 0, so it reads as "not scored" rather + # than as a region that scored badly. + rgba[..., 3] = np.where(finite, 150, 0) + self.score_overlay.setImage(rgba, autoLevels=False) + + def _refresh_select_note(self): + """Say what is limiting the selection, when something is. + + The point cap in particular has no other symptom: it just stops adding + points, and the result reads as though the threshold or the separation + did it. + """ + if not self.pipeline.mask.any(): + self.select_note.setText('No region acts as a mask, so nothing is selected. ' + 'Draw one in the Mask tab, or set a row to "mask".') + elif len(getattr(self, '_points', ())) >= self.max_points_spin.value(): + self.select_note.setText(f'Stopped at the cap of {self.max_points_spin.value()} points. ' + 'Raise the separation or the threshold to see the ' + 'whole selection.') + else: + self.select_note.setText('') + + # -- results ----------------------------------------------------------- + + def get_points(self): + """Run the pipeline and return the points. + + :return: ``(n_points, 2)`` integer array in ``(row, col)`` order, ready + for ``IDIMethod.set_points()`` + :rtype: numpy.ndarray + """ + return self.pipeline.get_points() + + @property + def points(self): + """The selected points, as :meth:`get_points` returns them. + + :rtype: numpy.ndarray + """ + return self.get_points() diff --git a/pyidi/GUIs/gui.py b/pyidi/GUIs/gui.py index 6f956ac..6da5901 100644 --- a/pyidi/GUIs/gui.py +++ b/pyidi/GUIs/gui.py @@ -7,7 +7,7 @@ warnings.simplefilter("default") from .. import tools -from . import selection +from ..selection_geometry import get_roi_grid from ..methods import SimplifiedOpticalFlow from ..methods import LucasKanade @@ -248,25 +248,43 @@ def displacement_widget(): self.ConfigWidget = viewer.window.add_dock_widget(lk_config_widget, name='Configure - LK', add_vertical_stretch=add_vertical_stretch) - def base_set_points_widget(self,viewer, subset_size, noverlap, show_subset_box): + def _gather_points(self, viewer, subset_size, noverlap): + """Collect the points from the napari layers. + + Individual picks are taken from the 'Points' layer; if an area has been + drawn in 'Area Selection', a regular grid is generated inside it instead. + """ #individual points selection if viewer.layers['Area Selection'].data == []: - self.method.points = np.round(viewer.layers['Points'].data).astype(int) - + return np.round(viewer.layers['Points'].data).astype(int) + #area selection for grid + border = viewer.layers['Area Selection'].data[0].T # shape data + + if viewer.layers['Area Deselection'].data == []: + deselect_border = [[],[]] else: - border = viewer.layers['Area Selection'].data[0].T # shape data - - if viewer.layers['Area Deselection'].data == []: - deselect_border = [[],[]] - else: - deselect_border = viewer.layers['Area Deselection'].data[0].T # deselection shape data + deselect_border = viewer.layers['Area Deselection'].data[0].T # deselection shape data + + return get_roi_grid( + polygon_points=border, + roi_size=subset_size, + noverlap=noverlap, + deselect_polygon=deselect_border) # get grid points - self.method.points = selection.get_roi_grid( - polygon_points=border, - roi_size=subset_size, - noverlap=noverlap, - deselect_polygon=deselect_border) # get grid points + def base_set_points_widget(self,viewer, subset_size, noverlap, show_subset_box): + points = self._gather_points(viewer, subset_size, noverlap) + + # Route through set_points so the GUI gets the same validation (bounds, + # dtype, shape) as the programmatic API, instead of writing .points directly. + if len(points): + try: + self.method.set_points(points) + except ValueError as e: + warnings.warn(f'Points were not set: {e}') + return + else: + self.method.points = points if 'Subsets' in viewer.layers: viewer.layers.pop('Subsets') # refresh ROI layer @@ -280,15 +298,23 @@ def base_set_points_widget(self,viewer, subset_size, noverlap, show_subset_box): if len(self.method.points) == 0: del self.method.points - - try: - viewer.window.remove_dock_widget(self.ConfigWidget) - except: - pass - + + self._remove_dock_widget(viewer, 'ConfigWidget') + self._remove_dock_widget(viewer, 'DisplacementWidget') + + + def _remove_dock_widget(self, viewer, attr_name): + """Remove a dock widget by attribute name, if it is currently docked. + + A widget that was never created, or has already been removed, is not an + error - the downstream widgets are rebuilt whenever the points change. + """ + widget = getattr(self, attr_name, None) + if widget is None: + return try: - viewer.window.remove_dock_widget(self.DisplacementWidget) - except: + viewer.window.remove_dock_widget(widget) + except (LookupError, ValueError, RuntimeError): pass diff --git a/pyidi/GUIs/result_viewer.py b/pyidi/GUIs/result_viewer.py index 81674dc..68012fd 100644 --- a/pyidi/GUIs/result_viewer.py +++ b/pyidi/GUIs/result_viewer.py @@ -61,7 +61,8 @@ def __init__(self, video, displacements=None, points=None, fps=30, magnification video : np.ndarray or VideoReader Array of shape (n_frames, height, width) containing the video frames. displacements : np.ndarray - Array of shape (n_frames, n_points, 2) for time-series displacements OR + Array of shape (n_points, n_frames, 2) for time-series displacements + (as returned by ``get_displacements``) OR Array of shape (n_points, 2) for mode shapes. points : np.ndarray Array of shape (n_points, 2) containing the grid points. @@ -97,7 +98,7 @@ def __init__(self, video, displacements=None, points=None, fps=30, magnification self.displacements = displacements[:, ::-1] # Flip x,y coordinates self.time_per_period = 1.0 # Seconds else: - # Time-series displacements: shape (n_frames, n_points, 2) + # Time-series displacements: shape (n_points, n_frames, 2) self.is_mode_shape = False self.displacements = displacements[:, :, ::-1] # Flip x,y coordinates diff --git a/pyidi/GUIs/selection.py b/pyidi/GUIs/selection.py deleted file mode 100644 index 2bfebba..0000000 --- a/pyidi/GUIs/selection.py +++ /dev/null @@ -1,391 +0,0 @@ -import numpy as np -import matplotlib.pyplot as plt -import matplotlib.patches as patches - -import tkinter as tk -from tkinter import ttk -from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk -from matplotlib.figure import Figure -from matplotlib.path import Path - -SELECTION_MODES = { - 'ROI grid': 0, - 'Deselect ROI polygon': 1, - 'Only polygon': 2, - 'Manual ROI select': 3 - } - -MODE_DESCRIPTION = { - 0: 'Use SHIFT + LEFT CLICK\nto select a polygon where\na regular grid of ROIs will\nbe generated.', - 1: 'Use SHIFT + LEFT CLICK\nto select a polygon where\nthe ROIs will be removed.', - 2: 'Use SHIFT + LEFT CLICK\nto select a polygon.', - 3: 'Use SHIFT + LEFT CLICK\nto manually position ROIs.' -} - -class SubsetSelection: - def __init__(self, video=None, roi_size=(11, 11), noverlap=0, polygon=None): - self.verbose = 0 - self.shift_is_held = False - - self.roi_size = roi_size - self.noverlap = int(noverlap) - self.cent_dist_0 = self.roi_size[0] - self.noverlap - self.cent_dist_1 = self.roi_size[1] - self.noverlap - - if polygon is None: - self.polygon = [[], []] - else: - self.polygon = polygon - self.deselect_polygon = [[], []] - self.points = [[], []] - - root = tk.Tk() - root.title('Selection') - - self.show_box = tk.IntVar(value=1) - - self.screen_width = root.winfo_screenwidth() - self.screen_height = root.winfo_screenheight() - root.geometry(f'{int(0.9*self.screen_width)}x{int(0.9*self.screen_height)}') - - # Create left frame for options - left_frame = tk.Frame(root, width=int(0.2 * self.screen_width)) - left_frame.pack(side='left', fill='y', padx=5, pady=5) - left_frame.grid_propagate(False) - - # Add options to the left frame - self.options = SelectOptions(left_frame, self) - - # Create main frame for the canvas and controls - main_frame = tk.Frame(root) - main_frame.pack(side='right', fill='both', expand=1) - - button1 = ttk.Button(main_frame, text='Confirm selection', command=lambda: self.on_closing(root)) - button1.pack(side='top', pady=5) - - self.fig = Figure(figsize=(10, 7)) - self.ax = self.fig.add_subplot(111) - self.ax.grid(False) - self.ax.imshow(video.get_frame(0), cmap='gray') - - # Initiate polygon - self.line, = self.ax.plot(self.polygon[1], self.polygon[0], 'C1.-') - self.line_deselect, = self.ax.plot(self.deselect_polygon[1], self.deselect_polygon[0], 'k.-') - self.line2, = self.ax.plot([], [], 'C0x') - - plt.show(block=False) - - # Embed figure in tkinter window - canvas = FigureCanvasTkAgg(self.fig, main_frame) - toolbar = NavigationToolbar2Tk(canvas, main_frame) - toolbar.pack(side='top', fill='x') # First pack the toolbar (it should be on top) - canvas.get_tk_widget().pack(side='top', fill='both', expand=1, padx=5, pady=5) # Then pack the canvas - - if self.verbose: - print('SHIFT + LEFT mouse button to pick a pole.\nRIGHT mouse button to erase the last pick.') - - # Connecting functions to event manager - self.fig.canvas.mpl_connect('key_press_event', self.on_key_press) - self.fig.canvas.mpl_connect('key_release_event', self.on_key_release) - - self.update_variables() - root.protocol("WM_DELETE_WINDOW", lambda: self.on_closing(root)) - tk.mainloop() - - - def _mode_selection_polygon(self, get_rois=True): - """Select polygon to compute the points based on ROI size and - ROI overlap.""" - def onclick(event): - if event.button == 1 and self.shift_is_held: - if event.xdata is not None and event.ydata is not None: - if self.polygon[0]: - del self.polygon[1][-1] - del self.polygon[0][-1] - - self.polygon[1].append(int(np.round(event.xdata))) - self.polygon[0].append(int(np.round(event.ydata))) - - if self.polygon[0]: - self.polygon[1].append(self.polygon[1][0]) - self.polygon[0].append(self.polygon[0][0]) - - if self.verbose: - print(f'y: {np.round(event.ydata):5.0f}, x: {np.round(event.xdata):5.0f}') - - elif event.button == 3 and self.shift_is_held: - if self.verbose: - print('Deleted the last point...') - del self.polygon[1][-2] - del self.polygon[0][-2] - - self.line.set_xdata(self.polygon[1]) - self.line.set_ydata(self.polygon[0]) - self.fig.canvas.draw() - - if get_rois: - self.plot_selection() - - self.cid = self.fig.canvas.mpl_connect('button_press_event', onclick) - - def _mode_selection_deselect_polygon(self): - """Select polygon to compute the points based on ROI size and - ROI overlap.""" - def onclick(event): - if event.button == 1 and self.shift_is_held: - if event.xdata is not None and event.ydata is not None: - if self.deselect_polygon[0]: - del self.deselect_polygon[1][-1] - del self.deselect_polygon[0][-1] - - self.deselect_polygon[1].append(int(np.round(event.xdata))) - self.deselect_polygon[0].append(int(np.round(event.ydata))) - - if self.deselect_polygon[0]: - self.deselect_polygon[1].append(self.deselect_polygon[1][0]) - self.deselect_polygon[0].append(self.deselect_polygon[0][0]) - - if self.verbose: - print(f'y: {np.round(event.ydata):5.0f}, x: {np.round(event.xdata):5.0f}') - - elif event.button == 3 and self.shift_is_held: - if self.verbose: - print('Deleted the last point...') - del self.deselect_polygon[1][-2] - del self.deselect_polygon[0][-2] - - self.line_deselect.set_xdata(self.deselect_polygon[1]) - self.line_deselect.set_ydata(self.deselect_polygon[0]) - self.fig.canvas.draw() - - self.plot_selection() - - self.cid = self.fig.canvas.mpl_connect('button_press_event', onclick) - - def _mode_selection_manual_roi(self): - """Select polygon to compute the points based on ROI size and - ROI overlap.""" - def onclick(event): - if event.button == 1 and self.shift_is_held: - if event.xdata is not None and event.ydata is not None: - self.points[0].append(int(np.round(event.ydata))) - self.points[1].append(int(np.round(event.xdata))) - - elif event.button == 3 and self.shift_is_held: - del self.points[1][-1] - del self.points[0][-1] - - self.fig.canvas.draw() - self.plot_selection() - - self.cid = self.fig.canvas.mpl_connect('button_press_event', onclick) - - def on_key_press(self, event): - """Function triggered on key press (shift).""" - if event.key == 'shift': - self.shift_is_held = True - - def on_key_release(self, event): - """Function triggered on key release (shift).""" - if event.key == 'shift': - self.shift_is_held = False - - def update_variables(self): - self.line2.set_xdata([]) - self.line2.set_ydata([]) - self.fig.canvas.draw() - - self.mode = self.options.combobox.get() - if SELECTION_MODES[self.mode] == 0: # ROI grid - self.clear_selection() - self._disconnect_mpl_onclick() - - self._mode_selection_polygon() - - self.roi_size = [int(self.options.roi_entry_y.get()), int(self.options.roi_entry_x.get())] - self.noverlap = int(self.options.noverlap_entry.get()) - - self.cent_dist_0 = self.roi_size[0] - self.noverlap - self.cent_dist_1 = self.roi_size[1] - self.noverlap - - self.plot_selection() - - elif SELECTION_MODES[self.mode] == 1: # Deselect ROI polygon - if len(self.points[0]) == 0: - tk.messagebox.showwarning("Warning", "No points have been selected yet.") - else: - self._disconnect_mpl_onclick() - - self._mode_selection_deselect_polygon() - self.plot_selection() - - elif SELECTION_MODES[self.mode] == 2: # Only polygon - self.clear_selection() - self._disconnect_mpl_onclick() - self._mode_selection_polygon(get_rois=False) - - elif SELECTION_MODES[self.mode] == 3: # Manual ROI select - self.clear_selection() - self._disconnect_mpl_onclick() - self._mode_selection_manual_roi() - self.roi_size = [int(self.options.roi_entry_y.get()), int(self.options.roi_entry_x.get())] - self.plot_selection() - - else: - raise Exception('Non existing mode...') - - self.options.description.configure(text=MODE_DESCRIPTION[SELECTION_MODES[self.mode]]) - - def _disconnect_mpl_onclick(self): - try: - self.fig.canvas.mpl_disconnect(self.cid) - except: - pass - - def plot_selection(self): - if len(self.polygon[0]) > 2 and len(self.polygon[1]) > 2: - - self.points = (get_roi_grid(self.polygon, self.roi_size, self.noverlap, self.deselect_polygon).T) - - if len(self.points[0]) >= 1 and len(self.points[1]) >= 1: - self.line2.set_xdata(np.array(self.points).T[:, 1]) - self.line2.set_ydata(np.array(self.points).T[:, 0]) - - self.options.nr_points_label.configure(text=f'{len(np.array(self.points).T)}') - - # if SELECTION_MODES[self.mode] == 3: - if self.show_box.get(): - [p.remove() for p in reversed(self.ax.patches)] - self.rectangles = [] - for i, (p0, p1) in enumerate(zip(self.points[0], self.points[1])): - self.rectangles.append(patches.Rectangle((p1 - self.roi_size[1]/2, p0 - self.roi_size[0]/2), - self.roi_size[1], self.roi_size[0], fill=False, color='C2', linewidth=2)) - self.ax.add_patch(self.rectangles[-1]) - else: - [p.remove() for p in reversed(self.ax.patches)] - - self.fig.canvas.draw() - - def clear_selection(self): - self.polygon = [[], []] - self.deselect_polygon = [[], []] - self.points = [[], []] - self.options.nr_points_label.configure(text='0') - self.clear_plot() - - def clear_plot(self): - self.line.set_xdata([]) - self.line.set_ydata([]) - self.line_deselect.set_xdata([]) - self.line_deselect.set_ydata([]) - self.line2.set_xdata([]) - self.line2.set_ydata([]) - [p.remove() for p in reversed(self.ax.patches)] - self.fig.canvas.draw() - - def on_closing(self, root): - self.points = np.array(self.points) - if self.points.shape[0] == 2: - self.points = self.points.T - root.destroy() - - def __repr__(self): - return f"SubsetSelection(roi_size={self.roi_size}, noverlap={self.noverlap}, n_points={len(self.points)})" - -class SelectOptions: - def __init__(self, parent_frame, parent: SubsetSelection): - self.running_options = True - self.parent = parent - - roi_x = tk.StringVar(parent_frame, value=str(parent.roi_size[1])) - roi_y = tk.StringVar(parent_frame, value=str(parent.roi_size[0])) - noverlap = tk.StringVar(parent_frame, value=str(parent.noverlap)) - - row = 0 - ttk.Label(parent_frame, text='Selection mode:').grid(row=row, column=0, padx=5, pady=5, sticky='W') - self.combobox = ttk.Combobox(parent_frame, values=list(SELECTION_MODES.keys())) - self.combobox.current(0) - self.combobox.grid(row=row, column=1, sticky='wens', padx=5, pady=5) - self.combobox.bind("<>", self.apply) # Auto apply when changing mode - - row = 1 - ttk.Label(parent_frame, text='Horizontal ROI size').grid(row=row, column=0, sticky='E') - self.roi_entry_x = tk.Entry(parent_frame, textvariable=roi_x) - self.roi_entry_x.grid(row=row, column=1, padx=5, pady=5, sticky='W') - - row = 2 - ttk.Label(parent_frame, text='Vertical ROI size').grid(row=row, column=0, sticky='E') - self.roi_entry_y = tk.Entry(parent_frame, textvariable=roi_y) - self.roi_entry_y.grid(row=row, column=1, padx=5, pady=5, sticky='W') - - row = 3 - ttk.Label(parent_frame, text='Overlap pixels').grid(row=row, column=0, sticky='E') - self.noverlap_entry = tk.Entry(parent_frame, textvariable=noverlap) - self.noverlap_entry.grid(row=row, column=1, padx=5, pady=5, sticky='W') - - row = 4 - ttk.Label(parent_frame, text='Show ROI box').grid(row=row, column=0, sticky='E') - self.show_box_checkbox = tk.Checkbutton(parent_frame, text='', variable=self.parent.show_box) - self.show_box_checkbox.grid(row=row, column=1, padx=5, pady=5, sticky='W') - - row = 5 - apply_button = ttk.Button(parent_frame, text='Apply', command=parent.update_variables) - apply_button.grid(row=row, column=0, sticky='we', padx=5, pady=5) - - clear_button = ttk.Button(parent_frame, text='Clear', command=parent.clear_selection) - clear_button.grid(row=row, column=1, sticky='w', padx=5, pady=5) - - row = 6 - ttk.Label(parent_frame, text='Number of selected points:').grid(row=row, column=0, sticky='E') - self.nr_points_label = ttk.Label(parent_frame, text='0') - self.nr_points_label.grid(row=row, column=1, sticky='W') - - row = 7 - ttk.Label(parent_frame, text=' ').grid(row=row, column=0) - - row = 8 - self.description = ttk.Label(parent_frame, text='Description') - self.description.grid(row=row, column=0, columnspan=2, pady=5) - - def apply(self, *args): - self.parent.update_variables() - - def on_closing(self): - self.running_options = False - self.parent.update_variables() - self.root1.destroy() - - -def get_roi_grid(polygon_points, roi_size, noverlap, deselect_polygon): - if len(roi_size) != 2: - raise Exception(f'roi_size must be a tuple of length 2') - - cent_dist_0 = roi_size[0] - noverlap - cent_dist_1 = roi_size[1] - noverlap - - points = np.array(polygon_points) - if points.shape[0] == 2: - points = points.T - - low_0 = np.min(points[:, 0]) - high_0 = np.max(points[:, 0]) - low_1 = np.min(points[:, 1]) - high_1 = np.max(points[:, 1]) - - candidates_0 = np.arange(low_0, high_0, cent_dist_0) - candidates_1 = np.arange(low_1, high_1, cent_dist_1) - candidates = np.concatenate([_.flatten()[:, None] for _ in np.meshgrid(candidates_0, candidates_1)], axis=1) - - path = Path(points) - mask = path.contains_points(candidates) - - if len(deselect_polygon[0]) and len(deselect_polygon[1]): - path_deselect = Path(np.array(deselect_polygon).T) - mask_deselect = path_deselect.contains_points(candidates) - mask = np.logical_and(mask, np.logical_not(mask_deselect)) - - return np.round(candidates[mask]).astype(int) - - - - diff --git a/pyidi/GUIs/subset_selection.py b/pyidi/GUIs/subset_selection.py index 236fc3f..202cd01 100644 --- a/pyidi/GUIs/subset_selection.py +++ b/pyidi/GUIs/subset_selection.py @@ -1,19 +1,127 @@ import sys +import warnings + import numpy as np -from PyQt6 import QtWidgets, QtCore +from PyQt6 import QtWidgets, QtCore, QtGui from pyqtgraph import GraphicsLayoutWidget, ImageItem, ScatterPlotItem import pyqtgraph as pg -from matplotlib.path import Path # import pyidi # Assuming pyidi is a custom module for video handling +from ..selection_geometry import points_along_polygon, rois_inside_polygon, rois_inside_mask, _as_size_pair + +#: Grab radius (in screen pixels) within which a click/drag is considered to hit an +#: existing grid/polyline vertex. Kept constant in screen space so hit-testing feels +#: the same regardless of the current zoom level. +VERTEX_GRAB_RADIUS_PX = 10 + +#: Row-label prefix for each non-manual selection-entry kind, used by +#: ``SelectionGUIOld.add_selection`` to generate monotonic labels ("Grid 3", ...). +PRETTY = {'line': 'Line', 'grid': 'Grid', 'brush': 'Brush'} + + class BrushViewBox(pg.ViewBox): def __init__(self, parent_gui, *args, **kwargs): super().__init__(*args, **kwargs) self.setMouseMode(self.PanMode) self.parent_gui = parent_gui + self._dragging_vertex = None # set while a vertex drag is in progress + + def _vertex_drag_container(self): + """Return the (entries, kind) vertex-list container for the active selection method. + + Only the "Grid" and "Along the line" methods support vertex dragging. The + returned ``entries`` list is filtered from ``gui.selections`` but holds the + same (mutable) entry dicts, so edits through it are visible in + ``gui.selections`` too. + + :return: the list of live selection entries of the active kind and a kind + tag ("grid" or "line"), or (None, None) if the current mode/method does + not support vertex dragging. + :rtype: tuple + """ + gui = self.parent_gui + if gui.mode != "selection": + return None, None + kind = gui.current_kind() + if kind not in ('grid', 'line'): + return None, None + entries = [e for e in gui.selections if e['kind'] == kind and e['visible']] + return entries, kind + + def _start_vertex_drag(self, ev): + """Hit-test the drag start position against existing vertices; begin a drag if one is hit. + + :param ev: the pyqtgraph mouse-drag event, with ``ev.isStart()`` True + :return: True if a vertex was grabbed and the event was accepted + :rtype: bool + """ + entries, kind = self._vertex_drag_container() + if entries is None: + return False + pos = ev.buttonDownScenePos() + if not self.sceneBoundingRect().contains(pos): + return False + point = self.parent_gui.view.mapSceneToView(pos) + entry_idx, vertex_idx = self.parent_gui.find_vertex_to_drag(entries, point.x(), point.y()) + if entry_idx is None: + return False + entry = entries[entry_idx] + self._dragging_vertex = { + 'kind': kind, + 'entry': entry, + 'vertex_index': vertex_idx, + 'original_position': entry['geometry'][vertex_idx], + } + ev.accept() + return True + + def _continue_vertex_drag(self, ev): + """Move the grabbed vertex for a mid-drag or finishing event; commit undo/recompute on finish. + + :param ev: the pyqtgraph mouse-drag event, with ``ev.isStart()`` False + :return: True if a vertex drag was in progress and the event was consumed + :rtype: bool + """ + drag = self._dragging_vertex + if drag is None: + return False + + pos = ev.scenePos() + if self.sceneBoundingRect().contains(pos): + point = self.parent_gui.view.mapSceneToView(pos) + drag['entry']['geometry'][drag['vertex_index']] = (point.x(), point.y()) + self.parent_gui.update_geometry_display() + + if ev.isFinish(): + self.parent_gui.push_undo({ + 'type': 'move', + 'kind': drag['kind'], + 'entry': drag['entry'], + 'vertex_index': drag['vertex_index'], + 'original_position': drag['original_position'], + }) + self.parent_gui.recompute_roi_points() + self._dragging_vertex = None + + ev.accept() + return True + + def _handle_vertex_drag(self, ev): + """Handle a plain left-drag that starts on an existing grid/polyline vertex. + + A drag starting near a vertex moves that vertex; a drag starting elsewhere is left + untouched so the caller falls back to panning the view. + + :param ev: the pyqtgraph mouse-drag event + :return: True if the event was consumed as a vertex drag + :rtype: bool + """ + if ev.isStart(): + return self._start_vertex_drag(ev) + return self._continue_vertex_drag(ev) def mouseClickEvent(self, ev): - if self.parent_gui.mode == "selection" and self.parent_gui.method_buttons["Brush"].isChecked(): + if self.parent_gui.mode == "selection" and self.parent_gui.current_kind() == 'brush': if self.parent_gui.ctrl_held: ev.accept() self.parent_gui.handle_brush_start(ev) @@ -22,67 +130,113 @@ def mouseClickEvent(self, ev): else: super().mouseClickEvent(ev) + def _handle_gradient_direction_drag(self, ev): + """Handle direction-line drag when Filter mode is setting the gradient direction. + + :param ev: the pyqtgraph mouse-drag event + :return: True if the event was consumed + :rtype: bool + """ + if not (self.parent_gui.mode == "filter" and self.parent_gui.setting_direction): + return False + + pos = ev.scenePos() + if not self.sceneBoundingRect().contains(pos): + return False + point = self.mapSceneToView(pos) + + if ev.isStart(): + self.parent_gui.gradient_direction_points = [(point.x(), point.y())] + self.parent_gui.gradient_direction_start = (point.x(), point.y()) + elif ev.isFinish(): + if hasattr(self.parent_gui, 'gradient_direction_start'): + self.parent_gui.gradient_direction_points = [ + self.parent_gui.gradient_direction_start, + (point.x(), point.y()) + ] + self.parent_gui.compute_direction_vector() + self.parent_gui.update_direction_line() + # Toggle off the direction selection mode + self.parent_gui.direction_button.setChecked(False) + self.parent_gui.set_gradient_direction_mode() + self.parent_gui.compute_candidate_points_gradient_direction() + else: + # During drag, update the line display + if hasattr(self.parent_gui, 'gradient_direction_start'): + temp_points = [ + self.parent_gui.gradient_direction_start, + (point.x(), point.y()) + ] + xs = [p[0] for p in temp_points] + ys = [p[1] for p in temp_points] + self.parent_gui.direction_line.setData(xs, ys) + + ev.accept() + return True + + def _handle_brush_drag(self, ev): + """Handle Ctrl+drag brush painting in Selection mode when Brush is the active method. + + :param ev: the pyqtgraph mouse-drag event + :return: True if the event was consumed + :rtype: bool + """ + if not (self.parent_gui.mode == "selection" and self.parent_gui.current_kind() == 'brush'): + return False + if not self.parent_gui.ctrl_held: + return False + + ev.accept() + if ev.isStart(): + self.parent_gui._painting = True + self.parent_gui._brush_path = [] + self.parent_gui.handle_brush_start(ev) + elif ev.isFinish(): + self.parent_gui._painting = False + self.parent_gui.handle_brush_end(ev) + else: + self.parent_gui.handle_brush_move(ev) + return True + def mouseDragEvent(self, ev, axis=None): - # Handle gradient direction selection - if self.parent_gui.mode == "filter" and self.parent_gui.setting_direction: - if ev.isStart(): - pos = ev.scenePos() - if self.sceneBoundingRect().contains(pos): - point = self.mapSceneToView(pos) - self.parent_gui.gradient_direction_points = [(point.x(), point.y())] - self.parent_gui.gradient_direction_start = (point.x(), point.y()) - ev.accept() - return - elif ev.isFinish(): - pos = ev.scenePos() - if self.sceneBoundingRect().contains(pos): - point = self.mapSceneToView(pos) - if hasattr(self.parent_gui, 'gradient_direction_start'): - self.parent_gui.gradient_direction_points = [ - self.parent_gui.gradient_direction_start, - (point.x(), point.y()) - ] - self.parent_gui.compute_direction_vector() - self.parent_gui.update_direction_line() - # Toggle off the direction selection mode - self.parent_gui.direction_button.setChecked(False) - self.parent_gui.set_gradient_direction_mode() - self.parent_gui.compute_candidate_points_gradient_direction() - ev.accept() - return - else: - # During drag, update the line display - pos = ev.scenePos() - if self.sceneBoundingRect().contains(pos): - point = self.mapSceneToView(pos) - if hasattr(self.parent_gui, 'gradient_direction_start'): - temp_points = [ - self.parent_gui.gradient_direction_start, - (point.x(), point.y()) - ] - xs = [p[0] for p in temp_points] - ys = [p[1] for p in temp_points] - self.parent_gui.direction_line.setData(xs, ys) - ev.accept() - return - - if self.parent_gui.mode == "selection" and self.parent_gui.method_buttons["Brush"].isChecked(): - if self.parent_gui.ctrl_held: - ev.accept() - if ev.isStart(): - self.parent_gui._painting = True - self.parent_gui._brush_path = [] - self.parent_gui.handle_brush_start(ev) - elif ev.isFinish(): - self.parent_gui._painting = False - self.parent_gui.handle_brush_end(ev) - else: - self.parent_gui.handle_brush_move(ev) - return + if self._handle_gradient_direction_drag(ev): + return + if self._handle_brush_drag(ev): + return + # Plain left-drag starting on an existing grid/polyline vertex moves that vertex. + if self._handle_vertex_drag(ev): + return # fallback: pan super().mouseDragEvent(ev, axis) -class SelectionGUI(QtWidgets.QMainWindow): +class SelectionGUIOld(QtWidgets.QMainWindow): + """The point-selection interface of pyIDI 1.3, kept for one release. + + .. deprecated:: 1.4 + Use :class:`~pyidi.SelectionGUI` instead, which is what the name + ``SelectionGUI`` now refers to. This class is frozen and will be + removed in 1.5. + + The replacement does everything this window does -- the same five tools, + the same two filters -- from a pipeline that scores the whole frame once + instead of one subset at a time, so it stays responsive while a slider is + still moving. The constructor signature is identical and ``get_points()`` + returns the same ``(row, col)`` array, so most scripts need only the name + changed. What does not carry over: + + - ``get_filtered_points()`` and ``get_selected_points()``. The replacement + has one ``get_points()``, because the filter is no longer a second pass + over an existing selection -- it is the selection. + - the internal attributes (``selections``, ``subset_size_spinbox``, ...), + which have no counterpart. + + Two behaviours also differ, both deliberately. Scores near the image border + are computed on real neighbours rather than reflected ones, so they are + slightly different -- and correct. And "Grid" is no longer a mode: draw a + polygon and set its row to the ``points`` role, or use the ``lattice`` + selector. + """ + def __init__(self, video, subset_size=11, subset_overlap=0): """Initialize the selection GUI for manual subset selection. @@ -92,17 +246,46 @@ def __init__(self, video, subset_size=11, subset_overlap=0): ---------- video : VideoReader or np.ndarray The video to be analyzed. If a VideoReader object, it should be initialized with the video file. + If a np.ndarray, it can be either a single 2-D image (height, width) or a 3-D frame stack + (n_frames, height, width), in which case the first frame is displayed. + subset_size : int or (height, width) tuple, optional + Initial side length (in pixels) of the subset drawn around each selected point. Either a + single int for a square subset, or a ``(height, width)`` pair for an anisotropic one, where + ``height`` is the vertical/row extent and ``width`` the horizontal/column extent -- the same + convention as ``LucasKanade.configure(roi_size=(vertical, horizontal))``, so a value that + works for one works for the other. Sets the starting values of the "Subset size" spinboxes/ + sliders used when computing ROI rectangles, grid/line spacing and automatic feature + filtering. Defaults to 11. Normalized and stored as ``self.subset_size``, a ``(height, + width)`` tuple of ints. The "Square subsets" checkbox starts checked if ``height == + width`` (so the width spinbox mirrors the height one) and unchecked otherwise. + subset_overlap : int, optional + Initial spacing (in pixels) between neighbouring subsets, used as the "Distance between + subsets" spinbox/slider value for the Grid, Along the line and Brush selection methods. + A positive value adds a gap between subsets, a negative value makes them overlap. This is a + single scalar applied to both axes -- the per-axis step is ``height + subset_overlap`` and + ``width + subset_overlap``, which is enough to get a sensible anisotropic grid spacing + without a second overlap control. Defaults to 0. """ + warnings.warn( + "SelectionGUIOld is deprecated and will be removed in pyIDI 1.5. " + "Use SelectionGUI, which as of 1.4 is the interface built on " + "pyidi.selection; it takes the same arguments and returns the same " + "(row, col) points.", + DeprecationWarning, + stacklevel=2, + ) + app = QtWidgets.QApplication.instance() if app is None: app = QtWidgets.QApplication([]) super().__init__() - self.setWindowTitle("ROI Selection Tool") + self.setWindowTitle("ROI Selection Tool (deprecated)") self.resize(1200, 800) - self.subset_size = subset_size + h, w = _as_size_pair(subset_size) + self.subset_size = (int(h), int(w)) self.subset_overlap = subset_overlap self._paint_mask = None # Same shape as the image @@ -111,19 +294,34 @@ def __init__(self, video, subset_size=11, subset_overlap=0): self.brush_deselect_mode = False self.installEventFilter(self) + # Bounded undo stack: covers adding a vertex, moving a vertex, and deleting a + # grid/polyline. Manual points, brush strokes and filter results are not undoable. + self.undo_stack = [] + self.undo_stack_limit = 50 + self.undo_shortcut = QtGui.QShortcut(QtGui.QKeySequence.StandardKey.Undo, self) + self.undo_shortcut.activated.connect(self.undo) + self.gradient_direction_points = [] self.gradient_direction = None self.setting_direction = False self.selected_points = [] - self.manual_points = [] self.candidate_points = [] - self.drawing_polygons = [{'points': [], 'roi_points': []}] - self.active_polygon_index = 0 - self.grid_polygons = [{'points': [], 'roi_points': []}] - self.active_grid_index = 0 - self.brush_masks = [] # Store brush masks for recomputation - self.brush_points = [] # Store computed brush points separately + # The threshold-and-show method of whichever filter produced candidate_points, + # so a change to the selection can re-derive them; None when no filter is live. + self._candidate_refresh = None + + # Single ordered list of selection entries, replacing the old parallel + # manual/line/grid/brush containers -- see add_selection()/entry_points() + # for the entry schema. Entries are created lazily on first click/stroke, + # so this starts empty (no "always >= 1 placeholder entry" invariant). + self.selections = [] + self.active_index = None # int index into self.selections, or None + self._label_counters = {'manual': 0, 'line': 0, 'grid': 0, 'brush': 0} + # Guards against re-entrant QListWidget signal handling: programmatic + # setCurrentRow()/setText()/setCheckState() calls set this so the + # corresponding currentRowChanged/itemChanged handlers bail out early. + self._syncing_list = False # Add status bar for instructions self.statusBar = self.statusBar() @@ -216,7 +414,18 @@ def __init__(self, video, subset_size=11, subset_overlap=0): from ..video_reader import VideoReader if isinstance(video, VideoReader): self.frame = video.get_frame(0) - + elif isinstance(video, np.ndarray) and video.ndim == 3: + # (n_frames, height, width) - take the first frame + self.frame = video[0] + elif isinstance(video, np.ndarray) and video.ndim == 2: + # (height, width) - a single image + self.frame = video + else: + raise TypeError( + f"`video` must be a VideoReader, or a 2-D (height, width) or 3-D " + f"(n_frames, height, width) np.ndarray, got {type(video).__name__!r}." + ) + self.image_item.setImage(self.frame.T) # axis 0 is x, while image axis 0 is y # Ensure method-specific widgets are visible on startup @@ -272,22 +481,53 @@ def ui_graphics(self): self.image_item = ImageItem() self.polygon_line = pg.PlotDataItem(pen=pg.mkPen('y', width=2)) self.polygon_points_scatter = ScatterPlotItem(pen=pg.mkPen(None), brush=pg.mkBrush(255, 255, 0, 200), size=6) - self.scatter = ScatterPlotItem(pen=pg.mkPen(None), brush=pg.mkBrush(255, 100, 100, 200), size=8) + self.grid_line = pg.PlotDataItem(pen=pg.mkPen('c', width=2)) + self.grid_points_scatter = ScatterPlotItem(pen=pg.mkPen(None), brush=pg.mkBrush(255, 200, 0, 200), size=6) self.roi_overlay = ImageItem() + # The subset borders, kept apart from roi_overlay (which carries only the + # translucent interior) so they can be stroked with a *cosmetic* pen: its width + # is measured in screen pixels rather than image pixels, so the borders stay one + # pixel thin however far you zoom in. A raster border cannot go below one image + # pixel, which turns into a thick band at high zoom. + self.roi_outline = QtWidgets.QGraphicsPathItem() + outline_pen = pg.mkPen(0, 255, 0, 150) + outline_pen.setCosmetic(True) + self.roi_outline.setPen(outline_pen) + self.roi_outline.setBrush(pg.mkBrush(None)) + self.scatter = ScatterPlotItem(pen=pg.mkPen(None), brush=pg.mkBrush(255, 100, 100, 200), size=8) + # Highlights the points of the entry currently selected in selection_list, + # drawn on top of the plain point scatter. Magenta with no fill: it is the one + # strong hue not already taken (green = ROI fill and filter candidates, cyan = + # grid outline, yellow = line outline and vertices, salmon = the points + # themselves), and unlike a white ring it stays visible against both the dark + # and the bright parts of a grayscale frame. No fill so the point underneath + # still reads through the ring. + self.highlight_scatter = ScatterPlotItem( + pen=pg.mkPen(255, 0, 255, 230, width=2), brush=pg.mkBrush(None), size=13 + ) self.candidate_scatter = ScatterPlotItem( pen=pg.mkPen(None), brush=pg.mkBrush(0, 255, 0, 200), size=6 ) + self.brush_overlay = ImageItem() self.direction_line = pg.PlotDataItem(pen=pg.mkPen('r', width=2)) self.view.addItem(self.image_item) self.view.addItem(self.polygon_line) self.view.addItem(self.polygon_points_scatter) + self.view.addItem(self.grid_line) + self.view.addItem(self.grid_points_scatter) + self.roi_overlay.setZValue(1) self.view.addItem(self.roi_overlay) # Add scatter for showing square points + self.roi_outline.setZValue(1) + self.view.addItem(self.roi_outline) self.view.addItem(self.scatter) # Add scatter for showing points + self.view.addItem(self.highlight_scatter) self.view.addItem(self.candidate_scatter) + self.brush_overlay.setZValue(2) + self.view.addItem(self.brush_overlay) self.view.addItem(self.direction_line) self.splitter.addWidget(self.pg_widget) @@ -321,6 +561,104 @@ def ui_right_menu(self): self.automatic_layout.addStretch(1) + def _make_subset_size_spinbox(self, initial_value: int) -> QtWidgets.QSpinBox: + """Create a subset-size QSpinBox with the styling shared by the height/width spinboxes. + + :param initial_value: starting value of the spinbox + :type initial_value: int + :rtype: QtWidgets.QSpinBox + """ + spinbox = QtWidgets.QSpinBox() + spinbox.setRange(1, 1000) + spinbox.setValue(initial_value) + spinbox.setAlignment(QtCore.Qt.AlignmentFlag.AlignRight) + spinbox.setSingleStep(2) + spinbox.setMinimum(1) + spinbox.setMaximum(999) + spinbox.setWrapping(False) + spinbox.setSuffix("px") + spinbox.setFixedWidth(80) + return spinbox + + def get_subset_size(self): + """Return the current subset size as a (height, width) tuple of ints. + + :rtype: tuple of int + """ + return (self.subset_height_spinbox.value(), self.subset_width_spinbox.value()) + + def toggle_square_subsets(self, checked: bool): + """Handle the "Square subsets" checkbox: lock/unlock the width axis. + + When checked, the width spinbox is disabled and snapped to the current height, and the + width slider is hidden, so the two axes are visually and functionally merged. When + unchecked, both become independently editable. + + :param checked: new checkbox state + :type checked: bool + """ + self.subset_width_spinbox.setEnabled(not checked) + self.subset_width_slider.setVisible(not checked) + + if checked: + self._set_subset_width_value(self.subset_height_spinbox.value()) + self.recompute_roi_points() + + def _set_subset_width_value(self, value: int): + """Set the width spinbox/slider to ``value`` without re-entering their handlers. + + :param value: new width value + :type value: int + """ + self.subset_width_spinbox.blockSignals(True) + self.subset_width_spinbox.setValue(value) + self.subset_width_spinbox.blockSignals(False) + + slider_value = min(100, max(1, value)) + self.subset_width_slider.blockSignals(True) + self.subset_width_slider.setValue(slider_value) + self.subset_width_slider.blockSignals(False) + + def _sync_square_width_and_recompute(self): + """If "Square subsets" is checked, mirror the height into the width; always recompute.""" + if self.square_subsets_checkbox.isChecked(): + self._set_subset_width_value(self.subset_height_spinbox.value()) + self.recompute_roi_points() + + def update_subset_height_from_slider(self, value): + """Update the height spinbox from the height slider value and recompute ROI points.""" + self.subset_height_spinbox.blockSignals(True) + self.subset_height_spinbox.setValue(value) + self.subset_height_spinbox.blockSignals(False) + + self._sync_square_width_and_recompute() + + def update_subset_height_from_spinbox(self, value): + """Update the height slider from the height spinbox value and recompute ROI points.""" + slider_value = min(100, max(1, value)) + self.subset_height_slider.blockSignals(True) + self.subset_height_slider.setValue(slider_value) + self.subset_height_slider.blockSignals(False) + + self._sync_square_width_and_recompute() + + def update_subset_width_from_slider(self, value): + """Update the width spinbox from the width slider value and recompute ROI points.""" + self.subset_width_spinbox.blockSignals(True) + self.subset_width_spinbox.setValue(value) + self.subset_width_spinbox.blockSignals(False) + + self.recompute_roi_points() + + def update_subset_width_from_spinbox(self, value): + """Update the width slider from the width spinbox value and recompute ROI points.""" + slider_value = min(100, max(1, value)) + self.subset_width_slider.blockSignals(True) + self.subset_width_slider.setValue(slider_value) + self.subset_width_slider.blockSignals(False) + + self.recompute_roi_points() + def ui_manual_right_menu(self): # Number of selected subsets self.points_label = QtWidgets.QLabel("Selected subsets: 0") @@ -362,32 +700,54 @@ def ui_manual_right_menu(self): config_group = QtWidgets.QGroupBox("Subset Configuration") config_layout = QtWidgets.QVBoxLayout(config_group) - # Subset size input + # Square subsets toggle: checked by default, but unchecked automatically if + # constructed with an already-anisotropic (h, w) pair, so the checkbox state + # matches the sizes it was started with. + square_default = self.subset_size[0] == self.subset_size[1] + self.square_subsets_checkbox = QtWidgets.QCheckBox("Square subsets") + self.square_subsets_checkbox.setChecked(square_default) + self.square_subsets_checkbox.toggled.connect(self.toggle_square_subsets) + config_layout.addWidget(self.square_subsets_checkbox) + + # Subset size input: height x width. The label sits on its own row above the + # spinboxes -- previously it shared a row with both spinboxes and got clipped + # to "Subset size (h" in the panel's default width. + config_layout.addWidget(QtWidgets.QLabel("Subset size (h x w):")) + self.subset_size_layout = QtWidgets.QHBoxLayout() - self.subset_size_layout.addWidget(QtWidgets.QLabel("Subset size:")) - - self.subset_size_spinbox = QtWidgets.QSpinBox() - self.subset_size_spinbox.setRange(1, 1000) - self.subset_size_spinbox.setValue(self.subset_size) - self.subset_size_spinbox.setAlignment(QtCore.Qt.AlignmentFlag.AlignRight) - self.subset_size_spinbox.setSingleStep(2) - self.subset_size_spinbox.setMinimum(1) - self.subset_size_spinbox.setMaximum(999) - self.subset_size_spinbox.setWrapping(False) - self.subset_size_spinbox.setSuffix("px") - self.subset_size_spinbox.setFixedWidth(80) - self.subset_size_spinbox.valueChanged.connect(self.update_subset_size_from_spinbox) - self.subset_size_layout.addWidget(self.subset_size_spinbox) - + + self.subset_height_spinbox = self._make_subset_size_spinbox(self.subset_size[0]) + self.subset_height_spinbox.valueChanged.connect(self.update_subset_height_from_spinbox) + self.subset_size_layout.addWidget(self.subset_height_spinbox) + # Kept as an explicit alias for the height spinbox for backward compatibility -- + # docs/source/quick_start/make_selection_animation.py and possibly user scripts + # reach for this name. + self.subset_size_spinbox = self.subset_height_spinbox + + self.subset_size_layout.addWidget(QtWidgets.QLabel("x")) + + self.subset_width_spinbox = self._make_subset_size_spinbox(self.subset_size[1]) + self.subset_width_spinbox.valueChanged.connect(self.update_subset_width_from_spinbox) + self.subset_width_spinbox.setEnabled(not square_default) + self.subset_size_layout.addWidget(self.subset_width_spinbox) + self.subset_size_layout.addStretch() # Push everything to the left config_layout.addLayout(self.subset_size_layout) - - self.subset_size_slider = QtWidgets.QSlider(QtCore.Qt.Orientation.Horizontal) - self.subset_size_slider.setRange(1, 100) - self.subset_size_slider.setValue(self.subset_size) - self.subset_size_slider.setSingleStep(1) - self.subset_size_slider.valueChanged.connect(self.update_subset_size_from_slider) - config_layout.addWidget(self.subset_size_slider) + + self.subset_height_slider = QtWidgets.QSlider(QtCore.Qt.Orientation.Horizontal) + self.subset_height_slider.setRange(1, 100) + self.subset_height_slider.setValue(self.subset_size[0]) + self.subset_height_slider.setSingleStep(1) + self.subset_height_slider.valueChanged.connect(self.update_subset_height_from_slider) + config_layout.addWidget(self.subset_height_slider) + + self.subset_width_slider = QtWidgets.QSlider(QtCore.Qt.Orientation.Horizontal) + self.subset_width_slider.setRange(1, 100) + self.subset_width_slider.setValue(self.subset_size[1]) + self.subset_width_slider.setSingleStep(1) + self.subset_width_slider.valueChanged.connect(self.update_subset_width_from_slider) + self.subset_width_slider.setVisible(not square_default) + config_layout.addWidget(self.subset_width_slider) # Show ROI rectangles self.show_roi_checkbox = QtWidgets.QCheckBox("Show subsets") @@ -460,28 +820,20 @@ def ui_manual_right_menu(self): self.brush_deselect_button.clicked.connect(self.activate_brush_deselect) method_controls_layout.addWidget(self.brush_deselect_button) - # Polygon manager (visible only for "Along the line") - self.polygon_list = QtWidgets.QListWidget() - self.polygon_list.setVisible(False) - self.polygon_list.currentRowChanged.connect(self.on_polygon_selected) - method_controls_layout.addWidget(self.polygon_list) - - self.delete_polygon_button = QtWidgets.QPushButton("Delete selected polygon") - self.delete_polygon_button.clicked.connect(self.delete_selected_polygon) - self.delete_polygon_button.setVisible(False) - method_controls_layout.addWidget(self.delete_polygon_button) - - # Grid polygon manager - self.grid_list = QtWidgets.QListWidget() - self.grid_list.setVisible(False) - self.grid_list.currentRowChanged.connect(self.on_grid_selected) - method_controls_layout.addWidget(self.grid_list) - - self.delete_grid_button = QtWidgets.QPushButton("Delete selected grid") - self.delete_grid_button.clicked.connect(self.delete_selected_grid) - self.delete_grid_button.setVisible(False) - method_controls_layout.addWidget(self.delete_grid_button) - + # Unified list of all selection entries (manual/line/grid/brush), always + # visible regardless of the active method -- replaces the separate + # polygon_list/grid_list. Each row's checkbox toggles that entry's + # visibility; the delete button below removes the current row. + self.selection_list = QtWidgets.QListWidget() + self.selection_list.setMinimumHeight(120) + self.selection_list.currentRowChanged.connect(self.on_entry_selected) + self.selection_list.itemChanged.connect(self.on_entry_item_changed) + method_controls_layout.addWidget(self.selection_list) + + self.delete_entry_button = QtWidgets.QPushButton("Delete selected") + self.delete_entry_button.clicked.connect(self.delete_selected_entry) + method_controls_layout.addWidget(self.delete_entry_button) + self.manual_layout.addWidget(method_controls_group) self.manual_layout.addStretch(1) @@ -528,6 +880,7 @@ def ui_auto_right_menu(self): self.show_points_checkbox.setChecked(False) def toggle_points_and_roi(state): self.roi_overlay.setVisible(state) + self.roi_outline.setVisible(state) self.scatter.setVisible(state) self.show_points_checkbox.stateChanged.connect(toggle_points_and_roi) display_options_layout.addWidget(self.show_points_checkbox) @@ -659,17 +1012,16 @@ def show_instruction(self, message: str): def method_selected(self, id: int): method_name = list(self.method_buttons.keys())[id] # print(f"Selected method: {method_name}") - is_along = method_name == "Along the line" - is_grid = method_name == "Grid" - is_brush = method_name == "Brush" + kind = self.current_kind() + self._reactivate_last_entry_of_kind(kind) + is_along = kind == 'line' + is_grid = kind == 'grid' + is_brush = kind == 'brush' show_spacing = is_along or is_grid or is_brush self.start_new_line_button.setVisible(is_along or is_grid) - self.polygon_list.setVisible(is_along) - self.delete_polygon_button.setVisible(is_along) - self.grid_list.setVisible(is_grid) - self.delete_grid_button.setVisible(is_grid) + self.start_new_line_button.setText("Start new grid" if is_grid else "Start new line") self.distance_widget.setVisible(show_spacing) self.distance_slider.setVisible(show_spacing) @@ -682,9 +1034,15 @@ def method_selected(self, id: int): if is_brush: self.show_instruction("Hold Ctrl and drag to paint selection area. Use distance slider to control subset spacing.") elif is_along: - self.show_instruction("Click to add points along the line. Click 'Start new line' to begin a new one.") + self.show_instruction( + "Click to add points along the line. Drag an existing point to move it. " + "Click 'Start new line' to begin a new one. Ctrl+Z to undo." + ) elif is_grid: - self.show_instruction("Click to define grid corners. Click 'Start new line' to begin a new grid.") + self.show_instruction( + "Click to define grid corners. Drag an existing corner to move it. " + "Click 'Start new grid' to begin a new grid. Ctrl+Z to undo." + ) elif method_name == "Manual": self.show_instruction("Click to add points manually.") elif method_name == "Remove point": @@ -709,7 +1067,9 @@ def switch_mode(self, mode: str): self.direction_line.clear() self.roi_overlay.setVisible(True) + self.roi_outline.setVisible(True) self.scatter.setVisible(True) + self.highlight_scatter.setVisible(True) self.show_instruction("Selection mode: choose a method on the left.") elif mode == "filter": @@ -720,74 +1080,481 @@ def switch_mode(self, mode: str): # Don't automatically compute anything - let user select method first self.show_points_checkbox.setChecked(False) self.roi_overlay.setVisible(False) + self.roi_outline.setVisible(False) self.scatter.setVisible(False) + # The active entry's highlight belongs to Select mode; leaving it on would + # ring points that are no longer drawn. + self.highlight_scatter.setVisible(False) self.show_instruction("Filter mode: choose a filter method and adjust settings.") def on_mouse_click(self, event): if self.mode == "filter": return - - if self.method_buttons["Manual"].isChecked(): + + kind = self.current_kind() + if kind == 'manual': self.handle_manual_selection(event) - elif self.method_buttons["Along the line"].isChecked(): + elif kind == 'line': self.handle_polygon_drawing(event) - elif self.method_buttons["Grid"].isChecked(): + elif kind == 'grid': self.handle_grid_drawing(event) + elif kind == 'brush': + self.handle_brush_start(event) elif self.method_buttons["Remove point"].isChecked(): self.handle_remove_point(event) - elif self.method_buttons["Brush"].isChecked(): - self.handle_brush_start(event) + + # ------------------------------------------------------------------ + # Selection-entry core helpers + # + # All selections (manual points, "along the line" polylines, grid + # polygons, brush strokes) live in one ordered list, self.selections, + # instead of four separate parallel containers. Each entry is a dict; + # see the module-level PRETTY dict and add_selection() below for the + # label scheme and entry schema. + # ------------------------------------------------------------------ + def current_kind(self): + """Return the selection-entry kind for the currently-checked method button. + + :return: ``'grid'``, ``'manual'``, ``'line'`` or ``'brush'`` for the + correspondingly-checked method button; ``None`` if "Remove point" is + checked (it has no entry kind of its own) or if no button is checked. + :rtype: str or None + """ + button_names = {'Grid': 'grid', 'Manual': 'manual', 'Along the line': 'line', 'Brush': 'brush'} + for name, kind in button_names.items(): + if self.method_buttons[name].isChecked(): + return kind + return None + + def add_selection(self, kind, geometry=None, label=None, make_active=True): + """Append a new entry of `kind`, register its list row, and return the entry dict. + + This is the entry point used both by the click/stroke handlers below and by + external callers (tests, the docs animation script) that want to build up a + selection programmatically. + + :param kind: ``'manual'``, ``'line'``, ``'grid'`` or ``'brush'`` + :type kind: str + :param geometry: initial geometry for the entry; defaults to ``[]`` for + manual/line/grid. Brush entries are always created with a real mask, so + leaving this ``None`` for ``kind='brush'`` is a programming error. + :type geometry: list or numpy.ndarray or None + :param label: explicit row label; if omitted, one is generated from the + per-kind monotonic counter (see the class/module docstring) + :type label: str or None + :param make_active: whether to make the new entry the active one and select + its row in ``selection_list`` + :type make_active: bool + :return: the newly created entry dict + :rtype: dict + :raises ValueError: if ``kind == 'brush'`` and ``geometry`` is ``None`` + """ + if geometry is None: + if kind == 'brush': + raise ValueError("add_selection('brush', ...) requires an explicit mask.") + geometry = [] + if label is None: + self._label_counters[kind] += 1 + label = 'Manual' if kind == 'manual' else f'{PRETTY[kind]} {self._label_counters[kind]}' + entry = { + 'kind': kind, + 'label': label, + 'geometry': geometry, + 'roi_points': [], + 'removed': set(), + 'visible': True, + } + self.selections.append(entry) + row = len(self.selections) - 1 + self._insert_row(row, entry) + if make_active: + self.active_index = row + self._syncing_list = True + self.selection_list.setCurrentRow(row) + self._syncing_list = False + return entry + + def _reactivate_last_entry_of_kind(self, kind): + """Make the most recent entry of `kind` active, if the active one is not already. + + There is a single ``active_index`` for all kinds, where the pre-list code kept a + separate active index per kind. Without this, switching away from Grid (say, to + drop a manual point) and back would leave a non-grid entry active, so the next + click would start a *new* grid instead of continuing the one being drawn. Nothing + happens when the active entry is already of `kind` -- that is what keeps clicking + a specific row in ``selection_list`` from being overridden by this. + + :param kind: the entry kind the tool has just switched to, or None for + "Remove point" (which owns no entries and leaves the active one alone) + :type kind: str or None + """ + if kind is None or self.active_entry(kind) is not None: + return + last = next((i for i in reversed(range(len(self.selections))) + if self.selections[i]['kind'] == kind), None) + if last is None: + return + self.active_index = last + self._syncing_list = True + self.selection_list.setCurrentRow(last) + self._syncing_list = False + self.update_highlight() + + def active_entry(self, kind=None): + """Return the active selection entry, or None. + + :param kind: if given, only return the active entry when its kind matches + :type kind: str or None + :return: the active entry dict, or None if there is no active entry (or its + kind does not match `kind`) + :rtype: dict or None + """ + if self.active_index is None or not (0 <= self.active_index < len(self.selections)): + return None + entry = self.selections[self.active_index] + if kind is not None and entry['kind'] != kind: + return None + return entry + + def entry_points(self, entry): + """The entry's contributed points: roi_points minus its `removed` set, order preserved. + + :param entry: a selection entry dict + :type entry: dict + :return: the entry's live points, in ``roi_points`` order + :rtype: list[tuple] + """ + return [p for p in entry['roi_points'] if tuple(p) not in entry['removed']] + + def _brush_points(self, mask, subset_size, spacing): + """ROI points inside a brush mask, returned as (x, y). + + ``mask`` is ``(n_x, n_y)`` indexed ``[x, y]`` (see ``handle_brush_move``), + but ``rois_inside_mask`` documents and assumes ``mask[y, x]`` and returns + ``(y, x)``. Transpose ``mask`` in, then flip the result back to ``(x, y)`` + out. Passing the mask untransposed happened to give correct *coordinates* + (the two transpositions canceled) but transposed the *per-axis grid step*, + so anisotropic subsets got the row/column spacing swapped -- fixed here. + + :param mask: boolean brush mask, ``(n_x, n_y)`` indexed ``[x, y]`` + :type mask: numpy.ndarray + :param subset_size: ``(height, width)`` subset size + :type subset_size: tuple + :param spacing: extra spacing added to the subset size to get the grid step + :type spacing: int + :return: ``(x, y)`` ROI points inside the mask + :rtype: list[tuple] + """ + return [(x, y) for (y, x) in rois_inside_mask(mask.T, subset_size, spacing)] + + def recompute_entry(self, entry, subset_size=None, spacing=None): + """Recompute `entry['roi_points']` from its geometry, in place. + + ``entry['removed']`` is deliberately NOT cleared here -- that is what lets + removed points survive a spacing/subset-size change (see ``entry_points``). + + :param entry: the selection entry to recompute + :type entry: dict + :param subset_size: ``(height, width)`` subset size; defaults to + ``self.get_subset_size()`` + :type subset_size: tuple or None + :param spacing: spacing between subsets; defaults to + ``self.distance_spinbox.value()`` + :type spacing: int or None + """ + if subset_size is None: + subset_size = self.get_subset_size() + if spacing is None: + spacing = self.distance_spinbox.value() + + kind, geom = entry['kind'], entry['geometry'] + if kind == 'manual': + entry['roi_points'] = list(geom) # no recompute; points are literal + elif kind == 'line': + entry['roi_points'] = points_along_polygon(geom, subset_size, spacing) if len(geom) >= 2 else [] + elif kind == 'grid': + entry['roi_points'] = rois_inside_polygon(geom, subset_size, spacing) if len(geom) >= 3 else [] + elif kind == 'brush': + entry['roi_points'] = self._brush_points(geom, subset_size, spacing) + + def _insert_row(self, row, entry): + """Insert a checkable QListWidgetItem for `entry` at `row` in `selection_list`. + + :param row: row index to insert at + :type row: int + :param entry: the selection entry the row represents + :type entry: dict + """ + item = QtWidgets.QListWidgetItem(f"{entry['label']} — {len(self.entry_points(entry))} pts") + item.setFlags(item.flags() | QtCore.Qt.ItemFlag.ItemIsUserCheckable) + self._syncing_list = True + item.setCheckState(QtCore.Qt.CheckState.Checked if entry['visible'] else QtCore.Qt.CheckState.Unchecked) + self.selection_list.insertItem(row, item) + self._syncing_list = False + + def refresh_row_labels(self): + """Refresh every row's text and checkbox from its entry's current state. + + Guarded by ``_syncing_list`` so the ``setText``/``setCheckState`` calls + below do not re-trigger ``on_entry_item_changed``. + """ + self._syncing_list = True + for row, entry in enumerate(self.selections): + item = self.selection_list.item(row) + if item is None: + continue + item.setText(f"{entry['label']} — {len(self.entry_points(entry))} pts") + item.setCheckState(QtCore.Qt.CheckState.Checked if entry['visible'] else QtCore.Qt.CheckState.Unchecked) + self._syncing_list = False + + def on_entry_item_changed(self, item): + """Sync an entry's `visible` flag from its row's checkbox state. + + :param item: the changed ``QListWidgetItem`` + :type item: QtWidgets.QListWidgetItem + """ + if self._syncing_list: + return + row = self.selection_list.row(item) + if 0 <= row < len(self.selections): + self.selections[row]['visible'] = item.checkState() == QtCore.Qt.CheckState.Checked + self.update_geometry_display() + self.update_selected_points() + + def on_entry_selected(self, row): + """Handle a row becoming the current row in `selection_list`. + + Switches the active tool to the row's kind, so its vertices become + immediately editable (drag/undo), and refreshes the display. + + :param row: the new current row, or -1 if the selection was cleared + :type row: int + """ + if self._syncing_list: + return + if not (0 <= row < len(self.selections)): + self.active_index = None + self.update_geometry_display() + self.update_selected_points() + return + + self.active_index = row + kind = self.selections[row]['kind'] + button_name = {'grid': 'Grid', 'line': 'Along the line', 'manual': 'Manual', 'brush': 'Brush'}[kind] + button = self.method_buttons[button_name] + if not button.isChecked(): + button.setChecked(True) + self.method_selected(self.button_group.id(button)) + self.update_geometry_display() + self.update_selected_points() + + def delete_selected_entry(self): + """Delete the currently-selected row/entry (any kind), pushing an undo action.""" + row = self.selection_list.currentRow() + if row < 0: + return + entry = self.selections[row] + self.push_undo({ + 'type': 'delete', 'kind': entry['kind'], 'entry': entry, 'row': row, 'label': entry['label'], + }) + del self.selections[row] + self.selection_list.takeItem(row) + self.active_index = min(row, len(self.selections) - 1) if self.selections else None + if self.active_index is not None: + self._syncing_list = True + self.selection_list.setCurrentRow(self.active_index) + self._syncing_list = False + self.update_geometry_display() + self.update_selected_points() + + def update_highlight(self): + """Highlight the active entry's points on top of the plain point scatter.""" + entry = self.active_entry() + if entry is None or not entry['visible']: + self.highlight_scatter.clear() + return + pts = self.entry_points(entry) + if not pts: + self.highlight_scatter.clear() + return + # Size/pen/brush come from the item's own defaults (see ui_graphics). + self.highlight_scatter.setData(pos=np.array(pts) + 0.5, symbol='o') + + def _line_display_data(self): + """Build nan-separated OPEN polyline coordinates and vertex list for 'line' entries. + + :return: ``(xs, ys, all_points)`` -- flattened, nan-separated polyline + coordinates and the flat list of all vertices, for every visible + ``line`` entry + :rtype: tuple + """ + xs, ys, all_points = [], [], [] + for entry in self.selections: + if entry['kind'] != 'line' or not entry['visible']: + continue + path = entry['geometry'] + all_points.extend(path) + if len(path) >= 2: + xs.extend([p[0] for p in path] + [np.nan]) + ys.extend([p[1] for p in path] + [np.nan]) + elif len(path) == 1: + xs.extend([path[0][0], path[0][0], np.nan]) + ys.extend([path[0][1], path[0][1], np.nan]) + return xs, ys, all_points + + def _grid_display_data(self): + """Build nan-separated CLOSED polygon coordinates and vertex list for 'grid' entries. + + :return: ``(xs, ys, all_points)`` -- flattened, nan-separated closed-polygon + coordinates and the flat list of all vertices, for every visible + ``grid`` entry + :rtype: tuple + """ + xs, ys, all_points = [], [], [] + for entry in self.selections: + if entry['kind'] != 'grid' or not entry['visible']: + continue + path = entry['geometry'] + all_points.extend(path) + if len(path) >= 2: + xs.extend([p[0] for p in path] + [path[0][0], np.nan]) # Close polygon + ys.extend([p[1] for p in path] + [path[0][1], np.nan]) + elif len(path) == 1: + xs.extend([path[0][0], path[0][0], np.nan]) + ys.extend([path[0][1], path[0][1], np.nan]) + return xs, ys, all_points + + def update_geometry_display(self): + """Redraw the line/grid outlines and vertex scatters from `self.selections`. + + Walks ``self.selections`` once per kind and builds the four display + datasets (line outline + vertices, grid outline + vertices), skipping + entries with ``visible == False``. + """ + line_xs, line_ys, line_points = self._line_display_data() + self.polygon_line.setData(line_xs, line_ys) + self.polygon_points_scatter.setData(pos=line_points) + + grid_xs, grid_ys, grid_points = self._grid_display_data() + self.grid_line.setData(grid_xs, grid_ys) + self.grid_points_scatter.setData(pos=grid_points) + + self.update_highlight() + + def clear_subset_rectangles(self): + """Remove both halves of the subset-rectangle display.""" + self.roi_overlay.clear() + self.roi_outline.setPath(QtGui.QPainterPath()) + + def draw_subset_rectangles(self, points, half_h, half_w): + """Draw a rectangle around each point, as a raster fill plus a hairline border. + + The two halves are drawn by different means because each is cheap in a + different way. The translucent interior goes into ``roi_overlay`` as a single + RGBA image, which costs one upload no matter how many subsets there are. The + borders go into ``roi_outline`` as one QPainterPath stroked with a cosmetic + pen, whose width is in *screen* pixels: that is what makes them a hairline at + any zoom. Drawing the borders into the raster instead, as this used to, pins + them to one *image* pixel, which is a thick band as soon as you zoom in. + + Both are built with whole-array numpy rather than a Python loop over the + points, which is what keeps the redraw quick for tens of thousands of subsets. + + :param points: the subset centres, as an ``(n, 2)`` array of real ``(x, y)`` + = (column, row) image coordinates + :type points: numpy.ndarray + :param half_h: half the subset height, in pixels (``subset_h // 2``) + :type half_h: int + :param half_w: half the subset width, in pixels (``subset_w // 2``) + :type half_w: int + """ + # image_item.image / roi_overlay are column-major (pyqtgraph's default + # axisOrder): array axis 0 is the image's x/width axis, axis 1 is its y/height + # axis. half_w therefore pairs with axis 0 and half_h with axis 1. + n_x, n_y = self.image_item.image.shape[:2] + w_x, w_y = 2 * half_w + 1, 2 * half_h + 1 + + ix0 = np.rint(points[:, 0]).astype(int) - half_w + iy0 = np.rint(points[:, 1]).astype(int) - half_h + # A subset whose rectangle would reach past the image edge is not drawn at all. + inside = (ix0 >= 0) & (iy0 >= 0) & (ix0 + w_x < n_x) & (iy0 + w_y < n_y) + ix0, iy0 = ix0[inside], iy0[inside] + + if not len(ix0): + self.clear_subset_rectangles() + return + + # Fill: mark every covered pixel at once by broadcasting the per-subset pixel + # index ranges against each other, giving an (n, w_x, w_y) index into the mask. + covered = np.zeros((n_x, n_y), dtype=bool) + covered[(ix0[:, None] + np.arange(w_x))[:, :, None], + (iy0[:, None] + np.arange(w_y))[:, None, :]] = True + overlay = np.zeros((n_x, n_y, 4), dtype=np.uint8) # RGBA + overlay[..., 1] = covered * np.uint8(180) # green + overlay[..., 3] = covered * np.uint8(40) # alpha + self.roi_overlay.setImage(overlay, autoLevels=False) + self.roi_overlay.setZValue(1) + + # Border: five corners per rectangle (the first repeated to close it) separated + # by a nan, which is how arrayToQPath is told to start a new sub-path. + x0, y0 = ix0.astype(float), iy0.astype(float) + x1, y1 = x0 + w_x, y0 + w_y + xs = np.empty((len(x0), 6)) + ys = np.empty((len(y0), 6)) + xs[:, 0] = xs[:, 3] = xs[:, 4] = x0 + xs[:, 1] = xs[:, 2] = x1 + ys[:, 0] = ys[:, 1] = ys[:, 4] = y0 + ys[:, 2] = ys[:, 3] = y1 + xs[:, 5] = ys[:, 5] = np.nan + self.roi_outline.setPath(pg.arrayToQPath(xs.ravel(), ys.ravel(), connect='finite')) + + def refresh_candidates_for_selection(self): + """Re-derive the filter candidates from what is currently selected. + + The Filter-mode filters score the subsets placed in Select mode, so their + result goes stale the moment one of those subsets disappears -- painted over + with the brush in deselect mode, clicked away with "Remove point", or removed + by deleting or unchecking a row. A stale candidate was not merely drawn in the + wrong place: ``get_points()`` returns the candidates whenever a filter has been + run, so a deselected subset stayed in the returned points. + + The cached per-subset scores are deliberately kept, so this is reversible: + re-checking a row, or undoing its deletion, brings its candidates back without + re-running the filter. + """ + if self._candidate_refresh is not None: + self._candidate_refresh() def update_selected_points(self): - polygon_points = [pt for poly in self.drawing_polygons for pt in poly['roi_points']] - grid_points = [pt for g in self.grid_polygons for pt in g['roi_points']] - self.selected_points = self.manual_points + polygon_points + grid_points + self.brush_points + # Order is creation order across all kinds (manual/line/grid/brush mixed + # together as entries were added) -- a deliberate change from the old + # manual+line+grid+brush concatenation order. + self.selected_points = [] + for entry in self.selections: + if entry['visible']: + self.selected_points.extend(self.entry_points(entry)) + + self.refresh_candidates_for_selection() if not self.selected_points: self.scatter.clear() - self.roi_overlay.clear() + self.clear_subset_rectangles() + self.refresh_row_labels() + self.update_highlight() return - subset_size = self.subset_size_spinbox.value() - half = subset_size // 2 + subset_h, subset_w = self.get_subset_size() + half_h = subset_h // 2 + half_w = subset_w // 2 # selected_points = np.round(np.array(self.selected_points) - 0.5) selected_points = np.array(self.selected_points) # --- Rectangles --- if self.show_roi_checkbox.isChecked(): - h, w = self.image_item.image.shape[:2] - overlay = np.zeros((h, w, 4), dtype=np.uint8) # RGBA - - for y, x in selected_points: - x0 = int(round(x - half)) - y0 = int(round(y - half)) - x1 = int(round(x + half+1)) - y1 = int(round(y + half+1)) - - # Ensure bounds - if x0 < 0 or y0 < 0 or x1 >= w or y1 >= h: - continue - - # Fill interior (semi-transparent green) - overlay[y0:y1, x0:x1, 1] = 180 # green - overlay[y0:y1, x0:x1, 3] = 40 # alpha - - # Outline (more opaque green) - overlay[y0, x0:x1, 1] = 255 # top - overlay[y1 - 1, x0:x1, 1] = 255 # bottom - overlay[y0:y1, x0, 1] = 255 # left - overlay[y0:y1, x1 - 1, 1] = 255 # right - - overlay[y0, x0:x1, 3] = 150 - overlay[y1 - 1, x0:x1, 3] = 150 - overlay[y0:y1, x0, 3] = 150 - overlay[y0:y1, x1 - 1, 3] = 150 - - self.roi_overlay.setImage(overlay, autoLevels=False) - self.roi_overlay.setZValue(1) + self.draw_subset_rectangles(selected_points, half_h, half_w) else: - self.roi_overlay.clear() + self.clear_subset_rectangles() # --- Center Dots --- self.scatter.setData( @@ -798,6 +1565,8 @@ def update_selected_points(self): pen=pg.mkPen(None) ) self.points_label.setText(f"Selected subsets: {len(self.selected_points)}") + self.refresh_row_labels() + self.update_highlight() def update_distance_from_slider(self, value): """Update distance spinbox from slider value and recompute ROI points.""" @@ -819,106 +1588,42 @@ def update_distance_from_spinbox(self, value): # Recompute ROI points self.recompute_roi_points() - def update_subset_size_from_slider(self, value): - """Update subset size spinbox from slider value and recompute ROI points.""" - # Update spinbox without triggering its signal - self.subset_size_spinbox.blockSignals(True) - self.subset_size_spinbox.setValue(value) - self.subset_size_spinbox.blockSignals(False) - - # Recompute ROI points and update display - self.recompute_roi_points() - - def update_subset_size_from_spinbox(self, value): - """Update subset size slider from spinbox value and recompute ROI points.""" - # Update slider, clamping to its range - slider_value = min(100, max(1, value)) - self.subset_size_slider.blockSignals(True) - self.subset_size_slider.setValue(slider_value) - self.subset_size_slider.blockSignals(False) - - # Recompute ROI points and update display - self.recompute_roi_points() - def recompute_roi_points(self): - subset_size = self.subset_size_spinbox.value() + subset_size = self.get_subset_size() spacing = self.distance_spinbox.value() - - # Update all "along the line" polygons - for poly in self.drawing_polygons: - if len(poly['points']) >= 2: - poly['roi_points'] = points_along_polygon(poly['points'], subset_size, spacing) - - # Update all "grid" polygons - for grid in self.grid_polygons: - if len(grid['points']) >= 3: - grid['roi_points'] = rois_inside_polygon(grid['points'], subset_size, spacing) - - # Update all brush masks - self.brush_points = [] - for mask in self.brush_masks: - self.brush_points.extend(rois_inside_mask(mask, subset_size, spacing)) - + for entry in self.selections: + self.recompute_entry(entry, subset_size, spacing) self.update_selected_points() def start_new_line(self): # print("Starting a new line...") - - if self.method_buttons["Along the line"].isChecked(): - self.drawing_polygons.append({'points': [], 'roi_points': []}) - self.active_polygon_index = len(self.drawing_polygons) - 1 - self.polygon_list.addItem(f"Polygon {self.active_polygon_index + 1}") - self.polygon_list.setCurrentRow(self.active_polygon_index) - self.update_polygon_display() - - elif self.method_buttons["Grid"].isChecked(): - self.grid_polygons.append({'points': [], 'roi_points': []}) - self.active_grid_index = len(self.grid_polygons) - 1 - self.grid_list.addItem(f"Grid {self.active_grid_index + 1}") - self.grid_list.setCurrentRow(self.active_grid_index) - self.update_grid_display() - - self.update_selected_points() + kind = self.current_kind() + if kind in ('grid', 'line'): + self.add_selection(kind) + self.update_geometry_display() + self.update_selected_points() def clear_selection(self): # print("Clearing selections...") - # Clear manual points - self.manual_points = [] + # Any pending undo actions reference the entries/rows being wiped out below, + # so they would no longer apply consistently after a full clear. + self.undo_stack = [] - # Clear brush data - self.brush_masks = [] - self.brush_points = [] + self.selections = [] + self.active_index = None + self._label_counters = {k: 0 for k in self._label_counters} + self.selection_list.clear() + self.selected_points = [] - # Clear line-based polygons - self.drawing_polygons = [{'points': [], 'roi_points': []}] - self.polygon_list.clear() - self.polygon_list.addItem("Polygon 1") - self.polygon_list.setCurrentRow(0) - self.active_polygon_index = 0 self.polygon_line.clear() self.polygon_points_scatter.clear() + self.grid_line.clear() + self.grid_points_scatter.clear() + self.highlight_scatter.clear() + self.scatter.clear() + self.clear_subset_rectangles() - # Clear grid-based polygons - self.grid_polygons = [{'points': [], 'roi_points': []}] - self.grid_list.clear() - self.grid_list.addItem("Grid 1") - self.grid_list.setCurrentRow(0) - self.active_grid_index = 0 - - if hasattr(self, 'grid_line'): - self.grid_line.clear() - if hasattr(self, 'grid_points_scatter'): - self.grid_points_scatter.clear() - - # Clear selected points and visual overlays - self.selected_points = [] - - if hasattr(self, 'scatter'): - self.scatter.clear() - if hasattr(self, 'roi_overlay'): - self.roi_overlay.clear() - # Clear candidate points from automatic filtering self.clear_candidates() @@ -955,6 +1660,121 @@ def get_selected_points(self): """Get all selected points from manual, polygons and grid.""" return np.array(self.selected_points)[:, ::-1] if self.selected_points else [] + # Vertex hit-testing (shared by Grid and "Along the line" click/drag handling) + def nearest_vertex(self, points, x, y): + """Find the vertex nearest to (x, y) in a list of (x, y) points. + + Distance is measured in screen pixels (via the view's current pixel size) so + that hit-testing behaves consistently regardless of the current zoom level. + + :param points: candidate vertices in view/data coordinates, native (x, y) order + :type points: list[tuple[float, float]] + :param x: query x coordinate in view/data units + :type x: float + :param y: query y coordinate in view/data units + :type y: float + :return: (index, distance in screen pixels) of the nearest vertex, or (None, None) + if `points` is empty + :rtype: tuple + """ + if not points: + return None, None + px_x, px_y = self.view.viewPixelSize() + px_x = px_x or 1e-9 + px_y = px_y or 1e-9 + arr = np.array(points, dtype=float) + dx = (arr[:, 0] - x) / px_x + dy = (arr[:, 1] - y) / px_y + distances = np.hypot(dx, dy) + idx = int(np.argmin(distances)) + return idx, float(distances[idx]) + + def vertex_within_grab_radius(self, points, x, y): + """Return the index of the vertex nearest to (x, y) if within the grab radius. + + :param points: candidate vertices in view/data coordinates, native (x, y) order + :type points: list[tuple[float, float]] + :param x: query x coordinate in view/data units + :type x: float + :param y: query y coordinate in view/data units + :type y: float + :return: index of the nearest vertex, or None if none is within the grab radius + :rtype: int or None + """ + idx, dist = self.nearest_vertex(points, x, y) + if idx is not None and dist <= VERTEX_GRAB_RADIUS_PX: + return idx + return None + + def find_vertex_to_drag(self, entries, x, y): + """Hit-test (x, y) against the vertices of every entry (grid or polyline). + + Searches across ALL entries in the container (not only the active one), so a + vertex of any grid/polyline can be grabbed and dragged. + + :param entries: list of selection entries (dicts with a 'geometry' key), + filtered to the active kind -- see + ``BrushViewBox._vertex_drag_container`` + :type entries: list[dict] + :param x: query x coordinate in view/data units + :type x: float + :param y: query y coordinate in view/data units + :type y: float + :return: (entry_index, vertex_index) of the closest vertex within the grab + radius across all entries, or (None, None) if none is close enough + :rtype: tuple + """ + best_entry_idx, best_vertex_idx, best_dist = None, None, None + for entry_idx, entry in enumerate(entries): + idx, dist = self.nearest_vertex(entry['geometry'], x, y) + if idx is None or dist > VERTEX_GRAB_RADIUS_PX: + continue + if best_dist is None or dist < best_dist: + best_entry_idx, best_vertex_idx, best_dist = entry_idx, idx, dist + return best_entry_idx, best_vertex_idx + + # Undo stack (add vertex / move vertex / delete a selection entry) + def push_undo(self, action): + """Push an action onto the bounded undo stack. + + :param action: description of the action; must include a 'type' key + ('add', 'move' or 'delete') and a 'kind' key (the entry's kind -- + 'grid', 'line', 'manual' or 'brush') + :type action: dict + """ + self.undo_stack.append(action) + if len(self.undo_stack) > self.undo_stack_limit: + self.undo_stack.pop(0) + + def undo(self): + """Undo the most recent undoable action. + + Covers adding a vertex, moving a vertex, and deleting a selection entry -- + of any kind, since delete is generic now (manual and brush entries are + undoable too, which they were not before). Filter results are not + undoable. A no-op when the undo stack is empty. + """ + if not self.undo_stack: + return + + action = self.undo_stack.pop() + + if action['type'] == 'add': + del action['entry']['geometry'][action['vertex_index']] + elif action['type'] == 'move': + action['entry']['geometry'][action['vertex_index']] = action['original_position'] + elif action['type'] == 'delete': + row = min(action['row'], len(self.selections)) + self.selections.insert(row, action['entry']) + self._insert_row(row, action['entry']) + self.active_index = row + self._syncing_list = True + self.selection_list.setCurrentRow(row) + self._syncing_list = False + + self.update_geometry_display() + self.recompute_roi_points() + # Grid selection def handle_grid_drawing(self, event): pos = event.scenePos() @@ -962,69 +1782,22 @@ def handle_grid_drawing(self, event): mouse_point = self.view.mapSceneToView(pos) x, y = mouse_point.x(), mouse_point.y() - # Add first grid polygon to the list if not yet shown - if self.grid_list.count() == 0: - self.grid_list.addItem("Grid 1") - self.grid_list.setCurrentRow(0) - - grid = self.grid_polygons[self.active_grid_index] - grid['points'].append((x, y)) + grid = self.active_entry('grid') + if grid is None: + grid = self.add_selection('grid') - # Compute ROI points only if closed polygon - if len(grid['points']) >= 3: - subset_size = self.subset_size_spinbox.value() - spacing = self.distance_spinbox.value() - grid['roi_points'] = rois_inside_polygon(grid['points'], subset_size, spacing) - - self.update_grid_display() - self.update_selected_points() + # Clicking on an existing vertex is a no-op (dragging is used to move it). + if self.vertex_within_grab_radius(grid['geometry'], x, y) is not None: + return - def on_grid_selected(self, index): - if 0 <= index < len(self.grid_polygons): - self.active_grid_index = index - - def delete_selected_grid(self): - row = self.grid_list.currentRow() - if row >= 0 and len(self.grid_polygons) > 1: - del self.grid_polygons[row] - self.grid_list.takeItem(row) - self.active_grid_index = max(0, row - 1) - self.grid_list.setCurrentRow(self.active_grid_index) - self.update_grid_display() - self.update_selected_points() + grid['geometry'].append((x, y)) + self.push_undo({ + 'type': 'add', 'kind': 'grid', 'entry': grid, 'vertex_index': len(grid['geometry']) - 1 + }) - def update_grid_display(self): - # Combine all points from all grid polygons for scatter - all_pts = [pt for poly in self.grid_polygons for pt in poly['points']] - - # Create or update scatter plot for grid polygon vertices - if not hasattr(self, 'grid_points_scatter'): - self.grid_points_scatter = ScatterPlotItem( - pen=pg.mkPen(None), - brush=pg.mkBrush(255, 200, 0, 200), - size=6 - ) - self.view.addItem(self.grid_points_scatter) - self.grid_points_scatter.setData(pos=all_pts) - - # Combine all polygon outlines with np.nan-separated segments - xs, ys = [], [] - for poly in self.grid_polygons: - path = poly['points'] - if len(path) >= 2: - xs.extend([p[0] for p in path] + [path[0][0], np.nan]) # Close polygon - ys.extend([p[1] for p in path] + [path[0][1], np.nan]) - elif len(path) == 1: - xs.extend([path[0][0], path[0][0], np.nan]) - ys.extend([path[0][1], path[0][1], np.nan]) - - # Create or update line plot for polygon outlines - if not hasattr(self, 'grid_line'): - self.grid_line = pg.PlotDataItem( - pen=pg.mkPen('c', width=2) # Cyan line - ) - self.view.addItem(self.grid_line) - self.grid_line.setData(xs, ys) + self.recompute_entry(grid) + self.update_geometry_display() + self.update_selected_points() # also refreshes this row's "N pts" label # Manual selection def handle_manual_selection(self, event): @@ -1033,9 +1806,17 @@ def handle_manual_selection(self, event): if self.view.sceneBoundingRect().contains(pos): mouse_point = self.view.mapSceneToView(pos) x, y = mouse_point.x(), mouse_point.y() - x_int, y_int = round(x-0.5), round(y-0.5) - self.manual_points.append((x_int, y_int)) - self.update_selected_points() + x_int, y_int = round(x - 0.5), round(y - 0.5) + + # Manual is a singleton entry -- every manual point lands in the same + # row, never a new one. + entry = next((e for e in self.selections if e['kind'] == 'manual'), None) + if entry is None: + entry = self.add_selection('manual') + + entry['geometry'].append((x_int, y_int)) + self.recompute_entry(entry) + self.update_selected_points() # also refreshes this row's "N pts" label # Along the line selection def handle_polygon_drawing(self, event): @@ -1044,52 +1825,22 @@ def handle_polygon_drawing(self, event): mouse_point = self.view.mapSceneToView(pos) x, y = mouse_point.x(), mouse_point.y() - # Add first polygon to the list if not yet shown - if self.polygon_list.count() == 0: - self.polygon_list.addItem("Polygon 1") - self.polygon_list.setCurrentRow(0) - - poly = self.drawing_polygons[self.active_polygon_index] - poly['points'].append((x, y)) - - # Update ROI points only for this polygon - if len(poly['points']) >= 2: - subset_size = self.subset_size_spinbox.value() - spacing = self.distance_spinbox.value() - poly['roi_points'] = points_along_polygon(poly['points'], subset_size, spacing) + poly = self.active_entry('line') + if poly is None: + poly = self.add_selection('line') - self.update_polygon_display() - self.update_selected_points() - - def delete_selected_polygon(self): - row = self.polygon_list.currentRow() - if row >= 0 and len(self.drawing_polygons) > 1: - del self.drawing_polygons[row] - self.polygon_list.takeItem(row) - self.active_polygon_index = max(0, row - 1) - self.polygon_list.setCurrentRow(self.active_polygon_index) - self.update_polygon_display() - self.update_selected_points() - - def update_polygon_display(self): - all_pts = [pt for poly in self.drawing_polygons for pt in poly['points']] - self.polygon_points_scatter.setData(pos=all_pts) - - xs, ys = [], [] - for poly in self.drawing_polygons: - path = poly['points'] - if len(path) >= 2: - xs.extend([p[0] for p in path] + [np.nan]) - ys.extend([p[1] for p in path] + [np.nan]) - elif len(path) == 1: - xs.extend([path[0][0], path[0][0], np.nan]) - ys.extend([path[0][1], path[0][1], np.nan]) + # Clicking on an existing vertex is a no-op (dragging is used to move it). + if self.vertex_within_grab_radius(poly['geometry'], x, y) is not None: + return - self.polygon_line.setData(xs, ys) + poly['geometry'].append((x, y)) + self.push_undo({ + 'type': 'add', 'kind': 'line', 'entry': poly, 'vertex_index': len(poly['geometry']) - 1 + }) - def on_polygon_selected(self, index): - if 0 <= index < len(self.drawing_polygons): - self.active_polygon_index = index + self.recompute_entry(poly) + self.update_geometry_display() + self.update_selected_points() # also refreshes this row's "N pts" label # Remove point selection def handle_remove_point(self, event): @@ -1107,48 +1858,52 @@ def handle_remove_point(self, event): idx = np.argmin(distances) closest = tuple(pts[idx]) - # Remove from manual if present - if closest in self.manual_points: - self.manual_points.remove(closest) - - # Remove from polygons - for poly in self.drawing_polygons: - if closest in poly['roi_points']: - poly['roi_points'].remove(closest) + # Locate the entry that actually contributed this point (first visible + # entry whose entry_points() contains it), so the removal is recorded + # against the right owner and survives a later recompute (spacing or + # subset-size change) instead of being silently undone by it -- see + # entry_points()/recompute_entry(). + entry = next((e for e in self.selections if e['visible'] and closest in self.entry_points(e)), None) + if entry is None: + return - # Remove from grid - for grid in self.grid_polygons: - if closest in grid['roi_points']: - grid['roi_points'].remove(closest) + if entry['kind'] == 'manual': + # Manual points are literal -- there is nothing to regenerate, so + # the point is removed from geometry outright instead of via + # `removed` (which stays empty for manual entries, see the class + # docstring / removed-point semantics notes). + entry['geometry'] = [p for p in entry['geometry'] if tuple(p) != closest] + self.recompute_entry(entry) + else: + entry['removed'].add(closest) - # Remove from brush points - if closest in self.brush_points: - self.brush_points.remove(closest) + self.update_selected_points() # also refreshes the owning row's "N pts" label - self.update_selected_points() - # Automatic filtering # Shi-Tomasi method def compute_candidate_points_shi_tomasi(self): """Compute good feature points using structure tensor analysis (Shi–Tomasi style).""" from scipy.ndimage import sobel - subset_size = self.subset_size_spinbox.value() - roi_size = subset_size // 2 + subset_h, subset_w = self.get_subset_size() + half_h = subset_h // 2 + half_w = subset_w // 2 img = self.image_item.image.astype(np.float32) candidates = [] + # img is column-major (pyqtgraph's default axisOrder): array axis 0 is + # the image's x/width axis, axis 1 is its y/height axis. # All selected points (not just manual) - for row, col in self.selected_points: - y, x = int(round(row)), int(round(col)) + for px, py in self.selected_points: + ix, iy = int(round(px)), int(round(py)) - if (y - roi_size < 0 or y + roi_size + 1 > img.shape[0] or - x - roi_size < 0 or x + roi_size + 1 > img.shape[1]): + if (ix - half_w < 0 or ix + half_w + 1 > img.shape[0] or + iy - half_h < 0 or iy + half_h + 1 > img.shape[1]): continue - roi = img[y - roi_size: y + roi_size + 1, - x - roi_size: x + roi_size + 1] + roi = img[ix - half_w: ix + half_w + 1, + iy - half_h: iy + half_h + 1] # Compute gradients gx = sobel(roi, axis=1) @@ -1164,10 +1919,14 @@ def compute_candidate_points_shi_tomasi(self): eigvals = np.linalg.eigvalsh(matrix) # sorted ascending min_eig = eigvals[0] - candidates.append((x + 0.0, y + 0.0, min_eig)) + # Stored (y, x, value): update_threshold_and_show_shi_tomsi (outside this + # function, unchanged) unpacks this as (x, y, e) and reverses it back to + # (x, y) -- keep that round-trip intact. + candidates.append((py + 0.0, px + 0.0, min_eig)) if not candidates: self.candidate_points = [] + self._candidate_refresh = None self.update_candidate_display() return @@ -1177,14 +1936,32 @@ def compute_candidate_points_shi_tomasi(self): self.candidates_shi_tomasi = candidates + self._candidate_refresh = self.update_threshold_and_show_shi_tomsi self.update_threshold_and_show_shi_tomsi() + def thresholded_candidates(self, cached, threshold): + """Turn cached filter scores into the points to show as candidates. + + :param cached: per-subset ``(y, x, score)`` tuples, as stored by the + ``compute_candidate_points_*`` methods + :type cached: list + :param threshold: keep only the subsets scoring strictly above this + :type threshold: float + :return: the surviving subsets, as rounded ``(x, y)`` tuples + :rtype: list + """ + selected = {(int(round(px)), int(round(py))) for px, py in self.selected_points} + points = [(round(y), round(x)) for (x, y, score) in cached if score > threshold] + # A subset deselected since the filter ran is dropped however well it scores. + # `cached` itself is left alone, so re-selecting it brings the candidate back. + return [p for p in points if p in selected] + def update_threshold_and_show_shi_tomsi(self): threshold_ratio = self.threshold_slider.value() / 1000.0 eig_threshold = self.max_eig_shi_tomasi * threshold_ratio - self.candidate_points = [(round(y), round(x)) for (x, y, e) in self.candidates_shi_tomasi if e > eig_threshold] + self.candidate_points = self.thresholded_candidates(self.candidates_shi_tomasi, eig_threshold) self.update_candidate_display() self.update_candidate_points_count() @@ -1199,15 +1976,6 @@ def update_candidate_points_count(self): def update_candidate_display(self): """Show candidate points as scatter dots on the image.""" - if not hasattr(self, 'candidate_scatter'): - self.candidate_scatter = ScatterPlotItem( - pen=pg.mkPen(None), - brush=pg.mkBrush(0, 255, 0, 150), # green with transparency - size=6, - symbol='o' - ) - self.view.addItem(self.candidate_scatter) - if self.candidate_points: self.candidate_scatter.setData(pos=np.array(self.candidate_points) + 0.5) else: @@ -1217,6 +1985,9 @@ def clear_candidates(self): """Clear candidate points.""" # print("Clearing candidate points...") self.candidate_points = [] + # Otherwise update_selected_points(), below, would re-derive them from the + # still-cached scores and undo the clear. + self._candidate_refresh = None self.update_candidate_points_count() if hasattr(self, 'candidate_scatter'): self.candidate_scatter.clear() @@ -1262,21 +2033,24 @@ def compute_candidate_points_gradient_direction(self): return dy, dx = self.gradient_direction - subset_size = self.subset_size_spinbox.value() - roi_size = subset_size // 2 + subset_h, subset_w = self.get_subset_size() + half_h = subset_h // 2 + half_w = subset_w // 2 img = self.image_item.image.astype(np.float32) candidates = [] - for row, col in self.selected_points: - y, x = int(round(row)), int(round(col)) + # img is column-major (pyqtgraph's default axisOrder): array axis 0 is + # the image's x/width axis, axis 1 is its y/height axis. + for px, py in self.selected_points: + ix, iy = int(round(px)), int(round(py)) - if (y - roi_size < 0 or y + roi_size + 1 > img.shape[0] or - x - roi_size < 0 or x + roi_size + 1 > img.shape[1]): + if (ix - half_w < 0 or ix + half_w + 1 > img.shape[0] or + iy - half_h < 0 or iy + half_h + 1 > img.shape[1]): continue - roi = img[y - roi_size: y + roi_size + 1, - x - roi_size: x + roi_size + 1] + roi = img[ix - half_w: ix + half_w + 1, + iy - half_h: iy + half_h + 1] gx = sobel(roi, axis=1) gy = sobel(roi, axis=0) @@ -1284,27 +2058,28 @@ def compute_candidate_points_gradient_direction(self): gdir = np.abs(gx * dx) + np.abs(gy * dy) strength = np.sum(np.abs(gdir)) - candidates.append((x + 0.0, y + 0.0, strength)) + # Stored (y, x, value): update_threshold_and_show_gradient_direction + # (outside this function, unchanged) unpacks this as (x, y, v) and + # reverses it back to (x, y) -- keep that round-trip intact. + candidates.append((py + 0.0, px + 0.0, strength)) if not candidates: self.candidate_points = [] + self._candidate_refresh = None self.update_candidate_display() return values = np.array([v[2] for v in candidates]) self.max_grad_dir = np.max(values) self.candidates_grad_dir = candidates + self._candidate_refresh = self.update_threshold_and_show_gradient_direction self.update_threshold_and_show_gradient_direction() def update_threshold_and_show_gradient_direction(self): threshold_ratio = self.gradient_thresh_slider.value() / 100.0 threshold = self.max_grad_dir * threshold_ratio - self.candidate_points = [ - (round(y), round(x)) - for (x, y, v) in self.candidates_grad_dir - if v > threshold - ] + self.candidate_points = self.thresholded_candidates(self.candidates_grad_dir, threshold) self.update_candidate_display() self.update_candidate_points_count() @@ -1328,7 +2103,7 @@ def handle_brush_move(self, ev): if self._paint_mask is None: return - pos = ev.pos() + pos = ev.scenePos() if self.view.sceneBoundingRect().contains(pos): mouse_point = self.view.mapSceneToView(pos) y, x = int(round(mouse_point.x())), int(round(mouse_point.y())) @@ -1347,60 +2122,75 @@ def handle_brush_end(self, ev): if self._paint_mask is None: return - subset_size = self.subset_size_spinbox.value() - spacing = self.distance_spinbox.value() - - # Generate (row, col) points inside the painted mask - brush_rois = rois_inside_mask(self._paint_mask, subset_size, spacing) - if self.brush_deselect_mode: - def point_inside_mask(pt, mask): - y, x = int(round(pt[0])), int(round(pt[1])) - h, w = mask.shape - return 0 <= y < h and 0 <= x < w and mask[y, x] - - # Remove from manual points - self.manual_points = [ - pt for pt in self.manual_points - if not point_inside_mask(pt, self._paint_mask) - ] - - # Remove from polygons - for poly in self.drawing_polygons: - poly['roi_points'] = [pt for pt in poly['roi_points'] if not point_inside_mask(pt, self._paint_mask)] - - # Remove from grid polygons - for grid in self.grid_polygons: - grid['roi_points'] = [pt for pt in grid['roi_points'] if not point_inside_mask(pt, self._paint_mask)] - - # Remove from brush points - self.brush_points = [ - pt for pt in self.brush_points - if not point_inside_mask(pt, self._paint_mask) - ] - - # Remove brush masks that are covered by the current mask - self.brush_masks = [mask for mask in self.brush_masks - if not np.any(mask & self._paint_mask)] - + self._apply_brush_deselect() self.brush_deselect_mode = False self.brush_deselect_button.setChecked(False) - else: - # Store the mask for future recomputation - self.brush_masks.append(self._paint_mask.copy()) - # Add points to brush_points - self.brush_points.extend(brush_rois) + # One entry per stroke. + entry = self.add_selection('brush', geometry=self._paint_mask.copy()) + self.recompute_entry(entry) self._paint_mask = None + self.update_geometry_display() self.update_selected_points() self.update_brush_overlay() - def update_brush_overlay(self): - if not hasattr(self, 'brush_overlay'): - self.brush_overlay = ImageItem() - self.view.addItem(self.brush_overlay) + def _apply_brush_deselect(self): + """Remove every point covered by the current deselect-mode brush stroke. + For a ``brush`` entry the stroke is subtracted from the painted mask itself, + so only the overlapping area is lost and the rest of the stroke survives; the + entry is dropped (and its row removed from ``selection_list``) only once + nothing is left painted. Editing the mask rather than the derived points is + what makes the deselection outlast a recompute. + + For every other kind the covered points are recorded in the entry's + ``removed`` set (``manual`` excepted -- see below), which ``entry_points()`` + applies on read, so those deselections survive a spacing/subset-size change + too. + """ + def point_inside_mask(pt, mask): + y, x = int(round(pt[0])), int(round(pt[1])) + h, w = mask.shape + return 0 <= y < h and 0 <= x < w and mask[y, x] + + active = self.active_entry() + rows_to_delete = [] + for row, entry in enumerate(self.selections): + if entry['kind'] == 'brush': + entry['geometry'] = entry['geometry'] & ~self._paint_mask + if not entry['geometry'].any(): + rows_to_delete.append(row) + else: + self.recompute_entry(entry) + continue + covered = [tuple(pt) for pt in self.entry_points(entry) if point_inside_mask(pt, self._paint_mask)] + if entry['kind'] == 'manual': + # Manual points are literal: drop them from geometry outright, the same + # way handle_remove_point does. Recording them in `removed` instead would + # make a later click on the very same pixel silently do nothing. + covered_set = set(covered) + entry['geometry'] = [p for p in entry['geometry'] if tuple(p) not in covered_set] + self.recompute_entry(entry) + else: + entry['removed'].update(covered) + + for row in reversed(rows_to_delete): + del self.selections[row] + self.selection_list.takeItem(row) + + # Deleting rows shifts every later index, so re-derive the active one from the + # entry object rather than leaving a stale (possibly out-of-range) index behind. + # Matched by identity: `==` on entry dicts compares their values, which raises + # on a brush entry's numpy mask ("truth value of an array is ambiguous"). + self.active_index = next((i for i, e in enumerate(self.selections) if e is active), None) + if self.active_index is not None: + self._syncing_list = True + self.selection_list.setCurrentRow(self.active_index) + self._syncing_list = False + + def update_brush_overlay(self): if self._paint_mask is not None: rgba = np.zeros((*self._paint_mask.shape, 4), dtype=np.uint8) if self.brush_deselect_mode: @@ -1486,70 +2276,6 @@ def set_y_direction_preset(self): self.show_instruction("Y (vertical) direction preset applied.") -def points_along_polygon(polygon, subset_size, spacing=0): - if len(polygon) < 2: - return [] - - step = subset_size + spacing - if step <= 0: - step = 1 - - result_points = [] - - for i in range(len(polygon) - 1): - p1 = np.array(polygon[i]) - p2 = np.array(polygon[i + 1]) - segment = p2 - p1 - length = np.linalg.norm(segment) - - if length == 0: - continue - - direction = segment / length - n_points = int(length // step) - - for j in range(n_points + 1): - pt = p1 + j * step * direction - result_points.append((round(pt[0] - 0.5), round(pt[1] - 0.5))) - - return result_points - -def rois_inside_polygon(polygon, subset_size, spacing): - if len(polygon) < 3: - return [] - - polygon = np.array(polygon) - min_x, max_x = int(np.floor(np.min(polygon[:, 0]))), int(np.ceil(np.max(polygon[:, 0]))) - min_y, max_y = int(np.floor(np.min(polygon[:, 1]))), int(np.ceil(np.max(polygon[:, 1]))) - - step = subset_size + spacing - if step <= 0: - step = 1 # minimum step to avoid infinite loop - xs = np.arange(min_x, max_x+1, step) - ys = np.arange(min_y, max_y+1, step) - - grid_x, grid_y = np.meshgrid(xs, ys) - points = np.vstack([grid_x.ravel(), grid_y.ravel()]).T - - mask = Path(polygon).contains_points(points) - return [tuple(p) for p in points[mask]] - -def rois_inside_mask(mask, subset_size, spacing): - step = subset_size + spacing - if step <= 0: - step = 1 - - h, w = mask.shape - xs = np.arange(0, w, step) - ys = np.arange(0, h, step) - grid_x, grid_y = np.meshgrid(xs, ys) - - candidate_points = np.vstack([grid_y.ravel(), grid_x.ravel()]).T # (y, x) - - # Only keep points where the mask is True - selected = [tuple(p) for p in candidate_points if mask[p[0], p[1]]] - return selected - if __name__ == "__main__": # import pyidi # filename = "data/data_showcase.cih" @@ -1561,7 +2287,6 @@ def rois_inside_mask(mask, subset_size, spacing): from PIL import Image import io import numpy as np - import matplotlib.pyplot as plt # Example black and white image (public domain) url = "https://raw.githubusercontent.com/scikit-image/scikit-image/main/skimage/data/camera.png" # Fetch the image @@ -1571,6 +2296,6 @@ def rois_inside_mask(mask, subset_size, spacing): example_image = (np.array(img).T)[:, ::-1] - Points = SelectionGUI(example_image.astype(np.uint8)) + Points = SelectionGUIOld(example_image.astype(np.uint8)) print(Points.get_points()) # # print selected points for testing diff --git a/pyidi/__init__.py b/pyidi/__init__.py index 2417506..63f28ef 100644 --- a/pyidi/__init__.py +++ b/pyidi/__init__.py @@ -22,13 +22,13 @@ # counts, as long as the thread pool has not been started yet. _sys.modules['numba'].config.THREADING_LAYER = 'forksafe' -# from .pyidi import * from .pyidi_legacy import pyIDI from . import tools from . import postprocessing from . import datasets from .load_analysis import load_analysis from .video_reader import VideoReader +from . import selection from .methods import * from .GUIs import * from .fiducial import * diff --git a/pyidi/load_analysis copy.py b/pyidi/load_analysis copy.py deleted file mode 100644 index bf34880..0000000 --- a/pyidi/load_analysis copy.py +++ /dev/null @@ -1,58 +0,0 @@ -import os -import json -import pickle -import warnings - -from .methods import LucasKanade, SimplifiedOpticalFlow, DirectionalLucasKanade, IDIMethod -from .video_reader import VideoReader - -method_mappings = { - "LucasKanade": LucasKanade, - "SimplifiedOpticalFlow": SimplifiedOpticalFlow, - "DirectionalLucasKanade": DirectionalLucasKanade, -} - -def load_analysis(analysis_path, input_file=None, load_results=True, root=None): - """Load the previous analysis and create a pyIDI object. - - :param analysis_path: Path to analysis folder (e.g. video_pyidi_analysis/analysis_001/) - :type analysis_path: str - :param input_file: new location of the cih file, if None, the location in settings.txt - is used, defaults to None - :type input_file: str or None, optional - :param load_results: if False, the displacements are not loaded, - only points and settings, defaults to True - :type load_results: bool, optional - :param root: root directory for the analysis (needed when the ``VideoReader`` requires it), - defaults to None. - :type root: str or None, optional - :return: pyIDI object and settings dict - :rtype: tuple - """ - with open(os.path.join(analysis_path, 'settings.json'), 'r') as f: - settings = json.load(f) - - if input_file is None: - video = VideoReader(settings['input_file'], root=root) - else: - video = VideoReader(input_file, root=root) - - method_name = settings['method'] - if method_name not in method_mappings: - raise ValueError(f"Method {method_name} not one of {list(method_mappings.keys())}") - - idi: IDIMethod = method_mappings[method_name](video) - - with open(os.path.join(analysis_path, 'points.pkl'), 'rb') as f: - points = pickle.load(f) - - if load_results: - with open(os.path.join(analysis_path, 'results.pkl'), 'rb') as f: - results = pickle.load(f) - - idi.displacements = results - - idi.set_points(points) - - return video, idi, settings['settings'] - diff --git a/pyidi/methods/_simplified_optical_flow.py b/pyidi/methods/_simplified_optical_flow.py index 848f395..25e8853 100644 --- a/pyidi/methods/_simplified_optical_flow.py +++ b/pyidi/methods/_simplified_optical_flow.py @@ -1,12 +1,7 @@ import numpy as np -import matplotlib.pyplot as plt -import matplotlib.patches as patches from scipy.signal import convolve2d from tqdm import tqdm -import tkinter as tk -from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk -from matplotlib.figure import Figure try: from qtpy.QtWidgets import QApplication except ImportError: @@ -225,175 +220,3 @@ def subset(self, data, subset_size): subset_image.append(subset_roll) return np.sum(np.asarray(subset_image), axis=0) - - # @staticmethod - # def get_points(video, **kwargs): - # """Determine the points. - # """ - # options = { - # 'subset': (20, 20), - # 'axis': 0, - # 'min_grad': 0., - # } - - # # # Change the docstring in `set_points` to show the options - # # docstring = video.set_points.__doc__.split('---') - # # docstring[1] = '- ' + '\n\t- '.join(options) + '\n\t' - # # video.set_points.__func__.__doc__ = '---\n\t'.join(docstring) - - # options.update(kwargs) - - # if isinstance(options['subset'], int): - # options['subset'] = 2*(options['subset'], ) - # elif type(options['subset']) not in [list, tuple]: - # raise Exception( - # f'keyword argument "subset" must be int, list or tuple (not {type(options["subset"])})') - - # polygon = PickPoints( - # video, subset=options['subset'], axis=options['axis'], min_grad=options['min_grad']) - - -class PickPoints: - """Pick the area of interest. - - Select the points with highest gradient in vertical direction. - """ - - def __init__(self, video, subset, axis, min_grad): - self.subset = subset - self.axis = axis - self.min_grad = min_grad - - image = video.get_frame(0) - self.gradient_0, self.gradient_1 = np.gradient(image.astype(float)) - - root = tk.Tk() # Tkinter - root.title('Pick points') # Tkinter - fig = Figure(figsize=(15, 7)) # Tkinter - ax = fig.add_subplot(111) # Tkinter - ax.grid(False) - ax.imshow(image, cmap='gray') - - self.polygon = [[], []] - line, = ax.plot(self.polygon[1], self.polygon[0], 'r.-') - - print('SHIFT + LEFT mouse button to pick a pole.\nRIGHT mouse button to erase the last pick.') - - self.shift_is_held = False - - def on_key_press(event): - """Function triggered on key press (shift).""" - if event.key == 'shift': - self.shift_is_held = True - - def on_key_release(event): - """Function triggered on key release (shift).""" - if event.key == 'shift': - self.shift_is_held = False - - def onclick(event): - if event.button == 1 and self.shift_is_held: - if event.xdata is not None and event.ydata is not None: - self.polygon[1].append(int(np.round(event.xdata))) - self.polygon[0].append(int(np.round(event.ydata))) - print( - f'y: {np.round(event.ydata):5.0f}, x: {np.round(event.xdata):5.0f}') - elif event.button == 3 and self.shift_is_held: - print('Deleted the last point...') - del self.polygon[1][-1] - del self.polygon[0][-1] - - line.set_xdata(self.polygon[1]) - line.set_ydata(self.polygon[0]) - fig.canvas.draw() - - def handle_close(event): - """On closing.""" - self.polygon = np.asarray(self.polygon).T - for i, point in enumerate(self.polygon): - print(f'{i+1}. point: x ={point[1]:5.0f}, y ={point[0]:5.0f}') - - # Add points to video object - video.points = self.observed_pixels() - video.polygon = self.polygon - - canvas = FigureCanvasTkAgg(fig, root) # Tkinter - canvas.get_tk_widget().pack(side='top', fill='both', expand=1) # Tkinter - NavigationToolbar2Tk(canvas, root) # Tkinter - - # Connecting functions to event manager - fig.canvas.mpl_connect('key_press_event', on_key_press) - fig.canvas.mpl_connect('key_release_event', on_key_release) - fig.canvas.mpl_connect('button_press_event', onclick) - # on closing the figure - fig.canvas.mpl_connect('close_event', handle_close) - - root.mainloop() - - def observed_pixels(self): - x = self.polygon[:, 1] - y = self.polygon[:, 0] - - _polygon = np.asarray([x, y]).T - - x_low = min(x) - x_high = max(x) - y_low = min(y) - y_high = max(y) - - # Get only the points in selected polygon - inside = [] - for x_ in range(x_low, x_high): - for y_ in range(y_low, y_high): - if self.inside_polygon(x_, y_, _polygon) is True: - inside.append([y_, x_]) # Change indices (height, width) - inside = np.asarray(inside) # Indices of points in the polygon - - # Points outside the polygon have gradient of value 0 - g0 = np.zeros_like(self.gradient_0) - g1 = np.zeros_like(self.gradient_1) - g0[inside[:, 0], inside[:, 1]] = self.gradient_0[inside[:, 0], inside[:, 1]] - g1[inside[:, 0], inside[:, 1]] = self.gradient_1[inside[:, 0], inside[:, 1]] - - if self.axis == 0: - g = g0 - elif self.axis == 1: - g = g1 - elif self.axis is None: - g = g0**2 + g1**2 - else: - raise Exception( - f'axis value {self.axis} is not valid. Please pick 0, 1 or None') - - indices = [] - for i in range(y_low, y_high, self.subset[0]): - for j in range(x_low, x_high, self.subset[1]): - _g = g[i:i+self.subset[0], j:j+self.subset[1]] - _ = np.argmax(np.abs(_g)) - ind = np.unravel_index(_, _g.shape) - if np.abs(_g[ind[0], ind[1]]) > np.max(np.abs(g))*self.min_grad: - indices.append([i+ind[0], j+ind[1]]) - - return np.asarray(indices) - - def inside_polygon(self, x, y, points): - """Return True if a coordinate (x, y) is inside a polygon defined by - a list of verticies [(x1, y1), (x2, x2), ... , (xN, yN)]. - - Reference: http://www.ariel.com.au/a/python-point-int-poly.html - """ - n = len(points) - inside = False - p1x, p1y = points[0] - for i in range(1, n + 1): - p2x, p2y = points[i % n] - if y > min(p1y, p2y): - if y <= max(p1y, p2y): - if x <= max(p1x, p2x): - if p1y != p2y: - xinters = (y - p1y) * (p2x - p1x) / \ - (p2y - p1y) + p1x - if p1x == p2x or x <= xinters: - inside = not inside - p1x, p1y = p2x, p2y - return inside diff --git a/pyidi/methods/idi_method.py b/pyidi/methods/idi_method.py index 88b3906..2ad625b 100644 --- a/pyidi/methods/idi_method.py +++ b/pyidi/methods/idi_method.py @@ -6,6 +6,7 @@ import glob import shutil import inspect +import warnings import matplotlib.pyplot as plt from ..video_reader import VideoReader @@ -269,17 +270,69 @@ def _make_comparison_dict(self): return settings def set_points(self, points): - from ..GUIs.selection import SubsetSelection - - if isinstance(points, list): - points = np.array(points) - elif isinstance(points, SubsetSelection): + """ + Set the points at which the displacements will be computed. + + Accepts a plain array-like of points, or a selection GUI object that + exposes a ``.points`` attribute/property (e.g. :class:`~pyidi.SelectionGUI`) + - duck-typed so this method does not need to import the GUI, and therefore + does not drag the optional Qt dependency into this code path. + + Points are given as row/column (y/x) image coordinates: ``points[:, 0]`` + is the row (y) coordinate and ``points[:, 1]`` is the column (x) + coordinate. + + :param points: Points to be set, as an array-like of shape ``(n_points, 2)``, + or an object with a ``.points`` attribute of that shape. + :type points: array_like or object + :raises ValueError: if `points` is empty, is not 2-dimensional, does not have + exactly 2 columns, or (when the video's image size is known) contains + coordinates outside of the image bounds. + """ + if hasattr(points, 'points'): points = np.array(points.points) + else: + points = np.array(points) + + if points.size == 0: + raise ValueError("Points must not be empty.") + + if points.ndim != 2: + raise ValueError( + f"Points must be a 2-dimensional array of shape (n_points, 2), got shape {points.shape}." + ) - points = np.array(points) if points.shape[1] != 2: - raise ValueError("Points must have two columns.") - + raise ValueError( + f"Points must have exactly two columns (row, column), got {points.shape[1]}." + ) + + if not np.issubdtype(points.dtype, np.integer): + rounded = np.rint(points) + n_changed = int(np.count_nonzero(np.any(rounded != points, axis=1))) + if n_changed: + warnings.warn( + f"{n_changed} of {points.shape[0]} points had non-integer (sub-pixel) " + "coordinates. They have been rounded to the nearest integer pixel." + ) + points = rounded.astype(np.int64) + + video = getattr(self, 'video', None) + if video is not None and hasattr(video, 'image_width') and hasattr(video, 'image_height'): + MAX_OFFENDERS_SHOWN = 5 + rows_ok = (points[:, 0] >= 0) & (points[:, 0] < video.image_height) + cols_ok = (points[:, 1] >= 0) & (points[:, 1] < video.image_width) + out_of_bounds = ~(rows_ok & cols_ok) + n_bad = int(np.count_nonzero(out_of_bounds)) + if n_bad: + bad = points[out_of_bounds][:MAX_OFFENDERS_SHOWN] + shown = min(MAX_OFFENDERS_SHOWN, n_bad) + raise ValueError( + f"{n_bad} of {points.shape[0]} points are outside the image bounds " + f"(0 <= row < {video.image_height}, 0 <= column < {video.image_width}). " + f"First offenders: {bad.tolist()} (showing {shown} of {n_bad})." + ) + self.points = points def show_points(self, figsize=(15, 5), cmap='gray', color='r'): diff --git a/pyidi/pyidi.py b/pyidi/pyidi.py deleted file mode 100644 index f6bc5d7..0000000 --- a/pyidi/pyidi.py +++ /dev/null @@ -1,280 +0,0 @@ -import os -import numpy as np -import matplotlib.pyplot as plt -import pickle -import datetime -import json -import glob -import warnings -warnings.simplefilter("default") - -from .methods import IDIMethod, SimplifiedOpticalFlow, LucasKanade, DirectionalLucasKanade #, LucasKanadeSc, LucasKanadeSc2, GradientBasedOpticalFlow -from .video_reader import VideoReader -from . import tools -from .GUIs import selection - -available_method_shortcuts = [ - ('sof', SimplifiedOpticalFlow), - ('lk', LucasKanade), - ('lk_1D', DirectionalLucasKanade) - # ('lk_scipy', LucasKanadeSc), - # ('lk_scipy2', LucasKanadeSc2) - # ('gb', GradientBasedOpticalFlow) - ] - - -class pyIDI(): - """ - The pyIDI base class represents the video to be analysed. - """ - def __init__(self, input_file, root=None): - """Constructor of the pyIDI class. - - .. versionremoved:: 1.0 - Since version 1.0 of pyIDI, this class is no longer used. Temporarily, the - ``pyIDI`` class from "pyidi_legacy.py" is available. - - - :param input_file: the video file to be analysed. Can be a name of the cih/cihx file, path to - images directory, video file, or a 3D numpy array. - :type input_file: str or np.ndarray - :param root: root directory of the video file. Only used when the input file is a np.ndarray. Defaults to None. - :type root: str - """ - raise NotImplementedError("This class has been removed in version 1.0. Temporarily, the pyIDI class from 'pyidi_legacy.py' \ - can be used (just import ``from pyidi import pyIDI``). However, the API has changed, \ - see the documentation for details.") - - if type(input_file) in [str, np.ndarray]: - self.reader = VideoReader(input_file, root=root) - self.cih_file = input_file - else: - raise ValueError('`input_file` must be either a image/video/cih filename or a 3D array (N_time, height, width)') - - self.available_methods = dict([ - (key, { - 'IDIMethod': method, - 'description': method.__doc__, - }) - for key, method in available_method_shortcuts - ]) - - # Fill available methods into `set_method` docstring - available_methods_doc = '\n' + '\n'.join([ - f"'{key}' ({method_dict['IDIMethod'].__name__}): {method_dict['description']}" - for key, method_dict in self.available_methods.items() - ]) - - tools.update_docstring(self.set_method, added_doc=available_methods_doc) - - - def set_method(self, method, **kwargs): - """ - Set displacement identification method on video. - To configure the method, use `method.configure()` - - Available methods: - --- - [Available method names and descriptions go here.] - --- - - :param method: the method to be used for displacement identification. - :type method: IDIMethod or str - """ - if isinstance(method, str) and method in self.available_methods.keys(): - self.method_name = method - self.method = self.available_methods[method]['IDIMethod'](self, **kwargs) - elif callable(method) and hasattr(method, 'calculate_displacements'): - self.method_name = 'external_method' - try: - self.method = method(self, **kwargs) - except: - raise ValueError("The input `method` is not a valid `IDIMethod`.") - else: - raise ValueError("method must either be a valid name from `available_methods` or an `IDIMethod`.") - - # Update `get_displacements` docstring - tools.update_docstring(self.get_displacements, self.method.calculate_displacements) - # Update `show_points` docstring - if hasattr(self.method, 'show_points'): - try: - tools.update_docstring(self.show_points, self.method.show_points) - except: - pass - - - def set_points(self, points=None, method=None, **kwargs): - """ - Set points that will be used to calculate displacements. - If `points` is None and a `method` has aready been set on this `pyIDI` instance, - the `method` object's `get_point` is used to get method-appropriate points. - """ - if points is None: - if not hasattr(self, 'method'): - if method is not None: - self.set_method(method) - else: - raise ValueError("Invalid arguments. Please input points, or set the IDI method first.") - self.method.get_points(self, **kwargs) # get_points sets the attribute video.points - else: - self.points = np.asarray(points) - - - def show_points(self, **kwargs): - """ - Show selected points on image. - """ - - if hasattr(self, 'method') and hasattr(self.method, 'show_points'): - self.method.show_points(self, **kwargs) - else: - figsize = kwargs.get('figsize', (15, 5)) - cmap = kwargs.get('cmap', 'gray') - marker = kwargs.get('marker', '.') - color = kwargs.get('color', 'r') - fig, ax = plt.subplots(figsize=figsize) - ax.imshow(self.reader.get_frame(0).astype(float), cmap=cmap) - ax.scatter(self.points[:, 1], self.points[:, 0], - marker=marker, color=color) - plt.grid(False) - plt.show() - - - def show_field(self, field, scale=1., width=0.5): - """ - Show displacement field on image. - - :param field: Field of displacements (number_of_points, 2) - :type field: ndarray - :param scale: scale the field, defaults to 1. - :param scale: float, optional - :param width: width of the arrow, defaults to 0.5 - :param width: float, optional - """ - max_L = np.max(field[:, 0]**2 + field[:, 1]**2) - - fig, ax = plt.subplots(1) - ax.imshow(self.reader.get_frame(0), 'gray') - for i, ind in enumerate(self.points): - f0 = field[i, 0] - f1 = field[i, 1] - alpha = (f0**2 + f1**2) / max_L - if alpha < 0.2: - alpha = 0.2 - plt.arrow(ind[1], ind[0], scale*f1, scale*f0, width=width, color='r', alpha=alpha) - - - def get_displacements(self, autosave=True, **kwargs): - """ - Calculate the displacements based on chosen method. - - Method docstring: - --- - Method is not set. Please use the `set_method` method. - --- - """ - if hasattr(self, 'method'): - self.method.calculate_displacements(self, **kwargs) - self.displacements = self.method.displacements - - # auto-save and clearing temp files - if hasattr(self.method, 'process_number'): - if self.method.process_number == 0: - - if autosave: - self.create_analysis_directory() - self.save(root=self.root_this_analysis) - - self.method.clear_temp_files() - - return self.displacements - else: - raise ValueError('IDI method has not yet been set. Please call `set_method()` first.') - - - def close_video(self): - """ - Close the .mraw video memmap. - """ - self.reader.close() - - - def create_analysis_directory(self): - if type(self.cih_file) == str: - cih_file_ = os.path.split(self.cih_file)[-1].split('.')[0] - else: - cih_file_ = 'ndarary_video' - self.root_analysis = os.path.join(self.reader.root, f'{cih_file_}_pyidi_analysis') - if not os.path.exists(self.root_analysis): - os.mkdir(self.root_analysis) - - analyses = glob.glob(os.path.join(self.root_analysis, 'analysis_*/')) - if analyses: - last_an = sorted(analyses)[-1] - print(last_an, last_an.split('\\')[-2]) - n = int(last_an.split('\\')[-2].split('_')[-1]) - else: - n = 0 - self.root_this_analysis = os.path.join(self.root_analysis, f'analysis_{n+1:0>3.0f}') - - os.mkdir(self.root_this_analysis) - - - def save(self, root=''): - with open(os.path.join(root, 'results.pkl'), 'wb') as f: - pickle.dump(self.displacements, f, protocol=-1) - with open(os.path.join(root, 'points.pkl'), 'wb') as f: - pickle.dump(self.points, f, protocol=-1) - - out = { - 'info': { - 'width': self.reader.image_width, - 'height': self.reader.image_height, - 'N': self.reader.N - }, - 'createdate': datetime.datetime.now().strftime("%Y %m %d %H:%M:%S"), - 'cih_file': self.cih_file if type(self.cih_file) == str else 'ndarray', - 'settings': self.method.create_settings_dict(), - 'method': self.method_name - } - - with open(os.path.join(root, 'settings.txt'), 'w') as f: - json.dump(out, f, sort_keys=True, indent=2) - - - def __repr__(self): - - rep = 'File name: ' + self.cih_file + ',\n' + \ - 'Image width: ' + str(self.image_width) + ',\n' + \ - 'Image height: ' + str(self.image_height) + ',\n' + \ - 'Total frame: ' + str(self.N) + ',\n' + \ - 'Record Rate(fps): ' + str(self.info['Record Rate(fps)']) - - if hasattr(self, 'method_name'): - rep +=',\n' + 'Method: ' + self.method_name - - if hasattr(self.method, 'subset_size'): - rep += ',\n' + 'Subset size: ' + str(self.method.subset_size) - - elif hasattr(self.method, 'roi_size'): - rep += ',\n' + 'ROI size: ' + str(self.method.roi_size) - - - if hasattr(self, 'points'): - rep +=',\n' + 'Number of points: ' + str(len(self.points)) - - return rep - - def gui(self): - from .GUIs import gui - self.gui_obj = gui.gui(self) - - @property - def mraw(self): - warnings.warn('`self.mraw` is deprecated and will be removed in the next version. Please use `self.reader.mraw` instead.', DeprecationWarning) - return self.reader.mraw - - @property - def info(self): - #warnings.warn('`self.info` is deprecated and will be removed in the next version. Please use `self.reader.info` instead.', DeprecationWarning) - return self.reader.info \ No newline at end of file diff --git a/pyidi/selection/__init__.py b/pyidi/selection/__init__.py new file mode 100644 index 0000000..a318b71 --- /dev/null +++ b/pyidi/selection/__init__.py @@ -0,0 +1,116 @@ +"""Automatic feature selection: mask, evaluate, select. + +Three steps, none of which needs a GUI: + +- **mask** (:mod:`~pyidi.selection.masks`) -- regions drawn on the image define + *where* points may go, and hand-picked entries name exact locations; +- **evaluate** (:mod:`~pyidi.selection.evaluate`) -- an evaluator scores every + pixel of the image at once, cached by name in + :mod:`~pyidi.selection.scores`; +- **select** (:mod:`~pyidi.selection.select`) -- a selector turns score plus + mask into points, with a threshold and a separation between them. + +The quick way in:: + + from pyidi.selection import Entry, select_points + + region = Entry('polygon', [(20, 20), (20, 200), (180, 200), (180, 20)]) + points = select_points(image, [region], subset_size=11, separation=15) + +and the stateful way, when scores should be reused across many parameter +changes:: + + from pyidi.selection import SelectionPipeline + + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('polygon', [(20, 20), (20, 200), (180, 200), (180, 20)]) + pipeline.selector_params['threshold'] = 95 + points = pipeline.points + +This module imports without PyQt6. The interactive interface built on it lives +in :mod:`pyidi.GUIs`. +""" + +from .evaluate import ( + Evaluator, + Parameter, + available_evaluators, + evaluate, + get_evaluator, + gradient_direction, + half_window, + register_evaluator, + shi_tomasi, + window_size, +) +from .masks import ( + DEFAULT_ROLE, + ROLES, + Entry, + all_literal_points, + apply_deselection, + combined_mask, + literal_points, + rasterize, +) +from .pipeline import DEFAULT_SELECTOR_PARAMS, PRETTY, SelectionPipeline, select_points +from .scores import ScoreSpec, ScoreStore +from .select import ( + DEFAULT_MAX_POINTS, + DEFAULT_SEPARATION, + DEFAULT_THRESHOLD, + ROBUST_MAXIMUM_PERCENTILE, + THRESHOLD_MODES, + SELECTORS, + as_point_array, + decimate, + merge_points, + occupancy, + select, + select_lattice, + select_peaks, + suppress, + threshold_value, +) + +__all__ = [ + 'DEFAULT_MAX_POINTS', + 'DEFAULT_SEPARATION', + 'DEFAULT_ROLE', + 'DEFAULT_SELECTOR_PARAMS', + 'DEFAULT_THRESHOLD', + 'ROBUST_MAXIMUM_PERCENTILE', + 'THRESHOLD_MODES', + 'Entry', + 'Evaluator', + 'PRETTY', + 'Parameter', + 'ROLES', + 'SELECTORS', + 'ScoreSpec', + 'ScoreStore', + 'SelectionPipeline', + 'all_literal_points', + 'apply_deselection', + 'as_point_array', + 'available_evaluators', + 'combined_mask', + 'decimate', + 'evaluate', + 'get_evaluator', + 'gradient_direction', + 'half_window', + 'literal_points', + 'merge_points', + 'occupancy', + 'rasterize', + 'register_evaluator', + 'select', + 'select_lattice', + 'select_peaks', + 'select_points', + 'shi_tomasi', + 'suppress', + 'threshold_value', + 'window_size', +] diff --git a/pyidi/selection/evaluate.py b/pyidi/selection/evaluate.py new file mode 100644 index 0000000..484fa55 --- /dev/null +++ b/pyidi/selection/evaluate.py @@ -0,0 +1,402 @@ +"""Whole-image evaluation of subset quality. + +An *evaluator* answers one question for every pixel of the image at once: how +well would a subset centred here track? The answer comes back as a *score +image* -- a ``float32`` array the shape of the input, with ``NaN`` wherever the +subset window would reach past the image edge. + +Everything here is vectorised over the whole image. The point-by-point +alternative (one Sobel and one 2x2 eigendecomposition per subset, as in +``pyidi/GUIs/subset_selection.py``) costs roughly 100 us per subset, which is +fine for a few hundred grid points and takes minutes at one megapixel. The +box-filter formulation below is a handful of separable O(1)-per-pixel passes, +so scoring every pixel of a megapixel frame is a matter of tens of +milliseconds. That is the whole reason a mask can go back to meaning "where I +want points" rather than "how much scoring I can afford". + +Coordinate convention: ``(row, col)`` throughout, i.e. plain numpy indexing. +The transpose that pyqtgraph's column-major image display needs belongs at the +GUI boundary, not here. + +Why ``NaN`` for the invalid border rather than a companion boolean array: every +comparison against ``NaN`` is already ``False``, so an invalid pixel can never +be picked without a single explicit check anywhere downstream, and +``np.nanmax``/``np.nanpercentile`` normalise correctly by construction. +""" + +from dataclasses import dataclass, field +from typing import Any, Callable, Optional, Tuple + +import numpy as np +from scipy.ndimage import sobel, uniform_filter + +from ..selection_geometry import _as_size_pair + +#: Radius, in pixels, of the gradient operator every evaluator here uses. The +#: score at a pixel therefore depends on the image up to ``half + this`` away, +#: which is what the bounding-box crop below has to pad by. +GRADIENT_RADIUS = 1 + +#: Evaluate the bounding box rather than the whole frame once the box is at +#: most this fraction of the frame. Below it the crop saves real time; above it +#: the bookkeeping costs more than the convolutions it avoids. +CROP_AREA_FRACTION = 0.25 + + +@dataclass(frozen=True) +class Parameter: + """A single evaluator parameter, described well enough to build a widget from. + + :param name: keyword name, as the evaluator function takes it + :type name: str + :param kind: ``'float'``, ``'int'`` or ``'direction'`` (a ``(row, col)`` + pair). A GUI switches on this to decide which control to create. + :type kind: str + :param default: value used when the caller does not supply one + :type default: object + :param minimum: lower bound, or ``None`` when unbounded + :type minimum: float or None + :param maximum: upper bound, or ``None`` when unbounded + :type maximum: float or None + :param description: one-line explanation, suitable for a tooltip + :type description: str + """ + + name: str + kind: str + default: Any + minimum: Optional[float] = None + maximum: Optional[float] = None + description: str = '' + + +@dataclass(frozen=True) +class Evaluator: + """A registered evaluator: the function plus everything needed to drive it. + + :param name: registry key, e.g. ``'shi_tomasi'`` + :type name: str + :param display_name: human-readable name for a menu + :type display_name: str + :param function: ``f(image, window, **params) -> ndarray``, where ``window`` + is the ``(rows, cols)`` scoring window. It scores the whole array and + need not care about the border -- :func:`evaluate` masks that off. + :type function: callable + :param parameters: descriptors for every keyword the function takes beyond + ``image`` and ``window`` + :type parameters: tuple[Parameter, ...] + :param description: one-line explanation of what the score means + :type description: str + """ + + name: str + display_name: str + function: Callable + parameters: Tuple[Parameter, ...] = field(default_factory=tuple) + description: str = '' + + +_REGISTRY = {} + + +def register_evaluator(evaluator): + """Add an evaluator to the registry, replacing any existing one of that name. + + :param evaluator: the evaluator to register + :type evaluator: Evaluator + :return: the evaluator, so this can be used as a decorator-ish one-liner + :rtype: Evaluator + """ + _REGISTRY[evaluator.name] = evaluator + return evaluator + + +def available_evaluators(): + """Every registered evaluator, keyed by name. + + :return: a copy of the registry, safe to iterate while registering + :rtype: dict[str, Evaluator] + """ + return dict(_REGISTRY) + + +def get_evaluator(name): + """Look up a registered evaluator by name. + + :param name: registry key + :type name: str + :return: the evaluator + :rtype: Evaluator + :raises ValueError: if no evaluator of that name is registered + """ + if name not in _REGISTRY: + known = ', '.join(sorted(_REGISTRY)) or '(none registered)' + raise ValueError(f"Unknown evaluator {name!r}. Registered evaluators: {known}.") + return _REGISTRY[name] + + +def resolve_parameters(evaluator, params): + """Fill in an evaluator's defaults and reject anything it does not accept. + + :param evaluator: the evaluator whose descriptors define the accepted keys + :type evaluator: Evaluator + :param params: caller-supplied parameters, possibly partial + :type params: dict + :return: every parameter the evaluator takes, defaults filled in + :rtype: dict + :raises ValueError: if ``params`` contains a key the evaluator does not take + """ + accepted = {p.name: p.default for p in evaluator.parameters} + unknown = set(params) - set(accepted) + if unknown: + known = ', '.join(sorted(accepted)) or '(none)' + raise ValueError( + f"Evaluator {evaluator.name!r} does not take {sorted(unknown)}. Parameters: {known}." + ) + accepted.update(params) + return accepted + + +def window_size(subset_size): + """The scoring window, per axis, for a subset size. + + Always odd, so the window is symmetric about the pixel it scores: an even + ``subset_size`` of 10 gives an 11-pixel window, exactly reproducing the + ``img[c - half : c + half + 1]`` slicing used for subsets elsewhere in the + package. An even-width box filter would instead sit half a pixel off centre. + + :param subset_size: scalar, or a ``(height, width)`` pair + :type subset_size: int or tuple + :return: ``(rows, cols)`` window extent, both odd + :rtype: tuple[int, int] + """ + h, w = _as_size_pair(subset_size) + return 2 * (int(h) // 2) + 1, 2 * (int(w) // 2) + 1 + + +def half_window(subset_size): + """Half the scoring window, per axis -- the depth of the invalid border. + + :param subset_size: scalar, or a ``(height, width)`` pair + :type subset_size: int or tuple + :return: ``(rows, cols)`` half-extent + :rtype: tuple[int, int] + """ + win_r, win_c = window_size(subset_size) + return win_r // 2, win_c // 2 + + +def _gradients(image): + """Sobel gradients of the whole image, as ``(d/drow, d/dcol)`` in float64. + + float64 rather than float32 because the box sums below reach ~1e13 for a + 16-bit image and an 11x11 window, which float32 cannot hold to the precision + the equivalence test against the per-subset reference asks for. The returned + score is narrowed back to float32. + """ + img = np.asarray(image, dtype=np.float64) + return sobel(img, axis=0), sobel(img, axis=1) + + +def _box_sum(values, window): + """Sum of ``values`` over ``window``, via the O(1)-per-pixel mean.""" + return uniform_filter(values, size=window, mode='constant') * (window[0] * window[1]) + + +def shi_tomasi(image, window): + """Smaller eigenvalue of the gradient structure tensor summed over the window. + + The Shi-Tomasi corner criterion: high where the image content inside the + subset constrains motion in *both* directions, so a subset on a plain edge + scores low (it can slide along the edge) and one on a corner scores high. + + Computed in closed form rather than through ``eigvalsh``, which needs a + Python-level loop:: + + lambda_min = (a + c)/2 - sqrt(((a - c)/2)**2 + b**2) + + with ``a = sum(gx**2)``, ``c = sum(gy**2)``, ``b = sum(gx*gy)``. The sums are + true sums over the window, not means, so the values are directly comparable + with the per-subset implementation in ``SelectionGUIOld``. + + :param image: 2-D image, indexed ``[row, col]`` + :type image: numpy.ndarray + :param window: ``(rows, cols)`` scoring window + :type window: tuple[int, int] + :return: score over the whole array, border included and meaningless + :rtype: numpy.ndarray + """ + g_row, g_col = _gradients(image) + a = _box_sum(g_col * g_col, window) + c = _box_sum(g_row * g_row, window) + b = _box_sum(g_col * g_row, window) + + half_trace = 0.5 * (a + c) + spread = np.sqrt(np.square(0.5 * (a - c)) + np.square(b)) + # The structure tensor is positive semi-definite, so the smaller eigenvalue + # is non-negative; clip away the rounding noise that would otherwise leave a + # flat region at -1e-20 instead of exactly zero. + return np.maximum(half_trace - spread, 0.0) + + +def gradient_direction(image, window, direction=(0.0, 1.0)): + """Summed squared image gradient projected onto one direction. + + Answers a narrower question than :func:`shi_tomasi`: not "can this subset be + tracked at all" but "can it be tracked *along this axis*". Useful when only + one component of the motion matters, e.g. a beam bending in one plane. + + :param image: 2-D image, indexed ``[row, col]`` + :type image: numpy.ndarray + :param window: ``(rows, cols)`` scoring window + :type window: tuple[int, int] + :param direction: ``(row, col)`` direction to project onto; normalised + internally, so only its orientation matters + :type direction: tuple[float, float] + :return: score over the whole array, border included and meaningless + :rtype: numpy.ndarray + :raises ValueError: if ``direction`` has zero length + """ + d = np.asarray(direction, dtype=np.float64).ravel() + if d.size != 2: + raise ValueError(f'direction must be a (row, col) pair, got {len(d)} values.') + norm = np.hypot(d[0], d[1]) + if norm == 0: + raise ValueError('direction must be non-zero.') + d = d / norm + + g_row, g_col = _gradients(image) + projected = d[0] * g_row + d[1] * g_col + return _box_sum(projected * projected, window) + + +register_evaluator(Evaluator( + name='shi_tomasi', + display_name='Shi-Tomasi', + function=shi_tomasi, + parameters=(), + description='Corner strength: high where the subset is constrained in both directions.', +)) + +register_evaluator(Evaluator( + name='gradient_direction', + display_name='Gradient in direction', + function=gradient_direction, + parameters=( + Parameter( + name='direction', + kind='direction', + default=(0.0, 1.0), + description='(row, col) direction the gradient is projected onto.', + ), + ), + description='Gradient strength along one chosen direction.', +)) + + +def _bounding_box(mask): + """Inclusive-exclusive ``(r0, r1, c0, c1)`` bounding box of a boolean mask. + + :return: the box, or ``None`` if the mask is empty + :rtype: tuple or None + """ + rows = np.flatnonzero(mask.any(axis=1)) + cols = np.flatnonzero(mask.any(axis=0)) + if not rows.size or not cols.size: + return None + return int(rows[0]), int(rows[-1]) + 1, int(cols[0]), int(cols[-1]) + 1 + + +def _evaluation_box(shape, mask, crop, half): + """Decide which slice of the image to run the evaluator on. + + Returns the *padded* box to evaluate together with the sub-box whose scores + that padding makes trustworthy. The padding is ``half + GRADIENT_RADIUS``: + the score at a pixel reads the gradient up to ``half`` away, and each + gradient reads the image one further. + + The crop follows the mask's bounding *box*, never its shape. Handing the + evaluator a mask-shaped image would let the mask boundary act as an image + edge and manufacture a strong gradient all along it, and it would force a + recompute on every brush stroke. + + :return: ``((r0, r1, c0, c1), (vr0, vr1, vc0, vc1))`` -- the box to evaluate + and the globally-valid box within it -- or ``None`` when nothing is + worth evaluating + :rtype: tuple or None + """ + h, w = shape + half_r, half_c = half + + box = None + if crop is not False and mask is not None: + bbox = _bounding_box(mask) + if bbox is None: + return None + auto = (bbox[1] - bbox[0]) * (bbox[3] - bbox[2]) <= CROP_AREA_FRACTION * h * w + if crop is True or auto: + box = bbox + + if box is None: + r0, r1, c0, c1 = 0, h, 0, w + else: + pad_r, pad_c = half_r + GRADIENT_RADIUS, half_c + GRADIENT_RADIUS + r0, r1 = max(0, box[0] - pad_r), min(h, box[1] + pad_r) + c0, c1 = max(0, box[2] - pad_c), min(w, box[3] + pad_c) + + # A score is trustworthy where the evaluated slice supplied every pixel it + # depends on. Where the slice edge *is* the image edge the slice saw exactly + # what a full-image evaluation would have, so only the subset-window rule + # applies there. + valid_r0 = max(half_r, r0 + half_r + GRADIENT_RADIUS if r0 > 0 else half_r) + valid_r1 = min(h - half_r, r1 - half_r - GRADIENT_RADIUS if r1 < h else h - half_r) + valid_c0 = max(half_c, c0 + half_c + GRADIENT_RADIUS if c0 > 0 else half_c) + valid_c1 = min(w - half_c, c1 - half_c - GRADIENT_RADIUS if c1 < w else w - half_c) + + if valid_r0 >= valid_r1 or valid_c0 >= valid_c1: + return None + return (r0, r1, c0, c1), (valid_r0, valid_r1, valid_c0, valid_c1) + + +def evaluate(image, evaluator='shi_tomasi', subset_size=11, mask=None, crop=None, **params): + """Score every subset position in the image. + + :param image: 2-D image, indexed ``[row, col]`` + :type image: numpy.ndarray + :param evaluator: registry name of the evaluator to run + :type evaluator: str + :param subset_size: scalar, or a ``(height, width)`` pair + :type subset_size: int or tuple + :param mask: boolean array the shape of ``image``; only used to decide the + bounding-box crop, never to restrict which pixels are scored inside it + :type mask: numpy.ndarray or None + :param crop: ``True`` to force the bounding-box crop, ``False`` to forbid it, + ``None`` (default) to crop when the box is at most + :data:`CROP_AREA_FRACTION` of the frame + :type crop: bool or None + :param params: evaluator-specific parameters; see its descriptors + :return: ``float32`` score image the shape of ``image``, ``NaN`` wherever the + subset window would leave the image or the crop leaves the score unknown + :rtype: numpy.ndarray + :raises ValueError: if ``image`` is not 2-D, the evaluator is unknown, or a + parameter is not one the evaluator takes + """ + image = np.asarray(image) + if image.ndim != 2: + raise ValueError(f'image must be 2-D (row, col), got shape {image.shape}.') + + spec = get_evaluator(evaluator) + resolved = resolve_parameters(spec, params) + window = window_size(subset_size) + half = (window[0] // 2, window[1] // 2) + + score = np.full(image.shape, np.nan, dtype=np.float32) + + boxes = _evaluation_box(image.shape, mask, crop, half) + if boxes is None: + return score + + (r0, r1, c0, c1), (vr0, vr1, vc0, vc1) = boxes + raw = spec.function(image[r0:r1, c0:c1], window, **resolved) + score[vr0:vr1, vc0:vc1] = raw[vr0 - r0:vr1 - r0, vc0 - c0:vc1 - c0] + return score diff --git a/pyidi/selection/masks.py b/pyidi/selection/masks.py new file mode 100644 index 0000000..c05ee3f --- /dev/null +++ b/pyidi/selection/masks.py @@ -0,0 +1,365 @@ +"""Selection entries, and the masks and literal points they contribute. + +The central idea of this package: a region drawn on the image defines an *area*, +not a set of points. Where the points go is decided later, by the selection +step, from the score image. That is what lets a filter find features the region +never sampled -- the failure mode of a regular grid on a random speckle pattern. + +Not every entry wants that treatment, though. A hand-clicked point or a line of +points placed by eye is a statement about exactly where a subset belongs, and +running it through a threshold would be perverse. So every entry carries a +**role**: + +``mask`` + the entry contributes its area to the combined mask, and contributes no + coordinates of its own; +``points`` + the entry contributes coordinates directly, bypassing evaluation and + selection entirely, and contributes nothing to the mask. + +Polygons and brush strokes default to ``mask``; manual points and polylines +default to ``points``. Any entry's role can be changed after the fact, which is +what makes a hand-drawn region usable either way without redrawing it. + +Coordinate convention: ``(row, col)`` throughout, and masks are boolean arrays +indexed ``[row, col]``. The helpers in :mod:`pyidi.selection_geometry` are +reused rather than reimplemented, with a flip at the call site for the three +that work in ``(x, y)``. +""" + +from dataclasses import dataclass, field +from typing import Any, Dict, Optional, Set + +import numpy as np +from matplotlib.path import Path + +from ..selection_geometry import points_along_polygon, rois_inside_mask, rois_inside_polygon + +#: Every entry kind, and the role it takes unless told otherwise. A polygon or a +#: painted area is a statement about a region; a clicked point or a line placed +#: by eye is a statement about specific locations. +DEFAULT_ROLE = { + 'polygon': 'mask', + 'brush': 'mask', + 'polyline': 'points', + 'points': 'points', +} + +ROLES = ('mask', 'points') + + +@dataclass +class Entry: + """One row of the selection: a piece of geometry plus how to use it. + + :param kind: ``'polygon'``, ``'brush'``, ``'polyline'`` or ``'points'`` + :type kind: str + :param geometry: ``(row, col)`` vertices for ``polygon``/``polyline``, + ``(row, col)`` coordinates for ``points``, or a boolean array indexed + ``[row, col]`` for ``brush`` + :type geometry: list or numpy.ndarray + :param label: the name shown in the selections list + :type label: str + :param role: ``'mask'`` or ``'points'``; defaults per :data:`DEFAULT_ROLE` + :type role: str or None + :param visible: whether the entry contributes at all + :type visible: bool + :param score_name: which named score this entry is filtered against; ``None`` + means the pipeline's default. Ignored when ``role == 'points'``. + :type score_name: str or None + :param selector: registry name of the selector to pick this entry's points + with; ``None`` means the pipeline's default + :type selector: str or None + :param selector_params: parameters for that selector; empty means the + pipeline's defaults + :type selector_params: dict + :param mask_width: width, in pixels, of the stroke a ``polyline`` rasterises + to when its role is ``mask``. Ignored for every other kind. + :type mask_width: int + :param erased: area subtracted from this entry by the deselect brush, as a + boolean array indexed ``[row, col]``, or ``None``. Kept separate from + ``geometry`` so the original shape stays intact and undo is a matter of + dropping this array. + + **Replace it, never write into it.** Both the pipeline's rasterisation + cache and the interface's undo stack identify this array by object, so + that neither has to read a frame's worth of booleans to notice a change + -- and an in-place write is a change neither of them can see. Growing an + erasure means ``entry.erased = entry.erased | more``, not ``|=``. + :type erased: numpy.ndarray or None + :param removed: ``(row, col)`` coordinates removed individually from a + ``points``-role entry + :type removed: set + """ + + kind: str + geometry: Any + label: str = '' + role: Optional[str] = None + visible: bool = True + score_name: Optional[str] = None + selector: Optional[str] = None + selector_params: Dict[str, Any] = field(default_factory=dict) + mask_width: int = 1 + erased: Optional[np.ndarray] = None + removed: Set = field(default_factory=set) + + def __post_init__(self): + if self.kind not in DEFAULT_ROLE: + known = ', '.join(sorted(DEFAULT_ROLE)) + raise ValueError(f'Unknown entry kind {self.kind!r}. Known kinds: {known}.') + if self.role is None: + self.role = DEFAULT_ROLE[self.kind] + if self.role not in ROLES: + raise ValueError(f"role must be one of {ROLES}, got {self.role!r}.") + + +def _stamp_discs(mask, centres, radius): + """Set a filled disc of ``radius`` in ``mask`` around each ``(row, col)`` centre.""" + h, w = mask.shape + r = int(radius) + if r <= 0: + for row, col in centres: + row, col = int(round(row)), int(round(col)) + if 0 <= row < h and 0 <= col < w: + mask[row, col] = True + return + + offsets = np.arange(-r, r + 1) + disc = offsets[:, None] ** 2 + offsets[None, :] ** 2 <= r * r + for row, col in centres: + row, col = int(round(row)), int(round(col)) + r0, r1 = max(0, row - r), min(h, row + r + 1) + c0, c1 = max(0, col - r), min(w, col + r + 1) + if r0 >= r1 or c0 >= c1: + continue + sub = disc[r0 - (row - r):r1 - (row - r), c0 - (col - r):c1 - (col - r)] + mask[r0:r1, c0:c1] |= sub + + +def _polyline_pixels(vertices): + """Every ``(row, col)`` pixel along an open polyline, densely sampled.""" + pixels = [] + for start, end in zip(vertices[:-1], vertices[1:]): + start = np.asarray(start, dtype=float) + end = np.asarray(end, dtype=float) + n = max(2, int(np.ceil(np.linalg.norm(end - start))) + 1) + t = np.linspace(0.0, 1.0, n)[:, None] + pixels.append(start + t * (end - start)) + if not pixels: + return np.empty((0, 2)) + return np.vstack(pixels) + + +def _polygon_mask(vertices, shape): + """Fill a polygon given by ``(row, col)`` vertices into a boolean array. + + Only the polygon's bounding box is tested, which matters when a small + polygon is drawn on a large frame -- ``contains_points`` over a full + megapixel grid is otherwise the slowest thing in the mask step. + """ + mask = np.zeros(shape, dtype=bool) + vertices = np.asarray(vertices, dtype=float) + if len(vertices) < 3: + return mask + + r0 = max(0, int(np.floor(vertices[:, 0].min()))) + r1 = min(shape[0], int(np.ceil(vertices[:, 0].max())) + 1) + c0 = max(0, int(np.floor(vertices[:, 1].min()))) + c1 = min(shape[1], int(np.ceil(vertices[:, 1].max())) + 1) + if r0 >= r1 or c0 >= c1: + return mask + + rows, cols = np.mgrid[r0:r1, c0:c1] + inside = Path(vertices).contains_points( + np.column_stack([rows.ravel(), cols.ravel()]) + ) + mask[r0:r1, c0:c1] = inside.reshape(r1 - r0, c1 - c0) + return mask + + +def rasterize(entry, shape): + """The area an entry covers, as a boolean array indexed ``[row, col]``. + + The entry's ``erased`` area, if any, is subtracted here rather than being + baked into the geometry, so that a deselection outlives a change of subset + size or selection parameters without the original shape being lost. + + :param entry: the entry to rasterise + :type entry: Entry + :param shape: ``(rows, cols)`` of the image + :type shape: tuple[int, int] + :return: the covered area + :rtype: numpy.ndarray + :raises ValueError: if a ``brush`` entry's mask does not match ``shape`` + """ + if entry.kind == 'brush': + mask = np.asarray(entry.geometry, dtype=bool) + if mask.shape != tuple(shape): + raise ValueError(f'brush mask has shape {mask.shape}, expected {tuple(shape)}.') + mask = mask.copy() + elif entry.kind == 'polygon': + mask = _polygon_mask(entry.geometry, shape) + elif entry.kind == 'polyline': + mask = np.zeros(shape, dtype=bool) + _stamp_discs(mask, _polyline_pixels(np.asarray(entry.geometry, dtype=float)), + max(0, (int(entry.mask_width) - 1) // 2)) + else: # 'points' + mask = np.zeros(shape, dtype=bool) + _stamp_discs(mask, entry.geometry, 0) + + if entry.erased is not None: + mask &= ~np.asarray(entry.erased, dtype=bool) + return mask + + +def combined_mask(entries, shape): + """Union of the areas of every visible entry whose role is ``mask``. + + :param entries: the selection entries + :type entries: iterable[Entry] + :param shape: ``(rows, cols)`` of the image + :type shape: tuple[int, int] + :return: the combined area; all-``False`` when nothing contributes + :rtype: numpy.ndarray + """ + mask = np.zeros(shape, dtype=bool) + for entry in entries: + if entry.visible and entry.role == 'mask': + mask |= rasterize(entry, shape) + return mask + + +def literal_points(entry, subset_size, spacing=0): + """The coordinates a ``points``-role entry contributes. + + Each kind places its points the way that kind always has: a polyline spaces + them along its segments, a polygon fills itself with a grid, a brush fills + its painted area with a grid, and a point list is taken literally. + + The entry's ``removed`` coordinates and ``erased`` area are both applied, so + a point deselected by hand stays deselected across a change of subset size + or spacing -- the reason those are recorded on the entry rather than being + deleted from the derived list. + + :param entry: the entry to read + :type entry: Entry + :param subset_size: ``(height, width)`` subset size, or a scalar + :type subset_size: int or tuple + :param spacing: extra spacing added to the subset size to get the step + between neighbouring points + :type spacing: int + :return: ``(row, col)`` coordinates + :rtype: list[tuple[int, int]] + """ + geom = entry.geometry + if entry.kind == 'points': + points = [tuple(p) for p in geom] + elif entry.kind == 'polyline': + flipped = [(c, r) for r, c in geom] + points = [(int(round(y)), int(round(x))) + for x, y in points_along_polygon(flipped, subset_size, spacing)] + elif entry.kind == 'polygon': + flipped = [(c, r) for r, c in geom] + points = [(int(round(y)), int(round(x))) + for x, y in rois_inside_polygon(flipped, subset_size, spacing)] + else: # 'brush' + mask = np.asarray(geom, dtype=bool) + points = [(int(r), int(c)) for r, c in rois_inside_mask(mask, subset_size, spacing)] + + if entry.removed: + points = [p for p in points if p not in entry.removed] + if entry.erased is not None: + erased = np.asarray(entry.erased, dtype=bool) + h, w = erased.shape + points = [p for p in points + if not (0 <= p[0] < h and 0 <= p[1] < w and erased[p[0], p[1]])] + return points + + +def all_literal_points(entries, subset_size, spacing=0): + """Every coordinate contributed by visible ``points``-role entries, in order. + + :param entries: the selection entries + :type entries: iterable[Entry] + :param subset_size: ``(height, width)`` subset size, or a scalar + :type subset_size: int or tuple + :param spacing: extra spacing added to the subset size + :type spacing: int + :return: ``(row, col)`` coordinates + :rtype: list[tuple[int, int]] + """ + points = [] + for entry in entries: + if entry.visible and entry.role == 'points': + points.extend(literal_points(entry, subset_size, spacing)) + return points + + +def apply_deselection(entries, stroke, shape, subset_size=11, spacing=0, area=None): + """Subtract a deselect-brush stroke from every entry it touches. + + Mask-role entries record the stroke in their ``erased`` array; the geometry + itself is left alone, so only the painted part is lost and the rest of the + region survives. An entry whose area is wiped out entirely is reported for + deletion rather than left as an empty row. + + Only entries the stroke actually reaches are given an ``erased`` array. A + stroke covers a few hundred pixels and an ``erased`` array covers the frame, + so handing one to every mask entry would make a single dab cost a megabyte + per region -- and cost it again in every undo snapshot. + + Point-role entries lose the covered coordinates. For a ``points`` entry + those are deleted from the geometry outright, the same way a click-to-remove + does: recording them as ``removed`` instead would make a later click on that + very pixel silently do nothing. + + :param entries: the selection entries, modified in place + :type entries: list[Entry] + :param stroke: the painted area, a boolean array indexed ``[row, col]`` + :type stroke: numpy.ndarray + :param shape: ``(rows, cols)`` of the image + :type shape: tuple[int, int] + :param subset_size: ``(height, width)`` subset size, needed to know where a + point-role entry currently places its points + :type subset_size: int or tuple + :param spacing: extra spacing added to the subset size + :type spacing: int + :param area: ``f(entry) -> ndarray`` giving an entry's covered area; + :func:`rasterize` when omitted. A caller holding a rasterisation cache + passes it here rather than filling every polygon a second time. + :type area: callable or None + :return: the entries left with nothing, in the order they appear + :rtype: list[Entry] + """ + stroke = np.asarray(stroke, dtype=bool) + if area is None: + def area(entry): + return rasterize(entry, shape) + emptied = [] + + for entry in entries: + if entry.role == 'mask': + if not (area(entry) & stroke).any(): + continue + # `|` rather than `|=`: `erased` is replaced wholesale and never + # written into, which is what lets both the rasterisation cache and + # an undo snapshot identify it by object rather than by reading a + # frame's worth of booleans. + entry.erased = stroke.copy() if entry.erased is None else (entry.erased | stroke) + if not area(entry).any(): + emptied.append(entry) + continue + + covered = {p for p in literal_points(entry, subset_size, spacing) + if 0 <= p[0] < shape[0] and 0 <= p[1] < shape[1] and stroke[p[0], p[1]]} + if not covered: + continue + if entry.kind == 'points': + entry.geometry = [p for p in entry.geometry if tuple(p) not in covered] + if not entry.geometry: + emptied.append(entry) + else: + entry.removed.update(covered) + + return emptied diff --git a/pyidi/selection/pipeline.py b/pyidi/selection/pipeline.py new file mode 100644 index 0000000..d6647e5 --- /dev/null +++ b/pyidi/selection/pipeline.py @@ -0,0 +1,528 @@ +"""The mask -> evaluate -> select pipeline, end to end and without Qt. + +Three steps, in the vocabulary settled in issue #51: + +1. **mask** -- regions drawn on the image say *where* points may go; +2. **evaluate** -- an evaluator scores every pixel of the image at once; +3. **select** -- a selector turns score plus mask into the points to track. + +Only step 2 is expensive, and it depends on nothing but the frame, the +evaluator and the subset size. So editing a mask, dragging a threshold or +changing the separation re-derives the points from a cached array and costs +nothing, which is what makes the interface on top of this able to update while +a slider is still moving. + +The entries are the pipeline. Each one carries not just its geometry but which +score it is filtered against and with what parameters, so two regions can be +treated differently without any of this changing shape. An interface that +offers only one global set of controls simply writes the same values into every +entry -- which is what the first version of the GUI does. +""" + +import numpy as np + +from .masks import Entry, all_literal_points, apply_deselection, literal_points, rasterize +from .scores import ScoreStore, _freeze +from .select import (DEFAULT_MAX_POINTS, DEFAULT_SEPARATION, DEFAULT_THRESHOLD, as_point_array, decimate, + merge_points, occupancy, select) + +#: Prefix used to label each kind of entry in the selections list. +PRETTY = { + 'polygon': 'Polygon', + 'brush': 'Brush', + 'polyline': 'Line', + 'points': 'Points', +} + +#: Selector parameters used by any entry that does not override them. +DEFAULT_SELECTOR_PARAMS = { + 'separation': DEFAULT_SEPARATION, + 'threshold': DEFAULT_THRESHOLD, + 'threshold_mode': 'quality', + 'max_points': DEFAULT_MAX_POINTS, + 'decimation': 1, +} + + +class SelectionPipeline: + """Entries, scores and selector settings, and the points they produce. + + :param image: 2-D reference frame, indexed ``[row, col]`` + :type image: numpy.ndarray + :param subset_size: scalar or ``(height, width)`` pair + :type subset_size: int or tuple + :param spacing: extra spacing added to the subset size when a + ``points``-role entry lays its points out + :type spacing: int + """ + + def __init__(self, image, subset_size=11, spacing=0): + self.store = ScoreStore(image, subset_size) + self.entries = [] + self.spacing = spacing + #: Score every mask entry is filtered against unless it names another. + self.default_score = None + #: Selector every mask entry uses unless it names another. + self.selector = 'peaks' + #: Selector parameters every mask entry uses unless it overrides them. + self.selector_params = dict(DEFAULT_SELECTOR_PARAMS) + self._label_counters = {kind: 0 for kind in PRETTY} + #: The whole-frame selection, kept alive across mask edits so that the + #: interface can show what the mask is leaving out without re-selecting + #: on every brush stroke. Holds the score array it was computed from, + #: which is what tells it it is stale. + self._candidate_cache = None + #: Rasterised area per entry, keyed by identity and validated against a + #: fingerprint of the geometry. Filling a polygon is a point-in-path test + #: over its bounding box, which on a full-frame region is the single most + #: expensive thing in a redraw -- and a threshold drag does not move a + #: single vertex. + self._raster_cache = {} + + # -- image and sizes --------------------------------------------------- + + @property + def image(self): + """The reference frame. + + :rtype: numpy.ndarray + """ + return self.store.image + + @property + def shape(self): + """``(rows, cols)`` of the reference frame. + + :rtype: tuple[int, int] + """ + return self.store.image.shape + + @property + def subset_size(self): + """The ``(height, width)`` subset size. + + :rtype: tuple[int, int] + """ + return self.store.subset_size + + def set_subset_size(self, subset_size): + """Change the subset size, invalidating every cached score. + + :param subset_size: scalar or ``(height, width)`` pair + :type subset_size: int or tuple + """ + self.store.set_subset_size(subset_size) + + def set_image(self, image): + """Change the reference frame, invalidating every cached score. + + :param image: 2-D frame, indexed ``[row, col]`` + :type image: numpy.ndarray + """ + self.store.set_image(image) + + # -- scores ------------------------------------------------------------ + + def define_score(self, name, evaluator='shi_tomasi', **params): + """Declare a named score. The first one declared becomes the default. + + :param name: the name entries refer to this score by + :type name: str + :param evaluator: registry name of the evaluator + :type evaluator: str + :param params: evaluator-specific parameters + :return: the spec now bound to ``name`` + :rtype: ScoreSpec + """ + spec = self.store.define(name, evaluator, **params) + if self.default_score is None: + self.default_score = name + return spec + + def ensure_default_score(self): + """The default score's name, declaring a Shi-Tomasi one if none exists. + + :rtype: str + """ + if self.default_score is None: + self.define_score('shi_tomasi', 'shi_tomasi') + return self.default_score + + # -- entries ----------------------------------------------------------- + + def add_entry(self, kind, geometry, label=None, **kwargs): + """Append an entry and return it. + + Labels are never reused: deleting ``Polygon 2`` and adding another + polygon gives ``Polygon 4``, not a second ``Polygon 3``, so a label in a + note or a screenshot always refers to the same thing. + + :param kind: ``'polygon'``, ``'brush'``, ``'polyline'`` or ``'points'`` + :type kind: str + :param geometry: the entry's geometry; see :class:`~pyidi.selection.masks.Entry` + :param label: explicit label; generated from a per-kind counter if omitted + :type label: str or None + :param kwargs: any other :class:`~pyidi.selection.masks.Entry` field + :return: the new entry + :rtype: Entry + """ + if label is None: + self._label_counters[kind] += 1 + label = f'{PRETTY[kind]} {self._label_counters[kind]}' + entry = Entry(kind=kind, geometry=geometry, label=label, **kwargs) + self.entries.append(entry) + return entry + + def remove_entry(self, entry): + """Delete an entry, by identity. + + :param entry: the entry to remove + :type entry: Entry + """ + self.entries = [e for e in self.entries if e is not entry] + + def deselect(self, stroke): + """Subtract a deselect-brush stroke, dropping entries it wipes out. + + :param stroke: the painted area, indexed ``[row, col]`` + :type stroke: numpy.ndarray + :return: the entries that were removed + :rtype: list[Entry] + """ + emptied = apply_deselection(self.entries, stroke, self.shape, self.subset_size, + self.spacing, area=self.area) + for entry in emptied: + self.remove_entry(entry) + return emptied + + @property + def mask(self): + """The union of every visible ``mask``-role entry's area. + + The same answer as :func:`~pyidi.selection.masks.combined_mask`, built + from the rasterised areas this pipeline has already cached rather than + by filling every polygon again -- this is asked for on every redraw. + + :rtype: numpy.ndarray + """ + covered = np.zeros(self.shape, dtype=bool) + for entry in self.entries: + if entry.visible and entry.role == 'mask': + covered |= self.area(entry) + return covered + + def _fingerprint(self, entry): + """A cheap value that changes whenever an entry's area would. + + Vertex lists are mutated in place, so they are compared by value; a + brush mask and an erased area are only ever replaced wholesale, so those + are compared by identity rather than by reading a megabyte of booleans. + """ + geometry = entry.geometry + shape = tuple(getattr(geometry, 'shape', ())) + return (entry.kind, entry.mask_width, self.shape, id(entry.erased), + id(geometry) if shape else tuple(map(tuple, geometry))) + + def area(self, entry): + """The area an entry covers, rasterised at most once per change. + + :param entry: the entry to rasterise + :type entry: Entry + :return: the covered area, as a boolean array indexed ``[row, col]``. + Owned by the cache, so callers must not modify it in place. + :rtype: numpy.ndarray + """ + fingerprint = self._fingerprint(entry) + cached = self._raster_cache.get(id(entry)) + if cached is not None and cached[1] == fingerprint: + return cached[2] + area = rasterize(entry, self.shape) + self._raster_cache = {id(e): self._raster_cache[id(e)] + for e in self.entries if id(e) in self._raster_cache} + # The entry itself is held alongside its area. The key is its `id`, and + # an `id` is only unique among live objects: without a reference here, a + # deleted entry could be collected and a new one allocated at the same + # address, which would then read the dead entry's area out of the cache. + self._raster_cache[id(entry)] = (entry, fingerprint, area) + return area + + # -- the pipeline ------------------------------------------------------ + + def entry_settings(self, entry): + """The score, selector and parameters an entry is actually run with. + + An entry that names none of them falls back to the pipeline's defaults, + which is what makes a single global control panel a special case of the + per-entry model rather than a different one. + + :param entry: the entry to resolve + :type entry: Entry + :return: ``(score name, selector name, parameters)`` + :rtype: tuple[str, str, dict] + """ + params = dict(self.selector_params) + params.update(entry.selector_params) + return (entry.score_name or self.ensure_default_score(), + entry.selector or self.selector, + params) + + def _mask_groups(self): + """Visible mask entries grouped by the settings they share. + + Grouping matters: two regions filtered the same way must compete for the + same separation, or a point in one could land right next to a point in + the other. + + :return: ``[((score, selector, params), mask), ...]`` in first-seen order + :rtype: list + """ + groups = {} + for entry in self.entries: + if not (entry.visible and entry.role == 'mask'): + continue + score_name, selector, params = self.entry_settings(entry) + key = (score_name, selector, tuple(sorted((k, _freeze(v)) for k, v in params.items()))) + if key not in groups: + groups[key] = [(score_name, selector, params), np.zeros(self.shape, dtype=bool)] + groups[key][1] |= self.area(entry) + return [tuple(value) for value in groups.values()] + + def in_frame(self, points): + """The coordinates that lie inside the reference frame. + + Applied to every hand-picked coordinate before it goes anywhere. A click + lands wherever the interface lets it land, and a subset centred outside + the frame is not a thing that can be tracked -- so it is dropped here, + once, rather than being caught by whichever array it is used to index + first. + + :param points: ``(row, col)`` coordinates + :type points: sequence + :return: those inside the frame, in the order given + :rtype: list[tuple[int, int]] + """ + array = as_point_array(points) + if not len(array): + return [] + height, width = self.shape + inside = ((array[:, 0] >= 0) & (array[:, 0] < height) + & (array[:, 1] >= 0) & (array[:, 1] < width)) + return [(int(row), int(col)) for row, col in array[inside]] + + def remove_point(self, entry, point): + """Take one displayed point away, and keep it away. + + How depends on where the point came from. A hand-picked coordinate is + simply deleted, so clicking that same pixel again puts it back. + + A selected point is not stored anywhere -- it is re-derived from the + score on every redraw -- so the only way to remove one is to take the + ground it stands on out of the mask. Erasing the single pixel does not + do it: the selector promotes the next-best pixel of the same block and + the point reappears a pixel or two away, which reads as the click having + nudged it rather than removed it. What is erased is the disc the point + was reserving, its separation, so nothing can land nearer to it than a + neighbouring point legitimately could have. + + :param entry: the entry the point is credited to + :type entry: Entry + :param point: the ``(row, col)`` coordinate to remove + :type point: tuple[int, int] + """ + row, col = int(point[0]), int(point[1]) + if entry.role == 'points': + if entry.kind == 'points': + entry.geometry = [p for p in entry.geometry if tuple(p) != (row, col)] + else: + entry.removed = set(entry.removed) | {(row, col)} + return + radius = max(0, int(self.entry_settings(entry)[2].get('separation', 0))) + blocked = occupancy([(row, col)], self.shape, radius) + # A new array rather than a write into the old one: `erased` is compared + # by object identity, both by the rasterisation cache and by undo. + entry.erased = blocked if entry.erased is None else (entry.erased | blocked) + + def literal_points(self): + """Every coordinate contributed by visible ``points``-role entries. + + :return: ``(row, col)`` coordinates inside the frame, in entry order + :rtype: list[tuple[int, int]] + """ + return self.in_frame(all_literal_points(self.entries, self.subset_size, self.spacing)) + + def picked_points(self, literal=None): + """The points the selector produces, group by group. + + Hand-picked points are stamped into the occupancy array before any group + runs, so they take precedence: nothing automatic can land within the + separation of one, and none of them can be displaced. + + :param literal: the hand-picked points to keep clear of; read from the + entries when omitted + :type literal: sequence or None + :return: ``(n_points, 2)`` integer array of ``(row, col)`` coordinates, + best first within each group + :rtype: numpy.ndarray + """ + if literal is None: + literal = self.literal_points() + + taken = np.zeros(self.shape, dtype=bool) + literal_taken = occupancy(literal, self.shape, 0) if len(literal) else None + picked = [] + groups = self._mask_groups() + for index, ((score_name, selector, params), mask) in enumerate(groups): + if not mask.any(): + continue + radius = max(0, int(params.get('separation', 0))) + occupied = taken + if literal_taken is not None: + occupied = taken | (literal_taken if radius <= 0 + else occupancy(literal, self.shape, radius)) + points = select(self.store.get(score_name), mask=mask, selector=selector, + occupied=occupied, **params) + # Everything selected is stamped, including what decimation is about + # to drop, so thinning one group leaves gaps rather than inviting the + # next group to fill them in. Only what a later group will read: the + # stamp is a Python loop over every selected point, which is 84 ms at + # seventeen thousand of them and is usually thrown away unread, since + # one set of settings for the whole image is one group. + if index + 1 < len(groups): + taken |= occupancy(points, self.shape, radius) + picked.append(as_point_array(decimate(points, stride=params.get('decimation')))) + return np.vstack(picked) if picked else as_point_array([]) + + def candidate_points(self): + """The points these settings would select over the whole frame. + + What the mask is leaving out, in other words -- an interface can show + the difference between "there is nothing there" and "you have masked it + away", which the selection alone cannot say. + + This is the whole frame every time, not the unmasked part, deliberately: + the answer then depends only on the score and the selector settings, so + it survives every mask edit and painting a region does not re-select + anything. The caller filters by the mask, which costs one lookup per + point. + + :return: ``(n_points, 2)`` integer array of ``(row, col)`` coordinates + :rtype: numpy.ndarray + """ + score_name = self.ensure_default_score() + score = self.store.get(score_name) + params = dict(self.selector_params) + key = (self.selector, tuple(sorted((k, _freeze(v)) for k, v in params.items()))) + cached = self._candidate_cache + # `is`, not `==`: the store replaces the array whenever the evaluator, + # its parameters or the subset size change, so identity is the version. + if cached is not None and cached[0] is score and cached[1] == key: + return cached[2] + points = select(score, mask=np.ones(self.shape, dtype=bool), + selector=self.selector, **params) + points = as_point_array(decimate(points, stride=params.get('decimation'))) + self._candidate_cache = (score, key, points) + return points + + def points_and_credits(self): + """The points, and which entry each one is credited to, from one pass. + + Anything drawing the selection needs both -- the total to plot and the + per-row counts to label the list with -- and asking for them separately + runs the whole selection twice for the same answer. + + A selected point is credited to the first visible mask entry whose area + covers it. With overlapping regions the attribution is arbitrary but + stable; the total is unaffected. + + :return: ``((n_points, 2) array, one (n, 2) array per entry)`` + :rtype: tuple[numpy.ndarray, list[numpy.ndarray]] + """ + credited = [as_point_array([]) for _ in self.entries] + literal = [] + for index, entry in enumerate(self.entries): + if entry.visible and entry.role == 'points': + own = self.in_frame(literal_points(entry, self.subset_size, self.spacing)) + credited[index] = as_point_array(own) + literal.extend(own) + + picked = self.picked_points(literal) + masks = [(index, self.area(entry)) + for index, entry in enumerate(self.entries) + if entry.visible and entry.role == 'mask'] + if len(picked) and masks: + # One lookup per mask over the whole point array, rather than one + # Python step per point: `inside` is (n_masks, n_points), and the + # first True down each column is the entry that gets the credit. + rows, cols = picked[:, 0], picked[:, 1] + inside = np.array([mask[rows, cols] for _, mask in masks]) + owner = inside.argmax(axis=0) + owned = inside.any(axis=0) + for position, (index, _) in enumerate(masks): + credited[index] = picked[owned & (owner == position)] + return merge_points(literal, picked), credited + + def points_by_entry(self): + """Which points each entry accounts for, aligned with :attr:`entries`. + + :return: one ``(n, 2)`` array of ``(row, col)`` coordinates per entry, + in :attr:`entries` order + :rtype: list[numpy.ndarray] + """ + return self.points_and_credits()[1] + + def get_points(self): + """Run the whole pipeline and return the points. + + :return: ``(n_points, 2)`` integer array of ``(row, col)`` coordinates, + hand-picked points first + :rtype: numpy.ndarray + """ + literal = self.literal_points() + return merge_points(literal, self.picked_points(literal)) + + @property + def points(self): + """The pipeline's points, as :meth:`get_points` returns them. + + :rtype: numpy.ndarray + """ + return self.get_points() + + +def select_points(image, entries=(), subset_size=11, evaluator='shi_tomasi', + selector='peaks', spacing=0, evaluator_params=None, **selector_params): + """Run mask, evaluate and select in one call. + + The headless entry point: no Qt, no interface, just an image and some + regions in, points out. + + :param image: 2-D reference frame, indexed ``[row, col]`` + :type image: numpy.ndarray + :param entries: the selection entries; an empty collection masks nothing and + so selects nothing + :type entries: iterable[Entry] + :param subset_size: scalar or ``(height, width)`` pair + :type subset_size: int or tuple + :param evaluator: registry name of the evaluator to score with + :type evaluator: str + :param selector: registry name of the selector to pick with + :type selector: str + :param spacing: extra spacing used when a ``points``-role entry lays out its + points + :type spacing: int + :param evaluator_params: parameters for the evaluator + :type evaluator_params: dict or None + :param selector_params: parameters for the selector + :return: ``(n_points, 2)`` integer array of ``(row, col)`` coordinates + :rtype: numpy.ndarray + """ + pipeline = SelectionPipeline(image, subset_size=subset_size, spacing=spacing) + pipeline.define_score('score', evaluator, **(evaluator_params or {})) + pipeline.selector = selector + pipeline.selector_params.update(selector_params) + pipeline.entries = list(entries) + return pipeline.get_points() + + +__all__ = ['SelectionPipeline', 'select_points', 'as_point_array', 'PRETTY', + 'DEFAULT_SELECTOR_PARAMS'] diff --git a/pyidi/selection/scores.py b/pyidi/selection/scores.py new file mode 100644 index 0000000..ef1c7c6 --- /dev/null +++ b/pyidi/selection/scores.py @@ -0,0 +1,277 @@ +"""Named, cached score images. + +A :class:`ScoreStore` holds one image and one subset size, and hands out score +images by name. The point of the name is that several scores coexist: a +Shi-Tomasi score for a speckled plate and a directional-gradient score for a +beam can both be live, each referenced by whichever selection entry wants it, +without either recomputing the other. + +Caching is keyed on the ``(evaluator, parameters, subset size)`` triple rather +than on the name, so two names describing the same computation share one array. + +What invalidates what is deliberate and coarse. Changing the image or the +subset size drops everything, because every cached array is wrong. Changing a +threshold, a mask or a selection parameter drops nothing, because none of them +enter a score. That asymmetry is the whole reason the interface can re-derive +points on every slider tick. + +The store deliberately evaluates the *full frame* and never the bounding-box +crop that :func:`~pyidi.selection.evaluate.evaluate` also offers. A cropped +score is only valid inside its box, so it would have to be invalidated whenever +the mask grew -- exactly the "a mask edit never costs an evaluation" guarantee +this class exists to provide. Callers scoring a small region of a very large +frame once, headlessly, can pass ``crop=True`` to ``evaluate`` directly. +""" + +from dataclasses import dataclass +from typing import Tuple + +import numpy as np + +from ..selection_geometry import _as_size_pair +from .evaluate import evaluate, get_evaluator, resolve_parameters + +#: How many score arrays a store keeps before dropping the least recently used. +#: +#: Each one is a ``float32`` the size of the frame -- 16 MB at 2560x1600 -- and +#: every distinct set of evaluator parameters is a different array. A spin box +#: dragged through sixty values therefore asks for sixty of them, and an +#: unbounded cache would hold the lot. Eight is comfortably more than the two or +#: three scores a session actually switches between, and recomputing one is tens +#: of milliseconds per megapixel. +DEFAULT_CACHE_SIZE = 8 + + +def _freeze(value): + """Turn a parameter value into something hashable and comparable. + + Sequences become tuples (recursively) so that ``[0, 1]`` and ``(0, 1)`` + produce the same cache key; a numpy scalar becomes a plain Python number so + that ``np.float64(0.5)`` and ``0.5`` do too. + """ + if isinstance(value, np.ndarray): + return tuple(_freeze(v) for v in value.tolist()) + if isinstance(value, (list, tuple)): + return tuple(_freeze(v) for v in value) + if isinstance(value, np.generic): + return value.item() + return value + + +@dataclass(frozen=True) +class ScoreSpec: + """What a score image is: an evaluator, its parameters, and a subset size. + + Two specs that compare equal describe the same array, which is what makes + this usable as a cache key. + + :param evaluator: registry name of the evaluator + :type evaluator: str + :param parameters: every parameter the evaluator takes, defaults filled in, + as a sorted tuple of ``(name, frozen value)`` pairs + :type parameters: tuple + :param subset_size: ``(height, width)`` + :type subset_size: tuple[int, int] + """ + + evaluator: str + parameters: Tuple[Tuple[str, object], ...] + subset_size: Tuple[int, int] + + @classmethod + def build(cls, evaluator, subset_size, **params): + """Normalise loose arguments into a spec. + + :param evaluator: registry name of the evaluator + :type evaluator: str + :param subset_size: scalar or ``(height, width)`` pair + :type subset_size: int or tuple + :param params: evaluator-specific parameters, possibly partial + :return: the normalised spec + :rtype: ScoreSpec + :raises ValueError: if the evaluator or a parameter name is unknown + """ + spec = get_evaluator(evaluator) + resolved = resolve_parameters(spec, params) + frozen = tuple(sorted((k, _freeze(v)) for k, v in resolved.items())) + h, w = _as_size_pair(subset_size) + return cls(evaluator=evaluator, parameters=frozen, subset_size=(int(h), int(w))) + + def as_kwargs(self): + """The parameters as a keyword dict, ready to pass to ``evaluate``. + + :rtype: dict + """ + return dict(self.parameters) + + +class ScoreStore: + """One image, one subset size, and every score computed from them. + + :param image: 2-D reference frame, indexed ``[row, col]`` + :type image: numpy.ndarray + :param subset_size: scalar or ``(height, width)`` pair + :type subset_size: int or tuple + :param max_cached: how many score arrays to keep; see + :data:`DEFAULT_CACHE_SIZE` + :type max_cached: int + """ + + def __init__(self, image, subset_size=11, max_cached=DEFAULT_CACHE_SIZE): + self._image = None + self._subset_size = None + self._definitions = {} + self._cache = {} + self.max_cached = max(1, int(max_cached)) + #: How many times an evaluator has actually run. Only ever increases, + #: including across an invalidation, so a test or a GUI assertion can + #: check "this interaction did not re-evaluate" by comparing before and + #: after. + self.n_evaluations = 0 + self.set_image(image) + self.set_subset_size(subset_size) + + @property + def image(self): + """The reference frame every score is computed from. + + :rtype: numpy.ndarray + """ + return self._image + + @property + def subset_size(self): + """The ``(height, width)`` subset size every score is computed for. + + :rtype: tuple[int, int] + """ + return self._subset_size + + @property + def names(self): + """The names of the scores currently defined, in definition order. + + :rtype: list[str] + """ + return list(self._definitions) + + def set_image(self, image): + """Replace the reference frame, discarding every cached score. + + :param image: 2-D frame, indexed ``[row, col]`` + :type image: numpy.ndarray + :raises ValueError: if ``image`` is not 2-D + """ + image = np.asarray(image) + if image.ndim != 2: + raise ValueError(f'image must be 2-D (row, col), got shape {image.shape}.') + self._image = image + self.invalidate() + + def set_subset_size(self, subset_size): + """Replace the subset size, discarding every cached score. + + The score definitions survive: a name still refers to the same evaluator + and parameters, and its array is recomputed at the new size on the next + request. + + :param subset_size: scalar or ``(height, width)`` pair + :type subset_size: int or tuple + """ + h, w = _as_size_pair(subset_size) + new_size = (int(h), int(w)) + if new_size == self._subset_size: + return + self._subset_size = new_size + self._definitions = { + name: ScoreSpec(spec.evaluator, spec.parameters, new_size) + for name, spec in self._definitions.items() + } + self.invalidate() + + def invalidate(self): + """Drop every cached array, keeping the definitions.""" + self._cache = {} + + def define(self, name, evaluator, **params): + """Declare a named score. Nothing is computed until it is requested. + + Redefining an existing name replaces its spec. + + :param name: the name to address this score by + :type name: str + :param evaluator: registry name of the evaluator + :type evaluator: str + :param params: evaluator-specific parameters + :return: the spec now bound to ``name`` + :rtype: ScoreSpec + """ + spec = ScoreSpec.build(evaluator, self._subset_size, **params) + self._definitions[name] = spec + return spec + + def remove(self, name): + """Forget a named score. The cached array survives if another name shares it. + + :param name: the name to drop + :type name: str + """ + self._definitions.pop(name, None) + + def spec(self, name): + """The spec bound to a name. + + :param name: a defined score name + :type name: str + :rtype: ScoreSpec + :raises KeyError: if no score of that name is defined + """ + if name not in self._definitions: + known = ', '.join(self._definitions) or '(none defined)' + raise KeyError(f"No score named {name!r}. Defined scores: {known}.") + return self._definitions[name] + + def get(self, name): + """The score image for a name, computing it only if it is not cached. + + :param name: a defined score name + :type name: str + :return: ``float32`` score image, ``NaN`` on the invalid border + :rtype: numpy.ndarray + :raises KeyError: if no score of that name is defined + """ + return self.get_for_spec(self.spec(name)) + + def get_for_spec(self, spec): + """The score image for a spec, computing it only if it is not cached. + + :param spec: what to compute + :type spec: ScoreSpec + :return: ``float32`` score image, ``NaN`` on the invalid border + :rtype: numpy.ndarray + """ + # Popped and reinserted, so that the dict's insertion order is + # least-recently-used first and the eviction below is its first key. + score = self._cache.pop(spec, None) + if score is None: + score = evaluate( + self._image, + evaluator=spec.evaluator, + subset_size=spec.subset_size, + crop=False, + **spec.as_kwargs(), + ) + self.n_evaluations += 1 + self._cache[spec] = score + while len(self._cache) > self.max_cached: + del self._cache[next(iter(self._cache))] + return score + + def is_cached(self, name): + """Whether a name's array is already computed. + + :param name: a defined score name + :type name: str + :rtype: bool + """ + return self.spec(name) in self._cache diff --git a/pyidi/selection/select.py b/pyidi/selection/select.py new file mode 100644 index 0000000..aca3ed6 --- /dev/null +++ b/pyidi/selection/select.py @@ -0,0 +1,583 @@ +"""Turning a score image and a mask into the points to track. + +A threshold on its own is not a selection. On the sparse grid the old filter +scored, "keep everything above 0.2 of the maximum" was a reasonable answer +because the grid had already spaced the candidates out. On a dense score image +it returns a solid blob of adjacent pixels around every strong corner -- a +thousand subsets stacked on top of each other, all tracking the same feature. + +So the selection has one other control: a **separation**, the distance no two +selected points may come closer than. Thinning the thresholded pixels any +other way does not work, and the numbers are worth recording, because "just keep +every n-th of them" is the obvious thing to reach for. On a 1024x1024 frame with +357k pixels above the threshold, thinned to twenty thousand points: + + ========================= ========== ================ + rule median gap pairs under 3 px + ========================= ========== ================ + every n-th, best first 2.0 px 78% + every n-th, in scan order 1.0 px 92% + separation >= n px 0% + ========================= ========== ================ + +Keeping every n-th of a list ordered by score fails because consecutive ranks +are neighbours on the same feature; keeping every n-th in scan order fails +because the stride aliases against the row length and lands in columns. Either +way the great majority of subsets end up on top of another one. + +Spacing is enforced by a greedy walk from best to worst, accepting a candidate +only if nothing already accepted is within the separation of it -- what +``goodFeaturesToTrack`` does. The walk uses a boolean occupancy array rather +than pairwise distances: accepting a point stamps a disc into it, and a +candidate is rejected by a single lookup. + +That walk is exact but it is linear in the *candidates*, and a loose threshold +leaves hundreds of thousands of them -- 40 ms to 300 ms, which no slider can +drag. So the candidates are reduced first, to the best pixel in each cell of a +grid half the separation across. It is an approximation, and what it costs is +yield: at a separation of 11 it finds 1708 points where the exact walk finds +2193, in 9 ms instead of 39. What it does not cost is the guarantee -- the walk +still runs, so the separation still holds exactly -- and a point count is what +the separation control is for adjusting anyway. + +The occupancy array also gives the merge with hand-picked points for free -- +stamp those into it before picking starts and no automatic point can ever crowd +one. +""" + +import inspect + +import numpy as np + +from ..selection_geometry import _as_size_pair + +#: Cap on the number of points a single selection returns. Finite by default: +#: a dense score image with a separation of 1 can otherwise produce tens of +#: thousands of subsets from one careless slider drag. +DEFAULT_MAX_POINTS = 20000 + +#: Default threshold: keep anything at least this good a fraction of the best +#: feature in the region. See :data:`ROBUST_MAXIMUM_PERCENTILE` for why "best" +#: is not the literal maximum. +DEFAULT_THRESHOLD = 0.01 + +#: How many candidates the suppression walk tests for occupancy at a time. +#: Large enough that the vectorised read dominates the Python loop, small enough +#: that a `max_points` cap still stops early rather than doing a whole pass. +SUPPRESS_CHUNK = 4096 + +#: Percentile of the eligible scores taken as "the best feature here". +#: +#: Not the maximum, which one dust mote or specular highlight can push an order +#: of magnitude above everything real, dragging every useful quality setting +#: into the bottom of the slider. The 99.9th percentile is the same number on a +#: well-behaved frame and survives a handful of outliers on a bad one. +ROBUST_MAXIMUM_PERCENTILE = 99.9 + +#: Default separation, in pixels: the distance no two selected points come closer +#: than. +DEFAULT_SEPARATION = 11 + +#: Cell size used to reduce the candidates before the suppression walk, as a +#: fraction of the separation. Half was measured against the exact walk: a third +#: recovers another 12% of the yield for 40% more time, and using the whole +#: separation is 25% faster again but throws away a third of the points. +CANDIDATE_CELL_FRACTION = 2 + +#: Thresholding rules. ``quality`` is the default and the one to reach for. +#: +#: ``percentile`` looks the most natural and is the least useful on a dense +#: score image, because it ranks *pixels* and pixels are overwhelmingly +#: background: on a typical frame the 90th percentile of the score is under +#: 1/500th of the best feature, so nine tenths of the slider's travel is spent +#: inside the featureless area and the whole selection collapses the moment you +#: leave the top percentile. It is kept because it is the right rule for a +#: lattice, where the candidates have already been spaced out. +#: +#: A third rule, a fraction of the literal maximum, was dropped: it is the same +#: rule as ``quality`` with a reference that one dust mote can move, so on any +#: frame worth using it is indistinguishable and on a bad one it is worse. +THRESHOLD_MODES = ('quality', 'percentile') + + +def threshold_value(score, mask=None, mode='percentile', value=DEFAULT_THRESHOLD): + """The absolute score a candidate must exceed. + + :param score: score image, ``NaN`` where invalid + :type score: numpy.ndarray + :param mask: boolean array restricting which scores are considered; ``None`` + considers the whole image + :type mask: numpy.ndarray or None + :param mode: ``'quality'`` (``value`` in 0..1, as a fraction of the robust + maximum) or ``'percentile'`` (``value`` in 0..100) + :type mode: str + :param value: the threshold in the units ``mode`` implies + :type value: float + :return: the absolute threshold; ``inf`` when nothing is eligible, so that + no candidate can pass + :rtype: float + :raises ValueError: if ``mode`` is not a known threshold mode + """ + if mode not in THRESHOLD_MODES: + raise ValueError(f"mode must be one of {THRESHOLD_MODES}, got {mode!r}.") + + finite = np.isfinite(score) + eligible = finite if mask is None else (finite & mask) + if not eligible.any(): + return np.inf + + values = score[eligible] + if mode == 'percentile': + if value <= 0: + # A slider at zero should keep everything. Taking the 0th percentile + # literally would return the smallest score, which the strict + # comparison below then excludes -- so a uniform score image would + # select nothing at all at the loosest setting. + return -np.inf + return float(np.percentile(values, value)) + if value <= 0: + return -np.inf # as for a zero percentile: keep everything + return float(value) * float(np.percentile(values, ROBUST_MAXIMUM_PERCENTILE)) + + +def _disc(radius): + """A boolean disc of the given radius, as a ``(2r+1, 2r+1)`` array.""" + offsets = np.arange(-radius, radius + 1) + return offsets[:, None] ** 2 + offsets[None, :] ** 2 <= radius * radius + + +def _mask_window(mask, cell): + """The slices bounding the mask's area, snapped back to a whole cell. + + Everything the selection does is linear in the pixels it is handed, and a + region drawn on a large frame is usually a small part of it. Nothing outside + the mask can be selected, so the work outside its bounding box -- a + full-frame threshold comparison, a full-frame block reduction, a full-frame + occupancy grid -- is spent on pixels that were never in the running. + + The start is floored to a multiple of ``cell`` so that the block grid of + :func:`_block_best` falls exactly where it would have on the whole frame. + The reduction has to pick the same winners as before, not merely similar + ones: a grid offset by a few pixels answers a slightly different question. + + :param mask: boolean array restricting where points may be placed + :type mask: numpy.ndarray + :param cell: the block size the reduction will use + :type cell: int + :return: ``(row_slice, col_slice)``, or ``None`` if the mask is empty + :rtype: tuple[slice, slice] or None + """ + rows = np.flatnonzero(mask.any(axis=1)) + if not rows.size: + return None + cols = np.flatnonzero(mask.any(axis=0)) + return (slice((rows[0] // cell) * cell, rows[-1] + 1), + slice((cols[0] // cell) * cell, cols[-1] + 1)) + + +def _block_best(score, eligible, cell): + """The best eligible pixel in each ``cell`` x ``cell`` block of the image. + + A cheap way to cut the candidate list down before the suppression walk. Cells + with nothing eligible in them contribute nothing, so a blank area costs no + points -- unlike a lattice, which places one wherever the grid falls. + + :param score: score image, ``NaN`` where invalid + :type score: numpy.ndarray + :param eligible: boolean array of candidate positions + :type eligible: numpy.ndarray + :param cell: block size in pixels; must be at least 2 + :type cell: int + :return: ``(rows, cols)`` of the winners, in row-major block order + :rtype: tuple[numpy.ndarray, numpy.ndarray] + """ + height, width = score.shape + down, across = -(-height // cell), -(-width // cell) + # Pad to a whole number of blocks with -inf, so the padding can never win a + # block and the reshape needs no special case at the right and bottom edges. + padded = np.full((down * cell, across * cell), -np.inf, dtype=np.float32) + padded[:height, :width] = np.where(eligible, score, -np.inf) + blocks = padded.reshape(down, cell, across, cell).transpose(0, 2, 1, 3) + blocks = blocks.reshape(down, across, cell * cell) + + within = blocks.argmax(-1) + best = np.take_along_axis(blocks, within[..., None], -1)[..., 0] + occupied = np.isfinite(best) + block_rows, block_cols = np.nonzero(occupied) + offset = within[occupied] + return block_rows * cell + offset // cell, block_cols * cell + offset % cell + + +def _ordered_candidates(score, eligible, keep=None): + """Candidate coordinates, best first, ties broken by row then column. + + :param score: score image + :type score: numpy.ndarray + :param eligible: boolean array of candidate positions + :type eligible: numpy.ndarray + :param keep: sort only this many of the best candidates. Only safe when the + caller will accept them all, since a candidate below the cut can still + be accepted once suppression has rejected the ones above it. + :type keep: int or None + :return: ``(rows, cols)`` arrays in acceptance order + """ + rows, cols = np.nonzero(eligible) + if not rows.size: + return rows, cols + values = score[rows, cols] + + if keep is not None and 0 < keep < rows.size: + # A loose threshold leaves hundreds of thousands of candidates and the + # sort dominates the whole selection, yet all but `keep` of them are + # discarded straight afterwards. Partitioning is linear, and taking + # everything tied with the worst survivor makes it exact: without that + # the tiebreak at the cut would fall to numpy's internal partition + # order instead of the row/column rule below. + cut = values[np.argpartition(-values, keep - 1)[:keep]].min() + head = np.flatnonzero(values >= cut) + rows, cols, values = rows[head], cols[head], values[head] + + # np.nonzero returns its indices in row-major order, so the candidates + # arrive sorted by row and then by column already. A *stable* sort by + # descending score therefore leaves ties in that order -- the same result a + # three-key lexsort gives, for a third of the work, and just as independent + # of numpy's internal ordering. + order = np.argsort(-values, kind='stable') + return rows[order], cols[order] + + +def suppress(rows, cols, shape, radius, max_points=None, occupied=None): + """Greedy non-maximum suppression over candidates already in priority order. + + :param rows: candidate row coordinates, best first + :type rows: numpy.ndarray + :param cols: candidate column coordinates, best first + :type cols: numpy.ndarray + :param shape: ``(rows, cols)`` of the image + :type shape: tuple[int, int] + :param radius: no accepted point comes within this distance of another; 0 + accepts every candidate + :type radius: float + :param max_points: stop after this many; ``None`` for no limit + :type max_points: int or None + :param occupied: boolean array of positions already taken, e.g. by + hand-picked points. Copied, not modified. + :type occupied: numpy.ndarray or None + :return: accepted ``(row, col)`` coordinates, in acceptance order + :rtype: list[tuple[int, int]] + """ + radius = int(radius) + if radius <= 0: + # Nothing suppresses anything, so the walk is just "drop what is already + # taken, then cut to the cap" -- two array operations instead of twenty + # thousand trips round a Python loop. + if occupied is not None: + free = ~np.asarray(occupied, dtype=bool)[rows, cols] + rows, cols = rows[free], cols[free] + if max_points is not None and len(rows) > max_points: + rows, cols = rows[:max_points], cols[:max_points] + return list(zip(rows.tolist(), cols.tolist())) + + taken = np.zeros(shape, dtype=bool) if occupied is None else np.asarray(occupied, dtype=bool).copy() + disc = _disc(radius) + height, width = shape + + accepted = [] + for start in range(0, rows.size, SUPPRESS_CHUNK): + chunk_rows = rows[start:start + SUPPRESS_CHUNK] + chunk_cols = cols[start:start + SUPPRESS_CHUNK] + # Occupancy only ever grows, so a candidate that is already covered now + # would still be covered when its turn came: dropping the whole batch of + # them in one vectorised read is exact, and it is what keeps the Python + # loop off the great majority of candidates. A tight separation on + # a dense score image rejects better than nine in ten. + free = ~taken[chunk_rows, chunk_cols] + for row, col in zip(chunk_rows[free].tolist(), chunk_cols[free].tolist()): + if taken[row, col]: + continue # taken by an earlier point in this chunk + accepted.append((row, col)) + if max_points is not None and len(accepted) >= max_points: + return accepted + r0, r1 = max(0, row - radius), min(height, row + radius + 1) + c0, c1 = max(0, col - radius), min(width, col + radius + 1) + taken[r0:r1, c0:c1] |= disc[r0 - row + radius:r1 - row + radius, + c0 - col + radius:c1 - col + radius] + return accepted + + +def select_peaks(score, mask=None, separation=DEFAULT_SEPARATION, threshold=DEFAULT_THRESHOLD, + threshold_mode='quality', max_points=DEFAULT_MAX_POINTS, occupied=None): + """Pick the strongest subsets from a score image, no two closer than ``separation``. + + :param score: score image, ``NaN`` where invalid + :type score: numpy.ndarray + :param mask: boolean array restricting where points may be placed; ``None`` + allows the whole image + :type mask: numpy.ndarray or None + :param separation: the distance, in pixels, no two selected points may come + closer than. 1 keeps every pixel above the threshold. + :type separation: int + :param threshold: threshold in the units ``threshold_mode`` implies + :type threshold: float + :param threshold_mode: one of :data:`THRESHOLD_MODES` + :type threshold_mode: str + :param max_points: keep at most this many, highest-scoring first + :type max_points: int or None + :param occupied: positions already taken, which no selected point may fall on + :type occupied: numpy.ndarray or None + :return: ``(row, col)`` coordinates, best first + :rtype: list[tuple[int, int]] + """ + separation = max(1, int(separation)) + # Two is the smallest cell that reduces anything, and the walk over an + # unreduced megapixel frame is 300 ms. + cell = 1 if separation <= 1 else max(2, separation // CANDIDATE_CELL_FRACTION) + + # Everything below runs on the mask's bounding box rather than the frame. + # The answer is the same -- nothing outside the mask was ever eligible, and + # `occupied` already carries the area blocked by points outside it. + offset_r = offset_c = 0 + if mask is not None: + window = _mask_window(mask, cell) + if window is None: + return [] + offset_r, offset_c = window[0].start, window[1].start + score, mask = score[window], mask[window] + if occupied is not None: + occupied = np.asarray(occupied, dtype=bool)[window] + + limit = threshold_value(score, mask, threshold_mode, threshold) + # NaN compares False against any threshold, so the invalid border is + # excluded here without a separate test. + eligible = score > limit + if mask is not None: + eligible &= mask + + if separation <= 1: + # Nothing to suppress, so there is no walk to feed and no reason to + # reduce anything: it is the pixels above the threshold, capped. What is + # already taken is dropped here rather than inside the walk, so that + # every remaining candidate is one that will be accepted -- which is + # what makes it safe to sort only the best `max_points` of them. + if occupied is not None: + eligible = eligible & ~np.asarray(occupied, dtype=bool) + rows, cols = _ordered_candidates(score, eligible, max_points) + points = suppress(rows, cols, score.shape, 0, max_points) + else: + rows, cols = _block_best(score, eligible, cell) + order = np.argsort(-score[rows, cols], kind='stable') + points = suppress(rows[order], cols[order], score.shape, separation, max_points, occupied) + + if points and (offset_r or offset_c): + shifted = np.asarray(points) + (offset_r, offset_c) + points = list(zip(shifted[:, 0].tolist(), shifted[:, 1].tolist())) + return points + + +def select_lattice(score, mask=None, pitch=12, threshold=DEFAULT_THRESHOLD, + threshold_mode='quality', max_points=DEFAULT_MAX_POINTS, occupied=None): + """Pick subsets on a regular grid, optionally dropping the weak ones. + + Uniform sampling is not always the wrong answer -- for full-field work the + point is to cover the surface evenly, not to find the best features. This + reproduces that behaviour inside the same pipeline, so a regular grid is a + choice of selector rather than a separate code path. + + :param score: score image, ``NaN`` where invalid + :type score: numpy.ndarray + :param mask: boolean array restricting where points may be placed + :type mask: numpy.ndarray or None + :param pitch: grid step, as a scalar or a ``(rows, cols)`` pair + :type pitch: int or tuple + :param threshold: threshold in the units ``threshold_mode`` implies. Use 0 + to keep every grid position with a finite score. + :type threshold: float + :param threshold_mode: one of :data:`THRESHOLD_MODES` + :type threshold_mode: str + :param max_points: keep at most this many + :type max_points: int or None + :param occupied: positions already taken + :type occupied: numpy.ndarray or None + :return: ``(row, col)`` coordinates, in row-major grid order + :rtype: list[tuple[int, int]] + """ + pitch_r, pitch_c = _as_size_pair(pitch) + pitch_r, pitch_c = max(1, int(pitch_r)), max(1, int(pitch_c)) + height, width = score.shape + + lattice = np.zeros(score.shape, dtype=bool) + lattice[::pitch_r, ::pitch_c] = True + + limit = threshold_value(score, mask, threshold_mode, threshold) + eligible = lattice & (score > limit) + if mask is not None: + eligible &= mask + + rows, cols = np.nonzero(eligible) + taken = None if occupied is None else np.asarray(occupied, dtype=bool) + points = [] + for row, col in zip(rows.tolist(), cols.tolist()): + if taken is not None and taken[row, col]: + continue + points.append((row, col)) + if max_points is not None and len(points) >= max_points: + break + return points + + +#: Selectors by name, for the same reason evaluators are registered by name: a +#: selection entry stores which one it wants, and the interface builds its +#: controls from the signature rather than hard-coding a panel per selector. +SELECTORS = { + 'peaks': select_peaks, + 'lattice': select_lattice, +} + + +def select(score, mask=None, selector='peaks', occupied=None, **params): + """Run a named selector. + + :param score: score image, ``NaN`` where invalid + :type score: numpy.ndarray + :param mask: boolean array restricting where points may be placed + :type mask: numpy.ndarray or None + :param selector: ``'peaks'`` or ``'lattice'`` + :type selector: str + :param occupied: positions already taken + :type occupied: numpy.ndarray or None + :param params: selector-specific parameters. Parameters this selector does + not take are ignored rather than raising, so that one set of defaults -- + ``separation`` for ``peaks``, ``pitch`` for ``lattice`` -- can be carried + around and handed to either. + :return: ``(n_points, 2)`` integer array of ``(row, col)`` coordinates + :rtype: numpy.ndarray + :raises ValueError: if ``selector`` is not a known selector + """ + if selector not in SELECTORS: + known = ', '.join(sorted(SELECTORS)) + raise ValueError(f"Unknown selector {selector!r}. Known selectors: {known}.") + function = SELECTORS[selector] + accepted = set(inspect.signature(function).parameters) + kwargs = {k: v for k, v in params.items() if k in accepted} + points = function(score, mask=mask, occupied=occupied, **kwargs) + return as_point_array(points) + + +def as_point_array(points): + """Normalise a list of ``(row, col)`` pairs to an ``(n, 2)`` integer array. + + :param points: the coordinates, possibly empty + :type points: sequence + :return: ``(n, 2)`` array of ``intp``; ``(0, 2)`` when empty, so that callers + can index columns without a special case + :rtype: numpy.ndarray + """ + if len(points) == 0: + return np.empty((0, 2), dtype=np.intp) + return np.asarray(points, dtype=np.intp).reshape(-1, 2) + + +def decimate(points, stride=None, count=None): + """Thin a list of coordinates, keeping their order. + + This drops points that were already chosen. It is not the same as asking + for fewer points up front: a wider separation re-selects, moving every point, + whereas decimation leaves the survivors exactly where they were and simply + keeps fewer of them. That is what you want when the selection is right and + only the count is too high for the computation you are about to run -- and it + is why the two are separate controls. It is also why it is not the separation + control: it thins a list, and a list thinned by score puts most of what + survives back-to-back on the same feature. + + ``select_peaks`` returns its points best first, so a stride keeps an even + sample across the whole quality range rather than the top slice of it. + + :param points: ``(row, col)`` coordinates + :type points: sequence + :param stride: keep every ``stride``-th point; ignored when ``None`` + :type stride: int or None + :param count: keep at most this many, spread evenly through the sequence; + applied after ``stride`` + :type count: int or None + :return: the kept coordinates, in the original order + :rtype: list + """ + points = list(points) + if stride is not None and stride > 1: + points = points[::int(stride)] + if count is not None and 0 <= count < len(points): + if count == 0: + return [] + keep = np.linspace(0, len(points) - 1, int(count)).round().astype(int) + points = [points[i] for i in dict.fromkeys(keep.tolist())] + return points + + +def merge_points(literal, picked): + """Combine hand-picked and automatically selected points, literals first. + + Duplicates are dropped, keeping the first occurrence, so a hand-picked point + that the selector would also have chosen appears exactly once. Crowding is + not handled here -- it is prevented earlier, by stamping the literal points + into the occupancy array before selection runs. + + :param literal: ``(row, col)`` coordinates contributed by ``points``-role entries + :type literal: sequence + :param picked: ``(row, col)`` coordinates from the selector + :type picked: sequence + :return: ``(n_points, 2)`` integer array + :rtype: numpy.ndarray + """ + combined = np.vstack([as_point_array(literal), as_point_array(picked)]) + if not len(combined): + return combined + # Deduplicating through a Python set costs more than the selection itself at + # twenty thousand points. Folding each coordinate pair into one integer makes + # it a single `unique`, and sorting the indices it returns puts the survivors + # back in the order they arrived -- which is the part `unique` alone loses. + # The fold is taken relative to the lowest coordinate, so that it stays + # one-to-one even if a caller hands in a negative one. + low_row, low_col = int(combined[:, 0].min()), int(combined[:, 1].min()) + span = int(combined[:, 1].max()) - low_col + 1 + flat = (combined[:, 0] - low_row) * span + (combined[:, 1] - low_col) + keep = np.sort(np.unique(flat, return_index=True)[1]) + return combined[keep] + + +def occupancy(points, shape, radius): + """Positions blocked by an existing set of points. + + Used to give hand-picked points precedence: the selector is handed this + array and cannot place anything within ``radius`` of one of them. + + :param points: ``(row, col)`` coordinates + :type points: sequence + :param shape: ``(rows, cols)`` of the image + :type shape: tuple[int, int] + :param radius: radius blocked around each point + :type radius: float + :return: boolean array indexed ``[row, col]`` + :rtype: numpy.ndarray + """ + taken = np.zeros(shape, dtype=bool) + radius = int(radius) + height, width = shape + points = as_point_array(points) + if radius <= 0: + # Nothing to stamp but the points themselves, so the whole thing is one + # indexed assignment rather than a Python step per point. + rows, cols = points[:, 0], points[:, 1] + inside = (rows >= 0) & (rows < height) & (cols >= 0) & (cols < width) + taken[rows[inside], cols[inside]] = True + return taken + + disc = _disc(radius) + for point in points: + row, col = int(point[0]), int(point[1]) + if not (0 <= row < height and 0 <= col < width): + continue + r0, r1 = max(0, row - radius), min(height, row + radius + 1) + c0, c1 = max(0, col - radius), min(width, col + radius + 1) + taken[r0:r1, c0:c1] |= disc[r0 - row + radius:r1 - row + radius, + c0 - col + radius:c1 - col + radius] + return taken diff --git a/pyidi/selection_geometry.py b/pyidi/selection_geometry.py new file mode 100644 index 0000000..e394dfe --- /dev/null +++ b/pyidi/selection_geometry.py @@ -0,0 +1,317 @@ +"""Pure-numpy geometry helpers for ROI/point selection GUIs. + +This module has no GUI-toolkit dependencies (no tkinter, no PyQt6) so it can +be imported from anywhere without pulling in optional GUI extras. It only +depends on ``numpy`` and ``matplotlib.path.Path``. + +The functions here were moved, unchanged in behaviour, out of the selection +GUIs, so that the geometry can be tested without a GUI toolkit and shared +between the napari GUI, ``SelectionGUI`` and ``SelectionGUIOld``. + +They do **not** share a common coordinate convention -- each docstring below +states explicitly which convention that function uses, since this differs +between ``get_roi_grid`` (row/column) and the rest (x/y). The two families +also differ in boundary handling: ``get_roi_grid`` steps with +``np.arange(low, high, step)`` and so excludes the far edge, while +``rois_inside_polygon`` uses ``np.arange(min, max + 1, step)`` and can +include it. They are therefore *not* axis-swapped versions of each other and +can return structurally different grids for the same geometry. Preserve this +when editing -- ``tests/test_selection_geometry.py`` pins it deliberately. + +All four functions accept an anisotropic ``subset_size``/``roi_size``, i.e. a +``(height, width)`` pair instead of a single scalar. ``get_roi_grid`` already +had this (``roi_size=(roi_size_y, roi_size_x)``) and its signature is +unchanged here. ``points_along_polygon``, ``rois_inside_polygon`` and +``rois_inside_mask`` now normalize a scalar or a ``(height, width)`` pair +through the private ``_as_size_pair`` helper; a scalar still produces +byte-identical output to before. +""" + +import numpy as np +from matplotlib.path import Path + + +def _as_size_pair(subset_size): + """Normalize a scalar or (height, width) subset size to a (h, w) pair. + + Integrality is preserved rather than always casting to float: if + ``subset_size`` is a scalar integer, or a pair of integers, ``h`` and + ``w`` are returned as Python ``int``; otherwise (any float involved) + they are returned as ``float``. This matters downstream -- + ``IDIMethod.set_points()`` treats non-integer point coordinates as + sub-pixel and warns about them, and GUI callers always pass integer + sizes (``QSpinBox`` values), so they must keep getting integer grid + coordinates out, not a spurious sub-pixel warning on every selection. + + Parameters + ---------- + subset_size : int, float, or a length-2 sequence of int/float + A scalar (broadcast to both axes) or a ``(height, width)`` pair, + i.e. ``(y_extent, x_extent)``. May be a plain Python number or a + 0-d/1-d numpy array or scalar. + + Returns + ------- + tuple of int or tuple of float + ``(h, w)``, as ``int`` if ``subset_size`` was integral, ``float`` + otherwise. + + Raises + ------ + ValueError + If ``subset_size`` is a sequence whose length is not 2. + """ + arr = np.asarray(subset_size) + + if arr.ndim != 0 and arr.shape != (2,): + raise ValueError(f'subset_size must be a scalar or a (height, width) pair, got shape {arr.shape}.') + + cast = int if np.issubdtype(arr.dtype, np.integer) else float + + if arr.ndim == 0: + s = cast(arr) + return s, s + + return cast(arr[0]), cast(arr[1]) + + +def get_roi_grid(polygon_points, roi_size, noverlap, deselect_polygon): + """Generate a regular grid of ROI centre points inside a polygon. + + Coordinate convention: this function works in (y, x) / (row, column) + order throughout. ``polygon_points`` is expected as an array of + (row, col) points (or, if given as a 2-row array, it is transposed so + that rows become points), ``roi_size`` is ``(roi_size_y, roi_size_x)``, + and the returned candidate points are ``(row, col)`` pairs. This is the + convention used by ``pyidi/GUIs/gui.py``. + + Parameters + ---------- + polygon_points : array_like + Vertices of the selection polygon, as an array of shape ``(N, 2)`` + with ``(row, col)`` points, or shape ``(2, N)`` (will be + transposed). + roi_size : tuple of int + ``(roi_size_y, roi_size_x)``, i.e. the ROI size in the (row, + column) directions. Must have length 2 -- the two entries may + differ (anisotropic ROI size). + noverlap : int + Overlap, in pixels, between neighbouring ROIs along each axis. The + centre-to-centre spacing along axis ``i`` is ``roi_size[i] - + noverlap``. + deselect_polygon : sequence of two sequences + ``(rows, cols)`` coordinates of a polygon whose interior should be + excluded from the returned grid. Pass two empty sequences (e.g. + ``[[], []]``) to disable deselection. + + Returns + ------- + numpy.ndarray + Integer array of shape ``(M, 2)`` with the ``(row, col)`` centre + points of the ROIs that fall inside ``polygon_points`` and outside + ``deselect_polygon``. + """ + if len(roi_size) != 2: + raise ValueError(f'roi_size must be a tuple of length 2, got length {len(roi_size)}.') + + cent_dist_0 = roi_size[0] - noverlap + cent_dist_1 = roi_size[1] - noverlap + + points = np.array(polygon_points) + if points.shape[0] == 2: + points = points.T + + low_0 = np.min(points[:, 0]) + high_0 = np.max(points[:, 0]) + low_1 = np.min(points[:, 1]) + high_1 = np.max(points[:, 1]) + + candidates_0 = np.arange(low_0, high_0, cent_dist_0) + candidates_1 = np.arange(low_1, high_1, cent_dist_1) + candidates = np.concatenate([_.flatten()[:, None] for _ in np.meshgrid(candidates_0, candidates_1)], axis=1) + + path = Path(points) + mask = path.contains_points(candidates) + + if len(deselect_polygon[0]) and len(deselect_polygon[1]): + path_deselect = Path(np.array(deselect_polygon).T) + mask_deselect = path_deselect.contains_points(candidates) + mask = np.logical_and(mask, np.logical_not(mask_deselect)) + + return np.round(candidates[mask]).astype(int) + + +def points_along_polygon(polygon, subset_size, spacing=0): + """Place evenly-spaced points along the segments of an open polygon. + + Coordinate convention: (x, y) throughout -- ``polygon`` is a sequence + of ``(x, y)`` vertices, as stored by ``SelectionGUIOld``, and the returned + points are ``(x, y)`` pairs (each shifted by -0.5 and rounded to the + nearest integer, to align with pixel centres). + + Parameters + ---------- + polygon : sequence of (x, y) + Vertices of an open polyline; points are generated along each + consecutive segment ``polygon[i] -> polygon[i + 1]``. + subset_size : float or (height, width) + Size of the subset/ROI, as a scalar or a ``(height, width)`` pair + (``height`` is the vertical/y extent, ``width`` the + horizontal/x extent). Combined with ``spacing``, this sets the + step between consecutive points along each segment: the step is + the extent of the subset projected along the segment's direction + (see implementation note below), which reduces to + ``subset_size + spacing`` for a square/scalar subset, regardless + of the segment's angle. + spacing : float, optional + Extra spacing added to the projected subset extent to get the + step between points. Default is 0. + + Returns + ------- + list of tuple + ``(x, y)`` points along the polygon, rounded to the nearest + integer (after a -0.5 pixel-centre shift). Returns an empty list + if ``polygon`` has fewer than 2 vertices. + """ + if len(polygon) < 2: + return [] + + h, w = _as_size_pair(subset_size) + + result_points = [] + + for i in range(len(polygon) - 1): + p1 = np.array(polygon[i]) + p2 = np.array(polygon[i + 1]) + segment = p2 - p1 + length = np.linalg.norm(segment) + + if length == 0: + continue + + direction = segment / length + + # Step = the elliptical extent of the (h, w) subset projected along + # the unit segment direction (dx, dy). For h == w == s this is + # s * sqrt(dx**2 + dy**2) == s for every angle (dx, dy is a unit + # vector), i.e. exactly the old isotropic behaviour -- do not + # "simplify" this to |dx|*w + |dy|*h, which is NOT equivalent (it + # gives 1.414*s on a 45-degree segment instead of s). + dx, dy = direction[0], direction[1] + extent = np.sqrt((dx * w) ** 2 + (dy * h) ** 2) + step = extent + spacing + if step <= 0: + step = 1 + + n_points = int(length // step) + + for j in range(n_points + 1): + pt = p1 + j * step * direction + result_points.append((round(pt[0] - 0.5), round(pt[1] - 0.5))) + + return result_points + + +def rois_inside_polygon(polygon, subset_size, spacing): + """Generate a regular grid of points inside a closed polygon. + + Coordinate convention: (x, y) throughout -- ``polygon`` is a sequence + of ``(x, y)`` vertices, as stored by ``SelectionGUIOld``, and the returned + points are ``(x, y)`` pairs. + + Parameters + ---------- + polygon : sequence of (x, y) + Vertices of a closed polygon. Must contain at least 3 points. + subset_size : float or (height, width) + Size of the subset/ROI, as a scalar or a ``(height, width)`` pair + (``height`` is the vertical/y extent, ``width`` the + horizontal/x extent). Combined with ``spacing`` this sets the + grid step along x (from ``width``) and y (from ``height``). + spacing : float + Extra spacing added to ``subset_size`` to get the grid step. + + Returns + ------- + list of tuple + ``(x, y)`` grid points that fall inside ``polygon``. Returns an + empty list if ``polygon`` has fewer than 3 vertices. + """ + if len(polygon) < 3: + return [] + + h, w = _as_size_pair(subset_size) + + polygon = np.array(polygon) + min_x, max_x = int(np.floor(np.min(polygon[:, 0]))), int(np.ceil(np.max(polygon[:, 0]))) + min_y, max_y = int(np.floor(np.min(polygon[:, 1]))), int(np.ceil(np.max(polygon[:, 1]))) + + step_x = w + spacing + if step_x <= 0: + step_x = 1 # minimum step to avoid infinite loop + step_y = h + spacing + if step_y <= 0: + step_y = 1 # minimum step to avoid infinite loop + xs = np.arange(min_x, max_x+1, step_x) + ys = np.arange(min_y, max_y+1, step_y) + + grid_x, grid_y = np.meshgrid(xs, ys) + points = np.vstack([grid_x.ravel(), grid_y.ravel()]).T + + mask = Path(polygon).contains_points(points) + return [tuple(p) for p in points[mask]] + + +def rois_inside_mask(mask, subset_size, spacing): + """Generate a regular grid of points inside a boolean mask. + + Coordinate convention: the input ``mask`` is indexed as ``mask[y, x]`` + (row, col), and the returned points are ``(y, x)`` pairs -- the + opposite convention from ``points_along_polygon`` and + ``rois_inside_polygon``. + + Parameters + ---------- + mask : numpy.ndarray + 2D boolean array of shape ``(h, w)``, indexed as ``mask[y, x]``. + subset_size : float or (height, width) + Size of the subset/ROI, as a scalar or a ``(height, width)`` pair + (``height`` is the vertical/y extent, ``width`` the + horizontal/x extent). Combined with ``spacing`` this sets the + grid step along y (from ``height``) and x (from ``width``). + spacing : float + Extra spacing added to ``subset_size`` to get the grid step. + + Returns + ------- + list of tuple + ``(y, x)`` grid points for which ``mask`` is True. + """ + size_h, size_w = _as_size_pair(subset_size) + + step_x = size_w + spacing + if step_x <= 0: + step_x = 1 + step_y = size_h + spacing + if step_y <= 0: + step_y = 1 + + h, w = mask.shape + # Defensive cast to int: these are pixel indices used directly to index + # `mask` below. For integer subset_size/spacing (the normal case -- + # _as_size_pair now preserves integrality) step_x/step_y and thus xs/ys + # are already int, so this is a no-op. But subset_size may legitimately + # be a float per the docstring above, in which case np.arange yields a + # float array that mask[...] cannot be indexed with -- cast defensively + # so a float subset_size degrades to truncated pixel indices instead of + # crashing. + xs = np.arange(0, w, step_x).astype(int) + ys = np.arange(0, h, step_y).astype(int) + grid_x, grid_y = np.meshgrid(xs, ys) + + candidate_points = np.vstack([grid_y.ravel(), grid_x.ravel()]).T # (y, x) + + # Only keep points where the mask is True + selected = [tuple(p) for p in candidate_points if mask[p[0], p[1]]] + return selected diff --git a/pyidi/tools.py b/pyidi/tools.py index d3b580a..1daf556 100644 --- a/pyidi/tools.py +++ b/pyidi/tools.py @@ -1,271 +1,13 @@ import numpy as np -import matplotlib.pyplot as plt -import matplotlib.patches as patches - -import tkinter as tk -from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk -from matplotlib.figure import Figure - -from multiprocessing import Pool -from tqdm import tqdm import logging import logging.handlers -class ManualROI: - """Manual ROI selection.""" - - def __init__(self, video, roi_size, single=False, verbose=0): - """Manually select region of interest. - - :param video: parent object - :type video: object - :param roi_size: size of region of interest (dy, dx) - :type roi_size: tuple, list - :param single: if True, ONLY ONE ROI can be selected, defaults to False - :type single: bool, optional - :param verbose: Show text, defaults to 0 - :type verbose: int, optional - """ - self.roi_size = roi_size - self.image = video.reader.mraw[0] - self.verbose = verbose - - # Tkinter root and matplotlib figure - root = tk.Tk() - root.title('Pick point') - fig = Figure(figsize=(15, 7)) - ax = fig.add_subplot(111) - ax.grid(False) - ax.imshow(self.image, cmap='gray') - plt.show() - - # Embed figure in tkinter winodw - canvas = FigureCanvasTkAgg(fig, root) - canvas.get_tk_widget().pack(side='top', fill='both', expand=1) - NavigationToolbar2Tk(canvas, root) - - if self.verbose: - print('SHIFT + LEFT mouse button to pick a pole.\nRIGHT mouse button to erase the last pick.') - - - self.point = [[], []] - line, = ax.plot(self.point[1], self.point[0], 'r.') - - self.rectangles = [] - self.rectangles.append(patches.Rectangle((0, 0), 10, 10, fill=False, alpha=0)) - ax.add_patch(self.rectangles[-1]) - - self.shift_is_held = False - def on_key_press(event): - """Function triggered on key press (shift).""" - if event.key == 'shift': - self.shift_is_held = True - - def on_key_release(event): - """Function triggered on key release (shift).""" - if event.key == 'shift': - self.shift_is_held = False - - def onclick(event): - if event.button == 1 and self.shift_is_held: - if event.xdata is not None and event.ydata is not None: - if single: - self.point[0] = [int(np.round(event.ydata))] - self.point[1] = [int(np.round(event.xdata))] - else: - self.point[0].append(int(np.round(event.ydata))) - self.point[1].append(int(np.round(event.xdata))) - if self.verbose: - print(f'y: {np.round(event.ydata):5.0f}, x: {np.round(event.xdata):5.0f}') - - elif event.button == 3 and self.shift_is_held and not single: - if self.verbose: - print('Deleted the last point...') - del self.point[1][-1] - del self.point[0][-1] - del self.rectangles[-1] - - line.set_xdata(self.point[1]) - line.set_ydata(self.point[0]) - - if self.point[0]: - [p.remove() for p in reversed(ax.patches)] - self.rectangles = [] - for i, (p0, p1) in enumerate(zip(self.point[0], self.point[1])): - self.rectangles.append(patches.Rectangle((p1 - self.roi_size[1]//2, p0 - self.roi_size[0]//2), - self.roi_size[1], self.roi_size[0], fill=False, color='r', linewidth=2)) - ax.add_patch(self.rectangles[-1]) - - fig.canvas.draw() - - def handle_close(event): - """On closing.""" - self.points = np.asarray(self.point).T - if self.verbose: - for i, point in enumerate(self.polygon): - print(f'{i+1}. point: x ={point[1]:5.0f}, y ={point[0]:5.0f}') - - # Connecting functions to event manager - fig.canvas.mpl_connect('key_press_event', on_key_press) - fig.canvas.mpl_connect('key_release_event', on_key_release) - fig.canvas.mpl_connect('button_press_event', onclick) - # on closing the figure - fig.canvas.mpl_connect('close_event', handle_close) - - root.mainloop() - - -class GridOfROI: - """ - Automatic simple ROI grid generation. - - Different from RegularROIGrid in that it gets a regular grid and only - then checks if all points are inside polygon. This yields a more regular - and full grid. Does not contain sssig filter. - """ - def __init__(self, video=None, roi_size=(7, 7), noverlap=0, verbose=0): - """ - - :param video: parent object of video - :type video: object - :param roi_size: Size of the region of interest (y, x), defaults to (7, 7) - :type roi_size: tuple, list, optional - :param noverlap: number of pixels that overlap between neighbouring ROIs - :type noverlap: int, optional - :param sssig_filter: minimum value of SSSIG that the roi must have, defaults to None - :type sssig_filter: None, float, optional - :param verbose: Show text, defaults to 1 - :type verbose: int, optional - """ - self.roi_size = roi_size - self.verbose = verbose - - self.noverlap = int(noverlap) - - self.cent_dist_0 = self.roi_size[0] - self.noverlap - self.cent_dist_1 = self.roi_size[1] - self.noverlap - - if video is not None: - self.image = video.reader.mraw[0] - self.pick_window() - else: - print('set the polygon points in self.polygon and call the `get_roi_grid` method') - - def pick_window(self): - # Tkinter root and matplotlib figure - root = tk.Tk() - root.title('Pick points') - fig = Figure(figsize=(15, 7)) - ax = fig.add_subplot(111) - ax.grid(False) - ax.imshow(self.image, cmap='gray') - plt.show() - - # Embed figure in tkinter winodw - canvas = FigureCanvasTkAgg(fig, root) - canvas.get_tk_widget().pack(side='top', fill='both', expand=1) - NavigationToolbar2Tk(canvas, root) - - # Initiate polygon - self.polygon = [[], []] - line, = ax.plot(self.polygon[1], self.polygon[0], 'r.-') - - if self.verbose: - print('SHIFT + LEFT mouse button to pick a pole.\nRIGHT mouse button to erase the last pick.') - - self.shift_is_held = False - - def on_key_press(event): - """Function triggered on key press (shift).""" - if event.key == 'shift': - self.shift_is_held = True - - def on_key_release(event): - """Function triggered on key release (shift).""" - if event.key == 'shift': - self.shift_is_held = False - - def onclick(event): - if event.button == 1 and self.shift_is_held: - if event.xdata is not None and event.ydata is not None: - self.polygon[1].append(int(np.round(event.xdata))) - self.polygon[0].append(int(np.round(event.ydata))) - if self.verbose: - print(f'y: {np.round(event.ydata):5.0f}, x: {np.round(event.xdata):5.0f}') - - elif event.button == 3 and self.shift_is_held: - if self.verbose: - print('Deleted the last point...') - del self.polygon[1][-1] - del self.polygon[0][-1] - - line.set_xdata(self.polygon[1]) - line.set_ydata(self.polygon[0]) - fig.canvas.draw() - - def handle_close(event): - """On closing.""" - self.polygon = np.asarray(self.polygon).T - if self.verbose: - for i, point in enumerate(self.polygon): - print(f'{i+1}. point: x ={point[1]:5.0f}, y ={point[0]:5.0f}') - - self.points = self.get_roi_grid() - - # Connecting functions to event manager - fig.canvas.mpl_connect('key_press_event', on_key_press) - fig.canvas.mpl_connect('key_release_event', on_key_release) - fig.canvas.mpl_connect('button_press_event', onclick) - # on closing the figure - fig.canvas.mpl_connect('close_event', handle_close) - - root.mainloop() - - def get_roi_grid(self): - points = self.polygon - - low_0 = np.min(points[:, 0]) - high_0 = np.max(points[:, 0]) - low_1 = np.min(points[:, 1]) - high_1 = np.max(points[:, 1]) - - rois = [] - for i in range(low_0+self.cent_dist_0, high_0-self.cent_dist_0, self.cent_dist_0): - for j in range(low_1+self.cent_dist_1, high_1-self.cent_dist_1, self.cent_dist_1): - if inside_polygon(i, j, self.polygon): - rois.append([i, j]) - return np.asarray(rois) - - -def inside_polygon(x, y, points): - """ - Return True if a coordinate (x, y) is inside a polygon defined by - a list of verticies [(x1, y1), (x2, x2), ... , (xN, yN)]. - - Reference: http://www.ariel.com.au/a/python-point-int-poly.html - """ - n = len(points) - inside = False - p1x, p1y = points[0] - for i in range(1, n + 1): - p2x, p2y = points[i % n] - if y > min(p1y, p2y): - if y <= max(p1y, p2y): - if x <= max(p1x, p2x): - if p1y != p2y: - xinters = (y - p1y) * (p2x - p1x) / \ - (p2y - p1y) + p1x - if p1x == p2x or x <= xinters: - inside = not inside - p1x, p1y = p2x, p2y - return inside - def update_docstring(target_method, doc_method=None, delimiter='---', added_doc=''): """ Update the docstring in target_method with the docstring from doc_method. - + :param target_method: The method that waits for the docstring :type target_method: method :param doc_method: The method that holds the desired docstring @@ -276,7 +18,7 @@ def update_docstring(target_method, doc_method=None, delimiter='---', added_doc= docstring = target_method.__doc__.split(delimiter) leading_spaces = len(docstring[1].replace('\n', '')) - len(docstring[1].replace('\n', '').lstrip(' ')) - + if doc_method is not None: if doc_method.__doc__: docstring[1] = doc_method.__doc__ @@ -291,7 +33,7 @@ def update_docstring(target_method, doc_method=None, delimiter='---', added_doc= def split_points(points, processes): """Split the array of points to different processes. - + :param points: Array of points (2d) :type points: numpy array :param processes: number of processes @@ -301,7 +43,7 @@ def split_points(points, processes): step = points.shape[0]//processes rest = points.shape[0]%processes points_split = [] - + last_point = 0 for i in range(processes): this_step = step @@ -315,10 +57,10 @@ def split_points(points, processes): # @nb.njit def get_gradient(image): """Fast gradient computation. - + Compute the gradient of image in both directions using central difference weights over 3 points. - + !!! WARNING: The edges are excluded from the analysis and the returned image is smaller then original. @@ -333,7 +75,7 @@ def get_gradient(image): im1 = image[:, 2:] im2 = image[:, :-2] Gx = (im1 - im2)/2 - + return Gx[1:-1], Gy[:, 1:-1] @@ -351,6 +93,6 @@ def setup_logger(logger_name, level="DEBUG", backup_count=1): logger.addHandler(file_handler) return logger - - - \ No newline at end of file + + + diff --git a/pyidi/video_reader.py b/pyidi/video_reader.py index c2bdd8d..313d282 100644 --- a/pyidi/video_reader.py +++ b/pyidi/video_reader.py @@ -154,11 +154,13 @@ def get_frame(self, frame_number, *args, **kwargs): :param frame_number: frame number :type frame_number: int - :param args: additional arguments to be passed to the image readers to handle - multiple channels in image - :param kwargs: additional keyword arguments forwarded to image/video reader methods + :param args: additional arguments to be passed to the image readers to + handle multiple channels in image + :param kwargs: additional keyword arguments forwarded to image/video + reader methods :type kwargs: dict :return: image (monochrome) + :rtype: numpy.ndarray """ if not 0 <= frame_number < self.N: raise ValueError("Frame number exceeds total frame number!") diff --git a/pyproject.toml b/pyproject.toml index cda9900..a08a584 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -58,6 +58,8 @@ dev = [ "sdypy-io", "sphinx-book-theme", "sphinx-copybutton", + "sphinx-design", + "myst-parser", "nbsphinx", "nbsphinx_link", "ipykernel", diff --git a/tests/test_feature_selection_gui.py b/tests/test_feature_selection_gui.py new file mode 100644 index 0000000..b2d5ab3 --- /dev/null +++ b/tests/test_feature_selection_gui.py @@ -0,0 +1,1840 @@ +"""Tests for ``SelectionGUI``, the interface over the selection pipeline. + +Constructed headlessly, by the same recipe as +``tests/test_selection_gui_anisotropic.py``: + +* ``QT_QPA_PLATFORM=offscreen`` before Qt is imported, so Qt renders to its + software framebuffer instead of opening a display; +* ``sys.ps1`` set, so the constructor takes its "interactive" branch rather + than ``sys.exit(...)``; +* ``QApplication.exec`` neutralised, since that branch still calls it and would + otherwise block in the event loop. + +What is worth testing here is not the widgets but the *contract between the +interface and the pipeline*: that a threshold or mask edit re-derives points +from a cached score while a subset-size change pays for a recomputation, that +roles and visibility do what the rows say they do, and that undo puts things +back. Those are the properties the whole design rests on. + +Note that a window opens with one seeded ``Whole image`` mask row, so the +helpers below clear it when a test wants to reason about a region of its own. +""" +import os + +os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") + +import sys # noqa: E402 + +import numpy as np # noqa: E402 +import pytest # noqa: E402 + +pytest.importorskip("PyQt6") + +from PyQt6 import QtCore, QtGui, QtWidgets # noqa: E402 + +sys.ps1 = getattr(sys, "ps1", ">>> ") +QtWidgets.QApplication.exec = lambda self=None: 0 + +from pyidi.GUIs.feature_selection import ( # noqa: E402 + REDRAW_BUDGET_MS, STEP_FIND, STEP_HINTS, STEP_MASK, SelectionGUI) + + +def make_image(): + """A speckled frame, so every subset has something to score.""" + rng = np.random.default_rng(4) + return rng.integers(0, 255, size=(160, 240), dtype=np.uint8) + + +def make_gui(**kwargs): + """A headless window on a fresh synthetic frame, as the user gets it.""" + return SelectionGUI(make_image(), **kwargs) + + +def empty_gui(**kwargs): + """A window with the seeded whole-image row removed, so nothing is masked.""" + gui = make_gui(**kwargs) + gui.pipeline.entries = [] + gui.active_index = None + gui.undo_stack = [] + gui.refresh() + return gui + + +def rect(r0, c0, r1, c1): + """A rectangular polygon as ``(row, col)`` vertices.""" + return [(r0, c0), (r0, c1), (r1, c1), (r1, c0)] + + +def gui_with_region(**kwargs): + """A window whose only mask is one polygon covering most of the frame.""" + gui = empty_gui(**kwargs) + gui.pipeline.add_entry('polygon', rect(20, 20, 140, 220)) + gui.active_index = 0 + gui.refresh() + return gui + + +# --------------------------------------------------------------------------- +# Construction +# --------------------------------------------------------------------------- + +def test_constructor_accepts_a_2d_image(): + gui = make_gui() + try: + assert gui.frame.shape == (160, 240) + assert gui.pipeline.subset_size == (11, 11) + finally: + gui.close() + + +def test_constructor_accepts_a_frame_stack(): + stack = np.stack([make_image(), make_image()]) + gui = SelectionGUI(stack) + try: + np.testing.assert_array_equal(gui.frame, stack[0]) + finally: + gui.close() + + +def test_constructor_rejects_an_unusable_input(): + with pytest.raises(TypeError, match='VideoReader'): + SelectionGUI('not a video') + + +def test_constructor_normalises_an_anisotropic_subset_size(): + gui = make_gui(subset_size=(21, 7)) + try: + assert gui.pipeline.subset_size == (21, 7) + assert not gui.square_check.isChecked() + assert gui.width_spin.isEnabled() + finally: + gui.close() + + +def test_the_image_is_not_transposed(): + """The whole module works in (row, col); nothing should flip the frame.""" + gui = make_gui() + try: + np.testing.assert_array_equal(gui.image_item.image, gui.frame) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Steps +# --------------------------------------------------------------------------- + +def test_both_tabs_are_separately_reachable(): + gui = make_gui() + try: + for name in (STEP_FIND, STEP_MASK): + gui.select_step(name) + assert gui.step == name + assert gui.step_stack.currentWidget() is gui.step_pages[name] + finally: + gui.close() + + +def test_the_window_opens_on_evaluate_and_select(): + """Evaluate and select do not depend on the mask, so they come first.""" + gui = make_gui() + try: + assert gui.step == STEP_FIND + assert list(gui.step_pages) == [STEP_FIND, STEP_MASK] + finally: + gui.close() + + +def test_the_tabs_are_not_numbered(): + """Numbering would imply an order the pipeline does not have.""" + gui = make_gui() + try: + for name, button in gui.step_buttons.items(): + assert button.text() == name + finally: + gui.close() + + +def test_exactly_one_tab_reads_as_active(): + gui = make_gui() + try: + for step in (STEP_FIND, STEP_MASK, STEP_FIND): + gui.select_step(step) + checked = [name for name, b in gui.step_buttons.items() if b.isChecked()] + assert checked == [step] + finally: + gui.close() + + +def test_the_checked_tab_and_tool_are_styled_not_left_to_the_theme(): + """A default theme separates checked from unchecked by a shade or two. + + Which is not a difference you can find across a panel, and this interface + asks the question twice -- which tab, and which tool. + """ + gui = make_gui() + try: + toolbar = gui.step_buttons[STEP_FIND].parent() + tools = gui.tool_buttons['polygon'].parent() + for widget in (toolbar, tools): + assert ':checked' in widget.styleSheet() + # Only the checked state, so unchecked buttons stay native and the + # window does not have to carry a theme of its own. + assert 'QPushButton {' not in widget.styleSheet() + finally: + gui.close() + + +def test_evaluate_and_select_share_one_panel(): + gui = make_gui() + try: + page = gui.step_pages[STEP_FIND] + titles = {box.title() for box in page.findChildren(QtWidgets.QGroupBox)} + assert {'Evaluate', 'Select'} <= titles + finally: + gui.close() + + +def test_the_highlight_is_a_mask_tab_cue(): + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + assert gui.highlight_scatter.isVisible() + gui.select_step(STEP_FIND) + assert not gui.highlight_scatter.isVisible() + finally: + gui.close() + + +def test_selecting_without_a_mask_says_so_rather_than_failing(): + gui = empty_gui() + try: + gui.select_step(STEP_FIND) + assert 'mask' in gui.select_note.text().lower() + assert gui.get_points().shape == (0, 2) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# The seeded whole-image row +# --------------------------------------------------------------------------- + +def test_the_window_opens_with_candidates_over_the_whole_frame(): + """There has to be something to trim before trimming is a workflow.""" + gui = make_gui() + try: + assert len(gui.pipeline.entries) == 1 + assert gui.pipeline.entries[0].label == 'Whole image' + assert gui.pipeline.entries[0].role == 'mask' + assert gui.pipeline.mask.all() + assert len(gui.get_points()) > 0 + finally: + gui.close() + + +def test_the_seeded_row_costs_one_evaluation_and_no_more(): + gui = make_gui() + try: + assert gui.pipeline.store.n_evaluations == 1 + gui.get_points() + assert gui.pipeline.store.n_evaluations == 1 + finally: + gui.close() + + +def test_the_seeded_row_is_an_ordinary_row(): + gui = make_gui() + try: + gui.entry_list.item(0).setCheckState(QtCore.Qt.CheckState.Unchecked) + assert gui.get_points().shape == (0, 2) + gui.entry_list.item(0).setCheckState(QtCore.Qt.CheckState.Checked) + assert len(gui.get_points()) > 0 + + gui.active_index = 0 + gui.delete_active() + assert gui.pipeline.entries == [] + assert gui.get_points().shape == (0, 2) + finally: + gui.close() + + +def test_the_seeded_row_can_be_trimmed_with_the_deselect_brush(): + gui = make_gui() + try: + before = gui.pipeline.mask.sum() + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.select_tool('erase') + gui.brush_start() + gui.brush_move((80, 120)) + gui.brush_end() + assert 0 < gui.pipeline.mask.sum() < before + finally: + gui.close() + + +def test_drawing_a_region_adds_to_the_seeded_one(): + """The combined mask is a union, so a new region does not replace it.""" + gui = make_gui() + try: + gui.pipeline.add_entry('polygon', rect(20, 20, 60, 60)) + gui.refresh() + assert len(gui.pipeline.entries) == 2 + assert gui.pipeline.mask.all() + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# The contract that makes the interface feel live +# --------------------------------------------------------------------------- + +def test_a_threshold_change_does_not_re_evaluate(): + gui = gui_with_region() + try: + evaluations = gui.pipeline.store.n_evaluations + assert evaluations >= 1 + before = len(gui.get_points()) + gui.threshold_slider.setValue(990) + assert gui.pipeline.store.n_evaluations == evaluations + assert len(gui.get_points()) < before + finally: + gui.close() + + +def test_a_separation_change_does_not_re_evaluate(): + gui = gui_with_region() + try: + evaluations = gui.pipeline.store.n_evaluations + before = len(gui.get_points()) + gui.separation_spin.setValue(30) + assert gui.pipeline.store.n_evaluations == evaluations + assert len(gui.get_points()) < before + finally: + gui.close() + + +def test_a_mask_edit_does_not_re_evaluate(): + gui = gui_with_region() + try: + evaluations = gui.pipeline.store.n_evaluations + gui.pipeline.entries[0].geometry = rect(30, 30, 90, 120) + gui.refresh() + assert gui.pipeline.store.n_evaluations == evaluations + finally: + gui.close() + + +def test_a_subset_size_change_does_re_evaluate(): + gui = gui_with_region() + try: + evaluations = gui.pipeline.store.n_evaluations + gui.height_spin.setValue(21) + gui.get_points() + assert gui.pipeline.store.n_evaluations > evaluations + finally: + gui.close() + + +def test_changing_the_evaluator_re_evaluates_and_keeps_the_old_score_cached(): + gui = gui_with_region() + try: + gui.get_points() + evaluations = gui.pipeline.store.n_evaluations + gui.evaluator_combo.setCurrentIndex(gui.evaluator_combo.findData('gradient_direction')) + gui.get_points() + assert gui.pipeline.store.n_evaluations == evaluations + 1 + + # Switching back must be free: the Shi-Tomasi array is still in the cache. + gui.evaluator_combo.setCurrentIndex(gui.evaluator_combo.findData('shi_tomasi')) + gui.get_points() + assert gui.pipeline.store.n_evaluations == evaluations + 1 + finally: + gui.close() + + +def test_the_parameter_panel_follows_the_evaluator(): + gui = make_gui() + try: + assert gui.param_widgets == {} # Shi-Tomasi takes no parameters + gui.evaluator_combo.setCurrentIndex(gui.evaluator_combo.findData('gradient_direction')) + assert set(gui.param_widgets) == {'direction'} + assert gui.param_widgets['direction']() == (0.0, 1.0) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Rows: visibility, roles, deletion +# --------------------------------------------------------------------------- + +def test_a_row_shows_its_label_role_and_count(): + gui = gui_with_region() + try: + text = gui.entry_list.item(0).text() + assert 'Polygon 1' in text + assert 'mask' in text + assert 'pts' in text + finally: + gui.close() + + +def test_unchecking_a_row_drops_its_contribution_without_deleting_it(): + gui = gui_with_region() + try: + assert len(gui.get_points()) > 0 + gui.entry_list.item(0).setCheckState(QtCore.Qt.CheckState.Unchecked) + assert gui.get_points().shape == (0, 2) + assert len(gui.pipeline.entries) == 1 + + gui.entry_list.item(0).setCheckState(QtCore.Qt.CheckState.Checked) + assert len(gui.get_points()) > 0 + finally: + gui.close() + + +def test_switching_a_role_changes_what_the_row_contributes(): + gui = gui_with_region() + try: + gui.active_index = 0 + automatic = gui.get_points() + gui.toggle_role() + assert gui.pipeline.entries[0].role == 'points' + literal = gui.get_points() + # As a points row the polygon lays out its own grid instead of being + # filtered, so the two results are different sets of points. + assert len(literal) > 0 + assert not np.array_equal(np.sort(literal, axis=0), np.sort(automatic, axis=0)) + assert not gui.pipeline.mask.any() + finally: + gui.close() + + +def test_deleting_a_row_removes_it(): + gui = gui_with_region() + try: + gui.active_index = 0 + gui.delete_active() + assert gui.pipeline.entries == [] + assert gui.entry_list.count() == 0 + finally: + gui.close() + + +def test_clear_all_goes_back_to_the_whole_frame(): + """Starting over means the state the window opens in, not a blank frame.""" + gui = gui_with_region() + try: + gui.pipeline.add_entry('points', [(50, 50)]) + gui.refresh() + gui.clear_all() + + assert [entry.label for entry in gui.pipeline.entries] == ['Whole image'] + assert gui.pipeline.mask.all() + assert len(gui.get_points()) + finally: + gui.close() + + +def test_clear_all_is_undoable(): + gui = gui_with_region() + try: + before = gui.get_points() + gui.clear_all() + gui.undo() + assert [entry.label for entry in gui.pipeline.entries] == ['Polygon 1'] + np.testing.assert_array_equal(gui.get_points(), before) + finally: + gui.close() + + +def test_deleting_the_whole_image_row_still_selects_nothing(): + """A different act from clearing: "not this area", not "forget what I did".""" + gui = make_gui() + try: + gui.active_index = 0 + gui.delete_active() + assert gui.pipeline.entries == [] + assert gui.get_points().shape == (0, 2) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Undo +# --------------------------------------------------------------------------- + +def test_undo_restores_a_deleted_row_at_its_original_position(): + gui = gui_with_region() + try: + first = gui.pipeline.entries[0] + gui.pipeline.add_entry('points', [(50, 50)]) + gui.refresh() + gui.active_index = 0 + gui.delete_active() + assert gui.pipeline.entries[0].kind == 'points' + + gui.undo() + assert gui.pipeline.entries[0] is first + assert gui.pipeline.entries[0].label == 'Polygon 1' + finally: + gui.close() + + +def test_undo_reverses_a_vertex_add(): + gui = empty_gui() + try: + gui.select_tool('polygon') + for position in [(20, 20), (20, 100), (100, 100)]: + gui.add_vertex(position) + entry = gui.pipeline.entries[0] + assert len(entry.geometry) == 3 + gui.undo() + assert len(entry.geometry) == 2 + finally: + gui.close() + + +def test_undo_reverses_a_brush_stroke(): + gui = empty_gui() + try: + gui.select_tool('brush') + gui.brush_start() + gui.brush_move((80, 120)) + gui.brush_end() + assert len(gui.pipeline.entries) == 1 + gui.undo() + assert gui.pipeline.entries == [] + finally: + gui.close() + + +def test_undo_reverses_a_deselection(): + gui = empty_gui() + try: + gui.select_tool('brush') + gui.brush_start() + gui.brush_move((80, 120)) + gui.brush_end() + painted = gui.pipeline.mask.sum() + assert painted > 0 + + gui.select_tool('erase') + gui.brush_start() + gui.brush_move((80, 120)) + gui.brush_end() + assert gui.pipeline.mask.sum() < painted + + gui.undo() + assert gui.pipeline.mask.sum() == painted + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Drawing +# --------------------------------------------------------------------------- + +def test_the_subset_border_pen_is_cosmetic(): + """Cosmetic means the width is in screen pixels, so it survives a zoom.""" + gui = gui_with_region() + try: + assert gui.roi_outline.pen().isCosmetic() + assert not gui.roi_outline.path().isEmpty() + finally: + gui.close() + + +def test_the_subset_overlay_covers_the_expected_area(): + gui = empty_gui(subset_size=(21, 7)) + try: + gui.pipeline.add_entry('points', [(80, 120)]) + gui.refresh() + overlay = gui.roi_overlay.image + covered = overlay[..., 3] != 0 + rows = np.flatnonzero(covered.any(axis=1)) + cols = np.flatnonzero(covered.any(axis=0)) + assert rows.max() - rows.min() + 1 == 21 + assert cols.max() - cols.min() + 1 == 7 + finally: + gui.close() + + +def test_hiding_the_subsets_clears_the_overlay(): + gui = gui_with_region() + try: + assert gui.roi_overlay.image is not None + gui.show_subsets.setChecked(False) + assert gui.roi_overlay.image is None + assert gui.roi_outline.path().isEmpty() + finally: + gui.close() + + +def test_the_score_overlay_is_transparent_on_the_invalid_border(): + gui = gui_with_region() + try: + gui.show_score.setChecked(True) + assert gui.score_overlay.isVisible() + rgba = gui.score_overlay.image + assert (rgba[:5, :, 3] == 0).all() + assert (rgba[80, 100:140, 3] > 0).all() + finally: + gui.close() + + +def test_the_score_overlay_sits_beside_the_selection_controls(): + """Seeing the score while thresholding is how you tell "too tight" from "nothing there".""" + gui = gui_with_region() + try: + page = gui.step_pages[STEP_FIND] + assert gui.show_score in page.findChildren(QtWidgets.QCheckBox) + assert gui.threshold_slider in page.findChildren(QtWidgets.QSlider) + finally: + gui.close() + + +def test_toggling_the_overlay_leaves_the_selection_alone(): + gui = gui_with_region() + try: + before = gui.get_points() + gui.show_score.setChecked(True) + gui.show_score.setChecked(False) + np.testing.assert_array_equal(gui.get_points(), before) + finally: + gui.close() + + +def test_a_large_selection_redraws_quickly(): + """Thousands of subsets must stay interactive, hence the raster-plus-path split.""" + import time + + gui = make_gui(subset_size=5) + try: + rng = np.random.default_rng(11) + rows = rng.integers(10, 150, 5000) + cols = rng.integers(10, 230, 5000) + points = np.column_stack([rows, cols]) + start = time.perf_counter() + gui.draw_subset_rectangles(points, 2, 2) + elapsed = time.perf_counter() - start + assert elapsed < 0.5, f'redraw of 5000 subsets took {elapsed:.3f} s' + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Results +# --------------------------------------------------------------------------- + +def test_points_are_row_col_integers_inside_the_frame(): + gui = gui_with_region() + try: + points = gui.points + assert points.dtype.kind == 'i' + assert points.shape[1] == 2 + assert (points[:, 0] < gui.frame.shape[0]).all() + assert (points[:, 1] < gui.frame.shape[1]).all() + np.testing.assert_array_equal(points, gui.get_points()) + finally: + gui.close() + + +def test_points_are_accepted_by_a_method_class(tmp_path): + import warnings + + from pyidi import SimplifiedOpticalFlow, VideoReader + + gui = gui_with_region() + try: + points = gui.points + assert len(points) + video = VideoReader(np.stack([gui.frame, gui.frame]), root=str(tmp_path)) + method = SimplifiedOpticalFlow(video) + with warnings.catch_warnings(): + warnings.simplefilter('error') + method.set_points(points) + np.testing.assert_array_equal(method.points, points) + finally: + gui.close() + + +def test_hand_picked_points_survive_a_high_threshold(): + gui = gui_with_region() + try: + gui.pipeline.add_entry('points', [(80, 120)]) + gui.threshold_slider.setValue(999) + points = gui.get_points() + assert (points == np.array([80, 120])).all(axis=1).any() + finally: + gui.close() + + +def test_the_count_label_follows_the_points(): + gui = gui_with_region() + try: + assert gui.count_label.text() == f'{len(gui.get_points())} points' + gui.threshold_slider.setValue(995) + gui.flush_refresh() + assert gui.count_label.text() == f'{len(gui.get_points())} points' + finally: + gui.close() + + +def test_the_lattice_selector_gives_regular_spacing(): + gui = gui_with_region() + try: + gui.selector_combo.setCurrentIndex(gui.selector_combo.findData('lattice')) + gui.pitch_spin.setValue(20) + gui.threshold_slider.setValue(0) + points = gui.get_points() + assert len(points) > 4 + assert set(np.diff(sorted(set(points[:, 0].tolist())))) <= {20} + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# The names +# --------------------------------------------------------------------------- + +def test_selection_gui_names_this_interface(): + """``SelectionGUI`` is this class, not the 1.3 window it replaced.""" + import pyidi + + assert pyidi.SelectionGUI is SelectionGUI + + +def test_the_old_interface_is_still_reachable_under_its_own_name(): + import pyidi + from pyidi.GUIs.subset_selection import SelectionGUIOld + + assert pyidi.SelectionGUIOld is SelectionGUIOld + assert pyidi.SelectionGUIOld is not pyidi.SelectionGUI + + +def test_the_old_interface_warns_on_construction(): + from pyidi import SelectionGUIOld + + with pytest.deprecated_call(): + gui = SelectionGUIOld(make_image()) + gui.close() + + +def test_the_working_name_says_where_it_went(): + """``FeatureSelectionGUI`` never shipped, but it is in the design notes.""" + import pyidi + + with pytest.raises(RuntimeError, match='now called SelectionGUI'): + pyidi.FeatureSelectionGUI() + + +# --------------------------------------------------------------------------- +# Where the controls live +# --------------------------------------------------------------------------- + +def test_the_subset_size_shows_on_every_tab(): + """It drives the score and it measures the drawn rectangles, so it is neither tab's.""" + gui = make_gui() + try: + for name in (STEP_FIND, STEP_MASK): + gui.select_step(name) + assert gui.height_spin.isVisible() + assert gui.width_spin.isVisible() + # ... by living outside the pages rather than being duplicated on each. + for page in gui.step_pages.values(): + assert gui.height_spin not in page.findChildren(QtWidgets.QSpinBox) + finally: + gui.close() + + +def test_the_point_spacing_stays_with_the_mask(): + """It only affects rows that lay points out, which is a masking concern.""" + gui = make_gui() + try: + assert gui.spacing_spin in gui.step_pages[STEP_MASK].findChildren(QtWidgets.QSpinBox) + finally: + gui.close() + + +def test_both_tabs_offer_the_score_overlay(): + gui = make_gui() + try: + assert gui.show_score in gui.step_pages[STEP_FIND].findChildren(QtWidgets.QCheckBox) + assert gui.show_score_mask in gui.step_pages[STEP_MASK].findChildren(QtWidgets.QCheckBox) + finally: + gui.close() + + +def test_the_score_toggles_are_ganged(): + """One overlay, two checkboxes: they must not disagree about its state.""" + gui = gui_with_region() + try: + gui.show_score_mask.setChecked(True) + assert gui.show_score.isChecked() + assert gui.score_overlay.isVisible() + + gui.show_score.setChecked(False) + assert not gui.show_score_mask.isChecked() + assert not gui.score_overlay.isVisible() + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Gradient direction +# --------------------------------------------------------------------------- + +def use_gradient(gui): + """Switch the evaluator to the direction-taking one.""" + gui.evaluator_combo.setCurrentIndex(gui.evaluator_combo.findData('gradient_direction')) + return gui + + +def test_the_direction_can_be_dragged_out_on_the_image(): + gui = use_gradient(make_gui()) + try: + gui.direction_button.setChecked(True) + assert gui.drawing_direction + + gui.set_direction_from_drag((10.0, 10.0), (10.0, 40.0)) # straight along +col + + row, col = (spin.value() for spin in gui.direction_spins) + assert row == pytest.approx(0.0, abs=1e-6) + assert col == pytest.approx(1.0, abs=1e-6) + assert not gui.drawing_direction # one drag sets it once + finally: + gui.close() + + +def test_the_dragged_direction_reaches_the_evaluator(): + gui = use_gradient(make_gui()) + try: + gui.set_direction_from_drag((0.0, 0.0), (30.0, 0.0)) # straight along +row + spec = gui.pipeline.store.spec(gui.pipeline.default_score) + assert spec.evaluator == 'gradient_direction' + assert dict(spec.parameters)['direction'] == pytest.approx((1.0, 0.0)) + finally: + gui.close() + + +def test_the_direction_line_shows_where_it_was_dragged(): + gui = use_gradient(make_gui()) + try: + gui.set_direction_from_drag((20.0, 30.0), (60.0, 90.0)) + xs, ys = gui.direction_line.getData() + np.testing.assert_allclose(xs, [30.0, 90.0]) # x is the column + np.testing.assert_allclose(ys, [20.0, 60.0]) + finally: + gui.close() + + +def test_a_preset_costs_one_evaluation_not_two(): + """Writing two components must not evaluate once per component.""" + gui = use_gradient(make_gui()) + try: + before = gui.pipeline.store.n_evaluations + gui.set_direction(1.0, 1.0) + assert gui.pipeline.store.n_evaluations == before + 1 + row, col = (spin.value() for spin in gui.direction_spins) + assert (row, col) == pytest.approx((2 ** -0.5, 2 ** -0.5), abs=1e-3) + finally: + gui.close() + + +def test_a_zero_direction_is_ignored(): + gui = use_gradient(make_gui()) + try: + before = [spin.value() for spin in gui.direction_spins] + gui.set_direction(0.0, 0.0) + assert [spin.value() for spin in gui.direction_spins] == before + finally: + gui.close() + + +def test_leaving_the_gradient_evaluator_drops_its_direction_line(): + gui = use_gradient(make_gui()) + try: + gui.set_direction_from_drag((0.0, 0.0), (0.0, 30.0)) + assert gui.direction_line.getData()[0] is not None + + gui.evaluator_combo.setCurrentIndex(gui.evaluator_combo.findData('shi_tomasi')) + assert gui.direction_line.getData()[0] is None + assert gui.direction_spins == [] + assert not gui.drawing_direction + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Masking is visible in the points +# --------------------------------------------------------------------------- + +def test_drawing_a_region_stands_the_whole_image_row_down(): + """Masks are a union, so a drawn region would otherwise change nothing.""" + gui = make_gui() + try: + seeded = gui.pipeline.entries[0] + assert seeded.label == 'Whole image' and seeded.visible + + gui.pipeline.add_entry('polygon', rect(20, 20, 60, 60)) + gui._retire_whole_image() + gui.refresh() + + assert not seeded.visible + assert gui.pipeline.mask[40, 40] + assert not gui.pipeline.mask[120, 200] # outside the drawn region + finally: + gui.close() + + +def test_standing_the_whole_image_row_down_is_undoable(): + gui = make_gui() + try: + seeded = gui.pipeline.entries[0] + gui.pipeline.add_entry('polygon', rect(20, 20, 60, 60)) + gui._retire_whole_image() + gui.undo() + assert seeded.visible + finally: + gui.close() + + +def test_an_empty_polygon_leaves_the_whole_image_row_alone(): + """Two vertices enclose nothing, so there is no region to make way for yet.""" + gui = make_gui() + try: + gui.pipeline.add_entry('polygon', [(20.0, 20.0), (20.0, 60.0)]) + gui._retire_whole_image() + assert gui.pipeline.entries[0].visible + finally: + gui.close() + + +def test_a_points_row_leaves_the_whole_image_row_alone(): + """A literal point adds to the selection rather than restricting it.""" + gui = make_gui() + try: + gui.pipeline.add_entry('points', [(80, 120)]) + gui._retire_whole_image() + assert gui.pipeline.entries[0].visible + finally: + gui.close() + + +def crossed_out(gui): + """How many points the deselect brush is currently showing as doomed.""" + xs = gui.doomed_scatter.getData()[0] + return 0 if xs is None else len(xs) + + +def test_a_deselect_stroke_crosses_out_the_points_it_covers(): + """Feedback while the stroke is being painted, not only when the mouse comes up.""" + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.select_tool('erase') + + gui.brush_start() + gui.brush_radius.setValue(40) + gui.brush_move((80.0, 120.0)) + + doomed = crossed_out(gui) + assert doomed > 0 + # The crosses go over the red points rather than replacing them: a stroke + # then costs only the points it has reached, not the whole cloud. + assert len(gui.point_scatter.getData()[0]) == len(gui.get_points()) + finally: + gui.close() + + +def test_painting_a_stroke_leaves_the_point_cloud_alone(): + """The cost of a mouse move is the points it reached, not every point drawn. + + Handing tens of thousands of positions back to the scatter item on every + move is what made a long stroke lag behind the cursor. + """ + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.select_tool('erase') + before = gui.point_scatter.getData()[0].copy() + + gui.brush_start() + gui.brush_radius.setValue(40) + gui.brush_move((80.0, 120.0)) + + assert crossed_out(gui) > 0 + assert np.array_equal(gui.point_scatter.getData()[0], before) + finally: + gui.close() + + +def test_the_stroke_is_drawn_without_rebuilding_a_full_frame_image(): + """A raster overlay costs the whole frame per move, however small the dab.""" + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.brush_start() + gui.brush_move((80.0, 120.0)) + assert not gui.brush_overlay.path().isEmpty() + gui.brush_end() + assert gui.brush_overlay.path().isEmpty() + finally: + gui.close() + + +def test_the_crossed_out_points_are_gone_once_the_stroke_lands(): + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.select_tool('erase') + before = len(gui.get_points()) + + gui.brush_start() + gui.brush_radius.setValue(40) + gui.brush_move((80.0, 120.0)) + stroke = gui._paint.copy() + gui.brush_end() + + after = gui.get_points() + assert len(after) < before + # The count is not simply `before - doomed`: freeing the area lets the + # minimum-distance rule admit points it had previously suppressed. + assert not stroke[after[:, 0], after[:, 1]].any() + assert crossed_out(gui) == 0 + finally: + gui.close() + + +def test_a_selecting_stroke_crosses_nothing_out(): + """Painting a new region takes nothing away, so nothing should look doomed.""" + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.brush_start() + gui.brush_radius.setValue(40) + gui.brush_move((80.0, 120.0)) + assert crossed_out(gui) == 0 + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Panel layout +# --------------------------------------------------------------------------- + +def test_no_group_box_contains_another(): + """Two nested frames cost two sets of margins out of an already narrow panel.""" + gui = make_gui() + try: + for page in gui.step_pages.values(): + for box in page.findChildren(QtWidgets.QGroupBox): + assert not box.findChildren(QtWidgets.QGroupBox), box.title() + finally: + gui.close() + + +def test_only_the_current_selectors_settings_are_shown(): + """A greyed-out row is a line of panel spent saying the line does not apply.""" + gui = make_gui() + try: + assert gui.separation_spin.isVisible() + assert not gui.pitch_spin.isVisible() + + gui.selector_combo.setCurrentIndex(gui.selector_combo.findData('lattice')) + assert gui.pitch_spin.isVisible() + assert not gui.separation_spin.isVisible() + finally: + gui.close() + + +def test_the_evaluator_describes_itself_in_a_tooltip(): + """It used to be a paragraph on the panel, which you read once.""" + gui = make_gui() + try: + assert 'corner' in gui.evaluator_combo.toolTip().lower() + gui.evaluator_combo.setCurrentIndex(gui.evaluator_combo.findData('gradient_direction')) + assert 'direction' in gui.evaluator_combo.toolTip().lower() + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Quality threshold and decimation +# --------------------------------------------------------------------------- + +def test_the_threshold_defaults_to_quality(): + """Percentile spends most of its travel inside the featureless background.""" + gui = make_gui() + try: + assert gui.threshold_mode.currentData() == 'quality' + assert gui.pipeline.selector_params['threshold_mode'] == 'quality' + assert gui.pipeline.selector_params['threshold'] == pytest.approx(0.01, rel=0.02) + finally: + gui.close() + + +def test_the_quality_slider_is_logarithmic(): + """The useful settings span three decades, so a linear slider wastes most of itself.""" + gui = make_gui() + try: + seen = [] + for position in (0, 250, 500, 750, 1000): + gui.threshold_slider.setValue(position) + seen.append(gui.pipeline.selector_params['threshold']) + assert seen[0] == pytest.approx(0.001) + assert seen[-1] == pytest.approx(1.0) + ratios = [b / a for a, b in zip(seen, seen[1:])] + assert all(r == pytest.approx(ratios[0], rel=1e-6) for r in ratios) + finally: + gui.close() + + +def test_switching_the_rule_moves_the_slider_to_that_rules_default(): + """The same position means 90 under one rule and 0.5 under another.""" + gui = make_gui() + try: + gui.threshold_mode.setCurrentIndex(gui.threshold_mode.findData('percentile')) + assert gui.pipeline.selector_params['threshold'] == pytest.approx(90.0) + + gui.threshold_mode.setCurrentIndex(gui.threshold_mode.findData('quality')) + assert gui.pipeline.selector_params['threshold'] == pytest.approx(0.01, rel=0.02) + finally: + gui.close() + + +def test_quality_keeps_the_points_off_the_blank_background(): + """The reason the default changed, end to end.""" + frame = np.full((200, 300), 240, dtype=np.uint8) + corners = np.zeros(frame.shape, dtype=bool) + for row in range(40, 180, 60): + for col in range(40, 280, 60): + frame[row:row + 20, col:col + 20] = 20 + corners[row - 9:row + 29, col - 9:col + 29] = True + frame = np.clip(frame.astype(int) + np.random.default_rng(3).integers(-5, 6, frame.shape), + 0, 255).astype(np.uint8) + + gui = SelectionGUI(frame, subset_size=11) + try: + for position in (1000, 750, 500, 333): + gui.threshold_slider.setValue(position) + points = gui.get_points() + assert len(points) + assert corners[points[:, 0], points[:, 1]].all(), gui.threshold_label.text() + finally: + gui.close() + + +def test_decimation_thins_without_moving_the_survivors(): + gui = gui_with_region() + try: + gui.separation_spin.setValue(6) + before = {tuple(point) for point in gui.get_points().tolist()} + assert len(before) > 20 + + gui.decimation_spin.setValue(3) + after = [tuple(point) for point in gui.get_points().tolist()] + assert set(after) <= before + assert len(after) == pytest.approx(len(before) / 3, rel=0.2) + finally: + gui.close() + + +def test_decimation_does_not_re_evaluate(): + gui = gui_with_region() + try: + before = gui.pipeline.store.n_evaluations + gui.decimation_spin.setValue(4) + assert gui.pipeline.store.n_evaluations == before + finally: + gui.close() + + +def test_a_redraw_runs_the_pipeline_once(): + """It used to run three times: the total, the row counts and the highlight.""" + gui = gui_with_region() + try: + calls = [] + original = gui.pipeline.picked_points + gui.pipeline.picked_points = lambda *a, **k: (calls.append(1), original(*a, **k))[1] + gui.select_step(STEP_MASK) # the highlight is drawn on this tab + calls.clear() + gui.refresh() + assert len(calls) == 1 + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Coalescing the redraws a dragged control produces +# --------------------------------------------------------------------------- + +def test_a_cheap_redraw_happens_immediately(): + """A live slider is the whole point; deferring a redraw we can afford buys nothing.""" + gui = gui_with_region() + try: + gui._last_refresh_ms = 0.0 + gui.threshold_slider.setValue(700) + assert not gui._refresh_timer.isActive() + assert len(gui._points) == len(gui.get_points()) + finally: + gui.close() + + +def test_an_expensive_redraw_is_deferred_and_coalesced(): + """Twenty positions on the way past cost one redraw, not twenty.""" + gui = gui_with_region() + try: + gui._last_refresh_ms = REDRAW_BUDGET_MS + 1.0 + stale = gui._points + for position in range(700, 720): + gui.threshold_slider.setValue(position) + assert gui._refresh_timer.isActive() + np.testing.assert_array_equal(gui._points, stale) # nothing redrawn yet + + gui.flush_refresh() + assert not gui._refresh_timer.isActive() + np.testing.assert_array_equal(gui._points, gui.get_points()) + finally: + gui.close() + + +def test_the_deferred_redraw_lands_on_the_value_the_control_stopped_at(): + gui = gui_with_region() + try: + gui._last_refresh_ms = REDRAW_BUDGET_MS + 1.0 + gui.threshold_slider.setValue(400) + gui.threshold_slider.setValue(900) + gui.flush_refresh() + assert len(gui._points) == len(gui.get_points()) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# One density control, and it is not a stride +# --------------------------------------------------------------------------- + +def test_the_separation_is_the_density_control(): + gui = gui_with_region() + try: + gui.separation_spin.setValue(6) + loose = gui.get_points() + gui.separation_spin.setValue(18) + tight = gui.get_points() + assert len(tight) < len(loose) + for points, separation in ((loose, 6), (tight, 18)): + gaps = np.hypot(points[:, None, 0] - points[None, :, 0], + points[:, None, 1] - points[None, :, 1]) + np.fill_diagonal(gaps, np.inf) + assert gaps.min() >= separation + finally: + gui.close() + + +def test_the_separation_cannot_be_turned_off(): + """Zero was the setting that made the point cap look like a tight cluster.""" + gui = make_gui() + try: + assert gui.separation_spin.minimum() == 1 + finally: + gui.close() + + +def test_the_threshold_menu_offers_quality_and_percentile_only(): + gui = make_gui() + try: + rules = [gui.threshold_mode.itemData(i) for i in range(gui.threshold_mode.count())] + assert rules == ['quality', 'percentile'] + finally: + gui.close() + + +def test_the_cap_says_so_rather_than_looking_like_a_threshold(): + gui = gui_with_region() + try: + gui.max_points_spin.setValue(20) + gui.separation_spin.setValue(2) + gui.flush_refresh() + assert '20' in gui.select_note.text() + assert 'cap' in gui.select_note.text().lower() + + gui.separation_spin.setValue(40) + gui.flush_refresh() + assert gui.select_note.text() == '' + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# What the mask is leaving out +# --------------------------------------------------------------------------- + +def rendered(gui): + """The canvas as three ``(h, w)`` integer channels: red, green, blue. + + Copied out of the QImage, which owns the buffer and frees it on the way out, + and signed, so that one channel can be subtracted from another. + """ + image = gui.pg_widget.grab().toImage().convertToFormat( + QtGui.QImage.Format.Format_RGB32) + raw = np.frombuffer(image.constBits().asarray(image.sizeInBytes()), dtype=np.uint8) + pixels = raw.reshape(image.height(), image.width(), 4).astype(np.int16) + return pixels[..., 2], pixels[..., 1], pixels[..., 0] + + +def test_the_two_point_layers_actually_paint(): + """Rendered, not just handed the right coordinates. + + The dots are one stroked path rather than a scatter item, and Qt draws + nothing at all for a zero-length subpath -- a failure no assertion about what + the item was *given* can see. + """ + gui = empty_gui() + try: + gui.pipeline.add_entry('polygon', rect(20, 20, 60, 60)) + gui.active_index = 0 + gui.select_step(STEP_MASK) + gui.show_subsets.setChecked(False) + gui.refresh() + + red, green, blue = rendered(gui) + # The red points, told apart from the magenta rings by their blue. + assert ((red > 150) & (red - blue > 60) & (red - green > 40)).sum() > 20 + # The dim blue candidates outside the polygon. + assert ((blue > 120) & (blue - red > 60)).sum() > 20 + finally: + gui.close() + + +def greyed(gui): + """The dimmed candidate positions currently drawn, as ``(row, col)``.""" + x, y = gui.candidate_scatter.getData() + if x is None or not len(x): + return np.zeros((0, 2), dtype=int) + return np.column_stack([y - 0.5, x - 0.5]).astype(int) + + +def test_the_mask_tab_shows_the_features_the_mask_leaves_out(): + """"No point here" is otherwise two different things wearing one face.""" + gui = empty_gui() + try: + gui.pipeline.add_entry('polygon', rect(20, 20, 60, 60)) + gui.active_index = 0 + gui.select_step(STEP_MASK) + outside = greyed(gui) + assert len(outside) + mask = gui.pipeline.mask + assert not mask[outside[:, 0], outside[:, 1]].any() + finally: + gui.close() + + +def test_masking_an_area_takes_its_points_out_of_the_dimmed_set(): + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + before = len(greyed(gui)) + gui.pipeline.add_entry('polygon', rect(20, 20, 140, 220)) + gui.refresh() + assert len(greyed(gui)) < before + finally: + gui.close() + + +def test_the_dimmed_candidates_are_a_mask_tab_cue(): + """On the other tab everything drawn is selected, so a second tier would only confuse.""" + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + assert len(greyed(gui)) + gui.select_step(STEP_FIND) + assert not len(greyed(gui)) + finally: + gui.close() + + +def test_editing_a_mask_does_not_move_the_candidates(): + """They are the whole-frame selection, so painting turns points red where it lands.""" + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + before = gui.pipeline.candidate_points() + gui.pipeline.add_entry('polygon', rect(20, 20, 140, 220)) + gui.refresh() + assert gui.pipeline.candidate_points() is before + finally: + gui.close() + + +def test_a_new_threshold_does_move_the_candidates(): + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + before = len(gui.pipeline.candidate_points()) + gui.threshold_slider.setValue(gui.threshold_slider.maximum()) + gui.flush_refresh() + assert len(gui.pipeline.candidate_points()) < before + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Tab names and where the panel's boxes live +# --------------------------------------------------------------------------- + +def test_the_tabs_are_named_for_the_pipeline_steps(): + """Not "find points": the steps have names, and the tabs hold exactly those.""" + assert STEP_MASK.lower() == 'mask' + assert 'evaluate' in STEP_FIND.lower() and 'select' in STEP_FIND.lower() + # An '&' would be read as a mnemonic and swallowed out of the button label. + assert '&' not in STEP_FIND + + +def test_the_selections_list_is_a_mask_tab_control(): + """Every row in it, and every button under it, acts on something drawn there.""" + gui = make_gui() + try: + gui.select_step(STEP_MASK) + assert gui.selection_box.isVisible() + gui.select_step(STEP_FIND) + assert not gui.selection_box.isVisible() + finally: + gui.close() + + +def test_the_subset_group_stays_on_both_tabs(): + """The counterpart to the list moving: this one really is read by both steps.""" + gui = make_gui() + try: + for name in (STEP_FIND, STEP_MASK): + gui.select_step(name) + assert gui.height_spin.isVisible() + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Tooltips +# --------------------------------------------------------------------------- + +def test_every_setting_explains_itself(): + gui = make_gui() + try: + for widget in (gui.selector_combo, gui.threshold_mode, gui.threshold_slider, + gui.separation_spin, gui.pitch_spin, gui.max_points_spin, + gui.decimation_spin, gui.spacing_spin, gui.height_spin, + gui.width_spin, gui.show_subsets, gui.show_score, + gui.role_button, gui.brush_radius): + assert widget.toolTip(), widget + finally: + gui.close() + + +def test_no_tooltip_or_hint_uses_a_name_the_interface_dropped(): + """Stale help is worse than none: it names a control that is not there.""" + gui = make_gui() + try: + texts = [w.toolTip() for w in gui.findChildren(QtWidgets.QWidget)] + texts += list(STEP_HINTS.values()) + for text in texts: + lowered = text.lower() + assert 'minimum distance' not in lowered + assert 'fraction of the maximum' not in lowered + assert 'find points' not in lowered + # 'Deselect painted area' was the checkable button that 'Remove + # area' replaced; nothing should still send anyone looking for it. + assert 'deselect painted area' not in lowered + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Odd subset sizes +# --------------------------------------------------------------------------- + +def test_the_subset_size_steps_in_odds(): + """A subset is centred on its point, so an even extent has no centre to be.""" + gui = make_gui(subset_size=11) + try: + gui.height_spin.stepUp() + assert gui.height_spin.value() == 13 + gui.height_spin.stepDown() + gui.height_spin.stepDown() + assert gui.height_spin.value() == 9 + finally: + gui.close() + + +def test_an_even_subset_size_cannot_be_typed_in(): + gui = make_gui() + try: + gui.height_spin.lineEdit().setText('12') + gui.height_spin.interpretText() + assert gui.height_spin.value() == 13 + finally: + gui.close() + + +def test_an_even_subset_size_is_rounded_up_on_the_way_in(): + """The pipeline and the control that shows it must not disagree.""" + gui = make_gui(subset_size=(10, 20)) + try: + assert gui.pipeline.subset_size == (11, 21) + assert (gui.height_spin.value(), gui.width_spin.value()) == (11, 21) + finally: + gui.close() + + +def test_the_square_toggle_keeps_both_odd(): + gui = make_gui(subset_size=(11, 21)) + try: + gui.square_check.setChecked(True) + assert gui.width_spin.value() == gui.height_spin.value() + assert gui.pipeline.subset_size == (11, 11) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Removing a point by clicking it +# --------------------------------------------------------------------------- + +def test_the_remove_tool_takes_a_point_away_every_time(): + """The regression: only the first click did anything. + + ``erased`` was grown with an in-place write, and the pipeline identifies + that array by object -- so once it existed, no later click changed anything + the rasterisation cache could see. + """ + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('remove') + for _ in range(4): + target = tuple(int(v) for v in gui._points[0]) + gui.remove_nearest_point(target) + gui.refresh() + assert not any(tuple(p) == target for p in gui._points) + finally: + gui.close() + + +def test_removing_a_point_is_undoable(): + gui = gui_with_region() + try: + gui.select_step(STEP_MASK) + gui.select_tool('remove') + before = len(gui._points) + target = tuple(int(v) for v in gui._points[0]) + + gui.remove_nearest_point(target) + gui.refresh() + assert not any(tuple(p) == target for p in gui._points) + + gui.undo() + assert len(gui._points) == before + assert any(tuple(p) == target for p in gui._points) + finally: + gui.close() + + +def test_a_click_far_from_every_point_removes_nothing(): + gui = gui_with_region() + try: + before = gui.pipeline.entries[0].erased + gui.remove_nearest_point((0, 0)) + assert gui.pipeline.entries[0].erased is before + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Clicks that land off the image +# +# The aspect is locked, so one axis always has a margin, and zooming out adds +# more. A subset centred off the frame is not something that can be tracked. +# --------------------------------------------------------------------------- + +def test_a_click_outside_the_image_adds_no_point(): + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + gui.select_tool('points') + rows, cols = gui.pipeline.shape + gui.add_point((rows + 40.0, 20.0)) + gui.add_point((20.0, -12.0)) + assert gui.pipeline.entries == [] + finally: + gui.close() + + +def test_a_click_on_the_image_still_adds_one(): + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + gui.select_tool('points') + gui.add_point((80.0, 120.0)) + assert gui.pipeline.entries[0].geometry == [(80, 120)] + finally: + gui.close() + + +def test_an_off_frame_point_cannot_reach_the_result(): + """Whatever route it arrived by -- it is an index error in every array.""" + gui = empty_gui() + try: + gui.pipeline.add_entry('points', [(80, 120), (999, 120)]) + gui.refresh() + assert [tuple(p) for p in gui.get_points()] == [(80, 120)] + + gui.select_tool('erase') + gui.brush_start() + gui.brush_move((80.0, 120.0)) # would index row 999 of a 160-row frame + gui.brush_end() + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# What an evaluator change costs +# +# This is the one control that can make a redraw expensive: a parameter the +# store has not scored before is a whole-frame evaluation. +# --------------------------------------------------------------------------- + +def select_evaluator(gui, name): + """Put the evaluator combo on a registry name.""" + for index in range(gui.evaluator_combo.count()): + if gui.evaluator_combo.itemData(index) == name: + gui.evaluator_combo.setCurrentIndex(index) + return + raise AssertionError(f'no {name!r} in the evaluator menu') + + +def test_a_parameter_change_is_coalesced_like_every_other_control(): + gui = gui_with_region() + try: + select_evaluator(gui, 'gradient_direction') + gui.refresh() + gui._last_refresh_ms = REDRAW_BUDGET_MS + 1 + gui.direction_spins[0].setValue(0.37) + assert gui._refresh_timer.isActive() + finally: + gui.close() + + +def test_dragging_a_parameter_does_not_hoard_a_score_per_value(): + """Each one is a full-frame float32; sixty of them is a gigabyte at 4 MP.""" + gui = gui_with_region() + try: + select_evaluator(gui, 'gradient_direction') + for step in range(40): + gui.direction_spins[0].setValue(0.02 * step) + gui.flush_refresh() + assert len(gui.pipeline.store._cache) <= gui.pipeline.store.max_cached + finally: + gui.close() + + +def test_the_score_overlay_follows_a_change_of_subset_size(): + """It is drawn from the score, so it has to be part of the redraw.""" + gui = gui_with_region() + try: + gui.show_score.setChecked(True) + before = gui.score_overlay.image.copy() + gui.height_spin.setValue(31) + gui.flush_refresh() + assert not np.array_equal(gui.score_overlay.image, before) + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Ctrl, and what a stroke costs to remember +# --------------------------------------------------------------------------- + +class FakeDrag: + """The parts of a pyqtgraph drag event the canvas actually reads.""" + + def __init__(self, ctrl=False, start=False, finish=False): + self._ctrl = ctrl + self._start = start + self._finish = finish + self.accepted = False + + def modifiers(self): + return (QtCore.Qt.KeyboardModifier.ControlModifier if self._ctrl + else QtCore.Qt.KeyboardModifier.NoModifier) + + def isStart(self): + return self._start + + def isFinish(self): + return self._finish + + def accept(self): + self.accepted = True + + def scenePos(self): + return QtCore.QPointF(0.0, 0.0) + + def buttonDownScenePos(self): + return QtCore.QPointF(0.0, 0.0) + + +def test_ctrl_is_read_off_the_event_not_tracked(): + """A panel widget with focus can swallow the key; a tracked flag then lies.""" + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + assert not gui.view._handle_brush_drag(FakeDrag(ctrl=False, start=True)) + assert gui.view._handle_brush_drag(FakeDrag(ctrl=True, start=True)) + assert gui.painting + finally: + gui.close() + + +def test_letting_go_of_ctrl_mid_stroke_still_finishes_the_stroke(): + """Otherwise the drag stops being handled and the stroke is silently lost.""" + gui = empty_gui() + try: + gui.select_step(STEP_MASK) + gui.select_tool('brush') + gui.view._handle_brush_drag(FakeDrag(ctrl=True, start=True)) + gui.brush_move((80.0, 120.0)) + + assert gui.view._handle_brush_drag(FakeDrag(ctrl=False)) + assert gui.view._handle_brush_drag(FakeDrag(ctrl=False, finish=True)) + assert not gui.painting + assert gui.pipeline.mask.sum() > 0 + finally: + gui.close() + + +def test_an_undo_snapshot_does_not_copy_the_erased_arrays(): + """One frame of booleans per region, in each of fifty undo slots. + + Safe to hold by reference because ``erased`` is always replaced, never + written into -- the contract on :class:`~pyidi.selection.masks.Entry`. + """ + gui = gui_with_region() + try: + gui.select_tool('brush') + gui.select_tool('erase') + gui.brush_start() + gui.brush_move((80.0, 120.0)) + gui.brush_end() + + entry = gui.pipeline.entries[0] + assert entry.erased is not None + held = {id(state[1]) for state in gui._snapshot()['state']} + assert id(entry.erased) in held + finally: + gui.close() + + +def test_the_seeded_row_is_recognised_by_identity_not_by_its_label(): + gui = make_gui() + try: + seeded = gui._whole_image + seeded.label = 'Renamed by hand' + gui.pipeline.add_entry('polygon', rect(20, 20, 140, 220)) + gui._retire_whole_image() + assert not seeded.visible + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# The mask tab shows only what the current tool and rows can act on +# --------------------------------------------------------------------------- + +def test_the_brush_radius_follows_the_tools_that_paint(): + """Painting is gated on the tool, so with any other one the radius does nothing.""" + gui = make_gui() + try: + gui.select_step(STEP_MASK) + + gui.select_tool('polygon') + assert not gui.brush_radius.isVisible() + + gui.select_tool('brush') + assert gui.brush_radius.isVisible() + + # Both brush tools share it: a stroke erases as wide as it paints. + gui.select_tool('erase') + assert gui.brush_radius.isVisible() + + gui.select_tool('points') + assert not gui.brush_radius.isVisible() + finally: + gui.close() + + +def test_erasing_is_a_tool_rather_than_a_mode_the_brush_is_in(): + """The two things that take away are tools; neither hides inside the other.""" + gui = make_gui() + try: + gui.select_step(STEP_MASK) + assert 'erase' in gui.tool_buttons + assert 'remove' in gui.tool_buttons + + gui.select_tool('brush') + assert not gui.deselect_mode + + gui.select_tool('erase') + assert gui.deselect_mode + + # Leaving the tool leaves the mode, because they are the same thing. + gui.select_tool('brush') + assert not gui.deselect_mode + finally: + gui.close() + + +def test_point_spacing_appears_only_for_a_row_that_lays_points_out(): + """It reaches ``literal_points`` and nowhere else.""" + gui = make_gui() + try: + gui.select_step(STEP_MASK) + # Only the seeded whole-image mask so far: nothing spacing can move. + assert not gui.spacing_spin.isVisible() + + entry = gui.pipeline.add_entry('polygon', rect(20, 20, 140, 220)) + gui.refresh() + assert not gui.spacing_spin.isVisible() + + entry.role = 'points' + gui.refresh() + assert gui.spacing_spin.isVisible() + + entry.role = 'mask' + gui.refresh() + assert not gui.spacing_spin.isVisible() + finally: + gui.close() + + +def test_point_spacing_stays_hidden_for_hand_clicked_points(): + """A ``points``-tool row is the coordinates you clicked; spacing has no say.""" + gui = make_gui() + try: + gui.select_step(STEP_MASK) + gui.pipeline.add_entry('points', [(40, 40), (60, 60)]) + gui.refresh() + assert not gui.spacing_spin.isVisible() + + gui.pipeline.add_entry('polyline', [(30, 30), (30, 200)]) + gui.refresh() + assert gui.spacing_spin.isVisible() + finally: + gui.close() diff --git a/tests/test_gui_availability.py b/tests/test_gui_availability.py new file mode 100644 index 0000000..c5f082f --- /dev/null +++ b/tests/test_gui_availability.py @@ -0,0 +1,57 @@ +"""Each GUI class checks its own dependencies, so a partial install still imports. + +``pyidi.GUIs`` used to gate every class on PyQt6 alone, then import the napari +``GUI`` unconditionally. With PyQt6 present and napari absent -- ``pip install +pyqt6 pyqtgraph`` without the extra -- ``import pyidi`` therefore died with a +``ModuleNotFoundError`` from inside a submodule, taking the whole package with +it rather than just the one class that was unusable. +""" + +import os +import sys + +import pytest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + '/../') + +import pyidi # noqa: E402 +from pyidi import GUIs # noqa: E402 + +EXPORTED = ('SelectionGUI', 'SelectionGUIOld', 'ResultViewer', 'Viewer', 'GUI') + + +@pytest.mark.parametrize('name', EXPORTED) +def test_every_gui_name_is_bound(name): + """Present as the real class or as a stub, but never missing.""" + assert getattr(pyidi, name, None) is not None + + +@pytest.mark.parametrize('name', EXPORTED) +def test_every_gui_name_declares_what_it_needs(name): + """A stub cannot say what is missing unless the requirement is recorded.""" + assert name in GUIs._REQUIREMENTS + + +def test_the_selection_windows_do_not_depend_on_napari(): + """The reason the package survives a PyQt6-without-napari install.""" + assert 'napari' not in GUIs._REQUIREMENTS['SelectionGUI'] + assert 'napari' not in GUIs._REQUIREMENTS['SelectionGUIOld'] + assert 'napari' in GUIs._REQUIREMENTS['GUI'] + + +def test_a_stub_names_the_extra_and_the_missing_package(): + """Constructing an unavailable class says what to install, and what is absent.""" + stub = GUIs._unavailable('GUI') + assert stub.__name__ == 'GUI' + + with pytest.raises(RuntimeError) as excinfo: + stub() + message = str(excinfo.value) + assert 'pip install pyidi[qt]' in message + assert 'GUI requires' in message + + +def test_a_stub_is_importable_rather_than_raising_on_definition(): + """The whole point: the failure waits for construction.""" + stub = GUIs._unavailable('ResultViewer') + assert isinstance(stub, type) diff --git a/tests/test_selection_geometry.py b/tests/test_selection_geometry.py new file mode 100644 index 0000000..b7624f8 --- /dev/null +++ b/tests/test_selection_geometry.py @@ -0,0 +1,412 @@ +""" +Tests for ``pyidi/selection_geometry.py``, the pure-numpy geometry helpers +behind the point/ROI selection GUIs. + +The module deliberately keeps two *different* coordinate conventions: +``get_roi_grid`` works in (row, col) / (y, x) order, while +``rois_inside_polygon`` and ``points_along_polygon`` work in (x, y) order, +and ``rois_inside_mask`` returns (y, x) again. These tests pin that split +down explicitly (see ``test_convention_split_is_intentional``), as well as +the anisotropic ``roi_size`` support that is the reason ``get_roi_grid`` +was kept as a separate function instead of being folded into the others. +""" + +import numpy as np +import pytest + +from pyidi.selection_geometry import ( + _as_size_pair, + get_roi_grid, + points_along_polygon, + rois_inside_polygon, + rois_inside_mask, +) + + +# --------------------------------------------------------------------------- +# _as_size_pair -- scalar/pair normalization shared by the (x, y) helpers +# --------------------------------------------------------------------------- + +def test_as_size_pair_broadcasts_scalar(): + assert _as_size_pair(10) == (10.0, 10.0) + assert _as_size_pair(7.5) == (7.5, 7.5) + + +def test_as_size_pair_passes_through_height_width_pair(): + assert _as_size_pair((5, 21)) == (5.0, 21.0) + assert _as_size_pair([5, 21]) == (5.0, 21.0) + + +def test_as_size_pair_raises_for_wrong_length(): + with pytest.raises(ValueError, match=r"subset_size"): + _as_size_pair((1, 2, 3)) + with pytest.raises(ValueError, match=r"subset_size"): + _as_size_pair([1]) + + +def test_as_size_pair_error_message_reports_shape_not_length(): + """A 2-D input like a (2, 2) array must report its actual shape, not "length 2". + + ``len(arr)`` on a (2, 2) array is 2, which is both wrong (the problem is + the shape, not the length) and confusing (2 looks like a valid pair + length). The message must show the real shape instead. + """ + with pytest.raises(ValueError, match=r"got shape \(2, 2\)"): + _as_size_pair(np.array([[1, 2], [3, 4]])) + + +def test_as_size_pair_preserves_integrality(): + """Integer input -> int output; any float involved -> float output. + + This is the property that keeps ``rois_inside_polygon`` / + ``rois_inside_mask`` returning integer coordinates for ordinary + (integer) GUI input, so ``IDIMethod.set_points()`` does not mistake + them for sub-pixel points and warn. + """ + h, w = _as_size_pair(10) + assert isinstance(h, int) and isinstance(w, int) + + h, w = _as_size_pair((5, 21)) + assert isinstance(h, int) and isinstance(w, int) + + h, w = _as_size_pair(np.int64(7)) + assert isinstance(h, int) and isinstance(w, int) + + h, w = _as_size_pair(10.5) + assert isinstance(h, float) and isinstance(w, float) + + h, w = _as_size_pair((5, 21.0)) + assert isinstance(h, float) and isinstance(w, float) + + +# --------------------------------------------------------------------------- +# get_roi_grid -- (row, col) convention +# --------------------------------------------------------------------------- + +def test_get_roi_grid_rectangle_known_count_and_containment(): + """A known rectangle + roi_size/noverlap produces an exact, known grid. + + The polygon is an axis-aligned rectangle spanning row 0..40, col 0..60. + ``matplotlib.path.Path.contains_points`` excludes points that fall + exactly on the polygon's own boundary, so the row/col equal to the + rectangle's minimum edge (0) drop out of the candidate grid, and the + grid step never reaches the maximum edge because ``np.arange`` excludes + its stop value. Both facts were verified empirically against the + implementation and are pinned here. + """ + poly = np.array([[0, 0], [0, 60], [40, 60], [40, 0]]) # (row, col) + pts = get_roi_grid(poly, roi_size=(10, 10), noverlap=0, deselect_polygon=[[], []]) + + expected_rows = [10, 20, 30] + expected_cols = [10, 20, 30, 40, 50] + assert set(pts[:, 0].tolist()) == set(expected_rows) + assert set(pts[:, 1].tolist()) == set(expected_cols) + assert len(pts) == len(expected_rows) * len(expected_cols) == 15 + + # Independent containment check (not reusing matplotlib.path): every + # returned point must be strictly inside the axis-aligned rectangle. + assert np.all((pts[:, 0] > 0) & (pts[:, 0] < 40)) + assert np.all((pts[:, 1] > 0) & (pts[:, 1] < 60)) + + +def test_get_roi_grid_anisotropic_roi_size_spaces_axes_differently(): + """An anisotropic roi_size=(7, 12) must space rows and cols differently. + + This is the capability that makes get_roi_grid worth keeping separate + from the (x, y) helpers below, which only take a scalar subset_size. + """ + poly = np.array([[0, 0], [0, 60], [40, 60], [40, 0]]) + pts = get_roi_grid(poly, roi_size=(7, 12), noverlap=0, deselect_polygon=[[], []]) + + rows = np.array(sorted(set(pts[:, 0].tolist()))) + cols = np.array(sorted(set(pts[:, 1].tolist()))) + + assert len(rows) > 1 and len(cols) > 1 + row_spacing = np.diff(rows) + col_spacing = np.diff(cols) + assert np.all(row_spacing == 7), row_spacing + assert np.all(col_spacing == 12), col_spacing + assert row_spacing[0] != col_spacing[0] + + +def test_get_roi_grid_deselect_polygon_removes_exactly_its_points(): + """deselect_polygon removes only the points inside it, nothing else.""" + poly = np.array([[0, 0], [0, 60], [40, 60], [40, 0]]) + base = get_roi_grid(poly, roi_size=(10, 10), noverlap=0, deselect_polygon=[[], []]) + + # small square (rows, cols) enclosing exactly the single point (10, 10) + deselect = [[5, 5, 15, 15], [5, 15, 15, 5]] + out = get_roi_grid(poly, roi_size=(10, 10), noverlap=0, deselect_polygon=deselect) + + base_set = set(map(tuple, base.tolist())) + out_set = set(map(tuple, out.tolist())) + + assert base_set - out_set == {(10, 10)} + # everything else must survive completely untouched + assert out_set == base_set - {(10, 10)} + + +def test_get_roi_grid_raises_for_wrong_length_roi_size(): + poly = np.array([[0, 0], [0, 1], [1, 1], [1, 0]]) + with pytest.raises(Exception, match=r"roi_size"): + get_roi_grid(poly, roi_size=(10,), noverlap=0, deselect_polygon=[[], []]) + with pytest.raises(Exception, match=r"roi_size"): + get_roi_grid(poly, roi_size=(10, 10, 10), noverlap=0, deselect_polygon=[[], []]) + + +def test_get_roi_grid_accepts_transposed_2xN_input(): + """A (2, N) polygon array is transposed, per the documented contract.""" + poly_nx2 = np.array([[0, 0], [0, 60], [40, 60], [40, 0]]) + poly_2xn = poly_nx2.T + a = get_roi_grid(poly_nx2, roi_size=(10, 10), noverlap=0, deselect_polygon=[[], []]) + b = get_roi_grid(poly_2xn, roi_size=(10, 10), noverlap=0, deselect_polygon=[[], []]) + assert np.array_equal(a, b) + + +# --------------------------------------------------------------------------- +# rois_inside_polygon -- (x, y) convention +# --------------------------------------------------------------------------- + +def test_rois_inside_polygon_rectangle_containment(): + """Every returned point must actually be inside the polygon, in (x, y).""" + poly = [(0, 0), (0, 40), (60, 40), (60, 0)] # (x, y), x in 0..60, y in 0..40 + pts = np.array(rois_inside_polygon(poly, subset_size=10, spacing=0)) + + assert len(pts) > 0 + # independent geometric check, not reusing matplotlib.path + assert np.all((pts[:, 0] >= 0) & (pts[:, 0] <= 60)) + assert np.all((pts[:, 1] >= 0) & (pts[:, 1] <= 40)) + + +def test_rois_inside_polygon_returns_empty_for_fewer_than_3_vertices(): + assert rois_inside_polygon([], 5, 0) == [] + assert rois_inside_polygon([(0, 0)], 5, 0) == [] + assert rois_inside_polygon([(0, 0), (1, 1)], 5, 0) == [] + + +def test_rois_inside_polygon_anisotropic_subset_size_spaces_axes_differently(): + """subset_size=(h, w) must space x by w and y by h, not by a shared step.""" + poly = [(0, 0), (0, 100), (100, 100), (100, 0)] # (x, y), x in 0..100, y in 0..100 + pts = np.array(rois_inside_polygon(poly, subset_size=(5, 21), spacing=0)) + + xs = np.array(sorted(set(pts[:, 0].tolist()))) + ys = np.array(sorted(set(pts[:, 1].tolist()))) + + assert len(xs) > 1 and len(ys) > 1 + assert np.all(np.diff(xs) == 21) # w + spacing + assert np.all(np.diff(ys) == 5) # h + spacing + + +def test_rois_inside_polygon_scalar_matches_equal_pair(): + poly = [(0, 0), (0, 40), (60, 40), (60, 0)] + scalar = rois_inside_polygon(poly, subset_size=10, spacing=0) + pair = rois_inside_polygon(poly, subset_size=(10, 10), spacing=0) + assert scalar == pair + + +def test_rois_inside_polygon_integer_input_returns_integer_coordinates(): + """Integer subset_size/spacing must not produce float coordinates. + + Regression guard: a naive scalar-or-pair normalizer that always casts + to float would make every ordinary (integer) GUI selection come back + as float points, which downstream trips ``IDIMethod.set_points()``'s + sub-pixel-rounding warning on input that was never sub-pixel. + """ + poly = [(0, 0), (0, 40), (60, 40), (60, 0)] + + scalar_pts = rois_inside_polygon(poly, subset_size=10, spacing=0) + assert len(scalar_pts) > 0 + for x, y in scalar_pts: + assert isinstance(x, (int, np.integer)) + assert isinstance(y, (int, np.integer)) + + pair_pts = rois_inside_polygon(poly, subset_size=(10, 15), spacing=0) + assert len(pair_pts) > 0 + for x, y in pair_pts: + assert isinstance(x, (int, np.integer)) + assert isinstance(y, (int, np.integer)) + + +# --------------------------------------------------------------------------- +# points_along_polygon -- (x, y) convention +# --------------------------------------------------------------------------- + +def test_points_along_polygon_spacing_on_a_straight_segment(): + """A known straight segment produces exactly the expected step points.""" + poly = [(0, 0), (30, 0)] + pts = points_along_polygon(poly, subset_size=10, spacing=0) + assert pts == [(0, 0), (10, 0), (20, 0), (30, 0)] + + +def test_points_along_polygon_returns_empty_for_fewer_than_2_vertices(): + assert points_along_polygon([], 5, 0) == [] + assert points_along_polygon([(0, 0)], 5, 0) == [] + + +def test_points_along_polygon_skips_zero_length_segments(): + """A degenerate (repeated-vertex) segment must not blow up or duplicate.""" + poly = [(0, 0), (0, 0), (20, 0)] + pts = points_along_polygon(poly, subset_size=10, spacing=0) + # only the second (non-degenerate) segment contributes points + assert pts == [(0, 0), (10, 0), (20, 0)] + + +def test_points_along_polygon_horizontal_segment_steps_by_width(): + """A purely horizontal segment must step by w (+ spacing), not h. + + The rounded (x, y) output has a small alternating +-1 rounding + artefact around the true step (a pre-existing property of the -0.5 + pixel-centre shift, not something introduced here), so the step is + checked through the point count -- ``int(length // step) + 1`` -- + which cleanly distinguishes a step of 21 (w) from one of 5 (h). + """ + poly = [(0, 0), (100, 0)] + pts = points_along_polygon(poly, subset_size=(5, 21), spacing=0) + assert len(pts) == int(100 // 21) + 1 == 5 + + +def test_points_along_polygon_vertical_segment_steps_by_height(): + """A purely vertical segment must step by h (+ spacing), not w.""" + poly = [(0, 0), (0, 100)] + pts = points_along_polygon(poly, subset_size=(5, 21), spacing=0) + assert len(pts) == int(100 // 5) + 1 == 21 + + +def test_points_along_polygon_45_degree_regression_square_subset_step_unchanged(): + """A square subset must still yield step == subset_size at 45 degrees. + + This pins the reason the extent formula in the implementation is + sqrt((dx*w)**2 + (dy*h)**2) and not |dx|*w + |dy|*h: for h == w == s, + the former reduces to exactly s for every segment angle (the old, + isotropic, pre-anisotropic-support behaviour), while the latter would + give s*sqrt(2) here (a step of ~14 instead of 10, i.e. roughly half as + many points along the diagonal). The expected points below were also + verified against the pre-change implementation for this exact input. + """ + poly = [(0, 0), (50, 50)] + pts = points_along_polygon(poly, subset_size=10, spacing=0) + + expected = [(0, 0), (7, 7), (14, 14), (21, 21), (28, 28), (35, 35), (42, 42), (49, 49)] + assert pts == expected + + +def test_points_along_polygon_scalar_matches_equal_pair(): + poly = [(0, 0), (30, 17), (5, 40)] + scalar = points_along_polygon(poly, subset_size=10, spacing=2) + pair = points_along_polygon(poly, subset_size=(10, 10), spacing=2) + assert scalar == pair + + +def test_points_along_polygon_integer_input_returns_integer_coordinates(): + """Integer subset_size/spacing (scalar and pair) -> integer points.""" + poly = [(0, 0), (30, 17), (5, 40)] + + for subset_size in (10, (10, 15)): + pts = points_along_polygon(poly, subset_size=subset_size, spacing=2) + assert len(pts) > 0 + for x, y in pts: + assert isinstance(x, (int, np.integer)) + assert isinstance(y, (int, np.integer)) + + +# --------------------------------------------------------------------------- +# rois_inside_mask -- mask[y, x] in, (y, x) out +# --------------------------------------------------------------------------- + +def test_rois_inside_mask_true_region_returns_yx_points(): + mask = np.zeros((20, 20), dtype=bool) + mask[5:15, 5:15] = True # rows (y) 5..14, cols (x) 5..14 + + pts = set(rois_inside_mask(mask, subset_size=5, spacing=0)) + assert pts == {(5, 5), (5, 10), (10, 5), (10, 10)} + + # every returned point must independently satisfy mask[y, x] is True + for y, x in pts: + assert mask[y, x] + + +def test_rois_inside_mask_all_false_returns_empty(): + mask = np.zeros((20, 20), dtype=bool) + assert rois_inside_mask(mask, subset_size=5, spacing=0) == [] + + +def test_rois_inside_mask_anisotropic_subset_size_spaces_axes_differently(): + """subset_size=(h, w) must space y by h and x by w, not by a shared step.""" + mask = np.ones((100, 100), dtype=bool) + pts = np.array(rois_inside_mask(mask, subset_size=(5, 21), spacing=0)) + + ys = np.array(sorted(set(pts[:, 0].tolist()))) + xs = np.array(sorted(set(pts[:, 1].tolist()))) + + assert len(ys) > 1 and len(xs) > 1 + assert np.all(np.diff(ys) == 5) # h + spacing + assert np.all(np.diff(xs) == 21) # w + spacing + + +def test_rois_inside_mask_scalar_matches_equal_pair(): + mask = np.zeros((20, 20), dtype=bool) + mask[5:15, 5:15] = True + scalar = rois_inside_mask(mask, subset_size=5, spacing=0) + pair = rois_inside_mask(mask, subset_size=(5, 5), spacing=0) + assert scalar == pair + + +def test_rois_inside_mask_integer_input_returns_integer_coordinates(): + """Integer subset_size/spacing (scalar and pair) -> integer points.""" + mask = np.ones((30, 30), dtype=bool) + + for subset_size in (5, (5, 8)): + pts = rois_inside_mask(mask, subset_size=subset_size, spacing=0) + assert len(pts) > 0 + for y, x in pts: + assert isinstance(y, (int, np.integer)) + assert isinstance(x, (int, np.integer)) + + +def test_rois_inside_mask_float_subset_size_does_not_crash(): + """A float subset_size is legitimate per the docstring and must not + crash the mask[y, x] indexing (it previously did, via a float-dtype + np.arange), even though it degrades to truncated pixel indices.""" + mask = np.ones((30, 30), dtype=bool) + pts = rois_inside_mask(mask, subset_size=10.5, spacing=0) + assert len(pts) > 0 + for y, x in pts: + assert isinstance(y, (int, np.integer)) + assert isinstance(x, (int, np.integer)) + assert mask[y, x] + + +# --------------------------------------------------------------------------- +# Explicit convention-split pin +# --------------------------------------------------------------------------- + +def test_convention_split_is_intentional(): + """get_roi_grid is (row, col); rois_inside_polygon is (x, y). + + This test feeds the SAME vertex list to both functions. Because the + rectangle is much longer along its first coordinate (0..100) than its + second (0..10), the two functions -- if they truly disagree about which + tuple slot is which axis -- must disagree about which axis of their + *output* is the "many points" one. + + get_roi_grid reads index 0 as row: the tall axis (0..100) becomes rows, + so almost all variation is in column 0 of the output and column 1 barely + varies (exactly 1 distinct value, verified empirically). rois_inside_polygon + reads the *same* raw vertex list as (x, y) instead -- its candidate + generation also differs slightly (it includes the far edge, get_roi_grid + does not), so its "thin" axis ends up with 2 distinct values rather than + 1. If someone "helpfully" unified the two functions to share one + convention and one candidate-generation scheme, this mismatch (1 vs 2) + would disappear or flip -- that is exactly what this test guards. + """ + verts = [(0, 0), (0, 10), (100, 10), (100, 0)] + + roi_pts = get_roi_grid(np.array(verts), roi_size=(5, 5), noverlap=0, deselect_polygon=[[], []]) + n_rows = len(set(roi_pts[:, 0].tolist())) + n_cols = len(set(roi_pts[:, 1].tolist())) + assert n_rows > 1 and n_cols == 1, (n_rows, n_cols) + + xy_pts = np.array(rois_inside_polygon(verts, subset_size=5, spacing=0)) + n_x = len(set(xy_pts[:, 0].tolist())) + n_y = len(set(xy_pts[:, 1].tolist())) + assert n_x > 1 and n_y == 2, (n_x, n_y) diff --git a/tests/test_selection_gui_anisotropic.py b/tests/test_selection_gui_anisotropic.py new file mode 100644 index 0000000..7fb2d18 --- /dev/null +++ b/tests/test_selection_gui_anisotropic.py @@ -0,0 +1,861 @@ +"""Tests for anisotropic (non-square) subset sizes in ``SelectionGUIOld``. + +``SelectionGUIOld`` is a full Qt application, but it can be constructed headlessly +for testing -- the same recipe used by +``docs/source/quick_start/make_selection_animation.py``: + +* ``QT_QPA_PLATFORM=offscreen`` must be set before Qt is imported, so Qt + renders to its software framebuffer instead of opening a real display. +* ``sys.ps1`` must be set before constructing ``SelectionGUIOld``, so its + constructor takes the "interactive" branch instead of ``sys.exit(...)``. +* ``sys.ps1`` alone is not enough: the "interactive" branch still calls + ``app.exec()``, which would block in the Qt event loop. So + ``QtWidgets.QApplication.exec`` is also monkeypatched to a no-op, letting + construction return immediately with a fully built (but not event-loop- + driven) window. +""" +import os + +os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") + +import sys # noqa: E402 + +import numpy as np # noqa: E402 +import pytest # noqa: E402 + +pytest.importorskip("PyQt6") + +from PyQt6 import QtCore, QtGui, QtWidgets # noqa: E402 + +sys.ps1 = getattr(sys, "ps1", ">>> ") # Make SelectionGUIOld think it's running interactively. +QtWidgets.QApplication.exec = lambda self=None: 0 # Neutralise the blocking event loop. + +from pyidi.GUIs.subset_selection import SelectionGUIOld # noqa: E402 +from pyidi.selection_geometry import rois_inside_polygon # noqa: E402 + +# Every test here constructs the deprecated interface on purpose, so its own +# DeprecationWarning is noise rather than a signal. The warning itself is +# covered in tests/test_feature_selection_gui.py. +pytestmark = pytest.mark.filterwarnings( + "ignore:SelectionGUIOld is deprecated:DeprecationWarning") + + +def make_image(): + """A synthetic grayscale image, large enough for the polygons used below.""" + rng = np.random.default_rng(0) + return rng.integers(0, 255, size=(200, 300), dtype=np.uint8) + + +def make_gui(**kwargs): + """Construct a headless ``SelectionGUIOld`` on a fresh synthetic image.""" + return SelectionGUIOld(make_image(), **kwargs) + + +def make_image_128x256(): + """A 128 (height) x 256 (width) synthetic image -- the size used in the bug report.""" + rng = np.random.default_rng(1) + return rng.integers(0, 255, size=(128, 256), dtype=np.uint8) + + +def overlay_extent(overlay): + """Return (extent along axis 0, extent along axis 1) of the non-zero region. + + ``overlay`` is the RGBA array from ``roi_overlay.image``; "non-zero" is + judged from the alpha channel. + """ + mask = overlay[..., 3] != 0 + axis0_idx = np.where(mask.any(axis=1))[0] + axis1_idx = np.where(mask.any(axis=0))[0] + assert axis0_idx.size and axis1_idx.size, "overlay is entirely empty" + extent0 = int(axis0_idx.max() - axis0_idx.min() + 1) + extent1 = int(axis1_idx.max() - axis1_idx.min() + 1) + return extent0, extent1 + + +def test_constructor_normalizes_scalar_subset_size(): + gui = make_gui(subset_size=11) + try: + assert gui.subset_size == (11, 11) + assert isinstance(gui.subset_size[0], int) + assert isinstance(gui.subset_size[1], int) + finally: + gui.close() + + +def test_constructor_normalizes_pair_subset_size(): + gui = make_gui(subset_size=(5, 21)) + try: + assert gui.subset_size == (5, 21) + finally: + gui.close() + + +def test_get_subset_size_reflects_spinboxes(): + gui = make_gui(subset_size=(5, 21)) + try: + # An anisotropic pair starts with Square unchecked, so both spinboxes reflect + # the values they were constructed with. + assert not gui.square_subsets_checkbox.isChecked() + assert gui.get_subset_size() == (5, 21) + + gui.subset_width_spinbox.setValue(25) + assert gui.get_subset_size() == (5, 25) + + gui.subset_height_spinbox.setValue(9) + assert gui.get_subset_size() == (9, 25) + finally: + gui.close() + + +def test_square_checkbox_toggle_locks_and_syncs_width(): + gui = make_gui(subset_size=11) + try: + assert gui.square_subsets_checkbox.isChecked() + assert not gui.subset_width_spinbox.isEnabled() + assert not gui.subset_width_slider.isVisible() + + gui.square_subsets_checkbox.setChecked(False) + assert gui.subset_width_spinbox.isEnabled() + + gui.subset_width_spinbox.setValue(31) + assert gui.get_subset_size() == (11, 31) + + # Toggling square back on snaps width back to height. + gui.square_subsets_checkbox.setChecked(True) + assert not gui.subset_width_spinbox.isEnabled() + assert gui.get_subset_size() == (11, 11) + finally: + gui.close() + + +def test_anisotropic_grid_roi_points_have_different_row_and_column_spacing(): + h, w, overlap = 5, 21, 2 + gui = make_gui(subset_size=(h, w), subset_overlap=overlap) + try: + # A rectangular polygon covering most of the synthetic image. + entry = gui.add_selection('grid', geometry=[(0, 0), (280, 0), (280, 180), (0, 180)]) + gui.recompute_roi_points() + + roi_points = entry['roi_points'] + assert len(roi_points) > 4, "expected a genuine grid of points, not a degenerate case" + + xs = sorted(set(p[0] for p in roi_points)) + ys = sorted(set(p[1] for p in roi_points)) + x_steps = set(round(b - a) for a, b in zip(xs, xs[1:])) + y_steps = set(round(b - a) for a, b in zip(ys, ys[1:])) + + assert x_steps == {w + overlap} + assert y_steps == {h + overlap} + assert x_steps != y_steps + finally: + gui.close() + + +def test_square_subset_size_matches_pre_change_geometry_reference(): + """Regression: a square subset_size must still produce exactly the old ROI points.""" + subset_size, overlap = 15, 3 + gui = make_gui(subset_size=subset_size, subset_overlap=overlap) + try: + polygon = [(10, 10), (250, 10), (250, 150), (10, 150)] + entry = gui.add_selection('grid', geometry=polygon) + gui.recompute_roi_points() + + expected = rois_inside_polygon(polygon, subset_size, overlap) + assert entry['roi_points'] == expected + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Rectangle-drawing / filter-ROI orientation (the "drawn transposed" bug) +# --------------------------------------------------------------------------- + +def test_rectangle_overlay_extent_matches_subset_size_axes(): + """subset_size=(5, 25) must draw 25 px along x (axis 0) and 5 px along y (axis 1). + + A manual selection entry's ``geometry`` uses the internal (x, y) convention: (128, 64) sits well + inside the 128 (height) x 256 (width) image used here, whereas (64, 128) + would not (x=64 < 256 is fine, but as a first coordinate it would be + read as a row and 128 is out of the 0..127 row range) -- see the class + docstring / bug report for why the ordering matters. + """ + gui = SelectionGUIOld(make_image_128x256(), subset_size=(5, 25)) + try: + entry = gui.add_selection('manual', geometry=[(128, 64)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + extent0, extent1 = overlay_extent(gui.roi_overlay.image) + assert extent0 == 25, f"expected 25 px along axis 0 (x/width), got {extent0}" + assert extent1 == 5, f"expected 5 px along axis 1 (y/height), got {extent1}" + finally: + gui.close() + + +def test_square_subset_overlay_matches_independent_reference(): + """Regression: a square subset_size must fill byte-identical overlay pixels. + + Only the translucent interior lives in ``roi_overlay``; the border is a separate + vector path (see ``test_subset_border_path_traces_the_filled_area``). + """ + subset_size = 15 + half = subset_size // 2 + gui = SelectionGUIOld(make_image_128x256(), subset_size=subset_size) + try: + px, py = 128, 64 + entry = gui.add_selection('manual', geometry=[(px, py)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + n_x, n_y = gui.image_item.image.shape[:2] + expected = np.zeros((n_x, n_y, 4), dtype=np.uint8) + ix0, iy0, ix1, iy1 = px - half, py - half, px + half + 1, py + half + 1 + expected[ix0:ix1, iy0:iy1, 1] = 180 + expected[ix0:ix1, iy0:iy1, 3] = 40 + + np.testing.assert_array_equal(gui.roi_overlay.image, expected) + finally: + gui.close() + + +def _path_subpaths(path): + """Split a QPainterPath into a list of (x, y) vertex arrays, one per sub-path.""" + subpaths, current = [], [] + for i in range(path.elementCount()): + el = path.elementAt(i) + if el.type == QtGui.QPainterPath.ElementType.MoveToElement and current: + subpaths.append(np.array(current)) + current = [] + current.append((el.x, el.y)) + if current: + subpaths.append(np.array(current)) + return subpaths + + +def test_subset_border_path_traces_the_filled_area(): + """The border path must outline exactly the pixels the overlay fills. + + The border is stroked with a cosmetic pen so it stays a hairline at any zoom; + that only reads as a subset boundary if its corners sit on the fill's corners. + """ + subset_size = 15 + half = subset_size // 2 + gui = SelectionGUIOld(make_image_128x256(), subset_size=subset_size) + try: + px, py = 128, 64 + entry = gui.add_selection('manual', geometry=[(px, py)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + assert gui.roi_outline.pen().isCosmetic() + + subpaths = _path_subpaths(gui.roi_outline.path()) + assert len(subpaths) == 1, f"expected one rectangle, got {len(subpaths)}" + corners = subpaths[0] + # Pixel ix is the view-coordinate band [ix, ix + 1), so the rectangle spanning + # pixels ix0..ix1-1 runs from ix0 to ix1 in view coordinates. + assert corners[:, 0].min() == px - half + assert corners[:, 0].max() == px + half + 1 + assert corners[:, 1].min() == py - half + assert corners[:, 1].max() == py + half + 1 + finally: + gui.close() + + +def test_subset_borders_are_dropped_together_with_the_fill(): + """A subset whose rectangle runs off the image edge gets neither fill nor border.""" + gui = SelectionGUIOld(make_image_128x256(), subset_size=15) + try: + entry = gui.add_selection('manual', geometry=[(128, 64), (2, 2)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + assert len(_path_subpaths(gui.roi_outline.path())) == 1 + assert overlay_extent(gui.roi_overlay.image) == (15, 15) + finally: + gui.close() + + +def _sobel_shapes(gui, monkeypatch): + """Patch ``scipy.ndimage.sobel`` to record every ROI shape it is called with.""" + import scipy.ndimage as ndi + + shapes = [] + original_sobel = ndi.sobel + + def spy_sobel(roi, axis): + shapes.append(roi.shape) + return original_sobel(roi, axis=axis) + + monkeypatch.setattr(ndi, "sobel", spy_sobel) + return shapes + + +def test_shi_tomasi_roi_shape_matches_subset_size_axes(monkeypatch): + """The ROI sliced inside compute_candidate_points_shi_tomasi must be (2w+1, 2h+1).""" + gui = SelectionGUIOld(make_image_128x256(), subset_size=(5, 25)) + try: + entry = gui.add_selection('manual', geometry=[(128, 64)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + shapes = _sobel_shapes(gui, monkeypatch) + gui.compute_candidate_points_shi_tomasi() + + half_h, half_w = 5 // 2, 25 // 2 + assert shapes, "sobel was never called -- point was skipped by the bounds check" + for shape in shapes: + assert shape == (2 * half_w + 1, 2 * half_h + 1), shape + finally: + gui.close() + + +def test_gradient_direction_roi_shape_matches_subset_size_axes(monkeypatch): + """The ROI sliced inside compute_candidate_points_gradient_direction must be (2w+1, 2h+1).""" + gui = SelectionGUIOld(make_image_128x256(), subset_size=(5, 25)) + try: + entry = gui.add_selection('manual', geometry=[(128, 64)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + gui.gradient_direction = (1.0, 0.0) + + shapes = _sobel_shapes(gui, monkeypatch) + gui.compute_candidate_points_gradient_direction() + + half_h, half_w = 5 // 2, 25 // 2 + assert shapes, "sobel was never called -- point was skipped by the bounds check" + for shape in shapes: + assert shape == (2 * half_w + 1, 2 * half_h + 1), shape + finally: + gui.close() + + +def test_square_subset_filter_roi_shape_unchanged(monkeypatch): + """Regression: a square subset_size must still produce a square filter ROI.""" + gui = SelectionGUIOld(make_image_128x256(), subset_size=15) + try: + entry = gui.add_selection('manual', geometry=[(128, 64)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + shapes = _sobel_shapes(gui, monkeypatch) + gui.compute_candidate_points_shi_tomasi() + + assert shapes + for shape in shapes: + assert shape == (15, 15), shape + finally: + gui.close() + + +def test_gradient_direction_dx_dy_convention_matches_real_axes(): + """The gradient-direction filter must respond to the real image gradient. + + ``compute_candidate_points_gradient_direction`` unpacks + ``dy, dx = self.gradient_direction`` (names swapped relative to the real + values) and treats ``sobel(roi, axis=1)`` as "gx" (also swapped, since + roi axis 1 is the real y/height axis). Algebraically the two swaps + cancel in ``|gx * dx| + |gy * dy|``, so this is NOT a bug -- pinned here + with an image that has a gradient in only one real axis, checked in both + directions. Uses a square subset_size so this is independent of the + anisotropic ROI-extent fix. + """ + width, height = 100, 60 + xs = np.arange(width) + frame = np.tile(xs, (height, 1)).astype(np.uint8) # frame[y, x] = x: varies only in x. + + gui = SelectionGUIOld(frame, subset_size=15) + try: + entry = gui.add_selection('manual', geometry=[(50, 30)]) + gui.recompute_entry(entry, gui.get_subset_size(), gui.distance_spinbox.value()) + gui.update_selected_points() + + gui.gradient_direction = (1.0, 0.0) # real x direction + gui.compute_candidate_points_gradient_direction() + strength_x = gui.candidates_grad_dir[0][2] + + gui.gradient_direction = (0.0, 1.0) # real y direction + gui.compute_candidate_points_gradient_direction() + strength_y = gui.candidates_grad_dir[0][2] + + assert strength_x > 0 + assert strength_y == pytest.approx(0, abs=1e-6) + assert strength_x > strength_y + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Unified selection-entry list (self.selections) -- see pyidi/GUIs/subset_selection.py +# --------------------------------------------------------------------------- + +class _FakeMouseEvent: + """Minimal stand-in for a pyqtgraph mouse-click event: just ``.scenePos()``.""" + + def __init__(self, scene_pos): + self._scene_pos = scene_pos + + def scenePos(self): + return self._scene_pos + + +def _lay_out(gui): + """Give the view a sane data range so simulated clicks land inside it. + + Until the view has been ranged, ``mapViewToScene`` sends image coordinates far + outside ``sceneBoundingRect()`` and every click handler bails out as "outside the + view". ``autoRange()`` does this synchronously. Pumping the event loop + (``QApplication.processEvents()``) has the same effect but intermittently aborts + the interpreter under the offscreen platform once the multiprocessing tests + earlier in the suite have run, so it is deliberately avoided here. + """ + gui.view.autoRange() + + +def _click(gui, x, y): + """Simulate a click at data coordinates (x, y) through the real on_mouse_click path.""" + _lay_out(gui) + scene_pos = gui.view.mapViewToScene(QtCore.QPointF(x, y)) + gui.on_mouse_click(_FakeMouseEvent(scene_pos)) + + +def _set_method(gui, name): + """Check `name`'s method button and run the real method_selected handler.""" + button = gui.method_buttons[name] + button.setChecked(True) + gui.method_selected(gui.button_group.id(button)) + + +def test_manual_selection_is_singleton(): + """Two manual clicks must land in ONE entry with two points, not two entries.""" + gui = make_gui() + try: + _set_method(gui, "Manual") + _click(gui, 10, 10) + _click(gui, 20, 20) + + manual_entries = [e for e in gui.selections if e['kind'] == 'manual'] + assert len(manual_entries) == 1 + assert len(manual_entries[0]['geometry']) == 2 + assert len(gui.selections) == gui.selection_list.count() + finally: + gui.close() + + +def _deselect(gui, mask): + """Run a deselect-mode brush stroke covering `mask` through the real handler.""" + gui.brush_deselect_mode = True + gui._paint_mask = mask + gui.handle_brush_end(_FakeMouseEvent(gui.view.mapViewToScene(QtCore.QPointF(0.0, 0.0)))) + + +def test_brush_deselect_only_removes_the_painted_part_of_a_stroke(): + """Deselecting part of a brush stroke must not discard the whole stroke. + + The stroke is subtracted from the painted mask, so the untouched part survives -- + and, because the mask itself is edited, the removal outlasts a spacing change. + """ + gui = make_gui() + try: + _lay_out(gui) + shape = gui.image_item.image.shape[:2] + mask = np.zeros(shape, bool) + mask[20:280, 20:180] = True + brush = gui.add_selection('brush', geometry=mask) + gui.recompute_roi_points() + before = len(gui.entry_points(brush)) + + deselected = np.zeros(shape, bool) + deselected[20:70, 20:70] = True + _deselect(gui, deselected.copy()) + + assert len(gui.selections) == gui.selection_list.count() == 1, "the whole stroke was discarded" + after = len(gui.entry_points(brush)) + assert 0 < after < before, "expected a partial removal, not all-or-nothing" + + # The mask edit -- not a derived-point filter -- is what makes this stick. + gui.distance_spinbox.setValue(gui.distance_spinbox.value() + 5) + still_inside = [p for p in gui.entry_points(brush) if deselected[int(p[0]), int(p[1])]] + assert not still_inside, "the deselected area came back after a recompute" + finally: + gui.close() + + +def test_brush_deselect_drops_a_stroke_only_once_nothing_is_left_painted(): + """A fully-covered brush row goes away, which shifts every later index. + + The active entry must then be re-derived from the entry object rather than left + as a stale index, and points of other kinds under the stroke must be removed. + """ + gui = make_gui() + try: + _lay_out(gui) + shape = gui.image_item.image.shape[:2] + grid = gui.add_selection('grid', geometry=[(20, 20), (280, 20), (280, 180), (20, 180)]) + first_mask = np.zeros(shape, bool) + first_mask[30:80, 30:80] = True + gui.add_selection('brush', geometry=first_mask) + second_mask = np.zeros(shape, bool) + second_mask[200:260, 100:160] = True + second_brush = gui.add_selection('brush', geometry=second_mask) + gui.recompute_roi_points() + + gui.selection_list.setCurrentRow(2) + gui.on_entry_selected(2) + + covers_first = np.zeros(shape, bool) + covers_first[25:85, 25:85] = True # strictly contains first_mask + _deselect(gui, covers_first) + + assert len(gui.selections) == gui.selection_list.count() == 2, "the fully-covered brush row should be gone" + assert gui.selections[gui.active_index] is second_brush, "active entry was not preserved" + assert grid['removed'], "grid points under the deselect stroke were not removed" + finally: + gui.close() + + +def test_highlight_tracks_the_active_row(): + """The highlight scatter must show exactly the active entry's points, and nothing + when that entry is hidden via its checkbox.""" + gui = make_gui() + + def n_highlighted(): + data = gui.highlight_scatter.getData()[0] + return 0 if data is None else len(data) + + try: + first = gui.add_selection('grid', geometry=[(20, 20), (120, 20), (120, 90)]) + second = gui.add_selection('grid', geometry=[(150, 20), (250, 20), (250, 90)]) + gui.recompute_roi_points() + + # add_selection made the second entry active. + assert n_highlighted() == len(gui.entry_points(second)) + + gui.selection_list.setCurrentRow(0) + gui.on_entry_selected(0) + assert n_highlighted() == len(gui.entry_points(first)) + + gui.selection_list.item(0).setCheckState(QtCore.Qt.CheckState.Unchecked) + assert n_highlighted() == 0 + finally: + gui.close() + + +def test_switching_tool_away_and_back_continues_the_same_grid(): + """Regression: leaving Grid mode and returning must not silently start a new grid. + + The pre-list code kept a separate ``active_grid_index``/``active_polygon_index`` + per kind; a single ``active_index`` loses that unless the tool switch + re-activates the most recent entry of the kind being switched to. + """ + gui = make_gui() + try: + _set_method(gui, "Grid") + for vertex in [(20, 20), (200, 20), (200, 150)]: + _click(gui, *vertex) + + _set_method(gui, "Manual") # step away... + _click(gui, 50, 50) + _set_method(gui, "Grid") # ...and back + _click(gui, 20, 150) + + grids = [e for e in gui.selections if e['kind'] == 'grid'] + assert len(grids) == 1, "a second grid was started instead of continuing the first" + assert len(grids[0]['geometry']) == 4 + finally: + gui.close() + + +def test_clicking_a_row_overrides_the_most_recent_entry_of_that_kind(): + """Selecting a specific row must win over the 'continue the latest one' rule.""" + gui = make_gui() + try: + _set_method(gui, "Grid") + for vertex in [(20, 20), (120, 20), (120, 90)]: + _click(gui, *vertex) + gui.start_new_line() + for vertex in [(150, 20), (250, 20), (250, 90)]: + _click(gui, *vertex) + + gui.selection_list.setCurrentRow(0) + gui.on_entry_selected(0) + _click(gui, 20, 90) + + assert len(gui.selections[0]['geometry']) == 4, "the explicitly selected row was overridden" + assert len(gui.selections[1]['geometry']) == 3 + finally: + gui.close() + + +def test_each_stroke_creates_its_own_entry_and_stays_in_sync(): + """Every grid/line click sequence and every brush stroke gets its own entry.""" + gui = make_gui() + try: + _set_method(gui, "Grid") + for (x, y) in [(20, 20), (150, 20), (150, 120), (20, 120)]: + _click(gui, x, y) + assert len(gui.selections) == gui.selection_list.count() == 1 + + _set_method(gui, "Along the line") + for (x, y) in [(30, 30), (200, 30)]: + _click(gui, x, y) + assert len(gui.selections) == gui.selection_list.count() == 2 + + _set_method(gui, "Brush") + _lay_out(gui) + for cx, cy in [(60, 60), (220, 150)]: + gui.handle_brush_start(_FakeMouseEvent(gui.view.mapViewToScene(QtCore.QPointF(cx, cy)))) + gui.handle_brush_end(_FakeMouseEvent(gui.view.mapViewToScene(QtCore.QPointF(cx, cy)))) + assert len(gui.selections) == gui.selection_list.count() == 4 + + assert [e['kind'] for e in gui.selections] == ['grid', 'line', 'brush', 'brush'] + finally: + gui.close() + + +def test_labels_are_monotonic_never_reused_after_delete(): + """Deleting Grid 2 and adding a new grid must give Grid 4, never a duplicate Grid 3.""" + gui = make_gui() + try: + gui.add_selection('grid') + gui.add_selection('grid') + gui.add_selection('grid') + assert [e['label'] for e in gui.selections] == ['Grid 1', 'Grid 2', 'Grid 3'] + + gui.selection_list.setCurrentRow(1) + gui.delete_selected_entry() + + gui.add_selection('grid') + labels = [e['label'] for e in gui.selections] + assert labels == ['Grid 1', 'Grid 3', 'Grid 4'] + assert len(set(labels)) == len(labels), "no two rows should share a label" + assert len(gui.selections) == gui.selection_list.count() + finally: + gui.close() + + +def test_unchecking_row_removes_and_rechecking_restores_its_points(): + """Toggling a row's visibility checkbox removes/restores exactly that entry's points.""" + gui = make_gui() + try: + entry = gui.add_selection('manual', geometry=[(10, 10), (20, 20)]) + gui.recompute_entry(entry) + gui.update_selected_points() + + n_before = len(gui.points) + assert n_before == 2 + + item = gui.selection_list.item(0) + item.setCheckState(QtCore.Qt.CheckState.Unchecked) + assert not gui.selections[0]['visible'] + assert len(gui.points) == 0 + + item.setCheckState(QtCore.Qt.CheckState.Checked) + assert gui.selections[0]['visible'] + assert len(gui.points) == n_before + assert len(gui.selections) == gui.selection_list.count() + finally: + gui.close() + + +def test_removed_point_survives_recompute(): + """The bug fix: a point removed via `removed` must not reappear after a recompute. + + Before the fix, ``handle_remove_point`` deleted the point straight out of + ``roi_points``, which ``recompute_roi_points`` regenerates from scratch -- so a + later spacing/subset-size change silently un-removed it. Now the removal is + recorded in ``entry['removed']`` and applied only at read time by + ``entry_points``, so it survives. + """ + gui = make_gui(subset_size=15, subset_overlap=2) + try: + polygon = [(10, 10), (250, 10), (250, 150), (10, 150)] + entry = gui.add_selection('grid', geometry=polygon) + gui.recompute_roi_points() + assert len(entry['roi_points']) > 1 + + target = tuple(entry['roi_points'][0]) + entry['removed'].add(target) + gui.update_selected_points() + assert target not in gui.selected_points + + # Recompute with the same subset size/spacing regenerates `roi_points` + # identically (`target` is back in it), but `removed` must still filter it + # out of the effective points -- this is the actual bug being fixed. + gui.recompute_roi_points() + assert target in entry['roi_points'], "sanity check: recompute regenerates the same point" + assert target not in gui.entry_points(entry) + assert target not in gui.selected_points + + # A genuine spacing change (as in the bug report) must not resurrect it either. + gui.distance_spinbox.setValue(gui.distance_spinbox.value() + 3) + assert target not in gui.entry_points(entry) + assert target not in gui.selected_points + finally: + gui.close() + + +def test_undo_restores_deleted_entry_at_original_row_and_label(): + """Delete + Ctrl+Z (undo()) must restore an entry at its original row/label. + + Checked for a `grid` entry and, since delete is now generic, a `brush` entry + too -- brush deletions were not undoable before this refactor. + """ + gui = make_gui() + try: + gui.add_selection('grid', geometry=[(10, 10), (50, 10), (50, 50)]) + gui.add_selection('brush', geometry=np.zeros(gui.image_item.image.shape[:2], dtype=bool)) + gui.add_selection('manual', geometry=[(5, 5)]) + + grid_label = gui.selections[0]['label'] + gui.selection_list.setCurrentRow(0) + gui.delete_selected_entry() + gui.undo() + assert gui.selections[0]['kind'] == 'grid' + assert gui.selections[0]['label'] == grid_label + assert len(gui.selections) == gui.selection_list.count() == 3 + + brush_label = gui.selections[1]['label'] + gui.selection_list.setCurrentRow(1) + gui.delete_selected_entry() + gui.undo() + assert gui.selections[1]['kind'] == 'brush' + assert gui.selections[1]['label'] == brush_label + assert len(gui.selections) == gui.selection_list.count() == 3 + finally: + gui.close() + + +def test_selected_points_order_matches_entry_creation_order(): + """`selected_points` order is creation order across kinds (manual/grid/line mixed).""" + gui = make_gui() + try: + manual_entry = gui.add_selection('manual', geometry=[(5, 5)]) + gui.recompute_entry(manual_entry) + + grid_entry = gui.add_selection('grid', geometry=[(10, 10), (60, 10), (60, 60), (10, 60)]) + gui.recompute_entry(grid_entry, subset_size=10, spacing=0) + + line_entry = gui.add_selection('line', geometry=[(70, 70), (150, 70)]) + gui.recompute_entry(line_entry, subset_size=10, spacing=0) + + gui.update_selected_points() + + expected = gui.entry_points(manual_entry) + gui.entry_points(grid_entry) + gui.entry_points(line_entry) + assert gui.selected_points == expected + finally: + gui.close() + + +def test_brush_roi_points_step_matches_subset_size_axes(): + """The brush anisotropic-spacing fix: (h=5, w=21) must step 21 px along x, 5 px along y. + + This fails before the ``_brush_points`` transpose fix and passes after it. + """ + h, w = 5, 21 + gui = make_gui(subset_size=(h, w), subset_overlap=0) + try: + n_x, n_y = gui.image_item.image.shape[:2] + mask = np.ones((n_x, n_y), dtype=bool) # paint the whole image + entry = gui.add_selection('brush', geometry=mask) + gui.recompute_entry(entry) + + roi_points = entry['roi_points'] + assert len(roi_points) > 4, "expected a genuine grid of points, not a degenerate case" + + xs = sorted(set(p[0] for p in roi_points)) + ys = sorted(set(p[1] for p in roi_points)) + x_steps = set(round(b - a) for a, b in zip(xs, xs[1:])) + y_steps = set(round(b - a) for a, b in zip(ys, ys[1:])) + + assert x_steps == {w} + assert y_steps == {h} + finally: + gui.close() + + +# --------------------------------------------------------------------------- +# Filter candidates following the selection +# --------------------------------------------------------------------------- + +def _run_shi_tomasi(gui, threshold=1): + """Run the Shi-Tomasi filter over the current selection and keep nearly everything.""" + gui.switch_mode("filter") + gui.compute_candidate_points_shi_tomasi() + gui.threshold_slider.setValue(threshold) + gui.update_threshold_and_show_shi_tomsi() + gui.switch_mode("selection") + + +def _candidates_outside_selection(gui): + selected = {(int(round(px)), int(round(py))) for px, py in gui.selected_points} + return [p for p in gui.candidate_points if (int(round(p[0])), int(round(p[1]))) not in selected] + + +def _brushed_gui(): + """A GUI with one brush stroke covering most of the image, already filtered.""" + gui = make_gui() + _lay_out(gui) + mask = np.zeros(gui.image_item.image.shape[:2], bool) + mask[20:280, 20:180] = True + entry = gui.add_selection('brush', geometry=mask) + gui.recompute_roi_points() + _run_shi_tomasi(gui) + assert gui.candidate_points, "the filter produced no candidates to test with" + return gui, entry + + +def test_brush_deselect_drops_the_filter_candidates_it_removes(): + """Deselected subsets must leave the candidates too, not just the selection. + + ``get_points()`` returns the candidates once a filter has been run, so a candidate + left behind by a deselect stays in the returned points. + """ + gui, _ = _brushed_gui() + try: + before = len(gui.candidate_points) + + deselected = np.zeros(gui.image_item.image.shape[:2], bool) + deselected[20:150, 20:100] = True + _deselect(gui, deselected.copy()) + + assert _candidates_outside_selection(gui) == [] + assert len(gui.candidate_points) < before, "no candidate was dropped" + assert len(gui.get_points()) == len(gui.candidate_points) + + # The threshold slider re-derives the candidates from the cached scores, so it + # is the obvious way for a dropped candidate to come back. + gui.update_threshold_and_show_shi_tomsi() + assert _candidates_outside_selection(gui) == [] + finally: + gui.close() + + +def test_unchecking_a_row_hides_its_candidates_and_rechecking_restores_them(): + """The row checkbox is a "try it in and out" control, so it must not be one-way.""" + gui, entry = _brushed_gui() + try: + before = len(gui.candidate_points) + + entry['visible'] = False + gui.update_selected_points() + assert gui.candidate_points == [] + + entry['visible'] = True + gui.update_selected_points() + assert len(gui.candidate_points) == before, "the filter result was not restored" + finally: + gui.close() + + +def test_clear_candidates_is_not_undone_by_the_next_selection_change(): + """Clearing must stick: the cached scores are still there to be re-derived from.""" + gui, _ = _brushed_gui() + try: + gui.clear_candidates() + assert gui.candidate_points == [] + + gui.update_selected_points() + assert gui.candidate_points == [], "the cleared candidates came back" + finally: + gui.close() diff --git a/tests/test_selection_pipeline.py b/tests/test_selection_pipeline.py new file mode 100644 index 0000000..c3abb4b --- /dev/null +++ b/tests/test_selection_pipeline.py @@ -0,0 +1,1459 @@ +""" +Tests for ``pyidi/selection/``, the headless mask -> evaluate -> select pipeline. + +Nothing here imports Qt. The whole point of the package is that the pipeline is +usable without an interface, so these tests exercise it the way a script would. + +The three steps are tested separately and then together: + +- **evaluate** -- the score image, its NaN border, and agreement with a direct + per-subset reference implementation. The vectorised form is the reason dense + evaluation is affordable at all, so "does it compute the same number" is the + load-bearing assertion in this file. +- **mask** -- what each entry kind rasterises to, how roles decide whether an + entry contributes an area or coordinates, and how deselection is applied + without destroying the geometry it came from. +- **select** -- threshold, minimum-distance suppression and the merge with + hand-picked points, which are what stop a dense score image from returning a + blob of adjacent pixels on every corner. +""" + +import sys + +import numpy as np +import pytest +from scipy.ndimage import sobel + +from pyidi.selection import ( + DEFAULT_ROLE, + Entry, + ScoreStore, + SelectionPipeline, + all_literal_points, + apply_deselection, + as_point_array, + available_evaluators, + combined_mask, + decimate, + evaluate, + get_evaluator, + half_window, + literal_points, + merge_points, + occupancy, + rasterize, + select, + select_lattice, + select_peaks, + ROBUST_MAXIMUM_PERCENTILE, + THRESHOLD_MODES, + select_points, + threshold_value, + window_size, +) + + +# --------------------------------------------------------------------------- +# Fixtures -- one image with a flat region, a straight edge, a corner and a +# speckle patch, so a single frame can answer most of the questions below. +# --------------------------------------------------------------------------- + +@pytest.fixture +def image(): + """160x200 uint16 frame: flat background, a bright square, a speckle patch.""" + img = np.full((160, 200), 40, dtype=np.uint16) + img[30:80, 30:80] = 220 # square: corners at (30, 30) etc, edges between + rng = np.random.default_rng(12345) + img[100:150, 100:190] = rng.integers(20, 240, (50, 90)) # speckle + return img + + +@pytest.fixture +def speckle(): + """A frame that is speckle everywhere, so every subset has something to score.""" + rng = np.random.default_rng(7) + return rng.integers(0, 255, (128, 128)).astype(np.uint16) + + +def reference_shi_tomasi(image, row, col, half): + """Score one subset the slow way, as ``SelectionGUIOld`` does it. + + Gradients are taken over the whole image rather than over the isolated ROI: + that is the one intentional difference between the old implementation and + the new one, and it is not what this reference is here to check. + """ + img = np.asarray(image, dtype=np.float64) + g_row, g_col = sobel(img, axis=0), sobel(img, axis=1) + box = (slice(row - half[0], row + half[0] + 1), slice(col - half[1], col + half[1] + 1)) + a = float((g_col[box] ** 2).sum()) + c = float((g_row[box] ** 2).sum()) + b = float((g_col[box] * g_row[box]).sum()) + return float(np.linalg.eigvalsh(np.array([[a, b], [b, c]]))[0]) + + +def rect(r0, c0, r1, c1): + """A rectangular polygon as ``(row, col)`` vertices.""" + return [(r0, c0), (r0, c1), (r1, c1), (r1, c0)] + + +# --------------------------------------------------------------------------- +# evaluate -- shape, dtype, window and border +# --------------------------------------------------------------------------- + +def test_score_image_has_the_image_shape_and_is_float32(image): + score = evaluate(image, 'shi_tomasi', 11) + assert score.shape == image.shape + assert score.dtype == np.float32 + + +def test_window_is_odd_even_for_an_even_subset_size(): + assert window_size(11) == (11, 11) + assert window_size(10) == (11, 11) # 2 * (10 // 2) + 1 + assert window_size((21, 7)) == (21, 7) + assert half_window((21, 7)) == (10, 3) + + +def test_border_is_nan_and_the_interior_is_finite(image): + score = evaluate(image, 'shi_tomasi', 11) + assert np.isnan(score[:5]).all() + assert np.isnan(score[-5:]).all() + assert np.isnan(score[:, :5]).all() + assert np.isnan(score[:, -5:]).all() + assert np.isfinite(score[5:-5, 5:-5]).all() + + +def test_anisotropic_border_depth_differs_per_axis(image): + score = evaluate(image, 'shi_tomasi', (21, 7)) + assert np.isnan(score[:10]).all() + assert np.isnan(score[:, :3]).all() + assert np.isfinite(score[10:-10, 3:-3]).all() + + +def test_nan_never_passes_a_threshold(image): + score = evaluate(image, 'shi_tomasi', 11) + # The invalid border is excluded by the comparison itself, which is why no + # separate validity test is threaded through the selectors. + assert not (score[:5] > -np.inf).any() + assert not (score > np.nanmin(score) - 1)[np.isnan(score)].any() + + +# --------------------------------------------------------------------------- +# evaluate -- the evaluators themselves +# --------------------------------------------------------------------------- + +def test_shi_tomasi_matches_the_per_subset_reference(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + half = half_window(11) + ours, reference = [], [] + for row in range(20, 110, 17): + for col in range(20, 110, 13): + ours.append(float(score[row, col])) + reference.append(reference_shi_tomasi(speckle, row, col, half)) + ours, reference = np.array(ours), np.array(reference) + np.testing.assert_allclose(ours, reference, rtol=1e-4, atol=1e-4 * reference.max()) + + +def test_flat_image_scores_exactly_zero(): + score = evaluate(np.full((60, 60), 17, dtype=np.uint16), 'shi_tomasi', 11) + finite = score[np.isfinite(score)] + assert finite.size + assert (finite == 0.0).all() + + +def test_a_corner_outscores_an_edge(image): + score = evaluate(image, 'shi_tomasi', 11) + corner = score[30, 30] # top-left corner of the bright square + edge = score[55, 30] # midpoint of its left edge + assert corner > edge + assert edge >= 0.0 + + +def test_gradient_direction_is_selective(speckle): + stripes = np.zeros((80, 80), dtype=np.uint16) + stripes[:, ::8] = 255 # intensity varies along columns only + along_cols = evaluate(stripes, 'gradient_direction', 11, direction=(0, 1)) + along_rows = evaluate(stripes, 'gradient_direction', 11, direction=(1, 0)) + interior = (slice(10, -10), slice(10, -10)) + assert along_cols[interior].mean() > along_rows[interior].mean() + + +def test_gradient_direction_normalises_its_direction(speckle): + one = evaluate(speckle, 'gradient_direction', 11, direction=(0, 1)) + five = evaluate(speckle, 'gradient_direction', 11, direction=(0, 5)) + np.testing.assert_allclose(one[10:-10, 10:-10], five[10:-10, 10:-10], rtol=1e-6) + + +def test_zero_direction_is_rejected(speckle): + with pytest.raises(ValueError, match='non-zero'): + evaluate(speckle, 'gradient_direction', 11, direction=(0, 0)) + + +# --------------------------------------------------------------------------- +# evaluate -- registry +# --------------------------------------------------------------------------- + +def test_registry_lists_the_built_in_evaluators(): + names = available_evaluators() + assert 'shi_tomasi' in names + assert 'gradient_direction' in names + + +def test_unknown_evaluator_names_the_registered_ones(speckle): + with pytest.raises(ValueError, match='shi_tomasi'): + evaluate(speckle, 'no_such_evaluator', 11) + + +def test_unknown_parameter_is_rejected(speckle): + with pytest.raises(ValueError, match='direction'): + evaluate(speckle, 'gradient_direction', 11, dirction=(0, 1)) + + +def test_every_evaluator_parameter_is_described(): + spec = get_evaluator('gradient_direction') + described = {p.name for p in spec.parameters} + assert described == {'direction'} + parameter = spec.parameters[0] + assert parameter.kind == 'direction' + assert parameter.default == (0.0, 1.0) + + +# --------------------------------------------------------------------------- +# evaluate -- bounding-box crop +# --------------------------------------------------------------------------- + +def test_cropped_evaluation_matches_uncropped_inside_the_mask(image): + mask = np.zeros(image.shape, dtype=bool) + mask[40:70, 40:70] = True + full = evaluate(image, 'shi_tomasi', 11, crop=False) + cropped = evaluate(image, 'shi_tomasi', 11, mask=mask, crop=True) + np.testing.assert_allclose(full[mask], cropped[mask], rtol=1e-5) + + +def test_cropped_evaluation_is_nan_outside_the_padded_box(image): + mask = np.zeros(image.shape, dtype=bool) + mask[40:70, 40:70] = True + cropped = evaluate(image, 'shi_tomasi', 11, mask=mask, crop=True) + assert np.isnan(cropped[:30]).all() + assert np.isnan(cropped[85:]).all() + + +def test_an_empty_mask_evaluates_to_all_nan(image): + empty = np.zeros(image.shape, dtype=bool) + assert np.isnan(evaluate(image, 'shi_tomasi', 11, mask=empty, crop=True)).all() + + +def test_dense_evaluation_is_fast_enough_to_be_interactive(): + import time + + rng = np.random.default_rng(3) + big = rng.integers(0, 255, (1000, 1000)).astype(np.uint16) + evaluate(big, 'shi_tomasi', 11) # warm scipy up + start = time.perf_counter() + evaluate(big, 'shi_tomasi', 11) + elapsed = time.perf_counter() - start + # A generous ceiling: the point is that this is not the minutes a per-subset + # loop would take, not to pin a particular machine's timing. + assert elapsed < 0.5, f'dense evaluation took {elapsed:.3f} s' + + +# --------------------------------------------------------------------------- +# scores -- caching +# --------------------------------------------------------------------------- + +def test_a_repeated_request_is_served_from_cache(speckle): + store = ScoreStore(speckle, 11) + store.define('corners', 'shi_tomasi') + first = store.get('corners') + assert store.n_evaluations == 1 + second = store.get('corners') + assert store.n_evaluations == 1 + assert first is second + + +def test_two_named_scores_coexist(speckle): + store = ScoreStore(speckle, 11) + store.define('corners', 'shi_tomasi') + store.define('sideways', 'gradient_direction', direction=(0, 1)) + corners = store.get('corners') + store.get('sideways') + assert store.is_cached('corners') + assert store.get('corners') is corners + assert set(store.names) == {'corners', 'sideways'} + + +def test_the_same_computation_under_two_names_is_computed_once(speckle): + store = ScoreStore(speckle, 11) + store.define('a', 'shi_tomasi') + store.define('b', 'shi_tomasi') + store.get('a') + store.get('b') + assert store.n_evaluations == 1 + + +def test_a_subset_size_change_invalidates_every_score(speckle): + store = ScoreStore(speckle, 11) + store.define('corners', 'shi_tomasi') + store.get('corners') + store.set_subset_size(21) + assert not store.is_cached('corners') + store.get('corners') + assert store.n_evaluations == 2 + + +def test_an_image_change_invalidates_every_score(speckle): + store = ScoreStore(speckle, 11) + store.define('corners', 'shi_tomasi') + store.get('corners') + store.set_image(speckle * 2) + assert not store.is_cached('corners') + + +def test_list_and_tuple_parameters_share_a_cache_entry(speckle): + store = ScoreStore(speckle, 11) + store.define('a', 'gradient_direction', direction=[0, 1]) + store.define('b', 'gradient_direction', direction=(0, 1)) + store.get('a') + store.get('b') + assert store.n_evaluations == 1 + + +# --------------------------------------------------------------------------- +# masks -- rasterisation and roles +# --------------------------------------------------------------------------- + +def test_polygon_rasterises_to_its_interior(): + entry = Entry('polygon', rect(10, 20, 40, 60)) + mask = rasterize(entry, (80, 100)) + assert mask[25, 40] + assert not mask[5, 40] + assert not mask[25, 70] + + +def test_brush_geometry_is_used_as_the_mask(): + painted = np.zeros((50, 50), dtype=bool) + painted[10:20, 10:20] = True + entry = Entry('brush', painted) + np.testing.assert_array_equal(rasterize(entry, (50, 50)), painted) + + +def test_a_brush_mask_of_the_wrong_shape_is_rejected(): + entry = Entry('brush', np.zeros((10, 10), dtype=bool)) + with pytest.raises(ValueError, match='expected'): + rasterize(entry, (50, 50)) + + +def test_default_roles_follow_the_kind(): + assert Entry('polygon', rect(0, 0, 5, 5)).role == 'mask' + assert Entry('brush', np.zeros((4, 4), bool)).role == 'mask' + assert Entry('polyline', [(0, 0), (5, 5)]).role == 'points' + assert Entry('points', [(1, 1)]).role == 'points' + assert DEFAULT_ROLE['polygon'] == 'mask' + + +def test_an_unknown_kind_is_rejected(): + with pytest.raises(ValueError, match='Unknown entry kind'): + Entry('rhombus', []) + + +def test_an_unknown_role_is_rejected(): + with pytest.raises(ValueError, match='role must be'): + Entry('polygon', rect(0, 0, 5, 5), role='sometimes') + + +def test_the_combined_mask_is_the_union_of_visible_mask_entries(): + a = Entry('polygon', rect(10, 10, 30, 30)) + b = Entry('polygon', rect(20, 20, 40, 40)) + mask = combined_mask([a, b], (60, 60)) + assert mask[15, 15] and mask[35, 35] and mask[25, 25] + + +def test_a_hidden_entry_contributes_nothing(): + a = Entry('polygon', rect(10, 10, 30, 30)) + b = Entry('polygon', rect(40, 40, 55, 55), visible=False) + mask = combined_mask([a, b], (60, 60)) + assert mask[15, 15] + assert not mask[45, 45] + + +def test_a_points_role_entry_contributes_no_mask(): + entry = Entry('polygon', rect(10, 10, 30, 30), role='points') + assert not combined_mask([entry], (60, 60)).any() + + +def test_a_mask_role_entry_contributes_no_literal_points(): + entry = Entry('points', [(5, 5)], role='mask') + assert all_literal_points([entry], 11) == [] + + +def test_changing_a_role_moves_the_contribution(): + entry = Entry('polygon', rect(10, 10, 40, 40)) + assert combined_mask([entry], (60, 60)).any() + assert all_literal_points([entry], 11) == [] + + entry.role = 'points' + assert not combined_mask([entry], (60, 60)).any() + assert len(all_literal_points([entry], 11)) > 0 + + +def test_literal_points_honour_the_removed_set(): + entry = Entry('points', [(5, 5), (9, 9)]) + entry.removed.add((5, 5)) + assert literal_points(entry, 11) == [(9, 9)] + + +# --------------------------------------------------------------------------- +# masks -- deselection +# --------------------------------------------------------------------------- + +def stroke_over(shape, r0, c0, r1, c1): + """A rectangular deselect stroke.""" + painted = np.zeros(shape, dtype=bool) + painted[r0:r1, c0:c1] = True + return painted + + +def test_partial_deselection_keeps_the_remainder(): + shape = (60, 60) + entry = Entry('polygon', rect(10, 10, 50, 50)) + apply_deselection([entry], stroke_over(shape, 10, 10, 30, 60), shape) + mask = rasterize(entry, shape) + assert not mask[20, 20] + assert mask[40, 20] + + +def test_deselection_survives_a_subset_size_change(): + shape = (60, 60) + entry = Entry('polygon', rect(10, 10, 50, 50)) + apply_deselection([entry], stroke_over(shape, 10, 10, 30, 60), shape) + # The stroke was recorded on the entry, not baked into a derived point list, + # so nothing about it depends on the subset size it was painted at. + assert not rasterize(entry, shape)[20, 20] + + +def test_deselection_leaves_the_original_geometry_intact(): + shape = (60, 60) + entry = Entry('polygon', rect(10, 10, 50, 50)) + original = list(entry.geometry) + apply_deselection([entry], stroke_over(shape, 10, 10, 30, 60), shape) + assert entry.geometry == original + + +def test_a_fully_covered_entry_is_reported_as_emptied(): + shape = (60, 60) + entry = Entry('polygon', rect(10, 10, 50, 50)) + emptied = apply_deselection([entry], stroke_over(shape, 0, 0, 60, 60), shape) + assert emptied == [entry] + + +def test_deselection_drops_covered_literal_points(): + shape = (60, 60) + entry = Entry('points', [(15, 15), (45, 45)]) + apply_deselection([entry], stroke_over(shape, 10, 10, 20, 20), shape) + assert entry.geometry == [(45, 45)] + + +# --------------------------------------------------------------------------- +# select -- threshold +# --------------------------------------------------------------------------- + +def test_percentile_threshold_keeps_the_top_decile(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + limit = threshold_value(score, None, 'percentile', 90) + points = select_peaks(score, separation=1, threshold=90, + threshold_mode='percentile', max_points=None) + assert points + assert all(score[r, c] > limit for r, c in points) + + +def test_the_fraction_of_the_maximum_rule_is_gone(speckle): + """It was `quality` with a reference one dust mote could move.""" + score = evaluate(speckle, 'shi_tomasi', 11) + assert 'fraction' not in THRESHOLD_MODES + with pytest.raises(ValueError, match='fraction'): + threshold_value(score, None, 'fraction', 0.5) + + +def test_an_unreachable_threshold_returns_an_empty_array(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + points = select(score, selector='peaks', threshold=100, threshold_mode='percentile') + assert points.shape == (0, 2) + assert points.dtype.kind == 'i' + + +def test_an_unknown_threshold_mode_is_rejected(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + with pytest.raises(ValueError, match='mode must be'): + threshold_value(score, None, 'quantile', 0.9) + + +# --------------------------------------------------------------------------- +# select -- a mask is worked inside its own bounding box +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize('separation', [1, 2, 5, 9, 14]) +@pytest.mark.parametrize('box', [(30, 90, 40, 100), (0, 200, 0, 200), (7, 23, 131, 149)]) +def test_a_masked_selection_is_worked_in_the_masks_bounding_box(monkeypatch, separation, box): + """The crop is an optimisation, so it has to give the identical answer. + + Cropping to the bounding box is only sound if the block grid the reduction + uses lands where it would have on the whole frame; an offset grid answers a + slightly different question and quietly returns different points. + """ + # By name, because ``pyidi.selection.select`` is the function: the package + # exports it under the same name as the module it lives in. + select_module = sys.modules['pyidi.selection.select'] + + rng = np.random.default_rng(7) + score = rng.random((200, 200)) * 10 + score[:3] = score[-3:] = score[:, :3] = score[:, -3:] = np.nan + r0, r1, c0, c1 = box + mask = np.zeros(score.shape, dtype=bool) + mask[r0:r1, c0:c1] = True + + cropped = select_peaks(score, mask=mask, separation=separation, threshold=0.2) + + monkeypatch.setattr(select_module, '_mask_window', + lambda mask, cell: (slice(0, mask.shape[0]), slice(0, mask.shape[1]))) + whole = select_peaks(score, mask=mask, separation=separation, threshold=0.2) + + assert cropped == whole + + +def test_an_empty_mask_selects_nothing(): + score = np.random.default_rng(0).random((60, 60)) + assert select_peaks(score, mask=np.zeros((60, 60), dtype=bool), separation=4) == [] + + +def test_a_hand_picked_point_outside_the_mask_still_keeps_its_distance(): + """The crop must not lose the area blocked by a point beyond its edge.""" + score = np.ones((60, 60)) + mask = np.zeros((60, 60), dtype=bool) + mask[30:50, 30:50] = True + blocked = occupancy([(30, 30)], score.shape, radius=8) + + points = select_peaks(score, mask=mask, separation=2, threshold=0, + max_points=None, occupied=blocked) + + assert points, 'the whole region cannot have been blocked' + distance = np.hypot(*(np.asarray(points, dtype=float) - (30, 30)).T) + assert distance.min() > 8 + + +# --------------------------------------------------------------------------- +# select -- suppression +# --------------------------------------------------------------------------- + +def pairwise_separation(points): + """Smallest distance between any two points, or ``inf`` for fewer than two.""" + points = np.asarray(points, dtype=float) + if len(points) < 2: + return np.inf + diff = points[:, None, :] - points[None, :, :] + distance = np.hypot(diff[..., 0], diff[..., 1]) + np.fill_diagonal(distance, np.inf) + return distance.min() + + +def test_the_separation_is_respected(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + points = select_peaks(score, separation=10, threshold=50, threshold_mode='percentile') + assert len(points) > 5 + assert pairwise_separation(points) >= 10 + + +def test_a_separation_of_one_keeps_every_candidate(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + limit = threshold_value(score, None, 'percentile', 99) + points = select_peaks(score, separation=1, threshold=99, + threshold_mode='percentile', max_points=None) + assert len(points) == int((score > limit).sum()) + + +def test_the_strongest_candidate_in_a_neighbourhood_wins(): + score = np.zeros((40, 40), dtype=np.float32) + score[20, 20] = 5.0 + score[20, 23] = 9.0 # closer than the separation, and stronger + points = select_peaks(score, separation=8, threshold=0, max_points=None) + assert (20, 23) in points + assert (20, 20) not in points + + +def test_a_dense_blob_yields_one_point(): + score = np.zeros((60, 60), dtype=np.float32) + rows, cols = np.mgrid[0:60, 0:60] + score += np.exp(-((rows - 30.0) ** 2 + (cols - 30.0) ** 2) / 40.0).astype(np.float32) + points = select_peaks(score, separation=5, threshold=99, + threshold_mode='percentile', max_points=None) + near = [p for p in points if abs(p[0] - 30) <= 5 and abs(p[1] - 30) <= 5] + assert len(near) == 1 + + +def test_the_point_cap_is_applied(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + many = select_peaks(score, separation=3, threshold=50, + threshold_mode='percentile', max_points=None) + capped = select_peaks(score, separation=3, threshold=50, + threshold_mode='percentile', max_points=5) + assert len(many) > 5 + assert len(capped) == 5 + assert capped == many[:5] + + +def test_results_are_deterministic(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + first = select(score, selector='peaks', separation=7, threshold=80, threshold_mode='percentile') + second = select(score, selector='peaks', separation=7, threshold=80, threshold_mode='percentile') + np.testing.assert_array_equal(first, second) + + +def test_ties_are_broken_by_position(): + score = np.zeros((40, 40), dtype=np.float32) + score[10, 20] = 1.0 + score[12, 15] = 1.0 # equal score, larger row -- must lose + points = select_peaks(score, separation=9, threshold=0.5, + threshold_mode='quality', max_points=None) + assert points == [(10, 20)] + + +def test_selected_points_stay_inside_the_mask(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + mask = np.zeros(speckle.shape, dtype=bool) + mask[30:90, 30:90] = True + points = select_peaks(score, mask=mask, separation=6, threshold=50, + threshold_mode='percentile') + assert points + assert all(mask[r, c] for r, c in points) + + +def test_the_nan_border_is_never_selected(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + points = select_peaks(score, separation=4, threshold=10, threshold_mode='percentile') + assert all(np.isfinite(score[r, c]) for r, c in points) + + +def test_selection_stays_interactive_on_a_dense_score_image(): + """A threshold drag re-selects on every step, so the whole thing is the budget. + + A loose threshold on a megapixel frame leaves ~10^5 pixels above it. Walking + them all is 40 ms to 300 ms depending on the separation, which is why the + candidates are reduced to the best in each cell first. + """ + import time + + rng = np.random.default_rng(3) + big = rng.integers(0, 255, (1000, 1000)).astype(np.uint16) + score = evaluate(big, 'shi_tomasi', 11) + assert (score > threshold_value(score, None, 'percentile', 90)).sum() > 50000 + + start = time.perf_counter() + points = select_peaks(score, separation=10, threshold=90, threshold_mode='percentile', + max_points=None) + elapsed = time.perf_counter() - start + assert len(points) > 100 + assert elapsed < 0.1, f'selection took {elapsed * 1000:.0f} ms' + + +def test_the_candidate_reduction_does_not_break_the_separation(): + """The cell grid is an approximation of the walk's input, never of its rule.""" + from pyidi.selection.select import CANDIDATE_CELL_FRACTION + + rng = np.random.default_rng(5) + score = rng.random((300, 400)).astype(np.float32) + for separation in (2, 4, 11, 30): + points = np.array(select_peaks(score, separation=separation, threshold=0.5, + threshold_mode='quality', max_points=None)) + assert len(points) > 10 + assert pairwise_separation(points) >= separation + # ...and it really did reduce, wherever the cell is worth having + if separation // CANDIDATE_CELL_FRACTION > 1: + assert len(points) < (score > threshold_value(score, None, 'quality', 0.5)).sum() + + +def test_the_cell_grid_keeps_the_best_pixel_in_each_cell(): + from pyidi.selection.select import _block_best + + score = np.zeros((12, 12), dtype=np.float32) + score[1, 1] = 1.0 + score[2, 2] = 5.0 # same 4x4 cell, stronger -- it should win + score[9, 5] = 3.0 + rows, cols = _block_best(score, score > 0, 4) + assert set(zip(rows.tolist(), cols.tolist())) == {(2, 2), (9, 5)} + + +def test_a_cell_with_nothing_eligible_contributes_nothing(): + """Unlike a lattice, which puts a point wherever the grid happens to fall.""" + from pyidi.selection.select import _block_best + + score = np.zeros((20, 20), dtype=np.float32) + score[3, 3] = 1.0 + rows, _ = _block_best(score, score > 0, 5) + assert len(rows) == 1 + + +def nearest_neighbour(points): + """Distance from each point to its closest other point.""" + points = np.asarray(points, dtype=float) + diff = points[:, None, :] - points[None, :, :] + distance = np.hypot(diff[..., 0], diff[..., 1]) + np.fill_diagonal(distance, np.inf) + return distance.min(axis=1) + + +def test_keeping_every_nth_is_not_a_substitute_for_the_separation(): + """The measurement the separation control exists because of. + + Thinning the pixels above the threshold by keeping every n-th of them, in + score order, is the obvious thing to reach for and it does not work: + consecutive ranks are neighbours on the same feature, so most of what + survives is still back-to-back with something else. On this noise field it + leaves two fifths of the subsets within three pixels of another, and on a + structured frame -- where the strong scores really are concentrated on a few + features -- three quarters. The separation leaves none. + """ + rng = np.random.default_rng(7) + image = rng.integers(0, 255, (300, 300)).astype(np.uint16) + score = evaluate(image, 'shi_tomasi', 11) + rows, cols = np.nonzero(score > threshold_value(score, None, 'quality', 0.05)) + order = np.argsort(-score[rows, cols], kind='stable') + stride = max(2, rows.size // 2000) + + ranked = np.column_stack([rows[order][::stride], cols[order][::stride]]) + spaced = np.array(select_peaks(score, separation=6, threshold=0.05, + threshold_mode='quality', max_points=None)) + + assert (nearest_neighbour(ranked) < 3).mean() > 0.4 + assert (nearest_neighbour(spaced) < 3).mean() == 0 + assert pairwise_separation(spaced) >= 6 + + +def test_a_zero_percentile_threshold_keeps_everything(): + """A slider at its loosest setting must not select nothing.""" + score = np.ones((40, 40), dtype=np.float32) + assert threshold_value(score, None, 'percentile', 0) == -np.inf + assert len(select_peaks(score, separation=1, threshold=0, max_points=None)) == 1600 + + +# --------------------------------------------------------------------------- +# select -- lattice and decimation +# --------------------------------------------------------------------------- + +def test_lattice_places_points_on_a_regular_grid(): + score = np.ones((60, 60), dtype=np.float32) + points = select_lattice(score, pitch=12, threshold=0, max_points=None) + rows = sorted({r for r, _ in points}) + cols = sorted({c for _, c in points}) + assert rows == [0, 12, 24, 36, 48] + assert cols == [0, 12, 24, 36, 48] + + +def test_lattice_honours_the_threshold(): + score = np.ones((60, 60), dtype=np.float32) + score[24, :] = 0.0 + points = select_lattice(score, pitch=12, threshold=0.5, threshold_mode='quality', + max_points=None) + assert not any(r == 24 for r, _ in points) + assert any(r == 12 for r, _ in points) + + +def test_unknown_selector_is_rejected(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + with pytest.raises(ValueError, match='Unknown selector'): + select(score, selector='vibes') + + +def test_selector_ignores_parameters_it_does_not_take(speckle): + """One set of defaults has to be usable with either selector.""" + score = evaluate(speckle, 'shi_tomasi', 11) + points = select(score, selector='lattice', pitch=10, separation=99, threshold=0) + assert len(points) > 0 + + +def test_decimation_by_stride(): + points = [(i, i) for i in range(100)] + assert decimate(points, stride=4) == points[::4] + + +def test_decimation_to_a_target_count(): + points = [(i, i) for i in range(100)] + thinned = decimate(points, count=30) + assert len(thinned) <= 30 + assert thinned[0] == points[0] + assert thinned[-1] == points[-1] + + +def test_decimation_leaves_a_short_list_alone(): + points = [(1, 1), (2, 2)] + assert decimate(points, count=10) == points + + +# --------------------------------------------------------------------------- +# select -- merging with hand-picked points +# --------------------------------------------------------------------------- + +def test_literal_points_survive_a_low_score(): + score = np.zeros((60, 60), dtype=np.float32) + score[40, 40] = 10.0 + literal = [(10, 10)] + picked = select_peaks(score, separation=5, threshold=99, threshold_mode='percentile', max_points=None, + occupied=occupancy(literal, score.shape, 5)) + merged = merge_points(literal, picked) + assert (merged == np.array([10, 10])).all(axis=1).any() + + +def test_selection_never_crowds_a_literal_point(): + score = np.ones((60, 60), dtype=np.float32) + literal = [(30, 30)] + picked = select_peaks(score, separation=8, threshold=0, max_points=None, + occupied=occupancy(literal, score.shape, 8)) + assert all(np.hypot(r - 30, c - 30) >= 8 for r, c in picked) + + +def test_a_coincident_point_appears_once(): + merged = merge_points([(5, 5)], [(5, 5), (9, 9)]) + assert merged.shape == (2, 2) + + +def test_as_point_array_of_nothing_is_shaped_for_indexing(): + empty = as_point_array([]) + assert empty.shape == (0, 2) + assert empty[:, 0].size == 0 + + +# --------------------------------------------------------------------------- +# pipeline -- end to end +# --------------------------------------------------------------------------- + +def test_end_to_end_returns_points_inside_the_polygon(image): + entry = Entry('polygon', rect(100, 100, 150, 190)) + points = select_points(image, [entry], subset_size=11, separation=8) + assert len(points) > 5 + assert points.dtype.kind == 'i' + assert points.shape[1] == 2 + assert ((points[:, 0] >= 100) & (points[:, 0] <= 150)).all() + assert ((points[:, 1] >= 100) & (points[:, 1] <= 190)).all() + + +def test_no_mask_entries_gives_no_points(image): + assert select_points(image, []).shape == (0, 2) + + +def test_a_points_role_entry_needs_no_score(image): + entry = Entry('points', [(50, 50), (60, 60)]) + points = select_points(image, [entry]) + np.testing.assert_array_equal(points, np.array([[50, 50], [60, 60]])) + + +def test_mask_edits_do_not_re_evaluate(image): + pipeline = SelectionPipeline(image, subset_size=11) + entry = pipeline.add_entry('polygon', rect(100, 100, 150, 190)) + pipeline.get_points() + evaluations = pipeline.store.n_evaluations + assert evaluations == 1 + + entry.geometry = rect(105, 105, 145, 185) + pipeline.selector_params.update({'threshold': 95, 'threshold_mode': 'percentile'}) + pipeline.get_points() + assert pipeline.store.n_evaluations == evaluations + + +def test_a_subset_size_change_does_re_evaluate(image): + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('polygon', rect(100, 100, 150, 190)) + pipeline.get_points() + pipeline.set_subset_size(21) + pipeline.get_points() + assert pipeline.store.n_evaluations == 2 + + +def test_hiding_and_unhiding_restores_the_points_without_re_evaluating(image): + pipeline = SelectionPipeline(image, subset_size=11) + entry = pipeline.add_entry('polygon', rect(100, 100, 150, 190)) + before = pipeline.get_points() + evaluations = pipeline.store.n_evaluations + + entry.visible = False + assert pipeline.get_points().shape == (0, 2) + + entry.visible = True + np.testing.assert_array_equal(pipeline.get_points(), before) + assert pipeline.store.n_evaluations == evaluations + + +def test_per_entry_settings_are_honoured(image): + pipeline = SelectionPipeline(image, subset_size=11) + loose = pipeline.add_entry('polygon', rect(100, 100, 125, 190)) + tight = pipeline.add_entry('polygon', rect(126, 100, 150, 190)) + loose.selector_params = {'separation': 4, 'threshold': 50, 'threshold_mode': 'percentile'} + tight.selector_params = {'separation': 20, 'threshold': 50, 'threshold_mode': 'percentile'} + + points = pipeline.get_points() + in_loose = points[points[:, 0] <= 125] + in_tight = points[points[:, 0] >= 126] + assert len(in_loose) > len(in_tight) + assert pairwise_separation(in_tight) >= 20 + + +def test_uniform_per_entry_settings_equal_global_settings(image): + shared = {'separation': 9, 'threshold': 70, 'threshold_mode': 'percentile'} + + globally = SelectionPipeline(image, subset_size=11) + globally.add_entry('polygon', rect(100, 100, 150, 190)) + globally.selector_params.update(shared) + + per_entry = SelectionPipeline(image, subset_size=11) + entry = per_entry.add_entry('polygon', rect(100, 100, 150, 190)) + entry.selector_params = dict(shared) + + np.testing.assert_array_equal(globally.get_points(), per_entry.get_points()) + + +def test_entries_sharing_settings_compete_for_the_same_separation(image): + """Two adjacent regions filtered alike must not place points on their shared edge.""" + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('polygon', rect(100, 100, 124, 190)) + pipeline.add_entry('polygon', rect(125, 100, 150, 190)) + pipeline.selector_params.update({'separation': 12, 'threshold': 40, + 'threshold_mode': 'percentile'}) + points = pipeline.get_points() + assert pairwise_separation(points) >= 12 + + +def test_literal_and_selected_points_combine(image): + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('polygon', rect(100, 100, 150, 190)) + pipeline.add_entry('points', [(60, 60)]) + pipeline.selector_params.update({'separation': 8, 'threshold': 60, + 'threshold_mode': 'percentile'}) + points = pipeline.get_points() + assert (points == np.array([60, 60])).all(axis=1).any() + assert len(points) > 1 + + +def test_labels_are_never_reused(image): + pipeline = SelectionPipeline(image) + first = pipeline.add_entry('polygon', rect(10, 10, 20, 20)) + second = pipeline.add_entry('polygon', rect(30, 30, 40, 40)) + assert (first.label, second.label) == ('Polygon 1', 'Polygon 2') + pipeline.remove_entry(second) + third = pipeline.add_entry('polygon', rect(50, 50, 60, 60)) + assert third.label == 'Polygon 3' + + +def test_deselecting_through_the_pipeline_drops_emptied_entries(image): + pipeline = SelectionPipeline(image) + entry = pipeline.add_entry('polygon', rect(100, 100, 150, 190)) + stroke = np.ones(image.shape, dtype=bool) + emptied = pipeline.deselect(stroke) + assert emptied == [entry] + assert pipeline.entries == [] + + +def test_points_property_matches_get_points(image): + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('polygon', rect(100, 100, 150, 190)) + np.testing.assert_array_equal(pipeline.points, pipeline.get_points()) + + +def test_output_is_accepted_by_a_method_class(image, tmp_path): + import warnings + + from pyidi import SimplifiedOpticalFlow, VideoReader + + entry = Entry('polygon', rect(100, 100, 150, 190)) + points = select_points(image, [entry], subset_size=11, separation=10) + assert len(points) + + video = VideoReader(np.stack([image, image]).astype(np.uint16), root=str(tmp_path)) + method = SimplifiedOpticalFlow(video) + with warnings.catch_warnings(): + warnings.simplefilter('error') + method.set_points(points) + np.testing.assert_array_equal(method.points, points) + + +# --------------------------------------------------------------------------- +# select -- the quality threshold +# +# The rule the interface defaults to, and the reason it does. A percentile +# ranks *pixels*, and on a dense score image the pixels are overwhelmingly +# background, so most of a percentile slider's travel is spent inside the +# featureless area. Quality is measured against the best feature instead. +# --------------------------------------------------------------------------- + +def flat_with_corners(): + """A frame like a real one: mostly blank, a few strong features, sensor noise.""" + img = np.full((200, 300), 240, dtype=np.uint8) + corners = np.zeros(img.shape, dtype=bool) + for row in range(40, 180, 60): + for col in range(40, 280, 60): + img[row:row + 20, col:col + 20] = 20 + corners[row - 9:row + 29, col - 9:col + 29] = True + noise = np.random.default_rng(3).integers(-5, 6, img.shape) + return np.clip(img.astype(int) + noise, 0, 255).astype(np.uint8), corners + + +def test_quality_is_a_fraction_of_the_robust_maximum(speckle): + score = evaluate(speckle, 'shi_tomasi', 11) + robust = np.nanpercentile(score, ROBUST_MAXIMUM_PERCENTILE) + assert threshold_value(score, None, 'quality', 0.25) == pytest.approx(0.25 * robust) + + +def test_a_lone_outlier_barely_moves_the_quality_scale(): + """A specular highlight must not drag every useful setting into the slider's floor. + + This is the whole difference between `quality` and taking a fraction of the + literal maximum, which is why the latter is not offered: on this score image + it moves by a factor of fifty where quality moves by 2%. + """ + score = np.random.default_rng(0).random((200, 300)) + before = threshold_value(score, None, 'quality', 0.1) + literal_before = 0.1 * score.max() + + score[100, 150] = 500.0 # one absurdly bright pixel + assert threshold_value(score, None, 'quality', 0.1) == pytest.approx(before, rel=0.02) + assert 0.1 * score.max() > 50 * literal_before + + +def test_a_zero_quality_keeps_everything(): + score = np.zeros((40, 40)) + assert threshold_value(score, None, 'quality', 0) == -np.inf + + +def test_quality_keeps_the_points_off_the_blank_background(): + """The headline behaviour: the whole slider stays inside the useful range.""" + image, corners = flat_with_corners() + score = evaluate(image, 'shi_tomasi', 11) + + for quality in (0.5, 0.1, 0.01): + points = np.array(select_peaks(score, separation=8, threshold=quality, + threshold_mode='quality', max_points=None)) + assert len(points) + assert corners[points[:, 0], points[:, 1]].all(), quality + + +def test_a_percentile_threshold_does_not(): + """Why the default changed: the same frame, ranked by pixel instead.""" + image, corners = flat_with_corners() + score = evaluate(image, 'shi_tomasi', 11) + + points = np.array(select_peaks(score, separation=8, threshold=50, + threshold_mode='percentile', max_points=None)) + assert corners[points[:, 0], points[:, 1]].mean() < 0.9 + + +# --------------------------------------------------------------------------- +# select -- candidate ordering and suppression, at speed +# --------------------------------------------------------------------------- + +def test_partitioning_the_candidates_matches_the_full_sort(): + """The `keep` shortcut must be an optimisation, not an approximation.""" + from pyidi.selection.select import _ordered_candidates + + rng = np.random.default_rng(5) + for score in (rng.random((120, 160)), rng.integers(0, 4, (120, 160)).astype(float)): + eligible = score > np.percentile(score, 20) + full_rows, full_cols = _ordered_candidates(score, eligible) + for keep in (1, 9, 400, 5000): + rows, cols = _ordered_candidates(score, eligible, keep) + np.testing.assert_array_equal(rows[:keep], full_rows[:keep]) + np.testing.assert_array_equal(cols[:keep], full_cols[:keep]) + + +def test_chunked_suppression_matches_a_plain_walk(): + """Batching the occupancy test is exact because occupancy only ever grows.""" + from pyidi.selection.select import _disc, _ordered_candidates, suppress + + shape = (90, 130) + score = np.random.default_rng(6).random(shape) + rows, cols = _ordered_candidates(score, score > np.percentile(score, 30)) + + for radius in (0, 1, 4, 11): + for max_points in (None, 5, 10000): + taken = np.zeros(shape, dtype=bool) + expected = [] + for row, col in zip(rows.tolist(), cols.tolist()): + if taken[row, col]: + continue + expected.append((row, col)) + if max_points is not None and len(expected) >= max_points: + break + radius = int(radius) + if radius == 0: + taken[row, col] = True + continue + disc = _disc(radius) + r0, r1 = max(0, row - radius), min(shape[0], row + radius + 1) + c0, c1 = max(0, col - radius), min(shape[1], col + radius + 1) + taken[r0:r1, c0:c1] |= disc[r0 - row + radius:r1 - row + radius, + c0 - col + radius:c1 - col + radius] + assert suppress(rows, cols, shape, radius, max_points) == expected + + +# --------------------------------------------------------------------------- +# select -- decimation +# --------------------------------------------------------------------------- + +def test_decimation_thins_the_points_without_moving_them(speckle): + """The distinction from a wider minimum distance, which re-selects instead.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.add_entry('brush', np.ones(speckle.shape, dtype=bool)) + pipeline.selector_params.update({'separation': 6, 'threshold': 0.05}) + + base = {tuple(point) for point in pipeline.points.tolist()} + assert len(base) > 20 + + for stride in (2, 3, 7): + pipeline.selector_params['decimation'] = stride + thinned = [tuple(point) for point in pipeline.points.tolist()] + assert set(thinned) <= base, stride + assert len(thinned) == pytest.approx(len(base) / stride, rel=0.15) + + +def test_a_wider_separation_moves_the_points_instead(speckle): + """The contrast decimation exists for.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.add_entry('brush', np.ones(speckle.shape, dtype=bool)) + pipeline.selector_params.update({'separation': 6, 'threshold': 0.05}) + base = {tuple(point) for point in pipeline.points.tolist()} + + pipeline.selector_params['separation'] = 18 + respaced = {tuple(point) for point in pipeline.points.tolist()} + assert not respaced <= base + + +def test_decimation_leaves_hand_placed_points_alone(): + """They were placed deliberately; thinning is for what the selector found.""" + image, _ = flat_with_corners() + pipeline = SelectionPipeline(image, subset_size=11) + pipeline.add_entry('brush', np.ones(image.shape, dtype=bool)) + pipeline.add_entry('points', [(100, 150), (100, 170), (100, 190)]) + pipeline.selector_params['decimation'] = 5 + + points = {tuple(point) for point in pipeline.points.tolist()} + assert {(100, 150), (100, 170), (100, 190)} <= points + + +def test_decimating_one_group_does_not_let_another_fill_the_gaps(speckle): + """What was selected is stamped before it is thinned, so gaps stay gaps.""" + def right_hand_points(decimation): + pipeline = SelectionPipeline(speckle, subset_size=11) + left = pipeline.add_entry('polygon', rect(10, 10, 90, 58)) + right = pipeline.add_entry('polygon', rect(10, 60, 90, 110)) + # Different minimum distances, so the two are always separate groups: + # entries that share their settings are deliberately selected together. + left.selector_params = {'separation': 5, 'threshold': 0.05, 'decimation': decimation} + right.selector_params = {'separation': 6, 'threshold': 0.05} + return pipeline.points_and_credits()[1][1] + + undecimated = right_hand_points(1) + assert len(undecimated) > 5 + np.testing.assert_array_equal(right_hand_points(4), undecimated) + + +def test_a_later_group_keeps_clear_of_an_earlier_ones_points(speckle): + """The stamp is skipped when no group follows, so prove one that does reads it.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + first = pipeline.add_entry('polygon', rect(10, 10, 90, 80)) + second = pipeline.add_entry('polygon', rect(10, 40, 90, 110)) # overlapping + first.selector_params = {'separation': 9, 'threshold': 0.05} + second.selector_params = {'separation': 4, 'threshold': 0.05} + + credited = pipeline.points_and_credits()[1] + mine, theirs = np.asarray(credited[0], dtype=float), np.asarray(credited[1], dtype=float) + assert len(mine) and len(theirs) + + gap = np.hypot(mine[:, None, 0] - theirs[None, :, 0], mine[:, None, 1] - theirs[None, :, 1]) + assert gap.min() > 9 + + +# --------------------------------------------------------------------------- +# One pass for points and per-entry credits +# --------------------------------------------------------------------------- + +def test_points_and_credits_agree_with_asking_separately(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.add_entry('polygon', rect(10, 10, 90, 110)) + pipeline.add_entry('points', [(50, 50)]) + + points, credited = pipeline.points_and_credits() + np.testing.assert_array_equal(points, pipeline.get_points()) + for mine, theirs in zip(credited, pipeline.points_by_entry()): + np.testing.assert_array_equal(mine, theirs) + + +# --------------------------------------------------------------------------- +# Candidates -- what the mask is leaving out +# --------------------------------------------------------------------------- + +def test_candidates_ignore_the_mask_entirely(speckle): + """They answer "what is there", which is the question a mask cannot.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.add_entry('polygon', rect(5, 5, 25, 25)) + candidates = pipeline.candidate_points() + assert len(candidates) > len(pipeline.points) + assert not pipeline.mask[candidates[:, 0], candidates[:, 1]].all() + + +def test_candidates_survive_a_mask_edit(speckle): + """Cached across mask changes, so painting a region re-selects nothing.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + before = pipeline.candidate_points() + pipeline.add_entry('polygon', rect(5, 5, 25, 25)) + assert pipeline.candidate_points() is before + + +def test_candidates_are_recomputed_when_the_score_changes(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.define_score('score', 'shi_tomasi') + before = pipeline.candidate_points() + pipeline.define_score('score', 'gradient_direction', direction=(0, 1)) + assert pipeline.candidate_points() is not before + + +def test_candidates_are_recomputed_at_a_new_subset_size(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + before = pipeline.candidate_points() + pipeline.set_subset_size(21) + assert pipeline.candidate_points() is not before + + +def test_candidates_are_recomputed_when_a_selector_setting_changes(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + before = pipeline.candidate_points() + pipeline.selector_params['separation'] = 3 + after = pipeline.candidate_points() + assert after is not before + assert len(after) > len(before) + + +# --------------------------------------------------------------------------- +# Removing a single point +# +# A selected point is not stored anywhere: it is re-derived from the score +# every time the pipeline runs. So "remove this one" cannot be a deletion -- +# it has to be an edit to the mask that the next selection will respect. +# --------------------------------------------------------------------------- + +def test_removing_a_point_leaves_no_replacement_beside_it(speckle): + """Erasing the pixel alone is not enough. + + The reduction picks the best pixel of each block, so taking the winner away + promotes its neighbour and the point comes back one or two pixels along -- + which reads as the click having nudged the point rather than removed it. + What is erased is the whole disc the point was reserving, so nothing can + land nearer to it than a neighbouring point legitimately could have. + """ + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('polygon', rect(10, 10, 118, 118)) + separation = pipeline.selector_params['separation'] + + target = tuple(int(v) for v in pipeline.points[0]) + pipeline.remove_point(entry, target) + + survivors = pipeline.points + assert not any(tuple(p) == target for p in survivors) + gaps = np.hypot(survivors[:, 0] - target[0], survivors[:, 1] - target[1]) + assert gaps.min() >= separation + + +def test_removing_a_point_works_more_than_once(speckle): + """The regression: the second click and every one after it did nothing. + + ``erased`` was grown with an in-place write, and the rasterisation cache + identifies that array by object -- so after the first click, which allocates + it, no later one changed anything the cache could see. + """ + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('polygon', rect(10, 10, 118, 118)) + + for _ in range(5): + target = tuple(int(v) for v in pipeline.points[0]) + pipeline.remove_point(entry, target) + assert not any(tuple(p) == target for p in pipeline.points) + + +def test_removing_a_hand_picked_point_deletes_it(speckle): + """Outright, so that clicking the same pixel again puts one back.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('points', [(30, 30), (60, 60)]) + pipeline.remove_point(entry, (30, 30)) + assert entry.geometry == [(60, 60)] + assert entry.erased is None + + +def test_removing_a_point_from_a_polyline_records_it(speckle): + """A polyline re-derives its points too, so the coordinate has to be kept.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('polyline', [(20, 20), (20, 110)]) + target = literal_points(entry, pipeline.subset_size)[1] + pipeline.remove_point(entry, target) + assert target in entry.removed + assert target not in literal_points(entry, pipeline.subset_size) + + +def test_removing_a_point_replaces_the_erased_array(speckle): + """Never a write into it: see the contract on ``Entry.erased``.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('polygon', rect(10, 10, 118, 118)) + pipeline.remove_point(entry, tuple(int(v) for v in pipeline.points[0])) + first = entry.erased + pipeline.remove_point(entry, tuple(int(v) for v in pipeline.points[0])) + assert entry.erased is not first + + +# --------------------------------------------------------------------------- +# What a deselect stroke costs +# +# An `erased` array covers the frame; a stroke covers a few hundred pixels of +# it. Handing one to every region on the list would make a single dab cost a +# megabyte per region -- and cost it again in every undo snapshot. +# --------------------------------------------------------------------------- + +def test_a_stroke_only_reaches_the_entries_it_covers(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + covered = pipeline.add_entry('polygon', rect(10, 10, 50, 50)) + missed = pipeline.add_entry('polygon', rect(80, 80, 120, 120)) + + stroke = np.zeros(pipeline.shape, dtype=bool) + stroke[20:25, 20:25] = True + pipeline.deselect(stroke) + + assert covered.erased is not None + assert missed.erased is None + + +def test_a_stroke_that_misses_everything_changes_nothing(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('polygon', rect(10, 10, 50, 50)) + before = pipeline.points + + stroke = np.zeros(pipeline.shape, dtype=bool) + stroke[100:105, 100:105] = True + assert pipeline.deselect(stroke) == [] + + assert entry.erased is None + np.testing.assert_array_equal(pipeline.points, before) + + +def test_deselection_still_erases_what_it_does_cover(speckle): + """The skip above must not have cost the stroke its job.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.add_entry('polygon', rect(10, 10, 118, 118)) + before = pipeline.mask.sum() + + stroke = np.zeros(pipeline.shape, dtype=bool) + stroke[40:60, 40:60] = True + pipeline.deselect(stroke) + + assert pipeline.mask.sum() == before - stroke.sum() + + +# --------------------------------------------------------------------------- +# Points that are not on the image +# +# A click lands wherever the interface lets it land, and the view is always +# larger than the frame. A subset centred off the frame is not something that +# can be tracked, and it is an index error waiting for whichever array reads it +# first. +# --------------------------------------------------------------------------- + +def test_a_hand_picked_point_off_the_frame_is_dropped(speckle): + pipeline = SelectionPipeline(speckle, subset_size=11) + pipeline.add_entry('points', [(-5, 40), (40, 40), (40, 999)]) + assert pipeline.literal_points() == [(40, 40)] + assert [tuple(p) for p in pipeline.points] == [(40, 40)] + + +def test_an_off_frame_point_is_dropped_from_its_row_too(speckle): + """Not just from the total, or the list would disagree with the canvas.""" + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('points', [(-5, 40), (40, 40)]) + credited = pipeline.points_by_entry()[pipeline.entries.index(entry)] + assert [tuple(p) for p in credited] == [(40, 40)] + + +def test_merging_survives_a_coordinate_off_the_top_of_the_frame(): + """The dedup folds a pair into one integer; the fold has to stay one-to-one.""" + merged = merge_points([(-5, 3), (-5, 3), (2, 7)], [(2, 7), (9, 1)]) + assert [tuple(p) for p in merged] == [(-5, 3), (2, 7), (9, 1)] + + +# --------------------------------------------------------------------------- +# The score cache is bounded +# +# Every distinct set of evaluator parameters is a separate full-frame float32. +# A spin box dragged through sixty values asks for sixty of them. +# --------------------------------------------------------------------------- + +def test_the_score_cache_stops_growing(speckle): + store = ScoreStore(speckle, subset_size=11, max_cached=4) + for k in range(20): + store.define('score', 'gradient_direction', direction=(1.0, k + 1.0)) + store.get('score') + assert len(store._cache) == 4 + assert store.n_evaluations == 20 + + +def test_the_cache_drops_the_least_recently_used(speckle): + store = ScoreStore(speckle, subset_size=11, max_cached=2) + for name, direction in (('a', (0.0, 1.0)), ('b', (1.0, 0.0))): + store.define(name, 'gradient_direction', direction=direction) + store.get(name) + + store.get('a') # 'b' is now the older of the two + store.define('c', 'gradient_direction', direction=(1.0, 1.0)) + store.get('c') + + assert store.is_cached('a') + assert store.is_cached('c') + assert not store.is_cached('b') + + +def test_a_repeated_request_is_still_free(speckle): + """The eviction must not have cost the cache its reason for existing.""" + store = ScoreStore(speckle, subset_size=11, max_cached=4) + store.define('score', 'shi_tomasi') + first = store.get('score') + assert store.get('score') is first + assert store.n_evaluations == 1 + + +# --------------------------------------------------------------------------- +# The point cap, when something is already taken +# --------------------------------------------------------------------------- + +def test_the_cap_is_filled_even_though_some_candidates_are_taken(speckle): + """At a separation of 1 the candidates are only sorted as far as the cap. + + That is exact when every one of them is accepted, and it was not: the + occupied positions were dropped *after* the cut, so a run with hand-picked + points returned fewer points than the cap allowed while more were eligible. + """ + score = evaluate(speckle, 'shi_tomasi', 11) + mask = np.zeros(score.shape, dtype=bool) + mask[20:110, 20:110] = True + + free = select_peaks(score, mask, separation=1, threshold=0, max_points=50) + taken = occupancy(free[:10], score.shape, 0) + blocked = select_peaks(score, mask, separation=1, threshold=0, max_points=50, + occupied=taken) + + assert len(blocked) == 50 + assert not set(blocked) & set(free[:10]) + + +# --------------------------------------------------------------------------- +# The rasterisation cache is keyed by identity +# --------------------------------------------------------------------------- + +def test_the_raster_cache_holds_the_entry_it_describes(speckle): + """An ``id`` is unique only among live objects. + + Without a reference here, a deleted entry could be collected and a new one + allocated at the same address, which would then read the dead entry's area + out of the cache. + """ + pipeline = SelectionPipeline(speckle, subset_size=11) + entry = pipeline.add_entry('polygon', rect(10, 10, 50, 50)) + pipeline.area(entry) + assert any(held is entry for held, _, _ in pipeline._raster_cache.values()) diff --git a/tests/test_set_points_validation.py b/tests/test_set_points_validation.py new file mode 100644 index 0000000..7f3f0f4 --- /dev/null +++ b/tests/test_set_points_validation.py @@ -0,0 +1,167 @@ +""" +Tests for the hardened ``IDIMethod.set_points()`` in +``pyidi/methods/idi_method.py``. + +set_points() now: + * accepts a plain array-like of (row, col) points, or any object exposing + a ``.points`` attribute (duck-typed, so both selection GUIs work without + idi_method importing either of them); + * raises ValueError for empty input, non-2D input, wrong column count, and + (when the method's video reports its size) out-of-bounds points; + * rounds non-integer points to the nearest int (not truncating) and warns + with UserWarning when it actually changes a value; + * skips the bounds check entirely when ``self.video`` doesn't expose both + ``image_width`` and ``image_height`` (see the ``getattr``/``hasattr`` + guard in the implementation). + +These tests avoid running any full displacement analysis; they only build +method instances and call ``set_points`` / ``configure``. +""" + +import types +import warnings + +import numpy as np +import pytest +import sys +import os + +my_path = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, my_path + '/../') + +import pyidi +from pyidi.methods.idi_method import IDIMethod + +DATA = os.path.join(my_path, '..', 'data', 'data_synthetic.cih') + + +@pytest.fixture +def video(): + return pyidi.VideoReader(input_file=DATA) + + +@pytest.fixture +def method(video): + # LucasKanade is a thin IDIMethod subclass; set_points() itself is + # implemented on IDIMethod and not overridden, so this exercises the + # real code path used by every method. + return pyidi.LucasKanade(video) + + +class DummySelectionGUI: + """Stand-in for a selection GUI: exposes only the `.points` contract + that set_points() duck-types against (both the tkinter and Qt subset + selection GUIs satisfy this).""" + + def __init__(self, points): + self.points = points + + +def _bare_method_without_video_info(): + """An object with a `set_points`-compatible `self.video` that lacks + `image_width`/`image_height`, so the bounds check must be skipped. + + Confirmed from the implementation: it does + ``video = getattr(self, 'video', None)`` then checks + ``hasattr(video, 'image_width') and hasattr(video, 'image_height')`` + before doing any bounds checking at all. + """ + obj = types.SimpleNamespace() + obj.video = types.SimpleNamespace() # no image_width / image_height + return obj + + +# --------------------------------------------------------------------------- +# ValueError cases +# --------------------------------------------------------------------------- + +def test_empty_points_raises_value_error(method): + with pytest.raises(ValueError, match=r"empty"): + method.set_points(np.array([])) + + +def test_non_2d_points_raises_value_error(method): + with pytest.raises(ValueError, match=r"2-dimensional"): + method.set_points(np.array([1, 2, 3])) + + +def test_wrong_column_count_raises_value_error(method): + with pytest.raises(ValueError, match=r"two columns"): + method.set_points(np.array([[1, 2, 3], [4, 5, 6]])) + + +def test_out_of_bounds_points_raise_value_error(method, video): + # image is 128 (height/rows) x 256 (width/cols); row 200 is out of bounds + with pytest.raises(ValueError, match=r"bounds"): + method.set_points(np.array([[10, 10], [200, 10]])) + + +def test_negative_points_are_out_of_bounds(method): + with pytest.raises(ValueError, match=r"bounds"): + method.set_points(np.array([[-1, 10]])) + + +# --------------------------------------------------------------------------- +# Rounding: nearest, not truncation +# --------------------------------------------------------------------------- + +def test_float_points_round_to_nearest_not_truncate(method): + """The specific regression being guarded: 1.7 must become 2, not 1. + + The old LucasKanade code truncated toward zero (int(x)); the old + SimplifiedOpticalFlow code hard-crashed on non-integer input. + """ + with pytest.warns(UserWarning): + method.set_points(np.array([[1.7, 2.2], [50.4, 60.6]])) + + assert np.issubdtype(method.points.dtype, np.integer) + np.testing.assert_array_equal(method.points, np.array([[2, 2], [50, 61]])) + + +def test_rounding_warning_fires_only_when_values_actually_change(method): + with pytest.warns(UserWarning, match=r"rounded"): + method.set_points(np.array([[1.5, 2.0], [3.0, 4.0]])) + + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter('always') + method.set_points(np.array([[1, 2], [3, 4]])) + assert not [w for w in caught if issubclass(w.category, UserWarning)] + + +def test_integer_points_round_trip_unchanged(method): + points = np.array([[10, 20], [50, 100], [0, 0], [127, 255]]) + method.set_points(points) + np.testing.assert_array_equal(method.points, points) + assert np.issubdtype(method.points.dtype, np.integer) + + +# --------------------------------------------------------------------------- +# Duck typing +# --------------------------------------------------------------------------- + +def test_accepts_object_with_points_attribute(method): + """This is what makes ``method.set_points(selection_gui)`` work.""" + gui = DummySelectionGUI(points=[[10, 20], [30, 40]]) + method.set_points(gui) + np.testing.assert_array_equal(method.points, np.array([[10, 20], [30, 40]])) + + +def test_duck_typed_points_are_still_validated(method): + """Duck-typed input goes through the same validation as a plain array.""" + gui = DummySelectionGUI(points=[[1, 2, 3]]) + with pytest.raises(ValueError, match=r"two columns"): + method.set_points(gui) + + +# --------------------------------------------------------------------------- +# Bounds check is skipped without usable video info +# --------------------------------------------------------------------------- + +def test_bounds_check_skipped_when_video_lacks_image_size(): + obj = _bare_method_without_video_info() + wild_points = np.array([[-999, 99999], [5, 5]]) + + # must not raise, even though these coordinates would be out of bounds + # for any real image + IDIMethod.set_points(obj, wild_points) + np.testing.assert_array_equal(obj.points, wild_points)