From 9e8091cbf5a6dffabe29f5dcc758415daa4449bc Mon Sep 17 00:00:00 2001 From: Alexander Refsum Jensenius Date: Sun, 23 Aug 2026 20:58:18 +0200 Subject: [PATCH 1/2] Actions: the layer between motion and meaning First part of #373. The toolbox measures motion and says nothing about actions, so this adds the middle term and leaves room for the third. An `Action` is a span with provenance, a `features` dict and a `labels` dict. `segment_actions` cuts spans out of a motion envelope; `action_type` describes the shape of one; `MgVideo.actions_from_motion()` does both and builds its envelope from POSE rather than pixels, which is what makes the result follow the person instead of the camera --- a pan moves every pixel and moves no landmark relative to the others. The separation of `features` from `labels` is the design rather than tidiness. Measuring fills features; naming fills labels. A shape is something the signal shows and a gesture is something a recogniser claims, and the two fail differently: a missed boundary loses an action outright, while a wrong name leaves the action there to be relabelled. It also lets the skeleton-based, video-model and dense-trajectory recognisers ARJ asked for coexist, since each writes under its own key and they are allowed to disagree. TWO ERRORS OF MINE, both caught by the known-answer tests rather than by reading. Half height was measured from the span's own minimum. A motion envelope has a meaningful zero, so for a rectangular action the minimum IS the peak, and the steadiest movement possible came out impulsive. Fixing that, a decaying impulse began reading as sustained, because segmentation cuts an action at a threshold and therefore hands this function an impulse already truncated to its loud third. Rather than tune the threshold until the three test signals passed, I measured all three after segmentation: time above half height reads 0.385 for a decay, 0.510 for a tremolo and 1.000 for a plateau, which does not separate them, while the energy centroid reads 0.332, 0.481 and 0.500, which does. The call rests on the centroid now, that being also the standard descriptor in the sound-object lineage this typology comes from, and the numbers are in the docstring so the threshold can be argued with. 706 tests pass, 4 skipped. mypy clean --- which caught me naming `musicalgestures` in an annotation without importing it, the same bug fixed in `_videoadjust.py` hours earlier, now that the step blocks rather than prints. Co-Authored-By: Claude Opus 5 --- musicalgestures/_actions.py | 296 ++++++++++++++++++++++++++++++++++++ musicalgestures/_video.py | 2 + tests/test_actions.py | 141 +++++++++++++++++ 3 files changed, 439 insertions(+) create mode 100644 musicalgestures/_actions.py create mode 100644 tests/test_actions.py diff --git a/musicalgestures/_actions.py b/musicalgestures/_actions.py new file mode 100644 index 00000000..ecc2a172 --- /dev/null +++ b/musicalgestures/_actions.py @@ -0,0 +1,296 @@ +"""Actions: the layer between motion and meaning. + +Three words, and this module owns the middle one. + +**Motion** is continuous displacement in space over time. The toolbox already measures it: +quantity of motion, optical flow, pose landmarks. + +**An action** is a segment of that motion, usually with a beginning and an end. Reaching for +a cup is an action; so is a drum stroke; so is a phrase of dance that is going nowhere in +particular. Actions need not be goal-directed, and in music they are often sound-producing. + +**A gesture** is an action carrying meaning, and meaning is not a property of the signal. A +recogniser can say a segment looks like someone waving; whether that wave means *hello*, +*stop* or nothing at all is not visible in the pixels. So this module derives actions from +motion, and then lets labels be *attached* to actions --- to some of them, not all --- rather +than pretending that recognition and meaning are one step. + +That layering is the design. :class:`Action` is a span with provenance and a place to hang +labels. :func:`segment_actions` produces spans from a motion envelope. :func:`action_type` +describes a span in terms of how the movement is distributed within it. Recognisers, of +which there can be several and which may disagree, add named labels to spans they did not +have to produce. + +The point of separating them is that the segmenter and the recogniser fail differently. A +segmenter that misses a boundary loses an action entirely; a recogniser that guesses wrong +leaves the action there to be relabelled. Keeping the record of *what happened when* apart +from *what it was* means the second can be revised without redoing the first. + +.. note:: + + Everything here works from a **motion envelope** --- a one-dimensional series saying how + much movement there was at each moment --- and not from pixels directly. That is + deliberate: issue #373 asks that recognition follow human activity rather than camera + motion or scene change, and an envelope computed from pose landmarks carries only the + body. Pass a pixel-derived envelope and the module will work, and a pan of the camera + will read as an action. +""" +from __future__ import annotations + +from dataclasses import dataclass, field + +import numpy as np + +import musicalgestures + + +@dataclass +class Action: + """One segment of motion, with somewhere to record what it was. + + Attributes: + start (float): Start time in seconds. + end (float): End time in seconds. + source (str): What produced this span, so that spans from different segmenters + can be told apart when they are pooled. + labels (dict): Names given to this action by recognisers, keyed by recogniser. + Empty is the normal state: most actions are never named, and an action with no + label is still an action. This is where a gesture would be recorded, if anything + could establish one. + features (dict): Numbers describing the span, from :func:`action_type` and anything + else that measures without naming. + """ + + start: float + end: float + source: str = "unknown" + labels: dict = field(default_factory=dict) + features: dict = field(default_factory=dict) + + @property + def duration(self) -> float: + """Length of the action in seconds.""" + return self.end - self.start + + def overlaps(self, other: "Action") -> bool: + """Whether this action shares any time with `other`.""" + return self.start < other.end and other.start < self.end + + def __repr__(self) -> str: + named = f" {self.labels}" if self.labels else "" + return f"" + + +def _as_envelope(x) -> np.ndarray: + e = np.asarray(x, float).ravel() + if not np.isfinite(e).all(): + e = np.interp(np.arange(len(e)), + np.flatnonzero(np.isfinite(e)), + e[np.isfinite(e)]) if np.isfinite(e).any() else np.zeros_like(e) + return e + + +def segment_actions(envelope, fs: float, threshold: float = 0.15, + min_duration: float = 0.1, min_gap: float = 0.1, + source: str = "envelope") -> list[Action]: + """Cut a motion envelope into actions, where movement rises above rest. + + An action begins where the envelope crosses `threshold` and ends where it falls back. + Short gaps are closed before short spans are dropped, in that order, because a single + action that dips momentarily below the threshold would otherwise be discarded as two + fragments rather than kept as one. + + The threshold is a fraction of the envelope's range, not an absolute value, so the same + setting transfers between recordings of different scale. Movement never rising above it + yields no actions, which is the correct answer for a still recording rather than an + error. + + Args: + envelope: Motion per frame, one dimension. Non-finite values are interpolated. + fs (float): Sampling rate of the envelope, in frames per second. + threshold (float): Level counting as movement, as a fraction of the envelope's + range. Defaults to 0.15. + min_duration (float): Spans shorter than this, in seconds, are discarded as noise. + Defaults to 0.1. + min_gap (float): Gaps shorter than this, in seconds, are closed. Defaults to 0.1. + source (str): Recorded on each Action, to identify what produced it. + + Returns: + list: The actions found, in time order. + """ + e = _as_envelope(envelope) + if len(e) < 2 or fs <= 0: + return [] + + lo, hi = float(np.min(e)), float(np.max(e)) + if hi <= lo: + return [] + level = lo + threshold * (hi - lo) + + active = e > level + if not active.any(): + return [] + + # run starts and ends, as sample indices + edges = np.diff(active.astype(np.int8)) + starts = list(np.flatnonzero(edges == 1) + 1) + ends = list(np.flatnonzero(edges == -1) + 1) + if active[0]: + starts.insert(0, 0) + if active[-1]: + ends.append(len(e)) + + spans = [[s / fs, t / fs] for s, t in zip(starts, ends)] + + # close short gaps first: a dip below the level in the middle of one action is not + # a boundary, and dropping short spans before merging would delete its halves + merged: list[list[float]] = [] + for span in spans: + if merged and span[0] - merged[-1][1] < min_gap: + merged[-1][1] = span[1] + else: + merged.append(span) + + return [Action(start=s, end=t, source=source) + for s, t in merged if t - s >= min_duration] + + +def action_type(envelope, fs: float, iterative_min_peaks: int = 3, + impulsive_centroid: float = 0.42) -> dict: + """Describe how movement is distributed inside one action. + + Three shapes, following the typology of *Sound Actions*: + + - **impulsive** --- energy arrives at once and decays. A hit, a tap, a clap. + - **sustained** --- energy is held across the span. A bowed note, a slow reach. + - **iterative** --- energy repeats within the span. A tremolo, a shake, a scrub. + + Decided on two measures rather than a classifier, so the call can be read and argued + with. `peaks` counts internal maxima above half the span's peak: several of them mean + the movement repeated. `centroid` is where the span's energy sits, 0 at its start and 1 + at its end: an impulse is front-loaded, a held movement is centred. + + Iterative is tested first, because a repeated movement is also a centred one, and the + repetition is the more specific description. + + The discriminator is the centroid rather than time spent above half height, and that is + a correction rather than a preference. Segmentation cuts an action at a threshold, so a + decaying impulse arrives here already truncated to its loud third, which raises the + fraction of it spent above half height until it is indistinguishable from a held + movement. Measured on the canonical shapes after segmentation: time-above-half reads + 0.385 for a decay against 0.510 for a tremolo and 1.000 for a plateau, while the + centroid reads 0.332, 0.481 and 0.500. Only the second separates the impulse. + + Args: + envelope: The motion envelope of ONE action, not of the whole recording. + fs (float): Sampling rate of the envelope, in frames per second. + iterative_min_peaks (int): Internal peaks needed to call a span iterative. + Defaults to 3. + impulsive_centroid (float): Energy centroid below which a span is called impulsive. + Defaults to 0.42, midway between a decay and a plateau as measured above. + + Returns: + dict: ``type`` as one of ``'impulsive'``, ``'sustained'``, ``'iterative'``, with the + ``peaks``, ``centroid`` and ``sustain`` behind it, so a disputed call can be + checked rather than merely disagreed with. + """ + e = _as_envelope(envelope) + out = {"type": "impulsive", "peaks": 0, "centroid": 0.0, "sustain": 0.0} + if len(e) < 3 or fs <= 0: + return out + + # Half of the peak, not half of the span's own range. A motion envelope has a + # meaningful zero --- no movement --- so "held" means "stayed near its peak", and + # measuring from the span minimum makes a perfectly steady action read as impulsive, + # because its minimum and its peak are the same number. + hi = float(np.max(e)) + if hi <= 0: + return out + half = 0.5 * hi + + total = float(np.sum(e)) + t = np.arange(len(e)) / (len(e) - 1) + centroid = float(np.sum(t * e) / total) if total > 0 else 0.0 + rising = (e[1:-1] > e[:-2]) & (e[1:-1] >= e[2:]) & (e[1:-1] > half) + + peaks = int(np.count_nonzero(rising)) + if peaks >= iterative_min_peaks: + shape = "iterative" + elif centroid >= impulsive_centroid: + shape = "sustained" + else: + shape = "impulsive" + + out["type"] = shape + out["peaks"] = peaks + out["centroid"] = centroid + out["sustain"] = float(np.mean(e > half)) + return out + + +def describe_actions(actions: list[Action], envelope, fs: float) -> list[Action]: + """Attach :func:`action_type` to each action, in place, and return them. + + Measuring is kept apart from naming on purpose: this fills `features`, never `labels`. + A shape is something the signal shows; a name is something a recogniser claims. + + Args: + actions (list): Actions to describe, as returned by :func:`segment_actions`. + envelope: The motion envelope the actions were cut from. + fs (float): Sampling rate of the envelope. + + Returns: + list: The same actions, with `features` filled in. + """ + e = _as_envelope(envelope) + for a in actions: + i, j = int(round(a.start * fs)), int(round(a.end * fs)) + a.features.update(action_type(e[i:j], fs)) + return actions + + +def mg_actions(self: "musicalgestures.MgVideo", envelope=None, fs: float | None = None, + threshold: float = 0.15, min_duration: float = 0.1, + min_gap: float = 0.1) -> list[Action]: + """Segment this video into actions and describe the shape of each. + + With no envelope given, one is built from the body rather than from the picture: pose + landmarks are extracted if they are not cached, and their quantity of motion becomes + the envelope. That is what makes the result follow the person and not the camera --- a + pan moves every pixel and moves no landmark relative to the others. + + Args: + envelope: A motion envelope to segment. Defaults to None, meaning build one from + pose. Pass your own to segment something else, and note that a pixel-derived + envelope will read camera movement as action. + fs (float, optional): Sampling rate of `envelope`. Defaults to the video's frame + rate, which is right for any envelope with one value per frame. + threshold (float): Level counting as movement, as a fraction of the envelope's + range. Defaults to 0.15. + min_duration (float): Shortest span kept, in seconds. Defaults to 0.1. + min_gap (float): Longest gap closed, in seconds. Defaults to 0.1. + + Returns: + list: The actions found, each carrying its shape in `features`. Also stored on the + video as `actions`. + """ + rate = float(fs) if fs is not None else float(self.fps) + if envelope is None: + from musicalgestures._qom import pose_qom + + cache = getattr(self, "_pose_keypoints", None) + if not cache: + self.pose() + cache = getattr(self, "_pose_keypoints", None) + if not cache: + raise RuntimeError( + "no pose landmarks are available, so there is no body to follow. Run " + "pose() first, or pass an envelope of your own.") + envelope, rate = pose_qom(cache["data"] if isinstance(cache, dict) else cache, rate) + + actions = segment_actions(envelope, rate, threshold=threshold, + min_duration=min_duration, min_gap=min_gap, + source="pose-qom" if fs is None else "envelope") + describe_actions(actions, envelope, rate) + self.actions = actions + return actions diff --git a/musicalgestures/_video.py b/musicalgestures/_video.py index 566472db..280ff24a 100644 --- a/musicalgestures/_video.py +++ b/musicalgestures/_video.py @@ -183,6 +183,7 @@ def __init__( mhi_image: MgImage motion_video: "musicalgestures.MgVideo" motiondescriptors_figure: MgFigure + actions: list motion_plot_image: MgImage motiongram_horizontal_image: MgImage motiongram_vertical_image: MgImage @@ -260,6 +261,7 @@ def motion_mp(self, *args, **kwargs): from musicalgestures._blurfaces import mg_blurfaces as blur_faces # type: ignore[misc] from musicalgestures._impacts import mg_impacts as impacts # type: ignore[misc] from musicalgestures._grid import mg_grid as grid # type: ignore[misc] + from musicalgestures._actions import mg_actions as actions_from_motion # type: ignore[misc] from musicalgestures._videoadjust import mg_resample as resample # type: ignore[misc] from musicalgestures._motionvideo import save_analysis # type: ignore[misc] diff --git a/tests/test_actions.py b/tests/test_actions.py new file mode 100644 index 00000000..b1d83568 --- /dev/null +++ b/tests/test_actions.py @@ -0,0 +1,141 @@ +"""Actions cut from a motion envelope, and the shape of what was cut. + +The signals here are built so the right answer is not a matter of opinion: three separated +bursts are three actions, a decaying spike is impulsive, a plateau is sustained, and a +tremolo is iterative. If the thresholds are ever retuned, these say what must not change. +""" +import numpy as np +import pytest + +from musicalgestures._actions import ( + Action, + action_type, + describe_actions, + segment_actions, +) + +FS = 100.0 + + +def _silence(seconds): + return np.zeros(int(seconds * FS)) + + +def _burst(seconds, shape="flat"): + n = int(seconds * FS) + if shape == "flat": + return np.ones(n) + if shape == "decay": + return np.exp(-np.linspace(0, 6, n)) + if shape == "tremolo": + return 0.5 + 0.5 * np.sin(2 * np.pi * 8 * np.linspace(0, seconds, n)) + raise ValueError(shape) + + +class TestSegmentation: + def test_three_separated_bursts_are_three_actions(self): + env = np.concatenate([_silence(1), _burst(0.5), _silence(1), + _burst(0.5), _silence(1), _burst(0.5), _silence(1)]) + assert len(segment_actions(env, FS)) == 3 + + def test_a_still_recording_has_no_actions(self): + assert segment_actions(_silence(5), FS) == [] + + def test_boundaries_land_where_the_movement_is(self): + env = np.concatenate([_silence(1), _burst(0.5), _silence(1)]) + a, = segment_actions(env, FS) + assert a.start == pytest.approx(1.0, abs=0.05) + assert a.end == pytest.approx(1.5, abs=0.05) + + def test_a_momentary_dip_does_not_split_one_action_in_two(self): + """A gap shorter than min_gap is closed, and the closing happens before short + spans are dropped, or the halves would be discarded rather than joined.""" + env = np.concatenate([_silence(1), _burst(0.3), _silence(0.05), + _burst(0.3), _silence(1)]) + assert len(segment_actions(env, FS, min_gap=0.2)) == 1 + + def test_a_flicker_shorter_than_min_duration_is_not_an_action(self): + env = np.concatenate([_silence(1), _burst(0.02), _silence(1)]) + assert segment_actions(env, FS, min_duration=0.1) == [] + + def test_the_threshold_is_relative_so_scale_does_not_matter(self): + env = np.concatenate([_silence(1), _burst(0.5), _silence(1)]) + assert len(segment_actions(env, FS)) == len(segment_actions(env * 1000, FS)) + + def test_movement_at_the_very_start_and_end_is_not_lost(self): + env = np.concatenate([_burst(0.5), _silence(1), _burst(0.5)]) + assert len(segment_actions(env, FS)) == 2 + + def test_source_is_recorded_so_segmenters_can_be_told_apart(self): + env = np.concatenate([_silence(1), _burst(0.5), _silence(1)]) + a, = segment_actions(env, FS, source="pose-qom") + assert a.source == "pose-qom" + + +class TestActionType: + def test_a_decaying_spike_is_impulsive(self): + assert action_type(_burst(0.4, "decay"), FS)["type"] == "impulsive" + + def test_a_plateau_is_sustained(self): + assert action_type(_burst(1.0, "flat"), FS)["type"] == "sustained" + + def test_a_tremolo_is_iterative(self): + assert action_type(_burst(1.0, "tremolo"), FS)["type"] == "iterative" + + def test_iterative_wins_over_sustained(self): + """A repeated movement is also a held one by the sustain measure; the repetition + is the more specific description, so it is tested first.""" + out = action_type(_burst(1.0, "tremolo"), FS) + assert out["sustain"] >= 0.35 and out["type"] == "iterative" + + def test_the_evidence_for_the_call_is_returned(self): + out = action_type(_burst(1.0, "tremolo"), FS) + assert out["peaks"] >= 3 + assert 0.0 <= out["sustain"] <= 1.0 + assert 0.0 <= out["centroid"] <= 1.0 + + def test_an_impulse_is_front_loaded_and_a_plateau_is_centred(self): + """The measure the impulsive/sustained call rests on.""" + assert action_type(_burst(0.4, "decay"), FS)["centroid"] < 0.42 + assert action_type(_burst(1.0, "flat"), FS)["centroid"] == pytest.approx(0.5, abs=0.02) + + def test_a_perfectly_steady_span_is_sustained(self): + """Its minimum and its peak are the same number, which an earlier version of this + measured from and so called the steadiest possible movement impulsive.""" + assert action_type(np.ones(50), FS)["type"] == "sustained" + + @pytest.mark.parametrize("n", [0, 1, 2]) + def test_too_few_samples_gives_a_default_rather_than_raising(self, n): + assert action_type(np.zeros(n), FS)["type"] == "impulsive" + + +class TestDescribe: + def test_each_action_gets_its_own_shape(self): + env = np.concatenate([_silence(0.5), _burst(0.4, "decay"), _silence(0.5), + _burst(1.0, "tremolo"), _silence(0.5)]) + actions = describe_actions(segment_actions(env, FS), env, FS) + assert [a.features["type"] for a in actions] == ["impulsive", "iterative"] + + def test_describing_fills_features_and_never_labels(self): + """A shape is measured; a name is claimed. The module does not confuse them.""" + env = np.concatenate([_silence(0.5), _burst(0.5), _silence(0.5)]) + a, = describe_actions(segment_actions(env, FS), env, FS) + assert a.features and a.labels == {} + + +class TestActionRecord: + def test_duration(self): + assert Action(1.0, 2.5).duration == pytest.approx(1.5) + + def test_overlap_is_symmetric(self): + a, b = Action(0, 2), Action(1, 3) + assert a.overlaps(b) and b.overlaps(a) + + def test_touching_actions_do_not_overlap(self): + assert not Action(0, 1).overlaps(Action(1, 2)) + + def test_labels_are_per_recogniser_so_two_may_disagree(self): + a = Action(0, 1) + a.labels["pose-gcn"] = "wave" + a.labels["video-i3d"] = "clapping" + assert len(a.labels) == 2 From f7c5f5d8bcd4455706b60e5cf3f4bae7f7ab2900 Mon Sep 17 00:00:00 2001 From: Alexander Refsum Jensenius Date: Sun, 23 Aug 2026 21:10:30 +0200 Subject: [PATCH 2/2] Handover for 2026-08-23 Written at the end of a session that ran across four repositories, so the state is not visible from any one of them. Leads with what is waiting rather than what was done, because one item is easy to lose: micromotion's `heading-persistence` branch is committed and never pushed, it has no upstream, and `git status -sb` therefore prints no ahead marker for it --- which reads like "in sync". It is three commits ahead of origin/main and two of those predate today, so pushing it publishes work that was already sitting there. Records the release-process correction as a procedure rather than a note: the version lives in three places, not the two RELEASING.md named, and the check to run before tagging is one command. Also records the argument for keeping the mypy gate closed, which is not the count. It is the table of what the count was hiding --- a crop window that crashed on its own instruction, an audio axis that crashed past an hour, a Flow you could not hold, and a public method that had never worked. Ends on the manuscript, which is where the session started and where it did not go. Co-Authored-By: Claude Opus 5 --- plans/2026-08-23-handover.md | 161 +++++++++++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) create mode 100644 plans/2026-08-23-handover.md diff --git a/plans/2026-08-23-handover.md b/plans/2026-08-23-handover.md new file mode 100644 index 00000000..1434b623 --- /dev/null +++ b/plans/2026-08-23-handover.md @@ -0,0 +1,161 @@ +# Handover — 2026-08-23 + +Written at the end of a long session across four repositories. Reading order: **In flight** +first, because two things are waiting on you and one of them is easy to lose. + +--- + +## In flight — needs you + +| what | where | state | +|---|---|---| +| Actions layer for #373 | MGT PR **#377** | CI green, unmerged | +| Heading persistence for #357 | micromotion branch `heading-persistence` | committed, **never pushed** | + +**The micromotion branch has no upstream.** `git status -sb` prints no ahead/behind marker for +it, which reads like "in sync" and is not. It is three commits ahead of `origin/main`, and only +the newest is from today: + +``` +13651bc 2026-08-23 Heading persistence … <- today +9040cef 2026-08-21 Regenerate the docs figures … <- was already unpushed +e6d0998 2026-08-21 Say "device" where the docs … <- was already unpushed +``` + +So pushing that branch publishes two older commits as well. Check you want them out before +you push. + +--- + +## Released today, and verified on PyPI + +| package | version | notes | +|---|---|---| +| `ambiscape` | 0.45.0 | Broad Sound Taxonomy sidecar for Freesound uploads | +| `musicalgestures` | 1.13.0 | result attributes declared and renamed; two bug fixes | + +Both tagged without the `v`, both published as GitHub releases rather than bare tags, both +confirmed present on PyPI through the JSON API. `pip index versions` serves a stale cache and +will tell you the previous version for a while; do not trust it. + +**`RELEASING.md` was wrong and is now fixed.** It said the version lives in two places. It +lives in three — `__version__`, `CITATION.cff`, and `docs/releases.md`, which +`tests/test_release_consistency.py` guards. The 1.13.0 release commit went red on five CI jobs +for exactly that. Run this before tagging anything: + +```bash +python3 -m pytest tests/test_release_consistency.py -q +``` + +`CITATION.cff` had also drifted to 1.11.3 across two releases and is corrected. + +--- + +## Closed today + +- **#350** — internal typing. **248 errors to zero**, and the CI mypy step **blocks merges now**. +- **#370** — `motion_mp()` retired. It raised on its first call and had never worked. +- **#346** — result attributes declared on `MgVideo` and named for what they hold. +- **#356** — was already solved in micromotion; closed with a pointer. +- **#323** — the capability asked for exists; closed with a worked example. + +### The reason #350 was worth doing + +Almost every "typing error" was a real defect. This is the list, and it is the argument for +keeping the gate closed: + +| symptom | what it was | +|---|---| +| crop window | `NameError` when you followed its own on-screen instruction | +| audio time axis | crashed on files over an hour; rendered `10:0` for whole minutes | +| `Flow` | `flow = mg.MgVideo(p).flow` was unusable — weakref to a collected video | +| four writers | crashed on a video that decoded to no frames | +| thirteen signatures | denied they could return `None` | +| `motion_mp()` | wholly broken | +| two frame rates | declared `int`; 29.97 is not an integer | +| `mg_motiondata` | declared `list`, returns `str` or `list` | +| four pose methods | stored `None` as though it were a figure | + +--- + +## Open, in the order I would take them + +### 1. Recognisers for #373 + +The foundation is PR #377. `Action` carries `features` (measured) and `labels` (claimed), and +recognisers write into `labels` under their own key, so several may disagree. You asked for all +four approaches; each is now a separate, additive piece of work: + +- skeleton-based classifier over the pose landmarks MGT already extracts +- off-the-shelf video models (I3D / SlowFast / VideoMAE) behind an optional extra +- dense trajectories and motion-boundary histograms +- the segmentation-first path, which #377 begins + +**Validate against SoundActions.** 365 daily recordings labelled impulsive 126 / sustained 83 / +iterative 125 / no-majority 31. An action typer that reproduces those labels is a testable +claim rather than a demonstration, and the corpus is yours. + +**The risk worth naming**: a Kinetics or AVA classifier brings a 400-class taxonomy of everyday +activities that is not the one in *Sound Actions*. Wrapping one is fine; letting its label set +become the toolbox's idea of an action is not. + +### 2. Remaining ports — #355, #358, #359 + +**#355 first, and it is nearly mechanical.** `2-Westney-comparisons/code/concert_fuse3d.py` and +`reh_fuse3d.py` are 82 lines each and differ by **one line**, a hardcoded `PIECES` list. One +function, one parameter. + +**#359 has the least behind it despite reading as the most concrete.** Only +`timing_summary_yolo.json` and two figures survive in the cymbal study; the code that made them +is gone. It is new work against Ultralytics plus a new optional dependency. The JSON is still +useful as a fixture. + +**Check micromotion before porting anything.** #356 and #357 both turned out to be mostly done +there already, and #357's issue text names a destination — `musicalgestures/_posture.py` — that +became a pure re-export shim on 2026-07-29, twelve days after the issue was filed. The +four-toolbox rule puts filtering, resampling and circular statistics in micromotion. + +### 3. 2.0 — but not yet + +The deprecated aliases (`ssm_fig`, `motion_plot`, the four grams, and the rest) come out at 2.0. +**Wait.** Those warnings only became visible today: `_audio.py` called +`warnings.filterwarnings("ignore")` at import, which silenced every warning in the process, so +nobody has ever actually been warned. 1.13.0 is the first release where they reach anyone. +Removing them in the next major would give one release of real notice. + +### 4. #311 JOSS — on hold at your request + +Closer than the plan assumed. It was sequenced after 2.0 for API stability, and that is now +achieved: renames done, typed, gated, released. 2.0 only removes aliases; it does not change +what a reviewer sees. `paper/paper.md` is still the 48-line stub dated 26 August 2020, and the +author list is a decision only you can make. + +### 5. #312 — in-memory workflow + +Untouched, from 2023. Decided in the road-to-2.0 design to land **additively** rather than in a +major bump, since nothing about skipping intermediate disk writes requires breaking anything. +Its own brainstorm should start by asking whether the 2023 proposal is still wanted. + +--- + +## Two things still true and worth fixing + +**The `warnings.filterwarnings("ignore")` is narrowed but the lesson generalises.** It is scoped +to `audioread` deprecations now. It had been hiding a crash on hour-long audio, tick labels +losing a digit, and matplotlib saying so. `tests/test_audio_time_axis.py` asserts the package +does not silence warnings process-wide, so it cannot come back unnoticed. + +**mypy is pinned, and must stay pinned while the gate is closed.** On identical source, mypy +1.20.1 reports 169 errors where 2.3.1 reports 90. Unpinning it lets a release turn CI red with +no change to the code. + +--- + +## And the thing that is not code + +The Sound Spaces manuscript has two commits from this session — the Broad Sound Taxonomy +paragraphs in ch2 and the supporting notes — and has not been touched since. `notes/TODO.md` +still lists ch11 at 128 words against a scaffold calling it the core thesis, Part IV at 4,557 +words in total, and twelve sentences that stop mid-thought. + +Everything above is instrumentation. That is the book.