Repository navigation
Multi-view pose fusion, from two scripts that differed by one line - #378
Merged
Merged
Conversation
concert_fuse3d.py and reh_fuse3d.py in the Westney-comparisons study held byte-identical copies of an Umeyama alignment and a visibility-weighted fusion, and differed by a hardcoded list of pieces. That algorithm is now fuse_pose_views(), and the scripts are drivers over it. Closes #355. It takes extract_pose_landmarks result dicts or plain (world, visibility) arrays, so nothing about it is tied to MediaPipe as a source, and it returns the fused skeleton, the per-view rotations and scales, and the cross-view residual per landmark as well as overall. VERIFIED AGAINST THE STUDY'S OWN OUTPUT, not only against synthetic data. All four concert excerpts re-fused through the rewritten driver reproduce the committed fused_*.npz: identical keys, identical NaN pattern, residual_mm and t equal to the bit, fused to 6e-8 m, rotations to 1e-9. The residuals are unchanged at 139.7, 143.2, 147.7 and 151.4 mm. TWO GUARDS THE STUDY CODE LACKED, and neither moves a published number. The rotation average projected the mean matrix back with a bare U @ Vt and no determinant check, so a mean falling nearer a reflection than a rotation would have mirrored the skeleton silently --- swapping its left and right, which nothing downstream would flag. It does not fire anywhere in the concert data, so this is insurance rather than a correction. And interp_nan's comment said "fill short gaps" while the code filled every gap of any length and held the ends flat, so a long dropout became a straight line with nothing marking it; max_gap bounds that now, defaulting to the old behaviour so results reproduce. Every guard's test was checked by removing the guard and watching the test fail, because all eight passed on first run and a check that cannot fail is worth nothing. The residual is documented as a consistency measure, not an accuracy one: views that agree closely can agree on a wrong pose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xg1f739wddu5M4s3UwdNkn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #355.
concert_fuse3d.pyandreh_fuse3d.pyin the Westney-comparisons study held byte-identical copies of an Umeyama alignment and a visibility-weighted fusion, and differed by a hardcoded list of pieces. That algorithm is nowfuse_pose_views(), and the scripts are drivers over it.It takes
extract_pose_landmarksresult dicts or plain(world, visibility)arrays, so nothing about it is tied to MediaPipe as a source. It returns the fused skeleton, the per-view rotations and scales, and the cross-view residual per landmark as well as overall.Verified against the study's own output
Not only against synthetic data. All four concert excerpts, re-fused through the rewritten driver and compared with the committed
fused_*.npz:residual_mm,tfusedPer-piece residuals unchanged: 139.7, 143.2, 147.7, 151.4 mm.
Two guards the study code lacked
Neither moves a published number.
U @ Vtand no determinant check. A mean falling nearer a reflection than a rotation would have mirrored the skeleton silently, swapping its left and right, which nothing downstream would flag. It does not fire anywhere in the concert data, so this is insurance rather than a correction.interp_nan's comment said "fill short gaps per coord" while the code filled every gap of any length and held the ends flat, so a long dropout became a straight line through it with nothing marking it.max_gapbounds that now, defaulting to the old behaviour so existing results reproduce.Testing
Eight tests, all synthetic known-answer — the study data lives outside the repo and is not a dependency. All eight passed on first run, so each guard's test was checked by removing the guard and watching it fail; a check that cannot fail is worth nothing.
mypy musicalgestures/clean across 70 files. Full suite 714 passed, 4 skipped.The residual is documented as a consistency measure, not an accuracy one: views that agree closely can agree on a wrong pose.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Xg1f739wddu5M4s3UwdNkn