Repository navigation
PERF: broadcast the Fourier probe shift over probe modes - #108
Merged
Merged
Conversation
With subpixel_shift_all_probe_modes=True, shift_unique_probes and adjoint_shift_probe_update_direction flattened the probes to (batch * modes, h, w) and repeated each shift once per mode, so translate built a dense complex multiplier for every (position, mode) pair and then multiplied the spectrum out of place: two batch x modes x N^2 arrays that are not needed, which made LSQML run out of memory with many modes at 512^2. translate now accepts (N, ..., H, W) images with one shift per leading index. The Fourier and periodic cubic path builds one multiplier per shift, broadcasts it over the dimensions in between, and multiplies the fresh FFT output in place unless autograd is recording or the product would promote. The spatial kernels flatten the images and repeat the shifts themselves. Both probe-shift callers pass the 4D tensor through. Results are unchanged (bitwise identical on CUDA). Closes #106 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mdw771
approved these changes
Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Features/fixes
With
subpixel_shift_all_probe_modes=True, the Fourier probe shift no longer builds a phase ramp for every (position, probe mode) pair.shift_unique_probesandadjoint_shift_probe_update_directionflattened the probes to(batch × modes, h, w)and repeated each shift once per mode. Sointerpolation.translatebuilt a dense complex multiplier for each of the batch × modes images, then multiplied the spectrum out of place. Those are two batch × modes × N² arrays that are not needed: 4.45 GiB each at 512², 15 modes and batch 152.interpolation.translateaccepts(N, ..., H, W)images with one(N, 2)shift per leading index.translate, so callers don't depend on the method.(batch, modes, h, w)tensor through instead of flattening it.translateandshift_imagesdocstrings describe the new shape.Results are unchanged. With the default
subpixel_shift_all_probe_modes=Falseonly mode 0 is shifted, so memory and speed don't change either.Tests:
tests/test_image_shift_adjoint.py. All of them fail onmain, whosetranslatetakes only(N, H, W). They check that:(N, M, H, W)stack gives the same result as the flattened stack with each shift repeatedMtimes. This covers Fourier, cubic, bilinear and nearest, forward and adjoint,pad0 and 1, and float32, complex64 and complex128. The input is not modified.test_probe_shift_adjoint_uses_forward_paddingalready runs both callers with 3 modes, and checks the adjoint against autograd of the forward shift.Checked
main(cdea57b) against this branch:PTYCHO_CI_DATA_DIR): the pass/fail lists are identical apart from the 51 new tests, which pass (1524 → 1575 passed). The same 49 tests fail in both because they need the CI data.ruff check src/(0.11.12) passes.shift_images, stock (flattened) vs this branch (4D): this is the comparison behind the new tests. On CUDA it is bitwise identical in all 32 cases. On CPU, the 8 padded Fourier/cubic cases differ by 1–2 ulp (≤ 2.4e-7), because the vectorized kernel rounds the loop remainder of a broadcast multiply differently. The new test's tolerance allows for that.mainexpandable_segments:Trueallow_nondeterministic_algorithms=False(3 epochs at 3 × 101), the final object and probe are bitwise identical.ifft2) in epoch 3. At that point 33.33 GiB is allocated, but 11.06 GiB of the reserved memory is in pieces too small for the 4.45 GiB block it needs.mainit fails in epoch 1, with 41.82 GiB allocated and 2.56 GiB in pieces.PYTORCH_ALLOC_CONF=expandable_segments:Trueremoves the fragmentation. With it the branch runs this case with 5.8 GiB to spare.mainfits only just, with 0.66 GiB to spare, and is 1.6× slower per epoch. As noted in subpixel_shift_all_probe_modes=True: the Fourier shift builds a phase ramp per probe mode (OOM at 15 modes, 512²) #106, the setting may be worth recommending in the docs for runs with many modes.The S0073 inputs and the scripts can be shared offline.
Related issues (optional)
Closes #106. Follow-up to #96.
Mentions
@mdw771 @hoidn
Checklist (Don't skip!)
Have you...
🤖 Generated with Claude Code