Skip to content

FEAT: detector response map - #98

Merged
mdw771 merged 2 commits into
mainfrom
detector_response
Nov 13, 2026
Merged

mdw771 merged 2 commits into
mainfrom
detector_response

Conversation

@mdw771

@mdw771 mdw771 commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Features/fixes

Added a detector_response_map=(multiplier, offset) argument in the PtychographyTask API. In the forward model, predicted intensity is transformed as intensity * multiplier + offset before it is compared with the measured intensity.

Related issues

#92

Mentions

@stevehenke

Checklist (Don't skip!)

To check a checklist item, change the space in the backets with an x, i.e., [ ] -> [x].

Have you...

  • Formatted your code properly adhering to PEP-8? Considering using RUFF to format your code automatically (https://docs.astral.sh/ruff/installation/).
  • Cleaned up your code to remove large commented-out blocks?
  • Properly documented any new API knobs and functions with non-trivial signatures?
  • Resolved all merge conflicts with the main branch?
  • Checked the diffs between your branch and the main branch to ensure that your changes are not introducing any regressions or unnecessary/irrelevant changes?
  • Checked the diffs to ensure your PR only includes one feature/fix? Multiple features/fixes/changes should be split into different PRs.
  • Added unit tests for new features if necessary?

@mdw771
mdw771 requested a review from stevehenke November 6, 2026 15:35

@stevehenke stevehenke left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The forward model y = m·blur(crop(Σ|ψ_far|²)) + b is correct, and the order (crop, then blur, then pixel response) is right. Autodiff applies the map exactly, and LSQML applies it approximately but converges to the right answer. The other analytic reconstructors accept the map but ignore it in their updates.

1. Most analytic solvers ignore the map

  • PIE, rPIE, mPIE, GM-rPIE, MAGPIE, GM-MAGPIE, DM and RAAR all update through replace_propagated_exit_wave_magnitude(psi_far, y_true) (reconstructors/base.py:727). It sets the amplitude to sqrt(d) and ignores m and b.
  • BH's gradientF/hessianF (reconstructors/bh.py:157) compare sqrt(y_true) against |ψ_far|, so BH ignores the map too.
  • In all of these, y is used only for loss tracking. The object and probe come out identical to a run without the map, while the reported loss changes.

Suggested fix: either reject the map for these solvers, as api/options/magpie.py:141 already does for diffraction_pattern_blur_sigma, or project with the ratio ψ·sqrt(d / y_pred).

  • The ratio reduces to today's behavior when the map is the identity and blur is off.
  • It matches what LSQML-Gaussian already does (below).
  • BH would need its gradient and Hessian re-derived, or a rejection.

2. LSQML: approximate, right fixed point

The true gradient carries a factor m that backward_to_psi_far omits: ∂L/∂ψ* ∝ m·(1 − sqrt(d/y))·ψ for Gaussian, m·(1 − d/y)·ψ for Poisson. The step is still zero exactly at y = d.

  • Gaussian. With the default σ = 0.5 and α = 0.5, the step is ψ' = sqrt(d/y)·ψ, so I' = d·I/(m·I + b).
    • The fixed point is I* = (d − b)/m.
    • With b = 0 it reaches the answer in one step.
    • With b > 0 the reciprocal-space step multiplies the error by b/d each iteration, so convergence is slower where the background dominates.
  • Poisson. The step-size search assumes y(α) = y·(1 − αξ)², but with the offset the true value is (y − b)·(1 − αξ)² + b.
  • Epoch-0 probe rescale. It uses c² = Σd / Σy, where exact would be (Σd − Σb) / Σ(m·I).

The step-size search and the rescale are both exact for m alone and approximate with b ≠ 0.

3. Validate the maps

  • Require finite values. One NaN in m poisons autodiff gradients even at masked pixels, because 0 × NaN = NaN through m·I + b. In LSQML a NaN makes the loss NaN, and through the epoch-0 rescale also the probe.
  • Require real values. to_tensor keeps complex input complex, and validation accepts it.
  • Mask pixels where m ≤ floor. In LSQML-Gaussian, a pixel with m = 0 has its intensity multiplied by d/b on every step. If b = 0 too, the step divides by eps = 1e-9 and blows up. The ratio projection in item 1 would inherit this.

4. The maps cannot be refined

detector_response_multiplier and detector_response_offset are registered as fixed buffers, not as a ReconstructParameter. No reconstructor, autodiff included, can refine them, and get_data cannot return them. For integrating detectors this is limiting: the pedestal drifts, and dark frames are noisy estimates of it.

Suggestions:

  • Make the offset a ReconstructParameter with its own OptimizationPlan and options (optimizable, start epoch, step size), like OPRModeWeights or ProbePositions, and add it to get_data.
    • Autodiff then refines it for free.
    • Analytic solvers need an update rule. For fixed predicted intensities I_k, the least-squares offset per pixel is b = mean_k(d_k − m·I_k) over all positions, using valid pixels only. Accumulate it across the epoch's batches and apply it once per epoch.
  • Refine the multiplier only with a constraint.
    • It has a global scale degeneracy with the probe: (m, P) → (c·m, P/sqrt(c)) leaves y unchanged, so it needs a normalization such as mean(m) = 1 over valid pixels, or a fixed probe power.
    • Per pixel, m and b are identifiable together by regression across positions: m = cov_k(I, d)/var_k(I) and b = mean_k(d) − m·mean_k(I). This is well conditioned only where I_k varies across the scan, so the multiplier is best left fixed by default.
  • Keep m > 0. An offset may legitimately be negative after pedestal subtraction, so a positivity constraint on b should be optional.

5. Smaller points

  • Tests. The new tests check forward() only, including for EPIE and BH. Add at least one update-step test per reconstructor family; it would have caught item 1.
  • Silent no-op on old versions. In ≤ 2.4.0, Task.__init__(options, *args, **kwargs) swallows unknown keywords, so code written for this feature silently does nothing there. Consider rejecting unknown keyword arguments.
  • Threshold meaning. exclude_measured_pixels_below still compares raw measured values; with a pedestal in the offset, the threshold no longer means what it used to.
  • Integrating detectors in ADU. With m as the gain g and b as the pedestal, the mean is right, but the Poisson NLL is misspecified: ADU variance is g²·I + σ_read², while Poisson assumes g·I + b. Worth documenting that the Gaussian model, or converting to photons first, is the safer choice.
  • Nits. Trailing whitespace in the new docstrings (api/task.py).

mdw771 added a commit that referenced this pull request Nov 10, 2026
Account for gain and fixed offsets in magnitude projections, BH derivatives, LSQML reciprocal steps, and initial probe rescaling. This addresses the analytical-reconstructor and LSQML review points in PR #98.

Validate response gradients, fixed-point steps, masked pixels, and reconstruction updates; 351 targeted tests pass.
mdw771 added a commit that referenced this pull request Nov 10, 2026
Account for gain and fixed offsets in magnitude projections, BH derivatives, LSQML reciprocal steps, and initial probe rescaling. This addresses the analytical-reconstructor and LSQML review points in PR #98.

Validate response gradients, fixed-point steps, masked pixels, and reconstruction updates; 351 targeted tests pass.
mdw771 added a commit that referenced this pull request Nov 10, 2026
Account for gain and fixed offsets in magnitude projections, BH derivatives, LSQML reciprocal steps, and initial probe rescaling. This addresses the analytical-reconstructor and LSQML review points in PR #98.

Validate response gradients, fixed-point steps, masked pixels, and reconstruction updates; 351 targeted tests pass.

mdw771 commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

@stevehenke Thanks for the review. I've addressed the points below and pushed the changes. I also renamed “multiplier” to “gain” in the code and docs.

  1. Analytical reconstructors. The PIE family, MAGPIE, DM, and RAAR now project onto the corrected signal intensity max((d - b) / g, 0), preserving masked and unmeasured pixels. This directly accounts for the offset and gain in the amplitude constraint. Detector blur is explicitly rejected for these projection methods, whether the response map is omitted or supplied. Crop and mask handling are shared between the two cases, so an omitted response now also works with cropped detectors. The historical amplitude stabilization is retained for an omitted response on a matching grid. BH's gradient and Hessian now both use the shared detector operators for crop, blur, and gain; its configured blur is also passed to the forward model.

  2. LSQML. backward_to_psi_far now includes the gain and the detector crop/blur adjoints. The reciprocal-space step-size calculation uses the actual trial prediction along that wavefield gradient, including the fixed offset. It retains a two-pass fixed-point approximation; the original step-size path is preserved when response and blur are absent. The epoch-0 probe rescaling now subtracts the offset from measured and predicted intensities and accumulates only valid pixels, with guards for nonpositive signal totals.

  3. Validation. Both maps must be real, finite, and have the detector shape, including at masked pixels. Supplied gain must now be strictly positive everywhere. Any zero or negative gain raises an exception, including at masked pixels. This is checked in the reconstruction dtype, so a positive input that rounds to zero is also rejected. Invalid detector pixels should be supplied through valid_pixel_mask; pixel validity comes solely from that mask. This removes the gain-dependent effective mask and its bookkeeping when response maps are copied or refined. Small gains that remain positive after dtype conversion are accepted; the positive gain floor is applied during gain refinement. See 0a437fc.

  4. Map refinement. Gain and offset are now stored together as a (2, H, W) DetectorResponse subclass of ReconstructParameter, included in PtychographyParameterGroup and available through get_data("detector_response"). DetectorResponseOptions is nested under ForwardModelOptions and contains the optimizer, step size, optimization plan, and refinement controls. Refinement is disabled by default; omitted maps give the identity response by default.

    AD uses the gradient of its configured reconstruction loss. Analytical reconstructors use the suggested intensity least-squares fit: offset-only refinement uses the mean residual, and joint gain/offset refinement uses regression across positions. Statistics are accumulated in a separate batched pass at epoch end so the object and probe stay fixed during the fit. The fitted displacement is stored with the appropriate sign in grad and applied through the optimizer once per enabled epoch. Unit-step SGD reaches the unconstrained fit before the constraints below are applied.

    LBFGS and other closure-based optimizers are supported as well. For analytical response refinement, the closure minimizes squared distance to the fixed fitted response, so its gradient agrees with the supplied fit-based update. Masks and frozen components are respected during closure evaluations.

    Gain refinement is off by default. When enabled, valid gains are floored positive and normalized to mean one to fix the scale ambiguity. Gain estimates with insufficient intensity variation retain their current values before normalization. Offset nonnegativity is optional and disabled by default.

    After the analytical update and gain normalization, the fitted offset is bounded so that g * I + b stays positive over constrained observations with the current object and probe. The bound uses the minimum intensity across positions and ranks, with a rounding margin. Feasible negative offsets are preserved. If a gain-only update would violate the bound with its fixed offset, the previous response is restored and an error is raised. Intensity-domain likelihoods and amplitude losses also reject negative or nonfinite predictions, so later object/probe changes that invalidate a response fail clearly. These checks are in cc6e331.

  5. Smaller points. Added update-step tests across the reconstructor families, Gaussian/Poisson gradient checks, and SGD/LBFGS refinement tests covering masks, frozen components, and COMPACT object updates alongside response refinement. Small in-code reference arrays generated with the previous routines check the Gaussian/Poisson gradients and LSQML reciprocal steps. Additional tests cover positive predictions after response fitting in float32/float64, feasible negative offsets, strict gain validation, and preservation of the destination mask when copying a response. Two-epoch comparisons of omitted and explicitly supplied identity maps cover EPIE, rPIE, DM, RAAR, and BH with matching grids and mixed odd/even detector crops; the geometry consolidation is in 1cb686e.

    Task.__init__ now rejects unknown arguments, so unsupported or misspelled keywords forwarded by a subclass raise an error. The threshold documentation now states that exclude_measured_pixels_below compares the supplied measured values in their original units, including any pedestal, before response correction. The docstring whitespace is also fixed.

    On ADU noise statistics, I'm keeping the current likelihood form for this PR: the noise models receive the response-transformed predicted intensity. The Poisson option therefore still does not account for integrating-detector gain/read-noise variance, and I haven't added automatic conversion to photon counts.

A few additional changes came out of this work:

  • Fixed pre-existing crop and blur adjoints, including mixed odd/even crop alignment and the transpose of the replicated-boundary blur. These fixes are in a separate commit from the detector-response adjoint changes.

  • Applied pixel masks consistently to loss/error tracking, preconditioner normalization, and epoch-0 rescaling, including with an identity detector response.

  • Updated the optimizer-closure paths introduced on main to use the shared pixel-mask utility.

  • Documented the map coordinates: input maps align with the supplied diffraction patterns and receive the same optional FFT shift; returned maps use the internal diffraction-data ordering.

  • Fixed a test-order device mismatch by creating the test's mask explicitly on CPU. All five affected cases passed with CUDA set as the default device before each test (02fca3d).

@mdw771
mdw771 requested a review from stevehenke November 10, 2026 22:10
@stevehenke

Copy link
Copy Markdown
Collaborator
  1. Normalizing mean gain to 1 changes predictions, and nothing compensates. After a joint fit, DetectorResponse.step_optimizer divides gain by its mean c over valid pixels. The probe is not rescaled by √c, and the offset is not refit, so the model no longer fits the data. Reproduced: data 2I + 5 fits to gain 1 and offset 5, leaving an RMS residual of 5.7 on a data mean of 14.7. A gain map supplied with mean ≠ 1 is also not normalized at construction, so predictions jump the first time refinement runs. Suggested fix: have update_detector_response move c into the probe, since the reconstructor owns both; the autodiff path needs the same. Normalize at build time too, or document that it doesn't happen.

  2. The default response has the wrong shape, which crashes LSQML. PtychographyParameterGroup.__post_init__ builds DetectorResponse(shape=probe.get_spatial_shape()). update_accumulated_intensities subtracts detector_response.offset unconditionally. When the detector is smaller than the probe, a parameter group built without a response (anything not built through PtychographyTask) fails: RuntimeError: The size of tensor a (5) must match the size of tensor b (6). Suggested fix: skip the subtraction when is_identity is set, or build the default with the detector shape and the dataset's mask.

  3. The LSQML step bound uses the largest gain anywhere, masked pixels included. _get_detector_model_step_size sets max_step = 1 / gain.abs().amax() over the whole map. Reproduced: one masked pixel with gain 1e4 drops α from 0.76 to 1e-4 for every position. The docs require positive gain at masked pixels, but nothing else bounds its value there. An unmasked hot pixel throttles every step the same way. Restrict the bound to constrained pixels, and consider a statistic less fragile than the max.

  4. Initial probe normalization ignores the response. get_probe_renormalization_factor, which seeds the preconditioner, now applies the mask but still sums the raw patterns: pedestal included, not divided by gain. update_accumulated_intensities does subtract the offset, so the two disagree. On an integrating detector the inferred power is inflated by roughly N_pixels × offset. Pass clamp((patterns − offset) / gain, 0) over valid pixels instead.

  5. An infeasible prediction aborts the run.

    • The analytical offset bound only holds for the current object and probe. A later object or probe update can drive a constrained prediction negative, and validate_predicted_intensity then raises partway through the run.
    • An infeasible gain-only fit raises too.
    • Autodiff has no feasibility guard, so one large step on a negative offset ends the reconstruction.

    A long beamline run needs a non-fatal policy, such as clamping predictions to a small positive floor (with zero gradient) inside the losses, or shrinking or rejecting the response step. At minimum, make that an option.

  6. The public helper get_constrained_pixel_mask was removed and inlined 15 times. IterativePtychographyReconstructor.get_constrained_pixel_mask (on main since all unmeasured pixels treated as unconstrained #64) is gone, replaced by 15 copies of the same five-line fm.NoiseModel.get_constrained_pixel_mask(...) call across the reconstructors. That removes public API, and the next change to the masking rule has to be made 15 times. Restore the method and keep the new call sites on it.

Minor

  • step_size (default 1.0) means a relaxation factor for the analytical solvers and a learning rate for autodiff. The docs say so, but one shared default can hardly suit both; consider separate fields or separate defaults.
  • is_identity records where the map came from, not its values: any set_data() call, including the multi-GPU copy, permanently leaves the identity fast path. With identity maps LSQML gives the same result either way (object difference ≤ 2e-8), but test_omitted_and_explicit_identity_response_reconstructions_match leaves LSQML out. Add it.

Use the exact crop transpose for mixed odd/even grids and the transpose of
replicated-edge Gaussian blur in likelihood gradients. Account for crop and
blur in LSQML reciprocal-space steps, with adjoint and legacy-reference tests.
Apply shared detector gain and offset across reconstruction algorithms and
support optional analytical and autodiff refinement. Include validation,
consistent pixel masks, prediction-preserving gain normalization, optimizer
closures, documentation, and regression tests.

mdw771 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

@stevehenke Thanks for the follow-up review. I’ve pushed the changes below.

  1. Gain normalization. DetectorResponse.normalize_gain(probe) now divides gain by its valid-pixel mean c and multiplies probe amplitude by sqrt(c). Both analytical and autodiff paths call this method. All gains share the normalization, including masked pixels, so predictions are preserved across the detector. Masked pixels still receive no fitted updates. Added prediction-invariance tests covering crop, blur, and masks. Supplied maps retain their original normalization at construction; this is now documented.

  2. Default response with cropped detectors. LSQML now skips offset subtraction when accumulating intensities for an identity response. This avoids the shape mismatch when a directly constructed parameter group has a default response on the probe grid. Added regression tests for this construction path with mixed odd/even detector crops.

  3. Maximum-gain step bound. The maximum is now calculated only over pixels constrained by at least one measurement in the current batch, respecting both valid_pixel_mask and exclude_measured_pixels_below. Tests verify that excluded high-gain pixels leave the step unchanged for Gaussian and Poisson noise. I’ve retained the maximum-based bound and the remaining step-size logic, so a high-gain pixel that genuinely contributes to the batch can still limit the step.

  4. Initial probe-power estimate. The detector response is now passed through the preconditioner initialization path. The power estimate uses clamp((patterns - offset) / gain, 0) over valid pixels. Added tests for gain/offset correction, clipping below-background measurements, and identity-response equivalence.

  5. Infeasible predictions. I’m keeping the current fail-fast policy for this PR. Analytical refinement retains the offset bound for the current object and probe, optional offset nonnegativity, and restoration of the previous response before raising on an infeasible gain-only fit. Likelihood validation remains in place. I accept the corner case where a later object/probe update makes a negative offset infeasible; an invalid autodiff update can also still stop the reconstruction. I haven’t added prediction clipping or a recovery option.

  6. Shared mask helper. I moved the implementation to ptychi.utils.get_constrained_pixel_mask and updated the noise models, reconstructors, and tests to call it. Masking behavior is preserved. I chose the utility as the shared interface and documented it as the replacement for the removed reconstructor method.

  7. step_size. I’ve retained the single field and its default of 1.0. The documentation continues to explain that it controls relaxation toward the analytical fit, while autodiff uses it as a gradient learning rate and may require a smaller value.

  8. Identity detection. is_identity now checks for exact gain-one/offset-zero values at construction and after set_data(), and caches the result. It also refreshes after state loading, device/dtype conversion, and changes to optimization status. Optimizable responses always use the full response path so gradients remain available. Added tests for data transitions and copying, plus LSQML identity-equivalence tests for both noise models on matching and cropped grids.

I also consolidated response-fitting ownership in DetectorResponse. It now owns the accumulation buffers, distributed reductions, least-squares update, feasibility checks, and gain/probe normalization. The reconstructor retains scheduling and the separate epoch-end forward pass, keeping the object and probe fixed while collecting observations. The buffers are allocated when a fit starts and excluded from checkpoints.

Validation included 127 refinement and feasibility tests, including CUDA buffer movement and empty-rank handling. For the ownership refactor, all 48 saved arrays—fitted responses, probes, and predictions from 16 fits across SGD/LBFGS, both noise models, and float32/float64—matched the previous implementation exactly. Lint also passed.

Finally, I consolidated the history into two commits:

  • 2329333: the independent crop/blur adjoint fixes and their tests.
  • ad6f1dd: detector-response support, refinement, and the follow-up changes.

The standalone adjoint commit passed 360 tests and lint. Before rewriting the history, I saved a snapshot and verified that the consolidated head has an identical Git tree.

@mdw771
mdw771 merged commit cdea57b into main Nov 13, 2026
2 checks passed
@mdw771
mdw771 deleted the detector_response branch November 13, 2026 19:51
@mdw771 mdw771 mentioned this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants