Better chunking strategies for constlat intersections and zonal routines. - #1624
Draft
cmdupuis3 wants to merge 68 commits into
Draft
Better chunking strategies for constlat intersections and zonal routines.#1624cmdupuis3 wants to merge 68 commits into
cmdupuis3 wants to merge 68 commits into
Conversation
…rite intersections, add 241 baseline testsgit status! - most came from accusphere
…us/mask, dispatcher
- benchmarks/geometry_kernels.py: ASV micro-benchmarks for all three layers of the EFT intersection stack (_accux_gca, _try_gca_gca_intersection, gca_gca_intersection, _accux_constlat, _try_gca_const_lat_intersection, gca_const_lat_intersection) plus EFT primitives and point-in-polygon; all functions warmed before timing so results reflect steady-state cost - test/test_plot.py: add test_to_raster_auto_extent verifying that the axis limits change and the raster contains finite data
…rectness fixes Review comments addressed: - Remove "near-double precision" / "sufficient" overclaims; say "roughly twice as accurate" and note the robustness tier boundary clearly - Explain _lon_bounds_from_vertices is required for UXarray antimeridian encoding and cannot be removed - Add block comment before _no_extreme functions clarifying they are pre-existing edge screeners unrelated to the EFT stack - Document SoS as explicit future work in _point_in_polygon_sphere docstring - L2 pos_fin/neg_fin: replace ternary with int(); exploit neg=-pos symmetry - Label computation: drop dead local*0 term, use integer mask arithmetic - Remove vertex-lat snap from bounds: _face_location_info already captures interior arc extrema accurately via the compensated kernel - _ON_MINOR_ARC_TOL: document intentional 1e-10 vs C++ 1e-8 divergence Bug fixes: - on_minor_arc: add antipodal-endpoint guard; a x b = 0 for antipodal inputs so every point on the great circle passes the collinearity test (false pos) - bounds.py: replace mask arithmetic use_ext*z_ext + (1-use_ext)*z_edge with plain if/else; 0*NaN = NaN propagates when norm=0, if/else does not - _point_in_polygon_sphere: ray-nudge now restarts the loop from i=0 so all edges are counted with the same ray (mid-loop nudge corrupted crossing parity) Cleanup: - Remove _flip_sign, _SIGN_NEG, _SIGN_POS, _SIGN_ZERO dead code from point_in_face.py; inline literals in _counts_as_crossing - Remove _SNAP_TOL_DEG constant and snap_tol_deg parameter throughout bounds.py - Notebook: fix Grid.get_point_on_face -> get_faces_containing_point; remove incorrect geometry.py row from Section 4 table; add accucross_pair and acc_sqrt_re to Section 2 building-blocks table
…PI name - ci/environment.yml: pin tornado<6.5.7 to avoid ssl.SSLError in panel 1.9.3 on Python 3.11 Windows (conda-forge regression, 2026-06-10) - intersections.py: remove _gca_gca_intersection_cartesian shim (dead code); add comment explaining _snap_const_lat_endpoint snap_sq constant - test_intersections.py: update 4 call sites to use gca_gca_intersection directly - spherical-geometry-accuracy.ipynb: fix stale Grid.get_point_on_face -> Grid.get_faces_containing_point (2 occurrences)
…rite intersections, add 241 baseline testsgit status! - most came from accusphere
…us/mask, dispatcher
Reconcile diverged accusphere branch. Resolutions: - intersections.py: restore inline=always on L1 kernels (_accux_constlat, _accux_gca) for allocation scalar-replacement - point_in_face.py: keep restart-loop ray casting (consistent parity), adopt named sign constants, drop unused _flip_sign - arcs.py: keep antipodal-endpoint guard in on_minor_arc - bounds.py: keep vertex-latitude snapping (snap_tol_deg) path - computing.py: keep detailed docstring with SIAM/EGUsphere references
Add an LLVM fma intrinsic and route two_prod through a single fused multiply-add for its error term on hardware that supports it, selected at import time and validated to be bit-exact against the Veltkamp split. Falls back to the portable Veltkamp form otherwise, so there is no hard FMA dependency. The FMA path is ~2x faster in the compensated geometry kernels (each two_prod drops from ~17 flops to one FMADD) and is numerically identical: all 241 AccuSphGeom baseline cases pass unchanged.
Add _accux_constlat_scalar, which takes the arc endpoints as six scalars and returns the candidate coordinates as scalars instead of two np.empty(3) arrays. _accux_constlat now wraps it so the array API is unchanged. Returning scalars lets Numba keep the candidates in registers, so a batch loop over many edges does no per-point heap allocation. On a 16M-point const-lat sweep this is ~2.7x faster than the array-returning path and drops the AccuX/FP64 cost ratio from ~19x to ~7x. Bit-identical results; all 241 AccuSphGeom baseline cases pass.
Move the uxarray kernel imports from module level into each benchmark class's setup, so a stale or mismatched environment build only errors the affected benchmark instead of aborting collection of every benchmark in the directory. Thanks @cmdupuis3 for catching the asv import failure.
* Honor quadrature kwargs in calculate_total_face_area calculate_total_face_area accepted quadrature_rule, order and latitude_adjusted_area but ignored them, always returning the cached default-parameter face_areas. As a result the gaussian/corrected path produced the same total as the triangular one. Keep the cached fast path for the default parameters (which also preserves the equal-area values used for HEALPix grids) and route any non-default quadrature settings through _compute_face_areas so the requested rule, order and latitude adjustment actually take effect. * Restore matplotlib backend after HoloViews matplotlib plot (#1538) * Restore matplotlib backend after HoloViews matplotlib plot plot(backend='matplotlib') calls hv.extension('matplotlib'), which switches the active matplotlib backend and clobbers the IPython inline display hook, silently breaking subsequent native matplotlib/xarray .plot() calls. Restore the original matplotlib backend right after the HoloViews extension switch; HoloViews objects still display via Store.current_backend, so this is safe. Closes #1537 * Address review: capture backend at switch, accurate docstring, effective test * Reconfigure IPython inline display hook when restoring backend mpl.use() restores the matplotlib backend name but does not re-register IPython's inline display integration that hv.extension('matplotlib') clobbers. In a Jupyter kernel without an explicit %matplotlib inline, native matplotlib/xarray .plot() calls after a uxarray matplotlib plot still failed to render. Re-run configure_inline_support when the restored backend is inline so the display hook is reinstated. See #1537. * Restore matplotlib backend via IPython shell reactivation to fix inline display --------- Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> * Allow SCRIP reader to respect units w/r/t radians (#1433) * Allow SCRIP reader to respect units wrt to radians * Add test file and unit test for SCRIP radian coordinate handling * minor formatting changes for scrip radians fix Moves meshfiles/scrip/scrip_radians.nc to meshfiles/scrip/scrip_radians/scrip_radians_grid.nc to match style of other meshfiles naming schemes. Renames the new _scrip._convert_to_degrees() to _scrip._values_in_degrees(). It doesn't always convert; and it also does more than just converting, because it gives numpy array from DataArray. Clarified docstring. ran the following, so it should now pass ruff checks: pre-commit run --all-files --------- Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Rajeev Jain <rajeeja@gmail.com> * Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) (#1563) * Upgrade YAC to v3.18, expose DNN remapping, drop pathlib backport - upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier, cython>=3.1 via conda for the bindings build) - fix add_average: reduction_type -> weight_type (renamed in YAC v3.18) - expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test - remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build) * Reference #1561 in numba version pin comment --------- Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com> * Bump actions/download-artifact in the actions group across 1 directory (#1572) Bumps the actions group with 1 update in the / directory: [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/download-artifact` from 7 to 8 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...v8) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#1565) updates: - [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.15.21](astral-sh/ruff-pre-commit@v0.15.20...v0.15.21) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Accusphere: gca_gca benchmarks and better arc sampling * Accusphere: fix sum_of_squares accuracy bug + tests * Accusphere: numba fma refactor and fix validation tautology * Accusphere: scalarized _counts_as_crossing --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Rajeev Jain <rajeeja@gmail.com> Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> Co-authored-by: zarzycki <colin.zarzycki@gmail.com> Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…k validity per AccuSphGeom
…_lat_intersection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains two post-accusphere optimizations, eliminating low-level hard materializations by using a vector-based masking strategy rather than individual conditionals, and reduced
zonal_meanpeakmem by building only the candidate faces instead of the whole grid.Partly addresses #1587
Waiting on accusphere merge to main #1513 first.
Overview
PR Checklist
General
Testing
Documentation