Skip to content

Fix OpenMP pragmas - #757

Merged
spossann merged 11 commits into
develfrom
fix-openmp-pragmas
Oct 14, 2026
Merged

spossann merged 11 commits into
develfrom
fix-openmp-pragmas

Conversation

@max-models

@max-models max-models commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Corresponding feectools PR struphy-hub/feectools#103

Fix OpenMP directives so all 128 Struphy kernel modules compile with GNU Fortran using struphy compile --openmp, and compile the feectools kernels with OpenMP as well.

  • Repair parallel regions, variable scoping, and accumulation reductions.
  • Give each thread its own mutable spline-evaluation workspace to prevent data races.
  • Force rebuilding when OpenMP settings change, and record compiler settings only after successful compilation.
  • Add CI compilation and numerical regression checks, plus documentation for thread counts and reduction memory requirements.

feectools

  • struphy compile --openmp now passes --openmp to psydac-accelerate, so the feectools kernels are threaded too (previously they were always compiled serially).
  • The feectools submodule points to the add-openmp-pragmas branch (Add OpenMP pragmas to the stencil and DOF kernels feectools#103), which adds OpenMP pragmas to the stencil dot, transpose, axpy and inner kernels (2D/3D) and the 3D DOF kernels.
  • The CI OpenMP step also runs pytest --pyargs feectools.linalg.tests.test_openmp.
  • install.rst mentions that the feectools kernels are compiled with OpenMP, and how to run their check.

Validation

  • Fresh OpenMP compilation of all 128 kernel modules.
  • Compiled results match serial reference loops with 1, 2, and 4 threads.
  • 575 tests passed, including pusher, SPH, compilation, and numerical parity tests.
  • console/tests/test_compile.py (16 tests) passes, including the new check that --openmp is forwarded to psydac-accelerate.
  • Ruff and whitespace checks passed.

Array reductions require temporary output arrays per thread; large problems may need additional memory and larger stack limits.

🤖 Generated with Claude Code

@max-models
max-models marked this pull request as draft October 12, 2026 15:26

@max-models max-models left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add openMP pragmas in feectools too

@max-models
max-models marked this pull request as ready for review October 13, 2026 15:20
@max-models
max-models requested a review from spossann October 13, 2026 15:20
spossann
spossann previously approved these changes Oct 13, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spossann
spossann previously approved these changes Oct 14, 2026
@spossann
spossann enabled auto-merge (squash) October 14, 2026 06:46
An uninitialized out (xp.empty) containing NaN/inf stayed NaN after
'*= 0.0', so test_spline_function_backends failed randomly on one MPI
rank while the other rank hung in the next Allreduce (CI timeout).
The test now pre-fills out with NaN to catch this deterministically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@spossann
spossann merged commit c9625e9 into devel Oct 14, 2026
28 checks passed
@spossann
spossann deleted the fix-openmp-pragmas branch October 14, 2026 08:28
spossann added a commit that referenced this pull request Oct 21, 2026
Corresponding feectools PR
struphy-hub/feectools#103

Fix OpenMP directives so all 128 Struphy kernel modules compile with GNU
Fortran using `struphy compile --openmp`, and compile the feectools
kernels with OpenMP as well.

- Repair parallel regions, variable scoping, and accumulation
reductions.
- Give each thread its own mutable spline-evaluation workspace to
prevent data races.
- Force rebuilding when OpenMP settings change, and record compiler
settings only after successful compilation.
- Add CI compilation and numerical regression checks, plus documentation
for thread counts and reduction memory requirements.

## feectools

- `struphy compile --openmp` now passes `--openmp` to
`psydac-accelerate`, so the feectools kernels are threaded too
(previously they were always compiled serially).
- The `feectools` submodule points to the `add-openmp-pragmas` branch
(struphy-hub/feectools#103), which adds OpenMP pragmas to the stencil
`dot`, `transpose`, `axpy` and `inner` kernels (2D/3D) and the 3D DOF
kernels.
- The CI OpenMP step also runs `pytest --pyargs
feectools.linalg.tests.test_openmp`.
- `install.rst` mentions that the feectools kernels are compiled with
OpenMP, and how to run their check.

## Validation

- Fresh OpenMP compilation of all 128 kernel modules.
- Compiled results match serial reference loops with 1, 2, and 4
threads.
- 575 tests passed, including pusher, SPH, compilation, and numerical
parity tests.
- `console/tests/test_compile.py` (16 tests) passes, including the new
check that `--openmp` is forwarded to `psydac-accelerate`.
- Ruff and whitespace checks passed.

Array reductions require temporary output arrays per thread; large
problems may need additional memory and larger stack limits.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants