You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Make examples and validation tests tell a missing backend from a failure - #246
This PR is the follow-up from the v0.4.0 release (#244, #245). Examples and validation tests now tell a backend that is missing from the build apart from a real failure. Before this change, a missing backend could make a test fail instead of skip, and a real failure could pass silently as a skip or as exit code 0.
Only examples/, tests/validation/, the examples guide and the CHANGELOG change. lib/ is unchanged.
Commits
Commit
Change
8a70882
Six examples report failures and exit nonzero (custom_source_terms, minimal_example, minimal_example_3d, runtime_comparison, solver_selection, steady_flow_multigrid). Each now checks every solver_init, solver_step and run_simulation_step status, skips a solver whose backend the build or machine lacks, and exits 1 on any other failure. Details for two of them below.
4551012
Validation tests skip only a missing backend. The new tests/validation/solver_availability.h creates solvers with cfd_solver_create_checked() and treats only CFD_ERROR_UNSUPPORTED as unavailable. Every harness that may be given an optional backend now uses it. tg_run_simulation() also checks solver_init and solver_step.
a270767
failure_reason() reports a failure with its own message. It now uses the library's stored message only when its status matches the failure; otherwise an earlier, unrelated message could be printed. Found by /code-review; also applied to the same helper in performance_comparison (from #244).
solver_selection changes:
It now checks cfd_backend_is_available() before switching to the AVX2 solver, because a failed switch only shows at the first step.
It creates the directory for its VTK file. Before, it wrote into a missing output/ and still printed "Output written to".
runtime_comparison is rebuilt on two helpers, create_sim() and time_solver(). Before, it stepped NULL simulations and divided by zero timings; a missing backend now prints n/a.
Validation tests and the behaviour each had before this PR:
poiseuille_3d_reference.h, taylor_green_3d_reference.h, test_poiseuille_flow, test_turbulent_channel, the cavity harness's scalar site
Exempt: they only create the scalar projection, and a NULL there already fails.
Testing
Two builds were used: the default options (AVX2 off) and one configured with -DCMAKE_DISABLE_FIND_PACKAGE_OpenMP=ON.
Affected validation tests: 20/20 pass on the default build and 21/21 without OpenMP. Without OpenMP, BackwardFacingStepRe400Test now skips.
Probe, validation: a misspelled solver in the Taylor-Green consistency test now fails with "not created: Resource not found". Before, it printed SKIPPED.
Examples: all six exit 0 on both builds. steady_flow_multigrid ran at n = 33 (see below).
Probe, examples: with a negative dt, minimal_example, runtime_comparison and solver_selection exit 1 and name the failing step.
Fast suite:ctest -LE "cross-arch|validation" passes 101/101 (Debug).
/pr-ready: passed, including /code-review medium; its one finding is fixed in a270767.
The full 129×129 steady_flow_multigrid run timed out at 600 s, so it was checked at n = 33 instead. This laptop is in the vendor "Silent" power profile with the CPU at 2.2 GHz, which made every timing on 10/4 roughly 5–6× slower. Before that the same run took 104 s. The change to that example only adds a status check on its warm-up steps.
Not in this PR (library issues found along the way)
simulation_set_solver() ignores solver_init()'s status, so simulation_set_solver_by_name() reports success when the new solver refuses to initialize, after the old solver has already been destroyed.
simulation_list_solvers() returns a hardcoded list. It omits RK2 and RK4, and lists the _gpu names even without CUDA.
write_vtk_flow_field() and the other VTK writers return void, so a caller cannot detect a failed write.
Six examples discarded solver_init, solver_step and run_simulation_step
statuses, so a solver that stopped working still printed results and
exited 0, and the release gate's example run could not see it. Each now
checks every status, skips a solver whose backend the build or machine
lacks, and exits 1 on anything else.
solver_selection checks cfd_backend_is_available() before switching to the
AVX2 solver: simulation_set_solver_by_name() reports success even when the
new solver's init fails, so the failure would only show at the first step.
It also creates the directory for its direct-API VTK file, which it wrote
into a missing output/ while printing "Output written to". runtime_comparison
is rebuilt on create_sim() and time_solver(), which separate a missing
backend from a failure; it used to step NULL simulations and divide by zero
timings, and now prints n/a.
Checked on a default build (AVX2 off) and one without OpenMP: all six exit
0, and with a negative dt injected minimal_example, runtime_comparison and
solver_selection exit 1 with the failing step named.
Several harnesses treated any NULL from cfd_solver_create() as "not
available" and skipped, which also hid an unregistered name or an
allocation failure (test_solver_architecture, test_taylor_green_decay,
test_natural_convection, test_ghia_projection_gpu, and the Taylor-Green
backend consistency test, which skipped on any failure at all), while
BackwardFacingStepRe400Test failed instead of skipping on a build without
OpenMP, where the _omp solvers are not registered and creation reports
CFD_ERROR_NOT_FOUND.
tests/validation/solver_availability.h creates solvers with
cfd_solver_create_checked(), which reports a missing backend as
CFD_ERROR_UNSUPPORTED before the name lookup, and marks only that status
unavailable. Every harness that may be handed an optional backend uses it,
including the two cavity-harness sites #245 fixed inline. tg_run_simulation()
also stops ignoring solver_init() and solver_step() statuses.
Exempt: poiseuille_3d_reference.h, taylor_green_3d_reference.h,
test_poiseuille_flow, test_turbulent_channel and the cavity harness's
scalar-only site only ever create the scalar projection, and a NULL there
already fails.
Checked on a default build and on one without OpenMP: the 20 and 21 affected
validation tests pass, BackwardFacingStepRe400Test now skips without
OpenMP, and a misspelled solver in the Taylor-Green consistency test fails
("not created: Resource not found") where it used to print SKIPPED.
failure_reason() printed cfd_get_last_error() whenever a message was stored,
so a failure that set none was reported with an earlier, unrelated message,
such as a backend skipped further up the run. It now uses the stored message
only when cfd_get_last_status() matches the status being reported.
Found by /code-review on this branch in solver_selection; the same helper in
performance_comparison (from #244) gets the same fix.
Review finding (#246): cfd_solver_create() reports a factory's NULL as
CFD_ERROR_NOMEM only while the last status is still CFD_SUCCESS, and that
status is sticky. An UNSUPPORTED left over from an earlier solver made a later
allocation failure read as unavailable, so validation_create_solver() skipped
exactly the failure it exists to expose.
Review findings (#246): runtime_comparison and solver_selection read "not in
the registry" as "backend unavailable", so a misspelled or removed solver
printed n/a or SKIPPED and the example still exited 0. solver_selection also
ignored a failed registry allocation, after which every solver read as
skipped and the run reported zero failures.
Both now probe with cfd_solver_create_checked(), which refuses an unavailable
backend with CFD_ERROR_UNSUPPORTED before the name lookup, and skip only that
status; any other creation failure, an unknown name included, is a failure. A
NULL registry in solver_selection is reported and counted as one.
A real initialization or step failure also sets ran to 0, so this prints n/a for both missing backends and failed runs. That makes the results table contradict the new distinction even though stderr and the exit code report the failure. Preserve each solver's unavailable flag in benchmark_result and print FAILED when !ran && !unavailable.
Preserve registry allocation failure diagnostics
examples/solver_selection.c:290
If registry allocation fails, cfd_registry_register_defaults(NULL) is a no-op and the subsequent solver creation overwrites the original CFD_ERROR_NOMEM with CFD_ERROR_INVALID. This new diagnostic then reports an invalid argument rather than the allocation failure. Check registry immediately, as run_solver_comparison() already does, before registering or creating a solver.
Unknown solver names misclassified as unsupported
tests/validation/solver_availability.h:41
cfd_solver_create_checked() infers the backend from name substrings before consulting the registry (lib/src/api/solver_registry.c:276-297, 2110-2122). On a build without OpenMP, for example, an unregistered name such as projection_omp_typo therefore returns CFD_ERROR_UNSUPPORTED and is marked unavailable here, contradicting this helper's contract that unregistered names fail. Validate that type is a recognized optional solver before treating UNSUPPORTED as skippable, or expose registry metadata that can distinguish a known-but-unbuilt solver from an unknown name.
Handle all solver initialization failures
docs/guides/examples.md:243
This example handles only the unsupported case. If initialization returns NULL for CFD_ERROR_NOT_FOUND, CFD_ERROR_NOMEM, or another real failure, the snippet falls through with a NULL simulation instead of reporting the error, which undermines the distinction this section documents. Handle every NULL result and treat only CFD_ERROR_UNSUPPORTED as a skip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is the follow-up from the v0.4.0 release (#244, #245). Examples and validation tests now tell a backend that is missing from the build apart from a real failure. Before this change, a missing backend could make a test fail instead of skip, and a real failure could pass silently as a skip or as exit code 0.
Only
examples/,tests/validation/, the examples guide and the CHANGELOG change.lib/is unchanged.Commits
8a70882custom_source_terms,minimal_example,minimal_example_3d,runtime_comparison,solver_selection,steady_flow_multigrid). Each now checks everysolver_init,solver_stepandrun_simulation_stepstatus, skips a solver whose backend the build or machine lacks, and exits 1 on any other failure. Details for two of them below.4551012tests/validation/solver_availability.hcreates solvers withcfd_solver_create_checked()and treats onlyCFD_ERROR_UNSUPPORTEDas unavailable. Every harness that may be given an optional backend now uses it.tg_run_simulation()also checkssolver_initandsolver_step.a270767failure_reason()reports a failure with its own message. It now uses the library's stored message only when its status matches the failure; otherwise an earlier, unrelated message could be printed. Found by/code-review; also applied to the same helper inperformance_comparison(from #244).solver_selectionchanges:cfd_backend_is_available()before switching to the AVX2 solver, because a failed switch only shows at the first step.output/and still printed "Output written to".runtime_comparisonis rebuilt on two helpers,create_sim()andtime_solver(). Before, it stepped NULL simulations and divided by zero timings; a missing backend now printsn/a.Validation tests and the behaviour each had before this PR:
test_solver_architecture,test_taylor_green_decay,test_natural_convection,test_ghia_projection_gpu, Taylor-Green backend consistencyBackwardFacingStepRe400Testprojection_ompis not registered.test_quiescent_3dpoiseuille_3d_reference.h,taylor_green_3d_reference.h,test_poiseuille_flow,test_turbulent_channel, the cavity harness's scalar siteTesting
Two builds were used: the default options (AVX2 off) and one configured with
-DCMAKE_DISABLE_FIND_PACKAGE_OpenMP=ON.BackwardFacingStepRe400Testnow skips.steady_flow_multigridran at n = 33 (see below).dt,minimal_example,runtime_comparisonandsolver_selectionexit 1 and name the failing step.ctest -LE "cross-arch|validation"passes 101/101 (Debug)./pr-ready: passed, including/code-review medium; its one finding is fixed ina270767.The full 129×129
steady_flow_multigridrun timed out at 600 s, so it was checked at n = 33 instead. This laptop is in the vendor "Silent" power profile with the CPU at 2.2 GHz, which made every timing on 10/4 roughly 5–6× slower. Before that the same run took 104 s. The change to that example only adds a status check on its warm-up steps.Not in this PR (library issues found along the way)
simulation_set_solver()ignoressolver_init()'s status, sosimulation_set_solver_by_name()reports success when the new solver refuses to initialize, after the old solver has already been destroyed.simulation_list_solvers()returns a hardcoded list. It omits RK2 and RK4, and lists the_gpunames even without CUDA.write_vtk_flow_field()and the other VTK writers returnvoid, so a caller cannot detect a failed write.🤖 Generated with Claude Code