GEOPY-2731: Replace InputFile for BaseUIJson class - #460
Open
domfournier wants to merge 23 commits into
Open
Conversation
# Conflicts: # docs/fundamentals/images/distributed_parallelization.svg
github-actions
Bot
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates simpeg-drivers to align with the newer geoh5py UIJson APIs by migrating away from legacy UIJson helpers (BaseUIJson, load_ui_json_as_dict, validate_out_group) and updating drivers/tests/assets accordingly.
Changes:
- Replace legacy UIJson reading/writing and output-group validation with
UIJson/SimPEGDriversUIJsonandto_ui_json_group(...)patterns across drivers and tests. - Update EM option models (TDEM/FDEM) with stricter typing and legacy-unit handling hooks, and refactor some out-group option updates.
- Repoint git dependencies/locks to
feature/uijsonbranches and refresh conda-lock outputs.
Reviewed changes
Copilot reviewed 78 out of 78 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/uijson_test.py | Adjust legacy UIJson test flow and driver instantiation; legacy version parametrization changed. |
| tests/run_tests/driver_mvi_test.py | Migrate UIJson usage (UIJson.read) and update expected output-group naming. |
| tests/run_tests/driver_joint_surveys_test.py | Migrate UIJson usage and out_group creation; fix expected group names. |
| tests/run_tests/driver_joint_pgi_homogeneous_test.py | Update expected group naming for magnetic forward outputs. |
| tests/run_tests/driver_joint_cross_gradient_test.py | Migrate UIJson usage/out_group creation and update expected group names. |
| tests/run_tests/driver_grav_test.py | Migrate UIJson usage/out_group creation and adjust test driver setup indentation. |
| tests/run_tests/driver_dc_2d_test.py | Update expected DC group names to include “(DC)”. |
| tests/run_tests/driver_dc_2d_rotated_gradients_test.py | Update expected DC forward group name to include “(DC)”. |
| tests/run_tests/init.py | Remove redundant header comment block. |
| tests/plate_simulation/runtest/sweep_test.py | Switch plate sweep tests to SimPEGDriversUIJson for template reads/writes. |
| tests/plate_simulation/runtest/match_test.py | Switch plate simulation tests to UIJson reads/writes. |
| tests/plate_simulation/runtest/leroi_test.py | Change starting_model test input to a scalar. |
| tests/plate_simulation/runtest/driver_test.py | Migrate to UIJson, adjust out-group validation approach via driver. |
| simpeg_drivers/utils/utils.py | Remove legacy UIJson dict loader; use SimPEGDriversUIJson for parallelization/Dask config. |
| simpeg_drivers/utils/tile_estimate.py | Centralize out-group option updates via update_out_group_options(). |
| simpeg_drivers/utils/synthetics/options.py | Exclude several mesh option defaults from serialization via Field(..., exclude=True). |
| simpeg_drivers/utils/meshes.py | Switch automesh out_group creation to ui_json.to_ui_json_group(...). |
| simpeg_drivers/uijson.py | Base UIJson class now subclasses UIJson; add parallelization fields + from_dict(validate=...). |
| simpeg_drivers/plate_simulation/sweep/options.py | Declare _ui_json_class = SimPEGDriversUIJson for sweep options. |
| simpeg_drivers/plate_simulation/sweep/driver.py | Drop legacy validate_out_group helper and use driver validation; set _out_group_class. |
| simpeg_drivers/plate_simulation/options.py | Declare _ui_json_class = SimPEGDriversUIJson for plate simulation options. |
| simpeg_drivers/plate_simulation/match/options.py | Declare _ui_json_class = SimPEGDriversUIJson for match options. |
| simpeg_drivers/plate_simulation/match/driver.py | Drop legacy validate_out_group helper and remove custom start implementation. |
| simpeg_drivers/plate_simulation/driver.py | Replace legacy InputFile path with UIJson.from_dict + updated params build flow. |
| simpeg_drivers/options.py | Set _ui_json_class; remove custom write_ui_json; exclude some line-selection fields. |
| simpeg_drivers/electromagnetics/time_domain/options.py | Add stricter typing for TDEM units and legacy-unit conversion validator hook. |
| simpeg_drivers/electromagnetics/frequency_domain/options.py | Add FEMSurvey typing and allow None conversion key. |
| simpeg_drivers/driver.py | Replace legacy ui.json loading and out_group validation flow; update CLI entrypoint parsing. |
| simpeg_drivers/depth_of_investigation/sensitivity_cutoff/driver.py | Remove unused validate_out_group import. |
| simpeg_drivers-assets/uijson/tipper_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/tipper_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/tile_estimator.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/tdem1d_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/tdem1d_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/tdem_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/tdem_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/sensitivity_cutoff.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/plate_sweep.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/plate_simulation.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/plate_match.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetotellurics_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetotellurics_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetic_vector_pde_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetic_vector_pde_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetic_vector_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetic_vector_forward.ui.json | Set template version to 0.0.0 (and “Magnetic Forward” title). |
| simpeg_drivers-assets/uijson/magnetic_scalar_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/magnetic_scalar_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/joint_surveys_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/joint_petrophysics_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/joint_cross_gradient_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/induced_polarization_3d_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/induced_polarization_3d_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/induced_polarization_2d_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/induced_polarization_2d_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/gravity_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/gravity_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/fdem1d_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/fdem1d_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/fdem_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/fdem_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/direct_current_3d_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/direct_current_3d_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/direct_current_2d_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/direct_current_2d_forward.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/apparent_conductivity_inversion.ui.json | Set template version to 0.0.0. |
| simpeg_drivers-assets/uijson/apparent_conductivity_forward.ui.json | Set template version to 0.0.0. |
| pyproject.toml | Point git dependencies to feature/uijson branches. |
| environments/py-3.13-win-64.conda.lock.yml | Refresh lock with updated git SHAs/versions. |
| environments/py-3.13-win-64-dev.conda.lock.yml | Refresh dev lock with updated git SHAs/versions. |
| environments/py-3.13-linux-64.conda.lock.yml | Refresh lock with updated git SHAs/versions. |
| environments/py-3.13-linux-64-dev.conda.lock.yml | Refresh dev lock with updated git SHAs/versions. |
| environments/py-3.12-win-64.conda.lock.yml | Refresh lock with updated git SHAs/versions. |
| environments/py-3.12-win-64-dev.conda.lock.yml | Refresh dev lock with updated git SHAs/versions. |
| environments/py-3.12-linux-64.conda.lock.yml | Refresh lock with updated git SHAs/versions. |
| environments/py-3.12-linux-64-dev.conda.lock.yml | Refresh dev lock with updated git SHAs/versions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+63
to
+84
| @field_validator("data_units", mode="before") | ||
| @classmethod | ||
| def validate_legacy_units(cls, value): | ||
| if "Airborne" not in value or "Ground" not in value: | ||
| original = value | ||
| if "dB/dt" in value: | ||
| value = "Airborne dB/dt (V/Am^4)" | ||
|
|
||
| elif "B" in value: | ||
| value = "Airborne B (T/Am^2)" | ||
|
|
||
| else: | ||
| value = "Ground B (T/A)" | ||
|
|
||
| logger.warning( | ||
| "Legacy data units '%s' converted to '%s'." | ||
| "Consider updating your ui.json file to the newest version.", | ||
| original, | ||
| value, | ||
| ) | ||
|
|
||
| return value |
Comment on lines
+664
to
668
| ui_json = SimPEGDriversUIJson.read(json_path) | ||
|
|
||
| n_workers = ui_json.get("n_workers", None) | ||
| n_threads = ui_json.get("n_threads", None) | ||
|
|
||
| if n_workers is None: | ||
| if ui_json.n_workers is None or ui_json.n_threads is None: | ||
| cpu_count = multiprocessing.cpu_count() | ||
|
|
Comment on lines
+701
to
707
| n_workers = ui_json.n_workers or n_workers | ||
| n_threads = ui_json.n_threads or n_threads | ||
| save_report = ui_json.performance_report or generate_report | ||
|
|
||
| if (n_workers is not None and n_workers > 1) or n_threads is not None: | ||
| if (n_workers is not None and n_workers > 1) and n_threads is not None: | ||
| cluster = LocalCluster( | ||
| processes=True, |
Comment on lines
+265
to
266
| @pytest.mark.parametrize("version", ["v0.2.1"]) # , "v0.3.0", "v0.4.0"]) | ||
| def test_legacy_uijson(tmp_path: Path, caplog, version): |
| @@ -1,5 +1,5 @@ | |||
| { | |||
| "version": "0.5.1", | |||
| "version": "0.0.0", | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GEOPY-2731 - Replace InputFile for BaseUIJson class