Skip to content

Drop 17 unreachable obs4REF reference files, and correct the obs4ref solving claim #863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lewisjared wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Drop 17 unreachable obs4REF reference files, and correct the obs4ref solving claim #863

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3b14b77
chore(datasets): drop 17 unreachable obs4REF reference files
lewisjared Aug 13, 2026
85d4a4f
docs: correct the claim that obs4REF datasets participate in solving
lewisjared Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions changelog/863.docs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Corrected the record on the `obs4ref` source type.
The 0.16.2 entry for [#814](https://github.com/Climate-REF/climate-ref/pull/814) stated that obs4REF datasets participate in solving.
They do not yet: no diagnostic declares an `obs4REF` data requirement,
and the solver matches a requirement against its own source type only,
so a collection ingested as `obs4ref` is never selected.
Ingest the obs4REF collection with `--source-type obs4mips`, as the ingest guide describes.
7 changes: 7 additions & 0 deletions changelog/863.trivial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Dropped 17 obs4REF reference files that no diagnostic can select:
four superseded by a newer version of the same dataset
(LORA-1-1, the pre-2023 RAPID record and the two unversioned Hoffman files),
and thirteen variables that no data requirement asks for
(WECANN-1-0 `hfls`/`hfss`, TropFlux-1-0 `hfns`/`tas`/`tauv`, Hoffman-1-0 `fgco2`,
and WOA-23 `no3`/`o2`/`po4`/`si`/`ohc`/`ohcJm2`).
A solve over the same catalog plans exactly the same executions as before the removal.
7 changes: 7 additions & 0 deletions docs/getting-started/03-ingest.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ ref datasets ingest --source-type obs4mips $REF_CONFIGURATION/datasets/obs4ref

Replace `$REF_CONFIGURATION/datasets/obs4ref` with the directory used when [fetched the obs4REF data](02-download-datasets.md#fetch-obs4ref-datasets).

!!! note "Why not the `obs4ref` source type?"

An `obs4ref` source type also exists, but ingesting this collection with it will leave the data unused:
no diagnostic declares an `obs4REF` data requirement yet,
and the solver matches a requirement against its own source type only.
Use `obs4mips` until that changes.

## 2. Ingest CMIP6 data

To ingest CMIP6 files, point the CLI at a directory of netCDF files and set `cmip6` as the source type:
Expand Down
26 changes: 0 additions & 26 deletions packages/climate-ref/src/climate_ref/dataset_registry/obs4ref_reference.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,8 @@
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/ARCCSS/LORA-1-0/mon/mrro/50km/v20250902/mrro_mon_LORA-1-0_REF_gn_198001-201212.nc
obs4REF/ARCCSS/LORA-1-0/mon/mrro/gn/v20250902/mrro_mon_LORA-1-0_REF_gn_198001-201212.nc md5:c1d290a26b4c0a094724b3dc19230f73
obs4REF/ARCCSS/LORA-1-1/mon/mrro/gn/20250516/mrro_mon_LORA-1-1_REF_gn_198001-201212.nc md5:4cfbbfa3be9632b14de99b18066fbffe
obs4REF/CNES/AVISO-1-0/mon/zos/gn/v20210727/zos_mon_AVISO-1-0_PCMDI_gn_199301-201912.nc md5:91252303cb65548fee5ff42dd3024825
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/ColumbiaU/WECANN-1-0/mon/gpp/100km/v20250902/gpp_mon_WECANN-1-0_REF_gn_200701-201512.nc
obs4REF/ColumbiaU/WECANN-1-0/mon/gpp/gn/v20250902/gpp_mon_WECANN-1-0_REF_gn_200701-201512.nc md5:eb4fcde64bcdf605b4e72c95e9bbdd54
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/ColumbiaU/WECANN-1-0/mon/hfls/100km/v20250902/hfls_mon_WECANN-1-0_REF_gn_200701-201512.nc
obs4REF/ColumbiaU/WECANN-1-0/mon/hfls/gn/v20250902/hfls_mon_WECANN-1-0_REF_gn_200701-201512.nc md5:dd3bd01233eb17cb6cb566c9eed7349f
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/ColumbiaU/WECANN-1-0/mon/hfss/100km/v20250902/hfss_mon_WECANN-1-0_REF_gn_200701-201512.nc
obs4REF/ColumbiaU/WECANN-1-0/mon/hfss/gn/v20250902/hfss_mon_WECANN-1-0_REF_gn_200701-201512.nc md5:1605e0d962c3794206e979e51c04f2f9
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/DWD/ESACCI-CLOUD-AVHRR-AMPM-3-0/mon/clivi/50km/v20250723/clivi_mon_ESACCI-CLOUD-AVHRR-AMPM-3-0_ESMValTool-REF_gn_198201-201612.nc
obs4REF/DWD/ESACCI-CLOUD-AVHRR-AMPM-3-0/mon/clivi/gn/v20250723/clivi_mon_ESACCI-CLOUD-AVHRR-AMPM-3-0_ESMValTool-REF_gn_198201-201612.nc md5:1eaf07b5d5c7d70c2c40ae9eea4bfd4a
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/DWD/ESACCI-CLOUD-AVHRR-AMPM-3-0/mon/clt/50km/v20250723/clt_mon_ESACCI-CLOUD-AVHRR-AMPM-3-0_ESMValTool-REF_gn_198201-201612.nc
Expand Down Expand Up @@ -36,14 +31,10 @@ obs4REF/DWD/ESACCI-CLOUD-AVHRR-AMPM-3-0/mon/rsutcs/gn/v20250723/rsutcs_mon_ESACC
obs4REF/DoESS-UCI/GFED-5-0/mon/burntFractionAll/gr/v20260128/burntFractionAll_mon_GFED-5-0_REF_gr_199701-202012.nc md5:7cddfd3b1d5e50e22d2f2621c9b0393e
obs4REF/ESSO/TropFlux-1-0/mon/hfls/gn/v20210727/hfls_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:2f05191d6727068e1500d8d4ed90098a
obs4REF/ESSO/TropFlux-1-0/mon/hfls/gn/v20250415/hfls_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:e607167a08a2521b65e55eb186182003
obs4REF/ESSO/TropFlux-1-0/mon/hfns/gn/v20210727/hfns_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:7a9019e51a41d9e4ab1fcfb072d8ca8d
obs4REF/ESSO/TropFlux-1-0/mon/hfss/gn/v20210727/hfss_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:1da9d8fe862c61bc49c36c18b6527213
obs4REF/ESSO/TropFlux-1-0/mon/hfss/gn/v20250415/hfss_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:6f766ca0332a8e566c408d237571a924
obs4REF/ESSO/TropFlux-1-0/mon/tas/gn/v20210727/tas_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:a6057931b5f6bc000a44514a1a8c891f
obs4REF/ESSO/TropFlux-1-0/mon/tas/gn/v20250415/tas_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:527ab1b9becf2a793df558532eccfe69
obs4REF/ESSO/TropFlux-1-0/mon/tauu/gn/v20210727/tauu_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:7c73a3deed3403fa9d21caef3a4d988d
obs4REF/ESSO/TropFlux-1-0/mon/tauu/gn/v20250415/tauu_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:0822e2002e61472277116d38e5e19498
obs4REF/ESSO/TropFlux-1-0/mon/tauv/gn/v20210727/tauv_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:8abc7a724a7a297826e2f783a4ea14f9
obs4REF/ESSO/TropFlux-1-0/mon/ts/gn/v20210727/ts_mon_TropFlux-1-0_PCMDI_gn_197901-201707.nc md5:8697d3d7862f6e3b72bb5a161aa75ee8
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/FMI/SAGE-CCI-OMPS-v00008/mon/o3/1000km/v20250718/o3_mon_SAGE-CCI-OMPS-v0008_SAGE-CCI-OMPS_REF_gnz_198410-202212.nc
obs4REF/FMI/SAGE-CCI-OMPS-v00008/mon/o3/gnz/v20250718/o3_mon_SAGE-CCI-OMPS-v0008_SAGE-CCI-OMPS_REF_gnz_198410-202212.nc md5:e8f320054b6fc9a92a6e6ed94bb54563
Expand All @@ -65,30 +56,13 @@ obs4REF/NOAA-ESRL-PSD/20CR/mon/ts/gn/v20210727/ts_mon_20CR_PCMDI_gn_187101-20121
obs4REF/NOAA-NCEI/CMAP-V1902/mon/pr/gn/v20210727/pr_mon_CMAP-V1902_PCMDI_gn_197901-201901.nc md5:9d943d2dd0645850b616820f246aedf3
obs4REF/NOAA-NCEI/GPCP-2-3/mon/pr/gn/v20210727/pr_mon_GPCP-2-3_PCMDI_gn_197901-201907.nc md5:0877f014868b83547448f96c3e7c83e9
obs4REF/NOAA-NCEI/GPCP-2-3/mon/pr/gn/v20231205/pr_mon_GPCP-Monthly-3-2_RSS_gn_198301-202303.nc md5:6970c22443e2097c45de5db8947318eb
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/monC/no3/1x1degree/v20250902/no3_monC_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/monC/no3/gn/v20250902/no3_monC_WOA-23_REF_gn_200501-202212.nc md5:fd8c3af50d5dc6fedec730d38879d2b4
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/monC/o2/1x1degree/v20250902/o2_monC_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/monC/o2/gn/v20250902/o2_monC_WOA-23_REF_gn_200501-202212.nc md5:8bc541d7fe602b156054863867ad6cc7
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/monC/po4/1x1degree/v20250902/po4_monC_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/monC/po4/gn/v20250902/po4_monC_WOA-23_REF_gn_200501-202212.nc md5:abc02f6a8606f276b27fe1ceb2557fe4
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/monC/si/1x1degree/v20250902/si_monC_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/monC/si/gn/v20250902/si_monC_WOA-23_REF_gn_200501-202212.nc md5:8a42fb90cc49bfa83aa285d9d0edf31b
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/monC/so/1x1degree/v20251024/so_monC_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/monC/so/gn/v20251024/so_monC_WOA-23_REF_gn_200501-202212.nc md5:28a6049a9ef0435bce0a9e13c83fd77c
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/monC/thetao/1x1degree/v20251024/thetao_monC_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/monC/thetao/gn/v20251024/thetao_monC_WOA-23_REF_gn_200501-202212.nc md5:a76187904de6d66281af2267d3dddfa3
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/yr/ohc/site/v20250903/ohc_yr_WOA-23_REF_gm_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/yr/ohc/gm/v20250903/ohc_yr_WOA-23_REF_gm_200501-202212.nc md5:98f20f959d9d297d50ac81a4e697b1fb
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOAA-NCEO-OCL/WOA-23/yr/ohcJm2/1x1degree/v20250903/ohcjm2_yr_WOA-23_REF_gn_200501-202212.nc
obs4REF/NOAA-NCEO-OCL/WOA-23/yr/ohcJm2/gn/v20250903/ohcjm2_yr_WOA-23_REF_gn_200501-202212.nc md5:1e4e7a86227d98d15baad7a06130cb7e
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NOC/RAPID-2023-1a/mon/msftmz/site/v20250902/msftmz_mon_RAPID-2023-1a_REF_gm_200404-202302.nc
obs4REF/NOC/RAPID-2023-1a/mon/msftmz/gm/v20250902/msftmz_mon_RAPID-2023-1a_REF_gm_200404-202302.nc md5:f5ff38176be71f717e3c4d7460938571
obs4REF/NOC/RAPID/mon/msftmz/NA/20250516/msftmz_mon_RAPID_REF_NA_200404-202302.nc md5:e9ee555d923d39112dcbf591440000b3
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/NR/CCI-CryoClim-FSC-1/mon/snc/0.5x0.5degree/v20250519/snc_mon_CCI-CryoClim-FSC-1_BE_gn_198201-201906.nc
obs4REF/NR/CCI-CryoClim-FSC-1/mon/snc/gn/v20250519/snc_mon_CCI-CryoClim-FSC-1_BE_gn_198201-201906.nc md5:d53cddd61a90776f353ee8627dc992a5
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/UCI-ORNL/Hoffman-1-0/yr/fgco2/site/v20251117/fgco2_yr_Hoffman-1-0_REF_gm_1850-2010.nc
obs4REF/UCI-ORNL/Hoffman-1-0/yr/fgco2/gm/v20251117/fgco2_yr_Hoffman-1-0_REF_gm_1850-2010.nc md5:1808ca5fd11be62934ba062df63d1be1
# source: https://dap.ceda.ac.uk/neodc/obs_for_ref_v2/data/UCI-ORNL/Hoffman-1-0/yr/nbp/site/v20251117/nbp_yr_Hoffman-1-0_REF_gm_1850-2010.nc
obs4REF/UCI-ORNL/Hoffman-1-0/yr/nbp/gm/v20251117/nbp_yr_Hoffman-1-0_REF_gm_1850-2010.nc md5:ea86edee4cb869699cdb9b15ed957d2b
obs4REF/UCI-ORNL/Hoffman/yr/fgco2/gm/20250516/fgco2_yr_Hoffman_REF_gm_185007-201007.nc md5:93419dd752e0ae20e69b7db3f8ea9f5b
obs4REF/UCI-ORNL/Hoffman/yr/nbp/gm/20250516/nbp_yr_Hoffman_REF_gm_185007-201007.nc md5:1c0b93c81e2608c668ec39c45ca828de
Loading