This project contains scripts and tools used to read ACS bias-corrected data and evaluate the DWI (Dry Windy Index) data in parallel and perform various evaluations on it.
- Evaluate and write netcdf files for BARRA (reference) data and projections data from multiple GCMs and RCMs
- Produce area-averaged (NRM regions)stats (median, 90th, 95th and 99th percentile) and bar plots
- Produce Q-Q plots at a location
- Produce intervariable correlations (T-RH, T-WS and WS-RH) netcdf files and plots
- Produce signal ratio and write netcdfs and plots
- Signal ratio For a metric (M) for example M=bias. e.g. M(UNI,OBS) = (data_qme-data_ref).mean(dim="time") Effect of doing BC at all (relative to RAW):\ [Δ_"UNI" =M("UNI" ,"OBS" )-M("RAW" ,"OBS" ), Δ_"MULTI" =M("MULTI" ,"OBS" )-M("RAW" ,"OBS" )](Negative is “good” if the metric is an error.) Difference between BC methods:\ [D_"UM" =M("MULTI" ,"OBS" )-M("UNI" ,"OBS" )] Which matters more? Compute a simple signal ratio per grid cell:\ [R=|D_"UM" |/max(|Δ_"UNI" |,,|Δ_"MULTI" |) ] If R ≪ 1, the choice of BC method matters little vs. doing any BC. If R ~ 1 (or >1), the two BC methods differ as much as (or more than) the act of bias correcting.
- Produce ensemble mean, stddev and IQR of signal ratio
- Produce signal ratio of intervariable correlations for each GCM, RCM combination. netcdfs and plots.
- Produce netcdf and plots of ensemble mean, stddev and IQR of the signal ratio of the intervariable correlations
- Parallel reading of ACS bias-corrected data
# Clone the repository
git clone git@git.nci.org.au:jp0715/acs_bc_dwi_evaluation.git
cd acs_bc_dwi_evaluationSummary of DWI scripts
This details the scripts in /g/data/mn51/users/jp0715/acs/code
-
- dwi_calc_write_bc_netcdf.ipynb and dwi_calc_write_bc_netcdf.py
-
1.1. reads all the datasets using a parellel read routine based on xr.open_mfdataset (parallel_read_acs_data and parallel_read_function)
-
1.2. calculates the DWI for the historical data (1960-2014) and the ssp370 data (2015-2099)
-
1.3. writes yearly data (due to memory constraints) for the MRNBC, QME and NOBC (non bias-corrected data or RCM data) to /g/data/ia39/users/jp0715/{RCM}/{GCM}/{SCENARIO}/{EXPERIMENT}/BARPA-R/{BC_METHOD}/BARRAR2. For example: /g/data/ia39/users/jp0715/BOM/ACCESS-CM2/ssp370/r4i1p1f1/BARPA-R/MRNBC/BARRAR2/*dwi_ffdi{YEAR}.nc
-
1.4. it takes four arguments as input --inst $INST --model $MODEL --bc $BC --scen $SCEN. The institution (BOM or CSIRO). The model (GCM), the BC method (QME or MRNBC) and the scenario (historical or ssp370)
-
1.5. it uses two auxillary routines to run: run_calc_write_bc_netcdf.pbs (which sets up the dask cluster and calls the python script) and submit_calc_write_bc_netcdf_jobs.sh (which loops through all the insts, models, bc methods and scenarios and submits to the queue via qsub)
-
1.6. NOTE! that the BC'd and non BC'd data are written separately and the appropriate line needs to be changed in line 243 of dwi_calc_write_bc_netcdf.py
-
1.7 DWI calculated for BARRA data in dwi_calc_write_barra_netcdf.ipynb.
-
- dwi_calc_bias.py and dwi_calc_bias.ipynb and dwi_calc_bias_quantiles.py (and corresponding *.ipynb scripts)
-
2.1. dwi_calc_bias.py is the earlier version that only calculates the bias of the mean (not the quantiles!)
-
2.2. produces csv files in /g/data/ia39/users/jp0715/dwi_rmse_calcs/
-
2.3. NOTE! These were moved into the directory metrics_summary_old
-
2.4. Contains script to produce NRM regions (dwi_calc_bias.ipynb)
-
2.5. The main script to compute the csv files is dwi_calc_bias_quantiles.py, which calculates the median, 90th, 95th and 99th percentiles of the DWI for an NRM region.
-
2.6. The header for the output files is: INST,GCM,RCM,EXPT,BC_METHOD,NRM,BIAS,RMSE,MAE,BIAS50,RMSE50,MAE50,BIAS90,RMSE90,MAE90,BIAS95,RMSE95,MAE95,BIAS99,RMSE99,MAE99
-
2.7. The filename is (for example): metrics_summary_quantiles_BOM_ACCESS-CM2_BARPA-R_expt.csv
-
2.8. NOTE, the calculations were only done for the historical data (the expt) value so all these files have the suffix expt. The root data directory for the expt is set explicitly in line 247. No evaluation of bias possible for projections.
-
2.9. NOTE: dwi_calc_bias,ipynb produces the NRM NRM regions (Figure 2)
-
- Dwi_analyse_bias.ipynb and dwi_
-
analyse_bias_quantiles.ipynb
-
3.1. Dwi_analyse_bias unused. Was an earlier version to analyse bias before the quantiles were produced.
-
3.2. Dwi_analyse_bias_quantiles.ipynb. Used to produce error box plots (Figure 3)
-
3.3. . Use cell 6 (which has the savefig line uncommented). Uses:bom_colors = sns.light_palette("red", len(bom_models)); csiro_colors = sns.light_palette("blue", len(csiro_models))
-
3.4. Uses files in /g/data/ia39/users/jp0715/dwi_rmse_calcs
-
-
DWI_qq_plots.ipynb
-
-
4.1. Reads BARRA data in /g/data/ia39/australian-climate-service/release/CORDEX/output-CMIP6/DD/AUST-05i/BOM/ERA5/historical/hres/BARRAR2/v1/day
-
4.2. Reads BC data in /g/data/kj66/CORDEX/output-CMIP6/bias-adjusted-output/AUST-05i/
-
4.3. Reads NOBC data in /g/data/kj66/CORDEX/output-CMIP6/DD/AUST-05i/
-
4.4. Uses parallel_read_function to read all models (like was developed in acs_sandbox.ipynb to read all the models (ACCESS-CM2 and ACCESS-ESM1)
-
4.5. Calculates dew point depression using internal function (same as one in acs_calc_functions.py)
-
4.6. Produces qq plots (Figure 4)
-
4.7. Note that we multiply the wind speed by 3.6 to convert it from m/s to km/hr. The calculations of the netcdf files of the DWI didn't have this conversion but it was only used to produce the plots in Figure 3, which are error metrics and don't depend on the conversion.
-
4.8. Note, if I need to place units on this can be easily added to cell 43 (second last cell)
-
- Dwi_calc_correlations_and_write_netcdf.py and .ipynb
-
5.1. Calculates correlations for each variable pair: T-RH, T_WSPD and RH-WSPD and writes netcdf files (dwi_correlation.nc). The ipynb file also has some exploratory plotting at the end of the file.
-
5.2. Writes data to /g/data/ia39/users/jp0715/correlation_calcs/ for each INST. GCM. BC_METHOD combo. For example to /g/data/ia39/users/jp0715/correlation_calcs/BOM/ACCESS-CM2/historical/r4i1p1f1/BARPA-R/MRNBC/dwi_correlation.nc
-
5.3. Ha helper functions run_calc_correlations_and_write_netcdf.pbs and submit_calc_correlations_and_write_netcdf.sh to submit the jobs in parallel for each INST, GCM and RCM combination.
-
- Dwi_analyse_correlation.ipynb
-
6.1. From the netcdf files output by the previous routines (dwi_correlation.nc files)
-
6.2. Calculates the bias (model-BARRA) and writes netcdf files for each INST, GCM,RCM, BC_METHOD combination to /g/data/ia39/users/jp0715/correlation_calcs/bias/bias_corr.nc
-
6.3. Reads in these files and calculates the nesemble bias for each bias correction method. These files are in (for example): /g/data/ia39/users/jp0715/correlation_calcs/bias/ensemble/MRNBC/ensemble_bias_corr.nc
-
6.4. Also plots these files as individual files (in e.g. /g/data/ia39/users/jp0715/correlation_calcs/bias/ensemble/MRNBC). Plots ofr individual correlations and aslo as all three correlations for one BC_METHOD (ensemble_bias_corrr_all.png)
-
6.5. Note it produces plots for Figure 5. However, this plot was combined with individual images from ensemble_bias_corr_all.png (without using labels as the program does) using Inkscape. If needs to be regenerated may need to get rid of labels. Actually the inkscape file may have been combined from the nine individual files! I think that is the case!
-
- Dwi_calc_signal_ratio.ipynb
-
7.1. NOT USED. SEEMS TO BE A VERSION OF calc_bias_quantiles.ipynb. NO SIGNAL RATIO CALCULATIONS DONE. REMOVED FROM GIT REPO!
-
- Dwi_calc_signal_ratio_and_write_netcdf.ipynb and dwi_calc_signal_ratio_and_write_netcdf.py
-
8.1. Calculates the signal ratio from the dwi_ffdi.nc files (produced in dwi_calc_write_bc_netcdf.py and writes netCDF files in /g/data/ia39/users/jp0715/signal_ratio_calcs/. For example in: /g/data/ia39/users/jp0715/signal_ratio_calcs/BOM/ACCESS-CM2/r4i1p1f1/BARPA-R/R_signal.nc
-
8.2. Also plots the signal ration for each INST, GCM, RCM combination in /g/data/ia39/users/jp0715/signal_ratio_calcs/plots/.
-
8.3. This produces the plots the figures used for Figure 6. For example in /g/data/ia39/users/jp0715/signal_ratio_calcs/plots/BOM/signal_ratio_BOM_ACCESS-CM2_r4i1p1f1_BARPA-R.png
-
- dwi_ensemble_signal_ratio_all_models.ipynb
-
9.1. Computes the ensemble mean, STDDEV and IQR for the ensemble of BOM and CSIRO models. Outputs a netcdf file to /g/data/ia39/jp0715/signal_ratio_calcs/ensemble_all
-
9.2. Produces plots of the ensemble mean. Stddev and IQR. Script to produce Figure 7
-
9.3. May need to change plot to just produce mean and standard deviation.
-
- Dwi_calc_signal_ratio_corr_and_write_netcdf.ipynb and dwi_calc_signal_ratio_corr_and_write_netcdf.py
-
10.1. Calculates the signal ratio for each INST, GCM and RCM combination and also plots this information. This is the signal ratio for the intervariable correlation
-
10.2. Note that there are helper functions to run in parallel for all INST, GCM and RCM combinations. These are run_calc_signal_ratio_corr_and_write_netcdf.pbs and submit_calc_signal_ratio_corr_and_write_netcdf.sh
-
10.3. Netcdf files are written to /g/data/ia39/users/jp0715/signal_ratio_corr_calcs/. For example to /g/data/ia39/users/jp0715/signal_ratio_corr_calcs/BOM/ACCESS-CM2/BARPA-R. The variables in the file are R-corr_T_RH (and corresponding gor T_WS and RH_WS)
-
10.4. The plots are written to /g/data/ia39/users/jp0715/signal_ratio_corr_calcs/plots
-
10.5. These are the plots for Figure 8. Note that they were combined using Inkscape.
-
- dwi_ensemble_signal_ratio_corr_all_models.ipynb
-
11.1. Makes ensemble of the signal ratio of the correlation for each variable combination. Produces a netcdf of the mean, STDDEV and IQR of the ensemble signal ratio. Written to /g/data/ia39/users/jp0715/signal_ratio_corr_calcs/ensemble_all/signal_ratio_corr_ensemble_all_models.nc
-
11.2. Also plots the results to /g/data/ia39/users/jp0715/signal_ratio_corr_calcs/plots_ensemble
-
11.3. Produces the plots for Figure 9. Note that Figure 9 is composited from the three output pngs using Inkscape.