Beltrami Validation CI Workflow

[cenamiller]

We need to integrate a check of these variables for the small (16kmx16km) Beltrami case:
RMSE U V W Vorticity Divergence
Small 0.03396 0.03396 0.03957 7.132e-5, 8.618e-5
Medium 0.03284 0.03284 0.04014 4.441e-5 4.477e-5
Large 0.03265 0.03265 0.04025 3.796e-5 3.256e-5

This check will be included in the CI pipeline, so that we can be confident our optimizations aren't producing bad results. NOTE: The Beltrami case is not an end-to-end test of the workflow and doesn't cover input/preprocessing or (of course) the RadarQC code

[cenamiller]

set param output_txt=TRUE
u,v,w,vorticity, divergence in samurai_XYZ_analysis.out? Need to check meaning of different output files (analysis/increment/background).

[cenamiller]

Add residual check as super basic check in meantime - can also be used for other test cases besides Beltrami

[cenamiller]

Currently code produces a residual number and prints out the number of iterations to solve. This number would be different for each test case. Supreeth currently has a python script that does this for Beltrami - need to find and add to CI workflow. Supreeth will add to CI workflow.

[cenamiller]

From Supeeth (Thanks!)

The validation script /glade/work/ssuresh/apar/samurai_validation/ is here in this directory.

Example usage: Usage: python validation.py refernce_log_file run_log_file

[cenamiller]

Supreeth will send docker file
need to rebuild it, and add something like 'python validation.py log1 log2' to the end
Don't need to change workflow yaml

in dockerfile, need to pipe output of ./samurai -params /app/samurai/ncar_scripts/TDRP/beltrami.tdrp
" > /app/run_samurai.sh to specific file name to be used

John will supply reference file

[cenamiller]

cenamiller@437a39d

This commit has the validation workflow EXCEPT THE GOLDEN REFERENCE FILE THAT CONTAINS THE BELTRAMI ANALYTICAL SOLUTION. A fake reference file (ncar_scripts/validation/beltrami/samurai_XYZ_wind_analysis_ref.nc) was used to test different scenarios.

ncar_scripts/validation/beltrami/bel_val_nc.py compares a reference netcdf file and the results of the containerized beltrami test run with respect to the RMSE of variables ['U', 'V', 'W', 'VORT', 'DIV']. The variables chosen for comparison can easily be changed in the script, along with the threshold RMSE for each variable. If the RMSE of a variable is above the threshold, the script returns an error.

Current settings:
rmse_thresholds = {
'U': 0.03396,
'V': 0.03396,
'W': 0.03957,
'VORT': 7.132e-5,
'DIV': 8.618e-5
}