Fail bindiff on shape mismatches; optionally reject extra variables (#1370)
* Add opt-in strict NetCDF structure checks to bindiff
* Fail bindiff comparisons on any shape mismatch
* Trim strict bindiff PR scope
* Clarify strict bindiff variable matching
* Making shape mismatches only a warning by default again - this is in case someone wants to change how metadata is stored in the netcdf. For example, new parameter is added to clubb_params, now the netcdf storage for clubb_params is 1 element bigger and the python will detect the sizes are not comparable (shape error) - we don't want this to cause this bindiff script to fail.
* Enable JAX microphysics with Fortran-shaped interfaces
Connect KK and Morrison cores through microphysics initialization, PDF preparation, transport, sedimentation, statistics, and delayed feedback. Retain explicit guards and dummy interfaces for unsupported features.
Add DYCOMS2 RF02 DS to the JAX-versus-Fortran harness at 240 timesteps. The four-column CPU double comparison passes at the existing 1e-7 threshold. Morrison validation remains limited to synthetic interface checks.
* Add RICO and LBA KK microphysics comparisons
Configure per-case timestep limits, timesteps and namelist overrides in the JAX-versus-Fortran harness. Add RICO at 380 steps of 60 seconds and the LBA warm-rain KK variant for its full 360-step, six-hour duration.
Match Fortran multicolumn hydrometeor hole-filling grid metrics, keep lazy KK module constants outside JAX tracing, resolve LBA radiation tables from the configured forcing path, and use the source LBA surface-flux pi constant. Update the existing surface-flux expectation and document coverage.
Validation: serial live comparisons pass for RICO 380, DYCOMS2 RF02 DS 240, and LBA 360 at the unchanged 1e-7 thresholds with four columns and standard statistics. All LBA prefixes and its comparison against release O2 pass. Focused checks: 72 RICO/integration checks and 51 LBA/radiation checks pass.
* Replace draft JAX Morrison core with source-transcribed physics
Restore Morrison process assembly, sedimentation, corrections and diagnostics with source float32 state and native arithmetic. Consolidate the implementation in module_mp_graupel.py, align public routine boundaries with Fortran, and remove unused draft helpers and their tests. Fix column snow integration and conversion of accumulated fallout to precipitation rates.
Add the lba_morrison strict startup case (60 steps at the unchanged 1e-7 threshold) and isolate named variants' output directories. Update support documentation.
Validation: fresh serial LBA Morrison strict comparison passes; 42 focused physics/interface checks pass. A separate six-hour, eight-column replay matches all 25 pre-cleanup NetCDF variables exactly. Active ice/graupel validation remains outstanding. Experimental ensemble tooling and investigation files are excluded.
* Little script/readme cleanup
* Little fixes for morrison code and adding morrison cases to jax_vs_fortran test, but with a short number of iterations since morrison is hard to compare (mainly because it's in single precision and that seems to hugely amplify floating-point error to be larger than our correctness tolerance that our test uses).
* Batching KK parabolic cylinder calls - this results in far less memory consumed because it's a big function and was called hundreds of times each timestep, and that consumes lots of memory because the jit step inlines each of those calls. Batching them allows us to compute them all in pretty much the same place with far less inlining. Also improving the jax_vs_fortran scripts handling of multiple processes - now it exits out correctly when using ctrl-c
* Little fix to prevent divergence in some fields with kk microphysics.
* Simplify JAX comparison CLI and reporting
* Adding a test, run_jax_comparison_mutation_test.py, that injects little code errors into the jax and runs the jax_vs_fortran script to ensure it fails.
* Little formatting changes
* Improving comment
* Update to the json return option from the runbindiff - clearer explaination and more functionality, making other scripts make use of it too
* Improving readme a bit
* Improving the jax_vs_fortran failure test by adding a case that tests what happens when a stat value is output by one file but not the other
