Skip to content
Success

Changes

Summary

  1. This commit contains code changes to implement modifications on limiters in three places: (details)
  2. Advance xm wpxp gpuization (#1077) (details)
  3. Small improvements to diff_netcdf_outputs.py, removing reliance on ncdiff, now it is entirely in python. Cleaning up linux_x86_64_nvhpc_gpu.bash, removing outdated parts, improving default parallel compilation, changing pgfortran to nvfortran. (details)
  4. Small tweaks to fix some GPU bugs. Some variables were uninitialized on the CPU while we were saving them. This could only have been caught by comparing consecutive runs and checking _zt and _zm files, even then few cases were having problems. (details)
  5. Fixing a labelling error in redirect_interpolated_azt_2D and similar procedures, since this interpolates to zt the input should be zm. I think this was my fault, so I cleaned all the zt2zm and zm2zt things up to make it a little nicer. Also ordered the routines _k _1D _2D to make it easier to jump around, it was a bit confusing as they were out of order and the typo really made it hard. (details)
Commit 4b16d833e86ad23ef6ea81de67c38c596ebccd25 by bmg2
This commit contains code changes to implement modifications on limiters in three places:

1. remove the limiters in denominator of equation for
   brunt_vaisala_freq_sqd_smth, which affects the computed
   eddy dissipation time scale in turbulent fluxes (wpxp).
   (in mixing_length.F90)

2. reduce the threshold values of limiters in the equation
   for richardson number (sqrt_Ri_zm)
   (in mixing_length.F90)

3. introduce the smoothed max/min function for limiters in
   equation of Cx_fnc_Richardson.
   (in advance_helper_module.F90).

After the modification, we also apply a zt2m(zm2zt) smoothing
on the calculated quantities. These modifications are found to
be benificial for improving solution convergence in CLUBB-SCM

The code changes are controlled by a newly introduced flag named
"l_modify_limiters_for_cnvg_test", which is set to .false. (meaning that
the modificaitons on limiters is turned off) by default.

Originally committed by Shixuan.
The file was modified src/G_unit_test_types/pdf_parameter_tests.F90 (diff)
The file was modified src/clubb_tuner.F90 (diff)
The file was modified input/tunable_parameters/configurable_model_flags.in (diff)
The file was modified src/G_unit_test_types/spurious_source_test.F90 (diff)
The file was modified src/clubb_driver.F90 (diff)
The file was modified src/CLUBB_core/model_flags.F90 (diff)
The file was modified src/CLUBB_core/clubb_api_module.F90 (diff)
The file was modified src/CLUBB_core/mixing_length.F90 (diff)
The file was modified src/CLUBB_core/advance_clubb_core_module.F90 (diff)
The file was modified src/CLUBB_core/advance_helper_module.F90 (diff)
Commit 49ec5990089b7f9728eb6707a2b653bbeca94c81 by noreply
Advance xm wpxp gpuization (#1077)

* Initial commit for GPUizing advance_xm_wpxp.

* GPUizing fill_holes_vertical

* Adding Skthl_zm to the update host list, I missed this in the last PR. I noticed this by comparing results with and without managed memory, now I've checked BFBness with arm, mpace_b, mc3e, and gabls2.

* Small GPU fixes (#1076)

* Fixing small things that I caught by adding the default(present) onto acc loops.

* Moving default(present) to the end because it looks nicer there.

* Adding default(present) to all acc loop statements. Also adding azt to a copyin statement, which was missed previously. All BFB.

* Incemental update, not well tested yet.

* Removing some copies and making the sclr_dim change.

* Fixing a bug that only seemed detectable with astex_a209. We need to pass only single arrays to functions, calling ddzt( nz, ngrdcol, gr, rho_ds_zt * K_zt_nu ) was resulting in rho_ds_zt * K_zt_nu being evluated on the CPU, but the values were only valid on the GPU. So we need to evaluate that expression on the GPU, save it into an array (currently K_zt_nu_tmp), then pass that to ddzt.

* GPUizing calc_turb_adv_range

* GPUizing mono_flux_limiter

* Cleaning up data statments and a couple other things.

* Updated for some different options.

* More updates needed for various options.

* Reverting accidental flag change

* Should be the final changes, all options tested now.

* Replacing some comments in monoflux limiter, and also modifying it to make it BFB on CPUs. Also changing incorrect error conditions on tridiag.

* Adding max_x_allowable to update host statement, missed previous.

* Properly naming tmp variables and variables calculated from ddzt and ddzm start with ddzt_ and ddzm_.

* Replacing constants with named ones from constants_clubb.

* Replacing hard coded numbers in lhs variables representing the number of bands they contain with fortran parameters.
The file was modified src/CLUBB_core/advance_helper_module.F90 (diff)
The file was modified src/CLUBB_core/diffusion.F90 (diff)
The file was modified src/CLUBB_core/parameters_model.F90 (diff)
The file was modified src/CLUBB_core/grid_class.F90 (diff)
The file was modified src/CLUBB_core/mono_flux_limiter.F90 (diff)
The file was modified src/CLUBB_core/fill_holes.F90 (diff)
The file was modified src/CLUBB_core/advance_xm_wpxp_module.F90 (diff)
The file was modified src/CLUBB_core/clip_explicit.F90 (diff)
The file was modified src/CLUBB_core/turbulent_adv_pdf.F90 (diff)
The file was modified src/CLUBB_core/matrix_solver_wrapper.F90 (diff)
The file was modified src/CLUBB_core/mean_adv.F90 (diff)
The file was modified src/CLUBB_core/advance_clubb_core_module.F90 (diff)
Commit f20b355f2ce4e686d8f3392ad148341f08fd6acc by Gunther Huebler
Small improvements to diff_netcdf_outputs.py, removing reliance on ncdiff, now it is entirely in python. Cleaning up linux_x86_64_nvhpc_gpu.bash, removing outdated parts, improving default parallel compilation, changing pgfortran to nvfortran.
The file was modified compile/config/linux_x86_64_nvhpc_gpu.bash (diff)
The file was modified run_scripts/diff_netcdf_outputs.py (diff)
Commit c0fb204097dab9c58399ebd3526835db482899cc by Gunther Huebler
Small tweaks to fix some GPU bugs. Some variables were uninitialized on the CPU while we were saving them. This could only have been caught by comparing consecutive runs and checking _zt and _zm files, even then few cases were having problems.
The file was modified src/CLUBB_core/advance_xm_wpxp_module.F90 (diff)
Commit 34329990202f44999b237263136111b0086963c4 by Gunther Huebler
Fixing a labelling error in redirect_interpolated_azt_2D and similar procedures, since this interpolates to zt the input should be zm. I think this was my fault, so I cleaned all the zt2zm and zm2zt things up to make it a little nicer. Also ordered the routines _k _1D _2D to make it easier to jump around, it was a bit confusing as they were out of order and the typo really made it hard.
The file was modified src/CLUBB_core/grid_class.F90 (diff)