Started by an SCM change75c0797a7f204bf6c3e61b5934d30c765329532605c0797a7f204bf6c3e61b5934d30c765329532605origin/clubb_silhs_develc0797a7f204bf6c3e61b5934d30c765329532605c0797a7f204bf6c3e61b5934d30c765329532605origin/clubb_silhs_devel712b097205a2cf81c88b1f24a64a493307e06a9f972b097205a2cf81c88b1f24a64a493307e06a9f97origin/changes_to_not_break2b097205a2cf81c88b1f24a64a493307e06a9f972b097205a2cf81c88b1f24a64a493307e06a9f97origin/changes_to_not_break239fd2404968435bfdb7afe440aed7382b6955d6d78fd2404968435bfdb7afe440aed7382b6955d6d78refs/remotes/origin/clubb_silhs_develfd2404968435bfdb7afe440aed7382b6955d6d78fd2404968435bfdb7afe440aed7382b6955d6d78refs/remotes/origin/clubb_silhs_develfd2404968435bfdb7afe440aed7382b6955d6d78fd2404968435bfdb7afe440aed7382b6955d6d78refs/remotes/origin/clubb_silhs_develhttps://github.com/larson-group/e3sm.githttp://carson.math.uwm.edu/jenkins/blue/organizations/jenkins/e3sm_run_gfortran_test/detail/e3sm_run_gfortran_test/239/artifactshttp://carson.math.uwm.edu/jenkins/blue/organizations/jenkins/e3sm_run_gfortran_test/detail/e3sm_run_gfortran_test/239/changeshttp://carson.math.uwm.edu/jenkins/blue/organizations/jenkins/e3sm_run_gfortran_test/detail/e3sm_run_gfortran_test/239/http://carson.math.uwm.edu/jenkins/blue/organizations/jenkins/e3sm_run_gfortran_test/detail/e3sm_run_gfortran_test/239/testsfalsefalse#23927459353428101e3sm_run_gfortran_test #239239false23919997SUCCESS1674119048360http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/239/mixing_length.F90saturation.F90advance_clubb_core_module.F90model_flags.F9038b63ff28ea145707d4ee1d9469013ee20b1697b1674081416000http://carson.math.uwm.edu/jenkins/user/noreplynoreplynoreply@github.comRestructuring and Porting of Compute_mixing_length subroutine(Phase 2) (#1054) * Restructuring and Porting of Compute_mixing_length subroutine(Phase 2) Restructure: sat_mixrat_liq_2D_acc is being directly called instead of calling the 1D version inside the column loop. Changing sat_mixrat_liq_2D_acc to a subroutine from a function and adding output array, start_index as additional argument. This is a workaround for passing sub-arrays. The OpenACC doen't like the sub-arrays being passed and fails the validation. Porting: OpenACC directives are added inside sat_mixrat_liq_2D_acc for porting Validation: Answers are Bit for Bit with arm-multicolumn case + nvhpc compiler * Fix for compilation issues Issue 1: Missed out declaring 'start_index' while intergrating the change Issue 2: The use of error_code module and the procedures inside it causes OpenACC compilation issues when run on the device. * Removing the sat_mixrat_liq_acc and sat_mixrat_liq_2D_acc, making the normal sat_mixrat_liq work for all current use cases, and making the other versions of sat_mixrat_liq (bolton,gfdl,lookup) functional with OPENACC. Co-authored-by: huebler <huebler@uwm.edu> 2023-01-18 16:36:56 -060038b63ff28ea145707d4ee1d9469013ee20b1697bRestructuring and Porting of Compute_mixing_length subroutine(Phase 2) (#1054)editmixing_length.F90editmodel_flags.F90editadvance_clubb_core_module.F90editsaturation.F90githttp://carson.math.uwm.edu/jenkins/user/noreplynoreplynoreplyfalseStarted by an SCM change75c0797a7f204bf6c3e61b5934d30c765329532605c0797a7f204bf6c3e61b5934d30c765329532605origin/clubb_silhs_develc0797a7f204bf6c3e61b5934d30c765329532605c0797a7f204bf6c3e61b5934d30c765329532605origin/clubb_silhs_devel712b097205a2cf81c88b1f24a64a493307e06a9f972b097205a2cf81c88b1f24a64a493307e06a9f97origin/changes_to_not_break2b097205a2cf81c88b1f24a64a493307e06a9f972b097205a2cf81c88b1f24a64a493307e06a9f97origin/changes_to_not_break24063941c86f53a10f74d1e45ca2ec5d22f79c408fd63941c86f53a10f74d1e45ca2ec5d22f79c408fdrefs/remotes/origin/clubb_silhs_devel63941c86f53a10f74d1e45ca2ec5d22f79c408fd63941c86f53a10f74d1e45ca2ec5d22f79c408fdrefs/remotes/origin/clubb_silhs_devel63941c86f53a10f74d1e45ca2ec5d22f79c408fd63941c86f53a10f74d1e45ca2ec5d22f79c408fdrefs/remotes/origin/clubb_silhs_develhttps://github.com/larson-group/e3sm.gitfalse#24027247423428101e3sm_run_gfortran_test #240240false24020073SUCCESS1674205449524http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/240/mono_flux_limiter.F909f86ec344fcdd14a084c6249520c33815e225c241674153381000http://carson.math.uwm.edu/jenkins/user/noreplynoreplynoreply@github.comBreaking up column loop in mono_flux_limiter. This may not be the final form for GPUization, but it's definitely a start, no longer do we have to copy single column variables to multicolumn ones anywhere. (#1051) 2023-01-19 12:36:21 -06009f86ec344fcdd14a084c6249520c33815e225c24Breaking up column loop in mono_flux_limiter. This may not be the final form for GPUization, but it's definitely a start, no longer do we have to copy single column variables to multicolumn ones anywhere. (#1051)editmono_flux_limiter.F90penta_lu_solver.F90model_flags.F90matrix_solver_wrapper.F90tridiag_lu_solver.F90lapack_wrap.F90a5ae081b3485ebbd6b470a1bf077eaf8ba2a29791674153404000http://carson.math.uwm.edu/jenkins/user/noreplynoreplynoreply@github.comAdding tridiag_lu solver (#1056) * Adding tridiag_lu solver * Changing low to lower in tridiag_lu and penta_lu, also fixing up some spacing. * Making variables that represent matrix solutions more consistently named. 2023-01-19 12:36:44 -0600a5ae081b3485ebbd6b470a1bf077eaf8ba2a2979Adding tridiag_lu solver (#1056)editmodel_flags.F90editmatrix_solver_wrapper.F90editlapack_wrap.F90addtridiag_lu_solver.F90editpenta_lu_solver.F90githttp://carson.math.uwm.edu/jenkins/user/noreplynoreplyfalse241http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/241/239http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/239/Started by an SCM change75c0797a7f204bf6c3e61b5934d30c765329532605c0797a7f204bf6c3e61b5934d30c765329532605origin/clubb_silhs_develc0797a7f204bf6c3e61b5934d30c765329532605c0797a7f204bf6c3e61b5934d30c765329532605origin/clubb_silhs_devel712b097205a2cf81c88b1f24a64a493307e06a9f972b097205a2cf81c88b1f24a64a493307e06a9f97origin/changes_to_not_break2b097205a2cf81c88b1f24a64a493307e06a9f972b097205a2cf81c88b1f24a64a493307e06a9f97origin/changes_to_not_break23802f066f5cbd140b9aa603b99c027095fa274697f02f066f5cbd140b9aa603b99c027095fa274697frefs/remotes/origin/clubb_silhs_devel02f066f5cbd140b9aa603b99c027095fa274697f02f066f5cbd140b9aa603b99c027095fa274697frefs/remotes/origin/clubb_silhs_devel02f066f5cbd140b9aa603b99c027095fa274697f02f066f5cbd140b9aa603b99c027095fa274697frefs/remotes/origin/clubb_silhs_develhttps://github.com/larson-group/e3sm.gitfalse#23827593913428101e3sm_run_gfortran_test #238238false23819905SUCCESS1673600650757http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/238/mixing_length.F90saturation.F90model_flags.F90constants_clubb.F9021426fea400c37cfd10dbde5beb0f96690a2f8381673549655000http://carson.math.uwm.edu/jenkins/user/noreplynoreplynoreply@github.comRestructuring and Porting of Compute_mixing_length subroutine(Phase 1) (#1052) * Restructuring and Porting of Compute_mixing_length subroutine(Phase 1) Restructure: The compute_mixing_length is one of the top most routine taking 35-50% of the total time in a single timestep. The subroutine has been restructured to push the i-loop further down to extract vectorization and parallelization. The restructuring also involves introduction of sat_mixrat_liq_acc routines to extract parallelism when called inside a OpenACC parallel region. Porting: OpenACC directives are inserted to port the restructured compute_mixing_length code on to the GPUs. This port is currently unoptimized and there is still room for improvement. NOTE: Currently, l_sat_mixrat_lookup = false and saturation_formula = saturation_flatau (Earthworks config options) case is supported on OpenACC build. Any other options works on CPUs as usual. OpenACC declare create directives are inserted in model_flags and constants_clubb, as these module variables are used inside the saturation routines. * Added debug message about only supporting l_sat_mixrat_lookup = false and saturation_formula = saturation_flatau on GPUs Answers are Bit for Bit with arm-multicolumn case + nvhpc compiler. * Changing CLUBB debug level 1 to 0 for the saturation formula support running on GPUs * Changing indentation to make gfortran happy, it wants ifdefs to start at the beginning of the line. * Adding use statements for error checks and printouts, also making the errors set err_code to clubb_fatal_error. Co-authored-by: huebler <huebler@uwm.edu> 2023-01-12 12:54:15 -060021426fea400c37cfd10dbde5beb0f96690a2f838Restructuring and Porting of Compute_mixing_length subroutine(Phase 1) (#1052)editmixing_length.F90editmodel_flags.F90editsaturation.F90editconstants_clubb.F90githttp://carson.math.uwm.edu/jenkins/user/noreplynoreplyfalse239http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/239/237http://carson.math.uwm.edu/jenkins/view/all/job/e3sm_run_gfortran_test/237/