Changes

I have passed clubb_params down into advance_xm_wpxp_module.F90.

The results from this revision are an exact match to the results from
the previous revision for all cases.

See ticket
#958

I have passed down CLUBB's tunable parameters into advance_wp2_wp3_module.F90.

The results in NetCDF output files from this revision are an exact match to the
results in NetCDF output files from the previous revision for all cases.

added some documentation in pdf_close_module.f90 and started writing of w_up_in_cloud to disk

Updates to separate out C4 and C14 in advance_xp2 module.  This will make it easier to control which invrs_tau_* variable multiplies each term for example.

This commit is BIT_CHANGING owing to order of operations differences, but the pyplotgen output for most cases looks identical to the naked eye.

Changing the name of dummy_1d variables for clarity.

I passed down CLUBB's tunable parameters into advance_xp2_xpyp_module.F90
for most parameters, excluding C4 and C14. Results are unchanged.

Add a blank line to trigger nightly tests.

Passing down more of CLUBB's tunable parameters.

I accidentally removed an important comment in the previous commit.

Update README.md

Adding comment to README for pyplotgen.

Passing down more CLUBB tunable parameters.

Everything is bit-for-bit.

More passing down CLUBB parameters ... whoop-de-doo

Passing down more tunable parameters and removing references to
parameters_tunable. Results are bit-for-bit.

Removing line to trigger the nightly tests for all repos.

See respective messages on the issue 958 on date 10/12/2021

Handling the abomination that is lmin.

Commented out the k=1 boundary conditions in diffusion_zm_lhs and diffusion_zt_lhs, since these are not currently fed into the final LHS matrix.  I included comments to describe why this was done.  Results are b4b.

See https://github.com/larson-group/clubb/issues/920#issuecomment-942735433.

Adding Jenkinsfile for new clubb backwards compatibility test. larson-group#960

Modifying jenkins script. larson-group/clubb#960

Passing tunable parameters into the new tau code.

Add a scatterplot of residual biases versus default-simulation biases.
All points lie along the one-one line if the tuner can remove all bias.

Add a calculation of the radius of curvature in order to assess
nonlinearity of global model to tunable parameter values.
However, the calculation code is still incomplete.

Add a second calculation of the error bars on parameter values
that is an alternative to the SVD calculation.

The code still doesn't pass the test_3x2_C8transformed unit test.

For #910.

Passed down more parameters. Results are bit-for-bit.

Passing through more clubb parameters. Bit-for-bit

Adding jenkins file

Changing name of jenkins tests. larson-group/clubb#960

Renaming

Passing down CLUBB's tunable parameters into the setup_pdf_parameters and
precipitation fraction portions of the code. Everything is bit-for-bit.

These updates incorporate some of Shixuan Zhang's changes, including:

(1) upwind scheme for the eddy diffusion code, which Shixuan says does not make a huge difference; and
(2) l_smooth_heaviside_tau_wpxp flag in the mixing_length.F90 code, which is intended to give
smoother results.

Both of the flags for these changes are turned off, but these updates are BIT_CHANGING because
of some minor changes to the way that factors of rho_ds_* and invrs_rho_ds_* are handled going
into the diffusion.F90 subroutines.  Most cases look identical in pyplotgen, but sensitive cases
like arm_97 and rico look slightly different.

These changes were originally in pull request #962, but because of conflicts with the master
branch code base that developed, they could not be rebased and merged, so I incorporated them
into this new commit.

I passed down CLUBB's tunable parameters into the advance_microphys portion of
the code. It's all bit-for-bit.

Passing down CLUBB's tunable parameters into the remainder of the
PDF subroutines. Everything is bit-for-bit.

I passed down the forgotten thlp2_rad_coef tunable parameter.
All bit-for-bit.

Passing down CLUBB's tunable parameters into CLUBB's
microphysics initialization.

Uncommenting diffusion.F90 k=1 lines again to fix Oracle uninit var test.  I've updated the accompanying note to state clearly that the lines have no effect on model output, but I guess we need to keep the lines to avoid uninitialized variables.

I am passing down the forgotten parameter, altitude threshold, in
advance_xm_wpxp_module.F90.

Adding code wrapped in a flag (l_use_tke_in_K_dfsn) to use u_i'u_i' in the wp2 and wp3 eddy diffusion terms, instead of just wp2 or wp3.  Relevant to CLUBB issue #920.

Results are b4b when the flag is turned off, which it currently is.

Updates as requested to the l_use_tke_in_K_dfsn code:

- The name of the flag is changed to l_use_tke_in_wp2_wp3_K_dfsn
- Comments are added to the main pieces of code in advance_wp2_wp3
- The flag is added to the "compatible_r8029" configurable flags file.

See this comment https://github.com/larson-group/clubb/issues/920#issuecomment-931446401 and following for more information.

Added WSUP from SAM to the VariableGroupWs.py in pyplotgen

I am committing the final stage of the "pass-down params" project.
The results are bit-for-bit identical.

Adding an option to use invrs_tau_N2_iso for C4 terms.  The same invrs_tau had been used for C4 and C14 terms, so some adjustments had to be made to separate them out (this was started with an earlier commit: https://github.com/larson-group/clubb/commit/ef3be1b1378bcd1a447f3866c9f88f8cc4764233).  The flag is currently set to false so results will be b4b.

See also the discussion here: https://github.com/larson-group/clubb/issues/920#issuecomment-920338746 and following comments.

Renaming invrs_tau_zt and invrs_tauw3t to "invrs_tau_wp3_zt" in advance_wp2_wp3 to avoid confusion.  This variable is only used for the wp3 C8 term, so one consistent name is simpler, and the name "invrs_tau_zt" is already used by another variable in advance_clubb_core.

Adding comments for clarity.

I forgot to add this necessary subroutine call to the clubb API.

Fixing a minor accounting bug related to the changes in https://github.com/larson-group/clubb/commit/987d63de489d2baa09ad6e1538b2c6dbb7551690.  There was some overcounting of the relevant terms.

Results are b4b since the l_use_tke_in_wp2_wp3_K_dfsn flag is false.

I have turned the nu_vert_res_dep family of variables into a derived
type and that variable is now being passed throughout the CLUBB code,
rather than accessing variables through a USE statement.

Everything is 100% bit-for-bit!

Add 3 new features to dashboard:
1) Add Huber (confusingly called "Ransac") regressor to find
a fit that de-weights outliers.  The goal is to identify tuning trade-offs.
2) Add ElasticNet regressor.  The goal here is to obtain
a good fit that doesn't perturb the parameters too much,
and that leaves some parameter values unchanged if possible.
3) Add a routine to create PCA biplots.  The goal here
is to identify outliers (i.e., tuning trade-offs) analytically.

For #910.

Updating tolerance_check.py to Python3.

Added sam file name to DYCOMS_RF02_DO case, fixed the clipping in calc_w_up_in_cloud in pdf_closure_module.f90, and changed long name of w_up_in_cloud in stats_zt_module.f90

Turn off l_Lscale_vert_avg because this flag has been deprecated.

We're not concerned about any damage to backwards compatibility because NCAR doesn't run SILHS.
Hopefully, this will allow the clubb_bin_diff_regression_backwards_compatibility_gfortran_test Jenkins test to pass.

larson-group/sys_admin#763

Resolved any issues in the pull request. See https://github.com/larson-group/clubb/pull/970 for details

Included a check in pdf_closure_module.f90 to check if flag for iw_up_in_cloud is set. Removed iw_up_in_cloud from standard_stats.in to prevent binary diffs in CLUBB output files after merge

fixed a typo in pdf_closure_module.f90, line 1284

See ticket
#958

added some documentation in pdf_close_module.f90 and started writing of w_up_in_cloud to disk

See respective messages on the issue 958 on date 10/12/2021

Added WSUP from SAM to the VariableGroupWs.py in pyplotgen

Added sam file name to DYCOMS_RF02_DO case, fixed the clipping in calc_w_up_in_cloud in pdf_closure_module.f90, and changed long name of w_up_in_cloud in stats_zt_module.f90

Resolved any issues in the pull request. See https://github.com/larson-group/clubb/pull/970 for details

Included a check in pdf_closure_module.f90 to check if flag for iw_up_in_cloud is set. Removed iw_up_in_cloud from standard_stats.in to prevent binary diffs in CLUBB output files after merge

fixed a typo in pdf_closure_module.f90, line 1284

Passing w_up_in_cloud out of advance_clubb_core

fixed compiler warning regarding w_up_in_cloud

Remove clipping from a3_coef.

In principle, removing the clipping should reduce noise.
In practice, it makes rico noiser but mpace_b less noisy.
It also allows twp_ice to run to completion with the gfortran compiler.

This commit is BIT_CHANGING.

For #965.

Add "subcolumns" key to LASSO cases in pyplotgen.

Adding a line to the readme to trigger the nightly tests.

Removing a line to trigger the nightly tests. larson-group/sys_admin#763

These changes allow you to call the 4 clubb "advance" subroutines in any
order that you want to try. In the default order (where the order is the
same as found in the current version of the CLUBB master), results are
bit-for-bit when compared to CLUBB master.

Placing the order variables in a nice, convenient place for testing use.

I reordered advance_clubb_core so that the "Advance xp3" (which is a
basic calculation of xp3 by the ansatz by default) section is moved out
of the way of the "advance loop" to a point that is after
advance_windm_edsclrm (or whatever will be called last in the advance
loop.

This change does not alter the default results. Everything is still
bit-for-bit.

I am committing the idiot-proofing code that forces l_min_xp2_from_corr_wx
and l_enable_relaxed_clipping to have opposite values (one must be true
and the other must be false). In this commit, both are set to true, so
the code will produce an exception.

I am indenting the portion of the code where the advance subroutines
are located within the new ordering loop.

Temporarily commenting out the idiot proofing in order to run some tests ...
The results from this branch are still bit-for-bit with the master branch.

Now official reinstating the idiot-proofing and turning off the
l_enable_relaxed_clipping flag. This commit is BIT_CHANGING

I am committing some idiot proofing in regards to the advance
subroutines ordering. No index can have a value of less than 1,
greater than 4, and no index can have the same value as any other
index. Each adv routine order index must have a unique value.

Updates to SAM-style budget plots for CLUBB output.

Adding new four-case tuner script and also some minor updates to error.F90 that avoid the "LES is zero" error for the cloud water and cloud fraction.

Adding comments and minor cleanup for my last commit about the tuner update.

Updating error.F90 to include the error statement about LES field being zero as a check.

Updating CLUBB to pass uprcp and vprcp into and out of advance_clubb_core.

Now that uprcp and vprcp have been passed through advance_clubb_core,
I can get rid of this idiot-proofing code that stops the run when
l_predict_upwp_vpwp is enabled and the PDF is in the "post" position.

I am adding more arguments to the argument list for advance_clubb_core
so that the model now runs with the PDF in the post position.

Result from this change are still bit-for-bit.

Modifying the generate subcolumn panels function to smoothly plot both CLUBB and WRF-LASSO SILHS subcolumns.

Insert print about overwriting default flags with namelist values

Insert print about overwriting default parameters with namelist values

Note that source-code flags and parameter values are overwritten by namelist values

Clarify comments in README

Making api version of setup_grid. larson-group/clubb#972

Making some paramters_silhs variables available from the silhs api

Comment out setting of fatal error

when l_min_xp2_from_corr_wx = F and l_enable_relaxed_clipping = F.  
The goal is to allow the clubb_backwards_compatibility_gfortran_test
Jenkins test to pass again.

For #846.

Adding a step to remove idio proofing into backwards compatibility test. larson-group/sys_admin#763