Removing the target attributes wherever possible. These are artifacts from when gr and the the stats types were globals, and got naively copied when these were pushed through the call stack. (details)
Fixing bug. This loop has a vertical dependency prevent the use of collapse(2). See https://github.com/larson-group/cam/issues/175\#issuecomment-2261283301 (details)
QuadTune: Create SVD plot, start to add checks on global means (unfinished), and start to add capability to include multiple fields (SWCF, LWCF, etc.) in metrics (unfinished). (details)
add Lscale intent(out) to advance_clubb_core and advance_clubb_core_api (#1190) (details)
QuadTune: De-bias matrix before calculating parameter correlations, add comments about definition of residual, include metric name in hover text. (details)
QuadTune: De-bias matrix before calculating parameter correlations, add comments about definition of residual, include metric name in hover text. (details)
QuadTune: Use SVD to approximate the sensitivity matrix, `normlzdSensMatrixPoly`, and curvature matrix, `normlzdCurvMatrix`. (details)
remove Lscale, Lscale_up and Lscale_down from stat_fields_reader and restart_clubb call and remove recalculation of Lscale Ticket #1166 (#1197) (details)
Adding multi_col cases to run_bindiff.py and adding new multicol BFBness test. (details)
Renaming some jenkins tests and making them use a new directory (details)
Changing test name to indicate that it runs on GPUs. (details)
I am making the loops for the wp3 ta term (using ADG1 PDF) symmetric. (details)
QuadTune: 1) Add beginnings of maps of SVD pattern, although this task is unfinished. 2) Normalize all regions by global mean, rather than normalizing each region separately. 3) Make sensitivities signed in sensitivity versus bias scatterplot. (details)
QuadTune: Fix commit bug and add numXBoxes. (details)
Moving Lscale from an acc create statement inside clubb to the acc output statement in clubb_api. This is bug fix that is BIT_CHANGING, but only for silhs cases running on the GPU. Also using this to add BIT_CHANGING:aff1bb12551c5b3a8ab1c4d773ef5c17fb96c53b, as that commit was BC but just missing the message. (details)
QuadTune: 1) Create new plot that shows tuning tradeoffs via U dot bias. 2) Improve bar chart showing the mean contribution of different parameters. (details)
Making a correction to the doubled plus sign from commit a1fcfe8. (details)
As part of generalizing the code to work with both an ascending and (details)
QuadTune: Added colorbars to regional map plots. (details)
QuadTune: 1) Change coloring of scatterpoints to represent either loss changes or residuals. 2) Add map of residuals to complement map of bias. (details)
QuadTune: 1) Add bar chart that shows |metric_changes| due to each parameter. 2) Fix colorbars on maps. 3) Allow download of hi-res figures. 4) Make background of scatterplots grey. (details)
Reduced the upper bound for wp2_min_array to 1.0 (#1214) (details)
Added up2 and vp2 to calculation of wp2_min_array (#1215) (details)
QuadTune: Add bar chart showing linear vs. nonlinear contributions of each parameter. (details)
Changed default setting of flag l_min_wp2_from_corr_wx to false (#1216) (details)
QuadTune: Add map plots of "linSoln" output from global E3SM runs. (details)
QuadTune: Include special regions (DYCOMS, HAWAII, etc.), in addition to 20x20 regions. However, the weights are set to epsilon, which wrecks the loss calculation. This will have to be remedied later. (details)
Recoded a loop within the mono flux limiter so that it's written (details)
Changed some variable names so that it is clear that, while they contain (details)
Within the mono flux limiter, a thermodynamic-level variable needed to (details)
I forgot to add the new variable to the Atlantic Coast Conference (details)
Changing a couple loop limits in mono flux limiter. (details)
QuadTune: Remedy loss function and facilitate plotting of special, non-20x20 regions (DYCOMS, HAWAII, etc.). (details)
Added new hole-filling method which takes TKE from up2 and vp2 (#1217) (details)
Finally fixing gpu_diffness test (#1219) (details)
Slightly relaxing the convergence criteria to fix convergence test. (details)
QuadTune: On matrix-eqn bar chart, plot only extraMetricsToPlot. Add sens-bias scatterplot of only extraMetricsToPlot. In parameter increments bar chart, change absolute values to squares. (details)
QuadTune: Add useLongTitle option, which adds extra information on the plotted quantity, if desired. (details)
QuadTune: Plot bias instead of negative bias on metricsBarChart and bias-vs-sensitivity plots. (details)
Swapping the loop order on a few kernels, I found this faster for the nvhpc compiler and it's BFB (details)
BIT_CHANGING bug fix that only affects l_t_dependent cases running with multiple columns, I detected this with cgils cases, but I'm not sure how it slipped through initially. (details)
Set l_wp2_fill_holes_tke to true by default (#1226) (details)
Removing the target attributes wherever possible. These are artifacts from when gr and the the stats types were globals, and got naively copied when these were pushed through the call stack.
- Removed redundant smooth_max call on Ri_zm in diagnose_Lscale_from_tau - Renamed Ri_zm to Ri_zm_Cx in compute_Cx_fnc_Richardson to distinguish it from the Ri_zm used to caluculate invrs_tau_wpxp_zm
Fixing bug. This loop has a vertical dependency prevent the use of collapse(2). See https://github.com/larson-group/cam/issues/175\#issuecomment-2261283301
I am copying the round-off level changes made to grid weights and grid interpolation functions from the generalized_grid branch over to the master branch. This is being done ahead of merging code from the master over to the generalized_grid branch.
This code is necessary to get results in the generalized grid branch bit-for-bit between ascending and descending grid directions when optimization is set to -O0.
This change produces changes at the numerical round-off level in the affected calculations, making this commit BIT_CHANGING.
Plots show lines overlap for almost all cases besides the most sensitive cases (ARM 97, TWP ICE, etc. -- looking at those shows you how much of a difference a simple numerical round-off can make).
QuadTune: Create SVD plot, start to add checks on global means (unfinished), and start to add capability to include multiple fields (SWCF, LWCF, etc.) in metrics (unfinished).
Extended verbosity functionality for run_bindiff_all.py CLUBB ticket #1181
I added a level system for the verbosity option. Now 0 gives just the general summary at the end, 1 is default and gives the same output as the old default, 2 is the same as the old `--verbose` option
* Adding multicolumn output support and improving stats code.
* Conditionally closing stats files. This is just a bug fix. I'm not sure how this ever worked without the conditionals.
* Improvements and some cleanup.
* Bug fix
* Merge with master
* Removing num_standalone_columns, it's no longer needed.
* Changes to make CPU vs GPU test work with new multicol mode. Columns now run with different parameter sets, tolerance script checks average diff instead, increasing number of cobra timestep run, removing mpace_b because it turns out that cobra has ice and is capable of detecting the same ice code errors that mpace_b was added to test.
* Jenkins script update
* Jenkins script update
* Jenkins script update
* Jenkins script update
* Jenkins script update
* Jenkins script update
* Tightening tolerance
* Script cleanup and improvement
* Some more slight changes and cleanup. Making monoflux script work with new multicol mode.
* Small bug fix that was breaking bugsrad when using multiple columns.
* Bug fix to script
* Small cleanup and renaming of read_parameters
* Slight update to ifdef NETCDF statement
* Better change to ifdefs
* Adding new name init_read_clubb_params_api to public api list
* Name changes and adding section in README
* Renaming script
* Renaming script
* Removing C1 from list of variables to change, it was breaking the CPU vs GPU test, and also clubb warns that 'l_damp_wp2_using_em = T requires C1=C14 and l_stability_correct_tau_zm = F', so perhaps chaning it is just unstable. I've confirmed that GPU bugs are still dectable though.
I am making the loops for the wp3 ta term (using ADG1 PDF) symmetric. Prior to this, they ran from thermo levels 2 to nzt-2. This change makes them symmetric by running them from 2 to nzt-1. There are no degradations to the results as a result of this change. This change makes it easier to generalize CLUBB's grid for both ascending and descending grid directions. The results of this change are BIT_CHANGING.
QuadTune: 1) Add beginnings of maps of SVD pattern, although this task is unfinished. 2) Normalize all regions by global mean, rather than normalizing each region separately. 3) Make sensitivities signed in sensitivity versus bias scatterplot.
* Pushing loops into compute_wpthlp_sfc and compute_wpthlp_sfc, also making them subroutines.
* Removing accidentially committed file.
* Fixing merge errors.
* Fixing bug that was breaking only dycoms2_rf02_so. It only broke this case since it is the only one that has l_cloud_sed=.true. and microphys_scheme=none
* Adding to rad if statement to prevent unneccesary calls.
* Moving stats statement for things like rcm_mc and rvm_mc out of the if microphys statement. This is an additional fix to the dycoms2_rf02_so issue.
* Chaning T_sfc_calc to T_sfc_interp where it's caluclated with interpolation.
* First bit of prescribe forcings GPUization. Only tested with arm so far.
* Cherry-pick merge
* Missed file
* GPUizing more cases
* Rest of cases GPUized
* Little more GPUization and fixes.
* Bug fix to minloc calculation affecting cases with l_modify_bc_for_cnvg_test=.true., unclear why this is a bug. Also adding ifdef around print in GPU code
Moving Lscale from an acc create statement inside clubb to the acc output statement in clubb_api. This is bug fix that is BIT_CHANGING, but only for silhs cases running on the GPU. Also using this to add BIT_CHANGING:aff1bb12551c5b3a8ab1c4d773ef5c17fb96c53b, as that commit was BC but just missing the message.
* Batching multicol output. Unfinished, need to make calls_p_out a namelist input and handle the case that it doesn't evenly divide the total number of timesteps.
* Pushin column loop into initialize_clubb_variables
* Adding explicit loops in main timestepping loop and pushing columns loop into l_calc_thlp2_rad
* Various small loop pushes and improvements.
* Adding argument to create_multi_col_params.py to add calls_p_out to namelist.
As part of generalizing the code to work with both an ascending and descending grid, I discovered an issue. The issue occurs in the xp2/xpyp "upwind" turbulent advection code and involves the "sgn_" variables. Ultimately, for other PDFs, I was setting this variable using equations of the form
sign( one, variable)
which set the value to either 1 or -1 no matter the value of variable.
However, for the ADG1 PDF, I was lazy and simply just set the value of sgn_ to be equal to wp3_on_wp2.
The issue arose when wp3_on_wp2 had a value of 0, and therefore is neither negative or positive.
To rectifiy the issue, all settings of "sgn_" variables now use equations of the form sign( one, variable ).
For a vast majority of cases, this change had absolutely no effect on the results and everything stayed bit-for-bit.
However, BIT_CHANGING results were detected for CGILS S6, MC3E, and TWP-ICE.
* Initial GPUization, and testing multi_col output method as single precision.
* advance_clubb_core tweaks
* Adding radht acc update
* Updates
* Adding new flags to the mono_flux test to prevent CPU and GPU divergence which breaks it.
* Small changes and improvements.
* Making new monoflux test lines
* Reworking some SILHS GPUization to make it more similar to the GPU code in the rest of clubb. This adds some extra parts that run on GPUs, so it is BIT_CHANGING
* Cleanup and comment update
* Cleanup
* Removing DCUDA flag from compile config scripts to help GPU and CPU results match for silhs cases
* Removing accidentally added file.
* Updating script
* Updating script
* Updating script
* Updating tolerance in script to handle rico_silhs differences, and hopefully final GPU updates
* Small cleanup
* Adding option to multi_col diff check script to scale the differences by the field avg. This is only needed (so far) for thlm differences that are slightly too large
Added a "quick" fix for the generation of the time-height plot colormaps when variables are not correlations (i.e. values outside of [-1;1]) Updated VariableGroupCorrelations to include all correlations that are usually output in CLUBB.
* Multi_col output update to enable averaging over timesteps and outputting with single precision. This replaces the timestep batched output method used before, which was only noticably faster at low column numbers. This is technically BC (edited during merge, PR no longer BC), because the single precision output will be interpretted as different from the current double precision output.
* Small cleanup
* Script updates
* Script update
* Script update
* Adding capability to output multi_col as double or single precision using new flag l_output_double_prec.
* Removing accidentally added file.
* Making new jenkins script run single and double precision tests.
Renamed threshold_array -> wp2_min_array Moved upper clipping of wp2 into clip_variance Added warning outputs where wp2 is hard-clipped Added debugging checks to hole-filling and wp2_min_array clipping Some necessary OpenACC-modifications
Added a "quick" fix for time-height plot colormaps when variables are not correlations (i.e. values in [-1;1]) Updated VariableGroupCorrelations to include all correlations that are usually output in CLUBB.
Fixed distinction for colormaps in time-height plots and added tick formatter. We now check the labels for the string "corr" to find correlation variables. Otherwise we just use one-sided or two-sided normed colormaps.
QuadTune: 1) Add bar chart that shows |metric_changes| due to each parameter. 2) Fix colorbars on maps. 3) Allow download of hi-res figures. 4) Make background of scatterplots grey.
Reduced the upper bound for wp2_min_array to 1.0 (#1214)
CLUBB ticket #1165
This is BIT_CHANGING for at least the cases ARM_97 and TWP_ICE, and likely any other more non-standard cases. This change will prevent the minimum clipping value for wp2 from exploding when l_min_wp2_from_corr_wx=.true.
Added up2 and vp2 to calculation of wp2_min_array (#1215)
CLUBB ticket #1165
This is BIT_CHANGING for the cases ARM_97, ATEX, CGILS_S6, GABLS3_NIGHT, LBA, MPACE_B, and TWP_ICE out of the priority cases. These lines of code were added since the fields `u`, `v`, `rt`, and `thl`, and their correlations with `w` should be treated equally.
Changed default setting of flag l_min_wp2_from_corr_wx to false (#1216)
CLUBB ticket #1165
BIT_CHANGING Changed default setting in configurable_model_flags.in and model_flags.F90 to decrease clipping contribution to `wp2` and increase clipping to `rtp2` and `thl2`.
QuadTune: Include special regions (DYCOMS, HAWAII, etc.), in addition to 20x20 regions. However, the weights are set to epsilon, which wrecks the loss calculation. This will have to be remedied later.
Fixed clipping in binormal correlation calc CLUBB ticket #1164
BIT_CHANGING
* Implemented new denominator with smooth_max clipping * Created subroutine `smooth_corr_quotient` in pdf_utilities.F90, outsourcing smooth correlation division * This subroutine is used `calc_comp_corrs_binormal` and `transform_pdf_chi_eta_component`
Added new hole-filling method which takes TKE from up2 and vp2 (#1217)
Added new hole-filling method which takes TKE from up2 and vp2 CLUBB ticket #1165
BIT_CHANGING if flag `l_wp2_fill_holes_tke` is set to true Introduced new flag `l_wp2_fill_holes_tke` to turn on new wp2 hole-filling subroutine that takes TKE from up2 and vp2.
QuadTune: On matrix-eqn bar chart, plot only extraMetricsToPlot. Add sens-bias scatterplot of only extraMetricsToPlot. In parameter increments bar chart, change absolute values to squares.
@LuisHasenauer added bootstrap sampling code to QuadTune. Bootstrap calculations are performed if `useBootstrap=T`. Luis and I checked that the best-fit parameter values and weighted chiSqd are unchanged either with `useBootstrap=T` or `useBootstrap=F`.
BIT_CHANGING bug fix that only affects l_t_dependent cases running with multiple columns, I detected this with cgils cases, but I'm not sure how it slipped through initially.
Set l_wp2_fill_holes_tke to true by default (#1226)
This code change is BIT_CHANGING for extreme cases (e.g. TWP_ICE, LBA, ARM_97). Changed configurable_model_flags.in files and default value in model_flags.F90.
![[Jenkins]](/jenkins/static/57c8a776/images/svgs/logo.svg){kind=logo}
