Started by an SCM change Obtained jenkins_tests/clubb_bin_diff_regression_gfortran_test_new/Jenkinsfile from git https://github.com/larson-group/clubb.git Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /home/jenkins/workspace/clubb_bin_diff_regression_gfortran_test_new [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout The recommended git tool is: git Cloning the remote Git repository Cloning repository https://github.com/larson-group/clubb.git > git init /home/jenkins/workspace/clubb_bin_diff_regression_gfortran_test_new # timeout=10 Fetching upstream changes from https://github.com/larson-group/clubb.git > git --version # timeout=10 > git --version # 'git version 1.8.3.1' using GIT_ASKPASS to set credentials A token based key used by Jenkins to preform Github actions, created 6/21/2021 > git fetch --tags --progress https://github.com/larson-group/clubb.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/larson-group/clubb.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision e8cfbf132e77e340988e90938742ec1912a371d1 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e8cfbf132e77e340988e90938742ec1912a371d1 # timeout=10 Commit message: "Adding blank line to note that my previous commit was BIT_CHANGING." > git rev-list --no-walk 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb # timeout=10 [Pipeline] } [Pipeline] // stage [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (Clone SysAdmin to tmp) [Pipeline] sh + rm -rf /home/jenkins/clubb_bit_diff_test_workspace/sys_admin + git clone https://github.com/larson-group/sys_admin.git /home/jenkins/clubb_bit_diff_test_workspace/sys_admin Cloning into '/home/jenkins/clubb_bit_diff_test_workspace/sys_admin'... [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Run Python Test) [Pipeline] sh + python3 /home/jenkins/clubb_bit_diff_test_workspace/sys_admin/python_nightly_test_suite/run_test.py -c /home/jenkins/clubb_bit_diff_test_workspace/bit_diff_clubb_config .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_gfdlact.a /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../src/CLUBB_core/advance_clubb_core_module.F90:132:23: hydromet, & ! Unused 1 Warning: Unused dummy argument ‘hydromet’ at (1) [-Wunused-dummy-argument] ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_param.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_lapack.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libmicrophys_utils.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_parabolic.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_KK_microphys.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_coamps.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_bugsrad.a /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../src/SILHS/silhs_api_module.F90:315:13: rho_ds_zt, & ! Unused 1 Warning: Unused dummy argument ‘rho_ds_zt’ at (1) [-Wunused-dummy-argument] ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libsilhs.a ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_morrison.a ../src/KK_microphys_module.F90:43:35: subroutine KK_local_microphys( gr, dt, nz, l_latin_hypercube, & ! In 1 Warning: Unused dummy argument ‘gr’ at (1) [-Wunused-dummy-argument] ar: creating /home/jenkins/clubb_bit_diff_test_workspace/repo/compile/../lib/libclubb_other.a CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. Directory 1 is /home/jenkins/clubb_bit_diff_test_workspace/1656d87e61d919b78bfa54fc2d30171d4a9b1601. Directory 2 is /home/jenkins/clubb_bit_diff_test_workspace/latest_revision_validated_8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb. Diffing arm netCDF (*.nc) files *** Differences detected in arm_zt.nc! *** *** Differences detected in arm_zm.nc! *** *** Differences detected in arm_sfc.nc! *** Diffing arm_97 netCDF (*.nc) files *** Differences detected in arm_97_zt.nc! *** *** Differences detected in arm_97_zm.nc! *** *** Differences detected in arm_97_sfc.nc! *** Diffing atex netCDF (*.nc) files *** Differences detected in atex_zt.nc! *** *** Differences detected in atex_zm.nc! *** *** Differences detected in atex_sfc.nc! *** Diffing bomex netCDF (*.nc) files *** Differences detected in bomex_zt.nc! *** *** Differences detected in bomex_zm.nc! *** *** Differences detected in bomex_sfc.nc! *** Diffing dycoms2_rf01 netCDF (*.nc) files *** Differences detected in dycoms2_rf01_zt.nc! *** *** Differences detected in dycoms2_rf01_zm.nc! *** *** Differences detected in dycoms2_rf01_sfc.nc! *** Diffing dycoms2_rf02_ds netCDF (*.nc) files *** Differences detected in dycoms2_rf02_ds_zt.nc! *** *** Differences detected in dycoms2_rf02_ds_zm.nc! *** *** Differences detected in dycoms2_rf02_ds_sfc.nc! *** Diffing gabls3_night netCDF (*.nc) files *** Differences detected in gabls3_night_zt.nc! *** *** Differences detected in gabls3_night_zm.nc! *** *** Differences detected in gabls3_night_sfc.nc! *** Diffing lba netCDF (*.nc) files *** Differences detected in lba_zt.nc! *** *** Differences detected in lba_zm.nc! *** *** Differences detected in lba_sfc.nc! *** Diffing rico netCDF (*.nc) files *** Differences detected in rico_zt.nc! *** *** Differences detected in rico_zm.nc! *** *** Differences detected in rico_sfc.nc! *** Diffing twp_ice netCDF (*.nc) files *** Differences detected in twp_ice_zt.nc! *** *** Differences detected in twp_ice_zm.nc! *** *** Differences detected in twp_ice_sfc.nc! *** Diffing wangara netCDF (*.nc) files *** Differences detected in wangara_zt.nc! *** *** Differences detected in wangara_zm.nc! *** *** Differences detected in wangara_sfc.nc! *** There were some differences detected! There were differences detected in netCDF (*.nc) files. .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. CLUBBStandardsCheck.pl has begun. CLUBBStandardsCheck.pl has finished. Directory 1 is /home/jenkins/clubb_bit_diff_test_workspace/e8cfbf132e77e340988e90938742ec1912a371d1. Directory 2 is /home/jenkins/clubb_bit_diff_test_workspace/1656d87e61d919b78bfa54fc2d30171d4a9b1601. Diffing arm netCDF (*.nc) files Diffing arm_97 netCDF (*.nc) files Diffing atex netCDF (*.nc) files Diffing bomex netCDF (*.nc) files Diffing dycoms2_rf01 netCDF (*.nc) files Diffing dycoms2_rf02_ds netCDF (*.nc) files Diffing gabls3_night netCDF (*.nc) files Diffing lba netCDF (*.nc) files Diffing rico netCDF (*.nc) files Diffing twp_ice netCDF (*.nc) files Diffing wangara netCDF (*.nc) files There were no differences detected! Directory 1 is /home/jenkins/clubb_bit_diff_test_workspace/1656d87e61d919b78bfa54fc2d30171d4a9b1601. Directory 2 is /home/jenkins/clubb_bit_diff_test_workspace/latest_revision_validated_8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb. Diffing arm netCDF (*.nc) files *** Differences detected in arm_zt.nc! *** *** Differences detected in arm_zm.nc! *** *** Differences detected in arm_sfc.nc! *** Diffing arm_97 netCDF (*.nc) files *** Differences detected in arm_97_zt.nc! *** *** Differences detected in arm_97_zm.nc! *** *** Differences detected in arm_97_sfc.nc! *** Diffing atex netCDF (*.nc) files *** Differences detected in atex_zt.nc! *** *** Differences detected in atex_zm.nc! *** *** Differences detected in atex_sfc.nc! *** Diffing bomex netCDF (*.nc) files *** Differences detected in bomex_zt.nc! *** *** Differences detected in bomex_zm.nc! *** *** Differences detected in bomex_sfc.nc! *** Diffing dycoms2_rf01 netCDF (*.nc) files *** Differences detected in dycoms2_rf01_zt.nc! *** *** Differences detected in dycoms2_rf01_zm.nc! *** *** Differences detected in dycoms2_rf01_sfc.nc! *** Diffing dycoms2_rf02_ds netCDF (*.nc) files *** Differences detected in dycoms2_rf02_ds_zt.nc! *** *** Differences detected in dycoms2_rf02_ds_zm.nc! *** *** Differences detected in dycoms2_rf02_ds_sfc.nc! *** Diffing gabls3_night netCDF (*.nc) files *** Differences detected in gabls3_night_zt.nc! *** *** Differences detected in gabls3_night_zm.nc! *** *** Differences detected in gabls3_night_sfc.nc! *** Diffing lba netCDF (*.nc) files *** Differences detected in lba_zt.nc! *** *** Differences detected in lba_zm.nc! *** *** Differences detected in lba_sfc.nc! *** Diffing rico netCDF (*.nc) files *** Differences detected in rico_zt.nc! *** *** Differences detected in rico_zm.nc! *** *** Differences detected in rico_sfc.nc! *** Diffing twp_ice netCDF (*.nc) files *** Differences detected in twp_ice_zt.nc! *** *** Differences detected in twp_ice_zm.nc! *** *** Differences detected in twp_ice_sfc.nc! *** Diffing wangara netCDF (*.nc) files *** Differences detected in wangara_zt.nc! *** *** Differences detected in wangara_zm.nc! *** *** Differences detected in wangara_sfc.nc! *** There were some differences detected! There were differences detected in netCDF (*.nc) files. =============================== Configuring Test =============================== Working directory set to: /home/jenkins/clubb_bit_diff_test_workspace Trying to clone repo... Cloned repo location: /home/jenkins/clubb_bit_diff_test_workspace/repo Output Save Mode: last Using branch: master Commits to test vs the baseline (8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb) 1656d87e61d919b78bfa54fc2d30171d4a9b1601 e8cfbf132e77e340988e90938742ec1912a371d1 Making list from 'UnresolvedCommits': Run Output Path: repo/output Run commands: ['repo/compile/compile.bash', 'rm -rf repo/output', 'mkdir repo/output', 'repo/run_scripts/run_scm_all.bash --min-cases'] Diffing command: repo/run_scripts/run_bindiff_all.bash =============================== Running Test =============================== Parsing message from commit: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 Parsing message from commit: e8cfbf132e77e340988e90938742ec1912a371d1 Found: BIT_CHANGING Checking Commit: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: e8cfbf132e77e340988e90938742ec1912a371d1 Checking out 1656d87e61d919b78bfa54fc2d30171d4a9b1601 Running: 'repo/compile/compile.bash' Running: 'rm -rf repo/output' Running: 'mkdir repo/output' Running: 'repo/run_scripts/run_scm_all.bash --min-cases' Output from 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb already saved Diffing: 'repo/run_scripts/run_bindiff_all.bash 1656d87e61d919b78bfa54fc2d30171d4a9b1601 latest_revision_validated_8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb'  ISSUE WITH COMMIT DETECTED: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 differs from 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb Checking out e8cfbf132e77e340988e90938742ec1912a371d1 Running: 'repo/compile/compile.bash' Running: 'rm -rf repo/output' Running: 'mkdir repo/output' Running: 'repo/run_scripts/run_scm_all.bash --min-cases' Output from 1656d87e61d919b78bfa54fc2d30171d4a9b1601 already saved Diffing: 'repo/run_scripts/run_bindiff_all.bash e8cfbf132e77e340988e90938742ec1912a371d1 1656d87e61d919b78bfa54fc2d30171d4a9b1601'  WARNING: commit 'e8cfbf132e77e340988e90938742ec1912a371d1' was supposed to be BIT_CHANGING, but is not. Finding all BIT_CHANGING commits in range: 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb...1656d87e61d919b78bfa54fc2d30171d4a9b1601 Output from 1656d87e61d919b78bfa54fc2d30171d4a9b1601 already saved Output from 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb already saved Diffing: 'repo/run_scripts/run_bindiff_all.bash 1656d87e61d919b78bfa54fc2d30171d4a9b1601 latest_revision_validated_8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb' Found: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 =============================== Test Results =============================== Commit Range Checked = 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb...e8cfbf132e77e340988e90938742ec1912a371d1 New Unresolved Commits: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 Fake BIT_CHANGING Commits: e8cfbf132e77e340988e90938742ec1912a371d1 Remaining Unresolved Commits: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 =============================== Updating Config File =============================== Saving into UnresolvedCommits: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 Saving into BaselineCommit: e8cfbf132e77e340988e90938742ec1912a371d1 =============================== Cleanup =============================== Removed repo. Removing all but final generated output. REMOVING: 8901b7ce5a715ef407f741ef8d42eebd7cb5bdcb REMOVING: 1656d87e61d919b78bfa54fc2d30171d4a9b1601 RENAMING e8cfbf132e77e340988e90938742ec1912a371d1 to latest_revision_validated_e8cfbf132e77e340988e90938742ec1912a371d1 Exiting with code: 1 [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] script [Pipeline] { [Pipeline] cleanWs [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Deferred wipeout is used... [WS-CLEANUP] done [Pipeline] } [Pipeline] // script [Pipeline] script [Pipeline] { [Pipeline] emailext Sending email to: messnermet@uwm.edu [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: script returned exit code 1 Finished: FAILURE