Started by an SCM change Obtained jenkins_tests/clubb_BFB_e3sm_flags_gfortran_test/Jenkinsfile from git https://github.com/larson-group/clubb.git [Pipeline] Start of Pipeline [Pipeline] node Running on malan in /home/jenkins/workspace/clubb_BFB_e3sm_flags_gfortran_test [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout The recommended git tool is: git Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/larson-group/clubb.git > git init /home/jenkins/workspace/clubb_BFB_e3sm_flags_gfortran_test # timeout=10 Fetching upstream changes from https://github.com/larson-group/clubb.git > git --version # timeout=10 > git --version # 'git version 2.30.2' > git fetch --tags --force --progress -- https://github.com/larson-group/clubb.git +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch Checking out Revision 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b (refs/remotes/origin/master) Commit message: "Small bug fix to make gfortran+debug happy, without the _core_rknd this could be a potential type error" > git config remote.origin.url https://github.com/larson-group/clubb.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > git config core.sparsecheckout # timeout=10 > git checkout -f 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b # timeout=10 > git rev-list --no-walk 040e9961b5b422803a0aa816c5921be9576aa4a8 # timeout=10 [Pipeline] } [Pipeline] // stage [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (Update sys_admin Repo) [Pipeline] sh + git -C /home/pub/jenkins_BFB_test_configs/sys_admin pull hint: Pulling without specifying how to reconcile divergent branches is hint: discouraged. You can squelch this message by running one of the following hint: commands sometime before your next pull: hint: hint: git config pull.rebase false # merge (the default strategy) hint: git config pull.rebase true # rebase hint: git config pull.ff only # fast-forward only hint: hint: You can replace "git config" with "git config --global" to set a default hint: preference for all repositories. You can also pass --rebase, --no-rebase, hint: or --ff-only on the command line to override the configured default per hint: invocation. Already up to date. [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Run Python Test) [Pipeline] sh + source /etc/profile.d/larson-group.sh ++ export GIT_EDITOR=vi ++ GIT_EDITOR=vi ++ export SVN_EDITOR=vi ++ SVN_EDITOR=vi ++ export OMP_STACKSIZE=1048579 ++ OMP_STACKSIZE=1048579 ++ export LMOD_ROOT=/opt/lmod/ ++ LMOD_ROOT=/opt/lmod/ ++ source /opt/lmod//lmod/lmod/init/bash +++ '[' -z '' ']' +++ case "$-" in +++ __lmod_vx=x +++ '[' -n x ']' +++ set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for this output (/usr/local/spack/opt/spack/linux-pop21-skylake_avx512/gcc-12.2.0/lmod-8.7.37-d6vyrvvncsuguygovgvimahv5kfyaunp/lmod/lmod/init/bash) Shell debugging restarted +++ unset __lmod_vx +++ find /usr/local/spack/share/spack/lmod/linux-pop21-x86_64/Core -print -quit ++ export MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop21-x86_64/Core ++ MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop21-x86_64/Core + module load gcc netcdf-fortran + '[' -z '' ']' + case "$-" in + __lmod_sh_dbg=x + '[' -n x ']' + set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for Lmod's output Lmod Warning: ------------------------------------------------------------------------------- The following dependent module(s) are not currently loaded: gcc-runtime/13.1.0-eheq3uy (required by: mpich/4.2.1, hdf5/1.14.3-2o6xqva, netcdf-c/4.9.2, netcdf-fortran/4.5.3), glibc/2.33-3y65nib (required by: mpich/4.2.1, hdf5/1.14.3-2o6xqva, netcdf-c/4.9.2, netcdf-fortran/4.5.3) ------------------------------------------------------------------------------- Shell debugging restarted + unset __lmod_sh_dbg + return 0 + python3 /home/pub/jenkins_BFB_test_configs/sys_admin/python_nightly_test_suite/run_test.py -c /home/pub/jenkins_BFB_test_configs/clubb_BFB_e3sm_flags_gfortran_test_config CMake Error at cmake/deps/NetcdfFortranDependency.cmake:14 (cmake_path): Unknown CMake command "cmake_path". Call Stack (most recent call first): CMakeLists.txt:29 (include) CMake Error at cmake/deps/NetcdfFortranDependency.cmake:14 (cmake_path): Unknown CMake command "cmake_path". Call Stack (most recent call first): CMakeLists.txt:29 (include) CMake Error at cmake/deps/NetcdfFortranDependency.cmake:14 (cmake_path): Unknown CMake command "cmake_path". Call Stack (most recent call first): CMakeLists.txt:29 (include) CMake Error at cmake/deps/NetcdfFortranDependency.cmake:14 (cmake_path): Unknown CMake command "cmake_path". Call Stack (most recent call first): CMakeLists.txt:29 (include) CMake Error at cmake/deps/NetcdfFortranDependency.cmake:14 (cmake_path): Unknown CMake command "cmake_path". Call Stack (most recent call first): CMakeLists.txt:29 (include) =============================== Configuring Test =============================== Working directory set to: /home/jenkins/workspace/clubb_BFB_e3sm_flags_gfortran_test Checking specified repo: /home/jenkins/workspace/clubb_BFB_e3sm_flags_gfortran_test Using /home/jenkins/workspace/clubb_BFB_e3sm_flags_gfortran_test with origin https://github.com/larson-group/clubb.git Output Save Mode: none Using branch: master Making list from 'UnresolvedCommits': CheckRange = True: Getting commit range Run Output Path: output Run commands: ['python compile.py -debug', '\nrm -rf output', '\nmkdir output', '\nmv input/tunable_parameters_e3sm/configurable_model_flags.in input/tunable_parameters', '\npython run_scripts/run_scm_all.py -min_cases'] Diffing command: run_scripts/run_bindiff_all.py =============================== Running Test =============================== Parsing message from commit: a5597665ed8bdd21902000712f8ad2685276eec0 Parsing message from commit: bba14c86a748da7993af72c056a9f6505cf43f1b Found: BIT_CHANGING Parsing message from commit: 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 Parsing message from commit: 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b Checking Commit: a5597665ed8bdd21902000712f8ad2685276eec0 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: bba14c86a748da7993af72c056a9f6505cf43f1b Checking out a5597665ed8bdd21902000712f8ad2685276eec0 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'mv input/tunable_parameters_e3sm/configurable_model_flags.in input/tunable_parameters' Running: 'python run_scripts/run_scm_all.py -min_cases' ================== WARNING ================== Commit: a5597665ed8bdd21902000712f8ad2685276eec0 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Checking out 040e9961b5b422803a0aa816c5921be9576aa4a8 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'mv input/tunable_parameters_e3sm/configurable_model_flags.in input/tunable_parameters' Running: 'python run_scripts/run_scm_all.py -min_cases' ================== WARNING ================== Commit: 040e9961b5b422803a0aa816c5921be9576aa4a8 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== ISSUE WITH COMMIT DETECTED: a5597665ed8bdd21902000712f8ad2685276eec0 differs from 040e9961b5b422803a0aa816c5921be9576aa4a8 Checking out bba14c86a748da7993af72c056a9f6505cf43f1b Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'mv input/tunable_parameters_e3sm/configurable_model_flags.in input/tunable_parameters' Running: 'python run_scripts/run_scm_all.py -min_cases' ================== WARNING ================== Commit: bba14c86a748da7993af72c056a9f6505cf43f1b There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit a5597665ed8bdd21902000712f8ad2685276eec0 already failed. Will not run again Commit 'bba14c86a748da7993af72c056a9f6505cf43f1b' is BIT_CHANGING and properly labelled as such. Checking Commit: 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b Running: final commit to check, not labelled BIT_CHANGING Checking out 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'mv input/tunable_parameters_e3sm/configurable_model_flags.in input/tunable_parameters' Running: 'python run_scripts/run_scm_all.py -min_cases' ================== WARNING ================== Commit: 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit bba14c86a748da7993af72c056a9f6505cf43f1b already failed. Will not run again ISSUE WITH COMMIT DETECTED: 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b differs from bba14c86a748da7993af72c056a9f6505cf43f1b Finding all BIT_CHANGING commits in range: 040e9961b5b422803a0aa816c5921be9576aa4a8...a5597665ed8bdd21902000712f8ad2685276eec0 Commit a5597665ed8bdd21902000712f8ad2685276eec0 already failed. Will not run again Commit 040e9961b5b422803a0aa816c5921be9576aa4a8 already failed. Will not run again Found: a5597665ed8bdd21902000712f8ad2685276eec0 Finding all BIT_CHANGING commits in range: bba14c86a748da7993af72c056a9f6505cf43f1b...988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b Checking out 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'mv input/tunable_parameters_e3sm/configurable_model_flags.in input/tunable_parameters' Running: 'python run_scripts/run_scm_all.py -min_cases' ================== WARNING ================== Commit: 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit bba14c86a748da7993af72c056a9f6505cf43f1b already failed. Will not run again Commit 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b already failed. Will not run again Commit 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 already failed. Will not run again Found: 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b =============================== Test Results =============================== Commit Range Checked = 040e9961b5b422803a0aa816c5921be9576aa4a8...988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b New Unresolved Commits: a5597665ed8bdd21902000712f8ad2685276eec0 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b No fake BIT_CHANGING commits found. Remaining Unresolved Commits (oldest commit is listed first): a5597665ed8bdd21902000712f8ad2685276eec0 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b =============================== Updating Config File =============================== Saving into UnresolvedCommits: a5597665ed8bdd21902000712f8ad2685276eec0 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b Ignoring 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b since it failed to produce output. Ignoring 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 since it failed to produce output. Ignoring bba14c86a748da7993af72c056a9f6505cf43f1b since it failed to produce output. Ignoring a5597665ed8bdd21902000712f8ad2685276eec0 since it failed to produce output. =============================== Cleanup =============================== Removing all generated output ================== Unresolved Commit Status ================== Remaining Unresolved Commits (oldest commit is listed first): a5597665ed8bdd21902000712f8ad2685276eec0 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b ================== FAILED Commits ================== These are the commits which entirely failed to produce output: a5597665ed8bdd21902000712f8ad2685276eec0 040e9961b5b422803a0aa816c5921be9576aa4a8 bba14c86a748da7993af72c056a9f6505cf43f1b 988ea0f2cc24e424e0a204b65bfd59bd4e9eee5b 14f5e6e323b11dc9e5fff8ca5982c6ab7f92f8c1 Exiting with code: 1 [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] script [Pipeline] { [Pipeline] emailext Sending email to: messnermet@uwm.edu [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: script returned exit code 1 Finished: FAILURE