Started by an SCM change Obtained jenkins_tests/clubb_BFB_varying_flags_gfortran_test/Jenkinsfile from git https://github.com/larson-group/clubb.git [Pipeline] Start of Pipeline [Pipeline] node Still waiting to schedule task ‘condella’ is offline ‘malan’ is offline ‘overlie’ is offline Running on mitch in /home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout The recommended git tool is: git Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/larson-group/clubb.git > git init /home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test # timeout=10 Fetching upstream changes from https://github.com/larson-group/clubb.git > git --version # timeout=10 > git --version # 'git version 2.34.1' using GIT_ASKPASS to set credentials larsongroupsysadmin github token > git fetch --tags --force --progress -- https://github.com/larson-group/clubb.git +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch Checking out Revision 955aa780e3a47b6accf7af68d85f76ad53baff39 (refs/remotes/origin/master) Commit message: "Removing unused variables" > git config remote.origin.url https://github.com/larson-group/clubb.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > git config core.sparsecheckout # timeout=10 > git checkout -f 955aa780e3a47b6accf7af68d85f76ad53baff39 # timeout=10 > git rev-list --no-walk 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb # timeout=10 [Pipeline] } [Pipeline] // stage [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (Update sys_admin Repo) [Pipeline] sh + git -C /home/pub/jenkins_BFB_test_configs/sys_admin pull Already up to date. [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Run Python Test) [Pipeline] sh + source /etc/profile.d/larson-group.sh ++ export GIT_EDITOR=vi ++ GIT_EDITOR=vi ++ export SVN_EDITOR=vi ++ SVN_EDITOR=vi ++ export OMP_STACKSIZE=1048579 ++ OMP_STACKSIZE=1048579 ++ export LMOD_ROOT=/opt/lmod/ ++ LMOD_ROOT=/opt/lmod/ ++ source /opt/lmod//lmod/lmod/init/bash +++ '[' -z '' ']' +++ case "$-" in +++ __lmod_vx=x +++ '[' -n x ']' +++ set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for this output (/usr/local/spack/opt/spack/linux-pop22-skylake_avx512/gcc-12.2.0/lmod-8.7.37-fq24mybyn2li6got2bxzk62ejh5d3p4z/lmod/lmod/init/bash) Shell debugging restarted +++ unset __lmod_vx +++ find /usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core -print -quit ++ export MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core ++ MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core + module load gcc netcdf-fortran + '[' -z '' ']' + case "$-" in + __lmod_sh_dbg=x + '[' -n x ']' + set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for Lmod's output Lmod Warning: ------------------------------------------------------------------------------- The following dependent module(s) are not currently loaded: gcc-runtime/13.1.0-wiuv7wc (required by: mpich/4.2.1, hdf5/1.14.3-nhc23hl, netcdf-c/4.9.2-xduzp77, netcdf-fortran/4.5.3), glibc/2.35-tdr5czg (required by: mpich/4.2.1, hdf5/1.14.3-nhc23hl, netcdf-c/4.9.2-xduzp77, netcdf-fortran/4.5.3) ------------------------------------------------------------------------------- Shell debugging restarted + unset __lmod_sh_dbg + return 0 + python3 /home/pub/jenkins_BFB_test_configs/sys_admin/python_nightly_test_suite/run_test.py -c /home/pub/jenkins_BFB_test_configs/clubb_BFB_varying_flags_gfortran_test_config Traceback (most recent call last): File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 359, in <module> main() File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 343, in main flag_dict = read_flag_settings(args.flag_config_file) File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 154, in read_flag_settings with open(path) as f: FileNotFoundError: [Errno 2] No such file or directory: 'run_scripts/run_bindiff_w_flags_config_core_flags.json' Traceback (most recent call last): File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 359, in <module> main() File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 343, in main flag_dict = read_flag_settings(args.flag_config_file) File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 154, in read_flag_settings with open(path) as f: FileNotFoundError: [Errno 2] No such file or directory: 'run_scripts/run_bindiff_w_flags_config_core_flags.json' Traceback (most recent call last): File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 359, in <module> main() File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 343, in main flag_dict = read_flag_settings(args.flag_config_file) File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 154, in read_flag_settings with open(path) as f: FileNotFoundError: [Errno 2] No such file or directory: 'run_scripts/run_bindiff_w_flags_config_core_flags.json' Traceback (most recent call last): File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 359, in <module> main() File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 343, in main flag_dict = read_flag_settings(args.flag_config_file) File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 154, in read_flag_settings with open(path) as f: FileNotFoundError: [Errno 2] No such file or directory: 'run_scripts/run_bindiff_w_flags_config_core_flags.json' Traceback (most recent call last): File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 359, in <module> main() File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 343, in main flag_dict = read_flag_settings(args.flag_config_file) File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 154, in read_flag_settings with open(path) as f: FileNotFoundError: [Errno 2] No such file or directory: 'run_scripts/run_bindiff_w_flags_config_core_flags.json' Traceback (most recent call last): File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 359, in <module> main() File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 343, in main flag_dict = read_flag_settings(args.flag_config_file) File "/home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test/run_scripts/run_clubb_w_varying_flags.py", line 154, in read_flag_settings with open(path) as f: FileNotFoundError: [Errno 2] No such file or directory: 'run_scripts/run_bindiff_w_flags_config_core_flags.json' =============================== Configuring Test =============================== Working directory set to: /home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test Checking specified repo: /home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test Using /home/jenkins/workspace/clubb_BFB_varying_flags_gfortran_test with origin https://github.com/larson-group/clubb.git Output Save Mode: none Using branch: master Making list from 'UnresolvedCommits': CheckRange = True: Getting commit range Run Output Path: output Run commands: ['python compile.py -debug', '\nrm -rf output', '\nmkdir output', '\npython run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json'] Diffing command: run_scripts/run_bindiff_varying_flags_output.py =============================== Running Test =============================== Parsing message from commit: e04956358685c2869358db8b6d46d16cf9744b03 Parsing message from commit: 5512bff0caa1b179b2a03a5512fc3de2c866b0cb Parsing message from commit: e1ca53920f7095d09c5dde5b9502f4056c481b48 Parsing message from commit: 0ed15803474cd7d3d8228a642e9e2599a840707d Parsing message from commit: 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 Parsing message from commit: 955aa780e3a47b6accf7af68d85f76ad53baff39 Checking Commit: e04956358685c2869358db8b6d46d16cf9744b03 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: 5512bff0caa1b179b2a03a5512fc3de2c866b0cb Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: e1ca53920f7095d09c5dde5b9502f4056c481b48 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: 0ed15803474cd7d3d8228a642e9e2599a840707d Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: 955aa780e3a47b6accf7af68d85f76ad53baff39 Running: final commit to check, not labelled BIT_CHANGING Checking out 955aa780e3a47b6accf7af68d85f76ad53baff39 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ================== WARNING ================== Commit: 955aa780e3a47b6accf7af68d85f76ad53baff39 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Checking out 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ISSUE WITH COMMIT DETECTED: 955aa780e3a47b6accf7af68d85f76ad53baff39 differs from 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb Finding all BIT_CHANGING commits in range: 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb...955aa780e3a47b6accf7af68d85f76ad53baff39 Checking out e04956358685c2869358db8b6d46d16cf9744b03 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ================== WARNING ================== Commit: e04956358685c2869358db8b6d46d16cf9744b03 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Output from 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb already saved Checking out 5512bff0caa1b179b2a03a5512fc3de2c866b0cb Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ================== WARNING ================== Commit: 5512bff0caa1b179b2a03a5512fc3de2c866b0cb There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit e04956358685c2869358db8b6d46d16cf9744b03 already failed. Will not run again Checking out e1ca53920f7095d09c5dde5b9502f4056c481b48 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ================== WARNING ================== Commit: e1ca53920f7095d09c5dde5b9502f4056c481b48 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit 5512bff0caa1b179b2a03a5512fc3de2c866b0cb already failed. Will not run again Checking out 0ed15803474cd7d3d8228a642e9e2599a840707d Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ================== WARNING ================== Commit: 0ed15803474cd7d3d8228a642e9e2599a840707d There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit e1ca53920f7095d09c5dde5b9502f4056c481b48 already failed. Will not run again Checking out 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 Running: 'python compile.py -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_clubb_w_varying_flags.py --min-cases -max_iters=360 -f run_scripts/run_bindiff_w_flags_config_core_flags.json' ================== WARNING ================== Commit: 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 There was a problem copying the output. Did the run complete? Is the output path correct? output The run will continue, and this commit will be considered unresolved. =========================================== Commit 0ed15803474cd7d3d8228a642e9e2599a840707d already failed. Will not run again Commit 955aa780e3a47b6accf7af68d85f76ad53baff39 already failed. Will not run again Commit 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 already failed. Will not run again Found: e04956358685c2869358db8b6d46d16cf9744b03 5512bff0caa1b179b2a03a5512fc3de2c866b0cb e1ca53920f7095d09c5dde5b9502f4056c481b48 0ed15803474cd7d3d8228a642e9e2599a840707d 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 955aa780e3a47b6accf7af68d85f76ad53baff39 =============================== Test Results =============================== Commit Range Checked = 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb...955aa780e3a47b6accf7af68d85f76ad53baff39 New Unresolved Commits: e04956358685c2869358db8b6d46d16cf9744b03 5512bff0caa1b179b2a03a5512fc3de2c866b0cb e1ca53920f7095d09c5dde5b9502f4056c481b48 0ed15803474cd7d3d8228a642e9e2599a840707d 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 955aa780e3a47b6accf7af68d85f76ad53baff39 No fake BIT_CHANGING commits found. Remaining Unresolved Commits (oldest commit is listed first): e04956358685c2869358db8b6d46d16cf9744b03 5512bff0caa1b179b2a03a5512fc3de2c866b0cb e1ca53920f7095d09c5dde5b9502f4056c481b48 0ed15803474cd7d3d8228a642e9e2599a840707d 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 955aa780e3a47b6accf7af68d85f76ad53baff39 =============================== Updating Config File =============================== Saving into UnresolvedCommits: e04956358685c2869358db8b6d46d16cf9744b03 5512bff0caa1b179b2a03a5512fc3de2c866b0cb e1ca53920f7095d09c5dde5b9502f4056c481b48 0ed15803474cd7d3d8228a642e9e2599a840707d 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 955aa780e3a47b6accf7af68d85f76ad53baff39 Ignoring 955aa780e3a47b6accf7af68d85f76ad53baff39 since it failed to produce output. Ignoring 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 since it failed to produce output. Ignoring 0ed15803474cd7d3d8228a642e9e2599a840707d since it failed to produce output. Ignoring e1ca53920f7095d09c5dde5b9502f4056c481b48 since it failed to produce output. Ignoring 5512bff0caa1b179b2a03a5512fc3de2c866b0cb since it failed to produce output. Ignoring e04956358685c2869358db8b6d46d16cf9744b03 since it failed to produce output. =============================== Cleanup =============================== Removing all generated output REMOVING: 2ca3ddc3ece9ae81f9b9acc08d3be93c4aab01fb ================== Unresolved Commit Status ================== Remaining Unresolved Commits (oldest commit is listed first): e04956358685c2869358db8b6d46d16cf9744b03 5512bff0caa1b179b2a03a5512fc3de2c866b0cb e1ca53920f7095d09c5dde5b9502f4056c481b48 0ed15803474cd7d3d8228a642e9e2599a840707d 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 955aa780e3a47b6accf7af68d85f76ad53baff39 ================== FAILED Commits ================== These are the commits which entirely failed to produce output: 955aa780e3a47b6accf7af68d85f76ad53baff39 e04956358685c2869358db8b6d46d16cf9744b03 5512bff0caa1b179b2a03a5512fc3de2c866b0cb e1ca53920f7095d09c5dde5b9502f4056c481b48 0ed15803474cd7d3d8228a642e9e2599a840707d 86f7a6081cc525e6d953cebc42d4e5962d7e24f2 Exiting with code: 1 [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] script [Pipeline] { [Pipeline] emailext Sending email to: messnermet@uwm.edu [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: script returned exit code 1 Finished: FAILURE