Started by an SCM change Obtained jenkins_tests/clubb_BFB_multi_col_gpu_nvhpc_test/Jenkinsfile from git https://github.com/larson-group/clubb.git [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout The recommended git tool is: git Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/larson-group/clubb.git > git init /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test # timeout=10 Fetching upstream changes from https://github.com/larson-group/clubb.git > git --version # timeout=10 > git --version # 'git version 2.34.1' using GIT_ASKPASS to set credentials larsongroupsysadmin github token > git fetch --tags --force --progress -- https://github.com/larson-group/clubb.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/larson-group/clubb.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision 034f61908b84f2574a06cc7da3db70d50696e69d (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 034f61908b84f2574a06cc7da3db70d50696e69d # timeout=10 Commit message: "The old mixing length calculation seems to be making dycoms2_rf02 fail when using the bicgstab solver. This case was always unstable with bicgstab, so it seemed more like luck before that it was passing. This change just removes the failed output data to prevent pyplotgen from crashing" > git rev-list --no-walk 765ae7c30cda88e01b27cd1ee4277d54e4fcb752 # timeout=10 [Pipeline] } [Pipeline] // stage [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (Update sys_admin Repo) [Pipeline] sh + git -C /home/pub/jenkins_BFB_test_configs/sys_admin pull Already up to date. [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Run Python Test) [Pipeline] lock Trying to acquire lock on [Resource: GPU_built-in] The resource [GPU_built-in] is locked by build clubb_plot_nvhpc_gpu_vs_cpu #204 #204 since Jul 29, 2026, 3:15 AM. [Resource: GPU_built-in] is not free, waiting for execution ... [Required resources: [GPU_built-in]] added into queue at position 0 Lock acquired on [Resource: GPU_built-in] [Pipeline] { [Pipeline] sh + ulimit -s 8388608 + source /etc/profile.d/larson-group.sh ++ export GIT_EDITOR=vi ++ GIT_EDITOR=vi ++ export SVN_EDITOR=vi ++ SVN_EDITOR=vi ++ export OMP_STACKSIZE=1048579 ++ OMP_STACKSIZE=1048579 ++ export LMOD_ROOT=/opt/lmod/ ++ LMOD_ROOT=/opt/lmod/ ++ source /opt/lmod//lmod/lmod/init/bash +++ '[' -z '' ']' +++ case "$-" in +++ __lmod_vx=x +++ '[' -n x ']' +++ set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for this output (/usr/local/spack/opt/spack/linux-pop22-cascadelake/gcc-12.2.0/lmod-8.7.37-bi3kyxcdrfgw3y7vv2k7c5rjxg75qzju/lmod/lmod/init/bash) Shell debugging restarted +++ unset __lmod_vx +++ find /usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core -print -quit ++ export MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core ++ MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core + module load nvhpc netcdf-fortran + '[' -z '' ']' + case "$-" in + __lmod_sh_dbg=x + '[' -n x ']' + set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for Lmod's output Shell debugging restarted + unset __lmod_sh_dbg + return 0 + python3 /home/pub/jenkins_BFB_test_configs/sys_admin/python_nightly_test_suite/run_test.py -c /home/pub/jenkins_BFB_test_configs/clubb_BFB_multi_col_gpu_nvhpc_test_config CMake Warning: Manually-specified variables were not used by the project: NetCDF_C_INCLUDE_DIR NetCDF_C_LIBRARY NetCDF_Fortran_INCLUDE_DIR NetCDF_Fortran_LIBRARY Directory 1 is 034f61908b84f2574a06cc7da3db70d50696e69d Directory 2 is 765ae7c30cda88e01b27cd1ee4277d54e4fcb752 Using reporting threshold: 0.0 The following cases will be compared: ['arm', 'cobra', 'gabls3', 'mpace_b', 'rico_silhs'] ================================================ arm ================================================ >No differences detected by the linux diff in arm_stats.nc< >>>The linux diff could not detect any differences in the file pairs for case arm.<<< ===================================================================================================== ================================================ cobra ================================================ >No differences detected by the linux diff in cobra_stats.nc< >>>The linux diff could not detect any differences in the file pairs for case cobra.<<< ===================================================================================================== ================================================ gabls3 ================================================ >No differences detected by the linux diff in gabls3_stats.nc< >>>The linux diff could not detect any differences in the file pairs for case gabls3.<<< ===================================================================================================== ================================================ mpace_b ================================================ >No differences detected by the linux diff in mpace_b_stats.nc< >>>The linux diff could not detect any differences in the file pairs for case mpace_b.<<< ===================================================================================================== ================================================ rico_silhs ================================================ >No differences detected by the linux diff in rico_silhs_stats.nc< >>>The linux diff could not detect any differences in the file pairs for case rico_silhs.<<< ===================================================================================================== SUMMARY: Linux diff did not detect any differences in the compared files. Pass: 5 cases ['arm', 'cobra', 'gabls3', 'mpace_b', 'rico_silhs'] Fail: 0 cases [] =============================== Configuring Test =============================== Working directory set to: /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test Checking specified repo: /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test Using /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test with origin https://github.com/larson-group/clubb.git Output Save Mode: all Using branch: master Making list from 'UnresolvedCommits': CheckRange = True: Getting commit range Run Output Path: output Run commands: ['python compile.py -gpu openacc -debug', '\nrm -rf output', '\nmkdir output', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 arm', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 rico_silhs', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 cobra', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 mpace_b', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 gabls3'] Diffing command: run_scripts/run_bindiff_all.py =============================== Running Test =============================== Parsing message from commit: c15480a71ed50936412eb66827a86bf69c490a37 Found: RESOLVED:8ab3902e812c2883976b4f1f57c085d682666d92 Parsing message from commit: 034f61908b84f2574a06cc7da3db70d50696e69d Checking Commit: c15480a71ed50936412eb66827a86bf69c490a37 Skipping: not final commit and not labelled BIT_CHANGING Checking Commit: 034f61908b84f2574a06cc7da3db70d50696e69d Running: final commit to check, not labelled BIT_CHANGING Checking out 034f61908b84f2574a06cc7da3db70d50696e69d Running: 'python compile.py -gpu openacc -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 arm' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 rico_silhs' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 cobra' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 mpace_b' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 gabls3' Checking out 765ae7c30cda88e01b27cd1ee4277d54e4fcb752 Running: 'python compile.py -gpu openacc -debug' Running: 'rm -rf output' Running: 'mkdir output' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 arm' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 rico_silhs' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 cobra' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 mpace_b' Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 gabls3' Diffing: 'run_scripts/run_bindiff_all.py 034f61908b84f2574a06cc7da3db70d50696e69d 765ae7c30cda88e01b27cd1ee4277d54e4fcb752' =============================== Test Results =============================== Commit Range Checked = 765ae7c30cda88e01b27cd1ee4277d54e4fcb752...034f61908b84f2574a06cc7da3db70d50696e69d No new Unresolved commits found. No fake BIT_CHANGING commits found. WARNING: 8ab3902e812c2883976b4f1f57c085d682666d92 labelled as RESOLVED, but was not listed as unresolved. No remaining Unresolved Commits. =============================== Updating Config File =============================== Saving into UnresolvedCommits: Saving into BaselineCommit: 034f61908b84f2574a06cc7da3db70d50696e69d =============================== Cleanup =============================== Not removing any generated output. RENAMING 034f61908b84f2574a06cc7da3db70d50696e69d to latest_revision_validated_034f61908b84f2574a06cc7da3db70d50696e69d ================== Unresolved Commit Status ================== No remaining Unresolved Commits. Exiting with code: 0 [Pipeline] } Lock released on resource [Resource: GPU_built-in] [Pipeline] // lock [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS