Started by an SCM change Obtained jenkins_tests/cam_coarse_res_r8029_flags_bin_diff_regression_gfortran_test/Jenkinsfile from git https://github.com/larson-group/cam.git [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /home/jenkins/workspace/cam_coarse_res_r8029_flags_bin_diff_regression_gfortran_test [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout The recommended git tool is: git > git rev-parse --resolve-git-dir /home/jenkins/workspace/cam_coarse_res_r8029_flags_bin_diff_regression_gfortran_test/.git # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/larson-group/cam.git # timeout=10 Fetching upstream changes from https://github.com/larson-group/cam.git > git --version # timeout=10 > git --version # 'git version 2.34.1' using GIT_ASKPASS to set credentials A token based key used by Jenkins to preform Github actions, created 6/21/2021 > git fetch --tags --force --progress -- https://github.com/larson-group/cam.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse refs/remotes/origin/clubb_silhs_devel^{commit} # timeout=10 Checking out Revision a6e8f378bc25c7cb5627d18a512fc00e8328c873 (refs/remotes/origin/clubb_silhs_devel) > git config core.sparsecheckout # timeout=10 > git checkout -f a6e8f378bc25c7cb5627d18a512fc00e8328c873 # timeout=10 Commit message: "Autoupdated CLUBB_core Commit 3f865924bdcf798fa45acaa67b57c44f40c32d89 Author: Gunther Huebler Date: Wed Jul 31 17:30:51 2024 -0500 Moving pdf_params copying to copyin. This is the data structure that contains pointers, and these pointers will be different on the host and device, so it is bad in theory to copy the structure back to the CPU as it might overwrite cpu memory pointers with gpu memory pointers. In practice though I've seen no problems caused by this, I'm just making this commit preemptively, and it has already been tested with the ECT test." > git rev-list --no-walk c0a710c144a2cb7367fd07d20be1f9ab92757a45 # timeout=10 [Pipeline] } [Pipeline] // stage [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (Clone SysAdmin to tmp) [Pipeline] sh + rm -rf /home/jenkins/cam_r8029_flags_bit_diff_test_workspace/sys_admin + git clone https://github.com/larson-group/sys_admin.git /home/jenkins/cam_r8029_flags_bit_diff_test_workspace/sys_admin Cloning into '/home/jenkins/cam_r8029_flags_bit_diff_test_workspace/sys_admin'... [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Run Python Test) [Pipeline] sh + source /etc/profile.d/larson-group.sh ++ export GIT_EDITOR=vi ++ GIT_EDITOR=vi ++ export SVN_EDITOR=vi ++ SVN_EDITOR=vi ++ export OMP_STACKSIZE=1048579 ++ OMP_STACKSIZE=1048579 ++ export LMOD_ROOT=/usr/local/spack/opt/spack/linux-pop22-cascadelake/gcc-12.2.0/lmod-8.7.37-bi3kyxcdrfgw3y7vv2k7c5rjxg75qzju ++ LMOD_ROOT=/usr/local/spack/opt/spack/linux-pop22-cascadelake/gcc-12.2.0/lmod-8.7.37-bi3kyxcdrfgw3y7vv2k7c5rjxg75qzju ++ source /usr/local/spack/opt/spack/linux-pop22-cascadelake/gcc-12.2.0/lmod-8.7.37-bi3kyxcdrfgw3y7vv2k7c5rjxg75qzju/lmod/lmod/init/bash +++ '[' -z '' ']' +++ case "$-" in +++ __lmod_vx=x +++ '[' -n x ']' +++ set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for this output (/usr/local/spack/opt/spack/linux-pop22-cascadelake/gcc-12.2.0/lmod-8.7.37-bi3kyxcdrfgw3y7vv2k7c5rjxg75qzju/lmod/lmod/init/bash) Shell debugging restarted +++ unset __lmod_vx ++ export MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core ++ MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core + rm -rf /home/jenkins/cam_output/scratch/UWM_debug_r8029_flags_f10_f10_mg37 + python3 /home/jenkins/cam_r8029_flags_bit_diff_test_workspace/sys_admin/python_nightly_test_suite/run_test.py -c /home/jenkins/cam_r8029_flags_bit_diff_test_workspace/bit_diff_cam_r8029_config ERROR: Unknown Job Queue specified use --force to set submit_jobs case.run Submit job case.run No timing file found in run directory check for resubmit dout_s False mach larson-group resubmit_num 0 Traceback (most recent call last): File "/home/jenkins/cam_r8029_flags_bit_diff_test_workspace/repo/run_scripts/set_up_cam_coarse_res.py", line 78, in <module> os.mkdir("ccs_config/machines/larson-group") FileExistsError: [Errno 17] File exists: 'ccs_config/machines/larson-group' ERROR: Unknown Job Queue specified use --force to set submit_jobs case.run Submit job case.run No timing file found in run directory check for resubmit dout_s False mach larson-group resubmit_num 0 PASS: All 3D output vars are zero. =============================== Configuring Test =============================== Working directory set to: /home/jenkins/cam_r8029_flags_bit_diff_test_workspace Trying to clone repo... Cloned repo location: /home/jenkins/cam_r8029_flags_bit_diff_test_workspace/repo Output Save Mode: none Using branch: clubb_silhs_devel Making list from 'UnresolvedCommits': Run Output Path: /home/jenkins/cam_output/scratch/UWM_debug_r8029_flags_f10_f10_mg37_larson-group_gnu/run/UWM_debug_r8029_flags_f10_f10_mg37_larson-group_gnu.cam.rh0.0001-01-02-00000.nc Run commands: ['rm -rf /home/jenkins/cam_output/scratch/UWM_debug_r8029_flags_f10_f10_mg37', '\nrepo/run_scripts/set_up_cam_coarse_res.py --caseroot /home/$USER/cam_output/caseroot/$CASE --reuse_build repo', '\nrepo/run_scripts/run_cesm_uwm_coarse_res_r8029_flags.sh'] Diffing command: python3 repo/run_scripts/netcdf_3D_vars_diff.py =============================== Running Test =============================== Parsing message from commit: a6e8f378bc25c7cb5627d18a512fc00e8328c873 Checking Commit: a6e8f378bc25c7cb5627d18a512fc00e8328c873 Running: final commit to check, not labelled BIT_CHANGING Checking out a6e8f378bc25c7cb5627d18a512fc00e8328c873 Running: 'rm -rf /home/jenkins/cam_output/scratch/UWM_debug_r8029_flags_f10_f10_mg37' Running: ' repo/run_scripts/set_up_cam_coarse_res.py --caseroot /home/$USER/cam_output/caseroot/$CASE --reuse_build repo' Running: ' repo/run_scripts/run_cesm_uwm_coarse_res_r8029_flags.sh' Checking out c0a710c144a2cb7367fd07d20be1f9ab92757a45 Running: 'rm -rf /home/jenkins/cam_output/scratch/UWM_debug_r8029_flags_f10_f10_mg37' Running: ' repo/run_scripts/set_up_cam_coarse_res.py --caseroot /home/$USER/cam_output/caseroot/$CASE --reuse_build repo' Running: ' repo/run_scripts/run_cesm_uwm_coarse_res_r8029_flags.sh' Diffing: 'python3 repo/run_scripts/netcdf_3D_vars_diff.py a6e8f378bc25c7cb5627d18a512fc00e8328c873 c0a710c144a2cb7367fd07d20be1f9ab92757a45' =============================== Test Results =============================== Commit Range Checked = c0a710c144a2cb7367fd07d20be1f9ab92757a45...a6e8f378bc25c7cb5627d18a512fc00e8328c873 No new Unresolved commits found. No fake BIT_CHANGING commits found. No remaining Unresolved Commits. =============================== Updating Config File =============================== Saving into UnresolvedCommits: Saving into BaselineCommit: a6e8f378bc25c7cb5627d18a512fc00e8328c873 =============================== Cleanup =============================== Removed repo. Removing all generated output REMOVING: c0a710c144a2cb7367fd07d20be1f9ab92757a45 REMOVING: a6e8f378bc25c7cb5627d18a512fc00e8328c873 ================== Unresolved Commit Status ================== No remaining Unresolved Commits. Exiting with code: 0 [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] script [Pipeline] { [Pipeline] sh + chmod -R 755 /home/jenkins/cam_output [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS