Started by an SCM change
Obtained jenkins_tests/clubb_BFB_multi_col_gpu_nvhpc_test/Jenkinsfile from git <a href='https://github.com/larson-group/clubb.git'>https://github.com/larson-group/clubb.git</a>
<span class="pipeline-new-node" nodeId="2" startId="2">[Pipeline] Start of Pipeline
</span><span class="pipeline-new-node" nodeId="3" startId="3" enclosingId="2">[Pipeline] node
</span><span class="pipeline-node-3">Running on <a href='/jenkins/computer/(built-in)/' class='jenkins-table__link model-link model-link--float'>Jenkins</a> in /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test
</span><span class="pipeline-new-node" nodeId="4" startId="4" enclosingId="3">[Pipeline] {
</span><span class="pipeline-new-node" nodeId="5" startId="5" enclosingId="4">[Pipeline] stage
</span><span class="pipeline-new-node" nodeId="6" startId="6" enclosingId="5" label="Declarative: Checkout SCM">[Pipeline] { (Declarative: Checkout SCM)
</span><span class="pipeline-new-node" nodeId="7" enclosingId="6">[Pipeline] checkout
</span><span class="pipeline-node-7">The recommended git tool is: git
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository <a href='https://github.com/larson-group/clubb.git'>https://github.com/larson-group/clubb.git</a>
 &gt; git init /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test # timeout=10
Fetching upstream changes from <a href='https://github.com/larson-group/clubb.git'>https://github.com/larson-group/clubb.git</a>
 &gt; git --version # timeout=10
 &gt; git --version # 'git version 2.34.1'
using GIT_ASKPASS to set credentials larsongroupsysadmin github token
 &gt; git fetch --tags --force --progress -- <a href='https://github.com/larson-group/clubb.git'>https://github.com/larson-group/clubb.git</a> +refs/heads/*:refs/remotes/origin/* # timeout=10
 &gt; git config remote.origin.url <a href='https://github.com/larson-group/clubb.git'>https://github.com/larson-group/clubb.git</a> # timeout=10
 &gt; git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
 &gt; git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 034f61908b84f2574a06cc7da3db70d50696e69d (refs/remotes/origin/master)
 &gt; git config core.sparsecheckout # timeout=10
 &gt; git checkout -f 034f61908b84f2574a06cc7da3db70d50696e69d # timeout=10
Commit message: "The old mixing length calculation seems to be making dycoms2_rf02 fail when using the bicgstab solver. This case was always unstable with bicgstab, so it seemed more like luck before that it was passing. This change just removes the failed output data to prevent pyplotgen from crashing"
 &gt; git rev-list --no-walk 765ae7c30cda88e01b27cd1ee4277d54e4fcb752 # timeout=10
</span><span class="pipeline-new-node" nodeId="8" startId="6">[Pipeline] }
</span><span class="pipeline-new-node" nodeId="9" startId="5">[Pipeline] // stage
</span><span class="pipeline-new-node" nodeId="10" startId="10" enclosingId="4">[Pipeline] withEnv
</span><span class="pipeline-new-node" nodeId="11" startId="11" enclosingId="10">[Pipeline] {
</span><span class="pipeline-new-node" nodeId="12" startId="12" enclosingId="11">[Pipeline] stage
</span><span class="pipeline-new-node" nodeId="13" startId="13" enclosingId="12" label="Update sys_admin Repo">[Pipeline] { (Update sys_admin Repo)
</span><span class="pipeline-new-node" nodeId="14" enclosingId="13">[Pipeline] sh
</span><span class="pipeline-node-14">+ git -C /home/pub/jenkins_BFB_test_configs/sys_admin pull
Already up to date.
</span><span class="pipeline-new-node" nodeId="15" startId="13">[Pipeline] }
</span><span class="pipeline-new-node" nodeId="16" startId="12">[Pipeline] // stage
</span><span class="pipeline-new-node" nodeId="17" startId="17" enclosingId="11">[Pipeline] stage
</span><span class="pipeline-new-node" nodeId="18" startId="18" enclosingId="17" label="Run Python Test">[Pipeline] { (Run Python Test)
</span><span class="pipeline-new-node" nodeId="19" startId="19" enclosingId="18">[Pipeline] lock
</span><span class="pipeline-node-19">Trying to acquire lock on [Resource: GPU_built-in]
The resource [GPU_built-in] is locked by build clubb_plot_nvhpc_gpu_vs_cpu #204 <a href='/jenkins/job/clubb_plot_nvhpc_gpu_vs_cpu/204/' class='jenkins-table__link model-link model-link--float'>#204</a> since Jul 29, 2026, 3:15 AM.
[Resource: GPU_built-in] is not free, waiting for execution ...
[Required resources: [GPU_built-in]] added into queue at position 0
Lock acquired on [Resource: GPU_built-in]
</span><span class="pipeline-new-node" nodeId="20" startId="20" enclosingId="19">[Pipeline] {
</span><span class="pipeline-new-node" nodeId="21" enclosingId="20">[Pipeline] sh
</span><span class="pipeline-node-21">+ ulimit -s 8388608
+ source /etc/profile.d/larson-group.sh
++ export GIT_EDITOR=vi
++ GIT_EDITOR=vi
++ export SVN_EDITOR=vi
++ SVN_EDITOR=vi
++ export OMP_STACKSIZE=1048579
++ OMP_STACKSIZE=1048579
++ export LMOD_ROOT=/opt/lmod/
++ LMOD_ROOT=/opt/lmod/
++ source /opt/lmod//lmod/lmod/init/bash
+++ '[' -z '' ']'
+++ case "$-" in
+++ __lmod_vx=x
+++ '[' -n x ']'
+++ set +x
Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for this output (/usr/local/spack/opt/spack/linux-pop22-cascadelake/gcc-12.2.0/lmod-8.7.37-bi3kyxcdrfgw3y7vv2k7c5rjxg75qzju/lmod/lmod/init/bash)
Shell debugging restarted
+++ unset __lmod_vx
+++ find /usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core -print -quit
++ export MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core
++ MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core
+ module load nvhpc netcdf-fortran
+ '[' -z '' ']'
+ case "$-" in
+ __lmod_sh_dbg=x
+ '[' -n x ']'
+ set +x
Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for Lmod's output
Shell debugging restarted
+ unset __lmod_sh_dbg
+ return 0
+ python3 /home/pub/jenkins_BFB_test_configs/sys_admin/python_nightly_test_suite/run_test.py -c /home/pub/jenkins_BFB_test_configs/clubb_BFB_multi_col_gpu_nvhpc_test_config
CMake Warning:
  Manually-specified variables were not used by the project:

    NetCDF_C_INCLUDE_DIR
    NetCDF_C_LIBRARY
    NetCDF_Fortran_INCLUDE_DIR
    NetCDF_Fortran_LIBRARY


Directory 1 is 034f61908b84f2574a06cc7da3db70d50696e69d
Directory 2 is 765ae7c30cda88e01b27cd1ee4277d54e4fcb752
Using reporting threshold:  0.0 


The following cases will be compared: ['arm', 'cobra', 'gabls3', 'mpace_b', 'rico_silhs']


================================================ arm ================================================
&gt;No differences detected by the linux diff in arm_stats.nc&lt;
&gt;&gt;&gt;The linux diff could not detect any differences in the file pairs for case arm.&lt;&lt;&lt;

=====================================================================================================

================================================ cobra ================================================
&gt;No differences detected by the linux diff in cobra_stats.nc&lt;
&gt;&gt;&gt;The linux diff could not detect any differences in the file pairs for case cobra.&lt;&lt;&lt;

=====================================================================================================

================================================ gabls3 ================================================
&gt;No differences detected by the linux diff in gabls3_stats.nc&lt;
&gt;&gt;&gt;The linux diff could not detect any differences in the file pairs for case gabls3.&lt;&lt;&lt;

=====================================================================================================

================================================ mpace_b ================================================
&gt;No differences detected by the linux diff in mpace_b_stats.nc&lt;
&gt;&gt;&gt;The linux diff could not detect any differences in the file pairs for case mpace_b.&lt;&lt;&lt;

=====================================================================================================

================================================ rico_silhs ================================================
&gt;No differences detected by the linux diff in rico_silhs_stats.nc&lt;
&gt;&gt;&gt;The linux diff could not detect any differences in the file pairs for case rico_silhs.&lt;&lt;&lt;

=====================================================================================================

SUMMARY:
Linux diff did not detect any differences in the compared files.
Pass: 5 cases ['arm', 'cobra', 'gabls3', 'mpace_b', 'rico_silhs']
Fail: 0 cases []

=============================== Configuring Test ===============================


Working directory set to: /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test
Checking specified repo: /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test

Using /home/jenkins/workspace/clubb_BFB_multi_col_gpu_nvhpc_test with origin <a href='https://github.com/larson-group/clubb.git'>https://github.com/larson-group/clubb.git</a>

Output Save Mode: all

Using branch: master

Making list from 'UnresolvedCommits':

CheckRange = True: Getting commit range

Run Output Path: output

Run commands: ['python compile.py -gpu openacc -debug', '\nrm -rf output', '\nmkdir output', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 arm', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 rico_silhs', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 cobra', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 mpace_b', '\npython run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 gabls3']

Diffing command: run_scripts/run_bindiff_all.py

=============================== Running Test ===============================

Parsing message from commit: c15480a71ed50936412eb66827a86bf69c490a37
	Found: RESOLVED:8ab3902e812c2883976b4f1f57c085d682666d92
Parsing message from commit: 034f61908b84f2574a06cc7da3db70d50696e69d

Checking Commit: c15480a71ed50936412eb66827a86bf69c490a37
	Skipping: not final commit and not labelled BIT_CHANGING

Checking Commit: 034f61908b84f2574a06cc7da3db70d50696e69d
	Running: final commit to check, not labelled BIT_CHANGING
	Checking out 034f61908b84f2574a06cc7da3db70d50696e69d
	Running: 'python compile.py -gpu openacc -debug'
	Running: 'rm -rf output'
	Running: 'mkdir output'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 arm'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 rico_silhs'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 cobra'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 mpace_b'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 gabls3'
	Checking out 765ae7c30cda88e01b27cd1ee4277d54e4fcb752
	Running: 'python compile.py -gpu openacc -debug'
	Running: 'rm -rf output'
	Running: 'mkdir output'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 arm'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 rico_silhs'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 cobra'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 mpace_b'
	Running: 'python run_scripts/run_scm.py -override lh_straight_mc=true -multicol 4 gabls3'
	Diffing: 'run_scripts/run_bindiff_all.py 034f61908b84f2574a06cc7da3db70d50696e69d 765ae7c30cda88e01b27cd1ee4277d54e4fcb752'

=============================== Test Results ===============================

Commit Range Checked = 765ae7c30cda88e01b27cd1ee4277d54e4fcb752...034f61908b84f2574a06cc7da3db70d50696e69d

No new Unresolved commits found.

No fake BIT_CHANGING commits found.

WARNING: 8ab3902e812c2883976b4f1f57c085d682666d92 labelled as RESOLVED,
	 but was not listed as unresolved.

No remaining Unresolved Commits.

=============================== Updating Config File ===============================


Saving into UnresolvedCommits: 

Saving into BaselineCommit: 034f61908b84f2574a06cc7da3db70d50696e69d

=============================== Cleanup ===============================

Not removing any generated output.
	RENAMING 034f61908b84f2574a06cc7da3db70d50696e69d to latest_revision_validated_034f61908b84f2574a06cc7da3db70d50696e69d

================== Unresolved Commit Status ==================

No remaining Unresolved Commits.

Exiting with code: 0
</span><span class="pipeline-new-node" nodeId="22" startId="20">[Pipeline] }
</span><span class="pipeline-node-22">Lock released on resource [Resource: GPU_built-in]
</span><span class="pipeline-new-node" nodeId="23" startId="19">[Pipeline] // lock
</span><span class="pipeline-new-node" nodeId="24" startId="18">[Pipeline] }
</span><span class="pipeline-new-node" nodeId="25" startId="17">[Pipeline] // stage
</span><span class="pipeline-new-node" nodeId="26" startId="11">[Pipeline] }
</span><span class="pipeline-new-node" nodeId="27" startId="10">[Pipeline] // withEnv
</span><span class="pipeline-new-node" nodeId="28" startId="4">[Pipeline] }
</span><span class="pipeline-new-node" nodeId="29" startId="3">[Pipeline] // node
</span><span class="pipeline-new-node" nodeId="30" startId="2">[Pipeline] End of Pipeline
</span>Finished: SUCCESS