Started by an SCM change Obtained jenkins_tests/clubb_column_mirror/Jenkinsfile from git https://github.com/larson-group/clubb.git ha:////4Cjittq30hkYZjMOAEpajcZR0V8yygtTgYIvukrI0KfJAAAAoh+LCAAAAAAAAP9tjTEOwjAQBM8BClpKHuFItIiK1krDC0x8GCfWnbEdkooX8TX+gCESFVvtrLSa5wtWKcKBo5UdUu8otU4GP9jS5Mixv3geZcdn2TIl9igbHBs2eJyx4YwwR1SwULBGaj0nRzbDRnX6rmuvydanHMu2V1A5c4MHCFXMWcf8hSnC9jqYxPTz/BXAFEIGsfuclm8zQVqFvQAAAA==[Pipeline] Start of Pipeline ha:////4OfoazyckbArggUo3ISfTGPxmxxRdP+stEFkv8dA98gsAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycohUghExsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jduZBmjwAAAAA==[Pipeline] node Running on ha:////4G0rVGZLwo0Ou9gETTdp4+2MbPAapBxm7bvz8E5cWqs8AAAAnB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAzWEgYB/eT83ILSktQi/dzMkuQMfQBx0n6exQAAAA==mitch in /home/jenkins/workspace/clubb_column_mirror ha:////4FDlcnFr9e5989JzoK959XjJGyMMLx4dByuovsdbc3KZAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0gA0xsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jfoP95RwAAAAA==[Pipeline] { ha:////4AE7Ft0kRt6yDHyl9I4e/bM1ttJPxUELiw6uNZ0QbR7MAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0gQkxsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jc09154wAAAAA==[Pipeline] stage ha:////4FkD70+B6a5J0827ONeUqfTiQDpR4+VE+gWaygTpm9OWAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0ggUxsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jek7ggRwAAAAA==[Pipeline] { (Declarative: Checkout SCM) ha:////4DWhhCSzqWCnqMgye1segLjeKHvm/Xy9xZ/t+WlCd57ZAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQTbaWGsbAmNJ0AWEZb8zwLrbuWJvJp3kLiJlZNMMm+a93rDOic4UbLcG+wdZu14DKOti0+U+lugiXu6ck2YKRguzSSpM+cFJRUDS1gDKwEbgzpQdmgLbIVXD9UGhba9lFS/o4DGdQM8gYlqLiqVL8wJdvexy4Q/z18BzLEA29ce4gfg7KmOvAAAAA==[Pipeline] checkout The recommended git tool is: git Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/larson-group/clubb.git > git init /home/jenkins/workspace/clubb_column_mirror # timeout=10 Fetching upstream changes from https://github.com/larson-group/clubb.git > git --version # timeout=10 > git --version # 'git version 2.43.0' using GIT_ASKPASS to set credentials larsongroupsysadmin github token > git fetch --tags --force --progress -- https://github.com/larson-group/clubb.git +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch Checking out Revision a8c952996409f8c94d6a28714171145ed7d2c2ae (refs/remotes/origin/master) Commit message: "Fixed bug in cubic equation solve function. Only relevant if pdf is L… (#1387)" > git config remote.origin.url https://github.com/larson-group/clubb.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > git config core.sparsecheckout # timeout=10 > git checkout -f a8c952996409f8c94d6a28714171145ed7d2c2ae # timeout=10 > git rev-list --no-walk 4292a70f53a6b6c4eab864fe27b8fd9af5d86909 # timeout=10 ha:////4M0lBXi8L0O4N1wFY5lRb5LBLalDgJBC9nNrfR5OtcBNAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQTbGRNjZUtoPAGyiLDkfxZYdytP5NW8g8RNrJxkknnTvNcb1jnBiZLl3mDvMGvHYxhtXXyi1N8CTdzTlWvCTMFwaSZJnTkvKKkYWMIaWAnYGNSBskNbYCu8eqg2KLTtpaT6HQU0rhvgCUxUc1GpfGFOsLuPXSb8ef4KYI6xADvU7j9Dg2gqvAAAAA==[Pipeline] } ha:////4C9gVc/RMYuh4P3VFxhWoDv34M8ROVTApORMFM74AFZeAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQRbWRhjZUtoPAGyiLDkfxZYdytP5NW8g8RNrJxkknnTvNcb1jnBiZLl3mDvMGvHYxhtXXyi1N8CTdzTlWvCTMFwaSZJnTkvKKkYWMIaWAnYGNSBskNbYCu8eqg2KLTtpaT6HQU0rhvgCUxUc1GpfGFOsLuPXSb8ef4KYI6xADvU7j9J+wGOvAAAAA==[Pipeline] // stage ha:////4OMUaFGtWHpXHrnJEqNuTaTTH7zYW1TXuR1OOwesYqpuAAAAph+LCAAAAAAAAP9tjTEOwjAQBM9BKWgpeYQDEh2iorXc8AITG+PEugv2haTiRXyNPxCIRMVWOyut5vmCMic4UPKycdgGzHWQXez91ORAqb1EGmRDZ1kTZopOajdosu44oyZ2MEcUsFCwdFhHygE9w0o15m6qaNBXJ07TtldQBHuDBwg1mdkk/sKYYH3tbSb8ef4KYOwYxI6h2G4+x/INtuQqUcEAAAA=[Pipeline] withEnv ha:////4POBlF/w66TdRjLOMaZmbKoF4qlP3EsxWZ8MwJufSwVuAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+DOrAycoiUzoiJNerCCUITQtLo/zZJSSdOxNW4Ay2VmPBg2Zas93pDGQOcKBjuNHYWY2t570czJ54pdDdPmTu68pYwkte80bkhpc9rbShpWFUw2AjYamw9RYsmwU44+ZCVl2iqSwrzdhTArBrgCYWYyUmG9C1TgP19VJHwx/kLgKlPwOrDYvXyLD8BobDcwgAAAA==[Pipeline] { ha:////4CZMEuZltkWrpbK+PirOYQRSCqpH7y+OE712fANE7n/gAAAAph+LCAAAAAAAAP9tjTEOwjAUQ3+LOrAycohUZUVMrFEWThCaEJJG/7dJSjpxIq7GHWipxIQHy7ZkvdcbqhjgRMEwp7GzGFvLej+aObFMobt5yszRlbWEkbxmQmdBSp/XKihpWFWUsOGw1dh6ihZNgh138iFrL9HUlxTm7cihtGqAJxR8JicZ0rdMAfb3UUXCH+cvAKY+Qdk0ix2WZ/UBbwyqm8IAAAA=[Pipeline] stage ha:////4KUMCWFfscl/TOIZTM5EUNA/xqZlP8Dml1BjSoC8QS+sAAAAph+LCAAAAAAAAP9tjTEOwjAUQ3+LOrAycohUwIiYukZZOEFoQkgb/V9+UtqJE3E17kBLJSY8WLYl673eUESGE7ETjcXWY6y96ELvpiQG4vYaaBANXURNGClYoeygyNhqqYqShUVZDisJa4t1oOjRJdjIRj90GTS68px42o4Scm/u8IRMTuSkOX3LyLC99SYS/jh/ATB2CfLdfrbD/Cw+y7kUycIAAAA=[Pipeline] { (Run Column Mirror Test) ha:////4ANXFJd3P2uH6oG6fHcMxklx/Urd1yvPwqSjWMs4SBztAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiBB2iSmu54QUmNsaJdWdsh6TiRXyNPxCIRMUWq51t5vmCZYpw5GixNdQ5So3D4Hs7LRw4dhfPA7Z8xoYpsTcozSBZm3pGydnAnKKEhYCVocZzcmQzrEWr7qryimx1ynH6DgJKp2/wgEJM5qxi/sIYYXPtdWL6ef4KYAwZyu3uU/vwBgq0rvS+AAAA[Pipeline] sh + source /etc/profile.d/larson-group.sh ++ export GIT_EDITOR=vi ++ GIT_EDITOR=vi ++ export SVN_EDITOR=vi ++ SVN_EDITOR=vi ++ export OMP_STACKSIZE=1048579 ++ OMP_STACKSIZE=1048579 ++ export LMOD_ROOT=/opt/lmod/ ++ LMOD_ROOT=/opt/lmod/ ++ source /opt/lmod//lmod/lmod/init/bash +++ '[' -z '' ']' +++ case "$-" in +++ __lmod_vx=x +++ '[' -n x ']' +++ set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for this output (/usr/local/spack/opt/spack/linux-pop22-skylake_avx512/gcc-12.2.0/lmod-8.7.37-fq24mybyn2li6got2bxzk62ejh5d3p4z/lmod/lmod/init/bash) Shell debugging restarted +++ unset __lmod_vx +++ find /usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core -print -quit ++ export MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core ++ MODULEPATH=/usr/local/spack/share/spack/lmod/linux-pop22-x86_64/Core + module load gcc netcdf-fortran + '[' -z '' ']' + case "$-" in + __lmod_sh_dbg=x + '[' -n x ']' + set +x Shell debugging temporarily silenced: export LMOD_SH_DBG_ON=1 for Lmod's output Lmod Warning: ------------------------------------------------------------------------------- The following dependent module(s) are not currently loaded: gcc-runtime/13.1.0-wiuv7wc (required by: mpich/4.2.1, hdf5/1.14.3-nhc23hl, netcdf-c/4.9.2-xduzp77, netcdf-fortran/4.5.3), glibc/2.35-tdr5czg (required by: mpich/4.2.1, hdf5/1.14.3-nhc23hl, netcdf-c/4.9.2-xduzp77, netcdf-fortran/4.5.3) ------------------------------------------------------------------------------- Shell debugging restarted + unset __lmod_sh_dbg + return 0 + ulimit -s 8388608 + python3 tests/check_mirrored_multi_col_output.py -multicol 3 -workers 2 Installing uv 0.11.32 in /home/jenkins/workspace/clubb_column_mirror/.clubb-python-tools/bin downloading uv 0.11.32 x86_64-unknown-linux-gnu installing to /home/jenkins/workspace/clubb_column_mirror/.clubb-python-tools/bin uv uvx everything's installed! Creating CLUBB virtualenv: /home/jenkins/workspace/clubb_column_mirror/.venv-python Using CPython 3.12.3 interpreter at: /usr/bin/python3.12 Creating virtual environment at: .venv-python Activate with: source .venv-python/bin/activate Installing python requirements into /home/jenkins/workspace/clubb_column_mirror/.venv-python Using Python 3.12.3 environment at: .venv-python Resolved 12 packages in 351ms Downloading pygments (1.2MiB) Downloading meson (1.1MiB) Downloading numpy (16.0MiB) Downloading netcdf4 (9.7MiB) Downloading cftime (1.6MiB) Downloaded cftime Downloaded meson Downloaded pygments Downloaded netcdf4 Downloaded numpy Prepared 12 packages in 674ms Installed 12 packages in 27ms + certifi==2026.7.22 + cftime==1.6.6 + iniconfig==2.3.1 + meson==1.12.1 + netcdf4==1.7.4 + ninja==1.13.2 + numpy==2.5.4 + packaging==26.3 + pluggy==1.6.0 + pygments==2.21.0 + pytest==9.1.1 + tabulate==0.10.0 Ready: /home/jenkins/workspace/clubb_column_mirror/.venv-python/bin/python Logs and output: /home/jenkins/workspace/clubb_column_mirror/output/column_mirror_test/run_yhotx_79 [1/4] Configuring and compiling (gfortran, CPU) [2/4] ABC run Cases: arm, arm_97, astex_a209, atex, bomex, cgils_s6, cgils_s11, cgils_s12, clex9_nov02, clex9_oct14, dycoms2_rf01, dycoms2_rf01_fixed_sst, dycoms2_rf02_do, dycoms2_rf02_ds, dycoms2_rf02_nd, dycoms2_rf02_so, fire, gabls2, gabls3, gabls3_night, jun25_altocu, lba, mc3e, mpace_a, mpace_b, mpace_b_silhs, nov11_altocu, rico, rico_silhs, twp_ice, wangara Grid: -multicol C8/0.2:0.8/3 C8 = [0.2, 0.5, 0.8] [3/4] CBA run Cases: arm, arm_97, astex_a209, atex, bomex, cgils_s6, cgils_s11, cgils_s12, clex9_nov02, clex9_oct14, dycoms2_rf01, dycoms2_rf01_fixed_sst, dycoms2_rf02_do, dycoms2_rf02_ds, dycoms2_rf02_nd, dycoms2_rf02_so, fire, gabls2, gabls3, gabls3_night, jun25_altocu, lba, mc3e, mpace_a, mpace_b, mpace_b_silhs, nov11_altocu, rico, rico_silhs, twp_ice, wangara Grid: -multicol C8/0.2:0.8/3 (columns reversed) C8 = [0.8, 0.5, 0.2] [4/4] Comparison PASS arm_97 PASS arm PASS astex_a209 PASS atex PASS bomex PASS cgils_s11 PASS cgils_s12 PASS cgils_s6 PASS clex9_nov02 PASS clex9_oct14 PASS dycoms2_rf01_fixed_sst PASS dycoms2_rf01 PASS dycoms2_rf02_do PASS dycoms2_rf02_ds PASS dycoms2_rf02_nd PASS dycoms2_rf02_so PASS fire PASS gabls2 PASS gabls3_night PASS gabls3 PASS jun25_altocu PASS lba PASS mc3e PASS mpace_a PASS mpace_b_silhs PASS mpace_b PASS nov11_altocu PASS rico_silhs PASS rico PASS twp_ice PASS wangara ha:////4HhaSfPPAiGCqYCWgOSOnTKEXCDVzERShhXph91HbicNAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIihKgQVVrLDS8wsTFOrDtjOyQVL+Jr/IFAJCq2WO1sM88XLFOEI0eLraHOUWocBt/baeHAsbt4HrDlMzZMib1BaQbJ2tQzSs4G5hQlLASsDDWekyObYS1adVeVV2SrU47TdxBQOn2DBxRiMmcV8xfGCJtrrxPTz/NXAGMIGcrt/lO7N7BnLt++AAAA[Pipeline] } ha:////4FVL20V4kFZmoupJ3yD+ARu1qGTPdJik5rudocilzqMlAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiKGgQVVrLDS8wsTFOrDtjOyQVL+Jr/IFAJCq2WO1sM88XLFOEI0eLraHOUWocBt/baeHAsbt4HrDlMzZMib1BaQbJ2tQzSs4G5hQlLASsDDWekyObYS1adVeVV2SrU47TdxBQOn2DBxRiMmcV8xfGCJtrrxPTz/NXAGMIGcrt/lO7N4glvS6+AAAA[Pipeline] // stage ha:////4FIJv5aYNj9RZ9LGa/7HbPrjKWUC3ws2EMS4om8Z+kWnAAAAox+LCAAAAAAAAP9tjTESwiAURH/iWNhaeggyqSwcK1uGxhNggghh/idAJJUn8mreQTQzVm6xs2+b93zBOgY4UtDMKhwMxs4w7yZdFssUhqujzCxdWEcYySkmVBbUq9OCgpKCJVUNKw4bhZ2jaFAn2HIr77JxEnVzTqF8Bw616Ud4QMWLOcmQvjAH2N2mPhL+PH8FMHufoG73n2rfl6fofL4AAAA=[Pipeline] } ha:////4Hr0qt3QJRHLTmkTa25Bgo6/aSivkdT3OmFC9mwZg5rmAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOICHRKiSmu54QUmNsaJdWdsh6TiRXyNPxCIRMUWq51t5vmCZYpw5GixNdQ5So3D4Hs7LRw4dhfPA7Z8xoYpsTcozSBZm3pGydnAnKKEhYCVocZzcmQzrEWr7qryimx1ynH6DgJKp2/wgEJM5qxi/sIYYXPtdWL6ef4KYAwhQ7nbf2r7BtQlufq+AAAA[Pipeline] // withEnv ha:////4OtOZIIm8cxFks9vP+yIYgHBWA+ZjTnNDhB5oUpXZZfYAAAApB+LCAAAAAAAAP9tjTEOwjAQBDdBFLSUPMIREg1CVLRWGl5gEmOcWHfBvpBUvIiv8QciIlGx1c4083pjmSKOHJ1qLLWeUuVVF3o3PTVwbK+BB9XwRVVMiYNVpR1Kru1pxpLFYl6WY6GxslQFTp6cYK0b8zBFMOSKs8TJHTRyX9/xRKanspgoXxgjNre+Tky/zt8Axq4T5Nu9INt9APRJVd+9AAAA[Pipeline] } ha:////4PsaE0auzp35cP/2Ybbyn+bNsrNW0x7Sv/SOTaJZoPqVAAAAoh+LCAAAAAAAAP9tjTEOwjAQBDdBFLSUPMJBUCIqWisNLzCJMU6su2BfSCpexNf4AxGRqNhqZ5p5vbFMEUeOTjWWWk+p8qoLvZueGji218CDaviiKqbEwarSDiXX9jRjyWIxL8ux0FhZqgInT06w1o15mCIYcsVZ4uQOGrmv73gi01NZTJQvjBGbW18npl/nbwBj1wny3VaQ7T+IzLXKvQAAAA==[Pipeline] // node ha:////4CEVHy/x8mt+tKte6MR4Mh2/XAZkgszUHqNq0IfxTwN6AAAAoh+LCAAAAAAAAP9tjTESgjAQRT84FraWHiKMtI6VbYbGE0SIMZDZxWQRKk/k1byDjMxY+av/XvNeb6xTxJGjU62lzlOqverD4OanRo7dNfCoWr6omilxsKqyY8WNPS1YsVgsy3KsNDaW6sDJkxNsdWsepgiGXHGWOLuDRu6bO57I9FwWE+ULU8TuNjSJ6df5G8DU94K83Auy8gOu1dKAvQAAAA==[Pipeline] End of Pipeline Finished: SUCCESS