Skip to content

Console Output

+ csh nightly_test/nightly_e3sm_ne4_silhs.csh

run_e3sm: ++++++++ run_e3sm starting (Thu Feb 10 14:41:12 CST 2022), version 3.0.22 ++++++++


run_e3sm: $case_name        = nelson.mg2_def.ne4_ne4

run_e3sm: -------- Starting create_newcase --------

run_e3sm: /home/jenkins/workspace/branch_e3sm_test//cime/scripts/create_newcase --case nelson.mg2_def.ne4_ne4 --compset FC5AV1C-L --script-root /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts --res ne4_ne4 --pecount M --handle-preexisting-dirs u --mach nelson --output-root /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4 --project condo
Compset longname is 2000_CAM5%AV1C-L_CLM45%SPBC_CICE%PRES_DOCN%DOM_SROF_SGLC_SWAV
Compset specification file is /home/jenkins/workspace/branch_e3sm_test/components/cam/cime_config/config_compsets.xml
Automatically adding SIAC to compset
Automatically adding SESP to compset
Compset forcing is 
Com forcing is present day:
ATM component is cam5 physics:CAM with complete set of E3SM atmospheric mods for V1 (72 layers model) and ACES4BGC SOAG emissions- v04p2:
LND component is clm4.5 physics:clm4.5 Satellite phenology:clm4.5 Satellite phenology with black carbon deposition:
ICE component is prescribed cice:
OCN component is DOCN   prescribed ocean mode
ROF component is Stub river component
GLC component is Stub glacier (land ice) component
WAV component is Stub wave component
IAC component is Stub iac component
ESP component is Stub external system processing (ESP) component
Pes     specification file is /home/jenkins/workspace/branch_e3sm_test/cime/config/e3sm/allactive/config_pesall.xml
Machine is nelson
Pes setting: grid          is a%ne4np4_l%ne4np4_oi%ne4np4_r%null_g%null_w%null_z%null_m%oQU240 
Pes setting: compset       is 2000_CAM5%AV1C-L_CLM45%SPBC_CICE%PRES_DOCN%DOM_SROF_SGLC_SWAV_SIAC_SESP 
Pes setting: tasks       is {'NTASKS_ATM': 16, 'NTASKS_ICE': 16, 'NTASKS_CPL': 16, 'NTASKS_LND': 16, 'NTASKS_WAV': 16, 'NTASKS_ROF': 16, 'NTASKS_OCN': 16, 'NTASKS_GLC': 16} 
Pes setting: threads     is {'NTHRDS_ICE': 1, 'NTHRDS_ATM': 1, 'NTHRDS_ROF': 1, 'NTHRDS_LND': 1, 'NTHRDS_WAV': 1, 'NTHRDS_OCN': 1, 'NTHRDS_CPL': 1, 'NTHRDS_GLC': 1} 
Pes setting: rootpe      is {'ROOTPE_OCN': 0, 'ROOTPE_LND': 0, 'ROOTPE_ATM': 0, 'ROOTPE_ICE': 0, 'ROOTPE_WAV': 0, 'ROOTPE_CPL': 0, 'ROOTPE_ROF': 0, 'ROOTPE_GLC': 0} 
Pes setting: pstrid      is {} 
Pes other settings: {}
Pes comments: none
 Compset is: 2000_CAM5%AV1C-L_CLM45%SPBC_CICE%PRES_DOCN%DOM_SROF_SGLC_SWAV_SIAC_SESP 
 Grid is: a%ne4np4_l%ne4np4_oi%ne4np4_r%null_g%null_w%null_z%null_m%oQU240 
 Components in compset are: ['cam', 'clm', 'cice', 'docn', 'srof', 'sglc', 'swav', 'siac', 'sesp', 'drv'] 
No charge_account info available, using value from PROJECT
e3sm model version found: 375404a
Batch_system_type is none
 Creating Case directory /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts

run_e3sm: -------- Finished create_newcase --------

run_e3sm: Project used for submission: condo

run_e3sm: Configuring E3SM to use the COSP simulator.


run_e3sm: -------- Starting case.setup --------

run_e3sm: /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts/case.setup
/home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts/env_mach_specific.xml already exists, delete to replace
/home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts/env_mach_specific.xml already exists, delete to replace
Creating batch scripts
Writing case.run script from input template /home/jenkins/workspace/branch_e3sm_test/cime/config/e3sm/machines/template.case.run
Creating file .case.run
Writing case.st_archive script from input template /home/jenkins/workspace/branch_e3sm_test/cime/config/e3sm/machines/template.st_archive
Creating file case.st_archive
Creating file .case.run.sh
Creating user_nl_xxx files for components and cpl
If an old case build already exists, might want to run 'case.build --clean' before building
Generating component namelists as part of setup
Creating component namelists
  2022-02-10 14:41:22 atm
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cam/cime_config/buildnml
  2022-02-10 14:41:22 lnd
   Calling /home/jenkins/workspace/branch_e3sm_test/components/clm/cime_config/buildnml
  2022-02-10 14:41:23 ice
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cice/cime_config/buildnml
  2022-02-10 14:41:23 ocn
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/data_comps/docn/cime_config/buildnml
  2022-02-10 14:41:23 rof
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/srof/cime_config/buildnml
  2022-02-10 14:41:23 glc
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sglc/cime_config/buildnml
  2022-02-10 14:41:23 wav
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/swav/cime_config/buildnml
  2022-02-10 14:41:23 iac
   Running /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/siac/cime_config/buildnml 
  2022-02-10 14:41:23 esp
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sesp/cime_config/buildnml
  2022-02-10 14:41:23 cpl
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/drivers/mct/cime_config/buildnml
Finished creating component namelists
You can now run './preview_run' to get more info on how your case will be run

run_e3sm: -------- Finished case.setup  --------

run_e3sm: Creating logical links to make navigating easier.
run_e3sm: Note: Beware of using ".." with the links, since the behavior of shell commands can vary.
run_e3sm: /home/jenkins/workspace/branch_e3sm_test/nightly_test/nightly_e3sm_ne4_silhs.csh=a_run_link
run_e3sm: run_root /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4
run_e3sm: run_dir /home/jenkins/workspace/branch_e3sm_test/nightly_test/nightly_e3sm_ne4_silhs.csh=a_run_link
setting case file to env_run.xml

run_e3sm: -------- Starting Build --------

run_e3sm: WARNING: Make sure there are no double-slashes (//) in the build command below: => Will cause a build error
run_e3sm: /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts/case.build
Building case in directory /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts
sharedlib_only is False
model_only is False
Generating component namelists as part of build
Creating component namelists
  2022-02-10 14:41:25 atm
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cam/cime_config/buildnml
  2022-02-10 14:41:25 lnd
   Calling /home/jenkins/workspace/branch_e3sm_test/components/clm/cime_config/buildnml
  2022-02-10 14:41:26 ice
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cice/cime_config/buildnml
  2022-02-10 14:41:26 ocn
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/data_comps/docn/cime_config/buildnml
  2022-02-10 14:41:26 rof
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/srof/cime_config/buildnml
  2022-02-10 14:41:26 glc
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sglc/cime_config/buildnml
  2022-02-10 14:41:26 wav
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/swav/cime_config/buildnml
  2022-02-10 14:41:26 iac
   Running /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/siac/cime_config/buildnml 
  2022-02-10 14:41:26 esp
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sesp/cime_config/buildnml
  2022-02-10 14:41:26 cpl
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/drivers/mct/cime_config/buildnml
Finished creating component namelists
Building gptl with output to file /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/gptl.bldlog.220210-144125
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/build_scripts/buildlib.gptl
Component gptl build complete with 1 warnings
Building mct with output to file /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/mct.bldlog.220210-144125
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/build_scripts/buildlib.mct
Building pio with output to file /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/pio.bldlog.220210-144125
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/build_scripts/buildlib.pio
Building csm_share with output to file /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/csm_share.bldlog.220210-144125
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/build_scripts/buildlib.csm_share
Component csm_share build complete with 4 warnings
         - Building atm Library 
Building atm with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/atm.bldlog.220210-144125
         - Building lnd Library 
Building lnd with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/lnd.bldlog.220210-144125
         - Building ice Library 
Building ice with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/ice.bldlog.220210-144125
         - Building ocn Library 
         - Building rof Library 
Building ocn with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/ocn.bldlog.220210-144125
         - Building glc Library 
Building rof with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/rof.bldlog.220210-144125
         - Building wav Library 
Building glc with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/glc.bldlog.220210-144125
Building wav with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/wav.bldlog.220210-144125
         - Building iac Library 
Building iac with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/iac.bldlog.220210-144125
         - Building esp Library 
Building esp with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/esp.bldlog.220210-144125
swav built in 0.894198 seconds
sglc built in 0.937862 seconds
siac built in 0.983303 seconds
sesp built in 0.988149 seconds
srof built in 1.038168 seconds
docn built in 2.953992 seconds
Component ice build complete with 9 warnings
Component ice build complete with 9 warnings
cice built in 30.829837 seconds
Component lnd build complete with 6 warnings
Component lnd build complete with 6 warnings
clm built in 164.210590 seconds
Component atm build complete with 15 warnings
Component atm build complete with 15 warnings
cam built in 178.895336 seconds
Building e3sm with output to /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/e3sm.bldlog.220210-144125 
Component e3sm exe build complete with 3 warnings
Time spent not building: 1.830473 sec
Time spent building: 250.712214 sec
MODEL BUILD HAS FINISHED SUCCESSFULLY

run_e3sm: -------- Finished Build --------

Creating component namelists
  2022-02-10 14:45:37 atm
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cam/cime_config/buildnml
  2022-02-10 14:45:38 lnd
   Calling /home/jenkins/workspace/branch_e3sm_test/components/clm/cime_config/buildnml
  2022-02-10 14:45:38 ice
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cice/cime_config/buildnml
  2022-02-10 14:45:38 ocn
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/data_comps/docn/cime_config/buildnml
  2022-02-10 14:45:38 rof
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/srof/cime_config/buildnml
  2022-02-10 14:45:38 glc
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sglc/cime_config/buildnml
  2022-02-10 14:45:38 wav
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/swav/cime_config/buildnml
  2022-02-10 14:45:38 iac
   Running /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/siac/cime_config/buildnml 
  2022-02-10 14:45:38 esp
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sesp/cime_config/buildnml
  2022-02-10 14:45:38 cpl
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/drivers/mct/cime_config/buildnml
Finished creating component namelists
run_e3sm: WARNING: This script does not have batch directives for $machine=nelson
run_e3sm:          Assuming default E3SM values.
run_e3sm: condo not recognized as a group, results must be shared manually

run_e3sm: $model_start_type = initial  (This is NOT necessarily related to RUN_TYPE)

run_e3sm: -------- Starting Submission to Run Queue --------

run_e3sm:          SUBMITTING JOB:
run_e3sm: /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts/case.submit
Creating component namelists
  2022-02-10 14:45:41 atm
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cam/cime_config/buildnml
  2022-02-10 14:45:41 lnd
   Calling /home/jenkins/workspace/branch_e3sm_test/components/clm/cime_config/buildnml
  2022-02-10 14:45:42 ice
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cice/cime_config/buildnml
  2022-02-10 14:45:42 ocn
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/data_comps/docn/cime_config/buildnml
  2022-02-10 14:45:42 rof
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/srof/cime_config/buildnml
  2022-02-10 14:45:42 glc
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sglc/cime_config/buildnml
  2022-02-10 14:45:42 wav
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/swav/cime_config/buildnml
  2022-02-10 14:45:42 iac
   Running /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/siac/cime_config/buildnml 
  2022-02-10 14:45:42 esp
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sesp/cime_config/buildnml
  2022-02-10 14:45:42 cpl
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/drivers/mct/cime_config/buildnml
Finished creating component namelists
Checking that inputdata is available as part of case submission
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/docn.input_data_list'
Loading input file list: 'Buildconf/clm.input_data_list'
Loading input file list: 'Buildconf/cam.input_data_list'
Loading input file list: 'Buildconf/cice.input_data_list'
Check case OK
submit_jobs case.run
Submit job case.run
Starting job script case.run
2022-02-10 14:45:42 CASE.RUN BEGINS HERE
2022-02-10 14:45:42 RUN_MODEL BEGINS HERE
2022-02-10 14:45:42 PRE_RUN_CHECK BEGINS HERE
2022-02-10 14:45:42 NAMELIST CREATION BEGINS HERE
Generating namelists for /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/case_scripts
Creating component namelists
  2022-02-10 14:45:42 atm
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cam/cime_config/buildnml
  2022-02-10 14:45:43 lnd
   Calling /home/jenkins/workspace/branch_e3sm_test/components/clm/cime_config/buildnml
  2022-02-10 14:45:43 ice
   Calling /home/jenkins/workspace/branch_e3sm_test/components/cice/cime_config/buildnml
  2022-02-10 14:45:43 ocn
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/data_comps/docn/cime_config/buildnml
  2022-02-10 14:45:43 rof
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/srof/cime_config/buildnml
  2022-02-10 14:45:43 glc
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sglc/cime_config/buildnml
  2022-02-10 14:45:43 wav
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/swav/cime_config/buildnml
  2022-02-10 14:45:43 iac
   Running /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/siac/cime_config/buildnml 
  2022-02-10 14:45:43 esp
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/components/stub_comps/sesp/cime_config/buildnml
  2022-02-10 14:45:43 cpl
   Calling /home/jenkins/workspace/branch_e3sm_test/cime/src/drivers/mct/cime_config/buildnml
Finished creating component namelists
2022-02-10 14:45:43 NAMELIST CREATION HAS FINISHED
-------------------------------------------------------------------------
 - Prestage required restarts into /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/run
 - Case input data directory (DIN_LOC_ROOT) is /home/pub/cam_inputdata 
 - Checking for required input datasets in DIN_LOC_ROOT
-------------------------------------------------------------------------
2022-02-10 14:45:43 PRE_RUN_CHECK HAS FINISHED
run command is /usr/local/mpi/MPICH-3.3.1-gfortran-8/bin/mpirun  -np 1 -prepend-rank /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/e3sm.exe   >> e3sm.log.$LID 2>&1  
2022-02-10 14:45:43 SAVE_PRERUN_PROVENANCE BEGINS HERE
2022-02-10 14:45:43 SAVE_PRERUN_PROVENANCE HAS FINISHED
2022-02-10 14:45:43 MODEL EXECUTION BEGINS HERE
2022-02-10 14:46:12 MODEL EXECUTION HAS FINISHED
Exception from case_run: ERROR: RUN FAIL: Command '/usr/local/mpi/MPICH-3.3.1-gfortran-8/bin/mpirun  -np 1 -prepend-rank /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/build/e3sm.exe   >> e3sm.log.$LID 2>&1 ' failed
See log file for details: /home/jenkins/E3SM_simulations/nelson.mg2_def.ne4_ne4/run/e3sm.log.220210-144542
Submitted job case.run with id None