2. Run CESM in four steps

paleoCAMP logo

2. Run CESM in four steps#

Tutorials at the 2025 paleoCAMP | June 16–June 30, 2025

Jiang Zhu
jiangzhu@ucar.edu
Climate & Global Dynamics Laboratory
NSF National Center for Atmospheric Research

Learning Objectives#

  • Know how to set up a simple experiment with CESM

  • Know how to customize details of the experiment using xmlchange and namelist modifications

Time to learn: 40 minutes

NOTE: This is a demonstration module, and you don’t need to do anything yet.

Run a preindustrial control simulation in four steps on Derecho#

  1. Create a new case b.e21.B1850.f19_g17.piControl.001 in the home directory (do this in the CESM Code Directory, the “Headquarter” and creates the “Production Control Room”)

cd /glade/work/jiangzhu/paleocamp/cesm2.1.5/cime/scripts/
./create_newcase --case ~/b.e21.B1850.f19_g17.piControl.001 --compset B1850 --res f19_g17 --project UAZN0042

  1. Set up your case (do this in the Case directory, the “Production Control Room” and creates the “Factory”)

cd ~/b.e21.B1850.f19_g17.piControl.001
./case.setup

  1. Build your case (do this in the Case directory, the “Production Control Room” and designs the manufacturing workflow)

./case.build

  1. submit your run (do this in the Case directory, the “Production Control Room” and starts the manufacturing)

./case.submit

Further explanations on create_newcase#

The create_newcase command has three required inputs. It creates your “Production Control Room”.

  1. --case: the name and directory path of your case (your Production Control Room)

  • Setting case to ~/b.e21.B1850.f19_g17.piControl.001 will make a simulation called b.e21.B1850.f19_g17.piControl.001 and it will be located in the home directory

  • Here is the CESM case naming conventions

  1. --compset: the CESM component set that you’d like to use.

  • CESM2 components can be combined in numerous ways to carry out various scientific experiments. A particular mix of components, along with component-specific configuration and/or namelist settings is called a component set or compset.

  • B1850 means fully coupled preindustrial control including active atmosphere, land, ocean, sea ice, etc.

  • F1850 means atmosphere/land-only simulation with fixed sea-surface temperature and sea ice cover.

  • Other supported compsets. Or $CESMROOT/cime/scripts/query_config --compsets

  1. --res: the resolution that you’d like to use.

  • f19_g17: ~2° atmosphere/land and ~1° ocean/sea ice.

  • f09_g17: ~1° atmosphere/land and ~1° ocean/sea ice

  • Other supported grid/resolution. Or $CESMROOT/cime/scripts/query_config --compsets

Further explanations on case.setup#

case.setup creates the Build and Run Directory, which is your Factory.
case.setup also creates files like user_nl_cam, where users can customize component namelist settings.
case.setup should be invoked in the Case Directory (your Production Control Room).

Further explanations on case.build#

case.build checks and builds component model namelists and libraries, and the final executable.
case.build may take 10 minutes!

Tips: If your first case.build fails somehow, run ./case.build --clean-all before trying to build the model again.
Note: It is recommended that you avoid building CESM directly on a shared login node. Instead, use qcmd -- ./case.build to build CESM on a computing node.

Further explanations on case.submit#

case.submit submits the job to the HPC Batch Job Scheduler.
case.submit also submits the case.st_archive script to archive the model output (move output from your Factory to Storage). This step is dependent on the successful completion of simulation.

Check job status#

qstat accesses the information in the Batch Job Scheduler to see the status of all jobs running on Derecho. To simplify the list, the -u option can be specified to filter by a particular user.

qstat can be used in any directory on Derecho.

Result of running qstat with an active job running: 

qstat -u $YOUR_USERNAME

Output: 

Job ID          Username Queue    Jobname    SessID NDS TSK Memory Time  S Time
--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
704942.chadmin* $USER regular  run.b.day*   8284  16 576    --  12:00 R 00:00
704945.chadmin* $USER regular  st_archiv*    --    1   1    --  00:20 H   --

NOTE: There may be a slight delay between submitting CESM cases and seeing the jobs appear in the queue.

Model output#

If the model run is successful, the CESM netcdf output history files are automatically moved to a short-term Archive Directory (Your Storage; /glade/derecho/scratch/jiangzhu/archive/b.e21.B1850.f19_g17.piControl.001).
Notes:

  • If a model run was unsuccessful the output remains in the Run Directory (your Factory) and the short-term archive is not created.

  • Both the Run Directory and the Archive Directory are in the NCAR HPC scratch space. This space is scrubbed, and files get deleted after a number of days. Thus, it is a good idea to move your model output files from the short-term archive to a more permanent location as soon as you are able.

More controls: XML change#

  • Use the ./xmlchange command in the Case Directory, your “Production Control Room”

./xmlchange VARIABLE=VALUE

  • For example, change the simulation length of a single submission to 1 year (the default setting is 5 days):

./xmlchange STOP_N=1,STOP_OPTION=nyears

  • STOP_OPTION controls the units in which you are specifying the length of the simulation

  • STOP_N is the length in those units

  • xmlchange of STOP_OPTION and STOP_N should be done beofore ./case.submit

  • You can check values of specific variables using:

./xmlquery STOP_N,STOP_OPTION

  • You can use the -p option with xmlquery to find a partial match. The following will show all available variable names that contain STOP:

./xmlquery -p STOP

  • You can find a listing of all the available XML variables along with their description in env_run.xml by running the following commands from your case directory

./xmlquery --listall --file env_run.xml --description

  • Note: Typically, supercomputers have a wallclock limit of 12 hours in real time, meaning that you can only run continuously for 12 hours. You can use STOP_OPTION and STOP_N to an appropriate length that can be completed wallclock limit.

More controls: namelist change#

  • Typically, it is used to control specific model parameters (CO2, cloud parameters, etc.)

  • Use text editor to edit the user_nl_* files in the Case Directory, your “Production Control Room”

  • For example, adding the following line into user_nl_cam will change the CO2 concentration in the atmosphere

co2vmr = 569.4e-6

  • Commonly used Unix text editors: vi, emacs, nano, etc.

  • If in JupyterHub, find the user_nl file on the left sidebar, click, edit, and save

  • Namelist changes are usually done before ./case.build

  • Complete documentation about the namelist variables can be found on the CESM webpage.

Summary#

  • Build and run a CESM case in four steps. What are they?

  • How do you further customize the simulation?

Optional advanced resources#

  • visualCaseGen: a GUI that runs on JupyterLab and guides the users through the process of creating CESM cases, e.g., choosing appropriate compsets and grids.

Contents