To run a CESM case, you must submit the batch script $CASE.run. In addition, you also need to modify env_run.xml for your particular needs.
The env_run.xml
file contains variables which may be modified at the
initialization of a model run and during the course of that model run.
These variables comprise coupler namelist settings for the model stop
time, model restart frequency, coupler history frequency and a flag to
determine if the run should be flagged as a continuation run. In
general, you only need to set the variables
$STOP_OPTION
and $STOP_N
. The other
coupler settings will then be given consistent and reasonable default
values. These default settings guarantee that restart files are
produced at the end of the model run.
In the following, we focus on the handling of run control (e.g. length of run, continuing a run) and output data. We also give a more detailed description of CESM restarts.
The case initialization type is set in env_run.xml. A CESM run can be initialized in one of three ways; startup, branch, or hybrid.
In a startup run (the default), all components are initialized using baseline states. These baseline states are set independently by each component and can include the use of restart files, initial files, external observed data files, or internal initialization (i.e., a "cold start"). In a startup run, the coupler sends the start date to the components at initialization. In addition, the coupler does not need an input data file. In a startup initialization, the ocean model does not start until the second ocean coupling (normally the second day).
In a branch run, all components are initialized using a
consistent set of restart files from a previous run (determined by
the $RUN_REFCASE
and $RUN_REFDATE
variables in env_run.xml).
The case name is generally changed for a branch run, although it does
not have to be. In a branch run, setting $RUN_STARTDATE
is ignored because the model components obtain the
start date from their restart datasets. Therefore, the start date
cannot be changed for a branch run. This is the same mechanism that is
used for performing a restart run (where $CONTINUE_RUN
is set to TRUE in the env_run.xml file).
Branch runs are typically used when sensitivity or parameter
studies are required, or when settings for history file output streams
need to be modified while still maintaining bit-for-bit
reproducibility. Under this scenario, the new case is able to produce
an exact bit-for-bit restart in the same manner as a continuation run
if no source code or component namelist inputs
are modified. All models use restart files to perform this type of
run. $RUN_REFCASE
and $RUN_REFDATE
are
required for branch runs.
To set up a branch run, locate the restart tar file or restart
directory for $RUN_REFCASE
and $RUN_REFDATE
from a previous run, then place those
files in the $RUNDIR
directory. See setting up a branch run for an
example.
A hybrid run indicates that CESM is initialized more like
a startup, but uses initialization datasets from a previous
case. This is somewhat analogous to a branch run with relaxed
restart constraints. A hybrid run allows users to bring together
combinations of initial/restart files from a previous case
(specified by $RUN_REFCASE
) at a given model output
date (specified by $RUN_REFDATE
). Unlike a branch
run, the starting date of a hybrid run (specified by
$RUN_STARTDATE
) can be modified relative to the
reference case. In a hybrid run, the model does not continue in a
bit-for-bit fashion with respect to the reference case. The resulting
climate, however, should be continuous provided that no model source
code or namelists are changed in the hybrid run.
In a hybrid initialization, the
ocean model does not start until the second ocean coupling (normally
the second day), and the coupler does a "cold start" without a restart file.
The variable $RUN_TYPE
determines the initialization
type. This setting is only important for the initial run of a
production run when the $CONTINUE_RUN
variable is set
to FALSE. After the initial run, the $CONTINUE_RUN
variable is set to TRUE, and the model restarts exactly using input
files in a case, date, and bit-for-bit continuous fashion. The
variable $RUN_TYPE
is the start date (in yyyy-mm-dd
format) either a startup or hybrid run. If the run is targeted to be a
hybrid or branch run, you must also specify values for
$RUN_REFCASE
and $RUN_REFDATE
. All run
startup variables are discussed in run start control
variables.
Before a job is submitted to the batch system, you need to first check that the batch submission lines in $CASE.run are appropriate. These lines should be checked and modified accordingly for appropriate account numbers, time limits, and stdout/stderr file names. You should then modify env_run.xml to determine the key run-time settings. See controlling run termination, controlling run restarts, and performing model restarts for more details. A brief note on restarting runs. When you first begin a branch, hybrid or startup run, CONTINUE_RUN must be set to FALSE. When you successfully run and get a restart file, you will need to change CONTINUE_RUN to TRUE for the remainder of your run. See performing model restarts for more details.
By default,
STOP_OPTION = ndays STOP_N = 5 STOP_DATE = -999 |
The default setting is only appropriate for initial testing. Before a
longer run is started, update the stop times based on the case
throughput and batch queue limits. For example, if the model runs 5
model years/day, set RESUBMIT
=30, STOP_OPTION
= nyears,
and STOP_N
= 5.
The model will then run in five year increments, and stop after 30
submissions.
In your $CASEROOT
directory, the subdirectory
$CASEROOT/Buildconf contains files to create the
component namelists, build the component libraries and create the
model executable.
Buildconf/$component.buildexe.csh creates the
component libraries and
Buildconf/$component.buildnml.csh creates the
component namelists. A new feature in the CESM1.1 and CESM1.2 release
series is that ALL CESM components now use a
component-specific build-namelist utility (similar
to that of CAM, CLM and CICE in the CESM1.0 series) to generate their
respective model namelists. In addition, CAM, CLM and CICE have an
associated configure utility that sets up compile
time configuration options and is also called from the corresponding
Buildconf/*.buildnml.csh (e.g. Buildconf/cam.buildnml.csh).
In the CESM1.2 series, user specific component namelist
changes should only be made only by editing the
$CASEROOT/user_nl_xxx files OR by
changing xml variables in env_run.xml or env_build.xml. A full
discussion of how to change the namelist variables for each component
is provided below. You can preview the case component namelists by
running preview_namelists in your $CASEROOT
.
Calling preview_namelists results in the creation
of component namelists (e.g. atm_in, lnd_in, .etc) in
$CASEROOT/CaseDocs/. A complete documentation of
all model component namelists
for CESM1.2 releases is now available. The namelist files
created in the CaseDocs/ are there only for user
reference and SHOULD NOT BE EDITED since they are
overwritten every time preview_namelists,
$CASE.run and $CASE.build
are called. In CESM1.2, (like CESM1.1 but unlike CESM1.0) the only
files that you should modify are in $CASEROOT
. No files in
Buildconf/ should be changed. The following represents a summary of
controlling and modifying component-specific run-time settings:
In CESM1.2, driver namelist are in two groups - those that are set directly from xml variables in env_case.xml, env_mach_pes.xml and env_run.xml, and those that are set by the driver build-namelist utility ($CCSMROOT/models/drv/bld/build-namelist) for the target compset and resolution. Except for the following driver namelist variables (see below), driver namelist variables that are in env_run.xml can be changed either by changing the xml variable OR by adding the correct key-word value pair at the end of user_nl_cpl, where any changes in user_nl_cpl will take precedence over values set in the xml file. For example, to change eps_frac to 1.0e-15, add the following line to the end of the user_nl_cpl, "eps_frac = 1.0e-15". To see the result of this modification to user_nl_cpl call preview_namelists and verify that this new value appears in CaseDocs/drv_in.
The following namelist variables MAY NOT be changed in user_nl_cpl - but must be changed in the appropriate $CASEROOT xml file. XXX refers to ATM,LND,ICE,OCN,ROF,GLC,WAV ====================================== drv namelist => xml variable variable ====================================== case_name => CASE username => CCSMUSER hostname => MACH model_version => CCSM_REPOTAG start_type => RUN_TYPE start_ymd => RUN_STARTDATE start_tod => START_TOD XXX_cpl_dt => XXX_NCPL XXX_ntasks => NTASKS_XXX XXX_nthreads => NTHRDS_XXX XXX_rootpe => ROOTPE_XXX XXX_pestride => PSTRID_XXX XXX_layout => NINST_XXX_LAYOUT |
CAM's configure and build-namelist utilities are called by Buildconf/cam.buildnml.csh. CAM_CONFIG_OPTS, CAM_NAMELIST_OPTS and CAM_NML_USECASE are used to set compset variables (e.g., "-phys cam5" for CAM_CONFIG_OPTS) and in general should not be modified for supported compsets. For a complete documentation of namelist settings, see CAM namelist variables. To modify CAM namelist settings, you should add the appropriate keyword/value pair at the end of the $CASEROOT/user_nl_cam file (see the documentation for each file at the top of that file). For example, to change the solar constant to 1363.27, modify the user_nl_cam file to contain the following line at the end "solar_const=1363.27". To see the result of adding this, call preview_namelists and verify that this new value appears in CaseDocs/atm_in.
CLM's configure and build-namelist utilities are called by Buildconf/clm.buildnml.csh. CLM_CONFIG_OPTS and CLM_NML_USE_CASE are used to set compset specific variables and in general should not be modified for supported compsets. For a complete documentation of namelist settings, see CLM namelist variables. To modify CLM namelist settings, you should add the appropriate keyword/value pair at the end of the $CASEROOT/user_nl_clm file (see the documentation for each file at the top of that file). To see the result of your change, call preview_namelists and verify that the changes appear correctly in CaseDocs/lnd_in.
RTM's build-namelist utility is called by Buildconf/rtm.buildnml.csh. For a complete documentation of namelist settings, see RTM namelist variables. To modify RTM namelist settings, you should add the appropriate keyword/value pair at the end of the $CASEROOT/user_nl_rtm file (see the documentation for each file at the top of that file). To see the result of your change, call preview_namelists and verify that the changes appear correctly in CaseDocs/rof_in.
CICE's configure and build-namelist utilities are now called by Buildconf/cice.buildnml.csh. Note that CICE_CONFIG_OPTS, and CICE_NAMELIST_OPTS are used to set compset specific variables and in general should not be modified for supported compsets. For a complete documentation of namelist settings, see CICE namelist variables. To modify CICE namelist settings, you should add the appropriate keyword/value pair at the end of the $CASEROOT/user_nl_cice file (see the documentation for each file at the top of that file). To see the result of your change, call preview_namelists and verify that the changes appear correctly in CaseDocs/ice_in.
In addition, cesm_setup creates CICE's compile time block decomposition variables in env_build.xml as follows:
./cesm_setup ⇓ Buildconf/cice.buildnml.csh and $NTASKS_ICE and $NTHRDS_ICE ⇓ env_build.xml variables CICE_BLCKX, CICE_BLCKY, CICE_MXBLCKS, CICE_DECOMPTYPE CPP variables in cice.buildexe.csh |
See POP2 namelist variables for a complete description of the POP2 run-time namelist variables. Note that OCN_COUPLING, OCN_ICE_FORCING, OCN_TRANSIENT are normally utilized ONLY to set compset specific variables and should not be edited. For a complete documentation of namelist settings, see CICE namelist variables. To modify POP2 namelist settings, you should add the appropriate keyword/value pair at the end of the $CASEROOT/user_nl_pop2 file (see the documentation for each file at the top of that file). To see the result of your change, call preview_namelists and verify that the changes appear correctly in CaseDocs/ocn_in.
In addition, cesm_setup also generates POP2's compile time compile time block decomposition variables in env_build.xml as follows:
./cesm_setup ⇓ Buildconf/pop2.buildnml.csh and $NTASKS_OCN and $NTHRDS_OCN ⇓ env_build.xml variables POP2_BLCKX, POP2_BLCKY, POP2_MXBLCKS, POP2_DECOMPTYPE CPP variables in pop2.buildexe.csh |
See CISM namelist variables for a complete description of the CISM run-time namelist variables. This includes variables that appear both in cism_in and in cism.config. To modify any of these settings, you should add the appropriate keyword/value pair at the end of the user_nl_cism file (see the documentation for each file at the top of that file). To see the result of your change, call preview_namelists and verify that the changes appear correctly in CaseDocs/cism_in and CaseDocs/cism.config.
There are also some run-time settings set via
env_run.xml, as documented in CISM run time variables
- in particular, the model resolution, set via
CISM_GRID
. The value of CISM_GRID
determines the default value of a number of other namelist parameters.
DATM is discussed in detail in Data Model's User's Guide. DATM is normally used to provide observational forcing data (or forcing data produced by a previous run using active components) to drive CLM (I compset), POP2 (C compset), and POP2/CICE (G compset). As a result, DATM variable settings are specific to the compset that will be targeted.
DATM can be user configured in three different ways.
You can set DATM run-time variables my modifying control settings for CLM and CPLHIST forcing.
You can edit user_nl_datm to change namelist settings namelists settings by adding all user specific namelist changes in the form of "namelist_var = new_namelist_value". Note that any namelist variable from shr_strdata_nml and datm_nml can be modified below using the this syntax. Use preview_namelists to view (not modify) the output namelist in CaseDocs.
You can modify the contents of a DATM stream txt file. To do this:
use preview_namelists to obtain the contents of the stream txt files in CaseDocs
place a copy of this file in $CASEROOT with the string "user_" prepended
Make sure you change the permissions of the file to be writeable (chmod 644)
Modify the user_datm.streams.txt.* file.
CASEROOT
should be
user_datm.streams.txt.CORE2_NYF.GISS. After
calling preview_namelists again, you should see your new
modifications appear in
CaseDocs/datm.streams.txt.CORE2_NYF.GISS.
DOCN is discussed in detail in Data Model's User's Guide.
DOCN running in prescribed mode assumes that the only field in the
input stream is SST and also that SST is in Celsius and must be
converted to Kelvin. All other fields are set to zero except for
ocean salinity, which is set to a constant reference salinity value.
Normally the ice fraction data (used for prescribed CICE) is found in
the same data files that provide SST data to the data ocean model
since SST and ice fraction data are derived from the same
observational data sets and are consistent with each other. For DOCN
prescribed mode, default yearly climatological datasets are provided
for various model resolutions. For multi-year runs requiring AMIP
datasets of sst/ice_cov fields, you need to set the variables for
DOCN_SSTDATA_FILENAME
,
DOCN_SSTDATA_YEAR_START
, and
DOCN_SSTDATA_YEAR_END
. CICE in prescribed mode
also uses these values.
DOCN running as a slab ocean model is used (in conjunction with CICE running in prognostic mode) in all E compsets. SOM ("slab ocean model") mode is a prognostic mode. This mode computes a prognostic sea surface temperature and a freeze/melt potential (surface Q-flux) used by the sea ice model. This calculation requires an external SOM forcing data file that includes ocean mixed layer depths and bottom-of-the-slab Q-fluxes. Scientifically appropriate bottom-of-the-slab Q-fluxes are normally ocean resolution dependent and are derived from the ocean model output of a fully coupled run. Note that while this mode runs out of the box, the default SOM forcing file is not scientifically appropriate and is provided for testing and development purposes only. Users must create scientifically appropriate data for their particular application. A tool is available to derive valid SOM forcing.
DOCN can be user-customized in three ways.
You can set DOCN run-time variables.
You can edit user_nl_docn to change namelist settings by adding all user specific namelist changes in the form of "namelist_var = new_namelist_value". Note that any namelist variable from shr_strdata_nml and datm_nml can be modified below using the this syntax. Use preview_namelists to view (not modify) the output namelist in CaseDocs.
You can modify the contents of a DOCN stream txt file. To do this:
use preview_namelists to obtain the contents of the stream txt files in CaseDocs/
place a copy of this file in $CASEROOT with the string "user_" prepended
Make sure you change the permissions of the file to be writeable (chmod 644)
Modify the user_docn.streams.txt.* file.
CASEROOT
should be user_docn.streams.txt.prescribed. After
changing this file and calling preview_namelists
again, you should see your new modifications appear in
CaseDocs/docn.streams.txt.prescribed.
DICE is discussed in detail in Data Model's User's Guide.
DICE can be user-customized in three ways.
You can set DICE run-time variables.
You can edit user_nl_dice to change namelist settings by adding all user specific namelist changes in the form of "namelist_var = new_namelist_value". Note that any namelist variable from shr_strdata_nml and datm_nml can be modified below using the this syntax. Use preview_namelists to view (not modify) the output namelist in CaseDocs/.
You can modify the contents of a DICE stream txt file. To do this:
use preview_namelists to obtain the contents of the stream txt files in CaseDocs/
place a copy of this file in $CASEROOT with the string "user_" prepended
Make sure you change the permissions of the file to be writeable (chmod 644)
Modify the user_dice.streams.txt.* file.
DLND is discussed in detail in Data Model's User's Guide. The data land model is different from the other data models because it can run as a purely data-land model (reading in coupler history data for atm/land fluxes and land albedos produced by a previous run), or to read in model output from CLM to send to CISM.
DLND can be user-customized in three ways:
You can set DLND run-time variables.
You can edit user_nl_dlnd OR user_nl_dsno depending on the component set, to change namelist settings namelists settings by adding all user specific namelist changes in the form of "namelist_var = new_namelist_value". Note that any namelist variable from shr_strdata_nml and dlnd_nml or dsno_nml can be modified below using the this syntax. Use preview_namelists to view (not modify) the output namelist in CaseDocs/.
You can modify the contents of a DLND stream txt file. To do this:
use preview_namelists to obtain the contents of the stream txt files in CaseDocs/
place a copy of this file in $CASEROOT with the string "user_" prepended
Make sure you change the permissions of the file to be writeable (chmod 644)
Modify the user_dlnd.streams.txt.* file.
DROF is discussed in Data Model's User's Guide. The data river runoff model reads in runoff data and sends it back to the coupler. In general, the data river runoff model is only used to provide runoff forcing data to POP2 when running C or G compsets
DROF can be user-customized in three ways:
You can set DROF run-time variables.
You can edit user_nl_drof to change namelist settings namelists settings by adding all user specific namelist changes in the form of "namelist_var = new_namelist_value". Note that any namelist variable from shr_strdata_nml and drof_nml can be modified using the this syntax. Use preview_namelists to view (not modify) the output namelist in CaseDocs/.
You can modify the contents of a DROF stream txt file. To do this:
use preview_namelists to obtain the contents of the stream txt files in CaseDocs/
place a copy of this file in $CASEROOT with the string "user_" prepended
Make sure you change the permissions of the file to be writeable (chmod 644)
Modify the user_drof.streams.txt.* file.
During a model run, each CESM component produces its own output datasets consisting of history, restart and output log files. Component history files and restart files are in netCDF format. Restart files are used to either exactly restart the model or to serve as initial conditions for other model cases.
Archiving is a phase of a CESM model run where the generated output data is moved from $RUNDIR to a local disk area (short-term archiving) and subsequently to a long-term storage system (long-term archiving). It has no impact on the production run except to clean up disk space and help manage user quotas. Although short-term and long-term archiving are implemented independently in the scripts, there is a dependence between the two since the short-term archiver must be turned on in order for the long-term archiver to be activated. In env_run.xml, several variables control the behavior of short and long-term archiving. See short and long term archiving for a description of output data control variables. Several important points need to be made about both short and long term archiving:
By default, short-term archiving is enabled and long-term archiving is disabled.
All output data is initially written to $RUNDIR.
Unless a user explicitly turns off short-term archiving, files will be moved to $DOUT_S_ROOT at the end of a successful model run.
If long-term archiving is enabled, files will be
moved to $DOUT_L_MSROOT
by
$CASE.l_archive, which is run as a
separate batch job after the successful completion of a model run.
Users should generally turn off short term-archiving when developing new CESM code.
If long-term archiving is not enabled, users must monitor quotas and
usage in the $DOUT_S_ROOT/
directory and should
manually clean up these areas on a frequent basis.
Standard output generated from each CESM component is saved in a
"log file" for each component in $RUNDIR
. Each time
the model is run, a single coordinated datestamp is incorporated in
the filenames of all output log files associated with that run. This
common datestamp is generated by the run script and is of the form
YYMMDD-hhmmss, where YYMMDD are the Year, Month, Day and hhmmss are
the hour, minute and second that the run began
(e.g. ocn.log.040526-082714). Log files are also copied to a user
specified directory using the variable $LOGDIR in env_run.xml. The
default is a 'logs' subdirectory beneath the case directory.
By default, each component also periodically writes history files
(usually monthly) in netCDF format and also writes netCDF or binary
restart files in the $RUNDIR
directory. The history and
log files are controlled independently by each component. History
output control (i.e. output fields and frequency) is set in the
Buildconf/$component.buildnml.csh
files.
The raw history data does not lend itself well to easy time-series analysis. For example, CAM writes one or more large netCDF history file(s) at each requested output period. While this behavior is optimal for model execution, it makes it difficult to analyze time series of individual variables without having to access the entire data volume. Thus, the raw data from major model integrations is usually postprocessed into more user-friendly configurations, such as single files containing long time-series of each output fields, and made available to the community.
As an example, for the following example settings
DOUT_S = TRUE DOUT_S_ROOT = /ptmp/$user/archive DOUT_L_MS = TRUE DOUT_L_MSROOT /USER/csm/$CASE |
the run will automatically submit the $CASE.l_archive to the queue upon its completion to archive the data. The system is not bulletproof, and you will want to verify at regular intervals that the archived data is complete, particularly during long running jobs.