K. \underline{How to Run the Model}

The NCAR CSM Ocean Model Job Scripts

Two job scripts are provided by the NCAR CSM ocean model developers: one which compiles the ocean model and creates a binary to be used in the coupled model system, and another which compiles and executes the stand-alone ocean model. Both scripts are intended to be self explanatory and are designed to reduce the possibility of set-up errors, but because there are certain features which may need some additional explanation, this section provides supplementary documentation.

Starting and Restarting the NCAR CSM Model

There are several procedures for starting and restarting the NCAR CSM ocean model, depending on the options chosen by the user. Originally, the CSM model had only one method for starting the model and one for restarting it, but the subsequent addition of an automatic restart option and an option to create a new run based on restart files from another case increased the number of start/restart procedures.

The MOM 1.1 produced a single restart file, containing all the necessary time levels of information. A major disadvantage of a single restart file is that the file size gets unmanageably large as the model resolution becomes finer. Consequently, the NCAR CSM ocean model produces three restart files. The R2 restart file contains the time information, barotropic streamfunction variables, and the accumulated ice formation (when sflxiceflx is defined). The required two time levels of the potential temperature, salinity, and horizontal velocity fields are stored in two of the R3, R4, or R5 restart files. The empty restart file is determined automatically by the code itself based on the number of time levels stored internally.

The autorestart feature was added for the user’s convenience: it allows the same job script to be submitted for multiple continuation runs, completely unaltered. The user need not provide restart file names in the script; rather, this information is read from a “pointer” file and the ocean model, during execution, then reads the most recent restart files from the disk location specified in the pointer file. The autorestart option is selected by setting the script-control variable autorestart = yes, which in turn adds autorestart to the list of all preprocessor options.

Whether or not the autorestart feature is selected, the standard method of starting an initial stand-alone CSM ocean model run is to set the script variable initial run = yes and provide initial conditions for the starting date of the model integration, for which, in the stand-alone model, mean January Levitus values have typically been used. The length of the model integration is controlled by setting numsteps, the total number of timesteps for the run. When initial run = yes, the job script sets the model variables nb=0, restart-flag = false, and nlast = numsteps.

K-1

Because there are situations in which modelers want to start a new model run from initial conditions provided from a prior run, but want to reset the starting time, a variant start-up option is available: set initial run = yes and nb=0, but set restart flag = true, and provide the names of the R2, R3, R4, and/or R5 restart files. This is the standard manner in which the ocean model is initialized in coupled model runs, and it is valid regardless of the autorestart setting.

The method used to continue a stand-alone model run depends on whether or not the autorestart option is selected. If autorestart is not in effect, the model run is continued by setting initial run = no, nb = itt, the timestep number from the last restart file, restart flag = true, autorestart = no, and by providing the names of the restart files, i.e., the R2, R3, R4, and/or R5 files in the appropriate part of the job script. For each subsequent continuation run, the user must set nb = itt, and provide the most recent restart-file names. The script will automatically set the value of nlast to nb + numsteps.

Continuing a stand-alone model run with the autorestart option is simple; the user needs only to set initial run = no and autorestart = yes. The user may decide to reset the value of numsteps during subsequent continuation runs, but if not, then all continuation runs may be initiated by submitting and resubmitting this continuation script entirely unaltered. The user should note that the script sets nlast to numsteps in this case; however, the autorestart preprocessor option activates code which sets nlast to the last timestep of the previous run, itt + nlast, thus effectively setting the total integration length to numsteps.

Continuing a coupled ocean-model run is even simpler, because the autorestart option is required, and the master job script (the “.nqs” script) controls whether the run is an initial run or a continuation. This allows the same ocean script to be used, unaltered, in either case.

Timing and Memory Requirements

Because time requirements for a model integration depend on the particular options chosen, it is impractical to provide precise model timing and memory information for all configurations. However, such information from the 2×2 and 3×3 models with the options presented in this document will provide a useful starting estimate for users who select other model options or run on other computers.

Memory requirements for the ocean model increase with the number of processors selected. Commonly, we run the models at NCAR on Cray C90, YMP, or J90 computers, with either 8 or 16 processors. The 2×2 model requires 32Mw of memory on a 16-processor machine and 23Mw of memory on an 8-processor machine. The figures for the 3×3 model are 11Mw and 8Mw, respectively. When the preprocessor option time average is chosen, the model memory requirements for the 2×2 model increase to 38Mw of memory on a 16-processor machine and 29Mw of memory on an 8-processor machine.

NCAR CSM ocean model timing requirements and cost comparisons with other global ocean-circulation models are discussed in detail in the timing section at the ocean-model

K-2 website (http://www.cgd.ucar.edu/csm/ocn-ncom). Timing figures are closely linked to the options selected. For our configuration, each 2×2 model timestep uses approximately 4 cpu seconds on a Cray C90, 9 cpu seconds on a Cray YM9, and 19 cpu seconds on a Cray J90. The comparable figures for the 3×3 model are approximately 1, 2.4, and 5 cpu seconds, respectively.

For a detailed, up-to-date summary of changes to the ocean model, its options and set-up scripts since our last public code release, please refer to the section “What’s New in NCOM” on the above referenced web page. We plan to periodically update this page, so it a good idea for NCAR CSM ocean-model users to check this page from time to time.

Time Manager

The time-manager subroutine, tmngr, has evolved at NCAR over several years, and was eventually substantially rewritten in order to incorporate these changes in an orderly way. In a departure from the MOM 1.1 time manager, all time-dependent decisions in the NCAR CSM ocean model are based on an integer timestep counter, itt, and the corresponding elapsed model time in seconds, ttsec. Some MOM 1.1 variables which are based on the number of model days have integer timestep-number counterparts in the CSM ocean model, but others were eliminated during the restructuring. Many features of the MOM 1.1 time manager remain in the CSM version, although the underlying computations have changed. Choices of a simple or Julian calendar, allowing leap years or not, and an arbitrary year length are all available.

New features have been added. Because model I/O has been customized to the NCAR computing environment, new time-manager variables have been added to control the writing and storing of files. Restart and history files are independently controlled, and may be written at the end of each calendar month, or after a specified number of timesteps, or both. Other new time-manager variables include logical flags which signal the present timestep is the end of a day, month, or year, and integer values of the present second, minute, and hour of the day and day, month, and year. All of these variables are available for use throughout the model upon the inclusion of the files switch.h and ctmngr2.h.

Controls for the model integration (starting and ending timesteps, the frequency of sampling for time-averaging, the frequency of writing diagnostics and history and restart files, and communication information for the Flux Coupler) are all set in the namelist contrl, which is located in the file model.data in the stand-alone job script and ocn.parm in the coupled-model job script. Namelist contrl members include:

nb (integer) – the beginning timestep number. Set nb = 0 for an initial run, nb = itt for a standard continuation run, and nb > 0 for an autorstart continuation run.

nlast (integer) – the last timestep number of the model integration; valid only for stand-alone integrations.

ntime avg freq (integer) – the number of timesteps between the accumulation of data used in time averaging.

K-3 write at reg int (logical) – if true, this option causes history files, restart files, and diagnostics to be written at regular intervals. Variables which must be set when this option is selected include:

ndgnstc (integer) – the number of timesteps between diagnostic calculations (global energetics and regional momentum and tracer term balances).

ndmph1,2,3,4 (integer) – the number of time steps between the sending of various history files to the NCAR mass store. Selecting ndmph as a multiple of nwrth allows for multiple writes onto one history file.

nfacdg (integer) – governs the frequency of printing the diagnostic computations in the subroutines diag.F and diag2.F (i.e., nfacdg*ndgnstc timesteps).

nprnstf (integer) – the number of timesteps between the printing of diagnostic stream-function information (mscan, resmax, and island values).

nrestrt (integer) – the number of timesteps between the creation of restart files (R files). Restart files are sent to the NCAR mass store immediately after creation.

ntime avg dump (integer) – the number of timesteps in the time-averaging interval. This parameter also controls the writing of the time-average history files, and should be an integer multiple of ntime avg freq. This is only valid when the logical switch write at reg int is set true.

ntravg (integer) – the number of timesteps between the creation of regional tracer averages.

ntsi (integer) – the number of timesteps between the printing of on-line integral analyses and timestep information.

nwrth1,2,3,4 (integer) – the number of time steps between the writing of various history files, if write at reg int = .true.

write at eom (logical) – if true, this option causes all history files, restart files, and diagnostics to be written at the end of each month. All variables related to write at reg int (nwrth1, etc.) are ignored unless write at reg int is also true.

write at eoy (logical) – if true, this option causes all history files, restart files, and diagnostics to be written at the end of each year. All variables related to write at reg int (nwrth1, etc.) are ignored unless write at reg int is also true.

The ocean-model user can further control the timing of the history, restart, and diagnostics output by selecting the write at eom and write at eoy suboptions:

write at eom diags (logical) – print diagnostics at the end of each month.

write at eoy diags (logical) – print diagnostics at the end of each year.

write at eom history (logical) – write history files at the end of each month.

K-4 write at eoy history (logical) – write history files at the end of each year. write at eom restart (logical) – write restart files at the end of each month. write at eoy restart (logical) – write restart files at the end of each year. The user can mix “eom” and “eoy” suboptions (e.g., write at eom history = .true., write at eoy restart = .true., write at eom diags = .true.); any conflict in the options is resolved in subroutine checks in favor of the “eoy” option. The selection of the length of regular and leap years, the calendar type (Julian or equally-spaced months), and specification of the starting model time are all controlled in the namelist tminfo, which is also set in the file model.data. At the beginning of a model run, the starting time is specified as integer values of day0:month0:year0 and hour0:minut0:second0. For example, a starting date of September 3, year 5 at 3:45:15pm would be initialized as year0=5, month0=9, day0=3, hour0=15, minut0=45, sec0=15. If the model is being restarted from restart files, these initial time values will be ignored, and the restart time will instead be read from the character variable stamp, which is read from the restart files. Variables in tminfo include: yrnorm (real) – the length of a normal year, in days. yrleap (real) – the length of a leap year, in days. Select yrleap = yrnorm if no leap year is desired. julian (logical) – if true, selects the Julian calendar. If false, each month will contain yrnorm/12 days, and, even if specified, leap years will not be allowed. sec0 (integer) – number of seconds; acceptable values: 0-59 minut0 (integer) – number of minutes; acceptable values: 0-59 hour0 (integer) – number of hours; acceptable values: 0-23 day0 (integer) – day number; acceptable values: 1-31 month0 (integer) – month number; acceptable values: 1-12 year0 (integer) – year number; acceptable values: 0,1, … New options have been added to the tsteps namelist, mainly for use in the coupledocean model. Instead of specifying dtts in seconds, the user may instead set compute dt (logical) – if true, the ocean model computes the timestep dtts at runtime: dtts = 1.0/n dt per day. n dt per day (integer) – the number of timesteps in one day. For coupled-ocean integrations, the user may set either nflux (integer) – the number of ocean-model timesteps between flux exchanges with the flux-coupler, or

K-5 n com per day (integer) – the number of communications (flux exchanges) with the flux coupler per model day. This option is useful in conjunction with compute dt = .true.

If both of these variables are set and one conflicts with the other, the ocean model will stop with an error message. Note that the relationship among the variables is nflux = n dt per day/n com per day.

K-6 Preprocessor Options This is a list of preprocessor options in effect for standard CSM ocean runs.

2×2 stand- 2×2 coupled 3×3 stand-
alone ocean  ocean/atmosphere alone ocean

Numerical options  congrad5pt  congrad5pt  congrad5pt
islands  islands  islands
fourfil  fourfil  fourfil
northpole  northpole  northpole
taperhrzvd  taperhrzvd  taperhrzvd
upwind3  upwind3
Model resolution  (default)  (default)  model3x3
Vertical sub-grid-scale implicitvmix  implicitvmix  implicitvmix
mixing  kmix  kmix  kmix
Horizontal and isopycnal consthmix  consthmix  consthmix
sub-grid-scale mixing isopycmix  isopycmix  isopycmix
isopycmixspatialvar  isopycmixspatialvar isopycmixspatialvar
Surface forcing options sflxcombo  sflxpvm  sflxcombo
sflxcomboavan  sflxpvmicetilt  sflxcomboavan
sflxiceflx  sflxiceflx  sflxiceflx
sflxsw  sflxsw  sflxsw

restormed

Boundary conditions  cyclic  cyclic  cyclic
Input/Output  diskless  diskless  diskless
Model performance  multitasking  multitasking  multitasking
and monitoring  readchunk  readchunk
timing  timing  timing
Conveniences  autorestart  autorestart  autorestart
readrmsk  readrmsk  readrmsk

time average

Hardware options  CRAY  CRAY  CRAY

K-7