K. How_to_Run_the_Model_____ The NCAR CSM Ocean Model Job Scripts Two job scripts are provided by the NCAR CSM ocean model developers: one wh* *ich compiles the ocean model and creates a binary to be used in the coupled model s* *ystem, and another which compiles and executes the stand-alone ocean model. Both scrip* *ts are intended to be self explanatory and are designed to reduce the possibility of s* *et-up errors, but because there are certain features which may need some additional explanati* *on, this section provides supplementary documentation. Starting and Restarting the NCAR CSM Model There are several procedures for starting and restarting the NCAR CSM ocean* * model, depending on the options chosen by the user. Originally, the CSM model had onl* *y one method for starting the model and one for restarting it, but the subsequent add* *ition of an automatic restart option and an option to create a new run based on restart * *files from another case increased the number of start/restart procedures. The MOM 1.1 produced a single restart file, containing all the necessary ti* *me levels of information. A major disadvantage of a single restart file is that the file* * size gets unmanageably large as the model resolution becomes finer. Consequently, the NCA* *R CSM ocean model produces three restart files. The R2 restart file contains the time* * information, barotropic streamfunction variables, and the accumulated ice formation (when sf* *lxiceflx is defined). The required two time levels of the potential temperature, salinity, * *and horizontal velocity fields are stored in two of the R3, R4, or R5 restart files. The empty* * restart file is determined automatically by the code itself based on the number of time leve* *ls stored internally. The autorestart feature was added for the user's convenience: it allows the* * same job script to be submitted for multiple continuation runs, completely unaltered. T* *he user need not provide restart file names in the script; rather, this information is * *read from a "pointer" file and the ocean model, during execution, then reads the most recen* *t restart files from the disk location specified in the pointer file. The autorestart opt* *ion is selected by setting the script-control variable autorestart = yes, which in turn adds au* *torestart to the list of all preprocessor options. Whether or not the autorestart feature is selected, the standard method of * *starting an initial stand-alone CSM ocean model run is to set the script variable initial_r* *un = yes and provide initial conditions for the starting date of the model integration, for * *which, in the stand-alone model, mean January Levitus values have typically been used. The le* *ngth of the model integration is controlled by setting numsteps, the total number of ti* *mesteps for the run. When initial_run = yes, the job script sets the model variables nb=0, * *restart-flag = false, and nlast = numsteps. K-1 Because there are situations in which modelers want to start a new model ru* *n from initial conditions provided from a prior run, but want to reset the starting ti* *me, a variant start-up option is available: set initial_run = yes and nb=0, but set restart_f* *lag = true, and provide the names of the R2, R3, R4, and/or R5 restart files. This is the stand* *ard manner in which the ocean model is initialized in coupled model runs, and it is valid * *regardless of the autorestart setting. The method used to continue a stand-alone model run depends on whether or n* *ot the autorestart option is selected. If autorestart is not in effect, the model run * *is continued by setting initial_run = no, nb = itt, the timestep number from the last restart f* *ile, restart_flag = true, autorestart = no, and by providing the names of the restart files, i.e.* *, the R2, R3, R4, and/or R5 files in the appropriate part of the job script. For each su* *bsequent continuation run, the user must set nb = itt, and provide the most recent resta* *rt-file names. The script will automatically set the value of nlast to nb + numsteps. Continuing a stand-alone model run with the autorestart option is simple; t* *he user needs only to set initial_run = noand autorestart = yes. The user may decide to* * reset the value of numsteps during subsequent continuation runs, but if not, then all con* *tinuation runs may be initiated by submitting and resubmitting this continuation script e* *ntirely unaltered. The user should note that the script sets nlast to numsteps in this * *case; however, the autorestart preprocessor option activates code which sets nlast to the last* * timestep of the previous run, itt + nlast, thus effectively setting the total integratio* *n length to numsteps. Continuing a coupled ocean-model run is even simpler, because the autoresta* *rt option is required, and the master job script (the ".nqs" script) controls whether the* * run is an initial run or a continuation. This allows the same ocean script to be used, un* *altered, in either case. Timing and Memory Requirements Because time requirements for a model integration depend on the particular * *options chosen, it is impractical to provide precise model timing and memory informatio* *n for all configurations. However, such information from the 2x2 and 3x3 models with the * *options presented in this document will provide a useful starting estimate for users wh* *o select other model options or run on other computers. Memory requirements for the ocean model increase with the number of process* *ors selected. Commonly, we run the models at NCAR on Cray C90, YMP, or J90 computer* *s, with either 8 or 16 processors. The 2x2 model requires 32Mw of memory on a 16-p* *rocessor machine and 23Mw of memory on an 8-processor machine. The figures for the 3x3 m* *odel are 11Mw and 8Mw, respectively. When the preprocessor option time_average is ch* *osen, the model memory requirements for the 2x2 model increase to 38Mw of memory on a 16-processor machine and 29Mw of memory on an 8-processor machine. NCAR CSM ocean model timing requirements and cost comparisons with other gl* *obal ocean-circulation models are discussed in detail in the timing section at the o* *cean-model K-2 website (http://www.cesm.ucar.edu/ocn-ncom). Timing figures are closely lin* *ked to the options selected. For our configuration, each 2x2 model timestep uses appro* *ximately 4 cpu seconds on a Cray C90, 9 cpu seconds on a Cray YM9, and 19 cpu seconds on* * a Cray J90. The comparable figures for the 3x3 model are approximately 1, 2.4, an* *d 5 cpu seconds, respectively. For a detailed, up-to-date summary of changes to the ocean model, its optio* *ns and set-up scripts since our last public code release, please refer to the section * *"What's New in NCOM" on the above referenced web page. We plan to periodically update this * *page, so it a good idea for NCAR CSM ocean-model users to check this page from time t* *o time. Time Manager The time-manager subroutine, tmngr, has evolved at NCAR over several years,* * and was eventually substantially rewritten in order to incorporate these changes in* * an orderly way. In a departure from the MOM 1.1 time manager, all time-dependent decision* *s in the NCAR CSM ocean model are based on an integer timestep counter, itt, and the corresponding elapsed model time in seconds, ttsec. Some MOM 1.1 variables whic* *h are based on the number of model days have integer timestep-number counterparts in * *the CSM ocean model, but others were eliminated during the restructuring. Many features* * of the MOM 1.1 time manager remain in the CSM version, although the underlying computa* *tions have changed. Choices of a simple or Julian calendar, allowing leap years or no* *t, and an arbitrary year length are all available. New features have been added. Because model I/O has been customized to the NCAR computing environment, new time-manager variables have been added to contr* *ol the writing and storing of files. Restart and history files are independently c* *ontrolled, and may be written at the end of each calendar month, or after a specified number o* *f timesteps, or both. Other new time-manager variables include logical flags which signal th* *e present timestep is the end of a day, month, or year, and integer values of the present* * second, minute, and hour of the day and day, month, and year. All of these variables ar* *e available for use throughout the model upon the inclusion of the files switch.h and ctmng* *r2.h. Controls for the model integration (starting and ending timesteps, the freq* *uency of sampling for time-averaging, the frequency of writing diagnostics and history a* *nd restart files, and communication information for the Flux Coupler) are all set in the n* *amelist contrl, which is located in the file model.data in the stand-alone job script a* *nd ocn.parm in the coupled-model job script. Namelist contrl members include: nb (integer) - the beginning timestep number. Set nb = 0 for an initial run* *, nb = itt for a standard continuation run, and nb > 0 for an autorstart continuation * *run. nlast (integer) - the last timestep number of the model integration; valid * *only for stand-alone integrations. ntime_avg_freq (integer) - the number of timesteps between the accumulation* * of data used in time averaging. K-3 write_at_reg_int (logical) - if true, this option causes history files, restart* * files, and diagnostics to be written at regular intervals. Variables which must be set whe* *n this option is selected include: ndgnstc (integer) - the number of timesteps between diagnostic calculations* * (global energetics and regional momentum and tracer term balances). ndmph1,2,3,4 (integer) - the number of time steps between the sending of va* *rious history files to the NCAR mass store. Selecting ndmph as a multiple of nwr* *th allows for multiple writes onto one history file. nfacdg (integer) - governs the frequency of printing the diagnostic computa* *tions in the subroutines diag.F and diag2.F (i.e., nfacdg*ndgnstc timesteps). nprnstf (integer) - the number of timesteps between the printing of diagnos* *tic stream-function information (mscan, resmax, and island values). nrestrt (integer) - the number of timesteps between the creation of restart* * files (R files). Restart files are sent to the NCAR mass store immediately after creation. ntime_avg_dump (integer) - the number of timesteps in the time-averaging in* *terval. This parameter also controls the writing of the time-average history files,* * and should be an integer multiple of ntime_avg_freq. This is only valid when t* *he logical switch write_at_reg_int is set true. ntravg (integer) - the number of timesteps between the creation of regional* * tracer averages. ntsi (integer) - the number of timesteps between the printing of on-line in* *tegral analyses and timestep information. nwrth1,2,3,4 (integer) - the number of time steps between the writing of va* *rious history files, if write_at_reg_int = .true. write_at_eom (logical) - if true, this option causes all history files, restart* * files, and diagnostics to be written at the end of each month. All variables related to wr* *ite_at_reg_int (nwrth1, etc.) are ignored unless write_at_reg_int is also true. write_at_eoy (logical) - if true, this option causes all history files, restart* * files, and diagnostics to be written at the end of each year. All variables related to wri* *te_at_reg_int (nwrth1, etc.) are ignored unless write_at_reg_int is also true. The ocean-model user can further control the timing of the history, restart, and diagnostics output by selecting the write_at_eom and write_at_eoy suboptions: write_at_eom_diags (logical) - print diagnostics at the end of each month. write_at_eoy_diags (logical) - print diagnostics at the end of each year. write_at_eom_history (logical) - write history files at the end of each month. K-4 write_at_eoy_history (logical) - write history files at the end of each year. write_at_eom_restart (logical) - write restart files at the end of each month. write_at_eoy_restart (logical) - write restart files at the end of each year. The user can mix "eom" and "eoy" suboptions (e.g., write_at_eom_history = .true* *., write_at_eoy_restart = .true., write_at_eom_diags = .true.); any conflict in th* *e options is resolved in subroutine checks in favor of the "eoy" option. The selection of the length of regular and leap years, the calendar type (Julia* *n or equally-spaced months), and specification of the starting model time are all co* *ntrolled in the namelist tminfo, which is also set in the file model.data. At the begin* *ning of a model run, the starting time is specified as integer values of day0:month0* *:year0 and hour0:minut0:second0. For example, a starting date of September 3, year 5 * *at 3:45:15pm would be initialized as year0=5, month0=9, day0=3, hour0=15, minut0=4* *5, sec0=15. If the model is being restarted from restart files, these initial tim* *e values will be ignored, and the restart time will instead be read from the character v* *ariable stamp, which is read from the restart files. Variables in tminfo include: yrnorm (real) - the length of a normal year, in days. yrleap (real) - the length of a leap year, in days. Select yrleap = yrnorm if n* *o leap year is desired. julian (logical) - if true, selects the Julian calendar. If false, each month w* *ill contain yrnorm/12 days, and, even if specified, leap years will not be allowed. sec0 (integer) - number of seconds; acceptable values: 0-59 minut0 (integer) - number of minutes; acceptable values: 0-59 hour0 (integer) - number of hours; acceptable values: 0-23 day0 (integer) - day number; acceptable values: 1-31 month0 (integer) - month number; acceptable values: 1-12 year0 (integer) - year number; acceptable values: 0,1, . . . New options have been added to the tsteps namelist, mainly for use in the coupl* *ed- ocean model. Instead of specifying dtts in seconds, the user may instead set compute_dt (logical) - if true, the ocean model computes the timestep dtts at r* *un- time: dtts = 1.0/n_dt_per_day. n_dt_per_day (integer) - the number of timesteps in one day. For coupled-ocean integrations, the user may set either nflux (integer) - the number of ocean-model timesteps between flux exchanges wi* *th the flux-coupler, or K-5 n_com_per_day (integer) - the number of communications (flux exchanges) with the flux coupler per model day. This option is useful in conjunction with compute_d* *t = .true. If both of these variables are set and one conflicts with the other, the ocean * *model will stop with an error message. Note that the relationship among the variables is n* *flux = n_dt_per_day/n_com_per_day. K-6 Preprocessor_Options_ This is a list of preprocessor options in effect for standard CSM ocean runs. 2x2 stand- 2x2 coupled 3x3 stand- alone ocean ocean/atmosphere alone ocean ______________________________________________________________________________* *________ Numerical options congrad5pt congrad5pt congrad5pt islands islands islands fourfil fourfil fourfil northpole northpole northpole taperhrzvd taperhrzvd taperhrzvd upwind3 upwind3 Model resolution (default) (default) model3x3 Vertical sub-grid-scaleimplicitvmix implicitvmix implicitvmix mixing kmix kmix kmix Horizontal and isopycnalconsthmix consthmix consthmix sub-grid-scale mixing isopycmix isopycmix isopycmix isopycmixspatialvar isopycmixspatialvar isopycmixspat* *ialvar Surface forcing optionssflxcombo sflxpvm sflxcombo sflxcomboavan sflxpvmicetilt sflxcomboavan sflxiceflx sflxiceflx sflxiceflx sflxsw sflxsw sflxsw restormed Boundary conditions cyclic cyclic cyclic Input/Output diskless diskless diskless Model performance multitasking multitasking multitasking and monitoring readchunk readchunk timing timing timing Conveniences autorestart autorestart autorestart readrmsk readrmsk readrmsk time_average Hardware options CRAY CRAY CRAY K-7