The ocean setup script, $SCRIPTS/ocn.setup.csh, is called by test.a1.run when an active ocean model is selected in the $MODELS environment variable array. ocn.setup.csh prepares the ocean model component for execution. To do so, it defines the run environment, collects the ocean input data sets, then builds the dynamic ocean model by calling gmake.
If ocn.setup.csh is unable to complete any of these tasks, it will abort with a non-zero error status. The test.a1.run script checks the error status and will halt if an error is detected.
#! /bin/csh -fv #=============================================================================== #=============================================================================== # Purpose: # (a) build an executable model (pop, a CSM active ocn component) # (b) gather or create necessary input files # # For help, see: http://www.cesm.ucar.edu/models/ocn #=============================================================================== cat $0;$TOOLS/ccsm_checkenvs || exit -1 # cat this file, check envs
This first section documents the ocean setup script.
The first line of this section identifies this as a C-shell script. The "-f" option prevents the user's personalized $HOME/.cshrc file from being executed to avoid introducing aliases that could adversely affect the operation of this script.
The CVS lines document the revision control version of this script.
The echo lines document the purpose of this script. These output from the echo commands will appear in the component log files.
The cat command combines two functions on one line. The "cat $0" command prints a copy of the entire setup script into the output log file in order to document the exact options set by this script. Then $TOOLS/ccsm_checkenvs writes the environment variables that have been set by test.a1.run into the same output log file. If any of the required environment variables are not set, the setup script will exit with an error status of -1.
echo ----------------------------------------------------------------- echo setting up some internal Env variables and path names echo ----------------------------------------------------------------- set SRCDIR = $CSMCODE/ocn/pop set my_path = $SCRIPTS/src.ocn if ( $OCN_GRID == gx1v3) then setenv DT_COUNT 23 setenv TIME_MIX_FREQ 17 setenv INIT_TS_FILE ts_PHC2_jan_20010711 else if ( $OCN_GRID == gx3) then setenv DT_COUNT 12 setenv TIME_MIX_FREQ 100 setenv INIT_TS_FILE ts_PHC2_jan_20011012 else echo "$0 ERROR: Cannot deal with GRID = $GRID" exit -1 endif
This section defines the ocean model source-code location and sets a few resolution dependent variables.
SRCDIR points to the directory for the POP model. Under SRCDIR are a number of subdirectories containing source code, documentation, input templates and other files needed by POP. SRCDIR is typically treated as an archive directory containing ``frozen'' code. Frozen source code refers to code that represents the exact released version of the model. Frozen code is never modified under any circumstances. To modify code, copy it from $SRCDIR into $my_path and make the necessary changes to the $my_path copy.
$my_path identifies a directory that contains user modified POP code. Any POP code that a user needs to modify should be placed in the $my_path directory. POP source code files in $my_path will have precedence over any copies of the same file that exist under SRCDIR. $my_path is designed to be only one level deep, with no subdirectories. If changes are so extensive that subdirectories are required, the ocn.setup.csh script will need to be modified accordingly.
Next, three resolution dependent variables are set. These variables will be input into the POP namelist via the pop_in file, which is modified in section b:
DT_COUNT is the number of timesteps per day.
TIME_MIX_FREQ is the number of timesteps between time-mixing
INIT_TS_FILE is the initial-condition temperature and salinity restart file used
for continuation or startup runs.
See the POP users guide for full details.
# branch- or hybrid-run setup
#---------------------------------------------
if ($RUNTYPE == branch || $RUNTYPE == hybrid) then
setenv INIT_TS_FILE ${REFCASE}.pop.r.$REFDATE-00000
setenv TAVG_TS_FILE ${REFCASE}.pop.rh.$REFDATE-00000
# position the branch or hybrid files
if !(-d rest ) mkdir -p rest
cd rest
$TOOLS/ccsm_getfile $REFCASE/ocn/rest/$INIT_TS_FILE || exit 99
$TOOLS/ccsm_getfile $REFCASE/ocn/rest/$INIT_TS_FILE.hdr || exit 99
$TOOLS/ccsm_getfile $REFCASE/ocn/rest/$TAVG_TS_FILE || exit 99
$TOOLS/ccsm_getfile $REFCASE/ocn/rest/$TAVG_TS_FILE.hdr || exit 99
cd ..
# generate the appropriate restart pointer files
cat >! $SCRIPTS/rpointer.ocn.restart << EOF
$cwd/rest/$INIT_TS_FILE.hdr
$cwd/rest/$INIT_TS_FILE
EOF
cat >! $SCRIPTS/rpointer.ocn.tavg << EOF
$cwd/rest/$TAVG_TS_FILE.hdr
$cwd/rest/$TAVG_TS_FILE
EOF
endif
In this section, a number of the initial data and restart pointer files are identified and positioned for branch or hybrid runs. For hybrid or branch runs, the initial (INIT_TS_FILE) and time-averaged (TAVG_TS_FILES) input filenames are determined by resolving the name based on the $REFCASE and $REFDATE variables set in the test.a1.run script. These data files and their associated header files are placed in the ocean restart directory. Finally the restart pointer files, $SCRIPTS/rpointer.ocn.restart and $SCRIPTS/rpointer.ocn.tavg, are created.
echo ------------------------------------------------------------------------- echo a. Build an executable in $OBJDIR echo ------------------------------------------------------------------------- if ( -e ocn && $?DONTBUILD ) then echo "Note: using an existing binary" else echo ------------------------------------------------------------------------- echo a.1 Calculate processor tiling based on $NTASK echo ------------------------------------------------------------------------- if ($NTASK == 4) setenv NX 2 if ($NTASK == 8) setenv NX 4 if ($NTASK == 16) setenv NX 4 if ($NTASK == 32) setenv NX 8 if ($NTASK == 40) setenv NX 5 if ($NTASK == 48) setenv NX 8 if ($NTASK == 64) setenv NX 8 if !($?NX) echo ERROR: ocn cant deal with NTASK=$NTASK if !($?NX) exit -1 @ ny = $NTASK / $NX; setenv NY $ny
The POP executable is built in this section. First the processor tiling is resolved, then the code necessary to build the model is copied in from a number of locations. Once the source code is in place, the POP executable is built with gmake.
This section attempts to automatically specify, for a variety of pre-determined processor counts, the values of NS and NY, the number of processors to assigned to the X and Y dimensions for a number of pre-determined processor counts. This is designed to be automatic. The values of NX and NY are passed as arguments to the gmake file, where they are used to set the values of the C-preprocessor parameters NPROC_X and NPROC_Y. Further documentation on NPROC_X and NPROC_Y is in the POP documentation: www.cesm.ucar.edu/models/ccsm2.0/pop/doc/POPusers_chap2.html#POPusers2.2.2
echo -----------------------------------------------------------------
echo a2. creating the internal directory structure
echo -----------------------------------------------------------------
set compile_dir = $OBJDIR/compile
set source_dir = $OBJDIR/source
if !(-d $source_dir ) mkdir -p $source_dir
if !(-d $compile_dir ) mkdir -p $compile_dir
echo ' '
echo '-----------------------------------------------------------------'
echo ' copy the necessary files into $source_dir '
echo '-----------------------------------------------------------------'
cd $source_dir
cp -fp $SRCDIR/input_templates/${OCN_GRID}_model_size.F model_size.F
cp -fp $SRCDIR/source/*.[FCc] .
cp -fp $SRCDIR/mpi/*.[FCc] .
cp -fp $CSMSHR/*.[F]* .
cp -fp $CSMSHR/*.[Cc] .
if (-d $my_path ) cp -fp $my_path/*.[FCc] .
The POP source code and compilation products are placed into two different directories, source_dir and compile_dir. Here, the two directories are created and all the necessary source code is copied into source_dir. Notice that the last copy is from $my_path, which overwrites any files that have the same name as the files in $my_path .
input_templates/gx1v3_model_size.F specifies the grid dimensions and number of tracers.
# # recompile if 2d decomp is changed # echo $OCN_GRID $NTASK $NX $NY >! $OBJDIR/ocnres.new diff $OBJDIR/ocnres.new $OBJDIR/ocnres.old || touch `grep -l NPROC $OBJDIR/source/*` echo $OCN_GRID $NTASK $NX $NY >! $OBJDIR/ocnres.old
The POP processor tiling is compiled into the model executable. Any changes to the number of processors assigned to POP will change the tiling, which requires recompiling POP. This section compares the currently requested tiling with the tiling specified during the last build. If the two tilings are different, a recompilation of POP is forced.
echo -----------------------------------------------------------------
echo a3. compile and copy the executable into $EXEDIR directory
echo -----------------------------------------------------------------
if ($BLDTYPE == 'true') then
cd $compile_dir
cc -o makdep $CSMBLD/makdep.c || exit 2
set EXEC = ocn_${NTASK}
set THREAD = FALSE ; if ($NTHRD > 1) set THREAD = TRUE
gmake -j 6 -f $CSMBLD/Makefile MACFILE=$CSMBLD/Macros.$OS \
MODEL=pop NX=$NX NY=$NY THREAD=$THREAD \
VPATH=$source_dir EXEC=$OBJDIR/$EXEC || exit 2
# gmake -f $CSMBLD/Makefile MACFILE=$CSMBLD/Macros.$OS mostlyclean
#--- cp & link into EXEDIR ---
rm -f $EXEDIR/ocn
cp $OBJDIR/$EXEC $EXEDIR/ocn
else
echo "BLDTYPE = $BLDTYPE"
endif
endif
A number of steps are required to build the POP executable using gmake. First, the dependency generator, makdep, is created. When gmake is called, makdep will build a list of source-code dependencies that identify the specific POP files that need to be recompiled if any of the source code is modified.
$EXEC specifies the name of the POP executable. The processor count NTASK is appended to the name to differentiate between executables built for different values of NTASK.
$THREAD allows for multiple MPI threads. However, since POP does not support threading, this should always be FALSE.
Finally, gmake is called using the generic CCSM gmake and macros files and the resulting executable is copied into the ocean-model execution directory.
echo ' ' echo '-------------------------------------------------------------------------' echo " b. gather or create necessary input files in $EXEDIR " echo '-------------------------------------------------------------------------' set IYEAR0 = `echo $BASEDATE | cut -c1-4 | sed -e 's/^0*//'` set IMONTH0 = `echo $BASEDATE | cut -c6-7 | sed -e 's/^0*//'` set IDAY0 = `echo $BASEDATE | cut -c9-10 | sed -e 's/^0*//'` if ($RUNTYPE == startup || $RUNTYPE == hybrid ) @ IDAY0 = $IDAY0 + 1
The POP model receives the initial date information is a slightly different format than it is defined in the $SCRIPTS/test.a1.run script. This short section does this text transformation.
IYEAR0, IMONTH0 and IDAY0 are the numeric representations for the coordinated base year, month and day set in test.a1.run. Startup and hybrid runs with POP require that the rest of the coupled system run for 1 day before POP starts. This is accomplished by incrementing the IDAY0 setting.
# sed commands are of the form s#OldString#NewString#
cat >! $EXEDIR/commands.sed << EOF
s#OUTPUTL#$EXEDIR#
s#OUTPUTR#$EXEDIR/rest/$CASE.pop.r#
s#OUTPUTH#$EXEDIR/hist/$CASE.pop.h#
s#OUTPUTD#$EXEDIR/hist/$CASE.pop.d#
s#OUTPUT/pop_pointer#$SCRIPTS/rpointer.ocn#
s#RUNID#$CASE#; s#LOG_FILENAME#$ocn_out#
s#INIT_TS_OPTION#$RUNTYPE#
s#INPUT#$EXEDIR/input#
s#DT_COUNT#$DT_COUNT#; s#TIME_MIX_FREQ#$TIME_MIX_FREQ#
s#IYEAR0#$IYEAR0#; s#IHOUR0#0#
s#IMONTH0#$IMONTH0#; s#IMINUTE0#0#
s#IDAY0#$IDAY0#; s#ISECOND0#0#
EOF
sed -f $EXEDIR/commands.sed $SRCDIR/input_templates/${OCN_GRID}_pop_in
>! $EXEDIR/pop_in || exit 3
# put pop_in where pop can find it (specific to mpirun, poe, etc)
ln -sf $EXEDIR/pop_in $EXEDIR/..
if (-d $EXEDIR/../all) ln -sf $EXEDIR/pop_in $EXEDIR/../all
Building the input namelist file for the ocean setup procedure differs from the other components. The text file ``pop_in'' contains a complete, resolution-dependent set of namelist input to POP, with several generically named values that are intended to be automatically modified by the ocean setup script. This section carries out this modification using the Unix stream editor, sed, to insert the desired namelist values into the pop_in file.
This set of sed commands modifies selected variables from the large generic pop_in namelist file in the directory $SRCDIR/input_templates. The sed commands are placed in the file commands.sed, then applied against the generic pop_in file. Additional changes to the pop_in file can be added here by following the examples in the script and appropriately modifying the pop_in file. The resulting pop_in file is linked into the directory in the next level up and to the /all directory for use on the SGI.
echo '-------------------------------------------------------------------------'
echo c. cp input datasets from $SRCDIR ...
echo '-------------------------------------------------------------------------'
if !( -d $EXEDIR/rest ) mkdir -p $EXEDIR/rest || exit 2
if !( -d $EXEDIR/hist ) mkdir -p $EXEDIR/hist || exit 2
if !( -d $EXEDIR/input ) mkdir -p $EXEDIR/input || exit 2
cd $EXEDIR/input
foreach FILE (depth_accel history_contents \
movie_contents region_ids scalar_contents tavg_contents \
transport_contents vert_grid )
$TOOLS/ccsm_cpdata $SRCDIR/input_templates/${OCN_GRID}_$FILE $FILE || exit 99
end
foreach FILE (eos_coefficients.ieeer8 )
$TOOLS/ccsm_cpdata $SRCDIR/input_templates/${OCN_GRID}_$FILE $FILE
end
This section collects the input datasets needed for the ocean model. The input datasets are all uniquely named by a description and a six digit number that documents the creation date (format: yymmdd) of the file. While the hard-wiring of filenames restricts the degree of automation, it ensures that the exact data that the user requests is input into the model.
First, the restart, history and input data directories are created if they don't already exist. Then text input files from the POP ``input_template'' directory are copied into the ocean input directory.
See the CCSM23.0 POP User's Guide, chapter 4, Model diagnostics and output, for a complete discussion of these files ( www.cesm.ucar.edu/models/ccsm2.0/pop/doc/POPusers_chap4.html ).
echo '-------------------------------------------------------------------------'
echo c. acquire initial/boundary datasets disk or MSS
echo '-------------------------------------------------------------------------'
set OCNDATA = ocn/pop/$OCN_GRID # point to disk copies
if ($RUNTYPE == startup ) then
$TOOLS/ccsm_getinput $OCNDATA/ic/${INIT_TS_FILE}.ieeer8 ts
$TOOLS/ccsm_getinput $OCNDATA/ic/${INIT_TS_FILE}.readme ts.readme
endif
if ($OCN_GRID == gx1v3) then
$TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20010402.ieeer8 horiz_grid
$TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20010402.readme horiz_grid.readme
$TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20010709.ieeei4 region_mask
$TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20010709.readme region_mask.readme
$TOOLS/ccsm_getinput $OCNDATA/grid/topography_20010702.ieeei4 topography
$TOOLS/ccsm_getinput $OCNDATA/grid/topography_20010702.readme topography.readme
$TOOLS/ccsm_getinput $OCNDATA/forcing/shf_mm_all_85-88_20010308.ieeer8 shf
$TOOLS/ccsm_getinput $OCNDATA/forcing/shf_mm_all_85-88_20010308.readme shf.readme
$TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_mm_all_85-88_20010320.readme shf.readme
$TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_mm_all_85-88_20010320.ieeer8 sfwf
else if ($OCN_GRID == gx3) then
$TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20001030.ieeer8 horiz_grid
$TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20001030.readme horiz_grid.readme
$TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20001030.ieeei4 region_mask
$TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20001030.readme region_mask.readme
$TOOLS/ccsm_getinput $OCNDATA/grid/topography_20001030.ieeei4 topography
$TOOLS/ccsm_getinput $OCNDATA/grid/topography_20001030.readme topography.readme
$TOOLS/ccsm_getinput $OCNDATA/forcing/shf_20011030.ieeer8 shf
$TOOLS/ccsm_getinput $OCNDATA/forcing/shf_20011030.readme shf.readme
$TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_20011030.ieeer8 sfwf
$TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_20011030.readme sfwf.readme
endif
wait
The variable OCNDATA defines the location of the collection of Initial and boundary datasets that are distributed with CCSM2.0. These variables are:
| horiz_grid_20010402.ieeer8 | Specifies the horizontal grid. |
|---|---|
| horiz_grid_20010402.readme | |
| region_mask_20010709.ieeei4 | Specifies the region mask. |
| region_mask_20010709.readme | |
| topography_20010702.ieeei4 | Specifies the topography (index of deepest |
| topography_20010702.readme | ocean level at each horizontal grid point) |
| shf_mm_all_85-88_20010308.ieeer8 | Contains monthly averaged fields of |
| shf_mm_all_85-88_20010308.readme | sea-surface temperature needed by the |
| surface heat flux forcing option ``restoring''. | |
| sfwf_mm_all_85-88_20010320.ieeer8 | Contains monthly averaged fields of |
| sfwf_mm_all_85-88_20010320.readme | sea-surface salinity needed by the surface |
| freshwater flux forcing option ``restoring''. |
The POP input datasets are copied of from $OCNDATA to the $EXEDIR/input directory. Note that in the case of a ``startup'' run, initial temperature and salinity initial conditions are also copied to the input directory.
unset echo echo ' ' echo '-----------------------------------------------------------------' echo ' End of setup shell script ' echo '-----------------------------------------------------------------' exit
Once the ocn.setup.csh script completes successfully, it will exit with a status value of 0.