Some of the Coupler's behavior can be controlled through the use of a Fortran namelist as described in Section 4. However, some necessary changes to the coupled system, such as adding new fields, requires editing the Coupler source code. In this section, we provide some guidance on how to change the Coupler code for likely situations. This section assumes some familiarity with the CCSM code and build system.
The complete list of fields exchanged between models is in cpl_fields_mod.F90. To add or subtract fields to existing data pathways in the Coupler (Section 9.2.4), only cpl_fields_mod.F90 needs to be edited.
For an example, we'll consider adding a field to the ocean-to-coupler fields, cpl_fields_o2c_*. The first step is to add a descriptive character string to the colon-delimited list of fields. The field must be added to cpl_fields_o2c_states if its a state and cpl_fields_o2c_fluxes if it is a flux. Adding a state variable So_new is shown below with changes in bold.
A new integer was added for So_new and the subsequent integers, cpl_fields_o2c_q were renumbered.
Finally, the integer parameter for the total number of fields must be updated.
NOTE: The programmer making these changes is responsible for ensuring that the character string cpl_fields_o2c_fields, the integer parameters for the individual fields and the integer parameter for the total are all consistent.
The above changes will only add a field to the ocean BUNDLE passed in to the Coupler. If you want this field passed out to another model, say the ice model, similar changes should be made to the cpl_fields_c2i_fields string and the string for any other models which need to receive this data.
To make the Coupler aware of these changes, recompile the code. On the next run of CCSM3 with the changes, the Coupler will automatically create extra storage for the new field and perform the same operations on So_new as on the rest of cpl_fields_o2c_states.
Because the Coupler is designed to work with models which may have very different internal data structures, additional editing must be done in the component models to complete the addition of the field. In the ocean model, code must be added to copy So_new from the ocean model's internal data structure into the buf argument of the ocean's cpl_interface_contractSend call. And any receiving models must also be edited to copy the received data into it's appropriate internal data structure. The steps needed to interface a component model with the Coupler are described below.
This section will review what is required to interface a component with the Coupler in place of a current component, e.g. replacing POP with a different ocean model or CAM with a different atmosphere, while sending and receiving the same fields. This only covers how to communicate with the Coupler and ignores other issues such as merging the model with the CCSM3 build system. Not every detail will be covered and programmers are encouraged to examine the code of current CCSM3 components to see how they interact with the Coupler. Clear examples of how a model is connected to the Coupler can be found in the routine msg.F90 in any of the data models (../models/atm/datm6, ../models/ocn/docn6/ etc.). msg.F90 contains all of the data model's interaction with the Coupler and mimics the steps performed by the full models. One can find all the references to the Coupler in any model within CCSM3 by searching for cpl_ in the code.
All the routines needed to connect a model to the CCSM system are in cpl_interface_mod.F90. Connecting a model to the CCSM Coupler involves placing calls to these routines within the model. No other editing or reorganization of a model's source code is required. Where exactly the calls go depends on the model code but there is some generality which can be used to guide their placement.
To use the routines in cpl_interface_mod.F90, a Fortran90 USE statement must be placed at the top of each model. Because references must also be made to the integer parameters in cpl_fields_mod.F90 and the CONTRACT datatype in cpl_contract_mod.F90, 3 use statements must appear in each model subroutine involved in Coupler communication:
use cpl_contract_mod
use cpl_interface_mod
use cpl_fields_mod
The first call which must be made is to cpl_interface_init:
call cpl_interface_init(cpl_fields_<modelname>,local_comm}
The first argument, cpl_fields_<modelname> must be one of
the component names defined at the top of cpl_fields_mod. The second
argument returns an MPI_Communicator which must be used by any MPI
calls made internally within the model. (cpl_interface_init also handles
calling MPI_Init.) This call only needs to be made once
and can be placed within one of the model's initialization routines.
The second and third calls are to cpl_interface_contractInit, one each to initialize the send and receive CONTRACT (Sec. 8.4). When called from a model, cpl_interface_contractInit should use the following set of arguments:
call cpl_interface_contractInit(contractS, cpl_fields_<modelname>,
cpl_fields_cplname, cpl_fields_<x>2c_fields, ibufi, buf, ibufr)
contractS will be the output send contract and is the only Fortran90
derived datatype a component model needs to declare to interact with the Coupler:
type(cpl_contract),save :: contractS
The other arguments of the INTERFACE routines are native scalars and arrays.
<x> should be replaced
with o, a, i, l, or r depending on if the calling model is an ocean, atmosphere, etc.
ibufi and ibufr are the integer and real parts of the INFOBUFFER
(Sec. 8.3) as simple arrays declared with one of the FIELDS parameters:
integer :: ibufi(cpl_fields_ibuf_total) real :: ibufr(cpl_fields_ibuf_total)During the CONTRACT initialization, only some of the integer parts of the infobuffer must be set before calling cpl_interface_contractInit:
ibufi(cpl_fields_ibuf_ncpl ) = number of communications per day
ibufi(cpl_fields_ibuf_gsize ) = the size of the global grid
ibufi(cpl_fields_ibuf_lsize ) = size of the local grid
ibufi(cpl_fields_ibuf_gisize ) = global size in i-index
ibufi(cpl_fields_ibuf_gjsize ) = global size in j-index
ibufi(cpl_fields_ibuf_nfields) = cpl_fields_grid_total
ibufi(cpl_fields_ibuf_dead) = 0 if not a dead model
The two-dimensional real array buf can be declared or allocated using another FIELDS parameter:
real :: buf(nx*ny, cpl_fields_grid_total)
where nx and ny are the local grid sizes in the x and y
directions. The buf array is used to tell the Coupler about the grid
the model is running on its information is used to initialize the DOMAIN
and MCT datatypes.
An example of filling buf can be found in datm6/msg.F90:
n=1
do j=1,ny ! local j index
do i=1,nx ! local i index
buf(n,cpl_fields_grid_lon ) = xc(i,j)
buf(n,cpl_fields_grid_lat ) = yc(i,j)
buf(n,cpl_fields_grid_area ) = area(i,j)
buf(n,cpl_fields_grid_mask ) = mask(i,j)
buf(n,cpl_fields_grid_index ) = n
n=n+1
enddo
enddo
In this case datm6 has very simple internal data structures to hold the local
values of longitude, latitude, area and mask. datm6's internal 2D array's,
xc, yc, etc., are ``unrolled''
into 1-dimension of buf while the other dimension of buf is used
for the different fields.
The latitude, longitude, area and mask values should be local values but
cpl_fields_grid_index contains a global index number given to each
point on the portion of the model's numerical grid communicating with the coupler (i.e.
the lowest level of a 3D atmosphere grid). Since datm6 is a single-processor application, this
assignment is trivial but care must be taken to assign the correct values
to cpl_fields_grid_index for a processor in a parallel application since
this information is used by MCT to perform the parallel communication between the model and
the Coupler. On each processor, cpl_fields_grid_index should contain the
globally-indexed values of points that processor ``owns''. The order should be
noted because later, when computed data is sent to the coupler, it must be placed in the
send buffer in the same order.
More detail on grid point numbering and its use in parallel communication
can be found in the User's Guide to the Model Coupling Toolkit.
The receive contract call to cpl_interface_contractInit can use the same input arguments as the send illustrated above but the first argument should be different, e.g. contractR. Like cpl_interface_init, the cpl_interface_contractInit calls also only need to be done once as part of the model's initialization. The last initialization step is to receive an INFOBUFFER from the Coupler with a call to cpl_interface_ibufRecv. The CONTRACTS initialized above must be passed to the routines which will communicate with the coupler during runtime.
Sending and receiving data during execution is very similar to the CONTRACT initialization. During a send, first real and/or integer arrays like ibufr and ibufi are updated with appropriate items in the INFOBUFFER. The a real array similar to buf is loaded with the latest calculated values intended for the Coupler using the indicies defined in cpl_fields_mod to place data in the correct location in buf. Then the INTERFACE send routine is called:
call cpl_interface_contractSend(cpl_fields_cplname, contractS, ibufi, buf)A receive reverses this order. First a call is made to cpl_interface_contractRecv, then data is copied out of buf using the indicies in cpl_fields_mod into the appropriate internal datatypes and scalars from the INFOBUFFER are also accessed using the indicies in cpl_fields_mod and the ibufi or ibufr arrays.