The CCSM2.0.1 Parallel Ocean Program (POP) User Guide is, to a large extent, identical to the standard Los Alamos National Laboratory (LANL) POP 1.4.3 User Guide. But because there are many minor modifications needed to run the POP model in the Community Climate System Model (CCSM) context, we produced a CCSM-specific User Guide to describe the changes and extensions to the original LANL POP 1.4.3 model which are necessary for its functioning as the ocean-model component of the CCSM.
Differences between the standard LANL POP code and the CCSM version are noted throughout this document, with references made to other CCSM documents where necessary.
POP was developed at LANL under the sponsorship of the Department of Energy's CHAMMP program, which brought massively parallel computers to the realm of climate modeling. POP is a descendant of the Bryan-Cox-Semtner (BCS) class of models (see Appendix 7.1 for the genealogy of BCS models). A number of improvements have been developed and incorporated in POP. Although originally motivated by the adaptation of POP for massively parallel computers, many of these changes improved not only its computational performance but the fidelity of the model's physical representation of the real ocean as well as. The most significant of these improvements are summarized below. Details can be found in articles by Smith et al. 1992; Dukowicz et al. 1993; and Dukowicz and Smith 1994.
Back to beginning of chapter 1
The barotropic streamfunction formulation in the standard BCS models required an additional equation to be solved for each continent and island that penetrated the ocean surface. This was costly even on machines like Cray parallel-vector-processor computers, which had fast memory access. To reduce the number of equations to solve with the barotropic streamfunction formulation, it was common practice to submerge islands, connect them to nearby continents with artificial land bridges, or merge an island chain into a single mass without gaps. The first modification created artificial gaps, permitting increased flow, while the latter two closed channels that should exist.
On distributed-memory parallel computers, these added equations were even more costly because each required gathering data from an arbitrarily large set of processors to perform a line-integral around each landmass. This computational dilemma was addressed by developing a new formulation of the barotropic mode based on surface pressure. The boundary condition for the surface pressure at a land-ocean interface point is local, which eliminates the non-local line-integral.
Consequently, the surface-pressure formulation permits any number of islands to be included at no additional computational cost, so all channels can be treated as precisely as the resolution of the grid permits.
Another problem with the barotropic streamfunction formulation is that the elliptic problem to be solved is ill-conditioned if bottom topography has large spatial gradients. The bottom topography must be smoothed to maintain numerical stability. Although this reduces the fidelity of the simulation, it does have the "desirable" side effect (given the other limitations of the streamfunction approach mentioned above) of submerging many islands, thereby reducing the number of equations to be solved. In contrast, the surface-pressure formulation allows more realistic, unsmoothed bottom topography to be used with no reduction in time step.
Back to beginning of chapter 1
An implicit free-surface boundary condition that allows the air-sea interface to evolve freely and makes sea-surface height a prognostic variable is implemented in POP. Optionally, the top-most layer thickness is allowed to change, thus permitting natural freshwater flux boundary conditions.
Back to beginning of chapter 1
Scaling of the horizontal diffusion coefficient by (cos(j))n was introduced, where j is latitude and n=1 for Laplacian mixing and n=3 for bi-harmonic mixing. This optional scaling prevents horizontal diffusion from limiting the time step severely at high latitudes, yet keeps diffusion large enough to maintain numerical stability.
Back to beginning of chapter 1
After the temperature and salinity have been updated to time-step n+1 in the baroclinic routines, the density rhon+1 and pressure pn+1 can be computed. By computing the pressure gradient with a linear combination of p at three time-levels (n-1, n, and n+1), a technique well known in atmospheric modeling (Brown and Campana, 1978), it is possible to increase the time-step by as much as a factor of two, if the internal gravity waves are the controlling factor.
Back to beginning of chapter 1
The code is written in Fortran90 and can be run on a variety of parallel and serial computer architectures. It uses domain decomposition in latitude and longitude, combined with MPI for inter-processor communications on distributed memory machines. SHMEM is also available on machines that support it (SGI Origin 2000 and Cray T3E).
Back to beginning of chapter 1
Because the code is written in Fortran90, it was relatively easy to reformulate and discretize the equations of motion to allow the use of any locally orthogonal horizontal grid. This provides alternatives to the standard latitude-longitude grid with its singularity at the North Pole.
Back to beginning of chapter 1
This generalization made possible the development of the displaced-pole grid, which moves the singularity arising from convergence of meridians at the North Pole into an adjacent landmass such as North America, Russia or Greenland. This leaves a smooth, singularity-free grid in the Arctic Ocean. That grid joins smoothly at the equator with a standard Mercator grid in the Southern Hemisphere.
Back to beginning of chapter 1
In the period 1994-97, POP was used to perform high resolution (0.28º at the Equator) global ocean simulations, running on the Thinking Machines CM5 computer then located at LANL's Advanced Computing Laboratory. (Output from these runs is available at http://climate.acl.lanl.gov/pop.) The primary motivation for performing such high-resolution simulations is to permit mesoscale eddies that play an important role in the dynamics of the ocean. Comparison of sea-surface height variability measured by the TOPEX/Poseidon satellite with that simulated by POP gave convincing evidence that still higher resolution was required (Fu and Smith 1996; Maltrud et al. 1998).
At the time, it was not possible to do a higher resolution calculation on the global scale, so an Atlantic Ocean simulation was done with 0.1º resolution at the Equator. This calculation agreed well with observations of sea-surface height variability in the Gulf Stream. Many other features of the flow were also well simulated (Smith et al. 1999). Using the 0.1º case as a benchmark, lower resolution cases were done at 0.2º and 0.4º; a comparison can be found at http://www.cgd.ucar.edu/oce/bryan/woce-poster.html.
Back to beginning of chapter 1
POP is the active ocean-model component of the Community Climate System Model; information on the CCSM coupled modelling efforts can be found at http://www.cesm.ucar.edu/.
Additionally, POP and the Los Alamos Elastic-Viscous-Plastic (EVP) sea-ice dynamics model have been coupled to the NCAR Community Climate Model (CCM) atmospheric and land-surface models, to form the Parallel Climate Model (PCM). This model is being used for climate research and global-warming studies (Washington 1999).
POP and CICE are also being used in coupled model development efforts at Colorado State University and UCLA. Information on CICE can be found at http://climate.acl.lanl.gov/models/cice/cice_frames.htm.