It is often of interest to validate that the difference between two model runs is due only to machine epsilon (i.e., roundoff) size differences that could have been introduced by, for example,
The nature of global atmospheric models such as CAM is that small solution differences grow rapidly, and hence can hide small errors due to coding, computer system, or porting. The perturbation growth test described here [Ros97] was designed as an inexpensive test to give reasonable confidence that solution differences are due only to rounding differences. There are, however, two important points to keep in mind.
PERGRO).
Hence, not all of CAM's production code is exercised by this test.
In spite of the preceding disclaimers, the perturbation growth test is a valuable tool because:
To perform the test requires three model simulations, each two days in duration. The first two are done with the control (trusted) model and differ due to a perturbation introduced into the lowest-order bits of the initial values of the temperature field of one of the runs. The difference between these simulations, as measured by the time evolution of the RMS temperature difference, is used to characterize the perturbation growth of the control model. The third run is done with the model being tested. The difference between the test simulation and either of the simulations from the control model characterizes the growth of the difference between the models.
We say that the test simulation is "within roundoff" of the control simulation if the difference growth does not substantially exceed the perturbation growth. Unfortunately, we can't be more precise than "substantially exceed". After gaining alot of practical experience doing tests of this type, we often find that the difference and perturbation growth curves will lie nearly on top of one another with the perturbation curve lying above the difference curve at times, and lying below it at other times.
The main reason we find this test so useful is that when it fails it generally fails spectacularly, i.e., the difference curve will exceed the perturbation curve by many orders of magnitude within a few model timesteps. On the other hand, a "passed" test may be fuzzy, for example, when the difference curve is consistently above the perturbation curve, but never exceeds it by more than an order of magnitude. As mentioned previously, at this point it's time to perform a full blown climate validation.
Below the procedure to perform a perturbation growth test is described in detail. The example illustrates the test for the T42 Eulerian dycore, but any dycore, resolution, or constituent scenario could be used.
-pergro option (in addition
to any options that are specific to the model configuration being tested).
% configure -pergro
The namelist for a two day run with no perturbation applied to the initial conditions is set up as follows:
% build-namelist -s -case camrun -runtype startup \ -namelist "&camexp stop_option='ndays', stop_n=2 nhtfrq=1 ndens=1 \ mfilt=97 hfilename_spec='h%t.nc' empty_htapes=.true. \ fincl1='T:I','PS:I' aqua_planet=.true. /"
This specifies a two-day initial run, in "aqua-planet" mode, with real*8 precision output of just the T and PS fields at every timestep. Additional variables set for convenience specify writing history output every timestep, writing all timesteps to one output file, use 64-bit NetCDF format for output, and renaming the history file to 'h0.nc'. Note that output of the PS field is required for computing the difference statistic which is a mass weighted RMS value.
The namelist for a two day run with a perturbation applied to the
initial temperature field is set up just as above, but with the added
namelist variable "pertlim=1.e-14".
Mass weighted RMS differences for all fields in two CAM history files
are computed using the tool cprnc which is distributed with
CAM (in the models/atm/cam/tools/cprnc directory). The output
from cprnc will contain a line formatted as follows for each
temperature timesample in the two files:
MASS WEIGHTED RMS T 6.6882E-12
A tool can be written in the user's favorite scripting language to strip
the RMS values from the cprnc output to make them available
for plotting. An example of such a tool, cprncdf, is a perl script
which, if given one filename argument, assumes that the file contains
output from cprnc and strips the RMS T values from the file.
If cprncdf is given two filename arguments it assumes they are
CAM history files on which it invokes cprnc and then processes
its output. cprncdf writes output which looks
like this to stdout:
e48pm_lf_aqpgro e48ph_aqpgro T 0 4.9115E-12 1 5.8276E-12 2 6.6468E-12 3 5.0961E-12 ...
The 3 header lines are comprised of the case names from each of the runs
and the name of the variable that the RMS values are for.
(cprncdf has a command-line option that allows outputting the
RMS differences of any variable in the two files.) Then there is a line
for each timesample in the files for which the RMS value is not zero. The
first value in the line is the timestep number, followed by the RMS value.
cprncplt, is a perl script which reads files in the format
generated by cprncdf (one file for each line in the plot) and
then invokes a scientific data processing and plotting tool called
yorick to do the plotting. yorick must be
installed on your system for this script to be used.