6.2 Testing Climate and Performance of Modified Code

For purposes of adding new code to the existing model, a tagged version of CCSM should be used, the more recent the better. The latest version of CCSM will usually contain the latest version of CSIM. It is a good idea to check if this is the case. These are the steps for integrating newly developed code into the existing sea ice model and testing:

1. Make modifications to the source code using one of the following options:
   1. Using CVS:
      1. Create a development branch from a CSIM tag on the main trunk, ideally the CSIM tag used in the control run.
   2. Not using CVS:
      1. Copy modules to be modified from the source code directory to
         ccsm3/scripts/$CASE/SourceMods/src.csim, and make modifications in this directory.

2. Compile, run for 5 days with minimum debugging turned on: array bounds check, Nan initialization, floating point trap.

3. Check that code still meets any requirements that may have been affected by the new code. The minimum tests are exact restart and conservation. If developed code involves any namelist variables, all options for those variables must be tested. Any tests available in the new code should be exercised.

4. Test on all supported platforms. Compilation and a 5 day run are the minimum.

5. Do a 1 year run, check results against control run.

6. Continue run out to 10-15 years, saving output in a permanent disk location. This may be used as the next control run.

7. Document performance, compare with control run and put difference plots on web.

8. If results are satisfactory (approved by PCWG), code may be inspected and/or reviewed. Development branch will be merged with the main trunk and tagged.

9. Model documentation (User's Guide, Code Reference and Scientific Description) must be updated.

Currently, there is no formal CCSM Requirements document, and the CSIM Requirements document is still in the draft stage. There is a somewhat complete list of tests for CSIM, but an automated test suite is still under development. All developers are strongly encouraged to read the CSIM Requirements Document and all of the Developer's Guide (this document).

Minimum Requirements

It is difficult to define all tests that are suitable for all code changes, and a user may not have access to all supported platforms. This section gives a list of basic requirements for the ice model, to provide guidance for testing. A more complete list of requirements is in the CSIM Requirements document (under development). As noted in section 6.2, any changes in climate or model performance due to the code modifications must be documented.

Minimum CCSM Requirements

Compiles, runs, and restarts exactly with at least one configuration, preferably the M component set (latm, dlnd, docn, csim in mixed layer ocean mode, cpl) or the B component set (cam, clm, pop, csim, cpl). Runs on at least one standard CCSM grid (gx1v3 or gx3v5). Runs on all CCSM supported platforms available to the modeler.

Conservation

checking water, salt and energy budgets in the log file. Any conservation errors or CFL violations that occurred in the 10-15 year climate year run should be documented.

Exact Restart

This test is automated in the CSIM and CCSM scripts. It does a 10 day startup run and a 5 day continuation run that begins at day 5 of the startup run. Output from days 5-10 of both runs are compared and must match bit-for-bit.

Debug Test

This test is also automated in the CSIM and CCSM scripts.

Input Data Files

If any new or modified input data files are necessary to run the modified code, they must be provided with the code.