Using create_test

The new $create_test tool is located in the $CCSMROOT/scripts/ directory and can be used to set up entire CESM test suites, as well as single standalone test cases. To see the list of test cases, and to view the available script options, execute create_test -help or create_test without any arguments. Creating a single test looks like:

> cd $CCSMROOT/scripts
> ./create_test -testname ERS.f19_g16.X.yellowstone_intel -testid t01
> cd ERS.f19_g16.X.yellowstone_intel.t01
> ERS.f19_g16.X.yellowstone_intel.t01.test_build
Check your test results. A successful test produces "PASS" as
  the first word in the file TestStatus

The above commands set up an exact restart test (ERS) at the 1.9x2.5_gx1v6 resolution using a dead model component set (X) for the machine yellowstone. The testid provides a unique tag for the test in case it needs to be rerun (i.e. using -testid t02).

As an example, to create an entire suite of tests on yellowstone for the 'prebeta' test category, do the following:

> cd $CCSMROOT/scripts
> ./create_test \
     -xml_mach yellowstone \
     -xml_compiler intel \
     -xml_category prebeta \
     -testid alpha01a \
     -testroot /glade/scratch/$USER/alpha01a 

The above command will run all of the 'prebeta' tests in the ccsm_utils/Testlistxml/testlist.xml file that are configured for the category prebeta using the Intel compiler on yellowstone. This is almost identical to the development testing that CSEG carries out on various machines. In addition to creating the suite of tests in your chosen test root, $create_test will also copy several helper scripts into the testroot directory. The first will be named cs.status.$testid.$machine. When run, this script will output the current status of all the tests. The second will be named cs.submit.$testid.$machine. This script is automatically run when all the test cases are created, and it builds and submits the suite of tests for you.

Some things to note about CESM tests:

The test status results have the following meaning

Test ResultDescription
BFAIL compare test couldn't find the baseline directory for the testcase
BUILD build succeeded, test not submitted
CFAIL env variable or build error
CHECK manual review of data is required
ERROR test checker failed, test may or may not have passed
FAIL test failed
GEN test has been generated
PASS test passed
PEND test has been submitted
RUN test is currently running OR it hung, timed out, exceeded its allotted walltime, or exited abnormally
SFAIL generation of test failed in scripts
TFAIL test setup error
UNDEF undefined result

The following tests are available at the time of writing:

SMS smoke test
ERS exact restart from startup, default 6 days initial + 5 days restart
ERB branch/exact restart test
ERH hybrid/exact restart test
ERI hybrid/branch/exact restart test
ERT exact restart from startup, default 2 months + 1 month
SEQ sequencing bit-for-bit test
PEA single processor testing with mpi and mpi-serial
PEM pe counts mpi bit-for-bit test
PET pe counts mpi/openmp bit-for-bit test
CME compare mct and esmf interfaces test
NCK single vs multi instance validation test
SBN smoke build namelist test