The create_test tool is located in the scripts directory and can be used to setup a standalone test case. The test cases are fixed and defined within the CESM scripts. To see the list of test cases or for additional help, type "create_test -help" from the scripts directory. To use create_test, do something like
> cd $CCSMROOT/scripts > ./create_test -testname ERS.f19_g16.X.yellowstone -testid t01 > cd ERS.f19_g16.X.yellowstone.t01 > ERS.f19_g16.X.yellowstone.t01.build submit ERS.f19_g16.X.yellowstone.t01.test Check your test results. A successful test produces "PASS" as the first word in the file TestStatus |
The above sets up an exact restart test (ERS) at the 1.9x2.5_gx1v6 resolution using a dead model compset (X) for the machine yellowstone. The testid provides a unique tag for the test in case it needs to be rerun (i.e. using -testid t02). Some things to note about CESM tests
For more information about the create_test tool, run "create_test -help".
Test results are set in the TestStatus file. The TestStatus.out file provides additional details.
Tests are not always easily re-runable from an existing test directory. Rather than rerun a previous test case, it's best to setup a clean test case (i.e. with a new testid).
The costs of tests vary widely. Some are short and some are long.
If a test fails, see the Section called Debugging Tests That Fail.
There are -compare and -generate options for the create_test tool that support regression testing.
There are extra test options that can be added to the test such as _D, _E, or _P*. These are described in more detail in the create_test -help output.
The test status results have the following meaning
| Test Result | Description |
|---|---|
| BFAIL | compare test couldn't find base result |
| CHECK | manual review of data is required |
| ERROR | test checker failed, test may or may not have passed |
| FAIL | test failed |
| GEN | test has been generated |
| PASS | test passed |
| PEND | test has been submitted |
| RUN | test is currently running OR it hung, timed out, or died ungracefully |
| SFAIL | generation of test failed in scripts |
| TFAIL | test setup error |
| UNDEF | undefined result |
The following tests are available at the time of writing
| Test | Description |
|---|---|
| SMS | 5 day smoke test |
| ERS | 10 day exact restart from startup |
| ERP | 2 month exact restart from startup |
| ERB | branch/exact restart test |
| ERH | hybrid/exact restart test |
| ERI | hybrid/branch/exact restart test |
| ERT | 2 month exact restart from startup, history file test |
| ERU | 2 month exact restart from initial conditions, history file test |
| SEQ | sequencing bit-for-bit test |
| PEA | single processor testing |
| PEM | pe counts mpi bit-for-bit test |
| PET | pe counts mpi/openmp bit-for-bit test |
| CME | compare mct and esmf interfaces test |