The create_test tool is located in the scripts directory and can be used to set up a standalone test case. The test cases are fixed and defined within the CCSM scripts. To see the list of test cases or for additional help, type "create_test -help" from the scripts directory. To use create_test, do something like
> cd $CCSMROOT/scripts > ./create_test -testname ERS.f19_g16.X.bluefire_ibm -testid t01 > cd ERS.f19_g16.X.bluefire_ibm.t01 > ERS.f19_g16.X.bluefire_ibm.t01.test_build submit ERS.f19_g16.X.bluefire.t01.test Check your test results. A successful test produces "PASS" as the first word in the file TestStatus |
The above sets up an exact restart test (ERS) at the 1.9x2.5_gx1v6 resolution using a dead model compset (X) for the machine bluefire. The testid provides a unique tag for the test in case it needs to be rerun (i.e. using -testid t02). Some things to note about CCSM tests
For more information about the create_test tool, run "create_test -help".
Test results are set in the TestStatus file. The TestStatus.out file provides additional details.
Tests are not always easily re-runable from an existing test directory. Rather than rerun a previous test case, it's best to set up a clean test case (i.e. with a new testid).
Tests are built using the .test_build script. This is different from cases which are built using the .build script. Some tests require more than one executable, the .test_build script builds all required executables upfront interactively.
The costs of tests vary widely. Some are short and some are long.
If a test fails, see the Section called Debugging Tests That Fail.
There are -compare and -generate options for the create_test tool that support regression testing.
There are extra test options that can be added to the test such as _D, _E, or _P*. These are described in more detail in the create_test -help output.
The test status results have the following meaning
| Test Result | Description |
|---|---|
| BFAIL | compare test couldn't find base result |
| BUILD | build succeeded, test not submitted |
| CFAIL | env variable or build error |
| CHECK | manual review of data is required |
| ERROR | test checker failed, test may or may not have passed |
| FAIL | test failed |
| GEN | test has been generated |
| PASS | test passed |
| PEND | test has been submitted |
| RUN | test is currently running OR it hung, timed out, or exitied ungracefully |
| SFAIL | generation of test failed in scripts |
| TFAIL | test setup error |
| UNDEF | undefined result |
The following tests are available at the time of writing
| Test | Description |
|---|---|
| SMS | smoke test |
| ERS | exact restart from startup, default 6 days + 5 days |
| ERB | branch/exact restart test |
| ERH | hybrid/exact restart test |
| ERI | hybrid/branch/exact restart test |
| ERT | exact restart from startup, default 2 months + 1 month |
| SEQ | sequencing bit-for-bit test |
| PEA | single processor testing with mpi and mpi-serial |
| PEM | pe counts mpi bit-for-bit test |
| PET | pe counts mpi/openmp bit-for-bit test |
| CME | compare mct and esmf interfaces test |
| NCK | single vs multi instance validation test |