Instructions to run CESM1.5: "36"

 



CESM1.5 (modern version: March 2016)

Control simulation "36".
"36" is basically the modern version of "28". It uses updated code and it produces a climate similar to "28".

The experimental only works for B1850. Other compsets WILL NOT produce a cesm1.5 simulation.
Below, we provide instructions to make a F2000 and FAMIP.

Instructions to run "36"

 


1. Pre-industrial control (B1850)

Experimental tag

On the svn repository
https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/cesm1_5_alpha06_cntlexp_tags/cesm1_5_alpha06_cntlexp07

There is a local copy of the tag at:

/glade/p/work/hannay/cesm_tags/cesm1_5_alpha06_cntlexp07/

Compset

In this tag, the B1850 compset uses:
- ATM: cam5.5 physics
- SEA ICE: prognostic cice5
- LAND ICE: cism1 land ice
- LAND: clm5.0 physics + clm5.0 bgc (cn and methane)
- RIVER MODEL: MOSART
- OCN: POP2 default+ BGC (CO2=diag)

create_newcase command

The create_newcase command is:
./create_newcase -case $yourcasename -compset B1850 -res f09_g16 -mach yellowstone -user_mods_dir b1850 -pecount X -project $project_number

pe_layout

For best performance, it is recommended to turn off hyperthreading.

./xmlchange MAX_TASKS_PER_NODE=16

Overall Metrics for this layout:
Model Cost: 4487.02 pe-hrs/simulated_year
Model Throughput: 20.20 simulated_years/day

problem with short-term archive

The short term archiver doesn't work properly in the experimental tag when doing an hybrid run.
A fix is provided in the section known problems.



2. CAM standalone with 2000 SSTs (F2000)

Experimental tag

On the svn repository
https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/cesm1_5_alpha06_cntlexp_tags/cesm1_5_alpha06_cntlexp06
By default, this tag will use cice4 and not cice5 for the prescribed sea-ice (this is hard-wired in the cice configure)

# make modifications ro run cice5 in standalone
As we want to use cice5 the case in standalone runs, we need to comment the lines in: components/cice/bld/configure

if ($cice_mode eq 'prescribed') {
$phys = 'cice4';
$cfg_ref->set('phys', $phys);
}

# define a new comspet F2000
# in /glade/p/work/hannay/cesm_tags/cesm1_5_alpha06_cntlexp06_cice5/components/cam/cime_config/config_compsets.xml

<compset>
<alias>F2000</alias>
<lname>2000_CAM55_CLM50%BGC_CICE%PRES_DOCN%DOM_MOSART_CISM1%NOEVOLVE_SWAV</lname>
</compset>

# local copy

There is a local copy of the tag that includes the modifications to the sea-ice code and to config_compsets.

/glade/p/work/hannay/cesm_tags/cesm1_5_alpha06_cntlexp06_cice5

 

Compset and create_newcase command

In this tag, the FC55 compset uses:
- ATM: cam5.5 physics
- SEA ICE: prescribed cice5
- LAND ICE: cism1 land ice
- LAND: clm5.0 physics + clm5.0 bgc (cn and methane)
- RIVER MODEL: MOSART
- OCN: SST

An example of the create_newcase command is:
./create_newcase -case $casename -compset F2000 -res f09_f09 -mach yellowstone -user_pes_setby cam -project CESM0005

pe_layout

Here I am suggesting 2 pe_layouts (the first one is cheaper and shoudl be used if you don't need an ultra fast turnaround)

(1) cheaper pe_layout

./xmlchange NTASKS_ATM=1728,NTASKS_CPL=1728,NTASKS_ICE=1728,NTASKS_LND=1728,NTASKS_OCN=1728,NTASKS_ROF=1728,NTASKS_WAV=1728
./xmlchange PES_PER_NODE=16,MAX_TASKS_PER_NODE=16

Overall Metrics for this layout
Model Cost: 2924.23 pe-hrs/simulated_year
Model Throughput: 14.18 simulated_years/day

(2) faster pe_layout

./xmlchange NTASKS_ATM=1728,NTASKS_CPL=1728,NTASKS_ICE=1728,NTASKS_LND=1728,NTASKS_OCN=1728,NTASKS_ROF=1728,NTASKS_WAV=1728
./xmlchange NTHRDS_ATM=2,NTHRDS_CPL=2,NTHRDS_GLC=1,NTHRDS_ICE=2,NTHRDS_LND=2,NTHRDS_OCN=2,NTHRDS_ROF=2,NTHRDS_WAV=2
./xmlchange PES_PER_NODE=16,MAX_TASKS_PER_NODE=16

Overall Metrics for this layout
Model Cost: 3970.64 pe-hrs/simulated_year
Model Throughput: 20.89 simulated_years/day

 

xml changes

./xmlchange RUN_STARTDATE=0001-01-01
./xmlchange RUN_TYPE=hybrid,RUN_REFCASE=b.e15.BHIST.f09_g16.20thC.28,RUN_REFDATE=2000-01-01,GET_REFCASE=TRUE

Change to user_nl_** files

# user_nl_clm
# Ideally we need to use a 2000 surface dataset but it is not available, so we are using the 1850 for now. We will update this when the dataset becomes available.

fsurdat = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/surfdata_0.9x1.25_16pftsmodwshrubwglcr_simyr1850_c151210.nc'
use_init_interp = .true.
init_interp_fill_missing_with_natveg = .true.

# user_nl_cpl

flux_diurnal = .true.

# user_nl_cam

solar_data_file = '/glade/p/cesmdata/cseg/inputdata/atm/cam/solar/SOLAR_SPECTRAL_Lean_1610-2008_annual_c090324.nc'
solar_data_type = 'FIXED'
solar_data_ymd = 20000101



3. CAM standalone AMIP type run (FAMIP)

Experimental tag

On the svn repository
https://svn-ccsm-models.cgd.ucar.edu/cesm1/exp/branch_tags/cesm1_5_alpha06_cntlexp_tags/cesm1_5_alpha06_cntlexp06
By default, this tag will use cice4 and not cice5 for the prescribed sea-ice (this is hard-wired in the cice configure)

# make modifications ro run cice5 in standalone
As we want to use cice5 the case in standalone runs, we need to comment the lines in: components/cice/bld/configure

if ($cice_mode eq 'prescribed') {
$phys = 'cice4';
$cfg_ref->set('phys', $phys);
}

# define a new comspet FAMIP
# in /glade/p/work/hannay/cesm_tags/cesm1_5_alpha06_cntlexp06_cice5/components/cam/cime_config/config_compsets.xml

<compset>
<alias>FAMIP</alias>
<lname>AMIP_CAM55_CLM50%BGC_CICE%PRES_DOCN%DOM_MOSART_CISM1%NOEVOLVE_SWAV</lname>
</compset>

# local copy

There is a local copy of the tag that includes the modifications to the sea-ice code and to config_compsets.

/glade/p/work/hannay/cesm_tags/cesm1_5_alpha06_cntlexp06_cice5

 

Compset and create_newcase command

In this tag, the FC55 compset uses:
- ATM: cam5.5 physics
- SEA ICE: prescribed cice5
- LAND ICE: cism1 land ice
- LAND: clm5.0 physics + clm5.0 bgc (cn and methane)
- RIVER MODEL: MOSART
- OCN: SST

An example of the create_newcase command is:
./create_newcase -case $casename -compset FAMIP -res f09_f09 -mach yellowstone -user_pes_setby cam -project CESM0005

pe_layout

Here I am suggesting 2 pe_layouts (the first one is cheaper and shoudl be used if you don't need an ultra fast turnaround)

(1) cheaper pe_layout

./xmlchange NTASKS_ATM=1728,NTASKS_CPL=1728,NTASKS_ICE=1728,NTASKS_LND=1728,NTASKS_OCN=1728,NTASKS_ROF=1728,NTASKS_WAV=1728
./xmlchange PES_PER_NODE=16,MAX_TASKS_PER_NODE=16

Overall Metrics for this layout
Model Cost: 2924.23 pe-hrs/simulated_year
Model Throughput: 14.18 simulated_years/day

(2) faster pe_layout

./xmlchange NTASKS_ATM=1728,NTASKS_CPL=1728,NTASKS_ICE=1728,NTASKS_LND=1728,NTASKS_OCN=1728,NTASKS_ROF=1728,NTASKS_WAV=1728
./xmlchange NTHRDS_ATM=2,NTHRDS_CPL=2,NTHRDS_GLC=1,NTHRDS_ICE=2,NTHRDS_LND=2,NTHRDS_OCN=2,NTHRDS_ROF=2,NTHRDS_WAV=2
./xmlchange PES_PER_NODE=16,MAX_TASKS_PER_NODE=16

Overall Metrics for this layout
Model Cost: 3970.64 pe-hrs/simulated_year
Model Throughput: 20.89 simulated_years/day

xml changes

./xmlchange RUN_TYPE=hybrid,RUN_REFCASE=b.e15.BHIST.f09_g16.20thC.28,RUN_REFDATE=1979-01-01,GET_REFCASE=TRUE
./xmlchange RUN_STARTDATE=1979-01-01

Change to user_nl_** files

# user_nl_clm
# Ideally we need to use a 2000 surface dataset but it is not available, so we are using the 1850 for now. We will update this when the dataset becomes available.

use_init_interp = .true.
init_interp_fill_missing_with_natveg = .true.
flanduse_timeseries = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/landuse.timeseries_0.9x1.25_rcp8.516pftsmodwshrub_simyr1850-2100_c160112.nc'
fsurdat = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/surfdata_0.9x1.25_16pftsmodwshrubwglcr_simyr1850_c151210.nc'

# user_nl_cpl

flux_diurnal = .true.


4. Known problems

Problem with short-term archiving for hybrid runs (2016-04-01)

The short term archiver doesn't work properly in the experimental tag when doing an hybrid run.
A fix is provided below.

In your sandbox, you need to replace the file:
cesm1_5_alpha06_cntlexp07/cime/scripts/Tools/st_archive
by the provided file st_archive.

You can also find a fix locally on yellowstone:
/glade/p/work/hannay/cesm_tags/cesm1_5_alpha06_cntlexp07/cime/scripts/Tools/st_archive