Transcript The CCSM2.0 Quick Start Guide Lawrence Buja CCSM Software Engineering Group [email protected] June 29 2002 www.ccsm.ucar.edu.

The CCSM2.0 Quick Start
Guide
Lawrence Buja
CCSM Software Engineering Group
[email protected]
June 29 2002
www.ccsm.ucar.edu
CCSM2.0 Quick Start Guide
•
•
•
•
•
•
•
What is needed to run CCSM2?
Downloading the CCSM2.0 distribution
The CCSM source code
Input datasets
Building the CCSM
Running the CCSM
CCSM output data
www.ccsm.ucar.edu
What is needed to run CCSM2?
•
•
•
•
•
•
OS: IBM AIX or SGI IRIX64
Compilers: Fortran 90, C
Tools: gunzip, gnumake, Perl5, Perl/Tk
Permanent & Temporary disk space
Libraries:
MPI and netCDF
Input data : 0.2 GB for T31_gx3 grid,
0.8 GB for T42_gx1v3 grid
• CCSM2 source code: 35 MB
www.ccsm.ucar.edu
The CCSM2.0 distribution
CCSM2.0 is available via the web from:
http://www.cgd.ucar.edu/csm/models/ccsm2.0
ccsm2.0.tar.gz
ccsm2.0.inputdata.T42_gx1v3.tar
ccsm2.0.inputdata.T31_gx3.tar
ccsm2.0.inputdata.T62.tar
ccsm2.0.inputdata.CLM_SPINUP_FILES.tar
CCSM2.0 source codes
input data for T42_gx1v3 resolution
input data for T31_gx3 resolution
latm data for T62 resolution
are CLM datasets
gunzip -c ccsm2.0.tar.gz | tar -xf tar -xf ccsm2.0.inputdata.T42_gx1v3.tar
www.ccsm.ucar.edu
The CCSM distribution
ccsm2.0.tar.gz
ccsm2.0.inputdata.T42_gx1v3.tar
(35 Mbytes)
(.8 Gbytes )
Scripts & Model Code
Initial data
$HOME/
/fs/$LOGNAME/
ccsm2.0/
$CSMROOT
inputdata/
$CSMDATA
www.ccsm.ucar.edu
The CCSM source code: inputdata/
inputdata/
cpl/
cpl5/
atm/
ocn/
ice/
lnd/
cam2/
pop/
csim4/
clm2/
datm5/
docn5/
dice5/
dlnd5/
latm5/
www.ccsm.ucar.edu
The CCSM source code: Top Level
$HOME/
ccsm2.0/
($CSMROOT)
CCSM2.0 Root Directory
doc/
documentation
models/
model source code
www.ccsm.ucar.edu
scripts/
build/run Scripts
The CCSM source code: models/
$CSMROOT/models/
cpl/ atm/
cpl5/
ocn/
ice/
lnd/
cam2/
pop/
csim4/
clm2/
datm5/
docn5/
dice5/
dlnd5/
latm5/
www.ccsm.ucar.edu
bld/
shr/
The CCSM source code: scripts/
$CCSMROOT/scripts/
test.a1/
test.a1.run
cpl.setup.csh
atm.setup.csh
ocn.setup.csh
ice.setup.csh
lnd.setup.csh
datm.setup.csh
docn.setup.csh
dice.setup.csh
dlnd.setup.csh
ccsm_archive
test.a1.har
gui_run/
ccsm_gui.pl
tools/
ccsm_checkenvs
ccsm_cpdata
ccsm_getfile
ccsm_getinput
ccsm_getrestart
ccsm_msmkdir
ccsm_msread
ccsm_mswrite
ccsm_splitdf
modules.AIX.seaborg
modules.IRIX64.nirvana
modules.IRIX64.ute
test.a1.mods.*
www.ccsm.ucar.edu
Building the CCSM
Three levels of c-shell scripts are used:
– Run script builds & runs CCSM (i.e. test.a1.run)
– Define the CCSM environment
– Run each component setup script
– Execute all components simultaneously
– Setup scripts builds each model (i.e. ice.setup.csh)
– Position any input data needed by that component
– Generate the component's namelist input file
– Build the component executable
– Utility tools (i.e. ccsm_getinput)
www.ccsm.ucar.edu
Building the CCSM
A sample run script (test.a1.run) is distributed.
Functional summary of test.a1.run
setenv A abc
setenv B xyz
foreach i (cpl atm ocn ice lnd)
$i.setup.csh
end
mpirun cpl atm ocn ice lnd
exit
www.ccsm.ucar.edu
Define
environment
Loop over
setup scripts
Run CCSM
Building the CCSM: $CASE
$CASE identifies a CCSM run.
The sample $CASE is test.a1,
$CASE for the CCSM control run is b20.007
To define a new $CASE
– Copy all of test.a1/ into $CASE/
– rename test.a1.run to $CASE.run
– Modify the main run script
– CASE CASESTR CSMROOT CSMDATA EXEROOT ARCROOT
– $HOME/ccsm2.0/scripts/$CASE/$CASE.run submitted to the batch
queue.
www.ccsm.ucar.edu
Building the CCSM
Scripts & Model Code
Initial data
$HOME/ccsm2.0/
$CSMROOT
doc/
models/
/fs/$LOGANME/inputdata/
$CSMDATA
scripts/
Large model build/execute directories
/ptmp/$LOGNAME/$CASE
$EXEROOT
www.ccsm.ucar.edu
Methods for starting the CCSM
New CCSM case is started as either a startup,
hybrid or branch run, depending on the
science requirements.
startup:
hybrid:
branch:
continue:
new run from arbitrary input files
new run from initial or restart files
new run from restart files
extend an existing run exactly
www.ccsm.ucar.edu
General concepts: Exact Restarts
• Wall-clock limits in the batch queues typically
restrict runs to a finite length, usually 1-3 model
years.
• At specified intervals, restart and initial files will be
created by all components
• Runs are extended as "continue" runs.
• A continued run must give exactly the same results
as if the run had never stopped.
www.ccsm.ucar.edu
Exact Restarts
Are your results reproducible?
Factors:
•Any change in machine architecture
•Any change in math libraries
•Running on different number of processors
•Changes you've made to the code
•Additional variables added to model not
properly written to restart files
www.ccsm.ucar.edu
Changing the code
Frozen Code
Keep the CCSM2.0 model code "frozen"
To introduce code changes:
• copy specific model files to new directory
• modify the new copies of model code
• tell the model setup scripts to get the modified
code from the new directory first, then get rest of
unchanged model code from the "frozen" version
This method encapsulates modified code
Use same "frozen code" for different work
www.ccsm.ucar.edu
Changing the code
Three levels of change:
1. "Bit-for-bit" (no change to answers)
• adding/subtracting output
2. Machine-level (answers similar at the bit level)
• Changes in operation order
• Changes in math libraries
• changes in compiler options/flags/versions
3. Algorithm changes (answers change)
• Different parameterization
• Rerun control integration
www.ccsm.ucar.edu
CCSM Components
atm
lnd
Coupler
ocn
www.ccsm.ucar.edu
ice
CCSM2: Performance
NERSC T42/gx1v3 (128Pes)
OCN
41secs
48
ICE
32
LND
8
ATM
32
CPL
8
3
7
2
14
7
4
Critical path
8
8
37
3
7
53 secs/day total
(~130 years/month)
www.ccsm.ucar.edu
Output log files
• The printed output from each CCSM
component is saved in a "log file" A
coordinated timestamp links filenames
$LID is of the form YYMMDD-hhmmss, where
YYMMDD are the Year, Month, Day and hhmmss
are hour, minute and second that the run began
(i.e. $EXEROOT/ocn/ocn.log.000626-082714) .
www.ccsm.ucar.edu
History and restart output data
• Default: netCDF monthly averaged history
files for all components in netCDF format.
• CCSM2.0 also writes out binary restart files
from all components at regular intervals.
• The total output volume of the model output
can vary greatly depending upon the output
frequencies and options selected.
www.ccsm.ucar.edu
CCSM Yearly Output (b20.014)
cpl
atm
ocn
ice
lnd
1.4 Gbytes
0.7 Gbytes
4.1 Gbytes
.6 Gbytes
.4 Gbytes
disk
disk
disk
7.4 Gbytes/year
www.ccsm.ucar.edu
Tape
Tape
Archive
Archive
Device
Devices
Getting help
• http://www.ccsm.ucar.edu forum
• [email protected]
www.ccsm.ucar.edu