Transcript CSEG Update Mariana Vertenstein CCSM Software Engineering Group Improvements to CCSM scripts  Addition of timing tool that provides automated information to help determine.

CSEG Update
Mariana Vertenstein
CCSM Software Engineering Group
Improvements to CCSM scripts
 Addition of timing tool that provides automated
information to help determine load balance,
throughput and cost of a run.
 Improvements made to CCSM script infrastructure
 Provide the ability to define new “component sets/modes”.
and as command line options. Makes it easier to run and test
new CCSM science (such as addition of component
biogeochemistry and atmospheric chemistry).
 Simplified user specification of non-default task/thread
settings.
 Changes improve extensibility and robustness of scripts.
2
Improvements to CCSM scripts (cont)
 CCSM test framework rewritten using new functionality.
 Introduced new testing functionality (e.g. auto-promotion test).
 Introduced new “test suites” that now permit different testing
levels:
 development, pre-tag, post-tag, monthly
 currently in process of defining contributions to each test
suite
 Benefits:
 Regression testing will be easier to perform.
 Straightforward addition of tests for new CCSM scenarios.
 Easier testing of new scenarios (e.g. CLM-CN with CO2
exchange).
 More frequent testing of new science.
 More robust CCSM code base.
3
Porting and Performance
 ORNL Cray X1 (phoenix)
 Both release and development code base have been ported. CAM
FV dycore ported in development code base. Porting this dycore in
CCSM proved to be very challenging.
 30 years/day on 200 procs (ignorning performance fluctuations)
 13 years/day on 108 procs on NCAR bluevista
 400 year FV control simulation carried out
 ORNL Cray XT3 (jaguar)
 Port of development code base is starting now. Release code base
is running - will require a validation.
 NCAR IBM (bluevista)
 Major new OS upgrade will permit over-subscription of nodes.
Provides an opportunity for major performance improvements - but
will also result in significantly larger parameter space for
performance optimization.
4
Porting and Performance (future)
NCAR IBM Bluegene
Will require single executable concurrent
system
Will require parallel I/O (NetCDF and
binary)
5
Data Model Project
Serial version rewrite of all data models
(datm7, dlnd7, dice7, docn7) completed!
datm7 can now duplicate stand-alone CLM
functionality (in serial mode).
docn7 now has both DOM and SOM functionality
(SOM scientific verification is close to complete).
All data components now have same
functionality.
Can perform spatial interpolation from input data
resolution to model resolution.
Can data cycle over subset of years.
6
Data Model Project
(what is next)
Parallelize the new serial data models.
Incorporate parallel data models into currently
developed sequential CCSM.
Replace component specific data models with
CCSM data models. As examples:
Replace SOM/DOM in CAM with DOCN7
Replace forcing atm driver in CLM with DATM7
Use of only one set of data models will result in
more consistent science and remove existing code
duplication.
7
Single Executable Concurrent CCSM
 CSEG is leveraging the work that Helen He and Chris
Ding (SciDAC) have done in order to create a single
executable concurrent implementation of CCSM
development code.
 A somewhat different implementation will be
produced to satisfy current CCSM requirements.
 Aim to have development CCSM tag (including all
CCSM components) by mid-July.
 Aim to have a CCSM3.0 release based version
released by later in the summer.
 Existence of single executable system should
improve CCSM portability and debugging.
8
Single Executable Sequential CCSM
 Major progress made in replacing stand-alone CAM
with “pseudo-sequential CCSM”.
 Both ESMF and MCT will be examined as coupling
frameworks. We are committed to creating a
sequential interoperable CCSM utilizing the ESMF
framework.
 ESMF superstructure coding for Stage 1 Evaluation
Plan is beginning now. Superstructure design is
already in place. Estimated completion date is end of
summer. Plan is to incorporate ESMF coupling
framework on CAM trunk upon successful completion
of Stage 1 evaluation effort.
9
Original stand-alone CAM
architecture (CAM3.0)
CAM Driver
Physics
CLM
CAM-ICE
Dynamics
CAM-OCN
Note that surface
models are invoked
from within CAM
physics - not from a
top level driver
10
Pseudo-Sequential CCSM Top level
Introduce top
Architecture
Introduce top level
framework dependent
(FD) application driver
to replace CAM driver
Application Driver
(FD)
level ESMF clock to
coordinate time
evolution of all
components
Couplers (FD)
ATM merger
(FD)
Thin coupling
layer (FD)
CAM
Phys Dyn
ICE merger
(FD)
OCN merger
(FD)
Thin coupling
layer (FD)
Thin coupling
layer (FD)
Thin coupling
layer (FD)
CLM
CAM-ICE
CAM-OCN
LND merger
(FD)
Introduce new flexible and
extensible “thin”
coupling layer design
Sequential CCSM (cont)
 New top level based application driver.
 Independent of CAM data structures
 Time evolution based upon ESMF general time management
utilities
 Utilizes CCSM share code for reading input
 Initial implementation: MCT
 Near term implementation: ESMF
 New surface coupling layer.
 Initial implementation: MCT
 Near term implementation: ESMF
 New directory structure for sequential CCSM.
 Applies to both MCT and ESMF
 Inter-component domain checking in coupling layer.
 Initial implementation: MCT
 Near term implementation: ESMF
12
CAM Update
 Implemented support for non lat-lon grids (Jim Edwards and Pat
Worley). Non lat-lon support added to
 Boundary data and aerosol data interpolation code.
 Physics/Dyamics coupling layer.
 Only aqua-planet mode is currently supported.
 Significant FV dycore interface refactoring implemented:
 Only data on the XY decomposition outside the portable
dycore is now exposed.
 Added dynamics import/export states, and dynamics
component module with init, run, final methods.
 CAM testing framework extended to add new platforms (e.g.
phoenix) and to perform overnight regression testing.
 New features added to CAM tropospheric MOZART.
13
CAM Update (what is next)
 Refactoring of CAM to run with new dycores.
Incorporation of HOMME dycore into CAM.
Refactoring CAM’s history module to encompass
non lat-lon output.
Ability to run pseudo-sequential CCSM (not just
aqua-planet) with non lat-lon CAM.
 Incorporation of parallel I/O into CAM.
Replace all binary I/O with NetCDF I/O.
 Creation of new tool to generate CAM namelists (tool
should easily extend to other models if desired).
14
CAM Update (what is next)
 SCAM (Single Column Atmosphere Model) refactoring
 Goal is to produce a more flexible and maintainable
interfaces. ,
 Adoption of new components
 Incorporation of new ice sheet model (GLIMMER).
Primary work will be done by Bill Lipscomb.
GLIMMER has already been incorporated into CAM build
system.
 Replacement of CAM-CSIM with CICE and CAM-SOM/DOM
with DOCN7.
15
CLM Update
 Finemesh grids have been implemented in CLM. This
gives CLM the new capability to run on its own
independent grid. The implementation follows the
scheme of Hahmann and Dickinson.
 CLM still couples to CAM or CCSM via the CAM coarse grid.
Mapping is done within the CLM code base.
 The implementation does not change answers when the
CLM fine grid is identical to the CAM grid.
 Stand-alone CAM runs have been run with a T42 coarse grid
and a half degree finemesh grid. Results are encouraging.
 New downscaling and upscaling interactions are being
implemented.
16
CLM Update (what is next)
 A nested grid capability is being added to CLM, so the model
can be run at spatially varied resolution to optimize cost.
 A new prognostic canopy air space scheme is being
implemented.
 Scientific formulation is being finalized.
 Software implementation (Forrest Hoffman) will start as soon as the
scientific formulation is completed.
 CLM offline testing framework will be rewritten in order to test
the numerous new features that are being introduced into the
system.
 Urban code will be incorporated into the main development line
of development.
17
POP2 Update
 Completed incorporation of major CCSM features
into the LANL pop2.1.alpha code. CCSM POP2 code
is now in CCSM SVN repository and current CCSM
scripts.
 Added
New tracer advection scheme (1D Lax-Wendroff
with 1D flux limiters).
Near-surface eddy flux, enhanced deep mixing.
Support for new 1-degree grid with Galapagos
Islands topography (gx21v4).
18
POP2 Update (what is next)
Inclusion of a more efficient barotropic solver
designed by John Dennis.
Reduced equatorial viscosity.
A new vertical grid.
Systematic exploration of the above features.
Plans for incorporating the ecosystem model
into POP2 will be developed during the
CCSM workshop.
19
CICE Update
 PCWG has decided to incorporate CICE as the new
standard CCSM ice model. New name will be
“Community Ice CodE”.
 Testing has been done using CICE 3.1 to establish
main differences with CSIM 5.0.
 CICE 3.1 changes answers significantly, but still
within the realm of the same climate.
 CICE 3.14 has additional bug fixes and a few new
physical parameterizations. This is the version that
will be adopt after more testing is completed.
 Goal is to incorporate CICE 4 (with new data
structures) into CCSM and into pseudo-sequential
CCSM.
20