Transcript SCEC Information Technology Overview for 2012 Philip J. Maechling Information Technology Architect Southern California Earthquake Center SCEC Board of Directors Meeting 30 Jan 2012

SCEC Information Technology
Overview for 2012
Philip J. Maechling
Information Technology Architect
Southern California Earthquake Center
SCEC Board of Directors Meeting
30 Jan 2012
SCEC3 Information Technology (IT) Work Areas
1.
Administrative IT including membership, awards, contracts
o John McRaney, Tran Huynh, John Yu, Karen Young and others
2. Collaboration IT supports collaborative research projects,
communications, meetings, creation and distribution of research
results.
o John Marquis, Tran Huynh, John Yu, John McRaney and others
3. Outreach IT prepares, presents, and preserves research results to
collaborative partners, funding agencies, and public
o Thomas Jordan, Mark Benthien, Philip Maechling, Tran Huynh, and
others
4. Computational Science IT provides access to computer hardware,
scientific software, and data management.
o Philip Maechling, Yifeng Cui, Ewa Deelman, Scott Callaghan, Maria
Liukis, Patrick Small, Kevin Milner, John Yu and others
SCEC Scientific Software Distributions
1.
2.
3.
4.
5.
6.
CSEP Software Framework
– Last release Jan 2012
CVM-S Community Velocity Model
– Last release Feb 2011
CVM-H Community Velocity Model
– Last release Oct 2011
UCVM Software Framework
– Last release Jan 2012
Broadband Platform
– Last release Oct 2011
SCEC-VDO visualization of 3D geo-referenced data
– Available on request
SCEC Scientific Software In Active Use
1.
AWP-ODC – Finite difference dynamic ruptures and anelastic wave propagation
–
Simulations Jan 2012
2. Hercules – Finite element anelastic wave propagation
– Simulations Jan 2012
3.
R. Graves Finite Difference Wave Propagation
– Simulations in Jan 2012
4.
OpenSHA probabilistic seismic hazard analysis
– Calculations in Jan 2012
5.
Tera3D full 3d tomography processing system
– California regional inversion iteration in Jan 2012
6.
CyberShake physics-based PSHA hazard curve calculation
– Research hazard curve in Jan 2012
7.
CSEP Testing Framework
– Daily use in CSEP, EEW, and TD Testing
8.
Broadband Platform
– Use on SCEC servers by PEER and NGA-E modelers in Jan 2012
9.
SCEC-VDO
– Used by UCERF Scientific Reviewers in Jan 2012
Unified Community Velocity Model
Development
UCVM development included (a) CVM evaluation tools, (b) CVM integration
framework, and (c) combining best of existing models into consistent
state-wide CVM for use with state-wide 1Hz 3D wave propagation
simulations. Standardizes query API, elevation DEM, bathymetry, Moho,
GTL, and distribution format
Inversion-based Update to CVM-S4
Performed two seismic inversion iterations using different data sets
1. CVM4SI1: improved CVM4 using earthquake data.
2. CVM4SI2: improved CVM4SI1 using ambient-noise Green’s function data.
Perturbation obtained in 2nd iteration enhances the perturbation obtained in the 1st
iteration. Waveform improvements for both earthquake recordings and ambient
noise Green’s functions
SCEC CyberShake system produced hazard curves with alternative
(improved) rupture generator and hazard curves based on CVM-H 11.2.
Used SCEC allocations on NSF supercomputers to calculate CyberShake
PSHA hazard curves for sites of interest (San Onofre and Diablo Canyon)
SCEC Broadband Platform 11.2.2 Released
SCEC released Broadband Platform 11.2.2 which can calculate
10Hz seismograms using rupture generators, 1D wave
propagations codes, and stochastic high frequency software
from multiple SCEC groups.
Broadband CyberShake
• Added R. Graves high-frequency stochastic modules from the
BBP into CyberShake Seismogram calculation
– Stochastic high-frequency
– Non-linear site response
– Filtered and combined with low-frequency (<0.5 Hz)
– Validate with SCSN, PBR sites
Perris
(PBR)
1 sec
0.1 sec
Current Broadband CyberShake Run
• 92 hazard curves
–
–
–
–
CVM-H 11.2 and CVM-S4, Graves and Pitarka (2010)
PBR, SCSN sites
64 SGTs, 92 post-processing
1.5 million SUs
• Data requirements
(broadband, 2component)
– SGTs: 2.5 TB
– Seismograms: 3.9 TB
PBR=red, SCSN=orange, near SCSN=yellow
SCEC Computational Science
Software Projects
The Quake Group at Carnegie Mellon
Current Efforts and Future Plans
The Effect of Minimum S-wave Velocity
Vsmin 200 m/s
Vsmin 500 m/s
Anelastic Wave Propagation (AWP)
Various applications
Hercules
Source model: Graves & Pitarka
Quadratic Elements
No Intrinsic Attenuation
fmax
Vsmin
Domain
Elements
Cores
Walltime
SUs
Short-Period Chino Hills Simulation
2 Hz
200 m/s
85 x 85 x 42.5 km3
226,670,602
18,000
4 hrs
72,000
Source model: Chen Ji
fmax
Vsmin
Domain
Elements
Cores
Walltime
SUs
4 Hz
200 m/s
180 x 135 x 62 km
5 billion
24,000
31 hrs
744,000
Blue – simulation
Red – data
1.5 trillion grid points
for a regular grid; 20
times the # of SUs
2011 SCEC CME Meeting
Jacobo Bielak, Ricardo Taborda, Haydar Karaoglu, Yigit Isbiliroglu
SCEC Computational Science
Forecast Testing Centers
1. Earthquake forecasts (CSEP)
– Active SCEC CSEP Operational Testing Center
2. Transient Detection (TD)
– Active Operational TD Testing Center
3. EEW CISN Testing Center
– Active Operational CTC Testing Center
CSEP Development and
SCEC CSEP Testing Center
CISN EEW Testing Center
SCEC Transient Detection Testing Center
SCEC Computational Science
Organizational Interactions
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
National Science Foundation (NSF)
United States Geological Survey (USGS)
W. M. Keck Foundation
Department of Homeland Security (DHS)
Pacific Gas and Electric (PG and E)
USC High Performance Computing and Communications (HPCC)
Argonne Leadership Computing Facility (ALCF)
Oak Ridge National Laboratory (ORNL)
National Institute for Computational Science (NICS)
Blue Waters NSF Track 1 Computing Facility (Univ. of Illinois)
Extreme Science and Engineering Discovery Environment (XSEDE)
San Diego Supercomputer Center (SDSC)
SCEC Computational Science
Organizational Interactions
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
Texas Advanced Computing Center (TACC)
Pittsburgh Supercomputing Center (PSC)
Open Science Grid (OSG)
National Center for Atmospheric Research (NCAR)
NCAR-Wyoming Supercomputing Center (NWSC)
USC Information Sciences Institute (ISI)
Data Intensive Cyber Environments (DICE) Center at the University of
North Carolina at Chapel Hill
Incorporated Research Institutions for Seismology (IRIS) Data
Management Center
California Integrated Seismic Network (CISN)
Computational Infrastructure in Geodynamics (CIG)
Global Earthquake Model (GEM)
SCEC Computational Science
In DOE Publication
SCEC CME researchers have access to three leadership class
Supercomputers (NSF Blue Waters (2012), DOE Titan (2012),
DOE Mira (2013)) to use for SCEC research calculations.
UCVM
Unified California Velocity Model
UCVM – Unified California Velocity
Model, a statewide 3-D velocity model
assembled from existing and improved
California CVM’s (Small, SCEC USR)
UCERF 3.0 Extended Earthquake
Rupture Forecast
(June 2012)
UCERF3.0 Extended Earthquake
Rupture Forecast contains slip-time
histories and rupture variations for all
UCERF3.0 ruptures.
CyberShake 1.0
Los Angeles Region Wave Propagationbased PSHA Map up to 0.5Hz (Fall 2009)
will increase in range and frequencies.
CyberShake 3.0
California Wave Propagation-based
PSHA Map up to 10Hz (Fall 2012)
CME CyberShake 3.0 calculation will build on UCERF3.0 in
order to calculate a “physics-based” California PSHA Map
CyberShake 3.0 Computational Estimates
CyberShake 3.0 Hazard Map Calculation Estimates:
Number of Sites on Map: 4240 (10km spacing)
Number of Jobs: 3.6 billion
Est. CPU-hours: 376.2 million
Data products (seismograms, spectral acceleration): 76.5 TB
Runtime on half-Jaguar: 3096 hrs (129 days)
Runtime on half-Blue Waters: 770 hrs (32.1 days)
Database entries: 6.95 billion
2011 Funded Projects (estimates per year)
1.
2.
3.
4.
5.
6.
7.
CSEP Software Framework Development and Operations (200k)
OpenSHA and SCEC-VDO in support of UCERF 3.0 (100k)
NSF PetaSHA-3 (850k)
NSF PRAC Blue Waters (5k)
USGS CISN EEW Project (50k)
Broadband Platform from PG and E (150k)
Transient Detection Testing (60k)
2012 Funded Projects (estimates per year)
1.
2.
3.
4.
5.
6.
7.
CSEP Software Framework Development and Operations (200k)
OpenSHA and SCEC-VDO in support of UCERF 3.0 (80k)
NSF SI2 SEISM Project (700k)
NSF PetaSHA-3 (425k)
NSF PRAC Blue Waters (5k)
USGS Supported CISN EEW Project (25k)
Broadband Platform from PG and E (150k)
NSF Software Infrastructure for Sustained Innovation Proposal
(Submitted July 2011)
SI2-SSI: A Sustainable Community Software Framework for Petascale Earthquake
Modeling
PI: T. H. Jordan (USC); Co-PIs: Jacobo Bielak (CMU), Y. Cui (SDSC), and K. Olsen (SDSU)
•
Year 1:
–
–
–
•
Year 2:
–
–
–
•
Computational milestone: simulation of Table 1 earthquake records at 0-4 Hz (deterministic) and 1-10 Hz
(stochastic) using the Broadband platform.
Validation effort: GMSV of these simulations against the earthquake records of Table 1 using linear and
nonlinear SDoF oscillators. UCVM comparison of tomographic results for Southern California.
Software releases: UCVM platform, Broadband platform; Kraken release of SEISM-IO.
Computational milestone: Time-independent CSHM-LA (UCEF2-based).
Validation efforts: GMSV for geotechnical systems; comparison of CSHM-LA with NGA hazard maps and
PBR data set. Full-3D evaluation of statewide UCVM.
Software releases: UCVM platform, Broadband platform; Blue Waters release of SEISM-IO.
Year 3:
–
–
–
Computational milestone: Time-dependent CSHM-CA (UCEF3-based).
Validation efforts: GMSV for nonlinear MDoF systems; comparison of CSHM-CA with NGA hazard maps
and PBR data set.
Software releases: UCVM platform, Broadband platform; Mira release of SEISM-IO.
NSF Geoinformatics Proposal
(Submitted Jan 13, 2012)
Geoinformatics: Community Computational Platforms for Developing ThreeDimensional Models of Earth Structure
PI: T. H. Jordan (USC); Co-PIs: P. Chen (U. Wyoming), Y. Cui (SDSC), and J. Tromp
(Princeton)
Objective: Establish an interoperable set of community computational platforms that will (a) implement
the techniques of full-3D waveform tomography, and (b) facilitate the development and delivery of
Earth models at a variety of scales. Two tomographic platforms will be built on highly scalable codes for
solving the forward problem of anelastic wave propagation (AWP):
•
AWP-ODC platform. AWP-ODC is 4th-order, staggered-grid, finite-difference code developed by K.
Olsen and his colleagues.
•
SPECFEM3D platform. SPECFEM3D is spectral element code, developed by D. Komatitsch and J.
Tromp.
•
UCVM platform developed by SCEC enables users to build meshes from models with topography,
hydrology, bathymetry, and standardized models of surficial (geotechnical) layers.
For More Information Please Visit:
http://www.scec.org/cme
End