Transcript Advancing Scientific Discovery through TeraGrid Scott Lathrop TeraGrid Director of Education, Outreach and Training University of Chicago and Argonne National Laboratory [email protected] www.teragrid.org August 2007

Advancing Scientific Discovery
through TeraGrid
Scott Lathrop
TeraGrid Director of Education, Outreach and Training
University of Chicago and Argonne National Laboratory
[email protected]
www.teragrid.org
August 2007
11 Resource Providers, One Facility
UW
Grid Infrastructure
Group (UChicago)
PSC
UC/ANL
NCAR
PU
NCSA
Caltech
IU
UNC/RENCI
ORNL
U Tenn.
USC/ISI
SDSC
LSU
TACC
Resource Provider (RP)
Software Integration Partner
August 2007
TeraGrid Objectives
• DEEP Science: Enabling Petascale Science
–Make Science More Productive through an integrated set
of very-high capability resources
•Address key challenges prioritized by users
• WIDE Impact: Empowering Communities
–Bring TeraGrid capabilities to the broad science
community
•Partner with science community leaders - “Science Gateways”
• OPEN Infrastructure, OPEN Partnership
–Provide a coordinated, general purpose, reliable set of
services and resources
•Partner with campuses and facilities
August 2007
TeraGrid Resources
• Computing - over 250 Tflops today and growing
– 400 Tflop system comes on-line in January at TACC
– U Tennesee system to come on-line in 2008
– Centralized help desk for all resource providers
• Visualization - Remote visualization servers and software
• Data
– Allocation of data storage facilities
– Over 100 Scientific Data Collections
• Access
– Shibboleth testbed to facilitate campus access
– Central allocations mechanism
• Human Support
– Advanced Support for TeraGrid Applications (ASTA)
– Education and training events and resources
– Over 20 Science Gateways
August 2007
Advanced Support for TeraGrid Applications
Virtualized
Resources,
Ensembles:
FOAM
Climate
Model
Liu (UWisc)
Coupled Simulation: Full Body Arterial Tree Simulation
Karniadakis (Brown)
Sources: Ian Foster (UC/ANL), Mike Papka (UC/ANL), George Karniadakis (Brown). Images by UC/ANL.
August 2007
Requesting Allocations of Time
• TeraGrid resources are provided for free to
academic researchers and educators
• Development Allocations Committee (DAC) for
start-up accounts up to 30,000 hours of time are
requests processed in two weeks - start-up and
courses
• Medium Resource Allocations Committee (MRAC)
for requests of up to 500,000 hours of time are
reviewed four times a year
• Large Resource Allocations Committee (LRAC) for
requests of over 500,000 hours of time are
reviewed twice a year
August 2007
TeraGrid Usage
275
Specific
Specific
Allocations
250
Roaming
Roaming Allocations
33% Annual Growth
225
200
200
Normalized
Units
(millions)
NUs (millions)
175
150
125
100
100
75
50
J
F
M
A
M
J
J
A
S
O
N
D
J
F
M
A
M
J
J
A
S
O
N
D
J
F
M
A
M
J
J
A
S
O
N
D
J
F
M
A
M
J
25
2004
TeraGrid currently delivers an
average of 420,000 cpu-hours per
day -> ~21,000 CPUs DC
2005
2006
2007
Dave Hart ([email protected])
August 2007
TeraGrid Usage Modes in CY2006
(est. number of
people/projects)
Batch Computing on Individual Resources
850
Exploratory and Application Porting
650
Workflow, Ensemble, and Parameter Sweep
160
Science Gateway Access
100
Remote Interactive Steering and Visualization
35
Tightly-Coupled Distributed Computation
10
August 2007
Grid-y Users
Use Modality
Community Size
Science Gateways
Broadening Participation in TeraGrid
• Increasing investment by
communities in their own
cyberinfrastructure, but
heterogeneous:
• Resources
• Users – from expert to K-12
• Software stacks, policies
• Science Gateways
– Provide “TeraGrid Inside”
capabilities
– Leverage community investment
Source: Dennis Gannon ([email protected])
OGCE
OGCEPortlets
Portlets
with
with Containe
Containerr
Serv
Service
ice
API
API
Grid
Grid
Serv
Service
ice
Stubs
Stubs
Local
Local
Portal
Portal
Serv
Services
ices
Remote
Remote
Content
Content
Serv
Services
ices
Java
CoG Kit
– Web-based Portals
– Application programs running on
users' machines but accessing
services in TeraGrid
– Coordinated access points
enabling users to move
seamlessly between TeraGrid and
other grids.
Build standard portals to meet the domain
Grid Resources
Workflow Composer
requirements of the biology communities
Dev elop f ederated databases to be
replicated and shared across TeraGrid
OGCE Science Portal
• Three common forms:
Technical Approach
Grid
Protocols
Grid
Serv ice
s
Open Source Tools
HTTP
Remote
Content
Serv ers
Apache
Apache Jetspeed
Jetspeed
Internal
Internal Services
Services
August 2007
Gateways are growing in numbers
• 10 initial projects as part of TG proposal
• >20 Gateway projects today
• No limit on how many gateways can use TG
resources
– Prepare services and documentation so developers
can work independently
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Open Science Grid (OSG)
Special PRiority and Urgent Computing
Environment (SPRUCE)
National Virtual Observatory (NVO)
Linked Environments for Atmospheric
Discovery (LEAD)
Computational Chemistry Grid (GridChem)
Computational Science and Engineering
Online (CSE-Online)
GEON(GEOsciences Network)
Network for Earthquake Engineering
Simulation (NEES)
SCEC Earthworks Project
Network for Computational Nanotechnology
and nanoHUB
GIScience Gateway (GISolve)
Biology and Biomedicine Science Gateway
Open Life Sciences Gateway
The Telescience Project
Grid Analysis Environment (GAE)
Neutron Science Instrument Gateway
TeraGrid Visualization Gateway, ANL
BIRN
Gridblast Bioinformatics Gateway
Earth Systems Grid
Astrophysical Data Repository (Cornell)
August 2007
TeraGrid as a Social Network
• Annual TeraGrid conference TeraGrid ‘08 - Las Vegas - June
• LRAC/MRAC liaisons
• SGW community very successful
– Transitioning to consulting model
• CI Days - campus outreach
– OSG/Internet2/NLR/EDUCAUSE/
MSI-CIEC partnership
• HPC University
– OSG, Shodor, Krell, OSC, NCSI,
MSI-CIEC partnership
• Education and Outreach
– Engaging thousands of people
August 2007
SC 07-09 Education Program Goals
•
•
Multi-year, year-long, Education Program to provide
continuity and broader, sustained impact in education
Increase participation of larger, more diverse
communities in the SC Conference
– Faculty, students, international, under-represented
•
Integrate HPC into undergraduate science, technology,
engineering and mathematics classrooms
• Significantly expanded digital
libraries of resources for
teaching and learning
•
Sponsors: ACM, IEEE, NCSI, CSERD
Intel, Wolfram Research, TeraGrid
August 2007
“HPC University”
• Advance researchers’ HPC skills
– Catalog of live and self-paced training
– Schedule series of training courses
– Gap analysis of materials to drive development
• Work with educators to enhance the curriculum
– Search catalog of HPC resources
– Schedule workshops for curricular development
– Leverage good work of others
• Offer Student Research Experiences
– Enroll in HPC internship opportunities
– Offer Student Competitions
• Publish Science and Education Impact
– Promote via TeraGrid Science Highlights, iSGTW
– Publish education resources to NSDL-CSERD
Quic kTime™ and a
TIFF (Unc ompres s ed) dec ompress or
are needed to s ee this pic ture.
August 2007
Broadening Participation in TeraGrid
• Broaden awareness of TeraGrid
– Campus Visits (coupled with CI Days)
– Professional Society Meetings
– Develop promotional materials
• Build human capacity for Terascale research
– In-depth consulting (5-8 consultants)
– TeraGrid Fellowship Program for faculty and students
– Mentoring Program
– Campus Champions
• Enhance the usability and access of TG via SGs
– Assess Science Gateway readiness and community requirements
– Develop replicable strategies for integrating TeraGrid resources
into SGs, with an emphasis on under-served community needs
August 2007
CI Days
• Working with campuses to take a leadership role
applying CI to accelerate scientific discovery
• First event held at UC Davis has helped catalyze
campus-wide discussions and planning
• Collaboration of Open Science Grid, Internet 2,
National Lamda Rail, EDUCAUSE, Minority Serving
Institution Cyberinfrastructure Empowerment
Coalition, TeraGrid, and local and regional
organizations
• Campus Champions Program
http://cidays.org
August 2007
For More Information
www.teragrid.org
www.computationalscience.org
www.s-education.org
www.nsdl.org
cserd.nsdl.org
www.nsf.gov/oci/
http://cidays.org
[email protected]
August 2007