Data- and Compute-Driven Transformation of Modern Science How e-Infrastructure & Policy Support Paradigm Shifts in Research Edward Seidel Senior Vice President, Research and Innovation Skolkovo Institute.

Download Report

Transcript Data- and Compute-Driven Transformation of Modern Science How e-Infrastructure & Policy Support Paradigm Shifts in Research Edward Seidel Senior Vice President, Research and Innovation Skolkovo Institute.

Data- and Compute-Driven
Transformation of Modern Science
How e-Infrastructure & Policy Support
Paradigm Shifts in Research
Edward Seidel
Senior Vice President, Research and Innovation
Skolkovo Institute of Science and Technology
1
Part 1: We are in a period of unprecedented
change in Science and Society…the crises &
opportunities this creates
2
Profound Transformation of Science
Collision of Two Black Holes
1972: Hawking. 1 1994: 10 people, 1998: 15 people,
NCSA Cray Y-MP, NCSA Origin,
person, no
computer
50 KB 50MB
50GB
3
3
Community Einstein Toolkit
“Einstein Toolkit : open
software for astrophysics to
enable new science,
facilitate interdisciplinary
research and use emerging
petascale computers
Many and
groups can do
advanced CI.”
this: field explodes!
 Consortium: 92 members, 46
sites, 15 countries Major triumph of
Computational
 Whole consortium engaged
in directions, support,
Science---solve EEs!
development
Community + software +
 Simulation: Luciano Rezzolla,
algorithms + hardware + …
Max Planck Institut für
4
Gravitationsphysik (AEI)
New Frontiers: Relativistic Matter

Nuclear equations of state
 Collab. with astrophysicists

General relativistic
magnetohydrodynamics
Zoom
in tohave
just
 Some groups
ideal MHD
 Radiation Transport
this
part: post BH
(neutrinos/photons)

Expensive and complicated!
formation
and
 Requires opacities/emissivities
evolution
of jet
 Chemical
reactions
(thermonuclear, chemical)
 SN community!

Computation:
 Multiphysics!!
 GRMHD: petascale problem
 Radiation transport beyond this
Schnetter et al, PetaScale Computing:
Algorithms and Applications, 2007
Post BH Formation/Evolution of Jet

Multiphysics
 GR, Neutrinos, MHD, Nuclear EOS
Multiphyiscs
framework needed
 10 level AMR, optimized for Blue
forWaters
fluids in
Schnetter, et al
 Science
astrophysics and
 200s evolution, analyze it allporous media…

Computer Science

Blue Waters
 6M Pflop@1PF sustained = 70days!
Rezzolla, et al
6
Theory and Observation of Universe

Gravitational Waves!

Gravity and general relativity are
transformed
New era of science
 Complex problems in relativistic
after a century! Dataastrophysics
 Relativity, hydrodynamics, nuclearand computephysics, radiation, neutrinos,
magnetic
dominated
gravitational
LIGO/VIRGO/GEO
fields: globally distributed collab!
wave
astronomy!
 Observe (PB), compute (PF) signals


4 centuries of small science, small data
culture
2-3 decades of radical change in both
data (factors of 1000 per~5 years) and
collaboration
7
US Council on Competitiveness

Ping Golf
 Moved from workstation to Cray, now
make prototype only at last stage of
design
 Too effective: had to simulate “less
effective” design!

Proctor & Gamble
 Pringles “flying” off manufacturing line
causing significant lost product and
revenue. Using CFD codes that Boeing
uses, airflow over the Pringle modeled,
design so Pringles did not “lift off”
8
Part 1a: The Growth of Data
“I’m still
here…”
But I’m your
new baby
big brother…
Data Tsunami
With millions
of processors…
9
Going Beyond a Community
Will require
Transient & Multi-Messenger
Astronomy
integration across

New era: seeing events
as they occur
 Here now




ALMA, EVLA in radio
Ice Cube neutrinos
24-42m optical?
LSST = SDSS (40TB)
every night!
SKA = exabytes
Communities need to
Simulations integrate all
share data, software,
physics
Data-intensive = knowledge, in real
compute-intensive
time


end
On horizon


Astronomy 1500-2010 was
disciplines,
end-topassive.
No longer!
10
NYT
Big Data vs The Long News
Tail ofFlash!
Science
How Drug
do weside
6/3/13:
 Many “Big Data” projects are “special”
harness
the power
effects
discovered
by
 Tend to be highly organized, have singular
ofcurated,
this web
long
tail?
mining
sources of data, professionally
a lot logs:
attention paid to them paroxetine + pravastatin
 What about the “Long Tail”
99%)?
= (the
highother
blood
sugar!
 Thousands of biologists sequencing communities
of organisms
 Thousands of chemist and materials scientists
developing a “materials genome”
 Millions of people “Tweeting”…
 Characteristics:
• Heterogeneous, perhaps hand generated
• Not curated, reused, served, etc…
11
Grand Challenge Communities Combine it All...
Where is it going to go?
Same CI
useful for
black holes,
hurricanes
12
12
Grand Challenge Communities for
Complex
Problems
Social,
behavioral and

Require many disciplines,
all scales
of collaborations
economic
sciences
will
 Individuals, groups, teams, communities
be critical
helping
 Multiscale Collaborations:
Beyondin
teams
us

these
Are dynamic and highlyunderstand
multidisciplinary

Drive sharing technologies and methodologies

 Time domain astronomy, emergency
forecasting,
issues…
metagenomincs, materials genome…
Researchers collaborate, work by sharing data.
Places requirements on eInfrastructre:
 Software, networks, collaborative environments, data,
sharing, computing, etc
 Scientific culture, reproducibility, access, university
structures
 “Publications.” What is a modern publication?
13
13
Scenarios like this in all fields
NEON+GIS
14
Framing the Challenge:
Science
and
Society
Transformed
by
Data
We still
 Modern science
think like
 Data- and computethis…
Students take
intensive
 Integrative, multiscale
note!


4 centuries of constancy,
4 decades 109-12 change!
Multi-disciplinary
Collaborations



Individuals (Galileo!)
Groups, teams, Grand
Challenge Communities
Big Data + Long Tail
 Sea

of Data
Age of Observation
…But such radical
change cannot be
adequately addressed
with (current)
incremental approach!
15
Part 2: Crises, Challenges,
Opportunities
Computing
Data
Cybe
r
Instruments &
Facilities
No, we
are not…
Organizational
structures
Education
Software
End-to-end
Networks
Five Crises
“CDSE” Community needs to address

Computing Technology




Data, provenance, and visualization



Multicore: processor is new transistor
Programming model, fault tolerance, etc
New models: clouds, grids, GPUs, …
How do we create “data scientists”?
What is an international data infrastructure?
Software treated as e-Infrastructure


Complex applications on coupled computedata-networked environments, tools needed
Modern apps: 106+ lines, many groups
contribute, take decades
17
Five Crises

Organization for Multidisciplinary &
Computational Science
 “Universities must significantly change organizational
structures: multidisciplinary & collaborative research are
needed [for US] to remain competitive in global science”
 “Itself a discipline, computational science advances all
science…inadequate/outmoded structures within Federal
government and the academy do not effectively support
this critical multidisciplinary field”

Education



The CI environment is running away from us!
How do we develop a workforce to work
effectively in this world?
How do universities transition?
18
Data Crisis: Information Big Bang
PCAST Digital Data
Industry
Wired, Nature
Storage Networking
Industry Association
(SNIA) 100 Year Archive
Requirements Survey
Report
NSF Experts Study
“there is a pending crisis
in archiving… we have to
create long-term methods
for preserving
information, for making it
available for analysis in
the future.” 80%
respondents: >50 yrs;
68% > 100 yrs
Scientific Computing and Imaging Institute,
University of Utah
The Shift Towards a “Sea of Data”
Implications
Science & society are now data-dominated
How
do we attribute
credit
 Experiment,
computation,
theory
Fundamental questions
new
publication
for
US this
mobile
phone
traffic exceeded 1 exabyte!
become focused
form?
How
are
data
peer
 Classes of data
around data: What to
reviewed?
What
is
a
 Collections, observations, experiments, simulations
curate, how to remove
publication
in
the
modern
 Software
boundaries? How to
data-rich world? What is a
 Publications
incentivize sharing? IP?
business model for OA?
 Totally new methodologies

 Algorithms, mathematics, culture

Data become the medium for
 Multidisciplinarity, communication, publication, science,
20
economic development…
Part 2a: Recommendations
21
ACCI Task Force Reports



Final recommendations
presented to the NSF Advisory
Committee on
Cyberinfrastructure Dec 2010
More than 25 workshops and
Birds of a Feather sessions,
1300 people involved
Final reports on-line
Grand
Challenge
Campus
Bridging
HPC
Data &
Viz
Learning Software
“Permanent programmatic activities in Computational and Data-Enabled Science &
Engineering (CDS&E) should be established within NSF.” Grand Challenges Task Force
“NSF should establish processes to collect community requirements and plan
long-term software roadmaps.” Software Task Force
“Higher education should adopt criteria for tenure and promotion that reward…the
production of digital artifacts of scholarship. Such artifacts include widely used22data
sets, scholarly services delivered online, and software.” Campus Bridging Task Force
Recommendation of NSF Advisory
Committee on Cyberinfrastructure
ACCI
"The National Science Foundation should create a program in
Computational and Data-Enabled Science and Engineering
(CDS&E), based in and coordinated by the NSF Office of
Cyberinfrastructure. The new program should be
collaborative with relevant disciplinary programs in other
NSF directorates and offices."
23
Part 3: Universities attempt to respond
We have to do all this
and revolutionize the
state/national
economy?
24
Skoltech: Example of a 21st Century University in
the Making
Skoltech at a Glance
•
•
Integrated data, compute,
A unique Russian institution instrumentation
in international
infrastructure and policy
context
– This decade: a community of 200
faculty,development
300 postunder
for
docs, 1200 graduate students
• Interdisciplinary research
Focused on science, engineering and technology
• inAccelerating
discovery
– Addressing problems and issues
IT, Energy,
Biomedicine, Space and Nuclear
• Economic development
• Interdisciplinary by design; no departments
– 15 centers organized around complex problems
• With strong programs in support of innovation
and entrepreneurship
– Creating a culture of innovation in every student,
professor, staff member
• Important part of the Skolkovo innovation
ecosystem
Part 3a: You can help lead this revolution
Kathryn Gray
27
Modern Research & Education Ecosystem
Blue Waters
Software
Software
Data
Data
XSEDE
Track 2
Track 2
Data
Track 2
Data
Campus
Campus
Campus
Education Crisis: I
need all of this to
my
Campusstart to solve
Campus
problem!
Campus
Campus
28
Campus
The Opportunity (US picture)!

Now have emerging national Integrated, High
Performance Research Architecture
 Blue Waters and beyond towards exascale: high end
• Extraordinary science continues lead at cutting edge
• Traditional and novel large data applications
• Few places can house, field, or drive such a facility
 XSEDE architecture can connect…
• Campus Bridging: campus to national CI…
– Campus Assets: MRI, Instruments, DNA sequencers…
– Facilities: Supercomputers, telescopes, accelerators,
light sources, NEON …
» ”More silicon than Steel”
– Networks: end-to-end connectivity
» Where are those optical network apps?
29
Much to do to build CDSE on this
Background: address the “5 Crises”

Education
 Many new opportunities and challenges
• CSE already has its struggles
• Now data: what is a “data scientist”?
• CDSE emerges
 Data opportunities for education and citizen science

Faculty development, curriculum development
 Needed on every campus

Talk to NSF, DOE, EC, your national agencies
 Recommendations of ACCI, MPSAC, etc
 New programs needed: See NSF CDS&E, CI TraCS,
CAREER, “LWD”, etc

You can help make this happen
30
Key Messages

Astounding rate of change of the “Triple
Helix” of Research, Education, and
Innovation
 Computing and Data radically change methods
 Culture of collaboration around complex
problems

These create many crises and opportunities
 From technology to methodology to culture…
 Deep integration required for science

Emergence of Computational and Dataenabled Science and Engineering as a
discipline and your role!
 A key part of the paradigm shift
31
& Data
32