Data- and Compute-Driven Transformation of Modern Science How e-Infrastructure & Policy Support Paradigm Shifts in Research Edward Seidel Senior Vice President, Research and Innovation Skolkovo Institute.
Download ReportTranscript Data- and Compute-Driven Transformation of Modern Science How e-Infrastructure & Policy Support Paradigm Shifts in Research Edward Seidel Senior Vice President, Research and Innovation Skolkovo Institute.
Data- and Compute-Driven Transformation of Modern Science How e-Infrastructure & Policy Support Paradigm Shifts in Research Edward Seidel Senior Vice President, Research and Innovation Skolkovo Institute of Science and Technology 1 Part 1: We are in a period of unprecedented change in Science and Society…the crises & opportunities this creates 2 Profound Transformation of Science Collision of Two Black Holes 1972: Hawking. 1 1994: 10 people, 1998: 15 people, NCSA Cray Y-MP, NCSA Origin, person, no computer 50 KB 50MB 50GB 3 3 Community Einstein Toolkit “Einstein Toolkit : open software for astrophysics to enable new science, facilitate interdisciplinary research and use emerging petascale computers Many and groups can do advanced CI.” this: field explodes! Consortium: 92 members, 46 sites, 15 countries Major triumph of Computational Whole consortium engaged in directions, support, Science---solve EEs! development Community + software + Simulation: Luciano Rezzolla, algorithms + hardware + … Max Planck Institut für 4 Gravitationsphysik (AEI) New Frontiers: Relativistic Matter Nuclear equations of state Collab. with astrophysicists General relativistic magnetohydrodynamics Zoom in tohave just Some groups ideal MHD Radiation Transport this part: post BH (neutrinos/photons) Expensive and complicated! formation and Requires opacities/emissivities evolution of jet Chemical reactions (thermonuclear, chemical) SN community! Computation: Multiphysics!! GRMHD: petascale problem Radiation transport beyond this Schnetter et al, PetaScale Computing: Algorithms and Applications, 2007 Post BH Formation/Evolution of Jet Multiphysics GR, Neutrinos, MHD, Nuclear EOS Multiphyiscs framework needed 10 level AMR, optimized for Blue forWaters fluids in Schnetter, et al Science astrophysics and 200s evolution, analyze it allporous media… Computer Science Blue Waters 6M Pflop@1PF sustained = 70days! Rezzolla, et al 6 Theory and Observation of Universe Gravitational Waves! Gravity and general relativity are transformed New era of science Complex problems in relativistic after a century! Dataastrophysics Relativity, hydrodynamics, nuclearand computephysics, radiation, neutrinos, magnetic dominated gravitational LIGO/VIRGO/GEO fields: globally distributed collab! wave astronomy! Observe (PB), compute (PF) signals 4 centuries of small science, small data culture 2-3 decades of radical change in both data (factors of 1000 per~5 years) and collaboration 7 US Council on Competitiveness Ping Golf Moved from workstation to Cray, now make prototype only at last stage of design Too effective: had to simulate “less effective” design! Proctor & Gamble Pringles “flying” off manufacturing line causing significant lost product and revenue. Using CFD codes that Boeing uses, airflow over the Pringle modeled, design so Pringles did not “lift off” 8 Part 1a: The Growth of Data “I’m still here…” But I’m your new baby big brother… Data Tsunami With millions of processors… 9 Going Beyond a Community Will require Transient & Multi-Messenger Astronomy integration across New era: seeing events as they occur Here now ALMA, EVLA in radio Ice Cube neutrinos 24-42m optical? LSST = SDSS (40TB) every night! SKA = exabytes Communities need to Simulations integrate all share data, software, physics Data-intensive = knowledge, in real compute-intensive time end On horizon Astronomy 1500-2010 was disciplines, end-topassive. No longer! 10 NYT Big Data vs The Long News Tail ofFlash! Science How Drug do weside 6/3/13: Many “Big Data” projects are “special” harness the power effects discovered by Tend to be highly organized, have singular ofcurated, this web long tail? mining sources of data, professionally a lot logs: attention paid to them paroxetine + pravastatin What about the “Long Tail” 99%)? = (the highother blood sugar! Thousands of biologists sequencing communities of organisms Thousands of chemist and materials scientists developing a “materials genome” Millions of people “Tweeting”… Characteristics: • Heterogeneous, perhaps hand generated • Not curated, reused, served, etc… 11 Grand Challenge Communities Combine it All... Where is it going to go? Same CI useful for black holes, hurricanes 12 12 Grand Challenge Communities for Complex Problems Social, behavioral and Require many disciplines, all scales of collaborations economic sciences will Individuals, groups, teams, communities be critical helping Multiscale Collaborations: Beyondin teams us these Are dynamic and highlyunderstand multidisciplinary Drive sharing technologies and methodologies Time domain astronomy, emergency forecasting, issues… metagenomincs, materials genome… Researchers collaborate, work by sharing data. Places requirements on eInfrastructre: Software, networks, collaborative environments, data, sharing, computing, etc Scientific culture, reproducibility, access, university structures “Publications.” What is a modern publication? 13 13 Scenarios like this in all fields NEON+GIS 14 Framing the Challenge: Science and Society Transformed by Data We still Modern science think like Data- and computethis… Students take intensive Integrative, multiscale note! 4 centuries of constancy, 4 decades 109-12 change! Multi-disciplinary Collaborations Individuals (Galileo!) Groups, teams, Grand Challenge Communities Big Data + Long Tail Sea of Data Age of Observation …But such radical change cannot be adequately addressed with (current) incremental approach! 15 Part 2: Crises, Challenges, Opportunities Computing Data Cybe r Instruments & Facilities No, we are not… Organizational structures Education Software End-to-end Networks Five Crises “CDSE” Community needs to address Computing Technology Data, provenance, and visualization Multicore: processor is new transistor Programming model, fault tolerance, etc New models: clouds, grids, GPUs, … How do we create “data scientists”? What is an international data infrastructure? Software treated as e-Infrastructure Complex applications on coupled computedata-networked environments, tools needed Modern apps: 106+ lines, many groups contribute, take decades 17 Five Crises Organization for Multidisciplinary & Computational Science “Universities must significantly change organizational structures: multidisciplinary & collaborative research are needed [for US] to remain competitive in global science” “Itself a discipline, computational science advances all science…inadequate/outmoded structures within Federal government and the academy do not effectively support this critical multidisciplinary field” Education The CI environment is running away from us! How do we develop a workforce to work effectively in this world? How do universities transition? 18 Data Crisis: Information Big Bang PCAST Digital Data Industry Wired, Nature Storage Networking Industry Association (SNIA) 100 Year Archive Requirements Survey Report NSF Experts Study “there is a pending crisis in archiving… we have to create long-term methods for preserving information, for making it available for analysis in the future.” 80% respondents: >50 yrs; 68% > 100 yrs Scientific Computing and Imaging Institute, University of Utah The Shift Towards a “Sea of Data” Implications Science & society are now data-dominated How do we attribute credit Experiment, computation, theory Fundamental questions new publication for US this mobile phone traffic exceeded 1 exabyte! become focused form? How are data peer Classes of data around data: What to reviewed? What is a Collections, observations, experiments, simulations curate, how to remove publication in the modern Software boundaries? How to data-rich world? What is a Publications incentivize sharing? IP? business model for OA? Totally new methodologies Algorithms, mathematics, culture Data become the medium for Multidisciplinarity, communication, publication, science, 20 economic development… Part 2a: Recommendations 21 ACCI Task Force Reports Final recommendations presented to the NSF Advisory Committee on Cyberinfrastructure Dec 2010 More than 25 workshops and Birds of a Feather sessions, 1300 people involved Final reports on-line Grand Challenge Campus Bridging HPC Data & Viz Learning Software “Permanent programmatic activities in Computational and Data-Enabled Science & Engineering (CDS&E) should be established within NSF.” Grand Challenges Task Force “NSF should establish processes to collect community requirements and plan long-term software roadmaps.” Software Task Force “Higher education should adopt criteria for tenure and promotion that reward…the production of digital artifacts of scholarship. Such artifacts include widely used22data sets, scholarly services delivered online, and software.” Campus Bridging Task Force Recommendation of NSF Advisory Committee on Cyberinfrastructure ACCI "The National Science Foundation should create a program in Computational and Data-Enabled Science and Engineering (CDS&E), based in and coordinated by the NSF Office of Cyberinfrastructure. The new program should be collaborative with relevant disciplinary programs in other NSF directorates and offices." 23 Part 3: Universities attempt to respond We have to do all this and revolutionize the state/national economy? 24 Skoltech: Example of a 21st Century University in the Making Skoltech at a Glance • • Integrated data, compute, A unique Russian institution instrumentation in international infrastructure and policy context – This decade: a community of 200 faculty,development 300 postunder for docs, 1200 graduate students • Interdisciplinary research Focused on science, engineering and technology • inAccelerating discovery – Addressing problems and issues IT, Energy, Biomedicine, Space and Nuclear • Economic development • Interdisciplinary by design; no departments – 15 centers organized around complex problems • With strong programs in support of innovation and entrepreneurship – Creating a culture of innovation in every student, professor, staff member • Important part of the Skolkovo innovation ecosystem Part 3a: You can help lead this revolution Kathryn Gray 27 Modern Research & Education Ecosystem Blue Waters Software Software Data Data XSEDE Track 2 Track 2 Data Track 2 Data Campus Campus Campus Education Crisis: I need all of this to my Campusstart to solve Campus problem! Campus Campus 28 Campus The Opportunity (US picture)! Now have emerging national Integrated, High Performance Research Architecture Blue Waters and beyond towards exascale: high end • Extraordinary science continues lead at cutting edge • Traditional and novel large data applications • Few places can house, field, or drive such a facility XSEDE architecture can connect… • Campus Bridging: campus to national CI… – Campus Assets: MRI, Instruments, DNA sequencers… – Facilities: Supercomputers, telescopes, accelerators, light sources, NEON … » ”More silicon than Steel” – Networks: end-to-end connectivity » Where are those optical network apps? 29 Much to do to build CDSE on this Background: address the “5 Crises” Education Many new opportunities and challenges • CSE already has its struggles • Now data: what is a “data scientist”? • CDSE emerges Data opportunities for education and citizen science Faculty development, curriculum development Needed on every campus Talk to NSF, DOE, EC, your national agencies Recommendations of ACCI, MPSAC, etc New programs needed: See NSF CDS&E, CI TraCS, CAREER, “LWD”, etc You can help make this happen 30 Key Messages Astounding rate of change of the “Triple Helix” of Research, Education, and Innovation Computing and Data radically change methods Culture of collaboration around complex problems These create many crises and opportunities From technology to methodology to culture… Deep integration required for science Emergence of Computational and Dataenabled Science and Engineering as a discipline and your role! A key part of the paradigm shift 31 & Data 32