CASC September Meeting Planning for CIF21 New Computational Infrastructure: CDS&E Software HPC

Gabrielle Allen, Eduardo Misawa, Manish Parashar Irene Qualters, Barry Schneider Office of CyberInfrastructure September, 2011

1

Work in Progress: Moving from Recommendations to Plan

• Key elements considered in computational infrastructure: • CDS&E • Software • HPC • Common themes: • Operating models for Sustainability- balanced with innovation/agility • Cross agency and international collaborations • Effective cross-directorate engagement • Alignment and overlap with other CIF aspects (e.g. data, access & security)

NSF-ACCI Task Force on Grand Challenges:

CYBERSCIENCE AND ENGINEERING

Summary of the TF Recommendations

J. Tinsley Oden

December 9, 2010

Summary and Recommendations: Computational Methods & Algorithms

General Recommendations

 A broad-based, comprehensive, long-term, and vigorous research program in advanced computational methods should be established to overcome the challenges faced in devising, analyzing, and scaling up new computational methods for critical CS&E problems on advanced computing systems.  This program should support multidisciplinary and interdisciplinary teams that bring together applied mathematicians, computer scientists, and computational scientists and engineers.

NSF Postdoc Fellowships

CI TraCS: Transformative Computational Science using Cyberinfrastructure

Image: W. B enger (LSU ) Image: M . Turk (C olumbia U.)

Aim: Support outstanding postdocs developing innovative computational and data-enabled science & engineering

· Use cyberinfrastructure to make revolutionary scientific advances · Research and develop cyberinfrastructure to drive innovations

Fellowships awarded directly to applicants! $240K over 3 years Research supplements for fellows moving to tenure-track faculty positions

www.nsf.gov/citracs Deadline: Jan 13 th 2012

Applicants must be U.S. citizens, nationals, or legally admitted permanent resident aliens of the United States

Summary and Recommendations: High Performance Computing

General Recommendations

 NSF needs to prioritize stable funding of • a diverse set of classical HPC resources, and, • innovative computing, networking and data analytics equipment to support the wide range of needs of the NSF research community.  HPC leadership needs development of innovative technologies including: • • • power-aware and application-sensitive architectures, new numerical algorithms to efficiently use petascale and exascale architectures, and data flow and data analysis at extreme scale.

Summary & Recommendations: Software

General Recommendations

It is recommended that NSF:  Support the creation of reliable, robust science and engineering applications and data analysis and visualization applications for Grand Challenges as well as the software development environment needed to create these applications.

 Provide support for the professional staff needed to create, maintain, evolve and disseminate the above applications as part of its grant funding.

 Establish best practices for the release of science and engineering applications and data as well as the workflows involved in their creation to ensure the reproducibility of computational results.

Software Infrastructure for Sustained Innovation (SI

2

)

http://www.nsf.gov/si2/

Software is Critical to CIF21

 SW as the modality for CF21 and Computational Science in the 21st Century  Software is essential to every aspect of CI – “the glue” • Drivers, middleware, runtime, programming systems/tools, applications, … CDS&E SW  Software crisis?

   Software complexity is impeding the use of CI • Science apps have 10 3 to 10 6+ lines, have bugs • Developed over decades – long lifecycles (~35 years) Software/systems design/engineering issues • Emergent rather than by design Quality of science in question CI

Task Force Recommendations

      Develop multilevel long-term program of support Promote verification, validation, sustainability and reproducibility Develop consistent policy on open source software Support software through collaboration of all NSF divisions, related federal agencies, industry Utilize Advisory Committees for input on priorities These recommendations considered and acted on through NSF-wide software working group.

Software Infrastructure for Sustained Innovations (SI

2

)

 Transform innovations into sustainable software that is an integral part of CIF21  Catalyze and nurture multidisciplinary software as a symbiotic “process” with ongoing evolution  Create a software ecosystem that scales from individual or small groups of software innovators to large hubs of software excellence   FY 10: ~$20M FY 11: $30M – SSI & SSE  Under review now

Software Infrastructure for Sustained Innovations (SI2) – Roadmap



SI

2

FY 12 and beyond…

 S2I2 Awards: Small number of S2I2 awards in areas that are strategically important to CIF21 • Ongoing discussions within the SI2WG  SSI & SSE Awards: Continued development of core SW infrastructure through cross-disciplinary SSI and SSE awards 

Multi-directorate/International Collaborations:

Anticipated formal linkages with NASA, China (NSFC)and UK (UKRC)  CI Integration: Activities to integrate SI 2 awards with core CI investments such as XD and DataNet

HPC Task Force Recommendations

1. Develop a sustainable model to provide the academic research community with access, by 2015 –2016, to a rich mix of HPC systems that: – deliver sustained performance of 20–100 petaflops on a broad range of scienceand engineering applications; – are integrated into a comprehensive national CI environment; – are supported at national, regional, and/or campus levels.

2. Invest now to prepare for exascale systems that will be available by 2018 –2020. NSF should consider the use of co-design partnerships to provide the HPC systems and data CI needed to enable data-driven science.

3. Broaden outreach to improve the preparation of HPC researchers and to engage industry, decision-makers, and new user communities in the use of HPC as a valuable tool.

4. Establish a continuing process for soliciting community input on plans for HPC investments.

HPC Strategy in the CIF21 Context

   Transition from TG to XD as the integrating fabric across HPC resources  Grow shared services; education, outreach, user support  Develop XD architecture as a highly adaptable service  Expand governance model to include new communities Separate CI services from hardware  Common user interface for science not dependent on specific hardware, geographic location or resource HPC resource strategy expands to encompass new classes and kinds of resources   XD Managed Resources and Community Provided Resources New architectures, clouds, data centers Align HPC strategy with software, data, networking, campus bridging and cybersecurity and to support CF21 programs

Blue Waters Update

 ~27 Science/Engineering Projects have been awarded allocations by NSF under NSF 08-529  On August 6 th , IBM and UIUC terminated their contract for delivery of the planned Blue Waters system  Questions regarding the reasons for termination or detail of their contract should be directed to NCSA or IBM as NSF was not a party to the contract.

 Numerous public presentations have been given on the intended configuration and are available online.

 The NSF award to UIUC is in the form of a Cooperative Agreement  UIUC has requested a change to the PEP – this change will be peer reviewed by external panel in Sept.

FY11 HPC Resource Solicitation

 Timeline – NSF 11-511     December, 2010 Solicitation Issued March 7, 2011 Proposals due September, 2011, One award of up to $30M anticipated January, 2013 Initial availability in XD  Includes Community Feedback from ACCI HPC taskforce • Option of an additional 4 year renewal award for upgrade/replacement • Both computational and data intensive  Voluntary Cost Sharing Prohibited per new GPG  O&M award (up to $6M/annum) separate  Introduction of “Innovative Capability” requirement  Optional $5M/$30M for this capability

HPC Strategy – 3 streams

   Transition from TG to XD as the integrating fabric across HPC resources Revisit HPC resource strategy to consider new classes and kinds of resources   XD Managed Resources and Community Provided Resources New and diverse architectures, clouds, data centers  Promote inclusion of all scientific domains  Promote flexible availability at all scales  Adjust /replace existing solicitation to match strategy Align with other CIF21 elements (e.g. software, data and campus bridging) thru CIF21 WG