Transcript What is campus bridging and what is XSEDE doing about it?

What is campus bridging and what is
XSEDE doing about it?
The beginnings of Campus Bridging as a
concept….
• In early 2009 National Science Foundation’s (NSF) Advisory Committee for
Cyberinfrastructure (ACCI) charged six different task forces: one of those
was called Campus Bridging.
• Cyberinfrastructure consists of computational systems, data and
information management, advanced instruments, visualization
environments, and people, all linked together by software and advanced
networks to improve scholarly productivity and enable knowledge
breakthroughs and discoveries not otherwise possible.
• The goal of campus bridging is to enable virtual proximity:
– the seamlessly integrated use among a scientist or engineer’s
personal cyberinfrastructure; cyberinfrastructure on the scientist’s
campus; cyberinfrastructure at other campuses; and
cyberinfrastructure at the regional, national, and international levels;
as if they were proximate to the scientist.
– When working within the context of a Virtual Organization (VO), the
goal of campus bridging is to make the ‘virtual’ aspect of the
organization irrelevant (or helpful) to the work of the VO.
3
http://pti.iu.edu/campusbridging/
4
Nationally accessible cyberinfrastructure as of
2011 - Estimated Computing Capacity (TFLOPS)
NSF Track 1
Track 2 and other
major facilities
Campus HPC/ Tier 3
systems
Workstations at
Carnegie research
universities
Volunteer
computing
Commercial cloud
(Iaas and Paas)
0
2000
4000
6000
8000
10000
12000
From Welch, V., Sheppard, R., Lingwall, M.J., Stewart, C.A. 2011.
Current structure and past history of US cyberinfrastructure (data set and figures).
http://hdl.handle.net/2022/13136
5
(In)adequacy of Research CI
Never
(10.6%)
Some of the
time (20.2%)
Most of the
time (40.2%)
All of the
time (29%)
Key observations from ACCI Campus
Bridging Task Force (paraphrased):
• Aggregate US cyberinfrastructure
inadequate to meet needs
• Existing CI not optimally utilized
• Many of the challenges have to
do with the existing state of
software, security, and policy
• Some reasonable choices well
executed now are better than
perfect solutions implemented
later
From: NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging. Final Report.
March 2011. http://www.nsf.gov/od/oci/taskforces/TaskForceReport_CampusBridging.pdf.
XSEDE campus bridging vision & strategy
• Promote better and easier use, via XSEDE and campus bridging tools, of the
nation’s aggregate CI resources
• Campus Bridging is more a mindset that should affect most of what XSEDE does
rather than a specific set of software modules within the overall set of XSEDE
services and products.
• Campus Bridging could be thought of as a very technical and broad approach to
useability
• Our goal is going to be to work with the various groups in XSEDE (particularly
XAUS, Campus Champions, Documentation / Training) to align activities and
communications so that XSEDE collectively does things in a way that achieves the
goals set in campus bridging area
• To be conscientiously targeted at Data, HPC, and HTC – probably in that order
• Strategy: conscientiously make a small number of reasoned choices, pursue them
with diligence, and reap economies of scale (if things go right) or clear learning
experiences (otherwise)
7
XSEDE, its components, ‘within’ and
‘beyond’ XSEDE
• XSEDE is “the most advanced, powerful, and robust collection of integrated
advanced digital resources and services in the world. It is a single virtual
system that scientists can use to interactively share computing resources,
data, and expertise.” The XSEDE grant award from NSF funds its core
organizing, support, and management entity.
• Service Providers (part of SP Forum):
– ‘Level 1’ Service Providers meet all XSEDE integration requirements . . .
access through the XSEDE allocation process. [e.g. Kraken & Stampede]
– ‘Level 2’ Service Providers make one or more digital services accessible
via XSEDE services and interfaces, share one or more digital services with
the XSEDE community along with the organization’s local users [IU
Quarry, Cornell Red Cloud are examples]
– ‘Level 3’ Service Providers are the most loosely coupled within the XSEDE
Federation; they will advertise the characteristics of one or more digital
services via XSEDE mechanisms [The CI for the Ocean Observatory
Initiative is a possible example]
Challenges regarding campus bridging
• It’s not a specific thing. You can’t point to a ‘campus bridge’
the way you can a supercomputer
• There is no such thing as a ‘campus bridger’ the way there is a
Campus Champion.
• It may make sense to talk about a ‘bridged resource’
• It’s more a mindset toward a particular form of technical
interoperability and usability than it is a specific thing
• The hardest thing about campus bridging: explaining a set of
use cases that affects several types of XSEDE activities as
campus bridging rather than having A thing to point to
• The second hardest thing: getting colleagues to abandon the
idea that groups interested in campus bridging are XSEDE
Service Provider wannabes.
XSEDE and systems engineering
• XSEDE is “an organization that delivers a series of
instantiations of services to the US research
community.”
• Development of these instantiations takes place via a
particular systems engineering methodology called
Architecture-centric systems engineering
• Use case descriptions =>
Use case quality attribute scenarios =>
Level 3 Decomposition documents (UML) =>
and then on to code and documents!
Campus Bridging use cases
• UCCB 1.0. InCommon-based Authentication. Consistent use of
community-accepted authentication mechanisms.
• UCCB 2.0. Economies of scale in training and usability
• UCCB 3.0. Long-term remote interactive graphic session
• UCCB 4.0. Use of data resources from campus on XSEDE, or from
XSEDE at a campus (Two approaches – one of which we hear
about from Ian Foster in the next talk)
• UCCB 5.0. Support for distributed workflows spanning XSEDE and
campus-based data, computational, and/or visualization resources
• UCCB 6.0. Shared use of computational facilities mediated or
facilitated by XSEDE
• UCCB 7.0 Access to resources on a service for money basis (___ on
demand).
• CB Prerequisite. XSEDE-wide unified trouble ticket handling
Economies of scale in training and
usability from more consistency in
cluster configurations
• In reality, the four cluster admins depicted
here being in agreement are all right.
• Experienced cluster admins all learned how
to use what they learn when the tools were
still developing, so the tool each sysadmin
knows the best is the tool that lets that
sysadmin do their work the best
• The only way to develop consistency is to
provide installers that will make their work
easier
• The XSEDE architecture group is
developing installers for file management
tools
• XSEDE campus bridging is developing
Rocks Rolls cluster build setups (and
documentation)
13
Economies of scale in training and
outreach
• Consistency in system
setups – local becoming
more like XSEDE –
should also lead to
economies of scale in
training
• Materials and trainer
expertise will be more
easily transportable and
extensible
• The campus bridging
group plans to work very
closely with the campus
champions
Image from TeraGridEOT: Education, Outreach, and
Training 2010. https://www.teragrid.org/web/news/
news#2010scihigh
Shared Virtual Compute Facilities
•
•
•
SVCF – virtual cluster independent of XSEDE
– Can we provide tools that will create authentication screens that look and
work like XSEDE login
– Doing this requires supporting multiple authentication mechanisms
– Remember: not everyone one wants to have an XSEDE label on their
organization!
SVCF – accepting jobs from XSEDE
– Requires ability for SVCFs to accept jobs (and trust) XSEDE
– Requires ability for XSEDE to trust SVCFs
– Requires trouble ticket exchange and security notification / response
processes
– This sort of SVCF may be a type of entity that one could meaningfully call a
‘bridged resource.’
This use case has High Performance Computing and High Throughput
Computing as two variants. The Open Science Grid = the solution to the High
Throughput Variants of this use case!
16
Campus Bridging GFFS pilot program





Texas A&M – use of GFFS with Brazos Cluster
Kansas – data transfer within Great Plans and with Indiana
University
CUNY – spanning campus to XSEDE resources
Univ of Miami – data sharing within campus and across WAN
Pilot projects are currently working with XSEDE Operations to
prepare Genesis II Infrastructure for beta usage
Five year goals
• At the end of 5 years, have implemented and
socialized a community vision of an integrated
national cyberinfrastructure with XSEDE as a critical
component – but just a component
• Working with XSEDE and the community as a whole,
implement the technology required to support as
many of the use cases described here as possible
Next deliverables and next steps
• Paper in the proceedings!
• Template for hybrid system description – generalization of the
TACC / IU templates
• Podcast
• Some real progress on pilots
• ROCKS Rolls – mid PY2 first visible results likely
• Continued work with Open Science Grid
• Campus Bridging objectives go in to the prioritization process
with everyone else’s priorities and we arrive at some estimate
of what is feasible over next four years
19
XSEDE Campus Bridging Staff
• Craig Stewart (reports to Scott Lathrop in Scott’s role leading outreach)
• Jim Ferguson (works 25% on campus bridging)
• Therese Miller – IU overall project lead for XSEDE activities (will be aiding,
expected particularly in regards to campus champions)
• Rich Knepper – Manager, Core Services, RT/PTI (0.25 FTE starting next
year)
• 1.0 FTE for “ROCKS Rolling” for PY2
20
Please cite as:
Stewart, C.A., R. Knepper, J.W. Ferguson, F. Bachmann, I. Foster, A.
Grimshaw, V. Hazlewood and D. Lifka. What is campus bridging and
what is XSEDE doing about it? 2012. Presentation. Presented at:
XSEDE12 (Chicago, IL, 16-20 Jul 2012).
http://hdl.handle.net/2022/14599
All slides (except where explicitly noted) are copyright 2012 by the
Trustees of Indiana University, and this content is released under the
Creative Commons Attribution 3.0 Unported license
(http://creativecommons.org/licenses/by/3.0/)
Thanks
•
•
•
•
•
•
•
•
•
Campus Champions – whose input has already shaped Campus Bridging activities
greatly.
All XSEDE staff
Guy Almes, Von Welch, Patrick Dreher, Jim Pepin, Dave Jent, Stan Ahalt, Bill
Barnett, Therese Miller, Malinda Lingwall, Maria Morris
Gabrielle Allen, Jennifer Schopf, Ed Seidel, all of the NSF program officers involved
in the campus bridging task force activities
All of the IU Research Technologies and Pervasive Technology Institute staff who
have contributed to this entire 2+ year process
Special thanks to CASC members who have participated in one of n information
gathering exercises (where n is large)
NSF for funding support (Awards 040777, 1059812, 0948142, 1002526, 0829462;
this material and ongoing work supported by Award 1053575)
Funding support provided by Lilly Endowment and the Indiana University Pervasive
Technology Institute
Any opinions presented here are those of the presenter or collective opinions of
members of the Task Force on Campus Bridging and do not necessarily represent
the opinions of the National Science Foundation or any other funding agencies
22