PlanetLab: An Open Laboratory for Introducing Disruptive Technology into the Internet David Culler University of California, Berkeley http://www.cs.berkeley.edu/~culler March 9, 2004
Download ReportTranscript PlanetLab: An Open Laboratory for Introducing Disruptive Technology into the Internet David Culler University of California, Berkeley http://www.cs.berkeley.edu/~culler March 9, 2004
PlanetLab: An Open Laboratory for Introducing Disruptive Technology into the Internet David Culler University of California, Berkeley http://www.cs.berkeley.edu/~culler March 9, 2004 1 Q: From where will the next 10 major advances in the internet emerge? March 9, 2004 2 Innovator’s Dilemma • The Internet is an enormous success story – commercially – impact on our daily lives – global reach • Success has an unexpected cost: ossification – difficult to deploy disruptive technologies correct vulnerabilities introduce new capabilities March 9, 2004 3 A: ideas honed through process of “rough consensus and running code” March 9, 2004 4 …PlanetLab UBC UW WI Toronto Chicago UPenn Harvard Utah Intel Seattle Intel MIT Intel OR Intel Berkeley Cornell CMU ICIR Princeton UCB St. Louis Columbia Duke UCSB Washu KY UA UCLA Rice GIT UCSD UT ISI Uppsala Lancaster Copenhagen Cambridge UCL Belas Berlin Nizhny Bologna HUJ Taiwan CUHK Towards 1,000 widely-distributed nodes – – – – today: 360 machines, 148 sites, 20 countries, 110 networks at edge sites and network cross-roads Universities, Companies, Nat’l labs March 9, 2004 5 Sydney Canterbury Today’s Internet Best-Effort Packet Delivery Service Limitations – the Internet is “opaque” making it difficult to adapt to network conditions – applications typically split into two pieces: client and server March 9, 2004 6 Internet services March 9, 2004 7 Tomorrow’s Internet Collection of Planetary-Scale Services Opportunities – multiple vantage points anomaly detection, robust routing Adapt to load, failures, $ – Proximity: close wherever you are low latency, high bandwidth, predictable, reliable content distribution, data fusion – multiple, independent domains survivable storage March 9, 2004 8 Towards Planetary-Scale Services • CDN and P2P just tip of the iceberg • Research Community developing the architectural building blocks to enable many kinds of distributed services –scalable translation, –dist. storage, –dist. events, –instrumentation, –management March 9, 2004 9 Key missing element – real hands-on experience • Researchers had no vehicle to try out their next n great ideas in this space • Lot’s of simulations • Lot’s of emulation on large clusters – emulab, millennium, modelnet • Lot’s of folks calling their 17 friends before the next deadline – RON testbed • but not the surprises and frustrations of experience at scale to drive innovation March 9, 2004 10 Quick Growth • “Underground” meeting March 2002 • Intel seeds effort – First 100 nodes, Operational support • First node up July 2002 • By SOSP (March 2003) 25% of accepted papers refer to PlanetLab • Each following conference has seen dramatic load – OSDI – NDSI March 9, 2004 11 Some Example Services March 9, 2004 12 Berkeley: OceanStore `RAID’ distributed over the whole Internet March 9, 2004 13 Washington - ScriptRoute UW Google • Internet measurement tool • open, unauthenticated, community measurement infrastructure vs closed, engineered service March 9, 2004 14 Princeton: CoDeeN Open Content Distribution Network CCC B A BBB A A C C B C B B C AA March 9, 2004 15 A Berkeley – Internet Indirection Infrastructure src dst Get ID Put v,ID • Multicast, anycast, mobility, … March 9, 2004 16 NetBait Serendipity • Brent Chun built a simple http server on port 80 to explain what planetlab was about and to direct inquiries to planet-lab.org • It also logged requests • Sitting just outside the firewall of ~40 universities... • the worlds largest honey pot • the number of worm probes from compromised machines was shocking • imagine the the epidemiology • see netbait.planet-lab.org March 9, 2004 17 1/ 5 1/ / 20 10 03 1/ /20 15 03 1/ /20 20 03 1/ /20 25 03 1/ /20 30 03 /2 2/ 00 4/ 3 2 2/ 00 9 3 2/ / 20 14 03 2/ /20 19 03 2/ /20 24 03 /2 3/ 00 1/ 3 2 3/ 00 6 3 3/ / 20 11 03 3/ /20 16 03 /2 00 3 Probes per day One example 250 March 9, 2004 Code Red Nimda 200 150 100 50 0 • The monthly code-red cycle in the large? • What happened in March 2003? 18 3/ 1/ 2 3/ 003 2/ 2 3/ 003 3/ 2 3/ 003 4/ 2 3/ 003 5/ 2 3/ 003 6/ 2 3/ 003 7/ 2 3/ 003 8/ 2 3/ 003 9/ 3/ 2 00 10 3 / 3/ 200 11 3 / 3/ 200 12 3 / 3/ 200 13 3 / 3/ 200 14 3 / 3/ 200 15 3 / 3/ 200 16 3 / 3/ 200 17 3 / 3/ 200 18 3 / 3/ 200 19 3 / 3/ 200 20 3 /2 00 3 Probes per day No, not Iraq 1400 Code Red Nimda 1200 Code Red II.F 1000 800 600 400 200 0 • A new voracious worm appeared and displaced the older Code Red March 9, 2004 19 Intel Netbait - March 2003 March 9, 2004 20 Internet Public Health March 9, 2004 21 Key Concept: Overlay networks March 9, 2004 22 Overlay network routing March 9, 2004 23 Evolving the Internet • Add a new layer to the network architecture – overlay networks purpose-built virtual networks that use the existing Internet for transmission the Internet was once deployed as an overlay on top of the telephony network overlay Internet • Challenge – how to innovate & deploy at scale March 9, 2004 24 PlanetLab is… UBC UW WI Toronto Chicago UPenn Harvard Utah Intel Seattle Intel MIT Intel OR Intel Berkeley Cornell CMU ICIR Princeton UCB St. Louis Columbia Duke UCSB Washu KY UA UCLA Rice GIT UCSD UT ISI Uppsala Lancaster Copenhagen Cambridge UCL Belas Berlin Nizhny Bologna HUJ Taiwan CUHK Sydney Canterbury More than a bunch of machines spread around the world March 9, 2004 25 PlanetLab is… A common software architecture • • • • OS running on each node Remote installation mechanism Remote management capability Central account management service March 9, 2004 26 PlanetLab is… A network testbed • 450 active research projects – experiment at scale – experiment under real-world conditions – potential for real workloads and users A deployment platform • 15 continuously running services design deploy measure March 9, 2004 27 PlanetLab is… A microcosm of the next Internet • Fold services back into PlanetLab – evolve core technologies to support overlays and slices – discover common sub-services • Long-term goals – become the way users interact with the Internet – define standards that support multiple “PlanetLabs” March 9, 2004 28 Software Architecture • Distributed virtualization – slice a network of virtual machines – isolation isolate services from each other protect the Internet from PlanetLab • Unbundled Management – OS defines only local (per-node) behavior global (network-wide) behavior implemented by services – multiple competing services running in parallel shared, unprivileged interfaces March 9, 2004 29 Slices March 9, 2004 30 Per-Node View Node Mgr Local Admin VM1 VM2 … VMn Virtual Machine Monitor (VMM) March 9, 2004 31 Virtualization • Hypervisors (e.g., VMWare) – don’t scale well – don’t need multi-OS functionality • Paravirtualization (e.g., Xen, Denali) – not yet mature • Virtualize at system call interface (e.g., Jail, Vservers) – reasonable compromise – doesn’t provide the isolation that hypervisors do • Unix processes – isolation is problematic • Java Virtual Machine – too high-level March 9, 2004 32 PlanetLab VMM • Linux: significant mind-share • Vservers: virtualizes at the system call interface – each vserver runs in its own security context private UID/GID name space limited superuser capabilities (e.g., no CAP_NET_RAW) – uses chroot for file system isolation – scales to 1000 vservers per node (29MB each) • Plkmod: enforces isolation – processor and link schedulers, disk quotas – network virtualization safe raw sockets (UDP, TCP, ICMP, GRE) rate limits exceptional packets, unique addrs/ports March 9, 2004 33 Infrastructure Services • Brokerage Services – buy, sell, trade, pool resources – least mature / greatest potential (market-based) • Environment Services – create / maintain programming environments – challenge: space efficiency • Monitoring Services – resource discovery – network / service health – adaptive applications • Routing Underlay – discover network topology March 9, 2004 34 PLC: Brokerage Service create_slice(name, credentials) delete_slice(name, credentials) assign_owners(name, ssh_keys[ ], credentials) set_state(name, boot_script, credentials) set_resources(name, share, credentials) instantiate_slice(name, nodes[ ], credentials) rcap = acquire(rspec) bind(name, rcap) March 9, 2004 35 Stork: Environment Service • Defines a boot_script to be run when VM inits – invokes Stork • Maintains a repository of packages – those registered by a client service + dependencies • Shared package area on each node – retrieves one copy of each required package – installation scripts run in context of client VM • Optionally… – manages user accounts for client slices – establishes tunnels to neighboring nodes March 9, 2004 36 Monitoring Service – Sophia, Pier,… Distributed query processor Information Plane observe analyze react S March 9, 2004 S … A S 37 A … A Pluto: Routing Underlay Expense Service Overlay Networks Library of Routing Services Primitives Topology Probing Kernel Raw Topology Information Scope March 9, 2004 38 Representative Sample of the Internet? March 9, 2004 39 A Rich Research Agenda • Global System Architecture – Slices, management, distribution, • Network measurement • Content Dist. Networks – CoDeeN, ESM, UltraPeer emulation, Gnutella mapping • Management and Monitoring – Ganglia, InfoSpect, Scout Monitor, BGP Sensors, etc. – Scriptroute, PlanetProbe, I3, etc. • Application-level multicast – ESM, Scribe, TACT, etc. • Overlay Networks – RON, ROM++, ESM, XBone, ABone, etc. • Distributed Hash Tables – Chord, Tapestry, Pastry, Bamboo, etc. • Virtualization and Isolation – Xen, Denali, VServers, SILK, Mgmt VMs, etc. • Wide-area distributed storage – Oceanstore, SFS, CFS, Palimpsest, IBP • Router Design implications – NetBind, Scout, NewArch, Icarus, etc. • Resource allocation – Sharp, Slices, XenoCorp, Automated contracts • Distributed query processing – PIER, IrisLog, Sophia, etc. March 9, 2004 • Testbed Federation – NetBed, RON, XenoServers • Etc., etc., etc. 40 Confluence of Technologies • Cluster-based scalable distribution, remote execution, management, monitoring tools – UCB Millennium, OSCAR, ..., Utah Emulab, ... • CDNS and P2Ps – Gnutella, Kazaa, ... • Proxies routine • Virtual machines & Sandboxing – VMWare, Janos, Denali,... web-host slices (EnSim) • Overlay networks becoming ubiquitous – xBone, RON, Detour... Akamai, Digital Island, .... • Service Composition Frameworks – yahoo, ninja, .net, websphere, Eliza • • • • • Established internet ‘crossroads’ – colos Web Services / Utility Computing Authentication infrastructure (grid) Packet processing (layer 7 switches, NATs, firewalls) Internet instrumentation March 9, 2004 41 Current Institutions (partial) Academia Sinica, Taiwan Boston University Caltech Carnegie Mellon University Chinese Univ of Hong Kong Columbia University Cornell University Datalogisk Institut Copenhagen Duke University Georgia Tech Harvard University HP Labs Intel Research Johns Hopkins Lancaster University Lawrence Berkeley Laboratory MIT Michigan State University National Tsing Hua Univ. New York University Northwestern University March 9, 2004 Princeton University Purdue University Rensselaer Polytechnic Inst. Rice University Rutgers University Stanford University Technische Universitat Berlin The Hebrew Univ of Jerusalem University College London University of Arizona University of Basel University of Bologna University of British Columbia UC Berkeley UCLA UC San Diego UC Santa Barbara University of Cambridge University of Canterbury University of Chicago University of Illinois 42 University of Kansas University of Kentucky University of Maryland University of Massachusetts University of Michigan University of North Carolina University of Pennsylvania University of Rochester USC / ISI University of Technology Sydney University of Tennessee University of Texas University of Toronto University of Utah University of Virginia University of Washington University of Wisconsin Uppsala University, Sweden Washington University in St Louis Wayne State University What Planet-Lab is about? • Create the open infrastructure for invention of the next generation of wide-area (“planetary scale”) services – post-cluster, post-yahoo, post-CDN, post-P2P, ... • Potentially, the foundation on which the next Internet can emerge – think beyond TCP/UDP/IP + DNS + BGP + OSPF... as to what the net provides – building-blocks upon which services and applications will be based – “the next internet will be created as an overlay in the current one” (NRC) • A different kind of network testbed – – – – not a collection of pipes and giga-pops not a distributed supercomputer geographically distributed network services alternative network architectures and protocols • Focus and Mobilize the Network / Systems Research Community to define the emerging internet March 9, 2004 43 Join the fun ... www.planet-lab.org • It is just beginning – towards a representative sample of the internet (1,000s) • PlanetLab consortium recently formed – Princeton, UCB, UWash – Intel, HP, Google, … • Hands-on experience with wide-area services at scale is mothering tremendous innovation – nothing “just works” in the wide-area at scale – Joining forces we all learn more March 9, 2004 44