Transcript Unique Opportunities in Experimental Computer Systems Research - the Berkeley Testbeds David Culler http://www.cs.berkeley.edu/~culler U.C.

Unique Opportunities in Experimental
Computer Systems Research
- the Berkeley Testbeds
David Culler
http://www.cs.berkeley.edu/~culler
U.C. Berkeley
Grad Student Presentations
8/27/1999
Emerging Problems of Scale
Massive Cluster
Clusters
Gigabit Ethernet
Server
Client
Scalable, Available
Internet Services
Info. appliances
8/26/99
Grad Presentation
2
Convergence at the Extremes
• Powerful Services on “Small” Devices
– massive computing and storage in the infrastructure
– active adaptation of form and content “along the way”
• Extremes more alike that either is to the middle
– More specialized in function
– Communication centric design
» wide range of networking options
– Federated System of Many Many Systems
– Hands-off operation, mgmt, development
– High Reliability, Availability
– Scalability
– Power and space limited
– simplicity
• They have to “work or die!”
8/26/99
Grad Presentation
3
100 node Ultra/Myrinet NOW
•GLUnix
•Active
Messages
•xFS
•Fast sockets,
•MPI, and SVM
•Titanium and
Split-C
•ScaLapack
8/26/99
Grad Presentation
4
Novel Systems Design
• Virtual networks
– integrate communication events into virtual memory system
• Implicit Co-scheduling
– cause local schedulers to co-schedule parallel computations
using a two-phase spin-block and observing round-trip
• Co-operative caching
– access remote caches, rather than local disk, and enlarge
global cache coverage by simple cooperation
•
•
•
•
•
Reactive Scalable I/O
Network virtual memory, fast sockets
ISAAC “active” security
Internet Server Architecture
TACC Proxy architecture
8/26/99
Grad Presentation
5
The Millennium Vision
• To work, think, and study in a computationally
rich environment with deep information stores
and powerful services
– test ideas through simulation
– explore and investigate data and information
– share, manipulate, and interact through natural actions
• Organized in a manner consistent with the
University setting
– clusters of clusters
– Computational Economy
• Novel modes of interacting with large amounts of
data
8/26/99
Grad Presentation
6
The Millennium Community
Business
School of Info. Mgmt and Sys.
BMRC
Chemistry
Computer Science
Electrical Eng.
Biology
Astro
Mechanical Eng.
Physics
Nuclear Eng.
IEOR
Civil
Eng.
8/26/99
MSME
Inst. Of
Transport
Grad Presentation
Economy
Math
7
NT Workstations for Sci. & Eng.
Business
SIMS
BMRC
Chemistry
C.S.
E.E.
Biology
Astro
M.E.
Physics
N.E.
IEOR
C. E.
8/26/99
Transport
MSME
Grad Presentation
Economy
Math
8
SMP => storage, small-scale parallelism
Business
SIMS
BMRC
Chemistry
C.S.
E.E.
Biology
Astro
M.E.
Physics
N.E.
IEOR
C. E.
8/26/99
Transport
MSME
Grad Presentation
Economy
Math
9
Group Cluster of SMPs => Parallelism
Business
SIMS
BMRC
Chemistry
C.S.
E.E.
Biology
Astro
NERSC
M.E.
Physics
N.E.
IEOR
C. E.
8/26/99
Transport
MSME
Grad Presentation
Economy
Math
10
Campus Cluster => large-scale Parallelism
Business
SIMS
BMRC
Chemistry
C.S.
E.E.
Biology
Astro
NERSC
M.E.
Physics
N.E.
IEOR
C. E.
8/26/99
Transport
MSME
Grad Presentation
Economy
Math
11
Gigabit Ethernet Connectivity
Business
SIMS
BMRC
Chemistry
C.S.
E.E.
Biology
Gigabit Ethernet
Astro
NERSC
M.E.
Physics
N.E.
IEOR
C. E.
8/26/99
Transport
MSME
Grad Presentation
Economy
Math
12
FIAT LUX: crossing areas
• Combines
–
–
–
–
Image Based Modeling and Rendering,
Image Based Lighting,
Dynamics Simulation and
Global Illumination in a completely novel fashion to achieve
unprecedented levels of scientific accuracy and realism
• Computing Requirements
– 15 Days of worth of time for development.
– 5 Days for rendering Final piece.
– 4 Days for rendering in HDTV resolution on 140 Processors
• Storage
– 72,000 Frames, 108 Gigabytes of storage
– 7.2 Gigs after motion blur
– 500 MB JPEG
• premiere at the SIGGRAPH 99 Electronic Theater
– http://fiatlux.berkeley.edu/
8/26/99
Grad Presentation
13
An upcoming Comp. Econ Experiment
• Two identical 32 proc Millennium Clusters
• One open shop
• One with usage based on bid-based proportional
share scheduling
8/26/99
Grad Presentation
14
Ninja: Push Services into an Active Infrastr.
Clients
Clients
Clients
Open
Infrastructure
Services
Clients
Clients
Servers
Clients
Servers
Servers
=> enable Distributed Innovation of Scalable, Avail. Services
8/26/99
Grad Presentation
15
Universal Computing Lab (464 Soda)
• Computing in the infra,
in the walls, on the desk,
in your hand, ...
• Not just a new project, a
new computing culture
8/26/99
Grad Presentation
16
universal
Function: adjective
1 : including or covering all or a whole collectively or distributively
without limit or exception
2 a : present or occurring everywhere b : existent or operative
everywhere or under all conditions <universal cultural patterns>
3 a : embracing a major part or the greatest portion (as of
mankind) <a universal state> <universal practices> b :
comprehensively broad and versatile <a universal genius>
4 a : affirming or denying something of all members of a class or
of all values of a variable b : denoting every member of a class
<a universal term>
5 : adapted or adjustable to meet varied requirements (as of use,
shape, or size)
8/26/99
Grad Presentation
17