Transcript Document

Future Parallel Computing Systems
– what to remember from the past
RAMP Workshop
FCRC 2007
Dr. Frederica Darema
Senior Science and Technology Advisor
NSF
1
Overview
• Systems Modeling and Analysis is important
– Design cycle and support the runtime
– measurements an important (static and runtime)
– performance, dependability, reliability,
power management, security, ….
• Multicore systems
– New opportunities and something from the past
– What we learned from shared memory systems
of the early-mid80’s
2
Two Components of CSR Program:
•* (AES) Advanced Execution Systems
•* (SMA) System Modeling and Analysis
Distributed Applications
Advanced
Execution
Systems
Prog.Models
Compilers
Libraries
Tools
Collaboration
Environments
Parallel and Distributed/
Visualization
Operating Systems
Scalable I/O
Data Management
Archiving/Retrieval
Services
Authenication
Authorization
Fault Recovery
Services
Distributed Systems Management
Distributed, Heterogeneous, Dynamic, Adaptive
Computing Platforms and Networks
Memory
Technology
3
CPU
Technology
Device
Technology
...
System Modeling and Analysis (SMA)
Seeks to develop methods and tools for modeling, measuring, analyzing,
evaluating, and predicting the performance and correctness of complex
computing and communications systems
SMA emphasizes the development of methods and tools for modeling,
measuring, analyzing, evaluating, and predicting the performance
and correctness of complex computing and communications systems
Topics of Interest
• Hardware and Software modeling
– methods tools and measurements, providing multimodal, hierarchical
or multilevel modeling and analysis capabilities of such systems;
– methods that describe components of the system, but also the
system as a total, and enable assessment of the effects of individual
hardware and software layers and components of these systems;
– ability to describe the system in multiple levels of detail
(characteristics and time-scales);
– combine different methods of describing components and layers
4
System Modeling and Analysis (SMA)
Topics of Interest (cont’d)
• Novel modeling and measurement approaches
– Develop capabilities to describe, analyze and predict the
behavior of the components as well as the systems;
Analysis and prediction due to changes in the application,
system software, hardware; multilevel approaches and
multi-modal approaches
• Performance Frameworks
– combine tools in “plug-and-play” fashion
– multiple views of the system
5
Emphasis on Multidisciplinary Research
(across sub-areas of CS)
Multiple views of the system
The applications’ view
Application
Distributed Applications
Models
...
IO / File
Models
OS
Scheduler
Models
Architecture /
Network
Models
Memory
Models
6
Languages
Compilers
Libraries
Tools
Collaboration
Visualization
Environments
Scalable I/O Authenication/
Data Management Authorization
Archiving/Retrieval Dependability
Services
Services
Other Services . . .
Distributed Systems Management
Distributed, Heterogeneous, Dynamic, Adaptive
Computing Platforms and Networks
Memory
Technology
CPU
Technology
Device
Technology
...
Multiple views of the system
The Operating Systems’ view
Application
Models
...
IO / File
Models
OS
Scheduler
Models
Architecture /
Network
Models
Memory
Models
7
Distributed Applications
Languages
Compilers
Libraries
Tools
Visualization Collaboration
Environments
Scalable I/O Authenication/
Data Management Authorization
Archiving/RetrievalDependability
Services
Services
Other Services. . .
Distributed Systems Management
Distributed, Heterogeneous, Dynamic, Adaptive
Computing Platforms and Networks
Memory
Technology
CPU
Technology
Device
Technology
...
The NGS Program developsTechnology for integrated feedback & control
Runtime Compiling System (RCS) and Dynamic Application Composition
Application
Model
Dynamic Analysis
Situation
Distributed
Programming
Model
Application
Program
Compiler
Front-End
Application
Intermediate
Representation
Compiler
Back-End
Launch
Application (s)
Dynamically
Link
&
Execute
Performance
Measuremetns
&
Models
Application
Components
&
Frameworks
Distributed Computing Resources
Distributed Platform
tac-com
alg accelerator
….
8
MPP
NOW
fire
cntl
data
base
data
base
fire
cntl
SAR
SP
Advanced Execution Systems (AES)
(a component of the Computer Systems Research Program)
(CSR Program)
Seeks to create systems software to facilitate the development and
runtime support of complex applications executing on large,
heterogeneous high-end computing and grid platforms
AES emphasizes runtime compiling systems and application
composition systems interface with the underlying operating systems
services and incorporating systems modeling and analysis methods
and tools.
Topics of Interest
• Novel Compiler Technology that go beyond the standard static notion of a
compiler
– for example by embedding a portion of the compiler in the runtime and
endowing the system with resource awareness and adaptive mapping
capabilities;
– new compiler techniques for determining functional and data
dependencies across multiple levels of memory hierarchy and across
platforms;
– mechanisms for matching an application’s resource needs to underlying
resources when both are changing as the application executes
9
Advanced Execution Systems
(AES)
Topics of Interest
• Programming models and tools
– expressing application partitioning across distributed,
heterogeneous computing platforms; application-level
checkpointing and recovery
• Application composition system (ACS) technology
– constructing applications to fit the available resources and to
adapt to changes in the underlying execution environment;
– methods for automatically selecting application components;
– creating knowledge bases for application components;
interfacing with the underlying computing platform models to
determine suitable application components;
– and developing appropriate application component libraries
and interfaces so the run-time portion of the RCS can link to
such libraries.
10
A number of projects are developing:
systems modeling methods and performance frameworks
&
runtime-compiler and application composition systems
Projects can be found in the proceedings of
the
Next Generation Software
(Workshop Series & PI-Meetings)
I organize every year in conjunction
with IPDPS
The RAMP effort
is very important
in enabling the modeling capabilities
needed for present and future
computer systems and environments
Can lead to a valuable
standard Performance Framework
usable by the broader
academic community and industry
With respect to the Emerging Multicore Systems:
Things to remember from the past
• Coherent caches – issues of scalability to large numbers of processors
• On-chip networks:
– Bus-based communication – contention - & scalability issues
– Cross-bars - can have contention for 100s procs (let alone 100Ks of
procs )
– Consider multistage switches
– Consider combining networks
• Test&Set/Compare& Swap - Fetch&op (e.g Fetch&Add) for
synchronization
• Shared data
– Synchronization (place in a “fast” shared memory - cache)
– Other application shared data (place in shared memory – stage in
cache)
– Past experiments showed that even if 99% of data are shared –
accesses to the shared data were ~20%
• Memory wall - remains challenge; reducing latency remains challenge
– Hardware – pipelining
– Software – compilers/prefetching
• Temporary cacheable shared data (compiler enabled)
• Performance Tools - Parallel tracing – parallel simulations
13