Transcript Multicores, Multiprocessors, and Clusters Dr. Mike Turi Pacific Lutheran University 5/12/2014 Uniprocessor vs. Multiprocessors • Power big issue with microprocessors • Large, inefficient processor vs.

Multicores, Multiprocessors, and
Clusters
Dr. Mike Turi
Pacific Lutheran University
5/12/2014
Uniprocessor vs. Multiprocessors
• Power big issue with microprocessors
• Large, inefficient processor vs. many smaller,
efficient processors
– Scalable performance
– Fault tolerance
• Requires changes to software
Parallelism
• Job- or Process-Level Parallelism
• Parallel Processing Program
• Speed-up still adheres to Amdahl’s Law
• Must balance load
Scaling
• Strong Scaling
– Problem size fixed as number of processors grow
• Weak Scaling
– Program size grows proportionate to number of
processors
SMPs
• Shared Memory Multiprocessors
– Single physical address space across all processors
– Data synchronization
• Uniform Memory Access (UMA)
• Nonuniform Memory Access (NUMA)
• Scalability?
Data Communication
• Data Synchronization
– Locks
• Explicit Message Passing
– Clusters
Hardware Multithreading
• Multiple threads share functional units of a
processor
• Fine-grained
– Switches between threads on each instruction
• Course-grained
– Switches only on costly stalls
• Simultaneous multithreading (SMT)
– Always executing instructions from multiple
threads
Data vs. Instruction Streams
• SISD: Single instruction, single data
– A uniprocessor
• MISD: Multiple instruction, single data
• SIMD: Single instruction, multiple data
– Operate on vectors of data
– Data-level parallelism
• MIMD: Multiple instruction, multiple data
– A multiprocessor
Multiprocessor Network Topologies
• Ring vs. Fully Connected and Crossbar
• Multistage Network
– Switch
– Processor