Transcript Slide 1

Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Komputasi Paralel
Kuliah 01:
Pendahuluan
Yeni Herdiyeni
http://www.cs.ipb.ac.id/~yeni/paralel
Departemen Ilmu Komputer IPB
Semester Genap 2010
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Deskripsi
• Membahas kebutuhan dan klasifikasi mesin
paralel (SISD, SIMD, MISD, MIMD, SPMD),
komunikasi antar prosesor, memori persekutuan
(shared memory), pengiriman pesan (message
passing), jaringan interkoneksi (interconnection
network), Desain algoritma paralel, efisiensi dan
percepatan pemrosesan paralel, dan contoh
aplikasi pemprosesan paralel.
• Perangkat lunak yang digunakan : MPI (Message
Passing Interface)
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Pengajar
• Dr. Yeni Herdiyeni, S.Si, M.Komp
• Hendra Rahmawan, S.Si, M.T
• Endang Purnama, S.Si, M.Komp
Komponen Penilaian
• UTS
• UAS
• Tugas
• Quiz
• Project
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Materi Kuliah
1. Pendahuluan
2. Definisi dan motivasi pemrosesan paralel
3. Arsitektur system, Shared memory multiprocessor system,
Message passing multicomputer distributed, Shared
memory dan klasifikasi memori persekutuan MIMD dan
SIMD
4. Topologi Network
5. Paradigma pengiriman pesan dengan menggunakan MPI
6. Prinsip-prinsip Desain Algoritme Paralel
7. Analisis kinerja Pemrosesan paralel
8. UTS
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Materi Kuliah #2
9. Pemrograman Paralel : Distributed Memory
10. Pemrograman Paralel
11. Tinjauan ulang critical section dengan menggunakan Pthread,
siknronisasi dengan Semaphore, Implementasi Semaphore
dilingkungan MPI
12. Sorting
13. Dense Matrix Algorithm
14. Aplikasi pemrosesan (shared memory): problema produsenkonsumen, problema writer reader, problema dining philosophy
15. Presentasi/diskusi proyek
16. UAS
Materi Kuliah dapat diakses di http://www.cs.ipb.ac.id/kulon
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Buku Ajar
• Grama, Ananth., Gupta, Anshul., Karypis, George.,
Kumar, Vipin. 2003. Introduction to Parallel
Computing. Second Edition. Pearson Addision Wesley.
• Quinn, Michael J. 2003. Parallel Programming in C
with MPI and OpenMP . International Edition,
McGraw-Hill.
• Wilkinson, Barry & Allen, Michael. 2005. Parallel
Programming . 2nd Edition,Pearson Educational
International.
• Jordan, Harry F., Alaghband Gita. 2003. Fundamentals
of Parallel Processing. Prentice Hall.
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Motivation : Classical Science
Nature
Observation
Physical
Experimentation
Theory
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Modern Scientific Method
Nature
Observation
Numerical
Simulation
Physical
Experimentation
Theory
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Modern Parallel Architectures
• Two basic architectural scheme:
Distributed Memory
Shared Memory
• Now most computers have a mixed
architecture
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
What is Parallel and Distributed computing?
– Solving a single problem faster using multiple
CPUs
– Parallel = Shared Memory among all CPUs
– Distributed = Local Memory/CPU
– Common Issues: Partition, Synchronization,
Dependencies
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Distributed Memory
CPU
CPU
CPU
memory
memory
CPU
CPU
node
CPU
node
memory
NETWORK
node
node
memory
node
memory
node
memory
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Shared Memory
memory
CPU
CPU
CPU
CPU
CPU
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Seeking Concurrency
•
•
•
•
•
Data dependence graphs
Data parallelism
Functional parallelism
Task Parallelism
Pipelining
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Interconnection Networks
• Uses of interconnection networks
– Connect processors to shared memory
– Connect processors to each other
• Interconnection media types
– Shared medium
– Switched medium
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Most Common Networks
switched
Cube, hypercube, n-cube
switch
Torus in 1,2,...,N Dim
Fat Tree
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Real Shared
Memory banks
System Bus
CPU
CPU
CPU
CPU
CPU
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Virtual Shared
Network
HUB
CPU
node
HUB
CPU
node
HUB
CPU
node
HUB
CPU
node
HUB
CPU
node
HUB
CPU
node
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Mixed Architectures
memory
memory
CPU
CPU
CPU
CPU
node
node
memory
CPU
CPU
node
NETWORK
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
General MPI Program Structure
MPI include file
variable declarations
Initialize MPI environment
Do work and make
message passing calls
Terminate MPI Environment
#include <mpi.h>
void main (int argc, char *argv[])
{
int np, rank, ierr;
ierr = MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Comm_size(MPI_COMM_WORLD,&np);
Do Some Works
*/
/*
ierr = MPI_Finalize();
}
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
The Message-Passing Programming Paradigm
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Message
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Foster’s Design Methodology
•
•
•
•
Partitioning
Communication
Agglomeration
Mapping
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Foster’s Methodology
Partitioning
Problem
Communication
Mapping
Agglomeration
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Example program (1)
Calculating the value of  by:
1
4

dx
2
1 x
0
24
OK!
Pemrosesan Paralel
OK!
Departemen Ilmu Komputer -IPB
=3.141...
Start calculation!
OK!
Calculated
by
process
Calculated
301
Calculatedby
byprocess
process2
OK!
25
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Sequential Algorithm
2
1
0
4
1
2
5
9
3
2
1
1
3
13
14
3
9

=
4
3
1
2
4
13
17
19
4
3
0
2
0
1
11
3
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Phases of Parallel Algorithm
Inner product computation
b
Row i of A
b
Row i of A
b
Row i of A
c
All-gather communication
ci
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Contoh
4x0
+6x1
2x0
+2x2
– 2x3
=
8
+5x2
– 2x3
=
4
–4x0
– 3x1
– 5x2
+4x3
=
1
8x0
+18x1
– 2x2
+3x3
=
40
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Partitioning
P0
P1
 4 6 2 2 8 
 2 0 5 2 4 


 4  3  5 4 1 


8
18

2
3
40


Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Communication
P0
P1
 4 6 2 2 8 
 2 0 5 2 4 


 4  3  5 4 1 


 8 18  2 3 40
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Communication (CONT..)
P0
P1
4 6 2  2 8 
0  3 4  1 0 


0 3  3 2 9 


0 6  6 7 24
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Communication (cont..)
P0
P1
4 6
0  3

0 0

0 0
2 2 8 

4 1 0 
1 1 9

2 5 24
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Communication (cont..)
P0
P1
4 6 2  2 8
0  3 5  1 0


0 0  5 1 9 


0 0 0 3 6
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Shared memory multiprocessor
using a single bus
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Process
• “A program in a run”, a program in the memory.
• A high level view of a UNIX process
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Threads
• A stream of control in a process.
• A high level view of threads in a UNIX process
Parallel Bubble Sort
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Iteration could start before previous iteration finished if does not overtake previous
bubbling action:
Slides for Parallel Programming
Techniques & Applications
Using Networked Workstations
& Parallel Computers 2nd ed.,
by B. Wilkinson & M. Allen, @
37
Pemrosesan Paralel
Departemen Ilmu Komputer -IPB
Virtual Topology
38