Transcript Mining Google Scholar Citations: An Exploratory Study

Parallel Computing in Matlab
PCT
•
Parallel Computing Toolbox
•
Offload work from one MATLAB
session (the client) to other MATLAB
sessions (the workers).
•
Run as many as eight MATLAB
workers (R2010b) on your local
machine in addition to your MATLAB
client session.
2
MDCS
•
MATLAB Distributed Computing Server
•
Run as many MATLAB workers on a remote cluster of computers as your
licensing allows.
•
Run workers on your client machine if you want to run more than eight
local workers (R2010b).
3
MDCS installing
4
Typical Use Cases
• Parallel for-Loops
– Many iterations
– Long iterations
• Batch Jobs
• Large Data Sets
5
Parfor
•
Parallel for-loop
•
Has the same basic concept with “for”.
•
Parfor body is executed on the MATLAB
client and workers.
•
The necessary data on which parfor
operates is sent from the client to workers,
and the results are sent back to the client
and pieced together.
•
MATLAB workers evaluate iterations in no
particular order, and independently of each
other.
6
Parfor
A = zeros(1024, 1);
for i = 1:1024
A(i) = sin(i*2*pi/1024);
end
plot(A)
parallelization
A = zeros(1024, 1);
matlabpool open local 4
parfor i = 1:1024
A(i) = sin(i*2*pi/1024);
end
matlabpool close
plot(A)
7
Timing
A = zeros(n, 1);
matlabpool open local 8
tic
parfor i = 1:n
A(i) = sin(i);
end
toc
A = zeros(n, 1);
tic
for i = 1:n
A(i) = sin(i);
end
toc
n
for
parfor
10000
0.003158
0.040542
1000000
0.080678
0.221070
100000000
23.161180
14.230125
8
When to Use Parfor?
Each loop must be
independent of other
loops.
Small number of
simple calculations.
Lots of iterations of
simple calculations.
Long iterations.
9
Classification of Variables
temporary variable
loop variable
reduction variable
sliced input variable
sliced output variable
broadcast variable
10
More Notes
d = 0; i = 0;
for i = 1:4
b = i;
d = i*2;
A(i)= d;
end
d = 0; i = 0;
parfor i = 1:4
b = i;
d = i*2;
A(i)= d;
end
A
[2,4,6,8]
A
[2,4,6,8]
d
8
d
0
i
4
i
0
b
4
b
/
11
More Notes
C = 0;
for i = 1:m
for j = i:n
C = C + i * j;
end
end
How to parallelize?
12
Parfor: Estimating an Integral
13
Parfor: Estimating an Integral
function q = quad_fun( m, n, x1, x2, y1, y2 )
q = 0.0;
u = (x2 - x1)/m;
v = (y2 - y1)/n;
for i = 1:m
x = x1 + u * i;
for j = 1:n
y = y1 + v * j;
fx = x^2 + y^2;
q = q + u * v * fx;
end
end
end
14
Parfor: Estimating an Integral
• Computation complexity: O(m*n)
• Each iteration is independent of other iterations.
• We can replace “for” with “parfor”, for either loop index i or loop
index j.
15
Parfor: Estimating an Integral
function q = quad_fun( m, n, x1, x2, y1, y2 )
q = 0.0;
u = (x2 - x1)/m;
v = (y2 - y1)/n;
tic
A = quad_fun(m,n,0,3,0,3);
toc
parfor i = 1:m
x = x1 + u * i;
for j = 1:n
y = y1 + v * j;
fx = x^2 + y^2;
q = q + u * v * fx;
end
end
end
(m, n)
1+0
1+1
1+2
1+3
1+4
(100, 100)
0.005
0.255
0.087
0.101
0.114
(1000, 1000)
0.035
0.066
0.046
0.045
0.053
(10000, 10000)
3.123
3.123
1.626
1.143
0.883
(100000, 100000)
308.282
309.926
157.393
108.819
85.185
16
Parfor: Estimating an Integral
function q = quad_fun( m, n, x1, x2, y1, y2 )
q = 0.0;
u = (x2 - x1)/m;
v = (y2 - y1)/n;
tic
A = quad_fun(m,n,0,3,0,3);
toc
for i = 1:m
x = x1 + u * i;
parfor j = 1:n
y = y1 + v * j;
fx = x^2 + y^2;
q = q + u * v * fx;
end
end
end
(m, n)
1+0
1+1
1+2
1+3
1+4
(100, 100)
0.005
1.754
1.975
2.126
2.612
(1000, 1000)
0.035
13.146
15.286
18.661
22.313
(10000, 10000)
3.123
113.368
139.568
178.425
220.155
(100000, 100000)
308.282
17
SPMD
• SPMD: Single Program Multiple Data.
– SPMD command is like a very simplified version of MPI.
– The spmd statement lets you define a block of code to run
simultaneously on multiple labs, each lab can have different, unique
data for that code.
– Labs can communicate
synchronization points.
directly via
messages,
they meet
at
– The client program can examine or modify data on any lab.
18
SPMD Statement
19
SPMD Statement
20
SPMD
• MATLAB sets up the requested number of labs, each with a copy of
the program. Each lab “knows" it's a lab, and has access to two
special functions:
– numlabs(), the number of labs;
– labindex(), a unique identifier between 1 and numlabs().
21
SPMD
22
Distributed Arrays
• Distributed()
– You can create a distributed array in the MATLAB client, and its data is
stored on the labs of the open MATLAB pool. A distributed array is
distributed in one dimension, along the last nonsingleton dimension, and
as evenly as possible along that dimension among the labs. You cannot
control the details of distribution when creating a distributed array.
23
Distributed Arrays
• Codistributed()
– You can create a codistributed array by executing on the labs
themselves, either inside an spmd statement, in pmode, or inside a
parallel job. When creating a codistributed array, you can control all
aspects of distribution, including dimensions and partitions.
24
Distributed Arrays
• Codistributed()
– You can create a codistributed array by executing on the labs
themselves, either inside an spmd statement, in pmode, or inside a
parallel job. When creating a codistributed array, you can control all
aspects of distribution, including dimensions and partitions.
25
Example: Trapezoid

b

b
a
a
1
1

f ( x)dx   f (a)  f (b)   (b  a)
2
2

1
f ( x)dx   f ( x1 )  f ( x2 ) 
2

1
 ba
f ( xn )  
2
 n 1
26
Example: Trapezoid
•
To simplify things, we assume interval is [0, 1] , and we'll let each lab define
a and b to mean the ends of its subinterval. If we have 4 labs, then lab
number 3 will be assigned [ ½, ¾].
27
Example: Trapezoid
28
Pmode
•
pmode lets you work interactively with a parallel job running simultaneously
on several labs.
•
Commands you type at the pmode prompt in the Parallel Command
Window are executed on all labs at the same time.
•
Each lab executes the commands in its own workspace on its own
variables.
pmode
spmd
Parallel computing synchronously
Each lab has a desktop
No desktop for labs
Can’t freely interleave serial
and parallel work
Can freely interleave serial
and parallel work
29
Pmode
30
Pmode
•
labindex() and numlabs()
still work;
•
Variables only have the same
name, they are independent of
each other.
31
Pmode
•
Aggregate the array segments into a coherent array.
codist = codistributor1d(2, [2 2 2 2], [3 8])
whole = codistributed.build(segment, codist)
32
Pmode
•
Aggregate the array segments into a coherent array.
whole = whole + 1000
section = getLocalPart(whole)
33
Pmode
•
Aggregate the array segments into a coherent array
combined = gather(whole)
34
Pmode
•
How to change distribution?
distobj = codistributor1d()
I = eye(6, distobj)
getLocalPart(I)
distobj = codistributor1d(1);
I = redistribute(I, distobj)
getLocalPart(I)
35
GPU Computing
• Capabilities
–
–
–
–
–
Transferring data between the MATLAB workspace and the GPU
Evaluating built-in functions on the GPU
Running MATLAB code on the GPU
Creating kernels from PTX files for execution on the GPU
Choosing one of multiple GPU cards to use
• Requirements
– NVIDIA CUDA-enabled device with compute capability of 1.3 or greater
– NVIDIA CUDA device driver 3.1 or greater
– NVIDIA CUDA Toolkit 3.1 (recommended) for compiling PTX files
36
GPU Computing
• Transferring data between workspace and GPU
N = 6;
M = magic(N);
G = gpuArray(M);
M2 = gather(G);
• Creating GPU data
37
GPU Computing
• Executing code on the GPU
– You can transfer or create data on the GPU, and use the resulting
GPUArray as input to enhanced built-in functions that support them.
– You can run your own MATLAB function file on a GPU.
result = arrayfun(@myFunction, arg1, arg2);
• If any of arg1 and arg2 is a GPUArray, the function executes on the GPU
and return a GPUArray
• If none of the input arguments is GPUArray, then arrayfun executes in CPU.
• Only element-wise operations are supported.
38
Review
• What is the typical use cases of parallel Matlab?
• When to use parfor?
• What’s the difference between worker(parfor) and lab(spmd)?
• What’s the difference between spmd and pmode?
• How to build distributed array?
• How to use GPU for Matlab parallel computing?
39