Transcript Lecture 9

Divide-and-Conquer
• Matrix multiplication and Strassen’s
algorithm
Matrix Multiplication
• How many operations are needed to
multiply two 2 by 2 matrices?
=
[ [
[
[
r s
t u
[ [
a b
c d
e f
g h
Traditional Approach
[
•
•
•
•
•
=
[ [
[
r s
t u
[ [
a b
c d
e f
g h
r = ae + bg
s = af + bh
t = ce + dg
u = cf + dh
8 multiplications and 4 additions
Extending to n by n matrices
=
[ [
[
[
RS
TU
[ [
AB
CD
• Each letter represents
an n/2 by n/2 matrix
• We can use the
breakdown to form a
divide and conquer
algorithm
•
•
•
•
E F
GH
R = AE + BG
S = AF + BH
T = CE + DG
U = CF + DH
• 8 multiplications of n/2 by
n/2 matrices
• T(n) = 8 T(n/2) + Q(n2)
• T(n) = Q(n3)
Example
[ [ [
[ [ [
=
•
•
•
•
R = AE + BG
S = AF + BH
T = CE + DG
U = CF + DH
1
5
9
13
2
6
10
14
3
7
11
15
4
8
12
16
17
21
25
29
18
22
26
30
19
23
27
31
20
24
28
32
• What are A, B, …, H?
• What happens at this level of
the D&C multiplication?
Strassen’s Approach
[
•
•
•
•
•
•
•
=
p1 = a(f – h)
p2 = (a+b)h
p3 = (c+d)e
p4 = d(g-e)
p5 = (a+d)(e + h)
p6 = (b-d)(g+h)
p7 = (a-c)(e+f)
[ [
[
r s
t u
[ [
a b
c d
•
•
•
•
e f
g h
r = p5 + p4 - p2 + p6
s = p1 + p2
t = p3 + p4
u = p5 + p1 – p3 – p7
• 7 multiplications
• 18 additions
Extending to n by n matrices
=
[ [
[
[
RS
TU
[ [
AB
CD
• Each letter represents
an n/2 by n/2 matrix
• We can use the
breakdown to form a
divide and conquer
algorithm
E F
GH
• 7 multiplications of n/2 by
n/2 matrices
• 18 additions of n/2 by n/2
matrices
• T(n) = 7 T(n/2) + Q(n2)
• T(n) = Q(nlg 7)
Example
[ [ [
[ [ [
=
•
•
•
•
•
•
•
•
•
•
•
p1 = a(f – h)
p2 = (a+b)h
p3 = (c+d)e
p4 = d(g-e)
p5 = (a+d)(e + h)
p6 = (b-d)(g+h)
p7 = (a-c)(e+f)
r = p5 + p4 - p2 + p6
s = p1 + p2
t = p3 + p4
u = p5 + p1 – p3 – p7
1
5
9
13
2
6
10
14
3
7
11
15
4
8
12
16
17
21
25
29
18
22
26
30
19
23
27
31
20
24
28
32
• What are A, B, …, H?
• What happens at this level of
the D&C multiplication?
Observations
• Comparison: n= 70
– Direct multiplication: 703 = 343,000
– Strassen: 70lg 7 is approximately 150,000
– Crossover point typically around n = 20
• Hopcroft and Kerr have shown 7 multiplications are necessary to
multiply 2 by 2 matrices
– But we can do better with larger matrices
• Current best is O(n2.376) by Coppersmith and Winograd, but it is not
practical
• Best lower bound is W(n2) (since there are n2 entries)
• Matrix multiplication can be used in some graph algorithms as a
fundamental step, so theoretical improvements in the efficiency of this
operation can lead to theoretical improvements in those algorithms