Transcript MS 101: Algorithms

Instructor
Neelima Gupta
[email protected]
Table of Contents
Divide and Conquer
 Divide the problem into smaller sub-problems.
 Solve the sub-problems recursively
 Combine the solution of the smaller sub-problems to
obtain the solution of the bigger problem.
 Size of subproblems must reduce
 There must be some initial conditions
 The two together ensures that the algorithm
terminates
Familiar Egs of divide and
conquer
Quick sort
 Merge sort (John von Neumann, 1945)

Time
If T(n) is the time to sort ‘n’ numbers :T(n)=T(n/2)+T(n/2)+Ө(n)
=2T(n/2) + cn
=Ө(nlog n)
Where Ө(n )= time required to merge two sorted lists,
each of size at most n/2
Multiplying 2 large integers
 Suppose for simplicity, numbers are of equal length
Egsuppose the 2 number are
2345 and 3854
 O(n2) time by the usual successive add algorithm.
 We can reduce this time using divide and conquer
strategy.
Multiplying 2 large integers
split the numbers into roughly 2 halves
Here x0 =45; x1 =23; y0 =54; y1 =38;
 Mathematically
x= x1 .10n/2 + x0 (1)
y= y1 .10n/2 + y0 (2)
xy=x1y1.10n+(x1y0+y1x0).10n/2+x0y0 (3)
In our eg n=4
x=23*102 + 45
y=38*102 + 54
x1y1=874;x1y0=1242;x0y1=1710;x0y0=2430;
Using (1),(2),(3) answer came out to be
= 874*104 +(2952)*102+2430
adding these numbers take linear time.
Time Analysis
 Computing
x1y1,(x1y0+y1x0),x0y0
T(n)=4T(n/2)+cn
=Ө(n2)
So, no improvement. We’ll use a smarter way to compute the
middle term.
(x0+x1)(y0+y1)=x1y1+(x1y0+y1x0)+x0y0
Thus,
x1y0+y1x0 = (x0+x1)(y0+y1) - x0y0 - x1y1
Thus, only 3 multiplications suffice
 So T(n)is reduced to
T(n)=3T(n/2)+cn
=Ө(nlog23)<Ө(n2)
where 3T(n/2) is the time required to multiply
x1y1 , (x0+x1)(y0+y1), x0y0
Complex Roots of Unity
 Consider xn = 1
 It has n distinct complex roots as follows:
ωj,n = e (2 π ij)/n , j = 1 to n – 1
where i = √-1
These numbers are called nth roots of unity
th
n
roots of unity
For example, for n = 2
x2=1
=> x = +1 , -1
e (2 π ij)/n , j = 0,1
j=0
j=1
e (2 π ij)/n = e0 = cos 0 + i sin 0
=1
e (2 π i)/2 = eπi = cos π + i sin π
= -1
Thanks to : Megha and Mitul
 These can be pictured as a set of equally spaced points
lying on a unit circle as shown in figure for n = 8.
 Figure by Mitul
 If ωj,2n ( j = 0, 1 … 2n-1) are (2n)th roots of unity then
clearly ω2j,2n ( j = 0, 1 … n-1) are nth roots of unity.
For j = n … 2n -1, roots repeat.
ω j,2n = e (2 π ij)/2n = e (π ij)/n
ω j,2n2 = (e (π ij)/n )2 = e (2 π ij)/n
It is also visible from the figure below
 Figure by Mitul
Discrete Fourier Transform
 DFT of a polynomial with coefficient vector
<a0,a1,….,an> is the vector y = <y0,y1,….,yn> where

y j  A e

2ij
n



 Θ(n2) time to compute DFT is trivial, each yi can be
computed in Θ(n) time.
 FFT is a method to compute DFT in Θ(n log n) time,
It makes use of special properties of complex roots of
unity.
Fast Fourier Transform
Instead of n, we will deal with 2n
Break the polynomial in 2 parts :
Aeven(x) = a0 + a2x + … + an-2 x (n-2)/2
Aodd(x) = a1 + a3x + a5x2 + ….+ an-1x(n-2)/2
A(x) = Aeven(x2) + x.Aodd(x2)
Let
p(x) = Aeven(x2) = a0+a2x2 + a4x4+….+ an-2xn-2
and
q(x) = Aodd(x2) + a1 + a3x2 + a5x4+….+an-1xn-2
x.q(x) = a1x + a3x3 + a5x5+…+an-1xn-1
Thanks to : Megha and Mitu
Fast Fourier Transform contd…
 Thus,
 yj = A(ωj,2n ) = Aeven(ω2j,2n ) + ωj,2n .Aodd(ω2j,2n )
(2n)th root of unity
nth root of unity
 Aeven and Aodd are polynomials with n terms, thus
they can be computed at nth roots of unity,
recursively.
FFT contd..
 Thus we arrive at the following recurrence to compute
the DFT
 T(n) = 2T(n/2) + O(n)
= n log n
Thus given a polynomial, its DFT can be computed
in O(n log n) time.
Vandermonde Matrix
 Computing DFT is equivalent to
1
...
1  a 
 y0   1
0
y  


1



1
1
,
n
2
,
n
n

1
,
n
a

 
 1 
2
2
...
n 1,n   ... .
...   1 1,n

 ...
  ... 
...
...
...
...

 


n 1
n 1
n 1
a
 yn1   1 1,n1 2,n 1 n 1,n 1   n 1 
Vandermonde matrix
 n is only a control variable and can be replaced by 2n
 Since DFT can be computed in O(n log n) time, this
matrix-vector product can be computed in O(n log n)
time.
Computing
-1
DFT
Theorem:For j,k = 0 …2n -1, (j,k) entry of V-12n is ω-kj,2n /2n
Thus, given yi ‘s , ai ‘s and hence the polynomial can be
computed as follows:
1
1
1


a
 0 
1,n
1
a  
n
1

2



1,n
...    1
n

 
...
...  ...
an1  
1,n( n1)
1
n

...
2,1n
n
1
...
...
...
...
...
2,(nn1)
n
...

n11,n  y
 0 
n  y 
n21,n   1 
... .

n 

...   ... 
n(1n,n1)   yn1 

n 
1
As before n can be replaced by 2n. This matrix-vector
multiplication is similar to the previous one and hence can be
computed in O(n log n) time.
Convolution
Let
A = <a0,a1,….,an> and
B = <b0,b1,…..,bn> be 2 vectors
C = A o B = <c0,c1,…..,c2n-2> (Length
= 2n -1)
C k = ∑ i+j = k aibj
AIM : to obtain convolution in (nlog(n)) time
Thanks to : Megha and Mitu
Applications: polynomial
multiplication
Suppose we have two polynomials
A(x) = a0 + a1x + ………+ an-1xn-1
B(x) = b0 + b1x + ………+ bn-1xn-1
Then, C(x) = A(x).B(x)
Algortihm for AoB
 Let A(x) and B(x) be two polynomials with coefficients
from the vectors A and B respectively.
1. Compute A and B at (2n)th roots of unity i.e.
compute A(ωj,2n ) and B(ωj,2n ) , j = 0,1 …2n-1 using
FFT.
O(n log n) time.
2. Compute C(ωj,2n ) = A(ωj,2n ) . B(ωj,2n ) …. O(n)
time.
3.
Reconstruct C(x) using FFT-1 ….O(n log n) time.
Submitted By:
Jewel Pruthi(18)
Juhi Jain(19)
Problem : Given a set of points p1,p2,--------pn,
our aim is to find out the closest pair
of points.
Finding Closest Pair in one-dimension
Complexity – O(nlogn)
How?
Thanks to
Jewel and Juhi
 Sort the points.(takes O(nlogn) time)
 Find distance between every pair of consecutive
points.
 In n comparisons,we will find the distance between
every pair of consecutive points.
 In another n comparisons,we will find minimum of the
distances found.
Thanks to Jewel
and Juhi
Finding Closest Pair in twodimension
Proposed algorithm:Sort the points p1,p2,------pn say on
increasing order of x-coordinates.
where pi=(xi,yi)
Distance between 2 consecutive points
d=sqrt((y2-y1)2 + (x2-x1)2 )
st x1≤x2≤x3------≤xn
Ques: Will this algorithm work?
Thanks to Jewel and
Juhi
 No
p2
p1
p3
Consider p1,p2,p3 st d(p1p2) > d(p2p3) after sorting p1,p2,p3
on x-coordinates.
Algorithm returns p2p3 but we can see that p1p3 are
closer.
Thanks to Jewel
and Juhi
Note : For minimum distance,the two points need not be
consecutive on x/y-axis.
Brute Force Approach
Calculate distance between every possible pair of points.
Time Complexity:- O(n2)
Can we improve on the time complexity?
Thanks to Jewel and Juhi
Divide and Conquer Approach
 Arrange the points in increasing order of x-coordinates say
Px & increasing order of y-coordinates say Py.
 Divide the set of n points into 2 halves Q and R (breaking
on middle of Px).
 Compute the closest pair in Q and in R recursively.
Q
Solve
recursively
Juhi
R
Solve recursively
Thanks to Jewel and
Q
(q1,q2)
Solve
recursively
R
(r1,r2)
Solve
recursively
Let (q1,q2) and (r1,r2) be the closest pairs obtained in Q
and R respectively.
Let δ=min { d(q1,q2) , d(r1,r2) }
Thanks to Jewel and Juhi
Ques: Does Ǝ a pair of points say (s1,s2) such that d(s1,s2)
<δ?
Soln: Let p* be the point with maximum x-coordinate in
Q and let x* be its x-coordinate.
Draw a line through p* described by the equation L :
x=x*
Thanks to Jewel
and Juhi
This distance is x*-qx
Q
δ
R
q(qx)
p*
δ
r(rx)
x*
L
Now in the highlighted triangle we can see that the length
of horizontal line
(x*-qx) < hypotenuse according to Pythagoras theorem.
x*-qx<hypotenuse<d(q,r)< δ
Similarly we can prove that rx-x* < d(q,r) < δ
Thanks to Jewel and Juhi
 Consider square boxes each of side δ/2 in this vertical
strip.
S
δ/2
qy
δ/2
Square of side δ/2
Thanks to Jewel and Juhi
ry
L
Claim : No box contains more than 1 point.
Proof: If the 2 farthest points in the box are the 2
diagonal points of the square, then
1. Distance between the 2 points =
√( (δ/2)2 + (δ/2)2 ) = (δ/√2) < δ
2. Both the points are within Q or both are within R
This implies that we have two points within Q (/R) with
distance < δ which is a contradiction to the definition of
δ.
Thanks to
Jewel and Juhi
Claim : Between any pair of 2 points q and r in S with
d(q,r) < δ , there can be no more than 12 points.
ṕr
ṕk
No of points ≤ 12
Thanks to Jewel and Juhi
ṕ2
ṕ1
Proof: Let Py1 … Pyt be the points of S in the increasing
order of y co-ordinates.
If there are more than 12 points between q and r in the
above list then there will be at least 3 rows between the
y co-ordinate of q and y co-ordinate of r.
Then,
the vertical distance between q and r is
> 3 × (δ/2) > δ
Thus, the actual distance which is > vertical distance
(show through fig.) > δ
---Contradiction to the definition of points q and r.
Thanks to Jewel and Juhi
 For i = 1 to t
 For j = i+1 to i + 12

Compute d(Pyi , Pyj)
 Compute the minimum of the 12t pairs above --- O(n)
time.
THANKYOU