Transcript Algorithms - Mount Holyoke College

Algorithm Analysis
Math Review – 1.2
•
Exponents
–
–
–
–
–
•
XAXB = XA+B
XA/XB=XA-B
(XA)B = XAB
XN+XN = 2XN ≠ X2N
2N+2N = 2N+1
Logarithms
– XA=B iff logXB=A
– logAB =logCB/logCA; A,B,C > 0, A ≠ 1
– logAB = logA + logB; A, B > 0
•
Series
N
– ∑ 2i = 2N+1-1
i=0
N
– ∑ Ai = AN+1-1/A-1
i=0
N
– ∑ i = N(N+1)/2
i=0
Running Time
• Why do we need to analyze the running
time of a program?
• Option 1: Run the program and time it
– Why is this option bad?
– What can we do about it?
Pseudo-Code
•
•
Used to specify algorithms
Part English, part code
Algorithm (arrayMax(A, n))
curMax = A[0]
for i=1 i<n i++
if curMax < A[i]
curMax = A[i]
return curMax
Counting Operations
Algorithm (arrayMax(A, n))
curMax = A[0] //2
for i=1 i<n i++ //1+n
if curMax < A[i] //4(n-1) 6(n-1)
curMax = A[i]
return curMax //1
• Best case – 5n
• Worst case – 7n-2
• Average case – hard to analyze
Asymptotic Notation
•
•
•
•
7n-2
n=5 -> 33
n=100 -> 698
n=1,000,000 -> 6,999,998
• Running time grows proportionally to n
• What happens as n gets large?
Big-Oh
• T(N) is O(f(N)) if there are positive
constants c and n0such that T(N) <= cf(N)
when N>=n0
• 7n2-2 is O(n2) n0>=1 c=8
• Upper bound
Omega
• T(N) is Ω(g(N)) if there are positive
constants c and n0such that T(N) >= cg(N)
when N>=n0
• 7n2-2 is Ω(n2) n0>=1 c=1
• Lower bound
Theta
• T(N) is Θ(h(N)) if and only if T(N) = Oh(N)
and T(N) = Ωh(N)
• 7n2-2 is Θ(n2)
• Tightest result possible
Little-Oh
• T(N) is o(p(N)) if T(N) = O(p(N)) and
T(N) ≠ Θ(p(N))
• 7n2-2 is o(n3)
– O when n0>=8 c=1
• Growth rate of T(N) is smaller than growth
rate of p(N)
Rules
• Rule 1
– If T1(N) = O(f(N) and T2(N) = O(g(N)), then
• T1(N) + T2(N) = max(O(f(N)), O(g(N)))
• T1(N)* T2(N) = O(f(N)*g(N))
• Rule 2
– If T(N) is a polynomial of degree k, then
T(N) = Θ(Nk)
• Rule 3
– logkN = O(N) for any constant k. This tells us that
logs grow very slowly.
Examples
• 87n4 + 7n
• 3nlogn + 12logn
• 4n4 + 7n3logn
Terminology
•
•
•
•
•
•
Logarithmic – O(logn)
Linear – O(n)
Linearithmic – O(nlogn)
Quadratic – O(n2)
Polynomial – O(nk) k>=1
Exponential – O(an) a>1
Rule 1 – for loops
• The running time of a for loop is at most
the running time of the statements inside
the for loop (including tests) times the
number of iterations.
for(i=0; i<5; i++)
cout << “hello\n”;
Rule 2 – nested loops
• Analyze these inside out. The total
running time of a statement inside a group
of nested loops is the running time of the
statement multiplied by the product of the
sizes if all of the loops.
for(i=0; i<n; i++)
for(j=0; j<n; j++)
cout << “hello\n”;
Rule 3 – consecutive statements
• These just add (which means that the max
is the one that counts)
for(i=0; i<n; i++)
a[i] = 0;
for(i=0; i<n; i++)
for(j=0; j<n; j++)
a[i]+=a[j]+i+j;
Rule 4 – if/else
• The running time of an if/else statement is never
more than the running time of the test plus the
larger of the running times of the statements,
if(condition)
S1
else
S2
Example
0
n-1
0
6
4
n-1
… 2
12
3
…
9
…
…
…
…
5
8
…
1
• Find maximum number in nxn matrix
• Algorithm:
Example
• What is the big-oh running time of this
algorithm?
Algorithm:
Input: A, n
curMax = A[0][0]
for i=0 i<n i++
for j=0 j<n j++
if curMax < A[i][j]
curMax = A[i][j]
return curMax
Another Example
0
2
0
6
4
n-1
… 6
8
n-1
… 3
• Determine how many elements of array 1
match elements of array 2
• Algorithm?
Another Example
0
2
0
6
4
n-1
… 6
8
n-1
… 3
Algorithm:
Input: A, B, n
for i=0 i<n i++
for j=0 j<n j++
if A[i] == A[j]
matches++
break
• What is the running time of the algorithm?
Logs in the Running Time
• An algorithm is O(logN) if it takes constant
time to cut the problem size by a fraction.
• Binary search
– Given an integer X and integers A0, A1, …,
AN-1, which are presorted and already in
memory, find i such that Ai=X, or return i = -1 if
X is not in the input.
Binary Search
• Algorithm?
• Running time is at most ceil(log(N-1))+2
which is O(logN)
Example
•
•
•
•
•
1 – (N-1) - 1
2 – (N-1)/2 - 2
3 – (N-1)/4 - 4
4 – (N-1)/8 - 8
i – (N-1)/2i-1 - 2i-1 = N-1
– difference between high and low is 1
• difference between high and low is 0
The Evil King
• King has N bottles of wine
• Exactly 1 bottle is poisoned
• How can the king figure out which bottle is
poisoned and only kill at most logN of his
testers?