Transcript Analysis of Algorithms - UCD School of Computer Science

COMP-2001 L4&5
Analysis of Algorithms
• Running Time
• Pseudo-Code
• Analysis of
Algorithms
• Asymptotic Notation
• Asymptotic Analysis
• Mathematical facts
Analysis of Algorithms
T(n)
n4
Input
Algorithm
Output
1
Portions copyright Goodrich & Tommassia!
Average Case vs. Worst Case
Running Timeof an algorithm
• An algorithm may run faster on certain data sets than on others.
• Finding the average case can be very difficult, so typically
algorithms are measured by the worst-case time complexity.
• Also, in certain application domains (e.g., air traffic control,
surgery, IP lookup) knowing the worst-case time complexity is of
crucial importance.
worst-case
5 ms
}
4 ms
average-case?
3 ms
best-case
2 ms
1 ms
Analysis of Algorithms
A
B
C
D
Input
E
F
G
2
Measuring the Running Time
• How should we measure the running time of an algorithm?
• Approach 1: Experimental Study
– Write a program that implements the algorithm
– Run the program with data sets of varying size and composition.
– Use a method like System.currentTimeMillis() to get an accurate
measure of the actual running time.
t (ms)
60
50
40
30
20
10
Analysis of Algorithms
3
0
n
50
100
Beyond Experimental Studies
• Experimental studies have several limitations:
– It is necessary to implement and test the algorithm in order to
determine its running time.
– Experiments can be done only on a limited set of inputs, and
may not be indicative of the running time on other inputs not
included in the experiment.
– In order to compare two algorithms, the same hardware and
software environments should be used.
Analysis of Algorithms
4
Beyond Experimental Studies
• We will now develop a general methodology for
analyzing the running time of algorithms. In
contrast to the "experimental approach", this
methodology:
– Uses a high-level description of the algorithm instead of
testing one of its implementations.
– Takes into account all possible inputs.
– Allows one to evaluate the efficiency of any algorithm in a
way that is independent from the hardware and software
environment.
Analysis of Algorithms
5
Pseudo-Code
• Pseudo-code is a description of an algorithm that is more structured
than usual prose but less formal than a programming language.
• Example: finding the maximum element of an array.
Algorithm arrayMax(A, n):
Input: An array A storing n integers.
Output: The maximum element in A.
currentMax  A[0]
for i 1 to n -1 do
if currentMax < A[i] then currentMax  A[i]
return currentMax
• Pseudo-code is our preferred notation for describing algorithms.
• However, pseudo-code hides program design issues.
Analysis of Algorithms
6
What is Pseudo-Code ?
• A mixture of natural language and high-level programming concepts that
describes the main ideas behind a generic implementation of a data
structure or algorithm.
-Expressions: use standard mathematical symbols to describe
numeric and boolean expressions -use  for assignment (“=” in Java)
-use = for the equality relationship (“==” in Java)
-Method Declarations:
-Algorithm name(param1, param2)
-Programming Constructs: - decision structures:
if ... then ... [else ... ]
- while-loops:
- repeat-loops:
- for-loop:
- array indexing:
-Methods:
Analysis of Algorithms
while ... do
repeat ... until ...
for ... do
A[i]
- calls:
object method(args)
- returns: return value
7
Analysis of Algorithms
• Primitive Operations: Low-level computations
independent from the programming language can be
identified in pseudocode.
• Examples:
– calling a method and returning from a method
– arithmetic operations (e.g. addition)
– comparing two numbers, etc.
• By inspecting the pseudo-code, we can count the number
of primitive operations executed by an algorithm.
8
Example analysis #1:
Algorithm arrayMax(A, n):
Input: An array A storing n integers.
Output: The maximum element in A.
2 operations
currentMax = A[0]
for i = 1 to n -1 do
2 operations
if currentMax < A[i] then
2 operations
currentMax  A[i]
return currentMax
n-1 iterations
Analysis:
Total time = 1 + (n-1)(loop time)
loop time = 4 (assume the worst case -- test is always true)
Total time = 1 + (n-1)4 = 4n
Analysis of Algorithms
- 3 = T(n)
9
Example
of
Asymptotic
Analysis
#2
Algorithm prefixAverages1(X):
Input: An n-element array X of numbers.
Output: An n -element array A of numbers such that A[i] is the average of
elements X[0], ... , X[i].
1 operation
Let A be an array of n numbers.
for i 0 to n - 1 do
1 operation
a0
for j  0 to i do
i iterations with
a  a + X[j]
3 operations i=0,1,2, …, n-1
A[i]  a/(i+ 1)
4 operations
return array A
n iterations
• Analysis ...
total time = 1 + 0i<n (outer loop body time)
outer loop body time = 1 + (inner loop time) + 4
inner loop time = 0j<i (inner loop body time)
inner loop body time = 3
 total time = 1 + 0i<n(1+ 0j<i 3 + 4)
1i n i =
= 1 + 0i<n(5+ 3i)
2+n)/2
(n
= 1 + 0i<n5+ 3 0i<ni
= 1 + 5(n-1) + 3((n-1)2+n-1)/2
10
= 1.5n2+4n-4 = T(n)
Example #3
• A better algorithm for computing prefix averages:
Algorithm prefixAverages2(X):
Input: An n-element array X of numbers.
Output: An n -element array A of numbers such that A[i] is the average of
elements X[0], ... , X[i].
Let A be an array of n numbers.
s 0
1 operation
for i  0 to n do
s  s + X[i]
7 operations
A[i]  s/(i+ 1)
return array A
n+1 iterations
• Analysis ...
Total time = 1 + (n+1)(loop time)
Loop time = 7
Total time = 1 + (n+1)7 = 7n+8 = T(n)
• Compare with T(n) = 1.5n2+4n-4 for previous alg
• Which is better?
Analysis of Algorithms
11
Asymptotic Notation
• Goal: to simplify analysis by getting rid of
unneeded information (like “rounding”
1,000,001≈1,000,000)
• We want to say in a formal way 3n2 ≈ n2
• The “Big-Oh” Notation:
– given functions f(n) and g(n), we say that
f(n) is O(g(n))
if and only if
there are positive constants c and n0 such that
f(n)≤ c g(n) for all n ≥ n0
Analysis of Algorithms
12
Example
For functions f(n)
and g(n) (to the
right) there are
positive constants c
and n0 such that:
f(n)≤c g(n) for n ≥ n0
f(n) = 2n + 6
conclusion:
2n+6 is O(n).
Analysis of Algorithms
13
Another Example
On the other hand…
n2 is not O(n) because there is
no c and n0 such that:
n2 ≤ cn for n ≥ n0
(As the graph to the right
illustrates, no matter how large
a c is chosen there is an n big
enough that n2>cn ) .
Analysis of Algorithms
14
Asymptotic Notation (cont.)
• Note: Even though it is correct to say “7n - 3 is O(n3)”, a
better statement is “7n - 3 is O(n)”, that is, one should make the
approximation as tight as possible
• Simple Rule: Drop lower order terms
and constant factors:
7n-3 is O(n)
8n2log n + 5n2 + n is O(n2log n)
Analysis of Algorithms
15
Asymptotic Notation
(terminology)
• Special classes of algorithms:
logarithmic:O(log n)
linear:
O(n)
quadratic: O(n2)
polynomial: O(nk), k ≥ 1
exponential:O(an), n > 1
• “Relatives” of the Big-Oh
– O(f(n)): Big-Oh -- asymptotic upper bound
–  (f(n)): Big Omega--asymptotic lower bound
–  (f(n)): Big Theta--asymptotic tight bound
Analysis of Algorithms
16
Asymptotic Analysis of The
Running Time
• Use the Big-Oh notation to express the number of primitive
operations executed as a function of the input size parameter n.
• For example, we saw that the arrayMax algorithm has
T(n) = 4n-3 = O(n)
#2 is better
2
2
• Similarly example #2: 1.5n +4n-4 = O(n )
(“asymptotically
example #3:
7n+8 = O(n)
faster”) than #3
• Comparing the asymptotic running time
-an algorithm that runs in O(n) time is better than one that runs in O(n2) time
-similarly, O(log n) is better than O(n)
-hierarchy of functions:
log n << n << n2 << n3 << 2n
• Caution! Beware of very large constant factors. An algorithm
running in time 1,000,000 n is still O(n) but might be less efficient
on your data set than one running in time 2n2, which is O(n2)
17
Summary
1.
2.
3.
4.
we want to predict running time of an algorithm
summarize all possible inputs with a single “size” parameter n
many problems with “empirical” approach (measure lots of
test cases with various n and then extrapolate)
prefer “analytical” approach - examine algorithm to derive a
execution time function T(n) for the number of primitive
operations executed as a function of n (err on the side of pessimism
by always select the worst case)
5.
6.
7.
To select best algorithm, compare their T(n) functions
To simplify this comparision “round” the function using
asymptotic (“big-O”) notation
Amazing fact: Even though asymptotic complexity analysis
makes many simplifying assumptions, it is remarkably useful
in practice: if A is O(n3) and B is O(n2) then B really will be
faster than A no matter how they’re implemented.
Analysis of Algorithms
18
Math You might Need to Review
Logarithms and Exponents (Appendix A)
• properties of logarithms:
logb(xy) = logbx + logby
logb (x/y) = logbx - logby
logbxa = alogbx
logba=
logxa/logxb
• properties of exponentials:
a(b+c) = aba c
abc = (ab)c
ab /ac = a(b-c)
b = a logab
bc = a c*logab
Analysis of Algorithms
19
More Math to Review
• Floor:
x = the largest integer ≤ x
• Ceiling: x = the smallest integer ≥ x
• Summations: (see Appendix A)
• Geometric progression: (see Appendix A)
Analysis of Algorithms
20