Transcript Analysis of Algorithms

Analysis of Algorithms
26-Jul-16
Time and space

To analyze an algorithm means:



developing a formula for predicting how fast an
algorithm is, based on the size of the input (time
complexity), and/or
developing a formula for predicting how much memory
an algorithm requires, based on the size of the input
(space complexity)
Usually time is our biggest concern

Most algorithms require a fixed amount of space
2
Basics

Before we attempt to analyze an algorithm, we need to
define two things:




How we measure the size of the input
How we measure the time (or space) requirements
Once we have done this, we find an equation that
describes the time (or space) requirements in terms of
the size of the input
We simplify the equation by discarding constants and
discarding all but the fastest-growing term
3
What about the constants?


An added constant, f(n)+c, becomes less and less
important as n gets larger
A constant multiplier, k*f(n), does not get less
important, but...




Improving k gives a linear speedup (cutting k in half cuts the
time required in half)
Improving k is usually accomplished by careful code
optimization, not by better algorithms
We aren’t that concerned with only linear speedups!
Bottom line: Forget the constants!
4
Simplifying the formulae

Throwing out the constants is one of two things we
do in analysis of algorithms



By throwing out constants, we simplify 12n2 + 35 to
just n2
Our timing formula may be a polynomial, and may
have terms of various orders (constant, linear,
quadratic, cubic, etc.)
We usually discard all but the highest-order term


We simplify n2 + 3n + 5 to just n2
Bottom line: Throw out the lower order terms!
5
y = x2 + 3x + 5, for x=1..20
6
Big O notation

When we have a polynomial that describes the time
requirements of an algorithm, we simplify it by:




Throwing out all but the highest-order term
Throwing out all the constants
If an algorithm takes 12n3+4n2+8n+35 time, we
simplify this formula to just n3
We say the algorithm requires O(n3) time


We call this Big O notation
(More accurately, it’s Big , but we’ll talk about that shortly)
7
Measuring requirements

Our formulae are in terms of n, the size of the input


For a Turing Machine, there is an obvious measure
Measuring time is usually done by counting
characteristic operations
 A characteristic operation is some operation that is executed
the most times
 Example: Counting the comparisons needed in an
array search

For a Turing Machine, there is an obvious measure
8
Average, best, and worst cases


Usually we would like to find the average time to perform an
algorithm
However,

Sometimes the “average” isn’t well defined

Example: Sorting an “average” array
 Time typically depends on how out of order the array is
How out of order is the “average” unsorted array?
Sometimes finding the average is too difficult



Often we have to be satisfied with finding the worst (longest)
time required


Sometimes this is even what we want (say, for time-critical operations)
The best (fastest) case is seldom of interest
9
Big-O and friends

Informal definitions:

Given a complexity function f(n),

(f(n)) is the set of complexity functions that are lower
bounds on f(n)

O(f(n)) is the set of complexity functions that are upper
bounds on f(n)

(f(n)) is the set of complexity functions that, given the
correct constants, correctly describes f(n)

Example: If f(n) = 17x3 + 4x – 12, then



(f(n)) contains 1, x, x2, log x, x log x, etc.
O(f(n)) contains x4, x5, 2x, etc.
(f(n)) contains x3
10
Formal definition of Big-O*



A function f(n) is O(g(n)) if
there exist positive constants c and N
such that, for all n > N,
0 < f(n) < cg(n)
That is, if n is big enough (larger than N—we don’t
care about small problems), then cg(n) will be bigger
than f(n)
Example: 5x2 + 6 is O(n3) because
0 < 5n2 + 6 < 2n3 whenever n > 3 (c = 2, N = 3)


We could just as well use c = 1, N = 6, or c = 50, N = 50
Of course, 5x2 + 6 is also O(n4), O(2n), and even O(n2)
11
Formal definition of Big-*



A function f(n) is (g(n)) if
there exist positive constants c and N
such that, for all n > N,
0 < cg(n) < f(n)
That is, if n is big enough (larger than N—we don’t
care about small problems), then cg(n) will be smaller
than f(n)
Example: 5x2 + 6 is (n) because
0 < 20n < 5n2 + 6 whenever n > 4 (c=20, N=4)


We could just as well use c = 50, N = 50
Of course, 5x2 + 6 is also O(log n), O(n), and even O(n2)
12
Formal definition of Big-*




A function f(n) is (g(n)) if
there exist positive constants c1 and c2 and N
such that, for all n > N,
0 < c1g(n) < f(n) < c2g(n)
That is, if n is big enough (larger than N), then c1g(n)
will be smaller than f(n) and c2g(n) will be larger
than f(n)
In a sense,  is the “best” complexity of f(n)
Example: 5x2 + 6 is (n2) because
n2 < 5n2 + 6 < 6n2 whenever n > 5
(c1 = 1, c2 = 6)
13
Graphs
f(n) is O(g(n))
cg(n)
f(n)
f(n) is (g(n))
f(n)
cg(n)
N
f(n) is (g(n))
N
c1g(n)
f(n)
c2g(n)

Points to notice:



N
What happens near the beginning
(n < N) is not important
cg(n) always passes through 0, but
f(n) might not (why?)
In the third diagram, c1g(n) and
c2g(n) have the same “shape” (why?)
14
Informal review

For any function f(n), and large enough values of n,




f(n) = O(g(n)) if cg(n) is greater than f(n),
f(n) = theta(g(n)) if c1g(n) is greater than f(n) and c2g(n) is
less than f(n),
f(n) = omega(g(n)) if cg(n) is less than f(n),
...for suitably chosen values of c, c1, and c2
15
The End
The formal definitions were taken, with some slight modifications, from
Introduction to Algorithms, by Thomas H. Cormen, Charles E. Leiserson,
Donald L. Rivest, and Clifford Stein
16