Transcript No Slide Title

CHAPTER 1
Fundamental Tools
Definition of an algorithm
(from CPSC 171):


.
An algorithm is a well-ordered
collection of unambiguous and
effectively computable operations
that, when executed, produces a
result and halts in a finite amount
of time.
The textbook has a similar
definition:
2
Textbook Definitions
• An algorithm is a step-by-step
procedure for performing some task in a
finite about of time.
• A data structure is a systematic way
of organizing and accessing data.
3
Running Time (§1.1)



Most algorithms
transform input objects
into output objects.
The running time of an
algorithm typically grows
with the input size.
Average case time is
often difficult to
determine.
We often focus on the
worst case running time.
• Easier to analyze
• Crucial to applications such
as games, finance and
robotics
best case
average case
worst case
120
100
Running Time

80
60
40
20
0
1000
2000
3000
4000
Input Size
4
Experimental Studies (§ 1.6)



Write a program
implementing the
algorithm
Run the program with
inputs of varying size
and composition
Use a method like
System.currentTimeMillis() to
get an accurate
measure of the actual
running time
Plot the results
9000
8000
7000
Time (ms)

6000
5000
4000
3000
2000
1000
0
0
50
100
Input Size
5
Limitations of Experiments





It is necessary to implement the algorithm,
which may be difficult
Results may not be indicative of the running
time on other input not included in the
experiment.
In order to compare two algorithms, the same
hardware and software environments must be
used
Different programmers have different abilities to
write “good, efficient” code.
We can’t examine all cases.
6
Theoretical Analysis




Uses a high-level description of the
algorithm instead of an
implementation
Characterizes running time as a
function of the input size, n.
Takes into account all possible
inputs
Allows us to evaluate the speed of
an algorithm independent of the
hardware/software environment
7
Pseudocode (§1.1)





Example: find max
High-level
element of an array
description of an
algorithm
More structured
than English prose Algorithm arrayMax(A, n)
Less detailed than a Input array A of n integers
Output maximum element of A
program
Preferred notation
currentMax  A[0]
for describing
for i  1 to n  1 do
algorithms
if A[i]  currentMax then
Hides program
currentMax  A[i]
design issues
return currentMax
8
Pseudocode Details

Control flow

Method call
var.method (arg [, arg…])
if … then … [else …]
 Return value
while … do …
return expression
repeat … until …
 Expressions
for … do …
Indentation replaces braces  Assignment
(like  in Java)
Method declaration
 Equality testing
Algorithm method (arg [, arg…])
(like  in Java)
n2 Superscripts and other
Input …
mathematical formatting
Output …
allowed
•
•
•
•
•

9
The Random Access Machine
(RAM) Model


A CPU
An potentially
unbounded bank of
2
1
memory cells, each of
0
which can hold an
arbitrary number or
character
 Memory cells are numbered and
accessing any cell in memory takes
unit time.
10
Primitive Operations





 Examples:
Basic computations
• Assigning a value to a
performed by an
variable
algorithm
• Calling a method
Identifiable in
• Performing an arithmetic
pseudocode
operation such as addition
Largely independent from • Comparing two numbers
• Indexing into an array
the programming
• Following an object
language
reference
Exact definition not
• Returning from a method
important (we will see
why later)
Assumed to take a
constant amount of time
in the RAM model
11
Counting Primitive Operations (§1.1)
By inspecting the pseudocode, we can determine
the maximum number of primitive operations
executed by an algorithm, as a function of the input
size
Algorithm arrayMax(A,n);
# operations
currentMax  A[0]
2 [index into array;
assign value]
for i  1 to n  1 do
1+n
[n comparisons;
initialization]
if A[i]  currentMax then
2(n  1) [n-1 loops-index;
compare]
currentMax  A[i] 2(n  1) [n-1 loops-index;
assign]
{ increment counter i }
2(n  1) [n-1 loops- add; assign]
return currentMax
1

5n (at least)
Total 7n  2 (worst case)
12
Estimating Running Time

Algorithm arrayMax executes 7n  2 primitive
operations in the worst case. Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive
operation


Let T(n) be worst-case time of arrayMax.
Then
a (7n  2)  T(n)  b(7n  2)
Hence, the running time T(n) is bounded by
two linear functions
13
Growth Rate of Running Time

Changing the hardware/ software
environment
• Affects T(n) by a constant factor, but
• Does not alter the growth rate of T(n)

The linear growth rate of the
running time T(n) is an intrinsic
property of the algorithm arrayMax
14
Growth Rates
Growth rates of
functions:
• Linear  n
• Quadratic  n2
• Cubic  n3

T (n )

In a log-log chart, the
slope of the line
corresponds to the
growth rate of the
function
1E+30
1E+28
1E+26
1E+24
1E+22
1E+20
1E+18
1E+16
1E+14
1E+12
1E+10
1E+8
1E+6
1E+4
1E+2
1E+0
1E+0
Cubic
Quadratic
Linear
1E+2
1E+4
1E+6
1E+8
1E+10
n
15
Constant Factors

The growth rate is not affected by
• constant factors or
• lower-order terms

Examples
• 102n + 105 is a linear function
• 105n2 + 108n is a quadratic function
16
Big-Oh Notation (§1.2)





Important definition: Given functions f(n)
and g(n), we say that f(n) is O(g(n)) if there
are positive constants c and n0 such that
f(n)  cg(n) for n  n0
O(n) is read as big-oh of n.
Example:
Prove: 2n + 10 is O(n)
• We want to find c and n0 such that
2n + 10  cn for n  n0
Realize the values found will not be unique!
17
Prove: 2n + 10 is O(n)
We want to find c and n0 such that
2n + 10  cn for n  n0
The yellow work below is scratch work to find a c and
n0 that works.
 Start with 2n + 10  cn and try to solve for n
 (c  2) n  10
 n  10/(c  2)
 So, one possible choice (but not the only one)
would be to let c = 3 and n0 = 10.
 This is not handed in when a proof is requested.
 Now we can construct a proof!

18
Prove: 2n + 10 is O(n)

We want to find c and n0 such that
2n + 10  cn for n  n0
• Proof: Pick c = 3 and n0 = 10. Then, for n  n0
= 10,
n ≥ 10/1 = 10/(3-2) = 10/(c - 2)
Thus, (c - 2) n ≥ 10 and
cn - 2n ≥ 10
or 2n + 10 ≤ cn
So, we have found c = 3 and n0 = 10 such that
2n + 10  cn for n  n0
Consequently, 2n + 10 is O(n)
19
Big-Oh Example

Example: the function n2 is not O(n)
• We want


n2  cn
n c
• The above inequality cannot be
satisfied since c must be a constant
20
More Big-Oh Examples

7n-2
Claim: 7n-2 is O(n)
need c > 0 and n0  1 such that 7n-2  cn for n  n0
this is true for c = 7 and n0 = 1
 3n3 + 20n2 + 5
3n3 + 20n2 + 5 is O(n3)
need c > 0 and n0  1 such that 3n3 + 20n2 + 5  cn3 for n  n0
this is true for c = 4 and n0 = 21

3 log n + log log n
3 log n + log log n is O(log n)
need c > 0 and n0  1 such that 3 log n + log log n  clog n for n  n0
this is true for c = 4 and n0 = 2
21
More Big-Oh Examples

7n-2
Claim: 7n-2 is O(n)
need c > 0 and n0  1 such that 7n-2  cn for n  n0
this is true for c = 7 and n0 = 1
 3n3 + 20n2 + 5
3n3 + 20n2 + 5 is O(n3)
need c > 0 and n0  1 such that 3n3 + 20n2 + 5  cn3 for n  n0
this is true for c = 4 and n0 = 21

3 log n + log log n
3 log n + log log n is O(log n)
need c > 0 and n0  1 such that 3 log n + log log n  clog n for n  n0
this is true for c = 4 and n0 = 2
22
More Big-Oh Examples

7n-2
Claim: 7n-2 is O(n)
Scratchwork: Need c > 0 and n0  1 such that 7n-2 
cn for n  n0
7n-2 ≤ cn
7n - cn ≤ 2
(7-c) n ≤ 2
this is true for c = 7 and n0 = 1
23
Big-Oh and Growth Rate



The big-Oh notation gives an upper bound on
the growth rate of a function
The statement “f(n) is O(g(n))” means that the
growth rate of f(n) is no more than the growth
rate of g(n)
We can use the big-Oh notation to rank
functions according to their growth rate
Assume:
f(n) is O(g(n))
g(n) is O(f(n))
g(n) grows
more
Yes
No
f(n) grows more
No
Yes
Same growth
Yes
Yes
24
Big-Oh Rules

If is f(n) a polynomial of degree d, then
f(n) is O(nd), i.e.,
1.Drop lower-order terms
2.Drop constant factors

Use the smallest possible class of
functions
• Say “2n is O(n)” instead of “2n is O(n2)”

Use the simplest expression of the class
• Say “3n + 5 is O(n)” instead of “3n + 5 is
O(3n)”
25
Asymptotic Algorithm Analysis


The asymptotic analysis of an algorithm determines
the running time in big-Oh notation
To perform the asymptotic analysis
• We find the worst-case number of primitive operations
executed as a function of the input size
• We express this function with big-Oh notation

Example:
• We determine that algorithm arrayMax executes at most 7n 
2 primitive operations
• We say that algorithm arrayMax “runs in O(n) time”

Since constant factors and lower-order terms are
eventually dropped anyhow, we can disregard them
when counting primitive operations. This means a bit
of slop is OK as long as you don’t mess up the counts
involving n
26
Computing Prefix Averages
We further illustrate
asymptotic analysis with
two algorithms for prefix
averages
 The i-th prefix average
of an array X is the
average of the first (i + 1)
elements of X:
A[i]  (X[0] + X[1] + … + X[i])/(i+1)


Computing the array A
of prefix averages of
another array X has
many applications.
35
30
X
A
25
20
15
10
5
0
1 2 3 4 5 6 7
27
Computing Prefix Averages

One application of the prefix average
calculation:
• Given the year-by-year returns on a stock, see
what the stock’s average annual returns were
for last year, the last three years, the last ten
years, etc.

As you can see by the previous graph, it
has the effect of smoothing out the data
especially when that data is varying a lot.
28
Prefix Averages (Quadratic)

The following algorithm computes prefix
averages in quadratic time by applying the
definition rather directly
Algorithm prefixAverages1(X, n)
Input array X of n integers
Output array A of prefix averages of X #operations
A  new array of n integers
for i  0 to n  1 do
n
s  X[0]
n
for j  1 to i do
1 + 2 + …+ (n  1)
s  s + X[j]
1 + 2 + …+ (n  1)
A[i]  s / (i + 1)
n
return A
1
29
Arithmetic Progression


The running time of
prefixAverages1 is
O(1 + 2 + …+ n)
The sum of the first n
integers is n(n + 1) / 2
• There is a simple
visual proof of this fact


Thus, algorithm
prefixAverages1 runs in
O(n2) time
Recall this sum is
Gauss’ formula that
was discussed in cpsc
171!
7
6
5
4
3
2
1
0
1
2
3
4
5
6
30
Prefix Averages (Linear)

The following algorithm computes prefix
averages in linear time by keeping a running
sum
Algorithm prefixAverages2(X, n)
Input array X of n integers
Output array A of prefix averages of X
A  new array of n integers
s0
for i  0 to n  1 do
s  s + X[i]
A[i]  s / (i + 1)
return A

#operations
Algorithm prefixAverages2 runs in O(n) time
1
n
n
n
1
31
Some math you will need





Summations (Sec. 1.3.1)
These often arise in analysis of algorithms
because they arise when loops are examined.
Some of these are standard ones seen (but
probably forgotten) in Algebra II in high school.
One, of course, is Gauss’ formula.
Another is the geometric summation
n
•i=0 ai = (1-an+1)/(1-a)
You should know the special case of
1 + 2 + 4 + 8 +...+ 2n-1 = 2n-1, the largest
interger representable in n bits.
32
Logarithms and Exponents (Sec. 1.3.2)


Definition: logba = c of a = bc
properties of logarithms:
logb(xy) = logbx + logby
logb (x/y) = logbx - logby
logbxa = alogbx
logba = logxa/logxb
properties of exponentials:
a(b+c) = aba c
abc = (ab)c
ab /ac = a(b-c)
b = a logab
bc = a c logab
33
Relatives of Big-Oh


big-Omega
• f(n) is (g(n)) if there is a constant c > 0
and an integer constant n0  1 such that
f(n)  cg(n) for n  n0
Intuitively, this says up to a constant factor, f(n)
asymptotically is greater than or equal to g(n)
big-Theta
• f(n) is (g(n)) if there are constants c’ > 0 and
c’’ > 0 and an integer constant n0  1 such that
c’g(n)  f(n)  c’’•g(n) for n  n0
Intuitively, this says up to a constant factor, f(n)
and g(n) are asymptotically the same.
34
Note: This is the concept I introduced in cpsc 171.
Relatives of Big-Oh


little-oh
• f(n) is o(g(n)) if, for any constant c > 0, there
is an integer constant n0  0 such that f(n) 
cg(n) for n  n0
Intuitively, this says f(n) is, up to a constant,
asymptotically strictly less than g(n).
little-omega
• f(n) is (g(n)) if, for any constant c > 0, there
is an integer constant n0  0 such that f(n) 
cg(n) for n  n0
Intuitively, this says f(n) is, up to a constant,
asymptotically strictly greater than g(n).
These are not used as much as the earlier
35
definitions, but they round out the picture.
Summary for Intuition for
Asymptotic Notation
Big-Oh
• f(n) is O(g(n)) if f(n) is asymptotically less than or equal to
g(n)
big-Omega
• f(n) is (g(n)) if f(n) is asymptotically greater than or
equal to g(n)
big-Theta
• f(n) is (g(n)) if f(n) is asymptotically equal to g(n)
little-oh
• f(n) is o(g(n)) if f(n) is asymptotically strictly less than g(n)
little-omega
• f(n) is (g(n)) if is asymptotically strictly greater than g(n)
36
Example Uses of the
Relatives of Big-Oh

5n2 is (n2)

f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0  1
such that f(n)  cg(n) for n  n0
let c = 5 and n0 = 1
5n2 is (n)

f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0  1
such that f(n)  cg(n) for n  n0
let c = 1 and n0 = 1
5n2 is (n)
f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n0 
0 such that f(n)  cg(n) for n  n0
need 5n02  c•n0  given c, the n0 that satisfies this is n0  c/5  0
37
A MORE PRECISE DEFINITION OF O, 
(only for those with calculus backgrounds)
Definition: Let f and g be functions defined on
the positive real numbers with real values.
We say g is in O(f) if and only if
lim
g(n)/f(n) = c
n -> 
for some nonnegative real number c--- i.e. the
limit exists and is not infinite.
We say f is in (g) if and only if
f is in O(g) and g is in O(f)
Note: Often to calculate these limits you need
L'Hopital's Rule.
38
Why Asymptotic Behavior is Important



1) Allows us to compare two algorithms
with respect to running time on large data
sets.
2) Helps us understand the maximum size
of input that can be handled in a given
time, provided we know the environment
in which we are running.
3) Stresses the fact that even dramatic
speedups in hardware do not overcome
the handicap of an asymtotically slow
algorithm.
39
ORDER WINS OUT
The TRS-80
Main language support: BASIC - typically a slow running
language
For more details on TRS-80 see:
http://mate.kjsl.com/trs80/
The CRAY-YMP
Language used in example: FORTRAN- a fast running language
For more details on CRAY-YMP see:
http://ds.dial.pipex.com/town/park/abm64/CrayWWWStuff/Cfaq
p1.html#TOC3
40
CRAY YMP
with FORTRAN
complexity is 3n3
n is:
TRS-80
with BASIC
complexity is 19,500,000n
microsecond (abbr µsec) One-millionth of a second.
millisecond (abbr msec) One-thousandth of a second.
10
3 microsec
100
3 millisec
200 millisec
2 sec
1000
3 sec
20 sec
2500
50 sec
50 sec
10000
49 min
3.2 min
1000000
95 years
5.4 hours
41
Proof techniques (Sec. 1.3.3)

By example
• You often hear profs say “you can’t prove by
an example”.
• That is true unless you have some very special
situations.

Situation 1: Prove that there is an element
x in a set S that has property P.
• To prove this, you need to



Define one element x
Show x is in the set S
Show x has property P
42
Proof techniques (Sec. 1.3.3)

Situation 2: Show that the claim “Every element
x in a set S has property P” is false.
• Now all you have to do is produce one counterexample
to the statement.
• i.e. define an element x
• Show x is in the set S
• Show x does NOT have property P

Example: Prove that the statement every number
of the form 2i – 1, for i >= 1, is prime is false.
• Let i= 4. Then 24 -1 = 15 = (3)(5) is not prime.
• Note- the choice of i is not unique. You need only find
one i which makes the statement false.
43
Proof techniques (Sec. 1.3.3)

Many theorems are of the form
• If P, then Q.
There are several approaches for proving this.

A direct proof does the following:
• Assume P is true.
• Using that information, deduce using known facts
that Q is true.
• Students often wonder why you assume P is true.

Logic dictates the following truth table:
P
T
T
F
F
Q
T
F
T
F
PQ
T
F
T
T
The last two entries often seem
strange to students. See class demo
which shows why those entries are
specified.
44
Proof techniques (Sec. 1.3.3)

A contrapositive proof does the following:
• The contrapositive of “If P, then Q” is
If not Q, then not P
• The contrapositive is logically equivalent to the
original, i.e. both have the same truth table:
P Q not Q not P not Q -> not P
T T
F
F
T
T F
T
F
F
F T
F
T
T
F F
T
T
T
• Now, assume not Q and deduce not P.
45
Proof techniques (Sec. 1.3.3)

A proof by contradiction does the
following:
• Assume P is true and Q is false.
• With these assumptions, try to reach a
contradiction--- i.e. logically deduce some
statement R and the statement not R.
• As we would always assume P is true when
proving “If P, then Q”, then our assumption
that Q is false must be wrong --- i.e. Q must
be true
46
Proof techniques (Sec. 1.3.3)

Which is approach is best?
• Direct proof
• Contrapositive proof
• Proof by contradiction



None of these- all are logically equivalent.
Which you choose depends upon the way you see
the proof.
Remember, a proof just tries to show others what
(acceptable) reasoning you used to draw a
conclusion. Therefore, any of these approaches
are fine.
47
Proof techniques (Sec. 1.3.3)



The age old question is, “What can I
assume is true when I am doing a proof?”
Normally, you don’t start from Peano’s
axioms for the integers and build
everything else from those (although, that
is an interesting approach to try sometime
in your lifetime!)
You need to use some judgment as to
what the course is assuming and what I
am asking you to prove.
48
Proof techniques (Sec. 1.3.3)

Many analyzes of complexity involve
statements of the form
• Q(n) is true for all n ≥ 1 (or for all n ≥ k, for
some k where n is an integer.
• This is a claim about an infinite set of
numbers.
• When the integers are defined using Peano’s
axioms, one of the axioms is the Induction
Principle which is assumed in one of two
equivalent forms:
49
Induction Principle


Let Q(n) be a statement about integers n and we
are to prove that “Q(n) is true for all integers n ≥
k, where k is an integer.”
One form of the Induction Principle says the
above is true if we can prove the following:
• 1) Q(k) is true
• 2) If Q(n) is true for i < n, then Q(n) is true.

An alternate form replaces (2) with
• 2) If Q(n) is true, then Q(n + 1) is true.

These forms are logically equivalent even though
the first form is called (unfortunately) strong
induction and the second form is called weak
50
induction.
Induction and Proofs Involving Recursion



Many algorithms use recursion instead of
iteration for control.
Analyzing these algorithms often entail a timing
function that is expressed as a recurrence
relation.
Example: (A recursive solution for finding the
maximum number in an array)
• Details are on page 12, but the basic step is
if n = 1 then
return A[0]
else return max {recursiveMax(A,n-1), A[n-1]}
where max is the maximum function for two elements.
51
Analyze the recursiveMax algorithm


Count every comparison, array reference,
recursive call, max calculation, or return as a
single primitive.
T(n) =
3 if n = 1 [check, access, return]
T(n-1) + 7 otherwise

[T(n-1) is the timing for the recursive call;
check n<> 1, access A[n-1], compute max (4
operations- if a < b return b else return a),
return ]
This formula is called a recurrence relation.
52
Induction and Proofs Involving Recursion





Often we want a closed form (i.e. an explicit
formula) for the recurrence relation.
Finding the closed form MAY be difficult.
But, after finding a possible candidate, we often
have to prove it is correct.
For example, we claim that the recurrence
relation derived for recursiveMax is the closed
form T(n) = 7n -4.
To show this is correct, an induction proof is often
used.
53
Proof techniques (Sec. 1.3.3)



Loop Invariants- a technique used to prove
statements about loops.
To prove some statement S about a loop, define
S in terms of a series of simpler statements S0,
S1, ..., Sk where
• 1) The initial claim S0 is true before the loop
begins
• 2) If Si-1 is true before iteration i begins, then
one can show that Si is true after iteration is is
over
• 3) The final statement, Sk, implies the
statement S is true.
Use this technique to prove that the arrayMax
algorithm is correct.
54
Basic probability (Sec. 1.3.4)



We will introduce these concepts later as
needed.
They are used predominately when we use
random data to analyze average cases.
The average case analysis is almost
always much more difficult than the worst
case or best case analysis.
• Some algorithms have not had there average
cases analyzed yet--- an example is an
interesting sort called the Shell Sort.
55
Amortization Techniques (Sect. 1.5)



We will postpone these, also, for a bit
later.
They are just now coming into use in
computer science.
The techniques come from financial
analysis and provide a way of performing
average-case analysis without using
probability theory.
56
Experimentation (Sect 1.6)





Asymptotic analysis is powerful, but does have
some limitations.
It does not provide insight into the constant
factors hiding behind the big-oh notation.
Another problem is that most analysis deals with
the worst case, as the average case is often
unbelievably difficult to analyze.
There may be other questions we wish to answer
about the algorithm other than what its worst
case is.
Some algorithms are just too complicated to even
find bounds for their performance.
57
First, focus on what question you want to
answer
Do we want to estimate the asymptotic running
time for the average case?
 Do we wish to see which of two algorithms is
faster for a given range of values?
 If an algorithm has constants that affect its
behavior, can we determine which ones yield the
best performance?
 If a function is supposed to minimize or maximize
its input, can we see how close it comes to its
optimal value?
You may choose different experiments depending
on the question.

58
Second, decide what you want to measure

Running time is often one of the poorest
choices as it is affected by so many
factors.
• However, this is a useful measure in some
cases.

Other possibilities:
•
•
•
•
Count memory references
Count comparisons
Count arithmetic operations
Count any operation (or operations) that seem
to contribute to the work of the algorithm.
59
Third, use good practices in choosing test
data



You want data that covers your possible
input space well.
Do not assume random data is what you
always need for average data.
You may need to analyze your input space
and determine the probability of certain
kinds of data occuring.
• Ex. How likely is it that quicksort will
encounter already sorted data?
60
Fourth, code the algorithm(s) and record
environmental data

A common problem with experimental
data is a poor implementation.
• Ex. Quicksort

You need to record carefully the
environmental data:
•
•
•
•
•
CPU type and speed
Number of CPUs
Memory size (including main and cache)
Disk speed (if external input from disk is used)
Memory bus speed
61
Fifth, present the resulting data in a useful
manner--- i.e. perform good data analysis
and visualization


A basic principle: Viewing data in tables is not as
useful as showing graphical plots.
Two useful methods are:
• The ratio test –


Tries to derive a function f(n) = nc for the main term in the
algorithm’s running time, for some constant c > 0.
Only useful for polynomial running times.
• The power test –


Can produce a good estimate for the running time t(n) of
an algorithm without producing a good guess for that
running time in advance.
Both these tests can at best achieve an accurate
c in the range [c=0.5, c+0.5], but no better.
62
Such a range, however, is not guaranteed.
The Ratio Test




Derive a function f(n) = nc, for c > 0, for
the main term in the algorithm.
We will test experimentally whether the
algorithm is Θ(nc) or not.
Let t(n) be the actual running time of the
algorithm on a specific problem instance
of size n. (You might want this to be the
average over several different types of
data of size n).
Plot the ratio r(n) = t(n)/f(n).
63
The Ratio Test- plotting r(n) = t(n)/f(n)

If r(n) increases as n increases, then
• f(n) underestimates the running time t(n)

If r(n) converges to 0, then
• f(n) is an overestimate.

If r(n) converges to some constant b >0,
then
• we have a good estimate for the growth rate of
t(n),
• Moreover, the constant b is a good estimate for
the constant factor in the running time t(n).

See example.
64
The Ratio Test- plotting r(n) = t(n)/f(n)
Problems


Can mislead you- for example, may look like it is
converging to 0, but with small data sets, you
may not see this.
In the selection sort example, working with n2,
the constant is ½, but on a small data set, it
looked as if convergence was to 0
• Recall- even with the best possibility, a 0.5 error is
possible.


We can rule out Θ(n) and Θ(n3) as possibilities,
however.
With a non-polynomial algorithm, all you will get
is an upper bound.
• Is it useful to know that quicksort is not Θ(n), but could
be Θ(n2) ?
65
The Power Test






With this approach, we don’t need to make an
estimate in advance of the complexity.
Given a collection of data points (x,y) where x is
the input size and y is t(x) from experimental
runs, plot (log x, log y).
If t(n) = bnc and (log x, log y) is (x’,y’), then y’ =
cx’ + b.
So, if we plot on log-log paper, we can roughly
determine b and c, if the plot is a straight line.
If the pairs grow without appearing to be a line,
then t(n) is most likely super-polynomial.
If the pairs converge to a constant, then t(n) is
most likely sublinear.
66
Example
1,000,000
n^2
100n
100,000
10n
n
10,000
1,000
100
10
1
1
10
100
n
1,000
67
Cautions on Using Experimental Analysis





You must choose good input test cases.
For a given value n, it is often wise to average
over different types of data.
You must plot enough points to see if something
is converging or not.
Even with the best of data, the accuracy is not
exact.
Nevertheless, the ratio test and the power test
are better than trying statistically to fit a
polynomial to data points by regression methods
which are very sensitive to noise.
68