Transcript CSC201Lecture04_BigO

CSC 201
Analysis and Design of Algorithms
Lecture 04:
Time complexity analysis in form of
Big-Oh
Dr.Surasak Mungsing
E-mail: [email protected]
Apr-15
1
Big-Oh Notation
 Big-Oh was introduced for functions’ growth rate
comparison in 1927 based on asymptotic
behavior
 Big-Oh notation characterizes functions
according to their growth rates: different
functions with the same growth rate may be
represented using the same O notation.
 A description of a function in terms of big-Oh
notation usually only provides an upper bound
on the growth rate of the function.
8-Apr-15
CSC201 Analysis and Design of Algorithms
2
Upper Bound
 In general a function
f(n) is O(g(n)) if  positive constants c and n0 such that
f(n)  c  g(n)  n  n0
e.g. if f(n)=1000n and g(n)=n2, n0 > 1000 and c = 1 then
f(n0) < 1.g(n0) and we say that f(n) = O(g(n))
 The O notation indicates 'bounded above by a constant
multiple of.'
8-Apr-15
CSC201 Analysis and Design of Algorithms
3
Big-Oh, the Asymptotic Upper Bound
 Because big O notation discards multiplicative constants on the running
time, and ignores efficiency for low input sizes, it does not always reveal
the fastest algorithm in practice or for practically-sized data sets, but the
approach is still very effective for comparing the scalability of various
algorithms as input sizes become large.
 If an algorithm‘s time complexity is in the order of O(n2) then it’s growth
rate of computation time will not be faster than a quadratic function for
input that is large enough
 Some upper bounds may be too broad, for example saying that 2n2 =
O(n3)

 By definition if c = 1 and n0 = 2, it is better to say that 2n2 = O(n2)
8-Apr-15
CSC201 Analysis and Design of Algorithms
4
Example 1
For all n>6, g(n) > 1 f(n).
f (n) is in big-O of g(n)
then, f(n) is in O(g(n)).
8-Apr-15
CSC201 Analysis and Design of Algorithms
5
Example 2
There exists n0 such that for all n>n0,
If f(n) < 1 g(n) then f(n) is in O(g(n))
8-Apr-15
CSC201 Analysis and Design of Algorithms
6
Example 3
There exists n0=5, c=3.5, for all n>n0,
if f(n) < c h(n) then f(n) is in O(h(n)).
8-Apr-15
CSC201 Analysis and Design of Algorithms
7
8-Apr-15
CSC201 Analysis and Design of Algorithms
8
Exercise on O-notation
 Show that f(n)=3n2+2n+5 is in O(n2)
10 n2 = 3n2 + 2n2 + 5n2
 3n2 + 2n + 5
 f(n)
for n  1
or f(n) ≤10 n2
consider
c = 10, n0 = 1
f(n) ≤c g(n2 ) for n  n0
then f(n) is in O(n2)
8-Apr-15
CSC201 Analysis and Design of Algorithms
9
Usage of Big-Oh
 We should always write Big-Oh in the most simple form
e.g.
3n2+2n+5 = O(n2)
It is not wrong to write these functions in term of Big-Oh
as below, but the most appropriate form should be in
the most simple form
• 3n2+2n+5 = O(3n2+2n+5)
• 3n2+2n+5 = O(n2+n)
• 3n2+2n+5 = O(3n2)
8-Apr-15
CSC201 Analysis and Design of Algorithms
10
Exercise on O-notation




8-Apr-15
f1(n) = 10 n + 25 n2
f2(n) = 20 n log n + 5 n
f3(n) = 12 n log n + 0.05 n2
f4(n) = n1/2 + 3 n log n
•
•
•
•
O(n2)
O(n log n)
O(n2)
O(n log n)
CSC201 Analysis and Design of Algorithms
11
Classification of Function : BIG-Oh
 A function f(n) is said to be of at most logarithmic growth




if f(n) = O(log n)
A function f(n) is said to be of at most quadratic growth
if f(n) = O(n2)
A function f(n) is said to be of at most polynomial growth
if f(n) = O(nk), for some natural number k > 1
A function f(n) is said to be of at most exponential
growth if there is a constant c, such that
f(n) = O(cn), and c > 1
A function f(n) is said to be of at most factorial growth if
f(n) = O(n!).
8-Apr-15
CSC201 Analysis and Design of Algorithms
12
Classification of Function : BIG-Oh (cont.)
 A function f(n) is said to have constant running time if
the size of the input n has no effect on the running
time of the algorithm (e.g., assignment of a value to a
variable). The equation for this algorithm is f(n) = c
 Other logarithmic classifications: f(n) = O(n log n)
f(n) = O(log log n)
8-Apr-15
CSC201 Analysis and Design of Algorithms
13
Big O Fact
A polynomial of degree k is O(nk)
Proof:
ถ้า
และให้
ดังนั้น
8-Apr-15
f(n) = bknk + bk-1nk-1 + … + b1n + b0
ai = | b i |
f(n)  aknk + ak-1nk-1 + … + a1n + a0
CSC201 Analysis and Design of Algorithms
14
Some Rules
Transitivity
f(n) = O(g(n)) and g(n) = O(h(n))  f(n) = O(h(n))
Addition
f(n) + g(n) = O(max { f(n) ,g(n)})
Polynomials
a0 + a1n + … + adnd = O(nd)
Heirachy of functions
n + log n = O(n);
8-Apr-15
2n + n3 = O(2n)
CSC201 Analysis and Design of Algorithms
15
Some Rules
Base of Logs ignored
logan = O(logbn)
Power inside logs ignored
log(n2) = O(log n)
Base and powers in exponents not ignored
3n is not O(2n)
a(n)2 is not O(an)
8-Apr-15
CSC201 Analysis and Design of Algorithms
16
Big-Oh Complexity
 O(1) The cost of applying the algorithm can be bounded




independently of the value of n. This is called constant
complexity.
O(log n) The cost of applying the algorithm to problems
of sufficiently large size n can be bounded by a function
of the form k log n, where k is a fixed constant. This is
called logarithmic complexity.
O(n) linear complexity
O(n log n) n lg n complexity
O(n2) quadratic complexity
8-Apr-15
CSC201 Analysis and Design of Algorithms
17
Big-Oh Complexity (cont.)







O(n3) cubic complexity
O(n4) quadratic complexity
O(n32) polynomial complexity
O(cn) If constant c>1, then this is called exponential complexity
O(2n) exponential complexity
O(en) exponential complexity
O(n!) factorial complexity
8-Apr-15
CSC201 Analysis and Design of Algorithms
18
Practical Complexity t < 500
500
f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n
0
1 2 3
8-Apr-15
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
CSC201 Analysis and Design of Algorithms
19
Practical Complexity t < 5000
5000
4000
f(n) = n
f(n) = log(n)
3000
f(n) = n log(n)
f(n) = n^2
2000
f(n) = n^3
f(n) = 2^n
1000
0
1
8-Apr-15
3
5
7
9
11
13
15
17
19
CSC201 Analysis and Design of Algorithms
20
Practical Complexity
10000000
1000000
100000
10000
1000
100
10
1
1
8-Apr-15
4
16
64
256
1024
4096
16384 65536
CSC201 Analysis and Design of Algorithms
21
Things to Remember in Analysis
 Constants or low-order terms are ignored
if f(n) = 2n2 then f(n) = O(n2)
 running time and memory are important resources for
algorithm and very large input affects algorithm
performance the most
 Parameter N, normally means size of input
N may refers to degree of polynomial, size of
input file for data sorting, or number of nodes in
graph
8-Apr-15
CSC201 Analysis and Design of Algorithms
22
Things to Remember in Analysis
 Worst case analysis means the worst-case execution time
of particular concern (it is important to know how much
time might be needed in the worst case to guarantee that
the algorithm will always finish on time)
 Average performance ,and also worst-case), performance
is the most used in algorithm analysis, using probabilistic
analysis techniques, especially expected value, to
determine expected running times (i.e. the case of typical
input data)
8-Apr-15
CSC201 Analysis and Design of Algorithms
23
General Rules for Analysis (1)
1. Consecutive statements
8-Apr-15

count only the most time required of the
consecutive block of statements

count only the most time required of the
consecutive loops
Block #1
t1
Block #2
t2
t1+ t2 = max(t1,t2)
CSC201 Analysis and Design of Algorithms
24
General Rules for Analysis(2)
2.
If/Else
if cond then S1
else
S2
Block #1
t1
Block #2
t2
Max(t1,t2)
8-Apr-15
CSC201 Analysis and Design of Algorithms
25
General Rules for Analysis(3)
3. For Loops
 Running time of a for-loop is at most the
running time of the statements inside the forloop times number of iterations
for (i = sum = 0; i < n; i++)
sum += a[i];
 for loop iterates n times, executes 2 assignment
statements each iteration ==> asymptotic
complexity of O(n)
8-Apr-15
CSC201 Analysis and Design of Algorithms
26
General Rules for Analysis(4)
4. Nested For-Loops
Analyze inside-out: total time is the product of
time required for each loop
for (i =0; i < n; i++)
for (j = 0, sum = a[0]; j <= i ; j++)
sum += a[j];
printf("sum for subarray - through %d is %d\n", i, sum);
8-Apr-15
CSC201 Analysis and Design of Algorithms
27
General Rules for Analysis
8-Apr-15
CSC201 Analysis and Design of Algorithms
28
General Rules for Analysis
 Analysis strategy :
 analyze from inside out
 analyze function calls first
8-Apr-15
CSC201 Analysis and Design of Algorithms
29
8-Apr-15
CSC201 Analysis and Design of Algorithms
30