Transcript Document

Chapter 1 Introduction
Definition of Algorithm An algorithm is a finite sequence of precise
instructions for performing a computation or for solving a problem.
Example 1 Describe an algorithm for finding the maximum (largest) value in a finite
sequence of integers.
Solution
1.
Set the temporary maximum equal to the first integer in the sequence.
2.
Compare the next integer in the sequence to the temporary maximum, and set
the larger one to be temporary maximum.
3.
Repeat the previous step if there are more integers in the sequence.
4.
Stop when there are no integers left in the sequence. The temporary maximum
at this point is the maximum in the sequence.
Algorithm1. Finding theMaximumElementin a FiniteSequence
//Input: n integersa1,a2 ,...,an
// Output : max (themaximumof a1,a2 ,...,an )
max : a1 ;
for i:  2 to n
if max  ai thenmax : ai
returnmax
Instead of using a
particular computer
language, we use a form
of pseudocode.
Algorithmic Problem Solving
Understand the problem
Design an algorithm and proper data structures
Analyze the algorithm
Code the algorithm
• Ascertaining the Capabilities of a Computational Device
• Choosing between Exact and Approximate Problem Solving
• Deciding on Appropriate Data Structures
• Algorithm Design
• Algorithm Analysis
• Coding
Important Problem Types
• Sorting
• Searching
• String processing (e.g. string matching)
• Graph problems (e.g. graph coloring problem)
• Combinatorial problems (e.g. maximizes a cost)
• Geometric problems (e.g. convex hull problem)
• Numerical problems (e.g. solving equations )
Fundamental Data Structures
1. Array
2. Link List
a[1] a[2] a[3]
Head of list
a1
a[n]
a3
a2
an
10
3. Binary Tree
6
4
1
9
7
17
13
4
9
20
Fundamental Abstract Data Structures
1. Stack
an
• Operations a stack supports: last-in-first-out (pop, push)
• Implementation: using an array or a list (need to check
underflows and overflows)
a2
a1
2. Queue
• Operations a queue supports: first-in-first-out (enqueue, dequeue)
• Implementation: using an array or a list (need to check underflows
and overflows)
am am1 am2
front
an
rear
3. Priority Queue (each elements of a priority queue has a key)
• Operations a priority queue supports: inserting an element, returning an element
with maximum/minimum key.
• Implementation: heap
Assignment
(1) Read the sections of the text book about array, linked list, stack, queue,
heap and priority queue.
(2)
(i)
Implement stack S and queue Q using both of array and linked list.
(ii) Write a main function to do the following operations for S: pop, push 10
times, pop, pop, repeat push and pop 7 times. Do the following
operations for Q: dequeue, enqueue 10 times, dequeue, depueue, repeat
enqueue and dequeue 7 times. When you use array, declare the size of the
array to be 10. Printout all the elements in S and in Q.
Some Advanced Data Structure
1. Graph
• Operations a graph support: finding neighbors
• Implementation: using list or matrix
G  (V , E )
V  (a, b, c, d , e, f )
E  {( a, c), (b, c), (d , a), (c, e), (e, c), (b, f ), (d , e), (e, f )}
a
c
b
d
e
f
c
a
c
f
b
e
c
a
e
d
f
c
e
f
Adjacent lists
 a b c d e f


a 0 0 1 0 0 0 
b 0 0 1 0 0 1 


c 0 0 0 0 1 0 
d1 0 0 0 1 0 


e 0 0 1 0 0 1 


f
0
0
0
0
0
0


Adjacent matrix
2. Binary Search Tree
Definition 1 Binary Search Tree is a Binary Tree
satisfying the following condition:
15
(1) Each vertex contains an item called as key
which belongs to a total ordering set and two
links to its left child and right child,
respectively.
(2) In each node, its key is larger than the keys of
all vertices in its left subtree and smaller than
the keys of all the vertices in its right subtree.
6
3
1
18
7
17
13
4
9
Operations a binary search tree support: search, insert, and delete an element
with a given key.
How to select a data structure for a given problem?
20
Example: Select a data structure for supporting Dynamic
Dictionary
Definition 2
A Dynamic Dictionary is a data structure of item
with keys that support the following basic operations:
(1) Insert a new item
(2) Remove an item with a given key
(3) Search an item with a given key
What data structure
is the best?
Chapter 2 Fundamentals of the Analysis of Algorithm
Implementation and Empirical Analysis
Challenge in empirical analysis:
•
Develop a correct and complete implementation.
•
Determine the nature of the input data and other factors
influencing on the experiment. Typically, There are three
choices: actual data, random data, or perverse data.
•
Compare implementations independent to programmers,
machines, compilers, or other related systems.
•
Consider performance characteristics of algorithms, especially
for those whose running time is big.
Can we analyze algorithms that haven’t run yet?
Example 1 Describe an algorithm for finding an element x in a
list of distinct elements a1 , a2 ,...,an .
Algot hm1 T he linearSearch Algorithm.
P rocedurelinear search( x : integer,a1,a2 ,...,an : distinct integers)
i:  1;
while (i  n and x  ai )
i:  i  1;
if i  n thenlocation: i
else locatiton: 0;
{locationis thesubscript of
term thatequals x, or is 0 if x is not found}
Algorit hm2 T he BinarySearch Algorit hm.
P rocedurebinary search( x : int eger,a1,a2 ,...,an : increasingint egers)
i:  1;
j:  n;
while (i  j )
begin
m :  (i  j)/ 2;
if x  am t heni:  m  1
else j:  m;
end;
if x  ai t henlocation: i
else location: 0;
{locationis t hesubscript of
t erm t hatequals x, or is 0 if x is not found}
Linear Search and Binary Search written in C++
1.
2.
Linear search
int lsearch(int a[], int v, int l, int r)
{
for (int i=l; i<=n; i++)
if (v==a[i]) return i;
return -1;
}
Binary search
int bsearch(int a[], int v, int l, int n)
{
while (r>=l)
{int m=(l+r)/2;
if (v==a[m]) return m;
if (v<a[m]) r=m-1; else l=m+1;
}
return -1;
}
Complexity of Algorithms
Assume that both algorithms A and B solve the problem P. Which
one is better?
• Time complexity: the time required to solve a problem of
a specified size.
• Space complexity: the computer memory required to
solve a problem of a specified size.
The time complexity is expressed in terms of the
number of operations used by the algorithm.
• Worst case analysis: the largest number of operations
needed to solve the given problem using this algorithm.
• Average case analysis: the average number of
operations used to solve the problem over all inputs.
Example 2 Analyze the time complexities of linear search
algorithm and binary search algorithm
LinearSearch Algorithm.
P rocedurelinear search( x : integer,a1,a2 ,...,an : distinct integers)
i:  1;
while (i  n and x  ai )
i:  i  1;
if i  n thenlocation: i
else locatiton: 0;
{locationis thesubscript of
term thatequals x, or is 0 if x is not found}
1
2(n+1)
n
2
3n+5
BinarySearch Algorit hm.
P rocedurebinary search( x : int eger,a1,a2 ,...,an : increasingint egers)
i:  1;
j:  n;
1
1
while (i  j )
begin
1
1
m :  (i  j)/ 2;
if x  am t heni:  m  1
else j:  m;
end;
if x  ai t henlocation: i
2
Let n  2k.
It repeats at most k (k  log2 n) times.
2
else location: 0;
{locationis t hesubscript of
t erm t hatequals x, or is 0 if x is not found}
Number of operations = 4 log n+4
2
Orders of Growth
Running time for a problem with size n  106
Running
Time
Operation
Per second
necessary
lg n
n
operations
n
2
2n
6
instant
1 second
11.5 days
Never end
2 59350 days
12
instant
Instant
1 second
Never end
2 59340 days
10
10
Using silicon computer, no matter how fast CPU will be you can
never solve the problem whose running time is exponential !!!
Asymptotic Notations: O-notation
Definition 2.1 A function t(n) is said to be O(g(n)) if there exist
some constant c0  0 and n0  0 such thatt (n) c0 g (n) for all n  n0 .
c0 g (n)
t(n)
n0
If limn
n
t ( n)
 c (c  0 is a constant) , thent(n)  O(g(n)).
g ( n)
Example 3 Prove 2n+1=O(n)
Example4 Prove10n2  12n  5  O(n2 )
Example 5
List the following function in O-notation in increasing order:
lg n, n, n2 , n lg n, n3 , n!,2n.
Example 6 What is thebig - oh of the following functions?
5n5  100n 2 1000,
n lg n 2  100n lg n  1000n,
0.0001 2 n  n
Asymptotic Analysis of algorithms (using O-notation)
Example 7 Analyze the time complexities of linear search
algorithm and binary search algorithm asymptotically.
LinearSearch Algorithm.
P rocedurelinear search( x : integer,a1,a2 ,...,an : distinct integers)
i:  1;
while (i  n and x  ai )
It repeats n times.
i:  i  1;
if i  n thenlocation: i
else locatiton: 0;
{locationis thesubscript of
term thatequals x, or is 0 if x is not found} Totally(addition): O(n)
BinarySearch Algorit hm.
P rocedurebinary search( x : int eger,a1,a2 ,...,an : increasingint egers)
i:  1;
j:  n;
while (i  j )
begin
m :  (i  j)/ 2;
if x  am t heni:  m  1
else j:  m;
end;
if x  ai t henlocation: i
Let n  2k.
It repeats at most k (k  log2 n) times.
else location: 0;
{locationis t hesubscript of
t erm t hatequals x, or is 0 if x is not found} Totally(comparison): O(log n)
Example 8 Analyze the time complexities of following algorithm
asymptotically.
Matrix addition algorithm
Procedure MatricAddition(A[0..n-1,0..n-1],B[0..n-1,0..n-1])
for i=0 to n-1 do
for j=0 to n-1 do
C[i,j] = A[i,j] + B[i,j];
return C;
Repeat n times
Repeat n times
2
Totally(addition): O(n )
Recursive Algorithms
Example 9 Computing the factorial function F(n)=n!.
F(n) can be defined recursively as follows:
F (0)  1

F (n)  F (n  1)  n
Factorial Algorithm
Procedure factorial(n)
Algorithm factorial calls itself
in its body!
if n = 0 return 1
else return factorial(n-1) * n;
Time complexity(multiplication): T(0) = 0
T(n) = T(n-1) +1 when n>0
recurrence
Basic Recurrences
Example 10
Solving the following recurrence
T(0) = 1
T(n) = T(n-1) + 1
T(n) = T(n-1) + 1
n>0
Example11
Solve thefollowingrecurrence
Tn  2Tn / 2  n
n 1
T1  c'
T (n)  2T (n / 2)  cn
 2(2T (n / 2 2 )  c(n / 2))  cn  2 2 T (n / 2 2 )  cn  cn
2
3
2
3
3
= T(n-2) + 1 + 1 = T(n-2) + 2  2 (2T (n / 2 )  c(n / 2 ))  cn  cn  2 T (n / 2 )  3cn
= T(n-3) + 1 + 2 = T(n-3) + 3 ......
…
= T(n-i) + i
 2i T (n / 2i )  icn
......
…
 2 k T (n / 2 k )  kcn (k  log n)
 nT1  cn log n  c' cn log n
= T(n-n) + n
 O(n log n)
=n
Example 12 Solve the recurrence Example13 Solve therecurrence
T (n)  T (n  1)  n n  1
T ( n )  T ( n / 2)  1
n 1
T1  1
T (1)  1
(Assume thatn is a power of 2.
T (n)  T (n  1)  n
 T (n  2)  (n  1)  n
 T (n  3)  (n  2)  (n  1)  n
......
T 12  3  ...  (n  2)  (n  1)  n
 1  2  3  ...  n
 n(n  1) / 2
T hatis, n  2 k , where k  log n).
T (n)  T (n / 2)  1
 T (n / 2 2 )  1  1
 T ( n / 23 )  1  1  1
......
 T ( n / 2i )  i
......
 T (n / 2 k )  k (k  log n)
 1  log n