Transcript Document

CSCE350 Algorithms and Data
Structure
Lecture 5
Jianjun Hu
Department of Computer Science and Engineering
University of South Carolina
2009.9.
Outline
 How to analyze the Time Efficiency of non-recursive
algorithms
 How to analyze time efficiency of recursive algorithms
Analyze the Time Efficiency of An
Algorithm
Nonrecursive Algorithm
ALGO RITHMFactorial(n )
f 1
Recursive Algorithm
for i  1 to n do
f  f *i
re turn f
ALGO RITHMFactorial(n )
if n  0
re turn1
e lse
re turnFactorial(n  1)* n
Element Uniqueness
Check whether all the elements in a given array are distinct
 Input: An array A[0…n-1]
 Output: Return “true” if all the elements in A are distinct and
“false” otherwise
ALGO RITHMUniqueElements( A[0..n  1])
for i  0 to n-2 do
for j  i  1 to n-1 do
if A[i ]  A[ j ] re turnfalse
re turntrue
n  2 n 1
2
1
C
(
n
)


(
n
)
  worst
i 0 j i 1
Selection sort
Cworst (n)  Cbest (n)  Caverage (n)  (n )
2
Insertion Sort
Cworst (n)  (n ), Cbest (n)  (n), Caverage (n)  ?
2
Insert Sorting—running time cases
 The best case input is an array that is already sorted. In this case
insertion sort has a linear running time (i.e., O(n)). During each
iteration, the first remaining element of the input is only
compared with the right-most element of the sorted subsection of
the array.
 The worst case input is an array sorted in reverse order. In this
case every iteration of the inner loop will scan and shift the entire
sorted subsection of the array before inserting the next element.
For this case insertion sort has a quadratic running time (i.e.,
O(n2)).
 The average case is also quadratic, which makes insertion sort
impractical for sorting large arrays. However, insertion sort is one
of the fastest algorithms for sorting arrays containing fewer than
ten elements.
Improvement of Insert sorting
 Shell sort: two simple variants requiring O(n3/2) and O(n4/3)
running time.
 binary insertion sort
 In 2004 Bender, Farach-Colton, and Mosteiro published a
new variant of insertion sort called library sort or gapped
insertion sort high probability in O(n log n) time
Example: Find the Number of Binary
Digits
Find the Number of Binary Digits in the Binary Representation
of a Positive Decimal Integer
ALGO RITHMBinary( n )
// In pu t: A positivede cim alin te ge rn
// O u tpu t: Th e n u m be rof bin arydigits
//
i n n' s bin aryre pre se n ta
tion
count  1
wh ilen  1 do
count  count  1
n  n/ 2
re tu rncount
How to identify inefficiency and speed
it up?
What is it doing?
Time complexity?
How to speed it up?
How to identify inefficiency and speed
it up?
2.4.7 Matrix multiplication
For i0 to n-1 do
n 1 n 1 n 1
for j 0 to n-1 do
C[i,j]0
i 0 j 0 k 0
for k0 to n-1 do
C[i,j] C[i,j]+A[i,k]*B[k,j]
1
n3    n3  n2
Example Recursive evaluation of n !
 Recursive algorithm for n!
ALGO RITHMFactorial(n )
 Input size: n
if n  0
 Basic operation: multiplication
re turn1
e lse
“*”
 Let M(n) be the number of
multiplications needed to
compute n!, then
re turnFactorial(n  1)* n
M (0)  0
M (n)  M (n  1)  1 for n  0
To compute Factorial(n-1)
To multiply Factorial(n-1) by n
Solve the Recurrence
 Therefore, the number of
M ( 0)  0
M ( n )  M ( n  1)  1 for n  0
 M (n )
 M ( n  2)  2
 M ( n  3)  3
 ...
 M (n  n)  n
n
multiplications needed to
compute n! in this
algorithm is n.
  The complexity of this
algorithm is (n )
Time Efficiency of Recursive
Algorithms
 Steps in mathematical analysis of recursive algorithms:
 Decide on parameter n indicating input size
 Identify algorithm’s basic operation
 Determine worst, average, and best case for input of size n
 Set up a recurrence relation and initial condition(s) for C(n)-
the number of times the basic operation will be executed for
an input of size n (alternatively count recursive calls).
 Solve the recurrence to obtain a closed form or estimate the
order of magnitude of the solution (see Appendix B)
Another Example: Tower of Hanoi
 n different-size disks, 3 pegs, move disks from the left peg to
the right one using the middle one as an auxiliary
 Rules: you can move one disk each time and it is forbidden to
place a larger disk on top of a smaller one
 Design and algorithm and analyze its complexity
Recursive Algorithm
 Input size: n (disks)
 Basic operation: one move of a disk
 Initial condition: n=1  only one direct move
 To build the recurrence: suppose you have a way to move n-1
disks.
 Then you can move the top n-1 disks from the left peg to the
middle peg using the right peg as an auxiliary.
 Move the bottom disk from the left peg to the right peg.
 Move n-1 disks from the middle peg to the right peg using the
left one as an auxiliary.
Illustration
1
3
2
Algorithm Complexity
 Let M(n) be the number of needed moves
 Initialization M(1)=1
 Recurrence
M (n)  M (n  1)  1  M (n  1)
M (n )  2 M ( n  1)  1
for n  1
for n  1
 2[2 M ( n  2)  1]  1  22 M (n  2)  2  1
 22 [2 M (n  3)  1]  2  1  23 M ( n  3)  22  2  1
 ...
n 2
 2n 1 M (1)   2i  2n 1  2n 1  1  2n  1
i 0
Important Recurrence Types:

One (constant) operation reduces problem size by one.
T(n) = T(n-1) + c
T(1) = d
Solution: T(n) = (n-1)c + d

A pass through input reduces problem size by one.
T(n) = T(n-1) + cn
T(1) = d
Solution: T(n) = [n(n+1)/2 – 1] c + d

quadratic
One (constant) operation reduces problem size by half.
T(n) = T(n/2) + c
Solution: T(n) = c lg n + d

linear
T(1) = d
logarithmic
A pass through input reduces problem size by half.
T(n) = 2T(n/2) + cn
Solution: T(n) = cn lg n + d n
T(1) = d
n log n
A General Divide-and-Conquer Recurrence:
Master Theorem

T(n) = aT(n/b) + f (n)
where f (n) ∈ Θ(nk)
a < bk
a = bk
T(n) ∈ Θ(nk)
T(n) ∈ Θ(nk lg n )
a > bk
T(n) ∈ Θ(nlog b a)

Note: the same results hold with O instead of Θ.
Example: Find the Number of
Binary Digits (Recursive Algorithm)
 Find the Number of Binary Digits in the Binary
Representation of a Positive Decimal Integer using a
recursive algorithm
ALGO RITHMBinRec( n )
// In pu t: A posi ti vede ci m ali n te ge rn
// O u tpu t: Th e n u m be rof bi n arydi gi ts
//
i n n' s bi n aryre pre se n ta
ti on
i f n  1 re tu rn1
e l sere tu rnBinRec( n/ 2)  1
 Recurrence
A(n)  A( n/ 2)  1, for n  1
A(1)  0
A(n)  log2 n  (logn)