Transcript Algorithms and Data Structures

Algorithms and Data
Structures
Lecture III
Simonas Šaltenis
Aalborg University
[email protected]
September 22, 2003
1
This Lecture

Divide-and-conquer technique for
algorithm design. Example problems:



Tiling
Searching (binary search)
Sorting (merge sort).
September 22, 2003
2
Tiling
A tromino tile:
And a 2nx2n board
with a hole:
September 22, 2003
A tiling of the board
with trominos:
3
Tiling: Trivial Case (n = 1)


Trivial case (n = 1): tiling a 2x2 board with
a hole:
Idea – try somehow to reduce the size of
the original problem, so that we eventually
get to the 2x2 boards which we know how
to solve…
September 22, 2003
4
Tiling: Dividing the Problem

To get smaller square boards let’s divide
the original board into for boards


September 22, 2003
Great! We have one problem
of the size 2n-1x2n-1!
But: The other three
problems are not similar to
the original problems – they
do not have holes!
5
Tiling: Dividing the Problem

Idea: insert one tromino at the center to
get three holes in each of the three smaller
boards


September 22, 2003
Now we have four boards with
holes of the size 2n-1x2n-1.
Keep doing this division, until
we get the 2x2 boards with
holes – we know how to tile
those
6
Tiling: Algorithm
INPUT: n – the board size (2nx2n board), L – location of the hole.
OUTPUT: tiling of the board
Tile(n, L)
if n = 1 then
Trivial case
Tile with one tromino
return
Divide the board into four equal-sized boards
Place one tromino at the center to cut out 3 additional
holes
Let L1, L2, L3, L4 denote the positions of the 4 holes
Tile(n-1, L1)
Tile(n-1, L2)
Tile(n-1, L3)
Tile(n-1, L4)
September 22, 2003
7
Divide and Conquer

Divide-and-conquer method for algorithm
design:

If the problem size is small enough to solve it
in a straightforward manner, solve it. Else:



Divide: Divide the problem into two or more
disjoint subproblems
Conquer: Use divide-and-conquer recursively to
solve the subproblems
Combine: Take the solutions to the subproblems
and combine these solutions into a solution for the
original problem
September 22, 2003
8
Tiling: Divide-and-Conquer

Tiling is a divide-and-conquer algorithm:




Just do it trivially if the board is 2x2, else:
Divide the board into four smaller boards
(introduce holes at the corners of the three
smaller boards to make them look like original
problems)
Conquer using the same algorithm recursively
Combine by placing a single tromino in the
center to cover the three introduced holes
September 22, 2003
9
Binary Search

Find a number in a sorted array:



Just do it trivially if the array is of one element
Else divide into two equal halves and solve each half
Combine the results
INPUT: A[1..n] – a sorted (non-decreasing) array of integers, s – an integer.
OUTPUT: an index j such that A[j] = s. NIL, if "j (1jn): A[j] s
Binary-search(A, p, r, s):
if p = r then
if A[p] = s then return p
else return NIL
q(p+r)/2
ret  Binary-search(A, p, q, s)
if ret = NIL then
return Binary-search(A, q+1, r, s)
else return ret
September 22, 2003
10
Recurrences


Running times of algorithms with Recursive
calls can be described using recurrences
A recurrence is an equation or inequality that
describes a function in terms of its value on
smaller inputs
solving_trivial_problem
if n  1

T (n)  
num_pieces T (n / subproblem_size_factor)  dividing  combining if n  1

Example: Binary Search
(1)
if n  1

T (n)  
2T (n / 2)  (1) if n  1
September 22, 2003
11
Binary Search (improved)


T(n) = (n) – not better than brute force!
Clever way to conquer:

Solve only one half!
INPUT: A[1..n] – a sorted (non-decreasing) array of integers, s – an integer.
OUTPUT: an index j such that A[j] = s. NIL, if "j (1jn): A[j] s
Binary-search(A, p, r, s):
if p = r then
if A[p] = s then return p
else return NIL
q(p+r)/2
if A[q]  s then return Binary-search(A, p, q, s)
else return Binary-search(A, q+1, r, s)
September 22, 2003
12
Running Time of BS
(1)
if n  1

T (n)  
T (n / 2)  (1) if n  1

T(n) = (lg n) !
September 22, 2003
13
Merge Sort Algorithm



Divide: If S has at least two elements (nothing
needs to be done if S has zero or one elements),
remove all the elements from S and put them
into two sequences, S1 and S2 , each containing
about half of the elements of S. (i.e. S1 contains
the first n/2elements and S2 contains the
remaining n/2elements).
Conquer: Sort sequences S1 and S2 using Merge
Sort.
Combine: Put back the elements into S by
merging the sorted sequences S1 and S2 into one
sorted sequence
September 22, 2003
14
Merge Sort: Algorithm
Merge-Sort(A, p, r)
if p < r then
q(p+r)/2
Merge-Sort(A, p, q)
Merge-Sort(A, q+1, r)
Merge(A, p, q, r)
Merge(A, p, q, r)
Take the smallest of the two topmost elements of
sequences A[p..q] and A[q+1..r] and put into the
resulting sequence. Repeat this, until both sequences
are empty. Copy the resulting sequence into A[p..r].
September 22, 2003
15
MergeSort (Example) - 1
September 22, 2003
16
MergeSort (Example) - 2
September 22, 2003
17
MergeSort (Example) - 3
September 22, 2003
18
MergeSort (Example) - 4
September 22, 2003
19
MergeSort (Example) - 5
September 22, 2003
20
MergeSort (Example) - 6
September 22, 2003
21
MergeSort (Example) - 7
September 22, 2003
22
MergeSort (Example) - 8
September 22, 2003
23
MergeSort (Example) - 9
September 22, 2003
24
MergeSort (Example) - 10
September 22, 2003
25
MergeSort (Example) - 11
September 22, 2003
26
MergeSort (Example) - 12
September 22, 2003
27
MergeSort (Example) - 13
September 22, 2003
28
MergeSort (Example) - 14
September 22, 2003
29
MergeSort (Example) - 15
September 22, 2003
30
MergeSort (Example) - 16
September 22, 2003
31
MergeSort (Example) - 17
September 22, 2003
32
MergeSort (Example) - 18
September 22, 2003
33
MergeSort (Example) - 19
September 22, 2003
34
MergeSort (Example) - 20
September 22, 2003
35
MergeSort (Example) - 21
September 22, 2003
36
MergeSort (Example) - 22
September 22, 2003
37
Merge Sort Summarized

To sort n numbers




if n=1 done!
recursively sort 2 lists of
numbers n/2 and n/2
elements
merge 2 sorted lists in (n)
time
Strategy



break problem into similar
(smaller) subproblems
recursively solve
subproblems
combine solutions to answer
September 22, 2003
38
Running time of MergeSort

Again the running time can be expressed
as a recurrence:
solving_trivial_problem
if n  1

T (n)  
num_pieces T (n / subproblem_size_factor)  dividing  combining if n  1
(1)
if n  1

T (n)  
2T (n / 2)  (n) if n  1
September 22, 2003
39
Repeated Substitution Method

Let’s find the running time of merge sort (let’s
assume that n=2b, for some b).
1
if n  1

T (n)  
2T (n / 2)  n if n  1
T (n)  2T  n / 2   n substitute
 2  2T  n / 4   n / 2   n expand
 22 T (n / 4)  2n substitute
 22 (2T ( n / 8)  n / 4)  2n expand
 23 T (n / 8)  3n
observe the pattern
T (n)  2i T (n / 2i )  in

September 22, 2003
2lg n T (n / n)  n lg n  n  n lg n
40
Example: Finding Min and Max

Given an unsorted array, find a minimum
and a maximum element in the array
INPUT: A[l..r] – an unsorted array of integers, l r.
OUTPUT: (min, max) such that "j (ljr): A[j]  min and A[j]  max
MinMax(A, l, r):
if l = r then return (A[l], A[r])
Trivial case
q(l+r)/2Divide
(minl, maxl) MinMax(A, l, q)
Conquer
(minr, maxr) MinMax(A, q+1, r)
if minl < minr then min = minl else min = minr
Combine
if maxl > maxr then max = maxl else max = maxr
return (min, max)
September 22, 2003
41
Next Week

Analyzing the running time of recursive
algorithms (such as divide-and-conquer)
September 22, 2003
42