Theory of Algorithms: Transform and Conquer James Gain and Edwin Blake
Download
Report
Transcript Theory of Algorithms: Transform and Conquer James Gain and Edwin Blake
Theory of Algorithms:
Transform and Conquer
James Gain and Edwin Blake
{jgain | edwin} @cs.uct.ac.za
Department of Computer Science
University of Cape Town
August - October 2004
Objectives
To introduce the transform-and-conquer mind set
To show a variety of transform-and-conquer
solutions:
Presorting
Horner’s Rule and Binary Exponentiation
Problem Reduction
To discuss the strengths and weaknesses of a
transform-and-conquer strategy
“The Secret of Life … to replace one worry with
another” - Charles M. Schultz
Transform and Conquer
1. In a transformation stage, modify the problem to
make it more amenable to solution
2. In a conquering stage, solve it
Often employed in mathematical problem solving
and complexity theory
Problem’s
Instance
Simpler Instance
OR
Another Representation
OR
Another Problem’s
Instance
Problem’s
Solution
Flavours of Transform and Conquer
1. Instance Simplification = a more convenient instance of the
same problem
Presorting
Gaussian elimination
2. Representation Change = a different representation of the
same instance
Balanced search trees
Heaps and heapsort
Polynomial evaluation by Horner’s rule
3. Problem Reduction = a different problem altogether
Reductions to graph problems
Presorting: Instance Simplification
Solve instance of problem by preprocessing the problem to
transform it into another simpler/easier instance of the
same problem
Many problems involving lists are easier when list is sorted:
Searching
Computing the median (selection problem)
Computing the mode
Finding repeated elements
Convex Hull and Closest Pair
Efficiency:
Introduce the overhead of an (n log n) preprocess
But the sorted problem often improves by at least one base
efficiency class over the unsorted problem (e.g., (n2) (n))
Example: Presorted Selection
Find the kth smallest element in A[1],…, A[n]
Special cases:
Minimum: k = 1
Maximum: k = n
Median: k = n/2
Presorting-based algorithm:
Sort list
Return A[k]
Partition-based algorithm (Variable Decrease & Conquer):
Pivot/split at A[s] using Partitioning algorithm from Quicksort
IF s=k RETURN A[s]
ELSE IF s<k repeat with sublist A[s+1],…A[n]
ELSE IF s>k repeat with sublist A[1],…A[s-1]
Notes on the Selection Problem
Presorting-based algorithm:
(n lgn) + (1) = (n lgn)
Partition-based algorithm (Variable decrease &
conquer):
Worst case: T(n) =T(n-1) + (n+1) (n2)
Best case: (n)
Average case: T(n) =T(n/2) + (n+1) (n)
Also identifies the k smallest elements (not just the kth)
Simpler linear (brute force) algorithm is better in
the case of max & min
Conclusion: Presorting does not help in this case
Finding Repeated Elements
Presorting algorithm for finding duplicated
elements in a list:
use mergesort: (n lgn)
scan to find repeated adjacent elements: (n)
(n lgn)
Brute force algorithm:
Compare each element to every other: (n2)
Conclusion: presorting yields significant
improvement
Similar improvement for mode
What about searching? What about amortised
searching?
Balancing Trees
Searching, insertion and deletion in a Binary
Search Tree:
Balanced = (log n)
Unbalanced = (n)
Must somehow enforce balance
Instance Simplification:
AVL and Red-black trees constrain imbalances by
restructuring trees using rotations
Representation Change:
2-3 Trees and B-Trees attain perfect balance by allowing
more than one element in a node
Heapsort: Representation Change
A heap is a binary tree with the following conditions:
It is essentially complete
The key at each node is ≥ keys at its children
The root has the largest key
The subtree rooted at any node of a heap is also a heap
Heapsort Algorithm:
1.
2.
3.
4.
Build heap
Remove root – exchange with last (rightmost) leaf
Fix up heap (excluding last leaf)
Repeat 2, 3 until heap contains just one node
Efficiency:
(n) + (n log n) = (n log n) in both worst and average cases
Unlike Mergesort it is in place
Horner’s Rule: Representation
Change
Addresses the problem of evaluating a polynomial
p(x) = an xn + an-1 xn-1 + … + a1x + a0 at a given point x = x0
Re-invented by W. Horner in early 19th Century
Approach: Convert to p(x) = (… (an x + an-1 ) x + …) x + a0
Algorithm:
Example:
p P[n]
FOR i n -1 DOWNTO 0 DO
p x p + P[i]
RETURN p
Q(x) = 2x3 - x2 - 6x + 5 at x = 3
P[ ]:
2
-1
-6
5
p:
2
3*2 + (-1) = 5
3*5 + (-6) = 9
3*9 + 5 = 32
Notes on Horner’s Rule
Efficiency:
Brute Force = (n2)
Transform and Conquer = (n)
Has useful side effects:
Intermediate results are coefficients of the quotient of
p(x) divided by x - x0
An optimal algorithm (if no preprocessing of
coefficients allowed)
Binary exponentiation:
Also uses ideas of representation change to calculate an
by considering the binary representation of n
Problem Reduction
If you need to solve a problem reduce it to another problem
that you know how to solve
Used in Complexity Theory to classify problems
Computing the Least Common Multiple:
The LCM of two positive integers m and n is the smallest integer
divisible by both m and n
Problem Reduction: LCM(m, n) = m * n / GCD(m, n)
Example: LCM(24, 60) = 1440 / 12 = 120
Reduction of Optimization Problems:
Maximization problems seek to find a function’s maximum.
Conversely, minimization seeks to find the minimum
Can reduce between: min f(x) = - max [ - f(x) ]
Reduction to Graph Problems
Algorithms such as Breadth First and Depth First
Traversal available after reduction to a graph rep
State-Space Graphs:
Vertices represent states and edges represent valid transitions
between states
Often with an initial and goal vertex
Widely used in AI
Example: The River Crossing Puzzle [(P)easant, (w)olf,
(g)oat, (c)abbage]
Pwgc | |
Pwc | | g
Pgc | | w
Pwg | | c
Pg | | wc
wc | | Pg
c | | Pwg
w | | Pgc
g | | Pwc
| | Pwgc
Strengths and Weaknesses of
Transform-and-Conquer
Strengths:
Allows powerful data structures to be applied
Effective in Complexity Theory
Weaknesses:
Can be difficult to derive (especially reduction)