Transcript Analysis of Algorithms Lecture 6 Approximation Algorithms UMass Lowell Computer Science 91.503

UMass Lowell Computer Science 91.503
Analysis of Algorithms
Prof. Karen Daniels
Fall, 2001
Lecture 6
Tuesday, 10/30/01
Approximation Algorithms
How to Treat NP-Complete Problems??
Approximation Algorithms
 Heuristic Upper or Lower Bounds



Mathematical Programming




Greedy, Simulated Annealing, Genetic “Alg”, AI
Linear Programming for part of problem
Integer Programming
Quadratic Programming...
Search Space Exploration:

Gradient Descent, Local Search, Pruning, Subdivision
Randomization, Derandomization
 Leverage/Impose Problem Structure
 Leverage Similar Problems

Overview

Approximation Algorithms
 Highlights
from Textbook
 Highlights from Garey & Johnson
 Literature Case Study
 Homework #4

Follow-up on NP-Completeness Literature
Case Study
Approximation Algorithms
Chapter 37
Basic Concepts
Definitions
Definitions

Approximation Algorithm


produces “near-optimal” solution
Algorithm has approximation ratio r(n) if:
 C C *
max 
,
  r ( n)
C* C 

C = cost of algorithm’s solution
C* = cost of optimal solution
n = number of inputs = size of instance
Approximation Scheme

(1+e)-approximation algorithm for fixed e


Polynomial-Time Approximation Scheme


e > 0 is an input
time is polynomial in n
Fully Polynomial-Time Approximation Scheme


time is also polynomial in (1/e)
constant-factor decrease in e causes only constant-factor running-time
increase
source: 91.503 textbook Cormen et al.
Overview



VERTEX-COVER
 Polynomial-time 2-approximation algorithm
TSP
 TSP with triangle inequality
 Polynomial-time 2-approximation algorithm
 TSP without triangle inequality
 Negative result on polynomial-time r(n)-approximation algorithm
SET-COVER
 polynomial-time r(n)-approximation algorithm



r(n) is a logarithmic function of set size
SUBSET-SUM
 Fully polynomial-time approximation scheme
MAX-3-CNF Satisfiability
 Randomized r(n)-approximation algorithm
Vertex Cover
Polynomial-Time 2-Approximation Algorithm
Vertex Cover
A
Vertex Cover of an undirected
graph G=(V,E) is a subset
B
vertex cover
C
D
of size 3
E
F
V '  V such that if (u, v)  E , then u V ' or v V ' or both
NP-Complete
source: Garey & Johnson
Vertex Cover
b
c
d
a
e
f
d
g
a
g
a
e
f
g
f
g
d
a
c
a
a
source: 91.503 textbook Cormen et al.
Vertex Cover
Theorem: APPROX-VERTEX-COVER is a
polynomial-time 2-approximation algorithm.
Proof: Let C * be an optimal cover
C be cover from APPROX - VERTEX - COVER
A  edges chosen by APPROX - VERTEX - COVER
Observe: no 2 edges of A share any vertices due to removal of incident edges
Any vertex cover must include >= 1 endpoint of each edge in A
APPROX-VERTEX-COVER adds both endpoints of each edge of A
C  2 A  2C*
transitivity
C  2C *
Algorithm runs in time polynomial in n.
C*  A
C  2A
Homework #4
APPROX-VERTEX-COVER instance
 Textbook: Exercise 37.1-1, p. 968

Traveling Salesman
TSP with triangle inequality
Polynomial-time 2-approximation algorithm
TSP without triangle inequality
Negative result on polynomial-time r(n)-approximation
algorithm
Hamiltonian Cycle
Hamiltonian Cycle of an undirected graph G=(V,E) is a
simple cycle that contains each vertex in V.
NP-Complete
source: 91.503 textbook Cormen et al.
Traveling Salesman Problem (TSP)
TSP Tour of a complete, undirected
graph G=(V,E) is a Hamiltonian
Cycle with a designated
starting/ending vertex.
u
v
x
w
{ G, c, k :
TSP Decision Problem:
cost function
c : V V  Z
k Z
G has TSP  tour of cost  k}
NP-Complete
source: 91.503 textbook Cormen et al.
Minimum Spanning Tree:
Greedy Algorithms
Time:
O(|E|lg|E|)
given fast
FIND-SET,
UNION
Invariant: Minimum weight
spanning forest
Produces minimum weight tree of
edges that includes every vertex.
Becomes single
tree at end
Time:
O(|E|lg|V|)
=
O(|E|lg|E|)
slightly
faster with
fast priority
queue
2
4
A
3
1
Spans all
vertices at end
G
5
6
E
Invariant: Minimum
weight tree
6
D
B
8
2
1
7
F
4
C
for Undirected, Connected,
Weighted Graph
G=(V,E)
source: 91.503 textbook Cormen et al.
TSP with Triangle Inequality
u
v
5
3
3
5
x
w
4
Cost Function Satisfies
Triangle Inequality
u , v, w  V
c(u , w)  c(u , v)  c(v, w)
source: 91.503 textbook Cormen et al.
TSP with Triangle Inequality
Theorem: APPROX-TSP-TOUR is a
polynomial-time 2-approximation algorithm
for TSP with triangle inequality.
Proof:
Algorithm runs in time polynomial in n.
Let H * be an optimal tour and T be MST
c(T )  c( H *)
(since deleting 1 edge from a tour creates a spanning tree)
Let W be a full walk of T (lists vertices when they are first visited and when returned to after visiting subtree)
c(W )  2c(T ) (since full walk traverses each edge of T twice)
c(W )  2c( H *)
c( H )  c(W )
Now make W into a tour using triangle inequality
(since H was formed by deleting vertices from W)
c( H )  2c( H *)
TSP without Triangle Inequality
Theorem: If P  NP , then for any constant r  1
polynomial-time approximation algorithm with ratio r
for TSP without triangle inequality.
Proof: (by contradiction) Suppose there is one --- call it A
Showing how to use A to solve NP-complete Hamiltonian Cycle problem
contradiction!
Convert instance G of Hamiltonian Cycle into instance of TSP (in polynomial time):
E '  {( u, v) : u, v V , u  v}
(u, v)  E 
 1
c(u, v)  

 r | V | 1 otherwise 
(G’=(V,E’) is complete graph on V)
(assign integer cost to each edge in E’)
For TSP problem (G’,c):
G has Hamiltonian Cycle
(G’,c) contains tour of cost |V|
Tour of G’ must use some edge not in E
G does not have Hamiltonian Cycle
Cost of that tour of G’  r | V | 1  | V | 1  r | V |  | V
Can use A on (G’,c)!
G has Hamiltonian Cycle
A finds tour of cost = |V|
G does not have Hamiltonian Cycle
A finds tour of cost > r |V|
| r | V |
Homework #4
TSP Closest-Point Heuristic proof
 Textbook: Exercise 37.2-2, p. 973

Set Cover
Greedy Approximation Algorithm
polynomial-time r(n)-approximation algorithm
r(n) is a logarithmic function of set size
Set Cover Problem
Instance ( X , F ) :
finite set X
family F of subsets of X
X  S
SF
Problem : Find a minimum - sized subset C  F
whose members cover all of X: X   S
SC
NP-Complete
source: 91.503 textbook Cormen et al.
Greedy Set Covering Algorithm
Greedy: select set that covers the most uncovered elements
source: 91.503 textbook Cormen et al.
Set Cover
Theorem: GREEDY-SET-COVER is a
polynomial-time r(n)-approximation algorithm
for r (n)  H (max{| S |: S  F })
d
Proof:
1
dth harmonic number H d    H ( d )
i 1 i
Algorithm runs in time polynomial in n.
Si  ith subset selected
cx  cost of element x  X
cx 
selecting Si costs 1
paid only when x is covered for the first time
1
Si  ( S1  S 2    Si 1 )
assume x is covered for the first time by Si
(spread cost evenly across all elements covered for first time by Si )
Number of elements covered for first time by Si
Set Cover (proof continued)
Theorem: GREEDY-SET-COVER is a
polynomial-time r(n)-approximation algorithm
for r (n)  H (max{| S |: S  F })
Proof: (continued)
Let C * be an optimal cover
C be cover from GREEDY - SET - COVER
Cost assigned to
optimal cover:


  cx 

S C * xS

C 
Each x is
in >= 1 S:


C     cx 
SC * xS

c
x X
x
1 unit is charged at
each stage of
algorithm


  cx    cx

SC * xS
 xX
Set Cover (proof continued)
Theorem: GREEDY-SET-COVER is a
polynomial-time r(n)-approximation algorithm
for r (n)  H (max{| S |: S  F })
Proof: (continued)
d
How does this relate to
harmonic numbers??
We’ll show that:
c
xS
And then
conclude that:
which will finish the proof
C 
1
dth harmonic number H d    H ( d )
i 1 i
x
 H( S )
 H ( S )  C * H (max{ S : S  F })
SC *
Set Cover (proof continued)
Proof of:
c
xS
x
 H( S )
ui | S  ( S1  S 2   Si ) |
For some set S: Number of elements of S
remaining uncovered after S1…Si selected

1

c

(
u

u
)


x
i 1
i

Si  ( S1  S 2    S i 1 )
xS
i 1 
k

1 


c

(
u

u
)


x
i
 i 1
ui 1 
xS
i 1 
k
k

i 1
u i 1
1

j  u i 1 u i 1
k
ui 1
Since, due to greedy
nature of algorithm:
1 since j <= u
i-1
 
i 1 j ui 1 j
k




u0 | S |
k = least index for
which uk=0.
| Si  ( S1  S 2   Si 1 ) | 
| S  ( S1  S 2   Si 1 ) | ui 1
 ui1 1 ui 1 
      
i 1  j 1 j
j 1 j 
k
  H (ui 1 )  H (ui )   H (u0 )  H (u k )  H (u0 )  H (0)  H (u0 )  H ( S )
i 1
telescoping sum
Homework #4
GREEDY-SET-COVER output
 Textbook: Exercise 37.3-1, p. 978

Subset-Sum
Exponential-Time Exact Algorithm
Fully Polynomial-Time Approximation Scheme
Subset-Sum Problem
Instance ( S , t ) :
Decision
Problem:
S  {x1,x2 ,...,xn }
positive integers
S '  S :  s  t ?
sS '
Optimization Problem seeks subset with largest sum <= t
NP-Complete
source: 91.503 textbook Cormen et al.
Exponential-Time
Exact Algorithm
Pi  set of values obtainable by selecting
subset of {x1,x2 ,...,xi } and summing elements.
Identity:
Pi  Pi 1  ( Pi 1  xi )
Li is sorted list of every element of Pi <= t
Li  list of sums of subsets of {x1,x2 ,...,xi }  t
MERGE-LISTS( L,L’ ) returns sorted list =
merge of sorted L, L’ with duplicates removed.
source: 91.503 textbook Cormen et al.
Fully Polynomial-Time
Approximation Scheme
Theorem:
APPROXSUBSET-SUM is a
fully polynomialtime
approximation
scheme for
subset-sum.
Proof:
see textbook
source: 91.503 textbook Cormen et al.
Homework #4
APPROX-SUBSET-SUM proof
 Textbook: Exercise 37.4-2, p. 983

 (eq.
37.12 only)
MAX-3-CNF Satisfiability
3-CNF Satisfiability Background
Randomized Algorithms
Randomized MAX-3-CNF SAT Approximation Algorithm
MAX-3-CNF Satisfiability

Background on Boolean Formula Satisfiability
 Boolean
Formula Satisfiability: Instance of
language SAT is a boolean formula f consisting of:
n boolean variables: x1, x2, ... , xn
 m boolean connectives: boolean function with 1 or 2
inputs and 1 output


e.g. AND, OR, NOT, implication, iff
    
parentheses
 truth, satisfying assignments notions apply

SAT  { f : f is a satisfiabl e boolean formula}
NP-Complete
source: 91.503 textbook Cormen et al.
MAX-3-CNF Satisfiability
(continued)

Background on 3-CNF-Satisfiability
 Instance
of language SAT is a boolean formula
f consisting of:
literal: variable or its negation
 CNF = conjunctive normal form




conjunction: AND of clauses
clause: OR of literal(s)
3-CNF: each clause has exactly 3 distinct literals
MAX-3-CNF Satisfiability: optimization version of 3-CNF-SAT
- Maximization: satisfy as many clauses as possible
- Input Restrictions:
- exactly 3 literals/clause
- no clause contains both variable and its negation
NP-Complete
source: 91.503 textbook Cormen et al.
Definition

Randomized Algorithm has approximation
ratio r(n) if, for expected cost C of solution
produced by randomized algorithm:
 C C *
max 
,
  r ( n)
C* C 
source: 91.503 textbook Cormen et al.
Randomized Approximation
Algorithm for MAX-3-CNF SAT
Theorem: Given an instance of MAX-3-CNF satisfiability with n
variables x1, x2,..., xn with m clauses, the randomized algorithm that
independently sets each variable to 1 with probability 1/2 and to 0 with
probability 1/2 is a randomized 8/7-approximation algorithm.
Proof: independen tly set each varia ble to 0 or 1 with  probabilit y  1/2
indicator random variable  Yi  I {clause i is satisfied}
Pr{clause i is not satisfied}  (1/2) 3  1 / 8 Pr{clause i is satisfied}  1 - (1 / 8)  7 / 8
Lemma : Given a sample space S and an event A in the sample space S ,
Let X A  I{A}. Then E[X A ]  Pr {A}.
By Lemma : E[Yi ]  7 / 8
Let Y  Y1  Y2    Ym
m
7 7m
m  m
E[Y ]  E  Yi    EYi   
8
i 1 8
 i 1  i 1
source: 91.503 textbook Cormen et al.
number of satisfied clauses  m
r (n)  m /( 7m / 8)  8 / 7
Homework #4

MAX-CNF 2-approximation algorithm
 New

Textbook: Exercise 35.4-2, p. 1043
EXTRA-CREDIT:
 Randomized Algorithm

Probability Analysis
New Textbook: Exercise 5.2-1, p. 98
Approximation Algorithms
Garey & Johnson
Bin-Packing
Bin-Packing Problems
Bin Packing
Input: A set of n items with sizes
d_1,...,d_n. A set of m bins with
capacity c_1,...,c_m.
Problem: How do you store the set of
items using the fewest number of
bins?
Bin-Packing Heuristics
source: Garey & Johnson
Bin-Packing Heuristics
source: Garey & Johnson
Bin-Packing Heuristics
source: Garey & Johnson
Bin-Packing Heuristics
source: Garey & Johnson
Approximation Algorithms
Literature Case Study
“Approximation Schemes for Covering and Packing
Problems in Image Processing and VLSI”
[Hochbaum and Maass, Journal of ACM, 1985]
Disk Covering Problem
Given a set of points N and many copies of a disk...
|N| = n = 25
How many disks are needed to cover the set of points?
Here 11 disks suffice. Is this optimal?
Notation
Area I
strip 1
strip 2
strip 3
strip 4
strip 5
Disk
Diameter = D
strip 6
|N| = n = 25
strip includes
left boundary
strip width = D
strip does not include
right boundary
Notation (continued)
strip 1
strip 2
strip 3
strip 4
strip 5
strip 6
Partition 1 = S1 = {(strip 1, strip 2, strip 3), (strip 4, strip 5, strip 6)}
Partition 2 = S2 = {(strip 2, strip 3, strip 4), (strip 5, strip 6, strip 1)}
Partition 3 = S3 = {(strip 3, strip 4, strip 5), (strip 6, strip 1, strip 2)}
Shifting Strategy
Total Area = I
Disk
Diameter = D
|N| = 25
The paper uses a shifting strategy to develop Polynomial Approximation
Scheme for Covering Problems. Strategy bounds error in generic divide-andconquer. For one partition, apply divide-and-conquer down to vertical strip
level and return union of disks in cover. This produces a globally feasible
solution. Repeat for each partition. Shift algorithm chooses “best partition.”
Approximation Algorithms
Homework #4
Chapter Dependencies
Ch 1-6
Math Review:
Asymptotics,
Recurrences,
Summations,
Sets, Graphs,
Counting,
Probability,
Calculus,
Proofs
Techniques (e.g.
Inductive)
Logarithms
Math: Linear
Algebra
Math: Geometry
(High School
Level)
Ch 28, 29
Parallel Comparison-Based
Sorting Networks, Arithmetic
Circuits
Ch 7-10
Sorting
Ch 16, 17, 18
Advanced Design &
Analysis Techniques
Ch 11-14, 15
Data Structures
Ch 23-25,26,27
Graph Algorithms
Ch 19-22
Advanced Data
Structures
Ch 31
Matrix Operations
Ch 35
Computational Geometry
Ch 30
Parallel Comparison-Based
Sorting Networks
Ch 36
NP-Completeness
Ch 37
Approximation
Algorithms
You’re responsible for all
sections of this chapter +
extra material from Garey &
Johnson + supplement from
new edition of text + literature
paper
Homework #4
APPROX-VERTEX-COVER instance
 TSP Closest-Point Heuristic proof
 GREEDY-SET-COVER output
 APPROX-SUBSET-SUM proof
 MAX-CNF 2-approximation algorithm
 EXTRA-CREDIT:

 Randomized Algorithm
Probability Analysis
NP-Completeness
Follow-up on
Literature Case Study
State-of-the-Art:
Covering Problems
covering
non-geometric covering
geometric covering
VERTEX-COVER, SET-COVER, EDGE-COVER,
VLSI logic minimization, facility location
2D translational covering
P: finite point sets
P: shapes
covering
P
Q: identical
Q: convex
Q: nonconvex
BOX-COVER
1D interval
covered by annuli
. . . .
. . .
. . .
 j (Q j )
.

1 j  m
.
P

1 j m
j
( Qj )
.
partition
cannot
: overlap  i (Qi )
Polynomial-time algorithms for
triangulation and some tilings
decomposition:
 j (Q j )
cover
may overlap
j
( Qj )
-Thin coverings of the plane with
congruent convex shapes
-Translational covering of a convex set by
a sequence of convex shapes
-Translational covering of nonconvex set
with nonidentical covering shapes
 i (Qi )
-NP-hard/complete polygon problems
-polynomial-time results for restricted
orthogonal polygon covering and
horizontally convex polygons
-approximation algorithms for boundary,
corner covers of orthogonal polygons
“Optimal Packing and Covering in the
Plane are NP-Complete” [Fowler et al.]
source: Fowler et al.
“Optimal Packing and Covering in the
Plane are NP-Complete” [Fowler et al.]
source: Fowler et al.
“Covering Polygons is Hard”
[Culberson, Reckhow]

Journal (prepublication) version of the
paper is on our web site, along with all
figures
source: Culberson, Reckhow
For More Information....

“Approximation Algorithms for NP-Hard
Problems”, editor Dorit Hochbaum, PWS
Publishing, Co., 1997.