Transcript Analysis of Algorithms Lecture 6 NP-Completeness UMass Lowell Computer Science 91.503

UMass Lowell Computer Science 91.503
Analysis of Algorithms
Prof. Karen Daniels
Fall, 2002
Lecture 6
Tuesday, 10/8/02
NP-Completeness
Overview

NP-Completeness
 Motivation
 Reduction
to Establish Lower Bounds
 Basics
 Literature
 How

to Treat NP- Hard or Complete Problems
Midterm Exam
Motivation
Why do we care??

When faced with a problem in practice, need to
know whether or not one can expect produce a
“fast” algorithm (i.e. O(nk)) to solve it
“Intractable” problems occur frequently in
practice
 Surface differences between tractable and
intractable problems are often hard to see

UMass Lowell Algorithms Lab
OS 305
Prof. Daniels
Design
Analyze
 PhD research
Undergraduate & Masters research
Apply
Manufacturing
Telecommunications
Data Mining &
Visualization
Industrial Motivation
Supporting Apparel Manufacturing
Maximum
Rectangle
Geometric Restriction
Distance-Based
Subdivision
Ordered
Containment
Limited Gaps
Containment
NP-complete
Maximal Cover
Two-Phase Layout
Minimal
Enclosure
Lattice
Packing
Column-Based Layout
Dynamic Covering Motivation
sensor coverage
clustering
shape recognition
channel
assignment
production allocation
facility location
layout
New Covering Research:
Objective

Practical solutions to dynamic hard core covering problems

dynamic instance = sequence of static instances
static instance0




static instance1
...
static instancei
hard : even static problem is computationally “intractable”
core : useful in multiple applications
covering ~ assignment ~ allocation
Practical : solve static instancei quickly if it is “easy”




combinatorially “small” or
clearly infeasible (for decision problem) or
clearly feasible (for decision problem) or
a “small” change from instancei-1
State-of-the-Art:
Dynamic Algorithms & Data Structures

Polynomial-time problems:


Data structures supporting fast updates
 Self-adjusting (e.g. splay trees) [Sleator, CMU]
 Kinetic Data Structures [Guibas, Stanford]
Hard problems:

Polynomial-time approximation schemes for optimization problems :

Practical algorithms for small and/or easy instances:
 Containment decision problem: [Daniels, Milenkovic]
“Greedy” can be sensitive to small changes [Gao, et al.]
 Meta-heuristics: no theoretical guarantee
static



Distributed heuristics
 Primarily for dynamic network routing or channel assignment
 [Wu et al.] connected dominating sets for network routing


successful building block for layout problems
leverages strong combinatorial optimization techniques
no theoretical guarantee; some empirical support
Randomized, distributed:
 [Gao, et al.] discrete mobile centers (clustering)

theoretical framework; assumes knowledge of change trajectories
State-of-the-Art:
Covering Problems
covering
non-geometric covering
geometric covering
VERTEX-COVER, SET-COVER, EDGE-COVER,
VLSI logic minimization, facility location
2D translational covering
P: finite point sets
P: shapes
covering
P
Q: identical
Q: convex
Q: nonconvex
BOX-COVER
1D interval
covered by annuli
. . . .
. . .
. . .
 j (Q j )
.

1 j  m
.
P

1 j m
j
( Qj )
.
partition
cannot
: overlap  i (Qi )
Polynomial-time algorithms for
triangulation and some tilings
decomposition:
 j (Q j )
cover
may overlap
j
( Qj )
-Thin coverings of the plane with
congruent convex shapes
-Translational covering of a convex set by
a sequence of convex shapes
-Translational covering of nonconvex set
with nonidentical covering shapes
 i (Qi )
-NP-hard/complete polygon problems
-polynomial-time results for restricted
orthogonal polygon covering and
horizontally convex polygons
-approximation algorithms for boundary,
corner covers of orthogonal polygons
New Covering Research Example 1:
Translational 2D Covering
[recent work with grad student Rajasekhar Inkulu (CCCG conference paper)]

Input:
STATIC
 Covering
polygons Q = {Q1, Q2 , ... , Qm}
 Target polygons (or point-sets) P = {P1, P2 , ... , Pn}

Output:
 Translations
 = { 1,  2 , ... ,  m} such that
P1
Q3
Q1
Q2

1 j  m
j
( Qj )
P2
Translational 2D Polygon Covering
P2
P
P1
Q3
Q2
Q1
Sample P and Q
NP-COMPLETE
Translated Q Covers P
New Covering Research Example 2:
2D Channel Assignment

Input:
[recent work with Masters student Sa Liu (GLOBECOM conference paper) ]
rx
STATIC
c cell grid
 q channels (frequencies)
 D(i,j) = cell(i,j) traffic demand

# cell-phone calls
 Co-channel
interference
constraints based on interference
threshold value e.g. no channel repetition

Output:
within any 2 x 2 square
 Feasible
assignment of channels
to cells that
satisfies demand and co-channel
interference constraints
 uses minimum number of channels

NP-COMPLETE
Reduction to Establish
Lower Bound
Tightening a Lower Bound for
Maximum-Area Axis-Parallel
Rectangle Problem
from W(n) to W(nlgn)
Approach

Go around lower bound brick wall by:
 examining
strong lower bounds for some similar
problems
 transforming a similar problem to our problem
[this process is similar to what we do when we prove problems NPcomplete]
worst-case bounds
on problem
1
n
n log2 n
n2
n5
2n
Lower Bounds for Related Problems
SmallestOuterCircle: Q (n)
point set, polygon
SmallestOuterRectangle: Q (n) point set, polygon
LargestInnerRectangle: W (n) polygon
LargestInnerCircle: Q (n log n) point set
LargestInnerRectangle: O (n log2(n)) polygon
1
n
n log(n)
n log2(n)
worst-case bounds on our problem
n5
2n
Lower Bound of W(n log n) by
Transforming a (seemingly unrelated) Problem
MAX-GAP instance: given n real numbers { x1, x2, ... xn } find the
maximum difference between 2 consecutive numbers in the
sorted list.
O(n) time transformation
Rectangle
specialized
area is a
polygon
solution to the
MAX-GAP
x2
instance
x4
x3 x1
Rectangle algorithm must take as least as much time as MAX-GAP.
MAX-GAP is known to be in W(n log n).
Rectangle algorithm must take W(n log n) time
for specialized polygons.
[Transforming yet another different problem yields bound for unspecialized polygons.]
NP-Completeness
Chapter 34
Basic Concepts
Polynomial Time & Problem Formalization
Polynomial-Time Verification
NP-Completeness & Reducibility
NP-Completeness Proofs
Expanding List of Hard Problems via Reduction
Polynomial Time &
Problem Formalization
Formalize notions of abstract and concrete
decision problems
 Encoding a problem
 Define class of polynomial-time solvable
decision problems
 Define complexity class
 Express relationship between decision
problems and algorithms that solve them
using framework of formal language theory

Polynomial Time &
Problem Formalization (continued)

Solvable in Polynomial Time
O(nk) for some positive constant k
 Tractable: k is typically small (e.g. 1, 2, 3)
 Typically holds across computation models
 Closure properties (chaining/composition)
 i.e.
Abstract Problem
 Decision Problem: yes/no answer

 impose
bound on optimization problem to recast
Polynomial Time &
Problem Formalization (continued)






Encoding of set S of abstract objects is mapping e from S to
set of binary strings
Concrete Problem: instance set is set of binary strings
Concrete Problem is Polynomial-Time Solvable if there
exists an algorithm to solve it in time O(nk) for some
positive constant k
Complexity Class P = set of polynomial-time solvable
concrete decision problems
Given abstract decision problem Q that maps instance set I
e : I  {0,1}
to {0,1}
can induce a concrete decision problem e(Q)
i  I is Q(i) {0,1}
If solution to abstract-problem instance

then solution to concrete-problem instance
e(i) {0,1}
is also Q(i)
Polynomial Time &
Problem Formalization (continued)
Goal: Extend definition of polynomial-time solvability from
concrete to abstract using encodings as bridge



f : {0,1}  {0,1}
is polynomial-time computable if there exists a
polynomial-time algorithm A that, given any input x {0,1} ,
produces f(x) as output.
For set I of problem instances, encodings are polynomially related if
there exist 2 polynomial-time computable functions f12, f21 such that
i  I , f12 (e1 (i))  e2 (i) and f 21(e2 (i))  e1 (i)
If 2 encodings e1, e2 of abstract problem are polynomially related, then
polynomial-time solvability of problem is encoding-independent:
Lemma 34.1: Let Q be an abstract decision problem on an instance set I
and let e1, e2 be polynomially related encodings on I.
Then,
e1 (Q)  P  e2 (Q)  P
Polynomial Time &
Problem Formalization (continued)

Formal Language Framework
S is finite set of symbols
 Language L over S is any set of strings consisting of
symbols from S
S for S  {0,1}
 Set of instances for decision problem Q is
L  {x  S : Q( x)  1}
 View Q as language L over S = {0,1}
x {0,1}
 Algorithm A accepts string
if, given input x,
output A(x) is 1
 Language accepted by A is set of strings A accepts
 Alphabet
L  {x {0,1} : A( x)  1}
Polynomial Time &
Problem Formalization (continued)

Formal Language Framework (continued)
 Language
L is decided by algorithm A if every binary
string in L is accepted by A and every binary string not in
L is rejected by A
 Language L is accepted in polynomial time by
algorithm A if it is accepted by A and if exists constant k
xL
such that, for any length-n string
, A accepts x in
time O(nk)
 Language L is decidable in polynomial time by
algorithm A if it is accepted by A and if exists constant k
x {0,1}*
such that, for any length-n string
, A correctly
xL
k
decides in time O(n ) if
Polynomial Time &
Problem Formalization (continued)

Formal Language Framework (continued)
 Complexity
Class is a set of languages whose
membership is determined by a complexity measure (i.e.
running time) of algorithm that determines whether a
given string belongs to a language.


We already defined: Complexity Class P = set of polynomialtime solvable concrete decision problems
Language Framework gives other definitions:
P = { L  {0,1} * : there exists an algorithm A that
decides L in polynomial time}
= { L : L is accepted by a polynomial-time algorithm}
Polynomial-Time Verification
Examine algorithms that “verify”
membership in languages
 Define class of decision problems whose
solutions can be verified in polynomial time

Polynomial-Time Verification
(continued)

Verification Algorithms
 algorithm
xL 
xL
A verifies language L if for any string
certificate y that A can use to prove that
for any string x  L  certificate y that A can
xL
use to prove that
 language verified by verification algorithm A is
 also,
L  {x {0,1}* :  y {0,1}* such that A( x, y )  1}
Polynomial-Time Verification
(continued)

The Complexity Class NP
 languages
that can be verified by a polynomialtime algorithm
such that A( x, y )  1}
L  {x {0,1}* :  certificate y such that | y | O(| x |c )
A

verifies language L in polynomial time
The Complexity Class co-NP
 set
of languages such that
L NP
Polynomial-Time Verification
(continued)
P
 NP coNP
P
NP coNP
P
34.3
source: 91.503 textbook Cormen et al.
NP-Completeness & Reducibility
Polynomial-Time Reducibility
 Define complexity class of NP-complete
problems
 Relationships among complexity classes
 Circuit satisfiability as an NP-complete
problem

NP-Completeness & Reducibility
(continued)


Q can be reduced to Q’ if any instance of Q can be “easily
rephrased as an instance of Q’, the solution to which provides
solution to instance of Q
L1 is polynomial-time reducible to L2 L1  P L2
if exists polynomial-time computable function
f : {0,1}*  {0,1} * such that x  {0,1} *
{0,1} *
{0,1} *
x  L1  f ( x)  L2
34.4
source: 91.503
textbook Cormen et al.
NP-Completeness & Reducibility
(continued)
If L1 , L2  {0,1}* are languages such that L1  P L2
Then L2  P  L1  P
YES , f ( x)  L2
f (x )
34.5
NO, f ( x)  L
2
YES , x  L1
NO, x  L1
34.3
source: 91.503 textbook Cormen et al.
NP-Completeness & Reducibility
(continued)
Language L  {0,1} * is NP-complete if
without this condition,
language is only NP-hard
popular
theoretical
computer
science view
L NP
NP
and
L'  P L,  L' NP
set of all NPcomplete
languages
Theorem:
- If any NP-complete problem is polynomial-time solvable, then P=NP.
- Equivalently, if any problem in NP is not polynomial-time solvable, then no
NP-complete problem is polynomial-time solvable.
source: 91.503 textbook Cormen et al.
NP-Completeness & Reducibility
(continued): Circuit

Satisfiability
circuit inputs
Satisfying
Assignment: truth
assignment inducing
output = 1
single
circuit
output
Boolean
Combinational
Gate Types:
AND, NOT, OR
Language Def: CIRCUIT-SAT =
{<C>:C is a satisfiable boolean
combinational circuit.}

GOAL: Show
Circuit Satisfiability
is NP-complete
34.8
source: 91.503 textbook Cormen et al.
NP-Completeness & Reducibility
(continued)

Circuit Satisfiability
CIRCUIT-SAT is in NP:

Construct 2-input polynomial-time algorithm A that verifies
CIRCUIT-SAT



Algorithm A :






Input 1: encoding of circuit C
Input 2: certificate = assignment of boolean values to all wires of C
For each logic gate A checks if certificate value for gate’s output wire
correctly computes gate’s function based on input wire values
If output of entire circuit is 1, algorithm outputs 1
Otherwise, algorithm outputs 0
For satisfiable circuit, there is a (polynomial-length)
certificate that causes A to output 1
For unsatisfiable circuit, no certificate can cause A to output 1
Algorithm A runs in polynomial time
source: 91.503 textbook Cormen et al.
NP-Completeness & Reducibility
(continued)

Circuit Satisfiability
CIRCUIT-SAT is NP-hard:







L represents some language in NP
Create polynomial-time algorithm F
computing reduction function f that
maps every binary string x to a
circuit C=f(x) such that
x  L  C  CIRCUIT  SAT
Must exist algorithm A verifying L
in polynomial time. (A from NP)
F uses A to compute f.
Represent computation of A as
sequence of configurations
F constructs single circuit that
computes all configurations
produced by initial configuration
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Proving a Language NP-Complete

Proving a Language NP-Hard
 all
steps except (1)
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Reducing Boolean Circuit Satisfiability to
Boolean Formula Satisfiability
 Boolean
Formula Satisfiability: Instance of
language SAT is a boolean formula f consisting
of:
n boolean variables: x1, x2, ... , xn
 m boolean connectives: boolean function with 1 or 2
inputs and 1 output


e.g. AND, OR, NOT, implication, iff





parentheses
 truth, satisfying assignments notions apply
SAT  { f : f is a satisfiabl e boolean formula}

source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Show: SAT is NP-Complete
 SAT
is in NP via argument similar to circuit case
 Reduce Boolean Circuit Satisfiability to Boolean
Formula Satisfiability
CIRCUIT  SAT P SAT
 Careful!
Naive approach might have shared
subformulas and cause formula size to grow
exponentially!
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Show: SAT is NP-Complete (continued)
 Reduce
Boolean Circuit Satisfiability to Boolean
Formula Satisfiability (continued) CIRCUIT  SAT P SAT
 Approach: For each wire xi in circuit C, formula f
has variable xi. Express gate operation.
f  x10  ( x4  x3 )
 ( x5  ( x1  x2 ))
 ( x6  x4 )
 ( x7  ( x1  x2  x4 ))
 ( x8  ( x5  x6 ))
 ( x9  ( x6  x7 ))
34.10
 ( x10  ( x7  x8  x9 ))
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Reducing Formula Satisfiability to 3-CNFSatisfiability


Boolean Formula Satisfiability: Instance of language
SAT is a boolean formula f
CNF = conjunctive normal form




conjunction: AND of clauses
clause: OR of literal(s)
3-CNF: each clause has exactly 3 distinct literals
Show: 3-CNF is NP-Complete
SAT  P 3CNF
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Show: 3-CNF is NP-Complete (continued) SAT 


P
3CNF
3CNF is in NP via argument similar to circuit case
Reduce Formula Satisfiability to 3-CNF-Satisfiability
 3 steps progressively transforming formula f closer to 3CNF

1) Similar to CIRCUIT  SAT  SAT reduction
P
Binary “ParseTree”
34.11
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Show: 3-CNF is NP-Complete (continued) SAT 


P
3CNF
3CNF is in NP via argument similar to circuit case
Reduce Formula Satisfiability to 3-CNF-Satisfiability
 3 steps progressively transforming formula f closer to 3CNF

2) Convert each clause fi’ into conjunctive normal form
 construct truth table for fi’
 using entries for 0, build DNF formula for f i '
 convert DNF into CNF formula fi” by applying DeMorgan’s laws
 each clause has at most 3 literals
f  (( x1  x2 )  ((x1  x3 )  x4 ))  x2
source: 91.503 textbook Cormen et al.
NP-Completeness Proofs (continued)

Show: 3-CNF is NP-Complete (continued) SAT 


P
3CNF
3CNF is in NP via argument similar to circuit case
Reduce Formula Satisfiability to 3-CNF-Satisfiability
 3 steps progressively transforming formula f closer to 3CNF


3) Convert each clause fi” so it has exactly 3 distinct literals
 add “padding variables”
 create fi”’
Size of resulting formula is polynomial in length of original
formula and reduction takes only polynomial time and 3-CNF
formula
f ' ' ' satisfiabl e  f satisfiabl e
source: 91.503 textbook Cormen et al.
Expanding List of Hard Problems
via Reduction

Relationships among some NP-complete problems
Circuit-SAT
source: 91.503 textbook Cormen et al.
source: Garey & Johnson
Vertex Cover
A
Vertex Cover of an undirected
graph G=(V,E) is a subset
B
vertex cover
of size 2
C
D
E
F
V '  V such that if (u, v)  E , then u V ' or v V ' or both
source: Garey & Johnson
Dominating Set
source: Garey & Johnson
Clique
Clique of an undirected graph
G=(V,E) is a complete subgraph of G.
A
B
C
clique of size 3
D
E
F
source: Garey & Johnson
Hamiltonian Cycle
34.2
Hamiltonian Cycle of an undirected graph G=(V,E) is a
simple cycle that contains each vertex in V.
source: 91.503 textbook Cormen et al.
Reducing 3-CNF-SAT to Clique
source: 91.503 textbook Cormen et al.
Construct graph G such that f is satisfiable iff G has clique of size k
x1
x
x
3
1
x1
x2
x2
34.14
For each clause Cr  (l1  l2  l3 ) f place triple of vertices v1r , v2r , v3r into V.
r
r
r
Put edge for vir , vjs if corresponding literals consistent and r not= s.
Reducing Clique to VertexCover
Construct complement G =(V, E ) of graph G such that G has
clique of size k iff graph G has a vertex cover of size |V|-k.
z
u
v
y
x
w
34.15
G has cliqueV '  V with | V ' | k  V  V ' is vertex cover in G
G has vertex cover V '  V with | V ' || V | k  V  V ' is clique in G
source: 91.503 textbook Cormen et al.
Literature
Resources
Recommended:
 Computers
& Intractability
by Garey & Johnson
 W.H.Freeman
 1990
 ISBN 0716710455.

How to Treat
NP-Hard or Complete Problems
How to Treat NP-Complete
Problems??
Approximation Algorithms
 Heuristic Upper or Lower Bounds



Mathematical Programming




Greedy, Simulated Annealing, Genetic “Alg”, AI
Linear Programming for part of problem
Integer Programming
Quadratic Programming...
Search Space Exploration:

Gradient Descent, Local Search, Pruning, Subdivision
Randomization, Derandomization
 Leverage/Impose Problem Structure
 Leverage Similar Problems
