Transcript Complexity classes - McGill University School of Computer

Computational complexity
Lecture 32
Hierarchy of problem difficulty
• Some problems are inherently more difficult than others
– Binary search runs in time O(log n)
it is impossible to do it faster
– FindMin runs in time O(n) and
it is impossible to do it faster
– Sorting takes time O(n log n) and
it is impossible to it faster
– Longest increasing subsequence (LIS)
takes time O(n2)
– The best algorithms known for the
following problems run in time (2n)
• Clique, graph coloring, hamiltonian cycle
• Task scheduling with constraints
• Satisfiability of boolean formula
These problems have
algorithms that run in
polynomial time:
Running time is O(nd)
These problems have
no known algorithms
that run in polynomial
time.
– The BIG question: are we just too stupid, or could it be that it is impossible
to find polynomial time algorithms for these problems?
Decision problems
• A decision problem is a problem whose
solution is either YES or NO
–
–
–
–
–
Is the smallest element of an array equal to 42?
Does a graph contain a clique so size k?
Does a graph contain a hamiltonian cycle?
Can a graph be colored with at most 3 colors?
Is a boolean formula satisfiable?
• Being able to solve the “decision” version
of a problem is usually enough to find the
actual solution (clique, cycle, coloring)
Satisfiability
• The Satisfiability Problem (SAT):
– Given a boolean formula with n variables x1,...,xn
– Is it possible to assign values (true of false) to the
variables to make the formula true?
• Example:
(x1  x2)  (x1  ( x3  x2 ) )  (x2  x3)
Satisfying assignment: x1 =
x2 =
x3 =
Complexity classes: P
• The class P is the set of decisions problems for
which a polynomial time algorithm exists
– The decision-problem version of findMin, shortest path,
change making, etc. are all in P
• Nobody knows if k-clique, hamiltonian cycle,
satisfiability,... are in P
– We don't have polynomial time algorithms for any of
these problems
– But we can't prove that such an algo. does not exist
Complexity classes: NP
• The class NP is the set of decision problems for
which, when the answer is YES, there exists a proof
of correctness (called a certificate) that can be
checked in polynomial type
• Example of certificates:
– k-clique: certificate = set of k vertices forming the clique.
To check correctness, simply verify that all vertices are
indeed connected by edges
– satisfiability: certificate = a truth value for each variable.
To check correctness, simply evaluate the expression.
• Most natural problems are in NP
• It is much easier to be in NP than to be in P
P vs NP
• P  NP, because if a poly-time algo exists to
find the solution to a problem, then we can use
it to check a solution, without even looking at
the certificate.
• The $1000000 question
Is P = NP ?
• In other words, is there a poly-time algorithm
for all problems in NP?
• Nobody knows, but most people believe that
P  NP, because...
Cook-Levin Theorem (1971)
Any decision problem X in NP of size n can be
encoded as a boolean formula that is satisfiable if
and only if the answer to X is YES. The size of
the boolean formula is polynomial in n.
Corollary: If we knew a poly-time algorithm for
SAT, all problems in NP could be solved in
polynomial time!
A decision problem X is NP-complete if
1) X is in NP
2) Any problem in NP can be reduced to X
SAT is NP-complete
Example of encoding
• To encode the 3-coloring problem:
• Let xij= true iff vertex i is colored with color j
 (x10  x20)   (x11  x21)   (x12  x22) 
 (x10  x30)   (x11  x31)   (x12  x32) 
 (x20  x30)   (x21  x31)   (x22  x32) 
 (x20  x40)   (x21  x41)   (x22  x42) 
 (x30  x40)   (x31  x41)   (x32  x42) 
(x10  x11  x12)  (x20  x21  x22)  (x30  x31  x32) 
(x40  x41  x42)  (x50  x51  x52)
Given any graph, we can build a boolean expression that
is satisfiable iff the graph is 3-colorable
Proving NP-completeness
To prove that a problem X is NP-complete:
1) Prove that X is in NP
2) Show it that some other NP-complete
problem Y can be reduced to X
NP
problems
Y, a NP-complete
problem
To
Problem X
prove
Other NP-complete problems
Example: To prove that k-CLIQUE is NP-complete
– We first prove that k-CLIQUE is in NP (easy!)
– We know that SAT is NP-complete. Thus all problems in NP
can be reduced to SAT
– We must give an algorithm to transform any boolean formula
into a graph that will contain a k-CLIQUE iff the formula is
satisfiable (harder, but possible)
• If we can do that, then k-CLIQUE is NP-complete,
because:
- any problem in NP can be encoded as SAT
- any SAT problem can be re-encoded as a k-CLIQUE
problem
- Thus any problem in NP can be encoded as k-CLIQUE.
Many NP-complete problems
• Using this method, the following problems have
been shown NP-complete:
–
–
–
–
–
k-clique
3-coloring
Hamiltonian cycle
Scheduling problems
Hundreds of others
• If we had a polynomial-time algorithm for one of
these NP-complete problems, this would directly
give a polynomial-time algorithm for all these
problems.
• But we don’t have such an algorithm for any...