Transcript Slide 1

P, NP and more
Towers of Hanoi
• Goal: transfer all n disks from peg A to peg C
• Rules:
– move one disk at a time
– never place larger disk above smaller one
• Recursive solution:
– transfer n - 1 disks from A to B
– move largest disk from A to C
– transfer n - 1 disks from B to C
• Total number of moves:
– T(n) = 2T(n - 1) + 1
Towers of Hanoi (2)
• Recurrence relation:
T(n) = 2 T(n - 1) + 1
T(1) = 1
• Solution by unfolding:
T(n) = 2 (2 T(n - 2) + 1) + 1 =
= 4 T(n - 2) + 2 + 1 =
= 4 (2 T(n - 3) + 1) + 2 + 1 =
= 8 T(n - 3) + 4 + 2 + 1 = ...
= 2i T(n - i) + 2i-1 +2i-2 +...+21 +20
• the expansion stops when i = n - 1
T(n) = 2n – 1 + 2n – 2 + 2n – 3 + ... + 21 + 20
Towers of Hanoi (3)
• This is a geometric sum, so that we have
T(n) = 2n - 1 = O(2n)
• The running time of this algorithm is
exponential (kn) rather than polynomial (nk),
where n is input size.
• Good or bad news?
– the Tibetans were confronted with a tower problem
of 64 rings...
– Assuming the priests move one ring per second, it
would take ~585 billion years to complete the
process!
Traveling Salesman Problem
• Given a set of cities, distances between all pairs of cities, and a
bound B, does there exist a tour (sequence of cities to visit) that
returns to the start and requires at most distance B to be traveled?
• TSP is in NP:
– given a candidate solution (a tour), add up all the distances and check if
total is at most B
Last Time
• The two theorems we proved showed an important
distinction.
– The difference between a single and multi-tape TM is at most a
square (or polynomial) difference.
– Moving to a nondeterministic TM gave an exponential speedup.
Polynomial Time
• From the perspective of time complexity, polynomial
differences are considered small, whereas exponential
differences are large.
– Exponential functions do grow incredibly fast; they grow much
faster than any polynomial function.
– However, different polynomials grow much faster than others. In
an algorithms course, you would be crazy to suggest that they
are equivalent:
• O(n log n) sorting algorithms are much better than
• O(n2) algorithms.
• O(n) and O(n2) are radically different.
• Nonetheless, there are some good reasons for assuming
polynomial-time equivalence.
Polynomial Time Algorithms
• What’s about?
– Halting problem
Not P
• Is your program slow or is in in infinite loop? (Can compiler help
you?)
– Chess (40 to 200 moves  ~ 400 positions)
• Very bushy trees (even with alpha-beta pruning)
Reasonable vs. Unreasonable
Processing 1 elements takes 0.01 sec
Growth rates
1E+40
5n
n^3
n^5
1.2^n
1E+30
2^n
n^n
Number of
microseconds
1E+20
1E+10
1
2
4
8
16
32
64
128
256
512
1024
Reasonable vs. Unreasonable
• ”Good”, reasonable algorithms
– algorithms bound by a polynomial function nk
– Tractable problems
• ”Bad”, unreasonable algorithms
– algorithms whose running time is above nk
– Intractable problems
intractable
problems
tractable
problems
problems not admitting
reasonable algorithms
problems admitting reasonable
(polynomial-time) algorithms
Background
• Exponential-time algorithms arise when we solve
problems by exhaustively searching a space of possible
solutions using brute force search.
• Polynomial-time algorithms require something other than
brute force.
• All reasonable computational models are polynomiallytime equivalent.
• So if we view all polynomial complexity algorithms as
equivalent, then the specific computational model
doesn't matter.
Definition of the class P
• P is the class of languages that are decidable in
polynomial time on a deterministic single-tape TM:
• That is:
The importance of P
• P plays a central role in the theory of
computation because:
– P is invariant over all models of computation that are
– polynomially equivalent to a single-tape DTM.
– P roughly corresponds to the class of problems that
are realistically solvable on a computer.
– Take this with a grain of salt.
• Some polynomial-time algorithms are bad: O(n10000) or
100000000000n2.
• Exponential algorithms may be okay for small n.
Verifying a Candidate Solution
• Suppose that you are organizing housing
accommodations for a group of four hundred university
students. Space is limited and only one hundred of the
students will receive places in the dormitory. To
complicate matters, the Dean has provided you with a
list of pairs of incompatible students, and requested that
no pair from this list appear in your final choice.
• Solving a problem: find possible people-room
distribution?
• Verifying a candidate solution: given a possible peopleroom distribution, verify whether it is correct?
Verifying a Candidate Solution vs.
Solving a Problem
• Intuitively it seems much harder (more time consuming)
in some cases to solve a problem from scratch than to
verify that a candidate solution actually solves the
problem.
• If there are many candidate solutions to check, then
even if each individual one is quick to check, overall it
can take a long time
Verifying a Candidate Solution
• Many practical problems in computer science, math,
operations research, engineering, etc. are polynomial
time verifiable but have no known polynomial time
algorithm
– Wikipedia lists problems in computational geometry, graph
theory, network design, scheduling, databases, program
optimization and more
Decision problems
• Are such long running times linked to the size of the
solution of an algorithm?
– No! To show that, we in the following consider only TRUE/FALSE
or yes/no problems – decision problems
• We can usually transform an optimization problem into
an easier decision problem:
– Optimization problem O : “Find a shortest path between vertices
u and v in a graph G.”
– Related decision problem D : “Determine if there is a path
between vertices u and v in a graph G shorter than k.”
– If we have an easy way to solve O, we can use it to solve D.
– If we show that D is hard, we know that O must be also hard.
The Class P
Why is it reasonable to consider all problems in P as
tractable (i.e., "easy")? What about n5000?
• If is not in P, then it certainly is not easy
• Polynomial time is mathematically convenient for
defining a class
– closed under composition
• Model independent notion
– poly time reductions between various formal models
• In practice, exponent is often small
The Class NP
• First, NP does not stand for not-P!!
• NP is the class of problems for which a
candidate solution can be verified in polynomial
time (nondeterministic solution in poly time)
• P is a subset of NP
More definitions
• P. Problems that can be solved in polynomial time. ("P"
stands for polynomial.) These problems have formed the
main material of this course.
• NP. This stands for "nondeterministic polynomial time"
where nondeterministic is just a fancy way of talking
about guessing a solution.
– A problem is in NP if you can quickly (in polynomial time) test
whether a solution is correct (without worrying about how hard it
might be to find the solution). Problems in NP are still relatively
easy: if only we could guess the right solution, we could then
quickly test it.
• NP does not stand for "non-polynomial". There are many
complexity classes that are much harder than NP.
P vs. NP
• Although polynomial time verifiability seems like a
weaker condition than polynomial time solvability, no one
has been able to prove that it is weaker (describes a
larger class of problems)
• So it is unknown whether P = NP
if 'yes'-answers to a 'yes'-or-'no'-question can be verified
"quickly" (in polynomial time), can the answers themselves also
be computed quickly?
– one of seven millennium prize problems
– if you solve it you will get $1,000,000
http://www.claymath.org/millennium/P_vs_NP/
P and NP
all problems
all problems
NP
or
P
P=NP
The Main Question
• If P=NP, then:
– Efficient algorithms for TSP, etc.
– Cryptography is impossible on conventional machines
– Modern banking system will collapse
The Main Question
• Yes? No?
– Thousands of researchers have spent four decades in search of
polynomial algorithms for many fundamental NP-complete
problems without success
– Consensus opinion: P  NP
• But maybe yes, since:
– No success in proving P  NP either
The Big Picture
• Summarizing: it is not known whether NP problems are
tractable or intractable
• But, there exist provably intractable problems
– Even worse – there exist problems with running times
unimaginably worse than exponential!
• More bad news: there are provably noncomputable
(undecidable) problems
– There are no (and there will not ever be!!!) algorithms to solve
these problems
Why should we care?
• Difficult problems come up all the time. Knowing they're hard lets
you stop beating your head against a wall trying to solve them, and
do something better:
– Use a heuristic (solving a reasonable fraction of the common cases)
– Solve the problem approximately instead of exactly.
– Use an exponential time solution anyway. If you really have to solve the
problem exactly, you can settle down to writing an exponential time
algorithm and stop worrying about finding a better solution.
– Choose a better abstraction. The NP-complete abstract problem you're
trying to solve presumably comes from ignoring some of the seemingly
unimportant details of a more complicated real world problem. Perhaps
some of those details shouldn't have been ignored, and make the
difference between what you can and can't solve.
P = NP Question
• Open question since about 1970
• Great theoretical interest
• Great practical importance:
– If your problem is NP-complete, then don't waste time
looking for an efficient algorithm
– Instead look for efficient approximations, heuristics,
etc.