Transcript Co-NP problems on random inputs Paul Beame University of Washington

Co-NP problems on random
inputs
Paul Beame
University of Washington
Basic idea
NP is characterized by a simple
property - having short certificates of
membership
Show that co-NP doesn’t have this
property
would separate P from NP so probably
quite hard
Lots of nice, useful baby steps towards
answering this question
2
Certifying language membership
Certificate of satisfiability
Satisfying truth assignment
Always short, SAT NP
Certificate of unsatisfiability
?????
transcript of failed search for satisfying truth
assignment
Frege-Hilbert proofs, resolution
Can they always be short? If so then NP=co-NP.
3
Proof systems
A proof system for L is a polynomial
time algorithm A s.t. for all inputs x
x is in L iff there exists a certificate
P s.t. A accepts input (P,x)
Complexity of a proof system
How big |P| has to be in terms of |x|
NP = {L: L has polynomial-size proofs}
4
Propositional proof systems
A propositional proof system is a
polynomial time algorithm A s.t.
for all formulas F
F is unsatisfiable iff
there exists a certificate P s.t.
A accepts input (P,F)
5
Sample propositional proof systems
Truth tables
Axiom/Inference systems, e.g.
modus ponens A, (A -> B) | B
excluded middle | (A v ~A)
Tableaux/Model Elimination systems
search through sub-formulas of input
formula that might be true simultaneously
e.g. if ~(A -> B) is true A must be true and
B must be false
6
Frege Systems
Finite # of axioms/inference rules
Proof of unsatisfiability of F - sequence
F1, …, Fr of formulas s.t.
F1 = F
each Fj is an axiom or follows from
previous ones via an inference rule
Fr = L trivial falsehood
All of equivalent complexity up to poly
7
Resolution
Frege-like system using CNF clauses only
Start with original input clauses of CNF F
Resolution rule
(A v x), (B v ~x) | (A v B)
Goal: derive empty clause L
Most-popular systems for practical theoremproving
8
Davis-Putnam (DLL) Procedure
Both
a proof system
a collection of algorithms for finding
proofs
As a proof system
 a special case of resolution where the
pattern of inferences forms a tree.
The most widely used family of
complete algorithms for satisfiability
9
Simple Davis-Putnam Algorithm
Refute(F)
While (F contains a clause of size 1)
set variable to make that clause true
simplify all clauses using this assignment
If F has no clauses then
output “F is satisfiable” and HALT
If F does not contain an empty clause then
Choose smallest-numbered unset variable x
Run Refute(Fx 0 )
splitting rule
Run Refute(Fx 1 )
10
Hilbert’s Nullstellensatz
System of polynomials
Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0
over field K has no solution in any
extension field of K
iff
there exist polynomials
P1(x1,…,xn),…,Pm(x1,…,xn) in K[x1,…,xn]
m
s.t.
 Pi Q i  1
i 1
11
Nullstellensatz proof system
Clause (x1 v ~x2 v x3)
becomes equation (1-x1)x2(1-x3)=0
Add equations xi2-xi =0 for each
variable
Proof: polynomials P1,…, Pm+n proving
unsatisfiability
12
Polynomial Calculus
Similar to Nullstellensatz except:
Begin with Q1,…,Qm+n as before
Given polynomials R and S can infer
a R + b S for any a, b in K
xi R
Derive constant polynomial 1
Degree = maximum degree of polynomial
appearing in the proof
Can find proof of degree d in time nO(d)
using Groebner basis-like algorithm
13
Cutting Planes
Introduced to relate integer and linear
programming:
Clause (x1 v ~x2 v x3)
becomes inequality x1+1-x2+x3
1
Add xi 0 and 1-xi
0



Derive 0  1 using rules for adding
inequalities and Division Rule:
acx+bcy  d implies ax+by  d/c
 
14
Some Proof System Relationships
ZFC
P/poly-Frege
Frege
AC0-Frege
Cutting Planes
Polynomial Calculus
Resolution
Davis-Putnam
Nullstellensatz
Truth Tables
15
Random k-CNF formulas
Make m independent choices of one of
n 
k
the 2   clauses of length k
k 
 D = m/n is the clause-density of the
formula
Distribution
F
k
n,D
16
Threshold behavior of random k-SAT
17
Contrast with ...
Theorem [CS]: For every constant D,
random k-CNF formulas almost
certainly require resolution proofs of
size 2W(n)
What is the dependence on D ?
18
Width of resolution proofs
If P is a resolution proof
width(P) = length of longest clause in P
Theorem [BW]: Every Davis-Putnam
(DLL) proof of size S can be
converted to one of width log2S
Theorem [BW]: Every resolution proof
of size S can be converted to one of
width O( n logS )
19
Sub-critical Expansion
 F - a set of clauses
s(F) - minimum size subset of F that is
unsatisfiable
 d F - boundary of F - set of variables
appearing in exactly one clause of F
e(F) - sub-critical expansion of F =
max
min { |d G|: G F, s/2
s  s(F)
< |G|
s}
20
Width and expansion
Lemma [CS] : If P is a resolution proof
of F then width(P)  e(F).
s/2 to s
s(F)
G
contains d G
L
21
Consequences
Corollaries:
Any Davis-Putnam (DLL) proof of F
requires size at least 2e(F)
Any resolution proof of F requires size
at least
2
W e (F) n
2










22
s(F) and e(F) for random formulas
If F is a random formula from F
then
s(F) is W (n/D1/(k-2)) almost certainly
k
n,D
e(F) is W (n/D2/(k-2)+e) almost
certainly
Proved for Hypergraph expansion
23
Hypergraph Expansion
 F - hypergraph
 d F - boundary of F - set of degree 1
vertices of F
sH(F) - minimum size subset of F that
does not have a System of Distinct
Representatives
eH(F) - sub-critical expansion of F max
min { |d G|: G  F, s/2
s  sH(F)
< |G|
s}
24
System of Distinct Representatives
variables/nodes
clauses/edges
sH(F)  s(F) so eH(F)  e(F)
25
Density and SDR’s
The density of a hypergraph is
#(edges)/#(vertices)
Hall’s Theorem: A hypergraph F has a system
of distinct representatives iff every
subgraph has density at most 1.
26
Density and Boundary
A k-uniform hypergraph of density
bounded below 2/k, say 2/k-e , has
average degree bounded below 2
 constant fraction of nodes are in
the boundary
27
Density of random formulas
Fix set S of vertices/variables of size r
Probability p that a single edge/clause
lands in S is at most (r/n)k
Probability that S contains at least q
edges is at most
q
 eDnp 
 eDr
   k-1
Pr B( Dn, p)  q  
 n
 q 
k-1



q
28
s(F) for random formulas
Apply for q=r+1 for all r up to s using
union bound:
 n   eDr
   k-1

r k  r   n
s
k 1



r 1
 ne 
  
r k  r 
s
r
 eDr
 k-1
 n
k 1
r  e Dr


k-2
en
n
r k

s
2
k 2






r 1
r 1
 o(1)
for s = O(n/D1/(k-2))
29
e(F) for random formulas
Apply for q=2r/k for all r between s/2
and s using union bound:
 n   eDr
   k-1

r  s/2  r   n
s
k 1



2r/k
 ne 
  
r  s/2  r 
s
r
 eDr
 k-1
 n
k 1
e
Dr
  
k-1-k/2
n
r  s/2 
s
1 k/2
k 1k/2



2r/k



2r/k
 o(1)
for s = Q(n/D2/(k-2))
30
Hypergraph Expansion and
Polynomial Calculus
Theorem [BI]: The degree of any
polynomial calculus or Nullstellensatz
proof of unsatisfiability of F is at
least eH(F)/2 if the characteristic is
not 2.

Groebner basis algorithm bound is
only nO(eH(F))
31
k-CNF and parity equations
Clause (x1 v ~x2 v x3)
is implied by x1+(x2+1)+x3 = 1 (mod 2)
i.e. x1+x2+x3 = 0 (mod 2)
Derive contradiction 0 = 1 (mod 2) by
adding collections of equations
# of variables in longest line is at least eH(F)
32
Parity equations and polynomial
calculus
Given equations of form
x1+x2+x3 = 0 (mod 2)
Polynomial equation yi2-1=0 for each variable
yi = 2xi-1
Polynomial equation y1 y2 y3-1=0
would be y1 y2 y3+1=0 if RHS were 1
Imply the old Nullstellensatz equations if
char(K) is not 2
33
Lower bounds
For random k-CNF chosen from Fn,kD
almost certainly for any e>0:
Any Davis-Putnam proof requires size
n/ Δ2/(k2)e
2
Any resolution proof requires size
n/ Δ 4/(k2)e
2
Any polynomial calculus proof requires
degree
2/(k 2)  e
n/ Δ
34
Upper Bound
Theorem [BKPS]: For F chosen from F
and D above the threshold, the simple
Davis-Putnam (DLL) algorithm almost
certainly finds a refutation of size
k
n,D

2
O n/ Δ1/(k2)
nO(1)
and this is a tight bound...
35
Idea of proof
2-clause digraph
(x v y)
y
x
y
x
Contradictory cycle: contains both x and x
After setting O(n/D1/(k-2)) variables,
> 1/2 the variables are almost certainly in
contradictory cycles of the 2-clause digraph
a few splitting steps will pick one almost certainly
setting clauses of size 1 will finish things off
36
Implications
Random k-CNF formulas are provably hard
for the most common proof search
procedures.
This hardness extends well beyond the
phase transition.
Even at clause ratio D=n1/3, current
algorithms on random 3-CNF formulas have
asymptotically the same running time as
the best factoring algorithms.
37
Random graph k-colourability
Random graph G(n,p) where each edge
occurs independently with probability p
Sharp threshold for whether or not graph
is k-colourable, e.g. p ~ 4.6/n for k=3
What about proofs that the graph is
not k-colourable?
38
Lower Bound
Theorem [BCM 99]: Non-k-colourability
requires exponentially large resolution proofs
Basic proof idea:
same outline as before
notion of boundary of a sub-graph
set of vertices of degree < k
s(G) smallest non-k-colourable sub-graph
39
Challenges
Better bound for e(F) for random F
Can it be Q(s(F)) ?
If so, the simple Davis-Putnam algorithm has
asymptotically best possible exponent of any DP
algorithm.
Extend lower bounds to other proof
systems
must be based on something other than expansion
since certain formulas with high expansion have
small Cutting Planes proofs.
40
Challenges
Conjecture: Random k-CNF formulas
are hard for Frege proofs
Extend to other random co-NP
problems
Independent Set?
Best algorithms only get within factor of 2 of
the largest independent set in a random graph
41
Sources
[Cook, Reckhow 79]
[Chvatal, Szemeredi 89]
[Mitchell, Selman, Levesque 93]
[Beame, Pitassi 97]
[Beame, Karp, Pitassi, Saks 98]
[Beame, Pitassi 98]
[Ben-Sasson, Wigderson 99]
[Ben-Sasson, Impagliazzo 99]
[Beame, Culberson, Mitchell 99]
42
Circuit Complexity
P/poly - polysize circuits
NC1 - polysize formulas
CNF - polysize CNF formulas
AC0 - constant-depth polysize circuits
using and/or/not
AC0[m] - also = 0 mod m tests
TC0 - threshold instead
43
C-Frege Proofs
Given circuit complexity class C can define
C-Frege proofs to be Frege-like proofs that
manipulate circuits in C rather than formulas
Frege = NC1-Frege
Resolution = CNF-Frege
Extended-Frege = P/poly-Frege
AC0-Frege
AC0[m]-Frege
TC0-Frege
44