Transcript rstut 6904

Discrete Aggregation and Ranking and
Gaussian Isoperimetric Inequalities
Elchanan Mossel
U.C. Berkeley
http://stat.berkeley.edu/~mossel
Discrete Aggregation and Ranking
• Motivation
– Aggregation of signals in order to rank is an important
practical problem (example: Google).
– Many important interesting variants
•
•
•
•
Today: Condorcet aggregations
Good probabilistic theory
Nice Gaussian Open problems
Same Theory applies to questions in computer science.
Gaussian Noise
•
•
Let -1    1 and f, g : Rn  Rm.
•
For sets A,B let: <A,B> := <1A,1B>
•
•
•
Let n
:= standard Gaussian volume
Let n
:= Lebesgue measure.
Let n-1, n-1 := corresponding (n-1)-dims areas.
Define <f, g> := E[<f(N) , g(M) >], where
N,M ~ Normal(0,I) with E[Ni Mj] =  (i,j).
Some isoperimetric results
•
•
•
I. Ancient: Among all sets with
n(A) = 1 the minimizer of n-1( A)
is A = Ball.
II. Recent (Borell, Sudakov-Tsierlson
70’s) Among all sets with n(A) = a
the minimizer of n-1( A) is A =
Half-Space.
III. More recent (Borell 85): For all
 > 0, among all sets with (A) = a
the maximizer of <A,A> is given by
A = Half-Space.
•
•
Influences and Noise in product
Spaces
Let X be a probability space.
Let f  L2(Xn,R). The i’th influence of f is given by:
Ii(f) := E[ Var[f | x1,…,xi-1,xi+1,…,xn] ]
•
(Ben-Or,Kalai,Linial; Efron-Stein 80s)
Given a reversible Markov operator T on X and
f, g: Xn  R define the T - noise form by
•
<f, g>T := E[f T n g]
The 2nd eigen-value (T) of T is defined by
(T) := max {|| :   spec(T),  < 1}
Influences and Noise in product
Spaces – Example 1
•
•
•
Let X = {-1, 1} with the uniform measure.
•
Let T be the “Beckner Operator” on X:
For the dictator function f(x) = xj: Ii(xj) = (i,j).
For the majority m(x) = sgn(1  i  n xi) function:
Ii(m)  (2  n)-1/2.
Ti,j =  (i,j) + (1-)/2.
• T xi =  xi and <xi, xi>T = .
• <m, m>T ~ 2 arcsin() / 
• (T) = .
Condorcet Paradox
•
n voters are to choose between 3 options /
candidates.
• Voter i ranks the three candidates A, B & C
via a permutation i  S3
• Let XABi = +1 if i(A) > i(B)
XABi = -1 if i(B) > i(A)
• Aggregate rankings via: f : {-1,1}n ! {-1,1}.
• Assume that f(-x1,…,-xn) = -f(x1,…,xn)
• Aggregation: A is better than B if f(xAB) = 1.
• A Condorcet Paradox occurs if:
f(xAB) = f(xBC) = f(xCA).
• Defined by Marquis de Condorcet in 18’th
century.
B
A
C
Arrow’s Impossibility Thm
•
Thm (Condorcet): If n > 2 is odd and
f is the majority function then there
exist rankings 1,…,n resulting in a
Paradox
• Thm (Arrow’s Impossibility): For all n
> 2, unless f is the dictator function,
there exist rankings 1,…,n resulting
in a paradox.
Kalai’s Random Ranking:
• Each voter tosses a dice.
• Vote according to the corresponding
order on A,B and C.
Probability of a Paradox
• Rankings are chosen uniformly in S3n
• What is the probability of a paradox:
• PDX(f) = P[f(xAB) = f(xBC) = f(xCA)]?
• Arrow’s:: If f  dictator then PDX(f) > 0.
• Note: If f = dictator then PDX(f) = 0.
• Thm(Kalai 02): PDX(f) = ¼ - ¾ <f, f>T where
Ti,j = 1/3 (i,j) + 1/3 is the Beckner 1/3 operator.
Low influence aggregation
• A question: What is the most rational aggregation function f
not allowing dictatorships or Juntas?
• For example, consider only monotone f that are invariant
under a transitive action on the coordinates.
• More generally: What is the minimum of PDX(f) among f
satisfying for all voters i:
Ii(f) = P[f(x1,…,xi,…,xn)  f(x1,…,-xi,…,xn)] < 
where n   and   0.
• Conjecture (Kalai 02): Among all low influence f’s, the one
that minimizes PDX(f) is given by the majority function.
(PDX(f)  0.0876 for f = majority)
X
X
•
•
•
•
•
•
Influences and Noise -Example 2
Let X = {0,1,2} with the uniform measure.
Let Ti,j = ½ (i  j)
Then (T) = ½ and
Claim (Colouring Graph): Consider Xn as a graph
where (x,y)  Edges(Xn) iff xi  yi for all i.
Let A,B  Xn. Then <A, B>T = 0 iff there are no
edges between A and B. In particular, A is an
independent set iff <A, A>T = 0.
Q: How do “large” independent sets look like?
Important for the analysis of complexity of graph
coloring.
Graph Colouring – An Algorithmic Problem
•
Let (G) := min # of colours needed
to colour the vertices of a graph G
so that no edge is monochramatic.
• ApxCol(q,Q):
Given a graph G, is (G)  q or (G)  Q ?
•
•
•
This is an algorithmic problem. How hard is it?
•
Let Q > 3. Suppose that   > 0 such that for all n if there
are no edges between A and B  {0,1,2}n (<A,B>T = 0) and
|A|,|B| > 3n/Q then there exists an i such that Ii(A) > 
and Ii(B) > .
Khot’s “games conjecture” + following claim  NP hard.
Claim: Consider {0,1,2}n as a graph G where
(x,y)  Edges(G) iff xi  yi for all i.
Graph Colouring – An Algorithmic Problem
u
Graph Colouring – An Algorithmic Problem
u
[u]
•
Gaussian Noise Bounds
Def: For a, b,   [0,1] , let
(a, b, ) := sup {< F,G > | F,G  R, [F] = a, [G] = b}
•
•
•
•
(a, b, ) := inf {< F,G > | F,G  R, [F] = a, [G] = b}
Thm: Let X be a finite space. Let T be a reversible
Markov operator on X with  = (T) < 1.
Then   > 0   > 0 such that for all n and all
f,g : Xn  [0,1] satisfying maxi min(Ii(f), Ii(g)) < 
It holds that <f, g>T  (E[f], E[g], ) +  and
<f, g>T  (E[f], E[g], ) - 
• M-O’Donnell-Oleskiewicz-05 + Dinur-M-Regev-06
Application: Example 1
•
•
Taking T on {-1,1} defined by Ti,j = (i,j)/3 + 1/3
•
•
•
<f, f>T  <F, F>1/3 +  where F(x) = sgn(x)
•
Thm : Claim:      f : {-1,1}n  {-1,1} with
Ii(f) <  for all i and E[f] = 0 it holds that:
<F, F>1/3 = 2 arcsin(1/3)/  (F is known by Borell-85)
 Kalai’s conjecture: The probability of Paradox in
Condorcet voting with low influences f is minimized
by f = majority.
Weaker results obtained by Bourgain 2001.
Application: Example 2
•
•
Taking T on {0,1,2} defined by Ti,j = ½ (i  j)
•
There exists an i s.t. Ii(A), Ii(B)  .
Thm  Claim:   > 0   > 0 s.t. if A,B  {0,1,2}n
have no edges between them and P[A], P[B]  
then
•
•
Proof follows from Borell-85 showing (,,1/2) > 0.
•
“For any constant K, it is NP hard to
Claim  Hardness of approximation
result for graph-colouring:
colour 3-colorable graphs using K colours”.
More results
•
Other results that use Noise-Stability bounds
include:
•
Social choice: Majority is most stable function to
errors in voting machines (M-O’Donnell-Oleskiewicz-05),
Most predictable from sample (Mossel 07).
•
•
Hardness of approximation:
Khot-Vishnoy etc.
•
Gaussian Noise Bounds
Def: For a, b,   [0,1] , let
(a, b, ) := sup {< F,G > | F,G  R, [F] = a, [G] = b}
•
•
•
•
(a, b, ) := inf {< F,G > | F,G  R, [F] = a, [G] = b}
Thm: Let X be a finite space. Let T be a reversible
Markov operator on X with  = (T) < 1.
Then   > 0   > 0 such that for all n and all
f,g : Xn  [0,1] satisfying maxi min(Ii(f), Ii(g)) < 
It holds that <f, g>T  (E[f], E[g], ) +  and
<f, g>T  (E[f], E[g], ) - 
• M-O’Donnell-Oleskiewicz-05 + Dinur-M-Regev-06
Gaussian Noise Bounds
•
•
Proof Idea:
•
More formally: Let H[a,b] be:
Low influence functions are close to functions in
L2() = L2(N1,N2,…).
n{ f : Xn  [a, b] |  i: Ii(f) < , E[f] = 0, E[f2] = 1}
• Then: H ““ {f  L2() : E[f] =0, E[f2] = 1, a  f  b}
•  noise forms in H [a,b] ~ noise forms of [a, b]
•
bounded functions in L2()
Note: Our results generalize and strengthen limit
results for U and V statistics.
•
An Invariance Principle
For example, we prove:
• Invariance Principle [M+O’Donnell+Oleszkiewicz(05)]:
• Let p(x) = 0 < |S| · k aS i 2 S xi be a degree k multilinear polynomial with |p|2 = 1 and Ii(p)   for all i.
• Let X = (X1,…,Xn) be i.i.d. P[Xi =  1] = 1/2 .
N = (N1,…,Nn) be i.i.d. Normal(0,1).
• Then for all t:
|P[p(X) · t] - P[p(N) · t]| · O(k 1/(4k))
•
•
Note: Noise form “kills” high order monomials.
Proof works for any hyper-contractive random vars.
Invariance Principle – Proof Sketch
•
Suffices to show that 8 smooth F (sup |F(4)| · C ),
E[F(p(X1,…,Xn)] is close to E[F(p(N1,…,Nn))].
•
Very similar to Lindberg proof of CLT (also Rotar,
Chatterjee in non-linear settings).
Invariance Principle – Proof Sketch
•
•
•
•
•
•
•
•
•
Write: p(X1,…,Xi-1, Ni, Ni+1,…,Nn) = R + Ni S
p(X1,…,Xi-1, Xi, Ni+1,…,Nn) = R + Xi S
F(R+Ni S) = F(R) + F’(R) Ni S + F’’(R) Ni2 S2/2 +
F(3)(R) Ni3 S3/6 + F(4)(*) Ni4 S4/24
E[F(R+ Ni S)] = E[F(R)] + E[F’’(R)] E[Ni2] /2 + E[F(4)(*)Ni4S4]/24
E[F(R + Xi S)] = E[F(R)] + E[F’’(R)] E[Xi2] /2 + E[F(4)(*)Xi4 S4]/24
|E[F(R + Ni S) – E[F(R + Xi S)|  C E[S4]
But, E[S2] = Ii(p).
And by Hyper-Contractivity, E[S4]  9k-1 E[S2]
So: |E[F(R + Ni S) – E[F(R + Xi S)|  C 9k Ii2
•
•
Thm1 (“Double-Bubble”):
•
•
Thm2 (“Peace Sign”):
Double bubbles
Among all pairs of disjoint sets A,B
with n(A) = a, n(B) = b, the minimizer
of -1( A   B) is a “Double Bubble”
Among all partitions A,B,C of Rn with
(A) = (B) = (C) = 1/3 , the minimum
of ( A   B   C) is obtained for
the “Peace Sign”
• 1.
Hutchings, Morgan, Ritore, Ros. + Reichardt,
Heilmann, Lai, Spielman 2. Corneli, Corwin, Hurder,
Sesum, Xu, Adams, Dvais, Lee, Vissochi
•
•
Conj:
•
•
The Peace-Sign Conjecture
For all 0    1,
all n  2
The maximum of
<A, A> + <B, B> + <C, C>
among all partitions (A,B,C) of Rn
with n(A) = n(B) = n(C) = 1/3 is
obtained for
(A,B,C) = “Peace Sign”
•
•
Conj:
•
•
The Peace-Sign Conjecture
For all 0    1,
all n  2
The maximum of
<A, A> + <B, B> + <C, C>
among all partitions (A,B,C) of Rn
with n(A) = n(B) = n(C) = 1/3 is
obtained for
(A,B,C) = “Peace Sign”
Summary
• Prove the “Peace Sign Conjecture” (Isoperimetry)
•  “Plurality is Stablest” (Low Inf Bounds)
•  MAX-3-CUT hardness (CS) and voting.
(more modest: almost optimal results using
isoperimetric theory).
•
•
•
Other possible application of invariance principle:
To Convex Geometry?
To Additive Number Theory?
•
Papers and Collaborators
Noise Stability of Functions with low influences:
Invariance and Optimality
M-O’Donnell-Oleszkiewicz.
•
Conditional Hardness for Approximate Coloring
Dinur-M-Regev.
•
Optimal Inapproximability results for MAX-CUT and 2CSPproblems?
Khot-Kindler-M-O’Donnell