Transcript On Gallager s problem: new bounds for noisy communication.

On Gallager’s problem:
New Bounds for Noisy Communication.
Guy Kindler
Microsoft Research
Joint work with
Navin Goyal & Mike Saks
“Noise is the chief product and the
authenticating sign of civilization”
Ambrose Bierce
1842 – 1914(?)
 In CS: Noise appears in the study of information theory,
network design, learning theory, cryptography, quantum
computation, hardness of approximation, theory of social
choice, embeddings of metric spaces, privacy in databases…
In this talk
 [El Gamal ’84]: The noisy broadcast network model.
 [Gallager ’88]: n¢loglog(n) algorithm for identity.
 Main result: Gallager’s algorithm is tight.
 Proof by reduction:
Generalized noisy decision trees (gnd-trees).
Lower bound for gnd-trees.
First, a Fourier-analytic result
 Definition (Fourier): Let f:{-1,1}n!{-1,1} be a Boolean function.
The i’th Fourier coefficient of f:
fi = Ex»U[f(x)¢xi].
 [Talagrand ’96]: Let p= Prx»U[f(x)=1], (p<1/2).
Then i (fi)2· p2log(1/p).
 Crucial for our result!
(as hinted in slide #26..)
What next:
Communication under noise - examples
The noisy broadcast model
Gallager: the algorithm and the problem
Gnd-trees: Generalized Noisy Decision Trees
Our results
About the proof
Noisy computation: case 1
01100
1.
Noiseless channel: n transmissions.
2. naïve: n¢log(n)
(error is polynomially small in n)
3. [Shannon ’48]: c¢n (error is exponentially small in n)
Aggregation of bits: Big advantage
Noisy computation: case 2
y=10101
1.
x=01100
Noiseless channel:
k transmissions.
2.
naïve: k¢log(k)
3. [Schulman ’96]: c¢k
Goal: compute f(x,y)
(error is exponentially small in k)
The Noisy Broadcast Model
[El Gamal ’84]
 Input: x1,..,xn.
x3
 One bit transmitted at a time.
0
 Error rate:  (small const.).
 Goal: compute g(x1,..,xn).
 In this talk: we want to
compute x1,..,xn.
01
x4
x5
0 x2
1 0
1
1
01
x6
Order of transmissions is predefined
x1
0
1
01
00
x7
1
1
01
x8
x9
x10
Some history
 Computing identity:
x4
0
Naïve solution: n log n (repetition)
[Gallager ’88]: n loglog n.
 [Yao 97]: Try thresholds first.
 [KM ’98]: Any threshold in O(n).
x3
1
x5
x6
x2
1 x
1
1
1
1 x1
1
0
1
Gallager’s problem:
1
x9
x
7
Can this be made
linear?
x
8
 In adversarial model:
[FK ’00]: OR in O(n¢log*n).
[N ’04]: OR in O(n).
Fails for “adversarial noise”.
what’s next:
Communication under noise - examples
The noisy broadcast model
Gallager: an algorithm and a problem
Gnd-trees: Generalized Noisy Decision Trees
Statement of results
About the proof
Generalized Noisy Decision (gnd) Trees
 Input: x, but access is to noisy
f(x1)
copies x1,x2, x3…
xi=x©Ni (Ni flips xj w.p. )
f0(x2)
f1(x1)
 Any Boolean queries!
Goal: compute g(x),
f00(x2)
f01(x1)
f10(x1)
f11(x2)
minimizing depth(T)
v=“01”
g(x)=.. g(x)=..
fv : Boolean function
g(x)=y
g(x)=.. g(x)=.. g(x)=.. g(x)=.. g(x)=..
Generalized Noisy Decision (gnd) Trees
 Noisy decision trees [FPRU ‘94]:
f(x1)
Query noisy coordinates of x.
 Identity computable in nlog(n).
f00(x2)
g(x)=.. g(x)=..
f0(x2)
f01(x1)
g(x)=y
f1(x1)
f10(x1)
f11(x2)
g(x)=.. g(x)=.. g(x)=.. g(x)=.. g(x)=..
Some bounds for noisy trees
f(x1)
f0(x2)
f00(x2)
g(x)=..
function
noisy trees
(n) [FPRU]
OR
MAJORITY (n log n) [FPRU]
PARITY
IDENTITY
(n log n) [FPRU]
g(x)=..
f01(x1)
g(x)=y
g(x)=..
gnd-trees
(n)
O(n) [FPRU]
O(n) [KM*]
(n)
(n)
[GKS]
(n log n) [FPRU] (n log n) [GKS]
f1(x1)
f10(x1)
g(x)=..
g(x)=..
f11(x2)
g(x)=..
g(x)=..
Our results
 Main theorem: (n¢loglog(n)) bound for identity in n.b. network.
 Lower-bound for gnd-tree : 3nlog(n) Lower bound for computing
identity in generalized decision trees.
 Reduction theorem:
kn time protocol in -noise n.b. network
)
2kn depth gnd-tree for noise ck.
 Proof of main theorem:
2kn ¸ ckn log n
2k(1/)ck ¸ log n
k= (loglog(n))
what’s next:
About communication under noise
The noisy broadcast model
Gallager: the algorithm and the problem
Generalized Noisy Decision Trees (gnd-trees)
Our results
About the proof
About the proof:
The reduction:

A series of transformations from a broadcast protocol into a
gnd-tree protocol.
About the proof:
The reduction:

A series of transformations from a broadcast protocol into a
gnd-tree protocol.
Gnd-tree lower bound:

Defining a knowledge measure.

Bounding knowledge measure by depth of tree.
Lower bound for gnd-trees
 Our claim: A gnd-tree which computes identity on x=x1,..,xn
requires (3n¢log n) depth.
 We actually prove: If depth(T)·3n¢log n then
Prx» U[T returns x] <(),
(lim!0 ()=0)
f(x1)
The big picture
f0(x2)
f00(x2)
 We prove: If depth(T)·3n¢log n then
Prx»U[T returns x] <(),
 Structure of such proofs:
1.
g(x)=..
g(x)=..
f01(x1)
g(x)=y
g(x)=..
g(x)=..
g(x)=..
f11(x2)
g(x)=..
 is the leaf reached by T.
each noisy copy is queried once…
2. Show: T correct only if w.h.p. Mx() > t
work needed in
case) M ()<t
3. Show: (more
If depth(T)<<nlog
n,general
then w.h.p.
x
In our case:
and typically
f10(x1)
( lim!0()=0 )
Define:Disclaimer:
Knowledge
Wemeasure
considerM
case
x(v) where
t = log(n),
f1(x1)
Mx(v,a)-Mx(v) · 1/(3¢n)
g(x)=..
f(x1)
Perceived probability
f0(x2)
f00(x2)
g(x)=..
 Perceived probability (“likelihood”) of x:
Lx(v)=Pr[x|visit(v)]
 Pr[x|visit(v)] is “multiplicative”.
g(x)=..
f01(x1)
g(x)=y
g(x)=..
f1(x1)
f10(x1)
g(x)=..
g(x)=..
f11(x2)
g(x)=..
g(x)=..
Knowledge measure: 1st attempt
f00(x2)
 Log likelihood of x:
g(x)=..
g(x)=..
f(x1)
f0(x2)
f01(x1)
g(x)=y
g(x)=..
LLx(v)= n + log(Lx(v))
LLx(root)=0,
LLx()¸ n – const
 We’d like to show: Typically, LLx(v,a)-LLx(v) < 1/log(n).
 But:
 Reason:
After n coordinate queries, LLx¼(n).
x is quickly separated from far away points.
Separating x from neighbors is the hardest.
f1(x1)
f10(x1)
g(x)=..
g(x)=..
f11(x2)
g(x)=..
g(x)=..
Knowledge measure: seriously
f(x1)
f0(x2)
f00(x2)
 Log likelihood “gradient” at x:
g(x)=..
g(x)=..
f1(x1)
f01(x1)
g(x)=y
g(x)=..
f10(x1)
g(x)=..
Mix(v)= log(Lx(v))-log(Lx©i(v))
Mx(v)= AVGi (Mix(v))
= log(Lx(v)) - AVGi ( log(Lx©i(v)) )
Mx(root)=0 ,
Mx()¸ log(n) - c
All that is left: typical gain in Mx is at most 1/n.
g(x)=..
f11(x2)
g(x)=..
g(x)=..
Gain in knowledge measure
v
f(x5)
a=1.
v,1
v,0
Gain in knowledge measure
Mix(v,a)-Mix(v)= log(Lx(v,a))-log(Lx©i(v,a))
-( log(Lx(v))-log(Lx©i(v)) )
v
f(x5)
a=1.
v,1
v,0
Gain in knowledge measure
Mix(v,a)-Mix(v)
Mx(v,a)-Mx(v)=
 The coup des grâce: For every query
fv, x,
3n)
E[M
(v)]· 1/(
x(v,a)-Mxdepends
Expression
only
on f, x !
E[(Mx(v,a)-Mx(v))2]· 1/(3n)
 Proof: Adaptation of [Talagrand ‘96].
v
f(x5)
v,1
v,0
Main open problem
 Show lower bound for computing a Boolean function.
 Not known even for a random function!
 Generalize for other network designs.
Thank You !
Gallager’s solution, simplified
1. Partition to groups of size log(n)
2. Each player sends its bit loglog(n) times.
1,1,1
1
0,0,0
1
1
1
0
1
0
1
Gallager’s solution, simplified
1. Partition to groups of size log(n)
2. Each player sends its bit loglog(n) times.
0,0,0
10
1
10
1
1
11
1
0
10
1
0
1
Gallager’s solution, simplified
1. Partition to groups of size log(n)
2. Each player sends its bit loglog(n) times.
 W.h.p., in all groups, almost all players know all bits.
1001
10
1001
10
1
1101
11
0
1001
10
0
1
Gallager’s solution, simplified
3. Each group transmits error correcting code of its bits:
* Each player transmits a constant number of bits.
4. W.h.p. all players now know all bits of all groups.
1,0,0
1001
1001
1
0,1,0
1,1,1
1101
0
0,1,1
1001
0
suppose code(1001)=100 111 100 011
1
The reduction
 The program:
Start with a noisy broadcast protocol with kn steps.
Gradually, simlulate protocol in more “tree-like” models.
 W.l.o.g., assume each node performs 10k transmissions.
 first step: each transmission is replaced by three, only one of
which is noisy.
The reduction
 First step: each transmission is replaced by three, only one
of which is noisy.
x3
b
Function of x3, and of
past receptions.
x3
x3, b(0),b(1)
b(0), b(1) transmitted
noise free.
The reduction
 Second step: noisy transmissions moved to beginning of
protocol.
x3
b
Function of x3, and of
past receptions.
x3
x3, x3, x3,..
x3
b(0),b(1)
The reduction
 Second step: noisy transmissions moved to beginning of
protocol.
x3
x3, x3, x3,..
x3
b(0),b(1)
 After noisy phase: each player has 10k noisy copies of each
bit.
 Equivalent to having an k-noisy copy of x.
The reduction
 Third step: each player begins with an k-noisy copy of x.
x3,x©N
x3, x33,..
Equivalent to
a gnd tree!
b(0),b(1)
 Each transmission depends on transmitter’s noisy copy,
and past transmissions (and perhaps a random decision).
Gain in progress measure
Mix(v)= log(Pr[x|visit(v)])-log(Pr[x© i|visit(v)])
v
f(x5)
v,1
v,0
Gain in progress measure
Mix(v)
a=f(x5) : a random variable
Only depends on f !.
Mix(v,a)-Mix(v)=
v
f(x5)
v,1
v,0