Transcript Amplification of stochastic advantage

Amplification of stochastic
advantage
Biased : known with certainty one of the
possible answer is always correct. Error
can be reduced by repeat the algorithm.
Unbiased example coin flip
for p-correct “advantage” is p - 1/2
Prabhas Chongstitvatana
1
Let MC be a 3/4-correct unbiased
What is the error prob. Of MC3 (mojority vote) ?
1
R
R
R
R
W
W
W
W
2
R
R
W
W
R
R
W
W
3
R
W
R
W
R
W
R
W
prob
27/64
9/64
9/64
3/64
9/64
3/64
3/64
1/64
MC3
R
R
R
W
R
W
W
W
MC3 is 27/32-correct
( > 84% )
Prabhas Chongstitvatana
2
What is the error prob. of MC with advantage
e > 0 in majority vote k times ?
Let Xi = 1 if correct answer, 0 otherwise
Pr[ Xi = 1 ] >= 1/2 + e
assume for simplicity Pr[ Xi = 1 ] = 1/2 + e ;
k is odd (no tie)
E( Xi ) = 1/2 + e;
Var(Xi ) = (1/2 + e) (1/2-e) = 1/4 - e2
Prabhas Chongstitvatana
3
k
Let
X   Xi
i 0
X is a random variable corresponds to the
number of correct answer is k trials.
 k  1

Pr[ X  i ]     e 

 i  2
E(X) = (1/2+e)k
i
1

  e
2 
k i
Var(X) = (1/4 - e2) k
Prabhas Chongstitvatana
4
error prob. Pr [ X <= k/2 ] can be calculated
k 2
 P r[X  i]
i 0
is normal distributed if k >= 30
Prabhas Chongstitvatana
5
If need error < 5%
Pr [ X < E(X) - 1.645 sqrt Var(X) ] ~ 5%
(from the table of normal distribution)
Pr [ X <= k/2 ] < 5% if
k/2 < E(X) - 1.645 sqrt Var(X)
k > 2.706 ( 1/(4e2) - 1 )
Prabhas Chongstitvatana
6
Example e = 5%, which is 55%-correct unbiased
Monte Carlo,
k > 2.706 ( 1/(4e2) - 1 )
k > 267.894,
majority vote repeat 269 times to obtain 95%correct.
Repetition turn 5% advantage into 5% error prob
Prabhas Chongstitvatana
7
It takes a large number of run for unbiased
Compare to biased
Run 55%-correct bias MC 4 times reduces the
error prob. To
0.454 ~ 0.041 ( 4.1% )
Prabhas Chongstitvatana
8
Not much more expensive to obtain more
confidence.
If we want 0.5% confidence (10 times more)
Pr[X < E(X) - 2.576 sqrt Var(X) ] ~ 5%
k > 6.636 (1/(4e2) - 1 )
This makes it 99.5%-correct with less than 2.5
times more expensive than 95%-correct.
Prabhas Chongstitvatana
9
To reduce error prob < del for an unbiased
Monte Carlo with advantage e;
the number of repetition is proportional to
1/e2, also to log 1/del
Prabhas Chongstitvatana
10