Beer Therapy

 At Oberwolfach in 2003, Ralf Kötter and Madhu Sudan a week long beer drinking competition. had  Who do you think won?

Ralf Madhu Vote early, vote often

Local Algorithms & Error-correction

Madhu Sudan MIT

Beer Therapy

 At Oberwolfach in 2003, Ralf Kötter and Madhu Sudan a week long beer drinking competition. had  Who do you think won?

Ralf Madhu Information Computation

Dedicated to Ralf Kötter

• Dear friend to many … • Wise beyond his age • Happy spirit •… I ’ ll miss him dearly.

•… I already do.

Prelude

• Algorithmic Problems in Coding Theory • New Paradigm in Algorithms • The Marriage: Local Error-Detection & Correction February 11, 2009 Local Error-Correction 5

 Code: E : § k !

§ n R(C) = k=n, ; Image(E ) = C µ § n ; ±(C) = normalized distance.

 Encoding: Fix Code C and associat ed E : Given m 2 § k , comput e E (m).

§ k !

§ n .

  Given x 2 § n , decide if 9m 2 § k s.t . x = E (m).

Given x 2 § n , decide if 9m 2 § k s.t .

±(E (m); x) · ².

Given x 2 § n , comput e m 2 § k t hat minimizes ±(E (m); x) (provided ±(E (m); x) · ²).

February 11, 2009 Local Error-Correction 6

Sublinear time algorithmics

 f : f 0; 1g o(k; n) k j !

f 0; 1g n x j f

f (x)

i 0 ) where x 0 i ¼ x i Answer 1: Clearly NO, since that is the time it 1.

Present input implicitly (by an oracle). 2.

Represent output implicitly 3.

Compute function on approximation to input.

Extends to computing relations as well.

February 11, 2009 Local Error-Correction 7

Sub-linear time algorithms

   Initiated in late eighties in context of  Program checking [ BlumKannan,BlumLubyRubinfeld ]  Interactive Proofs/PCPs [ BabaiFortnowLund ] Now successful in many more contexts   Property testing/Graph-theoretic algorithms Sorting/Searching   Statistics/Entropy computations (High-dim.) Computational geometry Many initial results are coding-theoretic!

February 11, 2009 Local Error-Correction 8

Sub-linear time algorithms & Coding

 Encoding: Not reasonable to expect in sub-linear time.

 Testing? Decoding? – Can be done in sublinear time.

 In fact many initial results do so!

 Codes that admit efficient …   … testing: Locally Testable Codes (LTCs) … decoding: Locally Decodable Codes (LDCs).

February 11, 2009 Local Error-Correction 9

Rest of this talk

     Definitions of LDCs and LTCs Quick description of known results The first result: Hadamard codes Some basic constructions Recent constructions of LDCs.

 [Yekhanin,Raghavendra,Efremenko] February 11, 2009 Local Error-Correction 10

Definitions

February 11, 2009 Local Error-Correction 11

Locally Decodable Code Code:

C : § k !

§ n is (q; ²)-Locally Decodable if 9 Decoder D s.t . given i 2 [k] and oracle w s.t . 9 m ±(w; C(m)) · ² · ±(C)=2, n w D (i ) reads q(n) random posit ions of w and out put s m i w.p. at least 2=3.

What if ² > ±(C)=2? Might need to report a list of upt o ` codewords.

February 11, 2009 Local Error-Correction 12

Locally List-Decodable Code Code:

C is ( ²; `)-list-decodable if 8w 2 § # codewords c 2 C s.t .

n , ±(w; c) · ² is at most `.

C is (q; ²; `)-locally list-decodable if 9 Decoder D s.t.

given i 2 [k] and j 2 [`] and oracle w s.t .

m 1 ; : : : ; m ` are all messages sat isfying ±(w; C(m j n )) · ² w D (i ; j ) reads q(n) random posit ions of w and out put s (m j ) i w.p. at least 2=3.

February 11, 2009 Local Error-Correction 13

History of definitions

   Constructions predate formal definitions  [Goldreich-Levin ’89].

  [Beaver-Feigenbaum ’90, Lipton ’91].

[Blum-Luby-Rubinfeld ’90].

Hints at definition (in particular, interpretation in the context of error-correcting codes): [Babai Fortnow-Levin-Szegedy ’91].

Formal definitions   [S.-Trevisan-Vadhan ’99] (local list-decoding).

[Katz-Trevisan ’00] February 11, 2009 Local Error-Correction 14

Locally Testable Codes Code:

C µ § n is (q; ²)-Locally Testable if 9 Test er T s.t .

n w T reads q(n) random posit ions: ² If w 2 C accepts w.p. 1.

² If w is ²-far from C, then rejects w.p. ¸ 1=2.

“Weak” definition: hinted at in [BFLS], explicit in [RS’96, Arora’94, Spielman’94, FS’95].

February 11, 2009 Local Error-Correction 15

Strong Locally Testable Codes Code:

C µ § n is (q; ²)-Locally Testable if 9 Test er T s.t .

n w T reads q(n) random posit ions: ² If w 2 C accepts w.p. 1.

² For every w 2 § n , T reject s w.p.

¸ ( ±(w; C)).

“Strong” Definition: [Goldreich-S. ’02] February 11, 2009 Local Error-Correction 16

Motivations

February 11, 2009 Local Error-Correction 17

Local decoding: Average-case vs. worst-case

² Suppose C µ § N is locally-decodable code for N = 2 n .

(Furt her assume can locally decode bit s of t he codeword, and not just bit s of t he message.) ² c 2 C can be viewed as function c : f 0; 1g n !

§ .

² Local decoding ¼) can compute c(x) for every x, if one can comput e c(x 0 ) for most x 0 . Relat es average case complexity t o worst -case. [Lipt on, ST V] ² Alternate interpretation: Compute c(x) without re vealing x. Leads t o Inst ance Hiding [BF], Privat e Informat ion Ret rieval [CGK S].

February 11, 2009 Local Error-Correction 18

Motivation for Local-testing

  No generic applications known.

However,    Interesting phenomenon on its own.

Intangible connection to Probabilistically Checkable Proofs (PCPs).

Potentially good approach to understanding limitations of PCPs (though all resulting work has led to improvements).

February 11, 2009 Local Error-Correction 19

Contrast between decoding and testing

  Decoding: Testing: Property of words near codewords.

Property of words far from code.

  Decoding:  Motivations happy with n = quasi-poly(k), and q = poly log n.  Testing: Better tradeoffs possible! Likely more useful in practice.

 Lower bounds show q = O(1) and n = nearly linear(k) impossible.

Even conceivable: n = O(k) with q = O(1)?

February 11, 2009 Local Error-Correction 20

Some LDCs and LTCs

February 11, 2009 Local Error-Correction 21

Hadamard (1 st Order RM) Codes Message:

(Coe ± cients of) Linear Functions L from F k 2 t o F 2 .

Encoding:

evaluat ions of L on all of F k 2 .

2 k -bit codewords ¡ L (y) 2-Locally Decodable [Folklore/ Exercise] 3-Locally Test able [BlumLubyRubinfeld] February 11, 2009 Local Error-Correction 22

Hadamard (1 st Order RM) Codes

 Conclusions:  There exist infinite families of codes  With constant locality (for testing and correcting).

February 11, 2009 Local Error-Correction 23

Codes via Multivariate Polynomials Message:

over coe ± cients of deg t, m-variate polynomial P ¯nite ¯eld F F P (Reed Muller code) F m

Encoding:

evaluat ions of P on all of F m .

Parameters:

k ¼ (t=m) m , n = jFj m , ± ¸ t=jFj.

February 11, 2009 Local Error-Correction 24

Basic insight to locality

 m-variat e polynomial of degree t rest rict ed t o m 0 < m-dim. (a ± ne) subspace is polynomial of degree t.

 Local Decoding: Pick subspace t hrough point x of int erest , and decode on subspace.

 Query complexity q = jFj m m 0 ¿ m ) sublinear!

0 ; T ime = poly(q).

Local Testing: Verify f rest rict ed t o space is of degree t.

Same complexity.

February 11, 2009 Local Error-Correction 25

 Many parameters:  Many tradeoffs possible: Locality q wit h n = exp(k 1=( q ¡ 1) ) Locality (log k) 2 p Locality wit h n = k k wit h n = O(k).

4 February 11, 2009 Local Error-Correction 26

Are Polynomial Codes (Roughly) Best?

 No! [ Ambainis97 ] [ GoldreichS.00

] … 

No!! [

Beimel,Ishai,Kushilevitz,Raymond

]

 Really … Seriously …

No!!!!

[ Yekhanin07,Raghavendra08,Efremenko09 ] February 11, 2009 Local Error-Correction 27

Recent LDCs

[Yekhanin ‘07, Raghavendra ‘08, Efremenko ‘09]

February 11, 2009 Local Error-Correction 28

The recent result:

  Fix q = 3; n = ??? (as function of $k$) Till 2007:

n n ¼ exp(k

1=5

) (non-binary).

¼ exp( p k) (binary).



¼ exp(k

0:0000001

) (binary).

  [Raghavendra ’08]:

¼ exp(exp( p log k)) (binary).

February 11, 2009 Local Error-Correction 29

  

Essence of the idea:

Z m Build “good” combinatorial matrix over Z m F Embed in multiplicative subgroup of F Get locally decodable code over February 11, 2009 Local Error-Correction 30

A =

k

0 … … … … … 0 … 0 … 0 … 0

£ n matrix over Z

m

Zeroes on diagonal Non-zero o ®-diagonal

Columns closed under addition February 11, 2009 Local Error-Correction [ arbitrary 31

Embedding into field

¡ Let A = [a i j ] be \ good\ over Z m ¡ Let ! = primitive mth root of unity in F.

¡ Let G = [!

a i j ].

T heorem [Yekhanin,Raghavendra,Efremenko]: G generat es m query LDC over F.

Highly non-intuitive!

February 11, 2009 Local Error-Correction 32

Improvements

¡ A = [a i j ]; G = [!

a i j ] .

¡ O®-diagonal entries of A from S ) G generat es jSj + 1-query LDC.

¡ !

S ) (Suffices for [Efremenko]) zeroes of t-sparse polynomial over F G generat es t-query LDC.

(Critical to [Yekhanin]) February 11, 2009 Local Error-Correction 33

“Good” Matrices?

  [Yekhanin]:  Picked m prime.   Achieved

n = exp(k

1=j Sj

)

  Optimal if m prime!

Managed to make S large with t=3.

[Efremenko] 

jSj = 3 and n = exp(exp( p

 Achieves  ([Beigel,Barrington,Rudich];[Grolmusz]) Optimal?

log k))

February 11, 2009 Local Error-Correction 34

 

Limits to Local Decodability: Katz-Trevisan

) n = k

1+ ( 1=q)

.

q queries Technique:  Recall D(x) computes C(x) whp for all x.

  Can assume (with some modifications) that query pattern uniform for any fixed x.

Can find many random strings such that their query sets are disjoint.

  In such case, random subset of n^{1-1/q} coordinates of codeword contain at least one query set, for most x. Yields desired bound.

February 11, 2009 Local Error-Correction 35

Some general results

 Sparse, High-Distance Codes:  Are Locally Decodable and Testable  [KaufmanLitsyn, KaufmanS]  2-transitive codes of small dual-distance:  Are Locally Decodable  [Alon,Kaufman,Krivelevich,Litsyn,Ron]  Linear-invariant codes of small dual-distance:  Are also Locally Testable  [KaufmanS] February 11, 2009 Local Error-Correction 36

Summary

 Local algorithms in error-detection/correction lead to interesting new questions.

 Non-trivial progress so far.

 Limits largely unknown  O(1)-query LDCs must have Rate(C) = 0  [Katz-Trevisan] February 11, 2009 Local Error-Correction 37

Questions

 Can LTC replace RS (on your hard disks)?

 Is a significant rate-loss necessary?

  Lower bounds?

Better error models?

   Simple/General near optimal constructions?

Other applications to mathematics/computation? (PCPs necessary/sufficient)?

Lower bounds for LDCs?/Better constructions?

February 11, 2009 Local Error-Correction 38

Thank You!

February 11, 2009 Local Error-Correction 39