Transcript Algebraic Property Testing Madhu Sudan MIT CSAIL Joint work with Tali Kaufman (IAS).

Algebraic Property Testing
Madhu Sudan
MIT CSAIL
Joint work with Tali Kaufman (IAS).
Classical Data Processing
Big computers
Small Data
Modern Data Processing
Small computers
Enormous Data
Needs new algorithmic paradigm



Imbalance not a question of technology :
 I.e., not because computing speeds are
growing less fast than memory capacity.
Imbalance is a function of expectations:
 E.g., Users expect to be able to “analyze” the
WWW, using a laptop. But WWW includes
millions other such laptops.
Need: Sublinear time algorithms
 That “estimate” rather than “compute” some
given function.
Property Testing
Dat a: f : D ! R
f given by a sampling box
x
Property: F µ f f : D ! Rg
f
f(x)
q-query Test : Samples f -box q t imes.
Accept s if f 2 F
Hope:
Reject s if f 6
2F
Impossible wit h q ¿ jD j
Reject s if f is ±-far from F
f ±-close t o F if 9 g 2 F s.t . Pr x 2 D [f (x) 6
= g(x)] · ±
Property Testing
Dat a: f : D ! R
f given by a sampling box
x
Property: F µ f f : D ! Rg
f
f(x)
q-query Test : Samples f -box q t imes.
Accept s if f 2 F
Hope:
Reject s if f 6
2F
Impossible wit h q ¿ jD j
Reject s if f is ±-far from F
f ±-close t o F if 9 g 2 F s.t . Pr x 2 D [f (x) 6
= g(x)] · ±
Example: Linearity Testing
f f : Fn ! F2 g
2

[Blum, Luby, Rubinfeld ’90]
D = Fn ; R = F2
2

Property = Linearity
F = f f j8x; yf (x) + f (y) = f (x + y)g

Test:
Pick random x; y
Accept if f (x) + f (y) = f (x + y)
F

Non-trivial analysis:
f ±-far from F ) reject w.p. 2±=9
Example: Linearity Testing
f f : Fn ! F2 g
2

[Blum, Luby, Rubinfeld ’90]
D = Fn ; R = F2
2

Property = Linearity
F = f f j8x; yf (x) + f (y) = f (x + y)g

Test:
Pick random x; y
Accept if f (x) + f (y) = f (x + y)
F

Non-trivial analysis:
f ±-far from F ) reject w.p. 2±=9
Property Testing: Abbreviated History





Prehistoric: Statistical sampling
 E.g., “Is mean/median at least X”.
Linearity Testing [BLR’90], Multilinearity Testing
[Babai, Fortnow, Lund ’91].
Graph/Combinatorial Property Testing [Goldreich,
Goldwasser, Ron ’94].
 E.g., Is a graph “close” to being 3-colorable.
Algebraic Testing [GLRSW,RS,FS,AKKLR,KR,JPSZ]
 Is multivariate function a polynomial (of bounded
degree).
Graph Testing [Alon-Shapira, AFNS, Borgs et al.]
 Characterizes graph properties that are testable.
This Talk



Abstracting Algebraic Property Testing
Generic
If
F µ FnTheorem:
! F is closed under addit ion,
and under a± ne t ranformat ions of t he coordinat es,
and is locally charact erized
t hen it is t est able.
Motivations:



Generalizes, unifies previous algebraic works
Initiates systematic study of testability for algebraic
properties
Sheds light on testing and invariances of properties.
Property Testing vs. “Statistics”

Classical Statistics (Mean, Median, Quantiles):




Also run in sublinear time.
So what’s special about “linearity testing”?
Classical statistics work on “symmetric”
properties:
F closed under arbit rary permut at ion on D .
Linear functions closed under much smaller
group of permutations:
F closed under linear maps L : Fn ! Fn
2
2
jD j l og j D j such maps vs. jD j!.

Simlarly, graph properties have nice invariances
Is testing a corollary of invariance?



Example 1:
D = R; F 1 = f I djI d(x) = xg

Invariant group trivial. Testing easy.

Invariant group still trivial. No O(1) local tests.
D = f 1; : : : ; ng; R = f 1; : : : ; n 9 g;
Example 2: F = f f jx < y ) f (x) < f (y)g
2
Conclusion: Testing not necessarily a
consequence of invariance.

If we believe this to be the case for the linearity test,
must prove it!!
Is testing a corollary of invariance?



Example 1:
D = R; F 1 = f I djI d(x) = xg

Invariant group trivial. Testing easy.

Invariant group still trivial. No O(1) local tests.
D = f 1; : : : ; ng; R = f 1; : : : ; n 9 g;
Example 2: F = f f jx < y ) f (x) < f (y)g
2
Conclusion: Testing not necessarily a
consequence of invariance.

If we believe this to be the case for the linearity test,
must prove it!!
Part II: Formal Definitions & Results
Linear Invariance
F is Linear Invariant if
² F is a linear subspace (of Fj Fj n )
² f 2 F and L : Fn ! Fn linear ) f ± L 2 F
(A± ne Invariance de¯ned similarly)

Examples:
²
²
²
²
Linear funct ions,
n-variat e polynomials of degree · d,
homogenous polynomialsof degree d,
F1 + F2
Testing, constraints, characterizations
² Suppose F has a k-query t est .
² T hen members of F sat isfy a k-local const raint .
Const raint : C= (x 1 ; : : : ; x k 2 Fn ; subspace V µ Fk )
8 f 2 F ; f sat is¯es C i.e., hf (x 1 ); : : : ; f (x k )i 2 V
² E.g., in t he linear case: f (®) + f (¯) = f (® + ¯)
C = (®; ¯; ® + ¯; V = f 000; 011; 101; 110g)
Charact erizat ion: C = f C1 ; : : : ; Cm g
f 2 F , f sat is¯es C1 ; : : : ; Cm :
(Linear-Invariant) Algebraic Characterizations


Characterizations require many constraints!
Linear (affine) invariance turns one constraint
into many.
C = (x 1 ; : : : ; x k ; V ) const raint and L linear (a± ne)
) C ± L = (L (x 1 ); : : : ; L (x k ); V ) is also a const raint .


(Linearity)
(L
(®); L (¯); Example:
L (® + ¯); V ) const raint for linearity
Algebraic Charact erizat ion: Single const raint C s.t .
f C ± L jL linear (a± ne) g charact erize F .
Main Theorems
T heorem: F a± ne-invariant
and has k-local algebraic charact erizat ion,
implies it has a k-query property t est .
² Uni¯es, simpli¯es, and ext ends previous algebraic t est s
T heorem: F a± ne-invariant
and has k-local const raint
) F has f F (k)-local algebraic charact erizat ion.
Pictorially
For a± ne-invariant
propert ies F
k-query t est
Theorem 1
k! k
By defn.
k-local alg. char.
k-local charact erizat ion

By defn.
k-local const raint
Theorem 2
k ! f F (k)
X
k! k
(with Grigorescu)
Pictorially
For a± ne-invariant
propert ies F
k-query t est
Theorem 1
k! k
By defn.
k-local alg. char.
k-local charact erizat ion

By defn.
k-local const raint
Theorem 2
k ! f F (k)
X
k! k
(with Grigorescu)
Part III: BLR (and our) analysis
BLR Analysis: Outline
² Have f s.t . Pr x ;y [f (x) + f (y) 6
= f (x + y)] = ± < 2=9.
Want t o show f close t o some g 2 F .
² De¯ne g(x) = most likely y f f (x + y) ¡ f (y)g.
² If f close t o F t hen g will be in F and close t o f .
² But if f not close? g may not even be uniquely de¯ned!
² St eps:
¡ St ep 0: Prove f close t o g
¡
Step 1: Prove “most likely” is overwhelming majority.
¡ St ep 2: Prove t hat g is in F .
BLR Analysis: Step 0
² De¯ne g(x) = most likely y f f (x + y) ¡ f (y)g.
Claim: Pr x [f (x) 6
= g(x)] · 2±
¡ Let B = f xj Pr y [f (x) 6
= f (x + y)f (y)] ¸
¡ Pr x ;y [linearity t est reject s jx 2 B ] ¸
) Pr x [x 2 B ] · 2±
¡ If x 6
2 B t hen f (x) = g(x)
1
2
1g
2
Vot ex (y)
BLR Analysis: Step 1
² De¯ne g(x) = most likely y f f (x + y) ¡ f (y)g.
² Suppose for some x, 9 two equally likely values.
Presumably, only one leads t o linear x, so which one?
² If we wish t o show g linear,
t hen need t o rule out t his case.
Lemma: 8 x, Pr y ;z [Vot ex (y) 6
= Vot ex (z))] · 4±
Vot ex (y)
BLR Analysis: Step 1
² De¯ne g(x) = most likely y f f (x + y) ¡ f (y)g.
² Suppose for some x, 9 two equally likely values.
Presumably, only one leads t o linear x, so which one?
² If we wish t o show g linear,
t hen need t o rule out t his case.
Lemma: 8 x, Pr y ;z [Vot ex (y) 6
= Vot ex (z))] · 4±
Vot ex (y)
BLR Analysis: Step 1
² De¯ne g(x) = most likely y f f (x + y) ¡ f (y)g.
Lemma: 8 x, Pr y ;z [Vot ex (y) 6
= Vot ex (z))] · 4±
?
f (z)
f (y)
¡ f (x + y)
f (y + z) ¡ f (y + 2z)
¡ f (x + z) ¡ f (2y + z) f (x + 2y + 2z)
Prob. Row/ column
sum non-zero · ±.
Vot ex (y)
BLR Analysis: Step 1
² De¯ne g(x) = most likely y f f (x + y) ¡ f (y)g.
Lemma: 8 x, Pr y ;z [Vot ex (y) 6
= Vot ex (z))] · 4±
?
f (z)
f (y)
¡ f (x + y)
f (y + z) ¡ f (y + 2z)
¡ f (x + z) ¡ f (2y + z) f (x + 2y + 2z)
Prob. Row/ column
sum non-zero · ±.
BLR Analysis: Step 2 (Similar)
Lemma: If ± <
g(x)
f (z)
1 ,
20
t hen 8 x; y, g(x) + g(y) = g(x + y)
g(y)
¡ g(x + y)
f (y + z) ¡ f (y + 2z)
¡ f (x + z) ¡ f (2y + z) f (x + 2y + 2z)
Prob. Row/ column
sum non-zero · 4±.
Our Analysis: Outline
² f s.t . Pr L [hf (L (x 1 ); : : : ; f (L (x k ))i 2 V ] = ± ¿ 1.
² De¯ne g(x) = ® t hat maximizes
Pr f L j L ( x ) = x g [h®; f (L (x 2 )); : : : ; f (L (x k ))i 2 V ]
1
² St eps:
¡ St ep 0: Prove f close t o g
¡
Step 1: Prove “most likely” is overwhelming majority.
¡ St ep 2: Prove t hat g is in F .
Our Analysis: Outline
² f s.t . Pr L [hf (L (x 1 ); : : : ; f (L (x k ))i 2 V ] = ± ¿ 1.
² De¯ne g(x) = ® t hat maximizes
Pr f L j L ( x ) = x g [h®; f (L (x 2 )); : : : ; f (L (x k ))i 2 V ]
1
Same as before
² St eps:
¡ St ep 0: Prove f close t o g
¡
Step 1: Prove “most likely” is overwhelming majority.
¡ St ep 2: Prove t hat g is in F .
Vot ex (L )
Matrix
² De¯ne Magic?
g(x) = ® t hat maximizes
Pr f L j L ( x
)= x g
1
[h®; f (L (x 2 )); : : : ; f (L (x k ))i 2 V ]
Lemma: 8 x, Pr L ;K [Vot ex (L ) 6
= Vot ex (K ))] · 2(k ¡ 1)±
x
K (x 2 )
..
.
K (x k )
L (x 2 )
?
¢¢¢
L (x k )
Matrix Magic?
x
K (x 2 )
..
.
L (x 2 )
?
¢¢¢
L (x k )
K (x k )
² Want marked rows t o be random const raint s.
² Suppose x 1 ; : : : ; x ` linearly independent ;
and rest dependent on t hem.
Matrix Magic?
`

Fill with random entries

Fill so as to make constraints

x
`
L (x 2 )
¢¢¢
Linear algebra implies final
columns are also constraints.
L (x k )
K (x 2 )
..
.
K (x k )
² Suppose x 1 ; : : : ; x ` linearly independent ;
and rest dependent on t hem.
Matrix Magic?
`

Fill with random entries

Fill so as to make constraints

x
`
L (x 2 )
¢¢¢
Linear algebra implies final
columns are also constraints.
L (x k )
K (x 2 )
..
.
K (x k )
² Suppose x 1 ; : : : ; x ` linearly independent ;
and rest dependent on t hem.
Conclusions



Linear/Affine-invariant properties testable if they
have local constraints.
Gives clean generalization of linearity and lowdegree tests.
Future work: What kind of invariances lead to
testability (from characterizations)?