1. of 39 February 22, 2012 Invariance in Property Testing: Yale 1 Invariance in Property Testing Madhu Sudan Microsoft Research TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A AA Based on: works with/of Eli Ben-Sasson, Elena Grigorescu, Tali Kaufman, Shachar Lovett, Ghid Maatouk, Amir Shpilka.

2. of 39 Property Testing Sublinear time algorithms: Sublinear time algorithms: Algorithms running in time o(input), o(output). Algorithms running in time o(input), o(output). Probabilistic. Probabilistic. Correct on (approximation) to input. Correct on (approximation) to input. Input given by oracle, output implicit. Input given by oracle, output implicit. Crucial to modern context Crucial to modern context (Massive data, no time). (Massive data, no time). Property testing: Property testing: Restriction of sublinear time algorithms to decision problems (output = YES/NO). Restriction of sublinear time algorithms to decision problems (output = YES/NO). Amazing fact: Many non-trivial algorithms exist! Amazing fact: Many non-trivial algorithms exist! February 22, 2012 Invariance in Property Testing: Yale 2

3. of 39 Example 1: Polling Is the majority of the population Red/Blue Is the majority of the population Red/Blue Can find out by random sampling. Can find out by random sampling. Sample size / margin of error Sample size / margin of error Independent of size of population Independent of size of population Other similar examples: (can estimate other moments …) Other similar examples: (can estimate other moments …) February 22, 2012 Invariance in Property Testing: Yale 3

4. of 39 Example 2: Linearity Can test for homomorphisms: Can test for homomorphisms: Given: f: G  H (G,H finite groups), is f essentially a homomorphism? Given: f: G  H (G,H finite groups), is f essentially a homomorphism? Test: Test: Pick x,y in G uniformly, ind. at random; Pick x,y in G uniformly, ind. at random; Verify f(x) ¢ f(y) = f(x ¢ y) Verify f(x) ¢ f(y) = f(x ¢ y) Completeness: accepts homomorphisms w.p. 1 Completeness: accepts homomorphisms w.p. 1 (Obvious) (Obvious) Soundness: Rejects f w.p prob. Proportional to its “distance” (margin) from homomorphisms. Soundness: Rejects f w.p prob. Proportional to its “distance” (margin) from homomorphisms. (Not obvious, [BlumLubyRubinfeld’90]) (Not obvious, [BlumLubyRubinfeld’90]) February 22, 2012 Invariance in Property Testing: Yale 4

5. of 39 February 22, 2012 Invariance in Property Testing: Yale 5 History ( slightly abbreviated ) [Blum,Luby,Rubinfeld – S’90] [Blum,Luby,Rubinfeld – S’90] Linearity + application to program testing Linearity + application to program testing [Babai,Fortnow,Lund – F’90] [Babai,Fortnow,Lund – F’90] Multilinearity + application to PCPs (MIP). Multilinearity + application to PCPs (MIP). [Rubinfeld+S.] [Rubinfeld+S.] Low-degree testing; Formal definition. Low-degree testing; Formal definition. [Goldreich,Goldwasser,Ron] [Goldreich,Goldwasser,Ron] Graph property testing; Systematic study. Graph property testing; Systematic study. Since then … many developments Since then … many developments More graph properties, statistical properties, matrix properties, properties of Boolean functions … More graph properties, statistical properties, matrix properties, properties of Boolean functions … More algebraic properties More algebraic properties

6. of 39 Aside: Story of graph property testing Initiated by [GoldreichGoldwasserRon] (dense graphs) and [GoldreichRon] (sparse graphs). Initiated by [GoldreichGoldwasserRon] (dense graphs) and [GoldreichRon] (sparse graphs). Focus of intensive research. Focus of intensive research. Near classification for dense graphs: Near classification for dense graphs: Works of Alon, Shapira and others (relate to Szemerdi’s regularity lemma) Works of Alon, Shapira and others (relate to Szemerdi’s regularity lemma) Works of Lovasz, B. Szegedy and others (theory of graph limits). Works of Lovasz, B. Szegedy and others (theory of graph limits). Significant progress in sparse graph case also: Significant progress in sparse graph case also: [Benjamini, Schramm, Shapira] [Benjamini, Schramm, Shapira] [Nguyen, Onak] [Nguyen, Onak] February 22, 2012 Invariance in Property Testing: Yale 6

7. of 39 Pictorial Summary February 22, 2012 Invariance in Property Testing: Yale 7 All properties Statistical Properties Linearity Low-degree Graph Properties Boolean functions Testable! Not-testable

8. of 39 Some (introspective) questions What is qualitatively novel about linearity testing relative to classical statistics? What is qualitatively novel about linearity testing relative to classical statistics? Why are the mathematical underpinnings of different themes so different? Why are the mathematical underpinnings of different themes so different? Why is there no analog of “graph property testing” (broad class of properties, totally classified wrt testability) in algebraic world? Why is there no analog of “graph property testing” (broad class of properties, totally classified wrt testability) in algebraic world? February 22, 2012 Invariance in Property Testing: Yale 8

9. of 39 Invariance? Property P µ {f : D  R} Property P µ {f : D  R} Property P invariant under permutation (function) ¼ : D  D, if Property P invariant under permutation (function) ¼ : D  D, if f 2 P ) f ο ¼ 2 P Property P invariant under group G if Property P invariant under group G if 8 ¼ 2 G, P is invariant under ¼. 8 ¼ 2 G, P is invariant under ¼. Observation: Different property tests unified/separated by invariance class. Observation: Different property tests unified/separated by invariance class. February 22, 2012 Invariance in Property Testing: Yale 9

10. of 39 Pictorial Summary February 22, 2012 Invariance in Property Testing: Yale 10 All properties Statistical Properties Linearity/Low-deg Graph Invariance Boolean invariance Testable! Not-testable

11. of 39 Invariances (contd.) Some examples: Some examples: Classical statistics: Invariant under all permutations. Classical statistics: Invariant under all permutations. Graph properties: Invariant under vertex renaming. Graph properties: Invariant under vertex renaming. Boolean properties: Invariant under variable renaming. Boolean properties: Invariant under variable renaming. Matrix properties: Invariant under mult. by invertible matrix. Matrix properties: Invariant under mult. by invertible matrix. Algebraic Properties = ? Algebraic Properties = ? Goals: Goals: Possibly generalize specific results. Possibly generalize specific results. Get characterizations within each class? Get characterizations within each class? In algebraic case, get new (useful) codes? In algebraic case, get new (useful) codes? February 22, 2012 Invariance in Property Testing: Yale 11

12. of 39 Abstracting Linearity/Low-degree tests Affine Invariance: Affine Invariance: Domain = Big field (GF(2 n )) Domain = Big field (GF(2 n )) or vector space over small field (GF(2) n ). or vector space over small field (GF(2) n ). Property invariant under affine transformations of domain (x  A.x + b) Property invariant under affine transformations of domain (x  A.x + b) Linearity: Linearity: Range = small field (GF(2)) Range = small field (GF(2)) Property = vector space over range. Property = vector space over range. February 22, 2012 Invariance in Property Testing: Yale 12

13. of 39 February 22, 2012 Invariance in Property Testing: Yale 13 Testing Linear Properties Algebraic Property = Code! (usually) Universe Universe: {f:D  R} P Don’t care Must reject Must accept P R is a field F; P is linear!

14. of 39 Why study affine-invariance? Common abstraction of properties studied in [BLR], [RS], [ALMSS], [AKKLR], [KR], [KL], [JPRZ]. Common abstraction of properties studied in [BLR], [RS], [ALMSS], [AKKLR], [KR], [KL], [JPRZ]. (Variations on low-degree polynomials) (Variations on low-degree polynomials) Hopes Hopes Unify existing proofs Unify existing proofs Classify/characterize testability Classify/characterize testability Find new testable codes (w. novel parameters) Find new testable codes (w. novel parameters) Rest of the talk: Brief summary of findings Rest of the talk: Brief summary of findings February 22, 2012 Invariance in Property Testing: Yale 14

15. of 39 Basic terminology Local Constraint: Local Constraint: Example: f(1) + f(2) = f(3). Example: f(1) + f(2) = f(3). Necessary for testing Linear Properties [BHR] Necessary for testing Linear Properties [BHR] Local Characterization: Local Characterization: Example: 8 x, y, f(x) + f(y) = f(x+y), f 2 P Example: 8 x, y, f(x) + f(y) = f(x+y), f 2 P Aka: LDPC code, k-CNF property etc. Aka: LDPC code, k-CNF property etc. Necessary for affine-invariant linear properties. Necessary for affine-invariant linear properties. Single-orbit characterization: Single-orbit characterization: One linear constraint + implications by affine- invariance. One linear constraint + implications by affine- invariance. Feature in all previous algebraic properties. Feature in all previous algebraic properties. February 22, 2012 Invariance in Property Testing: Yale 15

16. of 39 t-local constraint t-characterized Affine-invariance & testability February 22, 2012 Invariance in Property Testing: Yale 16 t-locally testable t-S-O-C

17. of 39 State of the art in 2007 [AKKLR]: t-constraint = t’-testable, for all linear affine-invariant properties? [AKKLR]: t-constraint = t’-testable, for all linear affine-invariant properties? February 22, 2012 Invariance in Property Testing: Yale 17

18. of 39 t-local constraint t-characterized Affine-invariance & testability February 22, 2012 Invariance in Property Testing: Yale 18 t-locally testable t-S-O-C

19. of 39 Some results [Kaufman+S.’07]: Single-orbit ) Testable. [Kaufman+S.’07]: Single-orbit ) Testable. Next few slides: the Proof Next few slides: the Proof February 22, 2012 Invariance in Property Testing: Yale 19

20. of 39 Property P (t-S-O-C) given by ® 1,…, ® t ; V 2 F t Property P (t-S-O-C) given by ® 1,…, ® t ; V 2 F t P = {f | f(A( ® 1 )) … f(A( ® t )) 2 V, 8 affine A:K n K n } P = {f | f(A( ® 1 )) … f(A( ® t )) 2 V, 8 affine A:K n K n } Rej(f) = Prob A [ f(A( ® 1 )) … f(A( ® t )) not in V ] Rej(f) = Prob A [ f(A( ® 1 )) … f(A( ® t )) not in V ] Wish to show: If Rej(f) < 1/t 3, Wish to show: If Rej(f) < 1/t 3, then δ(f,P) = O(Rej(f)). then δ(f,P) = O(Rej(f)). February 22, 2012 Invariance in Property Testing: Yale 20

21. of 39 Proof: BLR Analog Rej(f) = Pr x,y [ f(x) + f(y) ≠ f(x+y)] < ² Rej(f) = Pr x,y [ f(x) + f(y) ≠ f(x+y)] < ² Define g(x) = majority y {Vote x (y)}, Define g(x) = majority y {Vote x (y)}, where Vote x (y) = f(x+y) – f(y). where Vote x (y) = f(x+y) – f(y). Step 0: Show δ(f,g) small Step 0: Show δ(f,g) small Step 1: 8 x, Pr y,z [Vote x (y) ≠ Vote x (z)] small. Step 1: 8 x, Pr y,z [Vote x (y) ≠ Vote x (z)] small. Step 2: Use above to show g is well-defined and a homomorphism. Step 2: Use above to show g is well-defined and a homomorphism. February 22, 2012 Invariance in Property Testing: Yale 21

22. of 39 Proof: BLR Analysis of Step 1 Why is f(x+y) – f(y) = f(x+z) – f(z), usually? Why is f(x+y) – f(y) = f(x+z) – f(z), usually? February 22, 2012 Invariance in Property Testing: Yale 22 - f(x+z) f(y) - f(x+y) f(z) -f(y) f(x+y+z) -f(z) 0 ?

23. of 39 Proof: Generalization g(x) = ¯ that maximizes, over A s.t. A( ® 1 ) = x, g(x) = ¯ that maximizes, over A s.t. A( ® 1 ) = x, Pr A [ ¯,f(A( ® 2 ),…,f(A( ® t )) 2 V] Pr A [ ¯,f(A( ® 2 ),…,f(A( ® t )) 2 V] Step 0: δ(f,g) small. Step 0: δ(f,g) small. Vote x (A) = ¯ s.t. ¯, f(A( ® 2 ))…f(A( ® t )) 2 V Vote x (A) = ¯ s.t. ¯, f(A( ® 2 ))…f(A( ® t )) 2 V (if such ¯ exists) (if such ¯ exists) Step 1 (key): 8 x, whp Vote x (A) = Vote x (B). Step 1 (key): 8 x, whp Vote x (A) = Vote x (B). Step 2: Use above to show g 2 P. Step 2: Use above to show g 2 P. February 22, 2012 Invariance in Property Testing: Yale 23

24. of 39 Proof: Matrix Magic? February 22, 2012 Invariance in Property Testing: Yale 24 A( ® 2 ) B( ® t ) B( ® 2 ) A( ® t ) x l Say A( ® 1 ) … A( ® l ) independent; rest dependent l Random No Choice Doesn’t Matter!

25. of 39 Some results [Kaufman+S.’07]: Single-orbit ) Testable. [Kaufman+S.’07]: Single-orbit ) Testable. February 22, 2012 Invariance in Property Testing: Yale 25

26. of 39 t-local constraint t-characterized Affine-invariance & testability February 22, 2012 Invariance in Property Testing: Yale 26 t-locally testable t-S-O-C [KS’08]

27. of 39 Some results [Kaufman+S.’07]: Single-orbit ) Testable. [Kaufman+S.’07]: Single-orbit ) Testable. Unifies known algebraic testing results. Unifies known algebraic testing results. Converts testability to purely algebraic terms. Converts testability to purely algebraic terms. Yields “Constraints = Char. = Testability” for vector spaces over small fields. Yields “Constraints = Char. = Testability” for vector spaces over small fields. Left open: Domain = Big field. Left open: Domain = Big field. 9 Many “non-polynomial” testable properties 9 Many “non-polynomial” testable properties [GKS’08]: Over big fields, Constraint ≠ Char. [GKS’08]: Over big fields, Constraint ≠ Char. [BMSS’11]: Over big fields, Char ≠ Testability. [BMSS’11]: Over big fields, Char ≠ Testability. [BGMSS’11]: Many questions/conjectures outlining a possible characterization of affine- invariant properties. [BGMSS’11]: Many questions/conjectures outlining a possible characterization of affine- invariant properties. February 22, 2012 Invariance in Property Testing: Yale 27

28. of 39 [BS’10]t-local constraint t-characterized Affine-invariance & testability February 22, 2012 Invariance in Property Testing: Yale 28 t-locally testable t-S-O-C [KS’08] [GKS’08] [BMSS’11]weight-t degrees

29. of 39 State of knowledge over big fields February 22, 2012 Invariance in Property Testing: Yale 29

30. of 39 Quest in lower bound Proposition: For every affine-invariant property P, there exists a set of degrees D s.t. Proposition: For every affine-invariant property P, there exists a set of degrees D s.t. P = {polynomials supported on monomials in D} Quest: Given degree set D (shadow-closed, shift- closed) prove it has no S-O-C. Quest: Given degree set D (shadow-closed, shift- closed) prove it has no S-O-C. Equivalently: Prove there are no Equivalently: Prove there are no ¸ 1 … ¸ t є F, ® 1 … ® t є K such that ¸ 1 … ¸ t є F, ® 1 … ® t є K such that  i=1 t ¸ i ® i d = 0 for every d є D.  i=1 t ¸ i ® i d = 0 for every d є D.  i=1 t ¸ i ® i d ≠ 0 for every minimal d  D.  i=1 t ¸ i ® i d ≠ 0 for every minimal d  D. February 22, 2012 Invariance in Property Testing: Yale 30

31. of 39 Pictorially February 22, 2012 Invariance in Property Testing: Yale 31 ®1d®1d ®2d®2d ®td®td … M(D) = Is there a vector ( ¸ 1,…, ¸ t ) in its right kernel? Can try to prove “NO” by proving matrix has full rank. Unfortunately, few techniques to prove non-square matrix has high rank.

32. of 39 Structure of Degree sets February 22, 2012 Invariance in Property Testing: Yale 32

33. of 39 Non-testable Property - 1 AKKLR ( Alon,Kaufman,Krivelevich,Litsyn,Ron ) Conjecture: AKKLR ( Alon,Kaufman,Krivelevich,Litsyn,Ron ) Conjecture: If a linear property is 2-transitive and has a k- local constraint then it is testable. If a linear property is 2-transitive and has a k- local constraint then it is testable. [GKS’08]: For every k, there exists affine- invariant property with 8-local constraint that is not k-locally testable. [GKS’08]: For every k, there exists affine- invariant property with 8-local constraint that is not k-locally testable. Range = GF(2); Domain = GF(2 n ) Range = GF(2); Domain = GF(2 n ) P = Fam(Shift({0,1} [ {1+2,1+2 2,…,1+2 k })). P = Fam(Shift({0,1} [ {1+2,1+2 2,…,1+2 k })). February 22, 2012 Invariance in Property Testing: Yale 33

34. of 39 Proof (based on [BMSS’11]) F = GF(2); K = GF(2 n ); F = GF(2); K = GF(2 n ); P k = Fam(Shift({0,1} [ {1 + 2 i | i є {1,…,k}})) P k = Fam(Shift({0,1} [ {1 + 2 i | i є {1,…,k}})) Let M i = Let M i = If Ker(M i ) = Ker(M i+1 ), then Ker(M i+2 ) = Ker(M i ) If Ker(M i ) = Ker(M i+1 ), then Ker(M i+2 ) = Ker(M i ) Ker(M k+1 ) = would accept all functions in P k+1 Ker(M k+1 ) = would accept all functions in P k+1 So Ker(M i ) must go down at each step, implying Rank(M i+1 ) > Rank(M i ). So Ker(M i ) must go down at each step, implying Rank(M i+1 ) > Rank(M i ). February 22, 2012 Invariance in Property Testing: Yale 34 ®12i®12i … ®12®12 ®22®22 ®k2®k2 … ®22i®22i ®k2i®k2i

35. of 39 Stronger Counterexample GKS counterexample: GKS counterexample: Takes AKKLR question too literally; Takes AKKLR question too literally; Of course, a non-locally-characterizable property can not be locally tested. Of course, a non-locally-characterizable property can not be locally tested. Weaker conjecture: Weaker conjecture: Every k-locally characterized affine-invariant (2-transitive) property is locally testable. Every k-locally characterized affine-invariant (2-transitive) property is locally testable. Alas, not true: [BMSS] Alas, not true: [BMSS] February 22, 2012 Invariance in Property Testing: Yale 35

36. of 39 [BMSS] CounterExample Recall: Recall: Every known locally characterized property was locally testable Every known locally characterized property was locally testable Every known locally testable property is S-O-C. Every known locally testable property is S-O-C. Need a locally characterized property which is (provably) not S-O-C. Need a locally characterized property which is (provably) not S-O-C. Idea: Idea: Start with sparse family P i. Start with sparse family P i. Lift it to get Q i (still S-O-C). Lift it to get Q i (still S-O-C). Take intersection of superconstantly many such properties. Q =  i Q i Take intersection of superconstantly many such properties. Q =  i Q i February 22, 2012 Invariance in Property Testing: Yale 36

37. of 39 Example: Sums of S-O-C properties Suppose D 1 = Deg(P 1 ) and D 2 = Deg(P 2 ) Suppose D 1 = Deg(P 1 ) and D 2 = Deg(P 2 ) Then Deg(P 1 + P 2 ) = D 1 [ D 2. Then Deg(P 1 + P 2 ) = D 1 [ D 2. Suppose S-O-C of P 1 is C 1 : f(a 1 ) + … + f(a k ) = 0; and S-O-C of P 2 is C 2 : f(b 1 ) + … + f(b k ) = 0. Suppose S-O-C of P 1 is C 1 : f(a 1 ) + … + f(a k ) = 0; and S-O-C of P 2 is C 2 : f(b 1 ) + … + f(b k ) = 0. Then every g є P 1 + P 2 satisfies: Then every g є P 1 + P 2 satisfies:  i,j g(a i b j ) = 0  i,j g(a i b j ) = 0 Doesn’t yield S-O-C, but applied to random constraints in orbit(C 1 ), orbit(C 2 ) does! Doesn’t yield S-O-C, but applied to random constraints in orbit(C 1 ), orbit(C 2 ) does! Proof uses wt(Deg(P 1 )) ≤ k. Proof uses wt(Deg(P 1 )) ≤ k. February 22, 2012 Invariance in Property Testing: Yale 37

38. of 39 Hopes Get a complete characterization of locally testable affine-invariant properties. Get a complete characterization of locally testable affine-invariant properties. Use codes of (polynomially large?) locality to build better LTCs/PCPs? Use codes of (polynomially large?) locality to build better LTCs/PCPs? In particular move from “domain = vector space” to “domain = field”. In particular move from “domain = vector space” to “domain = field”. Codes invariant under other groups? Codes invariant under other groups? [KaufmanWigderson], [KaufmanLubotzky] [KaufmanWigderson], [KaufmanLubotzky] Yield symmetric LDPC codes, but not yet LTCs. Yield symmetric LDPC codes, but not yet LTCs. More broadly: Apply lens of invariance more broadly to property testing. More broadly: Apply lens of invariance more broadly to property testing. February 22, 2012 Invariance in Property Testing: Yale 38

39. of 39 Thank You! February 22, 2012 Invariance in Property Testing: Yale 39