March 20-24, 2010 Babai-Fest: Invariance in Property Testing 1 Invariance in Property Testing Madhu Sudan Microsoft/MIT TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A

Happy Birthday, Laci! Thanks for … Thanks for … All the wonderful results ! All the wonderful results ! (PCP, Alg. Group Theory) (PCP, Alg. Group Theory) The inspiring talks ! The inspiring talks ! The entertaining + educational writings ! The entertaining + educational writings ! March 20-24, 2010 Babai-Fest: Invariance in Property Testing 2

Modern challenge to Algorithm Design Data = Massive; Computers = Tiny Data = Massive; Computers = Tiny How can tiny computers analyze massive data? How can tiny computers analyze massive data? Only option: Design sublinear time algorithms. Only option: Design sublinear time algorithms. Algorithms that take less time to analyze data, than it takes to read/write all the data. Algorithms that take less time to analyze data, than it takes to read/write all the data. Can such algorithms exist? Can such algorithms exist? December 2, 2009 IPAM: Invariance in Property Testing 3

Yes! Polling … Is the majority of the population Red/Blue Is the majority of the population Red/Blue Can find out by random sampling. Can find out by random sampling. Sample size / margin of error Sample size / margin of error Independent of size of population Independent of size of population Other similar examples: (can estimate other moments …) Other similar examples: (can estimate other moments …) December 2, 2009 IPAM: Invariance in Property Testing 4

Recent “novel” example Can test for homomorphisms: Can test for homomorphisms: Given: f: G  H (G,H finite groups), is f essentially a homomorphism? Given: f: G  H (G,H finite groups), is f essentially a homomorphism? Test: Test: Pick x,y in G uniformly, ind. at random; Pick x,y in G uniformly, ind. at random; Verify f(x) ¢ f(y) = f(x ¢ y) Verify f(x) ¢ f(y) = f(x ¢ y) Completeness: accepts homomorphisms w.p. 1 Completeness: accepts homomorphisms w.p. 1 (Obvious) (Obvious) Soundness: Rejects f w.p prob. Proportional to its “distance” (margin) from homomorphisms. Soundness: Rejects f w.p prob. Proportional to its “distance” (margin) from homomorphisms. (Not obvious) (Not obvious) December 2, 2009 IPAM: Invariance in Property Testing 5

March 20-24, 2010 Babai-Fest: Invariance in Property Testing 6 Property Testing Data = a function from D to R: Data = a function from D to R: Property P µ {D  R} Property P µ {D  R} Distance Distance δ(f,g) = Pr x 2 D [f(x) ≠ g(x)] δ(f,g) = Pr x 2 D [f(x) ≠ g(x)] δ(f,P) = min g 2 P [δ(f,g)] δ(f,P) = min g 2 P [δ(f,g)] f is ε-close to g (f ¼ ² g) iff δ(f,g) · ε. f is ε-close to g (f ¼ ² g) iff δ(f,g) · ε. Local testability: Local testability: P is (k, ε, δ)-locally testable if 9 k-query test T P is (k, ε, δ)-locally testable if 9 k-query test T f 2 P ) T f accepts w.p. 1-ε. f 2 P ) T f accepts w.p. 1-ε. δ(f,P) > δ ) T f accepts w.p. ε. δ(f,P) > δ ) T f accepts w.p. ε. Notes: want k(ε, δ) = O(1) for ε,δ= (1). Notes: want k(ε, δ) = O(1) for ε,δ= (1).

March 20-24, 2010 Babai-Fest: Invariance in Property Testing 7 Brief History [Blum,Luby,Rubinfeld – S’90] [Blum,Luby,Rubinfeld – S’90] Linearity + application to program testing Linearity + application to program testing [Babai,Fortnow,Lund – F’90] [Babai,Fortnow,Lund – F’90] Multilinearity + application to PCPs (MIP). Multilinearity + application to PCPs (MIP). [Rubinfeld+S.] [Rubinfeld+S.] Low-degree testing Low-degree testing [Goldreich,Goldwasser,Ron] [Goldreich,Goldwasser,Ron] Graph property testing Graph property testing Since then … many developments Since then … many developments Graph properties Graph properties Statistical properties Statistical properties … More algebraic properties More algebraic properties

Graph Property Testing Initiated by [GoldreichGoldwasserRon] Initiated by [GoldreichGoldwasserRon] Initial examples: Initial examples: Is graph bipartite? Is graph bipartite? Is it 3-colorable? Is it 3-colorable? Is it triangle-free (underlying theorem dates back to 80s)? Is it triangle-free (underlying theorem dates back to 80s)? Many intermediate results Many intermediate results Close ties to Szemeredi’s regularity lemma Close ties to Szemeredi’s regularity lemma Culmination: [AlonFisherNewmanSzegedy]: Culmination: [AlonFisherNewmanSzegedy]: Characterization of all testable properties in terms of regularity. Characterization of all testable properties in terms of regularity. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 8

March 20-24, 2010 Babai-Fest: Invariance in Property Testing 9 Specific Directions in Algebraic P.T. Fewer results Fewer results More Properties More Properties Low-degree (d < q) functions [RS] Low-degree (d < q) functions [RS] Moderate-degree (q < d < n) functions Moderate-degree (q < d < n) functions q=2: [AKKLR] q=2: [AKKLR] General q: [KR, JPRZ] General q: [KR, JPRZ] Long code/Dictator/Junta testing [BGS,PRS] Long code/Dictator/Junta testing [BGS,PRS] BCH codes (Trace of low-deg. poly.) [KL] BCH codes (Trace of low-deg. poly.) [KL] Better Parameters (motivated by PCPs). Better Parameters (motivated by PCPs). #queries, high-error, amortized query complexity, reduced randomness. #queries, high-error, amortized query complexity, reduced randomness.

My concerns … Relatively few results … Relatively few results … Why can’t we get “rich” class of properties that are all testable? Why can’t we get “rich” class of properties that are all testable? Why are proofs so specific to property being tested? Why are proofs so specific to property being tested? What made Graph Property Testing so well- understood? What made Graph Property Testing so well- understood? What is “novel” about Property Testing, when compared to “polling”? What is “novel” about Property Testing, when compared to “polling”? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 10

March 20-24, 2010 Babai-Fest: Invariance in Property Testing 11 Contrast w. Combinatorial P.T. Algebraic Property = Code! (usually) Universe Universe: {f:D  R} P Don’t care Must reject Must accept P R is a field F; P is linear!

Basic Implications of Linearity [BHR] If P is linear, then: If P is linear, then: Tester can be made non-adaptive. Tester can be made non-adaptive. Tester makes one-sided error Tester makes one-sided error (f 2 P ) tester always accepts). (f 2 P ) tester always accepts). Motivates: Motivates: Constraints: Constraints: k-query test => constraint of size k: k-query test => constraint of size k: value of f at ® 1,… ® k constrained to lie in subspace. value of f at ® 1,… ® k constrained to lie in subspace. Characterizations: Characterizations: If non-members of P rejected with positive probability, then P characterized by local constraints. If non-members of P rejected with positive probability, then P characterized by local constraints. functions satisfying all constraints are members of P. functions satisfying all constraints are members of P. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 12

f = assgm’t to left f = assgm’t to left Right = constraints Right = constraints Characterization of P: Characterization of P: P = {f sat. all constraints} P = {f sat. all constraints} Pictorially March 20-24, 2010 Babai-Fest: Invariance in Property Testing D {0000,1100, 0011,1111}

Sufficient conditions? Linearity + k-local characterization Linearity + k-local characterization ) k-local testability? ) k-local testability? [BHR] No! [BHR] No! Elegant use of expansion Elegant use of expansion Rule out obvious test; but also any test … of any “q(k)”-locality Rule out obvious test; but also any test … of any “q(k)”-locality Why is characterization insufficient? Why is characterization insufficient? Lack of symmetry? Lack of symmetry? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 14

Example motivating symmetry Conjecture (AKKLR ‘96): Conjecture (AKKLR ‘96): Suppose property P is a vector space over F 2 ; Suppose property P is a vector space over F 2 ; Suppose its “invariant group” is “2-transitive”. Suppose its “invariant group” is “2-transitive”. Suppose P satisfies a k-ary constraint Suppose P satisfies a k-ary constraint 8 f 2 P, f( ® 1 ) +  + f( ® k ) = 0. 8 f 2 P, f( ® 1 ) +  + f( ® k ) = 0. Then P is (q(k), ² (k,δ),δ)-locally testable. Then P is (q(k), ² (k,δ),δ)-locally testable. Inspired by “low-degree” test over F 2. Implied all previous algebraic tests (at least in weak forms). Inspired by “low-degree” test over F 2. Implied all previous algebraic tests (at least in weak forms). March 20-24, 2010 Babai-Fest: Invariance in Property Testing 15

Invariances Property P invariant under permutation (function) ¼ : D  D, if Property P invariant under permutation (function) ¼ : D  D, if f 2 P ) f ο ¼ 2 P Property P invariant under group G if Property P invariant under group G if 8 ¼ 2 G, P is invariant under ¼. 8 ¼ 2 G, P is invariant under ¼. Can ask: Does invariance of P w.r.t. “nice” G leads to local testability? Can ask: Does invariance of P w.r.t. “nice” G leads to local testability? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 16

Invariances are the key? “Polling” works well when (because) invariant group of property is the full symmetric group. “Polling” works well when (because) invariant group of property is the full symmetric group. Modern property tests work with much smaller group of invariances. Modern property tests work with much smaller group of invariances. Graph property ~ Invariant under vertex renaming. Graph property ~ Invariant under vertex renaming. Algebraic Properties & Invariances? Algebraic Properties & Invariances? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 17

Abstracting Algebraic Properties [Kaufman & S.] [Kaufman & S.] Range is a field F and P is F-linear. Range is a field F and P is F-linear. Domain is a vector space over F (or some field K extending F). Domain is a vector space over F (or some field K extending F). Property is invariant under affine (sometimes only linear) transformations of domain. Property is invariant under affine (sometimes only linear) transformations of domain. “Property characterized by single constraint, and its orbit under affine (or linear) transformations.” “Property characterized by single constraint, and its orbit under affine (or linear) transformations.” March 20-24, 2010 Babai-Fest: Invariance in Property Testing 18

Invariance, Orbits and Testability Single constraint implies many Single constraint implies many One for every permutation ¼ 2 Aut(P): One for every permutation ¼ 2 Aut(P): “Orbit of a constraint C” “Orbit of a constraint C” = {C ο ¼ | ¼ 2 Aut(P)} = {C ο ¼ | ¼ 2 Aut(P)} Extreme case: Extreme case: Property characterized by single constraint + its orbit: “Single orbit feature” Property characterized by single constraint + its orbit: “Single orbit feature” Most algebraic properties have this feature. Most algebraic properties have this feature. W.l.o.g. if domain = vector space over small field. W.l.o.g. if domain = vector space over small field. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 19

Example: Degree d polynomials Constraint: When restricted to a small dimensional affine subspace, function is polynomial of degree d (or less). Constraint: When restricted to a small dimensional affine subspace, function is polynomial of degree d (or less). #dimensions · d/(K - 1) #dimensions · d/(K - 1) Characterization: If a function satisfies above for every small dim. subspace, then it is a degree d polynomial. Characterization: If a function satisfies above for every small dim. subspace, then it is a degree d polynomial. Single orbit: Take constraint on any one subspace of dimension d/(K-1); and rotate over all affine transformations. Single orbit: Take constraint on any one subspace of dimension d/(K-1); and rotate over all affine transformations. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 20

Some results If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. Unifies previous algebraic tests (in weak form) with single proof. Unifies previous algebraic tests (in weak form) with single proof. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 21

Analysis of Invariance-based test Property P given by ® 1,…, ® k ; V 2 F k Property P given by ® 1,…, ® k ; V 2 F k P = {f | f(A( ® 1 )) … f(A( ® k )) 2 V, 8 affine A:K n K n } P = {f | f(A( ® 1 )) … f(A( ® k )) 2 V, 8 affine A:K n K n } Rej(f) = Prob A [ f(A( ® 1 )) … f(A( ® k )) not in V ] Rej(f) = Prob A [ f(A( ® 1 )) … f(A( ® k )) not in V ] Wish to show: If Rej(f) < 1/k 3, Wish to show: If Rej(f) < 1/k 3, then δ(f,P) = O(Rej(f)). then δ(f,P) = O(Rej(f)). March 20-24, 2010 Babai-Fest: Invariance in Property Testing 22

BLR Analog Rej(f) = Pr x,y [ f(x) + f(y) ≠ f(x+y)] < ² Rej(f) = Pr x,y [ f(x) + f(y) ≠ f(x+y)] < ² Define g(x) = majority y {Vote x (y)}, Define g(x) = majority y {Vote x (y)}, where Vote x (y) = f(x+y) – f(y). where Vote x (y) = f(x+y) – f(y). Step 0: Show δ(f,g) small Step 0: Show δ(f,g) small Step 1: 8 x, Pr y,z [Vote x (y) ≠ Vote x (z)] small. Step 1: 8 x, Pr y,z [Vote x (y) ≠ Vote x (z)] small. Step 2: Use above to show g is well-defined and a homomorphism. Step 2: Use above to show g is well-defined and a homomorphism. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 23

BLR Analysis of Step 1 Why is f(x+y) – f(y) = f(x+z) – f(z), usually? Why is f(x+y) – f(y) = f(x+z) – f(z), usually? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 24 - f(x+z) f(y) - f(x+y) f(z) -f(y) f(x+y+z) -f(z) 0 ?

Generalization g(x) = ¯ that maximizes, over A s.t. A( ® 1 ) = x, g(x) = ¯ that maximizes, over A s.t. A( ® 1 ) = x, Pr A [ ¯,f(A( ® 2 ),…,f(A( ® k )) 2 V] Pr A [ ¯,f(A( ® 2 ),…,f(A( ® k )) 2 V] Step 0: δ(f,g) small. Step 0: δ(f,g) small. Vote x (A) = ¯ s.t. ¯, f(A( ® 2 ))…f(A( ® k )) 2 V Vote x (A) = ¯ s.t. ¯, f(A( ® 2 ))…f(A( ® k )) 2 V (if such ¯ exists) (if such ¯ exists) Step 1 (key): 8 x, whp Vote x (A) = Vote x (B). Step 1 (key): 8 x, whp Vote x (A) = Vote x (B). Step 2: Use above to show g 2 P. Step 2: Use above to show g 2 P. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 25

Matrix Magic? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 26 A( ® 2 ) B( ® k ) B( ® 2 ) A( ® k ) x t Say A( ® 1 ) … A( ® t ) independent; rest dependent t Random No Choice Doesn’t Matter!

Some results If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. If P is affine-invariant and has k-single orbit feature (characterized by orbit of single k-local constraint); then it is (k, δ/k 3, δ)-locally testable. Unifies previous algebraic tests with single proof. Unifies previous algebraic tests with single proof. If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) (explains the AKKLR optimism) (explains the AKKLR optimism) March 20-24, 2010 Babai-Fest: Invariance in Property Testing 27

Subsequent results [GrigorescuKaufmanS.; CCC08]: Counterexample to AKKLR Conjecture [GrigorescuKaufmanS.; CCC08]: Counterexample to AKKLR Conjecture [GrigorescuKaufmanS., Random09]: Single orbit characterization of some BCH (and other) codes. [GrigorescuKaufmanS., Random09]: Single orbit characterization of some BCH (and other) codes. [Ben-SassonS.]: Limitations on rate of affine- invariant codes. [Ben-SassonS.]: Limitations on rate of affine- invariant codes. [KaufmanWigderson]: LDPC codes with invariance (not affine-invariant) [KaufmanWigderson]: LDPC codes with invariance (not affine-invariant) [BhattacharyyaChenS.Xie, Shapira]: Affine- invariant non-linear properties. [BhattacharyyaChenS.Xie, Shapira]: Affine- invariant non-linear properties. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 28

Broad directions to consider Is every locally characterized affine-invariant property testable? Is every locally characterized affine-invariant property testable? Is every single-orbit characterized affine- invariant property testable? Is every single-orbit characterized affine- invariant property testable? What groups of invariances lead to testability? What groups of invariances lead to testability? In general … seek invariances In general … seek invariances March 20-24, 2010 Babai-Fest: Invariance in Property Testing 29

March 20-24, 2010 Babai-Fest: Invariance in Property Testing 30 Thanks

Results (contd.) If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) Proof Ingredients: Proof Ingredients: Analysis of all affine invariant properties. Analysis of all affine invariant properties. Rough characterization of locality of constraints, in terms of degrees of polynomials in the family. Rough characterization of locality of constraints, in terms of degrees of polynomials in the family. Infinitely many (new) properties … Infinitely many (new) properties … March 20-24, 2010 Babai-Fest: Invariance in Property Testing 31

More details Understanding invariant properties: Understanding invariant properties: Recall: all functions from K n to F are Traces of polynomials Recall: all functions from K n to F are Traces of polynomials ( Trace(x) = x + x p + x p 2 + … + x q/p ( Trace(x) = x + x p + x p 2 + … + x q/p where K = F q and F = F p ) where K = F q and F = F p ) If P contains Tr(3x 5 + 4x 2 + 2); then P contains Tr(4x 2 ) … If P contains Tr(3x 5 + 4x 2 + 2); then P contains Tr(4x 2 ) … So affine invariant properties characterized by degree of monomials in family. So affine invariant properties characterized by degree of monomials in family. Most of the study … relate degrees to upper and lower bounds on locality of constraints. Most of the study … relate degrees to upper and lower bounds on locality of constraints. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 32

Some results If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) If P is affine-invariant over K and has a single k- local constraint, then it is has a q-single orbit feature (for some q = q(K,k)) (explains the AKKLR optimism) (explains the AKKLR optimism) Unfortunately, q depends inherently on K, not just F … giving counterexample to AKKLR conjecture [joint with Grigorescu & Kaufman] Unfortunately, q depends inherently on K, not just F … giving counterexample to AKKLR conjecture [joint with Grigorescu & Kaufman] Linear invariance when P is not F-linear: Linear invariance when P is not F-linear: Abstraction of some aspects of Green’s regularity lemma … [ Bhattacharyya, Chen, S., Xie ] Abstraction of some aspects of Green’s regularity lemma … [ Bhattacharyya, Chen, S., Xie ] Nice results due to [Shapira] Nice results due to [Shapira] March 20-24, 2010 Babai-Fest: Invariance in Property Testing 33

More results Invariance of some standard codes Invariance of some standard codes E.g. “dual-BCH”: Have k-single orbit feature! So are “more uniformly” testable. E.g. “dual-BCH”: Have k-single orbit feature! So are “more uniformly” testable. [Grigorescu, Kaufman, S.] [Grigorescu, Kaufman, S.] Side effect: New (essentially tight) relationships between Rej AKKLR (f) and δ(f,Degree-d) over F 2 [with Bhattacharyya, Kopparty, Schoenebeck, Zuckerman] Side effect: New (essentially tight) relationships between Rej AKKLR (f) and δ(f,Degree-d) over F 2 [with Bhattacharyya, Kopparty, Schoenebeck, Zuckerman] March 20-24, 2010 Babai-Fest: Invariance in Property Testing 34

More results (contd.) One hope: Could lead to “simple, good locally testable code”? One hope: Could lead to “simple, good locally testable code”? (Sadly, not with affine-inv. [Ben-Sasson, S.]) (Sadly, not with affine-inv. [Ben-Sasson, S.]) Still … other groups could be used? [Kaufman+Wigderson] Still … other groups could be used? [Kaufman+Wigderson] Open: Is every locally characterized affine- invariant property locally testable? Open: Is every locally characterized affine- invariant property locally testable? March 20-24, 2010 Babai-Fest: Invariance in Property Testing 35

Conclusions Invariance seems to be a nice perspective on “property testing” … Invariance seems to be a nice perspective on “property testing” … Certainly helps unify many algebraic property tests. Certainly helps unify many algebraic property tests. But should be a general lens in sublinear time algorithmics. But should be a general lens in sublinear time algorithmics. March 20-24, 2010 Babai-Fest: Invariance in Property Testing 36