of 29 April 8, 2016 Two Decades of Property Testing 1 Madhu Sudan Harvard TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A AAA

of 29 Kepler’s Big Data Problem Tycho Brahe (~ ): Tycho Brahe (~ ): Wished to measure planetary motion accurately. Wished to measure planetary motion accurately. To confirm sun revolved around earth … (+ other planets around sun) To confirm sun revolved around earth … (+ other planets around sun) Spent 10% of Danish GNP Spent 10% of Danish GNP Johannes Kepler (~ s): Johannes Kepler (~ s): Believed Copernicus’s picture: planets in circular orbits. Believed Copernicus’s picture: planets in circular orbits. Addendum: Ratio of orbits based on Löwner-John ratios of platonic solids. Addendum: Ratio of orbits based on Löwner-John ratios of platonic solids. “Stole” Brahe’s data (1601). “Stole” Brahe’s data (1601). Worked on it for nine years. Worked on it for nine years. Disproved Addendum; Confirmed Copernicus (circle -> ellipse); discovered laws of planetary motion. Disproved Addendum; Confirmed Copernicus (circle -> ellipse); discovered laws of planetary motion. Nine Years? Nine Years? April 8, 2016 Two Decades of Property Testing 2 Source: Michael Fowler, “Galileo & Einstein”, U. Virginia

of 29 The challenge of analyzing big data Standard method: Standard method: Propose concept class. Propose concept class. LEARN (parameters of) best fitting concept in class to data in hand. LEARN (parameters of) best fitting concept in class to data in hand. TEST to see if this is a good enough fit. TEST to see if this is a good enough fit. Bottleneck Bottleneck LEARNing is expensive; wasted if TEST rejects. LEARNing is expensive; wasted if TEST rejects. Can we TEST before we LEARN? Can we TEST before we LEARN? Yes: This is PROPERTY TESTING!! Yes: This is PROPERTY TESTING!! April 8, 2016 Two Decades of Property Testing 3 Don’t be Ridiculous!

of 29 Property Testing April 8, 2016 Two Decades of Property Testing 4

of 29 Example 1: Polling April 8, 2016 Two Decades of Property Testing 5

of 29 Example 2: Linearity April 8, 2016 Two Decades of Property Testing 6

of 29 Linearity Analysis April 8, 2016 Two Decades of Property Testing 7

of 29 A subtle change April 8, 2016 Two Decades of Property Testing 8

of 29 Key step of BLR analysis April 8, 2016 Two Decades of Property Testing 9

of 29 April 8, 2016 Two Decades of Property Testing 10 History ( slightly abbreviated ) [Blum,Luby,Rubinfeld – S’90] [Blum,Luby,Rubinfeld – S’90] Linearity + application to program testing Linearity + application to program testing [Babai,Fortnow,Lund – F’90] [Babai,Fortnow,Lund – F’90] Multilinearity + application to PCPs (MIP). Multilinearity + application to PCPs (MIP). [Rubinfeld+S.] [Rubinfeld+S.] Low-degree testing + Definition Low-degree testing + Definition [Goldreich,Goldwasser,Ron] [Goldreich,Goldwasser,Ron] Graph property testing + systematic study Graph property testing + systematic study Since then … many developments Since then … many developments More graph properties, statistical properties, matrix properties, properties of Boolean functions … More graph properties, statistical properties, matrix properties, properties of Boolean functions … More algebraic properties More algebraic properties

of 29 April 8, 2016 Two Decades of Property Testing 11

of 29 Example 4: Long code/Junta testing April 8, 2016 Two Decades of Property Testing 12

of 29 Example 5: Distribution Testing April 8, 2016 Two Decades of Property Testing 13

of 29 What is Property Testing? April 8, 2016 Two Decades of Property Testing 14 Algebra Graphs + Regularity Statistics + CLT Matrices + Linear algebra

of 29 (Dense) Graph Property Testing April 8, 2016 Two Decades of Property Testing 15

of 29 Why no unification? Contrast with Low-degree testing April 8, 2016 Two Decades of Property Testing 16

of 29 Aside: Importance of Low-degree Testing April 8, 2016 Two Decades of Property Testing 17

of 29 Some (introspective) questions What is qualitatively novel about linearity testing relative to classical statistics? What is qualitatively novel about linearity testing relative to classical statistics? Why are the mathematical underpinnings of different themes so different? Why are the mathematical underpinnings of different themes so different? Why is there no analog of “graph property testing” (broad class of properties, totally classified wrt testability) in algebraic world? Why is there no analog of “graph property testing” (broad class of properties, totally classified wrt testability) in algebraic world? What is the context for low-degree testing? What is the context for low-degree testing? Answer to all: Invariance! Answer to all: Invariance! April 8, 2016 Two Decades of Property Testing 18

of 29 Invariance? April 8, 2016 Two Decades of Property Testing 19

of 29 Invariances (contd.) April 8, 2016 Two Decades of Property Testing 20

of 29 What is Property Testing? April 8, 2016 Two Decades of Property Testing 21 Algebra=? ?

of 29 Abstracting algebraic properties April 8, 2016 Two Decades of Property Testing 22

of 29 April 8, 2016 Two Decades of Property Testing 23 Testing Linear Properties Algebraic Property = Code! (usually) Universe Universe: {f:D  R} P Don’t care Must reject Must accept P R is a field F; P is linear!

of 29 Why study affine-invariance? Common abstraction of properties studied in [BLR], [RS], [ALMSS], [AKKLR], [KR], [KL], [JPRZ]. Common abstraction of properties studied in [BLR], [RS], [ALMSS], [AKKLR], [KR], [KL], [JPRZ]. (Variations on low-degree polynomials) (Variations on low-degree polynomials) Hopes Hopes Unify existing proofs Unify existing proofs Classify/characterize testability Classify/characterize testability Find new testable codes (w. novel parameters) Find new testable codes (w. novel parameters) Rest of the talk: Brief summary of findings Rest of the talk: Brief summary of findings April 8, 2016 Two Decades of Property Testing 24

of 29 Results 1: AKKLR Conjecture April 8, 2016 Two Decades of Property Testing 25

of 29 Results 2: Accidental +ve April 8, 2016 Two Decades of Property Testing 26

of 29 Results 3: Lifting April 8, 2016 Two Decades of Property Testing 27 Bad News + Bad News = Good News!

of 29 Result 3: Lifting (contd.) April 8, 2016 Two Decades of Property Testing 28

of 29 Conclusions April 8, 2016 Two Decades of Property Testing 29

of 29 Thank You April 8, 2016 Two Decades of Property Testing 30