Beer Therapy Madhu Ralf Vote early, vote often At Oberwolfach in 2003, Ralf Kötter and Madhu Sudan had a week long beer drinking competition. Who do you think won? Ralf Madhu Vote early, vote often

Local Algorithms & Error-correction Madhu Sudan MIT

Beer Therapy Madhu Ralf Information Computation At Oberwolfach in 2003, Ralf Kötter and Madhu Sudan had a week long beer drinking competition. Who do you think won? Ralf Madhu Information Computation

Dedicated to Ralf Kötter Dear friend to many … Wise beyond his age Happy spirit … I’ll miss him dearly. … I already do.

Local Error-Correction Prelude Algorithmic Problems in Coding Theory New Paradigm in Algorithms The Marriage: Local Error-Detection & Correction February 11, 2009 Local Error-Correction

Algorithmic Problems in Coding Theory Code: Encoding: Error-detection ( Testing): Error-correction (Decoding): E : § k ! n ; I m a g e ( ) = C µ R , ± o r l i z d s t c . F i x C o d e a n s c t E : § k ! . G v m 2 , p u ( ) ² - G i v e n x 2 § , d c f 9 m k s . t = E ( ) G i v e n x 2 § , d c f 9 m k s . t ± ( E ) ; · ² G i v e n x 2 § , c o m p u t k h a z s ± ( E ) ; r d · ² . February 11, 2009 Local Error-Correction

Sublinear time algorithmics Given can it be “computed” in time? Answer 1: Clearly NO, since that is the time it takes to even read the input/write the output f : ; 1 g k ! n o ( k ; n ) x - o r a c l e j f f ( x ) i w h e r ¼ f ( x ) i f ( x ) x i Answer 2: YES, if we are willing to Present input implicitly (by an oracle). Represent output implicitly Compute function on approximation to input. Extends to computing relations as well. February 11, 2009 Local Error-Correction

Sub-linear time algorithms Initiated in late eighties in context of Program checking [BlumKannan,BlumLubyRubinfeld] Interactive Proofs/PCPs [BabaiFortnowLund] Now successful in many more contexts Property testing/Graph-theoretic algorithms Sorting/Searching Statistics/Entropy computations (High-dim.) Computational geometry Many initial results are coding-theoretic! February 11, 2009 Local Error-Correction

Sub-linear time algorithms & Coding Encoding: Not reasonable to expect in sub-linear time. Testing? Decoding? – Can be done in sublinear time. In fact many initial results do so! Codes that admit efficient … … testing: Locally Testable Codes (LTCs) … decoding: Locally Decodable Codes (LDCs). February 11, 2009 Local Error-Correction

Local Error-Correction Rest of this talk Definitions of LDCs and LTCs Quick description of known results The first result: Hadamard codes Some basic constructions Recent constructions of LDCs. [Yekhanin,Raghavendra,Efremenko] February 11, 2009 Local Error-Correction

Local Error-Correction Definitions February 11, 2009 Local Error-Correction

Local Error-Correction Locally Decodable Code Code: C : § k ! n i s ( q ; ² ) - L o c a l y D e d b f 9 r . t g v 2 [ ] w m ± · = , n w D ( i ) r e a d s q n o m p t f w u . l 2 = 3 W h a t i f ² > ± ( C ) = 2 ? M g n e d o r p l s u ` c w . February 11, 2009 Local Error-Correction

Local Error-Correction Locally List-Decodable Code Code: C i s ( ² ; ` ) - l t d e c o a b f 8 w 2 § n , # r . ± · m C i s ( q ; ² ` ) - l o c a y t d e b f 9 D r . g v n 2 [ k ] j w m 1 : ± · n w D ( i ; j ) r e a d s q n o m p t f w u . l 2 = 3 February 11, 2009 Local Error-Correction

History of definitions Constructions predate formal definitions [Goldreich-Levin ’89]. [Beaver-Feigenbaum ’90, Lipton ’91]. [Blum-Luby-Rubinfeld ’90]. Hints at definition (in particular, interpretation in the context of error-correcting codes): [Babai-Fortnow-Levin-Szegedy ’91]. Formal definitions [S.-Trevisan-Vadhan ’99] (local list-decoding). [Katz-Trevisan ’00] February 11, 2009 Local Error-Correction

Local Error-Correction  Locally Testable Codes Code: C µ § n i s ( q ; ² ) - L o c a l y T e t b f 9 r . n w T r e a d s q ( n ) o m p i t : ² I f w 2 C c . 1 - , h j ¸ = “Weak” definition: hinted at in [BFLS], explicit in [RS’96, Arora’94, Spielman’94, FS’95]. February 11, 2009 Local Error-Correction

Local Error-Correction Strong Locally Testable Codes Code: C µ § n i s ( q ; ² ) - L o c a l y T e t b f 9 r . n w T r e a d s q ( n ) o m p i t : ² I f w 2 C c . 1 F v y § , j ¸ ­ ± ; “Strong” Definition: [Goldreich-S. ’02] February 11, 2009 Local Error-Correction

Local Error-Correction Motivations February 11, 2009 Local Error-Correction

Local decoding: Average-case vs. worst-case ² S u p o s e C µ § N i l c a y - d b f r = 2 n . ( F t h m w , j g ) ² c 2 C a n b e v i w d s f u t o : ; 1 g ! § . ² L o c a l d e i n g ¼ ) m p u t ( x f r v y , s . R - w [ S T V ] ² A l t e r n a i p o : C m u c ( x ) w h - v g . L d s I H [ B F ] , P f R G K S February 11, 2009 Local Error-Correction

Motivation for Local-testing No generic applications known. However, Interesting phenomenon on its own. Intangible connection to Probabilistically Checkable Proofs (PCPs). Potentially good approach to understanding limitations of PCPs (though all resulting work has led to improvements). February 11, 2009 Local Error-Correction

Contrast between decoding and testing Decoding: Property of words near codewords. Testing: Property of words far from code. Decoding: Motivations happy with n = quasi-poly(k), and q = poly log n. Lower bounds show q = O(1) and n = nearly-linear(k) impossible. Testing: Better tradeoffs possible! Likely more useful in practice. Even conceivable: n = O(k) with q = O(1)? February 11, 2009 Local Error-Correction

Local Error-Correction Some LDCs and LTCs February 11, 2009 Local Error-Correction

Local Error-Correction Hadamard (1st Order RM) Codes Message: Encoding: Parameters: Locality: ( C o e ± c i n t s f ) L a r F u m k 2 . e v a l u t i o n s f L F k 2 . k b i t m e s a g ! 2 - c o d w r L ( x ) = + y ¡ 2 - o c a l D e d b [ F k r / E i s ] 3 T t B u m R n f February 11, 2009 Local Error-Correction

Hadamard (1st Order RM) Codes Conclusions: There exist infinite families of codes With constant locality (for testing and correcting). February 11, 2009 Local Error-Correction

Local Error-Correction Codes via Multivariate Polynomials Message: Encoding: Parameters: c o e ± i n t s f d g , m - v a r p l y P ¯ F P F m (Reed Muller code) e v a l u t i o n s f P F m . k ¼ ( t = m ) , n j F ± ¸ . February 11, 2009 Local Error-Correction

Basic insight to locality Local Decoding: Local Testing: m - v a r i t e p o l y n f d g s c < . ( ± ) u b P i c k s u b p a e t h r o g n x f , d . Q y m l q = j F ; T ( ) ¿ ! V e r i f y s t c d o p a g . S m l x February 11, 2009 Local Error-Correction

Local Error-Correction Polynomial Codes Many parameters: Many tradeoffs possible: m , t F L o c a l i t y q w h n = e x p ( k 1 ¡ ) L o c a l i t y ( g k ) 2 w h n = 4 L o c a l i t y p k w h n = O ( ) . February 11, 2009 Local Error-Correction

Are Polynomial Codes (Roughly) Best? No! [Ambainis97] [GoldreichS.00] … No!! [Beimel,Ishai,Kushilevitz,Raymond] Really … Seriously … No!!!! [Yekhanin07,Raghavendra08,Efremenko09] February 11, 2009 Local Error-Correction

Recent LDCs [Yekhanin ‘07, Raghavendra ‘08, Efremenko ‘09] February 11, 2009 Local Error-Correction

Local Error-Correction The recent result: Fix q = 3; n = ??? (as function of $k$) Till 2007: [Yekhanin ’07]: [Raghavendra ’08]: [Efremenko ’09]: n ¼ e x p ( k 1 = 5 ) o - b i a r y . n ¼ e x p ( k ) b i a r y . n ¼ e x p ( k : 1 ) b i a r y . n ¼ e x p ( l o g k ) b i a r y . February 11, 2009 Local Error-Correction

Local Error-Correction Essence of the idea: Build “good” combinatorial matrix over Embed in multiplicative subgroup of Get locally decodable code over Z m Z m F F February 11, 2009 Local Error-Correction

“Good” Combinatorial matrix [ [ 0 … … 0 … … 0 … … 0 … … 0 A = k £ n m a t r i x o v e Z arbitrary Z e r o s n d i a g l N o n - z e r ® d i a g l Columns closed under addition February 11, 2009 Local Error-Correction

Local Error-Correction Embedding into field ¡ L e t A = [ a i j ] b \ g o d v r Z m ¡ L e t ! = p r i m v h o f u n y F . ¡ L e t G = [ ! a i j ] . T h e o r m [ Y k a n i , R g v d E f ] : G t s q u y L D C F . Highly non-intuitive! February 11, 2009 Local Error-Correction

Local Error-Correction Improvements ¡ A = [ a i j ] ; G ! . ¡ O ® - d i a g o n l e t r s f A m S ) G j + 1 q u y L D C . (Suffices for [Efremenko]) ¡ ! S z e r o s f t - p a l y n m i v F ) G g q u L D C . (Critical to [Yekhanin]) February 11, 2009 Local Error-Correction

Local Error-Correction “Good” Matrices? [Yekhanin]: Picked m prime. Hand-constructed matrix. Achieved Optimal if m prime! Managed to make S large with t=3. [Efremenko] m composite! Achieves ([Beigel,Barrington,Rudich];[Grolmusz]) Optimal? n = e x p ( k 1 j S ) j S = 3 a n d e x p ( l o g k ) February 11, 2009 Local Error-Correction

Limits to Local Decodability: Katz-Trevisan ) n = k 1 + ­ ( q . q queries Technique: Recall D(x) computes C(x) whp for all x. Can assume (with some modifications) that query pattern uniform for any fixed x. Can find many random strings such that their query sets are disjoint. In such case, random subset of n^{1-1/q} coordinates of codeword contain at least one query set, for most x. Yields desired bound. February 11, 2009 Local Error-Correction

Local Error-Correction Some general results Sparse, High-Distance Codes: Are Locally Decodable and Testable [KaufmanLitsyn, KaufmanS] 2-transitive codes of small dual-distance: Are Locally Decodable [Alon,Kaufman,Krivelevich,Litsyn,Ron] Linear-invariant codes of small dual-distance: Are also Locally Testable [KaufmanS] February 11, 2009 Local Error-Correction

Local Error-Correction Summary Local algorithms in error-detection/correction lead to interesting new questions. Non-trivial progress so far. Limits largely unknown O(1)-query LDCs must have Rate(C) = 0 [Katz-Trevisan] February 11, 2009 Local Error-Correction

Local Error-Correction Questions Can LTC replace RS (on your hard disks)? Is a significant rate-loss necessary? Lower bounds? Better error models? Simple/General near optimal constructions? Other applications to mathematics/computation? (PCPs necessary/sufficient)? Lower bounds for LDCs?/Better constructions? February 11, 2009 Local Error-Correction

Local Error-Correction Thank You! February 11, 2009 Local Error-Correction