List Decoding Product and Interleaved Codes Prasad Raghavendra Joint work with Parikshit Gopalan & Venkatesan Guruswami

Error-correcting codes Encoding E :  k   n Code C = { E(m) : m   k } Rate R of the code = k/n (Relative) distance of the code =  if every two codewords differ in  n positions Noise model : –Any subset of up to fraction p of symbols can be arbitrarily corrupted by the channel message m codeword E(m) What fraction of errors can various codes correct?

Decoding radii nn If error fraction p <  /2, correct codeword is unique closest codeword. When p >  /2, unambiguous recovery is not always possible. Pessimistic limit: even for p >>  /2, many error patterns have a unique close-by codeword. Relax decoding model: List decoding “Allow a small list of possible answers”

List Decoding [Elias’57, Wozencraft’58, Goldriech-Levin’89] List decoding C   n up to error fraction p Given: A received word r   n Output: A list of all codewords c  C s.t. HammingDist(r,c)  e =pn e r 1.Combinatorics: Code must guarantee that list is small (say, bounded by constant independent of n) for every r 2.Algorithmics: There are exp(n) many codewords; want to find close-by codewords in poly(n) time List Decoding Radius LDR(C): largest error rate for which the code can be (efficiently) list decoded

List decoding radius Natural question: Given a code, what is its list decoding radius? –combinatorial, algorithmic Johnson radius: LDR(C)  J(  ) = 1 - (1-  ) 1/2 –(1 - (1-2  ) 1/2 )/2 for binary codes –Johnson radius always   /2 –generic combinatorial bound as function of  alone Johnson bound tight in some cases, but specific codes may have higher LDR –Random codes in fact have LDR   w.h.p. –Determining LDR of even well-understood codes often very difficult

List decoding radii: previous salient results Hadamard (first order Reed-Muller) codes: LDR   = 1 - 1/q [Goldreich-Levin; Goldreich-Rubinfeld-Sudan] Reed-Solomon (RS) codes : Efficient algorithm to list decode up to Johnson radius [Sudan; G.-Sudan] ( Exact LDR open!) Binary Reed-Muller codes (of any order r): LDR   = 2 -r [Gopalan-Klivans-Zuckerman] “Correlated” RS codes: Efficient algorithm to list decode beyond J(  ) (for  close to 1) [Parvaresh-Vardy] Folded RS codes: LDR   = 1 - R (optimal) [G.-Rudra] –Some algebraic-geometric extensions [G.-Patthak’06] [G.-09] Group homomorphisms [Dinur-Grigorescu-Kopparty-Sudan] LDR   All results give algorithm to decode up to stated bound on LDR

Our work Efficient list decoding only known for specific well- structured codes –Mostly, algebraic codes (exceptions [G.-Indyk’03], [Trevisan’03]) Our work: General list decoding algorithms for a broad class of codes –Abstract look at two well-known “product” operations to combine codes 1.Tensoring 2.Interleaving –Algorithmic solutions (and combinatorial bounds) for associated list decoding problems

Tensor Products Given a linear code C  F q n, the (tensor) product code C x C  F q n x n is: C x C = { n x n matrices such that each row and each column is a codeword of C } Parameters: –Blocklength(C x C) = Blocklength(C) 2 –Rate(C x C) = Rate(C) 2 –  (C x C) =  (C) 2 –Alphabet size q remains same –Parameters in general worse, but can get longer codes

Tensoring of Reed Solomon Codes Basis for Reed Solomon Code C: the monomials = { 1,x,x 2, x 3, x 3,… x k-1 } Basis for tensor product C X C : the monomials = { 1,x,y,xy,x 2 y, x i y j,..,x k-1 y k-1 } C X C = { Evaluations of bivariate polynomials P(x,y) with degree of each variable bounded by k-1 }

"Applications" (Tensor) product codes arise in many contexts: Explicit codes for BSC: Product of Hamming codes [Elias] Hardness of Approximation [Dumer-Micciancio-Sudan] Construction of Locally Testable Codes [BenSasson-Sudan], [Meir] Local Testability of tensor products has received lot of attention: [BenSasson-Sudan], [G.-Rudra], [Dinur- Sudan-Wigderson], [Valiant], [BenSasson-Viderman]

Our Result: Tensor Products For every code C, LDR( C X C ) = LDR(C)   (C) Using list decoding algorithm for C as a blackbox, we get list decoding algorithm for C X C list size = 2 O((log L)^2) if list size for C is L. Corollary: For repeated tensoring LDR( C x m ) = LDR(C)   (C) m-1 Tensor products always decodable beyond Johnson radius! -  m-1 J(  ) >> J(  m ) Corollary: New list decoding algorithms for multivariate polynomial codes when degree of each variable is bounded. even decoding these up to Johnson radius was open Ratio LDR/Distance stays same under tensoring.

Next: Interleaved Product So far : List Decoding Tensor Products: Definition and Results

Interleaving Bursty Errors A few codewords incur large errors. Solution : Interleave the symbols from different codewords while transmitting. Bursty Errors get uniformly distributed across different codewords

Interleaved Product: Definition For a code C, Codeword of C m : m-wise interleaving of C is given by c1c1 c2c2 c3c3..cm..cm Parameters: Rate, distance, block length of C m same as those of C Alphabet of C m = [q] m where Alphabet of C = [q]

Reed Solomon Codeword Evaluation of a degree k polynomial P at n points. Degree k curve P(t) in the one- dimensional space [q]. Interleaving in Practice 3-interleaved Reed Solomon Codeword Evaluation of a degree k polynomials (P 1,P 2,P 3 ) at n points. Degree k curve (P 1 (t), P 2 (t), P 3 (t)) in the three dimensional space [q] 3. P(1)P(2)P(n) P 1 (1) P 1 (2)P 1 (n) P 2 (1)P 2 (2)P 2 (n) P 3 (1)P 3 (2)P 3 (n)

Interleaved codes list decoding Question: LDR(C ) = p → LDR(C m ) = ? Naive Approach: R 1 (1)R 1 (2)R 1 (n)R 2 (1)R 2 (2)R 2 (n)R 3 (1)R 3 (2)R 3 (n) List decode L 1 = {A 1,A 2,.. A m } L 2 = {B 1,B 2,.. B m } L 3 = {C 1,C 2,.. C m } p fraction of columns erroneous X X Drawback: List size and running time grows exponentially in number m of interleavings. Try all possibilities in L 1 X L 2 X L 3 Clearly at most p

Our Result: Interleaved codes For every code C, LDR(C m ) = LDR(C ) Specifically, we design list decoding algorithm, and bound the list size of C m List size is independent of # interleavings m !

Related work Interleaved Hadamard codes –Decoding linear transformations [Dinur-Grigorescu-Kopparty-Sudan], [Dwork-Shaltiel-Smith-Trevisan] Interleaved Reed-Solomon codes –Decoding under random noise (for cryptanalysis) [Coppersmith-Sudan], [Bleichenbacher-Kiayias-Yung] Parvaresh-Vardy codes & folded Reed-Solomon codes [G.-Rudra] are carefully chosen subcodes of interleaved RS codes

Other results: Better list size bounds Codewords of interleaved code C m and tensor product C x C are naturally viewed as matrices If rank of a codeword of C m is r, its Hamming weight equals r’th Generalized Hamming weight  r (C) –For binary codes large r,  r (C)  2  (C) “Deletion” argument: List size bound dominated by number of close-by low rank codewords –Bound low-rank list size by combinatorial techniques –Leads to better list size bounds for binary linear codes Eg., for decoding linear transformations over F q, get tight list size bound for fixed q – list size O q (1/  2 ) for error fraction (1-1/q-  )

Rest of talk List decoding interleaved codes Sketch of (tensor) product list decoding GHW and decoding linear transformations Summary

Interleaved codes For a code C, Codeword of C m : m-wise interleaving of C is given by c1c1 c2c2 c3c3..cm..cm

Naive Approach Revisited R 1 [1]R 1 [2]R 1 [n]R 2 [1]R 2 [2]R 2 [n]R 3 [1]R 3 [2]R 3 [n] Root A1A1 A2A2 AtAt B1B1 BtBt … … C1C1 CtCt … Each leaf is a candidate decoding. Running time and list size depends on the number of leaves.

Two simple ideas R 1 [1]R 1 [2]R 1 [n]R 2 [1]R 2 [2]R 2 [n]R 3 [1]R 3 [2]R 3 [n] Root A1A1 A2A2 AtAt B1B1 BtBt … … C1C1 CtCt … Idea 1: Erase columns containing an erroneous symbol. Idea 2: Prune branches with more than p fraction of errors.

Branching → Erasures v A1A1 A2A2 AtAt … Observation 1 At most one of the codewords A 1, A 2,.. A t is at a distance < δ(v)/2. For all but one A i e(A i ) ≥ e(v) + δ(v)/2 = ( δ + e(v) )/2 Observation 2 Even the nearest codeword has to be δ-p away (if t > 1) For the nearest codeword A i e(A i ) ≥ e(v) + δ - p e(v) = frac. of columns erased up to v We decode at v up to radius p - e(v) δ(v) = δ - e(v) = distance of punctured code with erased positions removed

Given a branching node v, along all children, the number of erasures increases by one of the following two rules : e(w i ) ≥ ( δ + e(v) )/2 OR e(w i ) ≥ e(v) + (δ – p) Erasures → Low depth v w1w1 w2w2 wtwt … pδ 0 e e+ δ – p (e+ δ)/2 In about δ/ (δ – p) steps, fraction of erasures reaches p Unique decode from p < δ erasures from then on

List size bound On every root to leaf path there are at most δ/(δ – p) branching nodes. Every branching node has at most L children, where L = list size for C. Number of leaves ≤ L (δ/ (δ – p)) –Careful bound: 2 (δ/(δ – p)) L log(δ/ (δ – p)) Proof is algorithmic Root A1A1 A2A2 AtAt B1B1 BtBt … … C1C1 CtCt …

Rest of talk List decoding interleaved codes Sketch of (tensor) product list decoding GHW and decoding linear transformations Summary

 =  (C)  (C x C) =  2 WrongCorrect Unique decoding product codes Decode along rows, then along columns Decoding up to radius  2 /4 Decode rows up to radius  /2 (at most  /2 rows wrong) Decode columns to radius  /2

Correct Advice Phase 1 : Given correct advice (codeword on random S x T), List decode to radius p along rows in S, and use advice to disambiguate. B : value on rows that didn’t fail; advice for next phase. Wrong   |S| Fail Correct  (1-  +  )|S| B List decoding product codes List decode along rows, then columns, then do it again! Goal: Decode up to radius p  -  (p = LDR(C)) S T

Phase 2 D – advice for next phase Fail B D List decode received word to radius p along all the columns, Given the noisy advice B, Use advice B to disambiguate ( pick codeword that differs from B in <  fraction rows in B) Correct  (1-  +  )n Wrong   n Averaging + code distance + sampling  Correct decoding on (1-  +  )n columns,   n mistakes

Phase 3 D is “noisy but sound” ErasuresCorrect D E Row-wise: Unique codeword (correct one) differs from D in <  n positions List decode received word to radius p along the rows, and use advice D to disambiguate. E : advice with only erasures.  (1-  +  )n rows correctly decoded! - rest fail (1-  +  )n

Phase 4 Given the codeword with erasures E, E Unique Decode each column of E from (<  n) erasures - easy for every linear code

Rest of talk List decoding interleaved codes Sketch of (tensor) product list decoding GHW and decoding linear transformations Summary

(Binary) Linear transformation code a1a1 a2a2 amam L  {0,1} m x k Had(a m ) Columns x  {0,1} k a1˙ xa1˙ x a2˙ xa2˙ x m rows Given recd. matrix R, # codewords that differ in (1/2-  ) columns? - For Hadamard codes (each row), list size is  (1/  2 ) - Our general algorithm gives upper bound (1/  ) O(log (1/  )) Consider rank of the codeword If rank > 1, Hamming weight is  3/4 (GHW connection) Johnson radius J(3/4) = 1/2. By “GKZ deletion argument,” list size  O(1/  ) L (1) (1/2-  ) - [Dinur-Grigorescu-Kopparty-Sudan] 1/  C for absolute constant C - We get tight O(1/  2 ) bound. (Also, min { c q /  2, C/  4 } over F q ) # close-by rank 1 codewords

Rank 1 list size Rank 1 codeword has simple form –1/2 columns are v, 1/2 are 0 Each candidate v must occur with frequency  in R –t  1/  candidates for v Reduces to t Hadamard decodings to radius (1/2-  ) –Naively, list size t/  2 = 1/  3 –Use erasures in decoding, list size O(1/  2 ) More complex argument: Rank 2 list size L (2) (1/2-  )  O(1/  2 ) Deletion argument: overall list size  C 0 · L (2) (1/2-  ) for absolute constant C 0 000vvv

Summary & Open questions Studied effect of two product operations on list decoding –Tensoring: LDR( C x m ) = LDR(C)   (C) m-1 –Interleaving: LDR( C m ) = LDR(C), irrespective of m Both decoding radii are tight, but our list size bounds could be improved –Lower bounds on list size? –List size C/  2 for linear transformations over F q ? –Construction of list-decodable codes via repeated tensoring? Combinatorial folded codes? LDR of Reed-Solomon codes? Non-binary (degree  3) Reed-Muller codes? –LDR   =1-2/q for quadratic case [Gopalan’09]