1. 1 Asymptotically good binary code with efficient encoding & Justesen code Tomer Levinboim Error Correcting Codes Seminar (2008)

2. 2 Outline Intro  codes  Singleton Bound Linear Codes Bounds  Gilbert-Varshamov  Hamming RS codes Code Concatention  Examples Wozencraft Ensemble Justesen Codes

3. 3 Hamming Distance Hamming Distance between The Hamming Distance is a metric  Non negative  Symmetric  Triangle inequality  =

4. 4 Weight The weight (wt) of Example (on board)

5. 5 Code An (n,k,d) q code C is a function such that:   For every

6. 6 Code (parameters) (n,k,d) q Parameters  n – block length  k – information length  d – minimum distance (actually, a lower bound)  q – size of alphabet  |C| = q k or k=log q |C|

7. 7 Code (parameters div n) Asymptotic view of parameters as n  ∞ :  The rate  Relative minimum distance Thus an (n,k,d) q can be written as (1,R, δ ) q Notation: (n,k,d) q vs. [n,k,d] q – latter reserved for linear code (soon)

8. 8 Trivial Code Example FEC3 = write each bit three time  R = ?  d = ? how many errors can we  Detect ? (d-1)  Correct ? t, where d=2t+1

9. 9 Goal Would like to:  Maximize δ – correct more  Maximize R – send more information * conflicting goals - would like to be able to construct an [n,k,d] q code s.t. δ>0, R>0 and both are constant.  Minimize q – for practical reasons  Maximize number of codewords while minimizing n and keeping d large.

10. 10 Singleton Bound Let C be an [n,k,d] q code then  k ≤ n – d + 1 equivalently  R ≤ 1 – δ + o(1) Proof: project C to first k-1 coordinates  On Board

11. 11 Visual intuition On board... Ball q (x,r)  r:=d  r:=t (where d=2t+1) Vol q (n,r) = |Ball q (x,r)|

12. 12 Linear Codes

13. 13 Linear Codes An [n,k,d] q code C:F q K  F q n is linear when:  F q is a field  C is linear function (e.g., matrix) Linearity implies:  C(ax+by) = aC(x) + bC(y)  0 n member of C

14. 14 Linear Codes (example) FEC3  [3,1,3] 2 Hadamard – longest linear code  [n,logn, n/2] 2  e.g., - [8,3,4] 2  (H - Matrix representation on board) Dimensions Asymptotic behavior

15. 15 Linear Codes – minimum distance Lemma: if C:F q K  F q n is linear then Note: for clarity C x means C(x) Proof:  ≤ - trivial  ≥ - follows from linearity (on board)

16. 16 Reed-Solomon code Idea: oversample a polynomial Let q be prime power and F q a finite field of size q. Let k<n and fix n elements of F q,  x 1,x 2,..x n Given a message m=(c 0..c k-1 ) interpret it has the coefficients of the polynomial p

17. 17 RS Codes Thus (c 0..c k-1 ) is mapped to (p(x 1 ),..p(x n ))  Linear mapping (Vandermonde) Using linearity, can show for x≠0  RS meet the Singleton bound Proof: on board  (# of roots of a k-1 degree poly) Encoding time

18. 18 Bounds

19. 19 Gilbert-Varshamov Bound Preliminaries Binary Entropy Stirling Implying that:

20. 20 Gilbert-Varshamov Bound Preliminaries Using the binary entropy we obtain On board

21. 21 Gilbert-Varshamov Bound bound statement For every n and d<n/2 there is an (n,k,d) q (not necessarily linear) code such that: In terms of rate and relative min-distance:

22. 22 Gilbert-Varshamov Bound Proof On Board Sketch of proof:  if C is maximal then:  And  Now use union bound and entropy to obtain result (we show for q=2, using binary entropy)

23. 23 GV-Bound Gilbert proved this with a greedy construction Varshamov proved for linear codes  proved using random generator matrices – most matrices are good error correcting codes

24. 24 Singleton / GV Plot 10.5 1 Singleton (upper) Gilbert-Varshamov (lower)

25. 25 Hamming Bound (Upper) With similar reasoning to GV bound but using For q=2 can show that

26. 26 Bounds plot *Madhu Sudan (Lecture 5, 2001)

27. 27 Code Concatenation

28. 28 Code Concatenation - Motivation RS codes imply we can construct good [n,k,d] q codes for any q=p k Practically would like to work with small q (2, 2 8 ) Consider the “obvious” idea for binary code generated from C – simply convert each symbol from Σ n to log 2 q, What’s the problem with this approach ? (write the new code!)

29. 29 Code Concatenation Due to Forney (1966) Two codes:  Outer:C out = [N,K,D] Q  Inner: C in = [n,k,d] q Inner code should encode each symbol of outer code  k = log q Q

30. 30 Code Concatenation How does it work ? * Luca Trevisan (Lecture 2)

31. 31 Code Concatenation What is the new code ?  d con = dD Proof:  On board

32. 32 Code Concatenation (Examples) Asymptotically  δ = ¼  R=logn/2n  0 

33. 33 Good Codes Can we “explicitly” build asymptotically good (linear) codes ?  asymptotically good = constant R, δ > 0 as n  ∞  Explicit = polytime constructable / logspace constructible

34. 34 Asymptotically Good Codes

35. 35 Asymptotically Good Codes GV tells us that most linear functions of a certain size are good error-correcting codes  Can find a good code in brute-force Use brute force on inner-code, where the alphabet is exponentially smaller! Do we really need to search ?

36. 36 Wozencraft Ensemble Consider the following set of codes: such that (  R=1/2) ( Notice that (on board)

37. 37 Wozencraft Ensemble Lemma: There exists an ensemble of codes c 1,..c N of rate ½ where N = q k -1 such that for at least (1-ε)N value of i, the code C i has distance d i s.t. Proof (on board), outline:  Different codes have only 0 n in common  Let y=C α (x), then, If wt(y)<d  y in Ball(0 n, d)  there are at most Vol(n,d) “bad” codes  For large enough n=2k, we have Vol(n,d) ≤ εN

38. 38 Wozencraft Ensemble Implications:  Can construct entire ensemble in O(2 k )=O(2 n )  There are many such good codes, but which one do we use ?

39. 39 Justesen Code Concatenation of:  C out - RS code over  a set of inner codes Justesen Code: C * = C out (C 1, C 2,.. C N )  Each symbol of C out is encoded using a different inner code C j  If RS has rate R  C * has rate R/2

40. 40 Justesen Code - δ Denote the outer RS code [N,K,D] Q Claim: C* has relative distance

41. 41 Justesen Code Proof Intuition: like regular concatenation, but εN bad codes. for x≠y, the outer code induces S={j | x j ≠y j },  |S| ≥D There are at most εN j’s such that Cj is bad and therefore at least |S|- εN ≥ D- εN ≥ (1-R- ε)N good codes  since RS implies D=N-(K-1) Each good code has relative distance ≥ d d * ≥ (1-R- ε)Nd

42. 42 Justesen Code The concatenated code C * is an asymptotically good code and has a “super” explicit construction Can take q=2 to get such a binary code