1 Asymptotically good binary code with efficient encoding & Justesen code Tomer Levinboim Error Correcting Codes Seminar (2008)

Slides:



Advertisements
Similar presentations
Mahdi Barhoush Mohammad Hanaysheh
Advertisements

On Complexity, Sampling, and -Nets and -Samples. Range Spaces A range space is a pair, where is a ground set, it’s elements called points and is a family.
Cyclic Code.
Applied Algorithmics - week7
Error Control Code.
296.3Page :Algorithms in the Real World Error Correcting Codes II – Cyclic Codes – Reed-Solomon Codes.
L. J. Wang 1 Introduction to Reed-Solomon Coding ( Part I )
Efficient Soft-Decision Decoding of Reed- Solomon Codes Clemson University Center for Wireless Communications SURE 2006 Presented By: Sierra Williams Claflin.
1. 2 Overview Review of some basic math Review of some basic math Error correcting codes Error correcting codes Low degree polynomials Low degree polynomials.
May 24, 2005STOC 2005, Baltimore1 Limits to List Decoding Reed-Solomon Codes Venkatesan Guruswami Atri Rudra (University of Washington)
15-853:Algorithms in the Real World
Information and Coding Theory
Probabilistic verification Mario Szegedy, Rutgers www/cs.rutgers.edu/~szegedy/07540 Lecture 4.
CHANNEL CODING REED SOLOMON CODES.
Information Theory Introduction to Channel Coding Jalal Al Roumy.
Cellular Communications
Asymptotic Enumerators of Protograph LDPCC Ensembles Jeremy Thorpe Joint work with Bob McEliece, Sarah Fogal.
Correcting Errors Beyond the Guruswami-Sudan Radius Farzad Parvaresh & Alexander Vardy Presented by Efrat Bank.
DIGITAL COMMUNICATION Coding
EEE377 Lecture Notes1 EEE436 DIGITAL COMMUNICATION Coding En. Mohd Nazri Mahmud MPhil (Cambridge, UK) BEng (Essex, UK) Room 2.14.
CS151 Complexity Theory Lecture 10 April 29, 2004.
Variable-Length Codes: Huffman Codes
DIGITAL COMMUNICATION Coding
Linear-Time Encodable and Decodable Error-Correcting Codes Jed Liu 3 March 2003.
15-853Page :Algorithms in the Real World Error Correcting Codes I – Overview – Hamming Codes – Linear Codes.
Mario Vodisek 1 HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Erasure Codes for Reading and Writing Mario Vodisek ( joint work.
CS151 Complexity Theory Lecture 9 April 27, 2004.
Hamming Code Rachel Ah Chuen. Basic concepts Networks must be able to transfer data from one device to another with complete accuracy. Data can be corrupted.
Hamming Codes 11/17/04. History In the late 1940’s Richard Hamming recognized that the further evolution of computers required greater reliability, in.
Linear codes 1 CHAPTER 2: Linear codes ABSTRACT Most of the important codes are special types of so-called linear codes. Linear codes are of importance.
DIGITAL COMMUNICATION Error - Correction A.J. Han Vinck.
CS151 Complexity Theory Lecture 9 April 27, 2015.
Information and Coding Theory Linear Block Codes. Basic definitions and some examples. Juris Viksna, 2015.
Combinatorial Algorithms Reference Text: Kreher and Stinson.
Codes Codes are used for the following purposes: - to detect errors - to correct errors after detection Error Control Coding © Erhan A. Ince Types: -Linear.
Analysis of Algorithms CSCI Previous Evaluations of Programs Correctness – does the algorithm do what it is supposed to do? Generality – does it.
1 Network Coding and its Applications in Communication Networks Alex Sprintson Computer Engineering Group Department of Electrical and Computer Engineering.
ERROR CONTROL CODING Basic concepts Classes of codes: Block Codes
MIMO continued and Error Correction Code. 2 by 2 MIMO Now consider we have two transmitting antennas and two receiving antennas. A simple scheme called.
Fields: Defns “Closed”: a,b in F  a+b, a.b in F Properties: – Commutative: a+b=b+a, a.b=b.a – Associative: a+(b+c)=(a+b)+c, a.(b.c) = (a.b).c – Distributive:
Great Theoretical Ideas in Computer Science.
Basic Characteristics of Block Codes
§6 Linear Codes § 6.1 Classification of error control system § 6.2 Channel coding conception § 6.3 The generator and parity-check matrices § 6.5 Hamming.
DIGITAL COMMUNICATIONS Linear Block Codes
Probabilistic verification Mario Szegedy, Rutgers www/cs.rutgers.edu/~szegedy/07540 Lecture 5.
Information and Coding Theory Cyclic codes Juris Viksna, 2015.
Information Theory Linear Block Codes Jalal Al Roumy.
Word : Let F be a field then the expression of the form a 1, a 2, …, a n where a i  F  i is called a word of length n over the field F. We denote the.
Perfect and Related Codes
Some Computation Problems in Coding Theory
Elementary Coding Theory Including Hamming and Reed-Solomom Codes with Maple and MATLAB Richard Klima Appalachian State University Boone, North Carolina.
Computer Construction of Quasi-Twisted Two-Weight Codes Eric Chen Dept. of Comp.Science Kristianstad University Kristianstad Sweden.
Error Control Coding. Purpose To detect and correct error(s) that is introduced during transmission of digital signal.
Richard Cleve DC 2117 Introduction to Quantum Information Processing QIC 710 / CS 667 / PH 767 / CO 681 / AM 871 Lecture (2011)
Reed-Solomon Codes Rong-Jaye Chen.
CHAPTER 8 CHANNEL CODING: PART 3 Sajina Pradhan
1 List decoding of binary codes: New concatenation-based results Venkatesan Guruswami U. Washington Joint work with Atri Rudra.
Class Report 林格名 : Reed Solomon Encoder. Reed-Solomom Error Correction When a codeword is decoded, there are three possible outcomes –If 2s + r < 2t (s.
RS – Reed Solomon Error correcting code. Error-correcting codes are clever ways of representing data so that one can recover the original information.
Umans Complexity Theory Lectures
Sublinear-Time Error-Correction and Error-Detection
Sublinear-Time Error-Correction and Error-Detection
Additive Combinatorics and its Applications in Theoretical CS
RS – Reed Solomon List Decoding.
The Curve Merger (Dvir & Widgerson, 2008)
Uncertain Compression
Block codes. encodes each message individually into a codeword n is fixed, Input/out belong to alphabet Q of cardinality q. The set of Q-ary n-tuples.
Cyclic Code.
Error Correction Coding
Theory of Information Lecture 13
Presentation transcript:

1 Asymptotically good binary code with efficient encoding & Justesen code Tomer Levinboim Error Correcting Codes Seminar (2008)

2 Outline Intro  codes  Singleton Bound Linear Codes Bounds  Gilbert-Varshamov  Hamming RS codes Code Concatention  Examples Wozencraft Ensemble Justesen Codes

3 Hamming Distance Hamming Distance between The Hamming Distance is a metric  Non negative  Symmetric  Triangle inequality  =

4 Weight The weight (wt) of Example (on board)

5 Code An (n,k,d) q code C is a function such that:   For every

6 Code (parameters) (n,k,d) q Parameters  n – block length  k – information length  d – minimum distance (actually, a lower bound)  q – size of alphabet  |C| = q k or k=log q |C|

7 Code (parameters div n) Asymptotic view of parameters as n  ∞ :  The rate  Relative minimum distance Thus an (n,k,d) q can be written as (1,R, δ ) q Notation: (n,k,d) q vs. [n,k,d] q – latter reserved for linear code (soon)

8 Trivial Code Example FEC3 = write each bit three time  R = ?  d = ? how many errors can we  Detect ? (d-1)  Correct ? t, where d=2t+1

9 Goal Would like to:  Maximize δ – correct more  Maximize R – send more information * conflicting goals - would like to be able to construct an [n,k,d] q code s.t. δ>0, R>0 and both are constant.  Minimize q – for practical reasons  Maximize number of codewords while minimizing n and keeping d large.

10 Singleton Bound Let C be an [n,k,d] q code then  k ≤ n – d + 1 equivalently  R ≤ 1 – δ + o(1) Proof: project C to first k-1 coordinates  On Board

11 Visual intuition On board... Ball q (x,r)  r:=d  r:=t (where d=2t+1) Vol q (n,r) = |Ball q (x,r)|

12 Linear Codes

13 Linear Codes An [n,k,d] q code C:F q K  F q n is linear when:  F q is a field  C is linear function (e.g., matrix) Linearity implies:  C(ax+by) = aC(x) + bC(y)  0 n member of C

14 Linear Codes (example) FEC3  [3,1,3] 2 Hadamard – longest linear code  [n,logn, n/2] 2  e.g., - [8,3,4] 2  (H - Matrix representation on board) Dimensions Asymptotic behavior

15 Linear Codes – minimum distance Lemma: if C:F q K  F q n is linear then Note: for clarity C x means C(x) Proof:  ≤ - trivial  ≥ - follows from linearity (on board)

16 Reed-Solomon code Idea: oversample a polynomial Let q be prime power and F q a finite field of size q. Let k<n and fix n elements of F q,  x 1,x 2,..x n Given a message m=(c 0..c k-1 ) interpret it has the coefficients of the polynomial p

17 RS Codes Thus (c 0..c k-1 ) is mapped to (p(x 1 ),..p(x n ))  Linear mapping (Vandermonde) Using linearity, can show for x≠0  RS meet the Singleton bound Proof: on board  (# of roots of a k-1 degree poly) Encoding time

18 Bounds

19 Gilbert-Varshamov Bound Preliminaries Binary Entropy Stirling Implying that:

20 Gilbert-Varshamov Bound Preliminaries Using the binary entropy we obtain On board

21 Gilbert-Varshamov Bound bound statement For every n and d<n/2 there is an (n,k,d) q (not necessarily linear) code such that: In terms of rate and relative min-distance:

22 Gilbert-Varshamov Bound Proof On Board Sketch of proof:  if C is maximal then:  And  Now use union bound and entropy to obtain result (we show for q=2, using binary entropy)

23 GV-Bound Gilbert proved this with a greedy construction Varshamov proved for linear codes  proved using random generator matrices – most matrices are good error correcting codes

24 Singleton / GV Plot Singleton (upper) Gilbert-Varshamov (lower)

25 Hamming Bound (Upper) With similar reasoning to GV bound but using For q=2 can show that

26 Bounds plot *Madhu Sudan (Lecture 5, 2001)

27 Code Concatenation

28 Code Concatenation - Motivation RS codes imply we can construct good [n,k,d] q codes for any q=p k Practically would like to work with small q (2, 2 8 ) Consider the “obvious” idea for binary code generated from C – simply convert each symbol from Σ n to log 2 q, What’s the problem with this approach ? (write the new code!)

29 Code Concatenation Due to Forney (1966) Two codes:  Outer:C out = [N,K,D] Q  Inner: C in = [n,k,d] q Inner code should encode each symbol of outer code  k = log q Q

30 Code Concatenation How does it work ? * Luca Trevisan (Lecture 2)

31 Code Concatenation What is the new code ?  d con = dD Proof:  On board

32 Code Concatenation (Examples) Asymptotically  δ = ¼  R=logn/2n  0 

33 Good Codes Can we “explicitly” build asymptotically good (linear) codes ?  asymptotically good = constant R, δ > 0 as n  ∞  Explicit = polytime constructable / logspace constructible

34 Asymptotically Good Codes

35 Asymptotically Good Codes GV tells us that most linear functions of a certain size are good error-correcting codes  Can find a good code in brute-force Use brute force on inner-code, where the alphabet is exponentially smaller! Do we really need to search ?

36 Wozencraft Ensemble Consider the following set of codes: such that (  R=1/2) ( Notice that (on board)

37 Wozencraft Ensemble Lemma: There exists an ensemble of codes c 1,..c N of rate ½ where N = q k -1 such that for at least (1-ε)N value of i, the code C i has distance d i s.t. Proof (on board), outline:  Different codes have only 0 n in common  Let y=C α (x), then, If wt(y)<d  y in Ball(0 n, d)  there are at most Vol(n,d) “bad” codes  For large enough n=2k, we have Vol(n,d) ≤ εN

38 Wozencraft Ensemble Implications:  Can construct entire ensemble in O(2 k )=O(2 n )  There are many such good codes, but which one do we use ?

39 Justesen Code Concatenation of:  C out - RS code over  a set of inner codes Justesen Code: C * = C out (C 1, C 2,.. C N )  Each symbol of C out is encoded using a different inner code C j  If RS has rate R  C * has rate R/2

40 Justesen Code - δ Denote the outer RS code [N,K,D] Q Claim: C* has relative distance

41 Justesen Code Proof Intuition: like regular concatenation, but εN bad codes. for x≠y, the outer code induces S={j | x j ≠y j },  |S| ≥D There are at most εN j’s such that Cj is bad and therefore at least |S|- εN ≥ D- εN ≥ (1-R- ε)N good codes  since RS implies D=N-(K-1) Each good code has relative distance ≥ d d * ≥ (1-R- ε)Nd

42 Justesen Code The concatenated code C * is an asymptotically good code and has a “super” explicit construction Can take q=2 to get such a binary code