1. exercise in the previous class (1) Define (one of) (15, 11) Hamming code: construct a parity check matrix, and determine the corresponding generator matrix 1 generator matrix parity check matrix m = 4 2 4 – 1 nonzero vectors

2. exercise in the previous class (2) Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4. 2 weight 1: Hu T = h i = 0...contradiction weight 2: Hu T = h i + h j = 0...contradiction weight 3: Hu T = h i + h i + h l = 0, but... column vectors have odd weight, and h i + h i has even weight h i + h i = h l never happens...contradiction weight 4: we have 10001110  C

3. previous class Hamming code one-bit error correcting, one of perfect codes minimum Hamming distance smallest distance between two codewords gives the maximum number of correctable errors easy but “rough” estimation of performance 3

4. today’s class two independent topics: performance evaluation using weight distribution slightly complicated ( ちょっと複雑 ) precise and detailed estimation is possible cyclic code a subclass of linear codes advantages in realizations 4

5. is the minimum distance everything? If the minimum Hamming distance of C is d min, then C can correct (d min – 1) / 2 or less bit errors. 5 C1C1 C2C2 C3C3 <= ? Consider three codes with the same code length... many neighborssmall # of neighbors d min C 2 seems better than C 3...

6. 1 codeword at distance 0 4 codewords at distance 3 3 codewords at distance 4 1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4 weight distribution 6 C={000000,100111,010110,110001, 001011,101100,011101,111010}. weight distribution in linear codes, all codewords have the same distance distribution 3 4 000000 distance distribution from 000000 from the view of 000000,

7. performance evaluation with weight distribution assume... BSC with bit error probability p one-bit error correcting send all-zero codeword and evaluate the probabilities of... correctly decoded wrongly decoded error detected 7 C={000000,100111,010110,110001, 001011,101100,011101,111010}. r 000000 decoder ? received vector 1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4

8. the domain of the received vector the received vector r  V 6 V 6 contains eight decoding territories 8 C={000000,100111,010110,110001, 001011,101100,011101,111010}. correctly decoded; if r falls in the territory of 000000 wrongly decoded; if r falls in the territory of the other codewords error detected; if r does not fall in the territories (not all vectors shown)

9. probability of correct decoding correctly decoded if r belongs to T = {000000, 000001, 000010, 000100, 001000, 010000, 100000} 9 r is decoded correctly with probability P c = (1 – p) 6 + 6p(1-p) 5 prob. that r  T and w H (r) = 0 prob. that r  T and w H (r) = 1 (1 – p) 6 6p( 1 – p) 5 (1 – p) 6 p( 1 – p) 5

10. probability of wrong decoding (1) case1: r belongs to the territory of a weight-3 codeword 001011 T = {001011, 001010, 001001, 001111, 000011, 011011, 101011} 10 weight 3 weight 2 weight 4 prob. that r  T and w H (r) = 3 prob. that r  T and w H (r) = 2 p 3 (1 – p) 3 3p 2 ( 1 – p) 4 prob. that r  T and w H (r) = 4 3p 4 ( 1 – p) 2 r is decoded to the wrong codeword 001011 with probability 3p 2 (1 – p) 4 + p 3 (1 – p) 3 + 3p 4 (1 – p) 2... the same discussion holds for other codewords with weight 3

11. probability of wrong decoding (2) case2: r belongs to the territory of a weight-4 codeword 100111 T = {100111, 100110, 100101, 100011, 101111, 110111, 000111} 11 prob. that r  T and w H (r) = 4 prob. that r  T and w H (r) = 3 p 4 (1 – p) 2 4p 3 ( 1 – p) 3 prob. that r  T and w H (r) = 5 2p 5 ( 1 – p) r is decoded to the wrong codeword 100111 with probability 4p 3 (1 – p) 3 + p 4 (1 – p) 2 + 2p 5 (1 – p)... the same discussion holds for other codewords with weight 4 weight 4 weight 3 weight 5

12. probability of wrong decoding (3) wrongly decoded; P e = 4×(falls in the territory of one of weight-3 codewords) + 3×(falls in the territory of one of weight-4 codewors) = 4×(3p 2 (1 – p) 4 + p 3 (1 – p) 3 + 3p 4 (1 – p) 2 ) + 3×(4p 3 (1 – p) 3 + p 4 (1 – p) 2 + 2p 5 (1 – p)) = 12p 2 (1 – p) 4 + 16p 3 (1 – p) 3 + 15p 4 (1 – p) 2 + 6p 5 (1 – p) error detected with probability P d = 1 – P c – P e 12 C={000000,100111,010110,110001, 001011,101100,011101,111010}. 1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4

13. the probabilities in a graph 13

14. comparison example (7,4) Hamming code: 0000000, 1000101, 0100111, 0010110, 0010110, 1010011, 0110001, 1110100, 0001011, 1001110, 0101100, 1101001, 0011101, 1011000, 0111010, 1111111 (9, 4) 2D code: 000000000, 000101011, 001001101, 001100110, 010010011, 010111000, 011011110, 011110101, 100010101, 100111110, 101011000, 101110011, 110000110, 110101101, 111001011, 111100000 14 weight 0 3 4 7 #codewords 1 7 1 weight 0 4 6 #codewords 1 9 6

15. comparison of two codes 15 Hamming code seems better, but...

16. “retransmission” re-transmission of broken data is allowed in some systems 16 2D code + one-time re-transmission is better for p ≤ 0.28.

17. half the way... two independent topics: performance evaluation using weight distribution slightly complicated ( ちょっと複雑 ) precise and detailed estimation is possible cyclic code a subclass of linear codes advantages in realizations 17

18. cyclic codes Linear codes are characterized by matrix operations. Encoders/decoders are realized by combinatorial circuits. simple but not scalable... O(n 2 ) for the code length n cyclic codes ( 巡回符号 ) a subclass of linear codes Encoders/decoders are realized by shift registers. complex but scalable... O(n) complexity 18 codes linear codes cyclic codes

19. preliminary (1) We represent binary vectors by binary polynomials ( 多項式 ). 19 11101x 4 + x 3 + x 2 + x + 1 addition of polynomials: (subtraction) x 4 + x 3 + x 2 + 1 x 3 + x + 1 +) x 4 + x 2 + x 11101 01011 10110 +) multiplication of polynomials: x 4 + x 3 + x 2 + 1 x 3 + x + 1 ×) x 4 + x 3 + x 2 + 1 x 5 + x 4 + x 3 + x x 7 + x 6 + x 5 + x 3 x 7 + x 6 + x 3 + x 2 + x + 1 11001111 11101 01011 ×) 11101 multiply by x m = left-shift of m bits

20. preliminary (2) 20 division of polynomials: x 6 + x 4 x 4 + x 3 + x 2 + 1) x 2 + x + 1 x 6 + x 5 + x 4 + x 2 x 5 + x 2 x 5 + x 4 + x 3 + x x 4 + x 3 + x 2 + x x 4 + x 3 + x 2 + 1 x + 1 11101 ) 1010000 11101 10010 11101 11110 11101 11 111 addition = subtraction = XOR of coefficients the division circuit is easily implemented by a shift register

21. division circuit (1) divide p(x) = x 6 + x 4 by q(x) = x 4 + x 3 + x 2 + 1... 21 11101 1 0 1 0 0 0 0 q(x)q(x) p(x)p(x) quotient ( 商 ) remainder ( 剰余 ) 1. store p(x) to the registers 2. if MSB = 1, then AND gates are activated, and registers are XOR’ed with q(x) 3. left-shift, and go to step 1

22. division circuit (2) 22 01001 11101 0 11101 ) 1010000 11101 010010 11101 11110 11101 11 111 10100 11101 0 one-cycle of operations = one-step of the calculus contents of the register = remainder of the division

23. definition of cyclic codes How to construct an (n, k) cyclic code: length n, k information symbols, m = n – k parity symbols Step 1: choose an order-m polynomial G(x) which divides x n + 1. Step 2: C = {multiples of G(x) with order < n} Example: n = 7, m = 4: G(x) = x 4 + x 3 + x 2 + 1 divides x 7 +1, because x 7 +1 = (x 3 + x 2 + 1) (x 4 + x 3 + x 2 + 1). G(x)...generator polynomial 23 1000000111101) 1101001 11101 0 1101 G(x)G(x) x 7 +1

24. construction of a cyclic code Step 1: choose an order-m polynomial G(x) which divides x n + 1. Step 2: C = {multiples of G(x) with order < n} n = 7, m = 4: G(x) = x 4 + x 3 + x 2 + 1 24 G(x)G(x) (x 2 +x+0)× C=C= 0000000 0011101 0111010 0100111 1110100 1101001 1001110 1010011 G(x)G(x) (x 2 +x+1)× G(x)G(x) G(x)G(x) G(x)G(x) G(x)G(x) G(x)G(x) G(x)G(x) =

25. properties of cyclic codes (1) Lemma: a cyclic code is a linear code proof: show that, for c 1, c 2  C, we have c 1 +c 2  C c 1  C  c 1 = f 1 (x)G(x) c 2  C  c 2 = f 2 (x)G(x) c 1 +c 2 = (f 1 (x)+f 2 (x)) G(x)  C 25

26. properties of cyclic codes (2) Lemma: if (a n-1, a n-2,..., a 0 )  C, then (a n-2,..., a 0, a n-1 )  C proof: Let W(x) = a n-1 x n-1 +... + a 0 and W’(x) = a n-2 x n-1 +... + a 0 x + a n-1. W(x) is a multiple of G(x) W’(x) = a n-2 x n-1 +... + a 0 x + a n-1 = a n-1 x n + a n-2 x n-1 +... + a 0 x + a n-1 + a n-1 x n = xW(x) + a n-1 (x n + 1) 26 multiple of G(x)  W’(x) is a multiple of G(x), and (a n-2,..., a 0, a n-1 )  C. A cyclic code C is closed for a cyclic shift.

27. three approaches for encoding three approaches for an encoding procedure: matrix approach use a generator matrix... no advantage of cyclic codes multiplication approach codeword = (info. symbols) × G(x) the code not systematic (  p.24) division approach slightly complicated (for human) make the code systematic easily implemented by shift registers 27

28. encoding by division 1. represent information symbols by a polynomial A(x) 2. divide A(x)x m by G(x), and let B(x) be the remainder 3. let A(x)x m + B(x) be a codeword encoding 011, with n = 7, k = 3, m = 4, G(x) = x 4 + x 3 + x 2 + 1 1. A(x) = x + 1 2. A(x)x 4 = x 5 + x 4 = x(x 4 + x 3 + x 2 + 1) + (x 3 + x), and B(x) = x 3 + x 3. A(x)x 4 + B(x)= x 5 + x 4 + x 3 + x, the codeword = 0111010 28 A(x)A(x) D(x)D(x) G(x)G(x) B(x)B(x) x4x4 ÷ divisor dividend quotient remainder W(x) = A(x) x 4 + B(x)

29. did we really make “encoding”? simple question: Is A(x)x m + B(x) really a codeword?  is A(x)x m + B(x) divided by G(x)? Yes, note that... B(x) is a remainder of A(x)x m in a binary world, A(x)x p + B(x) = A(x)x p – B(x) “A(x)x p – B(x)” = “remove the remainder” if A(x)x p – B(x) is divided by G(x), then there is no remainder... 29

30. example n = 7, k = 3, m = 4, G(x) = x 4 + x 3 + x 2 + 1 30 data 000 001 010 011 100 101 110 111 A(x) x 2 +x+0 x 2 +x+1 A(x)x 4 0000000 0010000 0100000 0110000 1000000 1010000 1100000 1110000 B(x) x 3 +x 2 +x+0 x 3 +x 2 +x+1 A(x)x m + B(x) 0000000 0011101 0100111 0111010 1001110 1010011 1101001 1110100 systematic code encoder ≈ division circuit  O(n) < O(n 2 ) of matrix operation

31. error “detection” of cyclic codes error “detection” is easy for cyclic codes u  C  u (in a polynomial representation) is divided by G(x) 31 G(x)G(x) = 0... no error  0... error ÷ divisor dividend quotient remainder received u one division circuit is used for encoding and error detection  reduces the cost of realization Cyclic Redundancy Check (CRC)... used in many communication systems

32. error “correction” of cyclic codes general algorithm for all cyclic codes error-trapping decoder [Kasami 61] 32 Tadao Kasami 1930-2007 E. Berlekamp, 1940-J. L. Massey, 1934- I. Reed, 1923- (left) and G. Solomon, 1930-1996 special algorithms for special cyclic codes Berlekamp-Massey algorithm for... BCH (Bose-Chaudhuri-Hocquenghem) codes Reed-Solomon Codes

33. summary performance evaluation using weight distribution slightly complicated ( ちょっと複雑 ) precise and detailed estimation is possible cyclic code a subclass of linear codes advantages in realizations 33

34. exercise Consider the following code C. determine the weight distribution of C compute the “three” probabilities (p. 7), and draw a graph 34 We want to construct a cyclic code with n = 7, k = 4, and m =3. confirm that G(x) = x 3 + x 2 + 1 can be a generator polynomial encode 0110 decide if 0001100 is a correct codeword or not