1. exercise in the previous class p: the probability that symbols are delivered correctly C: 1 00 → 00000 01 → 01011 10 → 10101 11 → 11110 What is the threshold ( 閾値 ) of p with which using C is good/bad? without coding, correct probability = p 2 = A with coding, correct probability = p 5 + 5p 4 (1 – p) = B B > A if p 5 + 5p 4 (1 – p) – p 2 = p 2 (p – 1)(–4p 2 + p + 1) > 0 (roots = –0.39, 0, 0.64, 1) Using C is better if 0.64 < p < 1.

2. what did we learn? motivation and models of communication channels simple examples of linear codes even parity check codes (a 1, …, a k ) → (a 1, …, a k, p), p = a 1 + … + a k mod 2 error detection only horizontal and vertical parity check (2D) code (a 1, a 2, a 3, a 4 ) → 2 a1a1 a2a2 a3a3 a4a4 x1x1 x2x2 y1y1 y2y2 z → (a 1, a 2, a 3, a 4, x 1, x 2, y 1, y 2, z) one-bit error correcting, two-bits error detecting

3. errata: additional remark (cnt’d) We expect that 2D codes detect all two-bit errors. If we don’t use the parity of parity, then... 3 000 000 000 000 011 010 codeword 000 010 000 000 010 010 1-bit err. to the nearest codeword 1-bit err. some two-bit errors are not detected, instead, they are decoded to a wrong codeword. 000 010 000 to the nearest codeword

4. additional remark (cnt’d) ， corrected We expect that 2D codes detect all two-bit errors. If we don’t use the parity of parity, then... 4 000 000 000 000 011 010 codeword 000 010 000 000 010 010 1-bit err. to the nearest codeword 1-bit err. some two-bit errors are not detected, instead, they are decoded to a wrong codeword. to the nearest codeword

5. today’s class linear codes definition encoding decoding (error detection and correction) 5

6. (one of) definition(s) of linear codes a binary code C is linear  C is a vector space  for any u, v  C, we have u + v  C parity check code with length 3: 6 000011101110 000000011101110 011011000110101 101101110000011 110110101011000 how about parity check code with other length?

7. linearity of parity check codes C: parity check code for length k data (x 1,..., x k )... codewords in C are (x 1,..., x k, p) with p = x 1 +... + x k Theorem: parity check codes are linear Proof: confirm that u + v  C for any u, v  C. 7 u = (u 1,..., u k, p),(p = u 1 +... + u k ), v = (v 1,..., v k, q),(q = v 1 +... + v k ), u + v = (u 1 + v 1,..., u k + v k, p + q). because p + q = u 1 +... + u k + v 1 +... + v k = (u 1 + v 1 ) +... + (u k + v k ), p + q is a valid parity bit for (u 1 + v 1,..., u k + v k ).  u + v is a codeword Theorem: 2D codes are linear (proof omitted)

8. computation of parity bits parity check codes, 2D codes... for information bits (x 1, x 2,..., x k ), parity bits are defined by linear equations p = a 1 x 1 + a 2 x 2 +... + a k x k (a 1, a 2,..., a k ∈ {0, 1}) 8 (x 1,..., x k, p)p = x 1 +... + x k (x 1,..., x ab, p 1,..., p b, q 1,..., q a, r) In both codes, parity bits are sum of some of information bits.

9. linear parity bits make the code linear Lemma: for a linear equation f(x 1, x 2,..., x k ) = a 1 x 1 + a 2 x 2 +...+ a k x k, f(u 1, u 2,..., u k ) + f(v 1, v 2,..., v k ) = f(u 1 + v 1, u 2 + v 2,..., u k + v k ) Theorem: If parity bits are defined by linear equations, then C is linear. 9 codeword (u 1, u 2,..., u k,..., f(u 1, u 2,..., u k ),...) codeword (v 1, v 2,..., v k,..., f(v 1, v 2,..., v k ),...) ＋） (u 1 + v 1, u 2 + v 2,..., u k + v k,..., f(u 1, u 2,..., u k ) + f(v 1, v 2,..., v k ),...) (u 1 + v 1, u 2 + v 2,..., u k + v k,..., f(u 1 + v 1, u 2 + v 2,..., u k + v k ),...) ＝ codeword

10. example of linear code construction k = 3: information bits are (x 1, x 2, x 3 ). determine parity bits as you like... p 1 = x 1 + x 2 p 2 = x 2 + x 3 10 ⇒ the codeword is (x 1, x 2, x 3, p 1, p 2 ) 0000000101010110111010010101111100111100 000000000000101010110111010010101111100111100 001010010100000011100101110111100101110011001 010110101101110000000010111001111001001010111 011100111001011001010000011100110011011110010 100101001010111110011110000000001010101101110 101111011110010111001100100101000000111001011 110011100111100100101011101011011100000000101 111001110011001101111001001110010110010100000

11. codewords for unit vectors Assume that... we define m parity bits for k-bit information x 1, x 2,..., x k, with the j-th parity bit defined by p j = a j,1 x 1 + a j,2 x 2 +... + a j,k x k (a j,i = 0 or 1) the codeword for the unit vector (0,...,0, 1, 0,..., 0) is... c i = (0,..., 1,..., 0, a 1,i, a 2,i,..., a m,i ) 11 ^^^ i1k c 1 = 1 0 0 1 0 c 2 = 0 1 0 1 1 c 3 = 0 0 1 0 1 p 1 = x 1 + x 2 p 2 = x 2 + x 3 example in p.10: k = 3, m = 2 (a 1,1 a 1,2 a 1,3 ) = (1 1 0) (a 2,1 a 2,2 a 2,3 ) = (0 1 1) transposition ( 転置 )

12. “basis” in a linear code a vector space has a basis ( 基底 ) b 1, b 2,..., b k : any u in the space is written as u = u 1 b 1 + u 2 b 2 +... + u k b k (u i ∈ {0, 1}) 12 Theorem: Codewords c 1, c 2,..., c k for unit vectors constitute the basis of C. any codeword c  C is written as u = u 1 c 1 + u 2 c 2 +... + u k c k (u i ∈ {0, 1})

13. basis of the p.8 example k = 3: information bits are (x 1, x 2, x 3 ) p 1 = x 1 + x 2 p 2 = x 2 + x 3 13 c 1 = 1 0 0 1 0 c 2 = 0 1 0 1 1 c 3 = 0 0 1 0 1 codewords 00000 00101 01011 01110 10010 10111 11001 11100 = 0 ・ 10010 + 0 ・ 01011 + 0 ・ 00101 = 0 ・ 10010 + 0 ・ 01011 + 1 ・ 00101 = 0 ・ 10010 + 1 ・ 01011 + 0 ・ 00101 = 0 ・ 10010 + 1 ・ 01011 + 1 ・ 00101 = 1 ・ 10010 + 0 ・ 01011 + 0 ・ 00101 = 1 ・ 10010 + 0 ・ 01011 + 1 ・ 00101 = 1 ・ 10010 + 1 ・ 01011 + 0 ・ 00101 = 1 ・ 10010 + 1 ・ 01011 + 1 ・ 00101

14. generator matrix if the j-th parity bit is p j = a j,1 x 1 + a j,2 x 2 +... + a j,k x k, then c i = (0,..., 1,..., 0, a 1,i, a 2,i,..., a m,i ) c 1, c 2,..., c k is a basis of C any codeword u is written as u = u 1 c 1 + u 2 c 2 +... + u k c k 14 generator matrix of C encoding = multiplication of the generator matrix and a vector

15. the structure of the generator matrix 15 the generator matrix k × k identity matrix row vectors = codewords for unit vector transposition of the coefficients of the linear equations of parity bits

16. example 16 parity check codep = x 1 + x 2 + x 3 2D code p 1 = x 1 + x 2 p 2 = x 3 + x 4 q 1 = x 1 + x 3 q 2 = x 2 + x 4 r = x 1 + x 2 + x 3 + x 4 x1x1 x2x2 x3x3 x4x4 p1p1 p2p2 q1q1 q2q2 r

17. encoding 17 encoding of a 2D code to encode 0111... the codeword is 011110101

18. hardware encoder 18 u1u1 u2u2 u3u3 u4u4 u1u1 u2u2 u3u3 u4u4 p1p1 p2p2 q1q1 q2q2 r data codeword XOR

19. summary of the first half A code is linear if and only if... it is a vector space parity bits are defined by linear equations codewords of unit vectors is a basis of the code the generator matrix... contains basis codewords as its row vectors gives mathematical principle for encoding  an encoder is realizable by a combinatorial circuit ( 組み合わせ回路 ) 19

20. condition for a vector to be a codeword (1) consider a 2D code 20 p 1 = x 1 + x 2 p 2 = x 3 + x 4 q 1 = x 1 + x 3 q 2 = x 2 + x 4 r = x 1 + x 2 + x 3 + x 4 a vector (x 1 x 2 x 3 x 4 p 1 p 2 q 1 q 2 r) is a codeword if and only if

21. condition for a vector to be a codeword (2) 21 p 1 = x 1 + x 2 p 2 = x 3 + x 4 q 1 = x 1 + x 3 q 2 = x 2 + x 4 r = x 1 + x 2 + x 3 + x 4 x 1 + x 2 – p 1 = 0 x 3 + x 4 – p 2 = 0 x 1 + x 3 – q 1 = 0 x 2 + x 4 – q 2 = 0 x 1 + x 2 + x 3 + x 4 – r = 0 in binary world, x – y = x ＋ y x 1 + x 2 + p 1 = 0 x 3 + x 4 + p 2 = 0 x 1 + x 3 + q 1 = 0 x 2 + x 4 + q 2 = 0 x 1 + x 2 + x 3 + x 4 + r = 0 parity check equations (x 1 x 2 x 3 x 4 p 1 p 2 q 1 q 2 r) is a codeword move the lhs to rhs...

22. condition for a vector to be a codeword (3) 22 x 1 + x 2 + p 1 = 0 x 3 + x 4 + p 2 = 0 x 1 + x 3 + q 1 = 0 x 2 + x 4 + q 2 = 0 x 1 + x 2 + x 3 + x 4 + r = 0 (x 1 x 2 x 3 x 4 p 1 p 2 q 1 q 2 r) is a codeword Is (x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 ) a codeword? transpose the vector, multiply to this matrix from the right, and see if the result is 0 or not. zero ⇒ it’s a codeword nonzero ⇒ it’s not a codeword parity check matrix

23. parity check matrix for error detection consider a 2D code Is 110101101 a codeword?... yes 23 Is 011011010 a codeword?... no

24. generator and check matrices 24 definition of parity bits p 1 = a 1,1 x 1 + a 1,2 x 2 +... + a 1,k x k p 2 = a 2,1 x 1 + a 2,2 x 2 +... + a 2,k x k : p m = a m,1 x 1 + a m,2 x 2 +... + a m,k x k generator matrix identitycoefficients transposed a 1,1 x 1 + a 1,2 x 2 +... + a 1,k x k + p 1 = 0 a 2,1 x 1 + a 2,2 x 2 +... + a 2,k x k + p 2 = 0 : a m,1 x 1 + a m,2 x 2 +... + a m,k x k + p m = 0 k rows parity check matrix identitycoefficients n － k rows

25. example of matrices 25 2D code ： n = 9, k = 4, m = n － k = 5. x1x1 x2x2 x3x3 x4x4 p1p1 p2p2 q1q1 q2q2 r generator matrixparity check matrix p 1 = x 1 + x 2 p 2 = x 3 + x 4 q 1 = x 1 + x 3 q 2 = x 2 + x 4 r = x 1 + x 2 + x 3 + x 4

26. syndrome For a parity check matrix H and a vector v, if Hv T = 0, then v  C if Hv T  0, then v  C the vector Hv T is called the syndrome ( シンドローム ) of v: if the syndrome of v is zero, then v  C if the syndrome of v is nonzero, then v  C The syndrome is more useful, because it contains the information of errors. 26

27. syndrome and error send a codeword u to a binary symmetric channel an error vector e is added to the codeword in the channel the received vector is v = u + e 27 u e v = u + e noise codewordreceived if e = 0 (no error), then the syndrome of v is... Hv T = Hu T = 0 if e  0 (error occurs), then the syndrome of v is... Hv T = H(u + e) T = Hu T + He T = He T the syndrome is solely determined from e, independently from u... if you see the syndrome, then you can say what e is.

28. error patterns determine the syndrome 28 000000000 is sent, 000100000 is received... H(0 0 0 1 0 0 0 0 0) T = (0 1 0 1 1) T. 110000110 is sent, 110100110 is received... H(1 1 0 1 0 0 1 1 0) T = (0 1 0 1 1) T. ⇒ if the syndrome is (0 1 0 1 1), then the fourth bit is in error independent from the sent codeword

29. error correction if you know the correspondence between error patterns and syndromes, then you can correct errors. 29 receivedsyndrome error pattern decoding result v = u + e compute the syndrome: s = Hv T s table of errors / syndromes..... e u

30. one-bit error Let n be a codeword, and let h i be the i-th column vector of H: 30 the syndrome of a one-bit error e = (0 0... 0 1 0... 0) is... only one-bit error at the i-th symbol position ⇔ syndrome equals to the i-th vector of H (the table of errors / syndromes not needed)

31. example of error correction 2D code 31 000000000, 000101011, 001001101, 001100110, 010010011, 010111000, 011011110, 011110101, 100010101, 100111110, 101011000, 101110011, 110000110, 110101101, 111001011, 111100000. codewords parity check matrix if 101001000 is received... ⇒ the syndrome is H(1 0 1 0 0 1 0 0 0) T = (1 0 0 0 0) T ⇒ this is the fifth column of H ⇒ the fifth-bit is in error, 101011000 must be sent if 101100110 is received... ⇒ the syndrome is H(1 0 1 1 0 0 1 1 0) T = (1 0 1 0 1) T ⇒ this is the first column of H ⇒ the first-bit is in error, 001100110 must be sent

32. parity check matrix and the ability of codes (1) one-bit error at the i-th symbol position ⇔ syndrome equals the i-th vector of H if several column vectors in H are the same, then different error patterns result in the same syndrome the error pattern is not uniquely determined 32 parity check codep = x 1 + x 2 p 1 = x 1 + x 2 p 2 = x 2 + x 3 the example code in p. 8 error correction NOT possible

33. parity check matrix and the ability of codes (2) one-bit error at the i-th symbol position ⇔ syndrome equals the i-th vector of H if all column vectors in H are different, then different error patterns result in different syndromes the error pattern is uniquely determined 33 error correction possible 2D code

34. summary definition of linear codes vector space, parity bits defined by linear equations generator matrix matrix of basis codewords contributes to the encoding parity check matrix represents constraints among symbols determines syndrome  error correction and error detection 34

35. exercise Consider an “odd” parity check code C whose codewords are (x 1, …, x k, p) with p = x 1 +…+x k +1. Is C a linear code? Construct a 2D code for 6-bit information (a 1,..., a 6 ) as follows. determine the generator and parity check matrices encode 011001 using the generator matrix correct an error in the sequence 110111001010 35 a1a1 a2a2 a3a3 a4a4 a5a5 a6a6 p1p1 p2p2 q1q1 q2q2 q3q3 r (a 1,..., a 6 ) → (a 1,..., a 6, p 1, p 2, q 1, q 2, q 3, r)