1. Introduction to Coding Theory

2. p2. Outline [1] Introduction [2] Basic assumptions [3] Correcting and detecting error patterns [4] Information rate [5] The effects of error correction and detection [6] Finding the most likely codeword transmitted [7] Some basic algebra [8] Weight and distance [9] Maximum likelihood decoding [10] Reliability of MLD [11] Error-detecting codes [12] Error-correcting codes

3. p3. Introduction to Coding Theory [1] Introduction Coding theory The study of methods for efficient and accurate transfer of information Detecting and correcting transmission errors Information transmission system Information Sink Receiver (Decoder) Transmitter (Encoder) Communication Channel Information Source Noise k-digit n-digit

4. p4. Introduction to Coding Theory [2] Basic assumptions Definitions Digit ： 0 or 1(binary digit) Word ： a sequence of digits Example ： 0110101 Binary code ： a set of words Example ： 1. {00,01,10,11}, 2. {0,01,001} Block code ： a code having all its words of the same length Example ： {00,01,10,11}, 2 is its length Codewords ： words belonging to a given code |C| ： Size of a code C(#codewords in C)

5. p5. Introduction to Coding Theory Assumptions about channel 3. The probability of any digit being affected in transmission is the same as the other one. Channel 1. Receiving word by word 2. Identifying the beginning of 1 st word

6. p6. Introduction to Coding Theory Binary symmetric channel reliabilityp: In many books, p denotes crossover probability. Here crossover probability(error prob.) is 1-p

7. p7. Introduction to Coding Theory [3] Correcting and detecting error patterns Any received word should be corrected to a codeword that requires as few changes as possible. Cannot detect any errors !!! Channel correct source parity-check digit Channel correct source

8. p8. Introduction to Coding Theory [4] Information rate Definition: information rate of code C Examples

9. p9. Introduction to Coding Theory [5] The effects of error correction and detection 1. No error detection and correction 1 wrong word / 10 sec 6 wrong words / min 360 wrong words / hr 8640 wrong words / day Reliability p=1-10 -8 Then Pr(a word is transmitted incorrectly) = 1-p 11 ≒ 11x10 -8 Transmission rate=10 7 digits/sec 11x10 -8 (wrong words/words)x10 7 /11(words/sec)=0.1 wrong words/sec Let C={0,1} 11 ={0000000000, …, 11111111111}

10. p10. Introduction to Coding Theory 2. parity-check digit added( Code length becomes 12 ) Any single error can be detected ! (3, 5, 7,..errors can be detected too !) Pr(at least 2 errors in a word)=1-p 12 -12 x p 11 (1-p) ≒ 66x10 -16 So 66x10 -16 x 10 7 /12 ≒ 5.5 x 10 -9 wrong words/sec one word error every 2000 days! The cost we pay is to reduce a little information rate + retransmission(after error detection!)

11. p11. Introduction to Coding Theory Task ： design codes with reasonable information rates low encoding and decoding costs some error-correcting capabilities 3. 3-repetition code Any single error can be corrected ! Code length becomes 33 and information rate becomes 1/3

12. p12. Introduction to Coding Theory [6] finding the most likely codeword transmitted Example ： BSC channel p ： reliability d ： #digits incorrectly transmitted n ： code length Code length = 5

13. p13. Theorem 1.6.3 Suppose we have a BSC with ½ < p < 1. Let and be codewords and a word, each of length. Suppose that and disagree in positions and and disagree in positions. Then Introduction to Coding Theory Assume is sent when is received if

14. p14. Introduction to Coding Theory Example channel d (number of disagreements with ) 011013 010014 101002 ← smallest d 101013

15. p15. Introduction to Coding Theory [7] Some basic algebra Addition ： Multiplication ： ： the set of all binary words of length n Addition ： Scalar multiplication ： ： zero word

16. p16. Introduction to Coding Theory K n is a vector space words of length n scalar

17. p17. Introduction to Coding Theory [8] Weight and distance Hamming weight ： the number of times the digit 1 occurs in Example ： Hamming distance ： the number of positions in which and disagree Example ：

18. p18. Introduction to Coding Theory Some facts ： words of length n digit

19. p19. Introduction to Coding Theory CMLD ： Complete Maximum Likelihood Decoding If only one word v in C closer to w, decode it to v If several words closest to w, select arbitrarily one of them IMLD ： Incomplete Maximum Likelihood Decoding If only one word v in C closer to w, decode it to v If several words closest to w, ask for retransmission channel k n decode CMLD IMLD [9] Maximum likelihood decoding Error pattern codeword Source string x w=v+u

20. p20. Introduction to Coding Theory The most likely codeword sent is the one with the error pattern of smallest weight error pattern Received w Error Pattern Decode v 0000 + w1010 + w0111 + w 0000 101001110000 1000 00101111- 0100 111000110000 0010 10000101- 0001 101101100000 1100 01101011- 1010 000011011010 1001 00111110- 0110 110000010111 0101 111100100111 Example ： Construct IMLD. |M|=3, C={0000, 1010, 0111}

21. p21. Introduction to Coding Theory [10] Reliability of MLD The probability that if v is sent over a BSC of probability p then IMLD correctly concludes that v was sent The higher the probability is, the more correctly the word can be decoded!

22. p22. Introduction to Coding Theory [11] Error-detecting codes + Error pattern Can ’ t detect Can detect Example ： Error Pattern uv = 000v = 111 000 111 100 011 010 101 001 110 001 101 010 011 100 111 000 Can ’ t detectCan detect

23. p23. Introduction to Coding Theory the distance of the code C ： the smallest of d(v,w) in C Theorem 1.11.14 A code C of distance d will at least detect all non-zero error patterns of weight less than or equal to d-1. Moreover, there is at least one error pattern of weight d which C will not detect. t error-detecting code It detects all error patterns of weight at most t and does not detect at least one error pattern of weight t+1 A code with distance d is a d-1 error-detecting code.

24. p24. Introduction to Coding Theory [12] Error-correcting codes Theorem 1.12.9 A code of distance d will correct all error patterns of weight less than or equal to. Moreover, there is at least one error pattern of weight 1+ which C will not correct. t error-correcting code It corrects all error patterns of weight at most t and does not correct at least one error pattern of weight t+1 A code of distance d is a error-correcting code. + Error pattern For all v in C, if it is closer to v than any other word in C, a code C can correct u.

25. p25. Introduction to Coding Theory ReceivedError PatternDecode w000 + w111 + wv 000000*111000 100100*011000 010010*101000 001001*110000 110 001*111 101 010*111 011 100*111 000*111 C corrects error patterns 000,100,010,001