Introduction to Coding Theory

p2. Outline [1] Introduction [2] Basic assumptions [3] Correcting and detecting error patterns [4] Information rate [5] The effects of error correction and detection [6] Finding the most likely codeword transmitted [7] Some basic algebra [8] Weight and distance [9] Maximum likelihood decoding [10] Reliability of MLD [11] Error-detecting codes [12] Error-correcting codes

p3. Introduction to Coding Theory [1] Introduction Coding theory The study of methods for efficient and accurate transfer of information Detecting and correcting transmission errors Information transmission system Information Sink Receiver (Decoder) Transmitter (Encoder) Communication Channel Information Source Noise k-digit n-digit

p4. Introduction to Coding Theory [2] Basic assumptions Definitions Digit ： 0 or 1(binary digit) Word ： a sequence of digits Example ： Binary code ： a set of words Example ： 1. {00,01,10,11}, 2. {0,01,001} Block code ： a code having all its words of the same length Example ： {00,01,10,11}, 2 is its length Codewords ： words belonging to a given code |C| ： Size of a code C(#codewords in C)

p5. Introduction to Coding Theory Assumptions about channel 3. The probability of any digit being affected in transmission is the same as the other one. Channel 1. Receiving word by word 2. Identifying the beginning of 1 st word

p6. Introduction to Coding Theory Binary symmetric channel reliabilityp: In many books, p denotes crossover probability. Here crossover probability(error prob.) is 1-p

p7. Introduction to Coding Theory [3] Correcting and detecting error patterns Any received word should be corrected to a codeword that requires as few changes as possible. Cannot detect any errors !!! Channel correct source parity-check digit Channel correct source

p8. Introduction to Coding Theory [4] Information rate Definition: information rate of code C Examples

p9. Introduction to Coding Theory [5] The effects of error correction and detection 1. No error detection and correction 1 wrong word / 10 sec 6 wrong words / min 360 wrong words / hr 8640 wrong words / day Reliability p= Then Pr(a word is transmitted incorrectly) = 1-p 11 ≒ 11x10 -8 Transmission rate=10 7 digits/sec 11x10 -8 (wrong words/words)x10 7 /11(words/sec)=0.1 wrong words/sec Let C={0,1} 11 ={ , …, }

p10. Introduction to Coding Theory 2. parity-check digit added( Code length becomes 12 ) Any single error can be detected ! (3, 5, 7,..errors can be detected too !) Pr(at least 2 errors in a word)=1-p x p 11 (1-p) ≒ 66x So 66x x 10 7 /12 ≒ 5.5 x wrong words/sec one word error every 2000 days! The cost we pay is to reduce a little information rate + retransmission(after error detection!)

p11. Introduction to Coding Theory Task ： design codes with reasonable information rates low encoding and decoding costs some error-correcting capabilities 3. 3-repetition code Any single error can be corrected ! Code length becomes 33 and information rate becomes 1/3

p12. Introduction to Coding Theory [6] finding the most likely codeword transmitted Example ： BSC channel p ： reliability d ： #digits incorrectly transmitted n ： code length Code length = 5

p13. Theorem Suppose we have a BSC with ½ < p < 1. Let and be codewords and a word, each of length. Suppose that and disagree in positions and and disagree in positions. Then Introduction to Coding Theory Assume is sent when is received if

p14. Introduction to Coding Theory Example channel d (number of disagreements with ) ← smallest d

p15. Introduction to Coding Theory [7] Some basic algebra Addition ： Multiplication ： ： the set of all binary words of length n Addition ： Scalar multiplication ： ： zero word

p16. Introduction to Coding Theory K n is a vector space words of length n scalar

p17. Introduction to Coding Theory [8] Weight and distance Hamming weight ： the number of times the digit 1 occurs in Example ： Hamming distance ： the number of positions in which and disagree Example ：

p18. Introduction to Coding Theory Some facts ： words of length n digit

p19. Introduction to Coding Theory CMLD ： Complete Maximum Likelihood Decoding If only one word v in C closer to w, decode it to v If several words closest to w, select arbitrarily one of them IMLD ： Incomplete Maximum Likelihood Decoding If only one word v in C closer to w, decode it to v If several words closest to w, ask for retransmission channel k n decode CMLD IMLD [9] Maximum likelihood decoding Error pattern codeword Source string x w=v+u

p20. Introduction to Coding Theory The most likely codeword sent is the one with the error pattern of smallest weight error pattern Received w Error Pattern Decode v w w w Example ： Construct IMLD. |M|=3, C={0000, 1010, 0111}

p21. Introduction to Coding Theory [10] Reliability of MLD The probability that if v is sent over a BSC of probability p then IMLD correctly concludes that v was sent The higher the probability is, the more correctly the word can be decoded!

p22. Introduction to Coding Theory [11] Error-detecting codes + Error pattern Can ’ t detect Can detect Example ： Error Pattern uv = 000v = Can ’ t detectCan detect

p23. Introduction to Coding Theory the distance of the code C ： the smallest of d(v,w) in C Theorem A code C of distance d will at least detect all non-zero error patterns of weight less than or equal to d-1. Moreover, there is at least one error pattern of weight d which C will not detect. t error-detecting code It detects all error patterns of weight at most t and does not detect at least one error pattern of weight t+1 A code with distance d is a d-1 error-detecting code.

p24. Introduction to Coding Theory [12] Error-correcting codes Theorem A code of distance d will correct all error patterns of weight less than or equal to. Moreover, there is at least one error pattern of weight 1+ which C will not correct. t error-correcting code It corrects all error patterns of weight at most t and does not correct at least one error pattern of weight t+1 A code of distance d is a error-correcting code. + Error pattern For all v in C, if it is closer to v than any other word in C, a code C can correct u.

p25. Introduction to Coding Theory ReceivedError PatternDecode w000 + w111 + wv * * * * * * * *111 C corrects error patterns 000,100,010,001