Lecture 18 The Main Coding Theory Problem (Section 4.7) Theory of Information Lecture 13 Theory of Information Lecture 18 The Main Coding Theory Problem (Section 4.7)
Transmission Rate Efficiency --- large number of codewords for a given length Reliability --- (here) high minimum distance. Alas, these two main goals are at odds with each other! For easy comparison, we may assume that the source symbols are strings over the same alphabet as the codewords are. Then, roughly, each codeword of an r-ary code of size M captures a source “symbol” of length logrM. Thus the longer the codewords, the less the efficiency of encoding the source. Based on these remarks, a fair measurement of efficiency is what is called “transmission rate”: Definition The transmission rate R(C) of an r-ary (n,M)-code C is (logrM)/n. What are the transmission rates for the following codes? {0000,0001,0010,0011,0100,0101,0110,0111,1000,1001,1010,1011,1100,1101,1110,1111} {0000,0011,0101,0110,1001,1010,1100,1111} {0000,1111} {00,01,02,10,11,12,20,21,22} {00,11,22}
Error Correction Rate The longer the codewords, the higher the requirements on how many errors per transmitted codeword a good encoding scheme should allow to detect. Hence, a fair measurement of reliability is what is called “error correction rate”: Definition The error correction rate (C) of an (n,M,d)-code C is (d-1)/2 / n. What are the error correction rates for the following codes? {0000,0001,0010,0011,0100,0101,0110,0111,1000,1001,1010,1011,1100,1101,1110,1111} {0000,0011,0101,0110,1001,1010,1100,1111} {0000,1111} {00,01,02,10,11,12,20,21,22} {00,11,22} {11111,00000} {111111111,000000000} Give me a code with the error correction rate of 3 Give me a code with the error correction rate of 4 How’bout the transmission rates of the last four codes? We can see that the two sorts of rates are really at odds with each other.
The Famous Numbers Ar(n,d) For given values n (codeword length) and d (minimum distance), Ar(n,d) denotes the largest possible size M for which there exists an r-ary (n,M,d) code: Ar(n,d)=max{M | there exists an r-ary (n,M,d)-code} Any r-ary (n,M,d)-code C such that M=Ar(n,d) is called optimal. An optimal code thus has the highest possible transmission rate among the (n,d)-codes. Ar(n,d) is thus a very important number, yet very little is known so far about how to find it in a general case. Determining the values of Ar(n,d) has become known as the main coding theory problem. In some limited cases we can still succeed though.
Some Theorems on Ar(n,d) Theorem 4.7.1 A2(4,3)=2. PROOF. Let C be a (4,M,3)-code. By Lemma 4.6.1, we can assume that 0000 is in C. Since d(C)=3, any other codeword in C must be at distance at least 3 from 0000. This leaves us with 1110, 1101, 1011, 0111, 1111 as the only possible values for those other words. But no pair of these has distance 3 apart, and so only one can be included in C. Thus, C can have at most 2 codewords, implying Ar(4,3)2. And since C={0000,1110} is a (4,2,3)-code, we conclude Ar(4,3)=2. Theorem 4.7.4 1. Ar(n,d) rn for all 1dn 2. Ar(n,1)=rn 3. Ar(n,n)=r
Theory of Information Lecture 13 Homework 1. Exercise 19 of Section 4.7 2. What are the transmission rates for the following codes? a) {00011,11000,00110,01100,11100,11001,10011,00111} b) {000,001,002,110,111,112,220,221,222} 3. What are the error correction rates for the following codes? 4. We did not define error detection rate. You do it now (provide a formula).