Lecture 4 What are Codes? (Section 1.2) Theory of Information Lecture 4 Theory of Information Lecture 4 What are Codes? (Section 1.2)

Theory of Information Lecture 4 Definition of a Code Theory of Information Lecture 4 DEFINITION Let A={a1,…,ar} be set, which we call a code alphabet. An r-ary code over A is a set CA*. The elements of C are called codewords. The number r is called the radix of the code. 2-ary codes are said to be binary, 3-ary codes are said to be ternary. DEFINITION Let S={s1,…,sq} be set, which we call a source alphabet. Let C be a code. An encoding function is a bijective function f: SC, from S onto C. The pair (C,f) is said to be an encoding scheme for S.

Theory of Information Lecture 4 An Example of a Code Theory of Information Lecture 4 Source alphabet: S={_,a,b,…,z} Code alphabet: A={0,1,…,9} Code: C={00,01,…,26} Encoding function f: f(_)=00, f(a)=01, f(b)=02, f(c)=03, f(d)=04, f(e)=05, f(f)=06, f(g)=07, f(h)=08, f(i)=09, f(j)=10, f(k)=11, f(l)=12, f(m)=13, f(n)=14, f(o)=15, f(p)=16, f(q)=17, f(r)=18, f(s)=19, f(t)=20, f(u)=21, f(v)=22, f(w)=23, f(x)=24, f(y)=25, f(z)=26. Encode the message “I am here”: Read 1000 1210 1105 0020 0810 1900 0315 2118 1905

Theory of Information Lecture 4 An Example of a Code Theory of Information Lecture 4 Why did not we choose the following encoding function f: f(_)=0, f(a)=1, f(b)=2, f(c)=3, f(d)=4, f(e)=5, f(f)=6, f(g)=7, f(h)=8, f(i)=9, f(j)=10, f(k)=11, f(l)=12, f(m)=13, f(n)=14, f(o)=15, f(p)=16, f(q)=17, f(r)=18, f(s)=19, f(t)=20, f(u)=21, f(v)=22, f(w)=23, f(x)=24, f(y)=25, f(z)=26. The above code is variable length. The code from the previous slide is a fixed length code, or a block code. Its length is 2. ASCII (American Standard Code for Information Exchange) is a block code of length 8.

Variable versus Fixed Length codes Theory of Information Lecture 4 The code of slide 4 is a variable length code: not every codeword has the same length. The code of slide 3 is a fixed length code, or a block code. Its length is 2. ASCII (American Standard Code for Information Exchange) is a block code of length 8. Main advantage of fixed length codes: Main advantage of variable length codes:

Theory of Information Lecture 4 Homework Theory of Information Lecture 4 Exercises 1,2,3,4,5,6,7 of Section 1.2.