1. Lecture 4 What are Codes? (Section 1.2)
Theory of Information Lecture 4
Theory of Information
Lecture 4 What are Codes? (Section 1.2)

2. Theory of Information Lecture 4
Definition of a Code
Theory of Information Lecture 4
DEFINITION Let A={a1,…,ar} be set, which we call a code alphabet.
An r-ary code over A is a set CA*. The elements of C are called
codewords. The number r is called the radix of the code.
2-ary codes are said to be binary, 3-ary codes are said to be ternary.
DEFINITION Let S={s1,…,sq} be set, which we call a source alphabet.
Let C be a code. An encoding function is a bijective function f: SC,
from S onto C. The pair (C,f) is said to be an encoding scheme for S.

3. Theory of Information Lecture 4
An Example of a Code
Theory of Information Lecture 4
Source alphabet: S={_,a,b,…,z}
Code alphabet: A={0,1,…,9}
Code: C={00,01,…,26}
Encoding function f:
f(_)=00, f(a)=01, f(b)=02, f(c)=03, f(d)=04, f(e)=05, f(f)=06,
f(g)=07, f(h)=08, f(i)=09, f(j)=10, f(k)=11, f(l)=12, f(m)=13,
f(n)=14, f(o)=15, f(p)=16, f(q)=17, f(r)=18, f(s)=19, f(t)=20,
f(u)=21, f(v)=22, f(w)=23, f(x)=24, f(y)=25, f(z)=26.
Encode the message “I am here”:
Read

4. Theory of Information Lecture 4
An Example of a Code
Theory of Information Lecture 4
Why did not we choose the following encoding function f:
f(_)=0, f(a)=1, f(b)=2, f(c)=3, f(d)=4, f(e)=5, f(f)=6,
f(g)=7, f(h)=8, f(i)=9, f(j)=10, f(k)=11, f(l)=12, f(m)=13,
f(n)=14, f(o)=15, f(p)=16, f(q)=17, f(r)=18, f(s)=19, f(t)=20,
f(u)=21, f(v)=22, f(w)=23, f(x)=24, f(y)=25, f(z)=26.
The above code is variable length.
The code from the previous slide is a fixed length code, or a block code.
Its length is 2.
ASCII (American Standard Code for Information Exchange) is a block
code of length 8.

5. Variable versus Fixed Length codes
Theory of Information Lecture 4
The code of slide 4 is a variable length code: not every codeword has the
same length.
The code of slide 3 is a fixed length code, or a block code. Its length is 2.
ASCII (American Standard Code for Information Exchange) is a
block code of length 8.
Main advantage of fixed length codes:
Main advantage of variable length codes:

6. Theory of Information Lecture 4
Homework
Theory of Information Lecture 4
Exercises 1,2,3,4,5,6,7 of Section 1.2.