This file contains figures from the book: Information Theory A Tutorial Introduction by Dr James V Stone 2015 Sebtel Press. Copyright JV Stone. These figures are released for use under the Creative Commons License specified on next slide. A copy of this file can be obtained from

Chapter 1

A B D = = = = = = = = 7 C Left Right

0 1 1 = = = = = = = = 7 Fish Bird Dog Car Van Truck Bus No Yes Q1 Q2 Q3 Cat

Chapter 2

Random variable X Experiment Coin flip Outcome = head Mapping X(head) = 1 Random variable X Experiment Coin flip Outcome = tail Mapping X(tail) = 0

Output y Noise  Channel Encoder x = g(s) Data s Decoder Input x

Chapter 3

Noise Channel Encoder Data Decoder Input Output X

A0.87 B0.04 C0.04 D0.03 E

from xkcd.com

From xkcd.com

Chapter 4

Output y Noise  Channel Encoder x = g(s) Data s Decoder Input x

a b c

H(Y) H(X) I(X,Y) H(Y|X) H(X|Y) H(X,Y)

H(Y) H(X) I(X,Y) H(Y|X) H(X|Y) H(X,Y)

2 H(Y|X) 2 H(X|Y) Number of outputs = 2 H(Y) Number of inputs = 2 H(X) Number of possible outputs for one input = 2 H(Y|X) Number of possible inputs for one output = 2 H(X|Y) Inputs X Outputs Y

Output entropy H(Y) Mutual information I(X,Y) Output noise entropy H(Y|X) Input entropy H(X) Mutual information I(X,Y) Input noise entropy H(X|Y) (a) (b)

ABCDEFGHIABCDEFGHI B E H

Chapter 5

Residual uncertainty Residual uncertainty Residual uncertainty b) After receiving 1 bit, U=50% a) After receiving 2 bits, U=25% c) After receiving 1/2 a bit, U=71%

Noise Channel Encoder Data Decoder Input Output

Chapter 6

Output y Noise  Channel Encoder x = g(s) Data s Decoder Input x

H(Y) H(X) I(X,Y) H(Y|X) H(X|Y) H(X,Y)

Chapter 7

Chapter 8

Warm AB

ColdHot AB

Chapter 9

a b c d e

82

83 Blue-Yellow channel Luminance channel Red-Green channel S-cone M-cone L-cone

THE END.