1. 1 Information Theory Nathanael Paul Oct. 09, 2002

2. 2 Claude Shannon: Father of Information Theory “Communication Theory of Secrecy Systems” (1949) Cryptography becomes science Why is information theory so important in cryptography?

3. 3 Some Terms (P,C,K,E,D) Computational Security Computational effort required to break cryptosystem Provable Security Relative to another, difficult problem Unconditional Security Oscar (adversary) can do whatever he wants, as much as he wants

4. 4 Applying probability to cryptography Each message p in P has a probability as well as each k in K has a probability Given a p in P and a k in K, a y in C is uniquely determined. Given a k in K and a y in C, an x in X is uniquely determined. Induce a probability on ciphertext space For the equation below, y is fixed.

5. 5 Some probability theory… Probability distribution on X Joint probability Conditional probability Bayes’ Theorem

6. 6 Probability Distribution of X p(x) – probability function of X X takes on a finite # (or countably infinite) of possible values – x Ex. x is a letter in substitution cipher, where X is plaintext space P(X=x) = p(x) >= 0 this sum is over all possible values of x

7. 7 Joint Probability Let X 1 and X 2 denote random variables p(x 1,x 2 ) = P(X 1 = x 1, X 2 = x 2 ) “The probability that X 1 will take on the value x 1 and X 2 will take on the value x 2 ” If X 1 and X 2 are independent, then p(x 1,x 2 ) = p(x 1 ) * p(x 2 )

8. 8 Conditional Probability “What is the probability of x given y?” p(x|y) = p(x,y)/p(y) If p(X = x|Y = y) = p(X = x), then X and Y are independent.

9. 9 Bayes’ Theorem  p(x,y)= p(x) * p(y | x) = p(y) * p(x | y)

10. 10 Perfect Secrecy Defined A cryptosystem (P,C,K,E,D) has perfect secrecy if “ciphertext yields no information about plaintext”

11. 11 Perfect Secrecy Defined Suppose a cryptosystem (P,C,K,E,D) has |K| = |C| = |P|. This cryptosystem has P.S. iff the following hold: -Each key chosen is truely random -For each x in P, y in C,  a unique key k  e k (x) = y.

12. 12 Perfect Secrecy (P.S.) implies |P| <= |K| and |C| <= |K| Claim: Perfect Secrecy (P.S.) implies |P| <= |K| and |C| <= |K| p P (x | y) = p P (x) > 0, where y is fixed. E k (x) = y, for a k in K (k is random) For each x  a k in K  E k (x) = y, since probability p P (x) > 0.

13. 13 Conclusion about Perfect Secrecy “Key size should be at least as large as message size, and key size should be at least as large as ciphertext size.”

14. 14 Perfect Secrecy Example P = C = K = Z 26 = {0,1,2,...,24,25} E k (x) = x + k mod 26 D k (x) = x – k mod 26 p(k) = 1/26 and p(x) = any distribution given note: key must be truely random

15. 15 Entropy Want to be able to measure the “uncertainty” or “information” of some random variable X. Entropy a measure of information “How much information or uncertainty is in a cryptosystem?”

16. 16 Entropy (cont.) Given: X, a random variable finite set of values of X: p 1,..., p n Entropy is:

17. 17 Entropy examples X:X 1,X 2 P:1,0 Entropy = 0, since there is no choice. X 1 will happen 100% of the time. H(X) = 0. X:X 1,X 2 X 1 is more likely than P:¾,¼X 2. H(X) = - (¾ log 2 (¾) + ¼ log 2 (¼))

18. 18 Entropy examples (cont.) X:X 1,X 2½ H(x) = - (½ log 2 (½) + ½ log 2 (½)) = 1 X:X 1,X 2,..., X n P:1/n,1/n,..., 1/n H(x) = - (1/n log 2 (1/n) * n) = log 2 (n)

19. 19 Entropy examples (cont.) If X is a random variable with n possible values: H(X) <= log 2 (n), with equality iff each value has equal probability (i.e. 1/n) By Jensen’s Inequality, log 2 (n) provides an upper bound on H(x) If x is the months of the year: H(x) = log 2 12  3.6 (about 4 bits needed to encode the year)

20. 20 Unicity Distance Assume in a given cryptosystem a msg is a string: x 1,x 2,...,x n where x i is in P (x i is a letter or block) Encrypting each x i individually with the same key k, y i = E k (x i ), 1 <= i <= n How many ciphertext blocks, y i ’s, do we need to determine k?

21. 21 Unicity Distance (cont.) Ciphertext only attack with infinite computing power Unicity Distance Smallest # n, for which n ciphertexts (on average) uniquely determine key One-time pad (infinite)

22. 22 Defining a language L: the set of all msgs, for n >= 1. “the natural language” p 2 : (x 1,x 2 ) : x 1, x 2 in P p n : (x 1,x 2,...,x n ), x i in P, so p n  L each p i inherits a probability distribution from L (digrams, trigrams,...) H(p i ) makes sense

23. 23 Entropy and Redundancy of a language What is the entropy of a language? What is the redundancy of a language?

24. 24 Application of Entropy and Redundancy 1 <= H L <= 1.5 in english H(P) = 4.18 H(P 2 ) = 3.90 R L = 1 – H L /log 2 26 about 70%, depends on H L

25. 25 Unicity in substitution cipher n o = log 2 |K|/(R L *log 2 |P|) |P| = 26 |K| = 26! (all permutations) n o = log 2 26!/(0.70 * log 2 26) which is about 26.8 Which means… on average, if one has 27 letters of ciphertext from a substitution cipher, then you should have enough information to determine the key!

26. 26 Ending notes... key equivocation “How much information is revealed by the ciphertext about the key?” H(K|C) = H(K) + H(P) – H(C) Spurious keys incorrect but possible So reconsider our question: “Why can’t cryptography and math be separated?”