1. February 1, 2011 1

2. 2 http://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpg

3. How many bit strings of length n? Adding one bit doubles the number February 1, 20113 lengthBit strings of that lengthCount 10, 1 2 = 2 1 200, 01 10, 11 4 = 2 2 3000, 001, 010, 011, 100, 101, 110, 111 8 = 2 3 n0 followed by strings of length n-1, 1 followed by strings of length n-1 2 n-1 + 2 n-1 = 2 x 2 n-1 =2 n

4. Text 8 bits per character “A” = 01000001 “(” = 00101000 How many combinations of 8 bits? 2· 2· 2· 2· 2· 2· 2· 2 = 2 8 = 256 February 1, 20114

5. Hexadecimal Digits February 1, 20115 00000001001000110100010101100111 01234567 10001001101010111100110111101111 89ABCDEF

6. ASCII American Standard Code for Information Interchange Character represented by Hex xy, e.g. 4B is “K” February 1, 20116 xyxy 0123456789ABCDEF 0 1 2 sp !"#$%&'()*+,-./ 30123456789:;<=>? 4@ABCDEFGHIJKLMNO 5PQRSTUVWXYZ[\]^_ 6`abcdefghijklmno 7pqrstuvwxyz{|}~ del

7. ASCII Underneath Emails Web pages February 1, 20117

8. 8 http://farm4.static.flickr.com/3021/2494096946_2bf86f8571.jpg?v=0

9. What if you need more than 256 characters? Unicode 32 bits per character February 1, 20119

10. 10

11. February 1, 201111

12. February 1, 201112

13. February 1, 201113

14. How many Unicode characters? 32 bits each, so there are in all February 1, 201114  2 32  4,294,967,296

15. February 1, 201115

16. Binary counting 1+1=10, or 1+1=0 and carry 1 February 1, 201116 +1 01100111 +1 01101000

17. Positive and Negative Numbers Signed and unsigned numbers Unsigned: 2 8 =256 bit patterns represent 0 … 255 Signed: 2 8 bit patterns represent -128 … +127 Leftmost bit = sign bit: 0 => 0 or pos, 1 => neg Largest 8-bit positive number = 01111111 = 127 0 = 00000000 Most negative negative number = 10000000 = -128 February 1, 201117

18. Negative numbers February 1, 201118 +1 11111111 +1 00000000 -1 = 11111111 so addition works the same for positive and negative numbers

19. Biggest Numbers Biggest positive number = 01111111 (like 999999 on a car odometer) Most negative negative number = 10000000 February 1, 201119

20. Biggest Positive Number + 1 “=” Most Negative Negative Number February 1, 201120 +1 01111111 +1 10000000 OVERFLOW!

21. The Comair Christmas “Glitch” 16 bits for monthly count of crew changes Biggest positive 16-bit number =32,767 December was a bad month, lots of snowstorms, lots of flights rescheduled As Christmas approached the count went from 32,767 to - 32,768 by adding 1 February 1, 201121

22. The Y2010 “Glitch” Binary representation of decimal 10 = 00001010 Binary Coded Decimal = write decimal 10 as sequence of 4- bit binary codes for digits Decimal 10 = BCD 0001 0000 What if you write decimal 10 in BCD but some other program reads it as decimal? Binary 0001 0000 = decimal 16 February 1, 201122

23. Bytes 1 byte = 8 bits = 2 hex digits = 1 character 2 10 =1024 bytes = 1 kilobyte = 1KB 2 20 =1,048,576 bytes = 1 megabyte = 1MB 2 30 bytes = 1 gigabyte = 1GB = “a billion” 2 40 bytes = 1 terabyte = 1TB = “a trillion” 2 50 bytes = 1 petabyte = 1PB = “a quadrillion” 2 60 bytes = 1 exabyte = 1EB = a quintillion bytes 2 70 = zetta 2 80 = yotta February 1, 201123

24. K All this terminology based on the accident that Which is 1K? There are new standard names: 1 kibibyte = 1000 bytes vs. 1 kilobyte = 1024 bytes But almost no one uses “kibi-”, “mebi-”, etc. February 1, 201124  1024  2 10  3  1000

25. February 1, 201125

26. February 1, 201126

27. Moore’s Law (1965) The number of transistors on a silicon chip doubles every 18 [or 12, or 24] months 1965: 64 = 2 6 2008 = 2 billion ~ 2 31 25 doublings in 43 years = one doubling every 20+ months February 1, 201127

28. Example of linear increase February 1, 201128

29. Example of exponential increase Now for the y axis use instead lg(y) = the exponent e such that 2 e =y February 1, 201129

30. Same plot, using lg(y) instead of y February 1, 201130

31. One of the Greatest Engineering Achievements An increase by a factor of 2 25 is about 30 millionfold If human speed had increased that much over the past 43 years, we would now be traveling faster than the speed of light February 1, 201131

32. Probabilities Fair coin: P(heads) = 1/2 Fair die: P(rolling 3) = 1/6 Fair card deck: P(hearts) = 1/4 P(ace) = 1/13 February 1, 2011 32

33. Probabilities of Independent Events Multiply P(heads and then heads) = 1/2 · 1/2 = 1/4 P(3 and then 4) = 1/6 · 1/6 = 1/36 P(ace and ace) = 1/13·1/13 = 1/169 ≈.0059 but only if the first card drawn is replaced and the deck is completely reshuffled, otherwise the events are not independent P(ace and ace without reshuffling) = 1/13 · 3/51 ≈.0045 February 1, 2011 33

34. Unlikely Events How likely are 100 heads in a row? (1/2) 100 ≈ 10 -32 =.00000000000000000000000000000001 February 1, 2011 34

35. How Small is 2 -100 ≈ 10 -30 ? Age of universe ≈ 10 18 sec = 10 27 nanoseconds (1 nanosecond = 1 ns = 1 billionth of a second = 10 -9 sec) If you do all 100 coin flips in a billionth of a second, you will get the 100-heads event about once every thousand lifetimes of the universe 10 30 = 10 3 ·10 27 This is “never” for all practical purposes February 1, 2011 35

36. Morse’s telegraph 1844 1848 February 1, 2011 36

37. Morse Code (1838) ABCDEFGHIJKLM.--…-.-.-.....-.--.…....----.-.-..-- NOPQRSTUVWXYZ -.---.--.--.-.-.…-..-…-.---..--.----.. February 1, 2011 37

38. Morse Code (1838) A.08 B.01 C.03 D.04 E.12 F.02 G.02 H.06 I.07 J.00 K.01 L.04 M.02.--…-.-.-.....-.--.…....----.-.-..-- N.07 O.08 P.02 Q.00 R.06 S.06 T.09 U.03 V.01 W.02 X.00 Y.02 Z.00 -.---.--.--.-.-.…-..-…-.---..--.----.. February 1, 2011 38

39. How Long are Morse Codes on Average? Not the average of the lengths of the letters: (2+4+4+3+…)/26 = 82/26 ≈ 3.2 We want the average a to be such that in a typical real sequence of say 1,000,000 letters, the number of dots and dashes should be about a·1,000,000 The weighted average: (freq of A)·(length of code for A) + (freq of B)·(length of code for B) + … =.08·2 +.01·4 +.03·4 +.04·3+… ≈ 2.4 February 1, 2011 39

40. Data vs. Information Message sequence: “yea,” “nay,” “yea,” “yea,” “nay,” “nay” … In ASCII, 3·8 = 24 bits of data per message But if the only possible answers are “yea” and “nay,” there is only 1 bit of information per message Entropy is a measure of the information content of a message, as opposed to its size Entropy of this message sequence = 1 bit/msg February 1, 2011 40

41. Squeezing out the “Air” Suppose you want to ship pillows in boxes and are charged by the size of the box Lossless data compression Entropy = lower limit of compressibility February 1, 2011 41

42. Claude Shannon (1916-2001) A Mathematical Theory of Communication (1948) February 1, 2011 42

43. Communication over a Channel February 1, 201143 Source Coded Bits Received Bits Decoded Message S X Y T Channel symbols bits bits symbols Encode bits before putting them in the channel Decode bits when they come out of the channel E.g. the transformation from S into X changes “yea” --> 1 “nay” --> 0 Changing Y into T does the reverse For now, assume no noise in the channel, i.e. X=Y