February 1,

2

How many bit strings of length n? Adding one bit doubles the number February 1, lengthBit strings of that lengthCount 10, 1 2 = , 01 10, 11 4 = , 001, 010, 011, 100, 101, 110, = 2 3 n0 followed by strings of length n-1, 1 followed by strings of length n-1 2 n n-1 = 2 x 2 n-1 =2 n

Text 8 bits per character “A” = “(” = How many combinations of 8 bits? 2· 2· 2· 2· 2· 2· 2· 2 = 2 8 = 256 February 1, 20114

Hexadecimal Digits February 1, ABCDEF

ASCII American Standard Code for Information Interchange Character represented by Hex xy, e.g. 4B is “K” February 1, xyxy ABCDEF sp !"#$%&'()*+,-./ :;<=>? 5PQRSTUVWXYZ[\]^_ 6`abcdefghijklmno 7pqrstuvwxyz{|}~ del

ASCII Underneath s Web pages February 1, 20117

8

What if you need more than 256 characters? Unicode 32 bits per character February 1, 20119

10

February 1,

February 1,

February 1,

How many Unicode characters? 32 bits each, so there are in all February 1,  2 32  4,294,967,296

February 1,

Binary counting 1+1=10, or 1+1=0 and carry 1 February 1,

Positive and Negative Numbers Signed and unsigned numbers Unsigned: 2 8 =256 bit patterns represent 0 … 255 Signed: 2 8 bit patterns represent -128 … +127 Leftmost bit = sign bit: 0 => 0 or pos, 1 => neg Largest 8-bit positive number = = = Most negative negative number = = -128 February 1,

Negative numbers February 1, = so addition works the same for positive and negative numbers

Biggest Numbers Biggest positive number = (like on a car odometer) Most negative negative number = February 1,

Biggest Positive Number + 1 “=” Most Negative Negative Number February 1, OVERFLOW!

The Comair Christmas “Glitch” 16 bits for monthly count of crew changes Biggest positive 16-bit number =32,767 December was a bad month, lots of snowstorms, lots of flights rescheduled As Christmas approached the count went from 32,767 to - 32,768 by adding 1 February 1,

The Y2010 “Glitch” Binary representation of decimal 10 = Binary Coded Decimal = write decimal 10 as sequence of 4- bit binary codes for digits Decimal 10 = BCD What if you write decimal 10 in BCD but some other program reads it as decimal? Binary = decimal 16 February 1,

Bytes 1 byte = 8 bits = 2 hex digits = 1 character 2 10 =1024 bytes = 1 kilobyte = 1KB 2 20 =1,048,576 bytes = 1 megabyte = 1MB 2 30 bytes = 1 gigabyte = 1GB = “a billion” 2 40 bytes = 1 terabyte = 1TB = “a trillion” 2 50 bytes = 1 petabyte = 1PB = “a quadrillion” 2 60 bytes = 1 exabyte = 1EB = a quintillion bytes 2 70 = zetta 2 80 = yotta February 1,

K All this terminology based on the accident that Which is 1K? There are new standard names: 1 kibibyte = 1000 bytes vs. 1 kilobyte = 1024 bytes But almost no one uses “kibi-”, “mebi-”, etc. February 1,  1024  2 10  3  1000

February 1,

February 1,

Moore’s Law (1965) The number of transistors on a silicon chip doubles every 18 [or 12, or 24] months 1965: 64 = = 2 billion ~ doublings in 43 years = one doubling every 20+ months February 1,

Example of linear increase February 1,

Example of exponential increase Now for the y axis use instead lg(y) = the exponent e such that 2 e =y February 1,

Same plot, using lg(y) instead of y February 1,

One of the Greatest Engineering Achievements An increase by a factor of 2 25 is about 30 millionfold If human speed had increased that much over the past 43 years, we would now be traveling faster than the speed of light February 1,

Probabilities Fair coin: P(heads) = 1/2 Fair die: P(rolling 3) = 1/6 Fair card deck: P(hearts) = 1/4 P(ace) = 1/13 February 1,

Probabilities of Independent Events Multiply P(heads and then heads) = 1/2 · 1/2 = 1/4 P(3 and then 4) = 1/6 · 1/6 = 1/36 P(ace and ace) = 1/13·1/13 = 1/169 ≈.0059 but only if the first card drawn is replaced and the deck is completely reshuffled, otherwise the events are not independent P(ace and ace without reshuffling) = 1/13 · 3/51 ≈.0045 February 1,

Unlikely Events How likely are 100 heads in a row? (1/2) 100 ≈ = February 1,

How Small is ≈ ? Age of universe ≈ sec = nanoseconds (1 nanosecond = 1 ns = 1 billionth of a second = sec) If you do all 100 coin flips in a billionth of a second, you will get the 100-heads event about once every thousand lifetimes of the universe = 10 3 ·10 27 This is “never” for all practical purposes February 1,

Morse’s telegraph February 1,

Morse Code (1838) ABCDEFGHIJKLM.--… … NOPQRSTUVWXYZ …-..-… February 1,

Morse Code (1838) A.08 B.01 C.03 D.04 E.12 F.02 G.02 H.06 I.07 J.00 K.01 L.04 M.02.--… … N.07 O.08 P.02 Q.00 R.06 S.06 T.09 U.03 V.01 W.02 X.00 Y.02 Z …-..-… February 1,

How Long are Morse Codes on Average? Not the average of the lengths of the letters: ( …)/26 = 82/26 ≈ 3.2 We want the average a to be such that in a typical real sequence of say 1,000,000 letters, the number of dots and dashes should be about a·1,000,000 The weighted average: (freq of A)·(length of code for A) + (freq of B)·(length of code for B) + … =.08·2 +.01·4 +.03·4 +.04·3+… ≈ 2.4 February 1,

Data vs. Information Message sequence: “yea,” “nay,” “yea,” “yea,” “nay,” “nay” … In ASCII, 3·8 = 24 bits of data per message But if the only possible answers are “yea” and “nay,” there is only 1 bit of information per message Entropy is a measure of the information content of a message, as opposed to its size Entropy of this message sequence = 1 bit/msg February 1,

Squeezing out the “Air” Suppose you want to ship pillows in boxes and are charged by the size of the box Lossless data compression Entropy = lower limit of compressibility February 1,

Claude Shannon ( ) A Mathematical Theory of Communication (1948) February 1,

Communication over a Channel February 1, Source Coded Bits Received Bits Decoded Message S X Y T Channel symbols bits bits symbols Encode bits before putting them in the channel Decode bits when they come out of the channel E.g. the transformation from S into X changes “yea” --> 1 “nay” --> 0 Changing Y into T does the reverse For now, assume no noise in the channel, i.e. X=Y