1. L01 - Introduction 1 Comp411 – Fall 2006 8/23/2006 Welcome to Comp 411! I thought this course was called “Computer Organization” David Macaulay 1) Course Mechanics 2) Course Objectives 3) Information (A course previously known as Comp 120)

2. L01 - Introduction 2 Comp411 – Fall 2006 8/23/2006 Meet the Crew… Lectures:Leonard McMillan (SN-258) Office Hours M 2-3 TA:TBA (Find out next Monday) Book:Patterson & Hennessy Computer Organization & Design 3 rd Edition, ISBN: 1-55860-604-1 (However, you won’t need it for the next couple of weeks)

3. L01 - Introduction 3 Comp411 – Fall 2006 8/23/2006 Problem sets will be distributed on Wednesdays and are due back on the next Wednesday *lecture* (before it begins). Usually you will have one week to complete each set. Late problem sets will not be accepted, but the lowest two problem-set scores will be dropped. I will attempt to make Lecture Notes, Problem Sets, and other course materials available on the web Before Class on the day they are given. Course Mechanics Grading: Best 8 of 10 problem sets 40% 2 Quizzes30% Final Exam30%

4. L01 - Introduction 4 Comp411 – Fall 2006 8/23/2006 Comp 411: Course Clickables Announcements, corrections, etc. On-line copies of all handouts http://www.unc.edu/courses/2006fall/comp/411/001

5. L01 - Introduction 5 Comp411 – Fall 2006 8/23/2006 Goal 1: Demystify Computers Strangely, most people (even some computer scientists) are afraid of computers. We are only afraid of things we do not understand! I do not fear computers. I fear the lack of them. - Isaac Asimov (1920 - 1992) Fear is the main source of superstition, and one of the main sources of cruelty. To conquer fear is the beginning of wisdom. - Bertrand Russell (1872 – 1970)

6. L01 - Introduction 6 Comp411 – Fall 2006 8/23/2006 Goal 2: Power of Abstraction Define a function, develop a robust implementation, and then put a box around it. Abstraction enables us to create unfathomable machines called computers. Why do we need ABSTRACTION… Imagine a billion --- 1,000,000,000

7. L01 - Introduction 7 Comp411 – Fall 2006 8/23/2006 The key to building systems with >1G components Personal Computer: Hardware & Software Circuit Board:  8 / system 1-2G devices Integrated Circuit:  8-16 / PCB.25M-16M devices Module:  8-16 / IC 100K devices Cell:  1K-10K / Module 16-64 devices Gate:  2-16 / Cell 8 devices Scheme for representing information MOSFET

8. L01 - Introduction 8 Comp411 – Fall 2006 8/23/2006 What do we See in a Computer? Structure –hierarchical design: –limited complexity at each level –reusable building blocks What makes a good system design? –“Bang for the buck”: minimal mechanism, maximal function –reliable in a wide range of environments –accommodates future technical improvements Interfaces –Key elements of system engineering; typically outlive the technologies they interface –Isolate technologies, allow evolution –Major abstraction mechanism Wait! I think I see a bug in the DIV logic. Got that one off the web. Sure hope it works.

9. L01 - Introduction 9 Comp411 – Fall 2006 8/23/2006 Computational Structures What are the fundamental elements of computation? Can we define computation independent of the substrate on which it is are implemented?

10. L01 - Introduction 10 Comp411 – Fall 2006 8/23/2006 Our Plan of Attack…  Understand how things work, by alternating between low-level (bottom-up) and high level (top- down) concepts  Encapsulate our understanding using appropriate abstractions  Study organizational principles: abstractions, interfaces, APIs.  Roll up our sleeves and design at each level of hierarchy  Learn engineering tricks - history - systematic design approaches - diagnose, fix, and avoid bugs

11. L01 - Introduction 11 Comp411 – Fall 2006 8/23/2006 What is “Computation”? Computation is about “processing information” -Transforming information from one form to another - Deriving new information from old - Finding information associated with a given input - “Computation” describes the motion of information through time - “Communication” describes the motion of information through space

12. L01 - Introduction 12 Comp411 – Fall 2006 8/23/2006 What is “Information”? information, n. Knowledge communicated or received concerning a particular fact or circumstance. Information resolves uncertainty. Information is simply that which cannot be predicted. The less predictable a message is, the more information it conveys! Duke won again. Tell me something new… “ 10 Problem sets, 2 quizzes, and a final!” A Computer Scientist’s Definition:

13. L01 - Introduction 13 Comp411 – Fall 2006 8/23/2006 Real-World Information Why do unexpected messages get allocated the biggest headlines? … because they carry the most information.

14. L01 - Introduction 14 Comp411 – Fall 2006 8/23/2006 What Does A Computer Process? A Toaster processes bread and bagels A Blender processes smoothies and margaritas What does a computer process? 2 allowable answers: –Information –Bits What is the mapping from information to bits?

15. L01 - Introduction 15 Comp411 – Fall 2006 8/23/2006 Quantifying Information (Claude Shannon, 1948) Suppose you’re faced with N equally probable choices, and I give you a fact that narrows it down to M choices. Then I’ve given you log 2 (N/M) bits of information Examples:  information in one coin flip: log 2 (2/1) = 1 bit  roll of a single die: log 2 (6/1) = ~2.6 bits  outcome of a Football game: 1 bit (well, actually, “they won” may convey more information than “they lost”…) Information is measured in bits (binary digits) = number of 0/1’s required to encode choice(s)

16. L01 - Introduction 16 Comp411 – Fall 2006 8/23/2006 Example: Sum of 2 dice 2 3 4 5 6 7 8 9 10 11 12 i 2 = log 2 (36/1) = 5.170 bits i 3 = log 2 (36/2) = 4.170 bits i 4 = log 2 (36/3) = 3.585 bits i 5 = log 2 (36/4) = 3.170 bits i 6 = log 2 (36/5) = 2.848 bits i 7 = log 2 (36/6) = 2.585 bits i 8 = log 2 (36/5) = 2.848 bits i 9 = log 2 (36/4) = 3.170 bits i 10 = log 2 (36/3) = 3.585 bits i 11 = log 2 (36/2) = 4.170 bits i 12 = log 2 (36/1) = 5.170 bits The average information provided by the sum of 2 dice: Entropy

17. L01 - Introduction 17 Comp411 – Fall 2006 8/23/2006 Show Me the Bits! Can the sum of two dice REALLY be represented using 3.274 bits? If so, how? The fact is, the average information content is a strict *lower-bound* on how small of a representation that we can achieve. In practice, it is difficult to reach this bound. But, we can come very close.

18. L01 - Introduction 18 Comp411 – Fall 2006 8/23/2006 Variable-Length Encoding Of course we can use differing numbers of “bits” to represent each item of data This is particularly useful if all items are *not* equally likely Equally likely items lead to fixed length encodings: –Ex: Encode a particular roll of 5? –{(1,4), (2,3), (3,2), (4,1)} which are equally likely if we use fair dice –Entropy = bits –00 – (1,4), 01 – (2,3), 10 – (3,2), 11 – (4,1) Back to the original problem. Let’s use this encoding: 2 - 100113 - 01014 - 0115 - 001 6 - 1117 - 1018 - 1109 - 000 10 - 100011 - 010012 - 10010

19. L01 - Introduction 19 Comp411 – Fall 2006 8/23/2006 Variable-Length Encoding Taking a closer look Decoding 2 - 100113 - 01014 - 0115 - 001 6 - 1117 - 1018 - 1109 - 000 10 - 100011 - 010012 - 10010 Unlikely rolls are encoded using more bits More likely rolls use fewer bits 2536583 Example Stream: 1001100101011110011100101

20. L01 - Introduction 20 Comp411 – Fall 2006 8/23/2006 Huffman Coding A simple *greedy* algorithm for approximating an entropy efficient encoding 1.Find the 2 items with the smallest probabilities 2.Join them into a new *meta* item whose probability is the sum 3.Remove the two items and insert the new meta item 4.Repeat from step 1 until there is only one item 36/36 11 2/36 3 2/36 4/36 4 3/36 7/36 9 4/36 5 4/36 8/36 15/36 12 1/36 2 1/36 2/36 7 6/36 11/36 8 5/36 6 5/36 10/36 21/36 10 3/36 5/36 Huffman decoding tree

21. L01 - Introduction 21 Comp411 – Fall 2006 8/23/2006 Converting Tree to Encoding 36/36 4 3/36 11 2/36 3 2/36 4/36 7/36 9 4/36 5 4/36 8/36 15/36 7 6/36 10 3/36 12 1/36 2 1/36 2/36 5/36 11/36 8 5/36 6 5/36 10/36 21/36 0 0 0 0 00 0 0 0 0 1 1 1 1 1 1 1 1 1 1 Huffman decoding tree Once the *tree* is constructed, label its edges consistently and follow the paths from the largest *meta* item to each of the real item to find the encoding. 2 - 100113 - 01014 - 0115 - 001 6 - 1117 - 1018 - 1109 - 000 10 - 100011 - 010012 - 10010

22. L01 - Introduction 22 Comp411 – Fall 2006 8/23/2006 Encoding Efficiency How does this encoding strategy compare to the information content of the roll? Pretty close. Recall that the lower bound was 3.274 bits. However, an efficient encoding (as defined by having an average code size close to the information content) is not always what we want!

23. L01 - Introduction 23 Comp411 – Fall 2006 8/23/2006 Encoding Considerations -Encoding schemes that attempt to match the information content of a data stream remove redundancy. They are data compression techniques. - However, sometimes our goal in encoding information is increase redundancy, rather than remove it. Why? -Make the information easier to manipulate (fixed-sized encodings) -Make the data stream resilient to noise (error detecting and correcting codes) -Data compression allows us to store our entire music collection in a pocketable device -Data redundancy enables us to store that *same* information *reliably* on a hard drive

24. L01 - Introduction 24 Comp411 – Fall 2006 8/23/2006 4 Error detection using parity Sometimes we add extra redundancy so that we can detect errors. For instance, this encoding detects any single-bit error: 2-1111000 3-1111101 4-0011 5-0101 6-0110 7-0000 8-1001 9-1010 10-1100 11-1111110 12-1111011 4 2* 5 4:001111100000101 4 10* 7 3:001111100000101 4 9* 7 2:001111100000101 4 6* 7 1:001111100000101 There’s something peculiar about those codings Same bitstream – w/4 possible interpretations if we allow for only one error ex:001111100000101

25. L01 - Introduction 25 Comp411 – Fall 2006 8/23/2006 Property 1: Parity The sum of the bits in each symbol is even. (this is how errors are detected) 2-1111000 = 1 + 1 + 1+ 1 + 0 + 0 + 0 = 4 3-1111101= 1 + 1 + 1 + 1 + 1 + 0 + 1 = 6 4-0011= 0 + 0 + 1 + 1 = 2 5-0101= 0 + 1 + 0 + 1 = 2 6-0110= 0 + 1 + 1 + 0 = 2 7-0000= 0 + 0 + 0 + 0 = 0 8-1001= 1 + 0 + 0 + 1 = 2 9-1010= 1 + 0 + 1 + 0 = 2 10-1100= 1 + 1 + 0 + 0 = 2 11-1111110= 1 + 1 + 1 + 1 + 1 + 1 + 0 = 6 12-1111011= 1 + 1 + 1 + 1 + 0 + 1 + 1 = 6 How much information is in the last bit?

26. L01 - Introduction 26 Comp411 – Fall 2006 8/23/2006 Property 2: Separation Each encoding differs from all others by at least two bits in their overlapping parts This difference is called the “Hamming distance” “A Hamming distance of 1 is needed to uniquely identify an encoding”

27. L01 - Introduction 27 Comp411 – Fall 2006 8/23/2006 A Short Detour It is illuminating to consider Hamming distances geometrically. Given 2 bits the largest Hamming distance that we can achieve between 2 encodings is 2. This allows us to detect 1-bit errors if we encode 1 bit of information using 2 bits. With 3 bits we can find 4 encodings with a Hamming distance of 2, allowing the detection of 1-bit errors when 3 bits are used to encode 2 bits of information. We can also identify 2 encodings with Hamming distance 3. This extra distance allows us to detect 2-bit errors. However, we could use this extra separation differently. 00 10 01 11 000001 010 100 011 111 101 110 Encodings separated by a Hamming distance of 2. 000001 010 100 011 111 101 110 Encodings separated by a Hamming distance of 3.

28. L01 - Introduction 28 Comp411 – Fall 2006 8/23/2006 Error correcting codes We can actually correct 1-bit errors in encodings separated by a Hamming distance of 2. This is possible because the sets of bit patterns located a Hamming distance of 1 from our encodings are distinct. However, attempting error correction with such a small separation is dangerous. Suppose, we have a 2-bit error. Our error correction scheme will then misinterpret the encoding. Misinterpretations also occurred when we had 2-bit errors in our 1-bit-error-detection (parity) schemes. A safe 1-bit error correction scheme would correct all 1-bit errors and detect all 2-bit errors. What Hamming distance is needed between encodings to accomplish this? 000001 010 100 011 111 101 110 This is just a voting scheme

29. L01 - Introduction 29 Comp411 – Fall 2006 8/23/2006 0000000100110010 0100010101110110 1100110111111110 1000100110111010 0000000100110010 0100010101110110 1100110111111110 1000100110111010 0000000100110010 0100010101110110 1100110111111110 1000100110111010 0000000100110010 0100010101110110 1100110111111110 1000100110111010 0000000100110010 0100010101110110 1100110111111110 1000100110111010 Quick look at 4D Using 4 bits we can separate 8 encodings with a Hamming distance of 1 to allow for 1-bit error detection, thus, encoding 3 bits of information with 4 bits. Bit patterns indicating errors differ from valid encodings by their parity (i.e. whether the number of 1’s used in the pattern is even or odd). Unfortunately, 4 bits are no better than 3 bits in terms of the number of encodings that can be separated with a Hamming distance of 3. The most encodings that we can fit is 2. We can, however, find 2 encodings with a Hamming distance of 4. These encodings allow 1-bit errors to be corrected, and 2-bit errors to be detected. 0000000100110010 0100010101110110 1100110111111110 1000100110111010 This is a flattened hypercube. Notice how the top 2 rows resemble a 3-bit 3D cube (we’ll see this again soon).

30. L01 - Introduction 30 Comp411 – Fall 2006 8/23/2006 (He wouldn’t try 5D!) It takes 5 bits before we can find more that 2 encoding separated by a Hamming distance of 3. Shown on the right are four 5-bit encodings {00000, 01111, 10101, 11010} separated by a Hamming distance of at least 3. With it we can correct any 1-bit error,and we can detect some (but not all) 2-bit errors. We’ll stop here, because we really need a computer to go much beyond 5D, and we’ll need to build one first! 10000100011001110010 10100101011011110110 11100011011111111110 11000110011101111010 00000000010001100010 00100001010011100110 01100011010111101110 01000010010101101010

31. L01 - Introduction 31 Comp411 – Fall 2006 8/23/2006 An alternate error correcting code We can generalize the notion of parity in order to construct error correcting codes. Instead of computing a single parity bit for an entire encoding we can allow multiple parity bits over different subsets of bits. Consider the following technique for encoding 25 bits. This approach is easy to implement, but it is not optimal in terms of the number of bits used. 01111 11100 01010 10100 01000 00101 0 1 0 0 1 0 01111 11100 01010 10100 01000 00101 0 1 0 0 1 0 1-bit errors will cause both a row and column parity error, uniquely identifying the errant bit for correction. An extra parity bit allows us to detect 1-bit errors in the parity bits. Many 2-bit errors can also be corrected. However, 2-bit errors involving the same row or column are undetectable. 10 11 01 01 11

32. L01 - Introduction 32 Comp411 – Fall 2006 8/23/2006 Summary Information resolves uncertainty Choices equally probable: N choices down to M  log 2 (N/M) bits of information Choices not equally probable: choice i with probability p i  log 2 (1/p i ) bits of information average number of bits =  p i log 2 (1/p i ) use variable-length encodings Next time: Technology and Abstraction What makes a computer tick…