King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 1 COE 308 Memory Hierarchy

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 2 COE 308 Memory Storage Area for Program and Data Programmers Always Want Unlimited Amount of Fast Memory Fast Memory is Expensive Solution: Create the Illusion of Unlimited Fast Memory

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 3 COE 308 Illustrative Example In the Library (Engineering or Math) Select a Collection of Books on the Subject Establish a List of Machines to Cover If a Machine is Not Covered Go and Look for the Book Student Writing Paper about Important Historical Development of Computer Hardware

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 4 COE 308 Information Access Being in the Library and not Elsewhere Having All the Selected Book on the Table and not Just One Book Saves Time of Constantly Going Back to Shelves to Look for Books Good Selection of Books means: –Good Probability that Many of the Needed Topics are Found in the Books –No Need to Bring All the Library Books on the Table Impossible: Table too Small for That Virtually All Library is on the Table because that’s All what is needed from the Whole Library

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 5 COE 308 Principle Of Locality Programs Access a Relatively Small Portion of Their Address Space at any instant of time (Student Accesses a Very Small Portion Of Library) Temporal Locality (Locality in Time): An item that is referenced now tends to be referenced again soon. Spatial Locality (Locality in Space): A referenced item close neighbors tend to be also referenced soon.

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 6 COE 308 Locality in Programs an Data Locality in Programs Arises from Simple and Natural Program Structures In Programs –Instructions are accessed sequentially –Loops are another reason why Instructions and Data are going to be accessed repeatedly In Data –Accesses to Elements of Array or Record –Variable Definitions are localized by the programmer in the source code and so are they in memory

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 7 COE 308 Memory Technology Typical Access TimePrice per Mbyte in 2005 SRAM DRAM Magnetic Disk 1-10 ns$70-$ ns$0.20-$ ms$0.001-$0.002 Faster = Smaller Capacity = Higher Cost Slower = Bigger Capacity = Lower Cost

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 8 COE 308 Memory Hierarchy Faster Memory Close to the Processor Slower Memory below faster in Hierarchy Because of Differences in Cost and Speed, It is Advantageous to build the Memory as a Hierarchy CPU Memory SpeedSizeCost($/bit) Fastest Slowest Smallest Biggest Highest Lowest

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 9 COE 308 Memory Hierarchy and Data May Consist of Multiple Levels Data copied at all levels Data update is copied between two adjacent levels at a time. Data found in upper level: hit Data not found in upper level: miss hit rate is the fraction of memory accesses that produced a hit miss rate is the fraction of memory accesses which outcome was a miss hit = (1- miss)

King Fahd University of Petroleum and Minerals King Fahd University of Petroleum and Minerals Computer Engineering Department Computer Engineering Department College of Computer Science And Engineering College of Computer Science And Engineering Memory Hierarchy 10 COE 308 Hits and Misses Data found in upper level: hit Data not found in upper level: miss hit rate is the fraction of memory accesses that produced a hit miss rate is the fraction of memory accesses which outcome was a miss hit = (1- miss) hit time: time needed to access upper level of memory miss penalty: time needed to replace a block in the upper level with the corresponding block from the lower level