1. CPE432 Chapter 5A.1Dr. W. Abu-Sufah, UJ Chapter 5A: Exploiting the Memory Hierarchy, Part 1 Adapted from Slides by Prof. Mary Jane Irwin, Penn State University And Slides Supplied by the textbook publisher Read Section 5.1: Introduction

2. CPE432 Chapter 5A.2Dr. W. Abu-Sufah, UJ Review: Major Components of a Computer Processor Control Datapath Memory Devices Input Output Cache Main Memory Secondary Memory (Disk)

3. CPE432 Chapter 5A.3Dr. W. Abu-Sufah, UJ Processor-Memory Performance Gap “Moore’s Law” µProc 55%/year (2X/1.5yr) DRAM 7%/year (2X/10yrs) Processor-Memory Performance Gap (grows 50%/year)

4. CPE432 Chapter 5A.4Dr. W. Abu-Sufah, UJ The “Memory Wall”  Processor vs DRAM speed disparity continues to grow Clocks per instruction Clocks per DRAM access  Good memory hierarchy (cache) design is increasingly important to overall performance

5. CPE432 Chapter 5A.5Dr. W. Abu-Sufah, UJ The Memory Hierarchy Goal  Fact: Large memories are slow and fast memories are small  How do we create a memory that gives the illusion of being large and fast (most of the time)? l With hierarchy l With parallelism Also cheap

6. CPE432 Chapter 5A.6Dr. W. Abu-Sufah, UJ Second Level Cache (SRAM) A Typical Memory Hierarchy Control Datapath Secondary Memory (Disk) On-Chip Components RegFile Main Memory (DRAM) Data Cache Instr Cache ITLB DTLB Speed (%cycles): ½’s 1’s 10’s 100’s 10,000’s Size (bytes): 100’s 10K’s M’s G’s T’s Cost: highest lowest  The memory system of a modern computer consists of a series of black boxes ranging from the fastest to the slowest.  Besides variation in speed, these boxes also vary in size (smallest to biggest) and cost.

7. CPE432 Chapter 5A.7Dr. W. Abu-Sufah, UJ Characteristics of the Memory Hierarchy Increasing distance from the processor in access time L1$ L2$ Main Memory (MM) Secondary Memory (SM) Processor (Relative) size of the memory at each level Inclusive– what is in L1$ is a subset of what is in L2$ is a subset of what is in MM that is a subset of is in SM 4-8 bytes (word) 1 to 4 blocks 1,024+ bytes (disk sector = page) 8-32 bytes (block)

8. CPE432 Chapter 5A.8Dr. W. Abu-Sufah, UJ Why Does the Concept of a Memory Hierarchy Work?  What makes this kind of hierarchical memory organization work is the principle of locality of memory references generated by programs.  The principle of locality states that programs access a relatively small portion of the address space at any instant of time.  A memory hierarchy takes advantage of the principle of locality to present the user with as much memory as is available in the cheapest technology at the speed offered by the fastest technology

9. CPE432 Chapter 5A.9Dr. W. Abu-Sufah, UJ Memory Hierarchy Technologies: the Cache  Caches use SRAM for speed and technology compatibility l Fast (typical access times of 0.5 to 2.5 nsec) l Low density (6 transistor cells), higher power, expensive ($2000 to $5000 per GB in 2008) l Static: content will last “forever” (as long as power is left on)

10. CPE432 Chapter 5A.10Dr. W. Abu-Sufah, UJ Memory Hierarchy Technologies: Main Memory  Main memory uses DRAM for size (density) l Slower (typical access times of 50 to 70 nsec) l High density (1 transistor cells), lower power, cheaper ($20 to $75 per GB in 2008) l Dynamic: needs to be “refreshed” regularly (~ every 8 ms) - consumes1% to 2% of the active cycles of the DRAM l Addresses divided into 2 halves (row and column) -RAS or Row Access Strobe triggering the row decoder -CAS or Column Access Strobe triggering the column selector

11. CPE432 Chapter 5A.11Dr. W. Abu-Sufah, UJ Principal of Locality  Temporal Locality (locality in time) l If a memory location is referenced then it will tend to be referenced again soon  Keep most recently accessed data items closer to the processor  Spatial Locality (locality in space) l If a memory location is referenced, the locations with nearby addresses will tend to be referenced soon  Move blocks consisting of contiguous words closer to the processor

12. CPE432 Chapter 5A.12Dr. W. Abu-Sufah, UJ The Memory Hierarchy: Terminology  Block (or line): the minimum unit of information that is present (or not present ) in a cache  Hit Rate: the fraction of memory accesses found in a level of the memory hierarchy l Hit Time: Time to access that level which consists of Time to determine hit/miss + Time to access the block

13. CPE432 Chapter 5A.13Dr. W. Abu-Sufah, UJ The Memory Hierarchy: Terminology II  Miss Rate: the fraction of memory accesses not found in a level of the memory hierarchy  1 - (Hit Rate) l Miss Penalty: Time to replace a block in that level with the corresponding block from a lower level which consists of: Time to access the block in the lower level + Time to transmit that block to the level that experienced the miss + Time to insert the block in that level + Time to pass the block to the requestor Hit Time <<< Miss Penalty

14. CPE432 Chapter 5A.14Dr. W. Abu-Sufah, UJ How is the Hierarchy Managed?  registers  memory l by compiler (programmer?)  cache  main memory l by the cache controller hardware  main memory  disks l by the operating system (virtual memory) l virtual to physical address mapping assisted by the hardware (TLB) l by the programmer (files)