1. Chapter 5 Memory
CSE 820

2. It’s the latency stupid!
CPU
performance
memory
time
Michigan State University
Computer Science and Engineering

3. Michigan State University Computer Science and Engineering
Vocabulary
Cache
Virtual memory
Memory stall cycles
Direct mapped
Valid bit
Block address
Write through
Michigan State University
Computer Science and Engineering

4. Michigan State University Computer Science and Engineering
Vocabulary
Instruction cache
Average memory access time
Cache hit
Page
Miss penalty
Fully associative
Dirty bit
Michigan State University
Computer Science and Engineering

5. Michigan State University Computer Science and Engineering
Vocabulary
Block offset
Write back
Data cache
Hit time
Cache miss
Page fault
Miss rate
Michigan State University
Computer Science and Engineering

6. Michigan State University Computer Science and Engineering
Vocabulary
N-way set associative
Least-recently used
Tag field
Write allocate
Unified cache
Misses per instruction
block
Michigan State University
Computer Science and Engineering

7. Michigan State University Computer Science and Engineering
Vocabulary
Locality
Address trace
Set
Random replacement
Index field
No-write allocate
Write buffer
Write stall
Michigan State University
Computer Science and Engineering

8. Michigan State University Computer Science and Engineering
Equations
CPU execution time = (CPU cycles + Memory-stall cycles) x clockCycleTime
Memory stall cycles = misses x penalty
= IC x miss/Inst x penalty
= IC x memAccess/Inst x missRate x penalty
Michigan State University
Computer Science and Engineering

9. Michigan State University Computer Science and Engineering
Hierarchy Questions
Block Placement
Where can a block be placed?
Block Identification
How is a block found?
Block Replacement
Which block should be replaced on a miss?
Write Strategy
What happens on a write?
Michigan State University
Computer Science and Engineering

10. Michigan State University Computer Science and Engineering
Cache Q1: Where to place?
Michigan State University
Computer Science and Engineering

11. Michigan State University Computer Science and Engineering
Cache Q2: How to find it?
block address
block offset
tag
index
Index selects the set
Tag checks all in the set
Offset selects within the block
Valid bit
Michigan State University
Computer Science and Engineering

12. Michigan State University Computer Science and Engineering
Cache Q3: Replacement?
Random
Simplest
FIFO
LRU
Approximation
Outperforms others for large caches
Michigan State University
Computer Science and Engineering

13. Michigan State University Computer Science and Engineering
Cache Q4: write?
Reads are most common and easiest
Write through
Write back
On replacement
Dirty bit
Write allocate
No-write allocate
Michigan State University
Computer Science and Engineering

14. Alpha 21264 Cache: Go through it on your own
Michigan State University
Computer Science and Engineering

15. Michigan State University Computer Science and Engineering
Equations
Fig 5.9 has 12 memory performance equations
Most are variations on
CPU = IC x CPI x cycles
AvgMemAccess = Hit + MissRate x MissPenalty
Michigan State University
Computer Science and Engineering

16. AvgMemAccess = Hit + MissRate x MissPenalty
17 Cache Optimizations
Reduce miss penalty
Multilevel, critical first, read-before-write, merging writes, victim cache
Reduce miss rate
Larger blocks and caches, higher associativity, way prediction, compiler optimizations
Reduce miss rate & penalty with parallelism
Non-blocking, hardware and software prefetch
Reduce hit time
Size, translation, pipelined, trace
Michigan State University
Computer Science and Engineering