1. Chapter 5 Memory II CSE 820

2. Michigan State University Computer Science and Engineering Equations CPU execution time = (CPU cycles + Memory-stall cycles) x clockCycleTime Memory stall cycles = misses x penalty = IC x miss/Inst x penalty = IC x memAccess/Inst x missRate x penalty

3. Michigan State University Computer Science and Engineering Hierarchy Questions 1.Block Placement Where can a block be placed? 2.Block Identification How is a block found? 3.Block Replacement Which block should be replaced on a miss? 4.Write Strategy What happens on a write?

4. Michigan State University Computer Science and Engineering Cache Q1: Where to place?

5. Michigan State University Computer Science and Engineering Cache Q2: How to find it? Index selects the set Tag checks all in the set Offset selects within the block Valid bit block address block offset indextag

6. Michigan State University Computer Science and Engineering Cache Q3: Replacement? Random –Simplest FIFO LRU –Approximation –Outperforms others for large caches

7. Michigan State University Computer Science and Engineering Cache Q4: write? Reads are most common and easiest Writes –Write through –Write back On replacement Dirty bit Allocate? –Write allocate –No-write allocate

8. Michigan State University Computer Science and Engineering Alpha 21264 Cache: Go through it on your own

9. Michigan State University Computer Science and Engineering Equations Fig 5.9 has 12 memory performance equations Most are variations on CPU = IC x CPI x cycles AvgMemAccess = Hit + MissRate x MissPenalty

10. Michigan State University Computer Science and Engineering AvgMemAccess = Hit + MissRate x MissPenalty 17 Cache Optimizations Reduce miss penalty Multilevel, critical first, read-before-write, merging writes, victim cache Reduce miss rate Larger blocks and caches, higher associativity, way prediction, compiler optimizations Reduce miss rate & penalty with parallelism Non-blocking, hardware and software prefetch Reduce hit time Size, translation, pipelined, trace

11. Michigan State University Computer Science and Engineering Reducing Cache Miss Penalty

12. Michigan State University Computer Science and Engineering Questions Why is a small cache fast? Why will a designer put L1 and L2 cache on a processor chip rather than simply having a large L1 cache? Why is L1 direct mapped?

13. Michigan State University Computer Science and Engineering Reducing Miss Penalty: Multilevel Caches How does an L2 cache reduce the miss penalty?

14. Michigan State University Computer Science and Engineering Multilevel Cache AvgMemAccessTime = HitTime L1 + MissRate L1 X MissPenalty L1 where MissPenalty L1 = HitTime L2 + MissRate L2 X MissPenalty L2

15. Michigan State University Computer Science and Engineering Terms Global miss rate = misses/CPUaccesses –e.g. for L2 = MissRate L1 X MissRate L2 Local miss rate = misses/CacheAccesses –e.g. MissRate L2 measured on the leftovers from L1

16. Michigan State University Computer Science and Engineering L1 vs L2 with 32K L1: small L2 is poor What is good about this graph? Hit time is good and so is avg memory latency.

17. Michigan State University Computer Science and Engineering Inclusive vs Exclusive Inclusion: L1 is subset of L2 pro: consistency between I/O and cache (or among multiprocessor caches) is easy—simply check L2 e.g. P4 Exclusive: L1 and L2 are disjoint pro: useful when L2 is only slightly larger than L1 e.g. AMD Athlon

18. Michigan State University Computer Science and Engineering Miss Penalty Reduction: Critical Word First & Early Restart Critical word first: In multiword block, bring in the desired word first. Early Restart: restart CPU when desired word is received

19. Michigan State University Computer Science and Engineering Miss Penalty Reduction: Priority to Read Reads first if no conflict with writes in write buffer Why?

20. Michigan State University Computer Science and Engineering Miss Penalty Reduction: Merging Write Buffer Within write buffer merge words into blocks before writing

21. Michigan State University Computer Science and Engineering Miss Penalty Reduction: Victim Cache Remember last block discarded Algorithm: on miss, if block is in victim cache, swap. Performance: a 4-entry victim cache can resolve ¼ of misses e.g. AMD Athlon (8 entries)

22. Michigan State University Computer Science and Engineering Victim Cache

23. Michigan State University Computer Science and Engineering Reducing Cache Miss Rate

24. Michigan State University Computer Science and Engineering Miss Classification Compulsory: first access cannot be in cache Capacity: if cache cannot contain all blocks, some accesses must miss Conflict: others e.g. two blocks conflict by mapping to same slot (fully associative has no conflict)

25. Michigan State University Computer Science and Engineering compulsory  Capacity = fully associative (decreases with size) Conflict decreases with set size.

26. Michigan State University Computer Science and Engineering Miss category as a percentage.