Caches in Systems Feb 2013 COMP25212 Cache 4

Learning Objectives To understand: “3 x C’s” model of cache performance Time penalties for starting with empty cache Systems interconnect issues with caching and solutions! Caching and Virtual Memory Feb 2013 COMP25212 Cache 4

Describing Cache Misses Compulsory Misses Capacity Misses Conflict Misses How can we avoid them? Feb 2013 COMP25212 Cache 4

Cache Performance again Today’s caches, how long does it take: a) to fill L3 cache? (8MB) b) to fill L2 cache? (256KB) c) to fill L1 D cache? (32KB) (e.g.) Number of lines = (cache size) / (line size) Number of lines = 32K/64 = 512 512 x memory access times at 20nS = 10 uS 20,000 clock cycles at 2GHz 131 072 x 20nS = 2.5 mS 4K x 20nS = 100 uS Feb 2013 COMP25212 Cache 4

Caches in Systems e.g. disk, network how often? L1 Inst CPU Cache fetch RAM Memory L1 Data Cache data e.g. SATA = 300 MB/s – one byte every 3 nS (64 bytes every 200 nS) e.g. 1G Ethernet = 1 bit every nS, or 64 bytes every 512 nS e.g. 10Gig E? On-chip internconnect stuff e.g. disk, network how often? Input/Output Feb 2013 COMP25212 Cache 4

Cache Consistency Problem 1 Inst Cache L2 “I” CPU fetch RAM Memory Data Cache “I” data “I” On-chip internconnect stuff Problem: I/O writes to mem; cache outdated Input/Output Feb 2013 COMP25212 Cache 4

Cache Consistency Problem 2 Inst Cache L2 “I” CPU fetch RAM Memory Data Cache “I” data “I” On-chip internconnect stuff Problem: I/O reads mem; cache holds newer Input/Output Feb 2013 COMP25212 Cache 4 COMP25212 Cache 4

Cache Consistency Software Solutions O/S knows where I/O takes place in memory Mark I/O areas as non-cachable (how?) O/S knows when I/O starts and finishes Clear caches before&after I/O? Feb 2013 COMP25212 Cache 4

Hardware Solutions:1 Unfortunately: tends to slow down cache L1 “I” Inst Cache L2 “I” CPU fetch RAM Memory Data Cache “I” data Issues? “I” On-chip internconnect stuff Unfortunately: tends to slow down cache Input/Output Feb 2013 COMP25212 Cache 4 COMP25212 Cache 4

Hardware Solutions: 2 - Snooping Inst Cache L2 “I” CPU fetch RAM Memory Data Cache “I” data Issues? “I” On-chip internconnect stuff L2 keeps track of L1 contents Snoop logic in cache observes every memory cycle Input/Output Feb 2013 COMP25212 Cache 4 COMP25212 Cache 4

Caches and Virtual Addresses CPU addresses – virtual Memory addresses – physical Recap – use TLB to translate v-to-p What addresses in cache? Feb 2013 COMP25212 Cache 4

Option 1: Cache by Physical Addresses TLB $ CPU address RAM Memory data On-chip BUT: Address translation in series with cache SLOW Feb 2013 COMP25212 Cache 4

Option 2: Cache by Virtual Addresses $ TLB CPU address RAM Memory data On-chip BUT: Snooping? Aliasing? More Functional Difficulties Feb 2013 COMP25212 Cache 4 COMP25212 Cache 4

3: Translate in parallel with Cache Lookup Translation only affects high-order bits of address Address within page remains unchanged Low-order bits of Physical Address = low-order bits of Virtual Address Select “index” field of cache address from within low-order bits Only “Tag” bits changed by translation Feb 2013 COMP25212 Cache 4 COMP25212 Cache 4

Option 3 in operation: 20 5 7 Virtual address virtual page no index within line TLB tag line data line What are my assumptions? Line size = 2^7 bytes – 128 bytes Index = 5 bits => cache has 2^5 lines (32 lines) – 4K byte cache Page size = 12 bits – 4K page How can we increase cache size? (multi-way set-associativity) – 8-way set associativity would give us 32K cache Physical address multiplexer compare = ? Hit? Data Feb 2013 COMP25212 Cache 4

The Last Word on Caching? On-chip L1 Data Cache CPU Inst fetch data L2 L3 On-chip L1 Data Cache CPU Inst fetch data L2 L3 On-chip L1 Data Cache CPU Inst fetch data L2 L3 RAM Memory Input/Output You ain’t seen nothing yet! Feb 2013 COMP25212 Cache 4