Cache Organization Topics Background Simple examples

– 2 – Typical Cache Organization =? cache line address tagindexoffset tag array data array

Cache Organization Details (S, E, B) E = 2 e blocks (lines) per set S=2 s sets set block 012 B-1 tagv valid bit B = 2 b bytes per cache block (the data) Cache size: S x E x B data bytes

– 4 – Example: Direct Mapped Cache (E = 1) S=2 s sets Direct mapped: One block per set Assume: cache block size 8 bytes t bits 100 Address of int: tagv v v v3 654 find set

– 5 – Example: Direct Mapped Cache (E = 1) t bits 100 Address of int: tagv3 654 match: assume yes = hit block offset tag Direct mapped: One block per set Assume: cache block size 8 bytes

– 6 – Example: Direct Mapped Cache (E = 1) t bits Address of int: tagv3 654 match: assume yes = hit block offset tag Direct mapped: One block per set Assume: cache block size 8 bytes int (4 Bytes) is here No match: old line is evicted and replaced 100

– 7 – E-way Set Associative Cache (E = 2) E = 2: Two lines per set Assume: cache block size 8 bytes t bits 100 Address of short int: tg v v v v v v v v3 654 find set

E-way Set Associative Cache (E = 2) t bits 100 Address of short int: tgv v3 654 compare both match: yes = hit block offset tg E = 2: Two lines per set cache block size 8 bytes tg

E-way Set Associative Cache (E = 2) t bits 100 Address of short int: v tg v3 654 compare both match: yes = hit block offset tg E = 2: Two lines per set cache block size 8 bytes short int (2 Bytes) is here No match: One line in set is selected for eviction and replacement Replacement policies: random, least recently used (LRU), … tg

– 10 – Assumed Simple Cache 2 ints per block 2-way set associative 2 blocks total 1 set i.e., same thing as fully associative Replacement policy: Least Recently Used (LRU) Cache Block 0 Block 1

– 11 – Array Access: Key Questions How many array elements are there per block? Does the data structure fit in the cache? Do I re-use blocks over time? In what order am I accessing blocks?

– 12 – Simple Array 1234 A Cache for (i=0;i<N;i++){ … = A[i]; } Miss rate = #misses / #accesses

– 13 – Simple Array 1234 A Cache for (i=0;i<N;i++){ … = A[i]; } Miss rate = #misses / #accesses = (N//2) / N = ½ = 50%

– 14 – Simple Array w outer loop 1234 A Cache for (k=0;k<P;k++){ for (i=0;i<N;i++){ … = A[i]; } Assume A[] fits in the cache: Miss rate = #misses / #accesses = Lesson: for sequential accesses with re-use, If fits in the cache, first visit suffers all the misses

– 15 – Simple Array w outer loop 1234 A Cache for (k=0;k<P;k++){ for (i=0;i<N;i++){ … = A[i]; } Assume A[] fits in the cache: Miss rate = #misses / #accesses = Lesson: for sequential accesses with re-use, If fits in the cache, first visit suffers all the misses (N//2) / N = ½ = 50%

– 16 – Simple Array A Cache for (i=0;i<N;i++){ … = A[i]; } Assume A[] does not fit in the cache: Miss rate = #misses / #accesses

– 17 – Simple Array A Cache for (i=0;i<N;i++){ … = A[i]; } Assume A[] does not fit in the cache: Miss rate = #misses / #accesses = (N/2) / N = ½ = 50% Lesson: for sequential accesses, if no-reuse it doesn’t matter whether data structure fits

– 18 – Simple Array with outer loop A Cache Assume A[] does not fit in the cache: Miss rate = #misses / #accesses = for (k=0;k<P;k++){ for (i=0;i<N;i++){ … = A[i]; } (N/2) / N = ½ = 50% Lesson: for sequential accesses with re-use, If the data structure doesn’t fit, same miss rate as no-reuse

– 19 – 2D array A Cache Assume A[] fits in the cache: Miss rate = #misses / #accesses = for (i=0;i<N;i++){ for (j=0;j<N;j++){ … = A[i][j]; } (N*N/2) / (N*N) = ½ = 50%

– 20 – 2D array A Cache for (i=0;i<N;i++){ for (j=0;j<N;j++){ … = A[i][j]; } Lesson: for 2D accesses, if row order and no-reuse, same hit rate as sequential, whether fits or not Assume A[] does not fit in the cache: Miss rate = #misses / #accesses = 50%

– 21 – 2D array A Cache for (j=0;j<N;j++){ for (i=0;i<N;i++){ … = A[i][j]; } Lesson: for 2D accesses, if column order and no-reuse, same hit rate as sequential if entire column fits in the cache Assume A[] fits in the cache: Miss rate = #misses / #accesses = (N*N/2) / N*N = ½ = 50%

– 22 – 2D array A Cache Assume A[] does not fit in the cache: Miss rate = #misses / #accesses for (j=0;j<N;j++){ for (i=0;i<N;i++){ … = A[i][j]; } = 100% Lesson: for 2D accesses, if column order, if entire column doesn’t fit, then 100% miss rate (block (1,2) is gone after access to element 9).

– 23 –