1. Cache Organization Topics Background Simple examples

2. – 2 – Typical Cache Organization =? cache line address tagindexoffset tag array data array

3. Cache Organization Details (S, E, B) E = 2 e blocks (lines) per set S=2 s sets set block 012 B-1 tagv valid bit B = 2 b bytes per cache block (the data) Cache size: S x E x B data bytes

4. – 4 – Example: Direct Mapped Cache (E = 1) S=2 s sets Direct mapped: One block per set Assume: cache block size 8 bytes t bits 100 Address of int: 012 7 tagv3 654 012 7 v3 654 012 7 v3 654 012 7 v3 654 find set

5. – 5 – Example: Direct Mapped Cache (E = 1) t bits 100 Address of int: 012 7 tagv3 654 match: assume yes = hit block offset tag Direct mapped: One block per set Assume: cache block size 8 bytes

6. – 6 – Example: Direct Mapped Cache (E = 1) t bits Address of int: 012 7 tagv3 654 match: assume yes = hit block offset tag Direct mapped: One block per set Assume: cache block size 8 bytes int (4 Bytes) is here No match: old line is evicted and replaced 100

7. – 7 – E-way Set Associative Cache (E = 2) E = 2: Two lines per set Assume: cache block size 8 bytes t bits 100 Address of short int: 012 7 tg v3 654 012 7 v3 654 012 7 v3 654 012 7 v3 654 012 7 v3 654 012 7 v3 654 012 7 v3 654 012 7 v3 654 find set

8. E-way Set Associative Cache (E = 2) t bits 100 Address of short int: 012 7 tgv3 654 012 7 v3 654 compare both match: yes = hit block offset tg E = 2: Two lines per set cache block size 8 bytes tg

9. E-way Set Associative Cache (E = 2) t bits 100 Address of short int: 012 7 v3 654 012 7 tg v3 654 compare both match: yes = hit block offset tg E = 2: Two lines per set cache block size 8 bytes short int (2 Bytes) is here No match: One line in set is selected for eviction and replacement Replacement policies: random, least recently used (LRU), … tg

10. – 10 – Assumed Simple Cache 2 ints per block 2-way set associative 2 blocks total 1 set i.e., same thing as fully associative Replacement policy: Least Recently Used (LRU) Cache Block 0 Block 1

11. – 11 – Array Access: Key Questions How many array elements are there per block? Does the data structure fit in the cache? Do I re-use blocks over time? In what order am I accessing blocks?

12. – 12 – Simple Array 1234 A Cache for (i=0;i<N;i++){ … = A[i]; } Miss rate = #misses / #accesses

13. – 13 – Simple Array 1234 A Cache for (i=0;i<N;i++){ … = A[i]; } Miss rate = #misses / #accesses = (N//2) / N = ½ = 50%

14. – 14 – Simple Array w outer loop 1234 A Cache for (k=0;k<P;k++){ for (i=0;i<N;i++){ … = A[i]; } Assume A[] fits in the cache: Miss rate = #misses / #accesses = Lesson: for sequential accesses with re-use, If fits in the cache, first visit suffers all the misses

15. – 15 – Simple Array w outer loop 1234 A Cache for (k=0;k<P;k++){ for (i=0;i<N;i++){ … = A[i]; } Assume A[] fits in the cache: Miss rate = #misses / #accesses = Lesson: for sequential accesses with re-use, If fits in the cache, first visit suffers all the misses (N//2) / N = ½ = 50%

16. – 16 – Simple Array 12345678 A Cache for (i=0;i<N;i++){ … = A[i]; } Assume A[] does not fit in the cache: Miss rate = #misses / #accesses

17. – 17 – Simple Array 56 78 12345678 A Cache for (i=0;i<N;i++){ … = A[i]; } Assume A[] does not fit in the cache: Miss rate = #misses / #accesses = (N/2) / N = ½ = 50% Lesson: for sequential accesses, if no-reuse it doesn’t matter whether data structure fits

18. – 18 – Simple Array with outer loop 12345678 A Cache Assume A[] does not fit in the cache: Miss rate = #misses / #accesses = for (k=0;k<P;k++){ for (i=0;i<N;i++){ … = A[i]; } (N/2) / N = ½ = 50% Lesson: for sequential accesses with re-use, If the data structure doesn’t fit, same miss rate as no-reuse

19. – 19 – 2D array A Cache Assume A[] fits in the cache: Miss rate = #misses / #accesses = for (i=0;i<N;i++){ for (j=0;j<N;j++){ … = A[i][j]; } 12 34 (N*N/2) / (N*N) = ½ = 50%

20. – 20 – 2D array A Cache for (i=0;i<N;i++){ for (j=0;j<N;j++){ … = A[i][j]; } Lesson: for 2D accesses, if row order and no-reuse, same hit rate as sequential, whether fits or not 1234 5678 9101112 13141516 Assume A[] does not fit in the cache: Miss rate = #misses / #accesses = 50%

21. – 21 – 2D array A Cache for (j=0;j<N;j++){ for (i=0;i<N;i++){ … = A[i][j]; } Lesson: for 2D accesses, if column order and no-reuse, same hit rate as sequential if entire column fits in the cache 12 34 Assume A[] fits in the cache: Miss rate = #misses / #accesses = (N*N/2) / N*N = ½ = 50%

22. – 22 – 2D array A Cache Assume A[] does not fit in the cache: Miss rate = #misses / #accesses 1234 5678 9101112 13141516 for (j=0;j<N;j++){ for (i=0;i<N;i++){ … = A[i][j]; } = 100% Lesson: for 2D accesses, if column order, if entire column doesn’t fit, then 100% miss rate (block (1,2) is gone after access to element 9).

23. – 23 –