1. Computer Architectures & Networks WS 2004/2005 Dec. 7 th 2005 The Memory Hierarchy / Caches Carsten Trinitis Lehrstuhl für Rechnertechnik und Rechnerorganisation (LRR) Technische Universität München

2. © C. Trinitis, Computer Architecture & Networks, Lecture @ CSE/TUM, WS 2005/2006 2 Programmability  Caches have no impact  Caches are designed to be transparent  Programmer has no influence  BUT: large performance impact  Need to use caches efficiently: High Locality!  E.g. Try to reuse data in caches  HPC Applications need to be tailored to caches  Adapt to cache sizes and cache line sizes  Good understanding of architecture required  BUT: Significant performance gains

3. © C. Trinitis, Computer Architecture & Networks, Lecture @ CSE/TUM, WS 2005/2006 3 Are Caches THE Solution?  Caches have been very effective  Transparently help to close the memory-CPU gap  Present in all modern computer systems  But: difficult to control  Adapt software to fully exploit caches  Can be very cumbersome (use analysis tools!)

4. © C. Trinitis, Computer Architecture & Networks, Lecture @ CSE/TUM, WS 2005/2006 4 Other Techniques / Pros & Cons  Prefetching  Try to preload data that will „potentially“ be used  Pro: Data can be pre-requested  Con: May waste bandwidth / not used loads  Controlled by Hardware  Speculative loads  Controlled by programmer / compiler  Insert the prefetching statements into the code.

5. © C. Trinitis, Computer Architecture & Networks, Lecture @ CSE/TUM, WS 2005/2006 5 Ideas for the Future  Enhance cache model  Higher associativity  Larger cache blocks  Adaptable cache parameters  Reconfigurable architectures  Adjust memory hierarchy to applications  Use of distinct local memory pools  Hardware remapping techniques  New model of computation  Away from CPU to memory centric models (PIM)

6. © C. Trinitis, Computer Architecture & Networks, Lecture @ CSE/TUM, WS 2005/2006 6 Summary  The Memory System  The Memory Wall Problems  Caches and their Problems  Issue of Locality & Alignment  Programmability & Software Techniques  Beyond caches  E.g. Prefetching