Memory Hierarchy and Cache Memory Jennifer Tsay CS 147 Section 3 October 8, 2009

Two Basic Types of Memory RAM (Random Access Memory) RAM (Random Access Memory)  Used to store programs and data that computer needs when executing programs  Volatile and loses information once power is turned off

2 Basic Types of Memory ROM (Read-Only Memory) ROM (Read-Only Memory)  Stores critical information necessary to operate the system, such as program necessary to boot computer  Not volatile and always retains its data  Also embedded in systems where programming does not need to change

Memory Hierarchy Hierarchal Memory Hierarchal Memory  Approach in which computer systems use combination of memory types to provide best performance at best cost  Basic types that constitute hierarchal memory system include registers, cache, main memory and secondary memory

Memory Hierarchy Today’s computers each have small amount of very high-speed memory, called cache where data from frequently used memory locations may be temporarily stored Today’s computers each have small amount of very high-speed memory, called cache where data from frequently used memory locations may be temporarily stored Cache is connected to main memory, which is typically medium-speed memory Cache is connected to main memory, which is typically medium-speed memory Main memory is complemented by secondary memory, composed of hard disk and various removable media Main memory is complemented by secondary memory, composed of hard disk and various removable media

The Memory Hierarchy

How Do We Classify Memory? Based on its “distance” from the processor Based on its “distance” from the processor Distance is measured by the number of machine cycles required for access Distance is measured by the number of machine cycles required for access The closer memory is to the processor, the faster it should be The closer memory is to the processor, the faster it should be

Locality of Reference Clustering of memory references into groups Clustering of memory references into groups Important because through use of cache technique, locality of reference can optimize performance Important because through use of cache technique, locality of reference can optimize performance

Cache Memory Small, temporary, but fast (and thus high-cost) memory Small, temporary, but fast (and thus high-cost) memory Processor uses cache memory for information it is likely to need again in very near future Processor uses cache memory for information it is likely to need again in very near future Non-computer Examples Non-computer Examples  Cell Phone Address Book vs. Phone Book  Desk vs. File Cabinet

MAIN MEMORY CACHE

Cache Mapping Schemes For cache to be functional, it must store useful data For cache to be functional, it must store useful data This data becomes useless if CPU can’t find it This data becomes useless if CPU can’t find it When accessing data or instructions, the CPU first generates main memory address When accessing data or instructions, the CPU first generates main memory address If data has been copied to cache, address of data in cache is not same as main memory address If data has been copied to cache, address of data in cache is not same as main memory address

Types of Mapping Schemes Direct Mapped Cache Direct Mapped Cache  Each main memory block has specific location to which it maps in cache  If block already occupies cache location where new block must be placed, block currently in cache is removed

Types of Mapping Schemes Fully Associative Cache Fully Associative Cache  Allows main memory block to be placed anywhere in cache  When cache is full, need replacement algorithm to decide which block to throw out of cache

Replacement Algorithms LRU (Least Recently Used) LRU (Least Recently Used)  Keep track of the last time each block was accessed  Select the block that has been used least recently to be the block that gets removed  Requires system to keep history of accesses for every cache block which requires significant space and slows down operation of cache FIFO (First-in, First out) FIFO (First-in, First out)  Block that has been in cache the longest (regardless of how recently it has been used) would be selected as block to be removed from cache memory

Measuring Performance How do we measure performance of two-level hierarchical memory?  By its Effective Access Time (EAT), or average time per access EAT = H x Access C + (1 – H) x Access MM H = cache hit rateAccess MM = main memory access time Access C = cache access time

Example Let’s say that: Let’s say that: Access C = 10ns Access MM = 200ns H = 99% EAT = H x Access C + (1 – H) x Access MM EAT = 0.99 x 10ns + (0.01) x 200ns ~ 12ns

Cache Writing Policies Designers have to determine what to do with blocks that have been modified Designers have to determine what to do with blocks that have been modified Cache write policy determines when actual main memory block is updated to match cache block Cache write policy determines when actual main memory block is updated to match cache block Write-Through Write-Through  Updates both cache and main memory simultaneously on every write Write-back Write-back  Only updates blocks in main memory when cache block is selected to be removed from cache

Data and Instruction Caches Data Cache Data Cache Instruction Cache Instruction Cache Advantages of separating Advantages of separating  Unified cache has only one port for data and instructions, resulting in conflicts between the two  Allows accesses to be less random and more clustered

Levels of Cache Multilevel Cache Memory Multilevel Cache Memory  Caches using caches for increased performance  Level 1 (L1) cache is term used for cache resident on chip and is fastest, smallest cache  L1 is always checked first, but if the data is not found in L1, L2 cache is checked  L3 cache refers to extra cache between processor and memory on processors

References ts.com/HON-210-Series Deep-Vertical-File-Cabinet-review- a897&usg=AFQjCNFogrb2UbcND7bTPbDviC8DvlTpeQ ts.com/HON-210-Series Deep-Vertical-File-Cabinet-review- a897&usg=AFQjCNFogrb2UbcND7bTPbDviC8DvlTpeQ ts.com/HON-210-Series Deep-Vertical-File-Cabinet-review- a897&usg=AFQjCNFogrb2UbcND7bTPbDviC8DvlTpeQ ts.com/HON-210-Series Deep-Vertical-File-Cabinet-review- a897&usg=AFQjCNFogrb2UbcND7bTPbDviC8DvlTpeQ main_Full.jpg main_Full.jpg main_Full.jpg main_Full.jpg tjliu.myweb.hinet.net/COA_CH_6.htm tjliu.myweb.hinet.net/COA_CH_6.htm tjliu.myweb.hinet.net/COA_CH_6.htm content/uploads/2009/08/2_mel-at-messy-desk.jpg content/uploads/2009/08/2_mel-at-messy-desk.jpg content/uploads/2009/08/2_mel-at-messy-desk.jpg content/uploads/2009/08/2_mel-at-messy-desk.jpg om/2008/09/ram-random-access-memory/&usg=AFQjCNF9SaCi- 7adm2Vu2fdpZF-hNVu4sw om/2008/09/ram-random-access-memory/&usg=AFQjCNF9SaCi- 7adm2Vu2fdpZF-hNVu4sw om/2008/09/ram-random-access-memory/&usg=AFQjCNF9SaCi- 7adm2Vu2fdpZF-hNVu4sw om/2008/09/ram-random-access-memory/&usg=AFQjCNF9SaCi- 7adm2Vu2fdpZF-hNVu4sw Null, L. & Lobur, J. The Essentials of Computer Organization and Architecture. 2006: Sudbury. Null, L. & Lobur, J. The Essentials of Computer Organization and Architecture. 2006: Sudbury.