Overview: Memory Memory Organization: General Issues (Hardware) –Objectives in Memory Design –Memory Types –Memory Hierarchies Memory Management (Software - OS) –Cache –Virtual Memory & Paging –Segmentation Memory Issues in Multiprogramming (Software - OS) –Memory allocation for each process –Swapping –Security

Memory Organization: Objectives High Capacity Fast Access Small Size to fit on chip Cheap Reusable Non-volatile Reliable

Memory Reality Fast Memory is very expensive The bigger memory, the slower it will be Need for memory grows Parkinson’s Law “ Programs expand to fill the memory available to hold them” You can’t run Windows XP on a 5 year old machine with 32 MB

Storage Devices Register Cache Main Memory (RAM) Hard disk Floppy Tape CD ROM …

Memory Types I Electric –Static RAM (Random Access Memory) D-Latch, holds information as long as the circuit has power Fast, Expensive, many transistors –Dynamic RAM One transistor and capacitor per bit Slower, need regular refreshing, cheap, small Magnetic –Hard Drive, Floppy Cheap, slower, stable, error prone

Hard Drive

Memory Types II Optical/Mechanical: CD-Rom, DVD –ROM (Read Only access) –PROM (Programmable ROM) –EPROM (Erasable PROM) – read/write! Cheap, large volume, slow access, stable, high reliability

Type Comparison TypeCategoryErasureVolatileUse SRAMRead/WriteElectricalYesRegister, Cache DRAMRead/WriteElectricalYesMain Memory ROM, PROMRead onlyNo Large Volume Appliances EPROMRead mostlyUV-LightNoDevice Prototyping MagneticRead/WriteMagneticNoHard Drive, Backup tapes

Question: How can you improve performance without higher cost? Answer: Build memory hierarchy Speed Fastest Slowest Cost $/bit Highest Lowest Size Smallest Biggest CPU Memory

Memory Hierarchy

Memory Management: OS Obvious question: What content of Memory should be in what level of memory? How to provide uniform access for all processes to memory content independent of its physical location?

Principle of Locality Locality: –Temporal: You will use again things you used recently –Spatial: Nearby items are more likely to be used (Array) The idea of hierarchy works only because of locality Why does code has locality? –Local variable space –Limited set of variables that will be used again –Arrays –Machine language is stored sequentially

Cache “A safe place for hiding or storing things” Webster Dictionary Small and very fast temporal memory between CPU and main memory that contains things (variables and instructions) that were recently used by CPU

Cache: Simple Example Example: Memory has 8 lines, cache has 4 –LOAD R1, x5 –Since x5 is not in the cache, it will be fetched from main memory and replace one current entry –Cache before request After request X3 X1 X4 X0 X3 X5 X4 X0

Terminology Hit: data requested could be found in one level Miss: Data could not be found -> has to be searched on lower level Miss Rate: percentage of memory access that resulted a miss Block: Minimum amount of data transferred between levels (in the example it was 1) Miss penalty: Additional time it takes to find item

Cache Parameter Block size: How many items do you replace? –Big if high degree of spatial locality –Small if low degree of spatial locality Which block will be discarded in case of a miss? –Replacement strategy: pick one that will hopefully not be used again Separation of data and instruction –Either cache for both or 2 caches, one for each

Unified Data and Instruction Cache

Separate Data and Instruction

Performance Access Time: Miss rate*Miss penalty + Hit time –Miss rate reduction: Bigger cache Bigger blocks Good replacement strategy –Miss penalty reduction Smaller blocks Simple replacement strategy –Hit time reduction Small and simple cache

Replacement Strategy Not recently used –Pick first block that was not used for certain time First in first out –Oldest of all blocks in cache Least recently used –Compare last use time or all blocks and pick the least used –Optimal but very expensive Cheaper approximation of LRU

Cache Entry What information do you need for each item/block? –Associated address –Valid bit (during initialization 0) –Accounting information for replacement: time of last use AddressValidTimeValue

Hit vs. Miss Read hit –Optimal Read miss –Stall CPU, fetch new block from memory, identify oldest block, replace, start CPU Write hit –Update cache, depending on strategy update memory Write miss –Similar to read miss, first update memory, fetch block, replace, start CPU

Write Strategies Write-through –Update memory whenever cache is changed Copy back –Update memory only when block is deleted from cache –Keep track of changes with dirty bit –Makes read miss more expensive but improves the write time significantly Buffered write-through –Keep track of updates in special buffer, buffer transfers data slowly to memory