1. Virtual Memory

2.  Next in memory hierarchy  Motivations:  to remove programming burdens of a small, limited amount of main memory  to allow efficient and safe sharing of memory among multiple programs the amount of main memory vs. the total amount of memory required by all the programmers  a fraction of the total required memory is active at any time  bring those active only to the physical memory  each program uses its own address space  VM implements the mapping of a program’s virtual address space to physical addresses

3. Motivations  Use physical DRAM as a cache for the disk  Address space of a process can exceed physical memory size  Sum of address spaces of multiple processes can exceed physical memory  Simplify memory management  Multiple processes resident in main memory  Each process with its own address space  Only “active” code and data is actually in memory  Allocate more memory to process as needed  Provide virtually contiguous memory space  Provide protection  One process can’t interfere with another  Because they operate in different address spaces  User process cannot access privileged information  Different sections of address spaces have different permissions. 3

4. Motivation 1: Caching  Use DRAM as a cache for the disk  Full address space is quite large:  32-bit addresses: 4 billion (~4 x 10 9 ) bytes  64-bit addresses: 16 quintillion (~16 x 10 18 ) bytes  Disk storage is ~300X cheaper than DRAM storage  160GB of DRAM: ~$32,000  160GB of disk: ~$100  To access large amounts of data in a cost-effective manner, the bulk of the data must be stored on disk 4 1GB: ~$200 DISK 160 GB: ~$100 DRAM SRAM 4 MB: ~$500

5. DRAM vs. Disk  DRAM vs. disk is more extreme than SRAM vs. DRAM  Access latencies:  DRAM: ~10X slower than SRAM  Disk: ~100,000X slower than DRAM  Importance of exploiting spatial locality:  Disk: First byte is ~100,000X slower than successive  DRAM: ~4X improvement for fast page-mode vs. regular accesses  Bottom line:  Design decisions made for DRAM caches driven by enormous cost of misses 5 1GB: 10X slower 160 GB: 100,000X slower 4 MB: DISK DRAM SRAM

6. Cache for Disk  Design parameters  Line (block) size?  Large, since disk better at transferring large blocks  Associativity? (Block placement)  High, to minimize miss rate  Block identification?  Using tables  Write through or write back? (Write strategy)  Write back, since can’t afford to perform small writes to disk  Block replacement?  Not to be used in the future  Based on LRU (Least Recently Used)  Impact of design decisions for DRAM cache (main memory)  Miss rate: Extremely low. << 1%  Hit time: Match DRAM performance  Miss latency: Very high. ~20ms  Tag storage overhead: Low, relative to block size 6

7. Locating an Object  DRAM Cache  Each allocated page of virtual memory has entry in page table  Mapping from virtual pages to physical pages  Page table entry even if page not in memory  Specifies disk address  OS retrieves information  Page fault handler places a page from disk in DRAM 7 Data 243 17 105 0: 1: N-1: X Object Name Location D: J: X: 1 0 On Disk “Main Memory”Page Table

8. If there is physical memory only  Examples  Most Cray machines, early PCs, nearly all embedded systems, etc.  Addresses generated by the CPU correspond directly to bytes in physical memory 8 CPU 0: 1: N-1: Memory Physical Addresses

9. System with Virtual Memory  Examples  Workstations, servers, modern PCs, etc.  Address translation: Hardware converts virtual addresses to physical addresses via OS-managed lookup table (page table) 9 CPU 0: 1: N-1: Memory 0: 1: P-1: Page Table Disk Virtual Addresses Physical Addresses

10. Page Fault (Cache Miss)  What if an object is on disk rather than in memory?  Page table entry indicates virtual addresses not in memory  OS exception handler invoked to move data from disk into memory  Current process suspends, others can resume  OS has full control over placement, etc. 10 Memory Page Table Disk Virtual Addresses Physical Addresses CPU Memory Page Table Disk Virtual Addresses Physical Addresses Before fault After fault CPU

11. Servicing a Page Fault  Processor signals controller  Read block of length P starting at disk address X and store starting at memory address Y  Read occurs  Direct Memory Address (DMA)  Under control of I/O controller  I/O controller signals completion  Interrupt processor  OS resumes suspended process 11 Disk Memory-I/O Bus Processor Cache Memory I/O Controller Reg (2) DMA Transfer (1) Initiate Block Read (3) Read Done Disk

12. Motivation 2: Management  Multiple processes can reside in physical memory  Simplifying linking  Each process see the same basic format of its memory image. 12 kernel virtual memory Memory mapped region for shared libraries runtime heap (via malloc) program text (.text) initialized data (.data) uninitialized data (.bss) stack forbidden 0 %esp memory invisible to user code the “brk” ptr Linux/x86 process memory image

13. Virtual Address Space  Two processes share physical pages by mapping in page table  Separate Virtual Address Spaces (separate page table)  Virtual and physical address spaces divided into equal-sized blocks  Blocks are called “pages” (both virtual and physical)  Each process has its own virtual address space  OS controls how virtual pages are assigned to physical memory 13 Virtual Address Space for Process 1: Physical Address Space (DRAM) VP 1 VP 2 PP 2 Address Translation 0 0 N-1 0 M-1 VP 1 VP 2 PP 7 PP 10 (e.g., read/only library code)... Virtual Address Space for Process 2:

14. Memory Management  Allocating, deallocating, and moving memory  Can be done by manipulating page tables  Allocating contiguous chunks of memory  Can map contiguous range of virtual addresses to disjoint ranges of physical addresses  Loading executable binaries  Just fix page tables for processes  Data in the binaries are paged in on demand  Protection  Store protection information on page table entries  Usually checked by hardware 14

15. Motivation 3: Protection  Page table entry contains access rights information  Hardware enforces this protection (trap into OS if violation occurs 15 Page Tables Process i: Physical AddrRead?Write? PP 9 Yes No PP 4 Yes XXXXXXX No VP 0: VP 1: VP 2: 0: 1: N-1: Memory Physical AddrRead?Write? PP 6Yes PP 9No XXXXXXXNo VP 0: VP 1: VP 2: No Yes SUP? No Yes No SUP? Yes No Process j:

16. VM Address Translation  Page size = 2 12 = 4 KB  # Physical pages = 2 18 => 2 (12+18) = 1 GB Main Mem.  Virtual address space = 2 32 = 4 GB 16

17. VM Address Translation: Hit 17 Processor Hardware Addr Trans Mechanism Main Memory a a' physical addressvirtual addresspart of the on-chip memory management unit (MMU)

18. VM Address Translation: Miss 18  Processor Hardware Addr Trans Mechanism fault handler Main Memory Secondary memory a a' page fault physical address OS performs this transfer (only if miss) virtual addresspart of the on-chip MMU

19. Page Table 19 virtual page number (VPN)page offset virtual address physical page number (PPN)page offset physical address 0p–1pm–1 n–10p–1p page table base register if valid=0 then page not in memory valid physical page number (PPN) access VPN acts as table index

20. Multi-Level Page Table  Given:  4KB (2 12 ) page size  32-bit address space  4-byte PTE  Problem:  Would need a 4 MB page table!  2 20 *4 bytes  Common solution  multi-level page tables  e.g., 2-level table (P6)  Level 1 table: 1024 entries, each of which points to a Level 2 page table.  Level 2 table: 1024 entries, each of which points to a page 20 Level 1 Table... Level 2 Tables

21. TLB  “Translation Lookaside Buffer” (TLB)  Small hardware cache in MMU  Maps virtual page numbers to physical page numbers  Contains complete page table entries for small number of pages 21 miss VA TLB Lookup Cache Main Memory PA hit data Trans- lation hit miss CPU

22. Address Translation with TLB 22 virtual address virtual page number page offset physical address n–10p–1p validphysical page numbertag validtagdata = cache hit tagbyte offset index = TLB hit TLB Cache...

23. MIPS R2000 TLB 23

24. VM Summary  Programmer’s View  Large “flat” address space  Can allocate large blocks of contiguous addresses  Process “owns” machine  Has private address space  Unaffected by behavior of other processes  System View  Use virtual address space created by mapping to set of pages  Need not be contiguous  Allocated dynamically  Enforce protection during address translation  OS manages many processes simultaneously  Continually switching among processes  Especially when one must wait for resources  e.g. disk I/O to handle page faults 24