1. February 25, 2010CS152, Spring 2010 CS 152 Computer Architecture and Engineering Lecture 11 - Virtual Memory and Caches Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste http://inst.eecs.berkeley.edu/~cs152

2. February 25, 2010CS152, Spring 2010 2 Today is a review of last two lectures Translation/Protection/Virtual Memory This is complex material - often takes several passes before the concepts sink in Try to take a different path through concepts today

3. February 25, 2010CS152, Spring 2010 3 VM features track historical uses: Bare machine, only physical addresses –One program owned entire machine Batch-style multiprogramming –Several programs sharing CPU while waiting for I/O –Base & bound: translation and protection between programs (not virtual memory) –Problem with external fragmentation (holes in memory), needed occasional memory defragmentation as new jobs arrived Time sharing –More interactive programs, waiting for user. Also, more jobs/second. –Motivated move to fixed-size page translation and protection, no external fragmentation (but now internal fragmentation, wasted bytes in page) –Motivated adoption of virtual memory to allow more jobs to share limited physical memory resources while holding working set in memory Virtual Machine Monitors –Run multiple operating systems on one machine –Idea from 1970s IBM mainframes, now common on laptops »e.g., run Windows on top of Mac OS X –Hardware support for two levels of translation/protection »Guest OS virtual -> Guest OS physical -> Host machine physical

4. February 25, 2010CS152, Spring 2010 4 Bare Machine In a bare machine, the only kind of address is a physical address PC Inst. Cache D Decode EM Data Cache W + Main Memory (DRAM) Memory Controller Physical Address

5. February 25, 2010CS152, Spring 2010 5 Base and Bound Scheme Logical address is what user software sees. Translated to physical address by adding base register. Physical Address Load X Program Address Space Main Memory data segment Data Bound Register Mem. Address Register Data Base Register  + Bounds Violation? Program Bound Register Program Counter Program Base Register  + Bounds Violation? program segment Logical Address

6. February 25, 2010CS152, Spring 2010 6 Base and Bound Machine PC Inst. Cache D Decode EM Data Cache W + Main Memory (DRAM) Memory Controller Physical Address Data Bound Register Data Base Register  + [ Can fold addition of base register into (base+offset) calculation using a carry-save adder (sum three numbers with only a few gate delays more than adding two numbers) ] Logical Address Bounds Violation? Physical Address Prog. Bound Register Program Base Register  + Logical Address Bounds Violation?

7. February 25, 2010CS152, Spring 2010 7 Memory Fragmentation As users come and go, the storage is “fragmented”. Therefore, at some stage programs have to be moved around to compact the storage. OS Space 16K 24K 32K 24K user 1 user 2 user 3 OS Space 16K 24K 16K 32K 24K user 1 user 2 user 3 user 5 user 4 8K Users 4 & 5 arrive Users 2 & 5 leave OS Space 16K 24K 16K 32K 24K user 1 user 4 8K user 3 free

8. February 25, 2010CS152, Spring 2010 8 Processor generated address can be interpreted as a pair A page table contains the physical address of the base of each page Paged Memory Systems Page tables make it possible to store the pages of a program non-contiguously. 0 1 2 3 0 1 2 3 Address Space of User-1 Page Table of User-1 1 0 2 3 page number offset

9. February 25, 2010CS152, Spring 2010 9 Private Address Space per User Each user has a page table Page table contains an entry for each user page VA1 User 1 Page Table VA1 User 2 Page Table VA1 User 3 Page Table Physical Memory free OS pages

10. February 25, 2010CS152, Spring 2010 10 Linear Page Table VPN Offset Virtual address PT Base Register VPN Data word Data Pages Offset PPN DPN PPN Page Table DPN PPN DPN PPN Page Table Entry (PTE) contains: –A bit to indicate if a page exists –PPN (physical page number) for a memory-resident page –DPN (disk page number) for a page on the disk –Status bits for protection and usage OS sets the Page Table Base Register whenever active user process changes

11. February 25, 2010CS152, Spring 2010 11 Page Tables in Physical Memory VA1 User 1 PT User 1 PT User 2 VA1 User 2

12. February 25, 2010CS152, Spring 2010 12 Size of Linear Page Table With 32-bit addresses, 4-KB pages & 4-byte PTEs:  2 20 PTEs, i.e, 4 MB page table per user  4 GB of swap needed to back up full virtual address space Larger pages? Internal fragmentation (Not all memory in a page is used) Larger page fault penalty (more time to read from disk) What about 64-bit virtual address space??? Even 1MB pages would require 2 44 8-byte PTEs (35 TB!) What is the “saving grace” ? sparsity of virtual address usage

13. February 25, 2010CS152, Spring 2010 13 Hierarchical (Two-Level) Page Table Level 1 Page Table Level 2 Page Tables Data Pages page in primary memory page in secondary memory Root of the Current Page Table p1 offset p2 Virtual Address (Processor Register) PTE of a nonexistent page p1 p2 offset 0 1112212231 10-bit L1 index 10-bit L2 index

14. February 25, 2010CS152, Spring 2010 14 Two-Level Page Tables in Physical Memory VA1 User 1 User1/VA1 User2/VA1 Level 1 PT User 1 Level 1 PT User 2 VA1 User 2 Level 2 PT User 2 Virtual Address Spaces Physical Memory

15. February 25, 2010CS152, Spring 2010 15 Address Translation & Protection Every instruction and data access needs address translation and protection checks A good VM design needs to be fast (~ one cycle) and space efficient Physical Address Virtual Address Address Translation Virtual Page No. (VPN)offset Physical Page No. (PPN) offset Protection Check Exception? Kernel/User Mode Read/Write

16. February 25, 2010CS152, Spring 2010 16 Translation Lookaside Buffers Address translation is very expensive! In a two-level page table, each reference becomes several memory accesses Solution: Cache translations in TLB TLB hitSingle Cycle Translation TLB miss Page Table Walk to refill VPN offset V R W D tag PPN physical address PPN offset virtual address hit? (VPN = virtual page number) (PPN = physical page number)

17. February 25, 2010CS152, Spring 2010 17 Handling a TLB Miss Software (MIPS, Alpha) TLB miss causes an exception and the operating system walks the page tables and reloads TLB. A privileged “untranslated” addressing mode used for walk Hardware (SPARC v8, x86, PowerPC) A memory management unit (MMU) walks the page tables and reloads the TLB If a missing (data or PT) page is encountered during the TLB reloading, MMU gives up and signals an exception for the original instruction

18. February 25, 2010CS152, Spring 2010 18 Page-Based Virtual Memory Machine (Hardware Page Table Walk) PC Inst. TLB Inst. Cache D Decode EM Data Cache W + Page Fault? Protection violation? Page Fault? Protection violation? Assumes page tables held in untranslated physical memory Data TLB Main Memory (DRAM) Memory Controller Physical Address P age Table Base Register Virtual Address Physical Address Virtual Address Hardware Page Table Walker Miss?

19. February 25, 2010CS152, Spring 2010 19 CS152 Administrivia Tuesday Mar 9, Quiz 2 –Cache and virtual memory lectures, L6-L11, PS 2, Lab 2

20. February 25, 2010CS152, Spring 2010 20 Virtual Memory More than just translation and protection Use disk to extend apparent size of main memory Treat DRAM as cache of disk contents Only need to hold active working set of processes in DRAM, rest of memory image can be swapped to disk Inactive processes can be completely swapped to disk (except usually the root of the page table) Combination of hardware and software used to implement this feature (ATLAS was first implementation of this idea)

21. February 25, 2010CS152, Spring 2010 21 Page Fault Handler When the referenced page is not in DRAM: –The missing page is located (or created) –It is brought in from disk, and page table is updated Another job may be run on the CPU while the first job waits for the requested page to be read from disk –If no free pages are left, a page is swapped out Pseudo-LRU replacement policy Since it takes a long time to transfer a page (msecs), page faults are handled completely in software by the OS –Untranslated addressing mode is essential to allow kernel to access page tables

22. February 25, 2010CS152, Spring 2010 22 Caching vs. Demand Paging CPU cache primary memory secondary memory Caching Demand paging cache entrypage frame cache block (~32 bytes)page (~4K bytes) cache miss rate (1% to 20%)page miss rate (<0.001%) cache hit (~1 cycle)page hit (~100 cycles) cache miss (~100 cycles)page miss (~5M cycles) a miss is handled in hardware mostly in software primary memory CPU

23. February 25, 2010CS152, Spring 2010 23 Address Translation: putting it all together Virtual Address TLB Lookup Page Table Walk Update TLB Page Fault (OS loads page) Protection Check Physical Address (to cache) miss hit the page is  memory  memory denied permitted Protection Fault hardware hardware or software software SEGFAULT Restart instruction

24. February 25, 2010CS152, Spring 2010 24 Address Translation in CPU Pipeline Software handlers need restartable exception on TLB fault Handling a TLB miss needs a hardware or software mechanism to refill TLB Need mechanisms to cope with the additional latency of a TLB: – slow down the clock – pipeline the TLB and cache access – virtual address caches – parallel TLB/cache access PC Inst TLB Inst. Cache D Decode EM Data TLB Data Cache W + TLB miss? Page Fault? Protection violation? TLB miss? Page Fault? Protection violation?

25. February 25, 2010CS152, Spring 2010 25 Virtual Address Caches one-step process in case of a hit (+) cache needs to be flushed on a context switch unless address space identifiers (ASIDs) included in tags (-) aliasing problems due to the sharing of pages (-) maintaining cache coherence (-) (see later in course) CPU Physical Cache TLB Primary Memory VA PA Alternative: place the cache before the TLB CPU VA (StrongARM) Virtual Cache PA TLB Primary Memory

26. February 25, 2010CS152, Spring 2010 26 Aliasing in Virtual-Address Caches VA 1 VA 2 Page Table Data Pages PA VA 1 VA 2 1st Copy of Data at PA 2nd Copy of Data at PA TagData Two virtual pages share one physical page Virtual cache can have two copies of same physical data. Writes to one copy not visible to reads of other! General Solution: Disallow aliases to coexist in cache Software (i.e., OS) solution for direct-mapped cache VAs of shared pages must agree in cache index bits; this ensures all VAs accessing same PA will conflict in direct- mapped cache (early SPARCs)

27. February 25, 2010CS152, Spring 2010 27 Concurrent Access to TLB & Cache Index L is available without consulting the TLB cache and TLB accesses can begin simultaneously Tag comparison is made after both accesses are completed Cases: L + b = kL + b k VPN L b TLB Direct-map Cache 2 L blocks 2 b -byte block PPN Page Offset = hit? DataPhysical Tag Tag VA PA Virtual Index k

28. February 25, 2010CS152, Spring 2010 28 Virtual-Index Physical-Tag Caches: Associative Organization Is this scheme realistic? VPN a L = k-b b TLB Direct-map 2 L blocks PPN Page Offset = hit? Data Phy. Tag VA PA Virtual Index k Direct-map 2 L blocks 2a2a = 2a2a After the PPN is known, 2 a physical tags are compared

29. February 25, 2010CS152, Spring 2010 29 Concurrent Access to TLB & Large L1 The problem with L1 > Page size Can VA 1 and VA 2 both map to PA ? VPN a Page Offset b TLB PPN Page Offset b Tag VA PA Virtual Index L1 PA cache Direct-map = hit? PPN a Data VA 1 VA 2

30. February 25, 2010CS152, Spring 2010 30 A solution via Second Level Cache Often, a common L2 cache backs up both Instruction and Data L1 caches L2 is “inclusive” of both Instruction and Data caches CPU L1 Data Cache L1 Instruction Cache Unified L2 Cache RF Memory

31. February 25, 2010CS152, Spring 2010 31 Anti-Aliasing Using L2: MIPS R10000 VPN a Page Offset b TLB PPN Page Offset b Tag VA PA Virtual Index L1 PA cache Direct-map = hit? PPN a Data VA 1 VA 2 Direct-Mapped L2 PA a 1 Data PPN into L2 tag Suppose VA1 and VA2 both map to PA and VA1 is already in L1, L2 (VA1  VA2) After VA2 is resolved to PA, a collision will be detected in L2. VA1 will be purged from L1 and L2, and VA2 will be loaded  no aliasing !

32. February 25, 2010CS152, Spring 2010 32 Virtually-Addressed L1: Anti-Aliasing using L2 VPN Page Offset b TLB PPN Page Offset b Tag VA PA Virtual Index & Tag Physical Index & Tag L1 VA Cache L2 PA Cache L2 “contains” L1 PA VA 1 Data VA 1 Data VA 2 Data “Virtual Tag” Physically-addressed L2 can also be used to avoid aliases in virtually- addressed L1

33. February 25, 2010CS152, Spring 2010 33 Atlas Revisited One PAR for each physical page PAR’s contain the VPN’s of the pages resident in primary memory Advantage: The size is proportional to the size of the primary memory What is the disadvantage ? VPN PAR’s PPN

34. February 25, 2010CS152, Spring 2010 34 Hashed Page Table: Approximating Associative Addressing hash Offset Base of Table + PA of PTE Primary Memory VPN PID PPN Page Table VPNd Virtual Address VPN PID DPN VPN PID PID Hashed Page Table is typically 2 to 3 times larger than the number of PPN’s to reduce collision probability It can also contain DPN’s for some non-resident pages (not common) If a translation cannot be resolved in this table then the software consults a data structure that has an entry for every existing page

35. February 25, 2010CS152, Spring 2010 35 Acknowledgements These slides contain material developed and copyright by: –Arvind (MIT) –Krste Asanovic (MIT/UCB) –Joel Emer (Intel/MIT) –James Hoe (CMU) –John Kubiatowicz (UCB) –David Patterson (UCB) MIT material derived from course 6.823 UCB material derived from course CS252