Virtual Memory Acknowledgment These slides are based on the slides of Prof. Weatherspoon of Cornell University and Dr Hutton and Dr Sheehan of Auckland University

Recommended Reading Patterson & Hennessy, Computer Organization & Design: The Hardware/Software Interface, Morgan Kaufmann, 5th edition (Chapter 5.7 and any other sections that appear relevant) (4th edition, 3rd or 2nd edition are also OK) Dr. Hutton’s Lecture Notes

Issues to be addressed How can the main memory be shared by multiple processes? Can a computer execute a program that needs more main memory than the computer has?

Agenda Virtual memory Translate a virtual memory address to a physical memory address Page Tables Translation lookaside buffer

Shared Memory Multiprocessors Shared Memory Multiprocessor (SMP) Typical (today): 2 – 8 cores each HW provides single physical address space for all processors Core0 Core1 Core2 Core3 Cache Cache Cache Cache Interconnect Memory I/O

Processor & Memory Memory CPU 0xfff…f 0x7ff…f Stack $$ Heap When a single program is executed, the program has the perception that it can use all the available memory Data Text 0x000…0 Memory

Multiple Processes Q: What happens when another program is executed concurrently on another processor? A: The addresses will conflict Even though, CPUs may take turns using memory bus CPU 0xfff…f 0x7ff…f Stack Stack $$ $$ Heap Heap CPU Data Data Text Text 0x000…0 Memory

Solution? Multiple processes/processors We relocate second program but… What if they don’t fit? What if not contiguous? Need to recompile/relink? … Stack CPU Data Stack Heap Heap CPU Data Text Text Memory

Solution? Multiple processes/processors A better solution: Map a Virtual Address (generated by CPU) to a Physical Address (in memory) automatically Stack CPU Data Stack Heap Heap CPU Data Text Text Memory

Not enough memory Some programs were too big to run in the memory available to them on their computers. 4GB main memory Some big data analysis programs need much bigger memory to run Something had to be done to enable programmers to work with memory which appeared to be larger than the memory available on their computers.

Virtual Memory vs Physical Memory Virtual Memory is an abstraction of the memory module. It appears to exist as main memory It corresponds to storage for information, without regard to its exact physical location. It supports multi-tasking---the ability to run more than one process at a time Physical memory corresponds to the physical chips in the computer.

Virtual Memory Each process has its own virtual address space A process is an instance of the execution of a program. Programmer can code as if they own all of memory Program/CPU can access any address from 0 .. 2N -1 (e.g. N=32) On-the-fly at runtime, for each memory access given in terms of virtual address, the memory management unit (MMU): translate fake virtual address to a real physical address redirect load/store to the physical address

Each virtual page fits one physical page (frame). To make it easier to map virtual memory to physical memory, the virtual memory is divided into blocks. It might be difficult to find a contiguous physical memory area to fit in a complete virtual address space Each block is called a page. The physical memory is also divided into chunks of the same size as the pages. These are called frames (physical page). Each virtual page fits one physical page (frame).

Physical Address Space B C Virtual Address Space Physical Address Space D Address Translation

Address Space Programs load/store to virtual addresses CPU CPU A X B Y C Z Virtual Address Space Virtual Address Space Physical Address Space Programs load/store to virtual addresses Actual memory uses physical addresses

Address Space Programs load/store to virtual addresses X CPU CPU C A X B B Y C Z Z MMU MMU Y Virtual Address Space Virtual Address Space A Physical Address Space Programs load/store to virtual addresses Actual memory uses physical addresses Memory Management Unit (MMU) Responsible for allocating virtual pages to physical pages

Address Space Programs load/store to virtual addresses X CPU CPU 0x1000 C 0x1000 A X B B Y C Z Z MMU MMU Y Virtual Address Space Virtual Address Space A Physical Address Space Programs load/store to virtual addresses Actual memory uses physical addresses Memory Management Unit (MMU) Responsible for allocating virtual pages to physical pages Responsible for translating on the fly Essentially, just a big array of integers: paddr = PageTable[vaddr];

Quiz Which of the following statements are correct? The size of a virtual memory address space must be exactly the same as the size of a physical memory address space. Physical memory corresponds to the main memory of a computer system. Virtual memory and physical memory are divided into pages. The order of the virtual pages must be preserved when the pages are loaded into the physical memory. On modern computer systems, programmers must specify the mapping between virtual pages and physical pages. On modern computer systems, programmers must make sure that their programs can fit into the main memory of their computers.

Quiz The memory management unit translates the virtual addresses to physical addresses. The mappings of the virtual pages to physical pages are stored in a page table. Using virtual memory makes the sharing of the physical memory by multiple programs easier.

Agenda Virtual memory Translate a virtual memory address to a physical memory address Page Tables Translation lookaside buffer

Size of Page Table How large is a PageTable? Granularity? Per byte… Per block… 232 = 4GB virtual memory

Size of Page Table How large should a PageTable be for a MMU? Granularity? Per byte… Per block… 232 = 4GB virtual memory 232 -> Need 4 billion entry PageTable!

Size of Page Table How large should a PageTable be for a MMU? Granularity? Per byte … Per block… Typical: 4KB – 16KB pages 232 = 4GB virtual memory e.g. 232 / 4 KB = 232 / 212 = 220 220 -> 1 million entry PageTable is better

Address Translation Page Offset Virtual page number 31 12 11 0 Each page is 4KB virtual memory address 0x00000000 0x00002000 0x00001000 0x00003000 0xFFFFF000 A virtual address can be partitioned into two parts, i.e virtual page number, page offset.

A physical address consists of two parts, i A physical address consists of two parts, i.e physical page number, page offset. A B Page Table virtual memory address physical page number 0x0000000 0x0002000 0x0001000 0x0003000 0xFFFF000 Disk A 0xFFFFF000 0x00003000 0xFFFF 0x00002000 Disk B 0x00001000 0x0003 0x00000000 0x0001

Address Translation For any access to virtual address: CPU generated 31 12 11 0 Page Offset Virtual page number vaddr For any access to virtual address: Extract virtual page number and page offset

Address Translation For any access to virtual address: CPU generated Page Offset Virtual page number 31 12 11 0 vaddr Lookup in PageTable Main Memory Physical page number For any access to virtual address: Extract virtual page number and page offset Lookup physical page number at PageTable[vpn]

Address Translation For any access to virtual address: CPU generated Page Offset Virtual page number 31 12 11 0 vaddr Lookup in PageTable Main Memory Physical page number Page offset paddr 12 11 0 For any access to virtual address: Extract virtual page number and page offset Lookup physical page number at PageTable[vpn] Form physical address as ppn:offset

Simple PageTable CPU MMU Assume each page is 4KB Read Mem[0x00002538] 0xC20A3000 CPU MMU 0x90000000 Assume each page is 4KB 0x4123B000 0x10045000 0x10044000 0x00000000

Simple PageTable CPU MMU Read Mem[0x00002538] PageOffset 0xC20A3000 0x4123B000 0x10045000 0x10044000 0x00000000

Simple PageTable CPU MMU Read Mem[0x00002538] PageOffset VPN: virtual page number 0xC20A3000 CPU MMU 0x90000000 0x4123B000 0x10045000 0x10044000 0x00000000

Simple PageTable CPU MMU Read Mem[0x00002538] PageOffset VPN: virtual page number 0xC20A3000 CPU MMU Q: Where to store page tables? A: In memory, … Special page table base register 0x90000000 PTBR 0x4123B000 0x10045000 0x10044000 0x00000000

Simple PageTable vaddr Physical Page Number 0x10045 0xC20A3 0x4123B vpn pgoff 0x10045000 vaddr 0x10044000 0x00000000 PTBR

Simple PageTable vaddr Physical Page Number 0x10045 0xC20A3 0x4123B 31 12 11 0x00002 0x538 0x10045000 vaddr 0x10044000 0x00000000 PTBR

Simple PageTable vaddr Physical Page Number 0x10045 0xC20A3 0x4123B 31 12 11 0x00002 0x538 0x10045000 vaddr 0x10044000 0x00000000 PTBR

Simple PageTable vaddr paddr Physical Page Number 0x10045 0xC20A3 0x4123B 0x10044 0xC20A3000 0x90000000 0x4123B000 31 12 11 0x00002 0x538 0x10045000 vaddr 0x10044000 0x4123B 0x538 paddr 0x00000000 PTBR

Simple PageTable vaddr paddr Physical Page Number 0x10045 0xC20A3 0x4123B 0x10044 0xC20A3000 0x90000000 0x4123B000 31 12 11 0x00002 0x538 0x10045000 vaddr 0x10044000 0x4123B 0x538 paddr 0x00000000 PTBR

Quiz Which of the following statements are correct? Assume that the virtual address consists of 32 bits, and the size of each virtual page is 8KB. The virtual page number field consists of 19 bits. Each entry of the page table is an address of the physical memory. Assume that the virtual address consists of 32 bits, the size of each virtual page is 8KB, the physical address consists of 30 bits, and the first five entries of the page table are as below. What is the physical address that corresponds to virtual address 0x00004321. 0x00891 0x1812E 0x0891A 0x16789 0x12345 4 3 2 1

Agenda Virtual memory Translate a virtual memory address to a physical memory address Page Tables Translation lookaside buffer

Invalid Pages Not every page is used. Some pages are not loaded into the physical memory due to inactivity or lack of space Inst. Data Heap unused Stack virtual address space

Invalid Pages Need valid bit for each page table entry V Physical Page Number Disk B 1 0x10045 0xC20A3 0x4123B 0x10044 Disk A 0xC20A3000 0x90000000 0x4123B000 Need valid bit for each page table entry A B 0x10045000 0x10044000 0x00000000

Page Permissions V R W X Physical Page Number 1 0x10045 0xC20A3 0x4123B 0x10044 0xC20A3000 0x90000000 0x4123B000 Keep R, W, X permission bits for each page table entry Make code read-only, executable; make data read-write but not executable; etc. 0x10045000 0x10044000 0x00000000

Page Permissions V KR KW KX UR UW UX Physical Page Number 1 0x10045 0xC20A3 0x4123B 0x10044 0xC20A3000 0x90000000 0x4123B000 The page tables of many modern systems contain many more permission bits to allow complicate control to be carried out. 0x10045000 0x10044000 0x00000000

Aliasing Map several virtual addresses to the same physical page W X Physical Page Number 1 0xC20A3 0x4123B 0x10044 0xC20A3000 0x90000000 0x4123B000 Map several virtual addresses to the same physical page Make different views of same data with different permissions 0x10045000 0x10044000 0x00000000

Paging/Swapping For some programs, the physical memory is not larger enough to hold all the instructions and the data of the programs. Memory Pyramid Disk (Many GB – few TB) Memory (128MB – few GB) L2 Cache (½-32MB) RegFile 100s bytes L1 Cache (several KB)

Paging/Swapping Paging allows us run process larger than physical memory. View memory as a “cache” for secondary storage (disk) Swap memory pages out to disk when not in use Page them back in when needed A page fault occurs when the accessed location is not in the physical memory Null entry: illegal address (the location is not allocated to the program to used) On disk: bring in the page from the disk Use Temporal/Spatial Locality Pages used recently most likely to be used again soon If a swapped out page is paged into the physical memory later, it might be stored at a different physical address.

Paging/Swapping V R W X D Physical Page Number invalid 1 0x10045 invalid 1 0x10045 disk sector 200 disk sector 25 0x00000 0xC20A3000 0x90000000 0x4123B000 0x10045000 When a page is swapped out, the contents of the page is written back to the disk if the page has been modified. Need more bits: Dirty, RecentlyUsed, … Why not use write-through? 0x00000000 200 25

Sharing Physical Memory Each process has its own page table. The page table base register always points to the starting address of the page table of the process that is currently running. When the CPU switches to execute another process, the operating system sets the page table base register to point to the page table of the process that is scheduled to execute.

Process B is executing B’s page table A’s page table PTBR 200 25

CPU switches to execute process A B’s page table PTBR A’s page table 200 25

Quiz Which of the following statements are correct? An invalid entry in the page table means the corresponding page has not be allocated to the program to use. The permission bits in a page table entry indicate the operations that can be carried out on the corresponding page. All computer systems have three permission bits in their page tables. Page aliasing means mapping several physical addresses to the same virtual page. Before a program starts its execution, all the instructions and the data of the program must be loaded into the main memory of the computer. A page fault occurs if the CPU tries to access an address that is not currently mapped to the main memory.

Quiz Page faults are all caused by accessing invalid addresses. A virtual page is always mapped to the same physical memory address. Virtual memory uses the write-back policy for handling update operations. If a page is not currently used by a program, it might be swapped out of the physical memory. The dirty bit in a page table entry indicates whether the page has been read recently. Most computer architectures have a page table base register that points to the starting address of the page table of the process that is currently running. When the CPU switches to execute another process, the page table base register is changed to point to the page table of the new process.

Agenda Virtual memory Translate a virtual memory address to a physical memory address Page Tables Translation lookaside buffer

Performance Every load/store is translated to physical addresses Every load/store needs two memory accesses Page table miss: load a swapped-out page and retry instruction, or kill program Memory is already slow, translation makes it slower Use a cache to improve the performance of address translation

Translation Lookaside Buffer (TLB) Hardware Translation Lookaside Buffer (TLB) A small, very fast cache of recent address mappings TLB hit: avoids PageTable lookup TLB miss: do PageTable lookup, cache result for later use

TLB Diagram V R W X D tag ppn V R W X D invalid 1

A TLB in the Memory Hierarchy Lookup Cache CPU Mem Disk PageTable Lookup Check TLB for vaddr (~ 1 cycle)

A TLB in the Memory Hierarchy Lookup Cache CPU Mem Disk PageTable Lookup Check TLB for vaddr (~ 1 cycle) TLB Hit compute paddr, send to cache

A TLB in the Memory Hierarchy Lookup Cache CPU Mem Disk PageTable Lookup Check TLB for vaddr (~ 1 cycle) TLB Hit compute paddr, send to cache TLB Miss: look up PageTables for vaddr

A TLB in the Memory Hierarchy Lookup Cache CPU Mem Disk PageTable Lookup Check TLB for vaddr (~ 1 cycle) TLB Hit compute paddr, send to cache TLB Miss: look up PageTables for vaddr PageTable has valid entry for in-memory page Load PageTable entry into TLB; try again (tens of cycles)

A TLB in the Memory Hierarchy Lookup Cache CPU Mem Disk PageTable Lookup Check TLB for vaddr (~ 1 cycle) TLB Hit compute paddr, send to cache TLB Miss: look up PageTables for vaddr PageTable has valid entry for in-memory page Load PageTable entry into TLB; try again (tens of cycles) PageTable has entry for swapped-out (on-disk) page Page Fault: load from disk, fix PageTable, try again (millions of cycles)

A TLB in the Memory Hierarchy Lookup Cache CPU Mem Disk PageTable Lookup Check TLB for vaddr (~ 1 cycle) TLB Hit compute paddr, send to cache TLB Miss: look up PageTables for vaddr PageTable has valid entry for in-memory page Load PageTable entry into TLB; try again (tens of cycles) PageTable has entry for swapped-out (on-disk) page Page Fault: load from disk, fix PageTable, try again (millions of cycles) PageTable has invalid null entry Page Fault: kill process

The TLB is a fast cache for address translations The TLB is a fast cache for address translations. A TLB hit is fast, miss is slow.

TLB Parameters TLB parameters (typical) very small (64 – 256 entries), so very fast fully associative, or at least set associative

TLB Coherency PageTable contents change swapping/paging activity Page Table Base Register changes Different processes are executed Flush TLB when PTBR changes (context switch) and invalidate entry when PTE changes

Quiz TLB is a cache for speeding up address translation. If a CPU cannot find a data item at a given address in the memory cache, the CPU uses the TLB to look up the physical address of the data item. In a TLB miss, reading a data item from the memory needs at least two memory accesses. If a page fault is caused by an invalid page table entry, the OS would kill the process that generates the page fault. The contents of TLB remain valid when the value in the page table base register is changed. Changes in the page table of a process might cause some of the entries in the TLB become invalid.