1. Virtual Memory Acknowledgment
These slides are based on the slides of Prof. Weatherspoon of Cornell University and Dr Hutton and Dr Sheehan of Auckland University

2. Recommended Reading
Patterson & Hennessy, Computer Organization & Design: The Hardware/Software Interface, Morgan Kaufmann, 5th edition (Chapter 5.7 and any other sections that appear relevant) (4th edition, 3rd or 2nd edition are also OK) Dr. Hutton’s Lecture Notes

3. Issues to be addressed
How can the main memory be shared by multiple processes? Can a computer execute a program that needs more main memory than the computer has?

4. Agenda
Virtual memory
Translate a virtual memory address to a physical memory address
Page Tables
Translation lookaside buffer

5. Shared Memory Multiprocessors
Shared Memory Multiprocessor (SMP)
Typical (today): 2 – 8 cores each
HW provides single physical address space for all processors
Core0
Core1
Core2
Core3
Cache
Cache
Cache
Cache
Interconnect
Memory
I/O

6. Processor & Memory
Memory
CPU
0xfff…f
0x7ff…f
Stack $$
Heap
When a single program is executed, the program has the perception that it can use all the available memory
Data
Text
0x000…0
Memory

7. Multiple Processes
Q: What happens when another program is executed concurrently on another processor?
A: The addresses will conflict
Even though, CPUs may take
turns using memory bus
CPU
0xfff…f
0x7ff…f
Stack
Stack $$ $$
Heap
Heap
CPU
Data
Data
Text
Text
0x000…0
Memory

8. Solution? Multiple processes/processors
We relocate second program
but…
What if they don’t fit?
What if not contiguous?
Need to recompile/relink? …
Stack
CPU
Data
Stack
Heap
Heap
CPU
Data
Text
Text
Memory

9. Solution? Multiple processes/processors
A better solution:
Map a Virtual Address (generated by CPU) to a Physical Address (in memory) automatically
Stack
CPU
Data
Stack
Heap
Heap
CPU
Data
Text
Text
Memory

10. Not enough memory
Some programs were too big to run in the memory available to them on their computers.
4GB main memory
Some big data analysis programs need much bigger memory to run
Something had to be done to enable programmers to work with memory which appeared to be larger than the memory available on their computers.

11. Virtual Memory vs Physical Memory
Virtual Memory is an abstraction of the memory module.
It appears to exist as main memory
It corresponds to storage for information, without regard to its exact physical location.
It supports multi-tasking---the ability to run more than one process at a time
Physical memory corresponds to the physical chips in the computer.

12. Virtual Memory Each process has its own virtual address space
A process is an instance of the execution of a program.
Programmer can code as if they own all of memory
Program/CPU can access any address from N -1 (e.g. N=32)
On-the-fly at runtime, for each memory access given in terms of virtual address, the memory management unit (MMU):
translate fake virtual address to a real physical address
redirect load/store to the physical address

13. Each virtual page fits one physical page (frame).
To make it easier to map virtual memory to physical memory, the virtual memory is divided into blocks.
It might be difficult to find a contiguous physical memory area to fit in a complete virtual address space
Each block is called a page.
The physical memory is also divided into chunks of the same size as the pages. These are called frames (physical page).
Each virtual page fits one physical page (frame).

14. Physical Address SpaceB C
Virtual Address
Space
Physical Address Space D
Address Translation

15. Address Space Programs load/store to virtual addresses
CPU
CPU A X B Y C Z
Virtual Address
Space
Virtual Address
Space
Physical Address Space
Programs load/store to virtual addresses
Actual memory uses physical addresses

16. Address Space Programs load/store to virtual addressesX
CPU
CPU C A X B B Y C Z Z
MMU
MMU Y
Virtual Address
Space
Virtual Address
Space A
Physical Address Space
Programs load/store to virtual addresses
Actual memory uses physical addresses
Memory Management Unit (MMU)
Responsible for allocating virtual pages to physical pages

17. Address Space Programs load/store to virtual addressesX
CPU
CPU
0x1000 C
0x1000 A X B B Y C Z Z
MMU
MMU Y
Virtual Address
Space
Virtual Address
Space A
Physical Address Space
Programs load/store to virtual addresses
Actual memory uses physical addresses
Memory Management Unit (MMU)
Responsible for allocating virtual pages to physical pages
Responsible for translating on the fly
Essentially, just a big array of integers: paddr = PageTable[vaddr];

18. Quiz Which of the following statements are correct?
The size of a virtual memory address space must be exactly the same as the size of a physical memory address space.
Physical memory corresponds to the main memory of a computer system.
Virtual memory and physical memory are divided into pages.
The order of the virtual pages must be preserved when the pages are loaded into the physical memory.
On modern computer systems, programmers must specify the mapping between virtual pages and physical pages.
On modern computer systems, programmers must make sure that their programs can fit into the main memory of their computers.

19. Quiz
The memory management unit translates the virtual addresses to physical addresses.
The mappings of the virtual pages to physical pages are stored in a page table.
Using virtual memory makes the sharing of the physical memory by multiple programs easier.

20. Agenda
Virtual memory
Translate a virtual memory address to a physical memory address
Page Tables
Translation lookaside buffer

21. Size of Page Table How large is a PageTable? Granularity? Per byte…
Per block…
232 = 4GB virtual memory

22. Size of Page Table How large should a PageTable be for a MMU?
Granularity?
Per byte…
Per block…
232 = 4GB virtual memory
232 -> Need 4 billion entry PageTable!

23. Size of Page Table How large should a PageTable be for a MMU?
Granularity?
Per byte …
Per block…
Typical:
4KB – 16KB pages
232 = 4GB virtual memory
e.g. 232 / 4 KB = 232 / 212 = 220
220 -> 1 million entry PageTable is better

24. Address Translation
Page Offset
Virtual page number
Each page is 4KB
virtual memory address 0x 0x 0x 0x
0xFFFFF000
A virtual address can be partitioned into two parts, i.e virtual page number, page offset.

25. A physical address consists of two parts, i
A physical address consists of two parts, i.e physical page number, page offset. A B
Page Table
virtual memory address
physical page number 0x 0x 0x 0x
0xFFFF000
Disk A
0xFFFFF000 0x
0xFFFF 0x
Disk B 0x
0x0003 0x
0x0001

26. Address Translation For any access to virtual address:
CPU
generated
Page Offset
Virtual page number
vaddr
For any access to virtual address:
Extract virtual page number and page offset

27. Address Translation For any access to virtual address:
CPU
generated
Page Offset
Virtual page number
vaddr
Lookup in PageTable
Main
Memory
Physical page number
For any access to virtual address:
Extract virtual page number and page offset
Lookup physical page number at PageTable[vpn]

28. Address Translation For any access to virtual address:
CPU
generated
Page Offset
Virtual page number
vaddr
Lookup in PageTable
Main
Memory
Physical page number
Page offset
paddr
For any access to virtual address:
Extract virtual page number and page offset
Lookup physical page number at PageTable[vpn]
Form physical address as ppn:offset

29. Simple PageTable CPU MMU Assume each page is 4KB Read Mem[0x00002538]
0xC20A3000
CPU
MMU 0x
Assume each page is 4KB
0x4123B000 0x 0x 0x

30. Simple PageTable CPU MMU Read Mem[0x00002538] PageOffset 0xC20A3000
0x4123B000 0x 0x 0x

31. Simple PageTable CPU MMU Read Mem[0x00002538] PageOffset
VPN: virtual page number
0xC20A3000
CPU
MMU 0x
0x4123B000 0x 0x 0x

32. Simple PageTable CPU MMU
Read Mem[0x ]
PageOffset
VPN: virtual page number
0xC20A3000
CPU
MMU
Q: Where to store page tables? A: In memory, … Special page table base register 0x
PTBR
0x4123B000 0x 0x 0x

33. Simple PageTable vaddr Physical Page Number 0x10045 0xC20A3 0x4123B
vpn
pgoff 0x
vaddr 0x 0x
PTBR

34. Simple PageTable vaddr Physical Page Number 0x10045 0xC20A3 0x4123B31 12 11
0x00002
0x538 0x
vaddr 0x 0x
PTBR

35. Simple PageTable vaddr Physical Page Number 0x10045 0xC20A3 0x4123B31 12 11
0x00002
0x538 0x
vaddr 0x 0x
PTBR

36. Simple PageTable vaddr paddr Physical Page Number 0x10045 0xC20A3
0x4123B
0x10044
0xC20A3000 0x
0x4123B000 31 12 11
0x00002
0x538 0x
vaddr 0x
0x4123B
0x538
paddr 0x
PTBR

37. Simple PageTable vaddr paddr Physical Page Number 0x10045 0xC20A3
0x4123B
0x10044
0xC20A3000 0x
0x4123B000 31 12 11
0x00002
0x538 0x
vaddr 0x
0x4123B
0x538
paddr 0x
PTBR

38. Quiz Which of the following statements are correct?
Assume that the virtual address consists of 32 bits, and the size of each virtual page is 8KB. The virtual page number field consists of 19 bits.
Each entry of the page table is an address of the physical memory.
Assume that the virtual address consists of 32 bits, the size of each virtual page is 8KB, the physical address consists of 30 bits, and the first five entries of the page table are as below. What is the physical address that corresponds to virtual address 0x
0x00891
0x1812E
0x0891A
0x16789
0x12345 4 3 2 1

39. Agenda
Virtual memory
Translate a virtual memory address to a physical memory address
Page Tables
Translation lookaside buffer

40. Invalid Pages Not every page is used.
Some pages are not loaded into the physical memory due to inactivity or lack of space
Inst.
Data
Heap
unused
Stack
virtual address space

41. Invalid Pages Need valid bit for each page table entry V
Physical Page Number
Disk B 1
0x10045
0xC20A3
0x4123B
0x10044
Disk A
0xC20A3000 0x
0x4123B000
Need valid bit for each page table entry A B 0x 0x 0x

42. Page Permissions V R W X
Physical Page Number 1
0x10045
0xC20A3
0x4123B
0x10044
0xC20A3000 0x
0x4123B000
Keep R, W, X permission bits for each page table entry Make code read-only, executable; make data read-write but not executable; etc. 0x 0x 0x

43. Page Permissions V KR KW KX UR UW UX
Physical Page Number 1
0x10045
0xC20A3
0x4123B
0x10044
0xC20A3000 0x
0x4123B000
The page tables of many modern systems contain many more permission bits to allow complicate control to be carried out. 0x 0x 0x

44. Aliasing Map several virtual addresses to the same physical pageW X
Physical Page Number 1
0xC20A3
0x4123B
0x10044
0xC20A3000 0x
0x4123B000
Map several virtual addresses to the same physical page
Make different views of same data with different permissions 0x 0x 0x

45. Paging/Swapping
For some programs, the physical memory is not larger enough to hold all the instructions and the data of the programs.
Memory Pyramid
Disk (Many GB – few TB)
Memory (128MB – few GB)
L2 Cache (½-32MB)
RegFile
100s bytes
L1 Cache
(several KB)

46. Paging/Swapping
Paging allows us run process larger than physical memory.
View memory as a “cache” for secondary storage (disk)
Swap memory pages out to disk when not in use
Page them back in when needed
A page fault occurs when the accessed location is not in the physical memory
Null entry: illegal address (the location is not allocated to the program to used)
On disk: bring in the page from the disk
Use Temporal/Spatial Locality
Pages used recently most likely to be used again soon
If a swapped out page is paged into the physical memory later, it might be stored at a different physical address.

47. Paging/Swapping V R W X D Physical Page Number invalid 1 0x10045
invalid 1
0x10045
disk sector 200
disk sector 25
0x00000
0xC20A3000 0x
0x4123B000 0x
When a page is swapped out, the contents of the page is written back to the disk if the page has been modified.
Need more bits:
Dirty, RecentlyUsed, …
Why not use write-through? 0x
200 25

48. Sharing Physical Memory
Each process has its own page table.
The page table base register always points to the starting address of the page table of the process that is currently running.
When the CPU switches to execute another process, the operating system sets the page table base register to point to the page table of the process that is scheduled to execute.

49. Process B is executing
B’s page table
A’s page table
PTBR
200 25

50. CPU switches to execute process A B’s page table PTBR A’s page table
200 25

51. Quiz Which of the following statements are correct?
An invalid entry in the page table means the corresponding page has not be allocated to the program to use.
The permission bits in a page table entry indicate the operations that can be carried out on the corresponding page.
All computer systems have three permission bits in their page tables.
Page aliasing means mapping several physical addresses to the same virtual page.
Before a program starts its execution, all the instructions and the data of the program must be loaded into the main memory of the computer.
A page fault occurs if the CPU tries to access an address that is not currently mapped to the main memory.

52. Quiz Page faults are all caused by accessing invalid addresses.
A virtual page is always mapped to the same physical memory address.
Virtual memory uses the write-back policy for handling update operations.
If a page is not currently used by a program, it might be swapped out of the physical memory.
The dirty bit in a page table entry indicates whether the page has been read recently.
Most computer architectures have a page table base register that points to the starting address of the page table of the process that is currently running.
When the CPU switches to execute another process, the page table base register is changed to point to the page table of the new process.

53. Agenda
Virtual memory
Translate a virtual memory address to a physical memory address
Page Tables
Translation lookaside buffer

54. Performance Every load/store is translated to physical addresses
Every load/store needs two memory accesses
Page table miss: load a swapped-out page and retry instruction, or kill program
Memory is already slow, translation makes it slower
Use a cache to improve the performance of address translation

55. Translation Lookaside Buffer (TLB)
Hardware Translation Lookaside Buffer (TLB)
A small, very fast cache of recent address mappings
TLB hit: avoids PageTable lookup
TLB miss: do PageTable lookup, cache result for later use

56. TLB Diagram V R W X D
tag
ppn V R W X D
invalid 1

57. A TLB in the Memory Hierarchy
Lookup
Cache
CPU
Mem
Disk
PageTable
Lookup
Check TLB for vaddr (~ 1 cycle)

58. A TLB in the Memory Hierarchy
Lookup
Cache
CPU
Mem
Disk
PageTable
Lookup
Check TLB for vaddr (~ 1 cycle)
TLB Hit
compute paddr, send to cache

59. A TLB in the Memory Hierarchy
Lookup
Cache
CPU
Mem
Disk
PageTable
Lookup
Check TLB for vaddr (~ 1 cycle)
TLB Hit
compute paddr, send to cache
TLB Miss: look up PageTables for vaddr

60. A TLB in the Memory Hierarchy
Lookup
Cache
CPU
Mem
Disk
PageTable
Lookup
Check TLB for vaddr (~ 1 cycle)
TLB Hit
compute paddr, send to cache
TLB Miss: look up PageTables for vaddr
PageTable has valid entry for in-memory page
Load PageTable entry into TLB; try again (tens of cycles)

61. A TLB in the Memory Hierarchy
Lookup
Cache
CPU
Mem
Disk
PageTable
Lookup
Check TLB for vaddr (~ 1 cycle)
TLB Hit
compute paddr, send to cache
TLB Miss: look up PageTables for vaddr
PageTable has valid entry for in-memory page
Load PageTable entry into TLB; try again (tens of cycles)
PageTable has entry for swapped-out (on-disk) page
Page Fault: load from disk, fix PageTable, try again (millions of cycles)

62. A TLB in the Memory Hierarchy
Lookup
Cache
CPU
Mem
Disk
PageTable
Lookup
Check TLB for vaddr (~ 1 cycle)
TLB Hit
compute paddr, send to cache
TLB Miss: look up PageTables for vaddr
PageTable has valid entry for in-memory page
Load PageTable entry into TLB; try again (tens of cycles)
PageTable has entry for swapped-out (on-disk) page
Page Fault: load from disk, fix PageTable, try again (millions of cycles)
PageTable has invalid null entry
Page Fault: kill process

63. The TLB is a fast cache for address translations
The TLB is a fast cache for address translations. A TLB hit is fast, miss is slow.

64. TLB Parameters TLB parameters (typical)
very small (64 – 256 entries), so very fast
fully associative, or at least set associative

65. TLB Coherency PageTable contents change
swapping/paging activity
Page Table Base Register changes
Different processes are executed
Flush TLB when PTBR changes (context switch) and invalidate entry when PTE changes

66. Quiz TLB is a cache for speeding up address translation.
If a CPU cannot find a data item at a given address in the memory cache, the CPU uses the TLB to look up the physical address of the data item.
In a TLB miss, reading a data item from the memory needs at least two memory accesses.
If a page fault is caused by an invalid page table entry, the OS would kill the process that generates the page fault.
The contents of TLB remain valid when the value in the page table base register is changed.
Changes in the page table of a process might cause some of the entries in the TLB become invalid.