1. Virtual Memory and Paging
Brian Kocoloski
CSE 422S - Operating Systems Organization
Washington University in St. Louis
St. Louis, MO 63130

2. Resource Virtualization
What resources do you use when you write programs?
The processor(s)
Memory
Files
Data (global variables, things like hardcoded strings)
What do you not have to worry about generally?
Existence of other processes
State of hardware resources
CSE 422S – Operating Systems Organization

3. Resource Virtualization
The operating system virtualizes physical hardware resources into virtual resources in the form of processes
Physical Memory
Physical Processors
CSE 422S – Operating Systems Organization

4. Resource Virtualization
Process 1
Process 2
Processor Virtualization:
Two (or more) processes may share the same set of physical CPUs
How to share them?
Context switching + CPU scheduling
Memory Virtualization:
Translate private memory addresses to physical addresses
Virtual memory
Physical Memory
Physical Processors
CSE 422S – Operating Systems Organization

5. CSE 422S – Operating Systems Organization
Virtual Memory
What is the purpose of virtual memory?
Give processes illusion of full dedicated access to entire system memory
Every process’ virtual address space is unique and independent from all other processes
Similar to how a process thinks it has dedicated access to processors
Preserve physical resources until they are actually needed
Many programs don’t need all of their code/data in memory at once (untaken branches, unused data, etc.)
Allocating virtual memory != allocating physical memory
Unify the differences in physical memory devices behind a common abstraction
Simplifies application programming
Similar to how ’processes’ and ‘threads’ exist on all different architectures despite core-level differences
CSE 422S – Operating Systems Organization

6. CSE 422S – Operating Systems Organization
Memory Pages
All physical memory is divided into pages
Page sizes are determined by the architecture
E.g. on both x86 and ARM, pages are 4KB in size
Total amount of physical memory: X bytes
Total number of pages: X/4KB
Physical Address:
0x0
0x1000
0x2000
0x3000
4KB
4KB
4KB
4KB
4KB
4KB …
4KB
CSE 422S – Operating Systems Organization

7. CSE 422S – Operating Systems Organization
Memory Pages
On all modern architectures, memory is accessed virtually via virtual addresses
Virtual addresses require translation
Virtual Address Translation:
Process A
Process B
Virtual Address
4KB
4KB
4KB
4KB
4KB
4KB
4KB
4KB
Virtual Memory
Page #
Offset
4KB
4KB
4KB
4KB
4KB
4KB
4KB
4KB
4KB
4KB
Physical Memory
Physical Page
Phys.
Addr +
CSE 422S – Operating Systems Organization

8. /proc/<pid>/maps
The valid virtual addresses in a process
$ cat /proc/self/maps
Executable code
Read-only static data
Read/write static data
CSE 422S – Operating Systems Organization

9. /proc/<pid>/maps
The valid virtual addresses in a process
$ cat /proc/self/maps
CSE 422S – Operating Systems Organization

10. /proc/<pid>/maps
The valid virtual addresses in a process
heap:
0x – 0x
0x21000 in length, or 132KB
stack:
0x7e7f6000 – 0x7e817000
0x21000 in length, also 132KB
CSE 422S – Operating Systems Organization

11. Memory Allocation/Mapping
In Linux, there are two primary system calls that perform memory allocation
brk() – expand the process’ heap
mmap() – similar to heap, but can have “holes” in the address space, and do other things like memory map files
CSE 422S – Operating Systems Organization

12. /proc/<pid>/maps
The valid virtual addresses in a process
$ cat /proc/self/maps
Anonymous Memory
(mmap)
CSE 422S – Operating Systems Organization

13. CSE 422S – Operating Systems Organization
Paging
malloc(…)
0x001b…
User space
Virtual address (x = 0x001b …)
# of bytes
Allocate new virtual memory area (VA)
Allocate physical memory pages (PA)
Update page tables to map VA -> PA
Kernel space
Page Tables
VA (x)
PA (y)
Hardware
CSE 422S – Operating Systems Organization

14. CSE 422S – Operating Systems Organization
Paging
write(“Blah”, 0x001b…);
malloc(…)
0x001b…
User space
Virtual address (x = 0x001b …)
# of bytes
Allocate new virtual memory area (VA)
Allocate physical memory pages (PA)
Update page tables to map VA -> PA
Address 0x001b
Kernel space
Page Tables
MMU
Hardware
CSE 422S – Operating Systems Organization

15. CSE 422S – Operating Systems Organization
Paging
write(“Blah”, 0x001b…);
malloc(…)
0x001b…
User space
Virtual address
# of bytes
Allocate new virtual memory area (VA)
Allocate physical memory pages (PA)
Update page tables to map VA -> PA
Address 0x001b
Kernel space
Page Tables
Physical Memory
MMU
Address 0x001b
Hardware
CSE 422S – Operating Systems Organization

16. CSE 422S – Operating Systems Organization
Demand Paging
malloc(…)
0x001b…
User space
Virtual address
# of bytes
Allocate new virtual memory area (VA)
DO NOT allocate physical memory
DO NOT update page tables X
Kernel space
Page Tables X
Hardware
CSE 422S – Operating Systems Organization

17. CSE 422S – Operating Systems Organization
Demand Paging
write(“Blah”, 0x001b…);
malloc(…)
0x001b…
User space
Virtual address
# of bytes
Allocate new virtual memory area (VA)
DO NOT allocate physical memory
DO NOT update page tables X
Kernel space
Page Tables
Page Tables X
MMU
Address 0x001b
Hardware
CSE 422S – Operating Systems Organization

18. CSE 422S – Operating Systems Organization
Demand Paging
write(“Blah”, 0x001b…);
malloc(…)
0x001b…
User space
Virtual address
PAGE FAULT
MMU cannot find VA
(0x001b ...) in the page tables
# of bytes
Allocate new virtual memory area (VA)
DO NOT allocate physical memory
DO NOT update page tables X
Kernel space
Page Tables
Page Tables X
MMU
Address 0x001b
Hardware
CSE 422S – Operating Systems Organization

19. CSE 422S – Operating Systems Organization
Demand Paging
write(“Blah”, 0x001b…);
User space
Query list of vm_area_struct for the process
If no VMA exists, invalid
If VMA exists, but no write permission, invalid
Else, OK
Page fault handler
Can user access VA 0x001b …?
Kernel space
Page Tables
Page Tables
MMU
Address 0x001b
Hardware
CSE 422S – Operating Systems Organization

20. CSE 422S – Operating Systems Organization
Demand Paging
write(“Blah”, 0x001b…);
Segmentation fault
User space
(1) Allocate physical pages and map them in
the page tables
(2) Reissue instruction
Page fault handler
Can user access VA 0x001b …? No
Kernel space
Page Tables
Page Tables
Yes
write(“Blah”, 0x001b…);
Physical Memory
MMU
Address 0x001b
Hardware
CSE 422S – Operating Systems Organization

21. CSE 422S – Operating Systems Organization
Address Translation
A virtual address is broken up into 2 pieces:
p: page number
d: page offset
64 bit virtual address space
4096 (4KB) byte page frame
12 bits to reference bytes within page frame
(2^12 = 4096)
That leaves 52 bits for p
Virtual Address Translation:
Virtual Address
Page #
Offset
Physical Page
Phys.
Addr +
CSE 422S – Operating Systems Organization

22. CSE 422S – Operating Systems Organization
Address Translation
Q: With 52 bits for the page number, how many different page numbers can we address?
In other words, how big must our translation table be?
Virtual Address Translation:
Virtual Address
Page number (52 bits)
Translation Table
Physical Page
CSE 422S – Operating Systems Organization

23. CSE 422S – Operating Systems Organization
Address Translation
Q: With 52 bits for the page number, how many different page numbers can we address?
A: 2^52
Q: How to design a lookup table that can efficiently perform translations of up to 2^52 different virtual page numbers?
Solution: multi-level page tables
Virtual Address Translation:
Virtual Address
Page number (52 bits)
Translation Table
Physical Page
CSE 422S – Operating Systems Organization

24. Multi-Level Paging … Virtual Address Translation: Virtual Address
Top level lookup directory
Physical Page
Second level lookup directory
Third level lookup directory …
CSE 422S – Operating Systems Organization

25. CSE 422S – Operating Systems Organization
Multi-Level Paging
Goal: We want each page table to fit into one page (4096 bytes) of memory
Proposed scheme:
Page Frame: 12 bits (2^12 = 4KB)
Top level page number: 9 bits
2nd level page number: 9 bits
3rd level page number: 9 bits
4th level page number: 9 bits
Unused: 16 bits
Note: These page numbers are really offsets into other page tables!
CSE 422S – Operating Systems Organization

26. CSE 422S – Operating Systems Organization
Example
Example 0x0000 cd c90
( c d c )
Top level page number (9 bits)
: 342
2nd level page number (9 bits)
: 0
3rd level page number (9 bits)
: 274
4th level page number (9 bits)
: 6
Page offset (12 bits)
: 2704
CSE 422S – Operating Systems Organization

27. Control Register (x86: CR3;
Example
CPU instruction: WR R1, 0x0000 cd c90
Step 1: Get Top-level page table address
Control Register (x86: CR3;
ARM: CP15 c2)
MMU
WR 0x0000 cd c90
Top level page table address (a)
CSE 422S – Operating Systems Organization

28. CSE 422S – Operating Systems Organization
Example 0x0000 cd c90
( ) a
Index
MetaData (r/w/x)
(present)
Page Address for next level of page table …
342
Present: 0
511
Hardware sees present bit set to 0: it CANNOT complete the VA -> PA translation
So it raises an exception called a PAGE_FAULT
CSE 422S – Operating Systems Organization

29. CSE 422S – Operating Systems Organization
Page Fault Handler
Hardware tells the kernel that a process tried to access a virtual address that either
Has not associated physical address in the page table
Or, accessed the memory in an invalid fashion (i.e., tried to write to read-only memory)
Kernel asks:
(1) Is this a valid virtual address for the process to be accessing?
i.e., if it’s a read, is the memory readable? If write, writable? If execution, executable?
(2) If not, raise SIGSEGV – the dreaded Segmentation Fault
(3) Else, update the page tables so the hardware knows how to complete the translation
CSE 422S – Operating Systems Organization

30. CSE 422S – Operating Systems Organization
Example 0x0000 cd c90
( ) a
Index
MetaData (r/w/x)
(present)
Page Address for next level of page table …
342
Present: 0
511
Hardware sees present bit set to 0: it CANNOT complete the VA -> PA translation
So it raises an exception called a PAGE_FAULT
CSE 422S – Operating Systems Organization

31. Kernel Page fault handler
Example 0x0000 cd c90
( ) a
idx P
Page
Address …
342 1 b
511
Kernel Page fault handler
Check if access is valid
Allocate a new page, say p
Determine physical address of, say b
Store b in page address of page table
Set present bit to 1
Hardware sees present bit set to 0: it CANNOT complete the VA -> PA translation
So it raises an exception called a PAGE_FAULT
CSE 422S – Operating Systems Organization

32. CSE 422S – Operating Systems Organization
Example 0x0000 cd c90
( ) a
idx P
Page
Address …
342 1 b
511 b
idx P
Page
Address …
511
Hardware sees present bit set to 0: it CANNOT complete the VA -> PA translation
So it raises an exception called a PAGE_FAULT
CSE 422S – Operating Systems Organization

33. CSE 422S – Operating Systems Organization
Example 0x0000 cd c90
( ) a
idx P
Page
Address …
342 1 b
511 b
idx P
Page
Address 1 c …
511 c
idx P
Page
Address …
274
511
Hardware sees present bit set to 0: it CANNOT complete the VA -> PA translation
So it raises an exception called a PAGE_FAULT
CSE 422S – Operating Systems Organization

34. CSE 422S – Operating Systems Organization
Example 0x0000 cd c90
( )
Index
Page …
342 b
511
Grab the
2704’th byte
in this page
Physical Page (4KB) a
Index
Page c …
511
Index
Page …
274 d
511
Index
Page … 6 e
511
CSE 422S – Operating Systems Organization

35. Kernel vs User Virtual Memory
When page tables are filled in on-demand like this, this is referred to as demand paging
By default, Linux uses demand paging for all user processes
Keeps memory allocation (virtual, i.e., malloc()) fast
Doesn’t allocate physical memory until process tries to access it
However, kernel virtual memory is always pre-paged
When the system boots, all physical memory is mapped into the kernel virtual address space
CSE 422S – Operating Systems Organization

36. Kernel vs User Virtual Memory
Kernel Virtual Memory
Physical Memory
Mappings created at runtime via memory allocation (e.g., mmap()) and page fault handling
Mappings created at system boot time
CSE 422S – Operating Systems Organization

37. Kernel vs User Virtual Memory
Given a physical address, how do I calculate the kernel virtual address to access it?
__va(unsigned long pa);
On arm 7 (R Pi 3): ((unsigned long)((x) - PHYS_OFFSET + PAGE_OFFSET))
PHYS_OFFSET: first address of physical memory
PAGE_OFFSET: (On our 32-bit R Pis) 0x
The point is, reverse PA->VA translation can be done in O(1) time
(You did this in studio 12)
Given a user virtual address, how would the kernel calculate its physical address?
Given a physical address, how do I calculate the user virtual address to access it?
CSE 422S – Operating Systems Organization