1. Virtual Memory Hardware

2. Outline Introduction to virtual memory Paging MMU Page tables
In these slides, we talk about virtual memory hardware

3. Problems with Simple Memory Mgmt
A simple memory management scheme allocates contiguous physical memory to programs
Problems
Growing a program requires copying entire program
Wastes memory due to internal and external fragmentation
Running a program requires loading entire program in memory
Maximum program size is limited by memory size
These problems occur because programs are allocated contiguous memory

4. Non-Contiguous Memory Allocation
Can we use non-contiguous memory for programs?
It is hard to program with non-contiguous memory (why?)
Challenge is to hide non-contiguous memory from programs
hard to program: think about an array. The reason it is cheap to access an array randomly (say a[i], for any ‘i’), is that the elements of the array are assumed to be in contiguous memory. Hence accessing any element requires some simple address calculations, which can be done in O(1) time. If the elements were not in contiguous memory, one would need a linked list like traversal to access the elements of the array, requiring O(n) time.

5. Use Memory Management H/W
Insight: Take advantage of MMU h/w to translate CPU virtual addresses to physical memory addresses
Program can use contiguous virtual addresses
Physical memory is allocated non-contiguously
1. Programs use virtual
memory addresses
2. CPU sends these virtual addresses to MMU
3. MMU translates virtual address to physical memory address
4. MMU accesses memory using physical addresses

6. Example: MMU with 2 Base Registers
Consider an MMU with two base and limit registers
Glossary
V = virtual address
P = physical address
B1 = base 1, L1 = limit 1
B2 = base 2, L2 = limit 2
Virtual address space
0 <= V < (L1 + L2)
Address translation
P = V + B1, when 0 <= V < L1
P = (V – L1) + B2, when L1 <= V < (L1 + L2)
Limit 2 P1
Base 2
Limit 1 P1
Base 1
Here P1 is a single program. It is located in 2 physical regions, but its virtual address space is contiguous. This is possible based on the address translation mechanism.

7. Paging MMU Problems with two base-register MMU
Limited: supports only two physical memory regions
Slow: with N registers, address translation needs O(N) time
A paging MMU supports a very large number of non-contiguous physical memory regions efficiently
Virtual address space consists of contiguous, fixed size chunks called pages
Page size = 2n bytes, fixed by paging MMU, e.g., 4KB
Paging MMU maps each page to a physical memory region called a frame
frame size = page size
frame = map(page)
To understand how paging works, we need to understand the format of virtual and physical addresses

8. Virtual and Physical Address
Virtual address = (page number, offset)
On 32 bit machine, virtual address space size = 232 bytes = 4GB
page size = 212 = 4KB, nr. of pages = 220
Physical address = (frame number, offset)
Say, physical memory size = 230 bytes = 1GB
frame size = 212 = 4KB, nr. of frames = 218
bit 0
bit 31
20 bits
12 bits
offset
page number
Low order n bits are the byte offset in the page/frame
Remaining high order bits are the page number/frame number
Note that the virtual and physical address space size may or may not be the same but the page size = frame size. As a result, the nr. of pages and frames may be different.
bit 0
bit 29
18 bits
12 bits
offset
frame number

9. Paging MMU Operation On each memory reference:
CPU produces vaddr = (page nr, offset)
Memory sees paddr = (frame nr, offset)
MMU maps vaddr to paddr, i.e., frame = map(page)
Note that offset remains the same
MMU

10. MMU translates virtual to physical address
Paging Example
Page 2^20 - 1
Stack
MMU
Frame 2^18 - 1
Unallocated pages
Unused frame
Heap
frame for another
address space
In this example, we have chosen page size = 2^12 (4 KB). The virtual address space is 2^32 bytes (on a 32 bit architecture). So the number of pages are 2^32/2^12 = 2^20.
Notice that the number of frames (e.g., 2^18) is unrelated to the number of pages.
Data
Text
Page 1
Frame 1
Page 0
Frame 0
Virtual
address space
MMU translates virtual to physical address
Physical
address
space

11. Contiguous Memory Allocation
Benefits of Paging
Contiguous Memory Allocation
Paging
Growing a program requires copying entire program
Growing a program requires allocating a page at a time
Wastes memory due to internal and external fragmentation
No external fragmentation, internal fragmentation is ½ page per region
Running a program requires loading entire program in memory
As program runs, pages can be loaded in memory (we will see this later)
Maximum program size is limited by memory size
Maximum program size is limited by disk size (we will see this later)
why is there no external fragmentation: because pages are of fixed size, and allocation is done at page granularity. so if a page is available, it can be allocated.

12. Page Tables
Paging MMU maintains its mapping information in a page table
A page table contains page table entries (PTE) that map a page to a frame
Typically, each entry is one word
E.g, 32 bits on 32 bit architecture, 64 bits on 64 bit architecture
Each entry contains frame number and various bits such as valid/invalid, writeable, dirty, etc.
Valid bit says whether mapping is valid or not
We will discuss how these bits are used later

13. Linear Page Table frames in memory page number offset
32 bit virtual address
frames in memory
20 bits
12 bits
page number
offset
Example architecture
Virtual address size: 32 bits
Page size: 212 B (4KB)
Example translation
vaddr = 0x01005a6f
offset = vaddr & 0xfff = 0xa6f
page = vaddr >> 12 = 0x1005
fr = page_table[page].frame = 0xe
paddr = (fr << 12) | offset = 0xea6f
0x1005
0xa6f 31 13 12
frame number
unused C D R W V
0xe000
PTE
The various bits of the PTE are described later.
In this example, page number is 0x1005. So we access the 0x1005 entry of the page table starting from PTR (0 index array). The frame number stored there is 0xe (15). So we access the 15th frame in physical memory (with address 0xe000, because the frame size is 12 bits). Then we add the offset to the frame address: 0xe xa6f = 0xea6f, which is the physical memory address associated with virtual address 0x1005a6f
0xe v
0x1005
page_table
(PTR)
Single-level
page table

14. Storing Page Table Page table is large and stored in memory Problem
MMU needs to access page table on every instruction
Each memory access requires two memory accesses
First one for page table entry in physical memory
Where is that shown in the example translation in previous slide?
Second one for the physical memory location being accessed
Solution
Cache PTE in MMU, handle misses from physical memory
Cache is called translation lookaside buffer (TLB)
Discussed later
example translation in previous slide: when reading (load instruction) page_table[page]. While the previous figure shows that we are reading this value twice, we would read it once into a register, and use then use it twice.

15. Accessing Page Tables
MMU needs to know the location of page table in physical memory
Paging MMU has a page table register (PTR) that stores the location of the start of the page table
Similar to base register
OS associates a separate page table for each process
Thus each process has its own address space
Implementing context switch
context switch = thread switch + address space switch
OS implements address space switching by changing PTR to the start of the appropriate page table for each process
each process needs its own page table because the page->frame mappings are different for each process. e.g., page 0 in two different processes maps to two different frames.

16. Page Table Size Page table size Effect of page size on performance
#PTE = #pages =
= virtual address space size / page size = 232 / 212 = 220
Typically, each PTE is word sized (4B)
Page table size = #PTE * PTE size = 220 * 4B = 4 MB
Each process needs 4MB for its own page table!
Effect of page size on performance
Smaller size => lower internal fragmentation
Larger size => fewer PTE
lower memory overhead for PTE, but more internal fragmentation
better performance (we will see this later)

17. Reducing Page Table Size
Consider a small process with one page of text, data and stack each
It requires 3 pages of memory (12KB)
It requires 4MB of memory for page tables!
With the linear page table design, we need a PTE even when the corresponding page is not in use
Notice that most of the page table entries has invalid
3 pages are valid, (220 – 3) entries are invalid
i.e., the page table array is sparely populated
Need a more space efficient data structure that avoids storing invalid entries
Use a tree data structure, instead of an array
A parent does not need to store a child sub-tree if all elements of the sub-tree are invalid

18. Multi-Level Page Table
32 bit virtual address
frames in memory
12 bits
10 bits
PT1
PT2
offset 4 5
0xa6f
0xe v
0xe000 5
0xa000
a multi-level page table is a tree data structure
In a two-level page table, the page number is broken into two parts, PT1 and PT2. PT1 (e.g., 4) serves as an index into the top (or first) level page table. The PTE that is obtained contains a frame number (e.g., 0xa shown above) that helps locate the frame at which the second level page table resides in physical memory (at address 0xa000). Then PT2 (e.g., 5) is used to index into the second level page table. This PTE contains the desired frame number (e.g., 0xe).
Note that there is one top level page table but there can be zero or more second level page tables.
In this example, the first level page table has 210 page table entries because PT1 has 10 bits. Similarly, each second level page table also has 210 page table entries because PT2 has 10 bits. In this example, since the first level page table has 210 entries, there can be as many as 210 second-level page tables. However, a second-level page table is not allocated when all its entries are empty.
You can think of this as a tree where each node has 210 child pointers. The top-level page table is the root of the tree. When a second-level page table does not exist, the corresponding first-level page table entry is marked invalid, similar to the first-level page entry being a NULL pointer.
0xa 4
PTR
Top-level
page table
Second level
page tables

19. Example Translation vaddr = 0x1005a6f offset = vaddr & 0xfff = 0xa6f
get 12 low bits of addr
pg2 = (vaddr >> 12) & 0x3ff = 0x5
get next 10 bits of addr
pg1 = vaddr >> (12+10) = 0x4
get top 10 bits of addr
pt_1 = page_table_register
base of top page table
pt_2 = pt_1[pg1].frame << 12 = 0xa000
look up top page table to find physical memory location of second level page table. pt_2 is stored page aligned
fr = pt_2[pg2].frame = 0xe
look up second page table
paddr = (fr << 12) | offset = 0xea6f
generate physical address
pt_1[pg1] and pt_2[pg2] are memory accesses

20. Comparing Single & Two-Level Page Table
Is address translation faster with a single-level page table or a two-level page table?
How does two-level page table save space compared to the single-level page table?
Not all pages within an address space are allocated
E.g., consider the region between heap and stack
This region does not need any frames
So there is no need to maintain mapping information
When a 2nd level page table is entirely invalid, it is not allocated, and corresponding entry in top-level page table is marked invalid
address translation: Single-level page table is faster. Single level requires one extra memory access for each memory access. Two level requires two extra memory accesses for each memory access. In previous slide, the two memory accesses are pt_1[pg1], and pt_2[pg2].

21. Summary
Contiguous memory allocation makes it hard to grow programs and causes external fragmentation
It is easier to allocate memory non-contiguously, but it is easier to program a contiguous address space
We can solve this dilemma by using MMU hardware to virtualize the address space of a process
Process sees contiguous (virtual) memory addresses
MMU translates these addresses to physical memory addresses that may be non-contiguous

22. Summary
A paging MMU breaks the virtual address space of a process into fixed size pages and maps each page to a physical memory frame
Enables growing programs at page granularity
No external fragmentation
Page mapping information is stored in memory
Single level page table stores mappings in an array
Inefficient because address space is generally sparse
Multi-level page table stores mappings in a tree structure
Reduces memory overhead significantly, but
Requires more time for looking up physical address

23. Think Time What is the purpose of a page table?
Where is the page table located?
How does the h/w locate the page table?
How many address bits are used for page offset when page size is 2KB?
Discuss advantages and disadvantages of linear and multi-level page tables
purpose of page table - stores virtual to physical page mappings
page table located - it is stored in physical memory
h/w locate page table – the OS programs a register called the page table register (PTR) to point to the start of the page table. the address stored in the PTR is a physical address. The MMU looks up the PTR to find the start of the page table in physical memory.
page size is 2KB – 11 bits for page offset
locate page table - using a dedicated register, e.g., page table register
linear page table: fast, large memory overhead