1. Virtual Memory

2. Virtual Memory
The main memory can act as a cache for the secondary storage
Advantages:
Illusion of having more physical memory
even a single user program can exceed the size of main memory.
Protection
safe and efficient sharing of memory.
Program relocation
Simplifies loading the program for execution. The relocation allows us to load the program into any location in main memory.

3. Basic Mechanism Virtual addresses Physical addresses
address translation
Disk addresses

4. Terminology Page: Virtual memory block.
Page fault: Virtual memory miss (i.e. a requested page is not in the main memory; retrieve it from the disk).
Physical address: The address that can directly be used to access main memory.
Virtual address: The address that the CPU produces, which must be translated to physical address.
Memory mapping or address translation: The process of calculating the physical address given the virtual address.

5. Address Translation Virtual Address Page size is 4 KB Physical Address
Virtual Page Number
Page Offset
……………… ……
Virtual Address
Translation
Page size is 4 KB
Physical Page Number
Page Offset
……………… ……
Physical Address

6. Page Fault A page fault can take millions of cycles to process.
Design choices:
Pages should be large enough to amortize the high access time. 4 KB to 16 KB are typical sizes. 32KB and 64KB pages are becoming popular.
embedded systems have 1KB page size
Fully associative placement of pages.
Page faults can be handled in software because the overhead is small compared to the disk access time.
Write-through will not work efficiently; instead write-back (copy-back) is used.

7. Page Tables Fully associative placement is used for pages.
Full search is impractical.
Pages are located using a full table, called page table, that indexes the memory.
Page table resides in memory and indexed with the page number from the virtual address.
Page table contains the corresponding physical page number.
Each program has its own page table, which maps virtual address space of the program to the main memory.

8. Page Tables
To indicate the location of the page table in memory, the hardware include a register, called page table register, that points to the start of the page table.
For sake of simplicity, assume for the time being that the page table is in a fixed and contiguous area of memory.
Page table contains a mapping for every possible virtual page
no tags are required.

9. If 0 then page is not in memory
Page Tables
Page table register
Virtual page number
Page offset … …
Virtual Address 20 12 V
Physical page number
Page Table
If 0 then page is not in memory 18
Physical page number
Page offset … …

10. Context Switching
The page table, PC, and the registers specify the state of a program, often referred as process.
The OS saves the state of a program in order to allow another program to use the CPU.
The OS makes a process active by loading the process’s state.
The process’s address space, and all the data it can access in memory, is defined by its page table, which resides in memory.
The OS loads the page table register to point to the page table of the process it wants to make active.

11. Page Faults
If a page fault occurs, the OS must be given control (through exception).
The OS finds the page in disk and decides where to place the requested page in main memory.
The location of the page on disk must also be kept track of in virtual memory systems.
The OS usually creates the space on disk for all the pages of a process when it creates the process (swap space)
It also creates a data structure to record where each virtual page stored on disk.
This data structure may be part of the page table or may be an auxiliary data structure indexed in the same way.

12. Finding the Virtual Pages1 V
Page Table
Physical page or disk address
Physical memory
Virtual page number
Disk Storage

13. Replacement Schemes in VM
The OS creates a data structure that records which processes and which virtual addresses use each physical page.
In a page fault if all the pages in main memory are in use, the OS must choose a page to replace.
LRU replacement scheme is usually implemented.
Every page has a use bit or reference bit which is set whenever a page is accessed.
The OS periodically clears reference bits.
If reference bit of a page is 0, it means that this page is not accessed during a particular period of time.

14. Size of Page Tables
With 32-bit virtual address, 4 KB pages, and 4 bytes per page table entry, the total page table size:
4 MB of memory for page table of each program in execution at any time
There may be tens to hundreds of active programs on a machine at the same time.

15. Smaller Page Tables
Page tables are stored in virtual memory rather than in physical memory
Page tables are also subject to paging.
Two-level table lookup (Pentium)
1024 PTE’s
Page Tables
Page table 0
Page table 1
Page table
1023
PDBR
PDE 0
PDE 1
PDE 1023
Page Directory

16. 3. Inverted Hash Table
Page table is as large as the size of the number of physical pages in main memory
VP #
Offset
Virtual Address
VP #
Entry
Page Table
Chain
Hashing
Hash Table
PP #
Offset

17. Writing to the Pages
Virtual memory systems use write-back scheme to write the updated pages to the disk.
A dirty bit is added to the page table to keep track whether a page has been written since it was read into memory.
When the OS chooses to replace a page, it checks the dirty bit to decide whether it is necessary to write back this page to the disk.

18. Translation-Lookaside Buffer (TLB)
Every memory access results in at least two accesses
One access to obtain the physical address
Second access to get the data
To improve, take advantage of locality of references to the page table
References to the words on that page have both temporal and spatial locality
Modern machines include a special cache, Translation-Lookaside Buffer (TLB), that keeps track of recently used translations.
A TLB is a cache that holds only page table mappings.

19. Translation-Lookaside Buffer (TLB)1 V
Tag
Physical page
number
TLB
VPN
Physical memory 1 V
Page Table
Physical page or disk address
Disk Storage

20. TLB Tag holds a portion of the virtual page numbers
Each data entry in TLB holds a physical page number.
TLB entries include additional information such as the reference bit and dirty bit.
When a miss occurs we need to determine if it is a page fault or merely a TLB miss.
If it is only a TLB miss, the CPU loads the translation from the page table into the TLB and tries the reference again.
TLB misses are much more frequent than the page faults, which are handled through exceptions.

21. TLB
When replacing an entry in TLB, only the reference and dirty bits of this entry are copied back to page table.
Typical values for a TLB
entries
Block size: 1-2 page table entries
Hit time: clock cycle
Miss penalty: clock cycles
Miss rate: 0.01% - 1%.
Fully-associative mapping in a TLB is not too costly when the TLB is small.
Random methods for replacing TLB entries provide reasonable performances.

22. Intrinsity FastMATH TLB
Virtual page number
Page offset … …
Virtual Address 20 = 12 V D
Tag
Physical page number
TLB
TLB
Hit
16 entry
Physical page number
Page offset
Physical address tag
Cache index
block
offset = 16 8 4 2
Tag
Data
Cache V
Cache Hit
Data 32

23. Reads & Writes Virtual address TLB Access TLB Hit? No TLB miss
exception
Write?
Yes
Physical address
Cache Hit?
Try to read data from cache No
Write access bit on?
Yes
Try to write
data to cache
Yes No
Write protection exception
Cache miss
stall No
Cache Hit?
write data into cache
update the dirty bit
Yes
Cache miss stall No
Yes
Deliver data to the CPU

24. If so, under what circumstances?
Hits & Misses
Cache
TLB
Virtual Memory
Possible?
If so, under what circumstances?
miss
hit
hit
Possible, although the page table is never really checked if TLB hits
hit
miss
hit
TLB misses, but the entry found in page table; after retry, data is found in the cache
miss
miss
hit
TLB misses, but entry found in the page table; after retry, data misses in cache.
miss
miss
miss
TLB misses, and followed by a page fault; after retry the data must miss in cache
miss
hit
miss
Impossible; cannot have a translation in TLB if page is not present in memory
hit
hit
miss
Impossible; cannot have a translation in TLB if page is not in memory
hit
miss
miss
Impossible; data cannot be allowed in cache if page is not in memory

25. Indexing Cache in VM 1/3 The cache is usually physically indexed.
Address translation
cache is accessed.
No need to flush cache content on context switch.
No aliasing problems.
No need for PID field.

26. MIPS Memory Allocation
sp 
7fff fffc
Stack
Dynamic Data
Static Data
gp 
Text
pc 
Reserved

27. Indexing Cache in VM 2/3
As an alternative, cache can be indexed by virtual address and it can be virtually tagged.
Address translation hardware (e.g. TLB) is not used.
When a cache miss occurs, address translation is needed.
PID is needed.
Two virtual addresses for the same page, aliasing, may occur (i.e. a word on such a page can be cached in two different locations).

28. Indexing Cache in VM 3/3
As a compromise, cache can be virtually indexed but physically tagged.
Address translation and cache indexing take place simultaneously.
Page address tag
page offset
31 …
Virtual Address
index
block offset
TLB
Cache
Physical Page no
hit + block number

29. Protection in VM
Each process has its own (virtual) address space and a process is not allowed to access other processes’ address spaces in normal circumstances.
OS guarantees that the independent virtual pages map to disjoint physical pages.
In order to implement this, a process shouldn’t be allowed to change table mapping.
The OS prevents a user process from modifying its own page tables.
The OS, however, must be able to do so.
Page tables are placed in the address space of OS

30. Protection in VM The hardware provides three basic capabilities:
Support at least two mode:
User process
OS process (kernel, supervisor or executive process).
Portion of CPU state inaccessible by user processes
user/supervisor bit, page table pointer, TLB
Provide a mechanism whereby the CPU can go from user mode to supervisor mode, and vice versa.
System call exception implemented as a special instruction (e.g. syscall in MIPS)
Return from exception (ERET) instruction to restore the state of the process that generated exception.

31. Data Sharing & Context Switch
When two processes share information in a limited way, the OS must assist them.
Write access bit is used to restrict the sharing to just reading.
Access bits must be included both in TLB and page table.
Context (process) switching from P1 to P2
It suffices to change Page Table Register to point to P2’s page table.
But the TLB must be reloaded with entries for P2.
A process (task) identifier (PID) is added to VA.
8-bit ID in Intrinsity FastMATH
A PID is sometimes included in cache for similar reasons.

32. Handling TLB Miss
A new TLB entry is created retrieving the physical address from the page table
Can be handled either in software or hardware.
MIPS does in software (exception)
Takes about 13 clock cycles
TLBmiss:
mfc0 $k1, Context #copy address of PTE into $k1
lw $k1, 0($k1) #put PTE into $k1
mtc0 $k1, EntryLo #put PTE into EntryLo
tlbwr #put EntryLo into TLB at random
eret #return from TLB miss exception

33. Page Fault The page is not in memory.
detected if the valid bit in page table entry is off.
Handled in software.
Page fault must be recognized in the same clock cycle as the memory access;
otherwise lw would destroy the content of the destination register.
In the next clock cycle, the exception must start.
In an exception due to a page fault, the OS saves the entire state of the active process:
all the general-purpose integer and floating-point registers, the page table register, the EPC, the exception Cause register.

34. Exceptions Due to Page Faults
Three steps are involved:
Find the location of the referenced page on disk
Choose a physical page to replace
If the chosen page is dirty, write it back to the disk.
Bring the referenced page from disk to memory.
The second and last steps may take millions of cycles  context switch
MIPS instructions are restartable.
But, in some processors, restartable instructions are impossible (e.g. block move instructions)