1. Virtual Memory
CSE451
Andrew Whitaker

2. Problem: Physical Memory Scarcity
Q: Which is bigger:
A 64 bit address space
Or, the number of hydrogen molecules in a star?
A: It’s the star
2^64 bytes in an address space
10^57 hydrogen atoms in a star
57 * log 10 > 64 * log 2
But, the fact we have to ask is significant!

3. Solution: Virtual Memory
Physical memory stores a subset of virtual address space
The rest is stored on disk
Main memory acts as a page cache
Implemented transparently by the OS and hardware
Application can’t tell which pages are in memory
virtual
memory
physical
memory

4. How Does Virtual Memory Work?1 1 1 2 20 V R M
prot
page frame number
Page table entry contains a valid bit
Says whether the mapping is valid
If valid bit is not set, the system raises a page fault
Behaves like an (involuntary) system call

5. What Happens on a Page Fault?
Hardware raises an exception
OS page fault handler runs
First, make sure the address is valid
Second, verify access type
Not allowed to write to a read-only page!
If access is legal, allocate a new page frame
What happens if physical memory is scarce… stay tuned!
OS reads page frame from disk
Process/thread is blocked during this process
Once read completes, OS updates the PTE and resumes the process/thread

6. Process Startup Two options for new processes:
Eagerly bring in pages
Lazily fault in pages
Called demand paging
Most OS’s prefer demand paging
Doesn’t require guessing or maintaining history
Slightly smarter approach: clustering
Bring in the faulting page and several subsequent pages

7. Dealing With Memory Scarcity
Physical memory is over-allocated
We can run out!
OS needs a page replacement strategy
Before diving into particular strategies, lets look at some theory…

8. Locality of Reference
Programs tend to reuse data and instructions they have used recently
These “hot” items are a small fraction of total memory
Rule of thumb: 90% of execution time is spent in 10% of the code
This is what makes caching work!

9. The Working Set Model
The working set represents those pages currently in use by a program
The working set contents change over time
So does the size of the working set
To get reasonable performance, a program must be allocated enough pages for its working set

10. A hypothetical web server
working
set
Not linear! Performance decays rapidly without enough memory
Request / second of throughput
Number of page frames allocated to process

11. Thrashing
Programs with an allocation beneath their working set will thrash
Each page fault evicts a “hot” page
That page will be needed soon…
Evicted page takes a page fault
[repeat]
Net result: no work gets done
All time is devoted to paging operations

12. Over-allocation
Giving a program more than its working set does very little good
Page eviction strategies take advantage of this

13. Generalizing to Multiple Processes
Let W to be the sum of working sets for all active processes
Let M be amount of physical memory
If W > M, the system as a whole will thrash
No page replacement policy will work :-(
If W <= M, the system might perform well
Key issue: is the page replacement policy smart enough to identify working sets?

14. Belady’s Algorithm
Evict the page that won’t be used for the longest time in the future
This page is probably not in the working set
If it is in the working set, we’re thrashing
This is optimal!
Minimizes the number of page faults
Major problem: this requires a crystal ball
We can’t “know” future memory sequence

15. Temporal Locality Assume the past is a decent indicator of the future
How good are these algorithms:
LIFO
Newest page is kicked out
FIFO
Oldest page is kicked out
Random
Random page is kicked out
LRU
Least recently used page is kicked out

16. Paging Algorithms Random is also pretty bad LIFO is horrendous
LRU is pretty good
FIFO is mediocre
VAX VMS used a form of FIFO because of hardware limitations

17. Implementing LRU: Approach #1
One (bad) approach:
on each memory reference:
long timeStamp = System.currentTimeMillis();
sortedList.insert(pageFrameNumber,timeStamp);
Problem: this is too inefficient
Time stamp + data structure manipulation on each memory operation
Too complex for hardware

18. Making LRU Efficient Use hardware support Trade off accuracy for speed
Reference bit is set when pages are accessed
Can be cleared by the OS
Trade off accuracy for speed
It suffices to find a “pretty old” page 1 1 1 2 20 V R M
prot
page frame number
Note: we don’t know the order of use of the referenced bits. So, our LRU estimate will definitely be approximate

19. Approach #2: LRU Approximation with Reference Bits
For each page, maintain a set of reference bits
Let’s call it a reference byte
Periodically, shift the HW reference bit into the highest-order bit of the reference byte
Suppose the reference byte was
If the HW bit was set, the new reference bit become
Frame with the lowest value is the LRU page

20. Analyzing Reference Bits
Pro: Does not impose overhead on every memory reference
Interval rate can be configured
Con: Scanning all page frames can still be inefficient
e.g., 4 GB of memory, 4KB pages => 1 million page frames

21. Approach #3: LRU Clock Use only a single bit per page frame
Basically, this is a degenerate form of reference bits
On page eviction:
Scan through the list of reference bits
If the value is zero, replace this page
If the value is one, set the value to zero

22. Why “Clock”?
Typically implemented with a circular queue 1 1 1 1

23. Analyzing Clock Pro: Very low overhead
Only runs when a page needs evicted
Takes the first page that hasn’t been referenced
Con: Isn’t very accurate (one measly bit!)
Degenerates into FIFO if all reference bits are set
Pro: But, the algorithm is self-regulating
If there is a lot of memory pressure, the clock runs more often (and is more up-to-date)

24. When Does LRU Do Badly? Example: Many database workloads:
LRU performs poorly when there is little temporal locality: 1 2 3 4 5 6 7 8
Example: Many database workloads:
SELECT *
FROM Employees
WHERE Salary < 25000