1. Virtual Memory CSC/EE/CPE 3760 Dr. Timothy Heil WINTER 2018 OMH 232
Some slides/material courtesy Dr. Kevin Bolding. All copyright Seattle Pacific University.
CSC/EE/CPE Dr. Timothy Heil
WINTER OMH 232
MW 3:00 PM – 5:00 pm (206)
OMH 246

2. “640KB ought to be enough for anybody.“*
Running Out of Memory
“640KB ought to be enough for anybody.“*
*Bill gates probably didn’t say this…
Memory is like money
More never seems to be enough!

3. Current Recommendations
Digital Trends – Jan 23, 2018
2GB: Only really found in budget tablet designs. Fine for them, but you’ll want more in a laptop or desktop.
4GB: Entry level memory that comes with even budget notebooks. Fine for Windows and Chrome OS.
8GB: Excellent for Windows and MacOS systems and most gaming settings. We recommend this for most people.
16GB: Ideal for professional work and the most demanding games.
32GB and beyond: Enthusiasts and purpose-built workstations only.
Servers
Typically 64GB to 256GB

4. Extending the Hierarchy
Registers
CPU
Load or I-Fetch
Store
Main
Memory
Cache
(Multi-level)
Cache: About $10/MB (can’t buy off the shelf!)
1/3 to 10ns access time
2-32MB typical (total)
Dominant technology: SRAM
Memory: About 0.94₵/MB
90ns (random) access time
4-16GB typical
Dominant technology: DRAM
Extend the hierarchy
Main memory acts like a cache for…
Disk: About 0.003₵/MB, 10ms (10,000,000 ns) access time
1-2TB typical
Dominant technology: magnetic disk
Flash (solid state drive): ₵, 100us (100,000ns)
Disk

5. Virtual Memory
Idea: Memory holds portions of program that are currently needed
Currently unused data is saved on disk
Appears as a very large virtual memory
Technically limited only by the disk size, but practically typically limited to 2x to 4x the size of DRAM
Advantages:
Programs that require large amounts of memory can be run
Unless they need it all at once
Multiple programs can be in virtual memory at once
Disadvantages:
The memory a program needs may all be on disk
The operating system has to manage virtual memory

6. The Virtual Memory Concept
Virtual Memory Space
Virtual Memory Space: All possible memory addresses (4GB in 32-bit systems, 16 Exabytes for 64-bit!) All that can be conceived
Disk Swap Space: Area on hard disk that can be used as an extension of memory. (Typically 1-4GB) All that can be used
Disk Swap Space
Main Memory: Physical memory (DRAM)
(Typically 8-16GB on a 64-bit system) All that physically exists
Main Memory

7. The Virtual Memory Concept
This address can be conceived of, but doesn’t correspond to any memory. Accessing it will produce an error.
Virtual Memory Space
Disk Swap Space
Main Memory
This address can be accessed. However, it currently is only on disk and must be read into main memory before being used. A table maps from its virtual address to the disk location.
Error
This address can be accessed immediately since it is already in memory. A table maps from its virtual address to its physical address. There will also be a backup location on disk.
Disk Address: Not in main memory
Physical Address: Disk Address:

8. Virtual vs. Physical Addresses
Memory locations now have two aliases
Virtual Address – The “public” alias for a memory location, used by almost all code and systems
Virtual addresses are used by almost all software
These are “fake” addresses
Physical Address – The “secret but real” name for a memory location, used only by hardware and the operating system
Physical addresses refer to the true location in memory
The only addresses that actually work with hardware
The OS must manage the aliases, converting virtual addresses to physical on the fly
Red/blue pill screen cap copyright Warner Bros.

9. Accessing Memory with a Virtual Address
In high-level languages and assembly alike, we use Virtual Addresses
Memory needs Physical Addresses!
Steps to accessing memory with a virtual address
1. Convert the virtual address to a physical address
Is the data we’re looking for in memory/disk or only on disk?
Page table keeps track
If the table indicates the data is on disk and not in physical memory
 Read the location from the disk into memory (may require eviction!)
2. Do the memory access using the physical address
Check the cache first (note: cache uses only physical addresses)
Update the cache if needed
Sound familiar?

10. Important Point
Data is not stored in the Page Table! Data is in memory, or on disk
Page table stores the address translation
Address Translation via Page Table
CPU, Code use Virtual Addresses
Main Memory: Physical Addresses
Disk

11. Making Virtual Memory Work
VM systems typically have a miss (page fault) rate of %
Making Virtual Memory Work
Virtual memory (V.M.) is like a cache system
Main memory is a cache for virtual memory
Differences
The miss penalty is huge (millions of cycles)
 Increase the “block size” to be 4KB or larger
Disk transfers have a large startup time, but data transfer is relatively fast after started
Blocks in V.M. are called pages
V.M. system has entries for all possible locations
Miss  V.M. system keeps track of where data is on disk
Hit  V.M. system keeps track of physical address in main memory, not the actual data
Saves room (one address rather than 8KB of data)

12. Virtual  Physical Addresses
Example
4GB (32-bit) Virtual Address Space
i.e., all memory we *could* have
32MB (25-bit) Physical Address Space
i.e., memory installed in the system
8 KB (13-bit) page size
Like block size in cache
(index)
No tag - All entries are unique
Virtual Page Number
Page Offset 12 13 31
Translation
Note: may involve reading from disk
Physical Page Number
Page Offset 12 13 24
A 32-bit virtual address is given to the V.M. hardware
The virtual page number (index) is derived from this by removing the page (block) offset
The Virtual Page Number is looked up in a page table
When found, entry is either:
The physical page number, if in memory
The disk address, if not in memory (a page fault)
If not found, the address is invalid

13. Virtual Memory: 8KB page size,16MB Memory12 13 31
Virtual Address 13 19
219=512K
4GB / 8KB = 512K entries
Page offset
Virt. Page #
Phys. Page #
Disk Address
Virt. Pg.# V 1 2
512K-1
... 11 12 13 23
Physical Address

14. VM Example
System with 20-bit V.A., 16KB pages, 256KB of physical memory
Page offset takes 14 bits, 6 bits for V.P.N. and 4 bits for P.P.N.
Page Table:
Access to:
Virtual Page # Valid Physical Page #/ (index) Bit Disk address sector sector 4323… sector
PPN = 0010
Physical Address:
Access to:
sector xxxx...
PPN = Page Fault to sector 1
1010
Pick a page to “kick out” of memory (use LRU)
Read data from sector 1239 into PPN 1010
Assume LRU is VPN for this example.

15. Page Table Computations
Example
32GB Virtual Address Space
1GB Physical Address Space
16 KB Pages
log2(32GB) = 35b
35 – 14 = 21b
log2(16KB) = 14b
Virtual Page Number
Page Offset
log2(1GB) = 30b
= 16b
Physical Page Number

16. Issues A slot in the page table for every possible virtual address?
With 8KB (213) pages, there are 2(32-13) = 512K entries for a 32-bit V.A.
Each entry takes around 15 bits (round up to 16 bits…)
That’s 1MB for the page table! Space on the CPU chip is already limited
Solutions
Put the page table itself in main memory rather than on the CPU chip
Make the page table only big enough for the amount of memory being used

17. I’ve Made a Huge Mistake
Each memory access has becomes two accesses
Read the page table for the translation
Do the actual access
But the page table is in memory
Off to Demaray hall!
Ok, 270 cycles for every translation is not going to work!
What do you think we should do?
Solution: Cache the page table entries in a special cache
The Translation Lookaside Buffer (TLB) is just a cache that holds recently accessed page table entries
A TLB hit means that we don’t have to actually have to look in the page table
GOB image copyright Fox, or whoever owns Arrested Development these days…

18. TLB Design The TLB stores everything in the page table
Physical page number
Valid bit, dirty bit
We want the TLB to have a high hit rate
Fortunately, pages are huge, providing super-high locality
A single entry “covers” 4KB of space, with only a small entry
TLB usually only has a small number of entries (i.e., 64) and is fully-associative
Typical hit rates are 98.0 to 99.9%

19. Now we know what those TLB thingys are!
How many do you see?

20. What Do Memory Access Look Like Now?
Ideal:
Miss in cache:
Miss in TLB:
Miss in TLB, page fault
Virtual Address
Virtual Address
Virtual Address
Virtual Address
TLB
TLB
TLB
TLB
Hit in TLB, hit in PT! Translate to physical address
Hit in TLB, hit in PT! Translate to physical address
Miss, pull page table from memory
Miss, pull page table from memory
L1 Cache
L1 Cache
Hit in PT, translate to physical address
Miss in PT (page fault), pull from disk (to memory), potentially write dirty page back…
Miss
Hit!
Look in L1, etc.
L2 Cache / L3 Cache /Memory
Where can we miss?

21. Write Issues Write Through - Update both disk and memory
- On every store…. ridiculously slow and disk bandwidth excessive
Write Back - Write only to main memory. Write to the disk only when block is replaced.
+ Writes are fast
+ Multiple writes to a page are combined into one disk write
- Must keep track of when page has been written (dirty bit)
- A read miss may cause page to be replaced and written back (if modified)