1. Memory – Virtual Memory, Virtual Machines
CS/COE 1541 (term 2174)
Jarrett Billingsley

2. Class Announcements
Slight grading policy updates/clarifications (on site):
Lowest of 6 HW grades will be dropped.
This mean each HW is 4% of your final grade.
Clarified the grading scale used for this class.
This means you must get at least a 73% for a C.
I don’t do “stunlock” grading. What I mean is…
Problem 2 depends on problem 1. Even if you get problem 1 wrong, you still get credit for problem 2 if you do it right…
…but I have to be able to see how you did problem 2.
Show your work! If I don’t know where you’re having trouble, I can’t help you.
2/27/2017
CS/COE 1541 term 2174

3. Project 1 Project 1 due dates:
3/8 (Wednesday) by midnight for full grade
3/10 (Friday) by midnight for 10% penalty
3/12 (Sunday) by midnight for 20% penalty – last chance!
I would just make it due Sunday but I gotta have time to grade the projects 
The penalties are absolute: if you got an 85%, and have a 10% penalty, your grade will be = 75%.
2/27/2017
CS/COE 1541 term 2174

4. Cache Clarifications (will be helpful on your project!)
2/27/2017
CS/COE 1541 term 2174

5. Terminology
Replacement means “evicting a valid block to put another block into the cache.” Empty (invalid) blocks are not replaced.
Which means in associative caches, the replacement scheme means “how do we pick a valid block to evict?”
The byte/word/row indices are used to select things in the cache.
They aren’t stored in the cache.
Just like in an array, we don’t store 0, 1, 2, 3… next to the data.
Allocation in caches means “for this memory address we’re trying to access, should we put an entry in the cache?”
E.g. in write-allocate, when we write to an address and it’s a miss in the cache, we make a new entry for that address.
On HW2, the last problem would be something like “read-no-allocate” – we read from memory but don’t cache it!
2/27/2017
CS/COE 1541 term 2174

6. Allocation policy vs. writing scheme
Allocation policies and writing schemes are two different things, though they often work in pairs. For misses on writes:
Write-allocate
Write-no-allocate
Write-through
Might make sense, if you’re dealing with memory that will be read soon!
Sometimes called “write-around”: write to memory, but not to cache.
Write-back
Makes it behave very similarly to hits on writes.
Invalid
The green cells are common pairings.
2/27/2017
CS/COE 1541 term 2174

7. Associative caches Blocks can’t be in an associative cache twice.
If a block is already in the cache when we access it, then it’s a hit!
LRU (least-recently-used) also means hits.
If you only go by which cache block has been in the cache the longest, that’s a different scheme known as FIFO (first-in, first-out).
When you hit a block that’s in the cache, it becomes the most-recently used, and the block that WAS the second-oldest becomes the oldest.
When you miss, the new block that’s brought in becomes the most-recently used as well.
2/27/2017
CS/COE 1541 term 2174

8. Performance Hit rate is not the same thing as hit time.
Hit rate = 100% - Miss rate. So improving hit rate is the same thing as improving miss rate.
In fact, some techniques to improve miss rate (and therefor hit rate) worsen hit time! e.g. associativity and larger caches.
Smaller caches do reduce the amount of data to look through, but that’s not why they improve hit time.
Searching through a cache always takes a constant time.
Even in associative caches, the search is done in parallel.
Smaller caches improve hit time because of physically shorter distance between the data and the circuits that need it.
In other words, the constant factor to search is smaller.
2/27/2017
CS/COE 1541 term 2174

9. Memory write buffers
A memory write buffer buffers data leaving the cache and going to memory.
This is in contrast to a cache write buffer, which buffers data coming into the cache from the CPU (used with write-back).
You can use memory write buffers with either write scheme!
The memory write buffer avoids memory stalls by allowing the cache to “trickle” writes to memory while the CPU continues.
Both write-back and write-through can benefit from this, but write-back reduces the amount of data moving from the cache to memory.
2/27/2017
CS/COE 1541 term 2174

10. Can a large write buffer be as good as write-back?
I accepted this answer, but there are complications…
Write to block A.
Read block B. Collides with A.
Read block A…?
Uh-oh.
Cache
Memory
Aold B
Anew B
Aold
Buffer
Anew
2/27/2017
CS/COE 1541 term 2174

11. Check that buffer
To ensure consistency, we have to check the buffer too.
Write buffers are essentially fully-associative FIFO caches.
Since they’re fully-associative, a large write buffer would require a LOT of comparators.
This is one reason why they’re usually 4-8 blocks in practice.
Write-back will still be a win, even if we have space for a large write buffer and its circuitry, because write-back can drastically reduce the number of writes to memory.
2/27/2017
CS/COE 1541 term 2174

12. Writing schemes Write-through:
When writing, both hits and misses take the same time.
If using write-allocate and we miss, we just smash the old entry.
In real life, we do have individual valid bits for each word in a block, and we can write single words through to memory.
Write-back:
Again, in real life we do have individual dirty bits for each word in a block and we can evict only the dirty words.
Write-back reduces frequency of memory writes because it only writes to memory when a dirty block is evicted.
2/27/2017
CS/COE 1541 term 2174

13. Intro to Virtual… things.
2/27/2017
CS/COE 1541 term 2174

14. Clearly this is impossible. But the programs believe it's true.
Virtual memory
Virtual memory is an illusion: the OS makes every process believe that it's the only program running on the entire computer!
Physical Memory
Code
Memory
Process 1
0x8000
0xFFFF
Process 2
...
Process 2's memory
Process 1's memory
0x0066_7FFF
0x0066_0000
0x0040_FFFF
Clearly this is impossible. But the programs believe it's true.
0x0040_8000
2/27/2017
CS/COE 1541 term 2174

15. ??? The nitty gritties Code Virtual Memory CPU Cache Physical Memory
The OS maintains this illusion with help from the hardware.
Programs see a virtual address space, and program code/registers hold virtual addresses.
The memory hardware uses a physical address space, and we use physical addresses to access it over the memory bus.
Code
Virtual Memory
Process
Virtual Addresses
CPU
???
Cache
Physical Addresses
Physical Memory
2/27/2017
CS/COE 1541 term 2174

16. How big does this table need to be…?
Address translation
We need a mechanism to translate virtual to physical.
In all modern architectures, this is the page table.
CPU
Cache
How big does this table need to be…?
Virtual Address
Physical Address
Valid
Virtual
Physical 1
8000
3300
BB030400
2/27/2017
CS/COE 1541 term 2174

17. Space problems
Let’s divide memory into blocks, just like we did with the cache.
This time, let’s use 4096 (212) bytes, or 4 KiB per block.
With 232 bytes of memory, how many blocks is that?
232 / 212 = 220 blocks, or about 1 million.
1 million entries in our page table. Each entry needs a 22-bit physical page number (concatenated with the lower 12 bits of the virtual address) and some other bits for valid, dirty etc. So ~4B.
4B x 220 blocks = 4 MB of storage. For one page table.
And what about CPUs with 64-bit addresses?
We’ll deal with this later…
2/27/2017
CS/COE 1541 term 2174

18. Cache the cache the cache the cache the cache the cache
Current physical memory tech has a big downside: it’s volatile.
Also, modern OSes allow hundreds of programs to run at once.
We might need more memory than is physically available!
The concept of virtual memory can be extended to treat the physical memory as a cache for the nonvolatile storage.
When we run out of space in physical memory…
Evict blocks to disk!
If we access a virtual address that isn’t in physical memory…
Bring it in from the disk!
Or error (segfault, “page fault”).
The caching concepts we’ve learned so far apply to this as well!
2/27/2017
CS/COE 1541 term 2174

19. We have to go deeper
Virtual Memory lets OSes give multiple processes the illusion of being the only program running on the computer.
Virtual Machines let multiple operating systems the illusion of being the only one running!
In this case we have a VMM: Virtual Machine Manager, or hypervisor, which runs on the hardware.
This manages the multiple operating systems, called guests.
There may also be a host OS: an operating system which is running the VMM!
If this sounds crazy, it kind of is…
2/27/2017
CS/COE 1541 term 2174

20. Problems
In order for an CPU architecture to be virtualizable (e.g. it can run OSes as a guests), it must:
Have two modes: user mode and kernel mode, where kernel mode has access to special privileged instructions used for managing the hardware.
All hardware must only be accessed through those instructions.
Almost all modern architectures satisfy the first requirement but not the second.
They all grew out of tiny personal computers…
In order to account for this, VMMs have to do more work and run the guest OSes less efficiently.
2/27/2017
CS/COE 1541 term 2174