1. Memory: Putting it all together
CS/COE 1541 (term 2174)
Jarrett Billingsley

2. Class Announcements HW4 tonight I had an exam to write by 1:00 today 
But that means I have no lecture to make for tomorrow so I can make the homework 
But I’m really sleepy cause I only got 6 hours of sleep 
Exam study guide + practice problems by Friday
Also, today is mostly a review
After the exam, you’ll have a week of break – no HW, no project
3/15/2017
CS/COE 1541 term 2174

3. <wistful montage>
<emotional piano music plays>
3/15/2017
CS/COE 1541 term 2174

4. Memory/storage technologies
Volatile
SRAM
DRAM
Speed
FAST OK
Price
Expensive
Power
Good!
Ehh…
Durability
Reliability
Pretty good!
Nonvolatile
HDDs
Flash
SLOW
Pretty good!
Cheap!
Ehh…
Bad OK
Good!
I’m using Durability to mean “how well it holds data after repeated use.”
I’m using Reliability to mean “how likely it is to break.”
3/15/2017
CS/COE 1541 term 2174

5. The memory hierarchy
Since the technologies vary so much in characteristics, we use them in conjunction with one another to exploit their strengths and minimize their weaknesses.
SRAM in the CPU
DRAM as main memory
HDDs and Flash as long-term storage
We use different sizes at each level to reduce cost.
We use caching to exploit each technology’s performance.
Multi-level CPU caches…
Paging to use DRAM as a cache for long-term storage…
Write buffers in the hardware of long-term storage…
3/15/2017
CS/COE 1541 term 2174

6. The fundamental conceit of caching
Caching is a form of prediction. It takes advantage of two things:
Temporal locality: data accesses repeat over time.
Spatial locality: data accesses are clustered by location.
But caching introduces a lot of complications:
The cache is limited in size compared to the next storage level.
We have to choose what data to cache.
The cached data is temporary.
We have to make sure it’s written out when it changes.
The cached data is a copy.
We have to make sure it’s consistent with the next level.
3/15/2017
CS/COE 1541 term 2174

7. Choosing cache parameters
As you saw in your project, varying the cache parameters can have complex and unexpected effects on the same workloads.
Bigger caches reduce capacity misses, but increase hit time.
Bigger blocks reduce compulsory misses, but increase bandwidth.
Associativity reduces conflict misses, but increases hit time.
LRU reduces miss rate, but increases hit time and complexity.
Write buffers amortize write bandwidth, but increase hit time and complexity.
Write-back reduces bandwidth, but increases hit time and complexity and is less dependable (more chance of data loss).
The particular needs of the problem at hand and the memory technologies being used will guide your choices of parameters.
3/15/2017
CS/COE 1541 term 2174

8. Virtual memory
To add to this complex layer cake… now we want to be able to support arbitrary mappings from VAs to PAs.
This requires support from both hardware and the OS.
Now we have to manage a cache for page table entries: the TLB.
And since we’re doing this virtual memory stuff to support multiprocessing…
We have to ensure the TLB/cache entries aren’t used by the wrong processes.
We can tag TLB/cache entries with process identifiers to avoid having to flush (invalidate) them.
3/15/2017
CS/COE 1541 term 2174

9. Paging A useful extension of virtual memory.
Now the main memory is a cache, too.
As if we didn’t have enough of those.
Paging is largely constrained by the performance of the nonvolatile storage being used.
Most paging techniques in use today were designed for use with magnetic spinning disks, but Flash is becoming widespread.
3/15/2017
CS/COE 1541 term 2174

10. My god, it’s full of caches
3/15/2017
CS/COE 1541 term 2174

11. Caches Rule Everything Around Me
VA Page#
Page offset
TLB
Physical address
CPU
Virtual address from lw/sw instructions or from program counter (PC)
Physical
Memory
PTE
PT walker
TLB miss
OS Page Fault Handler
Virtual
Address
space
Page table
page
Oh no!
Page fault
Cache
Block of a page
PTE into TLB
HDD/SSD
3/15/2017
Illustration courtesy Dr. Melhem