1. CS 704 Advanced Computer Architecture
Lecture 33
Memory Hierarchy Design
(Virtual Memory System)
Prof. Dr. M. Ashraf Chughtai
Welcome to the 33rd Lecture for the series of lectures on Advanced Computer Architecture

2. Lec. 33 Memory Hierarchy Design (9)
Today’s Topics
Recap: Main memory and Virtual memory Design
Virtual Memory Address Translation
Virtual Memory Performance
Protection of multiple processes sharing memory
Summary
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

3. Recap: Memory Hierarchy
Main memory organization
Organized using banks of memory arrays
Dual Inline Memory Modules - DIMMs
Fast page mode
Synchronous
Double Data Rate DRAMs
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

4. Recap: Main Memory Performance
Fast page mode
Synchronous DRAM (SDRAM)
Double Data Rate (DDR) DRAM
latency and bandwidth
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

5. Recap: Main Memory Performance
concern of caches
bandwidth
Inputs/outputs and multiprocessors
Wider Main Memory
Simple Interleaved Memory
Independent Memory Banks
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

6. Lec. 33 Memory Hierarchy Design (9)
Recap: Virtual Memory
Multiple processes
Dedicate a full address space
Virtual Memory
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

7. Recap: Virtual Memory … Cont’d
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

8. Recap: Virtual Memory … Cont’d
Fix-sized fragment
Variable-sized fragment
Contiguous pages in virtual memory
Physically available on the main memory
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

9. Recap: Virtual Memory .. Cont’d
Protection and Relocation
Protection
Relocation
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

10. Recap: Cache verses Virtual memory
Page fault or address fault
CPU produces virtual address
Mapping of a virtual address
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

11. Recap: Cache verses Virtual memory …. Cont’d
Replacement
The size of processor address
Secondary storage
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

12. Recap: Cache verses Virtual memory …. Cont’d
The page replacement strategies
FIFO – First –in-First Out
LRU – Least recently Used
Approximation to LRU
Bit
Resets the reference bit
Page with a reference bit
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

13. Recap: Cache verses Virtual memory …. Cont’d
VM Write strategies may be:
Write Back
Write Through
Write through is impossible because:
Too long access to disk
The write buffer
The I/O system
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

14. Recap: Virtual Memory operation
The CPU generates the Virtual Address
Lookup table
Location of the page or segment
Virtual addresses to physical addresses
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

15. Recap: Virtual Memory operation .. Cont’d
page fault
The OS has full control over placement
OS exception handler is invoked
current process suspends the data is to the main memory by the OS
The contents of the page table are updated
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

16. VM Address Translation Concept
Assume that Virtual Address space V comprises a set of N pages
V = {0, 1, …, N–1}
And, Physical Address space P comprises a set of M pages
P = {0, 1, …, M–1}
where M < N
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

17. VM Address Translation Concept
Assuming n-bit virtual address, m-bit physical address and p-bit page offset, the virtual and physical address limits and the page size can be expressed as
Virtual address limit = N = 2n
Physical address limit = M = 2m
page size (bytes) = PS = 2p
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

18. VM Address Translation Concept
page offset
page number
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

19. VM Address Translation Concept
p–1 p
virtual page number
page offset
virtual address
physical page number
physical address
address translation
mechanism
m–1
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

20. Memory resident page table (physical page or disk address)
Physical Memory
Disk Storage (swap file or
regular file system file)
Valid 1
Virtual Page
Number
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

21. Page Table Operation: 3 steps
1: Translation
2: Computing Physical Address
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

22. Address Translation via Page Table
virtual page number (VPN)
page offset
virtual address
physical page number (PPN)
physical address
p–1 p
m–1
n–1
page table base register
if valid=0
then page
not in memory
valid
access
VPN acts as
table index
Page entry Table
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

23. Lec. 33 Memory Hierarchy Design (9)
Page Table Operation
3: Checking Protection
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

24. Simple Memory System Example
Addressing
14-bit virtual addresses
12-bit physical address
Page size = 64 bytes 13 12 11 10 9 8 7 6 5 4 3 2 1
VPO
PPO
PPN
VPN
(Virtual Page Number)
(Virtual Page Offset)
(Physical Page Number)
(Physical Page Offset)
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

25. Simple Memory System Page Table
Only show first 16 entries
VPN
PPN
Valid 00 28 1 08 13 01 – 09 17 02 33 0A 03 0B 04 0C 05 16 0D 2D 06 0E 11 07 0F
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

26. Fast Address Translation
large and in the main memory
miss penalty
one memory access to obtain the physical address
second to get the data
Miss penalty can be reduced
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

27. Fast Translation with a TLB
CPU
TLB
Lookup
Cache
Main
Memory VA PA
miss
hit
data
Trans-
lation
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

28. Fast Address Translation .. Cont’d
Virtual Address Physical Address Dirty Ref Valid Access
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

29. Address Translation with a TLBp
p–1
virtual page number
page offset
virtual address
valid
tag
physical page number
TLB . . . =
TLB hit
physical address
tag
byte offset
index
valid
tag
data
Cache =
cache hit
data
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

30. Fast Address Translation .. Cont’d
Fully associative placement policy
Violation against protection information in the TLB
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

31. Fast Address Translation .. Cont’d
The physical address
The page offset
A full physical address
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

32. Fast Address Translation .. Cont’d
Merits of TLB:
Fully associative, set associative, or direct mapped
entries
Mid-range machines use small n-way set associative organizations.
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

33. Address Translation Example
virtual address of 64 bits
Physical Address: 41 bits
TLB – direct mapped with 256 entries
First Level Caches: direct mapped with 8KB entries, block size 64 byte
Second Level Cache: direct mapped 4MB direct mapped; block size 64 bytes
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

34. Address Translation Example VA – L2
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

35. Lec. 33 Memory Hierarchy Design (9)
VM Protection Process
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

36. VM Protection Mechanisms
The address translation mechanis
Protection attribute bits
(PTE) and TLB
Protection
Does not have permission
An exception is raised
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

37. VM Protection Mechanisms
The address is said to be valid if
Base <= address <= Bound
Base and Bound register
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

38. VM Protection Mechanisms
Page tables each pointing to the distinct pages of memory
Prevented from modifying these tables
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

39. Lec. 33 Memory Hierarchy Design (9)
Summary
Cache memories:
HW-management
Separate instruction and data caches permits simultaneous instruction fetch and data access
Four questions:
Block placement
Block identification
Block replacement
Write strategy
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

40. Lec. 33 Memory Hierarchy Design (9)
Summary
Virtual memory:
Software-management
Very high miss penalty
=> miss rate must be very low
Also supports:
program loading
memory protection
Multiprogramming
MAC/VU-Advanced Computer Architecture
Lec. 33 Memory Hierarchy Design (9)

41. Lec. 25 – Memory Hierarchy Design (1)
Summary
Memory hierarchy organization
Modules of DRAM and SRAM
design and working of disk storages
DRAM, SRAM and Disk
MAC/VU-Advanced Computer Architecture
Lec. 25 – Memory Hierarchy Design (1)

42. Recap: Memory Hierarchy Principles
Concept of Caching
Principle of Locality
MAC/VU-Advanced Computer Architecture
Lecture 27 Memory Hierarchy (3)

43. Recap: Principle of Locality
Data or instructions
Processor access a relatively small portion of the address space
Fastest memory closet to the processor
MAC/VU-Advanced Computer Architecture
Lecture 27 Memory Hierarchy (3)

44. Recap: Types of Locality
Temporal locality
Spatial locality
MAC/VU-Advanced Computer Architecture
Lecture 27 Memory Hierarchy (3)

45. Recap: Improving Cache Performance
The miss penalty
The miss rate
The miss Penalty or miss rate via Parallelism
The time to hit in the cache
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

46. Recap: Reducing Miss Penalty
Multilevel Caches
Critical Word first and Early Restart
Priority to Read Misses Over writes
Merging Write Buffers
Victim Caches
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

47. Recap: Reducing Miss Penalty
‘Multi level caches’
the more the merrier
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

48. Recap: Reducing Miss Penalty
“ Critical Word First and Early Restart’,
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

49. Recap: Reducing Miss Penalty
‘priority to read miss over the write miss’,
Favoritism
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

50. Recap: Reducing Miss Penalty
‘merging write-buffer,’
acquaintance
“victim cache’
salvage
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

51. Recap: Reducing Miss Penalty
Reducing miss rate
Cache-misses and methods to reduce the miss rate
MAC/VU-Advanced Computer Architecture
Lecture 30 Memory Hierarchy (6)

52. Summary – Cache Optimization
5 methods to reduce the miss penalty
7 ways to reduce 3Cs
3 methods for reducing miss rate and miss penalty via parallelism; and
4 techniques to reduce hit time
The performance of these methods is summarized here
MAC/VU-Advanced Computer Architecture
Lecture 31 Memory Hierarchy (7)

53. Lecture 31 Memory Hierarchy (7)
Allah Hafiz
MAC/VU-Advanced Computer Architecture
Lecture 31 Memory Hierarchy (7)