Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Miss Penalty( L1)

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Miss Penalty( L1) Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2)

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Miss Penalty( L1) Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2) Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Hit time (L2) + Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)

Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Hit time (L2) + Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2) Ex: Hit time (L1) = 1 clock cycle Miss rate(L1) = 10% Hit time ( L2) = 10 Clock Cycles Miss rate (L2) = 20% Miss penalty (L2) = 100 clock cyles If no secondary cache Ave Memory Access = * 100 = 11 clock cycles

Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Hit time (L2) + Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2) Ex: Hit time (L1) = 1 clock cycle Miss rate(L1) = 10% Hit time ( L2) = 10 Clock Cycles Miss rate (L2) = 20% Miss penalty (L2) = 100 clock cyles If no secondary cache Ave Memory Access = * 100 = 11 clock cycles With secondary cache Ave Memory Access = * *0.2* = 4 clock cycles

Characteristics of programs that makes efficient use of cache memory

Characteristics of programs that makes efficient use of cache memory Code has tight loops with lots of reuse

Characteristics of programs that makes efficient use of cache memory Code has tight loops with lots of reuse Code minimizes jumps and branches to far away

Characteristics of programs that makes efficient use of cache memory Code has tight loops with lots of reuse Code minimizes jumps and branches to far away Data marches through arrays

Processor Memory Hierarchy Cache Memory Main Memory Transfer in 1 clock cycle Transfer > 10 clock cycles SRAM DRAM

Processor Virtual Memory Hierarchy Cache Memory Main Memory Transfer in 10 + clock cycles Transfer > 100,000 clock cycles DRAM Hard Disk

Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM

Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM 2.Enabled Multiprocessing. Computer running multiple processes ( programs and environments) each with it’s own address space. ( Time Sharing )

Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM 2.Enabled Multiprocessing. Computer running multiple processes ( programs and environments) each with it’s own address space. ( Time Sharing ) 3.Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels of access.

Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM 2.Enabled Multiprocessing. Computer running multiple processes ( programs and environments) each with it’s own address space. ( Time Sharing ) 3.Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels of access. 4.Provides simple relocation by mapping to pages in physical memory

Virtual Address – 4 GB Virtual page number Page offset Physical page number Page offset Page = 2 12 = 4KB Translation Physical Address – 512 MB

Virtual Address Virtual Page No. Page Offset Page Table Main Memory Hard Disk Physical Address Page = Block What is the miss penalty? Called Page Fault

1. Where can a page ( block) be placed in Main Memory?

The miss penalty is very high, so minimize miss rate

1. Where can a page ( block) be placed in Main Memory? The miss penalty is very high, so minimize miss rate The long access time of the hard disk enables the OS to control placing the pages in Main Memory

1. Where can a page ( block) be placed in Main Memory? The miss penalty is very high, so minimize miss rate The long access time of the hard disk enables the OS to control placing the pages in Main Memory The lowest miss rate results from locating pages anywhere inMain Memory ( Fully Associative )

2. How is a page ( block) found if it is in Main Memory?

A page is located by the page table that contains the physical address of the page in Main Memory in the entry addressed by the virtual page number. ( The index is mapped by the software using the page table and no tag is required.)

2. How is a page ( block) found if it is in Main Memory? A page is located by the page table that contains the physical address of the page in Main Memory in the entry addressed by the virtual page number. ( The index is mapped by the software using the page table and no tag is required.) The page table has an entry for every virtual page number which can be 1 M entries. So, the page table is typically stored in Main Memory.

2. How is a page ( block) found if it is in Main Memory? A page is located by the page table that contains the physical address of the page in Main Memory in the entry addressed by the virtual page number. ( The index is mapped by the software using the page table and no tag is required.) The page table has an entry for every virtual page number which can be 1 M entries. So, the page table is typically stored in Main Memory. Accessing Main Memory twice each time takes to long, so another cache must be added.

3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault )

3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault ) Under OS control, so clever algorithms are feasible.

3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault ) Under OS control, so clever algorithms are feasible. Usually try to approximate least recently used (LRU) replacement strategy.

3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault ) Under OS control, so clever algorithms are feasible. Usually try to approximate least recently used (LRU) replacement strategy. May keep a use bit, which is set whenever a page is accessed to help estimate LRU.

4. What happens on a Write?

Write Through is not feasible due to the very slow speed of the Hard Disk

4. What happens on a Write? Write Through is not feasible due to the very slow speed of the Hard Disk The write strategy is always Write Back

4. What happens on a Write? Write Through is not feasible due to the very slow speed of the Hard Disk The write strategy is always Write Back Write Back # The word is written only to the page in Main Memory (cache). # The modified page is written to the Hard Disk only when it is replaced. A Dirty Bit indicates if a page has been altered.

Selecting a Page Size Reasons Pages should be larger: 1.Minimize miss rate, but too large can get no improvement or even increase. 2.Reduces the size of the page table 3.Transfers to / from Hard Disk are more efficient ( also over networks)

Selecting a Page Size Reasons Pages should be larger: 1.Minimize miss rate, but too large can get no improvement or even increase. 2.Reduces the size of the page table 3.Transfers to / from Hard Disk are more efficient ( also over networks) Reasons Pages should be smaller: 1.Wastes space because each process has several primary pages required. 2.Start up time is longer for small processes.

Selecting a Page Size Reasons Pages should be larger: 1.Minimize miss rate, but too large can get no improvement or even increase. 2.Reduces the size of the page table 3.Transfers to / from Hard Disk are more efficient ( also over networks) Reasons Pages should be smaller: 1.Wastes space because each process has several primary pages required. 2.Start up time is longer for small processes. Page size is 4 KB to 64 KB

Virtual Address Virtual Page No. Page Offset Page Table Main Memory Hard Disk Physical Address Page = Block Physical Page Address Valid Use Dirty Page Table Maps Virtual Page No. to Disk Addr

The Page Table is Large Usually in DRAM Main Memory Too Slow

The Page Table is Large Usually in DRAM Main Memory Too Slow So, provide a cache for the Page Table TLB – translation-lookaside buffer

Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Hit

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Hit Read Main Set Use Bit TLB

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Hit Write Main Set Dirty Bit TLB

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Miss

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Miss TLB Miss or Page Fault

TLB Miss –

1.Select entry to be replaced (LRU or Random)

TLB Miss – 1.Select entry to be replaced (LRU or Random) 2.Write Back Use and Dirty Bits to Page Table replaced entry

TLB Miss – 1.Select entry to be replaced (LRU or Random) 2.Write Back Use and Dirty Bits to Page Table replaced entry 3.Access requested entry in Page Table If Page is in Main Memory ( Valid), load the translation from the Page Table to TLB and try again

TLB Miss – 1.Select entry to be replaced (LRU or Random) 2.Write Back Use and Dirty Bits to Page Table replaced entry 3.Access requested entry in Page Table If Page is in Main Memory ( Valid), load the translation from the Page Table to TLB and try again If Page is not in Main Memory ( Valid), then it is a Page Fault : Write Back Replaced Page to Disk if Dirty Move page from Disk to Main Memory Update Page Table & TLB and try again

Virtual Address Virtual Page Number Page Offset = = = = Tag Physical Page Number 20 TLB Hit Valid Dirty To Main Memory 20 12

Virtual Address Virtual Page Number Page Offset = = = = Tag Physical Page Number 20 TLB Hit Valid Dirty To Main Memory Design Goals Hit Time – 1 clock cycle TLB Miss Penalty – clock cycles TLB Miss Rate - < 1%

Virtual Address Virtual Page Number Page Offset = = = = Tag Physical Page Number 20 TLB Hit Valid Dirty Tag - m Index-k Block Offset Byte Offset Main Memory Cache Physical Address

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U?

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 14

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 1426

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 1426

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U?

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? x 26

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? x 26

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? x 26 Tag Index

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? x 26 Tag Index-8

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? x 26 Tag-18 Index-8

TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? x 26 Tag-18 Index

CPU TLB On Chip Cache Secondary Cache DRAM Main Memory Hard Disk Virtual Address bits 4KB pages Microprocessors

CPU TLB On Chip Cache Secondary Cache DRAM Main Memory Hard Disk Virtual Address bits 4KB pages TLB Split Data and Instruction 4-Way Set Associative 64 – 128 entries Microprocessors

CPU TLB On Chip Cache Secondary Cache DRAM Main Memory Hard Disk Virtual Address bits 4KB pages TLB Split Data and Instruction 4-Way Set Associative 64 – 128 entries On Chip Cache Split Data and Instruction 8KB – 16KB each 4-Way Associative 32 Bytes / Block Microprocessors

Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor

Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor CPU supplies Read / Write and User / Supervisor signals for each access

Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor CPU supplies Read / Write and User / Supervisor signals for each access Comparisons can be made in the TLB. Non-compare can cause an exception

Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor CPU supplies Read / Write and User / Supervisor signals for each access Comparisons can be made in the TLB. Non-compare can cause an exception User programs cannot modify the protection bits in Page Table / TLB