Presentation is loading. Please wait.

Presentation is loading. Please wait.

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip.

Published byShannon Johnson Modified over 9 years ago

Similar presentations

Presentation on theme: "Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip."— Presentation transcript:

1

2 Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip

3 Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory

4 Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used

5 Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Miss Penalty( L1)

6 Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Miss Penalty( L1) Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2)

7 Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip A Second level cache is off chip that is larger and slower that interfaces with the Main Memory Same cache techniques can be used Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Miss Penalty( L1) Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2) Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Hit time (L2) + Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)

8 Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Hit time (L2) + Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2) Ex: Hit time (L1) = 1 clock cycle Miss rate(L1) = 10% Hit time ( L2) = 10 Clock Cycles Miss rate (L2) = 20% Miss penalty (L2) = 100 clock cyles If no secondary cache Ave Memory Access = 1 + 0.1 * 100 = 11 clock cycles

9 Ave Memory Access Time = Hit time(L1) + Miss rate (L1) * Hit time (L2) + Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2) Ex: Hit time (L1) = 1 clock cycle Miss rate(L1) = 10% Hit time ( L2) = 10 Clock Cycles Miss rate (L2) = 20% Miss penalty (L2) = 100 clock cyles If no secondary cache Ave Memory Access = 1 + 0.1 * 100 = 11 clock cycles With secondary cache Ave Memory Access = 1 + 0.1*10 + 0.1*0.2*100 1 + 1 + 2 = 4 clock cycles

10 Characteristics of programs that makes efficient use of cache memory

11 Characteristics of programs that makes efficient use of cache memory Code has tight loops with lots of reuse

12 Characteristics of programs that makes efficient use of cache memory Code has tight loops with lots of reuse Code minimizes jumps and branches to far away

13 Characteristics of programs that makes efficient use of cache memory Code has tight loops with lots of reuse Code minimizes jumps and branches to far away Data marches through arrays

14 Processor Memory Hierarchy Cache Memory Main Memory Transfer in 1 clock cycle Transfer > 10 clock cycles SRAM DRAM

15 Processor Virtual Memory Hierarchy Cache Memory Main Memory Transfer in 10 + clock cycles Transfer > 100,000 clock cycles DRAM Hard Disk

16 Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM

17 Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM 2.Enabled Multiprocessing. Computer running multiple processes ( programs and environments) each with it’s own address space. ( Time Sharing )

18 Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM 2.Enabled Multiprocessing. Computer running multiple processes ( programs and environments) each with it’s own address space. ( Time Sharing ) 3.Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels of access.

19 Virtual Memory The Illusion of Unlimited Amount of Memory 1.Invented to allow programmers to use the full addressing capability of the processor. Not limited by physical RAM 2.Enabled Multiprocessing. Computer running multiple processes ( programs and environments) each with it’s own address space. ( Time Sharing ) 3.Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels of access. 4.Provides simple relocation by mapping to pages in physical memory

20 Virtual Address – 4 GB 31 30 29... 14 13 12 11 10 9 8... 2 1 0 Virtual page number Page offset 28 27 26... 14 13 12 11 10 9 8... 2 1 0 Physical page number Page offset Page = 2 12 = 4KB Translation Physical Address – 512 MB

21 Virtual Address Virtual Page No. Page Offset Page Table Main Memory Hard Disk Physical Address Page = Block What is the miss penalty? Called Page Fault

22 1. Where can a page ( block) be placed in Main Memory?

23 The miss penalty is very high, so minimize miss rate

24 1. Where can a page ( block) be placed in Main Memory? The miss penalty is very high, so minimize miss rate The long access time of the hard disk enables the OS to control placing the pages in Main Memory

25 1. Where can a page ( block) be placed in Main Memory? The miss penalty is very high, so minimize miss rate The long access time of the hard disk enables the OS to control placing the pages in Main Memory The lowest miss rate results from locating pages anywhere inMain Memory ( Fully Associative )

26 2. How is a page ( block) found if it is in Main Memory?

27 A page is located by the page table that contains the physical address of the page in Main Memory in the entry addressed by the virtual page number. ( The index is mapped by the software using the page table and no tag is required.)

28 2. How is a page ( block) found if it is in Main Memory? A page is located by the page table that contains the physical address of the page in Main Memory in the entry addressed by the virtual page number. ( The index is mapped by the software using the page table and no tag is required.) The page table has an entry for every virtual page number which can be 1 M entries. So, the page table is typically stored in Main Memory.

29 2. How is a page ( block) found if it is in Main Memory? A page is located by the page table that contains the physical address of the page in Main Memory in the entry addressed by the virtual page number. ( The index is mapped by the software using the page table and no tag is required.) The page table has an entry for every virtual page number which can be 1 M entries. So, the page table is typically stored in Main Memory. Accessing Main Memory twice each time takes to long, so another cache must be added.

30 3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault )

31 3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault ) Under OS control, so clever algorithms are feasible.

32 3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault ) Under OS control, so clever algorithms are feasible. Usually try to approximate least recently used (LRU) replacement strategy.

33 3.Which page ( block) should be replaced on a Virtual Memory Miss ( Page Fault ) Under OS control, so clever algorithms are feasible. Usually try to approximate least recently used (LRU) replacement strategy. May keep a use bit, which is set whenever a page is accessed to help estimate LRU.

34 4. What happens on a Write?

35 Write Through is not feasible due to the very slow speed of the Hard Disk

36 4. What happens on a Write? Write Through is not feasible due to the very slow speed of the Hard Disk The write strategy is always Write Back

37 4. What happens on a Write? Write Through is not feasible due to the very slow speed of the Hard Disk The write strategy is always Write Back Write Back # The word is written only to the page in Main Memory (cache). # The modified page is written to the Hard Disk only when it is replaced. A Dirty Bit indicates if a page has been altered.

38 Selecting a Page Size Reasons Pages should be larger: 1.Minimize miss rate, but too large can get no improvement or even increase. 2.Reduces the size of the page table 3.Transfers to / from Hard Disk are more efficient ( also over networks)

39 Selecting a Page Size Reasons Pages should be larger: 1.Minimize miss rate, but too large can get no improvement or even increase. 2.Reduces the size of the page table 3.Transfers to / from Hard Disk are more efficient ( also over networks) Reasons Pages should be smaller: 1.Wastes space because each process has several primary pages required. 2.Start up time is longer for small processes.

40 Selecting a Page Size Reasons Pages should be larger: 1.Minimize miss rate, but too large can get no improvement or even increase. 2.Reduces the size of the page table 3.Transfers to / from Hard Disk are more efficient ( also over networks) Reasons Pages should be smaller: 1.Wastes space because each process has several primary pages required. 2.Start up time is longer for small processes. Page size is 4 KB to 64 KB

41 Virtual Address Virtual Page No. Page Offset Page Table Main Memory Hard Disk Physical Address Page = Block Physical Page Address Valid Use Dirty Page Table Maps Virtual Page No. to Disk Addr

42 The Page Table is Large Usually in DRAM Main Memory Too Slow

43 The Page Table is Large Usually in DRAM Main Memory Too Slow So, provide a cache for the Page Table TLB – translation-lookaside buffer

44 Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use

45 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use

46 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use

47 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Hit

48 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Hit Read Main Set Use Bit TLB

49 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Hit Write Main Set Dirty Bit TLB

50 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Miss

51 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Page Table Register + Physical Page Addr Physical Page Addr Tag Valid, Dirty, Use Miss TLB Miss or Page Fault

52 TLB Miss –

53 1.Select entry to be replaced (LRU or Random)

54 TLB Miss – 1.Select entry to be replaced (LRU or Random) 2.Write Back Use and Dirty Bits to Page Table replaced entry

55 TLB Miss – 1.Select entry to be replaced (LRU or Random) 2.Write Back Use and Dirty Bits to Page Table replaced entry 3.Access requested entry in Page Table If Page is in Main Memory ( Valid), load the translation from the Page Table to TLB and try again

56 TLB Miss – 1.Select entry to be replaced (LRU or Random) 2.Write Back Use and Dirty Bits to Page Table replaced entry 3.Access requested entry in Page Table If Page is in Main Memory ( Valid), load the translation from the Page Table to TLB and try again If Page is not in Main Memory ( Valid), then it is a Page Fault : Write Back Replaced Page to Disk if Dirty Move page from Disk to Main Memory Update Page Table & TLB and try again

57 Virtual Address 31 30 29... 14 13 12 11 10 9.... 2 1 0 Virtual Page Number Page Offset = = = = Tag Physical Page Number 20 TLB Hit Valid Dirty To Main Memory 20 12

58 Virtual Address 31 30 29... 14 13 12 11 10 9.... 2 1 0 Virtual Page Number Page Offset = = = = Tag Physical Page Number 20 TLB Hit Valid Dirty To Main Memory 20 12 Design Goals Hit Time – 1 clock cycle TLB Miss Penalty – 10 - 20 clock cycles TLB Miss Rate - < 1%

59 Virtual Address 31 30 29... 14 13 12 11 10 9.... 2 1 0 Virtual Page Number Page Offset = = = = Tag Physical Page Number 20 TLB Hit Valid Dirty Tag - m Index-k 20 12 Block Offset Byte Offset Main Memory Cache Physical Address

60 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address

61 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U?

62 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 14

63 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 1426

64 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 1426

65 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 1426 22

66 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address Size of page table assuming no disk address and 4 bits V,P,D,U? 1426 22 x 26

67 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? 1426 22 x 26

68 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? 1426 22 x 26 Tag Index

69 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? 1426 22 x 26 Tag Index-8

70 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? 1426 22 x 26 Tag-18 Index-8

71 TLB – translation-lookaside buffer Virtual Page No. Page Offset Page Table Main Memory Hard Disk TLB Physical Page Addr Tag Valid, Dirty, Use Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address TLB 2-way Associative with 256 entries. Index and Tag? 1426 22 x 26 Tag-18 Index-8 4 18 88

72 CPU TLB On Chip Cache Secondary Cache DRAM Main Memory Hard Disk Virtual Address 32-64 bits 4KB pages Microprocessors

73 CPU TLB On Chip Cache Secondary Cache DRAM Main Memory Hard Disk Virtual Address 32-64 bits 4KB pages TLB Split Data and Instruction 4-Way Set Associative 64 – 128 entries Microprocessors

74 CPU TLB On Chip Cache Secondary Cache DRAM Main Memory Hard Disk Virtual Address 32-64 bits 4KB pages TLB Split Data and Instruction 4-Way Set Associative 64 – 128 entries On Chip Cache Split Data and Instruction 8KB – 16KB each 4-Way Associative 32 Bytes / Block Microprocessors

75 Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor

76 Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor CPU supplies Read / Write and User / Supervisor signals for each access

77 Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor CPU supplies Read / Write and User / Supervisor signals for each access Comparisons can be made in the TLB. Non-compare can cause an exception

78 Implementing Protection with Virtual Memory Add protection bits to the TLB / Page Table Read / Write Access User(s) / Supervisor CPU supplies Read / Write and User / Supervisor signals for each access Comparisons can be made in the TLB. Non-compare can cause an exception User programs cannot modify the protection bits in Page Table / TLB

Download ppt "Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip."

Similar presentations

Virtual Memory In this lecture, slides from lecture 16 from the course Computer Architecture ECE 201 by Professor Mike Schulte are used with permission.

Virtual Memory In this lecture, slides from lecture 16 from the course Computer Architecture ECE 201 by Professor Mike Schulte are used with permission.
1 Lecture 13: Cache and Virtual Memroy Review Cache optimization approaches, cache miss classification, Adapted from UCB CS252 S01.

1 Lecture 13: Cache and Virtual Memroy Review Cache optimization approaches, cache miss classification, Adapted from UCB CS252 S01.
Lecture 8: Memory Hierarchy Cache Performance Kai Bu

Lecture 8: Memory Hierarchy Cache Performance Kai Bu
Practical Caches COMP25212 cache 3. Learning Objectives To understand: –Additional Control Bits in Cache Lines –Cache Line Size Tradeoffs –Separate I&D.

Practical Caches COMP25212 cache 3. Learning Objectives To understand: –Additional Control Bits in Cache Lines –Cache Line Size Tradeoffs –Separate I&D.
Computer Organization CS224 Fall 2012 Lesson 44. Virtual Memory  Use main memory as a “cache” for secondary (disk) storage l Managed jointly by CPU hardware.

Computer Organization CS224 Fall 2012 Lesson 44. Virtual Memory  Use main memory as a “cache” for secondary (disk) storage l Managed jointly by CPU hardware.
Lecture 34: Chapter 5 Today’s topic –Virtual Memories 1.

Lecture 34: Chapter 5 Today’s topic –Virtual Memories 1.
CSIE30300 Computer Architecture Unit 10: Virtual Memory Hsin-Chou Chi [Adapted from material by and

CSIE30300 Computer Architecture Unit 10: Virtual Memory Hsin-Chou Chi [Adapted from material by and
CSC 4250 Computer Architectures December 8, 2006 Chapter 5. Memory Hierarchy.

CSC 4250 Computer Architectures December 8, 2006 Chapter 5. Memory Hierarchy.
The Memory Hierarchy (Lectures #24) ECE 445 – Computer Organization The slides included herein were taken from the materials accompanying Computer Organization.

The Memory Hierarchy (Lectures #24) ECE 445 – Computer Organization The slides included herein were taken from the materials accompanying Computer Organization.
1 Lecture 20: Cache Hierarchies, Virtual Memory Today’s topics:  Cache hierarchies  Virtual memory Reminder:  Assignment 8 will be posted soon (due.

1 Lecture 20: Cache Hierarchies, Virtual Memory Today’s topics:  Cache hierarchies  Virtual memory Reminder:  Assignment 8 will be posted soon (due.
CSCE 212 Chapter 7 Memory Hierarchy Instructor: Jason D. Bakos.

CSCE 212 Chapter 7 Memory Hierarchy Instructor: Jason D. Bakos.
Chap. 7.4: Virtual Memory. CS61C L35 VM I (2) Garcia © UCB Review: Caches Cache design choices: size of cache: speed v. capacity direct-mapped v. associative.

Chap. 7.4: Virtual Memory. CS61C L35 VM I (2) Garcia © UCB Review: Caches Cache design choices: size of cache: speed v. capacity direct-mapped v. associative.
S.1 Review: The Memory Hierarchy Increasing distance from the processor in access time L1$ L2$ Main Memory Secondary Memory Processor (Relative) size of.

S.1 Review: The Memory Hierarchy Increasing distance from the processor in access time L1$ L2$ Main Memory Secondary Memory Processor (Relative) size of.
1  1998 Morgan Kaufmann Publishers Chapter Seven Large and Fast: Exploiting Memory Hierarchy.

1  1998 Morgan Kaufmann Publishers Chapter Seven Large and Fast: Exploiting Memory Hierarchy.
15-447 Computer ArchitectureFall 2008 © November 10, 2007 Nael Abu-Ghazaleh Lecture 23 Virtual.

Computer ArchitectureFall 2008 © November 10, 2007 Nael Abu-Ghazaleh Lecture 23 Virtual.
15-447 Computer ArchitectureFall 2007 © November 21, 2007 Karem A. Sakallah Lecture 23 Virtual Memory (2) CS 15-447: Computer Architecture.

Computer ArchitectureFall 2007 © November 21, 2007 Karem A. Sakallah Lecture 23 Virtual Memory (2) CS : Computer Architecture.
1 Chapter Seven Large and Fast: Exploiting Memory Hierarchy.

1 Chapter Seven Large and Fast: Exploiting Memory Hierarchy.
The Memory Hierarchy II CPSC 321 Andreas Klappenecker.

The Memory Hierarchy II CPSC 321 Andreas Klappenecker.
Technical University of Lodz Department of Microelectronics and Computer Science Elements of high performance microprocessor architecture Virtual memory.

Technical University of Lodz Department of Microelectronics and Computer Science Elements of high performance microprocessor architecture Virtual memory.
Virtual Memory. Why do we need VM? Program address space: 0 – 2^32 bytes –4GB of space Physical memory available –256MB or so Multiprogramming systems.

Virtual Memory. Why do we need VM? Program address space: 0 – 2^32 bytes –4GB of space Physical memory available –256MB or so Multiprogramming systems.

Similar presentations

Ads by Google