Presentation is loading. Please wait.

Presentation is loading. Please wait.

Optical Overlay NUCA: A High Speed Substrate for Shared L2 Caches

Published byHerbert Welch Modified over 6 years ago

Similar presentations

Presentation on theme: "Optical Overlay NUCA: A High Speed Substrate for Shared L2 Caches"— Presentation transcript:

1 Optical Overlay NUCA: A High Speed Substrate for Shared L2 Caches
OONUCA Optical Overlay NUCA: A High Speed Substrate for Shared L2 Caches Eldhose Peter*, Anuj Arora**, Akriti Bagaria* and Dr. Smruti R Sarangi* *IIT Delhi, **CISCO Bangalore

2 Motivation Overlay NUCA Architecture Results

3 Understand the problem - Cache
UCA Cache performance α 1/(hit latency) Cache performance α hit rate Sets Static => Not adaptable based on access pattern L2 L2 L2 L2 L2 Improved cache utilization L2 L2 L2 L2 NUCA Lower Level Memory Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

4 Understand the problem – Optical Communication
Electrical Optical 50-60 cycles 1-2 cycles Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

5 Optical Communication
S D1 D2 D3 Basic Components Reservation assisted Single Write Multi Read(R-SWMR) Methods to Leverage Optical Networks for Multicore Processors 11/22/2018

6 Prior Approaches No prior work in cache using optical NOC
Electrical NOC SNUCA DNUCA RNUCA L1 Migration near to the core L1 L1 L1 L1 Search L1 L1 L1 L1 Lower Level Memory 1000 100011 Tag Set Index Block Size HomeBank 10100 Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

7 Equidistant nodes Banks are equidistant in terms of delay(approx)
Dynamic creation of sets Improves the utilization of banks Improves hit rate I am near to S X cycles S I am also near to S X cycles Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

8 Phases of operation Phase 1: SNUCA + Profiling
Phase 2 : Reconfiguration Phase 3 : OONUCA Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

9 Optical Overlay Profiling information – Cache bank accesses, bank contention, cache lines used Experimentally determined that the ring topology 8 banks is the best Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

10 Creation of overlay High Low 1 2 20 3 4 5 13 6 7 8 30 Hybrid 6 9 10 11
14 12 17 13 14 19 15 16 8 3 Infreq 17 18 19 7 20 12 21 22 4 23 31 24 26 25 25 26 27 28 29 30 31 32 15 5 29 32 11 16 28 10 23 Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

11 Operations in Overlay NUCA – Search
Home Bank 2 23 15 Two-Side Incremental (TSI) 20 27 Broadcast 18 31 5 Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

12 Operations in Overlay NUCA - Eviction
Main Memory Home Bank Eviction from L2 18 20 21 24 26 29 31 32 Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

13 TSI - Protocol L1 Cache Miss L2 Cache Bank If message type is request?
If space available in message queue? Hit Reply If non home bank Kill to home bank Kill to opposite branch Yes - Search Remove RCB Entry Type of Message ? Miss No – NACK to sender(Exponential back off) Notify RCB Entry Notify Kill Miss Home bank? Add notify Create Entry in RCB (Home bank) Yes No Any Child? Remove MQ Entry If notify = 2 Send request to Main memory Hit Remove entry from RCB Remove entry from MQ Yes No Miss Send request message to children Notify message to home bank Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

14 Home Bank Controller Main memory Read Response Collection Buffer
Main memory Write Main Memory Message ID Block Addr MRBV Response to the sender Miss Eviction Logic Hit Victim Buffer Migrate block Miss Overlay Info store Evicted block Cache Bank NACK Message Fill Bank Read/Write NACK controller Response Forward request to other banks Search Logic Notify Full Message Queue Hit To core Read/Write Search Bank Kill controller Kill Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

15 Message Structure Control Message Data Message 1 Flit – 3 cycles
NACK, Kill, Notify Data Message 5 Flits – 7 cycles Request, Response Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

16 Clustered architecture – 16 stations
32 cores and 32 cache banks Clustered architecture – 16 stations Distribute directory Off chip laser Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

17 Story till now Optical Overlay Operations Home Bank Controller
Message Format Architecture Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

18 Configuration System L1 cache L2 cache Main memory
Cores – 32 Technology – 18nm Frequency – 3.4 GHz Laser – Off chip L1 cache Block size- 64 B Write mode – Write back Size – 32 KB MSHR - 32 L2 cache Banks – 32 Associativity – 8 Size – 256 KB Main memory Latency – 250 cycles Memory controllers – 4 Auxiliary Structures RCB – 128 MQ – 16 VB - 20 Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

19 Results Hits in Home Bank
More non-home bank hits => high performance in Optical Overlay NUCA More non home bank hits Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

20 Results Normalized Average Hit Latency 0.4-0.8 0.2-0.55
Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

21 Comparable to DNUCA, much better than SNUCA
Results More home bank hits L2 Hit Rate Comparable to DNUCA, much better than SNUCA Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

22 Results Normalized IPC 167% 161% 2-3% 50, 24,18%
High non-home bank hits Less L2 requests Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

23 What we achieved Flexible and efficient access mechanism for large shared L2 caches TSI protocol has less number of accesses compared to naive broadcast protocol Performance difference of TSI and broadcast is minimal(2-3%) Mean speedup of 50% over SNUCA Proposal is agnostic to the network topology Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

24 Thank you Optical overlay NUCA: A high speed substrate for shared L2 caches 11/22/2018

Download ppt "Optical Overlay NUCA: A High Speed Substrate for Shared L2 Caches"

Similar presentations

Virtual Hierarchies to Support Server Consolidation Michael Marty and Mark Hill University of Wisconsin - Madison.

Virtual Hierarchies to Support Server Consolidation Michael Marty and Mark Hill University of Wisconsin - Madison.
Cache coherence for CMPs Miodrag Bolic. Private cache Each cache bank is private to a particular core Cache coherence is maintained at the L2 cache level.

Cache coherence for CMPs Miodrag Bolic. Private cache Each cache bank is private to a particular core Cache coherence is maintained at the L2 cache level.
A Framework for Coarse-Grain Optimizations in the On-Chip Memory Hierarchy J. Zebchuk, E. Safi, and A. Moshovos.

A Framework for Coarse-Grain Optimizations in the On-Chip Memory Hierarchy J. Zebchuk, E. Safi, and A. Moshovos.
Exploiting Access Semantics and Program Behavior to Reduce Snoop Power in Chip Multiprocessors Chinnakrishnan S. Ballapuram Ahmad Sharif Hsien-Hsin S.

Exploiting Access Semantics and Program Behavior to Reduce Snoop Power in Chip Multiprocessors Chinnakrishnan S. Ballapuram Ahmad Sharif Hsien-Hsin S.
1 Lecture 17: Large Cache Design Papers: Managing Distributed, Shared L2 Caches through OS-Level Page Allocation, Cho and Jin, MICRO’06 Co-Operative Caching.

1 Lecture 17: Large Cache Design Papers: Managing Distributed, Shared L2 Caches through OS-Level Page Allocation, Cho and Jin, MICRO’06 Co-Operative Caching.
Managing Wire Delay in Large CMP Caches Bradford M. Beckmann David A. Wood Multifacet Project University of Wisconsin-Madison MICRO 2004 12/8/04.

Managing Wire Delay in Large CMP Caches Bradford M. Beckmann David A. Wood Multifacet Project University of Wisconsin-Madison MICRO /8/04.
ACM: An Efficient Approach for Managing Shared Caches in Chip Multiprocessors Mohammad Hammoud, Sangyeun Cho, and Rami Melhem Presenter: Socrates Demetriades.

ACM: An Efficient Approach for Managing Shared Caches in Chip Multiprocessors Mohammad Hammoud, Sangyeun Cho, and Rami Melhem Presenter: Socrates Demetriades.
Optimizing Shared Caches in Chip Multiprocessors Samir Sapra Athula Balachandran Ravishankar Krishnaswamy.

Optimizing Shared Caches in Chip Multiprocessors Samir Sapra Athula Balachandran Ravishankar Krishnaswamy.
Zhongkai Chen 3/25/2010. Jinglei Wang; Yibo Xue; Haixia Wang; Dongsheng Wang Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China This paper.

Zhongkai Chen 3/25/2010. Jinglei Wang; Yibo Xue; Haixia Wang; Dongsheng Wang Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China This paper.
Zhiguo Ge, Weng-Fai Wong, and Hock-Beng Lim Proceedings of the Design, Automation, and Test in Europe Conference, 2007 (DATE’07) April 2007 2015/4/17.

Zhiguo Ge, Weng-Fai Wong, and Hock-Beng Lim Proceedings of the Design, Automation, and Test in Europe Conference, 2007 (DATE’07) April /4/17.
Better than the Two: Exceeding Private and Shared Caches via Two-Dimensional Page Coloring Lei Jin and Sangyeun Cho Dept. of Computer Science University.

Better than the Two: Exceeding Private and Shared Caches via Two-Dimensional Page Coloring Lei Jin and Sangyeun Cho Dept. of Computer Science University.
1 Lecture 16: Large Cache Design Papers: An Adaptive, Non-Uniform Cache Structure for Wire-Dominated On-Chip Caches, Kim et al., ASPLOS’02 Distance Associativity.

1 Lecture 16: Large Cache Design Papers: An Adaptive, Non-Uniform Cache Structure for Wire-Dominated On-Chip Caches, Kim et al., ASPLOS’02 Distance Associativity.
112-12-2001CS752 Decoupled Architecture for Data Prefetching Jichuan Chang Kai Xu.

CS752 Decoupled Architecture for Data Prefetching Jichuan Chang Kai Xu.
1 Lecture 12: Large Cache Design Papers (papers from last class and…): Co-Operative Caching for Chip Multiprocessors, Chang and Sohi, ISCA’06 Victim Replication,

1 Lecture 12: Large Cache Design Papers (papers from last class and…): Co-Operative Caching for Chip Multiprocessors, Chang and Sohi, ISCA’06 Victim Replication,
HK-NUCA: Boosting Data Searches in Dynamic NUCA for CMPs Javier Lira ψ Carlos Molina ф Antonio González ψ,λ λ Intel Barcelona Research Center Intel Labs.

HK-NUCA: Boosting Data Searches in Dynamic NUCA for CMPs Javier Lira ψ Carlos Molina ф Antonio González ψ,λ λ Intel Barcelona Research Center Intel Labs.
1 Lecture 16: Large Cache Innovations Today: Large cache design and other cache innovations Midterm scores  91-80: 17 students  79-75: 14 students 

1 Lecture 16: Large Cache Innovations Today: Large cache design and other cache innovations Midterm scores  91-80: 17 students  79-75: 14 students 
1 Lecture 2: Snooping and Directory Protocols Topics: Snooping wrap-up and directory implementations.

1 Lecture 2: Snooping and Directory Protocols Topics: Snooping wrap-up and directory implementations.
1 Lecture 8: Large Cache Design I Topics: Shared vs. private, centralized vs. decentralized, UCA vs. NUCA, recent papers.

1 Lecture 8: Large Cache Design I Topics: Shared vs. private, centralized vs. decentralized, UCA vs. NUCA, recent papers.
1 E. Bolotin – The Power of Priority, NoCs 2007 The Power of Priority : NoC based Distributed Cache Coherency Evgeny Bolotin, Zvika Guz, Israel Cidon,

1 E. Bolotin – The Power of Priority, NoCs 2007 The Power of Priority : NoC based Distributed Cache Coherency Evgeny Bolotin, Zvika Guz, Israel Cidon,
1 Lecture 11: Large Cache Design Topics: large cache basics and… An Adaptive, Non-Uniform Cache Structure for Wire-Dominated On-Chip Caches, Kim et al.,

1 Lecture 11: Large Cache Design Topics: large cache basics and… An Adaptive, Non-Uniform Cache Structure for Wire-Dominated On-Chip Caches, Kim et al.,

Similar presentations

Ads by Google