Presentation is loading. Please wait.

Presentation is loading. Please wait.

Stream Chaining: Exploiting Multiple Levels of Correlation in Data Prefetching Pedro Díaz and Marcelo Cintra University of Edinburgh

Published byJanice Center Modified over 10 years ago

Similar presentations

Presentation on theme: "Stream Chaining: Exploiting Multiple Levels of Correlation in Data Prefetching Pedro Díaz and Marcelo Cintra University of Edinburgh"— Presentation transcript:

1 Stream Chaining: Exploiting Multiple Levels of Correlation in Data Prefetching Pedro Díaz and Marcelo Cintra University of Edinburgh http://www.homepages.inf.ed.ac.uk/mc/Projects/CELLULAR

2 ISCA 20092 Outline  Motivation  Correlation and Localization  Stream Chaining and Miss Graph Prefetching  Experimental Setup and Results  Related Work  Conclusions

3  The Memory Wall is still a problem –After decades of logic and DRAM technology disparity, memory access costs hundreds of processor cycles –On-chip cache quotas per processor unlikely to increase –Off-chip memory bandwidth quota per processor likely to decrease (unless some fancy memory technology succeeds)  (Hardware) Prefetching is a viable solution –Time-tested approach used in most commercial processors –Trades-off memory bandwidth for latency (especially good if some fancy memory technology succeeds) ISCA 20093 The “Memory Wall” and Prefetching

4 ISCA 20094 Prefetching  Prefetchers work by uncovering patterns in the miss address stream: correlation (e.g., address deltas)  Prefetchers often separate misses into multiple streams: localization (e.g., by instruction)  To eliminate more misses and hide longer latencies prefetchers often use prefetch degree greater than one  Prefetchers often measured against three metrics: –Accuracy: ratio of used prefetches over all prefetches –Coverage: ratio of used prefetches over original misses –Timeliness: data arrives too early, too late, or just in time

5 ISCA 20095 The Problem with Prefetching  Correlation on global miss stream often suffers from poor accuracy  Prefetching along localized streams often suffers from poor coverage and timeliness –Streams lose time ordering information of misses –“Cold” misses across stream boundaries  Deep prefetching suffers from diminishing accuracy  Applications access patterns exhibit different correlation patterns Ideally what we want is to combine multiple localized streams to improve coverage and timeliness while keeping accuracy high

6 ISCA 20096 Outline  Motivation  Correlation and Localization  Stream Chaining and Miss Graph Prefetching  Experimental Setup and Results  Related Work  Conclusions

7 ISCA 20097 Correlation  Establishing “relationship” among addresses of misses. For instance: –Sequential: miss to line L is followed by miss to line L+1 –Time : miss to address A is followed by miss to address B –Delta: miss to address A is followed by miss to address A+d –Markov: e.g., miss to address A is followed by miss to address B with probability p and miss to address C with probability (1-p)  Correlations are found by inspecting miss history and are used to predict next miss

8 ISCA 20098 Localization  Complete global history is undesirable in most cases –Misses from unrelated sources (e.g., from pointer chasing followed by data object manipulation) –“Wild” interleaving of misses (e.g., OOO execution, infrequent control flow) –Correlations over long traces  Localization: group misses according to some common property. For instance: –PC: misses from same static instruction –Temporal: misses that occur at about the same time –Spatial: misses to similar regions in memory address space  Attempts to exploit some high-level behaviour

9 9 Localization PC_A : A1 PC_B : A2 PC_A : A7 PC_D : A5 PC_B : A8 PC_A : A1 PC_B : A2 PC_C : A4 PC_E : A6 PC_A : A11 PC_B : A12 PC_A : A1 PC_B : A2 PC_A : A7 PC_B : A8 Miss Stream (PC : Addr) time A1 A2 A3 A4 A11 A12 A13 A14 A7 A8 A9 A10 Memory Address Space A5 A6 PC Localized Streams: A1 → A7 → A1 → A11 → A1 → A7 A2 → A8 → A2 → A12 → A2 → A8 PC Correlation ISCA 2009

10 10 Localization PC_A : A1 PC_B : A2 PC_A : A7 PC_D : A5 PC_B : A8 PC_A : A1 PC_B : A2 PC_C : A4 PC_E : A6 PC_A : A11 PC_B : A12 PC_A : A1 PC_B : A2 PC_A : A7 PC_B : A8 Miss Stream (PC : Addr) time A1 A2 A3 A4 A11 A12 A13 A14 A7 A8 A9 A10 Memory Address Space A5 A6 Temporal Correlation Time Localized Streams: A1 → A2 → A7 → A5 → A8 A1 → A2 → A4 → A6 → A11 → A12 A1 → A2 → A7 → A8 ISCA 2009

11 11 Localization PC_A : A1 PC_B : A2 PC_A : A7 PC_D : A5 PC_B : A8 PC_A : A1 PC_B : A2 PC_C : A4 PC_E : A6 PC_A : A11 PC_B : A12 PC_A : A1 PC_B : A2 PC_A : A7 PC_B : A8 Miss Stream (PC : Addr) time A1 A2 A3 A4 A11 A12 A13 A14 A7 A8 A9 A10 Memory Address Space A5 A6 Space Localized Streams: A1 → A2 A1 → A2 → A4 A7 → A8 Spatial Correlation A11 → A12 ISCA 2009

12 12 Outline  Motivation  Correlation and Localization  Stream Chaining and Miss Graph Prefetching  Experimental Setup and Results  Related Work  Conclusions

13 ISCA 200913 Stream Chaining: Idea and Operation  Chain streams: –Start from global, ordered, miss stream –Perform localization and build localized streams –Order and link streams according to program execution to partially reconstruct order of misses  Prefetch –On a miss to stream A follow chain and identify streams that commonly follow A –Perform correlation on each stream individually –Prefetch data for streams that follow A and, possibly, also for A itself

14 ISCA 200914 Benefits and Limitations + Recover chronological information following program’s stable memory access pattern + Still eliminate “spurious” misses + Still benefit from better predictability of localized streams + Prefetch across stream boundaries + Better use of large prefetch degrees - Stream chain patterns must be stable - Stream chains must be relatively small as to be manageable - Longer run time of algorithm as must correlate on multiple streams

15 ISCA 200915 Miss Graph Prefetcher  Based on Nesbitt and Smith’s GHB structure (HPCA’04)  Uses PC localization with delta correlation (PC/DC)  Represents stream chains as simple directed graphs –Nodes represent streams and edges represent time ordering (i.e., miss to stream A is followed by miss to stream B A→B) –Only 1 outgoing edge per node but multiple incoming edges possible –Edges only added to recurring sequences by using a threshold –Cycles allowed  Named PC/DC/MG

16 ISCA 200916 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time PC_A PC_B PC_C PC_D PC_E Index Table Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E

17 ISCA 200917 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time  Step 1: perform localization → already part of GHB funct. PC_A PC_B PC_C PC_D PC_E Index Table Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E

18 ISCA 200918 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time  Step 2: chain streams PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E 0 0 0 0 0

19 ISCA 200919 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time  Step 2: chain streams PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E 0 1 0 0 0

20 ISCA 200920 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time  Step 2: chain streams PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E 0 1 1 0 0

21 ISCA 200921 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time  Step 2: chain streams PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E 1 1 1 1 1

22 ISCA 200922 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time  Step 2: chain streams PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E 1 1 1 1 1

23 ISCA 200923 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E 3 2 1 1 3  Step 2: chain streams

24 ISCA 200924 Miss Graph Prefetcher PC_A : A1 PC_B : B1 PC_C : C1 PC_D : D1 PC_E : E1 PC_A : A2 PC_D : D2 PC_E : E2 PC_A : A3 PC_D : D3 PC_E : E3 PC_A : A4 Miss Stream (PC : Addr) time PC_A PC_B PC_C PC_D PC_E Index Table NextCtr current miss Global History Buffer A 1 B 1 C 1 D 1 E 1 A 2 D 2 E 2 A 3 D 3 E 3 A 4 PC_B PC_D PC_C PC_E  Step 3: perform correlations and prefetch along streams Note that we do not prefetch for A, but rely on “peers” (i.e., D and/or E) to prefetch for A

25 Miss Graph Example  perlbench (512KB L2) 25ISCA 2009

26 26 Outline  Motivation  Correlation and Localization  Stream Chaining and Miss Graph Prefetching  Experimental Setup and Results  Related Work  Conclusions

27 ISCA 200927 Experimental Setup  Simulator: –SESC: cycle-accurate architectural simulator from UIUC  Applications: SPEC2006 and BioBench  Architecture: –5GHz, 4-issue superscalar MIPS processor –64KB, 2-way L1 I-Cache and 64KB, 2-way L1 D-Cache –256KB/2MB, 8-way L2 cache –64bit, 1.25GHz memory bus –Main memory: 400 cycle latency

28 ISCA 200928 Performance Without Prefetching Some applications already perform well (within 15% of ideal) with 512KB L2 Many applications still perform poorly some even with large 2MB L2

29 ISCA 200929 Performance With Prefetching Best performing prefetching scheme varies across applications Overall, PC/DC/MG performs best or close to best in most applications

30 ISCA 200930 Prefetch Coverage PC/DC often has lowest coverage, and PC/DC/MG and G/DC vary across applications

31 ISCA 200931 … and Accuracy PC/DC/MG is often the most accurate, and PC/DC is often more accurate than G/DC

32 Benchmark UniqueNodes SubgraphsSnapshotCC (%)maxavg.maxavg. milc4.7157.773.6 lbm22207.9183.7 lbq0.82319187 zeusmp11181194.4 clustalw1.1109.3108.2 perl11168.693.3 namd2185.885 soplex2.83012103.6 bzip25.6382093.8 tiger5.44130184.2 hmmer125038335.4 gobmk20105.253.4 ISCA 200932 Miss Graphs Statistics Most graphs appear repeatedly during execution → potential for learning Moreover (results not shown) graphs are stable for long periods of time → potential to exploit patterns Number of nodes at any given time is small → manageable to keep track of Number of nodes per stream groups is small → small protocol execution overheads

33 ISCA 200933 Next-Stream Prediction Accuracy Miss-graph’s prediction accuracy is often very high 33ISCA 2009

34 34 Outline  Motivation  Correlation and Localization  Stream Chaining and Miss Graph Prefetching  Experimental Setup and Results  Related Work  Conclusions

35 ISCA 200935 (Closest) Related Work  K. Nesbit and J. Smith – HPCA’04 –Proposed GHB and introduced PC/DC  S. Somogyi, T. Wenisch, A. Ailamaki, and B. Falsafi – ISCA’09 –Combined spatial and temporal memory streaming –Can be seen as close to a PID/SMS/TMS prefetcher (except that PID is not used to index at prefetch time)

36 ISCA 200936 Outline  Motivation  Correlation and Localization  Stream Chaining and Miss Graph Prefetching  Experimental Setup and Results  Related Work  Conclusions

37 ISCA 200937 Conclusions  New strategy for creating prefetchers by composing (chaining) localization and correlation schemes  New prefetcher based on the Stream Chaining idea –Simple extension of GHB-based PC/DC of Nesbit and Smith (HPCA’04) –Captures most of the stable miss sequences in the programs tested –Overall better performance than PC/DC or G/DC  Stream Chaining could be applied to other localization and correlation schemes (we are working on it)

38 Stream Chaining: Exploiting Multiple Levels of Correlation in Data Prefetching Pedro Díaz and Marcelo Cintra University of Edinburgh http://www.homepages.inf.ed.ac.uk/mc/Projects/CELLULAR

39 ISCA 200939 Miss Distances Global miss distances are often in the order of tens or hundreds of cycles only PC localized miss distances are always much larger, often in the order of thousands or tens of thousands 39ISCA 2009

40 Miss graph prefetching  Prefetch operation Long enough, linear stream: Prefetch 1 item from PC_B onwards Not long enough or cyclic chains: Prefetch degree/length items per stream 40ISCA 2009

41 Miss Graph examples  bzip2 (2048KB L2) 41ISCA 2009

42 Miss Graph examples  lbm (512KB L2) 42ISCA 2009

43 Miss Graph examples  libquantum (256KB L2) 43ISCA 2009

Download ppt "Stream Chaining: Exploiting Multiple Levels of Correlation in Data Prefetching Pedro Díaz and Marcelo Cintra University of Edinburgh"

Similar presentations

A Preliminary Attempt ECEn 670 Semester Project Wei Dang Jacob Frogget Poisson Processes and Maximum Likelihood Estimator for Cache Replacement.

A Preliminary Attempt ECEn 670 Semester Project Wei Dang Jacob Frogget Poisson Processes and Maximum Likelihood Estimator for Cache Replacement.
Coherence Ordering for Ring-based Chip Multiprocessors Mike Marty and Mark D. Hill University of Wisconsin-Madison.

Coherence Ordering for Ring-based Chip Multiprocessors Mike Marty and Mark D. Hill University of Wisconsin-Madison.
1 A Hybrid Adaptive Feedback Based Prefetcher Santhosh Verma, David Koppelman and Lu Peng Louisiana State University.

1 A Hybrid Adaptive Feedback Based Prefetcher Santhosh Verma, David Koppelman and Lu Peng Louisiana State University.
1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.

1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.
Practical Caches COMP25212 cache 3. Learning Objectives To understand: –Additional Control Bits in Cache Lines –Cache Line Size Tradeoffs –Separate I&D.

Practical Caches COMP25212 cache 3. Learning Objectives To understand: –Additional Control Bits in Cache Lines –Cache Line Size Tradeoffs –Separate I&D.
1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.

1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.
Data Marshaling for Multi-Core Architectures M. Aater Suleman Onur Mutlu Jose A. Joao Khubaib Yale N. Patt.

Data Marshaling for Multi-Core Architectures M. Aater Suleman Onur Mutlu Jose A. Joao Khubaib Yale N. Patt.
Exploiting Spatial Locality in Data Caches using Spatial Footprints Sanjeev Kumar, Princeton University Christopher Wilkerson, MRL, Intel.

Exploiting Spatial Locality in Data Caches using Spatial Footprints Sanjeev Kumar, Princeton University Christopher Wilkerson, MRL, Intel.
Is SC + ILP = RC? Presented by Vamshi Kadaru Chris Gniady, Babak Falsafi, and T. N. VijayKumar - Purdue University Spring 2005: CS 7968 Parallel Computer.

Is SC + ILP = RC? Presented by Vamshi Kadaru Chris Gniady, Babak Falsafi, and T. N. VijayKumar - Purdue University Spring 2005: CS 7968 Parallel Computer.
Accurately Approximating Superscalar Processor Performance from Traces Kiyeon Lee, Shayne Evans, and Sangyeun Cho Dept. of Computer Science University.

Accurately Approximating Superscalar Processor Performance from Traces Kiyeon Lee, Shayne Evans, and Sangyeun Cho Dept. of Computer Science University.
CS7810 Prefetching Seth Pugsley. Predicting the Future Where have we seen prediction before? – Does it always work? Prefetching is prediction – Predict.

CS7810 Prefetching Seth Pugsley. Predicting the Future Where have we seen prediction before? – Does it always work? Prefetching is prediction – Predict.
Teaching Old Caches New Tricks: Predictor Virtualization Andreas Moshovos Univ. of Toronto Ioana Burcea’s Thesis work Some parts joint with Stephen Somogyi.

Teaching Old Caches New Tricks: Predictor Virtualization Andreas Moshovos Univ. of Toronto Ioana Burcea’s Thesis work Some parts joint with Stephen Somogyi.
Access Region Locality for High- Bandwidth Processor Memory System Design Sangyeun Cho Samsung/U of Minnesota Pen-Chung Yew U of Minnesota Gyungho Lee.

Access Region Locality for High- Bandwidth Processor Memory System Design Sangyeun Cho Samsung/U of Minnesota Pen-Chung Yew U of Minnesota Gyungho Lee.
Data Prefetching Mechanism by Exploiting Global and Local Access Patterns Ahmad SharifQualcomm Hsien-Hsin S. LeeGeorgia Tech The 1 st JILP Data Prefetching.

Data Prefetching Mechanism by Exploiting Global and Local Access Patterns Ahmad SharifQualcomm Hsien-Hsin S. LeeGeorgia Tech The 1 st JILP Data Prefetching.
112-12-2001CS752 Decoupled Architecture for Data Prefetching Jichuan Chang Kai Xu.

CS752 Decoupled Architecture for Data Prefetching Jichuan Chang Kai Xu.
Increasing the Cache Efficiency by Eliminating Noise Philip A. Marshall.

Increasing the Cache Efficiency by Eliminating Noise Philip A. Marshall.
CS 7810 Lecture 7 Trace Cache: A Low Latency Approach to High Bandwidth Instruction Fetching E. Rotenberg, S. Bennett, J.E. Smith Proceedings of MICRO-29.

CS 7810 Lecture 7 Trace Cache: A Low Latency Approach to High Bandwidth Instruction Fetching E. Rotenberg, S. Bennett, J.E. Smith Proceedings of MICRO-29.
WCED: June 7, 2003 Matt Ramsay, Chris Feucht, & Mikko Lipasti University of Wisconsin-MadisonSlide 1 of 26 Exploring Efficient SMT Branch Predictor Design.

WCED: June 7, 2003 Matt Ramsay, Chris Feucht, & Mikko Lipasti University of Wisconsin-MadisonSlide 1 of 26 Exploring Efficient SMT Branch Predictor Design.
Perceptron-based Global Confidence Estimation for Value Prediction Master’s Thesis Michael Black June 26, 2003.

Perceptron-based Global Confidence Estimation for Value Prediction Master’s Thesis Michael Black June 26, 2003.
1 Improving Branch Prediction by Dynamic Dataflow-based Identification of Correlation Branches from a Larger Global History CSE 340 Project Presentation.

1 Improving Branch Prediction by Dynamic Dataflow-based Identification of Correlation Branches from a Larger Global History CSE 340 Project Presentation.

Similar presentations

Ads by Google