Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.1 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Csaba Andras Moritz.

Slides:



Advertisements
Similar presentations
Dynamic History-Length Fitting: A third level of adaptivity for branch prediction Toni Juan Sanji Sanjeevan Juan J. Navarro Department of Computer Architecture.
Advertisements

Instruction-Level Parallelism compiler techniques and branch prediction prepared and Instructed by Shmuel Wimer Eng. Faculty, Bar-Ilan University March.
1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.
Computer Organization and Architecture (AT70.01) Comp. Sc. and Inf. Mgmt. Asian Institute of Technology Instructor: Dr. Sumanta Guha Slide Sources: Based.
CPE 631: Branch Prediction Electrical and Computer Engineering University of Alabama in Huntsville Aleksandar Milenkovic,
Dynamic Branch Prediction
Limits on ILP. Achieving Parallelism Techniques – Scoreboarding / Tomasulo’s Algorithm – Pipelining – Speculation – Branch Prediction But how much more.
3.13. Fallacies and Pitfalls Fallacy: Processors with lower CPIs will always be faster Fallacy: Processors with faster clock rates will always be faster.
Copyright 2001 UCB & Morgan Kaufmann ECE668.1 Adapted from Patterson, Katz and Culler © UCB Csaba Andras Moritz UNIVERSITY OF MASSACHUSETTS Dept. of Electrical.
CPE 731 Advanced Computer Architecture ILP: Part II – Branch Prediction Dr. Gheith Abandah Adapted from the slides of Prof. David Patterson, University.
CPE 631: Branch Prediction Electrical and Computer Engineering University of Alabama in Huntsville Aleksandar Milenkovic,
W04S1 COMP s1 Seminar 4: Branch Prediction Slides due to David A. Patterson, 2001.
Lecture 3: Branch Prediction Young Cho Graduate Computer Architecture I.
EECC551 - Shaaban #1 lec # 5 Spring Reduction of Control Hazards (Branch) Stalls with Dynamic Branch Prediction So far we have dealt with.
EECE476: Computer Architecture Lecture 20: Branch Prediction Chapter extra The University of British ColumbiaEECE 476© 2005 Guy Lemieux.
EECC551 - Shaaban #1 lec # 5 Fall Reduction of Control Hazards (Branch) Stalls with Dynamic Branch Prediction So far we have dealt with.
EENG449b/Savvides Lec /17/04 February 17, 2004 Prof. Andreas Savvides Spring EENG 449bG/CPSC 439bG.
Goal: Reduce the Penalty of Control Hazards
Branch Target Buffers BPB: Tag + Prediction
EECC551 - Shaaban #1 lec # 5 Winter Reduction of Control Hazards (Branch) Stalls with Dynamic Branch Prediction So far we have dealt with.
1 Lecture 8: Instruction Fetch, ILP Limits Today: advanced branch prediction, limits of ILP (Sections , )
Computer Architecture Instruction Level Parallelism Dr. Esam Al-Qaralleh.
Branch Prediction Dimitris Karteris Rafael Pasvantidιs.
So far we have dealt with control hazards in instruction pipelines by:
Dynamic Branch Prediction
EENG449b/Savvides Lec /25/05 March 24, 2005 Prof. Andreas Savvides Spring g449b EENG 449bG/CPSC 439bG.
CIS 429/529 Winter 2007 Branch Prediction.1 Branch Prediction, Multiple Issue.
Pipelined Datapath and Control (Lecture #15) ECE 445 – Computer Organization The slides included herein were taken from the materials accompanying Computer.
1 Lecture 7: Branch prediction Topics: bimodal, global, local branch prediction (Sections )
ENGS 116 Lecture 91 Dynamic Branch Prediction and Speculation Vincent H. Berk October 10, 2005 Reading for today: Chapter 3.2 – 3.6 Reading for Wednesday:
CPSC614 Lec 5.1 Instruction Level Parallelism and Dynamic Execution #4: Based on lectures by Prof. David A. Patterson E. J. Kim.
Ch2. Instruction-Level Parallelism & Its Exploitation 2. Dynamic Scheduling ECE562/468 Advanced Computer Architecture Prof. Honggang Wang ECE Department.
Branch Prediction High-Performance Computer Architecture Joe Crop Oregon State University School of Electrical Engineering and Computer Science.
1 Dynamic Branch Prediction. 2 Why do we want to predict branches? MIPS based pipeline – 1 instruction issued per cycle, branch hazard of 1 cycle. –Delayed.
CSCI 6461: Computer Architecture Branch Prediction Instructor: M. Lancaster Corresponding to Hennessey and Patterson Fifth Edition Section 3.3 and Part.
Branch.1 10/14 Branch Prediction Static, Dynamic Branch prediction techniques.
Branch Prediction Prof. Mikko H. Lipasti University of Wisconsin-Madison Lecture notes based on notes by John P. Shen Updated by Mikko Lipasti.
CPE 631 Session 17 Branch Prediction Electrical and Computer Engineering University of Alabama in Huntsville.
CS 6290 Branch Prediction. Control Dependencies Branches are very frequent –Approx. 20% of all instructions Can not wait until we know where it goes –Long.
Adapted from Computer Organization and Design, Patterson & Hennessy, UCB ECE232: Hardware Organization and Design Part 13: Branch prediction (Chapter 4/6)
Yiorgos Makris Professor Department of Electrical Engineering University of Texas at Dallas EE (CE) 6304 Computer Architecture Lecture #13 (10/28/15) Course.
Dynamic Branch Prediction
CMSC 611: Advanced Computer Architecture
Computer Organization CS224
CS203 – Advanced Computer Architecture
Computer Structure Advanced Branch Prediction
Dynamic Branch Prediction
UNIVERSITY OF MASSACHUSETTS Dept
COSC3330 Computer Architecture Lecture 15. Branch Prediction
CMSC 611: Advanced Computer Architecture
The processor: Pipelining and Branching
Lecture 14: Reducing Cache Misses
So far we have dealt with control hazards in instruction pipelines by:
CPE 631: Branch Prediction
Branch statistics Branches occur every 4-6 instructions (16-25%) in integer programs; somewhat less frequently in scientific ones Unconditional branches.
Dynamic Branch Prediction
Pipelining and control flow
So far we have dealt with control hazards in instruction pipelines by:
Lecture 10: Branch Prediction and Instruction Delivery
So far we have dealt with control hazards in instruction pipelines by:
So far we have dealt with control hazards in instruction pipelines by:
Adapted from the slides of Prof
Dynamic Hardware Prediction
So far we have dealt with control hazards in instruction pipelines by:
So far we have dealt with control hazards in instruction pipelines by:
Wackiness Algorithm A: Algorithm B:
So far we have dealt with control hazards in instruction pipelines by:
So far we have dealt with control hazards in instruction pipelines by:
So far we have dealt with control hazards in instruction pipelines by:
CPE 631 Lecture 12: Branch Prediction
Presentation transcript:

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.1 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Csaba Andras Moritz UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 668 Design of Power-Aware Dynamic Branch Prediction and Control Flow

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.2 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Re-evaluating Correlation  Several SPEC benchmarks have less than a dozen branches responsible for 90% of taken branches: program branch % static# = 90% compress14%23613 eqntott25%4945 gcc15% mpeg10% real gcc13%  Real programs + OS more like gcc  Small benefits of correlation beyond benchmarks?  Mispredict because either:  Wrong guess for that branch  Got branch history of wrong branch when indexing the table  For SPEC92, 4096 about as good as infinite table  Misprediction mostly due to wrong prediction  Can we improve using global history?

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.3 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Gselect and Gshare predictors  Keep a global register (GR) with outcome of k branches  Use that in conjunction with PC to index into a table containing 2-bit predictors  Gselect – concatenate  Gshare – XOR (better) Copyright 2007 CAM

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.4 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Tournament Predictors  Motivation for correlating branch predictors: 2-bit local predictor failed on important branches; by adding global information, performance improved  Tournament predictors: use two predictors, 1 based on global information and 1 based on local information, and combine with a selector  Hopes to select right predictor for right branch (or right context of branch)

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.5 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Tournament Predictor in Alpha  4K 2-bit counters to choose from among a global predictor and a local predictor  Global predictor also has 4K entries and is indexed by the history of the last 12 branches; each entry in the global predictor is a standard 2-bit predictor  12-bit pattern: ith bit is 0 => ith prior branch not taken; ith bit is 1 => ith prior branch taken ; 00,10,11 00,11 10 Use 1 Use 2 Use 1 00,01,11 00, K  2 bits Global (or #1)

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.6 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Tournament Predictor in Alpha  Local predictor consists of a 2-level predictor:  Top level a local history table consisting of bit entries; each 10-bit entry corresponds to the most recent 10 branch outcomes for the entry. 10-bit history allows patterns 10 branches to be discovered and predicted  Next level Selected entry from the local history table is used to index a table of 1K entries consisting a 3-bit saturating counters, which provide the local prediction  Total size: 4K*2 + 4K*2 + 1K*10 + 1K*3 = 29K bits! (~180K transistors) 1K  10 bits 1K  3 bits (Local or #2)

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.7 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann % of predictions from local predictor in Tournament Prediction Scheme 98% 100% 94% 90% 55% 76% 72% 63% 37% 69% 0%20%40%60%80%100% nasa7 matrix300 tomcatv doduc spice fpppp gcc espresso eqntott li

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.8 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann 94% 96% 98% 97% 100% 70% 82% 77% 82% 84% 99% 88% 86% 88% 86% 95% 99% 0%20%40%60%80%100% gcc espresso li fpppp doduc tomcatv Profile-based 2-bit counter Tournament Accuracy of Branch Prediction  Profile: branch profile from last execution (static in that is encoded in instruction, but profile) fig 3.40

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.9 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Accuracy v. Size (SPEC89) 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% Total predictor size (Kbits) Conditional branch misprediction rate Local - 2 bit counters Correlating - (2,2) scheme Tournament

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.10 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Need Address at Same Time as Prediction  Branch Target Buffer (BTB): Address of branch used as index to get prediction AND branch address (if taken)  Note: must check for branch match now, since can’t use wrong branch address Branch PCPredicted PC =? PC of instruction FETCH Prediction state bits Yes: instruction is branch; use predicted PC as next PC (if predict Taken) No: branch not predicted; proceed normally (PC+4)

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.11 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Branch Target “Cache”  Branch Target cache - Only predicted taken branches  “Cache” - Content Addressable Memory (CAM) or Associative Memory (see figure)  Use a big Branch History Table & a small Branch Target Cache Branch PCPredicted PC =? Prediction state bits (optional) Yes: predicted taken branch found No: not found PC

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.12 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Steps with Branch target Buffer Branch_CPI_Penalty = [Buffer_hit_rate x P{Incorrect_prediction}] x Penalty_Cycles + [(1- Buffer_hit_rate) x P{Branch_taken}] x Penalty_Cycles =.0.91x0.1x x0.6x2 =.29 Taken Branch? Entry found in branch- target buffer? Send out predicted PC Is instruction a taken branch? Send PC to memory and branch-target buffer Enter branch instruction address and next PC into branch-target buffer Mispredicted branch, kill fetched instruction; restart fetch at other target; delete entry from target buffer Normal instruction execution Branch correctly predicted; continue execution with no stalls No Yes No ID IF EX for the 5-stage MIPS

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.13 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann  Avoid branch prediction by turning branches into conditionally executed instructions: if (x) then A = B op C else NOP  If false, then neither store result nor cause interference  Expanded ISA of Alpha, MIPS, PowerPC, SPARC have conditional move; PA-RISC can annul any following instruction  Drawbacks to conditional instructions  Still takes a clock even if “annulled”  Stall if condition evaluated late: Complex conditions reduce effectiveness since condition becomes known late in pipeline x A = B op C Predicated Execution

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.14 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Special Case: Return Addresses  Register Indirect branch - hard to predict address  SPEC89 85% such branches for procedure return  Since stack discipline for procedures, save return address in small buffer that acts like a stack: 8 to 16 entries has small miss rate

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.15 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann How to Reduce Power  Reduce load capacitances switched  Smaller BTAC  Smaller local, global predictors  Less associativity  How do you know which branches? »Use static information »Combine runtime and compile time information »Add hints to be used at runtime  Also, predict statically  Branch folding »Runtime »Compile-time Copyright 2007 CAM & BlueRISC

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.16 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Predict early/ahead  Early: Prefetch buffer like in the BlueRISC CPU and recent ARM CPUs allows branches to be prefetched. This means we can predict a branch dynamically in time before entering pipeline.  Ahead: BlueRISC’s patented approach allows predicting several cycles ahead of the branch. So, one would not need to predict with every PC, one can wait to predict only branch instructions.  Issues: needs ISA support.  Cannot be done unless there is time (not possible after a flush or short basic blocks, requires prefetch buffer in both cases)

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.17 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Predict Statically  Use fixed static prediction when possible  Incurs limited power at runtime  Requires ISA support if not all branches are statically predicted

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.18 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Ignore  Sometimes the best is to not predict – this especially for branches that are hard to predict  Also, ignore branches that are hard to predict but that are NOT on critical path  Requires ISA support to best implement

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.19 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Phantom Branches  ARM removes branches to save power in the prefetch buffer  BlueRISC CPUs can remove them from the binary  Control-flow change 70% of the time needs fewer bits than a typical branch instruction would entail  This is most branches are short span requiring few bits only and a condition  Can be implemented cleverly and combined with other compiler exposed techniques

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.20 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Hybrid History Tables  Use simple predictor on simple branches and complex when needed  Do not store all history everywhere  E.g., 2-bit combined with Tournament/gshare  Small complex predictor preserved for hard & critical branches  Needs compiler support

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.21 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Hierarchical BTAC  Micro BTAC (mBTAC) a small CAM based BTAC with lower load capacitance  Use mBTAC when possible  Otherwise a BTAC can be similar to an I$ access and if you do every cycle it adds up  But, for every branch, either in hardware or with compiler we need to tell which predictor to use.  A compiler driven approach needs ISA support or some way to encode this. One could use a memory- mapped IO and preload this information but adds control which can impact clock speed if overdone 

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.22 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Pitfall: Sometimes dumber is better  Alpha uses tournament predictor (29 Kbits)  Earlier uses a simple 2-bit predictor with 2K entries (or a total of 4 Kbits)  SPEC95 benchmarks, outperforms  avg mispredictions per 1000 instructions  avg mispredictions per 1000 instructions  Reversed for transaction processing (TP) !  avg. 17 mispredictions per 1000 instructions  avg. 15 mispredictions per 1000 instructions  TP code much larger & hold 2X branch predictions based on local behavior (2K vs. 1K local predictor in the 21264)  What about power?  Large predictors give some increase in prediction rate but for a large power cost as discussed

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.23 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Finally  Understand that not all branches are made equal – those that are not on the critical path are simply not critical to do well on  Some branches are critical (part of critical loops and in top basic blocks) those need the best you got (advanced small predictors)  Performance is affected by branching a lot  The higher the issue width the more critical to do well  Next we look at some results based on these techniques

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.24 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann Power Consumption BlueRISC’s Compiler-driven Power-Aware Branch Prediction Comparison with 512 entry BTAC bimodal (patent-issued) Copyright 2007 CAM & BlueRISC

Copyright 2016 Csaba Andras MoritzECE668 Power Aware Branching.25 Few slides adapted from Patterson, et al © UCB and Morgan Kaufmann