Phase Capture and Prediction with Applications Martin Hock Brian Pellin Karthik Jayaraman Vivek Shrivastava University of Wisconsin-Madison

Phases Definition: A period of execution that exhibits the same characteristics

Motivation Programs go through different phases of their execution Phases are often repeated at different times in execution During each phase hardware is exercised differently

Sample Phase Behavior : gcc

Outline Phase Tracking Phase Prediction Applications Phase Based Branch Prediction Phase Based Cache Configuration Summary / Conclusions

Phase Tracking Goal: Identify program phases with different behavior Based on “Phase Tracking and Prediction” [Sherwood, Sair, Calder] Use reconfigurable hardware to take advantage of phase information Reconfigurable caches Instruction window size Dynamic branch predictor

Detecting Phases Track groups of 10 million instructions Collect information about instructions and store Build a phase footprint After each 10 m insts. Compare footprint with past footprints If footprint close enough, it is considered a repetition of the phase

Accumulator Branch PC Hash # of inst. since branch +

Accumulator Branch PC 2 Hash # of inst. since branch 20 + Branch occurs, must increment entry 2 by 20.

Accumulator Branch PC 20 3 Hash # of inst. since branch 80 + New branch, increment entry 3 by 10.

Accumulator Branch PC 20 80 Hash # of inst. since branch + After a phase completes we need somewhere to store data about previous phases.

Past Footprint Table Accumulator Branch PC 20 80 Hash # of inst. since branch + *At 100 instructions

Past Footprint Past Footprint Table Accumulator Branch PC 20 80 Hash # of inst. since branch + Accumulator Data is stored in Past Footprint table

Past Footprint Table Past Footprint Accumulator 90 Branch PC 20 5 80 Hash # of inst. since branch 5 + *At 200 instructions Take the Manhattan distance between accumulator and Past Footprints 90+20+75+5 = 190

Past Footprint Table Past Footprint Accumulator 90 Branch PC 20 80 5 Hash # of inst. since branch 5 + *At 200 instructions

Past Footprint Past Footprint Table Accumulator 90 Branch PC 21 20 79 80 5 Hash # of inst. since branch 5 + *At 300 instructions Manhattan distance between this phase and first phase is 2. This phase is close enough to the first phase to be considered the same as phase one.

Past Footprint Past Footprint Table Accumulator 430 Branch PC 21 20 9 10 80 Hash # of inst. since branch 70 + *At 30 million instructions Manhattan distance between this phase and first phase is 2. This phase is close enough to the first phase to be considered the same as phase one.

Outline Phase Tracking Phase Prediction Applications Phase Based Branch Prediction Phase Based Cache Configuration Summary / Conclusions

Phase prediction When we detect a phase, it’s over In order to adjust hardware, we need to know what phase we are in Three strategies Last seen Markov with RLE Perceptron

Last seen Predict next phase = last phase Because last seen is so simple, another predictor would have to beat it significantly to justify the added cost

RLE Markov Adapted from Sherwood Assumes that if we see phase X exactly Y times in a row, followed by phase Z, then if we see phase X exactly Y times again, it will again be followed by Z

Perceptron Individual perceptrons work in binary (±1) Given history h1, h2, …, hn (±1), weights w0, w1, w2, …, wn (integers), compute S = w0 + w1h1 + w2h2 + … + wnhn If S ≥ 0, predict “yes”, else predict “no” To train, if hi = current , increment wi, else decrement (for w0, add current) But there are many phases, not just 2 Combine perceptrons for multivalue prediction

Multivalue perceptron We have perceptrons P1, P2, …, Pn Perceptron Pi tries to predict phase i Train Pi only if in phase i History hi = 1 if it agrees with the current phase, -1 if disagrees Have the perceptrons vote for who is correct – most positive one wins

Phase prediction results GCC: Last phase: 96% accurate RLE Markov: 94% accurate Perceptron: much lower

Phase prediction comments Sherwood had lower accuracy for last phase (70%), perhaps due to oscillation Training cost of multiple perceptron means that it does not always adapt quickly Not worth improving due to the accuracy of last phase

Outline Phase Tracking Phase Prediction Applications Phase Based Branch Prediction Phase Based Cache Configuration Summary / Conclusions

Phase Based Dynamic Branch Predictor Previous research shows the usefulness of adapting branch predictors at run time “Dynamic history-length fitting: a third level of adaptivity for branch prediction” [Juan, Sanjeevan, Navarro]. “Combining Branch Predictors” [McFarling] Single branch predictor may not perform well within and across different executions. “A study of Branch Prediction Strategies” [Smith] Program behavior almost uniform within a phase -> choose best predictor for each phase

Methodology Select a small group of relevant predictors At the beginning of each new phase, sample all the predictors and choose the best Save the best for each phase and use it if a phase reoccurs

Methodology Select a small group of relevant predictors At the beginning of each new phase, sample all the predictors and choose the best Save the best for each phase and use it if a phase reoccurs

Methodology Select a small group of relevant predictors At the beginning of each new phase, sample all the predictors and choose the best Save the best for each phase and use it if a phase reoccurs

Methodology Select a small group of relevant predictors At the beginning of each new phase, sample all the predictors and choose the best Save the best for each phase and use it if a phase reoccurs

Methodology Select a small group of relevant predictors At the beginning of each new phase, sample all the predictors and choose the best Save the best for each phase and use it if a phase reoccurs Phase 1

Methodology Select a small group of relevant predictors At the beginning of each new phase, sample all the predictors and choose the best Save the best for each phase and use it if a phase reoccurs Phase 1 Phase 2

Dynamic Adaptations Possible dynamic adaptations Multiple Branch Predictors 2Level, Bimodal Sample each for one profiling period Select on basis of [miss rate, number of mis-speculated instructions, …] Varying History Lengths History lengths [0,12] Some workloads give better performance with smaller history

Multiple Branch Predictors Set of predictors 2level [1:1024:8] (Baseline predictor) Bimodal [1024] 2level [8: 512 :8] 2level [1: 512 :8] Profile period 10 million instructions

Multiple Branch Predictors Simulator Used Simplescalar v3.0d Set of benchmarks gcc, vpr, mcf, ammp, art Selection Criterion Least Miss Rate If miss rates of two predictors are within 1 %, select the less expensive (simpler) one

Multiple Branch Predictor : Results IPC (gcc)

Multiple Branch Predictors: Results Branch Predictor Misses (gcc)

Multiple Branch Predictor : Results IPC (vpr)

Multiple Branch Predictors: Results Branch Predictor Misses (vpr)

Multiple Branch Predictors: Results Branch Predictor Misses (mcf)

Multiple Branch Predictors IPC Comparison

Multiple Branch Predictors Branch Prediction Misses Comparison

Varying History Length G-share predictor with varying history lengths Set of history lengths sampled [0,3,6,8,12] Selection Criterion Least Miss Rate If miss rates of two predictors are within 1 %, select the less expensive (simpler) one

Varying History Length Set of benchmarks gcc, mcf Simulator Used Simplescalar v3.0d Profile Period 10 million instructions

Varying History Length: Results IPC (gcc)

Varying History Length: Results Branch Predictor Misses (gcc)

Varying History Length: Result Instruction Cache Misses(IL1) (gcc)

Outline Phase Tracking Phase Prediction Applications Phase Based Branch Prediction Phase Based Cache Configuration Summary / Conclusions

Cache optimization Smaller caches use less power Some phases of execution will use less memory or execute a smaller region of code and therefore need less cache We can use a smaller cache for these phases without affecting performance

Methodology Try 4 possibilities of data and instruction cache simultaneously Data cache and instruction cache misses should be independent Select the best combination Data Instr Phase 1 Phase 2

Cache optimization results GCC IPC Fixed 32K cache (16K + 16K): 1.807 Fixed 128K cache (64K + 64K): 1.896 Optimizer: 1.855 Average: 49K total

Cache comparison

Outline Phase Tracking Phase Prediction Applications Phase Based Branch Prediction Phase Based Cache Configuration Summary / Conclusions

Summary Significant reduction in branch mispredictions (29.88% - 44.35%) using phase based branch predictors Simple predictors beat more complex predictor in many phases Marginal gains in IPC using multiple branch predictor (2.24% - 4.70%) Marginal gains in IL1 misses using phase based multiple branch predictors.

Summary (cont...) Phase based dynamic history length fitting does not give good gains

Conclusions [1] Phase based optimizations provides scope for improvements using reconfigurable hardware Using phase specific branch predictor provides good improvements in mis predictions A good strategy for saving power as mis-predictions may result in reduction of mis- speculated instructions,

Conclusion [2] However, varying history length does not result in substantial savings More benchmarks need to be considered to understand the effect of history length adaptations

Questions??