Circuit-Level Timing Speculation: The Razor Latch Developed by Trevor Mudge’s group at the University of Michigan, 2003

We’ve Already Encountered Speculation in ECE 568 Branch prediction – When a branch is encountered, guess whether it is taken or not If the guess is correct, we have gained time If the guess is incorrect, we must undo any incorrectly executed instructions and move on Multi-word cache lines – When a cache miss is encountered, we bring in the entire cache line, not just the word we’re looking for If the access pattern shows spatial locality, we are prefetching other words that the program will soon ask for, thereby saving time. If the speculation is too aggressive (i.e., the cache lines are too long), we’ll fetch many words uselessly.

Speculation (contd.) Value Prediction – (Not covered in this course) – Idea is to predict what the value of a variable will be and use the predicted value. If the predicted value was right, we gain some time; if it was wrong, we did some useless execution. If this execution changed processor state, these changes will have to be undone. Not used in practice (to my knowledge): mainly an academic exercise so far.

Speculating on Time The pipeline clock cycle is the time by which each stage is guaranteed to complete its assigned operation This time is a function of – Actual hardware parameters: Gate and wire delays vary within the same die, from one die to another, and from one wafer to another. – Data involved in the computation: Example: Ripple-carry adder. Worst-case execution time is the time it takes to ripple the carry through from carry-in of the least significant, to the carry-out of the most significant, stage. Actual execution times may vary considerably. Requiring the worst-case delays to be accounted for often forces designers to be overly conservative in setting the clock rates

Timing Speculation: Basic Idea Suppose F is the frequency at which the pipeline is guaranteed to function correctly Run the pipeline at a somewhat higher rate, f. – Much of the time, this clock period, t_p=1/f, will be sufficient for all pipeline stages, and we’ll gain in execution speed – Some of the time, we may need more time: Need to discover when this is the case When this is the case, provide additional time by allowing the pipeline stages additional cycles to complete their operation

Implementation Recall that pipeline stages are separated by latches Duplicate each pipeline latch by introducing a shadow latch Consider any stage of the pipeline. Suppose it starts some activity at time 0. – At time t_p=1/f, latch the output of that stage into the regular pipeline latch. – At time T_p=1/F, latch the output of the stage into the shadow latch. – Compare the results of the regular and shadow latches – If they agree, do nothing: running at a higher speed has paid off – If they don’t agree, Use the result of the shadow latch as the correct one Squelch the computation that the following stage began on the basis of the incorrect shadow latch results Restart the computation in the following stage using the correct results, as stored in the shadow latch

Unless otherwise stated, all figures are from Ernst, et al., MICRO-36, 2003.

Issues to Consider How aggressive should we be? – If f is too high, a large fraction of the results will require correction with the shadow latch and we’ll actually lose time – If f is too low, the clock will be unnecessarily too slow and we won’t gain much

Issues to Consider (contd.) What about F? – Lower bound of F is given by the worst-case path (for the worst-case inputs) – What happens if F is too small? [This is one of the few instances in design when being too conservative at one level affects correctness of functioning!] F may be so small that the results of the next computation propagate through the stage and arrive at the shadow latch – We’d then be comparing the results of two different operations!

Metastability If the input data is not stable when the clock transition happens, the output of the latch may float at a voltage that is in neither the 0 nor in the 1 logic ranges – Duration of metastable stage is not bounded – Different gates may interpret such indeterminate voltages differently (in terms of logic values) Cannot reduce the probability of metastability to zero: all we can do is to keep it sufficiently low for all practical purposes

Recovery Technique 1: Global Clock Gating If any stage detects a timing problem – Stall the entire pipeline for one clock cycle. – Use this additional clock cycle to recompute using the correct shadow-latch values

Recovery Technique 2: Counterflow Pipelining When a mismatch (between regular and shadow latch contents) is detected: – Assert a bubble signal, to specify that the erring pipeline slot is now to be considered a bubble. – In the subsequent cycle, inject the shadow latch value into the next stage, allowing the errant operation to continue with the correct values – Trigger a flush train, traveling backwards from the errant stage, flushing operations at each stage it visits (Question: Is this flush operation necessary?? Can we do something else to avoid it?)

Power Consumption Using a Processor to Fry an Egg From:

Power Density From: Hsu and Feng, “A Power-Aware Real-Time System…”, 2005

Power Implications: Dynamic Power From Krishna & Lee: IEEE Trans. Computers, 2003.

Static Power Even when there is no switching, transistors leak current Leakage power is a strongly increasing function of temperature and supply voltage; it is inversely proportional to the threshold voltage.

Subthreshold leakage vs temperature From: Do, et al: Tech Report , Dept of CSE, Chalmers Instt of Tech

Leakage Current vs Vdd From Do et al., op cit.

Voltage Control for Razor Latch System