Anshul Kumar, CSE IITD CSL718 : Superscalar Processors Speculative Execution 2nd Feb, 2006

Anshul Kumar, CSE IITD slide 2 Handling Control Dependence Simple pipeline –Branch prediction reduces stalls due to control dependence Wide issue processor –Mere branch prediction is not sufficient –Instructions in the predicted path need to be fetched and EXECUTED (speculated execution)

Anshul Kumar, CSE IITD slide 3 What is required for speculation? Branch prediction to choose which instructions to execute Execution of instructions before control dependences are resolved Ability to undo the effects of incorrectly speculated sequence Preserving of correct behaviour under exceptions

Anshul Kumar, CSE IITD slide 4 Types of speculation Hardware based speculation –done with dynamic branch prediction and dynamic scheduling –used in Superscalar processors Compiler based speculation –done with static branch prediction and static scheduling –used in VLIW processors

Anshul Kumar, CSE IITD slide 5 Extending Tomasulo’s scheme for speculative execution Introduce re-order buffer (ROB) Add another stage – “commit” Normal execution Issue Execute Write result Speculative execution Issue Execute Write result Commit f x f x i i x x

Anshul Kumar, CSE IITD slide 6 Extending Tomasulo’s scheme for speculative execution – contd. Write results into ROB in the “write result” stage Write results into register file or memory in the “commit” stage Dependent instructions can read operands from ROB A speculative instruction commits only if the prediction is determined to be correct Instructions may complete execution out-of-order, but they commit in-order

Anshul Kumar, CSE IITD slide 7 Recall Tomasulo’s scheme......

Anshul Kumar, CSE IITD slide 8 IssueIssue Get next instruction from instruction queue Check if there is a matching RS which is empty –no: structural hazard, instruction stalls –yes: issue the instruction to that RS For each operand, check if it is available in RF –yes: put the operand in the RS –no: keep track of FU that will produce it

Anshul Kumar, CSE IITD slide 9 ExecuteExecute If one or more operands not available, wait and monitor CDB When an operand becomes available, it is placed in RS When all operands are available, start execution Choice may need to be made if multiple instructions become ready at the same time

Anshul Kumar, CSE IITD slide 10 Write result When result is available –write it on CDB and –from there into RF and relevant RSs Mark RS as available

Anshul Kumar, CSE IITD slide 11 More formal description......

Anshul Kumar, CSE IITD slide 12 RS and RF fields opbusyQjVjQkVk valQi

Anshul Kumar, CSE IITD slide 13 IssueIssue Get instruction from instruction queue Wait until  r  RS[r].busy = no if (RF[rs].Qi  0) {RS[r].Qj  RF[rs].Qi} else {RS[r].Vj  RF[rs].val; RS[r].Qj  0} similarly for rt RS[r].op  op; RS[r].busy  yes; RF[rd].Qi  r

Anshul Kumar, CSE IITD slide 14 ExecuteExecute Wait until RS[r].Qj = 0 and RS[r].Qk = 0 Compute result: operation is RS[r].op, operands are RS[r].Vj and RS[r].Vk

Anshul Kumar, CSE IITD slide 15 Write result Wait until execution complete at r and CDB available  x if (RF[x].Qi = r) {RF[x].val  result; RF[x].Qi  0}  x if (RS[x].Qj = r) {RS[x].Vj  result; RS[x].Qj  0} similarly for Qk / Vk RS[r].busy  no

Anshul Kumar, CSE IITD slide 16 Tomasulo’s scheme plus ROB......

Anshul Kumar, CSE IITD slide 17 IssueIssue Get next instruction from instruction queue Check if there is a matching RS which is empty and an empty slot in ROB –no: structural hazard, instruction stalls –yes: issue the instruction to that RS and mark the ROB slot, also put ROB slot number in RS For each operand, check if it is available in RF or ROB –yes: put the operand in the RS –no: keep track of FU that will produce it

Anshul Kumar, CSE IITD slide 18 Execute (no change) If one or more operands not available, wait and monitor CDB When an operand becomes available, it is placed in RS When all operands are available, start execution Choice may need to be made if multiple instructions become ready at the same time

Anshul Kumar, CSE IITD slide 19 Write result When result is available –write it on CDB with ROB tag and –from there into ROB RF and relevant RSs Mark RS as available

Anshul Kumar, CSE IITD slide 20 Commit (non-branch instruction) Wait until instruction reaches head of ROB Update RF Remove instruction from ROB

Anshul Kumar, CSE IITD slide 21 Commit (branch instruction) Wait until instruction reaches head of ROB If branch is mispredicted, –flush ROB –Restart execution at correct successor of the branch instruction else –Remove instruction from ROB

Anshul Kumar, CSE IITD slide 22 More formal description......

Anshul Kumar, CSE IITD slide 23 RS fields opbusyQiQjVjQkVk

Anshul Kumar, CSE IITD slide 24 RF fields valQibusy

Anshul Kumar, CSE IITD slide 25 ROB fields instbusyrdyvaldst

Anshul Kumar, CSE IITD slide 26 IssueIssue Get instruction from instruction queue Wait until  r  RS[r].busy=no and ROB[b].busy=no, where b = ROB tail if (RF[rs].busy) {h  RF[rs].Qi; if (ROB[h].rdy) {RS[r].Vj  ROB[h].val; RS[r].Qj  0} else {RS[r].Qj  h} } else {RS[r].Vj  RF[rs].val; RS[r].Qj  0} similarly for rt RS[r].op  op; RS[r].busy  yes; RS[r].Qi  b RF[rd].Qi  b; RF[rd].busy  yes; ROB[b].busy  yes ROB[b].inst  op; ROB[b].dst  rd; ROB[b].rdy  no

Anshul Kumar, CSE IITD slide 27 Execute (no change) Wait until RS[r].Qj = 0 and RS[r].Qk = 0 Compute result: operation is RS[r].op, operands are RS[r].Vj and RS[r].Vk

Anshul Kumar, CSE IITD slide 28 Write result Wait until execution complete at r and CDB available b  RS[r].Qi; RS[r].busy  no  x if (RF[x].Qi = r) {RF[x]  result; RF[x].Qi  0}  x if (RS[x].Qj = b) {RS[x].Vj  result; RS[x].Qj  0} similarly for Qk / Vk ROB[b].rdy  yes; ROB[b].val  result

Anshul Kumar, CSE IITD slide 29 Commit (non-branch instruction) Wait until instruction reaches head of ROB (entry h) and ROB[h].rdy = yes d  ROB[h].dst RF[d].val  ROB[h].val ROB[h].busy  no if (RF[d].Qi = h) {RF[d].busy  no}

Anshul Kumar, CSE IITD slide 30 Commit (branch instruction) Wait until instruction reaches head of ROB (entry h) and ROB[h].rdy = yes If branch is mispredicted, –clear ROB, RF[ ].Qi –fetch branch dest else –ROB[h].busy  no –if (RF[d].Qi = h) {RF[d].busy  no}