Spr 2016, Mar 9... ELEC / Lecture 7 1 ELEC / Computer Architecture and Design Spring 2016 Pipeline Control and Performance (Chapter 6) Vishwani D. Agrawal James J. Danaher Professor Department of Electrical and Computer Engineering Auburn University, Auburn, AL

Spr 2016, Mar 9... ELEC / Lecture 7 2 Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU Pipelined Datapath (without Jump) IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem PC

Spr 2016, Mar 9... ELEC / Lecture 7 3 PC Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU Mem. and Reg. File Need Controls IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead

Spr 2016, Mar 9... ELEC / Lecture 7 4 PC Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU Multiplexers Need Controls IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead ALUSrc RegDst Branch PCSrc MemtoReg

Spr 2016, Mar 9... ELEC / Lecture 7 5 PC Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU ALU Needs a Control IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead ALUSrc RegDst Branch PCSrc MemtoReg ALU cont. ALUOp 0-5

Spr 2016, Mar 9... ELEC / Lecture 7 6 Compare with Single-Cycle Control Control signals are the same as those needed for a single-cycle datapath. Control signals are generated using the Opcode in the ID (instruction decode) cycle and then distributed to other cycles. Let us reexamine the implementation of the single-cycle control (slides of Lecture 5).

Spr 2016, Mar 9... ELEC / Lecture 7 7 Hardwired CU: Single-Cycle Implemented by combinational logic. Control logic opcode Datapath ALU control 3 2 To ALU ALUOp Control signals funct. code 6 6

Spr 2016, Mar 9... ELEC / Lecture 7 8 Instr. mem. PC Add Reg. File Data mem. 1 mux 0 0 mux mux 0 Sign ext. Shift left 2 ALU Cont. CONTROL opcode MemWrite MemRead ALU Branch zero ALU 0 mux 1 Shift left Jump Single-cycle datapath MemtoReg ALUOp ALUSrc RegDst RegWrite

Spr 2016, Mar 9... ELEC / Lecture 7 9 Single-Cycle Control Logic InputsOutputs Instr.typeOpcode Instruction bits R lw lw sw sw101011X1X beq000100X0X J000010XXX0X0XXX1 ALUOp0 ALUOp1 RegDst ALUSrc MemtoReg RegWrite MemRead MemWrite Branch Jump

Spr 2016, Mar 9... ELEC / Lecture 7 10 Single-Cycle Control Circuit lw swbeqJR RegDst ALUSrc MemtoReg RegWrite MemRead MemWrite Branch ALUOp1 ALUOp0 Jump Op5 Op4 Op3 Op2 Op1 Op0

Spr 2016, Mar 9... ELEC / Lecture 7 11 ALU Control Logic Inputs Outputs to ALU Instr.type From CU Funct. Code from IR (bits 0-5) 3-bit code Opera- tion ALUOp1ALUOp0F5F4F3F2F1F0 lw, sw 00XXXXXX010Add B01XXXXXX110Subtract R 1XXX Add 1XXX Subtract 1XXX AND 1XXX OR 1XXX slt slt

Spr 2016, Mar 9... ELEC / Lecture 7 12 ALU Control ALU 3 zero result overflow Operation select from control Operation selectALU function 000AND 001OR 010Add 110Subtract 111Set on less than F3 F2 F1 F0 ALUOp1 ALUOp0 From Control Circuit ALU control

Spr 2016, Mar 9... ELEC / Lecture 7 13 Returning to Pipelined Control Opcode input to control is supplied by the pipeline register IF/ID in the ID (instruction decode) cycle. Nine control signals are generated in the ID cycle, but none is used. They are saved in the pipeline register ID/EX. ALUSrc, RegDst and ALUOp (2 bits) are used in the EX (execute) cycle. Remaining 5 control signals are saved in the pipeline register EX/MEM. Branch, MemWrite and MemRead are used in the MEM (memory access) cycle. Remaining 2 control signals are saved in the pipeline register MEM/WB. MemtoReg and RegWrite are used in the WB (write back) cycle. Pipelined control is shown without Jump.

Spr 2016, Mar 9... ELEC / Lecture 7 14 PC Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU Placing Control in Pipelined Datapath IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead ALUSrc RegDst Branch PCSrc MemtoReg ALU cont. ALUOp CONTROL 0-5 IF/IDID/EX

Spr 2016, Mar 9... ELEC / Lecture 7 15 PC Add Reg. File Data mem 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU Highlighted Pipelined Control IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead ALUSrc RegDst Branch PCSrc MemtoReg ALU cont. ALUOp CONTROL 0-5

Spr 2016, Mar 9... ELEC / Lecture 7 16 Single-Cycle Performance Assume 200 ps for memory access 100 ps for ALU operation 50 ps for register file read or write Cycle time set according to longest instruction: lw ≡ IF + ID/RegRead + ALU + MEM + RegWrite = = 600 ps  Av. instruction execution time= clock cycle time = 600 ps

Spr 2016, Mar 9... ELEC / Lecture 7 17 Multicycle Performance Consider SPECINT2000* instruction mix: 25% lw5 cycles 10% sw4 cycles 11% branch3 cycles 2% jump3 cycles 52% ALU instr.4 cycles Av. CPI= 0.25× × × × ×4 = 4.12 Clock cycle time determined from longest operation (memory access) = 200 ps Av. instruction execution time = 4.12×200 = 824 ps * Set of benchmark programs used for performance evaluation, to be discussed in a later lecture.

Spr 2016, Mar 9... ELEC / Lecture 7 18 Pipeline Performance Neglect initial latency (reasonable for long programs). One instruction completed every clock cycle unless delayed by hazard. Average CPI: lw2 cycles in 50% cases due to hazard1.5 cycles lw2 cycles in 50% cases due to hazard1.5 cycles sw1 cycle sw1 cycle ALU1 cycle ALU1 cycle branch2 cycles in 25% cases due to hazard 1.25 cycles branch2 cycles in 25% cases due to hazard 1.25 cycles jump2 cycles jump2 cycles For SPECINT2000 Av. CPI= 0.25× × × × ×1 = 1.17 Clock cycle time (longest operation: memory access) = 200 ps Av. instruction execution time = 1.17×200 = 234 ps

Spr 2016, Mar 9... ELEC / Lecture 7 19 Comparing Alternatives Type of datapath and control Clock cycle time Average CPI Av. instruction execution time Single- cycle 600 ps 1.00 Multicycle 200 ps ps Pipelined 200 ps ps

Spr 2016, Mar 9... ELEC / Lecture 7 20 Other Controls for Pipeline ForwardingStall Branch hazard and branch prediction Instruction flush Exceptions

Spr 2016, Mar 9... ELEC / Lecture 7 21 Forwarding Consider a data hazard: sub$2, $1, $3# computes result in CC3, writes in $2 in CC5 and$12, $2, $5# reads $2 in CC3, adds in CC4 IF: IM ID: REG. FILE READ EX: ALU MEM: DM WB : REG. WRITE IF/ID ID/EX MEM/WB EX/MEM CC1 CC2 CC3 CC4 CC5 CC6 CC7 CC8 sub $2, $1, $3 and $12, $2, $5 CC3: ALU saves new data in EX/MEM, to be written to $2 in CC5 IF: IM ID: REG. FILE READ EX: ALU MEM: DM WB : REG. WRITE IF/IDID/EX MEM/WB EX/MEM CC3: and reads $2 to ID/EX, but the correct data is in EX/MEM CC4: forwarding allows execution of “and” with correct data

Spr 2016, Mar 9... ELEC / Lecture 7 22 Understanding Forwarding Let’s ask following questions: Q: Why is there a hazard? A: Source register for the present instruction is the same as the destination register of the previous instruction. Q: When is the source register data needed? A: In the execute cycle (CC4). Q: Is source register data available in CC4? A: Yes – use forwarding. No – use stall. Q:Where is the required data in CC4? A: In the pipeline register EX/MEM as ALU output.

Spr 2016, Mar 9... ELEC / Lecture 7 23 Forwarding Hardware A forwarding unit is added to execute (ALU) cycle hardware. Functions of forwarding unit: –Hazard detection –Forward correct data to ALU Inputs to forwarding unit: –Source registers of the instruction in EX –Destination registers of instructions in DM and WB Outputs of forwarding unit: multiplexer controls to route correct data to the ALU.

Spr 2016, Mar 9... ELEC / Lecture 7 24 Recall Register Definitions R-type instruction (add, sub, and, or,... ) opcodeRsRtRdshamtfunct I-type instruction (beq, lw, sw, addi,... ) opcodeRsRtconstant_or_address J-type instruction (j, jal, jr) opcodea___d___d___r___e___s___s where Rs is the first source register Rt is the second source register Rd is the destination register

Spr 2016, Mar 9... ELEC / Lecture 7 25 Reg. File Data mem. 1 mux 0 0 mux 1 Sign ext. Shift left 2 opcode ALU zero ALU IF/ID ID/EXEX/MEMMEM/WB mux 0 Forwarding unit MUX Branch addr. Addr mem PC Rs Rt Forwarding Implemented Rd

Spr 2016, Mar 9... ELEC / Lecture 7 26 Stall Delay next instruction by sending nop through pipeline. Necessary when hazard not resolved by forwarding. IM ID, REG. FILE READ ALU DM REG. FILE WRITE IF/IDID/EX MEM/WB EX/MEM IM ID, REG. FILE READ ALU DM REG. FILE WRITE IF/ID ID/EX MEM/WB EX/MEM CC1 CC2 CC3 CC4 CC5 CC6 lw $2, 20($1) and $4, $2, $5 CC4: new data in MEM/WB, to be written to $2 CC4: execution of and is impossible; correct data unavailable until end of CC4

Spr 2016, Mar 9... ELEC / Lecture 7 27 Detecting Hazard Requiring Stall Consider instruction in IF/ID being decoded: If Previous instruction (lw) activated MemRead, and Instruction being decoded has a source register (Rs or Rt) same as the destination register (Rt for lw) of the previous instruction Then, stall the pipeline: Force all control outputs to 0 Prevent PC from changing Prevent IF/ID from changing

Spr 2016, Mar 9... ELEC / Lecture 7 28 Stall Implementation Data mem. 0 mux 1 Sign ext. Shift left 2 opcode zero IF/ID ID/EXEX/MEMMEM/WB mux 0 MUX Rs Rt Rd 1 mux 0 ALU Addr mem Hazard detection unit Control 0 MemRead PCWrite IF/IDWrite PC MUX Reg. File Forwarding unit Rs Rt

Spr 2016, Mar 9... ELEC / Lecture 7 29 Stall Execution with stall and forwarding: CC1 CC2 CC3 CC4 CC5 CC6 CC7 lw $2, 20($1) and $4, $2, $5 CC4: new data in MEM/WB, to be written to $2 IF: IM ID: REG. FILE READ EX: ALU MEM: DM WB : REG. WRITE IF/ID ID/EX MEM/WB EX/MEM IF: IM IF/ID IF: IM ID: REG. FILE READ EX: ALU MEM: DM WB : REG. WRITE IF/ID ID/EX MEM/WB EX/MEM next ID: REG. FILE READ EX: ALU MEM: DM WB : REG. WRITE IF/ID ID/EX MEM/WB EX/MEM IF: IM ID: REG. FILE READ EX: ALU MEM: DM WB : REG. WRITE IF/ID ID/EX MEM/WB EX/MEM State of IF/ID is frozen in CC3 bubble (nop) next is fetched twice since PC was frozen

Spr 2016, Mar 9... ELEC / Lecture 7 30 Branch Hazard Consider heuristic – branch not taken. Continue fetching instructions in sequence following the branch instructions. If branch is taken (indicated by zero output of ALU): –Control generates branch signal in ID cycle. –branch activates PCSource signal in the MEM cycle to load PC with new branch address. –Three instructions in the pipeline must be flushed if branch is taken – can this penalty be reduced?

Spr 2016, Mar 9... ELEC / Lecture 7 31 PC Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero ALU Branch Hazard IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead ALUSrc RegDst Branch PCSrc MemtoReg ALU cont. ALUOp0-5 beq CONTROL

Spr 2016, Mar 9... ELEC / Lecture 7 32 Branch Not Taken Branch fetchedBranch decodedBranch decisionPC keeps D (br. not taken) A fetchedA decodedA executedA continues B fetchedB decodedB executed C fetchedC decoded D fetched cycle b cycle b+1 cycle b+2 cycle b+3cycle b+4 Branch on condition to Z A B C D Z

Spr 2016, Mar 9... ELEC / Lecture 7 33 Branch Taken Branch fetchedBranch decodedBranch decisionPC gets Z (br. taken) A fetchedA decodedA executedNop B fetchedB decodedNop C fetchedNop Z fetched cycle b cycle b+1 cycle b+2 cycle b+3cycle b+4 Branch on condition to Z A B C D Z Three instructions are flushed if branch is taken Three-cycle penalty

Spr 2016, Mar 9... ELEC / Lecture 7 34 PC Add Reg. File Data mem. 1 mux 0 0 mux 1 4 Sign ext. Shift left 2 opcode ALU zero Add Branch Penalty Reduction IF/IDID/EXEX/MEMMEM/WB for I-type lw for R-type 1 mux 0 Instr mem RegWrite MemWrite MemRead ALUSrc RegDst Branch PCSrc MemtoReg ALU cont. ALUOp0-5 beq CONTROL

Spr 2016, Mar 9... ELEC / Lecture 7 35 Branch Taken Branch fetchedBranch decision PC gets Z A fetchedA flushedNopNop Z fetchedZ decodedZ executed cycle b cycle b+1 cycle b+2 cycle b+3cycle b+4 Branch to Z A B C D Z One instructions is flushed if branch is taken One-cycle penalty

Spr 2016, Mar 9... ELEC / Lecture 7 36 Pipeline Flush If branch is taken (as indicated by zero), then control does the following: –Change all control signals to 0, similar to the case of stall for data hazard, i.e., insert bubble in the pipeline. –Generate a signal IF.Flush that changes the instruction in the pipeline register IF/ID to 0 (nop). Penalty of branch hazard is reduced by –Adding branch detection and address generation hardware in the decode cycle – one bubble needed – a next address generation logic in the decode stage writes PC+4, branch address, or jump address into PC. –Using branch prediction. –Unrolling loops.

Spr 2016, Mar 9... ELEC / Lecture 7 37 Branch Prediction Useful for program loops. A one-bit prediction scheme: a one-bit buffer carries a “history bit” that tells what happened on the last branch instruction History bit = 1, branch was taken History bit = 0, branch was not taken Predict branch not taken 0 Predict branch taken 1 taken Not taken

Spr 2016, Mar 9... ELEC / Lecture 7 38 Branch Prediction = Prediction Logic 0 1 PC+4Next PC PC Low-order bits used as index Address of Target History recent branch addresses bit(s) instructions

Spr 2016, Mar 9... ELEC / Lecture 7 39 Branch Prediction for a Loop I = 0 I = I + 1 I – 10 = 0? Store X in memory X = X + R(I) Y N abcdeabcde Execu -tion seq. Oldhist.bit Next instr. New hist. bit Predi ction Pred.IAct. 10e1b1Bad 21b2b1Good 31b3b1Good 41b4b1Good 51b5b1Good 61b6b1Good 71b7b1Good 81b8b1Good 91b9b1Good 101b10e0 Execution of Instruction d h.bit = 0 branch not taken, h.bit = 1 branch taken.

Prediction Accuracy One-bit predictor: 2 errors out of 10 predictions Prediction accuracy = 80% To improve prediction accuracy, use two- bit predictor: A prediction must be wrong twice before it is changed Spr 2016, Mar 9... ELEC / Lecture 7 40

Spr 2016, Mar 9... ELEC / Lecture 7 41 Two-Bit Prediction Buffer Implemented as a two-bit counter. Can improve correct prediction statistics. Predict branch not taken 00 Predict branch taken 10 Predict branch taken 11 Predict branch not taken 01 taken Not taken

Spr 2016, Mar 9... ELEC / Lecture 7 42 Branch Prediction for a Loop I = 0 I = I + 1 I – 10 = 0? Store X in memory X = X + R(I) Y N Execu -tion seq. Old Pred. Buf Next instr. New pred. Buf Predi ction Pred.IAct Good Good Good Good Good Good Good Good Good Bad Execution of Instruction 4

Spr 2016, Mar 9... ELEC / Lecture 7 43 Exceptions A typical exception occurs when ALU produces an overflow signal. Control asserts following actions on exception: –Change the PC address to hex. This is the location of the exception routine. This is done by adding an additional input to the PC input multiplexer. –Overflow is detected in the EX cycle. Similar to data hazard and pipeline flush, Set IF/ID to 0 (nop). Generate ID.Flush and EX.Flush signals to set all control signals to 0 in ID/EX and EX/MEM registers. This also prevents the ALU result (presumed contaminated) from being written in the WB cycle.