Lecture 3: Review CPU Design Alvin R. Lebeck CPS 220 Fall 2001

2 © Alvin R. Lebeck 2001 CPS 220 Administrivia Read Chapter 3 Homework #1 Processor Design Control and Datapath Pipelining If you need to more information, please see Chapters 5 and 6 of Patterson & Hennessy, “Computer Organization & Design”

3 © Alvin R. Lebeck 2001 CPS 220 Basic ISA Classes Accumulator: 1 addressadd Aacc  acc + mem[A] 1+x addressaddx Aacc  acc + mem[A + x] Stack: 0 addressaddtos  tos + next (JAVA VM) General Purpose Register: 2 addressadd A BA  A + B 3 addressadd A B CA  B + C Load/Store: 3 addressadd Ra Rb RcRa  Rb + Rc load Ra RbRa  mem[Rb] store Ra Rbmem[Rb]  Ra

4 © Alvin R. Lebeck 2001 CPS 220 Variable format, 2 and 3 address instruction 32-bit word size, 16 GPR (four reserved) Rich set of addressing modes (apply to any operand) Rich set of operations – bit field, stack, call, case, loop, string, poly, system Rich set of data types (B, W, L, Q, O, F, D, G, H) Condition codes VAX-11

5 © Alvin R. Lebeck 2001 CPS 220 Kinds of Addressing Modes Register directRi Immediate (literal)v Direct (absolute)M[v] Register indirect M[Ri] Base+DisplacementM[Ri + v] Base+IndexM[Ri + Rj] Scaled IndexM[Ri + Rj*d + v] AutoincrementM[Ri++] AutodecrementM[Ri--] Memory IndirectM[M[Ri]] Ri Rj v memory reg. file

6 © Alvin R. Lebeck 2001 CPS 220 A "Typical" RISC 32-bit fixed format instruction (3 formats) bit GPR (R0 contains zero) 3-address, reg-reg arithmetic instruction Single address mode for load/store: base + displacement –no indirection Simple branch conditions Delayed branch (sometimes) see: SPARC, MIPS, MC88100, AMD2900, i960, i860 PARisc, POWERPC, DEC Alpha, Clipper, CDC 6600, CDC 7600, Cray-1, Cray-2, Cray-3

7 © Alvin R. Lebeck 2001 CPS 220 The Big Picture The Five Classic Components of a Computer Today’s Topic: Datapath and Control Design Control Datapath Memory Processor Input Output

8 © Alvin R. Lebeck 2001 CPS 220 The Big Picture: The Performance Perspective Performance of a machine was determined by: –Instruction count –Clock cycle time –Clock cycles per instruction Processor design (datapath and control) will determine: –Clock cycle time –Clock cycles per instruction In this lecture: –Single cycle processor: »Advantage: One clock cycle per instruction »Disadvantage: long cycle time –Multi cycle processor

9 © Alvin R. Lebeck 2001 CPS 220 The MIPS Instruction Formats All MIPS instructions are 32 bits long. The three instruction formats: R-type I-type J-type Fields : –op: operation of the instruction –rs, rt, rd: the source and destination registers specifier –shamt: shift amount –funct: selects the variant of the operation in the “op” field –address / immediate: address offset or immediate value –target address: target address of the jump instruction optarget address bits26 bits oprsrtrdshamtfunct bits 5 bits oprsrt immediate bits16 bits5 bits

10 © Alvin R. Lebeck 2001 CPS 220 An Abstract View of the Implementation Clk 5 RwRaRb bit Registers Rd ALU Clk Data In DataOut Data Address Ideal Data Memory Instruction Instruction Address Ideal Instruction Memory Clk PC 5 Rs 5 Rt 16 Imm 32 A B

11 © Alvin R. Lebeck 2001 CPS 220 The Steps of Designing a Processor Instruction Set Architecture => Register Transfer Language Register Transfer Language => –Datapath components –Datapath interconnect Datapath components => Control signals Control signals => Control logic

12 © Alvin R. Lebeck 2001 CPS 220 RTL: The ADD Instruction addrd, rs, rt –mem[PC]Fetch the instruction from memory –R[rd] <- R[rs] + R[rt]The actual operation –PC <- PC + 4Calculate the next instruction’s address oprsrtrdshamtfunct bits 5 bits

13 © Alvin R. Lebeck 2001 CPS 220 Combinational Logic Elements (Building Blocks) 32 A B Result Zero OP ALU 32 A B Y MUX Select 32 A B Sum Carry Adder CarryIn ADDER MUX ALU

14 © Alvin R. Lebeck 2001 CPS 220 Storage Element: Register (Building Block) Register –Similar to the D Flip Flop except »N-bit input and output »Write Enable input –Write Enable: »negated (0): Data Out will not change »asserted (1): Data Out will become Data In Clk Data In Write Enable NN Data Out

15 © Alvin R. Lebeck 2001 CPS 220 Storage Element: Register File Register File consists of 32 registers: –Two 32-bit output busses: busA and busB –One 32-bit input bus: busW Register is selected by: –RA selects the register to put on busA –RB selects the register to put on busB –RW selects the register to be written via busW when Write Enable is 1 Clock input (CLK) –The CLK input is a factor ONLY during write operation –During read operation, behaves as a combinational logic block: »RA or RB valid => busA or busB valid after “access time.” Clk busW Write Enable 32 busA 32 busB 555 RWRARB bit Registers

16 © Alvin R. Lebeck 2001 CPS 220 Storage Element: Idealized Memory Memory (idealized) –One input bus: Data In –One output bus: Data Out Memory word is selected by: –Address selects the word to put on Data Out –Write Enable = 1: address selects the memory word to be written via the Data In bus Clock input (CLK) –The CLK input is a factor ONLY during write operation –During read operation, behaves as a combinational logic block: »Address valid => Data Out valid after “access time.” Looks similar to register file. Why have registers? Clk Data In Write Enable 32 DataOut Address

17 © Alvin R. Lebeck 2001 CPS 220 Overview of the Instruction Fetch Unit The common RTL operations –Fetch the Instruction: mem[PC] –Update the program counter: »Sequential Code: PC <- PC + 4 »Branch and Jump: PC <- “something else” 32 Instruction Word Address Instruction Memory PC Clk Next Address Logic

18 © Alvin R. Lebeck 2001 CPS 220 Datapath for Register-Register Operations R[rd] <- R[rs] op R[rt] Example: add rd, rs, rt –Ra, Rb, and Rw comes from instruction’s rs, rt, and rd fields –ALUctr and RegWr: control logic after decoding the instruction 32 Result ALUctr Clk busW RegWr 32 busA 32 busB 555 RwRaRb bit Registers RsRtRd ALU oprsrtrdshamtfunct bits 5 bits

19 © Alvin R. Lebeck 2001 CPS 220 A Single Cycle Datapath We have everything except control signals (underline)

20 © Alvin R. Lebeck 2001 CPS 220 Instruction Fetch Unit at the Beginning of Add / Subtract 30 SignExt imm16 Mux 0 1 Adder “1” PC Clk Adder 30 Branch = previous Zero = previous “00” Addr Instruction Memory Addr 32 Mux PC Target 30 Fetch the instruction from Instruction memory: Instruction <- mem[PC] –This is the same for all instructions Jump = previous Instruction 30 Instruction

21 © Alvin R. Lebeck 2001 CPS 220 The Single Cycle Datapath during Add and Subtract 32 ALUctr = Add or Subtract Clk busW RegWr = 1 32 busA 32 busB 555 RwRaRb bit Registers Rs Rt Rd RegDst = 1 Extender Mux imm16 ALUSrc = 0 ExtOp = x Mux MemtoReg = 0 Clk Data In WrEn 32 Adr Data Memory 32 MemWr = 0 ALU Instruction Fetch Unit Clk Zero Instruction Jump = 0 Branch = 0 R[rd] <- R[rs] + / - R[rt] Imm16RdRsRt oprsrtrdshamtfunct

22 © Alvin R. Lebeck 2001 CPS 220 Instruction Fetch Unit at the End of Add and Subtract 30 SignExt imm16 Mux 0 1 Adder “1” PC Clk Adder 30 Branch = 0 Zero = x “00” Addr Instruction Memory Addr 32 Mux PC Target 30 PC <- PC + 4 –This is the same for all instructions except: Branch and Jump Jump = 0 Instruction 30 Instruction

23 © Alvin R. Lebeck 2001 CPS 220 The “Truth Table” for RegWrite R-typeorilwswbeqjump RegWrite op RegWrite = R-type + ori + lw = !op & !op & !op & !op & !op & !op (R-type) + !op & !op & op & op & !op & op (ori) + op & !op & !op & !op & op & op (lw) RegWrite

24 © Alvin R. Lebeck 2001 CPS 220 PLA Implementation of the Main Control op.... op.. op.. op.. op.. R-typeorilwswbeqjump RegWrite ALUSrc MemtoReg MemWrite Branch Jump RegDst ExtOp ALUop

25 © Alvin R. Lebeck 2001 CPS 220 Putting it All Together: A Single Cycle Processor

26 © Alvin R. Lebeck 2001 CPS 220 Drawback of this Single Cycle Processor Long cycle time: –Cycle time must be long enough for the load instruction: »PC’s Clock -to-Q + »Instruction Memory Access Time + »Register File Access Time + »ALU Delay (address calculation) + »Data Memory Access Time + »Register File Setup Time + »Clock Skew Cycle time is much longer than needed for all other instructions

27 © Alvin R. Lebeck 2001 CPS 220 Overview of a Multiple Cycle Implementation The root of the single cycle processor’s problems: –The cycle time has to be long enough for the slowest instruction Solution: –Break the instruction into smaller steps –Execute each step (instead of the entire instruction) in 1 clock cycle »Cycle time: time it takes to execute the longest step »Try to make all the steps have similar length –This is the essence of the multiple cycle processor The advantages of the multiple cycle processor: –Cycle time is much shorter –Different instructions take different number of cycles to complete »Load takes five cycles »Jump only takes three cycles –Allows a functional unit to be used more than once per instruction

Instr Decode / Reg Fetrch The Five Steps of a Load Instruction Clk PC Rs, Rt, Rd, Op, Func Clk-to-Q ALUctr Instruction Memory Access Time Old ValueNew Value RegWrOld ValueNew Value Delay through Control Logic busA Register File Access Time Old ValueNew Value busB ALU Delay Old ValueNew Value Old ValueNew Value Old Value ExtOpOld ValueNew Value ALUSrcOld ValueNew Value AddressOld ValueNew Value busWOld ValueNew Delay through Extender & Mux Data Memory Access Time Instruction FetchAddressReg WrData Memory Register File Write Time

29 © Alvin R. Lebeck 2001 Multiple Cycle Datapath

30 © Alvin R. Lebeck 2001 CPS 220 Where to get more information? Chapter 5 of CPS 104 text book: –David Patterson and John Hennessy, “Computer Organization & Design: The Hardware / Software Interface,” Morgan Kaufman Publishers, San Mateo, California, For a reference on the MIPS architecture: –Gerry Kane, “MIPS RISC Architecture,” Prentice Hall. Now: Pipelining

Introduction to Pipelining

32 © Alvin R. Lebeck 2001 Overview A Pipelined Processor : –Introduction to the concept of pipelined processor. –Pipelined Datapath –Pipeline example: Load Instruction Reading: Chapter 3 Or Chapters 5, 6 in the CPS104 text

33 © Alvin R. Lebeck 2001 Pipelining: It’s Natural! Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes “Folder” takes 20 minutes How long to do laundry? ABCD

34 © Alvin R. Lebeck 2001 Sequential Laundry Sequential laundry takes 6 hours for 4 loads If they learned pipelining, how long would laundry take? A B C D PM Midnight TaskOrderTaskOrder Time

35 © Alvin R. Lebeck 2001 Pipelined Laundry: Start work ASAP Pipelined laundry takes 3.5 hours for 4 loads A B C D 6 PM Midnight TaskOrderTaskOrder Time

36 © Alvin R. Lebeck 2001 Pipelining doesn’t help latency of single task, it helps throughput of entire workload Pipeline rate limited by slowest pipeline stage Multiple tasks operating simultaneously Potential speedup = Number pipe stages Unbalanced lengths of pipe stages reduces speedup Time to “fill” pipeline and time to “drain” it reduces speedup A B C D 6 PM 789 TaskOrderTaskOrder Time Pipelining Lessons

37 © Alvin R. Lebeck 2001 Review: a Multiple-Cycle Implementation The main single cycle processor’s problem: –The cycle time has to be long enough for the slowest instruction to complete execution. Solution: –Break instruction execution into smaller steps –Execute each step in one cycle (instead of the entire instruction). »Short cycle time: time it takes to execute the longest step »Make all the steps have similar length –This is the essence of the multiple cycle processor Multiple-cycle processor advantages: –Cycle time is much shorter –Different instructions take different number of cycles to complete »Load takes five cycles »Jump only takes three cycles –Allows a functional unit to be used more than once per instruction

38 © Alvin R. Lebeck 2001 MCP: If a functional unit is used more than once per instruction -> cannot pipeline -> lower performance Multiple Cycle Processor

39 © Alvin R. Lebeck 2001 Ifetch: Instruction Fetch –Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: Calculate the memory address Mem: Read the data from the Data Memory WrB: Write the data back to the register file Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5 IfetchReg/DecExecMemWrBLoad The Five Stages of a Load

40 © Alvin R. Lebeck 2001 Key Ideas Behind Instruction Execution Pipelining The load instruction has 5 stages: –Five independent functional units to work on each stage »Each functional unit is used only once! –A 2nd load can start doing Ifetch as soon as the 1st load finishes its Ifetch stage. –Each load still takes five cycles to complete. »latency is still 5 cycles –The throughput is much higher: »CPI is 1 with ~1/5th the cycle time. –Instructions start executing before previous instructions complete execution. IfetchReg/DecExecMemWrBLoad

41 © Alvin R. Lebeck 2001 The five independent pipeline stages are: –Read Next Instruction: The Ifetch stage. –Decode Instruction and fetch register values: The Reg/Dec stage –Execute the operation: The Exec stage. –Access Data-Memory: The Mem stage. –Write Data to Destination Register: The WrB stage One instruction enters the pipeline every cycle –One instruction comes out of the pipeline (completed) every cycle –The “Effective” Cycles per Instruction (CPI) is 1; ~1/5 cycle time Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7 IfetchReg/DecExecMemWrB1st lw IfetchReg/DecExecMemWrB2nd lw IfetchReg/DecExecMemWrB3rd lw Pipelining the Load Instruction

42 © Alvin R. Lebeck 2001 Ifetch: Instruction Fetch –Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: ALU operates on the two register operands WrB: Write the ALU output back to the register file Cycle 1Cycle 2Cycle 3Cycle 4 IfetchReg/DecExecWrB R-type The Four Stages of R-Type

43 © Alvin R. Lebeck 2001 We have a problem called a structural hazard or pipeline conflict: –Two instructions try to write to the register file at the same time! Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9 IfetchReg/DecExecWrR-type IfetchReg/DecExecWrR-type IfetchReg/DecExecMemWrLoad IfetchReg/DecExecWrR-type IfetchReg/DecExecWrR-type OOPS! We have a problem! Pipelining the R-type and Load Instruction

44 © Alvin R. Lebeck 2001 Each functional unit can only be used once per instruction. Each functional unit must be used at the same stage for all instructions: –Load uses Register File’s Write Port during its 5th stage. –R-type uses Register File’s Write Port during its 4th stage. IfetchReg/DecExecMemWrBLoad IfetchReg/DecExecWrBR-type 1234 °How do we solve this pipeline hazard? Important Observations

45 © Alvin R. Lebeck 2001 Delay R-type’s register write by one cycle: –Now R-type instructions also use Reg File’s write port at Stage 5 –Mem stage is a NO-OP stage: nothing is being done. Effective CPI? Clock Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9 IfetchReg/DecMemWrBR-type IfetchReg/DecMemWrBR-type IfetchReg/DecExecMemWrBLoad IfetchReg/DecMemWrBR-type IfetchReg/DecMemWrBR-type Exec IfetchReg/DecExecWrR-type Mem Solution: Delay R-type’s Write by One Cycle

46 © Alvin R. Lebeck 2001 Ifetch: Instruction Fetch –Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: Calculate the memory address Mem: Write the data into the Data Memory Cycle 1Cycle 2Cycle 3Cycle 4 IfetchReg/DecExecMemStoreWrB The Four Stages of a Store

47 © Alvin R. Lebeck 2001 Ifetch: Instruction Fetch –Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: ALU compares the two register operands –Adder calculates the branch target address Mem: If the registers we compared in the Exec stage are the same, –Write the branch target address into the PC Cycle 1Cycle 2Cycle 3Cycle 4 IfetchReg/DecExecMemBeqWrB The Four Stages of BEQ

48 © Alvin R. Lebeck 2001 A Pipelined Datapath

49 © Alvin R. Lebeck 2001 MemWr You are here! IF/ID: lw $1, 100 ($2) ID/Ex Register Ex/Mem Register Mem/Wr Register PC = 12 Data Mem WA Di RADo IF_Unit A I RFile Di Ra Rb Rw RegWr ExtOp Exec Unit busA busB Imm16 ALUOp ALUSrc Mux 1 0 MemtoReg 1 0 RegDst Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 Clk IfetchReg/DecExecMem Location 8: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100] The Instruction Fetch Stage

50 © Alvin R. Lebeck 2001 Location 8: lw $1, 0x100($2) lw $1, 100 ($2) PC = 12 “8” Adder Instruction Memory “4” Instruction Address Clk Ifetch You are here! Reg/Dec PC+4 32 Detailed View of the Instruction Fetch Unit

51 © Alvin R. Lebeck 2001 Location 8: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100] The Decode / Register Fetch Stage

52 © Alvin R. Lebeck 2001 OP rs rt rd func PC + 4 Rw Control Rb Ra rt rs Register File rt rd Imm16 Bus-A Bus-B PC+4 Din Clk Detailed View of the Fetch/Decode Stage

53 © Alvin R. Lebeck 2001 Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100] IF/ID: ID/Ex Register Ex/Mem: Load’s Address Mem/Wr Register PC Data Mem WA Di RADo IUnit A I RFile Di Ra Rb Rw MemWr RegWr ExtOp=1 Exec Unit busA busB Imm16 ALUOp=Add ALUSrc=1 Mux 1 0 MemtoReg 1 0 RegDst=0 Rt Rd Imm16 PC+4 Rs Rt PC+4 Zero Branch 1 0 Clk IfetchReg/DecExecMem You are here! Load’s Address Calculation Stage

54 © Alvin R. Lebeck 2001 ID/Ex Register Ex/Mem: Load’s Memory Address ALU CONTROL ALUctr 32 busA 32 busB Extender Mux 16 imm16 ALUSrc=1 ExtOp=1 3 ALU Zero ALUout 32 Adder 3 ALUOp=Add << 2 32 PC+4 Target 32 Clk Exec You are here! Mem Detailed View of the Execution Unit

55 © Alvin R. Lebeck 2001 You are here! Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100] Load’s Memory Access Stage

56 © Alvin R. Lebeck 2001 Location 10: lw $1, 0x100($2) $1 <- Mem[($2) + 0x100] Load’s Write Back Stage

57 © Alvin R. Lebeck 2001 Next Time Pipeline Complications