CECS 440 Pipelining.1(c) 2014 – R. W. Allison [slides adapted from D. Patterson slides with additional credits to M.J. Irwin]

Slides:



Advertisements
Similar presentations
CMSC 611: Advanced Computer Architecture Pipelining Some material adapted from Mohamed Younis, UMBC CMSC 611 Spr 2003 course slides Some material adapted.
Advertisements

Review: Pipelining. Pipelining Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer.
Review: MIPS Pipeline Data and Control Paths
ENEE350 Ankur Srivastava University of Maryland, College Park Based on Slides from Mary Jane Irwin ( )
ENEE350 Ankur Srivastava University of Maryland, College Park Based on Slides from Mary Jane Irwin ( )
Mary Jane Irwin ( ) [Adapted from Computer Organization and Design,
ENEE350 Ankur Srivastava University of Maryland, College Park Based on Slides from Mary Jane Irwin ( )
Datorsystem 1 och Datorarkitektur 1 – föreläsning 10 måndag 19 November 2007.
1  2004 Morgan Kaufmann Publishers Chapter Six. 2  2004 Morgan Kaufmann Publishers Pipelining The laundry analogy.
©UCB CS 162 Computer Architecture Lecture 3: Pipelining Contd. Instructor: L.N. Bhuyan
1 Stalling  The easiest solution is to stall the pipeline  We could delay the AND instruction by introducing a one-cycle delay into the pipeline, sometimes.
Computer ArchitectureFall 2007 © October 24nd, 2007 Majd F. Sakr CS-447– Computer Architecture.
331 Lec18.1Fall :332:331 Computer Architecture and Assembly Language Fall 2003 Lecture 18 Introduction to Pipelined Datapath [Adapted from Dave.
Chapter 6 Pipelining to Increase Effective Computer Speed.
CS 152 L10 Pipeline Intro (1)Fall 2004 © UC Regents CS152 – Computer Architecture and Engineering Fall 2004 Lecture 10: Basic MIPS Pipelining Review John.
Computer ArchitectureFall 2007 © October 22nd, 2007 Majd F. Sakr CS-447– Computer Architecture.
1 CSE SUNY New Paltz Chapter Six Enhancing Performance with Pipelining.
Computer ArchitectureFall 2008 © October 6th, 2008 Majd F. Sakr CS-447– Computer Architecture.
CS 300 – Lecture 24 Intro to Computer Architecture / Assembly Language The LAST Lecture!
Control Hazards.1 Review: Datapath with Data Hazard Control Read Address Instruction Memory Add PC 4 Write Data Read Addr 1 Read Addr 2 Write Addr Register.
Spring W :332:331 Computer Architecture and Assembly Language Spring 2005 Week 11 Introduction to Pipelined Datapath [Adapted from Dave Patterson’s.
Chapter 4 Sections 4.5 – 4.8 Dr. Iyad F. Jafar
1 Pipelining Reconsider the data path we just did Each instruction takes from 3 to 5 clock cycles However, there are parts of hardware that are idle many.
Pipeline Data Hazards: Detection and Circumvention Adapted from Computer Organization and Design, Patterson & Hennessy, © 2005, and from slides kindly.
EEL5708 Lotzi Bölöni EEL 5708 High Performance Computer Architecture Pipelining.
CPE432 Chapter 4B.1Dr. W. Abu-Sufah, UJ Chapter 4B: The Processor, Part B-2 Read Section 4.7 Adapted from Slides by Prof. Mary Jane Irwin, Penn State University.
Analogy: Gotta Do Laundry
CSE431 Chapter 4A.1Irwin, PSU, 2008 CSE 431 Computer Architecture Fall 2008 Chapter 4A: The Processor, Part A Mary Jane Irwin ( )
CSE 340 Computer Architecture Summer 2014 Basic MIPS Pipelining Review.
Basic Pipelining & MIPS Pipelining Chapter 6 [Computer Organization and Design, © 2007 Patterson (UCB) & Hennessy (Stanford), & Slides Adapted from: Mary.
CS.305 Computer Architecture Enhancing Performance with Pipelining Adapted from Computer Organization and Design, Patterson & Hennessy, © 2005, and from.
Computer Organization CS224 Chapter 4 Part b The Processor Spring 2010 With thanks to M.J. Irwin, T. Fountain, D. Patterson, and J. Hennessy for some lecture.
CMPE 421 Parallel Computer Architecture
1 Designing a Pipelined Processor In this Chapter, we will study 1. Pipelined datapath 2. Pipelined control 3. Data Hazards 4. Forwarding 5. Branch Hazards.
Computer Architecture and Design – ELEN 350 Part 8 [Some slides adapted from M. Irwin, D. Paterson. D. Garcia and others]
Electrical and Computer Engineering University of Cyprus LAB3: IMPROVING MIPS PERFORMANCE WITH PIPELINING.
ECE 232 L18.Pipeline.1 Adapted from Patterson 97 ©UCBCopyright 1998 Morgan Kaufmann Publishers ECE 232 Hardware Organization and Design Lecture 18 Pipelining.
CMPE 421 Parallel Computer Architecture Part 2: Hardware Solution: Forwarding.

Cs 152 L1 3.1 DAP Fa97,  U.CB Pipelining Lessons °Pipelining doesn’t help latency of single task, it helps throughput of entire workload °Multiple tasks.
Chap 6.1 Computer Architecture Chapter 6 Enhancing Performance with Pipelining.
CSIE30300 Computer Architecture Unit 04: Basic MIPS Pipelining Hsin-Chou Chi [Adapted from material by and
CSE431 L07 Overcoming Data Hazards.1Irwin, PSU, 2005 CSE 431 Computer Architecture Fall 2005 Lecture 07: Overcoming Data Hazards Mary Jane Irwin (
1  1998 Morgan Kaufmann Publishers Chapter Six. 2  1998 Morgan Kaufmann Publishers Pipelining Improve perfomance by increasing instruction throughput.
CSIE30300 Computer Architecture Unit 05: Overcoming Data Hazards Hsin-Chou Chi [Adapted from material by and
CSE431 L06 Basic MIPS Pipelining.1Irwin, PSU, 2005 MIPS Pipeline Datapath Modifications  What do we need to add/modify in our MIPS datapath? l State registers.
Lecture 9. MIPS Processor Design – Pipelined Processor Design #1 Prof. Taeweon Suh Computer Science Education Korea University 2010 R&E Computer System.
CPE432 Chapter 4B.1Dr. W. Abu-Sufah, UJ Chapter 4B: The Processor, Part B-1 Read Sections 4.7 Adapted from Slides by Prof. Mary Jane Irwin, Penn State.
CSE 340 Computer Architecture Spring 2016 Overcoming Data Hazards.
Computer Organization
Exceptions Another form of control hazard Could be caused by…
Stalling delays the entire pipeline
Single Clock Datapath With Control
Chapter 4 The Processor Part 4
ECE232: Hardware Organization and Design
ECE232: Hardware Organization and Design
Chapter 4 The Processor Part 3
Review: MIPS Pipeline Data and Control Paths
Chapter 4 The Processor Part 2
Pipelining review.
Pipelining in more detail
CS-447– Computer Architecture Lecture 14 Pipelining (2)
The Processor Lecture 3.6: Control Hazards
The Processor Lecture 3.4: Pipelining Datapath and Control
Control unit extension for data hazards
The Processor Lecture 3.5: Data Hazards
Control unit extension for data hazards
Pipelining Appendix A and Chapter 3.
Control unit extension for data hazards
Presentation transcript:

CECS 440 Pipelining.1(c) 2014 – R. W. Allison [slides adapted from D. Patterson slides with additional credits to M.J. Irwin]

CECS 440 Pipelining.2(c) 2014 – R. W. Allison Actual time saved Review: Single Cycle vs. Multiple Cycle Timing Clk Cycle 1 Multiple Cycle Implementation: IFetchDecExecMemWB Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9Cycle 10 IFetchDecExecMem lwsw IFetch R-type Clk Single Cycle Implementation: lwsw Waste Cycle 1Cycle 2 multicycle clock slower than 1/5 th of single cycle clock due to stage register overhead

CECS 440 Pipelining.3(c) 2014 – R. W. Allison How Can We Make It Even Faster?  Split the multiple instruction cycle into smaller and smaller steps l There is a point of diminishing returns where as much time is spent loading the state registers as doing the work  Start fetching and executing the next instruction before the current one has completed l Pipelining – (all?) modern processors are pipelined for performance  Fetch (and execute) more than one instruction at a time (out-of-order superscalar and VLIW)  Fetch (and execute) instructions from more than one instruction stream (multithreading [hyperthreading])

CECS 440 Pipelining.4(c) 2014 – R. W. Allison A Pipelined MIPS Processor  Start the next instruction before the current one has completed l improves throughput - total amount of work done in a given time l instruction latency (execution time, delay time, response time, i.e. the time from the start of an instruction to its completion) is not reduced Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5 IFetchDecExecMemWB lw Cycle 7Cycle 6Cycle 8 sw IFetchDecExecMem WB R-type IFetchDecExec Mem WB -clock cycle (pipeline stage time) is limited by the slowest stage -for some instructions, some stages are wasted cycles

CECS 440 Pipelining.5(c) 2014 – R. W. Allison Actual time saved Single Cycle, Multiple Cycle, vs. Pipeline Multiple Cycle Implementation: Clk Cycle 1 IFetchDecExecMemWB Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9Cycle 10 IFetchDecExecMem lwsw IFetch R-type lw IFetchDecExecMemWB Pipeline Implementation: IFetchDecExecMem WB sw IFetchDecExec Mem WB R-type Clk Single Cycle Implementation: lwsw Waste Cycle 1Cycle 2

CECS 440 Pipelining.6(c) 2014 – R. W. Allison Write Data Read Addr 1 Read Addr 2 Write Addr Register File Read Data 1 Read Data Dec/Exec ID:Dec Sign Extend MIPS Pipeline Datapath Modifications  What do we need to add/modify in our MIPS datapath? l State registers between each pipeline stage to isolate them Data Memory Address Write Data Read Data MEM: MemAccess Mem/WB Read Address Instruction Memory Add PC 4 IFetch/Dec IF:IFetch System Clock WB: WriteBack Exec/Mem EX:Execute ALU Shift left 2 Add

CECS 440 Pipelining.7(c) 2014 – R. W. Allison MIPS Pipeline Control Path Modifications  All control signals can be determined during Decode l and held in the state registers between pipeline stages Read Address Instruction Memory Add PC 4 Write Data Read Addr 1 Read Addr 2 Write Addr Register File Read Data 1 Read Data ALU Shift left 2 Add Data Memory Address Write Data Read Data IF/ID Sign Extend ID/EX EX/MEM MEM/WB Control

CECS 440 Pipelining.8(c) 2014 – R. W. Allison  Pass control signals along just like the data Pipeline Control

CECS 440 Pipelining.9(c) 2014 – R. W. Allison Pipelining the MIPS ISA  What makes it easy l all instructions are the same length (32 bits)  can fetch in the 1 st stage and decode in the 2 nd stage l few instruction formats (three) with symmetry across formats  can begin reading register file in 2 nd stage l memory operations can occur only in loads and stores  can use the execute stage to calculate memory addresses l each MIPS instruction writes at most one result (i.e., changes the machine state) and does so near the end of the pipeline (MEM and WB)  What makes it hard l structural hazards: what if we had only one memory? l control hazards: what about branches? l data hazards: what if an instruction’s input operands depend on the output of a previous instruction?

CECS 440 Pipelining.10(c) 2014 – R. W. Allison Graphically Representing MIPS Pipeline  Can help with answering questions like: l How many cycles does it take to execute this code? l What is the ALU doing during cycle 4? l Is there a hazard, why does it occur, and how can it be fixed? ALU IM Reg DM Reg

CECS 440 Pipelining.11(c) 2014 – R. W. Allison Why Pipeline? For Performance! I n s t r. O r d e r Time (clock cycles) Inst 0 Inst 1 Inst 2 Inst 4 Inst 3 Once the pipeline is full, one instruction is completed every cycle so complete an instruction every cycle (CPI = 1) Time to fill the pipeline ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.12(c) 2014 – R. W. Allison Can Pipelining Get Us Into Trouble?  Yes: Pipeline Hazards l structural hazards: attempt to use the same resource by two different instructions at the same time l data hazards: attempt to use data before it is ready  An instruction’s source operand(s) are produced by a prior instruction still in the pipeline l control hazards: attempt to make a decision about program control flow before the condition has been evaluated and the new PC target address calculated  branch and jump instructions, exceptions  Can always resolve hazards by waiting pipeline control must detect the hazard and take action to resolve hazards

CECS 440 Pipelining.13(c) 2014 – R. W. Allison ALU IM Reg DM Reg ALU IM Reg DM Reg I n s t r. O r d e r Time (clock cycles) lw Inst 1 Inst 2 Inst 4 Inst 3 A Single Memory Would Be a Structural Hazard Reading data from memory Reading instruction from memory  Can fix with separate instruction and data memories ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.14(c) 2014 – R. W. Allison ALU IM Reg DM Reg How About Register File Access? I n s t r. O r d e r Time (clock cycles) add $1, Inst 1 Inst 2 add $2,$1, ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.15(c) 2014 – R. W. Allison How About Register File Access? I n s t r. O r d e r Time (clock cycles) Inst 1 Inst 2 Fix register file access hazard by doing writes in the first half of the cycle and reads in the second half add $1, add $2,$1, clock edge that controls register writing clock edge that controls loading of pipeline state registers ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.16(c) 2014 – R. W. Allison Register Usage Can Cause Data Hazards I n s t r. O r d e r add $1, sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9  Dependencies going “backward in time” cause hazards  Read before write data hazard ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.17(c) 2014 – R. W. Allison Register Usage Can Cause Data Hazards add $1, sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9  Read before write data hazard ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg  Dependencies going “backward in time” cause hazards

CECS 440 Pipelining.18(c) 2014 – R. W. Allison Loads Can Cause Data Hazards I n s t r. O r d e r lw $1,4($2) sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9  Dependencies going “backward in time” cause hazards ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.19(c) 2014 – R. W. Allison Loads Can Cause Data Hazards I n s t r. O r d e r lw $1,4($2) sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9  Load-use data hazard ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg  Dependencies going “backward in time” cause hazards

CECS 440 Pipelining.20(c) 2014 – R. W. Allison noop (stall) One Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, sub $4,$1,$5 and $6,$1,$7 ALU IM Reg DM Reg ALU IM Reg DM Reg Can fix data hazard by waiting – stall ALU IM Reg DM Reg

CECS 440 Pipelining.21(c) 2014 – R. W. Allison Another Way to “Fix” a Data Hazard I n s t r. O r d e r add $1, sub $4,$1,$5 and $6,$1,$7 Fix data hazards by forwarding results as soon as they are available to where they are needed xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.22(c) 2014 – R. W. Allison Another Way to “Fix” a Data Hazard Fix data hazards by forwarding results as soon as they are available to where they are needed I n s t r. O r d e r add $1, sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.23(c) 2014 – R. W. Allison How to Implement Forwarding  The main idea (some details not shown) Rs, Rt from “Instr” Rd from “Instr-1” Rd from “Instr-2” Forwarding Unit must detect data hazards Forward Source 0 0 ID/EX 1 0 EX/MEM 0 1 MEM/WB

CECS 440 Pipelining.24(c) 2014 – R. W. Allison Forwarding with Load-use Data Hazards  Will we still need one stall cycle even with forwarding? I n s t r. O r d e r lw $1,4($2) sub $4,$1,$5 and $6,$1,$7 xor $4,$1,$5 or $8,$1,$9 ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.25(c) 2014 – R. W. Allison Hazard Detection Unit  Stalling by creating a no-operation (noop) instruction Consider this sequence: 0x100 lw $t1, 24[$s1] 0x104 add $t3, $t1, $t2 0x108 sub $t4, $t1, $t2 PC=0x108. Fetching the “sub” instruction EA calc for instruction at 0x100 (lw) is being done in this stage; thus, MemRead=1. Reg codes and operands for “lw” Getting $t2 and $t1 for the “add” instruction at 0x104 add $t3, $t2, $t1 Hazard detected, so (1)the mux in ID stage selects all “0’s” (i.e nop) (2) PCWrite=0, so PC not change, remaining at 0x108 (3) IF/Dwrite=0, so the add instruction remains. Thus, the sub instruction will be fetched after “stalling” for one clock cycle Rt = $t1 Rs = $t1 Rt = $t2 MemRead=1 Hazard Detection: If ( Ex.MemRead & (Ex.Rt==ID.rs | Ex.Rt == ID.Rt) )

CECS 440 Pipelining.26(c) 2014 – R. W. Allison Control Hazards  When the flow of instruction addresses is not sequential (i.e., PC = PC + 4) l Conditional branches ( beq, bne ) l Unconditional branches ( j, jal, jr ) l Exceptions  Possible “solutions” l Stall (impacts performance) l Move branch decision point as early in the pipeline as possible, thereby reducing the number of stall cycles l Delay decision (requires compiler support) l Predict and hope for the best !  Control hazards occur less frequently than data hazards, but there is nothing as effective against control hazards as forwarding is for data hazards

CECS 440 Pipelining.27(c) 2014 – R. W. Allison flush Jumps Incur One Stall I n s t r. O r d e r j j target  Fortunately, jumps are very infrequent – only 3% of the SPECint instruction mix  Jumps not decoded until ID, so one flush is needed l To flush, set IF.Flush=1 to “zero” the instruction field of the IF/ID pipeline register (turning it into a noop) Fix jump hazard by waiting – flush ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.28(c) 2014 – R. W. Allison Review: MIPS Pipeline Control & Datapath  All control signals can be determined during Decode l and held in the state registers between pipeline stages Read Address Instruction Memory Add PC 4 Write Data Read Addr 1 Read Addr 2 Write Addr Register File Read Data 1 Read Data ALU Shift left 2 Add Data Memory Address Write Data Read Data IF/ID Sign Extend ID/EX EX/MEM MEM/WB Control

CECS 440 Pipelining.29(c) 2014 – R. W. Allison ALU IM Reg DM Reg ALU IM Reg DM Reg Branches Cause Control Hazards I n s t r. O r d e r lw Inst 4 Inst 3  Dependencies backward in time cause hazards ALU IM Reg DM Reg ALU IM Reg DM Reg beq

CECS 440 Pipelining.30(c) 2014 – R. W. Allison flush One Way to “Fix” a Branch Control Hazard I n s t r. O r d e r beq beq target ALU Inst 3 IM Reg DM Fix branch hazard by waiting – flush – but affects CPI ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.31(c) 2014 – R. W. Allison flush Another Way to “Fix” a Branch Control Hazard I n s t r. O r d e r beq beq target Inst 3 Fix branch hazard by waiting – flush  Move branch decision hardware back to as early in the pipeline as possible – i.e., during the decode cycle ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg

CECS 440 Pipelining.32(c) 2014 – R. W. Allison 8 sub $4,$1,$5 flush Yet Another Way to “Fix” a Control Hazard 4 beq $1,$2,2 I n s t r. O r d e r ALU IM Reg DM Reg 16 and $6,$1,$7 20 or r8,$1,$9 ALU IM Reg DM Reg ALU IM Reg DM Reg ALU IM Reg DM Reg  To flush, set IF.Flush=1 to zero the instruction field of the IF/ID pipeline register (turning it into a noop )  “Predict branches are always not taken – and take corrective action when wrong (i.e., taken)

CECS 440 Pipelining.33(c) 2014 – R. W. Allison Two “Types” of Stalls  Noop instruction inserted between two instructions in the pipeline (e.g., load-use hazards) l Keep the instructions earlier in the pipeline (later in the code) from progressing down the pipeline for a cycle (“bounce” them in place with write control signals) l Insert noop instruction by zeroing control bits in the pipeline register at the appropriate stage l Let the instructions later in the pipeline (earlier in the code) progress normally down the pipeline  Flushes (or instruction squashing) where an instruction in the pipeline is replaced with a noop instruction (as done for instructions located sequentially after j and beq instructions) l Zero the control bits for the instruction to be flushed

CECS 440 Pipelining.34(c) 2014 – R. W. Allison Many Other Pipeline Structures Are Possible  What about the (slow) multiply operation? l Make the clock twice as slow or … l let it take two cycles (since it doesn’t use the DM stage) ALU IM Reg DM Reg MUL ALU IM Reg DM1 Reg DM2  What if the data memory access is twice as slow as the instruction memory? l make the clock twice as slow or … l let data memory access take two cycles (and keep the same clock rate)

CECS 440 Pipelining.35(c) 2014 – R. W. Allison Pipelining Summary  All modern day processors use pipelining  Pipelining doesn’t help latency of single task, it helps throughput of entire workload  Potential speedup: a really fast clock cycle and able to complete one instruction every clock cycle (CPI)  Pipeline rate limited by slowest pipeline stage l Unbalanced pipe stages makes for inefficiencies l The time to “fill” pipeline and time to “drain” it can impact speedup for deep pipelines and short code runs  Must detect and resolve hazards l Stalling negatively affects CPI (makes CPI less than the ideal of 1)