1. Introduction to Pipelining Rabi Mahapatra Adapted from the lecture notes of Dr. John Kubiatowicz (UC Berkeley)

2. Pipelining is Natural! Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes “Folder” takes 20 minutes ABCD

3. Sequential Laundry Sequential laundry takes 6 hours for 4 loads ABCD 304020304020304020304020 6 PM 789 10 11 Midnight TaskOrderTaskOrder Time

4. Pipelined Laundry: Start work ASAP Pipelined laundry takes 3.5 hours for 4 loads ABCD 6 PM 789 10 11 Midnight TaskOrderTaskOrder Time 3040 20

5. Pipelining Lessons Latency vs. Throughput Question –What is the latency in both cases ? –What is the throughput in both cases ? Pipelining doesn’t help latency of single task, it helps throughput of entire workload ABCD 3040 20

6. Pipelining Lessons [contd…] Question –What is the fastest operation in the example ? –What is the slowest operation in the example Pipeline rate limited by slowest pipeline stage ABCD 3040 20

7. Pipelining Lessons [contd…] ABCD 3040 20 Multiple tasks operating simultaneously using different resources

8. Pipelining Lessons [contd…] Question –Would the speedup increase if we had more steps ? ABCD 3040 20 Potential Speedup = Number of pipe stages

9. Pipelining Lessons [contd…] Washer takes 30 minutes Dryer takes 40 minutes “Folder” takes 20 minutes Question –Will it affect if “Folder” also took 40 minutes Unbalanced lengths of pipe stages reduces speedup

10. Pipelining Lessons [contd…] ABCD 3040 20 Time to “fill” pipeline and time to “drain” it reduces speedup

11. Five Stages of an Instruction Ifetch: Instruction Fetch –Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: Calculate the memory address Mem: Read the data from the Data Memory Wr: Write the data back to the register file Cycle 1Cycle 2Cycle 3Cycle 4Cycle 5 IfetchReg/DecExecMemWrLoad

12. Conventional Pipelined Execution Representation IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB Program Flow Time

13. Example

14. Example [contd…] Time pipeline = Time non-pipeline / Pipe stages –Assumptions Stages are perfectly balanced Ideal conditions Ideally, speedup = 8/5 = 1.6 Most cases are not ideal !!!

15. Example [contd…] Speedup in this case = 24/14 = 1.7 Lets add 1000 more instructions –Time (non-pipelined) = 1000 x 8 + 24 ns = 8000 ns –Time (pipelined) = 1000 x 2 + 14 ns = 2014 ns –Speedup = 8000 / 2014 = 3.98 = 4 (approx) = 8/2 Instruction throughput is important metric (as opposed to individual instruction) as real programs execute billions of instructions in practical case !!!

16. Pipeline Hazards Structural Hazard IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB IFetchDcdExecMemWB Program Flow

17. Pipeline Hazard [contd…] Control Hazard Example –add $4, $5, $6 –beq$1, $2, 40 –lw$3, 300($0)

18. Pipleline Hazard [contd…] Data Hazards Example –add$s0, $t0, $t1 –sub$t2, $s0, $t3

19. Summary Pipelining Lessons Pipelining doesn’t help latency of single task, it helps throughput of entire workload Pipeline rate limited by slowest pipeline stage Multiple tasks operating simultaneously using different resources Potential speedup = Number pipe stages Unbalanced lengths of pipe stages reduces speedup Time to “fill” pipeline and time to “drain” it reduces speedup Stall for Dependences ABCD 6 PM 789 TaskOrderTaskOrder Time 3040 20

20. Summary of Pipeline Hazards Structural Hazards –Hardware design Control Hazard –Decision based on results Data Hazard –Data Dependency