1. Computer Organization
Lecture Set – 06
Chapter 6
Huei-Yung Lin
Lectures 1 & 2

2. Roadmap for the Term: Major Topics
Overview / Abstractions and Technology
Performance
Instruction sets
Logic & arithmetic
Processor Implementation
Single-cycle implemenatation
Multicycle implementation
Pipelined Implementation \
Memory systems
Input/Output
H.Y. Lin, CCUEE
Computer Organization 4

3. Computer Organization
Pipelining Outline
Introduction
Defining Pipelining \
Pipelining Instructions
Hazards
Pipelined Processor Design
Datapath
Control
Advanced Pipelining
Superscalar
Dynamic Pipelining
Examples
H.Y. Lin, CCUEE
Computer Organization

4. Computer Organization
What is Pipelining?
A way of speeding up execution of instructions
Key idea: overlap execution of multiple instructions
Analogy: doing your laundry
1. Run load through washer
2. Run load through dryer
3. Fold clothes
4. Put away clothes
5. Go to 1
Observation: we can start another load as soon as we finish step 1!
H.Y. Lin, CCUEE
Computer Organization

5. Computer Organization
The Laundry Analogy
Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold
Washer takes 30 minutes
Dryer takes 30 minutes
“Folder” takes 30 minutes
“Stasher” takes 30 minutes to put clothes into drawers A B C D
H.Y. Lin, CCUEE
Computer Organization

6. If we do laundry sequentially...
6 PM 7 8 9 10 11 12 1
2 AM 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 T a s k O r d e
Time A B C D
Time Required: 8 hours for 4 loads
H.Y. Lin, CCUEE
Computer Organization

7. To Pipeline, We Overlap Tasks
6 PM 7 8 9 10 11 12 1
2 AM
Time 30 30 30 30 30 30 30 T a s k O r d e B A D C
Time Required: 3.5 Hours for 4 Loads
Latency remains 2 hours
Throughput improves by factor of 2.3 (decreases for more loads)
H.Y. Lin, CCUEE
Computer Organization

8. Pipelining a Digital System
Key idea: break big computation up into pieces
Separate each piece with a pipeline register
1ns
200ps
Pipeline
Register
H.Y. Lin, CCUEE
Computer Organization

9. Pipelining a Digital System
Why do this?
Because it's faster for repeated computations
1ns
Non-pipelined:
1 operation finishes
every 1ns
200ps
Pipelined:
1 operation finishes
every 200ps
H.Y. Lin, CCUEE
Computer Organization

10. Comments about pipelining
Pipelining increases throughput, but not latency
Answer available every 200ps, BUT
A single computation still takes 1ns
Limitations:
Computations must be divisible into stage size
Pipeline registers add overhead
H.Y. Lin, CCUEE
Computer Organization

11. Pipelining a Processor
Recall the 5 steps in instruction execution:
1. Instruction Fetch
2. Instruction Decode and Register Read
3. Execution operation or calculate address
4. Memory access
5. Write result into register
Review: Single-Cycle Processor
All 5 steps done in a single clock cycle
Dedicated hardware required for each step
What happens if we break execution into multiple cycles, but keep the extra hardware?
H.Y. Lin, CCUEE
Computer Organization

12. Review - Single-Cycle ProcessorIF
Instruction Fetch ID
Instruction Decode EX
Execute/ Address Calc.
MEM
Memory Access WB
Write Back
H.Y. Lin, CCUEE
Computer Organization

13. Computer Organization
Pipelining - Key Idea
Question:
What happens if we break execution into multiple cycles, but keep the extra hardware?
Answer:
In the best case, we can start executing a new instruction on each clock cycle – this is pipelining
Pipelining stages:
IF - Instruction Fetch
ID - Instruction Decode
EX - Execute / Address Calculation
MEM - Memory Access (read / write)
WB - Write Back (results into register file)
H.Y. Lin, CCUEE
Computer Organization

14. Basic Pipelined Processor
Pipeline Registers
IF/ID
ID/EX
EX/MEM
MEM/WB
H.Y. Lin, CCUEE
Computer Organization

15. Single-Cycle vs. Pipelined Execution
Non-Pipelined
Pipelined
H.Y. Lin, CCUEE
Computer Organization

16. Comments about Pipelining
The good news
Multiple instructions are being processed at same time
This works because stages are isolated by registers
Best case speedup of N
The bad news
Instructions interfere with each other – hazards
Example: different instructions may need the same piece of hardware (e.g., memory) in same clock cycle
Example: instruction may require a result produced by an earlier instruction that is not yet complete
Worst case: must suspend execution – stall
H.Y. Lin, CCUEE
Computer Organization

17. Pipelined Example – Executing Multiple Instructions
Consider the following instruction sequence:
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
H.Y. Lin, CCUEE
Computer Organization

18. Executing Multiple Instructions Clock Cycle 1
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10 LW
H.Y. Lin, CCUEE
Computer Organization

19. Executing Multiple Instructions Clock Cycle 2
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10 SW LW
H.Y. Lin, CCUEE
Computer Organization

20. Executing Multiple Instructions Clock Cycle 3
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
ADD SW LW
H.Y. Lin, CCUEE
Computer Organization

21. Executing Multiple Instructions Clock Cycle 4
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
SUB
ADD SW LW
H.Y. Lin, CCUEE
Computer Organization

22. Executing Multiple Instructions Clock Cycle 5
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
SUB
ADD SW LW
H.Y. Lin, CCUEE
Computer Organization

23. Executing Multiple Instructions Clock Cycle 6
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
SUB
ADD SW
H.Y. Lin, CCUEE
Computer Organization

24. Executing Multiple Instructions Clock Cycle 7
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
SUB
ADD
H.Y. Lin, CCUEE
Computer Organization

25. Executing Multiple Instructions Clock Cycle 8
lw $r0, 10($r1)
sw $sr3, 20($r4)
add $r5, $r6, $r7
sub $r8, $r9, $r10
SUB
H.Y. Lin, CCUEE
Computer Organization

26. Alternative View – Multicycle Diagram
H.Y. Lin, CCUEE
Computer Organization

27. Computer Organization
Pipeline Hazards
Where one instruction cannot immediately follow another
Types of hazards
Structural hazards – attempt to use same resource twice
Control hazards – attempt to make decision before condition is evaluated
Data hazards – attempt to use data before it is ready
Can always resolve hazards by waiting
H.Y. Lin, CCUEE
Computer Organization

28. Computer Organization
Structural Hazards
Attempt to use same resource twice at same time
Example: Single Memory for instructions, data
Accessed by IF stage
Accessed at same time by MEM stage
Solutions
Delay second access by one clock cycle, OR
Provide separate memories for instructions, data
This is what the book does
This is called a “Harvard Architecture”
Real pipelined processors have separate caches
H.Y. Lin, CCUEE
Computer Organization

29. Example Structural Hazard – Single Memory
Memory Conflict
H.Y. Lin, CCUEE
Computer Organization

30. Computer Organization
Control Hazards
Attempt to make a decision before condition is evaluated
Example: beq $s0, $s1, offset
Assume we add hardware to second stage to:
Compare fetched registers for equality
Compute branch target
This allows branch to be taken at end of second clock cycle
But, this still means result is not ready when we want to load the next instruction!
H.Y. Lin, CCUEE
Computer Organization

31. Control Hazard Solutions
Stall – stop loading instructions until result is available
Predict – assume an outcome and continue fetching (undo if prediction is wrong)
Delayed branch – specify in architecture that following instruction is always executed
H.Y. Lin, CCUEE
Computer Organization

32. Computer Organization
Control Hazard – Stall
beq
writes PC
here
new PC
used here
H.Y. Lin, CCUEE
Computer Organization

33. Control Hazard – Correct Prediction
Fetch assuming
branch taken
H.Y. Lin, CCUEE
Computer Organization

34. Control Hazard – Incorrect Prediction
“Squashed”
instruction
H.Y. Lin, CCUEE
Computer Organization

35. Control Hazard – Delayed Branch
always executes
correct PC avail. here
H.Y. Lin, CCUEE
Computer Organization

36. Summary – Control Hazard Solutions
Stall – stop fetching instr. until result is available
Significant performance penalty
Hardware required to stall
Predict – assume an outcome and continue fetching (undo if prediction is wrong)
Performance penalty only when guess wrong
Hardware required to "squash" instructions
Delayed branch – specify in architecture that following instruction is always executed
Compiler re-orders instructions into delay slot
Insert "NOP" (no-op) operations when can't use (~50%)
This is how original MIPS worked
H.Y. Lin, CCUEE
Computer Organization

37. Computer Organization
Data Hazards
Attempt to use data before it is ready
Solutions
Stalling – wait until result is available
Forwarding – make data available inside datapath
Reordering instructions – use compiler to avoid hazards
Examples: add $s0, $t0, $t1 ; $s0 = $t0+$t1 sub $t2, $s0, $t3 ; $t2 = $s0-$t2 lw $s0, 0($t0) ; $s0 = MEM[$t0] sub $t2, $s0, $t3 ; $t2 = $s0-$t2
H.Y. Lin, CCUEE
Computer Organization

38. Computer Organization
Data Hazard – Stalling
H.Y. Lin, CCUEE
Computer Organization

39. Data Hazards – Forwarding
Key idea: connect new value directly to next stage
Still read s0, but ignore in favor of new result
Problem: what about load instructions?
H.Y. Lin, CCUEE
Computer Organization

40. Data Hazards – Forwarding
STALL still required for load – data avail. after MEM
MIPS architecture calls this delayed load, initial implementations required compiler to deal with this
H.Y. Lin, CCUEE
Computer Organization

41. Data Hazards – Reordering Instructions
Assuming we have data forwarding, what are the hazards in this code? lw $t0, 0($t1) lw $t2, 4($t1) sw $t2, 0($t1) sw $t0, 4($t1)
Reorder instructions to remove hazard: lw $t0, 0($t1) lw $t2, 4($t1) sw $t0, 4($t1) sw $t2, 0($t1)
H.Y. Lin, CCUEE
Computer Organization

42. Summary - Pipelining Overview
Pipelining increase throughput (but not latency)
Hazards limit performance
Structural hazards
Control hazards
Data hazards
H.Y. Lin, CCUEE
Computer Organization

43. Computer Organization
Pipelining Outline
Introduction
Pipelined Processor Design \
Datapath
Control
Dealing with Hazards & Forwarding
Branch Prediction
Exceptions
Performance
Advanced Pipelining
Superscalar
Dynamic Pipelining
Examples
H.Y. Lin, CCUEE
Computer Organization

44. Computer Organization
Pipelining in MIPS
MIPS architecture was designed to be pipelined
Simple instruction format (makes IF, ID easy)
Single-word instructions
Small number of instruction formats
Common fields in same place (e.g., rs, rt) in different formats
Memory operations only in lw, sw instructions (simplifies EX)
Memory operands aligned in memory (simplifies MEM)
Single value for writeback (limits forwarding)
Pipelining is harder in CISC architectures
H.Y. Lin, CCUEE
Computer Organization

45. Pipelined Datapath with Control Signals
H.Y. Lin, CCUEE
Computer Organization

46. Next Step: Adding Control
Basic approach: build on single-cycle control
Place control unit in ID stage
Pass control signals to following stages
Later: extra features to deal with:
Data forwarding
Stalls
Exceptions
H.Y. Lin, CCUEE
Computer Organization

47. Control for Pipelined Datapath
RegDst
ALUOp[1:0]
ALUSrc
MemRead
MemWrite
Branch
RegWrite
MemtoReg
Source: Book Fig. 6.29, p 469
H.Y. Lin, CCUEE
Computer Organization

48. Control for Pipelined Datapath
Source: Book Fig. 6.25, p 401
H.Y. Lin, CCUEE
Computer Organization

49. Datapath and Control Unit
H.Y. Lin, CCUEE
Computer Organization

50. Tracking Control Signals - Cycle 1LW
H.Y. Lin, CCUEE
Computer Organization

51. Tracking Control Signals - Cycle 2SW LW
H.Y. Lin, CCUEE
Computer Organization

52. Tracking Control Signals - Cycle 301 1
ADD SW LW
H.Y. Lin, CCUEE
Computer Organization

53. Tracking Control Signals - Cycle 41
SUB
ADD SW LW
H.Y. Lin, CCUEE
Computer Organization

54. Tracking Control Signals - Cycle 51
SUB
ADD SW LW
H.Y. Lin, CCUEE
Computer Organization

55. Computer Organization
References
Portions of these slides are derived from:
Textbook figures © 1998 Morgan Kaufmann Publishers all rights reserved
Tod Amon's COD2e Slides © 1998 Morgan Kaufmann Publishers all rights reserved
Dave Patterson’s CS 152 Slides – Fall 1997 © UCB
Rob Rutenbar’s Slides – Fall 1999 CMU
John Nestor’s ECE 313 Slides – Fall 2004 LC
T.S. Chang’s DEE 1050 Slides – Fall 2004 NCTU
Other sources as noted
H.Y. Lin, CCUEE
Computer Organization