1. Chapter 4 The Processor Part 2
박능수

2. Performance Issues Longest delay determines clock period
Critical path: load instruction
Instruction memory  register file  ALU  data memory  register file
Not feasible to vary period for different instructions
Violates design principle
Making the common case fast
We will improve performance by pipelining

3. Overview of Pipeline
Pipelining provides a method for executing multiple instructions at the same time.
Laundry Example
Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold
Washer takes 30 minutes
Dryer takes 40 minutes
“Folder” takes 20 minutes A B C D

4. Sequential Laundry 6 PM 7 8 9 10 11 Midnight 30 40 20 30 40 20 30 40
Time 30 40 20 30 40 20 30 40 20 30 40 20 T a s k O r d e A B C D
Sequential laundry takes 6 hours for 4 loads
If they learned pipelining, how long would laundry take?

5. Pipelined Laundry: Start work ASAP
6 PM 7 8 9 10 11
Midnight
Time 30 40 20 T a s k O r d e A B C D
Pipelined laundry takes 3.5 hours for 4 loads

6. Pipelining Lessons 6 PM 7 8 9 30 40 20 A B C D
Pipelining doesn’t help latency of single task, it helps throughput of entire workload
Pipeline rate limited by slowest pipeline stage
Multiple tasks operating simultaneously using different resources
Potential speedup = Number pipe stages
Unbalanced lengths of pipe stages reduces speedup
Time to “fill” pipeline and time to “drain” it reduces speedup
Stall for Dependences
6 PM 7 8 9
Time 30 40 20 T a s k O r d e A B C D

7. Pipelining Lessons Five stages, one step per stage
IF: Instruction fetch from memory
ID: Instruction decode & register read
EX: Execute operation or calculate address
MEM: Access memory operand
WB: Write result back to register

8. One Way to Break up the Datapath Operations
Ifetch: Instruction Fetch
Fetch the instruction from the Instruction Memory
ID(ID/Reg): Instruction Decode and Registers Fetch
Exec: Calculate the memory address
Mem: Read the data from the Data Memory
Wr: Write the data back to the register file
Cycle 1
Cycle 2
Cycle 3
Cycle 4
Cycle 5
Load
Ifetch
ID/Reg
Exec
Mem Wr

9. Pipelined Execution
On a processor multiple instructions are in various stages at the same time.
Assume each instruction takes five cycles
Time
IFetch ID
Exec
Mem WB
IFetch ID
Exec
Mem WB
IFetch ID
Exec
Mem WB
IFetch ID
Exec
Mem WB
IFetch ID
Exec
Mem WB
Program Flow
IFetch ID
Exec
Mem WB

10. Single Cycle, Multiple Cycle, vs. Pipeline
Clk
Single Cycle Implementation:
Load
Store
Cycle 1
Cycle 2
Waste
Cycle 1
Cycle 2
Cycle 3
Cycle 4
Cycle 5
Cycle 6
Cycle 7
Cycle 8
Cycle 9
Cycle 10
Clk
Multiple Cycle Implementation:
Ifetch ID
Exec
Mem Wr
Load
Store
R-type
Load
Ifetch ID
Exec
Mem Wr
Pipeline Implementation:
Store
R-type

11. Pipeline Performance Assume time for stages is
100ps for register read or write
200ps for other stages
Compare pipelined datapath with single-cycle datapath
Instr
Instr fetch
Register read
ALU op
Memory access
Register write
Total time lw
200ps
100 ps
800ps sw
700ps
R-format
600ps
beq
500ps

12. Ideal speedup is number of stages in the pipeline. Do we achieve this?
Pipelining
Improve performance by increasing instruction throughput
Ideal speedup is number of stages in the pipeline. Do we achieve this?

13. If all stages are balanced
i.e., all take the same time
𝑇𝑖𝑚𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑖𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛𝑠 𝑝𝑖𝑝𝑒𝑙𝑖𝑛𝑒𝑑 = 𝑇𝑖𝑚𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑖𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛𝑠 𝑛𝑜𝑛𝑝𝑖𝑝𝑒𝑙𝑖𝑛𝑒𝑑 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑡𝑎𝑔𝑒𝑠
If not balanced, speedup is less
Speedup due to increased throughput
Latency (time for each instruction) does not decrease

14. Why Pipeline? Suppose Single Cycle Machine Multicycle Machine
100 instructions are executed
The single cycle machine has a cycle time of 45 ns
The multicycle and pipeline machines have cycle times of 10 ns
The multicycle machine has a CPI of 3.6
Single Cycle Machine
45 ns/cycle x 1 CPI x 100 inst = 4500 ns
Multicycle Machine
10 ns/cycle x 3.6 CPI x 100 inst = 3600 ns
Ideal pipelined machine
10 ns/cycle x (1 CPI x 100 inst + 4 cycle drain) = 1040 ns
Ideal pipelined vs. single cycle speedup
4500 ns / 1040 ns = 4.33
What has not yet been considered?

15. Pipelining and ISA Design
MIPS ISA designed for pipelining
All instructions are 32-bits
Easier to fetch and decode in one cycle
c.f. x86: 1- to 17-byte instructions
Few and regular instruction formats
Can decode and read registers in one step
Load/store addressing
Can calculate address in 3rd stage, access memory in 4th stage
Alignment of memory operands
Memory access takes only one cycle

16. MIPS Pipelined Datapath
MEM
Right-to-left flow leads to hazards WB

17. Pipeline registers Need registers between stages
To hold information produced in previous cycle

18. Morgan Kaufmann Publishers
10 November, 2018
Pipeline Operation
Cycle-by-cycle flow of instructions through the pipelined datapath
“Single-clock-cycle” pipeline diagram
Shows pipeline usage in a single cycle
Highlight resources used
c.f. “multi-clock-cycle” diagram
Graph of operation over time
We’ll look at “single-clock-cycle” diagrams for load & store
Chapter 4 — The Processor

19. Morgan Kaufmann Publishers
10 November, 2018
IF for Load, Store, …
Chapter 4 — The Processor

20. Morgan Kaufmann Publishers
10 November, 2018
ID for Load, Store, …
Chapter 4 — The Processor

21. Morgan Kaufmann Publishers
10 November, 2018
EX for Load
Chapter 4 — The Processor

22. Morgan Kaufmann Publishers
10 November, 2018
MEM for Load
Chapter 4 — The Processor

23. Morgan Kaufmann Publishers
10 November, 2018
WB for Load
Wrong register number
Chapter 4 — The Processor

24. Corrected Datapath for Load
Morgan Kaufmann Publishers
10 November, 2018
Corrected Datapath for Load
Chapter 4 — The Processor

25. Morgan Kaufmann Publishers
10 November, 2018
EX for Store
Chapter 4 — The Processor

26. Morgan Kaufmann Publishers
10 November, 2018
MEM for Store
Chapter 4 — The Processor

27. Morgan Kaufmann Publishers
10 November, 2018
WB for Store
Chapter 4 — The Processor

28. Single-Cycle Pipeline Diagram
Morgan Kaufmann Publishers
10 November, 2018
Single-Cycle Pipeline Diagram
State of pipeline in a given cycle
Chapter 4 — The Processor

29. Multi-Cycle Pipeline Diagram
Morgan Kaufmann Publishers
10 November, 2018
Multi-Cycle Pipeline Diagram
Form showing resource usage
Chapter 4 — The Processor

30. Multi-Cycle Pipeline Diagram
Morgan Kaufmann Publishers
10 November, 2018
Multi-Cycle Pipeline Diagram
Traditional form
Chapter 4 — The Processor

31. Graphically Representing Pipelines
Can help with answering questions like:
how many cycles does it take to execute this code?
what is the ALU doing during cycle 4?
use this representation to help understand datapaths

32. Why Pipeline? For Performance!
Time (clock cycles)
Once the pipeline is full, one instruction is completed every cycle, so CPI = 1
ALU IM
Reg DM
Inst 0 I n s t r. O r d e
ALU IM
Reg DM
Inst 1
ALU IM
Reg DM
Inst 2
ALU IM
Reg DM
Inst 3
ALU IM
Reg DM
Inst 4
Time to fill the pipeline

33. MIPS Pipeline Datapath Additions/Mods
State registers between each pipeline stage to isolate them
IF:IFetch
ID:Dec
EX:Execute
MEM:
MemAccess
WB:
WriteBack
Read
Address
Instruction
Memory
Add PC 4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Data 1
Data 2 16 32
ALU
Shift
left 2
Data
IF/ID
Sign
Extend
ID/EX
EX/MEM
MEM/WB
Any info that is needed in a later pipe stage must be passed to that stage via a pipeline register (e.g., need to preserve the destination register address in the pipeline state registers)
Note two exceptions to right-to-left flow
WB that writes the result back into the register file in the middle of the datapath
Selection of the next value of the PC, one input comes from the calculated branch address from the MEM stage
Only later instructions in the pipeline can be influenced by these two REVERSE data movements.
The first one (WB to ID) leads to data hazards.
The second one (MEM to IF) leads to control hazards.
All instructions must update some state in the processor – the register file, the memory, or the PC – so separate pipeline registers are redundant to the state that is updated (not needed).
PC can be thought of as a pipeline register: the one that feeds the IF stage of the pipeline. Unlike all of the other pipeline registers, the PC is part of the visible architecture state – its content must be saved when an exception occurs (the contents of the other pipe registers are discarded).
System Clock

34. Pipelined Control (Simplified)
Morgan Kaufmann Publishers
10 November, 2018
Pipelined Control (Simplified)
Chapter 4 — The Processor

35. Pipeline control
We have 5 stages. What needs to be controlled in each stage?
Instruction Fetch and PC Increment
Instruction Decode / Register Fetch
Execution
Memory Stage
Write Back
How would control be handled in an automobile plant?
a fancy control center telling everyone what to do?
should we use a finite state machine?

36. Morgan Kaufmann Publishers
10 November, 2018
Pipelined Control
Control signals derived from instruction
Pass control signals along just like the data implementation
Chapter 4 — The Processor

37. Morgan Kaufmann Publishers
10 November, 2018
Pipelined Control
Chapter 4 — The Processor

38. MIPS Pipeline Data and Control Paths1
PCSrc
ID/EX
EX/MEM
Control
IF/ID
Add
MEM/WB
Branch
Add 4
RegWrite
Shift
left 2
Instruction
Memory
Read Addr 1
Data
Memory
Register
File
Read
Data 1
Read Addr 2
MemtoReg
Read
Address
ALUSrc PC
Read
Data
Address 1
Write Addr
ALU
Read
Data 2
Write Data
Write Data 1
ALU
cntrl
How many bits wide is each pipeline register?
PC = 32 bits
IF/ID = 64 bits
ID/EX = x = 147
EX/MEM = x3 + 5
= 107
MEM/WB = x2 + 5
= 71
MemWrite
MemRead
Sign
Extend 16 32
ALUOp 1
RegDst