1. Pipelining
Chapter 6

2. Introduction to Pipelining
Pipelining is overlapping of tasks to realize improvement in overall performance
Consider 4 sub-tasks making up a major task. Lets consider the example given in your text: wash, dry, iron and fold clothes (W D I F)
Now consider n-students want to do this WDIF operation this weekend.
WDIFWDIFWDIFWDIF
WDIF

3. Instruction Cycle Fetch: Fetch instruction from memory
Read: Read registers while decoding the instructions
Execute: Execute the operation or calculate an address
Access Memory: Read memory
Write: Write result to register
Assume each of the above operation takes clock cycle.
Assume read and write to register happen in different halves of the cycle. Now we can overlap register read and write.

4. Pipelining
Time between instructions in pipelined = time between instructions in non-pipelined / # pipelined stages
We want a balanced set of instructions to realized best performance by pipelining
Lets examine the MIPS instruction pipelining page: 373
How do we design instruction set for pipelining?
MIPS:
instructions of same length
Only few instruction formats
Memory operands only in load and store
Operands must be aligned in the memory

5. Life is not simple It is full of hazards
There are situations in pipelining where the next instruction cannot execute in the following cycle.
These are called hazards and there are three different types.
Structural hazards: instruction fetch and data access of memory
Data hazards:
add $s0,$t0,$t1
sub $t2,$s0,$t3
Solution: data forwarding
Control hazards: branch…delayed branch, rearranging instructions
Lets look at some examples

6. How to address pipeline hazards?
Stalls in the pipeline occur when instructions due to
structural hazards (two instructions needing memory at the same time),
control hazards (branch instruction), and
data hazards (results from an instruction needed as data in another instruction).
Solution 1: Forwarding… need to be made during the design of the datapath
Solution 2: introducing a delay or bubble in the pipeline; this is usually done after load and store; delayed load;
Example:

7. Rendering Code to Avoid Pipeline Stalls
Original code
Rearranged code
A = B + E
C = B + F
lw $t1,0(t0)
lw $t2,4(t0)
add $t3, $t1, $t2
sw $t3, 12($t0)
lw $t4, 8($t0)
add $t5, $t1, $t4
sw $t5,16($t0)
A = B + E
C = B + F
lw $t1,0(t0)
lw $t2,4(t0)
lw $t4, 8($t0)
add $t3, $t1, $t2
sw $t3, 12($t0)
add $t5, $t1, $t4
sw $t5,16($t0)

8. Control Hazards
There are benchmark program that are used for evaluating the performance of the hardware called SPEC benchmarks
SPECint2000 is one of them. According to this benchmark 13% of the instructions executed are branch.
After a branch we a nop to stall; 13% of the time one extra cycle is added to the time.
Also the instructions loaded into the pipeline need to flushed if the branch is taken.
Branch prediction is another solution: based on the prediction you may want to stall or prefetch.

9. Revisit and redesign Datapath
Lets redesign our datapath to allow pipelined execution:
See. Figs., 6.9, 6.10, 6.11…

10. Issues: how to accommodate more than 1 instruction in the datapath?IF ID EX MM WB IF ID EX MM WB IF ID EX MM WB IF ID EX MM WB

11. Add buffer before each stage
IF/ID buffer : 64 bits
ID/EX buffer : 128 bits
EX/MM buffer : 97 bits : 1 for carry/zero
MM/WB buffer: 64 bits
Fig. 6.9 (without control)
Reason out the size of these pipeline registers
How about load register address in a load instruction?
Add 5 more bits to choose the load register; this extra bits will be in ID/EX, EX/MM, MM/WB
See fig. 6.17

12. Pipelined execution instruction
Instructions:
lw $t1,20($t2)
sub $t3, $t4, $t5
add $t6, $t5,$t7
lw $t8,24($t2)
add $t9,$t10,$t11
Lets draw the multi-cycle pipeline diagram of five instructions.
Fig,6.19, 6.20, 6.21
Fig with control line buffers at ID/EX and EX/MM

13. Pipelined control Control gets complex Remember, life is not simple
Consider the sequence given below; lets analyze the data forwarding requirement of these instructions.
sub $t2,$t1,$t3
and $t12, $t2,$t5
or $t13,$t6,$t2
add $t14,$t2,$t2
sw $t15,100($t2)
Fig. 6.28
How to solve this dependency problem? Detect dependency and resolve at the hardware level.