Pipeline Design Problems

Job Sequencing and Collision Prevention for the Design of Static Pipeline

Job Sequencing and Collision Prevention Consider reservation table given below at t=0   1 2 3 4 5 Sa A Sb Sc

Job Sequencing and Collision Prevention Consider next initiation made at t=1 The second initiation easily fits in the reservation table   1 2 3 4 5 6 7 Sa A1 A2 Sb Sc

Job Sequencing and Collision Prevention Now consider the case when first initiation is made at t = 0 and second at t = 2. Here both markings A1 and A2 falls in the same stage time units and is called collision and it must be avoided   1 2 3 4 5 6 7 Sa A1 A2 Sb A1A2 Sc

Terminologies

Terminologies Latency: Time difference between two initiations in units of clock period Forbidden Latency: Latencies resulting in collision Forbidden Latency Set: Set of all forbidden latencies

General Method of finding Latency Considering all initiations: Forbidden Latencies are 2 and 5   1 2 3 4 5 6 7 8 9 10 Sa A1 A2 A3 A4 A5 A6A1 A6 Sb A1A3 A2A4 A3A5 A4A6 Sc

Shortcut Method of finding Latency Forbidden Latency Set = {0,5} U {0,2} U {0,2} = { 0, 2, 5 }

Terminologies Initiation Sequence : Sequence of time units at which initiation can be made without causing collision Example : { 0,1,3,4 ….} Latency Sequence : Sequence of latencies between successive initiations Example : { 1,2,1….} For a RT, number of valid initiations and latencies are infinite

Terminologies Initiation Rate : The average number of initiations done per unit time It is a positive fraction and maximum value of IR is 1 Average Latency : The average of latency of a given latency sequence IR = 1/AL

Terminologies Latency Cycle: Among the infinite possible latency sequence, the periodic ones are significant. E.g. { 2, 3, 4, 2, 3, 4,… } The subsequence that repeats itself is called latency cycle. E.g. {2, 3, 4}

Terminologies Period of cycle: The sum of latencies in a latency cycle (2+3+4=9) Average Latency: The average taken over its latency cycle (AL=9/3=3) To design a pipeline, we need a control strategy that maximize the throughput (no. of results per unit time) Maximizing throughput is minimizing AL

Terminologies Control Strategy Initiate pipeline as specified by latency sequence. Latency sequence which is aperiodic in nature is impossible to design Thus design problem is arriving at a latency cycle having minimal average latency.

Terminologies Stage Utilization Factor (SUF): SUF of a particular stage is the fraction of time units the stage used while following a latency sequence. Example: Consider 5 initiations of function A as below   1 2 3 4 5 6 7 8 9 10 11 12 13 Sa A1 A2 A3 A4 A5 Sb Sc

Terminologies SUF of stage Sa is number of markings present along Sa divided by the time interval over which marking is counted. SUF(Sa) = SUF(Sb) = SUF(Sc) = 10/14

Terminologies Let SU(i) be the stage utilization factor of stage i Let N(i) be no. of markings against stage i in the reservation table Suppose we initiate pipeline with initiation rate (IR), then SU(i) is given by

SUF

Terminologies Minimum Average Latency (MAL) Thus SU(i) = IR x N(i) N(i) ≤ 1/IR  N(i) ≤ AL Therefore

State Diagram Suppose a pipeline is initially empty and make an initiation at t = 0. Now we need to check whether an initiation possible at t=i for i > 0. bi is used to note possibility of initiation bi = 1  initiation not possible bi = 0  initiation possible

State Diagram bi 1 0 1 0 0 1

State Diagram The above binary representation (binary vector) is called collision vector(CV) The collision vector obtained made at first initiation is called initial collision vector(ICV) ICVA = (101001) The graphical representation of states (CVs) that a pipeline can reach and the relation is given by state diagram

State Diagram States (CVs) are denoted by nodes The node representing CVt-1 is connected to CVt by a directed graph from CVt-1 to CVt and similarly for CVt* with a * on arc

Procedure to draw state diagram Start with ICV For each unprocessed state, say CVt-1, do as follows: Find CVt from CVt-1 by the following steps Left shift CVt-1 by 1 bit Drop the leftmost bit Append the bit 0 at the right-hand end

Procedure to draw state diagram If the 0th bit of CVt is 0, then obtain CV* by logically ORing CVt with ICV. Make a new node for CVt and join with CVt-1 with an arc if the state CVt does not already exist. If CV* exists, repeat step (c), but mark the arc with a *.

State Diagram 1 0 1 0 0 1

State Diagram 1 0 1 0 0 1 0 1 0 0 1 0 Left Shift

State Diagram 1 0 1 0 0 1 0 1 0 0 1 0 Zero  CV* exists

State Diagram 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * ICV – 101001 OR CVi – 010010 CV* 111011

State Diagram 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 No CV* 1 1 0 1 1 0 Left Shift

State Diagram No CV* Left Shift 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * ICV – 101001 OR CVi – 001000 CV* 101001 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 Zero  CV* exists 1 0 1 1 0 0

State Diagram 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * ICV – 101001 CVi – 010000 CV* 111001 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 1 0 0 1 1 1 0 0 1 Zero  CV* exists

1 0 0 1 0 0 ICV – 101001 CVi – 011000 CV* 111001 0 1 0 0 1 0 * 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 0 0 1 1 1 1 0 1 1 1 1 0 1 1 0 1 0 1 1 0 0 0 1 1 0 0 0 Zero  CV* exists

1 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 No CV* *

* 0 1 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 1 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 0 No CV*

1 1 0 0 0 0 * 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 0 0 1 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 *

* 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 1 1 0 0 0 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 1 0 1 1 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 1 1 0 0 0 0 1 0 0 0 0 0

1 0 0 1 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0

State Diagram From the above diagram, closed loops can be identified as latency cycles. To find the latency corresponding to a loop, start with any initial * count the number of states before we encounter another * and reach back to initial *.

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (3)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (1,3,3)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (4,3)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (1,6)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (1,7)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (4)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (6)

1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 1 1 * 1 0 0 1 0 0 1 1 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 Latency = (7)

State Diagram The state with all zeros has a self-loop which corresponds to empty pipeline and it is possible to wait for indefinite number of latency cycles of the form (1,8), (1,9),(1,10) etc. Simple Cycle: latency cycle in which each state is encountered only once. Complex Cycle: consists of more than one simple cycle in it. It is enough to look for simple cycles

State Diagram In the above example, the cycle that offers MAL is (1, 3, 3) From A cycle arrived so is called greedy cycle, which minimize latency between successive initiation

Modified State Diagram The state diagram becomes cumbersome for longer ICVs. In modified state diagrams, we represent only states obtained of initiations.

Modified State Diagram The procedure is as follows: Start with the ICV For each unprocessed state, For each bit I in the CVi which is 0, do the following: Shift CVi left by i bits Drop i leftmost bits

Modified State Diagram Append zeros to right Logically OR with ICV If step(d) results in a new state then form a new node for this state and join it with node of CVi by an arc with a marking i. Join this new node with node of ICV with an arc having the marking ≥ d (length of ICV)

Modified State Diagram 1 0 1 0 0 1

Modified State Diagram 1 0 1 0 0 1 1 1 1 0 1 1 ICV – 101001 OR CVi – 010010 CV* 111011 i =1 1

Modified State Diagram 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 1

Modified State Diagram 1 0 1 0 0 1 1 1 1 0 1 1 i = 3 ≥6 1 ICV – 101001 OR CVi – 001000 CV* 101001

Modified State Diagram 3 1 0 1 0 0 1 1 1 1 0 1 1 i = 3 ≥6 1

Modified State Diagram 3 1 0 1 0 0 1 1 1 1 0 1 1 i = 4 ≥6 1 ICV – 101001 OR CVi – 010000 CV* 111001

Modified State Diagram 3 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 4 1 1 1 1 0 0 1 ICV – 101001 OR CVi – 010000 CV* 111001

Modified State Diagram 3 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 1 1 1 1 0 0 1

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 1 1 1 1 0 0 1

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 1 1 1 1 0 0 1 i = 3 ICV – 101001 OR CVi – 011000 CV* 111001

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 1 3 1 1 1 0 0 1

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 1 3 1 1 1 0 0 1 i = 3 ICV – 101001 OR CVi – 001000 CV* 101001

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 3 1 3 1 1 1 0 0 1

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 3 1 3 1 1 1 0 0 1 i = 4 ICV – 101001 OR CVi – 010000 CV* 111001

Modified State Diagram 3 ≥6 1 0 1 0 0 1 1 1 1 0 1 1 ≥6 ≥6 4 3 1 3 1 1 1 0 0 1 4

Dynamic Pipeline and Reconfigurability Two methods to improve the throughput of dynamic pipeline: Insertion of non-compute delays Use of Internal Buffers

End