1. ELEC692 VLSI Signal Processing Architecture Lecture 6
Array Processor Structure

2. Introduction
Regular and recursive algorithms are common in DSP applications.
Repetitive apply identical series of pre-determined operations
E.g. matrix operations
Regular architectures based on identical processing elements
Utilizing massive parallel processing and intensive pipelining
Systems with programmable processors – multiprocessor systems
Systems with application-specific processors – array processors
Global clock synchronized array – systolic arrays
Self-timed asynchronous data transfer – wavefront arrays
Issues
How is the array processor design dependent on the algorithm?
How is the algorithm best implemented in the array processors?

3. What is a systolic array
Computational networks with distributed data storage and distributed arrangement of processing elements, so that a high number of operations are carried out simultaneously.
Multiple PEs to maximize processing per memory access
memory
memory
10ns
10ns PE PE PE PE PE
Array processor: 400MOPS
Conventional: 100MOPS (Million Operations Per Second)

4. Characteristics of Array processors
Parallelism
Both data operation and data transfers
Locality
Connection exists only to directly neighbouring PEs
Regularity and modularity
Both computation and communications structures
Processing elements can be simple (e.g. a single addition/multiplication) or complex
Why call systolic array?
Analogy to the circulatory system with its two phases, systolic and diastole
System that are entirely pipelined and have periodic computation and data transfer cycles
Synchronous clocking and control signals

5. Array structure examplesPE PE PE PE
1D array PE PE PE PE PE PE PE PE PE PE PE PE
3D array
2D array

6. Drawback of Array processing
Not all algorithms can be mapped and implemented using systolic array architecture
Only fixed operations and operands to be processed are fixed prior to run-time
Adaptive algorithms in which the particular operations and operands depend on the data to be processed, cannot be used.
Cost in hardware and area is high
Cost in latency

7. Data dependency Express the algorithm in inherent dependency
Single-assignment code and local data dependency
E.g. Matrix multiplication
FOR i:=1 to n Do
For j:=1 to n Do
BEGIN
c(i,j,0) :=0
For k:=1 to n Do
if i=1, then b(1,j,k):=b_in(k,j)
else b(i,j,k):=b(i-1,j,k)
if j=1, then a(i,1,k):=a_in(i,k)
else a(i,j,k):=a(i,j-1,k)
c(i,j,k) := c(i,j,k-1)+a(i,jk)*b(i,j,k);
END;
c_out(I,j):=c(i,j,n);
FOR i:=1 to n Do
For j:=1 to n Do
BEGIN
c(i,j) :=0
For k:=1 to n Do
c(i,j) := c(i,j)+a(i,k)*b(k,j);
END;

8. Data dependency graph (DG)
A graph specifies the data dependencies in an algorithm
E.g. DG for the matrix multiplication

9. DG of matrix-vector multiplication

10. Systolic Array Fundamentals
Systolic architecture are designed by using linear mapping techniques on regular dependency graph (DG).
Regular Dependency Graph: the presence of an edge in a certain direction at any node in the DG represents presence of an edge in the same direction at all nodes in the DG.
DG corresponds to space representation  no time instance is assigned to any computation
Systolic architectures have a space-time representation where each node is mapped to a certain PE and is scheduled at a particular time instance.
Systolic design methodology maps an N-dimensional DG to a lower dimensional systolic architecture

11. Example of DG Space representation for a FIR filter
y(n) = w0x(n)+w1x(n-1)+w2x(n-2)
(0,1)T
x(0)
x(1)
x(2)
x3)
x(4)
x(5)
j’=j
Processor axis w2
(1,0)T w1 w0
y(0)
y(1)
y(2)
y(3)
y(4)
y(5)
(1,-1)T 1 2 3 4 5
Time axis

12. Linear Mapping Methods
algorithm
Design an application-specific array processor architecture for a given algorithm
Satisfy the demands regarding computational performance and throughput
Minimize hardware cost
Regular structure for VLSI implementation
Dependency
graph
Assignment of the operations to processors and points in time (assignment and scheduling)
Signal flow
graph
architecture

13. Estimation of # of PE
# of PE nPE is generally smaller than the # of nodes in the dependency graph nDG.
Important to know both since they specify how many nodes of the DG must be mapped to a PE.
The processing time of a PE = TPE (including transfer time of register).
Within TPE, each PE carries out a total of nOP/PE operations.
The computational rate of the processor array is then given by
Desired throughput = RT,target and the # of operations per sample nOP/SAMPLE, so we have

14. Estimation of # of PE
# of PE can be furthered reduced through pipelining within the PEs. Given np as the # of pipeline stages along the datapath of the PE, then the new T’PE is
Example
Samples of a colour TV signal (27MHz sampling rate) are to be transformatted into 8X8 blocks, and each of the blocks is to be multiplied by an 8X8 matrix
Sampling and matrix coefficents are 8-bits wide
Results of the accumulated product is 19-bit
A PE contains 1 MUL and 1ADD and a register at its output

15. Example of Estimation of # of PE
(2*83 operations for 82 samples)
With intensive pipelining
Assume tL=50ps)

16. Some definitions Projection vector (also called iteration vector)
Two nodes that are displaced by d or multiples of d are executed by the same processor.
Scheduling vector: sT=(s1,s2)
Any node with index I would be executed at time STI
Processor space vector pT=(p1,p2)
Any node with index IT=(i,j) would be exeucted by processor
Hardware Utilization Efficiency, HUE = 1/|sTd|
This is because two tasks executed by the same processor are spaced |sTd| time units apart.

17. Systolic Array Design Methodology
Systolic architectures are designed by selecting different project, processor space and scheduling vectors,
Feasibility constraints
Processor space vector and projection vector must be orthogonal to each other.
Points A and B differ by the projection vector, i.e. IA-IB is same as d, then they must be executed by the same processor, i.e. PTIA = PTIB and
If A and B are mapped to the same processor, then they cannot be executed at the same time, i.e.
Edge mapping: If an edge e exists in the space representation or DG, then an edge pTe is introduced in the systolic array with sTe delays.

18. Array Architecture Design
Step 1: mapping algorithm to DG
Based on the space-time indices in the recursive algorithm
Shift-Invariance (Homogeneity) of DG
Localization of DG: broadcast vs. transmitted data
Step 2: mapping DG to SFG
Processor assignment: a projection method may be applied (project vector d)
Scheduling: a permissible linear schedule may be applied (Schedule vector s)
Preserve the inter-dependence
Nodes on an equitemporal hyperplane should not be projected to the same PE
Step 3: mapping an SFG onto an array processor

19. Example: FIR Filter y(0) y(1) y(2) . D 2D D k D 2D D y(4) y(5) y(6) d
h(4)
y(2) s
h(3) D 2D
y(1) D
h(2)
y(0) D
h(1) 2D D
h(0) n
x(0)
x(1)
x(2)
x(0)
x(1)
x(2)
x(3)
x(4)
x(5)
x(6)
Equitemporal hyperplanes

20. Space time transformation
Space representation or DG can be transformed to a space-time representation by interpreting one of the spatial dimensions as temporal dimension.
For a two-dimensional (2D) DG, the general transformation is described by i’=t=0, j’=pTI and t’=sTI or equivalently
In the space-time representation, the j’ axis represents the processor axis and t’ represents the scheduling time instance.

21. FIR Systolic Array (Design B1)
B1 design is derived by selecting projection vector, processor vector and scheduling vector as follows:
Any node with index IT=(I,j) is mapped to processor
Therefore all nodes on a horizontal line are mapped to the same processor
Any node with index IT=(i,j) is executed at time
Since then
Edge mapping eT
pTe
sTe
Weight (wt(1 0)) 1
Input ( i/p(0 1))
Result (1 -1) -1

22. B1 design Low-level implementation Space-time representation
Input x(n)
Processor
axis D w1
result w0 D w2 D
weight D
Result D
Processor 1
Low-level implementation D
j’=j
Processor
axis
x(0)
x(1)
x(2)
x(3)
x(4) w2
Space-time
representation
of B1 design w1 D w0
y(0)
y(1)
y(2)
y(3)
y(4)
input D
Time axis 1 2 3 4
Block diagram

23. FIR Systolic Array (Design B2)
B2 design is derived by selecting projection vector, processor vector and scheduling vector as follows:
Any node with index IT=(I,j) is mapped to processor
Any node with index IT=(i,j) is executed at time
Since then
Edge mapping eT
pTe
sTe
Weight (wt(1 0)) 1
Input ( i/p(0 1))
Result (1 -1)
Weights move instead of the results as in B1.
Inputs are broadcast.

24. B2 design Low-level implementation
Input x(n) (…x(3),x(2),x(1),x(0)
Processor
axis D D D
result D w0 w1 w2 D D D D
Low-level implementation
Applying space-time transformation
j’=i+j
Processor
axis
x(0)
x(1)
x(2)
x(3)
x(4) D w2 w1 w0
y(3) D
y(2) D
y(1)
y(0)
input D
weight
Time axis t’=I 1 2 3 4
Block diagram
Space-time representation of B2 design

25. FIR Systolic Array (Design F)
F design is derived by selecting projection vector, processor vector and scheduling vector as follows:
Since then
Edge mapping eT
pTe
sTe
Weight (wt(1 0)) 1
Input ( i/p(0 1))
Result (1 -1) -1
Weights fixed in space, input vector moves from left to right with 1 delay;
Output moves from right to left with no delay elements.

26. F design Low-level implementation
Input x(n) D D
Processor
axis w0 D w1 D w2 D
result D
weight
Result D
Low-level implementation D
Applying space-time transformation
j’=j
Processor
axis D
x(0)
x(1)
x(2) D w2 D w1 w0
input
y(0)
y(1)
y(2)
y(3)
Time axis t’=i+j 1 2 3 4
Block diagram
Space-time representation of B2 design

27. Selection of Scheduling Vector
Finding feasible scheduling vectors using scheduling inequalities.
Based on the selected scheduling vector ST, the projection vector d and the processor space vector pT can be found by
Scheduling inequalities:
Consider the dependency relation XY:
We have
Ix and Iy are the indices of node X and Y.
Where Tx is the time to compute node X and Sx, Sy are the scheduling times for nodes X and Y

28. Scheduling Equations Linear Scheduling
Affine scheduling ( A transformation followed by a translation
and we have
Defining edge from XY as ex-y = Iy-Ix, then scheduling inequality for an edge is:
Scheduling vector sT can be obtained by solving these inequalities

29. Scheduling Inequalities
Two steps for select the scheduling vectors:
Capture all the fundamental edges. The reduced dependence graph (RDG) is used to capture the fundamental edges and the regular iterative algorithm (RIA) description of the corresponding problem is used to construct RDGs
Construct the scheduling inequalities and solve them for feasible sT.

30. Regular Iteration Algorithm (RIA)
Standard input RIA form
If the index of the inputs are the same for all equations.
Standard output RIA form
If the output indices, i.e. indices on the left side, are the same
E.g. FIR filter, RIA description is
W(i+1,j) = W(i,j)
X(i,j+1) = X(i,j)
Y(i+1,j-1)=Y(i,j) + W(i+1,j-1)*X(i+1,j-1)
Cannot expressed as standard input RIA form, but can be expressed as standard output RIA form
W(i,j) = W(i-1,j)
X(i,j) = X(i,j-1)
Y(i,j)=Y(i-1,j+1) + W(i,j)*X(i,j)
(0,1)
(1,0) w x
(0,0)
(0,0)
Reduced DG y
(1,-1)

31. Example Tmult = 5, Tadd = 2, Tcom = 1 w x y (1,0) (0,1) (1,-1) (0,0)
We have 5 edges in the RDG and so
Select d=(1,-1) such that sTd=8¹ 0 and select pT = (1,1) so that pTd = 0. HUE = 1/8 eT
pTe
sTe
Weight (wt(1 0)) 1 9
Input ( i/p(0 1))
Result (1 -1) 8
Edge mapping = X D D 8D 8D 8D
For linear scheduling, gx= gy=gw=0 W 9D 9D
We have
One of the solution is s2=1,s1=8+1=9 and sT=(9,1)

32. Matrix-Matrix Multiplication
DG is a three-dimensional (3D) space representation
Linear projection is used to design 2D systolic arrays
Given 2 matrices A and B (e.g. dimension of 2X2) and C= AB,
Standard output RIA form

33. Matrix-Matrix Multiplication Exampleb c
(0,1,0)
(1,0,0)
(0,0,1)
(0,0,0)
Let Tmult-add=1 and Tcom = 0, we have
RDG of the matrix multi. example
Solution 1:
For linear scheduling, ga= gb=gc=0
HUE = 1

34. Solution 1 Edge mapping = i a a a c D j D D beT
pTe
sTe
a(0,1,0)
(0,1) 1
b(1,0,0)
(1,0)
C(0,0,1)
(0,0)
Edge mapping = i a a a c D j D D b
2-dimensional systolic array
By S. Y. Kung c D D D D D D b c D D D D D D b

35. Solution 2 Edge mapping = i a a a c c c j b D D D c D D D D D D b D ceT
pTe
sTe
a(0,1,0)
(0,1) 1
b(1,0,0)
(1,0)
C(0,0,1)
(1,1) i a a a c c c j b D D D c D D D D D D b D c D D D D b D D