1. Pattern Programming with the Seeds Framework Workpool pattern Matrix addition and multiplication
© B. Wilkinson/Clayton Ferner SIGCSE 2013 Workshop 31 session2a.ppt Modification date: Feb 18, 2013

2. Matrix Addition, C = A + B
Add corresponding elements of each matrix to form elements of result matrix. Given elements of A as ai,j and elements of B as bi,j, each element of C computed as:
Add A B C
Easy to parallelize – each processor computes one C element or group of C elements

3. Workpool Implementation
Slave computation
Adds one row of A with one row of B to create one row of C
(rather than each slave adding single elements)
Add A B C

4. Slaves (one for each row)
Return one row of C C A B
Send one row of A and B to slave
Master
Compute node
Source/sink
Following example 3 x 3 arrays and 3 slaves

5. MatrixAddModule.java Continues on several sides
package edu.uncc.grid.example.workpool; import … public class MatrixAddModule extends Workpool { private static final long serialVersionUID = 1L; int[][] matrixA; int[][] matrixB; int[][] matrixC; public MatrixAddModule() { matrixC = new int[3][3]; } public void initMatrices(){ matrixA = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; matrixB = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; public int getDataCount() { return 3; public void initializeModule(String[] args) { Node.getLog().setLevel(Level.WARNING);
MatrixAddModule.java Continues on several sides
In this example matrices are 3 x 3
Some initial values
Required method. Number of data objects (Slaves)

6. DataMap d returned are pairs of string key and associated array
DiffuseData method
public Data DiffuseData(int segment) { int[] rowA = new int[3]; int[] rowB = new int[3]; DataMap<String, int[]> d =new DataMap<String, int[]>(); int k = segment; for (int i=0;i<3;i++) { rowA[i] = matrixA[k][i]; rowB[i] = matrixB[k][i]; } d.put("rowA",rowA); d.put("rowB",rowB); return d;
DataMap d returned are pairs of string key and associated array
segment variable used to select rows
Copy one row of A and one row of B into rowA, rowB to be sent to slaves
rowA and rowB put in d DataMap to send to slaves

7. Compute method
public Data Compute(Data data) { int[] rowC = new int[3]; DataMap<String, int[]> input = (DataMap<String,int[]>)data; DataMap<String, int[]> output = new DataMap<String, int[]>(); int[] rowA = (int[]) input.get("rowA"); int[] rowB = (int[]) input.get("rowB"); for (int i=0;i<3;i++) { rowC[i] = rowA[i] + rowB[i]; } output.put("rowC",rowC); return output;
Get two rows from data received
Add rows
Put result row into output with key to be sent back to master

8. GatherData method Note segment variable and Data from slave
public void GatherData(int segment, Data dat) { DataMap<String,int[]> out = (DataMap<String,int[]>) dat; int[] rowC = (int[]) out.get("rowC"); for (int i=0;i<3;i++) { matrixC[segment][i]= rowC[i]; }
Get C row sent from slave
Place row into result matrix
Segment variable associated with Data used to choose correct row

9. Bootstrap class - RunMatrixAddModule.java
package edu.uncc.grid.example.workpool; import … public class RunMatrixAddModule { public static void main (String [] args ) { try { long start = System.currentTimeMillis(); Seeds.start( args[0] ,false); MatrixAddModule m = new MatrixAddModule(); m.initMatrices(); PipeID id = Seeds.startPattern(new Operand ((String[])null,new Anchor (args[1], Types.DataFlowRoll.SINK_SOURCE),m)); Seeds.waitOnPattern(id); m.printResult(); Seeds.stop(); long stop = System.currentTimeMillis(); double time = (double) (stop - start) / ; System.out.println("Execution time = " + time); …
In this example the path to Seeds and local host name are command line arguments

10. Matrix Multiplication, C = A * B
Multiplication of two matrices, A and B, produces matrix C
whose elements, ci,j (0 <= i < n, 0 <= j < m), computed as follows:
where A is an n x l matrix and B is an l x m matrix.

11. Parallelizing Matrix Multiplication
Assume throughout that matrices square (n x n matrices).
Sequential code to compute A x B could simply be
for (i = 0; i < n; i++) // for each row of A
for (j = 0; j < n; j++) { // for each column of B
c[i][j] = 0;
for (k = 0; k < n; k++)
c[i][j] = c[i][j] + a[i][k] * b[k][j]; }
Requires n3 multiplications and n3 additions
Sequential time complexity of O(n3).
Very easy to parallelize as each result independent

12. Slaves (one for each element of result)
Return one element of C C A
Send one row of A and one column of B to slave B
Master
Compute node
Source/sink
Following example 3 x 3 arrays and 9 slaves

13. MatrixAddModule.java Continues on several sides
package edu.uncc.grid.example.workpool; import … public class MatrixAddModule extends Workpool { private static final long serialVersionUID = 1L; int[][] matrixA; int[][] matrixB; int[][] matrixC; public MatrixAddModule() { matrixC = new int[3][3]; } public void initMatrices(){ matrixA = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; matrixB = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; public int getDataCount() { return 9; public void initializeModule(String[] args) { Node.getLog().setLevel(Level.WARNING);
MatrixAddModule.java Continues on several sides
In this example matrices are 3 x 3
Some initial values
Required method. Number of data objects (Slaves)

14. Note on mapping rows and columns to segments
Arow Bcol segment segment segment segment segment segment segment segment segment 8 2 2
int Arow =segment/3;
Int Bcol segment%3;

15. DataMap d returned are pairs of string key and associated array
DiffuseData method
public Data DiffuseData(int segment) { int[] rowA = new int[3]; int[] colB = new int[3]; DataMap<String, int[]> d =new DataMap<String, int[]>(); int a=segment/3,b = segment%3 ; for (int i=0;i<3;i++) { rowA[i] = matrixA[a][i]; colB[i] = matrixB[i][b]; } d.put("rowA",rowA); d.put(“colB",colB); return d;
DataMap d returned are pairs of string key and associated array
segment variable used to select element in A and B
Copy one row of A and one column of B into rowA, colB to be sent to slaves
rowA and colB put in d DataMap to send to slaves

16. Compute method
public Data Compute(Data data) { int[] rowC = new int[3]; DataMap<String, int[]> input = (DataMap<String,int[]>)data; DataMap<String, Integer> output = new DataMap<String, Integer>(); int[] rowA = (int[]) input.get("rowA"); int[] colB = (int[]) input.get(“colB"); int out = 0; for (int i=0;i<3;i++) { out += rowA[i]*colB[i]; } output.put(“out",out); return output;
Get two rows from data received
Matrix multiplication, one result
Put result into output with key to be sent back to master

17. GatherData method Note segment variable and Data from slave
public void GatherData(int segment, Data dat) { DataMap<String,Integer> out = (DataMap<String,Integer>) dat; int answer = out.get("out"); int a=segment/3, b=segment%3; matrixC[a][b]= answer; }
Get result sent from slave*
Place element into result matrix
Segment variable associated with Data used to choose correct row
* Cast from Integer to int not necessary

18. Bootstrap class - RunMatrixMultiplyModule.java
package edu.uncc.grid.example.workpool; import … public class RunMatrixMultiplyModule { public static String localhost = "T5400"; // name of local machine public static String seedslocation = "C:\\seeds_2.0\\pgaf"; public static void main (String [] args ) { try { long start = System.currentTimeMillis(); Seeds.start( seedslocation ,false); MatrixMultiplyModule m = new MatrixMultiplyModule(); m.initMatrices(); PipeID id = Seeds.startPattern(new Operand ((String[])null,new Anchor (localhost, Types.DataFlowRoll.SINK_SOURCE),m)); Seeds.waitOnPattern(id); m.printResult(); Seeds.stop(); long stop = System.currentTimeMillis(); double time = (double) (stop - start) / ; System.out.println("Execution time = " + time); …
In this example, local host and path to Seeds are hardcoded.

19. Questions