Pattern Programming with the Seeds Framework Workpool pattern Matrix addition and multiplication © 2013 B. Wilkinson/Clayton Ferner SIGCSE 2013 Workshop 31 session2a.ppt Modification date: Feb 18, 2013

Matrix Addition, C = A + B Add corresponding elements of each matrix to form elements of result matrix. Given elements of A as ai,j and elements of B as bi,j, each element of C computed as: Add A B C Easy to parallelize – each processor computes one C element or group of C elements

Workpool Implementation Slave computation Adds one row of A with one row of B to create one row of C (rather than each slave adding single elements) Add A B C

Slaves (one for each row) Return one row of C C A B Send one row of A and B to slave Master Compute node Source/sink Following example 3 x 3 arrays and 3 slaves

MatrixAddModule.java Continues on several sides package edu.uncc.grid.example.workpool; import … public class MatrixAddModule extends Workpool { private static final long serialVersionUID = 1L; int[][] matrixA; int[][] matrixB; int[][] matrixC; public MatrixAddModule() { matrixC = new int[3][3]; } public void initMatrices(){ matrixA = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; matrixB = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; public int getDataCount() { return 3; public void initializeModule(String[] args) { Node.getLog().setLevel(Level.WARNING); MatrixAddModule.java Continues on several sides In this example matrices are 3 x 3 Some initial values Required method. Number of data objects (Slaves)

DataMap d returned are pairs of string key and associated array DiffuseData method public Data DiffuseData(int segment) { int[] rowA = new int[3]; int[] rowB = new int[3]; DataMap<String, int[]> d =new DataMap<String, int[]>(); int k = segment; for (int i=0;i<3;i++) { rowA[i] = matrixA[k][i]; rowB[i] = matrixB[k][i]; } d.put("rowA",rowA); d.put("rowB",rowB); return d; DataMap d returned are pairs of string key and associated array segment variable used to select rows Copy one row of A and one row of B into rowA, rowB to be sent to slaves rowA and rowB put in d DataMap to send to slaves

Compute method public Data Compute(Data data) { int[] rowC = new int[3]; DataMap<String, int[]> input = (DataMap<String,int[]>)data; DataMap<String, int[]> output = new DataMap<String, int[]>(); int[] rowA = (int[]) input.get("rowA"); int[] rowB = (int[]) input.get("rowB"); for (int i=0;i<3;i++) { rowC[i] = rowA[i] + rowB[i]; } output.put("rowC",rowC); return output; Get two rows from data received Add rows Put result row into output with key to be sent back to master

GatherData method Note segment variable and Data from slave public void GatherData(int segment, Data dat) { DataMap<String,int[]> out = (DataMap<String,int[]>) dat; int[] rowC = (int[]) out.get("rowC"); for (int i=0;i<3;i++) { matrixC[segment][i]= rowC[i]; } Get C row sent from slave Place row into result matrix Segment variable associated with Data used to choose correct row

Bootstrap class - RunMatrixAddModule.java package edu.uncc.grid.example.workpool; import … public class RunMatrixAddModule { public static void main (String [] args ) { try { long start = System.currentTimeMillis(); Seeds.start( args[0] ,false); MatrixAddModule m = new MatrixAddModule(); m.initMatrices(); PipeID id = Seeds.startPattern(new Operand ((String[])null,new Anchor (args[1], Types.DataFlowRoll.SINK_SOURCE),m)); Seeds.waitOnPattern(id); m.printResult(); Seeds.stop(); long stop = System.currentTimeMillis(); double time = (double) (stop - start) / 1000.0; System.out.println("Execution time = " + time); … In this example the path to Seeds and local host name are command line arguments

Matrix Multiplication, C = A * B Multiplication of two matrices, A and B, produces matrix C whose elements, ci,j (0 <= i < n, 0 <= j < m), computed as follows: where A is an n x l matrix and B is an l x m matrix.

Parallelizing Matrix Multiplication Assume throughout that matrices square (n x n matrices). Sequential code to compute A x B could simply be for (i = 0; i < n; i++) // for each row of A for (j = 0; j < n; j++) { // for each column of B c[i][j] = 0; for (k = 0; k < n; k++) c[i][j] = c[i][j] + a[i][k] * b[k][j]; } Requires n3 multiplications and n3 additions Sequential time complexity of O(n3). Very easy to parallelize as each result independent

Slaves (one for each element of result) Return one element of C C A Send one row of A and one column of B to slave B Master Compute node Source/sink Following example 3 x 3 arrays and 9 slaves

MatrixAddModule.java Continues on several sides package edu.uncc.grid.example.workpool; import … public class MatrixAddModule extends Workpool { private static final long serialVersionUID = 1L; int[][] matrixA; int[][] matrixB; int[][] matrixC; public MatrixAddModule() { matrixC = new int[3][3]; } public void initMatrices(){ matrixA = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; matrixB = new int[][]{{2,5,8},{3,4,9},{1,5,2}}; public int getDataCount() { return 9; public void initializeModule(String[] args) { Node.getLog().setLevel(Level.WARNING); MatrixAddModule.java Continues on several sides In this example matrices are 3 x 3 Some initial values Required method. Number of data objects (Slaves)

Note on mapping rows and columns to segments Arow Bcol segment 0 0 0 segment 1 0 1 segment 2 0 2 segment 3 1 0 segment 4 1 1 segment 5 1 2 segment 6 2 0 segment 7 2 1 segment 8 2 2 int Arow =segment/3; Int Bcol segment%3;

DataMap d returned are pairs of string key and associated array DiffuseData method public Data DiffuseData(int segment) { int[] rowA = new int[3]; int[] colB = new int[3]; DataMap<String, int[]> d =new DataMap<String, int[]>(); int a=segment/3,b = segment%3 ; for (int i=0;i<3;i++) { rowA[i] = matrixA[a][i]; colB[i] = matrixB[i][b]; } d.put("rowA",rowA); d.put(“colB",colB); return d; DataMap d returned are pairs of string key and associated array segment variable used to select element in A and B Copy one row of A and one column of B into rowA, colB to be sent to slaves rowA and colB put in d DataMap to send to slaves

Compute method public Data Compute(Data data) { int[] rowC = new int[3]; DataMap<String, int[]> input = (DataMap<String,int[]>)data; DataMap<String, Integer> output = new DataMap<String, Integer>(); int[] rowA = (int[]) input.get("rowA"); int[] colB = (int[]) input.get(“colB"); int out = 0; for (int i=0;i<3;i++) { out += rowA[i]*colB[i]; } output.put(“out",out); return output; Get two rows from data received Matrix multiplication, one result Put result into output with key to be sent back to master

GatherData method Note segment variable and Data from slave public void GatherData(int segment, Data dat) { DataMap<String,Integer> out = (DataMap<String,Integer>) dat; int answer = out.get("out"); int a=segment/3, b=segment%3; matrixC[a][b]= answer; } Get result sent from slave* Place element into result matrix Segment variable associated with Data used to choose correct row * Cast from Integer to int not necessary

Bootstrap class - RunMatrixMultiplyModule.java package edu.uncc.grid.example.workpool; import … public class RunMatrixMultiplyModule { public static String localhost = "T5400"; // name of local machine public static String seedslocation = "C:\\seeds_2.0\\pgaf"; public static void main (String [] args ) { try { long start = System.currentTimeMillis(); Seeds.start( seedslocation ,false); MatrixMultiplyModule m = new MatrixMultiplyModule(); m.initMatrices(); PipeID id = Seeds.startPattern(new Operand ((String[])null,new Anchor (localhost, Types.DataFlowRoll.SINK_SOURCE),m)); Seeds.waitOnPattern(id); m.printResult(); Seeds.stop(); long stop = System.currentTimeMillis(); double time = (double) (stop - start) / 1000.0; System.out.println("Execution time = " + time); … In this example, local host and path to Seeds are hardcoded.

Questions