1 Friday, November 03, 2006 “The greatest obstacle to discovery is not ignorance, but the illusion of knowledge.” -D. Boorstin

2 Solving a System of Linear Equations §This is written as Ax = b, where A is an n x n matrix with A[i, j] = a i,j, b is an n x 1 vector [ b 0, b 1, …, b n ] T, and x is the solution.

3 Solving a System of Linear Equations We write this as: Ux = y. A commonly used method for transforming a given matrix into an upper-triangular matrix is Gaussian Elimination.

4 A 00 A 01 A 02 A 03 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

5 A 00 A 01 A 02 A 03 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

6 1A 01 / A 00 A 02 / A 00 A 03 / A 00 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

7 1A 01 / A 00 A 02 / A 00 A 03 / A 00 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

8 1A 01 / A 00 A 02 / A 00 A 03 / A 00 0A` 11 A` 12 A` 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

9 1A 01 / A 00 A 02 / A 00 A 03 / A 00 0A` 11 A` 12 A` 13 0A` 21 A` 22 A` 23 A 30 A 31 A 32 A 33

10 1A 01 / A 00 A 02 / A 00 A 03 / A 00 0A` 11 A` 12 A` 13 0A` 21 A` 22 A` 23 0A` 31 A` 32 A` 33 k+1 to n-1 1 to 3

11 1A 01 / A 00 A 02 / A 00 A 03 / A 00 01A` 12 A` 13 00A` 22 A` 23 00A` 32 A` 33 k+1 to n-1 2 to 3

12 1A 01 / A 00 A 02 / A 00 A 03 / A 00 01A` 12 A` A` A` 33 k+1 to n-1 3 to 3

13

14 Gaussian Elimination Serial Gaussian Elimination

15 Gaussian Elimination §What are the opportunities of parallelization?

16 Gaussian Elimination §1-D partitioning §Pi stores elements A[i,j] for 0<=j<=n

17 n Division Step

18

19 Elimination Step

20 Overall parallel runtime

21 §Idle processes in previous algorithm?

22 §Pipelined communication.

23 A 00 A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=0 has started

24 1A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

25 1A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

26 1A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

27 1A 01 A 02 A 03 A 04 0A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=1 has started

28 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 0A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

29 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 0A 21 A 22 A 23 A 24 0A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Iteration k=0 has ended Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

30 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 0A 21 A 22 A 23 A 24 0A 31 A 32 A 33 A 34 0A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

31 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 00A 22 A 23 A 24 0A 31 A 32 A 33 A 34 0A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=2 has started

32 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A 24 00A 32 A 33 A 34 0A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=1 has ended

33 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A 24 00A 32 A 33 A 34 00A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

34 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A 24 00A 32 A 33 A 34 00A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

35 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A A 33 A 34 00A 42 A 43 A 44 Division step k=3 Elimination step k=3 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=3 has started

36 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A A A 43 A 44 Division step k=3 Elimination step k=3 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

37 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A A A 43 A 44 Division step k=3 Elimination step k=3 Division step k=4 Elimination step k=4 Division step k=2 Elimination step k=2 Iteration k=3 has ended

38 1A 01 A 02 A 03 A 04 01A 12 A 13 A A 23 A A A 44 Division step k=3 Elimination step k=3 Division step k=4 Elimination step k=4 Division step k=2 Elimination step k=2 Iteration k=4 has started

39 1-D partitioning with less than n processes

40

41

42 §2-D partitioning

43

44

45

46

47 §Analysis of nxn 2D partitioning on nxn processors

48

49

50

51