1. 1 Friday, November 03, 2006 “The greatest obstacle to discovery is not ignorance, but the illusion of knowledge.” -D. Boorstin

2. 2 Solving a System of Linear Equations §This is written as Ax = b, where A is an n x n matrix with A[i, j] = a i,j, b is an n x 1 vector [ b 0, b 1, …, b n ] T, and x is the solution.

3. 3 Solving a System of Linear Equations We write this as: Ux = y. A commonly used method for transforming a given matrix into an upper-triangular matrix is Gaussian Elimination.

4. 4 A 00 A 01 A 02 A 03 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

5. 5 A 00 A 01 A 02 A 03 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

6. 6 1A 01 / A 00 A 02 / A 00 A 03 / A 00 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

7. 7 1A 01 / A 00 A 02 / A 00 A 03 / A 00 A 10 A 11 A 12 A 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

8. 8 1A 01 / A 00 A 02 / A 00 A 03 / A 00 0A` 11 A` 12 A` 13 A 20 A 21 A 22 A 23 A 30 A 31 A 32 A 33

9. 9 1A 01 / A 00 A 02 / A 00 A 03 / A 00 0A` 11 A` 12 A` 13 0A` 21 A` 22 A` 23 A 30 A 31 A 32 A 33

10. 10 1A 01 / A 00 A 02 / A 00 A 03 / A 00 0A` 11 A` 12 A` 13 0A` 21 A` 22 A` 23 0A` 31 A` 32 A` 33 k+1 to n-1 1 to 3

11. 11 1A 01 / A 00 A 02 / A 00 A 03 / A 00 01A` 12 A` 13 00A` 22 A` 23 00A` 32 A` 33 k+1 to n-1 2 to 3

12. 12 1A 01 / A 00 A 02 / A 00 A 03 / A 00 01A` 12 A` 13 001A` 23 000A` 33 k+1 to n-1 3 to 3

13. 13

14. 14 Gaussian Elimination Serial Gaussian Elimination

15. 15 Gaussian Elimination §What are the opportunities of parallelization?

16. 16 Gaussian Elimination §1-D partitioning §Pi stores elements A[i,j] for 0<=j<=n

17. 17 n Division Step

18. 18

19. 19 Elimination Step

20. 20 Overall parallel runtime

21. 21 §Idle processes in previous algorithm?

22. 22 §Pipelined communication.

23. 23 A 00 A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=0 has started

24. 24 1A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

25. 25 1A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

26. 26 1A 01 A 02 A 03 A 04 A 10 A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

27. 27 1A 01 A 02 A 03 A 04 0A 11 A 12 A 13 A 14 A 20 A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=1 has started

28. 28 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 0A 21 A 22 A 23 A 24 A 30 A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

29. 29 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 0A 21 A 22 A 23 A 24 0A 31 A 32 A 33 A 34 A 40 A 41 A 42 A 43 A 44 Iteration k=0 has ended Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

30. 30 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 0A 21 A 22 A 23 A 24 0A 31 A 32 A 33 A 34 0A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

31. 31 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 00A 22 A 23 A 24 0A 31 A 32 A 33 A 34 0A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=2 has started

32. 32 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 00A 32 A 33 A 34 0A 41 A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=1 has ended

33. 33 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 00A 32 A 33 A 34 00A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

34. 34 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 00A 32 A 33 A 34 00A 42 A 43 A 44 Division step k=0 Elimination step k=0 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

35. 35 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 000A 33 A 34 00A 42 A 43 A 44 Division step k=3 Elimination step k=3 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2 Iteration k=3 has started

36. 36 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 0001A 34 000A 43 A 44 Division step k=3 Elimination step k=3 Division step k=1 Elimination step k=1 Division step k=2 Elimination step k=2

37. 37 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 0001A 34 000A 43 A 44 Division step k=3 Elimination step k=3 Division step k=4 Elimination step k=4 Division step k=2 Elimination step k=2 Iteration k=3 has ended

38. 38 1A 01 A 02 A 03 A 04 01A 12 A 13 A 14 001A 23 A 24 0001A 34 0000A 44 Division step k=3 Elimination step k=3 Division step k=4 Elimination step k=4 Division step k=2 Elimination step k=2 Iteration k=4 has started

39. 39 1-D partitioning with less than n processes

40. 40

41. 41

42. 42 §2-D partitioning

43. 43

44. 44

45. 45

46. 46

47. 47 §Analysis of nxn 2D partitioning on nxn processors

48. 48

49. 49

50. 50

51. 51