Of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Kathryn Mohror Karen L. Karavanic Portland State University Enabling MPI-2 Support in Paradyn.

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Introduction Goal Increase the level of performance tool support for MPI programs on Linux clusters Multiple MPI implementations MPI-1 and MPI-2 Approach Leverage an existing tool Paradyn LAM/MPI, MPICH2 MPI-2 features 1

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn MPI-2 Programming flexibility and possible performance improvements New important features Dynamic process creation, one-sided communication, MPI-I/O, multi-threaded MPI programs, MPI object naming Challenges application programmers performance tool developers 2

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Related Work Available Parallel Performance Tools MPE/Jumpshot, Vampir, SeeWithin/Pro, Paraver, mpiP, Pablo, TAU, KOJAK, Prophesy, SCALEA, DPCL, TAU, Paradyn, SIGMA, DynaProf, Autopilot, KOJAK, PE Benchmarker, DEEP/MPI, Sun One Studio, SGI SpeedShop Tools that Measure MPI Performance on Linux MPE/Jumpshot, KOJAK, Vampir, Paraver, SeeWithin/Pro, mpiP, Pablo, Paradyn, TAU, DEEP/MPI, SCALEA, Sun One Studio Tools with MPI-2 Support TAU, SCALEA, Vampir, Pablo, mpiP, Prophesy, and KOJAK 3

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Whole ProgramCode Default Module File.c Library.so.2 Myprog.cfunction2function1mainMachineNode1Node2proc1proc2proc3 proc4 SyncObject SpinLock Semaphore Barrier Message 19 1 23 1 Resource Hierarchy metrics cpu_time sync_wait msg_bytes_sent Paradyn Performance Measurement 4

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Whole ProgramCode Default Module File.c Library.so.2 Myprog.cfunction2function1mainMachineNode1Node2proc1proc2proc3 proc4 SyncObject SpinLock Semaphore Barrier Message 19 1 23 1 Resource Hierarchy metricsResource name /Code/Myprog.c/function1cpu_time sync_wait msg_bytes_sent Paradyn Performance Measurement 4

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Whole ProgramCode Default Module File.c Library.so.2 Myprog.cfunction2function1mainMachineNode1Node2proc1proc2proc3 proc4 SyncObject SpinLock Semaphore Barrier Message 19 1 23 1 Resource Hierarchy metricsfocus /Code/Myprog.c/function1, /Machine,/Process/proc1, /SyncObject/Message/23 cpu_time sync_wait msg_bytes_sent Paradyn Performance Measurement 4

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Multiple MPI Implementations Extended to support MPICH mpd LAM/MPI MPICH2 Primary considerations Command line arguments MPI Object Identifiers 5

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn MPI Object Representation MPI Object context IDs may be reused X-Y X = MPI implementation given ID Y = counter 6 SyncObject SpinLock Semaphore Barrier Message SyncObject SpinLock Semaphore Barrier Message 0-1 1 0-0 SyncObject SpinLock Semaphore Barrier Message 1 0-0 1 1 MPI_Comm_freeMPI_Comm_create

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn MPI Object Naming User-defined names for MPI objects Facilitates understanding of performance data main(){… MPI_Comm_create(MPI_COMM_WORLD, group, newComm); MPI_Comm_set_name(newComm, “myCommunicator”); … } Set name example Get name example MPI_Send(…){… MPI_Comm_get_name(newComm, &name_of_comm, &length); … } 7 Detect object naming Forward the new name to user interface

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn MPI Object Naming 8 Whole ProgramCode Default Module File.c Library.so.2 Myprog.cfunction2function1mainMachineNode1Node2proc1proc2proc3 proc4 SyncObject SpinLock Semaphore Barrier Message Resource Hierarchy 0-0 1 focus /Code/Myprog.c/function1, /Machine,/Process/proc1, /SyncObject/Message/0-0

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn MPI Object Naming 8 Whole ProgramCode Default Module File.c Library.so.2 Myprog.cfunction2function1mainMachineNode1Node2proc1proc2proc3 proc4 SyncObject SpinLock Semaphore Barrier Message Resource Hierarchy MPI_COMM_WORLD 1 focus /Code/Myprog.c/function1, /Machine,/Process/proc1, /SyncObject/Message/MPI_COMM_WORLD

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Remote Memory Access 2 Process 0 Process 1 Process 2 35 Process 3 Data Transfer Routines MPI_Get() MPI_Put - remote write MPI_Get - remote read 9

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Remote Memory Access 2 Process 0 Process 1 Process 2 35 Process 3 Data Transfer Routines MPI_Put - remote write MPI_Accumulate - remote update MPI_Get - remote read MPI_Accumulate(3,MPI_SUM) 9

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Remote Memory Access 2 Process 0 Process 1 Process 2 3 MPI_Accumulate(3,MPI_SUM) 8 Process 3 Data Transfer Routines MPI_Put - remote write MPI_Accumulate - remote update MPI_Get - remote read 9

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Remote Memory Access MPI_Put - remote write 2 Process 0 Process 1 Process 2 38 Process 3 Data Transfer Routines MPI_Accumulate - remote update User demand Performance improvements and increased programming flexibility Performance tool support desired Performance Tool Developer Interest MPI_Get - remote read 9

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn RMA: Passive Target Synchronization Origin Process A Origin Process B Target Process MPI_Win_lock lock MPI_Put MPI_Win_unlock unlock MPI_Win_lock lock MPI_Get MPI_Win_unlock unlock 10

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn RMA: Passive Target Synchronization Origin Process A Origin Process B Target Process MPI_Win_lock lock MPI_Put MPI_Win_unlock unlock MPI_Win_lock lock MPI_Get MPI_Win_unlock unlock pt_rma_sync_wait 10

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn RMA: Active Target Synchronization Target ProcessOrigin Process A MPI_Win_start MPI_Win_wait MPI_Win_post MPI_Put MPI_Win_complete Origin Process B MPI_Win_start MPI_Get MPI_Win_complete MPI_Win_wait 12

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn RMA: Active Target Synchronization Target ProcessOrigin Process A MPI_Win_start MPI_Win_wait MPI_Win_post MPI_Put MPI_Win_complete Origin Process B MPI_Win_start MPI_Get MPI_Win_complete MPI_Win_wait at_rma_sync_wait 12

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn RMA: Resource Hierarchy Changes Whole ProgramCodeMyprog.cMachine Node1 proc1 SyncObject SpinLock Semaphore BarrierMessage Default Module File.c Library.so.2 function2function1main 1 proc119 Resource Hierarchy Window 13

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Whole ProgramCodeMyprog.cMachine Node1 proc1 SyncObject SpinLock Semaphore BarrierMessage Default Module File.c Library.so.2 function2function1mainWindow MyWindow_1 MyWindow_2 1 proc119 Resource Hierarchy RMA: Resource Hierarchy Changes 13

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Whole ProgramCodeMyprog.cMachine Node1 proc1 SyncObject SpinLock Semaphore BarrierMessage Default Module File.c Library.so.2 function2function1mainWindow MyWindow_1 MyWindow_2 1 proc119 Resource Hierarchy New metrics rma_ops, rma_sync_wait, rma_bytes, … RMA: Resource Hierarchy Changes 13

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Testing Plan PPerfMark - programs with known behavior oned - Using MPI-2, Gropp, Lusk, and Thakur ASCI Purple Benchmark Program: Presta rma Modified version of Paradyn 4.0 Wyeast 48-node cluster of dual SMPs running Linux LAM/MPI 7.0.4 MPICH2 0.96p2 beta 14

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Paradyn Performance Consultant Hypotheses ExcessiveSyncWaitingTime CPUBound Hypothesis : Focus ExcessiveSyncWaitingTime : /Code,/Machine,/SyncObject CPUBound : /Code,/Machine,/SyncObject Resource Hierarchy TopLevelHypothesis Our Representation CodeMachineSyncObject Whole Program 15

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Paradyn Performance Consultant Hypotheses ExcessiveSyncWaitingTime CPUBound Hypothesis : Focus ExcessiveSyncWaitingTime : /Code,/Machine,/SyncObject CPUBound : /Code,/Machine,/SyncObject Resource Hierarchy TopLevelHypothesis Our Representation CodeMachineSyncObject Whole Program CPUBound 15

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Paradyn Performance Consultant Hypotheses ExcessiveSyncWaitingTime CPUBound Hypothesis : Focus ExcessiveSyncWaitingTime : /Code,/Machine,/SyncObject CPUBound : /Code/myProg.c/main,/Machine,/SyncObject CPUBound : /Code,/Machine/Node1,/SyncObject CPUBound : /Code,/Machine/Node2,/SyncObject Resource Hierarchy TopLevelHypothesis CPUBound Node2Node1main Our Representation CodeMachineSyncObject Whole Program 15

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Paradyn Performance Consultant Hypotheses ExcessiveSyncWaitingTime CPUBound Hypothesis : Focus ExcessiveSyncWaitingTime : /Code,/Machine,/SyncObject CPUBound : /Code/myProg.c/main,/Machine,/SyncObject CPUBound : /Code,/Machine/Node1,/SyncObject CPUBound : /Code,/Machine/Node2,/SyncObject Resource Hierarchy TopLevelHypothesis CPUBound Node2Node1main Our Representation CodeMachineSyncObject Whole Program mainNode2 15

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Paradyn Performance Consultant Hypotheses ExcessiveSyncWaitingTime CPUBound Hypothesis : Focus ExcessiveSyncWaitingTime : /Code,/Machine,/SyncObject CPUBound : /Code/myProg.c/main/fun1,/Machine,/SyncObject CPUBound : /Code/myProg.c/main/fun2,/Machine,/SyncObject CPUBound : /Code,/Machine/Node2/proc2,/SyncObject Resource Hierarchy TopLevelHypothesis CPUBound fun1fun2proc2 main Node2Node1main Node2 Our Representation CodeMachineSyncObject Whole Program 15

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Results: Presta rma Throughput Unidir PutUnidir GetBidir PutBidir Get MB/sDiffMB/sDiffMB/sDiffMB/sDiff LAM: Paradyn Presta 9.920.0311.190.0117.890.0419.230.01 9.8911.1817.8519.24 MPICH2: Paradyn Presta 9.930.059.83-0.0616.21-0.0112.21-0.01 9.889.8916.2212.22 18 Per Operation Time Unidir PutUnidir GetBidir PutBidir Get ops/sDiffops/sDiffops/sDiffops/sDiff LAM: Paradyn Presta 103.630.1491.45-0.02112.73-1.65106.47-0.29 103.4991.47114.38106.76 MPICH2: Paradyn Presta 102.46-0.64103.78-0.38126.030.00167.28-0.27 103.10104.16126.03167.55

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Conclusions Future Work Other MPI-2 feature support MPI-I/O dynamic process creation Paradyn Extensions Multiple MPI implementations New MPI object representation One-sided communication MPI Object Naming 19

of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Acknowledgements Many thanks to the Paradyn group for their help and support! This work was supported in part by UC/LLNL contracts B524838 and B539302. 20

Of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Kathryn Mohror Karen L. Karavanic Portland State University Enabling MPI-2 Support in Paradyn.

Similar presentations

Presentation on theme: "Of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Kathryn Mohror Karen L. Karavanic Portland State University Enabling MPI-2 Support in Paradyn."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Kathryn Mohror Karen L. Karavanic Portland State University Enabling MPI-2 Support in Paradyn.

Similar presentations

Presentation on theme: "Of 24©2005 Kathryn Mohror Enabling MPI-2 Support in Paradyn Kathryn Mohror Karen L. Karavanic Portland State University Enabling MPI-2 Support in Paradyn."— Presentation transcript:

Similar presentations

About project

Feedback