1. Comparative Study of Parallel Performance Visualization Tools By J. Ramphis Castro December 4, 2002

2. Presentation Overview Introduction Introduction Evaluation Criteria Evaluation Criteria Aims Aims ParaGraph ParaGraph VAmpir VAmpir Conclusions Conclusions

3. Introduction The reasons for poor performance of parallel message-passing codes can be varied and complex and users need to be able to understand and correct performance problems. The reasons for poor performance of parallel message-passing codes can be varied and complex and users need to be able to understand and correct performance problems. Performance tools can help by monitoring a program's execution and producing performance data that can be analyzed to locate and understand areas of poor performance. Performance tools can help by monitoring a program's execution and producing performance data that can be analyzed to locate and understand areas of poor performance.

4. Introduction (cont) There are many performance visualization tools available to the parallel programming community. The purpose of this review is to analyze the distinctive features that separate them from each other. There are many performance visualization tools available to the parallel programming community. The purpose of this review is to analyze the distinctive features that separate them from each other.

5. Evaluation Criteria There must always be some guidelines with which to separate and gauge the value of a given piece of software. Therefore, we used numerous review standards in order to accurately gauge the individual software’s value. There must always be some guidelines with which to separate and gauge the value of a given piece of software. Therefore, we used numerous review standards in order to accurately gauge the individual software’s value.

6. Usability Intuitive easy-to-use interface Intuitive easy-to-use interface Adequate documentation and support Adequate documentation and support On-line help and man pages On-line help and man pages

7. Portability Amount of platforms it can be used on Amount of platforms it can be used on Ease of building for a new host Ease of building for a new host MPI implementations and languages it could handle MPI implementations and languages it could handle

8. Versatility Ability to analyze performance data in different ways and to display performance information using different views Ability to analyze performance data in different ways and to display performance information using different views Ability for new analysis methods and views to be added easily Ability for new analysis methods and views to be added easily

9. Scalability Ability to handle large numbers of processes Ability to handle large numbers of processes Ability large or long-running programs Ability large or long-running programs Acceptable overhead increases Acceptable overhead increases

10. Maturity Handles error conditions without crashing Handles error conditions without crashing Few or no penalties by incurring in mistakes while using the program Few or no penalties by incurring in mistakes while using the program Level of support Level of support Size of user base Size of user base

11. Automated Instrumentation and Monitoring System (AIMS) AIMS is a software toolkit for measurement and analysis of Fortran 77 and C message-passing programs written using the NX, PVM, or MPI communication libraries. AIMS is a software toolkit for measurement and analysis of Fortran 77 and C message-passing programs written using the NX, PVM, or MPI communication libraries.

12. Usability Software distribution included readme files and man pages Software distribution included readme files and man pages User guide available from main web page User guide available from main web page User interface was functional, not aesthetically pleasing User interface was functional, not aesthetically pleasing

15. Portability Platforms (HP, SGI, IBM, and SUN) Platforms (HP, SGI, IBM, and SUN) University of Tennessee is porting AIMS to DoD MSRC University of Tennessee is porting AIMS to DoD MSRC An ASCI effort at LANL is porting AIMS to ASCI An ASCI effort at LANL is porting AIMS to ASCI Data files are in ASCII and thus portable Data files are in ASCII and thus portable AIMS supporting HPF is being developed jointly with the Portland Group AIMS supporting HPF is being developed jointly with the Portland Group Developers of AIMS are working on a port for the SGI Origin 2000 Developers of AIMS are working on a port for the SGI Origin 2000

16. Versatility Instrumentation done automatically (xinstrument) Instrumentation done automatically (xinstrument) There are a few display types: VK, tally, MK, and Xisk. (provide different display choices) There are a few display types: VK, tally, MK, and Xisk. (provide different display choices)

17. Scalability PC is the utility name that attempts to remove overhead PC is the utility name that attempts to remove overhead No data available on the tool’s impact on overhead No data available on the tool’s impact on overhead

18. Maturity From what information we could find about AIMS support and development we can conclude that it is still a relatively young program and counts on minimal developer support From what information we could find about AIMS support and development we can conclude that it is still a relatively young program and counts on minimal developer support There a numerous projects to port AIMS to different platforms There a numerous projects to port AIMS to different platforms

19. Verdict on AIMS Although there are a few research groups trying to improve upon its availability to different platforms and its robustness, AIMS suffers from reliability issues. Although there are a few research groups trying to improve upon its availability to different platforms and its robustness, AIMS suffers from reliability issues. Has good features for all around visualization. Has good features for all around visualization.

20. ParaGraph This tool was created for the purpose of providing a detailed, dynamic, graphical animation of the behavior of message- passing parallel programs and graphical summaries of their performance. This tool was created for the purpose of providing a detailed, dynamic, graphical animation of the behavior of message- passing parallel programs and graphical summaries of their performance.

21. Usability Most features can be accessed through “Point and Click” operations Most features can be accessed through “Point and Click” operations Easy manipulation of the trace file in use Easy manipulation of the trace file in use Offers intuitive appeal and aesthetic quality of its displays Offers intuitive appeal and aesthetic quality of its displays Vast amount of documentation available from the developers Vast amount of documentation available from the developers

22. Portability Based on X Windows Based on X Windows Is available for most unix based systems Is available for most unix based systems Its availability to different platforms rivals even the most accomplished of commercial tools Its availability to different platforms rivals even the most accomplished of commercial tools

23. Versatility Three categories - utilization, communication, and task information Three categories - utilization, communication, and task information 25 different displays in its current form 25 different displays in its current form Allows for users to create their own visualization windows Allows for users to create their own visualization windows

24. Scalability Due to its association and dependency with PICL, its overhead is basically a function of the frequency and volume of communication traffic Due to its association and dependency with PICL, its overhead is basically a function of the frequency and volume of communication traffic Visualization still provides accurate and useful information Visualization still provides accurate and useful information

25. Maturity Fairly mature tool Fairly mature tool Minor bugs (“features”) Minor bugs (“features”) PG has a display limitation of how many processors can be displayed at one time (1024) PG has a display limitation of how many processors can be displayed at one time (1024)

26. Verdict on PG Regardless of its few setbacks and limitations, PG is a full featured and powerful visualization tool for small to medium sized clusters. Its ease of use, display features, developer support, open-source nature, and its huge community support (which allows for numerous improvements over a shorter period of time) it is the best visualization tool available freely over the internet short of developing one yourself. Regardless of its few setbacks and limitations, PG is a full featured and powerful visualization tool for small to medium sized clusters. Its ease of use, display features, developer support, open-source nature, and its huge community support (which allows for numerous improvements over a shorter period of time) it is the best visualization tool available freely over the internet short of developing one yourself.

27. Visualization and Analysis of MPI Programs (Vampir) VAMPIR is a graphical tool for analyzing the performance and message passing characteristics of parallel programs that use the MPI and/or SHMEM communication libraries. It is the top commercial tool available today and is developed and supported by Pallas. VAMPIR is a graphical tool for analyzing the performance and message passing characteristics of parallel programs that use the MPI and/or SHMEM communication libraries. It is the top commercial tool available today and is developed and supported by Pallas.

28. Usability On-the-fly segment zoom On-the-fly segment zoom Built-in snapshot capabilities Built-in snapshot capabilities Graph to source code linking Graph to source code linking Shortcuts to most of the commands and displays Shortcuts to most of the commands and displays “Point and Click” interface “Point and Click” interface

31. Portability Architectures supported is simply impressive Architectures supported is simply impressive Supported on T3E, IBM SP2, SGI PowerChallenge, and Sun Solaris (just to name a few) Supported on T3E, IBM SP2, SGI PowerChallenge, and Sun Solaris (just to name a few) Only limited by its demand on any specific platform or research group Only limited by its demand on any specific platform or research group

32. Versatility The amount of displays available is just staggering (if you need it, they probably have it or are in the process of making it) The amount of displays available is just staggering (if you need it, they probably have it or are in the process of making it)

33. Scalability Causes the least amount of disturbance on the program behavior out of all the tools (its effects are more noticeable on large clusters) Causes the least amount of disturbance on the program behavior out of all the tools (its effects are more noticeable on large clusters) Logistic curve overhead Logistic curve overhead

34. Maturity Monthly fee, they can also provide 24/7 customer service and support Monthly fee, they can also provide 24/7 customer service and support Has new releases or patches almost every month Has new releases or patches almost every month Robustness is one of its best qualities (i.e.- systems without a globally consistent clock, VAMPIRtrace automatically corrects clock offset and skew so that timestamps in the merged trace file are consistent) Robustness is one of its best qualities (i.e.- systems without a globally consistent clock, VAMPIRtrace automatically corrects clock offset and skew so that timestamps in the merged trace file are consistent) Cannot benefit from their user base efforts to improve vampir at a faster rate Cannot benefit from their user base efforts to improve vampir at a faster rate

35. Verdict on Vampir If you have the money required to purchase and maintain such a product, vampir is the best visualization tool available on the market today. It has the most features, easiest and most intuitive UI, and the largest support infrastructure (funded by its customers). If you have the money required to purchase and maintain such a product, vampir is the best visualization tool available on the market today. It has the most features, easiest and most intuitive UI, and the largest support infrastructure (funded by its customers). Works best for very large cluster, where savings in overhead are more noticeable Works best for very large cluster, where savings in overhead are more noticeable Displays contain a lot of information and legends are very informative Displays contain a lot of information and legends are very informative Visualization features are easy to use and understand Visualization features are easy to use and understand

36. Overall Conclusion AIMS seemed like a very promising tool, but its lack of support, available platforms and features is very disappointing AIMS seemed like a very promising tool, but its lack of support, available platforms and features is very disappointing Vampir has the most features, support and platform availability out of all the tools. Its display flexibility and the sheer volume of data it can display accurately on screen at any given time, make it the perfect tool for very large cluster of computers. Vampir has the most features, support and platform availability out of all the tools. Its display flexibility and the sheer volume of data it can display accurately on screen at any given time, make it the perfect tool for very large cluster of computers.

37. Overall Conclusion (cont) ParaGraph is the clear winner. Nothing more can be said of a FREE tool that rivals Vampir in features, robustness and versatility whose open-source nature and design structure allow it to be a dynamic and highly adaptive tool for today’s fast paced research market. ParaGraph is the clear winner. Nothing more can be said of a FREE tool that rivals Vampir in features, robustness and versatility whose open-source nature and design structure allow it to be a dynamic and highly adaptive tool for today’s fast paced research market.