1. Building PetaScale Applications and Tools on the TeraGrid Workshop December 11-12, 2007 Scott Lathrop and Sergiu Sanielevici

2. Workshop Activities Attendees: ~ 80 participants; 19 peta-apps and SDCI awards; TeraGrid staff; NSF Pre-workshop survey to identify key topics Discussions of goals, needs, and challenges Recommendations for action Post-workshop survey You have draft of workshop report

3. Algorithms and Techniques Visualization Parallel compilers/languages Parallel solvers and AMR Multi-core programming Workflow Performance tools Communications overhead Parallel I/O Discussion Topics

4. Recommendations 1) Support a Forum for exchanging experiences among petascale efforts - TG task –Support wiki for information exchange - TG task –Utilize TeraGrid ’08 to bring projects together to share findings - TG task –Regularly solicit needs and findings from peta-apps and SDCI projects - TG task –Invite all PetaApps and SDCI projects to participate in a joint scaling paper - Community Task - proposed –Facilitate scientific mentoring to support petascale development and dissemination among the community - TG and Community Task - near term

5. Recommendations 2) Provide a repository of petascale info - TG task –Track 2 and 1 detailed tech. specifications - TG task –Mechanisms for making community contributed readers, clients, etc. widely available - TG task –PetaApps algorithmic needs - Community Task - started –Collect performance data, anecdotes, and community wisdom - TG and Community Task - will take time –Libraries, tools, workflow tools - Characterize advantages/strengths to help users select tools - Community Task - will take time –Exemplars, case studies, best practices, etc. - TG and Community Task - will take time –Create “workflow ready” indicator list (e.g. 100K files, 100K jobs, data transfer, visualization); notify projects when workflow tech may be appropriate - TG and Community Task - will take time

6. Recommendations 3) Enact policies that support advancements in petascale computing - TG Tasks –Improve allocations process - TG Task - evolving –Develop special allocation and job scheduling policy for debugging massively parallel applications - TG task »Sufficient access to test and verify scaling to progressively larger Track 2 and 1 machines –Provide traces (e.g. disk failures to understand failing patterns and to develop proactive schemes) - TG task –Single interface - unified login, password, no certificates - TG task - will take time –Transparent single file system - TG task - long-term –Address scheduling capabilities - TG task - long-term »Dynamic processor allocation for variable-load runs »Dynamic and predictive load balancing and repartitioning

7. Recommendations 4) Provide petascale support services to advance petascale applications - TG tasks –Provide consulting and mentoring - TG task »Review ASTA support requests and recommend projects that would benefit from the emerging technology »Scientific expertise/support/mentoring to suggest algorithm alternatives and/or adjustments; tools; workflows –Expand training efforts - TG task »HPC University training to get new people involved »Provide expert guidance on effective use of petascale architectures, tools, workflow –Provide petascale related documentation and guides - TG task »How to compile, link, submit, and monitor jobs »Example application readings and problem solutions to show advantages of various petascale tools, workflows »Provide non-technical information to potential petascale users as to advantages of petascale technology

8. Recommendations 5) Identify and support key software developments –Development of scalable components: subject to Usability*Reusability=constant. - Community Task - will take time –Identify kinds of tools needed to solve current problems - Community Task - will take time »download data »push data back and forth »calculate and move images –Conduct verification and validation of Petascale applications - Community Task - will take time –Support for asynchronous as well as synchronous visualization - TeraGrid Task –Easily customizable clients for visualization applications, for specific domains - TeraGrid Task

9. Planning Additional Community Events Convene regular events for sharing experiences- TG and Community Tasks – TeraGrid ‘08 - June 9-13, 2008 – SC’08 - November, 2008 Planning for Petascale Workshop Series – Target Community Peta-apps, SDCI, and other interested parties Applications scientists, computer scientists, vendors – Update of applications challenges – Updates of new and emerging tools and capabilities – Identify challenges, barriers, and gaps – Recommendations for further action

10. Q&A Session Which are roles for TeraGrid? for the community? for NSF? Which recommendations are most critical? A number of actions depend on active community engagement. How can we best facilitate community engagement and sharing?