1. Introduction to Grid Application On-Boarding Nick Werstiuk werstiuk@platform.com

2. Presentation Objective and Contents Objective Provide a Grid software vendor perspective on the challenges, approaches and techniques used with our customers to on-board applications to their Enterprise Grid environments Presentation Contents Context to Frame the Discussion Enterprise Motivations – Why On-board? Enterprise Challenges Choices Bring Flexibility and Complexity Example Scenario Conclusions

3. Context to Frame the Discussion Context This presentation is made in the context of Enterprise users across multiple industries (Electronic Design, Industrial Manufacturing, Financial Services, etc) who use their Enterprise grid for multiple applications. When I say Grid on-boarding… what I mean is… taking an application (either homegrown or commercial ISV) and enable it to run that on a distributed infrastructure within the 4 walls of the Enterprise Typical Enterprise customer has a range of application types and patterns that bring different challenges. Sequential vs. Parallel Short Running vs. Long Running Job Oriented vs. Service Oriented Home Grown vs. ISV

4. Enterprise User Motivations – Why Onboard? Application Business Owner Driven Application Performance Need results faster – leverage parallel execution Demand growth beyond the boundaries of a single server IT /Infrastructure Driven Economy of scale – more applications means more resources, means (generally) opportunity for a more efficient infrastructure Aggressive promotion of centralized/managed grid infrastructure for an Enterprise Utility/Shared service.

5. Enterprise Challenges with On-boarding to the Grid Balancing between needs of the Application Owners, Architects, Developers and the IT Infrastructure Application and language specific middleware and toolkits vs. single approach for internal standards Server and Cluster Huggers Multiple applications on the same infrastructure introduces additional management challenges Different Application Middleware – Common Resources Service Levels Resource Sharing and Policies Minimize Application Development and Integration Maintenance Leverage new resource and processor types without major application migration pain

6. Choices Bring Flexibility and Complexity Application Integration Driven In many cases on-boarding is a script integration between the ISV and the grid infrastructure Easiest to do with Job based applications Quick assessment of the market says there are probably 100+ commercial ISVs across the high performance computing market that have this type of deployment. Maintaining the integrations becomes a challenge for the ecosystem (A standards opportunity) Application Development Driven How best to decompose the problem? Single Program Multiple Data (SPMD) Multi-Program Multiple Data (MPMD) Whats the Best middleware for the application and programming model OR the corporate grid standard Another Silo or an Enterprise Grid. How do I enable non Grid developers to easily transition their applications to the grid

7. Choices Bring Flexibility and Complexity Which one is the Best? MPI OpenMP Task-Oriented / Service Oriented HPC Message Oriented Middleware JavaSpaces Map-Reduce Spring Batch JSR-237 Work Manager for Application Servers Multi-core/GPU Programming Models Emerging Parallel Languages Complex Event Processing CEP

8. Sample Paradigms for Parallel Applications ModelParadigmDevelopmentDeploymentRun Map- Reduce (MR) Scatter & aggregate for data parallel app Split data into files, write functions that map input files into intermediate output files, then aggregate into result files Copy MR functions to a file system like Need a special distributed file system Limited node sharing with other apps MOM, JMS, AMQP Message- oriented master and workers A master and its workers send & receive messages via a message server or bus Manually copy programs to nodes Nodes are dedicated to a single app run Multi-coreMulti-core shared memory multi- threading Convert a serial app in the master thread into parallel tasks for worker threads, with memory & cache access optimization Manually copy a program to multi-core boxes Multi-core box is dedicated to a single app run HPC service- oriented middleware Scatter & gather; generic task- oriented service pool Parallel tasks are scattered to a distributed service pool, results are gathered by clients or other tasks. Service programs are automatically deployed to grid nodes on-demand Nodes & service instances are pooled & shared with other apps

9. Example Scenario Split the Problem Developers Develop without knowledge of the grid infrastructure Seamless use of their IDEs and development tools IT Provides the Infrastructure without getting into the details of the application Leverage application developer package – promote from test environment into production Enable Wider Enterprise Developer Usage Open Access to broad range of developers Enable IT to promote grid on-boarding to their application developer communities

10. Example Scenario Symphony Developer Edition: A grid-ready application middleware to allow developers to code, tune, test applications before deployment onto a grid infrastructure Development & testing of the application is done outside of the grid Eclipse IDE Integration No further code changes required to deploy to the grid No IT involvement & effort is needed Supports service oriented and batch applications Increases developer productivity - hides grid from developers Middleware delivers the highest application performance Support millisecond round trip latency Enables developers to target applications with a range of runtime characteristics No restrictions access for developers at my.platform.com Go ahead and sign in and try it

11. Other Considerations Capacity Planning New Resources vs. Fitting into Existing Grid Data and I/O Impacts Data Cache Database Filesystem End User Experience Desktop Integration ISV Integration End User Portal

12. Conclusions Customer Objectives Balance between driving more applications and dealing with political/organizational issues Challenges Maintaining the traditional well understood application integrations as the infrastructure and ISV evolves Emerging application patterns and approaches