Introduction to Grid Computing Ed Seidel Max Planck Institute for Gravitational Physics

Grid Intro 2 Outline n What is the Grid. n What does the Grid mean for application users? n What is a Virtual Organization? n Why is the Grid different from the Internet? n What should application users and developers be doing right now to be ready for the Grid? n High level overview of today’s testbeds.

Grid Intro 3 History and Motivation n 1970s - 80s: Internet l , ftp, etc n 1980s - 90s: Remote-, Distributed-, Meta-computing l Beginning to think of ways to exploit distributed resources l US SC Centers Programs –Had to access remotely –Think of harnessing together l See examples later n SC95: I-Way l First large scale experiments l Most failed, but I-Way a success! n Post I-Way: The Grid l Explosion of activity, but still immature. Get prepared for future

Grid Intro 4 Grid Computing: A New Paradigm n Computational Resources Scattered Across the World l Compute servers (double each 18 months) l Handhelds l File servers l Networks (double each 9 months) l Playstations, cell phones etc… n How to take advantage of this for scientific/engineering simulations? l Harness multiple sites and devices l Simulations at new level of complexity and scale, interacting with data

Grid Intro 5 Grid vs. Internet/Web Services? n We’ve had computers connected by networks for 20 years n We’ve had web services for 5-10 years n The Grid combines these things, and brings additional notions l Virtual Organizations l Infrastructure to enable computation to be carried out across these –Authentication, monitoring, information, resource discovery, status, coordination, etc n Can I just plug my application into the Grid? l No! Much work to do to get there!

Grid Intro 6 All have to work for real applications Communities n Teams with a need to share data, or develop common code n Communities can be very diverse l Experimentalists, computer scientists, simulation scientists, network researchers l All may need to work together to solve a specific problem l May co-develop a code, or contribute modules that should work together n These communities may have very different resources l Within company or university l Across the world n Grids aim to bring them together, allow them to harness their resources n These communities may form a type of (virtual) organization Components for Grid Computing

Grid Intro 7 Virtual Organizations Resources n Machines, networks, archives, file systems, etc at different sites n What is a Virtual Organization (VO)? l groups of organizations that use the Grid to share resources for specific purposes –EU DataGrid, Alliance, TeraGrid, SC02 Global Grid Testbed, etc l Typically deploy same technology l Deploy directory service: resources registered, and may be “discovered”. Globus has MDS (Monitoring and Discovery Service) –GIIS (Grid Index Information Service) provides info for entire VO –GRIS (Grid Resource Information Service), installed locally, reports to GIIS so people or applications can search GIIS for info l Authentication: Certificates and Gridmap file –Sometimes a VO offers a “certificate” for individuals and resources –Sometimes they use other VO’s certificates n So far, there is no Global GIIS that links together all VOs

Grid Intro 8 Software Infrastructure Infrastructure: Globus Metacomputing Toolkit n Low Level l Fundamental technologies needed to build computational grids. l Security: logins, data transfer l Communication l Information (GRIS, GIIS) l More generally: Grid Services n Middleware l Data movers l Resource Brokers l Portals l Application Monitoring systems n High Level l Application Toolkits n We focus today on Globus, Cactus, and GridLab l Other: Legion, Unicore, Juxta…

Grid Intro 9 Grid Aware Applications (Cactus example): n Grid Enabled Modular Toolkits for Parallel Computation: Provide to Scientist/Engineer, etc.. n Plug your Science/Eng. Applications in! n Must Provide, Register as Grid Services l Ease of Use: automatically find resources, given need! l Distributed simulations: use as many machines as needed! l Remote Viz and Steering, tracking: watch what happens! l Take advantage of infrastructure n Collaborations of groups with different expertise: no single group can do it! Grid is natural for this… Components for Grid Computing

Grid Intro 10 Example Grids n GridLab Testbed l Ten machines in Europe for developers of Grid tools n SC2001 ARG Testbed & Global Grid Testbed Collaboration l Hastily assembled loose federation of world machines for SC2001 and now SC2002 demonstrations n NCSA Virtual Machine Room and PACI Grid l Production resources n TeraGrid ( l USA distributed terascale facility at 4 sites for open scientific research n Information Power Grid ( l NASAs high performance computing grid

Grid Intro 11 Example Grids NSF PACI Grid NorduGrid

Grid Intro 12 The NSF TeraGrid n Some Grids are tightly coupled facilities, like TeraGrid n Some are testbeds, like Global Grid Testbed Collaboration n Some are long term, loosely coupled: PACI Grid (includes TeraGrid) n All try to provide organized access to distributed resources n Most are depending on Globus as the base infrastructure 13.6 Tflop facility

Grid Intro 13 What does this mean now for users and developers? n There is a grand vision of the future l Collecting resources around the world into Vos l Seamless access to them, with a single signon l NEW applications to exploit of them in unique ways! l Today we want to help you prepare for this n There is a frustrating reality of the present l These technologies are not yet fully mature l Not fully deployed l Not consistent across even single Vos n But centers and funding agencies worldwide are pushing this very, very hard l Better get ready now l You can help! Work with your centers to get this deployed

Grid Intro 14 Getting Ready for the Grid n Need to start imagining how your applications can exploit the Grid (simplify use of HPC, provide more processing power, better access to data, allow new scenarios to forward your science, better collaborations). n Applications codes will need modifications for the Grid, depending on much of it you want to be able to exploit! n Write new codes with the Grid in mind. n Consider using frameworks which are already Grid-compatible.