Extreme! Computing Lab, Dept. of Computer Science, Indiana University 1 Programming the Grid with Components Madhu Govindaraju Aleksander Slominski Dennis Gannon Sriram Krishnan

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 2 Outline Software Components XCAT –Component and Services model – Web Services – Programming model – OGSI

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 3 Software Components Analogy with hardware Standard design paradigm for app development Goal: to build applications by composition of well tested and well behaved subsystems (components). – simplify application design process – promote code reusability – plug and play

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 4 Component Architecture A Component Architecture consists of two parts: – Components software objects that implement a set of required behaviors – Frameworks A runtime environment A set of services used by components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 5 Industry Standards COM/DCOM – interoperability for Microsoft applications Java Beans, Enterprise Java Beans (EJB) –Java based desktop and enterprise applications OMG CORBA Component Model (CCM) –language neutral superset of EJB

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 6 Academic Research SciRun from Utah – scalable parallel applications and viz Webflow from Syracuse – graphical composition palette CCAT and XCAT from Indiana University – framework for Grid based applications

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 7 CCA: Common Component Architecture  CCA: Initiative to define minimal specification for scientific components  Department of Energy (National Labs)  Few universities  Aim: build components for high performance computing  Draws ideas from CCM and other models as a baseline  Targeting  Parallel  Distributed

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 8 The Common Component Architecture Key Idea: dynamic composition by linking “ports” – Provides ports: interface of “services” provided – Uses ports: a call-site for a service to be provided by another component. Uses port- Provides Port

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 9 What is new about CCA? Minimal specification Envision connections as dynamic – Can add, remove and connect ports at run-time Tailored to build Problem Solving Environments – End user manipulates connections Not specific to any underlying distributed object model Notion of collective ports – logically one connection – implementation: multiple network connections

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 10 Components on the Grid: Issues Discovering static information about components Launching components Discovering references for running instances Communication protocols Messages and Events Authentication and Authorization Encapsulating legacy applications Efficient scheduling Run-time environment

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 11 Grid Programming Model: Needs Need a set of APIs, protocols, libraries and tools that allow access to Grid resources Examples: Globus, Condor – provide infrastructure – do not provide programming model What is missing? – How can we make it easier to “program” the Grid?

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 12 XCAT Implementation of the CCA specification Designed for distributed applications Allows creation of Grid application from components Wraps Grid Services as components Test-bed for CCA – Is there an alternative to OMG IDL?

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 13 Component Communication How do components communicate? – Use Remote Procedure Call (RPC) Mechanism XCAT uses SOAP 1.1 XCAT ports can serve as web services – Events/Messages Objects encoded as XML documents

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 14 XCAT Services Architecture Default services for all components XCAT services – Directory locate components based on port types and other attributes – Registry locate running instances of components – Creation create running instance of a component – Connection connect ports of two running instances – Events publish/subscribe framework for messages

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 15 IDS: Information Directory Service Store Component descriptions in XML Used to search databases and remote repositories Information Directory Service Component MySQL Database File system LDAP RDBS Request component info Return XML specs.

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 16 X Creation Service Creates a running instance of another component Encapsulates authentication issues Supports GRAM, SSH and Local-Exec Creation Service Component Launch an instance of component X on resource Y Returns: remote reference to new component instance Globus resource Y

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 17 Registry Service  Stores instance information (runtime configurations)  Used to advertise and search other instances  Tradeoff: distributed vs. centralized Find a running instance of component X Returns: remote reference to component instance Registry Service Component

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 18 Connection Service A component that can be used to connect a “uses” port of one component to the “provides” port of another Can export ports of other components as one’s own Y Connection Service Component X Connect port A of component X to port B of component Y A B

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 19 Event Service Channel for publish-subscribe event model Creation and connection events Application users can use it for asynchronous information transfer B Event Channel Service Component C A Subscribe to listen for events of type X xx x x Notification of publication of events of type X

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 20 Features of Events Event/Notification Services are an essential part of all distributed systems Application Events –“I am done.” “I just wrote to a file,” “here is a result.” Event Publishers & Listeners – Some Listeners subscribe, others poll – Must have persistent event channels record events for later analysis application history logs allow for retrieval in chunks

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 21 XCAT: Other Features Security: –Every remote method call is intercepted – port, method and parameter level – authentication based on SSL – authorization based on simple ACL Component Handle: – in XML and can be converted to WSDL – GSH /GSR Exceptions –Supports throw and catch across components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 22 XCAT Processes Creation Service Other Services Connection Service Framework Impl. Scripts, Front-end Tools Application Code Framework Impl. Application Code Scripts, Front-end, Tools Creation Service Connection Service Other Services Port Wrapper Code

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 23 XCAT-Web Services Stack Framework Layer: Creation, Connection,… Service Discovery Layer: LDAP Service Description Layer: Subset of WSDL Service Messaging Layer: XSOAP Service Transport Layer: HTTP

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 24 XCAT Programming: Examples Builder: A tool used to select and link components together Builder Options – Swing based GUI – Jython scripting – Portal XCAT – Jython scripting

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 25 CCAT Gui image

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 26 Scripting XCAT Applications import xcat stringDump = xcat.createComponent(‘StringDump’) printer = xcat.createComponent(‘Printer’) xcat.setCreationMechanism(stringDump, ‘gram’) xcat.setCreationMechanism(printer, ‘ssh’) xcat.setMachineName(stringDump, “exodus”) xcat.setMachineName(printer, “rainier”)

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 27 Scripting XCAT Applications: contd printer.put(“exec-fqn”, “samples.printer.Printer”) stringDump.put(“exec-fqn”, “samples.stringDump.StringDump) xcat.createInstance(printer) xcat.createInstance(stringDump) xcat.connectPorts(stringDump, ‘outputString’, printer, ‘inputString’)

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 28 Encapsulating Legacy Apps Common Case –Legacy App that reads files and writes files –Use a “scripted component” Component runs a python script –The App Script Stages files Launches and monitors application Writes output files publishes event streams application XCAT comp input files output files App Script Event Stream Control

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 29 Scripting Grid Computations Grid Scripts are executable documents that – Orchestrate Computations Select resources Supply application parameters and launch remote jobs Subscribe to event streams Create components Connect components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 30 Composition Using Components Two ways of composing applications – Composition in space: one component/service directly invokes the services of another – Composition in time: A workflow engine schedules tasks that involve accessing remote services and responding to events

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 31 XCAT, OGSI and Web Services GS Component A GS Component B Standard Grid Service Grid Service Client

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 32 XCAT and OGSI compliance Add a GridService port to each component Merge component handle with GSH/GSR OGSI messaging is non-reliable push model – XCAT has a reliable, persistent model OGSI has factory portType for instantiating new services – XCAT has an extended distributed factory model