Grid of Grids Information Management Meeting Anabas October

General Goal Build Net Centric Core Enterprise Services in fashion compatible with GGF/OGF and Industry Add key additional services including those for sensors and GIS Support Systems of Systems by federating Grids of Grids supporting a heterogeneous software production model allowing DoD greater sustainability and choice of vendors Build tool to allow easy construction of Grids of Grids

Database SS SSSSSSSSS FS FSFS Portal FSFS OSOS OSOS OSOS OSOS OSOS OSOS OSOS OSOS OSOS OSOS OSOS OSOS MD MetaData Filter Service Sensor Service Other Service Another Grid Raw Data  Data  Information  Knowledge  Wisdom Decisions S S Another Service S Another Grid S SS FS SOAP Messages Portal

The Grid and Web Service Institutional Hierarchy OGSA GS-* and some WS-* GGF/W3C/…. XGSP (Collab) WS-* from OASIS/W3C/ Industry Apache Axis.NET etc. Must set standards to get interoperability 2: System Services and Features (WS-* from OASIS/W3C/Industry) Handlers like WS-RM, Security, UDDI Registry 3: Generally Useful Services and Features (OGSA and other GGF, W3C) Such as “Collaborate”, “Access a Database” or “Submit a Job” 4: Application or Community of Interest (CoI) Specific Services such as “Map Services”, “Run BLAST” or “Simulate a Missile” 1: Container and Run Time (Hosting) Environment (Apache Axis,.NET etc.) XBML XTCE VOTABLE CML CellML

The Ten areas covered by the 60 core WS-* Specifications WS-* Specification AreaExamples 1: Core Service ModelXML, WSDL, SOAP 2: Service InternetWS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM 3: NotificationWS-Notification, WS-Eventing (Publish-Subscribe) 4: Workflow and TransactionsBPEL, WS-Choreography, WS-Coordination 5: SecurityWS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation 6: Service DiscoveryUDDI, WS-Discovery 7: System Metadata and StateWSRF, WS-MetadataExchange, WS-Context 8: ManagementWSDM, WS-Management, WS-Transfer 9: Policy and AgreementsWS-Policy, WS-Agreement 10: Portals and User InterfacesWSRP (Remote Portlets)

Activities in Global Grid Forum Working Groups GGF AreaGS-* and OGSA Standards Activities 1: ArchitectureHigh Level Resource/Service Naming (level 2 of slide 6), Integrated Grid Architecture 2: ApplicationsSoftware Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval, 3: ComputeJob Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling 4: DataDatabase and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management 5: InfrastructureNetwork measurements, Role of IPv6 and high performance networking, Data transport 6: ManagementResource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model 7: SecurityAuthorization, P2P and Firewall Issues, Trusted Computing

Net-Centric Core Enterprise Services Core Enterprise ServicesService Functionality NCES1: Enterprise Services Management (ESM) including life-cycle management NCES2: Information Assurance (IA)/Security Supports confidentiality, integrity and availability. Implies reliability and autonomic features NCES3: MessagingSynchronous or asynchronous cases NCES4: DiscoverySearching data and services NCES5: MediationIncludes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents NCES6: CollaborationProvision and control of sharing with emphasis on synchronous real-time services NCES7: User AssistanceIncludes automated and manual methods of optimizing the user GiG experience (user agent) NCES8: StorageRetention, organization and disposition of all forms of data NCES9: ApplicationProvisioning, operations and maintenance of applications.

Produce the Needed Core Services We can classify services in many ways and following 2 charts are one way; slightly changed from proposal as NCOW and our work changed a little. Green is “in hand”; we know a lot Orange is “in hand” with outside but available solutions Red has problems – Security does not have industry consensus while current Scheduling work does not address DoD real-time service and network requirements

The Core Features/Service Areas I Service or FeatureWS-* GS-* NCES (DoD) Comments A: Broad Principles FS1: Use SOA: Service Oriented Arch. WS1Core Service Architecture, Build Grids on Web Services. Industry best practice FS2: Grid of GridsDistinctive Strategy for legacy subsystems and modular architecture B: Core Services FS3: Service Internet, Messaging WS2NCES3Streams/Sensors. Team FS4: NotificationWS3NCES3JMS, MQSeries. FS5 WorkflowWS4NCES5Grid Programming FS6 : SecurityWS5GS7NCES2Grid-Shib, Permis Liberty Alliance... FS7: DiscoveryWS6NCES4UDDI FS8: System Metadata & State WS7Globus MDS Semantic Grid, WS-Context FS9: ManagementWS8GS6NCES1CIM FS10: PolicyWS9ECS

The Core Feature/Service Areas II Service or FeatureWS-*GS-*NCESComments B: Core Services (Continued) FS11: Portals and User assistance WS10NCES7Portlets JSR168, NCES Capability Interfaces FS12: ComputingGS3 FS13: Data and StorageGS4NCES8NCOW Data Strategy Federation at data/information layer major research area; CGL leading role FS14: InformationGS4JBI for DoD, WFS for OGC FS15: Applications and User Services GS2NCES9Standalone Services Proxies for jobs FS16: Resources and Infrastructure GS5Ad-hoc networks FS17: Collaboration and Virtual Organizations GS7NCES6XGSP, Shared Web Service ports FS18: Scheduling and matching of Services and Resources GS3Current work only addresses scheduling “batch jobs”. Need networks and services

Additional Services Sensors have low level support listed as FS3; higher level integration using SensorML and Filters well understood. Some work in phase I GIS Grid services pioneered by team and already shown in phase I Mediation (Interoperability) Services needed to link Grids (defined as a collection of ≥ 1 Services) –Need to generalize existing solutions for Sensor Grids and for MQSeries-SOAP Mediation –View NaradaBrokering as a SOAP Intermediary

Out of Scope for Phase II Many areas are still evolving significantly –Mediation/Interoperation –Security –Scheduling of non-compute Resources –Data/Information Federation –Semantic Grid and management We will not test scalability on large number of services, sensors and component Grids Integrating legacy systems not addressed Grid of Grids building tool is “new idea” – can expect will benefit from further work

Research Tasks The R&D Effort is divided into five major tasks: Task 1: Implementation of Collaboration Grid Middleware Task 2: Enhanced NCOW Core Enterprise Services (NCES) with Enterprise Control Services and Metadata Services Task 3: Design and implementation of Grid of Grids mediation algorithms and NCOW services Task 4: Design and Implementation of Net-Centric Collaboration Grid Builder Tool Task 5: Technology Demonstration

Grid of Grids Building Tool This will provide a graphical interface to build grids from existing libraries of Services and Grids Meta-data (provenance) needs to be specified Grids (services) need to be linked This will be built by extending an existing workflow engine which is aimed at a more tightly couple version of the builder problem –We will consider HPSearch (CGL), Taverna (Open source from UK OMII), BPEL with user interface (OMII or IU LEAD project), Eclipse –Semantic Grid provenance “add-ons” –We have experience with all these approaches

Applications Indiana University will apply to Earthquake Science and Emergency Response Ball Aerospace will apply to DoD applications using possibly Matrix to support people involved But a key problem is how do we package results? –Looks perhaps that GridBuilder is key Take above 3 (or other?) applications and make scenarios as to how to use GridBuilder Who will use GridBuilder – DoD or contractors building software