1. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 1 ANABAS Use of Grids in DoD Applications Geoffrey Fox, Alex Ho, Marlon Pierce SBIR Briefing August 25, 2005

2. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 2 What do Web Services Prescribe? The specify interfaces for system services (and generally useful services like database) They specify an interface language (WSDL) for all services They develop containers and frameworks to use to host services They specify a message format (SOAP) for ALL messages that defines both application and system actions precisely They imply a process be started to define domain specific services There are multiple competing activities from Microsoft and IBM to Apache, IU and Anabas (for example) developing system and application services Unlike for RTI and CORBA, services from different vendors should interoperate H1H4H3H2Body F1F2F3 F4 Service Container Handlers Container System Processing

3. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 3 What do Grids Add? Grids use all of the Web Services They address management and deployment of large distributed systems of services They address security and management issues of virtual organizations crossing multiple administrative domains GGF is developing specific services of relevance including job management, many aspects of data and scheduling GGF has a good process for developing new higher level specifications –For example GGF15 will address Cross enterprise security, Campus Grids, Enterprise Grids, High Performance messaging, GIS Grids, Portals and continue OGSA work integrating data, compute and management

4. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 4 The Grid and Web Service Institutional Hierarchy 1: Container and Run Time (Hosting) Environment 2: System Services and Features Handlers like WS-RM, Security, Programming Models like BPEL or Registries like UDDI 3: Generally Useful Services and Features Such as “Access a Database” or “Submit a Job” or “Manage Cluster” or “Support a Portal” or “Collaborative Visualization” 4: Application or Community of Interest Specific Services such as “Run BLAST” or “Look at Houses for sale” OGSA and other GGF/W3C/ ……… WS-* from OASIS/W3C/ Industry Apache Axis.NET etc.

5. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 5 The Grid and Web Service Functional Hierarchy B: Resources C: Electronic Proxy Services for Resources G: User Interface F: Portal: Aggregation, Profiles E: Manipulating and Linking Services A: Pervasive System Services: Security, Collaboration, Messaging, Metadata D: Brokering Monitoring and Managing Resources and Services

6. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 6 Composing Functionality and Resources in the Grid of Grids Overlay and Compose Grids of Grids MethodsServicesFunctional Grids CPUsClusters Compute Resource Grids MPPs Databases Federated Databases SensorSensor Nets Data Resource Grids

7. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 7 Critical Infrastructure (CI) Grids built in composite fashion and linked to an NCOW (GiG) Grid DoD Services and Filters Physical Network RegistryMetadata Flood Services and Filters Flood CIGrid NCOW Grid … Electricity CIGrid … Data Access/Storage SecurityWorkflowNotificationMessaging Portals Visualization GridCollaboration Grid Sensor GridCompute GridGIS Grid

8. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 8 Mediation and Transformation in a Grid of Grids and Simple Services Port Internal Interfaces Subgrid or service Port Internal Interfaces Subgrid or service Port Internal Interfaces Subgrid or service Messaging Mediation and Transformation Services External facing Interfaces

9. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 9 DoD Services as Part of Grid of Grids ANABAS

10. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 10 The Global Information Grid Core Enterprise Services Core Enterprise ServicesService Functionality CES1: Enterprise Services Management (ESM) including life-cycle management CES2: Information Assurance (IA)/Security Supports confidentiality, integrity and availability. Implies reliability and autonomic features CES3: MessagingSynchronous or asynchronous cases CES4: DiscoverySearching data and services CES5: MediationIncludes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents CES6: CollaborationProvision and control of sharing with emphasis on synchronous real-time services CES7: User AssistanceIncludes automated and manual methods of optimizing the user GiG experience (user agent) CES8: StorageRetention, organization and disposition of all forms of data CES9: ApplicationProvisioning, operations and maintenance of applications.

11. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 11 The Ten areas covered by the core WS-* Specifications WS-* Specification AreaExamples 1: Core Service ModelXML, WSDL, SOAP 2: Service InternetWS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM 3: NotificationWS-Notification, WS-Eventing (Publish-Subscribe) 4: Workflow and TransactionsBPEL, WS-Choreography, WS-Coordination 5: SecurityWS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation 6: Service DiscoveryUDDI, WS-Discovery 7: System Metadata and StateWSRF, WS-MetadataExchange, WS-Context 8: ManagementWSDM, WS-Management, WS-Transfer 9: Policy and AgreementsWS-Policy, WS-Agreement 10: Portals and User InterfacesWSRP (Remote Portlets)

12. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 12 Activities in Global Grid Forum Working Groups GGF AreaStandards Activities 1: ArchitectureHigh Level Resource/Service Naming (level 2 of fig. 1), Integrated Grid Architecture 2: ApplicationsSoftware Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval, 3: ComputeJob Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling 4: DataDatabase and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management 5: InfrastructureNetwork measurements, Role of IPv6 and high performance networking, Data transport 6: ManagementResource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model 7: SecurityAuthorization, P2P and Firewall Issues, Trusted Computing

13. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 13 Core Services in Action I NCOW Service or FeatureWS-* Service areaGGF and OthersDemo Status A: General Principles Use Service Oriented ArchitectureCore Service Model (#1) Build Grids on Web Services YES Grid of Grids CompositionDTA7 B: NCOW Core Services (to be continued) CES 1: Enterprise Services Management WS-* #8 ManagementGGF #6: Management CIM Phase II (DTA9) CES 2: Information Assurance(IA)/Security WS-* #5 WS-Security GGF #7, Grid-Shib, Permis Liberty Alliance etc. Phase II (Not in DTA) CES 3: MessagingWS-* #2, #3JMS, MQSeries,Streaming /Sensor Technologies SBIR (DTA1) CES 4: DiscoveryWS-* #6SBIR (DTA8) CES 5: MediationWS-* #4 workflowTreatment of Legacy systems. Data Transformations SBIR (DTA6 DTA7) CES 6: CollaborationVOGGF VO. XGSP, Shared Web Service ports SBIR (DTA2 DTA3) CES 7: User assistanceWS- * #10Portlets, JSR168 NCOW Capability Interfaces SAB (DTA10)

14. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 14 Core Services in Action II NCOW Service or FeatureWS-* Service areaGGF and OthersDemo Status B: NCOW Core Services Continued CES 8: Storage (not real-time streams) GGF #4 Data NCOW Data Strategy Phase II (Not in DTA) CES 9: ApplicationGGF #2 ; Best Practice in building Grid/Web services SAB (DTA6 DTA7) Environmental Control Services ECS WS-*, #9Phase II (Not in DTA) Resource InfrastructureGGF #5; Ad-hoc networks important NO C: Key NCOW Capabilities not directly in CES Meta-dataWS-* #7SAB (DTA8) Resource/Service Matching/Scheduling Distributed Scheduling and SLA’s (GGF # 3) GGF scheduling work extended to networks Phase II with scheduled messaging (DTA1) Sensors (real-time data)OGC Sensor standardsSBIR (DTA6) GISOGC GIS standardsSBIR (DTA4 DTA5)

15. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 15 Analysis of Grid Technology for DoD We developed four significant documents DoD Grid Opportunities for the GiG and NCOW –Grids for the GiG and Real Time Simulations (Proceedings of Ninth IEEE International Symposium DS-RT 2005 on Distributed Simulation and Real Time Applications' Montreal October 10-12 2005) is subset Appendix on Grid and Web Services covering all basic Grid and Web services Grid Technology Overview and Status reviews Grids Grid Application Areas within DoD is a general study of applications of Grids in DoD Complete list of References for Reports on Grids for GiG and NCOW contains all references in above reports Implementing some Grid Application Areas within NCOW 1.1 of DoD is not started but is intended to write up specific applications such as demos

16. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 16 Major Conclusions I One can map “broadly” NCOW and GiG core capabilities into Web Service (WS-*) and Grid (GGF) architecture and core services –Analysis of Grids in NCOW document inaccurate (confuse Grids and Globus and only consider early activities) Important “mismatches” on both NCOW and Grid sides Grid/WS-* do not have messaging and collaboration NCOW does not have system metadata and resource/service scheduling and matching Higher level services of importance include GIS (Geographical Information Systems), Sensors and data- mining Criticisms of Web services in a recent paper by Birman seem to be addressed by Grids or reflect immaturity of initial technology implementations

17. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 17 Major Conclusions II NCOW does not seem to have any analysis of how to build their systems on WS-*/Grid technologies in a layered fashion; they do have a layered service architecture so this can be done –In particular do not clearly endorse critical header/body structure of SOAP messages Grid of Grids allows modular architectures and natural treatment of legacy systems

18. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 18 Areas in Birman Paper Information Architecture and Service Description Document-Centric Nature Time-critical Events Life Cycle Support for Services Reliable Messages Security Scalability

19. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 19 Grids and HLA/RTI I HLA through IEEE1516 has specified the interfaces for its key services that are supported by RTI (Run Time Infrastructure) HLA does not specify each message semantics or core system services –RTI implementations are NOT interoperable although each one should support any HLA federation –RTI implementations become a full distributed system environment as need metadata, reliable messaging etc. with simulation support only a small part Grids can be used in HLA with –Dynamic assignment of compute resources to support federates –Building web service interfaces to federates (XMSF) –Infrastructure to build a new generation of RTI that will use Web system services and just add simulation support

20. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 20 Grids and HLA/RTI II HLA specifies –Declaration management – achieved through use of publish/subscribe Grid Messaging (NaradaBrokering) –Data Distribution management – corresponds to geometry sensitive publish and subscribe model (add to NaradaBrokering) –Time management – corresponds to simulation framework (use best event driven and time stepped models – as infrastructure generic, one can support broad range of simulations including classic parallel computing and agent-based simulations) –Object management - Very specific to HLA and should be built as per IEEE1516 –Ownership management - could use Grid virtualization and use metadata catalog catalogs to handle properties – might be generalizable –Federation management - Could generalize to support of general simulation models (federates and federations are a general concept)

21. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 21 Demo Technology Areas DTA 1)NaradaBrokering: Grid Messaging 2)Anabas Collaborative Applications 3)GlobalMMCS/XGSP Collaborative Stream Management Framework and A/V Conferencing 4)Geographical Information Systems (GIS) Web Feature Services (WFS) 5)GIS Web Map Services (WMS) and Clients 6)GIS Sensor Enablement Services 7)Wrapping of non Grid Applications as Grids of grids or Grids of Services 8)Information and meta-data Services 9)HPSearch Workflow and Management 10)Portals and Portlets

22. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 22 IN1 Data Mining and GIS Grid WMS handling Client requests WMS Client UDDI WFS2 Databases with NASA, USGS features SERVOGrid Faults WFS1 NASA WMS HTTP SOAP WFS3 Data Mining Grid WMS Client

23. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu23 California fault data from Quake Tables fault database via Web Feature Service. Standard Open Geospatial Consortium WMS Clients

24. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu24 Get Feature Info allows users to get map information. This can also be used to read feature info off the map when creating input data for applications Standard Open Geospatial Consortium WMS Clients

25. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 25 IN1 Data Mining Grid HPSearch Workflow UDDI Databases with NASA,USGS features SERVOGrid Faults WFS4 SOAP WS-Context WFS3 PI Data Mining Filter GIS Grid Filter Narada Brokering Pipeline System Services

26. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu26 PI demo combines WFS, WMS, and HPSearch for service orchestration. Tool bar items allows you to adjust maps Users set up problems by adding filtered seismic archives from WFS as map layers.

27. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu27 Hot spots calculations-- areas of increased earthquake probability in the forecast time-- calculations are re-plotted on the map as features.

28. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 28 Typical use of Grid Messaging in NASA Datamining Grid Sensor Grid Grid Eventing GIS Grid

29. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 29

30. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 30 Typical use of Grid Messaging HPSearch Manages Narada Brokering Sensor Grid WS-Context Stores dynamic data Filter or Datamining WFS Post before Processing Post after Processing Notify Subscribe Database Archivess

31. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 31 IN2 Google Map Client Google Central Google Map Client UDDI WFS2 Databases with SERVOGrid Faults WFS1 SOAP Sensor Grid HTTP Helper Services ArchivedReal Time

32. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 32 IN2: Real Time GPS and Google Maps Subscribe to live GPS station. Position data from SOPAC is combined with Google map clients. Select and zoom to GPS station location, click icons for more information.

33. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 33 Google maps can be integrated with Web Feature Service Archives to filter and browse seismic records. IN2: Integrating Archived Web Feature Services and Google Maps

34. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 34 IN2: Integrating Archived Web Feature Services and Google Maps Google maps can be integrated with Web Feature Service Archives to browse earthquake fault records. Faults are typically stored by segment number, so map interfaces are convenient for both verifying continuity and setting up input files for computing problems.

35. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu35 IN2: Same Screens with Hybrid Maps

36. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 36 IN3 Google Maps as a WMS Web service Google Central WMS handling Client requests WMS Client UDDI WFS2 ….. Databases with NASA,USGS features SERVOGrid Faults WFS1 Google Maps WMS (Gateway) HTTP SOAP Other Services using results

37. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 37 Google Maps as Service accessed from our WMS Client

38. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 38 Collaborative Streaming Grid IN4 Server-side Streaming from a WMS WMS handling Client requests UDDI SOAP XGSP Media Service Narada Brokering WFS2 Databases with NASA, USGS features SERVOGrid Faults WFS1 NASA WMS WFS3 Data Mining Grid

39. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 39 Collaborative Streaming Grid IN4 Shared Display Streaming from a WMS WMS handling Client requests UDDI SOAP XGSP Media Service Narada Brokering WFS2 Databases with NASA, USGS features SERVOGrid Faults WFS1 NASA WMS WFS3 Data Mining Grid WMS Client GMC GMC = GlobalMMCS Client

40. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 40 Web Service Collaboration Web Service Narada Brokering WS1 WS2 WS3 Narada Brokering Shared Input Port with replicated services Shared Output port with replicated recipients

41. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 41 Pipelined Web Service Collaboration In a workflow, one can invoke collaborative streams on any flow and this splitting is between output port of one and input of next Web Service in chain WS1 WS2 WS3 Narada Brokering WS4 WS5 WS6 WS-BWS-A Shared Input Port Shared Output Port

42. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 42 Collaboration Grid UDDI Narada Broker HPSearch WS-Context Gateway WS-Security Narada Broker Gateway XGSP Media Service Video Mixer Transcoder Audio Mixer Replay Record Annotate Thumbnail WhiteBoard SharedDisplay SharedWS

43. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 43 Annotation of GIS Maps from WMS converted into H261 Video Stream

44. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 44 New GlobalMMCS Client Chat TV WebcamVideo Mixer GIS

45. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 45 Collaborative Map Pages Shared Input Port Anabas Endpoint XML Sharedlet interface similar to WSDL Narada Brokering Anabas Endpoint Anabas Endpoint Google Central Input to Google web page

46. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 46 Impromptu Google Interface

47. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 47 Impromptu Google Map on Whiteboard

48. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 48 Clients and Services Thick clients can be Grid (WSDL) endpoints and act as services Can break thick client up as MVC with Model becoming a service – Message-based MVC ClientGrid Service SOAP Model as Service Grid Service SOAP Portal HTTP Portlet View

49. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 49 Collaborative “Legacy” Object Shared Output Port with Anabas Endpoint providing Service wrapping of RMI Object Anabas Endpoint XML Sharedlet interface similar to WSDL Narada Brokering Anabas Endpoint Anabas Endpoint Java Distributed Object RMI Output from Java Object

50. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 50 Collaborative Annotation of Map Pages Shared Input Port with replicated white board XML Sharedlet interface similar to WSDL Narada Brokering Anabas WB Service Google Central White Board changes Anabas Google Endpoint WB Service Anabas WB Service

51. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 51 Collaborative Search Shared Input Port for annotation with replicated search display (shared output port for search service) Narada Brokering Anabas Annotatable Text Service Search Results and annotation Anabas Search Endpoint Annotatable Text Service Anabas Annotatable Text Service Google Search Yahoo Search MetaSearch

52. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 52 Impromptu Yahoo Search

53. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 53 Impromptu Google Search

54. August 25 2005 Briefing gcf@iu.edu alexho@anabas.com mpierce@cs.indiana.edu 54 Impromptu MetaSearch