Managing Metadata in Service Architectures Mehmet S. Aktas Advisor: Prof. Geoffrey C. Fox

Outline  Introduction  Motivation  Requirements  Research Issues  Architecture  Performance Evaluation  Conclusions  Contribution 2 of 34

3 of 34 Context as Service Metadata  Context interaction-independent  slowly varying, quasi-static service metadata interaction-dependent  dynamically generated metadata as result of interaction of services  information associated to a single service, or a session (service activity) or both  Dynamic Grid/Web Service Collections loosely assembled collections of services assembled to support a specific task generate metadata and have limited life-time

4 of 34 Motivating Cases  Multimedia Collaboration Grids Global Multimedia Collaboration System- Global MMCS widely distributed services, session service metadata, session metadata, stream-specific metadata mostly read-only  Workflow-style applications in Geographical Information System/Sensor Grids Pattern Informatics (PI) – UC Davis, Interdependent Energy Infrastructure Simulation System (IEISS) – LANL widely distributed services conversation metadata, transient multiple writers

5 of 34 Problems with Grid Information Services  Standardization and Unification Issues Customized Grid Information Services Differences in application requirements Thick clients  Performance and Centralization Issues Low performance Low fault tolerance  Dynamic Metadata Management Issues Point-to-point service communication approaches

6 of 34 Requirements for Grid Information Services  Greater Interoperability Unified platform for communication Shared communication protocol Thin clients  Greater Capabilities High Performance Fault-tolerant  Dynamic Grid/Web Service Collections Distributed state management Collaboration session management

7 of 34 Research Issues I  Unification of Grid Information Services How to combine different information services?  Federation of Grid Information Services What is a common data model and communication protocol?  Flexibility and extensibility Accommodating broad range of application domains  read-dominated, read/write dominated Ability to add/support more information services  Interoperability Being compatible with wide range of applications

8 of 34 Research Issues II  Performance Efficient centralized metadata management strategies  high performance and persistency Efficient decentralized metadata management strategies  Efficient request distribution strategies  Adaptation to instantaneous client-demand changes  Fault-tolerance Efficient replica-content creation strategies  Consistency How to provide consistency across the copies of the same data?

 Unification  Uniform Access  Extensibility  Interoperability  Extended UDDI  WS-Context  Federation  Unified Schema  Query/Publish XML API Hybrid Grid Information Service 9 of 34

10 of 34 UDDI instance WS-Context instance Unified schema instance

11 of 34  Decentralized  Fault-tolerant  Efficient distribution  Look-ahead caching  Consistency enforced 11 of 34

12 of 34 Support for interaction-independent metadata: Extended UDDI Service  It supports different types of metadata Geographical Information System Metadata Catalog (functional metadata) User-defined metadata ((name, value) pairs)  It enables advanced query capabilities Geo-spatial queries Metadata oriented queries Domain independent queries  It provides additional capabilities Up-to-date service registry information (leasing) Dynamic aggregation of capabilities of services  Ex: geospatial capabilities

Support for interaction-dependent metadata: WS-Context Service  Context Manager Service Data model and communication protocol Session-related metadata  It supports Dynamic Web Service Collections Support for distributed state based systems  collaboration grids  workflow-style grids  It provides various capabilities Asynchronous communication capability Up-to-date service registry information (leasing) 13 of 34

Support for federated service metadata: Unified Information Service  Federating Grid Information Services Unified data model and communication protocol Extended UDDI, WS-Context and Glue Schemas  Approach taken Union of schemas vs. separate schemas Reuse common concepts  Ex1: business, session, site => category Combine disjoined concepts  Ex1: UDDI’s tModel  It enables hybrid query capabilities “Give me list of services satisfying C:{a,b,c..} QoS requirements and participating S:{x,y,z..} sessions” 14 of 34

Collaboration Grid Sensor Grid WSDL HYBRID Service Database WS-Context Topic Based Publish-Subscribe Messaging System Subscriber Publisher WSDL HYBRID Service Database Ext-UDDI Federating Grid Information Services 15 of 34

16 of 34 Features of the Distributed System  Cache Strategy Memory-in storage  Access Distribution Redirecting client request to an appropriate replica server  Look-ahead caching Moving/replicating metadata to where they wanted  Replica Content Placement Replicating data on an appropriate replica server  Consistency enforcement Ensuring all replicas of a data to be the same 16 of 34

17 of 34 Tuple Spaces & Publish-Subscribe Paradigms  Publish-Subscribe paradigm Message based asynchronous communication Participants are decoupled both in space and in time Open source NaradaBrokering software  topic based publish/subscribe messaging system  Tuple Spaces paradigm [Gelernter-99] a data-centric asynchronous communication paradigm communication units are tuples (data structure) JavaSpaces [Sun Microsystems]- object oriented implementation specification

18 of 34 Caching Strategy  Light-weight implementation of JavaSpaces Data sharing, associative lookup, and persistency  Integrated caching capability for all types of service metadata Ex: UDDI-type, WS-Context-type, Unified Schema-type metadata We assume that today’s servers are capable of holding such small size metadata in cache.  All metadata accesses happen in memory  Persistency All metadata is backed-up into appropriate Information Service back-end every so often for persistency

Persistency investigation 19 of 34

20 of 34 Performance investigation

21 of 34 Message rate scalability investigation

22 of 34 Message size scalability investigation

23 of 34 Access Distribution Look-ahead Caching  Broadcast-based request dissemination Pub-sub system for message broadcast Broadcast requests only to those servers that can answer No need to keep track of metadata locations Dynamic migration/replication [Rabinovich et al, 1999] Popular copies are moved/replicated where they wanted Autonomous decisions, self-awareness

24 of 34 Access Distribution Experiment Test Methodology T1T2T3 Time = T1 + T2 + T3 Simulation parameters Backup frequencyevery 10 seconds Message size2.7 Kbytes

Distribution experiment result  Overhead of access distribution is only few milliseconds.  Continuous access distribution operation does not degrade the performance.  The overhead of distribution remains the same regardless of the network distances between nodes. 25 of 34

26 of 34 T1T2T3 Time = T1 + T2 + T3 Simulation parameters message size / message rate2.7 Kbytes / 10 msg/sec replication decision frequencyevery 100 seconds deletion / replication threshold0.03 request/second and 0.18 request/second registry size1000 metadata in Indianapolis Dynamic Replication Performance Test Methodology

27 of 34  The decrease in average latency shows that the algorithm manages to move replica copies to where they wanted.

Replica content placement Consistency enforcement  Replica-content placement  Each node keeps information about other servers  Selection of Replica Server(s) Selection policy based on a) geographical (proximity) and b) topical (number of topics) information  Consistency Enforcement - Primary-copy approach  Update distribution: updates labeled with synchronized timestamps reflected (unicast) to primary-copy  Update propagation: primary-copy pushes (broadcast) updates only to those replica servers holding the context Hybrid Service 1 Hybrid Service 2 Hybrid Service 3 Hybrid Service 4 Hybrid Service 1 28 of 34

29 of 34 Fault-tolerance experiment Testing Setup Simulation parameters Backup frequencyevery 10 seconds Message size2.7 Kbytes T1T2T3 Time = T1 + T2 + T3

30 of 34 Fault-tolerance experiment result  Overhead of replica creation is only few milliseconds.  Continuous replica creation operation does not degrade the performance.  Overhead of replica creation increases in the order of milliseconds as the fault-tolerance level increase.

31 of 34 Consistency Enforcement Experiment Test Methodology T1T2T3 Time = T1 + T2 + T3 Simulation parameters Backup frequencyevery 10 seconds Message size2.7 Kbytes

32 of 34 Consistency Enforcement Test Result  Overhead of consistency enforcement is few milliseconds.  Continuous operation does not degrade the performance.  The cost of consistency enforcement remains the same regardless of distribution of the network nodes.

Conclusions 33 of 34  Efficient decentralized metadata strategies TupleSpaces & Pub-Sub communication paradigms Distribution Replication for fault-tolerance Replication for performance Consistency Enforcement  Efficient centralized metadata management strategies TupleSpaces Paradigm based memory-in storage

Contributions 34 of 34  Federated Grid Information Service Architecture Unified data model and communication protocol Support for both interaction independent and conversation- based service metadata Support for greater interoperability  Unified Grid Information Service Architecture Flexible and extendable architecture Support for High Performance and Fault-tolerance Uniform access to all kinds of service metadata  Efficient decentralized metadata systems can be built by integrating TupleSpaces and Publish-Subscribe paradigms Fault-tolerance, distribution and consistency can be succeeded with few milliseconds system processing overhead. Self-awareness can be achieved in decentralized metadata management.  Communication among services can be achieved with efficient mediator metadata strategies  A metadata management approach for Dynamic Web/Grid Service Collections  Collective operations such as queries on subsets of all available metadata in service conversation.

35 of 34 Information ServiceUsage Cases WS-ContextFast SOAP transfer in Mobile Computing (Sangyoon Oh Thesis) WS-Context Extended UDDI Geographical Information Service & Sensor Grids (Galip Aydin’s Thesis) WS-ContextSession Metadata Management (Hasan Bulut’s Thesis) WS-ContextFault-Tolerant Registry (Harshawardhan Gadgil’ s Thesis) WS-ContextVLab Project – Univ. of Minesota, Florida State University Extended UDDIChemical Informatics and Cyberinfrastructure Collaboratory Project WS-Context Extended UDDI Pattern Informatics – UC – Davis IEISS - LANL

Selected Publication List focusing on a) Metadata, b) Information Services, and c) Metadata Discovery 36 of 34  Mehmet S. Aktas, Geoffrey Fox, Marlon Pierce, Information Services for Dynamically Assembled Semantic Grids [SKG-05, 2005]  Mehmet S. Aktas, Geoffrey Fox, Marlon Pierce, Managing Dynamic Metadata as Context [ICCSE, 2005]  Mehmet S. Aktas et al., Web Service Information Systems and Applications [GGF-16, 2006]  Mehmet S. Aktas, Geoffrey C. Fox, Marlon Pierce, Fault Tolerant High Performance Information Services for Dynamic Collections of Grid and Web Services [FGCS Journal, 2006]  Mehmet S. Aktas, Sangyoon Oh, Geoffrey C. Fox, Marlon Pierce, XML Metadata Services [SKG-2006, Concurrency and Computation: Practice and Experience Journal-2007]  Mehmet S. Aktas, Marlon Pierce, and Geoffrey C.Fox, Designing Ontologies and Distributed Resource Discovery Services for an Earthquake Simulation Grid [ GGF11, 2004]  Mehmet S. Aktas, M. Pierce, G. Fox, and D. Leake, A Web based Conversational Case-Based Recommender System for Ontology aided Metadata Discovery [GRID Workshop -2004]  Sangyoon Oh, Mehmet S. Aktas, Geoffrey C. Fox, Marlon Pierce, Architecture for High-Performance Web Service Communications Using an Information Service [WSEAS Journal -2006]