METEOR-S: investigations on semantics empowerment of processes Amit Sheth LSDIS LabLSDIS Lab, Dept of Computer Science, University of Georgia with the METEOR-S team; special thanks: Kunal Verma, Meena NagarajanMETEOR-S

Motivation Evolution of business needs drives IT innovation Initial focus on automation led to workflow technology The current and future needs include: –Aligning business goals and IT processes –Streamlining business processes –Having ability to quickly work with new partners –Creating adaptive process that react to changing conditions Three of the enablers for realizing these goals: –Interoperability (with Semantic Annotation) –Dynamic process configuration –Process Adaptation

Research RoadMap Semantic Web Technologies (OWL, SWRL, RDF) Web Services Standards (WSDL, UDDI, BPEL) Annotation/ Representation DiscoveryComposition Execution/ Adaptation Semantic Web Processes Self Configuration Self Healing Self Optimization Self Protection Semantic and Autonomic Web Processes Standards in WS and Semantic Web Our Current Focus: METEOR-S: Applying Semantics to Complete Web Process Lifecycle The Future

Outline Interoperability and WSDL-S –Work by Meena Nagarajan, Kunal Verma, with IBM Dynamic Process Configuration –Work by Kunal, Karthik Gomadam Process Adaptation –Work by Kunal, Prashant Doshi Some Results Conclusions

Interoperability and WSDL-S

Interoperability in Web services Impediments beyond semantic composition of Web services –Message level heterogeneities between communicating Web services

Message level Heterogeneities Syntactic - differences in the language used for representing the elements Model/Representational - differences in the underlying models (database, ontologies) or their representations (relational, object-oriented, RDF, OWL) Structural - differences in the types, structures of the elements Semantic - where the same real world entity is represented using different terms (or structures) or vice versa Resolved by the XML based environment WSDL-S; Semi-automatic solution

Matching Mapping A lot of early work on heterogeneous database integration is still quite useful

WSDL-S Offer an evolutionary and compatible upgrade of existing Web services standards Externalize the semantic domain models –agnostic to ontology representation languages –reuse of existing domain models –allows annotation using multiple ontologies (same or different domain) updating tools around WSDL is relatively easier

Semantic annotations on WSDL elements

Using WSDL-S to interoperate (...) (...) Address StreetAddress xsd:string OWL ontology hasCity hasStreetAddress hasZip WSDL complex type element 1.modelReference to establish a semantic association 2.schemaMapping to resolve structural heterogeneities beyond a semantic match semantic match

Using WSDL-S to interoperate Associate mappings using the 'schemaMapping' attribute on Web service message (input and output) elements. Use mappings as follows

Implementationusing AXIS2 Data mediation implemented as module+handler in Axis2 –Validation of our philosophical choice (reusing existing WS infrastructure) Heterogeneous Web service messages intercepted at AXIS and transformed to facilitate interoperation Nagrajan et al: Semantic Interoperability in Web services - Challenges and Experiences

WSDL-S - Use Cases and Standardization Activity International Bank Use Case Agriculture Produce Market Committee (APMC India Use Case) Bioinformatics Use Case

International Bank Use Case This bank is considering moving to SOA based architecture They feel WSDL has following shortcomings –Schema level Unable to define well known restrictions: , credit card number Unable to define detail description for enumerations: SPD for Summary Plan Description WSDL operation level pre-conditions and post-conditions of a service operation restrictions on elements / complexTypes that are operation specific (e.g. customerId in CustomerType must be null for AddCustomer; but it's mandatory for GetCustomer)

Use Case Details A search service is defined to search by either personal name or commercial name. The search engine would return at least one element of names, or a SOAP fault

Adding Contracts to WSDL In the use case, it’s expressed as the following: –A name has to be provided. –If it’s a personal name, either last name or personal name must exist. –If it’s a commercial name, either corporate name or stock ticker must exist. –Either at least one or no more than 100 names would be returned, or an error “not found” will occur.

Some of the Preconditions These conditions can be represented as preconditions in WSDL-S

Current Agri-Marketing scenario in India Seller: Farmer Agriculture Produce Market Committee Brokers associated With APMC Buyers

A farmer can sell his produce to either Agriculture Produce Market Committees or Brokers associated with APMC’s. APMC’s sell the produce either by retail or in open markets. Research is underway in creating SOA based architectures to realize the buyer seller interactions as services. Current Agri-Marketing scenario in India

Farmers use kiosks to interact with the buyer services Farmers need to locate the right APMC for their products –Some APMC’s may not have refrigeration making them unsuitable for fresh vegetables, diary products etc. –Farmers might want to get paid in cash the same day whilst some APMC’s may not be willing to do so. Farmers use the web based interface to then sell their produce to the APMC. Current Agri-Marketing scenario in India

Why WSDL-S? Uses semantics to provide richer descriptions of the services offered by APMC’s. –An APMC buys wheat, potatoes, fresh meat and dairy products. The APMC can use WSDL-S to represent this information in his service description. –Allows for capturing policies such as “Refrigeration is free for 2 business days” or “Same day payment will be issued in cash” Various APMC’s have varying data definitions. It is hard to create a client that can interoperate, due to heterogeneities that are present. WSDL-S help address them by mediation.

Using WSDL-S in Bioinformatics ProPreO - Experimental Proteomics Process Ontology (CCRC / LSDIS) data sequence peptide_sequence Excerpt: ProPreO – process ontology <wsdl:definitions targetNamespace="urn:ngp" …… xmlns:wssem=" xmlns:ProPreO=" > <schema targetNamespace="urn:ngp" xmlns=" …… Excerpt: Bio-informatics Web service WSDLS CCRC – Complex Carbohydrate Research Center ProPreO -

Dynamic Process Configuration

Sample Supply Chain Scenario Consider a simplified supply chain process of a computer manufacturer –Most parts are multiple sourced Overseas goods cheaper but greater lead times than internal/local suppliers –Need to deal with part compatibility constraints Choosing a certain motherboard restricts choices of RAMs, processors –Must respect relationship with preferred suppliers Suppliers characterized as preferred or secondary –Sometimes important to maintain production schedule in the presence of delayed orders

Dynamic Process Configuration Dynamic configuration Problem Find optimal partners for the process based on process run time– cost, supply time, etc. Conceptual Approach 1.Create framework to capture & represent domain knowledge 2.Represent constraints on the domain knowledge 3.Ability to reason on the constraints and configure the process Proposed Solution 1.Use of ontologies to represent domain knowledge 2.Use semantic knowledge captured in ontologies across the process lifecycle 3.A multi-paradigm constraint analysis based approach to handle quantitative and logical constraints

Dynamic Process Configuration –Finding optimal partners for a process based on service and process constraints Research Challenges –Capturing functional and non-functional requirements of the Web process (Abstract process specification) –Discovering service partners based on functional requirements (Semantic Web service discovery) –Choosing optimal partners that satisfy non-functional requirements (Constraint Analysis)

Abstract Process Specification Specify process control flow by using virtual partners Capture Functional Requirements of Services using Semantic Templates Specify Process Constraints

Abstract Process Specification 1.Specify process control flow by using virtual partners 2.Capture Functional Requirements of Services using Semantic Templates 3.Specify Process Constraints Semantic Templates capture the functionality of a Web service with the help of ontologies/other domain models Find a service that sells RAM in Athens, GA. It must allow the user to return and cancel, if needed The template can also have non-functional (QoS) requirements such as response time, security, etc.

Semantic Templates Sample Semantic Template Service Level MetaData IndustryCategory = NAICS:Electronics ProductCategory = DUNS:RAM Location = Athens, GA Semantically Defined Operations Operation1 = Rosetta#requestPurchaseOrder Input = Rosetta#PurchaseOrderDetails Output = Rosetta#PurchaseConfirmation ResponseTime < 5s Operation2 = Rosetta#CancelOrder … Operation3 = Rosetta#ReturnProduct ….. Part of Rosetta Net Ontology Data Semantics Functional Semantics Non-Functional Semantics * WSDL-S is used to capture semantic templates Semantic Templates capture the functionality of a Web service with the help of ontologies/other domain models Find a service that sells RAM in Athens, GA. It must allow the user to return and cancel, if needed The template can also have non- functional (QoS) requirements such as response time, security, etc.

Abstract Process Specification 1.Specify process control flow by using virtual partners 2.Capture Functional Requirements of Services using Semantic Templates 3.Specify Process Constraints

Process Constraints Constraints can be specified at an appropriate level: an activity (operation of a partner), a partner, or the process as a whole. An objective function can also be specified e.g., minimize cost and supply-time, etc. Two types of constraints: –Quantitative (Q) (Time < 5 sec) –Logical (L) (preferredPartner, Security, etc.)

Process Constraints TrueSatisfyPartner 1PreferredSupplier(P1) (Logical) TrueSatisfyProcessCompatible (P1, P2) (Logical) Activity Process Scope ΣDollars<1000SatisfyCost (Quantitative) MAXDays< 7SatisfySupplytime (Quantitative) ΣDollarsMinimizeCost (Quantitative) AggregationUnitValueGoalFeature

Constraint Analysis Multi-paradigm approach used –ILP for quantitative constraints –SWRL for logical constraints Discovered Services first given to ILP solver –It returns ranked sets of services Then each set is checked for logical constraints using a SWRL reasoner –Sets not satisfying the criteria are rejected Verma et al: Semantics-enabled Configuration of Web Processes

Configuration Architecture

Semantic Discovery Finds actual services matching semantic templates Implemented as a layer over UDDI Current implementation based on ontological representation of operations, inputs and outputs. Returns ranked of services for each semantic template Builds upon following previous discovery implementations –Extends matching presented in [1] to consider operations and service level metadata –Extends the approach presented “WSDL to UDDI Mapping” [2] to support operation level discovery [1] M. Paolucci, T. Kawamura, T. Payne and K. Sycara, Semantic Matching of Web Services Capabilities, ISWC [2] Using WSDL in a UDDI Registry, Version Technical Note, spec/doc/tn/uddi-spec-tc-tn-wsdl-v pdf

Quantitative Constraint Analysis A service is used (1) or not used (0) –Create a binary variable x ij for each candidate service. Set up constraints such that one service is chosen for each activity. –N(i) is the number of candidate services of activity ‘i’ and M is the number of activities.

Quantitative Constraint Analysis Set the cost constraint on activity 1 Set the supply time constraint Set up the objective function

Logical Constraint Analysis Use a SWRL reasoner to perform logical constraint analysis Domain knowledge is captured as ontologies Rules are created with the help of the knowledge in the ontology Implemented using IBM’s ABLE and SNOBASE –SNOBASE stores OWL ontologies using ABLE Rule Language (ARL) –Our implementation is based on SWRL rules written in ARL

Domain Ontology

Rules Supplier 1 should be a preferred supplier. –“if S1 is a supplier and its supplier status is preferred then the S1 is a preferred supplier”. Supplier (?S1) and partnerStatus (?S1, “preferred”) => preferredSupplier (?S1) Supplier 1 and supplier 2 should be compatible. –if S1 and S2 are suppliers and they supply parts P1 and P2, respectively, and the parts work with each other, then suppliers S1 and S2 are compatible for parts P1 and P2. Supplier (?S1) and supplies (?S1, ?P1) and Supplier (?S2) and supplies (?S2, ?P2) and worksWith (?P1, ?P2) => compatible (?S1, ?S2, ?P1, ?P2)

Configuration Example Discovery Results After ILP After SWRL

Implementation Details Entities –Process Manager (PM) Responsible for global process configuration –Service Manager (SM) Responsible for interaction of process with service –Configuration Module (CM) Discovery and constraint analysis Infrastructure –Implemented as modules in Axis 2.0 Phases –Pre-binding Number of services bound to same service manager. Used for information gathering for constraint analysis –Binding Constraint Analysis and binding optimal partner to each SM –Post-binding Normal process execution with optimal partner

Runtime Configuration Example

Incorporating Configuration support in Axis 2.0

Process Adaptation

Adaptation Problem Optimally react to events like delays in ordered goods Conceptual Approach 1.Maintain state of the process 2.Capture costs while transitioning from abnormal states to goal state(s) 3.Ability to decide optimal actions on the basis of state Proposed Solution 1.Use of stochastic decision making framework to deduce optimal actions

Process Adaptation Ability to adapt the processes from failures, unexpected events Two kinds of failures –Failures of physical components like services, processes, network Can replace services using dynamic configuration –Logical failures like violation of SLA constraints/Agreements such as Delay in delivery, partial fulfillment of order Need additional decision making capabilities

Revisiting the Supply Chain Scenario After order for the parts are placed, they may get delayed The manufacturer may have severe costs (losses) if assembly is halted. –It must evaluate whether it is cheaper to cancel/return and reorder or take the penalty of delay –Caveat: possible that reordered goods may be delayed too

Process Adaptation Research Challenges –Creating a model to recover from failures and handle future events –Model must deal with two important factors Uncertainty about when a failure occurs Cost based recovery Proposed Solution –Use a stochastic decision making framework to deal with such failures Currently we use Markov Decision Processes

State Based Representation (costs not shown) S1 - Ordered = True (All other flags false) S4 - Ordered = True and Received = false S5 - Ordered = True and Delayed = false ---Transition due to action - - Exogenous events (example probabilities of occurrences of events conditioned on the states)

Generating States using preconditions and effects Actions Events Flags Use an algorithm similar to reachability analysis to generate states

Handling Inter-Service dependencies Since the RAM and Motherboard must be compatible, the actions of service managers (SMs) must be coordinated For example, if MB delivery is delayed, and MB SM wants to cancel order and change supplier, the RAM SM must do the same Hence, coordination must be introduced in SM- MDPs K. Verma, P. Doshi, K. Gomadam, J. Miller, A. Sheth, Optimal Adaptation in Autonomic Web Processes with Inter-Service Dependencies, LSDIS Lab, Technical Report, November 2005

Centralized Approach State space created by Cartesian product of transition diagrams Joint actions from each state Transition probability created by multiplying states Costs created by adding cost per action from each state –Compatible actions given rewards –Incompatible actions given penalties Optimal but exponential with number of manager

Decentralized Approach Simple coordination mechanism If one service manager changes suppliers –All dependent managers must change suppliers Low complexity but sub-optimal

Hybrid Approach If the policy of some SM dictates it to change suppliers, the following actions happen: – it sends a coordinate request to PM – PM gets the current cost of changing suppliers or current optimal action by polling all SMs It takes the cheapest action (change supplier or continue) A bit like decentralized voting- will change suppliers if majority are delayed It mirrors performance of centralized approach and has complexity like the decentralized approach

Empirical Evaluation

Evaluating Dynamic Configuration Evaluation with help of the supply chain scenario Use the variations in currency exchange rates of China and Taiwan as the primary factor affecting supplier prices Assume that process is dynamically configured every fortnight

Variations in Chinese and Taiwanese Currency Source for graphs and data:

Observations Static binding –Configured at the first run and same partners are persisted with for all subsequent runs –Cost changes due to variations in currency Dynamic binding –Dynamically configured with latest prices for all runs –With just ILP (DB-ILP) Always the lowest cost, logical constraints not guaranteed –With ILP and SWRL (DB-ILP+SWRL) Lowest cost for partners satisfying all constraints

Results – Process Configuration

Evaluating Process Adaptation Evaluation with the help of the supply chain scenario Two main parameters used for the evaluation –Probability of Delay – (probability that an item ordered from a supplier will be delayed) –Penalty of Delay – (cost for the manufacturer for not reacting to delay) Total process cost = $1000 and cost of changing suppliers (CS) =$200

Evaluating Adaptation KEY M-MDP: Centralized Random: Random process (changes suppliers for 50% of delays) Hyb. Com: Hybrid MDP-Com: Decentralized

Observations Results –For Penalty = 200 (cost of CS = cost of delay), MDP always waits –For Penalty = 300, 400 (cost of CS < cost of delay), MDP changes at lower prob., waits at higher prob. Conclusions –Thus MDP makes intelligent decisions and outperforms random process that changes suppliers 50% of the time it is delayed –Centralized MDP performs the best, followed by Hybrid MDP

Conclusions, Related Work and Future Agenda

Summary - Dynamic Process Configuration Allows optimal selection of partners for a process –The optimal process is not always the cheapest process as the domain constraints must also be respected Processes can be configured to handle the following cases: –In volatile environments where changes may render older configurations sub-optimal (e.g. changes in currency exchange rates, new discounts by suppliers) –When the constraints change –Can replace services in case of physical failures by caching results from configuration module

Summary - Adaptation Adaptation is need to handle logical failures and events Whether to adapt or not depends on the cost of the failure –For this evaluation it was the cost of the delay Intelligent decision making can reduce costs for adaptation

Related Work Semantic Web Services –OWL-S, WSMO, FLOWS Quality driven composition [1] –Uses ILP for optimizing processes –Our work uses a multi-paradigm approach to considering a broader set of constraints Support in Websphere [2] and Oracle BPEL Engine for runtime binding. –Based on replacing services with same interfaces. Service selection is not the focus –Our focus is on finding optimal services based on process constraints Automated workflow composition –Plethora of work based on automatically generating processes based on high level goals. [3] –Our focus is on configuring pre-existing processes. [1] L. Zeng, B. Benatallah, M. Dumas, J. Kalagnanam, Q. Sheng: Quality driven Web services composition, WWW 2003 [2] Dynamic service binding with WebSphere Process Choreographer, ibm.com/developerworks/webservices/library/ws-dbind/ 128.ibm.com/developerworks/webservices/library/ws-dbind/ [3] J. Rao and X. Su. "A Survey of Automated Web Service Composition Methods". SWSWPC 2004.

Related work Focus on correctness of changes to control flow structure –Adept[1], Workflow inheritance [2], METEOR Use of ECA rules [3] to automatically make changes We extend previous work in this area –Cost based adaptation –Coordination Constraints [1] M. Reichert and P. Dadam. Adeptflex-supporting dynamic changes of workflows without losing control. Journal of Intelligent Information Systems, 10(2):93–129, 1998 [2] W. van der Aalst and T. Basten. Inheritance of workflows: an approach to tackling problems related to change. Theoretical Computer Science, 270(1-2):125–203, [3] R. Muller, U. Greiner, and E. Rahm. Agentwork: a workflow system supporting rule-based workflow adaptation. Journal of Data and Knowledge Engineering, 51(2):223–256, 2004.

Conclusions Provided a framework for configuration and adaptation of Semantic Web Processes Contributions –Proposed WSDL-S which is now an important input to two W3C charters for Semantic Web Services Specifications –Handled process configuration with both quantitative and logical constraints using a multi-paradigm approach Typical real world use cases require handling both –Studied the utility of Markov Decision Processes for optimal adaptation of Web processes Future Directions –Autonomic Web Processes –Applying Semantics to Service Sciences –Semantics and Lightweight services – AJAX, REST

Publications Dynamic Process Configuration –K. Verma, R. Akkiraju, R. Goodwin, P. Doshi, J. Lee, On Accommodating Inter Service Dependencies in Web Process Flow Composition, Proceedings of the AAAI Spring Symposium on Semantic Web Services, March, 2004, pp –R. Aggarwal, K. Verma, J. A. Miller, Constraint Driven Composition in METEOR-S, SCC –K. Verma, K.Gomadam, J. Miller and A. Sheth, Configuration and Execution of Dynamic Web Processes, LSDIS Lab Technical Report, Adaptation –K. Verma, A. Sheth, Autonomic Web Processes, ICSOC 2005 –K. Verma, P. Doshi, K. Gomadam, A. Sheth, J. Miller, Optimal Adaptation of Web Processes with Co-ordination Constraints, LSDIS Lab Technical Report, Semantic Policy/SLA Representation and Matching –K. Verma, R. Akkiraju, R. Goodwin, Semantic Matching of Web Service Policies SDWP 2005 & Filed Patent –N. Oldham, K. Verma, A. Sheth, Semantic WS-Agreement Based Partner Selection, WWW 2006

Publications Semantic Web Service Discovery –K. Verma, K. Sivashanmugam, A. Sheth, A. Patil, S. Oundhakar and John Miller, METEOR-S WSDI: A Scalable Infrastructure of Registries for Semantic Publication and Discovery of Web Services, JITM –K. Sivashanmugam, K. Verma, A. Sheth, Discovery of Web Services in a Federated Registry Environment, ICWS04 Semantic Annotation/Representation –Rama Akkiraju, Joel Farrell, John Miller, Meenakshi Nagarajan, Amit Sheth, and Kunal Verma, Web Service Semantics, WSDL-S W3C Member Submission –K. Sivashanmugam, Kunal Verma, Amit Sheth, John A. Miller, Adding Semantic to Web Service Standards, ICWS Semantic Web Composition –K. Sivashanmugam, J. Miller, A. Sheth, and K. Verma, Framework for Semantic Web Process Composition, International Journal of Electronic Commerce, Winter , Vol. 9(2) pp

Resources METEOR-S (also tools and downloads): WSDL-S: Publications/Resources: