WS-DREAM: A Distributed Reliability Assessment Mechanism for Web Services Zibin Zheng, Michael R. Lyu Department of Computer Science & Engineering The Chinese University of Hong Kong Hong Kong, China DSN 2008, Anchorage, Alaska, USA, 25 June, 2008

2 Outlines 1.Introduction 2.Design 3.Implementation 4.Experiments 5.Conclusion

3 1. Introduction Service-Oriented Architecture (SOA) is becoming popular. –Usually built using Web services. Reliability of the service-oriented applications becomes difficult to be guaranteed. –Remote Web services may contain faults. –Remote Web services may become unavailable. –The Internet environment is unpredictable. We need to know whether the target Web services are reliable or not before using them.

4 1. Introduction Performance of Web services is different from different user locations. Service-oriented applications may be deployed to different locations after developed. Distributed reliability assessment on Web services is necessary. Difficult. Time consuming. Expensive.

5 1. Introduction WS-DREAM: A Distributed REliability Assessment Mechanism for Web Services. – User-collaboration YouTube: sharing videos. Wikipedia: sharing knowledge. WS-DREAM: sharing assessment results of target Web services. –Obtain performance information of individual Web service from different locations for Web service selection and ranking. –Assess fault tolerance replication strategies.

6 2. Design 1. Assessment request 2. Load Applet 3. Create test cases 4. Test task scheduling 5. Client get test plans 6. Client run test plans 7. Send back results 8. Analyzing and return final results to client.

7 2. Design Fairness. Different Web Services should have fair chances to be assessed. Distribution. Web Services should be assessed by users in as many geography locations as possible. Feasibility. Task assignment should dynamically adjust to the frequently changed number of users and number of test plans. Efficiency. The algorithm should be efficient and it should not slow down the testing progress.

8 2. Design Identical and similar Web Services are becoming available in Internet  redundant replicas for fault tolerance  cheaper. Basic replication strategies. 1.Parallel. The application sends requests to different replicas at the same time. 2.Retry. The same Web Service will be tried one more time if it fails at first. 3.Recovery Block (RB). Another standby Web Service will be tried in sequence if the primary Web Service fails. ParallelRetryRB Parallel1.Parallel4.Parallel+Retry6. Parallel+RB Retry5.Retry+Parallel2.Retry8.Retry+RB RB7.RB+Parallel9.RB+Retry3.RB

9 2. Design 4. Parallel+Retry5. Retry+Parallel 6. Parallel+RB.7. RB+Parallel 8. Retry+RB 9. RB+Retry

10 2. Design Assess performance of different replication strategies. Includes several test cases. Created by WS-DREAM server and executed in the client-side. XML-based test plan design. 6. Parallel+RB. 7. RB+Parallel

11 3. Implementation JDK + Eclipse Client-side: –Java Applet Server-side: –an HTTP Web site (Apache HTTP Server) –a TestCaseGenerator (JDK6.0 + Axis library) –a TestCoodinator (Java Servlet + Tomcat 6.0) –a MySQL database (Record testing results)

12 4. Experiments A service user plans to employ several Web services in his commercial Web site. –Six identical Amazon book displaying and selling Web Service for fault tolerance purpose. (a-us, a-jp, a-de, a-ca, a-fr and a-uk) –A Global Weather Web Service to display currently weather information. –A GeoIP Web Service to get geography information of Website visitors.

13 4. Experiments Among all the 5443 failure cases –2986 failure cases are due to timeout (of larger than 10 seconds) –2456 failure cases are due to unavailable service (http code 503) –1 failure case is due to bad gateway (http code 502). 1. Assess the reliability of individual Web Services.

14 4. Experiments

15 4. Experiments Strategy 1 (Parallel) provides the best RTT performance. The sequential-type strategies (2:Retry, 3:RB, 8:Retry+RB, and 9:RB+Retry) can provide good RTT performance in the normal environment, however, their performances are not so good in the faulty environment. 2. Measure the performance of different replication strategies.

16 4. Experiments Two replicas are enough to provide high availability in the normal Internet environment, while three replicas are needed to ensure high availability in the 5% faulty Internet environment. 3. Determine the optimal number of replicas.

17 5. Conclusion and future work Conclusion  WS-DREAM Reliability assessment of individual Web services. Performance assessment of fault tolerance replication strategies.  Experiment More than 1,000,000 test plans. Users from five locations. Web Services located in six countries. Future work Assessment of stateful Web services. Enhancement of system feature in facilitating user test case contributions

WS-DREAM: A Distributed Reliability Assessment Mechanism for Web Services Zibin Zheng, Michael R. Lyu Department of Computer Science & Engineering The Chinese University of Hong Kong Hong Kong, China DSN 2008, Anchorage, Alaska, USA, 25 June, 2008