1. Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer Science & Engineering The Chinese University of Hong Kong Hong Kong, China ICWS 2010, Miami, Florida, USA July 05 - 10, 2010

2. 2 Outlines Introduction Distributed Evaluation of Web Services How to Use the Datasets Conclusion and Future Work

3. 1. Introduction

4. 4 4 Introduction Web applications are becoming more and more important!

5. 5 5 Introduction The age of Web 2.0 –Web pages and Web services Web services are Web APIs that can be accessed over a network and executed on remote systems. –Open standards –Interoperability

6. 6 Quality-of-Service Quality-of-Service (QoS): Non-functional performance. –User-independent QoS properties. price, popularity. No need for evaluation –User-dependent QoS properties. failure probability, response time, throughput. Different users receive different performance.

7. 7 QoS-Driven Approaches Web service selection Web service composition Fault tolerant Web services Web service ranking Web service recommendation Limited real-world Web service QoS datasets for experimental studies

8. 8 Real-world Web Service Evaluation Obtain 21,358 publicly available Web services from the Internet. WS invocation and evaluation: –235,262,555 lines of Java codes. Two large-scale distributed evaluations are conducted and first hand experiences are provided. –Dataset 1: 150 users * 100 Web services –Dataset 2: 339 users * 5825 Web services

9. 2. Distributed Evaluation of Web services

10. 10 Location Information 21,358 Web services from 89 countries. The top 3 countries provide 55.5% of the obtained Web services. –United States: 8867 Web services, –United Kingdom: 1657 Web services, –Germany: 1246 Web services

11. 11 Obtaining Web Service Address Web service portals or directories – xmethods.net, –webservicex.net, –webservicelist.com, Web service searching engines –seekda.com, –esynaps.com, Obtain 21,358 addresses of WSDL files

12. 12 WSDL File Infomation

13. 13 Java Code Generation Axis 2 to generate Java codes for 13,108 Web services. Totally 235,262,555 lines of Java codes are produced.

14. 14 Dataset 1: Failure Probability

15. 15 Overall Performance Mean of failure probability Standard deviation of failure probability

16. 16 Overall Performance Average failure probabilities of all of the 100 Web services and all the 150 service users are larger than 0. The standard deviation first becomes larger with the increase of mean and begins to decrease after a certain threshold. The standard deviations are large, indicating that performance of the same Web service observed by different service users can vary widely.

17. 17 Failure Types (1) Web service invocations can fail easily. (2) Providing reliable Web services is not enough for building reliable service- oriented system. (3) The Web service invocation failures are unavoidable in the unpredictable Internet environment; therefore, service fault tolerance approaches are becoming important. (4) Service fault tolerance mechanisms should be developed at the client-side.

18. 18 Dataset 2: Response-time & Throughput Each service user makes one invocation on each Web services.

19. 19 Overall Response-Time Web services with poor average response time performance tend to have large performance variance to different users. Large response time of a Web service can be caused by the long data transferring time or the long request processing time at the server-side. Influenced by the client-side network conditions, different service users observe quite different average response time performance on the same Web services.

20. 20 Overall Throughput Influenced by the poor server-side network conditions, there is a small part of Web services providing very poor average throughput performance (<1 kbps). Service users with large average throughput values are more likely to observe large throughput performance variance on different Web services.

21. 3. How to Use the Datasets?

22. 22 Web Service Recommendation Zibin Zheng, Hao Ma, Michael R. Lyu, Irwin King, “WSRec: A Collaborative Filtering based Web Service Recommender System”, ICWS2009.

23. 23 Fault Tolerant Web Services Zibin Zheng and Michael Lyu, “A QoS-Aware Fault Tolerant Middleware for Dependable Service Composition ”, DSN2009. Global constraint: Success-rate of the whole service plan > 99%. Stateless Web services Stateful Web services

24. 24 More Research on the Datasets Web service selection. Web service search. Web service ranking. Other QoS-driven approaches of Web services.

25. 4. Conclusion and Future Work

26. 26 Conclusion and Future Work Conclusion  Distributed evaluation of Web services  Dataset 1: 150 * 100, failure probability  Dataset 2: 339 * 5825, response-time and throughput Future work  Investigating more QoS properties  Incentive mechanisms for collecting user data  Evaluating more Web services from more locations

27. Thank you! Web service QoS datasets: http://www.wsdream.nethttp://www.wsdream.net