1. Research @ UC Marco Vieira University of Coimbra mvieira@dei.uc.pt

2. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[1] ::.. Outline  Fault Injection for Failure Prediction Methods Validation  Robustness Testing for Web Services  Other Research Topics  Questions & Comments

3. Fault Injection for Failure Prediction Methods Validation

4. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[3] ::.. Why do computers fail?

5. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[4] ::.. Hardware problems

6. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[5] ::.. Environment problems

7. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[6] ::.. Bad configuration

8. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[7] ::.. Misuse

9. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[8] ::.. Not proven design

10. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[9] ::.. But most of the times it is due to…

12. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[11] ::.. What can we do?  Don’t use computers Sooner or later they will fail!!!  Or: Build better software Many tried… most have failed Find ways to identify failure-prone situations And react accordingly…  But… How do we know when a failure is about to happen?

13. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[12] ::.. Predict!

14. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[13] ::.. But prediction is hard…  Prophets do not exist!  Failure prediction methods are complex  Needs lots of data  How to improve this? That is the goal of our research…

15. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[14] ::.. Our idea Injection of realistic software faults to validate and improve failure prediction methods

16. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[15] ::.. What is failure prediction?  The goal is to identify failure-prone situations i.e. situations that will probably evolve to a failure  The output of online failure prediction can either be: A decision that a failure is imminent or not Some continuous measure evaluating the current situation as failure prone  Short-term predictions are made on the basis of runtime monitoring

17. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[16] ::.. Using failure prediction  Steps for building a high availability system based on failure prediction: 1. Observe 2. Reason 3. React

18. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[17] ::.. Research goals  Use the injection of realistic software faults to validate and improve failure prediction mechanisms  Four scenarios: 1. Help identifying symptoms for failure prediction 2. Accelerate the learning/training phase of prediction algorithms 3. Evaluate the figures of merit of prediction algorithms 4. Integrate fault injection in the prediction algorithm, as a form of continuous training

19. Robustness Testing for Web Services

20. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[19] ::.. The problem Web Services must provide a robust service to the client applications  Development tools lack mechanisms to: Characterize the robustness of Web Services code Compare robustness of alternative Web Services

21. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[20] ::.. Web Services robustness testing  Erroneous Web Services call parameters Generated using a set of predefined rules Based on the data types of each parameter Injected during the Web Services execution GetWeather(city, day) → GetWeather(“Coimbra”, null)  Key components needed: Workload Robustness tests Failure modes classification

22. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[21] ::.. Preparing the tests  Obtain web service definitions List of operations Parameters Data types Domains  The WSDL file is processed automatically to obtain the required information  The domain for each parameter cannot be deduced from the WSDL description Must be provided by the user

23. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[22] ::.. Workload  A workload is needed to exercise each operation of the web service  A generic workload that fits all Web Services is not feasible We need to generate a workload for each web service  Workload generation: User defined workload Random workload

24. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[23] ::.. Parameters values mutation TypeParameter Mutation String Replace by null value Replace by empty string Replace by predefined string Replace by string with nonprintable characters Add nonprintable characters to the string Replace by alphanumeric string Add characters to overflow max size Number… List… Date… Boolean…

25. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[24] ::.. Experimental evaluation  Tested 100 Web Services publicly available The majority is listed at http://xmethods.net/  Web Services owned by different parties E.g., Microsoft and Xara  Some services implement the same functionally e.g., Text Disguise and Free Captcha Service

26. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[25] ::.. Some details on the results…  About 35% of the Web Services tested presented abort failures 30% null references 30% SQL problems 13% conversion problems 7% arithmetic problems 20% others  No Silent failures were observed  Most of the services assume non-null input parameters

27. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[26] ::.. The wsrbench tool…  Implements the Web Services testing approach  Available online: http://wsrbench.dei.uc.pt

28. Other Research Topics

29. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[28] ::.. Security  Vulnerability & attack injection Validation of security mechanisms  Tools for developing non-vulnerable web services Penetration testing Static analysis Anomaly detection  Security benchmarking for transactional systems Comparing systems in terms of security features

30. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[29] ::.. Dynamic systems  Resilience benchmarking for self-adaptive systems Accessing resilience-related metrics Compare systems with autonomic capabilities  V&V of large-scale, dynamic service systems Support traceability to evolving requirements Cope with agile software development process Explore the notion of regression in V&V Cope withsuccessive software releases Dynamic and evolving system V&V

31. COIMBRA, 10TH MARCH 2010 RESEARCH @ UC[30] ::.. Questions & Comments Marco Vieira Center for Informatics and Systems University of Coimbra mvieira@dei.uc.pt