SIMULATING ERRORS IN WEB SERVICES International Journal of Simulation: Systems, Sciences and Technology 2004 Nik Looker, Malcolm Munro and Jie Xu

Outline Introduction Quality of Service Fault Injection Fault Model Test Case Conclusion and Future Work

Introduction Web Service technology is a key factor in the success of any e-Science or e-Commerce project. High Quality of Service (QoS) to allow the production of highly reliable systems. Fault injection is a well-proven method of assessing the dependability of a system

Introduction This fault injection method is a modified version of Network Level Fault Injection. This method differs from standard Network Level Fault Injection techniques Fault injector decodes the SOAP messages and can inject meaningful faults into a message This enables API-level parameter value modification to be performed as well as standard Network Level Fault Injection.

Quality of Service Availability  the quality aspect of whether a Web Service is present and ready for use. Accessibility  the quality aspect that represents the degree the Web Service is capable of servicing a request. Integrity  the quality of the Web Service maintaining the correctness of any interaction. Performance  the quality aspect that is defined in terms of the throughput of a Web Service and the latency.

Quality of Service Reliability  the quality aspect that represents the capability of maintaining the service and service quality. Regulatory  the quality aspect that the service corresponds to rules, laws, standards and specifications. Security  the quality aspect that defines confidentiality for parties using a service

Fault Injection Techniques Network Level Fault Injection  Corruption, loss or reordering of network messages at the network interface.  Runtime injection technique that injects faults into captured network packets.  Software Implemented Fault Injection (SWIFI) instrumenting the operating system protocol stack Perturbation  Modify program states without mutating existing code statements.  testing fault tolerance mechanisms

Fault Injection Techniques Corruption of Memory Space  Platform Specific Code Mutation  Certification Problems because code changed Syscall Interposition  Platform Specific

Fault Injection Tools Ferrari (Fault and Error Automatic Real-Time Injection)  uses a system based around software traps to inject errors into a system.  traps are activated by either a call to a specific memory location or a timeout. FTAPE (Fault Tolerance and Performance Evaluator)  inject faults into memory, registers and disk accesses

Fault Injection Tools Xception  processor’s exception handling capabilities are used to trigger fault injection. DOCTOR (IntegrateD sOftware Fault InjeCTiOn EnviRonment)  allows injection of memory and register faults, and network communication faults.  combination of timeout, trap and code modification.

Fault Injection Tools Orchestra  a script driven fault injector which is based around Network Level Fault Injection.

WS-FIT Web Service – Fault Injection Technology Method and tools implement mechanism of fault injection and also provide a framework for the creation and execution of test cases Fault injector that allows network level fault injection Dependability Assessment of SOAP based SOA  SOAP Based  WSDL Defined Interfaces  Combined with conventional specification

Fault Model Fault Types that can effect a SOAP based system:  Physical Faults Effecting memory or processor registers  Software Faults Programming errors and design errors  Resource Management Faults Memory leakage and exhaustion of resource  Communication Faults Message deletion, duplication, delay, reordering or corruption  Life-Cycle Faults Premature object destruction through starvation of keep-alive messages and delayed asynchronous responses

Enhanced Fault Model

Failure Modes Crash of a service Crash of a hosting web server (or the host itself) Hang of a service Corruption of input data Corruption of output data Duplication of messages Omission of messages Delay of messages

Why WS-FIT testing? Particularly interested in  Assessing Quality of Service  Assessing Fault Tolerance of Systems  Developing Methods and Tools So that we can  Detect defects in platform code and design  Gather dependability metrics on platforms to allow comparisons  Use WS-FIT to test individual Web Services

WS-FIT Consists of 3 parts:  Hook code in SOAP Library  Fault Injector Framework  Script Trigger Injection Results

SOAP API Hooks  One for sending messages.  One for receiving messages. Hooks consisted of simple socket code  To pass messages to fault injector.  Receive (possibly) modified message from injector.

Instrumenting a Service

What the Framework Provides Logging Function (XML format for easy analysis)  Detects faults and logs faults  Logs injected fault packets Trigger and Injection mechanism:  Trigger on Message Type and parameter.  Inject Fault through User Script. Implemented  Framework implemented in Java for portability  Scripts implemented in Python Executed in the JVM by Jython

User Script Result class  One instance of this for duration of test.  Used to store static results.  Currently only tag count but will be enhanced in later experiments. Trigger Class  One instance per packet.  Returns Boolean value to indicate if injector class should be run.  Use data from result class to determine if the correct point in message stream has been reached. Injector Class  One instance per packet  Inject fault into packet

WS-FIT

GUI The GUI provides  An easy to use environment to create and execute tests  Ability to create a skeleton test script from WSDL definitions  Populate tests through user defined Fault Model  Execution of test scripts from tool  Real-time visualization of RPC message parameters

Quality of Service Experiment to demonstrate WS-FIT can be used to assess QoS criteria WS-FIT will be used to modify the performance of a system by injecting a latency (without code modification being required)

Test Scenario The test system implements a simple, self regulating, heater unit. The hardware used is simple and relies on the software drivers to prevent failure states being reached. Under normal operation the ‘Unit’ will give a logarithmic temperature rise so the Controller uses timers to modify to a linear temperature rise. Will be used in a chemical experiment  A substance must be heated to 60°C  Must reach 60ºC in a specified period of time  Must have a linear temperature rise The test will introduce a latency to decrease the throughput and hence the performance of the system.

Test System

Natural Operation of Heater

Model

Applying a Fault Model to a Parameter

Injecting Latency

Summary WS-FIT can be effectively used to:  Introduce SOAP protocol faults Introduce both random and non-random protocol faults  Assess Quality of Service Such as throughput modification WS-FIT  Relatively non-invasive  Can be used to assess both middleware and individual services  Gives real-time feedback through visualization of RPC parameters