Testing AJAX functionality with UniTESK Yevgeny Gerlits, a postgraduate student from Lomonosov Moscow State University SYRCoSE 2010

2 AJAX vs. Classic web application Client side (browser) Server side HTTP response (HTML, CSS…) User interface event User interface Web server Back end User interface AJAX engine Web server Back end HTTP request Asynchronous HTTP request HTTP response (XML, JSON…) DOM change ClassicAJAX

3 Object of testing function onEventHandler() { } function callback() { } Client side (browser) Server side User interface AJAX engine Web server Back end Possible results of JavaScript execution: client side state changes including DOM, global JavaScript variables, and Cookies; page reloads; asynchronous HTTP requests; …

4 A problem: AJAX engine itself may initiate JavaScript processing JavaScript functions to create delayed actions: setTimeout (script, timeout); setInterval (script, timeout); clearTimeout (script, timeout); clearInterval (script, timeout) Client side (browser) Server side User interface AJAX engine Web server Back end

5 Technique for testing AJAX engine UniTESK – a general purpose, model based test development technology Test suite for systems with synchronous interfaces: 1. The next stimulus may be only after the reaction to the previous one. 2. SUT may not initiate interaction with its environment. Test suite for systems with asynchronous interfaces: 1. SUT may simultaneously interact with a couple of systems. 2. SUT may initiate interaction with its environment.

6 UniTESK test suite architecture for systems with asynchronous interfaces Formal descriptions:  Specification Test scenario Mediator   Components in the target programming language: Catcher  Model state Hyper oracle State mediator Test scenario Interaction register Action mediator Application under test Catcher Oracle

7 Elaboration of the specification for AJAX engine 1. Represent possible interactions of AJAX engine with its environment as a set of atomic stimuli and reactions. 2. Extract functional requirements to the behavior of AJAX engine. 3. Represent the requirements in the form of pre- conditions and post- conditions of stimuli and reactions, and model state invariants. Web server User interface AJAX engine Catcher Browser 1 Proxy server User interface AJAX engine Browser N Stimulus Reaction Detects reactions Action mediator

8 UniTESK tests for AJAX engine A test suite (covers the requirements in specification by traversing a FSM): A single test (Test-1): 1. mediator and proxy server apply a set of stimuli; catcher detects the reactions and saves their results; 2. hyper oracle verifies the results of the stimuli and reactions. Application states: Appropriate model states: Model-State-1 post(R 2 ) pre(S 1 )post(S 1 ), pre(R 1 )post(R 1 ), pre(R 2 ) Model-State-4 S1S1 R1R1 R2R2 App-State-1App-State-4 Test-2Test-1 Test-5 Test-N f (App-State-1, Model-State-1) f (App-State-4, Model-State-4) Test-3 Test-4 Test-6

9 Approbation process 1. Collect 8 AJAX design patterns. 2. Choose a pattern and implement it in a simple AJAX application. 3. Develop a test system for the application using UniTESK. 4. Assess the fault revealing capability of the test system: a. Inject a single fault into the AJAX engine of the application. b. Run the test system. c. See if the fault has been revealed. 5 times 8 times

10 AJAX design patterns and their implementations An AJAX design pattern: How to design an AJAX application so that it would behave in a certain way or a particular interactivity effect could be achieved? Pattern Predictive Fetch: Problem: How can you make an AJAX application respond quickly to user activity? Solution: Have the application anticipate likely user actions and call the server in preparation. Implementation: A multi-page article  12N

11 Injected faults Fault examples: removal of user interface event handlers; building incorrect HTTP requests; changing unique identifiers of HTML elements; removal of an HTTP request object; faults that result to the wrong processing of data in callback functions; … Characteristics of the injection process: Faults are injected at the client side of AJAX applications. There is no special fault injection procedure.

12 Test results AXAX application forIntroducedRevealed% 1. Explicit Submission55100% 2. Periodic Refresh5480% 3. Submission Throttling5480% 4. Predictive Fetch55100% 5. Browser-side cache55100% 6. Guesstimate5360% 7. Pseudo-threading5480% 8. Multi-stage download5480% TOTAL403485%

13 Conclusion and future work An approach to testing the client side of AJAX applications (AJAX engine) with UniTESK. Future work: enhance the automation level of the UniTESK approach; apply the UniTESK approach to a couple of AJAX applications available on WWW; design a new approach for testing both the client side and the server side of AJAX applications

14 Thank you!

15 References I. Bourdonov, A. Kossatchev, V. Kuliamin, and A. Petrenko, “UniTesK test suite architecture,” Proc. FME LNCS 2391, Springer-Verlag, 2002, pp I. Bourdonov, A. Kossatchev, V. Kuliamin, and A. Petrenko, “UniTesK: Model Based Testing in Industrial Practice,” Proc. the 1st European Conference on Model- Driven Software Engineering (ECMDSE), Nuremberg, Germany, Dec , 2003, pp A. Khoroshilov., “Specification and Testing Systems with Asynchronous Interfaces,” Preprint of the Inst. for System Programming, Russ. Acad. Sci., Moscow, V. Kuliamin, A. Petrenko, and N. Pakoulin, “Practical Approach to Specification and Conformance Testing of Distributed Network Applications,” Proc. ISAS'2005, Berlin, Germany, April 25-26, 2005, pp N. Pakulin and A. Khoroshilov, “Development of formal models and conformance testing for systems with asynchronous interfaces and telecommunications protocols,” Programming and Computer Software, vol. 33, number 6, Nov. 2007, pp , doi: /S

16 Contact Yevgeny Gerlits Lomonosov Moscow State University, Faculty of Computational Mathematics and Cybernetics, System Programming department

17 Comparison with the existing techniques Proposed by the scientific community: Invariant Based Testing; State Based Testing; Search Based Testing Used in industrial practice: Capture and Playback; Unit testing framework + software library to simulate user actions and programmatically access resulting DOM states FSM based approaches Advantages of our approachDisadvantages of our approach Modular test suite architectureLabor intensive, i.e. the automation level is low High fault exposing capability Ability to test concurrent HTTP requestMay reveal only client-side faults Specification-based test coverage measurement

18 First AJAX engine testing problem - a real situation of a set of users working with a single application

19 Solution to the first AJAX engine testing problem - a real situation of a set of users working with a single application Proxy server simulates server-side state changes that may be carried out by the users 2…N. Web server User interface AJAX engine Catcher Browser 1 Proxy server Stimulus Reaction Action mediator User interface AJAX engine Browser 2 User interface AJAX engine Browser N

20 Second AJAX engine testing problem - concurrent asynchronous HTTP requests Web serverDOMAJAX engine Asynchronous HTTP request 1 Asynchronous HTTP request 2 Response to the HTTP request 2 Removal of DOM element X Response to the HTTP request 1 Modification of DOM element X

21 Solution to the second AJAX engine testing problem - concurrent asynchronous HTTP requests A test for concurrent HTTP requests: 1. Action mediator applies stimulus 1; 2. Proxy server intercepts HTTP request 1; 3. Action mediator applies stimulus 2; 4. Proxy server intercepts HTTP request 2; 5. Proxy server emulates response to HTTP request 2; 6. Proxy server emulates response to HTTP request 1. Web server User interface AJAX engine Catcher Browser 1 Proxy server User interface AJAX engine Browser N Stimulus Reaction Detects reactions Action mediator

22 Third AJAX engine testing problem - AJAX engine itself may initiate JavaScript processing JavaScript functions to create delayed actions: setTimeout (script, timeout); setInterval (script, timeout); clearTimeout (script, timeout); clearInterval (script, timeout)

23 Solution to the third AJAX engine testing problem - AJAX engine itself may initiate JavaScript processing We impose a set of restrictions. An AJAX application may demonstrate a set of reactions but: after an external stimulus; the set of reactions is finite; there is an upper bound of time intervals, during which the reactions appear.

24 Fourth AJAX engine testing problem - how to programmatically detect AJAX engine reactions and get client side state changes? user interface event handlers setTimeout (CODE, timeout) callback functions Client side (browser) Server side User interface AJAX engine Web server Back end

25 Solution to the fourth AJAX engine testing problem - detecting AJAX engine reactions Existing approaches: Selenium RC testing tool - waitForCondition(sctipt, timeout); SWEA testing tool – AJAX complete event; Our approach: User interface event handlers, callback functions, and functions working with timers are programmatically instrumented to monitor the results of their execution.