1. 1-6-2015 @ IPA Herfstdagen Software Engineering Research Group Delft University of Technology Dynamic Analysis and Testing of Ajax User Interfaces Ali Mesbah, Arie van Deursen Software Engineering Research Group Delft University of Technology, The Netherlands

2. 2 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Classical Web Applications Shortcomings of the Multi-page style: Low lever of user interactivity Redundant data transfer High user-perceived latency Client is passive Server Client Browser Client Browser HTTP GET HTML

3. 3 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Ajax Asynchronous JavaScript and XML Asynchronous server communication through XMLHttpRequest Web standards-based (HTML, CSS, XML) Dynamic display through DOM JavaScript binds everything together Features: No full page refresh! Single-page web interface Google Suggest, Gmail, Yahoo! Mail

4. 4 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Ajax-based Web Applications Client-side execution State changes & navigation Dynamic DOM Delta-communication Clickables: Elements having event-listeners attached to them $(".news").click(function() { $("#content").load("news.php"); });

5. 5 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Research Question How can we automatically analyze (test) Ajax web applications?

6. 6 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Current Approaches General Approach: extract link, send a request to the server and analyze the response. i.e., HTML (based on the Multi-page style) Capture/Replay: demands a substantial amount of manual effort Static Analysis: misses the complex run-time behavior Dynamic Analysis: promising but detecting the various doorways to different dynamic states and providing proper interface mechanisms for input values is challenging.

7. 7 Dynamic Analysis and Testing of Ajax User Interfaces | 26 A Method for Deriving Ajax States Conduct dynamic analysis of the user interface to find Clickable elements and state changes Reverse-engineer a State-flow Graph to abstract the navigation paths and state changes

8. 8 Dynamic Analysis and Testing of Ajax User Interfaces | 26 User Interface State Changes In classic web applications, each UI state is represented by a URL and the corresponding web page In Ajax, it is the internal change of the DOM tree on the (single-page) user interface that represents a UI state change To adopt a generic solution, we define a UI state change as: a change on the DOM tree caused either by server-side state changes propagated to the client, or client-side events handled by the Ajax engine

9. 9 Dynamic Analysis and Testing of Ajax User Interfaces | 26 State-flow Graph

10. 10 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Inferring the State Machine (Crawljax)

11. 11 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Detecting Clickables Find a set of Candidate Elements (CC) Expose each to en event (onclick, onmouseover, …) Analyze the effect on the DOM We distinguish between 3 ways of obtaining CCs: 1.Full Auto Scan: HTML tagname (div, span, a, …) with attribute constraints, e.g., div:{class=“menuitem”} 2.Annotation: element attribute crawljax=“true”, or “false” 3.A DSL: Crawling Ajax Specification Language (CASL)

12. 12 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Detecting States Input: DOM Trees before and after the event: Compare DOM Trees: As is Textual: extract textual content and compare Structural: extract structural content and compare Meta-model: reverse-engineer the Schema for each DOM, then compare the schemas Comparison method: Edit Distance, Levenshtein method with a similarity threshold

13. 13 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Processing Document Tree Deltas Upon every new state entry: Compute the differences between the previous document tree and the current one We use an enhanced Diff algorithm to find the “delta updates” Delta updates may be due, for example, to a server request call that injects new elements into the DOM Find new candidate elements in the delta updates, and examine them in a recursive depth-first manner

14. 14 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Navigating the States Problem: going ‘Back’ in the browser Solutions: 1.Register the state explicitly and use Back 2.Click through from the initial state Use element IDs: not always present, not always persistent Use XPath Expressions: much more reliable, not 100% water-proof

15. 15 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Data Entry Points Detecting DOM forms: For each state change, extract all the forms For each form, calculate a hashcode based on the form properties Check if form is already present in the database If not, store the form completely in the database with the hashcode as its ID If present, use the corresponding custom values in the database (if present) to fill the form in the browser and submit Upon submission, the resulting state change is recursively analyzed.

16. 16 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Full Auto Scan For each state: 1.Retrieve all candidate clickables 2.Fire event on each candidate element 3.Compare DOM trees: Edit Distance. Is DOM changed? 4.Make a new State and add to the State-flow graph 5.Find the XPath Expression of the Clickable Element 6.Add the clickable as an edge to the SFG between state before and after the click 7.Analyze forms 8.Do all this recursively

17. 17 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Testing Ajax States With access to different dynamic DOM states we can check the user interface against different constraints We use invariants as oracle Invariants on the DOM tree Invariants between DOM states, and Application-specific invariants

18. 18 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Generic DOM Invariants Generic DOM Invariants: Validated DOM No error messages in DOM Accessible State Secure State Discoverable links No Dead Clickables Consistent Back-Button

19. 19 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Application-specific Invariants Examples: //case one: warn about collapsable divs within expandable items String xpathCase1 = "//LI[contains(@class,’expandable’)]/DIV[contains(@class,’collapsable’)]"; //case two: warn about collapsable items within expandable items String xpathCase2 = "//LI[contains(@class,’expandable’)]/UL/LI[contains(@class,’collapsable’)]";

20. 20 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Testing Ajax Paths While running the crawler to derive the state machine can be considered as a first full test pass, the state machine itself can be further used for testing purposes. We generate test cases from the state machine: K shortest paths from Index to sinks (nodes with no outgoing edges) Cycles included only once, All-transitions coverage Transform each path found into a JUnit test case

21. 21 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Testing Ajax Paths @Test public void testcase1 () { browser.goToUrl (url); /*Element-info: SPAN class=expandable-hitarea */ browser.fireEvent(new Eventable (new Identification("xpath", "//DIV[1]/SPAN[4]"), "onclick")); Comp.assertEquals(oracle.getState ("S_1"). getDom(), browser.getDom ()); /*Element-info: DIV class=hitarea id=menuitem2 */ browser.fireEvent(new Eventable(new Identification("xpath", "//P[2]/ DIV[2]"), "onmouseover")); … }

22. 22 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Testing Ajax Paths Generated test suite can be used in several ways: It can be run as is on the current version of the Ajax application, but for instance with a different browser to find browser incompatibilities It can be applied to altered versions of the Ajax application to support regression testing (against a Gold-standard) The typical use of the derived test suite will be to take apart specific generated test cases, and augment them with application- specific assertions, capturing specific fault-sensitive click trails.

23. 23 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Tool Implementation Crawljax: crawling through dynamic states ATUSA: plugin-based framework on top of Crawljax: preCrawling inCrawling postCrawling Java 1.6 Watij (IE), XULRunner (Mozilla) Numerous libraries Java Plugin Framework Open source

24. 24 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Tool Implementation

25. 25 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Empirical Evaluation 4 case studies (3 open source), 1 industrial TUDU, FreakTask, PetStore, CoachJeZelf Instrumented the server side code with Clover Instrumented the client-side JavaScript code with JSCoverage Fault seeding and code coverage Code Coverage: 73% server-side, 70% client-side More than 180 Violations per case 80% fault detection

26. 26 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Concluding Remarks We have proposed a method for testing Ajax applications automatically through 1.A crawler that can detect clickables and forms and analyze state changes 2.A testing framework that can check the application behavior through generic invariants as well as application-specific invariants 3.A test suite generation from the inferred state machine. Tools are available for download: http://spci.st.ewi.tudelft.nl/content/software

27. 27 Dynamic Analysis and Testing of Ajax User Interfaces | 26 Publications Ali Mesbah, Engin Bozdag, and Arie van Deursen (2008). Crawling Ajax by Inferring User Interface State Changes. In Proceedings of the 8th International Conference on Web Engineering (ICWE'08), IEEE Computer Society. Ali Mesbah and Arie van Deursen (2009). Invariant-Based Automatic Testing of Ajax User Interfaces. In Proceedings of the 31st International Conference on Software Engineering (ICSE'09), IEEE Computer Society.