1 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 GUI.

Slides:

Advertisements

Similar presentations

A Method for Validating Software Security Constraints Filaret Ilas Matt Henry CS 527 Dr. O.J. Pilskalns.

Advertisements

Unit-V testing strategies and tactics.

Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.

Systems V & V, Quality and Standards

Ossi Taipale, Lappeenranta University of Technology

Towards Self-Testing in Autonomic Computing Systems Tariq M. King, Djuradj Babich, Jonatan Alava, and Peter J. Clarke Software Testing Research Group Florida.

1 Program Slicing Purvi Patel. 2 Contents Introduction What is program slicing? Principle of dependences Variants of program slicing Slicing classifications.

Prioritizing User-session-based Test Cases for Web Applications Testing Sreedevi Sampath, Renne C. Bryce, Gokulanand Viswanath, Vani Kandimalla, A.Gunes.

Selecting Preservation Strategies for Web Archives Stephan Strodl, Andreas Rauber Department of Software.

CSE 322: Software Reliability Engineering Topics covered: Architecture-based reliability analysis.

A GOAL-BASED FRAMEWORK FOR SOFTWARE MEASUREMENT

Design of Fault Tolerant Data Flow in Ptolemy II Mark McKelvin EE290 N, Fall 2004 Final Project.

Hierarchical GUI Test Case Generation Using Automated Planning Atif M. Memon, Student Member, IEEE, Martha E. Pollack, and Mary Lou Soffa, Member, IEEE.

Testing Components in the Context of a System CMSC 737 Fall 2006 Sharath Srinivas.

Company LOGO B2C E-commerce Web Site Quality: an Empirical Examination (Cao, et al) Article overview presented by: Karen Bray Emilie Martin Trung (John)

University of Jyväskylä – Department of Mathematical Information Technology Computer Science Teacher Education ICNEE 2004 Topic Case Driven Approach for.

Unit Testing CS 414 – Software Engineering I Don Bagert Rose-Hulman Institute of Technology January 16, 2003.

Introduction to Software Testing

1 Software Testing Techniques CIS 375 Bruce R. Maxim UM-Dearborn.

TEST CASE DESIGN Prepared by: Fatih Kızkun. OUTLINE Introduction –Importance of Test –Essential Test Case Development A Variety of Test Methods –Risk.

A Semantic Workﬂow Mechanism to Realise Experimental Goals and Constraints Edoardo Pignotti, Peter Edwards, Alun Preece, Nick Gotts and Gary Polhill School.

System/Software Testing

Reverse Engineering State Machines by Interactive Grammar Inference Neil Walkinshaw, Kirill Bogdanov, Mike Holcombe, Sarah Salahuddin.

1. Topics to be discussed Introduction Objectives Testing Life Cycle Verification Vs Validation Testing Methodology Testing Levels 2.

Validation Metrics. Metrics are Needed to Answer the Following Questions How much time is required to find bugs, fix them, and verify that they are fixed?

CMSC 345 Fall 2000 Unit Testing. The testing process.

CS4311 Spring 2011 Unit Testing Dr. Guoqiang Hu Department of Computer Science UTEP.

1 Software Quality CIS 375 Bruce R. Maxim UM-Dearborn.

A Taxonomy of Evaluation Approaches in Software Engineering A. Chatzigeorgiou, T. Chaikalis, G. Paschalidou, N. Vesyropoulos, C. K. Georgiadis, E. Stiakakis.

Overview of Software Testing 07/12/2013 WISTPC 2013 Peter Clarke.

Composing Adaptive Software Authors Philip K. McKinley, Seyed Masoud Sadjadi, Eric P. Kasten, Betty H.C. Cheng Presented by Ana Rodriguez June 21, 2006.

Agenda Introduction Overview of White-box testing Basis path testing

Software Engineering Research paper presentation Ali Ahmad Formal Approaches to Software Testing Hierarchal GUI Test Case Generation Using Automated Planning.

Verification and Validation in the Context of Domain-Specific Modelling Janne Merilinna.

Lecture 7: Requirements Engineering

CSE403 Software Engineering Autumn 2001 More Testing Gary Kimura Lecture #10 October 22, 2001.

©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 22 Slide 1 Software Verification, Validation and Testing.

Information Systems Engineering. Lecture Outline Information Systems Architecture Information System Architecture components Information Engineering Phases.

1 Graphical User Interfaces Graphical User Interfaces (GUIs) have become an important and accepted way of interacting with today's software. Although.

1 Program Testing (Lecture 14) Prof. R. Mall Dept. of CSE, IIT, Kharagpur.

1 UP MBT Extending the Unified Process with Model-Based Testing Fabrice Bouquet, Stéphane Debricon, Bruno Legeard and Jean-Daniel Nicolet MoDeV 2 a 2006.

1 Checking Interaction Consistency in MARMOT Component Refinements Yunja Choi School of Electrical Engineering and Computer Science Kyungpook National.

Chair of Software Engineering Exercise Session 6: V & V Software Engineering Prof. Dr. Bertrand Meyer March–June 2007.

Using Social Network Analysis Methods for the Prediction of Faulty Components Gholamreza Safi.

Comparing model-based and dynamic event-extraction based GUI testing techniques : An empirical study Gigon Bae, Gregg Rothermel, Doo-Hwan Bae The Journal.

Center for Reliability Engineering Integrating Software into PRA B. Li, M. Li, A. Sinha, Y. Wei, C. Smidts Presented by Bin Li Center for Reliability Engineering.

Towards a Reference Quality Model for Digital Libraries Maristella Agosti Nicola Ferro Edward A. Fox Marcos André Gonçalves Bárbara Lagoeiro Moreira.

Software Engineering1  Verification: The software should conform to its specification  Validation: The software should do what the user really requires.

SOFTWARE TESTING. Introduction Software Testing is the process of executing a program or system with the intent of finding errors. It involves any activity.

Survey of Tools to Support Safe Adaptation with Validation Alain Esteva-Ramirez School of Computing and Information Sciences Florida International University.

Dynamic Testing.

Dynamic White-Box Testing What is code coverage? What are the different types of code coverage? How to derive test cases from control flows?

National Aeronautics and Space Administration Jet Propulsion Laboratory March 17, 2009 Workflow Orchestration: Conducting Science Efficiently on the Grid.

SOFTWARE TESTING LECTURE 9. OBSERVATIONS ABOUT TESTING “ Testing is the process of executing a program with the intention of finding errors. ” – Myers.

Software Testing. SE, Testing, Hans van Vliet, © Nasty question  Suppose you are being asked to lead the team to test the software that controls.

1 Software Testing. 2 What is Software Testing ? Testing is a verification and validation activity that is performed by executing program code.

CS223: Software Engineering Lecture 25: Software Testing.

Testing Integral part of the software development process.

Designing a framework For Recommender system Based on Interactive Evolutionary Computation Date : Mar 20 Sat, 2011 Project Number :

Introduction to Software Testing (2nd edition) Chapter 5 Criteria-Based Test Design Paul Ammann & Jeff Offutt

Software Testing.

Deriving Test Data for Web Applications from User Session Logs

Software Engineering (CSI 321)

Graph Coverage for Specifications CS 4501 / 6501 Software Testing

Types of Testing Visit to more Learning Resources.

Goal, Question, and Metrics

Software Testing (Lecture 11-a)

Graph Coverage for Specifications CS 4501 / 6501 Software Testing

By Hyunsook Do, Sebastian Elbaum, Gregg Rothermel

Unit III – Chapter 3 Path Testing.

Presentation transcript:

1 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | GUI Testing Term paper in Scientific Programming

2 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Overview Motivation Introduction to software testing GUI representation Coverage criteria for GUI testing Optimizing the test suite Conclusion Future work

3 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Motivation Software testing is a proven method to increase the quality of software Software bugs can have severe consequences  Loss of money or material e.g. Ariane 5 launch  Loss of human life e.g. Toyota’s electronic throttle control system GUIs are an ubiquitous instrument for interacting with software

4 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Software Testing A process of executing a program to determine if it meets its specifications Categories of software testing  Correctness testing  Performance testing  Reliability testing  Security testing  Usability testing Whitebox or blackbox

5 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Challenges of GUI Testing Event based Sequence of events  Large number of combinations of events in the sequence  Possibly unbound

6 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Requirements for GUI Representation Conceptually at high level of abstraction  Independent of the platform for which the GUI is developed  Guarantees that generated testing information is portable across platforms Possible to represent a variaty of GUIs Low level details of GUIs must be captured Representation must be scalable Easy to develop and use

7 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | GUI Representation Directed graphs Nodes represent events Edges represent transitions from one event to another

8 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Example GUI Xie and Memon, 2008

9 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Event Sequence Graph (ESG) Hierarchical structure of a GUI is contained in the graph Subgraphs are represented as a node of the graph Uses explicit entry and exit nodes [Xie and Memon, 2008] Belli et al., 2005

10 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Complete Event Sequence Graph (CESG) Belli et al., 2005

11 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Event Flow Graph (EFG) No hierarchical structure contained in the graph Uses integration tree to represent hierarchy Entry and exit nodes are implicit [Belli et al., 2011] Memon and Science 2001

12 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Integration Tree Gives a hierarchical overview of the GUI components of a software Each leaf invokes a GUI component or triggers an action Memon and Science, 2001

13 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Comparison of Graphs [Xie and Memon, 2008]

14 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Coverage Criteria Determine what to test  Straightforward answer: all possibilities  impractical Determine when to stop testing Set of rules that determine whether a software has been tested enough

15 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Intra-Component Coverage Focuses on events that are bound to one GUI component Event coverage  Each event has to be performed at least once  Checks if each event behaves as expected Event-interaction coverage  Checks the interaction between all possible pairs of events Length-n event sequence coverage  Behavior of events can change depending on previous events  Captures the contextual impact of events  There are an infinite amount of contexts  limit of the length of event sequences is defined

16 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Inter-Component Coverage Invocation coverage  Ensures that each component in the GUI can be invoked Invocation-termination coverage  Ensures that GUI components can be terminated Inter-component length-n event sequence coverage  Tests all event sequences that start in one component and end in another component Uses integration tree to identify interactions between components

17 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Metric of Coverage Intra-component coverage  Percentage of covered intra-component length-n event sequences Inter-component coverage  Percentage of covered inter-component length-n event sequences

18 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Why is Optimizing Necessary? Thoroughly testing can result in a large test suite Adopting the test suite to evolving software can result in redundant tests Test suites can contain many similar tests

19 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Efficient and Flexible GUI Test Execution via Test Merging Goal reducing the number of test Merging redundant or similar tests Constraint:  Preserves fault localization capabilities of a test suite Benefits:  Reduces number of failures from the same source  Reduces testing time

20 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Redundant Tests Tests are identical if they start in the same state and end in the same state A test is redundant if all entities exercised by a test case are also covered by another test case from the test suite

21 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Example of Similar Tests Comparing rating a book with buying a book

22 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Example for Similar Tests (Rating a book)

23 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Example for similar tests (Rating a book)

24 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Example for similar tests

25 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Example for similar tests (Merged test)

26 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Results of merging Empirical study with five web application Number of test steps were reduced by 29% Execution time was reduced by 39%

27 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Conclusion Software testing important to ensure high quality software Testing GUI is challenging because it uses events It is possible to use the ESG for the coverage criteria Using the methods described  GUIs can be modeled using one of the GUI representations  Tests can be created based on the coverage criteria  Test suites can be optimized

28 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Future Work

29 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Future Work

30 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | Thank you for your attention. Do you have any questions?

31 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | References [Devaki et al., 2013] Devaki, P., Thummalapenta, S., Singhania, N., and Sinha, S. (2013). Efficient and flexible GUI test execution via test merrging. Proceedings of the 2013 International Symposium on Software Testing and Analysis - ISSTA 2013, page 34. [Memon et al., 2001] Memon, A. M., Soffa, M. L., and Pollack, M. E. (2001). Coverage criteria for GUI testing. ACM SIGSOFT Software Engineering Notes, 26(5):256. [Xie and Memon, 2008] Xie, Q. and Memon, A. M. (2008). Using a pilot study to derive a GUI model for automated testing. ACM Transactions on Software Engineering and Methodology, 18(2):1–35. [Belli et al., 2011] Belli, F., Beyaz, M., and Güler, N. (2011). Eventbased GUI Testing and Reliability Assessment Techniques An Experimental Insight and Preliminary Results. Software Testing, Verification and Validation Workshops (ICSTW), 2011 IEEE Fourth International Conference on, pages 212–221. [Belli et al., 2005] Belli, F., Nissanke, N., Budnik, C. J., and Mathur, A. (2005). Technical Reports and Working Papers Test Generation Using Event Sequence Graphs. (September). [Memon and Science, 2001] Memon, A. M. and Science, C. (2001). A COMPREHENSIVE FRAMEWORK FOR TESTING.