Test-Driven Development of Graphical User Interfaces: A Pilot Evaluation Thedore D. Hellmann, Ali Hosseini-Khayat, Frank Maurer XP 2011

Outline  Capture and Replay Tools  Introduction  Difficulties  A Prototype-Driven Approach to UITDD  Pilot Evaluation  Conclusion  Capture and Replay Tools  Introduction  Difficulties  A Prototype-Driven Approach to UITDD  Pilot Evaluation  Conclusion

Capture and Replay Tools  Capture and Replay Tools(CRTs)  Tools that record users’ interactions with an application.  Can be used to record test scripts.  Capture and Replay Tools(CRTs)  Tools that record users’ interactions with an application.  Can be used to record test scripts.

Introduction  Test-driven development (TDD) has proven benefits for the software engineering process  increased developer confidence  increased software quality  In the approach, user interface prototypes are created in such a way as to allow capture/replay tools to record interactions with them.  These recordings can then be replayed on the actual GUI as it is being developed in a test-driven fashion.  TDD of GUI-based applications (UITDD)  Test-driven development (TDD) has proven benefits for the software engineering process  increased developer confidence  increased software quality  In the approach, user interface prototypes are created in such a way as to allow capture/replay tools to record interactions with them.  These recordings can then be replayed on the actual GUI as it is being developed in a test-driven fashion.  TDD of GUI-based applications (UITDD) Does having tests for a GUI written before the functionality of the GUI is implemented help developers?

Difficulty  What Makes GUI Testing and UITDD Difficult? 1.Complexity 2.Verification 3.Finding Widgets from Test code  What Makes GUI Testing and UITDD Difficult? 1.Complexity 2.Verification 3.Finding Widgets from Test code

Difficulty - Complexity  Modern GUIs are composed of a set of user interface elements, or widgets. Like most classes in object-oriented systems, widgets tend to have a long chain of inheritances.  Even a simple GUI has a huge number of possible user interactions, GUI states, and events.  For Example:  Modern GUIs are composed of a set of user interface elements, or widgets. Like most classes in object-oriented systems, widgets tend to have a long chain of inheritances.  Even a simple GUI has a huge number of possible user interactions, GUI states, and events.  For Example:

Difficulty - Verification  A test is separated into two parts:  test procedure  test oracle  The test procedure interacts with part or all of an application in order to generate a state that is interesting from a testing perspective.  The test oracle verifies that the system behaves as expected in response to this interaction.  The effectiveness of automated tests is directly limited by the difficulty of writing useful test oracles.  A test is separated into two parts:  test procedure  test oracle  The test procedure interacts with part or all of an application in order to generate a state that is interesting from a testing perspective.  The test oracle verifies that the system behaves as expected in response to this interaction.  The effectiveness of automated tests is directly limited by the difficulty of writing useful test oracles.

Difficulty - Finding Widgets from Test code  In GUI testing, it is necessary to look up widgets when a test is run based on information about the widget that is recorded when a test is created.  This process of look up widgets is in large part responsible for the fragility of GUI tests.  There are two ways of performing this search:  testing with object maps  keyword-based testing  In GUI testing, it is necessary to look up widgets when a test is run based on information about the widget that is recorded when a test is created.  This process of look up widgets is in large part responsible for the fragility of GUI tests.  There are two ways of performing this search:  testing with object maps  keyword-based testing

A Prototype-Driven Approach to UITDD  ExpenseManager  ExpenseManager is a GUI-based application.  Vies Totals  New Report  Modify Report  ExpenseManager  ExpenseManager is a GUI-based application.  Vies Totals  New Report  Modify Report

A Prototype-Driven Approach to UITDD(2)  After defining the features of ExpenseManager, a prototype of the application was created. This prototype was created using SketchFlow.  SketchFlow  Prototypes are created by dragging-and-dropping widgets onto a canvas.  Widgets can also be assigned behaviors such as transitioning to a different state.  we used LEET (LEET Enhances Exploratory Testing) to record events from interactions with prototypes of ExpenseManager.  LEET  It is based on the Automation API  Using keyword-based identification to determine which widget to interact with for each step in the test script.  After defining the features of ExpenseManager, a prototype of the application was created. This prototype was created using SketchFlow.  SketchFlow  Prototypes are created by dragging-and-dropping widgets onto a canvas.  Widgets can also be assigned behaviors such as transitioning to a different state.  we used LEET (LEET Enhances Exploratory Testing) to record events from interactions with prototypes of ExpenseManager.  LEET  It is based on the Automation API  Using keyword-based identification to determine which widget to interact with for each step in the test script.

Pilot Evaluation  We designed a pilot evaluation to determine whether our approach is actually useful to developers.  Based on 3 particepants:  We designed a pilot evaluation to determine whether our approach is actually useful to developers.  Based on 3 particepants: 1.Researchers completed the development of ExpenseManager, including its GUI and event handlers, based on the tests. 2.The researchers removed all of the code contained in ExpenseManager’s 7 event handlers. 1.Researchers completed the development of ExpenseManager, including its GUI and event handlers, based on the tests. 2.The researchers removed all of the code contained in ExpenseManager’s 7 event handlers.

Pilot Evaluation(2)  Interestingly, all three participants entered into the same development cycle during their development of ExpenseManager.  Participants used the provided automated GUI tests as the sole indication of completeness of functionality.  Interestingly, all three participants entered into the same development cycle during their development of ExpenseManager.  Participants used the provided automated GUI tests as the sole indication of completeness of functionality. Observations Collected during the Study

Pilot Evaluation(3)  The participants asked to fill out a survey, which perception of the usefulness and usability of this approach. Observations Collected from Post-Study Surveys  The first and second questions were ranked from A (Very Useful) to E (Useless).  These responses imply that the participants saw the potential of this approach to be of some benefit in their own work.  The first and second questions were ranked from A (Very Useful) to E (Useless).  These responses imply that the participants saw the potential of this approach to be of some benefit in their own work.

Pilot Evaluation(4)  Participants were also asked to rank the resources they were provided with in terms of decreasing usefulness for their development task.  Instead of ranking features by frequency of use, participants seem to have ranked resources based on the value they provided to the development effort.  Participants were also asked to rank the resources they were provided with in terms of decreasing usefulness for their development task.  Instead of ranking features by frequency of use, participants seem to have ranked resources based on the value they provided to the development effort. Observations Collected from Post-Study Surveys  This implies that one way to immediately make this approach to UITDD provide higher value to users :  Making technical details of widgets being tested more visible so that users can understand the expected details of the system they are building.  This implies that one way to immediately make this approach to UITDD provide higher value to users :  Making technical details of widgets being tested more visible so that users can understand the expected details of the system they are building.

Study Limitations 1.Only three participants took part in this study. 2.The time allotted for each study session was limited to one hour in order to avoid inconveniencing participants. 3.The application used for this study was, based on Table 2, too complicated for a one- hour study. 1.Only three participants took part in this study. 2.The time allotted for each study session was limited to one hour in order to avoid inconveniencing participants. 3.The application used for this study was, based on Table 2, too complicated for a one- hour study.

Conclusion  In this paper we present a pilot evaluation of an approach to test-driven development of GUI-based applications.  we hope to test our approach to UITDD on multi-touch and gesture-based interfaces.  we were able to determine that participants seemed to feel that UITDD had benefits, and would be a useful practice.  In this paper we present a pilot evaluation of an approach to test-driven development of GUI-based applications.  we hope to test our approach to UITDD on multi-touch and gesture-based interfaces.  we were able to determine that participants seemed to feel that UITDD had benefits, and would be a useful practice.

Question Time