1. Capture-Replay and Test Automation Jerzy.Nawrocki@put.poznan.pl www.cs.put.poznan.pl/jnawrocki/models/ Models and Analysis of Software Lecture 9 Copyright, 2003  Jerzy R. Nawrocki

2. The Scribble application 1 Belgium 2 Denmark 3 Netherlands New item added in order 1 Netherlands Initial state 1 Netherlands 2 Belgium New item added – end 1 Belgium 2 Netherlands Sorted 1 Denmark 2 Netherlands Old item deleted Enter: Belgium Enter: Denmark Enter: 1 Sort

3. Manual test for Scribble Belgium Netherlands UK Add 2 new items to the sorted list. Move an item (the list gets unsorted). Add an item to the unsorted list. Delete an item. Try to delete an invalid item (position) countries.dcm countries2.dcm

4. The screen shots – Starting the Scribble Scribble File List

5. The screen shots – Openning a file Scribble File List New Close Print Exit Open Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK

6. The screen shots – Adding an item (Sweden) Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK Add Item OK Cancel Sweden| Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK New List Sort List Del Item Mov Item Add Item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK

7. The screen shots – Adding an item (USA) Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK Add Item OK Cancel USA| Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK New List Sort List Del Item Mov Item Add Item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK USA

8. The screen shots – Moving an item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK USA New List Sort List Add Item Del Item Mov Item Scribble File List File Name countries List Sorted Countries on our mailing list are: 1 Belgium 2 Netherlands 3 Sweden 4 UK 5 USA Move Item OK Cancel 4 From 1 To Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA

9. The screen shots – Adding an item (Norway) Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA Add Item OK Cancel Norway| Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA New List Sort List Del Item Mov Item Add Item Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA Norway

10. The screen shots – Deleting a valid item Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Belgium 3 Netherlands 4 Sweden 5 USA 6 Norway Delete Item From Position OK Cancel 2 Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA Norway New List Sort List Add Item Mov Item Del Item Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Netherlands Sweden USA Norway

11. The screen shots – Deleting an invalid item Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Netherlands 3 Sweden 4 USA 5 Norway Delete Item From Position OK Cancel 7 Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Netherlands Sweden USA Norway New List Sort List Add Item Mov Item Del Item Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Netherlands 3 Sweden 4 USA 5 Norway Delete Error Position Not Valid OK

12. The screen shots – Closing Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Netherlands Sweden USA Norway Open New Print Exit Close Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Netherlands 3 Sweden 4 USA 5 Norway Close Save the document? OK i i No

13. The screen shots – Exiting Scribble File List Scribble File List Open New Close Print Exit

14. What was the test input? 1 Move mouse to Scribble icon 2 Double click

15. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu Scribble File List

16. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click Scribble File List

17. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option Scribble File List New Close Print Exit Open

18. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click Scribble File List New Close Print Exit Open

19. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm

20. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click

21. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK

22. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK

23. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK New List Sort List Del Item Mov Item Add Item

24. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK New List Sort List Del Item Mov Item Add Item

25. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click 13 Type Sweden Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK Add Item OK Cancel Sweden|

26. What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click 13 Type Sweden 14 Move mouse to OK button Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK Add Item OK Cancel Sweden|

27. The manual test process 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click 13 Type Sweden 14 Move mouse to OK button The amount of effort to automate a test case: 2 – 10 times the effort required to run the test manually (!)

28. The manual test process Unscripted manual testing: 1.Identify conditions to test 2.Think up specific inputs 3.Enter the inputs 4.Checked it worked OK

29. The manual test process Unscripted manual testing Vague manual scripts: 1.Read what to do 2.Think up specific inputs 3.Enter the inputs 4.Checked it worked OK

30. Vague manual script 1 Run up Scribble File menu displayed 2 Open file with sorted ListFile contents displayed 3 Add some items to ListItems added in order 4 Move an itemItem moved, List now unsorted 5 Add an itemItem added at end of List 6 Delete item form listItem deleted 7 Use invalid position numberError message displayed to delete an item 8 Save changes in new fileEnd of test

31. The manual test process Unscripted manual testing Vague manual scripts Detailed manual scripts: 1.Read what to do 2.Enter the inputs 3.Checked it worked OK

32. Detailed manual script 1 Click on Scribble iconScribble opened, File menu enabled 2 Move mouse to File menu, clickOptions available: Open, New, Exit 3 Move mouse to Open option,Filenames displayed including clickcountries.dcm 4 Move mouse to countries.dcmText plus List of 3 countries: Belgium, Netherlands, UK, File and List menus displayed...

33. Automating test execution Automated scripts are not like manual scripts Don’t automate testing by simply recording Automated execution with manual verification? How much shoul be compared (whole, minimum, something between)? Dynamic comparison and post-execution comparison Automated comparison messages have to be manually checked While tests fail the second time?

34. Conclusion Automated is not automatic Automated is not automatic

35. Bibliography M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999. M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999.

36. Questions?