1. Automated Comparison Jerzy.Nawrocki@put.poznan.pl www.cs.put.poznan.pl/jnawrocki/models/ Models and Analysis of Software Lecture 12 Copyright, 2003  Jerzy R. Nawrocki

2. Plan wystąpienia Summary of scripting techniques Reference testing Complex comparison

3. Summary of scripting techniques Linear scripts Structured scripting Shared scripts Data-driven scripts Keyword-driven scripts

4. Summary of scripting techniques Data-driven scripting = Test inputs stored in a separate (data) file. OpenFile ‘ScribbleData’ For each record in ScribbleData Read INPUTFILE Read NAME1 Read NAME2 Read OUTPUTFILE Call ScribbleOpen (INPUTFILE) FocusOn ‘Scribble’ SelectOption ‘List/Add Item’ FocusOn ‘Add Item’ Type NAME1 LeftMouseClick ‘OK’... EndFor countries, Sweden, USA, countries2 countries, France, Germany, test2 countries, Austria, Italy, test3 ScribbleData Control script

5. A more sofisticated data-driven script OpenFile ‘ScribbleData’ Read INPUTFILE Call ScribbleOpen (INPUTFILE) Go to next record (ie row) For each record in ScribbleData Read ADDNAME If ADDNAME <> Blank Then { FocusOn ‘Scribble’ SelectOption ‘List/Add Item’ FocusOn ‘Add Item’ Type ADDNAME LeftMouseClick ‘OK} Read MOVEFROM Read MOVETO... EndFor ScribbleData countries Sweden USA 41 Norway 2 7 countries2 Control script

6. Keyword-driven scripts Keyword-driven script = A data-driven scripts augmented with keywords representing user actions. SQABasic??? ScribbleOpen countries AddToList Sweden USA SaveAs countries2 For each TEST_ID OpenFile TEST_ID For each record in test file Read KEYWORD Call KEYWORD EndFor CloseFile TEST_ID EndFor Test file Control script ScribbleOpen... AddToList... SaveAs... Supporting scripts

7. Plan wystąpienia Summary of scripting techniques Reference testing Complex comparison

8. Reference testing 1.Run an application and capture the outcomes 2.Review the outcomes 3.Use them in subsequent tests as a reference point It is a kind of regression testing.

9. Reference testing 1.Run an application and capture the outcomes 2.Review the outcomes 3.Use them in subsequent tests as a reference point Reference testing (RT) or expected outcomes prepared in advance (ExOPA)? The amount of expected data: a little > ExOPA; a lot > RT Is the software under test available: No > ExOPA Verification quality is higher with ExOPA.

10. Plan wystąpienia Summary of scripting techniques Reference testing Complex comparison

11. Kinds of comparison Dynamic comparison Post-execution comparison passive active Simple (dumb) comparison: requires identical match Complex (intelligent) comparison: takes into account known differences

12. Complex comparison When we need complex comparison: date and time unique identity code or number ?

13. Complex comparison Sales Invoice No. 03/11803 Date: 26-May-2003 Code Description Price -------------------------- CL/3 Chain link 2.00 HK/1 Hook 3.50 -------------------------- Total: 5.50 Payment due by 09-Jun-2003

14. Complex comparison Sales Invoice No. 03/11803 Date: 26-May-2003 Code Description Price -------------------------- CL/3 Chain link 2.00 HK/1 Hook 3.50 -------------------------- Total: 5.50 Payment due by 09-Jun-2003 IGNORE(1, 19:26) IGNORE(2, 7:17) IGNORE(11, 16:26)

15. Complex comparison Sales Invoice No. 03/11803 Date: 26-May-2003 Code Description Price -------------------------- CL/3 Chain link 2.00 HK/1 Hook 3.50 -------------------------- Total: 5.50 Payment of $5.50 due by 09-Jun-2003 IGNORE(1, 19:26) IGNORE(2, 7:17) IGNORE(11, 16:26) ???

16. Complex comparison Sales Invoice No. 03/11803 Date: 26-May-2003 Code Description Price -------------------------- CL/3 Chain link 2.00 HK/1 Hook 3.50 -------------------------- Total: 5.50 Payment due by 09-Jun-2003 IGNORE_AFTER("Invoice No. ", 8) IGNORE_AFTER("Date: ", 11) IGNORE_AFTER("Due by ", 11)

17. Complex comparison Sales Invoice No. 03/11803 Date: 26-May-2003 Code Description Price -------------------------- CL/3 Chain link 2.00 HK/1 Hook 3.50 -------------------------- Total: 5.50 Payment due by 09-Jun-2003 [0-9]{2}\/[0-9]{4} DD”-”LLL”-”DDDD

18. Complex comparison Sales Invoice No. @InvId Date: @Date Code Description Price -------------------------- CL/3 Chain link 2.00 HK/1 Hook 3.50 -------------------------- Total: 5.50 Payment due by @Date @InvId [0-9]{2}\/[0-9]{4} @Date DD”-”LLL”-”DDDD

19. Complex comparison What to do with the ignored data? Remove them from a copy of the actual output Replace them with a constant

20. Summary Simple masking comparison Simple masking comparison Prefix-based comparison Prefix-based comparison Regular expressions Regular expressions Template-based comparison Template-based comparison

21. Bibliography M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999. M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999.

22. Questions?

23. Quality assessment 1. What is your general impression? (1 - 6) 2. Was it too slow or too fast? 3. What important did you learn during the lecture? 4. What to improve and how?