Acceptance Testing (1) Copyright, 2006  Jerzy R. Nawrocki Requirements Engineering & Project Management Lecture 7

J.Nawrocki, Acceptance Testing The V Model Requirements spec. Coding Design Integration tests Acceptance tests Unit tests

J.Nawrocki, Acceptance Testing Model V Requirements spec. Acceptance tests Design Integration tests Coding Unit tests Customer

J.Nawrocki, Acceptance Testing Introduction Test-driven development Acceptance tests: customer Automated accept. testing A. testing: customer+tester Progress measurement based on acceptance testing

J.Nawrocki, Acceptance Testing Extreme Programming Acceptance testing -> Progress measurement Weeks Number of tests

J.Nawrocki, Acceptance Testing Introduction – What is Rational Robot? Baseline Robot

J.Nawrocki, Acceptance Testing What is test automation? That’s simple! 1. Buy a test execution tool. 2. Record the manual tests. 3. Play the tests back whenever you want to. „Just as there is more to software design than knowing a programming language, there is more to automating testing than knowing a testing tool.” -- M. Fewster & D. Graham

J.Nawrocki, Acceptance Testing Manual or Automatic Testing? The amount of effort to automate a test case: 2 – 10 times the effort required to run the test manually (!)

J.Nawrocki, Acceptance Testing Introduction – Rational Robot and test-first coding Remarks: When you start testing an application is ready. Rational Robot is for regression testing, not for test-first coding. The only chance for test-first coding is SQABasic.

J.Nawrocki, Acceptance Testing Testing HTML Applications StartBrowser [ URL$, ] [ WindowTag= Name$ ] Starts the browser, enables HTML testing, and loads a page if specified. Web page address Identifies this instance of the browser StartBrowser " "WindowTag= Parent"

J.Nawrocki, Acceptance Testing Testing HTML Applications Browser action$, recMethod$, parameters$ Performs an action on a Web browser Back, Forward, NewPage, SetFrame, CloseWin,.. Wait= % default time is 30s HTMLTitle= $, Name= $, "".. Browser NewPage, "" Wait for the topmost frame

J.Nawrocki, Acceptance Testing Testing HTML Applications HTML action%, recMethod$, parameters$ Performs a mouse action on an HTML tag. Click,.. Coords= x, y HTML Click, "Name= Autor", "Coords= 10,12" HTMLTitle= $, Name= $, ""..

J.Nawrocki, Acceptance Testing TestDatastore\DefaultTestScriptDatastore\TMS_Script\vp\test7. Weryfikacja_daty. base.grd SQABasic HTMLDocument Click, "Type=HTMLDocument;Index=1", " Coords=25,14 " Result = HTMLTableVP (CompareData, "Type=HTMLTable;Index=1", "VP= Weryfikacja_daty ")

J.Nawrocki, Acceptance Testing An alternative to Rational Robot httpUnit Based on jUnit For programmers!

J.Nawrocki, Acceptance Testing Data-driven scripts Data-driven scripting = Test inputs stored in a separate (data) file. countries, Sweden, USA, countries2 countries, France, Germany, test2 countries, Austria, Italy, test3

J.Nawrocki, Acceptance Testing Data-driven scripts Data-driven scripting = Test inputs stored in a separate (data) file. OpenFile ‘ScribbleData’ For each record in ScribbleData Read INPUTFILE Read NAME1 Read NAME2 Read OUTPUTFILE Call ScribbleOpen (INPUTFILE) FocusOn ‘Scribble’ SelectOption ‘List/Add Item’ FocusOn ‘Add Item’ Type NAME1 LeftMouseClick ‘OK’... EndFor countries, Sweden, USA, countries2 countries, France, Germany, test2 countries, Austria, Italy, test3 ScribbleData Control script

J.Nawrocki, Acceptance Testing A more sophisticated data-driven script OpenFile ‘ScribbleData’ Read INPUTFILE Call ScribbleOpen (INPUTFILE) Go to next record (ie row) For each record in ScribbleData Read ADDNAME If ADDNAME <> Blank Then { FocusOn ‘Scribble’ SelectOption ‘List/Add Item’ FocusOn ‘Add Item’ Type ADDNAME LeftMouseClick ‘OK} Read MOVEFROM Read MOVETO... EndFor ScribbleData countries Sweden USA 41 Norway 2 7 countries2 Control script

J.Nawrocki, Acceptance Testing Data-driven scripts               Software under test Key automated regression test cases Additional automated regression test cases

J.Nawrocki, Acceptance Testing Keyword-driven scripts Keyword-driven script = A data-driven scripts augmented with keywords representing user actions. SQABasic??? ScribbleOpen countries AddToList Sweden USA SaveAs countries2 Test file For each TEST_ID OpenFile TEST_ID For each record in test file Read KEYWORD Call KEYWORD EndFor CloseFile TEST_ID EndFor Control script ScribbleOpen... AddToList... SaveAs... Supporting scripts

J.Nawrocki, Acceptance Testing Shared scripts Shared script = A script used (shared) by more than one test case. SQABasic Call ScribbleOpen (‘countries’) FocusOn ‘Scribble’ SelectOption ‘List/Add Item’... Call ScribbleSaveAs (‘countries2’)

J.Nawrocki, Acceptance Testing Automating test execution Automated scripts are not like manual scripts Don’t automate testing by simply recording Automated execution with manual verification? How much shoul be compared (whole, minimum, something between)? Dynamic comparison and post-execution comparison Automated comparison messages have to be manually checked Why tests fail the second time?

J.Nawrocki, Acceptance Testing Test activities Identify test conditions (‘what’ to test) and prioritize Design test cases (‘how’ to test) Build test cases (scripts, data etc.) Execute test cases Compare test outcomes to expected outcomes

J.Nawrocki, Acceptance Testing Questions?