Semester 1, 2003 Week 7 CSE9020 / 1 Software Testing and Quality Assurance With thanks to Shonali Krishnaswamy and Sylvia Tucker

Semester 1, 2003 Week 7 CSE9020 / 2 Reading and References n “Software Testing - A Craftsman’s Approach”, (1995) by Paul C. Jorgensen n “The Complete Guide to Software Testing”, (1988), by Bill Hetzel

Semester 1, 2003 Week 7 CSE9020 / 3 Presentation Outline n Introduction n Terminology n Levels of Testing n Defect Fixing Strategies n Test Process n Bottom line

Semester 1, 2003 Week 7 CSE9020 / 4 Testing and Quality n Hetzel - “Testing is the process of establishing confidence that a program or system does what it is supposed to do” n Quality “Meets requirements” n Quality is not tangible

Semester 1, 2003 Week 7 CSE9020 / 5 Testing and Quality n The purpose of testing is to make quality visible n Testing is the measurement of software quality

Semester 1, 2003 Week 7 CSE9020 / 6 Terminology n Test Plan: A document describing the testing approach –Description of the system to be tested –Objectives of testing

Semester 1, 2003 Week 7 CSE9020 / 7 Terminology - Test Plan –Project team –Test environment –Resource requirements –Test Schedule

Semester 1, 2003 Week 7 CSE9020 / 8 Terminology n Test Procedure: A document defining the steps required to carry out the test plan (or part of it) - that is, execute a set of test cases –Test case name, objective –Description of input required –Description of expected output

Semester 1, 2003 Week 7 CSE9020 / 9 Terminology - Test Procedure Some aspects to watch : –Commands or Macros to run the test case –Location and format of the actual output –Assumptions, Conditions

Semester 1, 2003 Week 7 CSE9020 / 10 Terminology n Test Case: A specific set of test data and the corresponding expected results for a particular test objective. –Invalid and Valid test cases n Test Script: The code written to implement the test case

Semester 1, 2003 Week 7 CSE9020 / 11 Levels of Testing n Unit Testing: Testing of individual components of a system (sometimes called Basic Functionality Testing)

Semester 1, 2003 Week 7 CSE9020 / 12 Levels of Testing n System Testing: Testing in a wider scope - a group of components and the sequence of operations

Semester 1, 2003 Week 7 CSE9020 / 13 Levels of Testing n Acceptance Testing: Testing to determine if a system is ready for implementation (or, - would the ‘user’ buy this ? - would I buy it ? )

Semester 1, 2003 Week 7 CSE9020 / 14 Automated vs. Manual Testing n Manual –Every test case is executed manually –Comparison of actual and expected is done manually

Semester 1, 2003 Week 7 CSE9020 / 15 Automated vs. Manual Testing n Manual –Suitable for interactive tests –Tedious and time consuming –Simpler

Semester 1, 2003 Week 7 CSE9020 / 16 Automated vs. Manual Testing n Automated –A test harness runs the test scripts –Analysis of the results is automated and the results are logged –Testing effort is reduced and it is more efficient in the longer term n Note: The actual test scripts are not generated automatically

Semester 1, 2003 Week 7 CSE9020 / 17 Boundary Value Analysis n Guidelines: –Break up the system in to components that have to be tested –Determine the boundary values for the components –Design test cases based on the boundary values n Limitation: Works well with independent variables (or components) with well defined boundaries

Semester 1, 2003 Week 7 CSE9020 / 18 Equivalence Classes n Motivations: Completeness with minimal or no redundancies n An equivalence class is a partition of a set n Guidelines: –Identify equivalence classes –Design test cases for each equivalence class n Example: A variable can take only values from 0 to 9 inclusive.

Semester 1, 2003 Week 7 CSE9020 / 19 Decision Tables n Condition/Action rules are presented in a tabular form n Test Cases: Conditions are inputs, Actions are outputs Fixed rate accountTTFFF Variable rate accountFFTTF Consumption < 100 KWHTFTF Consumption >= 100 KWHFTFT Minimum monthly chargeX Schedule A billingXX Schedule B billingX Other treatmentX

Semester 1, 2003 Week 7 CSE9020 / 20 Defect Fixing Strategies n Document Defects –full and complete information so the defect can be reproduced –Pass it back to the developer n Fix –Document the fix –Pass it back to the tester n Re-test, regression tests

Semester 1, 2003 Week 7 CSE9020 / 21 Test Process n Develop a test plan (and agree on it) n Specify the test procedure n Perform boundary value analysis / equivalence classes etc.

Semester 1, 2003 Week 7 CSE9020 / 22 Test Process n Design and implement test cases n Run the tests n Analyse and document results n Fix Defects then Run the tests etc etc -and agree that the fixes have worked

Semester 1, 2003 Week 7 CSE9020 / 23 Testing Web Sites n Are essentially client/server applications n Consider: –Interaction between html pages –internet connections –firewalls –applications that run in web pages (client) applets javascript.net approach

Semester 1, 2003 Week 7 CSE9020 / 24 Testing Web Sites –applications that run on the server cgi scripts database interfaces logging applications dynamic page generators –variety of servers, browsers (versions) –expected loads on the server, expected performance on client, and server investigate web testing tools n

Semester 1, 2003 Week 7 CSE9020 / 25 Testing Web Sites n Who is the target audience? –Browsers, connection speed intranet (fast) or internet (may be sloooooow!) –Usability guidelines Pages should be 3-5 screens max., otherwise provide internal links within the page Use a consistent layout, navigation Make pages browser-independent, or generate for a specific browser Include page owner, revision date, contact link NO dead-end pages!

Semester 1, 2003 Week 7 CSE9020 / 26 Testing Web Sites n On-going –Performance Testing –Link Testing –Security Testing –HTML validation (browsers change!)

Semester 1, 2003 Week 7 CSE9020 / 27 In Summary... n Why? n Who? n When? n What? n How?

Semester 1, 2003 Week 7 CSE9020 / 28 Bottom Line n Testing is expensive n Testing is necessary n You are never going to test everything –use Risk Analysis to determine where testing should be focused

Semester 1, 2003 Week 7 CSE9020 / 29 Bottom Line n Your IE Project is not complete until your client has completed the Acceptance Testing to their satisfaction, and signed-off the project!

Semester 1, 2003 Week 7 CSE9020 / 30 Bottom Line n Most Essential Make sure that –the system –and the group members maintain versioning control (link Software Testing and Quality Assurance with with Risks and Ethics)

Semester 1, 2003 Week 7 CSE9020 / 31 More Maintenance Pilot Entry: Left inside main tyre almost needs replacement Mechanic Entry: Almost replaced left inside main tyre Pilot Entry: Test flight OK, except autoland very rough Mechanic entry Autoland not installed on this aircraft