Topic 13Summer ICS 52: Introduction to Software Engineering Lecture Notes for Summer Quarter, 2003 Michele Rousseau Topic 13 Partially based on lecture notes written by Sommerville, Frost, Van Der Hoek, Taylor & Tonne. Duplication of course material for any commercial purpose without the written permission of the lecturers is prohibited

Topic 13Summer Today’s Objectives l Testing continued l How to use a Test Matrix to organize test cases l More examples of test coverage l Formal Specification based testing

Topic 13Summer Recap: Choosing Test Cases l Identify what to test functionality from requirements (black box) classes, methods, statements (white box) l Determine set of all inputs l Identify one or more bases l For each basis, define equivalence partitions l For each eq. partition, choose test cases l Use combinations of eq. partitions when appropriate l For each test input, determine correct output l Use a matrix to organize your test cases Rows: test cases Columns: equivalence partitions

Topic 13Summer Example 1float homeworkAverage(float[] scores) { 2 float min = 99999; 3 float total = 0; 4 for (int i = 0 ; i < scores.length ; i++) { 5 if (scores[i] < min) 6 min = scores[i]; 7 total += scores[i]; 8 } 9 total = total – min; 10 return total / (scores.length – 1); 11}

Topic 13Summer Testing Matrix Test case (input) Basis (subdomain) Equivalence Partitions Expected output Notes

Topic 13Summer Testing Matrix - homeworkAverage 1 Test case (input) Basis: Array length Expected output Notes EmptyOneSmallLarge ()x0.0 (87.3)x87.3 (90,95,85)x92.5 (80,81,82,83, 84,85,86,87, 88,89,90,91) x86.0

Topic 13Summer Testing Matrix - homeworkAverage 2 Test case (input) Basis:Position of minimum Expected output Notes FirstMiddleLast (80,87,88,89)x88.0 (87,88,80,89)x88.0 (99,98,0,97,96)x97.5 (87,88,89,80)x88.0

Topic 13Summer Testing Matrix - homeworkAverage 3 Test case (input) Basis:Number of minimaExpected output Notes OneSeveralAll (80,87,88,89)x88.0 (87,86,86,88)x87.0 (99,98,0,97,0)x73.5 (88,88,88,88)x88.0

Topic 13Summer Security Dialog Example

Topic 13Summer Testing Matrix - Security Dialog 1 Test case (input) Choice of Button (Black Box) Expected output Notes Ok ButtonCancel Button Press Ok x See Security Dialog #2 Enter ID, Press Cancel x Text Boxes are cleared, dialog remains Don’t Enter ID, Press Cancel x Text Boxes are cleared, dialog remains

Topic 13Summer Testing Matrix - Security Dialog 2 Test case (input) Basis: Correctness of Input, with OK Button Press Expected output Notes Valid ID, Correct PW Valid ID, Incorrect PW Invalid. ID. Test1, x Proceed to Main Screen Test1, x Audible Beep, Dialog Remains Tes1, x Audible Beep, Dialog Remains

Topic 13Summer Another Structural Testing Example // In a linked list class Node getSecondElement() { 1 Node head = getHead(); 2 if (head == null) 3 return null; 4 if (head.next == null) 5 return null; 6 return head.next.node; }

Topic 13Summer Basis _______________________Test Case (Input) Expected Output Notes Testing Matrix - getSecondElement (White) All-Statement (white box) null list x x xnull (A  null) x x x x null (A  B  x x x x B null) (A  B  x x x x B C  null)

Topic 13Summer Basis _______________________Test Case (Input) Expected Output Notes Testing Matrix-homeworkAverage 3(White) All statements (White box) 123,4,5,6,7,8,11,4,5,8,11,4,13,14,15 Scores[30,30] x“all scores are the same” 30

Topic 13Summer Test Oracles l Where does “expected output” come from? l Critical to testing l Difficult to create systematically l Typically done with a lot of guesswork great dependence on the intuition of testers l Formal specifications make it possible to automate oracles A test oracle is a mechanism for deciding whether a test case execution failed or succeeded

Topic 13Summer Formal Verification l Techniques for proving consistency between two software descriptions subject to assumptions of proof system only as good as formal specification to prove consistency of specification to prove correctness of implementation does not show other qualities, especially the subjective ones has not been shown to scale up to large-scale software systems only informal techniques for validating against user needs

Topic 13Summer Verification with Formal Specifications

Topic 13Summer Testing with Formal Specifications

Topic 13Summer Specification-based testing should augment implementation-based testing, not replace it Specification-based testing should augment implementation-based testing, not replace it Formal Specification-Based Testing l Formalize functional testing for formal specifications l Extend implementation-based techniques to be applicable to formal specifications l Dependent on the specification language Model-based specifications: test cases selected to structurally cover specification Algebraic specifications: test cases selected to test against axioms or axioms tested by interpretation on test data State-based specifications: test cases selected to cover operations’ state changes

Topic 13Summer Something to think about: Testing Process/Integration l It is generally accepted that no single technique is sufficient to verify and validate software l How do you choose? strengths and weaknesses integration issues incremental issues