Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 1 Informatics 43 Introduction to Software Engineering Lecture 9-1 May 26, 2015 Emily Navarro Duplication of course material for any commercial purpose without the explicit written permission of the professor is prohibited.

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 2 Announcement No discussion this week

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 3 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 4 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 5 Two Approaches Black box testing – Specification-based testing – Test cases designed, selected, and ran based on specifications – Scale: tests the higher-level system behavior – Drawback: less systematic White box testing – Structural testing – Test cases designed, selected, and ran based on structure of the code – Scale: tests the nitty-gritty – Drawbacks: need access to source

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 6 Black-box/Specification-Based Testing

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 7 Black-box/Specification-Based Testing Use specifications to derive test cases – Requirements – Design – Function signature Assume no access to source code Based on some kind of input domain Choose test cases that guarantee a wide range of coverage – Typical values – Boundary values – Special cases – Invalid input values

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 8 Equivalence Class Partitioning – A Systematic Approach 1.Identify the set of all possible inputs (to what is being tested) 2.Identify a basis for subdividing the set of inputs – Possible bases Size Order Structure Correctness Your creative thinking 3.Use this basis to divide the set of all possible inputs into subsets/subdomains – Subdomains may overlap slightly

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 9 Equivalence Class Partitioning – A Systematic Approach (Cont.) 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) – One test case may suffice 5.Test for each partition – “Normal” values – Boundary or edge input values (Boundary Value Analysis) Extreme values for a partition, beyond which new partitions begin Experience has shown that many errors are made at the boundaries rather than under normal conditions

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 10 Example: quizAverage Input: A list of numbers – Scores must be between 0 and 1000 (inclusive) Output: a single number which is the average of the numbers on the input list, not counting the lowest number on the list.

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 11 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) 2.Identify a basis for subdividing the set of inputs 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 12 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 13 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) Equivalence Class Partitioning with quizAverage

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 14 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 0 elements, 1, 2-10, 11+, first, middle, last, 1, a few, all, 0-10, , From each subdomain, select [a] representative(s) to be [a] test case input(s)

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 15 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 0 elements, 1, 2-10, 11+, first, middle, last, 1, a few, all, 0-10, , From each subdomain, select [a] representative(s) to be [a] test case input(s) [], [87.3], [90,95,85], [80,81,82,83,84,85,86,87,88,89,90,91], [80,87,88,89], [87,88,80,89], [87,88,89,80], [80,87,88,89], [87,86,86,88], [88,88,88,88], [0,4,6,5,8], [17, 43, 98, 10, 100], [101, 1000]

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 16 Possible Bases List length – Empty list – One element – Two or three elements – Lots of elements Input domain: float[] Basis: list length one small empty large

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 17 Possible Bases Position of minimum score – Smallest element first – Smallest element in middle – Smallest element last Input domain: float[] Basis: position of minima somewhere in middle first last

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 18 Possible Bases Number of minima – Unique minimum – A few minima – All minima Input domain: float[] Basis: number of minima all data equal 1 minimum 2 minima

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 19 Possible Bases Magnitude of numbers – 0-10 – – Input domain: float[] Basis: magnitude of numbers

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 20 Testing Matrix Test case (input) Basis (subdomain) Expected output Notes

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 21 quizAverage 1 Test case (input) Basis: List lengthExpected output Notes EmptyOneSmallLarge ()x ! (87.3)x87.3 crashes! (90,95,85)x92.5 (80,81,82,83, 84,85,86,87, 88,89,90,91) x86.0

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 22 quizAverage 2 Test case (input) Position of minimumExpected output Notes FirstMiddleLast (80,87,88,89)x88.0 (87,88,80,89)x88.0 (99,98,0,97,96)x97.5 (87,88,89,80)x88.0

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 23 quizAverage 3 Test case (input) Number of minimaExpected output Notes OneSeveralAll (80,87,88,89)x88.0 (87,86,86,88)x87.0 (99,98,0,97,0)x73.5 (88,88,88,88)x88.0

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 24 quizAverage 4 Test case (input) Magnitude of NumbersExpected output Notes (0,4,6,5,8)x5.75 (17,43,98,11,100)x64.5 (99,98,11,97,50)x86 (101,1000)x1000

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 25 Example: Hotel Management System Consider a hotel management system that takes phone numbers as input while gathering data about the guest Imagine we want to test the “input phone number” function of the system Specification: Should give a descriptive error message if – input is less than 10 digits – input is more than 20 digits – input contains non-numeric characters What are the properties about phone numbers that we can exploit to create “valuable” partitions? – length, content (types of characters, position of invalid characters)

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 26 Input Phone Number 1 SubdomainsTest Case Input DataExpected Output 0{}Error Error Success Error Basis: Length

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 27 Input Phone Number 2 SubdomainsTest Case Input DataExpected Output all numbers success contains error contains (( error contains )123) error contains other invalid characters #76!abcerror Basis: Types of characters

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 28 Input Phone Number 3 SubdomainsTest Case Input DataExpected Output beginning& error middle123$4569error end #error Basis: Position of invalid characters

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 29 Example: Gmail Imagine we are testing the login functionality of Gmail – Input: username, password – Output: login successful or error message Two users: – Mary; maryspassword – Joe; joespassword What possible bases can we use to divide our testing into partitions? – whether the username is valid (no user matching in the system, invalid characters, length) – whether the password is valid (no no user matching in the system, invalid characters, length)

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 30 Login SubdomainsTest Case Input DataExpected Output matches the user Mary, maryspassword Joe, joespassword success doesn’t match any user Mary, newpassword Joe, anothernewpassword error matches another user Mary, joespassword Joe, maryspassword error Basis: whether the password matches the user

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 31 Example: Room Scheduler System Imagine we are testing a classroom scheduler program that handles M-F scheduling for five classrooms Room capacities – Room A: 500 – Room B: 300 – Room C: 100 – Room D: 50 – Room E: 20 All classes are 1-hour long, once per week, and can be scheduled between 8am-10pm

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 32 Example: Room Scheduler System Input – The current schedule – The number of students in the class to be scheduled – The desired time of the class to be scheduled Output – A list of available rooms that can hold the number of students, ordered from most appropriate (number of students is as close as possible to room capacity without going over) to least appropriate

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 33 Example: Room Scheduler System Example – Input: {Current schedule: Room A: M-F: 8-11am, 2-4pm Room B: T-F: 9-10am, 5-8pm Room C: F: 10am-3pm Room D: M-F: 8am-10pm Room E: M-F: 10am-5pm, 8pm-10pm; Num students: 73 Desired time: W 5-6pm} – Expected output: {Room C, Room A} Room capacities Room A: 500 Room B: 300 Room C: 100 Room D: 50 Room E: 20

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 34 Example: Room Scheduler System What possible bases can we use to divide our testing into partitions?

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 35 Schedule Room 1 SubdomainsTest Case Input DataExpected Output Basis:

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 36 Schedule Room 2 SubdomainsTest Case Input DataExpected Output Basis:

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 37 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 38 Homework 3 You will be designing test cases for ZotMyHealth using a black-box/specification-based approach – You will be provided with a specification upon which to base your testing – Your document will mainly consist of testing matrices Homework 3 is posted Deadline is Tuesday, June 2, 11:55pm to EEE

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 39 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 40 Quiz 6 Study Guide Testing (Lectures 8-2 and 9-1) – Validation/verification – How do we know when we are done? – Test-driven development – Error, fault, failure – Testing process model – Testing goals – Difference between white-box and black-box testing – Levels of testing (unit, functional/integration, system/acceptance) – Oracles – Test drivers/stubs

Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 41 Next Time White-box (Structural) Testing