(1) Unit Testing and Test Planning CS2110: SW Development Methods These slides design for use in lab. They supplement more complete slides used in lecture on the topic of testing and verification.

(2) Some Definitions Verification: showing that something meets specifications –We verify “against” something –Important: if you don’t have requirements specifications documented, then you can’t verify! –If behavior is not defined, then what counts as incorrect behavior? Verification can happen on any kind of software work- product –designs, code, etc.

(3) Types of Testing Unit testing –An individual class or function is tested thoroughly –Done as part of coding process –Done by the programmer him/herself –Usually by a test harness (or driver) or using a test-class Integration testing –Multiple parts of a system are combined and tested to see if they work together Acceptance testing –The customer is involved. (Alpha or beta testing)

(4) Lab on Monday/Tuesday The JUnit framework is a tool to help people generate and execute unit tests in a systematic way. Stand-alone and also built into Eclipse Example JUnit test –A class contains a set of test methods –setUp() runs before each test –Methods with name starting with “test…” are run –“assert” methods check a program condition public class CoffeeMakerTest extends TestCase { private CoffeeMaker cm; private Recipe recipe1; public void setUp() { cm = new CoffeeMaker(); recipe1 = new Recipe(); recipe1.setName("Coffee"); recipe1.setPrice(50); recipe1.setAmtCoffee(6); recipe1.setAmtMilk(1); } public void testAddRecipe() { assertTrue(cm.addRecipe(recipe1)); }

JUnit and a Test Plan JUnit is good for: –Creating tests and keeping them with your code –Running them easily and seeing results quickly –Running them over and over. (Can run inside and outside Eclipse, e.g. in an automatic way.) But it’s a tool. You can plan and design your tests first so you use the tool well. The plan includes: –List of test cases –Each with a purpose/goal, what you expect to happen for a given input (5)

(6) Planning Your Tests How do we plan a “test case”? Plan: Define three parts first: –A purpose (why are we doing this particular test?) –An input –An expected result Execute: We run the test to see if observed output matched the expected result –It’s really just like a scientific experiment When testing, have a plan –At least in your head, but for serious testing, write it out ahead of time. –Don’t test blindly or randomly – you’ll waste your time

(7) Planning Your Tests These are poorly specified test cases! –Is any move OK? Be completely precise about situation, inputs, and expected outputs! Test IDDescriptionExpected Results Actual Results 1Player 1 rolls dice and moves. Player 1 moves on the board. 2Player 2 rolls dice and moves. Player 2 moves on the board.

(8) Planning Your Tests Why are these test cases better than the previous ones? Another good example later in the slides Test IDDescriptionExpected Results Actual Results 1Precondition: Game is in test mode, GameBoard is start configuration, and game begins. Number of Players: 2 Player 1 dice roll: 3 Player 1 moves to square Blue 3. 2Precondition: TestID 1 completed successfully. Player 2 dice roll: 3 Player 2 moves to square Blue 3.

How to Define Good Test Cases How do you come up with a good set of test cases? For each class –Test each method –Consider different situations of the “state” of the object and the parameters passed to the method Example: class BookShelf stored Book objects –Test method: boolean remove(int) in BookShelf –Should test: “Valid” removal calls: first item, last item, middle item Other calls: index too large or small What if BookShelf is empty? Make sure list actually changes! Make sure return value is correct! (9)

Types of Errors to Consider Details of these types of error or testing given in the following slides: –Boundary values –Off-by-one errors –Invalid inputs (10)

(11) Unit Testing: Boundary values Often programs misbehave because we make mistakes dealing with the “extremes” Examples: –The last item or first item in a collection –The start or end of a range of values Equivalent classes of test cases –Don’t have test cases or data that test the same situation (e.g. the middle of a range) Boundary or extreme conditions –An empty list, an empty file –One item in a list or file

(12) Unit Testing: Off-by-one errors Related to boundary conditions Using the nth value instead of n-1 Using 0 instead of 1 Executing a loop body once when it the body should not be executed at all Etc.

(13) Unit Testing: Invalid Inputs Rely on pre-conditions defined for your functions –They specify what the agreement is about what you are responsible for when its time to handle mis-use If a function is mis-used in this way –Return an error condition if the method provides a way to do this –Otherwise, halt program! Better than producing unexpected results Java has assertions that will test a condition and halt program. (These are not the same a JUnit assertions.)

(14) Example: Program with Student and Course classes Homework 0 this term –Define class Student methods include: –Define class Course methods include: Focus on defining: –Black-box test-cases for the method boolean drop(Course c) in class Student Let’s make a list of possible test-cases for this method!

(15) Example of Full Test-Case Description Test Case #1: Drop last course in list with >1 item –Purpose: Drop Course c for Student s where that course is the last course stored in the Student object's courses list. This tests the boundary condition of last item in list for Student.drop(c). –Input: Student s is enrolled in three courses, and the course to be dropped is the last of these stored in the list. –Expected output: The Student s is unchanged except that only two courses are stored for that student. Also, the list of students stored for that course is altered by having that student removed.

(16) Other Test Cases for this Program There are a lot! Boundary conditions? –for Student.drop(): end of list, start of list –same for Course.drop() –Question: is this affected by whether you used ArrayList instead of Java arrays? (yes) –Note: also test “typical” case, item in middle of list –Empty lists: Course.cancel(), Student.dropAll() Off-by-one errors? –Really included in boundary condition testing here

(17) Other Test Cases for this Program Invalid inputs? –See the specifications! –Test return values For boolean, test both success and failure Remember: make it fail! Test that your program handles that. Test of equals() methods? –If you get this wrong, trouble! –Easy to test. (Easy to forget to.)