In the Senior Design Center Software Testing In the Senior Design Center few easy questions: 1. Why test software? 2. Can you ship untested software in industry?

Senior Design Center - Fall 2015 Software Testing Why now? Development method requires it Create & maintain a test plan as you develop Possible need to design for testing Senior Design Center - Fall 2015

Senior Design Center - Fall 2015 What We Expect Your project includes a test plan Documented in your written reports With updates included in your oral reports What’s a test plan? “Oh, we definitely plan to thoroughly test our project.” “We’ll be using Selenium and JUnit to test our project.” Not a Test Plan Also Not a Test Plan Senior Design Center - Fall 2015

Senior Design Center - Fall 2015 What We Expect Unit testing Usually automated / easily repeatable Usually white-box tests So, you can use metrics like code coverage to show you’ve done a good job reasonably thorough job executing paths that may surface faults, failures, or errors Acceptance testing Usually manual (still repeatable, but with effort) You can automate yours, if you want. Usually black box tests of the whole system You can compare against requirements to show you’ve done a good job. Senior Design Center - Fall 2015

Senior Design Center - Fall 2015 Acceptance Testing A suite of multiple acceptance test cases Usually a few for each functional requirement. Maybe a few supporting non-functional requirements Performance / resource usage? Response Time? These can be written early As you develop requirements Revise and extend as the requirements mature What does a test case look like? Senior Design Center - Fall 2015

(Bad) Acceptance Test Cases Test ID Description Expected Results Actual Results restaurantSearch1 Preconditions: The system is loaded. Jason is a registered user of the system. Jason has authenticated and is at the homepage. Steps: Enter a type of restaurant. System reports all the nearby restaurants of that type. (Even if they are not as expected, or not yet implemented) Unique IDs! Specific Steps and Results?

(Better) Acceptance Test Cases Test ID Description Expected Results Actual Results restaurantSearch1 Preconditions: The system is loaded. Jason is a registered user of the system. Jason has authenticated and is at the homepage. Steps: Enter “sushi” as the restaurant type. System reports all the nearby sushi restaurants. FAIL Why? Still vague. Better

(OK) Acceptance Test Cases Test ID Description Expected Results Actual Results restaurantSearch1 Preconditions: The system is loaded. Jason is a registered user of the system. Jason has authenticated and is at the homepage. Steps: Enter “sushi” as the restaurant type. System reports: Sushi Hut (1 mile away) McSushi (3 miles away) Sushi King (4 miles away) FAIL: The interface does not have an input field that allows a user to indicate restaurant type Good But, how do you know you’ll get this?

Senior Design Center - Fall 2015 Acceptance Testing Acceptance tests are usually supported by test data Created by the team Example users, example accounts, example data Should be simple (so you and anyone else can tell what the right system behavior should be) Acceptance test plan should include Explanation of what you’re testing How to set up for each suite of tests What the test data is (are) like The table of test cases Discussion of test results. Senior Design Center - Fall 2015

Pro-Tip for Acceptance Testing Don’t copy-and-paste From expected to actual results  Senior Design Center - Fall 2015

Pro-Tip for Acceptance Testing Don’t copy-and-paste From expected to actual results  From one test case to another  This creates a maintenance problem. Tests should be independent of each other Senior Design Center - Fall 2015

Pro-Tip for Acceptance Testing Don’t copy-and paste From expected to actual results  From one test case to another  This creates a maintenance problem. Tests should be independent of each other Instead, factor common steps into the test set-up. Senior Design Center - Fall 2015

Senior Design Center - Fall 2015 Unit Testing With unit testing, we take advantage of what we know about the implementation. We have access to the source code We know how it’s organized into components We know the dependency structure We can do things we couldn’t do in acceptance testing We can test individual components (in an order that makes sense) We can measure code coverage (to see if we’ve done a good job reasonably thorough job executing paths that may surface faults, failures, or errors) Senior Design Center - Fall 2015

Senior Design Center - Fall 2015 Unit Testing Coverage expectations The Course Pack says to plan for 70% statement coverage But, the right coverage targets depend on the component under test You could achieve 100% statement coverage … and still have bugs Coverage != Quality Senior Design Center - Fall 2015

Senior Design Center - Fall 2015 Unit Testing Your project will probably have different components Maybe in different languages Maybe requiring different approaches to unit testing This is going to require some explanation Unit test plan should include Explanation of what you’re testing Explanation of your choice of testing tools How to run the unit tests Code coverage report Discussion of test results Senior Design Center - Fall 2015

Testing in OPR 1 You can report on: Testing strategy Testing tools Progress so far A glimpse into your acceptance test plan? Number of tests, how they’re organized, number passing? Preliminary unit test results? Number of tests, how they’re organized, coverage?

Traceability to Requirements The quantity of black-box tests is not important! Instead: are you thoroughly testing against the requirements? A mapping of test cases to requirements can identify areas that need more testing traceability matrix FR 1.1 FR 1.2 UC2-S3 UC3-AF1 testID1 X testID2 testID3 testID4 Senior Design Center - Fall 2015

Testing in Senior Design Every project is different You’ll need to figure out the right way to assure the quality of your project With help from your sponsor … and your technical advisors This may include some additional forms of testing Performance testing? Usability testing? Stress testing? Ask questions and get help early Testing resources on the SDC website Senior Design Center - Fall 2015