1. 1 March 12, 2008 1 Testing William Cohen NCSU CSC 591W March 12, 2008

2. 2 2 Reasons for Testing ● Make sure that the software works as expected ● Minimize obvious failures ● Ensure that previous failures do not creep back into software.

3. 3 March 12, 2008 3 Caveat on Testing ● Testing only demonstrate instants of failure ● Testing does not prove that the software will work in all cases

4. 4 March 12, 2008 4 Black Box/White Box Testing ● White box: able to inspect code and base tests on that information ● Black box: unable to inspect codes operation

5. 5 March 12, 2008 5 Types of testing ● Unit testing: check that building block work ● Integration testing: make sure modules work together ● Functional testing: check that things operate as expected ● Regression testing: previous problems converted into test cases ● System testing ● System integration testing ● Performance testing ● Acceptance testing

6. 6 March 12, 2008 6 Test Case ● Specific inputs ● Action or procedure to perform ● Expected output (for success or failure) ● Need to determine pass or fail ● Want to make it possible for the machine to determine success (who wants to manually run tests)

7. 7 March 12, 2008 7 Test Suite ● Collection of test cases ● Often automated scripts ● Want to make as easy as possible for people to run: ● Want to run them frequently to catch bugs early, e.g. Firefox tinderbox ● Want people running them in many environments ● Want to make it easy for people to add tests: ● New feature added to code, new tests added to testsuite

8. 8 March 12, 2008 8 Writing Testsuite ● May use simple script to check whether things work ● Example tools: ● Dejagnu, for batch oriented testing ● Dogtail, scripting for GUI applications

9. 9 March 12, 2008 9 Testsuite Example

10. 10 March 12, 2008 10 Code Coverage ● Show which areas of code are exercised during testing ● Can indicate which sections of program have not executed ● GCC: ● Generate with “-fprofile-arcs -ftest-coverage” options ● Render with lcov

11. 11 March 12, 2008 11 Code Coverage Example