1 March 12, Testing William Cohen NCSU CSC 591W March 12, 2008

2 2 Reasons for Testing ● Make sure that the software works as expected ● Minimize obvious failures ● Ensure that previous failures do not creep back into software.

3 March 12, Caveat on Testing ● Testing only demonstrate instants of failure ● Testing does not prove that the software will work in all cases

4 March 12, Black Box/White Box Testing ● White box: able to inspect code and base tests on that information ● Black box: unable to inspect codes operation

5 March 12, Types of testing ● Unit testing: check that building block work ● Integration testing: make sure modules work together ● Functional testing: check that things operate as expected ● Regression testing: previous problems converted into test cases ● System testing ● System integration testing ● Performance testing ● Acceptance testing

6 March 12, Test Case ● Specific inputs ● Action or procedure to perform ● Expected output (for success or failure) ● Need to determine pass or fail ● Want to make it possible for the machine to determine success (who wants to manually run tests)

7 March 12, Test Suite ● Collection of test cases ● Often automated scripts ● Want to make as easy as possible for people to run: ● Want to run them frequently to catch bugs early, e.g. Firefox tinderbox ● Want people running them in many environments ● Want to make it easy for people to add tests: ● New feature added to code, new tests added to testsuite

8 March 12, Writing Testsuite ● May use simple script to check whether things work ● Example tools: ● Dejagnu, for batch oriented testing ● Dogtail, scripting for GUI applications

9 March 12, Testsuite Example

10 March 12, Code Coverage ● Show which areas of code are exercised during testing ● Can indicate which sections of program have not executed ● GCC: ● Generate with “-fprofile-arcs -ftest-coverage” options ● Render with lcov

11 March 12, Code Coverage Example