1. IMPORTANT NOTICE TO STUDENTS:
These slides are NOT to be used as a replacement for student notes.
These slides are sometimes vague and incomplete on purpose to spark class discussions.
Software Unit Testing
CS 360
Lecture 17

2. Model of the Software testing process (documentation)

3. General characteristics of software testing
Development teams should conduct effective formal reviews (technical reviews, code reviews, integration/system testing, etc.)
The objective is to arrive at a superior version of the product being reviewed.
By correcting defects or developing new functionality
Testing begins at the component level
Make sure individual functions/interfaces of all components work correctly, then move toward integration testing of the entire system.
Testing is conducted mostly by the development team.
Testing and debugging are different activities
Testing: “based on requirements, does function A do X?”
Debugging: function A doesn’t work with function B because function A calls function B incorrectly. Correct the call for function B in function A.

4. Software Levels of Testing (documentation)
Unit (Component) testing
Concentrates on each implemented component/function of the software.
Used to generate documentation of how each function works.
Integration testing
Focuses on the design and construction of the software architecture.
Individual components are combined and tested as a group.
System testing
The software and other system elements are tested as a whole.
System testing should require no knowledge of the system architecture or code.
Validation testing
Requirements (Functional/Non-Functional) are validated against the implemented software to make sure it meets stakeholder requirements.
Acceptance/User Testing
The software is tested by members outside of the development team

5. When is Testing Complete?
There is no definitive answer to this question.
Every time a user executes the software, the program is being tested.
Sadly, testing usually stops when a project is running out of time, money, or both.
System/Software testing is one of the most demanded abilities for new industry employees.

6. Unit Testing Focuses on generating test cases for:
Functional correctness/completeness
Internal program logic and data structures
Optimizing algorithms and performance
Error handling
There is some debate about what constitutes a “unit”.
Here some common definitions of a unit:
A single class/method/function scope
The smallest chunk that can be compiled by itself
A single component scope (including interfaces)

7. Benefits of Unit Testing
Improves the quality of code by identifying defects before integration.
Provides documentation on all tested components.
Allows the programmer/team to refactor code and make sure it still works correctly.
Testing the smaller parts of a program before integration makes integration testing easier.

8. Unit Testing Techniques

9. Targets for Unit Test Cases (documentation)
Component interfaces
Local data structures
Boundary conditions
Independent paths (basis paths)
Error handling paths

10. Static and dynamic testing
Static testing:
Testing techniques that does not include execution of the software.
May be manual or use computer tools.
Dynamic testing:
Testing by executing the software with trial data.
Debugging to remove errors.

11. Static testing - Reviews
Form of static testing that is carried out throughout the software development process.

12. Project Team (Technical and Code) Review Goals
Understanding the problem
Project consistency
Collective ownership of the code
Code reuse
Knowledge sharing in the team
Improving the programmer!
Error diagnosis
Scheduling and management control

13. Static testing: Code Reviews
Formal program reviews whose objectives are to verify functionality and detect faults
Code is read and reviewed line by line.
150 to 250 lines of code in 2 hour meeting.
Use checklist of common errors.
Requires team commitment and face to
face meetings.

14. Dynamic testing: Unit testing, Basis paths (documentation)
The objective of path testing is to build a set of test cases so that each path through the program is executed at least once.
Ensures statement/branch coverage.
If every condition in a compound condition is considered:
Condition coverage can be achieved as well.
Steps for basis path testing:
Draw a (control) flow graph using the source code.
Line numbers can dictate each node in the graph.
Calculate the cyclomatic complexity using the flow graph.
Studies have shown:
CC is directly related to the number of errors in the source code.
CC of 10 is a practical upper limit for testing a single component.
Determine the basis set of linearly independent paths.
Design test cases to exercise each path in the basis set.

15. Dynamic testing: Unit testing, Basis paths (documentation)
Flow Graphs
Used to depict program control structure.
Can be drawn from a piece of source code.
Flow Graph Notation: composed of edges and nodes.
An edge starts from one node and ends at another node.

16. Dynamic testing: Unit testing, Basis paths (documentation)
Flow graph for extreme algorithm

17. Dynamic testing: Unit testing, Basis paths (documentation)
Calculating cyclomatic complexity:
E = number of edges
N = number of nodes
P = set of connected components
CC = E – N + 2P
CC = 11 – = 3
Independent paths through the program:
12, 1, 2, 3, 5, 6, 7, 10 \\one element
12, 1, 2, 3, 5, 6, 7, 8, 7, 10\\max in 1st position
12, 1, 2, 3, 5, 6, 7, 8, 9, 7, 10\\searching for max
Test cases should be derived so that all of these paths are executed

18. Dynamic testing: Unit testing, Basis paths (documentation)
Designing the test cases:
Path 1 test case:
12, 1, 2, 3, 5, 6, 7, 10
Input data: [4]
Expected output: 4
Path 2 test case:
12, 1, 2, 3, 5, 6, 7, 8, 7, 10
Input data: [6, 2, 5, 1, 3]
Expected output: 6
Path 3 test case:
12, 1, 2, 3, 5, 6, 7, 8, 9, 7, 10
Input data: [5, 2, 1, 1, 8, 3, 4]
Expected output: 8

19. Dynamic testing: Unit testing, Basis paths Example 2 (documentation)
Flow graph for Fibonacci algorithm

20. Dynamic testing: Unit testing, Basis paths Example 2 (documentation)
Calculating cyclomatic complexity:
CC = E – N + 2P
CC = 13 – = 3
Independent paths through the program:
1, 2, 3, 4, 7, 8, 9 \\ n == 0
1, 2, 3, 4, 7, 8, 9, 10, 15 \\ n == 1
1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13 \\ n > 1

21. Dynamic testing: Unit testing, Basis paths Example 2 (documentation)
Designing the test cases: 0,1, 1, 2, 3, 5, 8, …
Path 1 test case:
1, 2, 3, 4, 7, 8, 9
Input data: 0
Expected output: 0
Path 2 test case:
1, 2, 3, 4, 7, 8, 9, 10, 15
Input data: 1
Expected output: 1
Path 3 test case:
1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13
Input data: 2

22. Team Activity: Unit testing, Basis paths
Generate the flow graph, calculate the cyclomatic complexity, and define the independent paths for the following program:

23. Documentation of testing (documentation)
Every project needs a test plan that documents the testing procedures for completeness, visibility, and for future maintenance.
It should include:
Description of testing approach.
List of test cases and related bugs.
Procedures for running the tests.
Test analysis report.

24. Unit Test Cases (documentation)
Unit Test Plan Scope (In Scope – Out of Scope)
In Scope
Out of Scope
In Scope List
features/functions that are tested (numbered test cases).
Out of Scope List features/functions that are not tested.

25. Unit Test Cases (documentation)ID
Test Cases
Input Value
Expected Output
Pass/Fail
Date Tested #
Identify the test cases along with the expected results.
mm/dd/yyyy 1
Example:
Test Procedure:
Login with a user account name and password.
Username: abc
Password: abc
Expected Results:
Successful user login. otherwise, an error will be displayed for the wrong credentials.
Id: abc
Pw: abc
Pw: abb
Id: aba
Successful login
Username/Password doesn’t match
Pass
11/08/2018

26. Guidelines for Unit Testing
Unit tests should be fully automated.
These tests should be executed often to verify correct functionality.
Keep unit tests small and fast.
Ideally, unit tests should be executed before every code check in. Small/fast tests reduce development turnaround time.
Make unit tests easy to run.
See guideline 1.

27. Guidelines for Unit Testing
Analyze and fix failing unit tests and test cases immediately.
Test developers are responsible for making sure a new unit tests run successfully before using them.
If a test case fails, the entire team should move their focus to the problem to make sure it gets resolved.
Keep unit tests independent.
Test cases should never depend on other test cases, nor should they depend on the ordering in which they are executed.
Name unit tests properly.
Each unit test should verify the functionally of one distinct component feature. Typical naming convention: test[name], testAddUser, testDBConnection, etc.

28. Guidelines for Unit Testing
Provide negative unit test cases.
These tests intentionally misuse the code to verify robustness and appropriate error handling.
Write unit test cases to reproduce errors.
When an error is uncovered, isolate the test case to reproduce the error and use this test as a success criteria when fixing the code.
Know unit testing limitations.
Unit tests can never prove 100% correctness of code.
Prioritize unit testing (and other types of testing).
Understand the trade-off between the cost of writing test cases, and the cost of not writing test cases.