IMPORTANT NOTICE TO STUDENTS: These slides are NOT to be used as a replacement for student notes. These slides are sometimes vague and incomplete on purpose to spark class discussions. Software Unit Testing CS 360 Lecture 17

Model of the Software testing process (documentation)

General characteristics of software testing Development teams should conduct effective formal reviews (technical reviews, code reviews, integration/system testing, etc.) The objective is to arrive at a superior version of the product being reviewed. By correcting defects or developing new functionality Testing begins at the component level Make sure individual functions/interfaces of all components work correctly, then move toward integration testing of the entire system. Testing is conducted mostly by the development team. Testing and debugging are different activities Testing: “based on requirements, does function A do X?” Debugging: function A doesn’t work with function B because function A calls function B incorrectly. Correct the call for function B in function A.

Software Levels of Testing (documentation) Unit (Component) testing Concentrates on each implemented component/function of the software. Used to generate documentation of how each function works. Integration testing Focuses on the design and construction of the software architecture. Individual components are combined and tested as a group. System testing The software and other system elements are tested as a whole. System testing should require no knowledge of the system architecture or code. Validation testing Requirements (Functional/Non-Functional) are validated against the implemented software to make sure it meets stakeholder requirements. Acceptance/User Testing The software is tested by members outside of the development team

When is Testing Complete? There is no definitive answer to this question. Every time a user executes the software, the program is being tested. Sadly, testing usually stops when a project is running out of time, money, or both. System/Software testing is one of the most demanded abilities for new industry employees.

Unit Testing Focuses on generating test cases for: Functional correctness/completeness Internal program logic and data structures Optimizing algorithms and performance Error handling There is some debate about what constitutes a “unit”. Here some common definitions of a unit: A single class/method/function scope The smallest chunk that can be compiled by itself A single component scope (including interfaces)

Benefits of Unit Testing Improves the quality of code by identifying defects before integration. Provides documentation on all tested components. Allows the programmer/team to refactor code and make sure it still works correctly. Testing the smaller parts of a program before integration makes integration testing easier.

Unit Testing Techniques

Targets for Unit Test Cases (documentation) Component interfaces Local data structures Boundary conditions Independent paths (basis paths) Error handling paths

Static and dynamic testing Static testing: Testing techniques that does not include execution of the software. May be manual or use computer tools. Dynamic testing: Testing by executing the software with trial data. Debugging to remove errors.

Static testing - Reviews Form of static testing that is carried out throughout the software development process.

Project Team (Technical and Code) Review Goals Understanding the problem Project consistency Collective ownership of the code Code reuse Knowledge sharing in the team Improving the programmer! Error diagnosis Scheduling and management control

Static testing: Code Reviews Formal program reviews whose objectives are to verify functionality and detect faults Code is read and reviewed line by line. 150 to 250 lines of code in 2 hour meeting. Use checklist of common errors. Requires team commitment and face to face meetings.

Dynamic testing: Unit testing, Basis paths (documentation) The objective of path testing is to build a set of test cases so that each path through the program is executed at least once. Ensures statement/branch coverage. If every condition in a compound condition is considered: Condition coverage can be achieved as well. Steps for basis path testing: Draw a (control) flow graph using the source code. Line numbers can dictate each node in the graph. Calculate the cyclomatic complexity using the flow graph. Studies have shown: CC is directly related to the number of errors in the source code. CC of 10 is a practical upper limit for testing a single component. Determine the basis set of linearly independent paths. Design test cases to exercise each path in the basis set.

Dynamic testing: Unit testing, Basis paths (documentation) Flow Graphs Used to depict program control structure. Can be drawn from a piece of source code. Flow Graph Notation: composed of edges and nodes. An edge starts from one node and ends at another node.

Dynamic testing: Unit testing, Basis paths (documentation) Flow graph for extreme algorithm

Dynamic testing: Unit testing, Basis paths (documentation) Calculating cyclomatic complexity: E = number of edges N = number of nodes P = set of connected components CC = E – N + 2P CC = 11 – 10 + 2 = 3 Independent paths through the program: 12, 1, 2, 3, 5, 6, 7, 10 \\one element 12, 1, 2, 3, 5, 6, 7, 8, 7, 10\\max in 1st position 12, 1, 2, 3, 5, 6, 7, 8, 9, 7, 10\\searching for max Test cases should be derived so that all of these paths are executed

Dynamic testing: Unit testing, Basis paths (documentation) Designing the test cases: Path 1 test case: 12, 1, 2, 3, 5, 6, 7, 10 Input data: [4] Expected output: 4 Path 2 test case: 12, 1, 2, 3, 5, 6, 7, 8, 7, 10 Input data: [6, 2, 5, 1, 3] Expected output: 6 Path 3 test case: 12, 1, 2, 3, 5, 6, 7, 8, 9, 7, 10 Input data: [5, 2, 1, 1, 8, 3, 4] Expected output: 8

Dynamic testing: Unit testing, Basis paths Example 2 (documentation) Flow graph for Fibonacci algorithm

Dynamic testing: Unit testing, Basis paths Example 2 (documentation) Calculating cyclomatic complexity: CC = E – N + 2P CC = 13 – 12 + 2 = 3 Independent paths through the program: 1, 2, 3, 4, 7, 8, 9 \\ n == 0 1, 2, 3, 4, 7, 8, 9, 10, 15 \\ n == 1 1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13 \\ n > 1

Dynamic testing: Unit testing, Basis paths Example 2 (documentation) Designing the test cases: 0,1, 1, 2, 3, 5, 8, … Path 1 test case: 1, 2, 3, 4, 7, 8, 9 Input data: 0 Expected output: 0 Path 2 test case: 1, 2, 3, 4, 7, 8, 9, 10, 15 Input data: 1 Expected output: 1 Path 3 test case: 1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13 Input data: 2

Team Activity: Unit testing, Basis paths Generate the flow graph, calculate the cyclomatic complexity, and define the independent paths for the following program:

Documentation of testing (documentation) Every project needs a test plan that documents the testing procedures for completeness, visibility, and for future maintenance. It should include: Description of testing approach. List of test cases and related bugs. Procedures for running the tests. Test analysis report.

Unit Test Cases (documentation) Unit Test Plan Scope (In Scope – Out of Scope) In Scope Out of Scope In Scope List features/functions that are tested (numbered test cases).   Out of Scope List features/functions that are not tested.

Unit Test Cases (documentation) ID Test Cases Input Value Expected Output Pass/Fail Date Tested # Identify the test cases along with the expected results. mm/dd/yyyy 1 Example: Test Procedure: Login with a user account name and password. Username: abc Password: abc Expected Results: Successful user login. otherwise, an error will be displayed for the wrong credentials.   Id: abc Pw: abc Pw: abb Id: aba Successful login Username/Password doesn’t match Pass 11/08/2018

Guidelines for Unit Testing Unit tests should be fully automated. These tests should be executed often to verify correct functionality. Keep unit tests small and fast. Ideally, unit tests should be executed before every code check in. Small/fast tests reduce development turnaround time. Make unit tests easy to run. See guideline 1.

Guidelines for Unit Testing Analyze and fix failing unit tests and test cases immediately. Test developers are responsible for making sure a new unit tests run successfully before using them. If a test case fails, the entire team should move their focus to the problem to make sure it gets resolved. Keep unit tests independent. Test cases should never depend on other test cases, nor should they depend on the ordering in which they are executed. Name unit tests properly. Each unit test should verify the functionally of one distinct component feature. Typical naming convention: test[name], testAddUser, testDBConnection, etc.

Guidelines for Unit Testing Provide negative unit test cases. These tests intentionally misuse the code to verify robustness and appropriate error handling. Write unit test cases to reproduce errors. When an error is uncovered, isolate the test case to reproduce the error and use this test as a success criteria when fixing the code. Know unit testing limitations. Unit tests can never prove 100% correctness of code. Prioritize unit testing (and other types of testing). Understand the trade-off between the cost of writing test cases, and the cost of not writing test cases.