1. Outline Types of errors Component Testing Testing Strategy
Unit testing
Integration testing
Testing Strategy
Design Patterns & Testing
System testing
Function testing
Structure Testing
Performance testing

2. How do we deal with Errors & Faults?

3. Verification?

4. Modular Redundancy?

5. Declaring the Bug a Feature?

6. Patching?

7. Testing?

8. Examples of Faults Interface specification Faults Algorithmic Faults
Expectation mismatch between the client & the server
Cannot happen with Jini
Mismatch between requirements & implementation
Algorithmic Faults
Mechanical Faults
Documentation does not match actual conditions
Timing errors
Throughput / performance errors

9. Some Observations
It is impossible to completely test any nontrivial module
Theoretical limitations: Halting problem
Practical limitations: Prohibitive in time & cost
Testing only shows the presence of bugs, not their absence

10. Testing Requires Creativity
Is testing dirty work?
An effective test requires:
Detailed understanding of the system
Knowledge of test techniques
Skill to apply these techniques in an effective & efficient manner
Book says: Testing is done best by independent testers
XP says: developers devise/do unit tests

11. Testing Activities Unit T est Unit T est Integration Functional Test
Requirements
Analysis
Document
Code
Unit
Requirements
Analysis
Document
System
Design
Document T
est
Tested
Subsystem
User
Manual
Code
Unit T
est
Tested
Subsystem
Integration
Functional
Test
Test
Functioning
System
Integrated
Subsystems
Tested Subsystem
Code
Unit T
est
All tests by developer

12. Testing Activities … Performance Acceptance Installation Test Test
Client’s
Understanding
of Requirements
Global
Requirements
User
Environment
Validated
System
Accepted
System
Functioning
System
Performance
Acceptance
Installation
Test
Test
Test
Tests by developer
Tests by client

13. Fault Handling Techniques
Fault Avoidance
Fault Detection
Fault Tolerance
Design
Methodology
Reviews
Atomic
Transactions
Modular
Redundancy
Verification
Configuration
Management
Testing
Component
Testing
Integration
Testing
System
Testing
Correctness
Debugging
Performance
Debugging

14. Component Testing Unit Testing Integration Testing
Developers confirm that class functions correctly.
Integration Testing
Developers confirm that subsystem interface functions correctly

15. System Testing System Testing Acceptance Testing
Developers confirm that the system meets its requirements
Acceptance Testing
Clients confirm that the system meets its requirements

16. Unit Testing Informal Static Analysis Testing Incremental coding
Hand execution: Read the source code
Walk-Through: Informal presentation to others
Code Inspection: Formal presentation to others
Testing
Black-box testing: Test the input/output behavior
White-box testing: Test the internal logic of the class

17. Black-box Testing I/O behavior
If for any input, we can predict the output, the module passes the test.
Generally intractable to test all possible inputs
Reduce number of tests by equivalence partitioning
Partition the inputs into equivalence classes
Devise a test case for each equivalence class.
Example: If an object is supposed to accept a negative number, include a test of 1 negative number.

18. Black-box Testing (Continued)
Selection of equivalence classes
No rules, only guidelines
Input is valid across range of values. Select 4 test cases from 3 equivalence classes:
Just below the lower boundary
Just above the lower boundary
Just below the upper boundary
Just above the upper boundary
Input is valid if it is from a discrete set. Select test cases from 2 equivalence classes:
Valid discrete value
Invalid discrete value

19. White-box Testing Thoroughness (Coverage) 4 types of white-box testing
Every statement in the component is executed at least once.
4 types of white-box testing
Statement Testing
Loop Testing
Path Testing
Branch Testing

20. White-box Testing … Statement Testing Loop Testing
Test single statements
Loop Testing
Execution of the loop is skipped.
Exception: Do-While loops
Loop is executed exactly once
Loop is executed more than once
Path testing (implies conditional testing)
Make sure all paths in the program are executed

21. White-box Testing … public int sumPositives( int[] a ) {
int total = 0;
for ( int i = 0; i < a.length; i++ )
if ( a[i] > 0 )
total += a[i];
return total; }
t1 = sumPositives( null );
t2 = sumPositives( { 0 } );
t3 = sumPositives( { 1 } );
t5 = sumPositives( { 1, 2, 3 } );
t6 = sumPositives( { 2, 0, 0 } ); // ?

22. Comparison of White & Black-box Testing
Black-box tests are devised when:
Functional requirement is defined
Class is defined
White-box tests are devised when class is implemented.
Create a test harness: A program that executes the tests.
First examination of test results is by eye.
Test Oracle: Save output of tests in a file.
Automate regression testing.

23. Bottom-up Testing Strategy
Subsystem classes in the lowest layer of the call hierarchy are tested individually
The next subsystems are tested that call the previously tested subsystems
Do until all subsystems are included in the testing
Test driver
A routine that passes a test case to a subsystem.

24. Bottom-up IntegrationC D G F E
Layer I
Layer II
Layer III
Test E
Test B, E, F
Test F
Test
A, B, C, D,
E, F, G
Test C
Test D,G
Test G

25. Bottom-Up Integration Testing
Useful for integrating:
Object-oriented systems
Systems with strict performance requirements
Real-time systems

26. Top-down Testing Strategy
Test the top layer subsystem first
Then test that layer + all subsystems called by that layer.
Do until all subsystems are incorporated into the test
A test stub is needed
A program or a method that simulates a missing method or subsystem by returning fake data.

27. Top-down Integration TestingB C D G F E
Layer I
Layer II
Layer III
Test
A, B, C, D,
E, F, G
Test A
Test A, B, C, D
Layer I
Layer I + II
All Layers

28. Sandwich Testing Strategy
Combines top-down strategy with bottom-up strategy
The system is viewed as having 3 layers
A target layer in the middle
A layer above the target
A layer below the target
The target layer is tested.

29. 1. Unit test all the classes in the selected component.
2. Do functional testing:
Define test cases that exercise all uses cases with the selected component
3. Execute performance tests
Repeat steps 1 to 3 until the full system is tested.
Keep records of the test cases.

30. System Testing Functional Testing Structure Testing
Performance Testing
Acceptance Testing
Installation Testing
Impact of requirements on system testing:
The more explicit the requirements, the easier they are to test.
Quality of use cases determines the ease of functional testing
Quality of subsystem decomposition determines the ease of structure testing
Quality of nonfunctional requirements & constraints determines the ease of performance tests

31. Structure Testing Essentially the same as white box testing.
Cover all paths in the system design
Exercise all input / output parameters of each component.
Exercise all components & methods
Use conditional & iteration testing as in unit testing.

32. Functional Testing Essentially the same as black box testing .
Test functionality of system
Test cases are designed from the RAD, center around requirements & use cases
The system is treated as black box. . .

33. Performance Testing Timing testing Stress Testing Environmental test
Stress limits of system (maximum # of users, data, peak demands, extended operation)
Configuration testing
Test the various software and hardware configurations
Security testing
Try to violate security requirements
Timing testing
Evaluate response times and time to perform a function
Environmental test
Test tolerances for heat, humidity, motion, portability
Recovery testing
Tests system’s response to presence of errors or loss of data.

34. Testing has its own Life Cycle (?)
Establish the test objectives
Design the test cases
Write the test cases
Test the test cases
Execute the tests
Evaluate the test results
Change the system
Do regression testing

35. Test Team Test Team Professional Tester Analyst System Designer User
too familiar
Programmer
with code
Analyst
System
Designer
Test
User
Team
Configuration
Management
Specialist

36. Summary Testing is still a black art Testing consists of:
component-testing (unit testing, integration testing)
system testing
Testing has its own lifecycle (?)