1. The Software Development Life Cycle: An Overview Presented by Maxwell Drew and Dan Kaiser Southwest State University Computer Science Program

2. Last Time Introduction to the Principles of Object Technology Object Oriented Design Object Technology and MSF Object Technology and RUP

3. Session 6: Testing Introduction to the Principles of Testing The Testing Process Schwan’s Development Standards MSF Testing RUP Implementation and Testing

4. Purpose of Testing In most engineering disciplines the purpose of testing is to verify that the product meets its specifications. Most physical products have only a finite number of failure modes. If your toaster doesn’t get hot then either: the cord is defective the heating element is broken the connections are bad

5. Purpose of Testing In software engineering, the purpose of testing is to find errors. When a piece of software does not work there can be an almost unlimited number of causes.

6. Terminology Two views of error. Failure - The manifestation of the error. The software does something that is contrary to its required behavior. Fault - The cause of the error. A human mistake made during some software activity that produces a failure.

7. Formal Definition of Testing According to the IEEE The purpose of software testing is to find failures so that once the software has failed under test, faults responsible for the failures can be found and fixed.

8. Table 8.1. IBM orthogonal defect classification. Fault typeMeaning FunctionFault that affects capability, end-user interfaces, product interfaces, interface with hardware architecture, or global data structure InterfaceFault in interacting with other components or drivers via calls, macros, control blocks or parameter lists CheckingFault in program logic that fails to validate data and values properly before they are used AssignmentFault in data structure or code block initialization. Timing/serializationFault that involves timing of shared and real-time resources Build/package/mergeFault that occurs because of problems in repositories, management changes, or version control DocumentationFault that affects publications and maintenance notes AlgorithmFault involving efficiency or correctness of algorithm or data structure but not design

9. Hewlett Packard Fault Classification Scheme

11. Overall Goal Our overall goal is still to validate and verify the software.

12. Validation "Are we building the right product?" The software should do what the user really requires

13. Verification "Are we building the product right?” The software should conform to its specification

14. Is a whole life-cycle process - V & V must be applied at each stage in the software process. Has two principal objectives The discovery of defects in a system The assessment of whether or not the system is usable in an operational situation. The V & V process

15. Dynamic verification Concerned with exercising and observing product behavior (testing) Static verification Concerned with analysis of the static system representation to discover problems Dynamic vs Static Verification

16. Static and Dynamic V&V

17. Statistical testing tests designed to reflect the frequency of user inputs. Used for reliability estimation. Defect testing Tests designed to discover system defects. A successful defect test is one which reveals the presence of defects in a system. Types of Testing

18. Defect testing and debugging are distinct processes Defect testing is concerned with confirming the presence of errors Debugging is concerned with locating and repairing these errors Debugging involves formulating a hypothesis about program behavior then testing these hypotheses to find the system error Testing and Debugging

19. The Debugging Process

20. Unit testing testing of individual components Module testing testing of collections of dependent components Sub-system testing testing collections of modules integrated into sub-systems Testing Stages

21. System testing testing the complete system prior to delivery Acceptance testing testing by users to check that the system satisfies requirements. Sometimes called alpha testing.

22. The Testing Process

23. Object-oriented System Testing Less closely coupled systems. Objects are not necessarily integrated into sub- systems Cluster testing. Test a group of cooperating objects Thread testing. Test a processing thread as it weaves from object to object.

24. Describe major phases of the testing process Describe trace-ability of tests to requirements Estimate overall schedule and resource allocation Describe relationship with other project plans Describe recording method for test results Test Planning and Scheduling

25. The Test Plan The testing process Requirements trace-ability Tested items Testing schedule Test recording procedures Hardware and software requirements Constraints

26. The V-model of Development

27. Testing Strategies Testing strategies are ways of approaching the testing process Different strategies may be applied at different stages of the testing process Strategies covered Top-down testing Bottom-up testing Thread testing Stress testing Back-to-back testing

28. Top-down testing

29. Start with the high-levels of a system and work your way downwards Testing strategy which is used in conjunction with top-down development Finds architectural errors May need system infrastructure before any testing is possible May be difficult to develop program stubs

30. Bottom-up testing

31. Necessary for critical infrastructure components Start with the lower levels of the system and work upward Needs test drivers to be implemented Does not find major design problems until late in the process Appropriate for object-oriented systems

32. Thread testing Suitable for real-time and object-oriented systems Based on testing an operation which involves a sequence of processing steps which thread their way through the system Start with single event threads then go on to multiple event threads Complete thread testing is impossible because of the large number of event combinations

33. Process interactions

34. Thread testing

35. Multiple-thread testing

36. Exercises the system beyond its maximum design load. Stressing the system often causes defects to come to light Stressing the system test failure behavior. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or data Particularly relevant to distributed systems which can exhibit severe degradation as a network becomes overloaded Stress testing

37. Back-to-back testing Present the same tests to different versions of the system and compare outputs. Differing outputs imply potential problems Reduces the costs of examining test results. Automatic comparison of outputs. Possible when a prototype is available or with regression testing of a new system version

38. Back-to-back testing

39. Incremental testing

40. Key Points Verification and validation are not the same thing Testing is used to establish the presence of faults and to show fitness for purpose Testing activities include unit testing, module testing, sub-system testing, integration testing and acceptance testing Object classes should be tested in O-O systems

41. Key Points Testing should be scheduled as part of the planning process. Adequate resources must be made available Test plans should be drawn up to guide the testing process Testing strategies include top-down testing, bottom-up testing, stress testing, thread testing and back-to-back testing

42. Questions?

43. Schwan’s Development Standards

45. Business Test Cases (260) Objective The objective of the Business Test Cases is to have a written plan to confirm after creation that the system meets the business needs of the customer. This plan should include a list of functionality needed at the business level. Customers should help create the Business Test Cases. Required: ProjectsSupport Optional: Deliverable: Business Test Cases Deliverable to: Systems Development Customer (Optional) Responsibility: Business Systems Planning SAP Tie: 2.4

46. Create Test Plan (440) Objective The objective of the Test Plan is to take the Business Test Cases created by the Business Systems Planning group during project analysis and create a system test plan to be used during the test phase of the project development cycle. Required: Projects Support Optional: Deliverable: Test Plan Deliverable to: Systems Development Responsibility: Systems Development SAP Tie: 3.1

47. Unit Test (530) Objective The objective of the Unit Testing is to run the application through a rigorous test using the test cases to verify all functionality works as expected. A unit test is required on all programs and should be repeatable. Required: Projects Optional: Support Deliverable: Documented Repeatable Unit Test Deliverable to: Systems Development Responsibility: Systems Development SAP Tie: 3.5

48. System Test (540) Objective The objective of the System Testing is to test the application(s) with the system to verify it has the desired effects and no undesired affects. Required: Projects Optional: Support Deliverable: Documented Repeatable System Test Deliverable to: Systems Development Responsibility: Joint Responsibility SAP Tie: 4.3

49. Acceptance Test Objective The objective of the Acceptance Testing is to get approval from customers and IS on final product to be moved to production. Support Services may be involved with this step in preparation for support of the final system. Required: Projects Optional: Support Deliverable: Customer Approval Deliverable to: Systems Development Responsibility: Joint Responsibility SAP Tie: 4.3, 4.6

50. Questions?