1. INFO 637Lecture #81 Software Engineering Process II Integration and System Testing INFO 637 Glenn Booker

2. INFO 637Lecture #82 Testing & Documentation This week covers integration and system testing, and user documentation

3. INFO 637Lecture #83 Testing Principles The purpose of testing is to assess the quality of the product While defects found in testing need to be fixed, few defects should be left by the beginning of testing If you find a lot of defects in testing, there are probably lots of others hiding!

4. INFO 637Lecture #84 Testing Principles Even good software engineers, following a poor process for development, will tend to produce poor quality software Testing is not a substitute for using a good process!

5. INFO 637Lecture #85 Testing Strategy Build the system from the unit-tested parts Integration test the system to ensure it is correctly built, and all parts function together System test the product to demonstrate it meets system requirements

6. INFO 637Lecture #86 Testing Strategy While doing all of this:  Identify low quality components and return them to the quality/process manager for assessment and cleanup  If they still cause problems, return them to the quality/process manager for rework or replacement  In later cycles, regression tests show you didn’t break anything that worked earlier

7. INFO 637Lecture #87 Build and Integration Strategies Several approaches may be used for building the system Integration testing is to verify that all parts are present and communicating with each other – testing for functionality is done by system testing

8. INFO 637Lecture #88 Build and Integration Strategies There are four main integration strategies  Big Bang Strategy  One-at-a-time Strategy  Cluster Strategy  Flat-system Strategy For good quality code, almost any strategy could work well

9. INFO 637Lecture #89 Big Bang Strategy The simplest integration strategy is to simply put all of the pieces together and see if everything works Rarely works for complex or low quality systems – only try if you have known high quality components

10. INFO 637Lecture #810 One-at-a-time Strategy Another simple approach is to start with one component, and test it Then add another component, and test them Keep repeating one component at a time until the entire system is present Good for identifying problems, but very time intensive

11. INFO 637Lecture #811 Cluster Strategy Another common strategy is to implement groups of related modules together Choices of groups are either by type of function (printing, file management, reports, etc.) or by portion of the system (ATM balance verification, then withdrawal, then deposit)

12. INFO 637Lecture #812 Flat-system Strategy This is a top-down integration strategy First build the highest level modules Then add the next level modules called by them Then keep adding lower and lower level modules until the system is complete Good for finding system-level issues

13. INFO 637Lecture #813 System Test Strategy System testing seeks to answer some fundamental questions about the system  Does the system perform its intended functions?  Does the system meet its quality goals?  Will the system work under normal conditions?  Will the system work under adverse conditions?

14. INFO 637Lecture #814 System Test Strategy Other key concerns include:  Can the system be installed?  Does it start properly?  Will it recover from software or hardware failures?  Is its performance adequate?  Can users use the system easily?

15. INFO 637Lecture #815 System Test Strategy System testing cannot be complete for any complex system, so test cases need to be chosen to address the most important concerns Test planning should address several related functions or characteristics with each test

16. INFO 637Lecture #816 System Test Strategy Most common priorities for system testing include:  System functionality  Stress testing  Usability testing  Performance testing

17. INFO 637Lecture #817 Second System Test Strategy Another approach is to do all aspects of testing for one functional area at a time Do function testing, stress testing, usability, and performance testing for one set of functions Then repeat for each functional area Loses overall feel for the system

18. INFO 637Lecture #818 Third System Test Strategy Combine previous two strategies Do normal and stress testing for lower level functions Then test higher and higher level functions until the entire system is covered Good for poor quality systems, but time consuming

19. INFO 637Lecture #819 Fourth System Test Strategy Do a top down version of the previous strategy Test highest level functions under normal and stressed conditions Work down to lower and lower level functions until whole system tested Only works with high quality product

20. INFO 637Lecture #820 Test Planning We have seen test planning appear in many phases of the TSP Test plans need to identify  The materials needed for testing (including the system configuration tested)  What tests will be run  What order the tests will be run

21. INFO 637Lecture #821 Test Completeness TSP focuses on measuring system testing by the percent of requirements which were tested Other methods focus on the percent of logical paths which were tested, or the percent of code exercised

22. INFO 637Lecture #822 Test Planning Test plans also need to:  Name each test  Predict the outcome of each test  Predict how long the test should take  Scripts and test cases will be needed for most tests Estimate the number of defects to be found in each phase, and their repair time

23. INFO 637Lecture #823 Test Tracking Test effectiveness is measured by the number of defects found per hour of running tests A test log (e.g. p. 172) records the details of testing

24. INFO 637Lecture #824 Defect-prone Modules Modules (components) which have a lot of defects in testing have a lot of defects after release Hence it is critical for testing to identify parts of the system which need to be fixed or reworked before they become a problem later

25. INFO 637Lecture #825 Defect Data SUMP records the defects found and removed for each (life cycle) phase SUMQ captures quality for each module Hence these two forms are key to identifying defect-prone modules, and knowing when they first became that way

26. INFO 637Lecture #826 Test Scripts To start testing you need to have:  Completed SRS and SDS documents  You have some components which have been implemented and passed unit testing  Later cycles also need prior versions of the product

27. INFO 637Lecture #827 Test Scripts Test script TEST1 (p. 178) includes:  Develop build, integration test, and system test procedures  Determine size, time, and materials needed for each test  Build the product  Conduct integration testing  Conduct system testing

28. INFO 637Lecture #828 Test Scripts  Produce user documentation, review and fix it Outputs include  An integrated and tested product for this cycle  Completed LOGD and LOGTEST forms for all tests  Completed user documentation  Time, size, and defect data in TSP system

29. INFO 637Lecture #829 Documentation Documentation for a system may include manuals for various purposes and audiences  Installation  User  Maintenance (troubleshooting, upgrades, etc.)  Training  Marketing Here we focus on end user documentation

30. INFO 637Lecture #830 Documentation Documentation captures the mindset of the design – its philosophy or train of thought This helps the user know what should be expected next Documentation can also capture little used features, and ones whose usage are hard to guess correctly

31. INFO 637Lecture #831 Documentation Approach Documentation needs to be written from the user’s viewpoint, not the designer’s Good user documentation focuses on groups of functions which the user may wish to use, generally increasing in complexity Best to start writing documentation right after design of each type of function

32. INFO 637Lecture #832 Documentation Features Good user documentation includes:  A glossary for unusual terminology  Error messages, troubleshooting, and recovery information  Index of key topics  And a detailed table of contents Good to outline the documentation first, and make sure key functions are accounted for

33. INFO 637Lecture #833 Documentation Features Documentation should use short, simple paragraphs and sentences Use lots of bulleted lists to break up concepts Write to a consistent level of detail, and make sure documentation uses a consistent expectation of what the user knows

34. INFO 637Lecture #834 Documentation Review Check documentation for:  Organization  Terminology  Content  Accuracy  Readability  Understandability

35. INFO 637Lecture #835 Why Here? Why is documentation discussed with integration and system testing?  User documentation is often the basis for some functionality (system) testing  Need to make sure the examples provided in the documentation really do work exactly as stated!