1. CS 5150 1 CS 5150 Software Engineering Lecture 19 Acceptance and Delivery

2. CS 5150 2 Administration Presentations All presentations on Wednesday, April 1 will be in Upson 4135. Quiz 3 averages Question 1 7.0 Question 2 7.3

3. CS 5150 3 Quiz 3 Question 1 (b) The plan is to develop the system in two phases. In Phase 1, the product database will remain unchanged but the web interface will be replaced by the ordering system. In Phase 2 a new product database will be created. For the interface between the ordering system and the product database, select a design pattern that will allow the product database to be replaced in Phase 2 without modifications to the ordering system.

4. CS 5150 4 Quiz 3 Question 1 (b) The Problem LegacyProductDB Ordering System OldClient Phase2ProductDB In Phase 1, the Ordering System must work with the LegacyProductDB Future (Phase 2)Legacy System

5. CS 5150 5 Quiz 3 Question 1 (b) Phase 1: Adapter Design Pattern OderingSystemInterfaceAdapterLegacyProductDB Ordering System abstract class shown in italic inheritance

6. CS 5150 6 Quiz 3 Question 1 (b) The Bridge design pattern is not suitable for wrapping legacy code, because the legacy code does not subclass the Implementor abstract class. Implementor Phase2ProductDB LegacyProductDB

7. CS 5150 7 Sequential Development: The Waterfall Model Requirements System design Testing Operation & maintenance Program design Implementation (coding) Acceptance & release Requirements Design Implementation Feasibility study

8. CS 5150 8 Changes to the Syllabus Lecture 19Acceptance testing, delivery, and release Lecture 20-22Reliability: building reliable systems, including testing Lecture 23Testing large systems

9. CS 5150 9 Acceptance Testing Acceptance testing The complete system, including documentation, training materials, installation scripts, etc. is tested against the requirements by the client, assisted by the developers. Each requirement is tested separately. Scenarios are used to compare the expected outcomes with what the system produces. Emphasis is placed on how the system handles problems, errors, restarts, and other difficulties. Is the system that we have built, the system that you wanted? Have we met your requirements?

10. CS 5150 10 Acceptance Testing Closed box: by the client without knowledge of the internals The entire system is tested as a whole The emphasis is on whether the system meets the requirements Uses real data in realistic situations, with actual users, administrators, and operators The acceptance test must be successfully completed before the new system can go live or replace a legacy system. Completion of the acceptance test may be a contractual requirement before the system is paid for.

11. CS 5150 11 The Testing Process Acceptance Testing is a major part of a software project It requires time on the schedule It may require substantial investment in test data, equipment, and test software. Good testing requires good people! Documentation, including help systems an training materials, is an important part of acceptance testing.

12. CS 5150 12 Techniques for Release The transition from the previous version of a production system to a new release is challenging. Alpha Testing: Clients operate the system in a realistic but non-production environment Beta Testing: Clients operate the system in a carefully monitored production environment Parallel Testing: Clients operate new system alongside old production system with same data and compare results

13. CS 5150 13 Release: Transition from a Legacy System Parallel Testing Parallel Testing For data processing systems, such as financial systems, payroll, etc., the old and the new systems are run together for several productions cycles to check that the new system replicates the functionality of the old. Requires two sets of everything (staff, equipment, etc.). Requires software to control changeover (e.g., do not mail two sets of payments). Requires automated scripts to compare results. Parallel testing may take several months. If possible, the new system will be brought into production incrementally.

14. CS 5150 14 Delivery: Summary A good delivery package results in: happy client happy users less expense in support or maintenance But most projects rush the packaging, help systems, and training materials, give them to the least experienced members of the team, do not test them properly, and generally neglect this part of the software process.

15. CS 5150 15 Delivering the System: Themes Different categories of software product need different packaging and delivery procedures. Packaging, support, maintenance and major failures are expensive. Trade-offs must be made. Pressures to get products to market and in operation often lead to bad decisions. (In my experience, the pain of being late is often less than the pain of putting a bad system into operation.) Services, such as installation, training, configuration, etc. may be paid for separately.

16. CS 5150 16 Delivery of Software: Categories of Product Shrink-Wrapped Package Installation scripts -- automatic -- varieties of hardware and operating systems -- uninstall, reinstall, etc. Support (very expensive when it requires staff) -- staff training -- documentation (user, system administrator, expert user) Maintenance -- client does not have source code -- no bug fixing except with new release

17. CS 5150 17 Delivery of Software: Categories of Product Data Processing System Acceptance -- acceptance period may cover several months -- client should be comfortable with complete system Support -- client should be self-sufficient -- documentation and training for system administrators and operators -- well organized source code for maintenance -- maintenance and support contracts

18. CS 5150 18 Delivery of Software: Categories of Product Embedded System Acceptance -- hardware and software developed together -- acceptance tests combination Maintenance -- bug fixes require servicing the hardware -- errors may be expensive or dangerous Support -- training for support personnel -- documentation and training for users

19. CS 5150 19 Training Time and money spent on training is usually well spent: one-on-one in-house training training courses distance education online tutorials Development team needs to provide training materials: users (perhaps several categories) system administrators system maintainers trainers

20. CS 5150 20 Training and Usability A well-designed system needs less training good conceptual model intuitive interfaces Different skill levels need different types of training skilled users work from the conceptual model less-skilled users prefer cookbook sets of instructions occasional users will forget complex details, but remember general structure

21. CS 5150 21 Help Systems Resources A good help system is a major sub-project (time-consuming, expensive) A good help system saves user time and support staff (time- saving, cost-saving) Help system design Users need many routes to find information (index by many terms, examples, mini-tutorials, etc.) Help systems need to be tested with real users

22. CS 5150 22 Documentation Online documentation Much cheaper than print Fewer restrictions on numbers of pages, colors, etc. Easy to update (e.g., over the Internet) but... Cannot be used if the user's system is down

23. CS 5150 23 Categories of Documentation Software development Requirements, design Source code, test plans and results User Introductory (various audiences) User manual Web site of known problems, FAQ, etc. System administrator and operator System manuals Business License, contract, etc.

24. CS 5150 24 Installation Tools Creating installation scripts may be a major sub-project Different scripts, tools and procedures for different categories of software Testing must be extensive with real users in their own environment

25. CS 5150 25 Delivery: Maintenance Most production programs are maintained by people other than the programmers who originally wrote them. (a) What factors make a program easy for somebody else to maintain? (b) What factors make a program hard for somebody else to maintain?

26. CS 5150 26 CS 5150: What Materials should you Deliver? When you leave, all that the client has is your documentation and your software. Imagine that you work for the client and are asked to take over this system. What would you want? Materials can be in any format, need not be huge, but must be current. Place your materials on a project site web site.

27. CS 5150 27 Delivery: Check List for CS 5150 Projects Documentation Requirements, updated to reflect delivered system System & program design, updated to reflect delivered system Instructions for: users, administrators, operators Presentation slides, updated to reflect delivered system Business documentation, e.g., copyright license System Source code and matching binary for all programs Installation scripts, etc. Test scripts, test data, and test reports Different projects will have different deliverables

28. CS 5150 28 User Interface Testing User interfaces need several categories of testing. During the design phase, user interface testing is carried out with trial users to ensure that the design is usable. Design testing is also used to develop graphical elements and to validate the requirements. During the implementation phase, the user interface goes through the standard steps of unit and system testing to check the reliability of the implementation. Acceptance testing is carried out with users, on the completed system.

29. CS 5150 29 How we’re user testing: -One-on-one, 30-45 min user tests with staff levels -Specific tasks to complete -No prior demonstration or training -Pre-planned questions designed to stimulate feedback -Emphasis on testing system, not the stakeholder! -Standardized tasks / questions among all testers User Interface Testing: A CS 5150 Project The next few slides are from a presentation by a team in 2007 (second milestone)

30. CS 5150 30 How we’re user testing: Types of questions we asked: -Which labels, keywords were confusing? -What was the hardest task? -What did you like, that should not be changed? -If you were us, what would you change? -How does this system compare to your paper based system -How useful do you find the new report layout? (admin) -Do you have any other comments or questions about the system? (open ended) User Interface Testing: A CS 5150 Project

31. CS 5150 31 What we’ve found: Issue #1, Search Form Confusion! User Interface Testing A CS 5150 Project

32. CS 5150 32 User Interface Testing A CS 5150 Project What we’ve found: Issue #2, Inconspicuous Edit/ Confirmations!

33. CS 5150 33 User Interface Testing A CS 5150 Project What we’ve found: Issue #3, Confirmation Terms

34. CS 5150 34 User Interface Testing A CS 5150 Project What we’ve found: Issue #4, Entry Semantics

35. CS 5150 35 User Interface Testing A CS 5150 Project What we’ve found: #5, Search Results Disambiguation & Semantics