2. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 1 Practice 5: Verify Software Quality Control Changes Develop Iteratively Use Component Architectures Manage Requirements VerifyQuality Quality, as used within Rational Unified Process, is defined as “The characteristic of having demonstrated the achievement of producing a product which meets or exceeds agreed upon requirements as measured by agreed upon measures and criteria And is produced by an agreed upon process. If done this way, the process can be repeated and managed In most organizations, testing accounts for 30-50% of development costs! Yet most people believe software is not adequately tested when delivered. Testing is difficult; complete testing is impossible; a good process and automated tools help!

3. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 2 Software problems are 100 to 1000 times more costly to find and repair after deployment Development Deployment Cost Practice 5: Verify Software Quality Test early and continuously! Test functionality, reliability; performance; Test architectural decisions early.

4. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 3 Iterative Development Permits Continuous Testing T I M E Test T C D R Iteration 1Iteration 2Iteration 3 T C D R T C D R Test Life Cycle Evaluate Plan Design Implement Execute Evaluate Plan Design Implement Execute Evaluate Plan Design Implement Execute

5. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 4 Iterative Development – and Continuous Testing (cont.)   Rather than test one time, spread testing out continuously.  Part of each iteration – BUT (see below)   Each iteration produces an executable release (not a product release…)  Don’t think of an ‘executable’ as just an.exe or.dll. The executables may be part of an architecture…..   Each iteration is tested and integrated into an evolving application.   Note: Each ‘phase’ has iteration(s), and each phase has milestones! (A phase may have zero or more iterations prior to arriving at the phase milestone!)   Be careful: The ‘degree’ of R, D, C, T depends on which phase the iteration is in!  See your drawings of the RUP (There are MANY). See pg. 66 in RUP, third edition, as one of many examples…) Keep this handy! Will be referencing this many times.

6. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 5 More…recall – talking about Verifying Quality   Cannot ‘engineer in’ quality; Must be threaded throughout development!   Notice: Continuous Testing and integration!  Distributes testing….   At end, entire system tested as a whole.   Many errors can be found early and fixed while repair costs are inexpensive   Architectural decisions (key decisions) tested early avoiding disastrous problems later.   These features greatly reduce risks and liability of delivering poor quality systems.   Early iterations are used to mitigate risk and address core functionalities (whether in elaboration, construction…)

7. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 6 Testing in an Iterative Environment Requirements Test Suite 1 Iteration 2 Iteration 3 Iteration 4 Test Suite 2 Test Suite 3 Test Suite 4 Iteration 1 Automated Tests Continuous integration!!! (one of the major problems of SDLC!) We will produce automated tests. As new requirements are added in iterations, new tests are generated and run. This means that some tests will be rerun – part of ‘Regression Testing.’

8. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 7 Automation Reduces Testing Time and Effort One Manual Test Cycle 13,000 Tests2 Weeks6 People 13,000 Tests 6 hours 1 Person 13,000 Tests 6 hours 1 Person One Manual Test Cycle 13,000 Tests2 Weeks6 People Test Automation Run More Tests More Often Manual preparation of tests is very expensive and usually results in missed ‘opportunities.’

9. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 8 Dimensions of Software Quality Functionality Reliability Application Performance System Performance Does my app do what’s required? Does my app leak memory? Does my app respond acceptably? Does my system perform under production load? Test cases for each scenario implemented Analysis tools and code instrumentation Check performance for each use-case/scenario implemented Test performance of all use- cases under authentic and worst-case load TypeWhy?How? For each iteration, do the ‘above’ software quality checks. Remember: tests are ‘driven’ by Use Cases and Supplementary Specifications!

10. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 9 Problems Addressed by Verifying Quality Testing provides objective project status assessment Objective assessment exposes inconsistencies early (continuous integration helps!) Testing and verification are focused on high risk areas Defects are found earlier and are less expensive to fix (because ‘testing’ is distributed… Automated testing tools provide testing for reliability, functionality, and performance Root Causes Solutions  Insufficient requirements  Ambiguous communications  Brittle architectures  Overwhelming complexity  Subjective assessment  Undetected inconsistencies  Poor testing  Waterfall development  Uncontrolled change  Insufficient automation

11. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 10 Practice 6: Control Changes to Software Control Changes Develop Iteratively Use Component Architectures Manage Requirements Verify Quality Must recognize that we CANNOT STOP CHANGE The only ‘constant’ is ‘change!’ But, we must be able to Manage Change! Must control How and When control is introduced and who introduces the changes. This DOES NOT MEAN that we absolutely accept ALL changes, But…(Discuss!) Want a process that can respond to change…(RUP) Must synchronize Change across development teams and locations too. (What impacts do proposed changes have on our architecture!)

12. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 11 Without explicit control, parallel development degrades to chaos!!!! Practice 6: Control Changes to Software  Multiple developers  Multiple teams  Multiple sites  Multiple iterations  Multiple releases  Multiple projects  Multiple platforms May have multiple developers organized into different teams at multiple sites all working together on multiple iterations, releases, products, and platforms (mostly based on the software architecture)

13. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 12 Three Major Aspects of a CM System Controlling Change involves a Change Management System and a Configuration Management System for version control releases, etc. (this is beyond where we will go in this course…) CR = change request (version control; evolving products…)

14. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 13 Concepts of Configuration & Change Management  Decompose the architecture into subsystems and assign responsibility for each subsystem to a team  Establish secure workspaces for each developer  Provide isolation from changes made in other workspaces  Control all software artifacts - models, code, docs, etc.  Establish an integration workspace  Establish an enforceable change control mechanism  Know which changes appear in which releases  Release a tested baseline at the completion of each iteration  Versioning; baselines; …

15. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 14 Change Control Supports All Other Best Practices  Develop iteratively  Manage requirements  Use component architectures  Model visually  Verify quality  Progress is incremental only if changes to artifacts are controlled  To avoid scope creep, assess the impact of all proposed changes before approval  Components must be reliable, i.e., the correct versions of all constituent parts found  To assure convergence, incrementally control models as designs stabilize  Tests are only meaningful if the versions of the items under test are known and the items protected from changes Italicized items – verbally discussed in class. Not necessarily more important than others…

16. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 15 Problems Addressed by Controlling Changes Requirements change workflow is defined and repeatable Change requests facilitate clear communications Isolated workspaces reduce interference from parallel work Change rate statistics are good metrics for objectively assessing project status Workspaces contain all artifacts, facilitating consistency Change propagation is controlled Changes maintained in a robust, customizable system Root Causes Solutions  Insufficient requirements  Ambiguous communications  Brittle architectures  Overwhelming complexity  Subjective assessment  Undetected inconsistencies  Poor testing  Waterfall development  Uncontrolled change  Insufficient automation

17. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 16 Best Practices Reinforce Each Other ControlChanges DevelopIteratively Use Component Architectures ModelVisually VerifyQuality Ensures users involved as requirements evolve Validates architectural decisions early on. Drives development, planning, change control. …. Addresses complexity of design/implementation incrementally Need tools/support environment! Measures quality early and often Continuous testing and integration Evolves baselines incrementally Architecture  teams  localizing changes; Need CMS, Conf Control… ManageRequirements Remember: these best practices yield the best results WHEN USED COLLECTIVELY!

18. Unified Software Practices v 5.0 - D Copyright  1998 Rational Software, all rights reserved 17 Summary: Best Practices of Software Engineering  The result is software that is  On Time  On Budget  Meets Users Needs Project Manager Performance Engineer Release Engineer Analyst Developer Tester Control Changes Develop Iteratively Use Component Architectures ManageRequirements ModelVisually VerifyQuality