211 Continuing Best Practices ► Slide information taken in large part from former Rational Corporation slides. Considerably modified and supplemented for classroom use.

212 Practice 5: Verify Software Quality Control Changes Develop Iteratively Use Component Architectures Manage Requirements VerifyQuality Quality, as used within Unified Process, is defined as “The characteristic of having demonstrated the achievement of producing a product which meets or exceeds agreed upon requirements as measured by an agreed upon measures and criteria And is produced by an agreed upon process. If done this way, the process can be repeated and managed  In most organizations, testing accounts for 30-50% of development costs! Yet most people believe software is not adequately tested when delivered. Testing is difficult; complete testing is impossible; a good process and automated tools help!

213 Software problems are 100 to 1000 times more costly to find and repair after deployment Development Deployment Cost Practice 5: Verify Software Quality One of the ways to improve quality: Test early and continuously! Test functionality, reliability; performance; Test architectural decisions early.

214 Iterative Development Permits Continuous Testing – ensuring higher Quality T I M E Test T C D R Iteration 1Iteration 2Iteration 3 T C D R T C D R Test Life Cycle Assess Plan Design Implement Execute Assess! Plan Design Implement Execute Assess! Plan Design Implement Execute

215 Verifying Quality – and Continuous Testing (continued) ► ► Rather than test one time, spread testing out continuously.  Part of each iteration – BUT (see below) ► ► Each iteration produces an executable release (not necessarily a product release…)  Don’t think of an ‘executable’ as just an.exe or.dll. The executables may be part of an architecture……   Each iteration is tested and integrated into an evolving application. ► ► Note: In the RUP, Each ‘phase’ has one or more iterations, and each phase has milestones! ► ► Be careful: The ‘extent (how much)’ of R, D, C, T depends on which phase the iteration is in! ► See your drawings of the UP) ► (See your drawings of the UP)

216 Testing in an Iterative Environment Requirements Test Suite 1 Iteration 2 Iteration 3 Iteration 4 Test Suite 2 Test Suite 3 Test Suite 4 Iteration 1 Automated Tests Continuous integration!!! (one of the major problems of SDLC!) We will produce automated tests (IF AVAILABLE in your ‘environment’) As new requirements are added in iterations, new tests are generated and run. This means that some tests will be rerun – part of ‘Regression Testing.’

217 Quality – What is it? ► It is an elusive term ► Means different things to different people Next few slides from OOSE text slides; modified

218 Software Quality... ► Usability - Users can learn it fast and get their job done easily (usually addressed in Interface) job done easily (usually addressed in Interface) ► Efficiency - It doesn’t waste resources like CPU time and memory (addressed in execution) time and memory (addressed in execution) ► Reliability - It does what it is required to do without failing (MTBF, MTTR…) without failing (MTBF, MTTR…) ► Maintainability - It can be easily changed ► Reusability - Its parts can be used in other projects, so reprogramming is not needed projects, so reprogramming is not needed

219 Software Quality...again, means different things to different stakeholders QUALITY SOFTWARE Developer: easy to design; easy to maintain; easy to reuse its parts User: easy to learn; efficient to use; helps get work done Customer: solves problems at an acceptable cost in terms of money paid and resources used Development manager: sells more and pleases customers while costing less to develop and maintain

2110 Software Quality ► The different qualities can conflict.  Increasing efficiency can reduce maintainability or reusability  Increasing usability can reduce efficiency  Increasing usability can reduce maintainability  Increasing maintainability can reduce efficiency, etc! ► Setting objectives for quality is a key engineering activity  Then design to meet these objectives  Avoids ‘over-engineering’ which wastes money  Obtain the highest possible reliability using a fixed budget

2111 Short Term Vs. Long Term Quality ► Short term:  Does the software meet the customer’s immediate needs?  Is it sufficiently efficient for the volume of data we have today? ► Long term: (more in terms of ‘quality factors’  Maintainability  Customer’s future needs  Scalability

2112 Dimensions of Software Quality Functionality Reliability Application Performance System Performance Does my app do what’s required? Does my app leak memory? Does my app respond acceptably? Does my system perform under production load? Test cases for each scenario implemented Analysis tools and code instrumentation Check performance for each use-case/scenario implemented Test performance of all use- cases under authentic and worst-case load TypeWhy?How? For each iteration, do the ‘above’ software quality checks. Remember: tests are ‘driven’ by Use Cases and Supplementary Specifications!

2113 Problems Addressed by Verifying Quality Testing provides objective project status assessment Objective assessment exposes inconsistencies early (continuous integration helps!) Testing and verification are focused on high risk areas Defects are found earlier and are less expensive to fix (because ‘testing’ is distributed… Automated testing tools provide testing for reliability, functionality, and performance Root Causes Solutions  Insufficient requirements  Ambiguous communications  Brittle architectures  Overwhelming complexity  Subjective assessment  Undetected inconsistencies  Poor testing  Waterfall development  Uncontrolled change  Insufficient automation

2114 Practice 6: Control Changes to Software Control Changes Develop Iteratively Use Component Architectures Manage Requirements Verify Quality Must recognize that we CANNOT STOP CHANGE. Our goal is to Manage Change! The only ‘constant’ is ‘change!’ Must control How and When control is introduced and who introduces the changes. This DOES NOT MEAN that we absolutely accept ALL changes, But…(Discuss!) Want a process that can respond to change…(UP) Must synchronize Change across development teams and locations too. (What impacts do proposed changes have on our architecture!)

2115 Without explicit control, parallel development degrades to chaos!!!! Practice 6: Control Changes to Software ► Consider: we often have:  Multiple developers  Multiple teams  Multiple sites  Multiple iterations  Multiple releases  Multiple projects  Multiple platforms May have multiple developers organized into different teams at multiple sites all working together on multiple iterations, releases, products, and platforms (mostly based on the software architecture) All at the same time!

2116 Three Major Aspects of a CM System Controlling Change involves a Change Management System and a Configuration Management System for version control, releases, etc. (this is beyond where we will go in this course…) CR = change request (version control; evolving products…)

2117 Concepts of Configuration & Change Management ►  Decompose the architecture into layers, packages, subsystems etc., and assign responsibility for these architectural elements to team/teams. ► Establish secure workspaces for each developer  Provide isolation from changes made in other workspaces  Control all software artifacts - models, code, docs, etc. ► Establish an integration workspace ► Establish an enforceable change control mechanism ► Know which changes appear in which releases ► Release a tested baseline at the completion of each iteration  Versioning; baselines; …

2118 Problems Addressed by Controlling Changes Requirements change; workflow is defined and repeatable Change requests facilitate clear communications Isolated workspaces reduce interference from parallel work Change rate statistics are good metrics for objectively assessing project status Workspaces contain all artifacts, facilitating consistency Change propagation is controlled Changes maintained in a robust, customizable system Root Causes Solutions  Insufficient requirements  Ambiguous communications  Brittle architectures  Overwhelming complexity  Subjective assessment  Undetected inconsistencies  Poor testing  Waterfall development  Uncontrolled change  Insufficient automation

2119 Best Practices Reinforce Each Other ControlChanges DevelopIteratively Use Component Architectures ModelVisually VerifyQuality Ensure users involved as requirements evolve Validate architecture decisions early on. Drives development, planning, change control. …. Addresse complexity of design/implementation incrementally. Need tools/support environment! Measure quality early and often. Continuous testing and integration Evolve baseline incrementally Architecture  teams  localizing changes; Need CMS, Config Control… ManageRequirements Remember: best practices yield best results WHEN USED COLLECTIVELY! What does iterative development do??

2120 Summary: Best Practices of Software Engineering The result is software that is  On Time  On Budget  Meets/Exceeds Users Needs Project Manager Performance Engineer Release Engineer Analyst Developer Tester Control Changes Develop Iteratively Use Component Architectures ManageRequirements ModelVisually VerifyQuality