CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 1 SMU CSE 8314 / NTU SE 762-N Software Metrics and Quality Engineering Module 02 Engineering Quality

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 2 Contents Peer Reviews Fundamental Concepts of Process Engineering Quality Summary

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 3 Peer Reviews An example of quality engineering

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 4 Software DevelopersSoftware Managers Peer Reviews Motivation: – How to find problems in your work in a non-threatening way Method: – Evaluation of one individual’s work by others at the same organizational level (i.e., team members, not supervisors)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 5 Benefits of Peer Reviews Reduced rework and faster development due to a more effective evaluation – The peers understand the software best – and they will not be as much of a threat when it comes to promotions and raises – reciprocity -- you help them and they help you

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 6 Cost of Peer Reviews May require 10-20% of the total development time – this must be planned into the schedule – it must also be planned into progress measures and reward systems

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 7 Role of Quality Professionals in Peer Reviews Participants – Part of the team – They see issues from other projects Coaches and Facilitators – Make sure the process is followed correctly Recorders and Measurers – Keep track of results – Track action items to closure

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 8 Two Potential Problems with Peer Reviews People don’t want to attend others’ peer reviews because they are behind in their own schedules – Solution lies in reward system Smarter people tend to dominate others!! – Solution lies in teamwork and team reward systems

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 9 Fundamental Concepts of Process (A review. For more information, see other courses in SMU curriculum)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 10 The Process is What You Do The steps you take to accomplish something such as software development The proper inputs and outputs of each step The correct sequencing among the steps The right decisions and the right decision criteria as you proceed

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 11 A Process Description or Process Model is like a Roadmap It tells everyone what to do It synchronizes all of the participants It enables you to analyze what is happening and make changes if appropriate

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 12 Process, Methods and Tools METHODS DETAILED INSTRUCTIONS - HOW to Do It Eg. - Object Oriented Analysis - Structured Walkthroughs TOOLS AUTOMATION - Do it Efficiently Eg. - Compiler, - CASE, - Test Generator, - Measurement Tool PROCESS (description) Eg. - Design, then Review - Plan Tests with Requirements A MAP -- WHAT to do

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 13 Process vs Tools Tools without processes or methods are like a fast car with an inexperienced driver – You may drive fast – But who knows where you will end up? – Wrecks are more likely “If you automate without a process, you screw up more efficiently” -- Anonymous

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 14 Process vs Methods Methods tell you how to do the job - as driving lessons tell you how to drive But methods without processes are like a good driver who doesn’t know where he is or where he is going – You may eventually get there – And you may go fast – But you may not get to the right place very soon

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 15 Process Gives Knowledge Where to go How to get there Which shortcuts will work Which shortcuts will NOT work Risks of each option Why, not just what

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 16 Knowledge is Not Limited to Technical Matters Much of what contributes to project success or failure has little to do with technical decisions – Communication – Political issues – Management of resources – Realistic estimating – Keeping track of what is happening

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 17 Logic vs. Emotion Altruism vs. Politics AltruismPolitics LogicEmotion Philosopher King Practical Realist

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 18 Example of Knowledge Test before inspect – Natural tendency – More fun – Efficient (if testing is cheap) – Tends to waste test time Inspect before test – Usually more efficient overall – Avoids tying up expensive test resources – May be less efficient if testing is cheap

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 19 Process Descriptions Include... Tasks or activities -- what to do Sequencing -- when to do Information or data (artifacts) Flow if information and material (compare with a computer program)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 20 Process Descriptions Often Also Include Information to Help People Use Them... Entry and/or exit criteria Why Who What and when to measure Risks References...

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 21 Representing a Process It Depends on Who will Enact the Process Process Description Agent Process

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 22 Alternative Representations Process Description Computer Program RoadmapRecipe AgentComputerDriverCook Process Enaction A Specific Invocation of the Program A Specific Trip Preparation of A Specific Dish Several SW Developer Development of Software

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 23 Goals of a Process Representation Specific enough to enable a correct outcome – An accurate map – A correct recipe Flexible enough to allow judgment

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 24 Flexibility vs. Exactness Staying in the Lane Without Constant Correction (Experienced Driver vs. Novice)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 25 What is the Software Process The Process by which we Develop Software

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 26 The Hierarchical Decomposition Approach to Software Construction Analysis and Design Implementation and Testing

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 27 Software Development using the Hierarchical Decomposition Approach - I Requirements Design of Major Parts or Components Top Level Design -- Determine the Parts Design of Units or Modules Detailed Design -- Determine the Modules or Units of Each Part Coding & Test -- Build Each Module and Test It Coded Units or Modules Continued on Next Slide

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 28 Software Development using the Hierarchical Decomposition Approach - II Software Product Integrated Components Integration of Modules into Components Coding & Test -- Build Each Module and Test It Coded Units or Modules Continued from Previous Slide Integration of Components into Final Product

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 29 Waterfall Model Analyze Requirements Preliminary Design Detailed Design Code and Unit Test Integration and Test Customer Requirements Specification Top Level Design Specification Requirements Problems Design Problems Coding Problems

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 30 There are Many Variations on This Basic Approach Cycles of learning -- build small, learn, build a little bigger, learn, build a little bigger,... Early integration -- build “shells” of each component and integrate them, then flesh out the details of each component until you are finished, but integrate each step of the way Pre-define components into a library, then assemble new products from these components using high level programming tools.....

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 31 Spiral Model Plan Next Cycle Determine What to Do Evaluate the Results Develop the Next Version of Software

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 32 R = Understand Requirements D = Design I = Implement E = Evaluate These are the Basic Steps of Any Development Effort Essentially Similar to One Turn of the Spiral Model or One Cycle through the Waterfall Tornado Model Basic Development Cycle R I DE

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 33 Tornado Model Sequential Application This Application is Like the Spiral Model Incremental Development: Each Cycle Adds New Features Evolutionary Development: Each Cycle Improves the Previous One R I DE

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 34 Tornado Model Other Applications Requirements – Simulation or Prototype – To Analyze Alternatives or Quantify Risks Design – Build a Prototype – Test Out a Concept Implementation – Two Incremental Pre- releases R I DE R I DE R I DE R I DE R I DE

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 35 Microsoft “Integrate Every Day” Incremental Process Define interfaces first Build dummies for all parts Integrate all parts every day, gradually replacing dummies with real software Interfaces are more important than the specific functions of the components

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 36 Applying the Process Divide required features/functions by priority Implement top priority functions first Then later priorities – If schedule slips, you can still deliver something important – Forces you to understand the real market need

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 37 Tornado Model of Microsoft Process Top Priority Requirements R I DE R I DE R I DE Medium Priority Requirements Low Priority Requirements Release Date Integrate Every Day

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 38 Concurrent Engineering as a Larger Process Concept Exploration Demonstration / Validation Engineering & Manufacturing Development Production Support & Maintenance Product Development Process Department of Defense, Defense System Software Development, Dod-STD- 2167A, 29 Feb. 1988, Department of Defense, Washington D.C.,

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 39 Software Process Quality A Quality Software Development Process is one that Consistently produces Value to Customers (or meets some other definition of quality, per the application)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 40 Why All the Attention to Process? Software disasters are costing lots of money and getting more attention as software becomes a larger part of almost every manufactured product Process management is being recognized as the core issue behind many software development problems

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 41 Quality Engineering Principle Software is no exception to this basic principle of quality engineering Process is the proper focus of attention for effective quality improvement The quality of a product is determined to a large extent by the quality of the process used to develop and produce the product.

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 42 Quality Management Principle The way to improve quality is to: – Define (document) the process – Measure and evaluate the process as it is executed – Improve the process based on these measurements and evaluations

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 43 Process Automation Principle Know your process before automating it If you automate the wrong process, you simply do the wrong thing more efficiently.

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 44 Productivity Principle Design the process for real people Automate the parts that people don’t do well. If you design a process for perfect people, it will fail when executed by human beings

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 45 Engineering Quality The rest of the course

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 46 Mapping the Rest of the Course The first half of this course will examine many aspects of quality engineering Among other things, we will show how quality engineering relates directly to cycle time and productivity improvements

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 47 In The First Few Modules we have Discussed... The importance of defining quality – What does it mean for your product? – What goals are most important Quality engineering -- integrating quality processes into the software development processes Understand process – The role it plays in achieving quality

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 48 In Future Modules we will Discuss... Understanding and dealing with cultural issues – Cutting through excuses – Understanding the change process Developing plans for process improvement – Understanding how to use models – Assessing – Deciding whether/what/how Implementing the plans

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 49 Later, We will Discuss... Evaluating all elements of quality – Reliability – Maintainability – Verification – Validation – Testing and evaluation – Safety – Supportability Learning lessons from failures

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 50 In Cost of Quality and Value Added Analysis we will... Determine what matters to the customer – Eliminate what does not add value Determine the costs and payoffs of quality improvements – Show the return on investment – Quantify the benefits

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 51 In Software Reliability we will... Define software reliability Evaluate how important it is to your project Evaluate and apply reliability models

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 52 In Six Sigma and Other Industrial Programs for Quality Improvement we will... Discuss how to set aggressive quality goals - and meet them Apply successful techniques that were originally developed for manufacturing

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 53 Summary Peer reviews are a good example of whole team focus on quality Process must be the focal point to achieve effective quality, cycle time, and productivity improvement There are many aspects and dimensions of quality improvement

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M02 8/20/2001Slide 54 END OF MODULE 02