Software Quality Engineering An Introduction

Lecture Objectives Provide some basic concepts about software quality engineering Quality measurement, assessment, and improvement Quality engineering activities and processes The importance of metrics interpretation Provide a focus for this course

Premise of a Quality Focus Customer and Organization Success Process Quality Product Quality

Customer and Organization Measure and Improve Customer and Organization Success Process Quality Product Quality Measure quality, analyze results, and identify improvements

Quality Engineering Balance cost of quality with cost of poor quality in the context of business objectives The ultimate measure of quality is customer satisfaction Quality systems and frameworks (quality principles and practices) guide quality activities, but they cannot guarantee quality Only people can deliver excellence

Scope of Software Quality Engineering Quality planning, process improvement Software Quality Engineering Inspection, formal methods, defect prevention, fault tolerance, etc. Quality Assurance Testing Execute software, compare observed behavior to specs.

Process and Product Quality Course Summary Thinking Software Quality Quality systems/frameworks Measurement fundamentals Software metrics catalog Defects and Reliability Defect detection, prevention, and removal Defect removal metrics and models Software Reliability Engineering Product Quality and Customers Product quality engineering Customer satisfaction Project and Process Quality Project and process activity metrics Process assessment and improvement Testing (Defect Detection)

Applying the Concepts: Course Project Creating a Quality Systems Improvement Plan Quality objectives Quality approach Product quality Defects Other quality attributes Customer satisfaction Process quality Incremental implementation plans Assessment plans

Activities in a Quality Engineering System

Implementing a Quality Improvement System Quality attribute Definition: Identify measurements for quality assessment and improvement Execution: Measure, analyze, and improve quality Operational definition, metrics Data analysis and interpretation Measurements Data collection

Quality Engineering Process Defined quality goals Quality Planning Quality Assurance Activities Entry Feedback & adjustments No Yes ? Selected QA activities Exit Quality goals satisfied? Quality Assessment & Improvement Selected measurements & models Analysis & modeling results

Quality Planning Activities Set quality goals by balancing customer expectations with project economics (cost, schedule, scope, risk) Identify customer quality views and attributes Including customer balance of their cost of quality Select direct measures of the quality attributes Set achievable and acceptable goal values of the quality measures For the stated quality goals ... Select specific QA activities to achieve quality goals Balance the cost and benefit of the QA activities against the quality goals Select direct and indirect product and process quality measurements and quality models for quality assessment and analysis

Quality Assurance Activities Defect prevention Remove (human) error sources Block defects from being injected into software artifacts Defect reduction Detect defects Inspection Testing Remove defects “Debugging”—iterate on the software engineering activity Rework requirements, design, code, etc. Defect containment Fault tolerance Fault containment

Quality Assessment and Improvement Activities Measurement Defect and other product quality measurements Process quality measurements Quality analysis and modeling Analyze measurement data Fit data to analytical models of quality Estimate current and future quality (quality trends) Identify problematic software components or process activities Feedback for immediate product and process improvement Identify problematic product components and process activities Adjust quality goals, project plan, QA plan Adjust quality models Feedback for organizational process improvement Improve techniques for quality assurance, quality engineering, and overall software engineering process

Right-Size Your Process A Quick Review from SE Process and Project Management

Processes (Systematic) steps for accomplishing a task “Structured” approach to getting things done The best process for a task is that which accomplishes the task most effectively, that is, optimizes across task objectives

More Process vs. Best Process Processes maximize probability of successful task accomplishment, that is, prevent problems “More process” (more formality and ceremony) improves probability of success, but often runs counter to other objectives (such as cost, flexibility) “Best” process optimizes across task objectives More process is not always good Less process is not always good, either

Process Design Designing good processes requires: Understanding the various task objectives Understanding the impact of the steps involved (process design decisions) on all the different objectives Creatively identifying different possible approaches (process designs) and picking the best A typical engineering design problem!

Limitations of Process Processes are designed to prevent problems Reduce product defects, reduce process variance, etc. Process is not free – there is a cost in time and effort, as well as in flexibility Processes incorporate assumptions about the nature of the task and about the objectives But every situation is a little different The more the difference, the less effective the process Process customization (tailoring) is not free!

Metrics

Metrics The purpose of metrics is to provide evaluation and feedback Try walking to the door with your eyes closed They can provide an “objective” view to complement the subjective view of the people doing the job When skillfully used, metrics can reveal longer-term trends that are harder to spot otherwise

Metrics Interpretation Metrics are indicators that tell us something about some quality attribute of interest To make sure that the numbers don’t mislead us, we need to do a lot of additional work to understand the state of the quality attribute Metrics analysis and interpretation Doing this requires: Metrics understanding Domain understanding Understanding the specific process, product, and organization of the projects

Metrics Interpretation… Any conclusion and supporting data and graphs should be accompanied by comments that point out what lies behind the numbers and conclusions This is the real value added by the quality engineer!

Famous Lines About Metrics “There are three kinds of lies: lies, damned lies and statistics” (attributed to Benjamin Disraeli and made popular by Mark Twain) “What statistics reveal is interesting, but what they conceal is vital”

Another Famous Line “When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.” - (Lord Kelvin, Scottish Physicist – 1884) Paraphrased: “If you can’t measure it, you probably don’t understand it, and you certainly can’t control it” The idea is that measurement and analysis helps to close the feedback loop, which is necessary for quality control and improvement

You Must Interpret the Metrics Flip side: “If you manage purely by the numbers, all you manage is the numbers” Objectives that are not measured will not be met, and some of them may be the most important ones Numbers don’t reveal the entire truth Don’t measure what is easy to measure, measure what is essential to measure

Software Quality Engineering A systematic approach to measuring and improving software product and process quality will help ensure organizational success Customer and Organization Success Process Quality Product Quality Measure quality, analyze results, and identify improvements

Conclusion Quality engineering is about effective ways to achieve project and organizational objectives Measure, control and improve product and process quality Processes and metrics are enablers to achieve these objectives Defining good processes and selecting good metrics is a challenging engineering design problem Metrics interpretation is critical If not done well, all work collecting data is useless