1. Software Quality Engineering
An Introduction

2. Lecture Objectives
Provide some basic concepts about software quality engineering
Quality measurement, assessment, and improvement
Quality engineering activities and processes
The importance of metrics interpretation
Provide a focus for this course

3. Premise of a Quality Focus
Customer and Organization
Success
Process Quality
Product Quality

4. Customer and Organization
Measure and Improve
Customer and Organization
Success
Process Quality
Product Quality
Measure quality, analyze results, and identify improvements

5. Quality Engineering
Balance cost of quality with cost of poor quality in the context of business objectives
The ultimate measure of quality is customer satisfaction
Quality systems and frameworks (quality principles and practices) guide quality activities, but they cannot guarantee quality
Only people can deliver excellence

6. Scope of Software Quality Engineering
Quality planning, process improvement
Software Quality Engineering
Inspection, formal methods, defect prevention, fault tolerance, etc.
Quality Assurance
Testing
Execute software, compare observed behavior to specs.

7. Process and Product Quality Course Summary
Thinking Software Quality
Quality systems/frameworks
Measurement fundamentals
Software metrics catalog
Defects and Reliability
Defect detection, prevention, and removal
Defect removal metrics and models
Software Reliability Engineering
Product Quality and Customers
Product quality engineering
Customer satisfaction
Project and Process Quality
Project and process activity metrics
Process assessment and improvement
Testing (Defect Detection)

8. Applying the Concepts: Course Project
Creating a Quality Systems Improvement Plan
Quality objectives
Quality approach
Product quality
Defects
Other quality attributes
Customer satisfaction
Process quality
Incremental implementation plans
Assessment plans

9. Activities in a Quality Engineering System

10. Implementing a Quality Improvement System
Quality attribute
Definition: Identify measurements for quality assessment and improvement
Execution: Measure, analyze, and improve quality
Operational definition, metrics
Data analysis and interpretation
Measurements
Data collection

11. Quality Engineering Process
Defined quality goals
Quality
Planning
Quality
Assurance
Activities
Entry
Feedback & adjustments No
Yes ?
Selected QA activities
Exit
Quality goals satisfied?
Quality Assessment & Improvement
Selected measurements & models
Analysis & modeling results

12. Quality Planning Activities
Set quality goals by balancing customer expectations with project economics (cost, schedule, scope, risk)
Identify customer quality views and attributes
Including customer balance of their cost of quality
Select direct measures of the quality attributes
Set achievable and acceptable goal values of the quality measures
For the stated quality goals ...
Select specific QA activities to achieve quality goals
Balance the cost and benefit of the QA activities against the quality goals
Select direct and indirect product and process quality measurements and quality models for quality assessment and analysis

13. Quality Assurance Activities
Defect prevention
Remove (human) error sources
Block defects from being injected into software artifacts
Defect reduction
Detect defects
Inspection
Testing
Remove defects
“Debugging”—iterate on the software engineering activity
Rework requirements, design, code, etc.
Defect containment
Fault tolerance
Fault containment

14. Quality Assessment and Improvement Activities
Measurement
Defect and other product quality measurements
Process quality measurements
Quality analysis and modeling
Analyze measurement data
Fit data to analytical models of quality
Estimate current and future quality (quality trends)
Identify problematic software components or process activities
Feedback for immediate product and process improvement
Identify problematic product components and process activities
Adjust quality goals, project plan, QA plan
Adjust quality models
Feedback for organizational process improvement
Improve techniques for quality assurance, quality engineering, and overall software engineering process

15. Right-Size Your Process
A Quick Review from SE Process and Project Management

16. Processes (Systematic) steps for accomplishing a task
“Structured” approach to getting things done
The best process for a task is that which accomplishes the task most effectively, that is, optimizes across task objectives

17. More Process vs. Best Process
Processes maximize probability of successful task accomplishment, that is, prevent problems
“More process” (more formality and ceremony) improves probability of success, but often runs counter to other objectives (such as cost, flexibility)
“Best” process optimizes across task objectives
More process is not always good
Less process is not always good, either

18. Process Design Designing good processes requires:
Understanding the various task objectives
Understanding the impact of the steps involved (process design decisions) on all the different objectives
Creatively identifying different possible approaches (process designs) and picking the best
A typical engineering design problem!

19. Limitations of Process
Processes are designed to prevent problems
Reduce product defects, reduce process variance, etc.
Process is not free – there is a cost in time and effort, as well as in flexibility
Processes incorporate assumptions about the nature of the task and about the objectives
But every situation is a little different
The more the difference, the less effective the process
Process customization (tailoring) is not free!

20. Metrics

21. Metrics The purpose of metrics is to provide evaluation and feedback
Try walking to the door with your eyes closed
They can provide an “objective” view to complement the subjective view of the people doing the job
When skillfully used, metrics can reveal longer-term trends that are harder to spot otherwise

22. Metrics Interpretation
Metrics are indicators that tell us something about some quality attribute of interest
To make sure that the numbers don’t mislead us, we need to do a lot of additional work to understand the state of the quality attribute
Metrics analysis and interpretation
Doing this requires:
Metrics understanding
Domain understanding
Understanding the specific process, product, and organization of the projects

23. Metrics Interpretation…
Any conclusion and supporting data and graphs should be accompanied by comments that point out what lies behind the numbers and conclusions
This is the real value added by the quality engineer!

24. Famous Lines About Metrics
“There are three kinds of lies: lies, damned lies and statistics”
(attributed to Benjamin Disraeli and made popular by Mark Twain)
“What statistics reveal is interesting, but what they conceal is vital”

25. Another Famous Line
“When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.” - (Lord Kelvin, Scottish Physicist – 1884)
Paraphrased: “If you can’t measure it, you probably don’t understand it, and you certainly can’t control it”
The idea is that measurement and analysis helps to close the feedback loop, which is necessary for quality control and improvement

26. You Must Interpret the Metrics
Flip side:
“If you manage purely by the numbers, all you manage is the numbers”
Objectives that are not measured will not be met, and some of them may be the most important ones
Numbers don’t reveal the entire truth
Don’t measure what is easy to measure, measure what is essential to measure

27. Software Quality Engineering
A systematic approach to measuring and improving software product and process quality will help ensure organizational success
Customer and Organization
Success
Process Quality
Product Quality
Measure quality, analyze results, and identify improvements

28. Conclusion
Quality engineering is about effective ways to achieve project and organizational objectives
Measure, control and improve product and process quality
Processes and metrics are enablers to achieve these objectives
Defining good processes and selecting good metrics is a challenging engineering design problem
Metrics interpretation is critical
If not done well, all work collecting data is useless