1. Introduction to Quality Engineering
RIT Software Engineering

2. RIT Software Engineering
Quality: Two views
Conformance to requirements (absence of defects)
Narrow definition (sometime referred to as q)
Fitness for use (relative to needs)
Broader definition (referred to as Q)
Relative to actual needs, not just written requirements
Includes other attributes (product quality, project business objectives, organizational objectives)
RIT Software Engineering

3. RIT Software Engineering
Quality Engineering
“Optimize” quality (not maximize)
Preferred tradeoff among multiple objectives
E.g. Achieve desired quality levels within cost bounds
Aim is to design systems and systematic approaches that continually work towards this optimum
RIT Software Engineering

4. Limitation of Quality Engg
Quality frameworks define what to do and how to do it, and measure the outcomes.
They can identify and eliminate problems.
But their effectiveness depends on the people who do the activities involved.
Frameworks cannot deliver excellence. Only people can deliver excellence.
RIT Software Engineering

5. RIT Software Engineering
Processes
(Systematic) steps for accomplishing a task
“Structured” approach to getting things done
The best process for a task is that which accomplishes the task most effectively i.e. optimizes across task objectives
RIT Software Engineering

6. RIT Software Engineering
More vs. best process
Processes maximize probability of successful task accomplishment i.e. prevent problems
“More process” (i.e. more formality) improves probability of success, but runs counter to other objectives (cost, flexibility)
“Best” process optimizes across task objectives, hence more process is not always good
RIT Software Engineering

7. RIT Software Engineering
Process Design
Designing good processes requires
understanding the various task objectives
understanding the impact of the steps involved (process design decisions) on all the different objectives
Creatively identifying different possible approaches (designs) and picking the best
A typical engineering design problem!
RIT Software Engineering

8. RIT Software Engineering
Limitation of process
Processes are designed to prevent problems (“reduce variance of output”)
Process is not free – there is a cost in time and effort, as well as in flexibility
Processes incorporate assumptions about the nature of the task and about the objectives – but every situation is a little different. The more the difference, the less effective the process.
Process customization (tailoring) is not free!
RIT Software Engineering

9. RIT Software Engineering
Metrics
The purpose of metrics is to provide evaluation and feedback (“try walking to the door with your eyes closed”)
They can provide an “objective” view to complement the subjective view of the people doing the job
When skillfully used, metrics can reveal longer-term trends that are harder to spot otherwise (“filter out random variation”)
RIT Software Engineering

10. RIT Software Engineering
Using Metrics
Problem:
We want to compare the prosperity of two communities.
We use metrics:
Average (per capita) income in city X is 40% more than average (per capita) income in city Y.
Conclusion:
X is a more prosperous place than Y.
Comments?
RIT Software Engineering

11. Limitations of Metrics
Exercise:
Identify reasons (as many as you can) why the conclusion may possibly not be valid.
“Average (per capita) income in city X is 40% greater than average (per capita) income in city Y. Therefore X is a more prosperous place than Y.”
RIT Software Engineering

12. RIT Software Engineering
Metrics design
Follow-up exercise:
If we wanted to determine which city was more prosperous using a metrics-based approach, how should we go about it?
RIT Software Engineering

13. Metrics Interpretation
Moral of the story: Metrics tell us something, but to make sure that the numbers don’t mislead us, we need to do a lot of additional work “behind the scenes”
Doing this requires:
Metrics understanding
Domain understanding
Familiarity with the specifics of the situation
RIT Software Engineering

14. Metrics Interpretation…
“Any chart should be accompanied by comments that point out what lies behind the numbers” (rule at Motorola)
This is the real value added by the quality engineer!
RIT Software Engineering

15. Famous lines about Metrics
“There are three kinds of lies: lies, damned lies and statistics”
“What statistics reveal is interesting, but what they conceal is vital”
RIT Software Engineering

16. RIT Software Engineering
Another famous line
“If you can’t measure it, you can’t control it”
(The idea is that measurement helps to close the feedback loop, which is necessary)
Flip side:
“If you manage purely by the numbers, all you manage is the numbers”
(Objectives that are not measured will not be met, and some of them may be the most important ones. And as we have seen, numbers don’t reveal the entire truth)
To ponder: Is the goal to “control” the outcomes, or to facilitate achievement of better outcomes?
RIT Software Engineering

17. RIT Software Engineering
Conclusion
Quality engineering is about effective ways to achieve project objectives
Processes, metrics are enablers for this
Defining good processes and selecting good metrics is a challenging design problem
Metrics interpretation is critical
RIT Software Engineering