1. Managing Software Quality
Main issues:
Quality cannot be added as an afterthought
To measure is to know
Product quality vs process quality
© SE, Quality, Hans van Vliet

2. Commitment to quality pays off
SE, Quality, Hans van Vliet, ©2008
© SE, Quality, Hans van Vliet

3. Approaches to quality
Quality of the product versus quality of the process
Check whether (product or process) conforms to certain norms
Improve quality by improving the product or process
SE, Quality, Hans van Vliet, ©2008

4. Approaches to quality Conformance Improvement Product ISO 9126
‘best practices’
CMM
SPICE
Bootstap
ISO 9001
SQA
Process
SE, Quality, Hans van Vliet, ©2008

5. + What is quality? software measures
SE, Quality, Hans van Vliet, ©2008

6. SE, Quality, Hans van Vliet, ©2008

7. SE, Quality, Hans van Vliet, ©2008

8. How to measure “complexity”?
The length of the program?
The number of goto’s?
The number of if-statements?
The sum of these numbers?
Yet something else?
SE, Quality, Hans van Vliet, ©2008

9. Software Metrics Product vs. process Most metrics are indirect:
No way to measure property directly or
Final product does not yet exist
For predicting, need a model of relationship of predicted variable with other measurable variables.
We can accurately measure some property of software or process.
A relationship exists between what we can measure and what we want to know.
This relationship is understood, has been validated, and can be expressed in terms of a formula or model.
Few metrics have been demonstrated to be predictable or related to product or process attributes.
SE, Quality, Hans van Vliet, ©2008

10. A measurement framework
Formal world
“Real” world
scale type
unit
value
attribute-relation model
attribute relation
attribute
entity
belongs to
expressed in
computes
used in
has
part of
formalizes
measures
holds for
SE, Quality, Hans van Vliet, ©2008

11. Scale types
Nominal Attributes are merely classified: the color of my hair is gray, white or black.
Ordinal There is a (linear) ordering in the possible values of an attribute: one type of material is harder than another, one program is more complex than another.
Interval The same as ordinal, but the ‘distance’ between successive values of an attribute is the same, as in a calendar, or the temperature measured in degrees Fahrenheit.
Ratio The same as interval, with the additional requirement that there exists a value 0, as in the age of a software system, or the temperature measured in degrees Kelvin.
Absolute In this case we simply count the number of occurrences, as in the number of errors detected in a program.
SE, Quality, Hans van Vliet, ©2008

12. Attribute-relation model
If there exists a relation between different attributes of, possibly different, entities in the ‘real’ world, we may express that relation in a formal model.
This model computes (predicts) the value of an attribute in which we are interested from the values of one or more other attributes from the model.
More than one model is possible
SE, Quality, Hans van Vliet, ©2008
© SE, Quality, Hans van Vliet

13. Representation condition
A measure M is valid if it satisfies the representation condition, i.e. if A>B in the real world, then M(A)>M(B)
E.g. if we measure complexity as the number of if-statements, then:
Two programs with the same number of if-statements are equally complex
If program A has more if-statements than program B, then A is more complex than B
SE, Quality, Hans van Vliet, ©2008

14. Internal versus external attributes
Internal attributes of an entity can be measured purely in terms of that entity itself:
Modularity, size, defects encountered, and cost
External attributes of an entity are those which can be measured only with respect to how that entity relates to its environment.
Maintainability and usability
Most quality factors we discuss in this chapter are external attributes.
External attributes can be measured only indirectly, since they involve measurement of othere attributes
SE, Quality, Hans van Vliet, ©2008

15. Direct versus indirect measures
Example
the speed of a car can be determined from the values of two other attributes:
a distance and the time it takes the car to travel that distance.
The speed is then measured indirectly, by taking the quotient of two direct measures.
In this case, the attribute-relation model formalizes the relation between the distance traveled, time, and speed.
SE, Quality, Hans van Vliet, ©2008

16. More on measures
Scale type of a combined measure is the ‘weakest’ of the scale types of its constituents
This is often violated; see cost estimation models
SE, Quality, Hans van Vliet, ©2008

17. Quality
IEEE Glossary of Software Engineering Terminology : ‘the degree to which a system, component, or process meets customer or user needs or expectations’.
Software quality should be measured primarily against the degree to which user requirements are met:
correctness, reliability, usability, etc
Software lasts a long time and is adapted from time => important to do this within reasonable costs.
maintainability, testability, portability, etc
SE, Quality, Hans van Vliet, ©2008

18. McCall’s Triangle of Quality
3 categories
11 factors
SE, Quality, Hans van Vliet, ©2008

19. Quality attributes (McCall)
Product operation
Correctness does it do what I want?
Reliability does it do it accurately all of the time?
Efficiency will it run on my hardware as well as it can?
Integrity is it secure?
Usability can I use it?
Product revision
Maintainability can I fix it?
Testability can I test it?
Flexibility can I change it?
Product transition
Portability will I be able to use it on another machine?
Reusability will I be able to reuse some of the software?
Interoperability will I be able to interface it with another system?
SE, Quality, Hans van Vliet, ©2008

20. McCall’s Quality factors Defined
Correctness: The extent to which a program satisfies its specifications and fulfils the user’s mission objectives.
Reliability: The extent to which a program can be expected to perform its intended function with required precision.
Efficiency: The amount of computing resources and code required by a program to perform a function.
Integrity: The extent to which access to software or data by unauthorized persons can be controlled.
Usability: The effort required to learn, operate, prepare input, and interpret output of a program.
Maintainability: The effort required to locate and fix an error in an operational program.
Testability: The effort required to test a program to ensure that it performs its intended function.
Flexibility: The effort required to modify an operational program.
Portability:The effort required to transfer a program from one hardware and/or software environment to another.
Reusability: The extent to which a program (or parts thereof) can be reused in other applications.
Interoperability: The effort required to couple one system with another.
SE, Quality, Hans van Vliet, ©2008

21. Taxonomy of quality attributes (ISO 9126)
Functionality
Reliability
Usability
Efficiency
Maintainability
Portability
The ISO quality characteristics strictly refer to a software product. Their definitions do not capture process quality issues.
An important objective of the software architecture phase is to bring these quality factors to the forefront and make the tradeoffs explicit, so that the stakeholders know what they are in for
SE, Quality, Hans van Vliet, ©2008

22. Definition of ISO quality attributes
Functionality: The capability of the software product to provide functions which meet stated and implied needs when the software is used under specified conditions.
Reliability: The capability of the software product to maintain a specified level of performance when used under specified conditions.
Usability: The capability of the software product to be understood, learned, used and be attractive to the user, when used under specified conditions.
Efficiency: The capability of the software product to provide appropriate performance, relative to the amount of resources used, under stated conditions.
Maintainability: The capability of the software product to be modified. Modifications may include corrections, improvements or adaptation of the software to changes in environment, and in requirements and functional specifications.
Portability: The capability of the software product to be transferred from one environment to another.
SE, Quality, Hans van Vliet, ©2008

23. ISO Quality Scheme
ISO scheme is hierarchical: each sub-characteristic is related to exactly one characteristic.
The sub-characteristics concern quality aspects that are visible to the user. Reusability, for example, is not included in the ISO scheme.
SE, Quality, Hans van Vliet, ©2008

24. SE, Quality, Hans van Vliet, ©2008

25. ISO 9126 (cnt’d)
ISO 9126 measures ‘quality in use’: the extent to which users can achieve their goal
Quality in use is modeled in four characteristics:
Effectiveness
Productivity
Safety
Satisfaction
SE, Quality, Hans van Vliet, ©2008

26. Perspectives on quality (Different definitions)
Transcendent (“I really like this program”)
User-based (“fitness for use”)
Product-based (based on attributes of the software)
Manufacturing-based (conformance to specs)
Value-based (balancing time and cost vs profits)
SE, Quality, Hans van Vliet, ©2008

27. Qyality attribute scenario to reconcile the different perspectives
SE, Quality, Hans van Vliet, ©2008

28. ISO 9001
Model for quality assurance in design, development, production, installation and servicing
Basic premise: confidence in product conformance can be obtained by adequate demonstration of supplier’s capabilities in processes (design, development, …)
ISO registration by an officially accredited body, re-registration every three years
SE, Quality, Hans van Vliet, ©2008

29. Capability Maturity Model (CMM)
Initial level: software development is ad-hoc
Repeatable level: basic processes are in place
Defined level: there are standard processes
Quantitatively managed level: data is gatheread and analyzed routinely
Optimizing level: stable base, data is gathered to improve the process
SE, Quality, Hans van Vliet, ©2008

30. Initial  repeatable level
Requirements management
Project planning
Project monitoring and control
Supplier agreement management
Measurement and analysis
Process and product quality assurance
Configuration management
SE, Quality, Hans van Vliet, ©2008

31. Repeatable  defined level
Requirements development
Technical solution
Product integration
Verification
Validation
Organization process focus
Organization process definition
Organizational training
Integrated project management
Risk management
Decision analysis and resolution
SE, Quality, Hans van Vliet, ©2008

32. CMM: critical notes Most appropriate for big companies
Pure CMM approach may stifle creativity
Crude 5-point scale (now: CMMI)
SE, Quality, Hans van Vliet, ©2008

33. Get started on Software Process Improvement (SPI)
Formulate hypotheses
Carefully select metrics
Collect data
Interpret data
Initiate improvement actions
Iterate
SE, Quality, Hans van Vliet, ©2008

34. Lessons w.r.t. data collection
Closed loop principle: result of data analysis must be useful to supplier of data
Do not use data collected for other purposes
Focus on continuous improvement
Only collect data you really need
SE, Quality, Hans van Vliet, ©2008

35. Summary Product quality versus process quality
Quality conformance versus quality improvement
Quality has to be actively pursued
There are different notions of quality
Quality has many aspects
Quality is hard to measure
SE, Quality, Hans van Vliet, ©2008