Software Quality : The Elusive Target Kitchenham B. & Pfleeger S. L. January, 1996 IEEE Software Zhenyu Wu Software Quality CS5391 Spring 2003

Introduction Questions: What do you really mean by software quality (SQ)? Is your definition adequate? How to evaluate your product’s SQ ? How to measure SQ? How to achieve SQ? Software Quality CS5391 Spring 2003

Meaning of SQ A good definition should let us measure quality in a meaningful way Measurements let us know if techniques improve quality how process quality affects product quality how quality methods affect product quality during use Is the investment in SQ methods profitable? Software Quality CS5391 Spring 2003

Meaning of SQ… Depends on how you approach software quality Different groups involved have different views Product-based Process-based People believe quality is important and can be improved Software Quality CS5391 Spring 2003

Views of SQ SQ can be perceived in various domains (philosophy, economics, marketing and operations management) SQ is complex and multifaceted concept and can be described from five different perspectives Transcendental view User view Manufacturing view Product view Value-based view Software Quality CS5391 Spring 2003

Transcendental View See quality as something that can be recognized but not defined Quality as something toward which we strive as an ideal, but may never implement completely Abstract sense Software Quality CS5391 Spring 2003

User View More concrete, grounded in the product characteristics that meet user’s needs Evaluate product in a task context can be personalized view Asses product behavior with respect to operational profiles Related to product usability and reliability Software Quality CS5391 Spring 2003

Manufacturing View Focus on product quality during production and after delivery Concerns whether or not the product was constructed “right the first time” Can lead to quality assessment independent of the product Adapted by ISO 9001 and CMM, advocates conformance to process rather than to specification Software Quality CS5391 Spring 2003

Product View Consider the product’s inherent characteristics, looks inside Assumption : measuring and controlling internal product properties will result in improved external product behavior More research needed to conform this assumption Software Quality CS5391 Spring 2003

Value-Based View Different views can be held by different groups involved in software development user’s requirement for a useful product conflict with manufacturer’s goal of minimizing rework consider the trade-offs between cost and quality manage conflict  design to cost compare with potential benefits This view tries to bring in compromise between different groups and views Software Quality CS5391 Spring 2003

Measuring Quality We need to measure quality so we can: Establish baselines Predict likely quality Monitor improvement Product attributes contribute to user satisfaction are a mixture of: product’s functions product’s nonfunctional qualities constraints ISO definition of quality: the totality of characteristics of an entity that bear on its ability to satifsfy stated and implied needs. Software Quality CS5391 Spring 2003

Measuring user's view Reliability: how long the product functions properly between failures Usability: including ease of installation, learning and use Tom Gilb’s technique: characteristics can be measured directly Example: Learning time = average elapsed time (in hours) for a typical user to achieve stated level of competence Software Quality CS5391 Spring 2003

Measuring user's view… This technique can be generialized to any quality feature The quality concept is broken down into component parts until each can be stated in terms of directly measurable attributes Software Quality CS5391 Spring 2003

Measuring the Manufacturer's View Defects count Count the number of known defects recorded during development and use Can compare modules, products or projects (with same way and same time) Relationship between defects count and operational failures is unclear Can indicate test efficiency and identify process improvement areas Software Quality CS5391 Spring 2003

Measuring the Manufacturer's View… Rework costs Any additional effort required to find and fix problems after documents and code are formally signed-off as part of configuration management Count debugging effort during integration and testing Not count end-phase verification and validation Pre-release rework: measure manufacturing efficient and process improvement Post-release rework: measure of deliverd quality Software Quality CS5391 Spring 2003

Modeling Quality Several models have been built to understand and measure quality Old Models McCall’s model ISO 9126 New model Dromey’s model Software Quality CS5391 Spring 2003

McCall's Model Defines product qualities as hierarchy of factors, criteria and metrics: Quality factor represents behavior characteristic of the system Quality criterion is an attribute of a quality factor that is related to software production and design Quality metric is a measure that captures some aspect of a quality criterion. Software Quality CS5391 Spring 2003

McCall's quality Model… The metrics have to be answered with ‘yes’ or ‘no’ answer The final quality is assessed by dividing the ‘yes’ answers by the total number of questions chosen to describe the quality Software Quality CS5391 Spring 2003

McCall’s Model… Software Quality CS5391 Spring 2003

ISO 9126 Six characteristics, completely hierarchical Quality charactertistics are divided into sub-characteristics which can be measured using metrics Recommends measuring the char-acteristics directly, but does not indicate clearly how to do it Software Quality CS5391 Spring 2003

Quality Characteristic Definition Functionality A set of attributes that bear on the existence of a set of functions and their specified propertise. The functions are those that satisfy stated or implied needs. Relibity A set of attributes that bear on the capability of software to maintain its performance level under stated conditions for a stated period of time. Usablity A set of attributes that bear on the effort needed for use and on the individual assessment of such use by a stated or implied set of users. Software Quality CS5391 Spring 2003

Efficiency Maintainability Portability A set of attributes that bear on the relationship between the software’s performance and the amount of resources used under stated conditions. Maintainability A set of attributes that bear on the effect needed to make specified modifications (which may include correction, improvements, or adaptations of software to environmental changes and changes in the requirements and functonal specifications). Portability A set of attributes that bear on the ability of software to be transferred from one environment to another (this includes the organizational, hardware or software environment). Software Quality CS5391 Spring 2003

ISO 9126… Functionality Reliebility Usability Suitability Accurecy Interoperability Secuity Functionality Materity Fault Telerance Recoverability Reliebility Understandability Learnability Operability Usability Characteristics Sub-characteristics Software Quality CS5391 Spring 2003

ISO 9126… Effiency Maintainability Portability Time behavior Resource behavior Analyzability Changeability Stability Testability Maintainability Adaptability Installability Conformance Replaceability Portability Characteristics Sub-characteristics Software Quality CS5391 Spring 2003

Main Differences between Mccall’s and ISO 9126 model with different quality frame quality factor not hierarchical ISO 9126 work and terminology quality characteristic and subcharacteristics completely hierarchical Software Quality CS5391 Spring 2003

Model Problems Lack rationale for determining which factor should be included in the quality definition Lack rationale for deciding which criteria relate to a particular factor The measure of metrics can be too subjective Does not describe how lower level metrics are composed into an overall assessment of higher level quality characteristics Software Quality CS5391 Spring 2003

Dromey's models (new model) Higher level quality attributes cannot be measured or built directly Higher level quality can be achieved by building components which exhibit a consistent, harmonious and complete set of product properties that result in manifestations of quality attributes Can allow us to verify models Software Quality CS5391 Spring 2003

Business Value of Quality Businesses invest valuable money for obtaining good software, we have to pay them back in good We should be careful in assessing and providing software with good quality How much "less than perfect" software are the businesses willing to accept should be determined Software Quality CS5391 Spring 2003

Conclusion Quality is a complex concept; It means different things to different people You must define aspects of quality in which you are interested Quality should be defined in a measurable way, so that it can be understood and verified Quality must be related to business goals Software Quality CS5391 Spring 2003