1. Software Methods Mö/3.6.2002 slide 1 Methods and Techniques of Software Quality Management ICEL Quality Management Systems: Methods and Techniques of Software Quality Management Karl Heinrich Möller Gaertnerstr. 29 D-82194 Groebenzell Tel: +49(8142)570144 Fax: +49(8142)570145 Email: karl-heinrich.moeller@t-online.de

2. Software Methods Mö/3.6.2002 slide 2 Methods and Techniques of Software Quality Management ICEL Definitions User friendliness Software Development as a Process Faults as a cost factor Fault causing factors Strategies to fault reduction Topics

3. Software Methods Mö/3.6.2002 slide 3 Methods and Techniques of Software Quality Management ICEL Quality: Degree to which a set of inherent characteristics fulfils requirements Note 1: The term quality can be used with adjectives such as poor or excellent Definition of Quality ISO 9000:2000; Par. 3.1.1

4. Software Methods Mö/3.6.2002 slide 4 Methods and Techniques of Software Quality Management ICEL Characteristic: Distinguishing feature Note 1: A characteristic can be inherent or assigned Note 2: A characteristic can be qualitative or quantitative Note 2: There are various classes of characteristics (physical, sensory, behavioural, temporal, etc.) Definition of Characteristic ISO 9000:2000; Par. 3.5.1

5. Software Methods Mö/3.6.2002 slide 5 Methods and Techniques of Software Quality Management ICEL Nonconformity: Non - fulfilment of a requirement Definition of Nonconformity ISO 9000:2000; Par. 3.6.2

6. Software Methods Mö/3.6.2002 slide 6 Methods and Techniques of Software Quality Management ICEL Defect: Non - fulfilment of a requirement related to an intended or specified use Note: The distinction between the concepts defect and nonconformity is important as it has legal connotations, particularly those associated with product liability. Definition of Defect ISO 9000:2000; Par. 3.6.3

7. Software Methods Mö/3.6.2002 slide 7 Methods and Techniques of Software Quality Management ICEL Quality-related Costs : Those costs incurred in ensuring and assuring satisfactory quality, as well as the losses incurred when satisfactory quality is not achieved Quality losses: Losses caused by not realizing the potential of resources in processes and activities Definition of Quality Cost and Losses ISO 8402:1995; Par. 4.2 and 4.3

8. Software Methods Mö/3.6.2002 slide 8 Methods and Techniques of Software Quality Management ICEL Defect Prevention Costs: Group of quality cost elements to collect costs which will be caused by taking measures of prevention and correction in order of QM Test Costs Group of quality cost elements to collect costs which will be caused by all planned quality tests Defect Costs Group of quality cost elements to collect costs which will be caused by the losses incurred when satisfactory quality is not achieved Definition of Quality Cost Elements Internationally not defined

9. Software Methods Mö/3.6.2002 slide 9 Methods and Techniques of Software Quality Management ICEL Development of software should incorporate two characteristics: Development process of high quality (will result in high productivity, efficiency) Products with high customer value (will result in high sales) --> Assuring the economic survival Quality Objectives of Software Development

10. Software Methods Mö/3.6.2002 slide 10 Methods and Techniques of Software Quality Management ICEL 1. Functionality Suitability, Accuracy, Interoperability, Security, Functionality Compliance 2. Reliability Maturity, Fault Tolerance, Recoverability, Reliability Compliance 3. Usability Understand ability, Learn ability, Operability, Attractiveness, Usability Compliance 4. Efficiency Time Behaviour, Resource Utilisation, Efficiency Compliance 5. Maintainability Analysability, Changeability, Stability, Testability, Maintainability Compliance 6. Portability Adaptability, Install ability, Co-Existence, Replace ability, Portability Compliance ISO 9126:2001 Software Engineering; Quality Model

11. Software Methods Mö/3.6.2002 slide 11 Methods and Techniques of Software Quality Management ICEL User Friendliness Requirement Product Characteristic Usability Portability Reliability Efficiency Design Characteristic Hardware Independency Insularity Completeness Robustness Consistency Hardware Efficiency Accessibility Connect ability Self-explanatoribility Diagnose ability Upgradeability Accuracy Readability User Adaptability Testability Comprehensibility Changeability

12. Software Methods Mö/3.6.2002 slide 12 Methods and Techniques of Software Quality Management ICEL Development Process as Waterfall Model Market Require- ments Product Concept Product Defini- tion Design Implem. & Test Integr. & funct. Test System Test Real Product Product Maintenance Customer Satisfaction Development Process Product Planning Field Defects Realisation

13. Software Methods Mö/3.6.2002 slide 13 Methods and Techniques of Software Quality Management ICEL Schedule =fct(Product Structure + Process) Development Process Product Planning Field Realisation Product Structure Product Part A Part C Part B Part D Schedule Realisation Part A Product Planning Realisation Part B Realisation Part C Realisation Part D System Test

14. Software Methods Mö/3.6.2002 slide 14 Methods and Techniques of Software Quality Management ICEL Quality Assurance for Software Development By using metrics and setting objectives: Without Clear Objectives Quality Cannot be Improved

15. Software Methods Mö/3.6.2002 slide 15 Methods and Techniques of Software Quality Management ICEL Two Approaches to Quality Measurement Quality improvement through measurement –Presence of quality How good is this product? - Subjective metrics –Absence of quality What is wrong with this product? - Objective metrics

16. Software Methods Mö/3.6.2002 slide 16 Methods and Techniques of Software Quality Management ICEL Open and Hidden Resistance Resistance –Open Metrics are too expensive Metrics are not obvious Metrics are not connected to the business –Hidden Management Developers

17. Software Methods Mö/3.6.2002 slide 17 Methods and Techniques of Software Quality Management ICEL Open and Hidden Resistance Overcoming Resistance –Involve Everyone –Build Trust –Metrics Program Champion –Work Circles

18. Software Methods Mö/3.6.2002 slide 18 Methods and Techniques of Software Quality Management ICEL Faults as a Cost Factor (to Boehm) Requirements Architecture Design Detailed Design Coding Functional Test Integration Test & System Test Management Quality Assurance, Config.-Management 18 9 10 6 12 4 10 1 5 9 6 Corrections in integration test and system test amount to 18% of the costs Correct. in... Corrections in... amount to 6% of the costs Corrections in... amount to 10% of the costs Total amount of costs for finding and correcting the defects: 39%

19. Software Methods Mö/3.6.2002 slide 19 Methods and Techniques of Software Quality Management ICEL Fault Stream Development Process Requirement Review Design Review Code Review Functional Test System Test Deploy- ment Require- ments Design Implemen- tation Time 10% 40% 50% Stream of Faults 3% 5%7% 25% 50% 10% Goal: <5% Goal: <25% Fault Finding Process

20. Software Methods Mö/3.6.2002 slide 20 Methods and Techniques of Software Quality Management ICEL Improvement Potential Fault Origin Fault Origin Fault detection Fault detection Require- ments Design Coding Functional Test System Test Deploy- ment Costs per Fault 10% 40% 50% 3% 5% 7% 25% 50% 10% 1 kDM 6 kDM 12 kDM 20 kDM

21. Software Methods Mö/3.6.2002 slide 21 Methods and Techniques of Software Quality Management ICEL Fault Distribution in Moduls Number of faults Number of modules 100% 10% 50% 90% fault concentration in a small number of modules

22. Software Methods Mö/3.6.2002 slide 22 Methods and Techniques of Software Quality Management ICEL Fault rates as a function of the change rate faults/k DNLCC DNLCC NLCC fault driving factors fault rate

23. Software Methods Mö/3.6.2002 slide 23 Methods and Techniques of Software Quality Management ICEL Fault rates as a function of the module length faults/kNLCC log (NLCC) fault driving factors fault rate for new modules

24. Software Methods Mö/3.6.2002 slide 24 Methods and Techniques of Software Quality Management ICEL Fault rates as a function of the module length faults/k DNLCC log (DNLCC) fault driving factors faults rate for changed modules

25. Software Methods Mö/3.6.2002 slide 25 Methods and Techniques of Software Quality Management ICEL Particular Factors for High Fault Rates Code changes Short modules „Old“ Programming Languages Data flow instead of control flow More missing than defect Late changes, corrections Incomprehensible Messages Logic expressions Missing initialization

26. Software Methods Mö/3.6.2002 slide 26 Methods and Techniques of Software Quality Management ICEL Early Fault Removal for a Operating System

27. Software Methods Mö/3.6.2002 slide 27 Methods and Techniques of Software Quality Management ICEL Fault Rates and Productivity Size [kNLCC] Productivity [DNLCC/MD] Fault Rate [Faults/kDNLCC] 100350.3 200250.4 500170.9 1000101.2 150071.5 300051.6

28. Software Methods Mö/3.6.2002 slide 28 Methods and Techniques of Software Quality Management ICEL Strategies for Reducing Faults 1. Introducing less faults in developing (will be about 5% per year for experienced development teams) 2. Finding faults earlier (Reducing field defects at about 50% in particular years possible) 3. Fault Prevention by re-use

29. Software Methods Mö/3.6.2002 slide 29 Methods and Techniques of Software Quality Management ICEL Strategies for Reducing Faults Fault Introduction Analysis Where Faults are Introduced within the Process Target Goals For Faults Found in Development and the Field within all Functions For all Products Monitoring the Target Goals Through Reporting and Projection Comparison with Plan Corrective Action Supporting Measures - Found Faults Counts for all Phases - Management Support of Quality Improvement - Improved Review Methods - Methods Training - Tools Use - Many Small Steps

30. Software Methods Mö/3.6.2002 slide 30 Methods and Techniques of Software Quality Management ICEL Strategies for Reducing Faults Forecasting Methods 1. Forecasting rest faults by faults found 2. Forecasting faults based on the defect input in older versions 3. Forecasting faults with a database based on experience 4. Forecasting faults by introducing faults synthetically

31. Software Methods Mö/3.6.2002 slide 31 Methods and Techniques of Software Quality Management ICEL Strategies for Reducing Faults Years 0 1 2 3 4 5 6 Detected faults in total

32. Software Methods Mö/3.6.2002 slide 32 Methods and Techniques of Software Quality Management ICEL Strategies for Reducing Faults faults found in % of all faults Time Fault found F(t) 100% 90- 97% t(x) 85- 95% faults to find faults found System Test Pilot Deployment Development 60- 75% F (t) = N(1-e -  t )

33. Software Methods Mö/3.6.2002 slide 33 Methods and Techniques of Software Quality Management ICEL Strategies for Reducing Faults Fault forecasting for product lines Version n Version n - 1 Analysis of the fault rates between defined milestones l Programming language l Change rate l Group l Module group Version n + 1 l Forecasting of the faults to expect l New agreement with the development group for number of faults to find Difference of the fault rates of the version n-1 and n