1. CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved CSE7315M33 Slide 1 August 13, 2006 SMU CSE 7315 Planning and Managing a Software Project Module 33 Software Quality Control

2. CSE7315M33 Slide # 2August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Objectives of This Module To examine the meaning of software quality To review quality control and testing techniques

3. CSE7315M33 Slide # 3August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Software Quality Control References to text: Managing Quality16 Software Testing11 Other Relevant Material8

4. CSE7315M33 Slide # 4August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved SW Quality Typical Symptoms of a Problem We have a procedure to prevent that! Why did it happen? The procedure was not followed. Nobody even knew about it

5. CSE7315M33 Slide # 5August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Software Quality Symptoms of Another Problem They’ve violated company policy and we’re facing a lawsuit! Did they know about the policy? Who authorized what they did?

6. CSE7315M33 Slide # 6August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Other Typical Symptoms of Quality Problems Upper management was sure embarrassed when the product failed at the customer demo Didn’t anyone tell them it wasn’t tested?

7. CSE7315M33 Slide # 7August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Other Typical Symptoms of Quality Problems The customers are complaining that it doesn’t work as advertised. But we tested it for three weeks!

8. CSE7315M33 Slide # 8August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Poor Quality Costs Money In a world-wide marketplace, higher quality means more profit Higher quality can mean lower cost (see next module) The costs associated with poor quality are often hard to quantify – Loss of customers – Low pride of workmanship –...

9. CSE7315M33 Slide # 9August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Motivation of the Software Developer Pride in Workmanship and Skill Fear of Failure Quality Software Software Developer

10. CSE7315M33 Slide # 10August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved What Is Software Quality?

11. CSE7315M33 Slide # 11August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved There Are Many Definitions Crosby: Quality is conformance to requirements Juran: Quality is fitness for intended use Webster: – (1) Quality is that which makes something what it is – (2) Quality is the degree of excellence Weinberg: Quality is value to someone

12. CSE7315M33 Slide # 12August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved The Essential Characteristics The product does what it is supposed to do The customer gets what they paid for

13. CSE7315M33 Slide # 13August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved External vs Internal Quality External Quality: – Is determined by factors whose presence or absence may be observed by the users of the software – Is used as the ultimate criterion to judge the quality of a system Internal Quality: – Factors invisible to the end users – Help achieve the external quality

14. CSE7315M33 Slide # 14August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved External Quality Factors Correctness – Does the system conform to its specs? Robustness – Does the software system react appropriately to abnormal conditions? Extendibility – Ease of adapting the software product to changes in its specifications

15. CSE7315M33 Slide # 15August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved External Quality (continued) Compatibility – The ease with which the system can be combined with other systems Efficiency – The ability of the software to put as little demand on hardware as possible Portability – The ease of transferring software products to different hardware/software platforms

16. CSE7315M33 Slide # 16August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved External Quality (continued) Functionality – The extent of possibilities supported by the software system Timeliness & Economy – The ability of the software to be cost effectively developed and released in a timely manner

17. CSE7315M33 Slide # 17August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved External Quality (continued) Ease of use – The ease with which people with different backgrounds can learn to use the software and then effectively apply it to their benefit Verifiability – How easily and cost effectively can the correctness of the system be judged?

18. CSE7315M33 Slide # 18August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Internal Quality Factors Modularity – The degree to which elements of the software are independent from each other Readability – How hard is it to read and understand the source code? Reusability – The ability of the elements of the software to be used in later systems

19. CSE7315M33 Slide # 19August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Internal and External Quality Factors Some of the external quality factors have an equally strong impact on the internal quality – Timeliness & Economy – Compatibility – Portability – Extendibility

20. CSE7315M33 Slide # 20August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Evolution of Software Quality Quality Control Quality Assurance Quality Engineering This Module Next Module Introduced in Next Module

21. CSE7315M33 Slide # 21August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Software Quality Control Purposes: – To prevent defective products from being shipped to the customer – To arbitrate disputes when there are debates about whether the product is “good enough”

22. CSE7315M33 Slide # 22August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Software Quality Control Typical Practices: – Inspections – Tests – Measurements – Independent confirmation of compliance Standards Requirements

23. CSE7315M33 Slide # 23August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Quality Control Goal: Keep quality at an acceptable level by rejecting unacceptable products Requirements DevelopmentQC Inspection Pass Fail Standards of Quality Control Quality Assurance Quality Engineering

24. CSE7315M33 Slide # 24August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Software Quality Control Typical criteria for a QC inspection – How many requirements are met? – How many tests have passed / failed? Problems (maybe) unique to software – Are the tests adequate? – Do the fixes inject more errors? – Did the developers intentionally fail to test or avoid testing, to save time or because they are overconfident?

25. CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved CSE7315M33 Slide 25 August 13, 2006 Testing

26. CSE7315M33 Slide # 26August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Why Do We Test Software? To find defects in the software And, ultimately, to convince ourselves that most of the defects have been removed (when tests do not uncover defects) It is very easy in the stress of a project to focus on the second objective instead of the first.

27. CSE7315M33 Slide # 27August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Sample Of An Ineffective Software Test This is based on a hardware test technique Why is it ineffective? “Run the software for 7 days and see whether it fails” Class discussion

28. CSE7315M33 Slide # 28August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved What Would Effective Software Tests Look Like? Test the different ways the software might fail Test under different operating conditions Test all of the requirements – Stated – Unstated Reflect actual customer use Others?... Class discussion

29. CSE7315M33 Slide # 29August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved What Would Effective Software Tests Do For Us? Provide programmers with information they can use to prevent errors – Where the errors happen – Where the errors come from Give management information it needs to rationally assess the risk in releasing the software Validate the software – Make sure it does what is should do

30. CSE7315M33 Slide # 30August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Testing Strategies A good testing strategy provides a systematic way to select and generate tests to detect the faults that are most serious or most likely – Determines what needs to be tested – Decides whether to base tests on specs (Black Box Testing) or the structure of the software(Glass Box Testing) – Determines what test data should be used

31. CSE7315M33 Slide # 31August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Testing Vocabulary Failure A particular case where the system does not meet its specification Fault or Defect or Bug The problem in the system that may cause a failure Error A conceptual mistake in the mind of the programmer that causes faults The mapping may not be 1-1

32. CSE7315M33 Slide # 32August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Black Box (Behavioral) Testing Testing that verifies whether the system behaves in accordance with its specifications No information is needed about the structure of the system to generate test cases Typically, one creates a test case to cover each requirement in the system specification

33. CSE7315M33 Slide # 33August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Characteristics of Black Box Testing Tends to find bugs that directly affect the end-users Particularly suitable for larger systems – where it is impossible to test every possible way the software could fail Tends to miss defects that stem from special cases or complex conditions

34. CSE7315M33 Slide # 34August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Glass Box (Structural) Testing Verifies the system behavior against its design Generally suitable to test small portions of the system, where the testers understand the software’s structure Typically you develop test cases for each path through the software, each potential value of each parameter, etc. Knowing the structure you know what to test

35. CSE7315M33 Slide # 35August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Comparing The Two Types of Testing Both techniques are applicable to testing at any level Each technique tends to have limitations that the other can compensate for The techniques used for quality control purposes are often hybrid

36. CSE7315M33 Slide # 36August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Testing Techniques Different testing techniques may be used for either black box or glass box testing, depending on the nature of the specification – Control flow testing – Loop testing – Data flow testing – Transaction flow testing – Syntax testing – Finite state testing

37. CSE7315M33 Slide # 37August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved The Nature of Bugs (Faults) - I Unit/Component Bugs – In principle, these should not escape the unit test process If unit testing is performed – The cost of out-of-phase detection is generally much more than in-phase detection – But these kinds of bugs can be detected earlier, via code reviews and inspections

38. CSE7315M33 Slide # 38August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved The Nature of Bugs (Faults) - II Integration Bugs – Arise due to wrong assumptions about interfaces or interaction between otherwise correct components – The more the design is coupled (i.e., the less modular it is), the harder it is to find the cause of integration bugs – These bugs can be reduced by good interface design and effective management of interface changes

39. CSE7315M33 Slide # 39August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved The Nature of Bugs (Faults) - III System Testing – Qualitative in nature – Detects problems where all correctly cooperating components fail to work together to achieve a system specification The limited capability of humans to handle complexity is the major cause of bugs. A good software process can reduce bugs significantly The limited capability of humans to handle complexity is the major cause of bugs. A good software process can reduce bugs significantly

40. CSE7315M33 Slide # 40August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Two Approaches to System Testing Coverage based testing – The test cases are generated to cover the whole system’s specifications with equal rigor Usage based testing – The test cases focus on more frequently used features – Ensures better reliability – Gains in schedule time

41. CSE7315M33 Slide # 41August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Testing Can Detect the Presence of Errors, but Not Their Absence DevelopTest ? Errors Injected Errors Reduced How Many Errors are Left?

42. CSE7315M33 Slide # 42August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved The Fundamental Problem of Quality Control... It does not improve anything

43. CSE7315M33 Slide # 43August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Other Problems with Quality Control Adversarial relationship between QC and developers Weak motive to improve Little help with improvement – You know it is defective, but not why

44. CSE7315M33 Slide # 44August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Evolution of Software Quality Quality Control Quality Assurance Quality Engineering Next Module Introduced in Next Module

45. CSE7315M33 Slide # 45August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Module Summary You have to define what you mean by quality Poor quality can cost money Quality control focuses on the end product Quality control does not improve the product A systematic method of testing yields more effective results

46. CSE7315M33 Slide # 46August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Note for Assignment 5 Testing belongs in section 3 of the SDP The Quality Engineering appendix should describe issues beyond testing (see next module)

47. CSE7315M33 Slide # 47August 13, 2006 CSE 7315 - SW Project Management / Module 33 - Software Quality Control Copyright © 1995-2006, Dennis J. Frailey, All Rights Reserved END OF MODULE 33