The Organizational Impacts on Software Quality and Defect Estimation Stephen Lopez-Couto

Discussion Topics Introduction Defect Concepts Definition of a defect Static vs. Dynamic Discovery Defect Estimation Methods Linear Regression Capture/Recapture AI Defect Estimation Variables Further Research Conclusion

Introduction Defect Estimation The act of “guessing” the number of defects that exist in a current software baseline Purpose of this paper is to determine the usefulness of metrics that are not explicitly tied to the code to make the estimates

Introduction Organizational Elements: Development Team Size Developer Experience Institutional Processes CMMI level for example Development Tools Development Schedule Programming Language Software Architecture

Defect Concepts: What is a Defect? There is no common definition of a defect Consider the following: Program is supposed to add the variables ‘y’ and ‘i’ together and put the value into ‘x’ int i = 5; int y = 6; int x = y+1 Print(x);

Defect Concepts: What is a Defect? The programmer introduced a defect Correct code: int x = y + i There is no common definition of a defect Consider the following: Program is supposed to add the variables ‘y’ and ‘i’ together and put the value into ‘x’ int i = 5; int y = 6; int x = y+1 Print(x);

Defect Concepts: What is a Defect? Two things occur before a defect can be detected Fault – Incorrect value in the internal state of the program Failure – When a fault is realized as output int i = 5; int y = 6; int x = y+1 // Causes a Fault Print(x); // Causes a Failure A Failure may not be detected until well after the fault has occurred!

Defect Concepts: Static Vs. Dynamic Discovery Static – Does not utilize executing code Software Inspections Walkthroughs Complexity Mapping Dynamic – Utilizes executing, compiled code Runtime tests Automated defect discovery tools

Defect Estimation Methods Estimating the number of defects is by no means an exact science There are lots of different ways to determine an estimate Most of the methods are highly tied to a specific organization or software baseline Not good enough for general use Three general methods will be discussed Linear Regression Capture/Recapture Artificial Intelligence Bayesian Belief Networks Neural Networks

Defect Estimation Methods Linear Regression Determines a mathematical expression that relates some number of independent variables (the input metrics) and the dependent variable (defect estimation) Variable Type: Numeric Only

Defect Estimation Methods Capture/Recapture Some number of defects are seeded into the software at development time. When testing occurs ratio of seeded to unseeded defects found is used to estimate the total number. Variable Type: N/A. Only track defect counts.

Defect Estimation Methods AI: Bayesian Belief Networks A network that weighs the relationship among different variables is created (using expert knowledge) and processed using Bayesian probabilities to determine the quality of the software. Variable Type: any

Defect Estimation Methods AI: Neural Networks A multilayer perceptron using the back propagation algorithm is trained on legacy defect data and then provides estimates based on input data. Variable Type: Numeric Only

Defect Estimation Variables

Further Research Three main areas Transformation of non numerical data into a numerical form Additional estimation methods that do not utilize numeric only data Determination of common relationships among the input metrics, independent of which method is used

Conclusion Organizational Element Data is useful when paired with the proper estimation method Limitations of the most common method (linear regression) has led to a general shunning of these metrics Defect Estimation approaches are not ready for real world use They are too tied to specific cases

Questions Questions?