“How to Measure the Impact of Specific Development Practices on Fielded Defect Density” by Ann Marie Neufelder Presented by: Feride Padgett

Outline Introduction Results Conclusion

Introduction The author evaluated 17 organizations based on a questionnaire of 177 parameters The 17 organizations were: –Defense systems(5) –Process control manufacturing equipment(11) –Mathematical software(1)

The Questionnaire It had 5 sections: –Organization commitment –Life cycle practices –Product characteristics –Change control practices –Informational questions

Scoring Mathematical correlation of observed practices vs. observed defect density in terms of KSLOC of assembler A correlation of -1 means the practice perfectly correlates to LOWER defect density

High Scorers The common features of organizations with the lowest defect densities: –Software engineering is a part of the process, not an art form –A well-rounded sound set of development practices vs. a “single bullet” –Formal and informal reviews of requirements prior to design and code –Testers are involved in requirements phase

Low Scorers The common features of organizations with the highest defect densities: –Lack of software management –Misconception of programmer knows what customer needs better than the customer –Complete void of requirements definition phase –Insufficient testing methods

Ten Parameters with the Highest Correlation Consistent and documented formal and informal reviews of the software and system requirements prior to design and code-.87 The language and operating system is well supported by industry-.86 Existence and use of test beds-.85

Ten Parameters with the Highest Correlation Incremental testing (as opposed to “big bang” testing.)-.84 Scheduled regression testing consistently performed by independent testers-.83 Defect and failure tracking systems that are used by testers for test plan development-.81

Ten Parameters with the Highest Correlation There is a defined life cycle model that best suits the application, market, ad organization-.81 Testers involved during requirements and design. Test plans are started during the requirements phase-.76

Ten Parameters with the Highest Correlation Automated unit testing tools-.76 Explicit test cases for user documentation -.76

Reviews Requirements reviews-.87 Design reviews-.65 Coding reviews-.12

Results Software managers and engineers often assume engineering practices that decrease defects will also increase development time Her study shows that this assumption is not true Organizations with the best practices had the highest probability of making their schedules

Results Some companies were striving for more releases while their customer surveys showed that their customer wanted fewer releases with higher quality More corrective action releases mean more downtime for customers

Results This correlation can and is used to: Predict defect density Improve software development practices for the best return on investment

Conclusion This is an ongoing work. New data is added on a yearly basis The study shows that making software engineering principles a part of the process decreases the defect density and increases the probability of on time delivery