Software Engineering Modern Approaches Eric Braude and Michael Bernstein

Chapter 9: Quality and Metrics in Project Management

The Software Development Lifecycle Planning Maintenance Testing The Software Development Lifecycle Requirements analysis Implementation Learning Goals Of This Chapter How do you cultivate a quality mentality in a project? What are examples of project metrics? How do you use metrics for to improve projects? What is a software verification and validation plan? How are verification and validation planned for in a case study? Design Phase most relevant to this chapter is shown in bold © 2010 John Wiley & Sons Ltd.

Managing Quality Internal quality activities Plan project Introduce continuous quality attitude Perform requirements analysis Include tests for each requirement Create design Include tests for each unit Implement Perform unit tests Perform inspections External quality activities Plan QA , V&V Perform quality assurance Perform System Testing

Example of Responsibilities for Documentation, with Backup Name Primary Responsibility Backup Responsibility Alice Jarman Team leader Configuration management Bruce Stern Security Bob Crowder Internal quality Sarah Fournier Hans Lightman Requirements Release Vladimir Norsk Design John Green Implementation Susan Klein © 2010 John Wiley & Sons Ltd.

Defect Injection/Detection Defects detected: Per 100 requirements / per ... in the design / per KLoC, etc. This project / norm Phase in which defect was detected Detailed require-ments Design Implemen- tation Deploy-ment Phase in which defect was injected 2 / 5 0.5 / 1.5 0.1 / 0.3 3 / 1 1 / 3 3 / 2 2 / 2 5 / 3 3 / 12

Defect Plan TROUBLE! © 2010 John Wiley & Sons Ltd.

Project Dashboard Source: Software Program Manager’s Network (www.spmn.com)

Data on Activities Relating to Document Creation and Error Rates Research Meeting DraftingT0 ReviewingT1 FinalizingT2 Post-mortem TOTAL TimeL1 120 30 210 130 140 660 % Time 18% 5% 32% 15% 21% 10% N/A (Average % time)L2 -14% -7% -15% -30% -16% -18% QuantityL3 22 28 Productivity TBDI1 6.3 2.5I2 (Average productivity) 18.3I2 Self-assessed qualityL4 3 5 2 1 9 Defect rateL5 1.5I3 TBDI4 (Average defect rate) (1.1) TBD Process improvement note # (1) (2) (3) (4) © 2010 John Wiley & Sons Ltd.

Using Metrics For Process Improvement Our self-evaluation gives scores of 3 to review and 8 to research out of a forced average of 5. We spent 15% of our time on review and 25% on research. We will spend 20% on each of these activities for the next phase. Our defect rate declined steadily except for this phase, when it rose. This seemed to be due to a lack of common vision prior to dividing the writing. In past phases we succeeded in establishing a common vision of what we wanted to do before beginning to write our parts. Before beginning to write the next document we will confirm a shared vision. The ratio of requirements time to design time was 1.2, which is lower than this ratio from past successful projects in the company. Our design self-evaluation was 6, more than average. On our next project, we plan to spend 10% more time on requirements analysis at the expense of design time.

Improvement Across Projects Category Category Metric Metric Description Description Improvement Goal Improvement Quality Quality Defects/KLOC Defects/KLOC New defects found during formal QA testing, per churned KLOC 10% Predictability Predictability % Schedule % Schedule % improvement across releases 5% Accuracy Accuracy Improvement Improvement Efficiency Efficiency MM/KLOC MM/KLOC Pre Pre - - QA development effort per QA development effort per 10% churned KLOC churned KLOC Time to Market Time to Market Calendar time / Calendar time / Pre Pre - - QA calendar time per QA calendar time per 15% KLOC KLOC churned KLOC churned KLOC

IEEE 1012-2004 Software Verification & Validation Plans Table of Contents 1. Purpose 2. Referenced documents 3. Definitions 4. V&V overview 4.1 Organization* 4.2 Master schedule 4.3 Software integrity level scheme 4.4 Resource summary 4.5 Responsibilities 4.6 Tools, techniques & methodologies 5. V&V processes 5.1 Management of V&V 5.2 Acquisition V&V 5.3 Development V&V 5.4 Operation V&V 5.5 Maintenance V&V 6. V&V reporting requirements 6.1 Reporting 6.2 Administrative 6.3 Documentation 7. V&V administrative requirements 7.1 Anomaly reporting & resolution 7.2 Task iteration policy 7.3 Deviation policy 7.4 Standards, practices & conventions 8. V&V documentation requirements * Subheadings are typical examples (IEEE) © 2010 John Wiley & Sons Ltd.