Measuring Software Quality Norm Goodkin Quality Matrix International, Inc.

Why is there never enough time to do it right, but always enough time to do it over?

Requirements Management RequisitePro tm, Doors tm, Caliber-RM tm What about Quality? Triple Constraint? Scope Earned Value MS Project Primavera Cost Schedule

“Would you tell me, please, which way I ought to go from here?” That depends a good deal on where you want to get to,” said the Cat. I don’t much care where--” said Alice. Then it doesn’t matter which way you go,” said the Cat. “--so long as I get somewhere,” Alice added as an explanation. “Oh, you’re sure to do that,” said the Cat, “If only you walk long enough.”

You can’t manage what you can’t measure.

1D, 2D, 3D 1D = Total Number of Bugs –But some bugs are more equal than others 2D = Bugs by level of Severity –But one area can suffer while another is great 3D = Levels of Severity + Area Affected –Severity is not the same as Urgency for Fix.

Severity = worst case scenario 8 = pretty bad 7 = service outage 6 = old feature fails 5 = new feature fails … 0 = spelling error in an internal operations message

Accumulation of Low Level Bugs Lots of Level 3 and Level 4 bugs can give you a tummy ache. Set limits at all levels, for all areas. Take appropriate action if it looks like you are going to exceed the limits. –Fast Track or Crash to allow more fixing –Prepare users to deal with the problems

Setting the Bar Set realistic goals Base goals on past performance Enjoy the success of achieving goals Gold Plating is as wrong for Quality as it is for Scope.

Zero Defects/Zero Downtime? Is ZD 2 a realistic requirement? Goals vs. Objectives Continuous Improvement – a form of Gold Plating? Set Quality Targets based on project history Achieve Acceptable Quality

Quality Review Board Represent Stakeholders Not too senior Subject Matter Experts max Quantify Problem Severity Establish Quality Targets

Quality Review Board?

The Quality Matrix

3D Quality Matrix – Sample

Apply Feedback New Perceived Quality Update Historical Quality Data Set Quality Targets

Weigh the Risk Which bug fixes will be the least risky? If two have equal severity, fix the one with less risk. Generally, the larger the change, the larger the risk. Fixing a low severity problem can be risky. There’s no such thing as zero risk with software.

Summary Measure Quality Set reasonable quality targets Represent Stakeholders Balance Schedule, Scope, Cost and Quality Use feedback to get it right