Personal Software Process Software Quality CIS 376 Bruce R. Maxim UM-Dearborn

These notes are based on: Introduction to the Personal Software Process Watts S. Humphrey Addison-Wesley Longman (1997)

Software Quality Product meets the customer’s needs (not necessarily the same as customer’s wants) Defects are the primary measure of quality in PSP Yield Management –defect removal –defect prevention

Hierarchy of Needs Software performs its required tasks Product meets its performance requirements Software is usable Development is economical and timely Product is dependable and reliable

Software Bugs Latent bugs must –be operationally insignificant –not be destructive –not be observed often Bugs are not important to the customer if they do not –affect operations –cause inconvenience –cost time or money –cause loss of confidence in software results

PSP Quality Focus Low defect content is an essential prerequisite to quality software process Low defect rates can best be achieved at the PSP level (e.g. individual SE’s) SE’s are the ultimate source of defects are injected SE’s need to remove defects, determine their causes, and learn to prevent them

Testing and Inspections The sooner a defect is located the cheaper it is to repair and faster the repair Design inspections will –improve product quality –lead to a more predictable schedule –allow more time to focus on quality issues It is important for SE’s to review their own work before giving it to others to review

Some Fix Time Data Some typical fix time ratios –IBM rules of thumb - coding: 1.5; testing: 60; usage: 100 –Boehm - design: 1; development test: 15 to 40; acceptance test: 30 to 70; operation: 40 to 1000 –Remus - design: 1, code: 20, test: 82 –Ackerman - test: times inspection time –Russell - inspection: 1, test: 2 to 4, use: 33 –PSP research - unit test takes 12 times longer than code review to find and fix defects

Cost of Quality - part 1 Failure costs –repair, rework, scrap –in PSP includes all compile and testing time Appraisal costs –cost of defect inspections –in PSP includes all design and code review time

Cost of Quality - part 2 Prevention costs –finding and resolving defect causes –handle before project starts –should be a process (not product) activity –in PSP this includes formal specification and design work prototyping process analysis and improvement

PSP Quality Strategy - part 1 Identify PSP quality objectives, e.g. –remove all defects before first compile Establish PSP process quality measures, e.g. –overall process yield –LOC reviewed per hour Examine products reviewed –determine their ratings on the measures –see which behaviors impacted these results

PSP Quality Strategy - part 2 Based on these data, identify your most effective work practices. Incorporate these practices in your process artifacts –process scripts –checklists –forms Identify measures that predict process quality –use these as control variables –set specifications for these variables

PSP Quality Strategy - part 3 Track your performance against these specifications. Track your process to determine –when and if to change these specifications –actions to take for further process improvement

Appraisal to Failure Cost Ratio Appraisal COQ = 100*(design review time + code time)/ (total development time) Failure COQ (Cost of Quality) = 100*(compile time + test time)/(total development time) A/FR (Appraisal to Failure Cost Ratio) ratio = (Appraisal COQ)/(Failure COQ)

Yield vs. A/FR High A/FR ratios appear to lead to higher yields 70+% yields not achieved without A/FRs near 1.0 or above high A/FR does not guarantee high yield - the appraisal time must be spent effectively

Test Defects vs. A/FR Defects are reduced by high A/FR ratios To get very low numbers of test defects, A/FR values of above 2.0 appear required. With A/FRs between 1.0 and 2.0, low test defects are occasionally obtained. With an A/FR of below 1.0, low test defects are rare.

Yield and A/FR vs. Productivity Productivity has considerable variation among individuals. In some cases, high yield produces higher productivity but in others it does not. High A/FR also sometimes results in high productivity and sometimes not. Yield and A/FR are somewhat independent of productivity.

Process Benchmarks It is desirable to have high values for –yield –A/FR –productivity Since yield and A/FR are closely related, a yield vs A/FR benchmarking chart would not be useful. Yield vs. productivity or A/FR vs. productivity would likely be useful benchmarking charts.

Defect Removal Strategies - part 1 Focused inspections and reviews –high level design reviews - structural issues –detail level design reviews - logic correctness –code reviews - implementation issues Do not address –system issues in DLD –design issues in code reviews

Defect Removal Strategies - part 2 Do thorough reviews the first time and then trust them. Do thorough unit tests –black box –white box Do system tests once unit testing is done –integration –system –regression

Defect Prevention Finding defects is expensive so it is better to avoid them in the first place Defect prevention costs are only incurred once, but their savings impact every future project For PSP, defect prevention actions include improved design methods and prototyping

Defect Prevention Strategies Set priorities for defect types that are –found most frequently –the most troublesome –easily prevented In setting priorities consider the defects most often found in integration and system testing Incorporate your prevention actions in your process scripts, checklists, and forms

Defect Prevention Process Follow an established schedule Select 1 or 2 defect types for initial action Track and evaluate the effectiveness of the defect prevention actions Make needed adjustments and continue