Statistical Software Quality Assurance Implies –Information about defects is collected and categorized –An attempt is made to trace each defect to underlying cause –Use of Pareto Principle to identify vital causes (80% of defects can be traced to 20% of causes/mistakes) –Move to correct the problems that have caused the defects

Software Engineering II Lecture 30 Fakhar Lodhi

Recap

Example Information about defects is collected for one year and categorized as follows: 1.Incomplete or erroneous specifications (IES) 2.Misinterpretation of customer communication (MCC) 3.Intentional deviation from specifications (IDS) 4.Violation of programming standards (VPS) 5.Error in data representation (EDR) 6.Inconsistent component interface (ICI) 7.Error in digital logic (EDL) 8.Incomplete or erroneous testing (IET) 9.Inaccurate or incomplete documentation (IID) 10.Error in programming language translation or design (PLT) 11.Ambiguous or inconsistent human computer interface (HCI) 12.Miscellaneous (MIS)

Error CategorySeriousModerateMinorSub Total IES MCC IDS VPS EDR ICI EDL IET IID PLT HCI MIS Total

Error CategorySub Total % of Total errors IES20522 MCC15617 IDS485 VPS253 EDR13014 ICI586 EDL455 IET9510 IID364 PLT606 HCI283 MIS566 Total942 IES, MCC and EDR are vital errors - cause 53% of all errors

Error CategorySerious % of Serious errors IES3427 MCC129 IDS11 VPS00 EDR2620 ICI97 EDL1411 IET129 IID22 PLT1512 HCI32 MIS00 Total128 IES, EDR, PLT and EDL constitute about 80% of serious errors

Excel example Now start corrective action focused on vital few For example for EDR –Review the data representation techniques to identify the possible improvement areas –Adopt a use case tool for data modeling and perform stringent data design reviews

Error Index (EI) Used to develop an overall indication of improvement in software quality E i – the total number of errors uncovered during the ith step in the SE process S i – number of serious errors M i – number of moderate errors T i – number of minor errors PS i – product size at the ith step w s, w m, w t – weighting factors for serious, moderate, and minor errors Recommended values – 10, 3, 1 respectively

Error Index (EI) At each step of the software process a Phase Index is computed PI i = w s (S i /E i ) + w m (M i /E i ) + w t (T i /E i ) EI – cumulative effect on each PI i = ∑(i x PI i )/PS i –Weighting errors encountered in the SE processes more heavily than those encountered earlier –Used to develop an overall indication of improvement in software quality

Software Reliability Defined as: –Probability of failure free operation of a computer program in a specified environment for a specified time –E.g. program X is estimated to have a reliability of 0.96 over 8 elapsed hours. What is meant by the term failure? –Failure is non-conformance to software requirements –Grading From annoying to catastrophic Time to fix from minutes to months Ripples from fixing

Software reliability Hardware versus software reliability –Hardware reliability is predicted on failure due to wear, rather than failure due to design –Software – no wear and tear Mean time between failure – MTBF –MTBF = MTTF + MTTR where MTTF is mean time to failure and MTTR is mean time to repair

Software reliability Arguably MTBF is far better than defects/kloc –Each error does not have the same failure rate –User is concerned with failure and not with total error count