© USC-CSE Feb Keun Lee ( & Sunita Chulani COQUALMO and Orthogonal Defect Classification(ODC)

© USC-CSE Feb Current COQUALMO Model - Results and Challenges COQUALMO – ODC Research Approach Example Results Issues and Research Plans COQUALMO and Orthogonal Defect Classification(ODC)

© USC-CSE Feb COCOMO II Current COQUALMO System COQUALMO Defect Introduction Model Defect Removal Model Software platform, Project, product and personnel attributes Software Size Estimate Defect removal profile levels Automation, Reviews, Testing Software development effort, cost and schedule estimate Number of residual defects Defect density per unit of size

© USC-CSE Feb Partion of COQUALMO Rating Scale Very LowLowNominalHighVery HighExtra High Automated Analysis Simple compiler syntax checking Basic compiler capabilities Compiler extension Basic req. and design consistency Intermediate- level module Simple req./design More elaborate req./design Basic dist- processing Formalized specification, verification. Advanced dist- processing Peer Reviews No peer review Ad-hoc informal walk-through Well-defined preparation, review, minimal follow-up Formal review roles and Well-trained people and basic checklist Root cause analysis, formal follow Using historical data Extensive review checklist Statistical control Execution Testing and Tools No testingAd-hoc test and debug Basic test Test criteria based on checklist Well-defined test seq. and basic test coverage tool system More advance test tools, preparation. Dist- monitoring Highly advanced tools, model- based test COCOMO II p.263

© USC-CSE Feb COQUALMO Defect Removal Estimates - Nominal Defect Introduction Rates Delivered Defects / KSLOC Composite Defect Removal Rating

© USC-CSE Feb Multiplicative Defect Removal Model - Example : Code Defects; High Ratings Analysis : 0.7 of defects remaining Reviews : 0.4 of defects remaining Testing : 0.31 of defects remaining Together : (0.7)(0.4)(0.31) = 0.09 of defects remaining How valid is this? - All catch same defects : 0.31 of defects remaining - Mostly catch different defects : ~0.01 of defects remaining

© USC-CSE Feb Example UMD-USC CeBASE Data Comparisons “Under specified conditions, …” Peer reviews are more effective than functional testing for faults of omission and incorrect specification(UMD, USC) Functional testing is more effective than reviews for faults concerning numerical approximations and control flow(UMD,USC) Both are about equally effective for results concerning typos, algorithms, and incorrect logic(UMD,USC)

© USC-CSE Feb ODC Data Attractive for Extending COQUALMO - IBM Results (Chillarege, 1996)

© USC-CSE Feb COQUALMO Extension Research Approach Extend COQUALMO to cover major ODC categories Collaborate with industry ODC users - IBM, Motorola underway - Two more sources being explored Obtain first-land experience on USC digital library projects - Completed IBM ODC training - Initial front-end data collection and analysis

© USC-CSE Feb Artifacts - Operational Concept, Requirements, Software Architecture documents Activities - Perspective-based Fagan inspections Triggers - Environment or condition that causes defect Digital Library Analysis to Date - in ODC terms

© USC-CSE Feb Front End (Information Development) Triggers 1.Clarity – confusing or difficulty to understand information. 2.Style – inappropriate or difficulty to understand the manner of expression 3.Accuracy – incorrect information 4.Task Orientation - inappropriate presentation to perform task 5.Organization – relationship between parts is not conveyed 6.Completeness – missing information. 7.Consistency – the expression manner is not displayed in a consist manner

© USC-CSE Feb Initial Digital Library Project ODC Analysis - Trigger percentage Distribution by Team

© USC-CSE Feb Initial Digital Library Project ODC Analysis - Number of Triggers Defects by Team

© USC-CSE Feb Understand anomalies in Digital Library Data - Number of Team 22 defects - Team 4 completeness defects - Due to differences in artifacts or procedures? Continue Digital Library ODC collection & analysis - Detailed Design, code, test Obtain, analyze industry ODC data - Looking for more sources of ODC Data Issues and Research Plans