Empirical Validation of OO Metrics in Two Different Iterative Software Processes Mohammad Alshayeb Information and Computer Science Department King Fahd University of Petroleum and Minerals Alshayeb, M. and Wei Li, “ An Empirical Validation of Object- Oriented Metrics in Two Iterative Processes, ” IEEE Transactions on Software Engineering, Vol. 29, No. 11, November 2003.

4 January Agenda  Research Objective.  Introduction.  Research Data.  Research Hypotheses.  Validation Studies.  Conclusion.  Future Work.

4 January Objective  Validate the Predictive Capability of Object-Oriented (OO) Metrics in Two Iterative Software Processes.

4 January Measurement Acceptance  No Wide Acceptance of Software Measurements Because :  Management personnel rarely know how to analyze, interpret or use the software metrics.  Various ways of analyzing software metrics.  Lack of measurement validation.

4 January Measurement Validation  Validation Assures That Measurements Assess What They Are Supposed to Measure.  Types of Validation:  Theoretical: the measure is a proper numerical characterization of a claimed attribute.  Empirical: establishes the accuracy of a prediction system by comparing the model performance with known data in a given environment.

4 January Software Processes  The Waterfall Model  Prototype Model  Rapid Application Development Model  The Incremental Model  The Spiral Model  Component Assembly Model  The Iterative Development Process Model  Extreme Programming

4 January The Iterative Development Process Domain Analysis Release Iteration Prototype Highest Risk Risk Assessment Software Architecture Requirements definition Test Suite & Environment Development Integrate with Previous Iteration Customer Requirements Iteration Luckey et al., “ Iterative Development Process with Proposed Applications, ” 1992

4 January Agile Process: Extreme Programming (XP)  “ A lightweight, efficient, low-risk, flexible, predictable, scientific, and fun way to develop software ” -Beck,  XP Applicability:  Projects with vague or changing requirements.  Small groups of programmers.  The need for customer input.

4 January XP Is Different  Continuity of Feedback Throughout Its Short Cycles.  Incremental Planning.  Flexibility in Scheduling on the Basis of Business Needs.  Dependency on Unit Tests Written by Both Developers and Customers.  Pair Programming.

4 January XP Breakdown Customer System Priority Organization of user stories Story 1Story 2Story 3 Division of system into user stories Iteration 1 Iteration 2Iteration 3 Iterations assigned for the development of each story Story 2Story 3Story 1 Release 2 Release 3 Task 1Task 3 Test Case 2Test Case 1Test Case 3 Release 1 Working, tested system with story 2 integrated Passed, integrate with the system Task 2

4 January XP and Other Software Processes Analysis Design Implementation Testing WaterfallLong Cycled Iterative ProcessShort Cycled Iterative Process XP Beck, “ Embracing changes with Extreme programming, ” 1999

4 January Frameworks  “ A Framework is an OO class hierarchy plus a built-in model which defines how the objects derived from the hierarchy interact with one another ” – Lewis et al,

4 January OO Metrics Suites C&K suite  Weighted Methods per Class (WMC)  Depth of Inheritance Tree (DIT)  Number Of Children (NOC)  Coupling Between Objects (COB)  Response For a Class (RFC)  Lack of Cohesion of Methods (LCOM) Li Suite  Number of Ancestor Classes (NAC)  Number of Local Methods (NLM)  Class Method Complexity (CMC)  Number of Descendent Classes (NDC)  Coupling Through Abstract data types (CTA)  Coupling Through Message passing (CTM)

4 January Research Data Iterative Software Process Process: Extreme Programming (XP)Process: the Framework Iterative Data: Two Java ApplicationsData: JDK Releases

4 January Research Data System 1 Story 1 Story 2 Story 3 Story 4 Story 5 Story 6 JDK JDK 1.0 JDK 1.1 JDK 1.2 JDK 1.3 JDK 1.4 System 2 Story 1 Story 2 Story 3 Story 4 Story 5 Story 6 Story 7

4 January Data Collection  The tasks planned for the current iteration cycle. The tasks come from the user stories, which are supplied by the customer.  A description of the progress or failure of progress towards completing the tasks.  The time spent on each task measured by hours.  A description of the problems encountered during implementing a task and the attempted solutions.  A description of the changes made on the system, the reasons for the changes, and the affected classes.

4 January Data Analysis  Tools support  Webgain ’ s Quality Analyzer  Metamata Metrics  Object-Oriented metrics collection.  Collects 12 metrics.  Krakatau Project Manager for Java  Collecting development/maintenance effort measured by lines of code.

4 January Metrics Validation Criteria Li and Henry Basili et al.Briand et al. EL Emam et al. Briand and W ü st Suite of metrics CK metrics Cohesion and Coupling C&K, Lorenz and Kidd Size, Cohesion and Coupling CK& and Li metrics Type of products OO dialect of Ada C++AdaJavaC++Java Dependent variable Number of changes in components Fault occurrence in C++ classes Fault occurrence in Ada packages Fault- proneness of a class effortDesign Changes (Lines added, changed, and deleted), in a class. Maintenance effort, refactoring effort Statistical technique Least- square regression Logistic regression Poisson regression and regression trees Least-square regression Long-Cycled and Short-Cycled Iterative Processes Short-Cycled Iterative Processes Alshayeb and Li Process?

4 January Multiple Regression  Investigate the Relationship Between a Dependent Variable and Multiple Independent Variables y =     x  +   x    x     The x ’ s are the independent (predictor or regressor) variables.  y is the dependent (response) variable.   — the statistical error — is the difference between the observed value y and the line (     x  +   x     x  )    regression coefficient.  R 2 : The variance percent in the dependent variable based on the knowledge of the independents.

4 January Empirical Validation of Object- Oriented Metrics  Empirically Validate the Predictive Capability of OO Metrics in the Long- Cycled (Framework) and Short-Cycled (Agile) Iterative Software Processes  Investigate the relationship between metrics and development/maintenance efforts in two forms of iterative processes.  Types of Changes:  Lines added (LA), Lines changed (LC), and Lines deleted (LD).  Refactoring and Error-Fix efforts.

4 January Example of Lines Added, Deleted and Changed // Example.java version 1 public class Example{ public static void main(String args[]){ System.out.println("Welcome to java"); System.out.println("Hello"); System.out.println("World"); } // Example.java version 2 public class Example{ public static void main(String args[]){ System.out.println("Hello"); System.out.println("Java"); System.out.println("new World"); } Deleted Added Changed

4 January Research Hypotheses  Hypothesis 1: Using OO metrics, we can predict design changes indicated by added, changed, and deleted source lines of code in classes from one iteration (release) to the next in the long-cycled framework iterative process.  Hypothesis 2: Using OO metrics, we can predict added, changed, and deleted source lines of code in classes from one iteration to the next in the short-cycled XP iterative process.

4 January Research Hypotheses  Hypothesis 3: Using OO metrics, we can predict error-fix maintenance effort (measured in man-hours) in classes from one iteration to the next in the short-cycled XP iterative process.  Hypothesis 4: Using OO metrics, we can predict refactoring effort (measured in man- hours) in classes from one iteration to the next in the short-cycled XP iterative process.

4 January Dependant and Independent Variables -Hypotheses 1 and 2  Dependant Variables Are:  LA, LC, and LD.  Independent Variables  WMC, DIT, LCOM, NLM, CTA, and CTM metrics.

4 January Research Results -Hypothesis 1  Hypothesis 1 Dependent Variables JDK JDK JDK JDK R 2 % LA LC LD

4 January Research Results -Hypothesis 1  Object-Oriented Metrics Cannot Predict Design Changes Indicated by Added, Changed, and Deleted, Source Lines of Code in Classes From One Iteration (Release) to the Next in the Long-cycled Framework Iterative Process.

4 January Research Results -Hypothesis 2  Hypothesis 2 Dependent Variables Story 5-6 R 2 % LA 95.6 LC 97.9 LD 88.5

4 January Research Results -Hypothesis 2  Hypothesis 2 Dependent Variables Story 4-5Story 6-7 R 2 % LA LC LD

4 January Research Results -Hypothesis 2  Object-Oriented Metrics May Predict Design Changes Indicated by Added, Changed, and Deleted Source Lines of Code in Classes From One Iteration to the Next in the Short-cycled XP Iterative Process Only When System Design Accumulates to Certain Mass.

4 January Dependant and Independent Variables -Hypotheses 3 and 4  Dependant Variables Are:  Refactoring and Error-Fix efforts.  Independent Variables  WMC, DIT, LCOM, NLM, CTA, and CTM metrics.

4 January Research Results – Hypotheses 3,4  Hypotheses 3 and 4 System 1RefactoringError Fix R 2 % Story Story

4 January Research Results – Hypotheses 3,4  Hypotheses 3 and 4 System 2RefactoringError Fix R 2 % Story Story Story Story

4 January Research Results – Hypotheses 3,4  Object-oriented Metrics Can Predict the Refactoring Effort and Error-fix Maintenance Effort in the XP-like Process When the System Design Accumulates to Certain Mass.

4 January Conclusion  The Predictive Capability of OO Metrics:  OO metrics are reasonably effective in predicting design changes in the short-cycled XP-like process, but they are largely ineffective in the long-cycled framework evolution process.  OO metrics' predictive capability is limited to the design and implementation changes during the development iterations, not the long-term evolution of an established system in different releases.  OO metrics are effective in predicting error-fix and refactoring efforts in the short-cycled agile process.

4 January Conclusion  The Development Software Process is Related Directly to the Predictive and Project-Progress Indicative Capabilities of OO Metrics:  It is thus essential to clearly identify the software process when validating OO metrics.

4 January Future Work  Studies Can Target OO Metrics Capabilities in Different Software Processes.  Validation Studies More Diverse Data Sets Are Needed to Test the Prediction Accuracy.  Automate the Steps Followed in This Study and Build a Tool to Gather, to Analyze, and to Use Metrics and the Statistical Models in a CASE Tool.

4 January Questions