1. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 1 COCOMO II Maintenance Model Upgrade Vu Nguyen, Barry Boehm Center for Systems and Software Engineering (CSSE) CSSE Annual Research Review 2010 Mar 8 th, 2010

2. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 2 Outline Motivation Research Problem and A Solution Major Software Maintenance Estimation Models COCOMO II for Maintenance Research Validation and Preliminary Results Next Steps Expected Main Contributions References

3. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 3 Magnitude of Software Maintenance Majority of software costs incur after the first operational release [Boehm 1981] Fig. 1. Software maintenance cost versus total software cost Maintenance vs. Total Software Cost 0 10 20 30 40 50 60 70 80 90 100 Zelkowitz et al. (1979) McKee (1984) Moad (1990) Erlikh (2000) Studies % of Software Cost Others Maintenance

4. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 4 Importance of Software Estimation in Managing Software Projects Estimation is a key success factor of software projects –Two out of three most-cited project failures are related to resource estimation, according to a CompTIA survey in 2007 [Rosencrance 2007] Cost estimate is the key information for investment, project planning and control, etc. Many software estimation approaches have been proposed and used in industry –E.g., COCOMO, SEER-SEM, SLIM, PRICE-S, Function Point Analysis

5. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 5 The Problem These models are built on the assumptions of new development projects Problem is that these assumptions do not always hold in software maintenance due to differences between new development and maintenance –Low estimation accuracies achieved

6. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 6 A Solution Improved COCOMO models that allow estimators to better determine the equivalent size of maintained software and estimate maintenance effort can improve the accuracy of effort estimation of software maintenance.

7. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 7 Major Software Maintenance Estimation Models Estimate cost for the maintenance phase after the software is delivered All types of regular maintenance tasks are included Most of the models use SLOC as a size input Lack of empirical evaluation of these models ModelsKey size metrics COCOMOCode added, modified; equivalent SLOC KnowledgePlanCode added, reused, deleted, changed PRICE-SCode added, adapted, reused SEER-SEMEffective SLOC or FP SLIMCode added and modified

8. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 8 Outline Motivation Research Problem and A Solution Major Software Maintenance Estimation Models COCOMO II for Maintenance –Modeling Process –A Software Maintenance Sizing Method –Effort Model –Calibration Techniques Research Validation and Preliminary Results Next Steps Expected Main Contributions References

9. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 9 The Modeling Process Perform Behavioral Analysis Identify relative significance of factors Determine sizing method for maintenance Perform Expert-judgment and Delphi assessment Gather project data Calibrate model parameters Determine parameter variability Evaluate model performance 3A Analyze existing literature 1 3B Determine form of effort model 4 5A 5B 7 8 Test hypotheses about impact of parameters 6 Perform maintenance experiment to validate some size measures 2 Fig. 2. The Modeling Process, Adapted from the COCOMO Modeling Process [Boehm 2000]

10. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 10 COCOMO II for Maintenance – A Glance An extension of COCOMO II –COCOMO is the non-proprietary most popular model –COCOMO has attracted many independent validations and extensions Two Components –A Unified Reuse and Maintenance Model Determining equivalent SLOC for reuse and maintenance –COCOMO Effort Model for Maintenance Using a different set of parameters and constants

11. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 11 Software Maintenance Sizing Size is a key determinant of effort Sizing method has to take into account different types of code Automatically translate External Modules Existing System Modules Reused Modules Adapted Modules New Modules Manually develop and maintain Automatically Translated Modules Preexisting CodeDelivered Code Fig.3. Types of Code

12. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 12 A Unified Reuse and Maintenance Model – 1/3 Objectives –Provide a consistent size measure for both reuse and maintenance –Better account for different types of code Changes –Use a single model for both reuse and maintenance –Redefine the reuse model equation and parameters to account for code expansion allow equivalent SLOC to be determined from completed code include deleted SLOC in modified modules, but excluded SLOC in deleted modules smooth the curve representing nonlinear effects

13. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 13 A Unified Reuse and Maintenance Model – 2/3 New AAF and AAM equations Major changes AAF – Adaptation Adjustment Factor (% of modification) AAM – Adaptation Adjustment Multiplier DM – Percentage of Design Modified, accounting for only design changes made to the preexisting modules IM – Percentage of Integration changed, relative to the integration of the preexisting modules CM – Percentage of Code Modified, including added, modified, deleted

14. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 14 A Unified Reuse and Maintenance Model – 3/3 Compute Equivalent SLOC (ESLOC): –New Modules: –Adapted Modules: –Reused Modules: –Total Equivalent KSLOC: : KSLOC of the adapted modules before changes : KSLOC of the reused modules

15. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 15 COCOMO Effort Model for Maintenance Follows the same COCOMO II non-linear form Where, PM – project effort measured in person-month A – a multiplicative constant, calibrated using data sample B – an exponent constant, calibrated using data sample Size – software size measured in SLOC EM – effort multipliers, cost drivers that have an multiplicative effect on effort SF – scale factors, cost drivers that have an exponential effect on effort Linearize the model using log-transformation log(PM) =  0 +  1 log(Size) +  i SF i log(Size) +  j log(EM j )

16. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 16 Calibration Process of fitting data to the model to adjust its parameters and constants Rating scales for cost drivers Sample data Delphi survey of experts (Expert-judgment estimates) Model Calibration New rating scales for cost drivers and Constants Calibration Techniques: - Ordinary Least Squares Regression (OLS) - Bayesian Analysis - Constrained Regression Technique [Nguyen 2008] Fig.4. Calibration Process

17. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 17 Outline Motivation Research Problem and A Solution Major Software Maintenance Estimation Models COCOMO II for Maintenance Research Validation and Preliminary Results Next Steps Expected Main Contributions References

18. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 18 Data Collection – 1/2 Collect data of completed maintenance projects from industry –Maintenance type: error corrections, enhancements, etc. Excluding reengineering and language-migration projects CodeCount tool (UCC) is used for size collection [Nguyen 2007] Release N Project starts for Release N+1 Release N+1Project starts for Release N+2 Baseline 1 Baseline 2 Timeline Maintenance project N+1 Fig.5. Release Period to be Collected

19. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 19 Data Collection – 2/2 Metrics MetricDescription ProductName of software product. ReleaseRelease number. A software product has multiple releases, each associated with a maintenance project. EffortTotal time in person-month for the maintenance project delivering the release. SLOC adaptedSum of SLOC added, modified, and deleted of adapted modules. SLOC pre-adaptedSLOC count of the preexisting modules to be adapted. SLOC addedSLOC count of new modules. SLOC reusedSLOC count of reused modules. CMThe percentage of code added, modified, and deleted. DMThe percentage of design modified. IMThe percentage of implementation and test needed for the preexisting modules. SUSoftware Understanding UNFMProgrammer Unfamiliarity Cost driversRating levels for 22 cost drivers.

20. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 20 Current Status Completed the controlled experiment Collected 83 projects/releases –64 from a large organization member of CSSE Affiliates –14 from a CMMI-Level 5 company in Vietnam –4 from a CMMI-Level 3 company in Thailand Generated preliminary results using Bayesian and constrained regression techniques –Use cost-driver rating scales from Delphi exercise for COCOMOII.2000 (Delphi COCOMOII.2000 ) Delphi survey has yet to be completed

21. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 21 Preliminary Results – 1/2 Relative Impact of Cost Drivers on Effort –Rating scales: Delphi COCOMOII.2000 COCOMO II.2000 Values [Boehm 2000] Values Generated by COCOMO II for Maintenance Using the Bayesian Analysis Fig. 6. Productivity Ranges Sample data: 161 projectsSample data: 83 projects +18% +16% -17% -37%

22. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 22 Preliminary Results – 2/2 ModelMMREPRED(0.3) COCOMO II.200064%34% COCOMO II for Maintenance: Bayesian51%39% COCOMO II for Maintenance: CMRE48%46% Estimation Accuracies –Estimated 83 projects using COCOMO II.2000 COCOMO II for Maintenance: Bayesian analysis COCOMO II for Maintenance: Constrained regression –Rating scales: Delphi COCOMOII.2000 –Computed MMRE and PRED(0.3) values:

23. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 23 Next Steps Perform Delphi survey to obtain expert- judgment rating scales for maintenance Continue data collection –COCOMO data for both new development and maintenance projects Validate the research hypotheses Analyze and validate the models

24. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 24 Expected Main Contributions A model for sizing maintenance and reuse An extended COCOMO model for maintenance A set of cost drivers and levels of their impact on maintenance cost Empirical validations on the impact of cost drivers for software maintenance

25. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 25 References Boehm B.W. (1981), “Software Engineering Economics”, Prentice-Hall, Englewood Cliffs, NJ, 1981. Erlikh L. (2000). “Leveraging legacy system dollars for E-business”. (IEEE) IT Pro, May/June, 17-23. McKee J. (1984). “Maintenance as a function of design”. Proceedings of the AFIPS National Computer Conference, 187-193. Moad J. (1990). “Maintaining the competitive edge”. Datamation 61-62, 64, 66. Nguyen V., Deeds-Rubin S., Tan T., Boehm B.W. (2007), “A SLOC Counting Standard,” The 22nd International Annual Forum on COCOMO and Systems/Software Cost Modeling. Nguyen V., Steece B., Boehm B.W. (2008), “A constrained regression technique for COCOMO calibration”, Proceedings of the 2nd ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM), pp. 213-222 Rosencrance L. (2007), "Survey: Poor communication causes most IT project failures," Computerworld Zelkowitz M.V., Shaw A.C., Gannon J.D. (1979). “Principles of Software Engineering and Design”. Prentice-Hall

26. University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 26 Backup Slides