1. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 1 Assessing and Estimating Corrective, Enhancive, and Reductive Maintenance Tasks: A Controlled Experiment * Presented by Vu Nguyen on behalf of Vu Nguyen, Barry Boehm, Phongphan Danphitsanuphan (*) paper accepted for Asia-Pacific Software Engineering Conference 2009 (APSEC 2009)

2. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 2 Outline Motivation and Background Experiment Design Results and Explanatory Models Conclusions

3. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 3 Motivation and Background

4. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 4 Maintenance is crucial in software engineering Systems are tightly coupled with their environment –Environment changes require changing its software systems Technologies and requirements are continuously changing –Software systems are outdated quickly –Software systems must be updated and upgraded to maintain their values Maintenance is important in market competition –New software has more advantages than the existing one –Software system must be upgraded to keep its market share

5. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 5 Majority of software costs incur after the first operational release (Boehm ’81) Maintenance cost is usually 2x to 100x as much as new development cost (Sommerville, 2006) K$ Time 100200300 System 1 System 2 Release 1Release 2Release 3 Release 1Release 2 Adapted from (Summerville, 2006)New Development Maintenance Release N

6. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 6 Software estimation community has paid little attention to software maintenance Most estimation models regard maintenance estimation as secondary –COCOMO, SEER-SEM, SLIM, PRICE-S models were built using mainly data of new development projects Use of SLOC metrics for models is inconsistent –Some models use SLOC added, modified, deleted –Others use only SLOC added and modified Impact of different SLOC metrics on productivity has not been investigated Projects tend to use experience or expert judgment methods to estimate software effort and cost instead

7. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 7 Research Questions and Hypotheses RQ1: Are there any differences in the productivity of enhancive, corrective, and deductive maintenance? –Hypothesis 1 (H1): no difference RQ2: Are there any differences in the effort distribution among the maintenance types? –Hypothesis 2 (H2): no difference Explanatory models to estimate participant’s maintenance effort

8. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 8 Software Maintenance –“Modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment” [IEEE ‘98] Types of Maintenance –Swanson ’76: Adaptive, Corrective, Perfective –IEEE ’98: all Swanson’s plus Preventive –Chapin et al, 2001: 12 types, including three business rules types: Enhancive Corrective Reductive

9. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 9 Experimental Design

10. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 10 Experiment Description Task Group# Tasks# Participants Enhancive57 Corrective69 Reductive68 23 masters’ students and 1 senior, computer science major Participants worked on tasks individually in the lab –Enhancive: add new capabilities –Corrective: fix the existing capabilities –Reductive: remove the existing capabilities UCC as a target program –5K+ source statements (logical SLOC) in 20 C++ classes MS Visual Studio 2005 was used for maintenance

11. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 11 Calculating Maintenance SLOC Equivalent SLOC = TRCF x AAM TRCF = the total SLOC of task-relevant code fragments S = the size in SLOC (added, modified, or deleted) SU = the software understandability UNFM = the level of programmer unfamiliarity with the program Task-relevant code fragment (TRCF)

12. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 12 Results and Models

13. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 13 Resulted Data 24 students participated Each task requires four activities –Task comprehension –Code isolation –Editing code –Unit test Timesheet has 490 activity records, totaling 77.02 hours Total of 909 SLOC added, modified, and deleted 402 added 216 modified 291 deleted

14. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 14 Effort distribution is different among the groups Corrective group spent much time for code isolation –twice as much as that of the enhancive group Enhancive group spent majority of time for editing code Effort distribution is statistically different among three groups (p-value = 0.0013) –H2 is rejected Kruskal-Wallis rank-sum test

15. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 15 Productivity is significantly different among the groups Corrective group has lowest productivity Reductive group has highest productivity Productivity between groups are statistically different (p-value = 0.0004) –H1 is rejected Enhancive Reductive Corrective Productivity (SLOC/Hour) 40 30 20 10

16. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 16 Participant Time Explanatory Models Two models for participant effort M1 Effort = 78.1 + 2.2 * S * EAF M2 Effort = 43.9 + (2.8*Add + 5.3*Mod + 1.3*Del) * EAF Effort = Time spent by the participant on all maintenance tasks EAF = Effort adjustment factor, a product of Programmer capability (PCAP), Language experience (LTEX), and Platform Experience (PLEX) Add, Mod, Del = Equivalent SLOC added, modified, and deleted by the participant, respectively S = Add + Mod + Del All the estimates of coefficients are significant (p-value < 0.05) R 2 = 0.5 R 2 = 0.75

17. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 17 Model M 2 outperforms M 1 M1M1 M2M2 MMRE33%20% PRED(.25)46%71% PRED(.3)58%79%

18. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 18 Threats to Validity Internal design –Groups have imbalanced skills and capability –Imbalanced complexity of tasks on three groups –Incorrect time recorded Generalizability –Professional programmers are more experienced –Professional programmers are more familiar with the software maintained –Maintenance process is different in industry

19. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 19 Conclusions Productivity and effort distribution are significantly different among the maintenance types SLOC metrics are relevant factors for estimating effort Three SLOC metrics (added, modified, and deleted) have different impact on effort –SLOC deleted is an important factor for estimating effort –It is more expensive to modify than to add or delete a statement Assigning experienced programmers to fixing defect can save up to 40% of effort

20. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 20 References Barry W. Boehm, “Software Engineering Economics”, Prentice Hall, 1991 Ian Sommerville, “Software Engineering,” 8 th Ed., Addison- Wesley, 2006 Ned Chapin, et al., “Types of software evolution and software maintenance,” Journal of Software Maintenance: Research and Practice, v.13 n.1, p.3-30, Jan. 2001 IEEE, IEEE Standard Glossary of Software Engineering Terminology. Institute of Electrical and Electronics Engineers: New York NY, 1990: 83 pp IEEE Std 1219-1998, IEEE Standard for Software Maintenance, IEEE Computer Society, 1998