University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE1 Predicting Understandability of a Software Project Using COCOMO II Model Drivers Ali Afzal Malik Barry Boehm A. Winsor Brown {alimalik, boehm, 23 rd International Forum on COCOMO and Systems/Software Cost Modeling

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE2 Outline Introduction Motivation & Related Work Methodology Results Future Work References

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE3 Introduction Understandability –“Degree of clarity of the purpose and requirements of a software system to the developers of that system at the end of the Inception phase” Basic idea –Quantification enables prediction –Reuse inputs of software cost estimation Empirical Study –Projects

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE4 RUP Hump Chart (Kruchten, 2003)

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE5 Empirical Study SE I (Fall) and SE II (Spring) 2004 – real-client, MS-student, team projects (SE I 2008, SE II 2008) Process: MBASE/RUP (Boehm et al. 2005, Kruchten 2003) Projects –Development-intensive –Used COCOMO II

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE6 S#YearProjectType 12004Bibliographies on Chinese Religions in Western LanguagesWeb-based database 22004Data Mining of Digital Library Usage DataData mining 32004Data Mining from Report FilesData mining 42005Data Mining PubMed ResultsData mining 52005EBay Notification SystemStand-alone application 62005Rule-based EditorGUI 72005CodeCount™ Product Line with XML and C++Code Counter Tool 82006California Science Center Newsletter SystemWeb-based database 92006California Science Center Event RSVP SystemWeb-based database USC Diploma Order/ Tracking Database SystemWeb-based database USC Civic and Community Relations web applicationWeb-based database Student's academic progress web applicationWeb-based database New Economics for Woman (NEW)Web-based database Web Portal for USC Electronic ResourcesWeb-based GUI Early Medieval East Asian TombsWeb-based database USC CONIPMOCost model An Eclipse Plug-in for Use Case AuthoringStand-alone application USC COINCOMOCost model BTI Appraisal ProjectsStand-alone database LAMAS Customer Service ApplicationWeb-based database BID review SystemStand-alone database Proctor and Test Site Tracking SystemWeb-based database E-Mentoring programWeb-based application Los Angeles County Generation Web InitiativeWeb-based database

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE7 Motivation & Related Work Some important considerations –18% of software project failures due to unclear objectives and incomplete R&S (Standish Group 1995) –Escalation in cost of fixing requirements defects: rapid for large and considerable for smaller projects (Boehm 1981, Boehm and Turner 2004) –Requirement changes have significant impact on project’s budget and schedule (Zowghi and Nurmuliani 2002)

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE8 Motivation & Related Work (2) An objective mechanism to predict understandability enables –Minimization of resource wastage due to rework –Answering “How much RE is enough?” Related previous work –“Expert COCOMO” (Madachy 1997) Uses COCOMO II cost factors to quantify risk

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE9 Methodology Identified 8 relevant COCOMO II model drivers S#Model DriverDescription 1PRECProduct precedentedness 2RESLArchitecture/Risk resolution 3CPLXProduct complexity 4ACAPAnalyst capability 5PCAPProgrammer capability 6APEXApplications experience 7PLEXPlatform experience 8LTEXLanguage and tool experience

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE10 Methodology (2) Weighted-sum formula –UNDR – understandability –MD i – i th Model Driver’s value –w i – weight of MD i –n i – nature of MD i ; Є {-1, +1} -1 for CPLX; +1 for the rest

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE11 Methodology (3) Model driver rating scale Driver RatingRating SymbolNumerical Value Very LowVL1 LowL2 NominalN3 HighH4 Very HighVH5 Extra HighXH6

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE12 Methodology (4) Voting for model driver weights –22 students from SE II class –Rating scale 1 (least important) – 5 (most important) iMD i wiwi 1PREC3.64 2RESL3.23 3CPLX3.64 4ACAP3.55 5PCAP3.95 6APEX3.64 7PLEX3.32 8LTEX3.36

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE13 Methodology (5) Determine the lowest and highest numerical values of understandability

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE14 Methodology (6) min() and max() min (MD i ) { if (n i = = +1) return minimum numerical value of MD i else return maximum numerical value of MD i } max (MD i ) { if (n i = = +1) return maximum numerical value of MD i else return minimum numerical value of MD i }

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE15 Methodology (7) iMD i nini wiwi min(MD i )max(MD i ) 1PREC RESL CPLX ACAP PCAP APEX PLEX LTEX

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE16 Methodology (8) Project categories using UNDR ranges –Vaguely-understood applications (VUA) –Intermediate-clarity applications (ICA) –Well-understood applications (WUA) VUAICAWUA UNDR Low UNDR High

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE17 Methodology (9) Ranges defining VUA, ICA, and WUA groups are adjustable –End-points of spectrum depend on weights –Range sizes can be non-uniform

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE18 Results S#UNDR ValuePredicted UNDR GroupActual UNDR GroupMatches reality? ICA Yes ICA Yes ICA Yes ICA Yes ICAVUANo ICA Yes ICA Yes ICA Yes ICA Yes ICA Yes ICA Yes ICA Yes ICA Yes VUA Yes ICA Yes WUA Yes VUA Yes ICA Yes ICAVUANo ICA Yes ICA Yes ICAWUANo ICA Yes ICAWUANo

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE19 Results (2) Prediction accuracy –ICA: 100% –VUA: 50% (2 out of 4) –WUA: 33% (1 out of 3) –Overall: 83% (20 out of 24) Possible reasons for discrepancies –Inappropriate COCOMO II model drivers –Voters are different from developers Projects # 5 & 22

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE20 Future Work Weights for commercial projects using techniques such as Wideband Delphi (Boehm 1981) Investigate other widely-used models e.g. SLIM (Putnam 1978) and PRICE-S (Freiman and Park 1979) Analyze understandability’s contribution towards degree of requirements elaboration (Malik and Boehm 2008)

University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE21 References Books –Boehm, B., Software Engineering Economics, Prentice-Hall, –Boehm, B., Abts, C., Brown, A., Chulani, S., Clark, B, Horowitz, E., Madachy, R., Reifer, D., and Steece, B., Software Cost Estimation with COCOMO II, Prentice Hall, –Boehm, B. and Turner, R., Balancing Agility and Discipline: A Guide for the Perplexed, Addison- Wesley, –Kruchten, P., The Rational Unified Process: An Introduction, Addison-Wesley, Conference papers –Boehm, B., “Anchoring the Software Process”, IEEE Software 13(4), 1996, pages –Freiman, F.R. and Park, R. E., “PRICE Software Model–Version 3: An Overview”, Proc. IEEE-PINY Workshop on Quantitative Software Models, 1979, pages –Madachy, R., “Heuristic Risk Assessment Using Cost Factors”, IEEE Software 4 (3), 1997, pages –Malik, A. A. and Boehm, B., “An Empirical Study of Requirements Elaboration”, The 22nd Brazilian Symposium on Software Engineering (SBES’08), –Putnam, L. H., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem”, IEEE Trans. Software Engr., 1978, pages 345–361. –Zowghi, D. and Nurmuliani, N., “A Study of the Impact of Requirements Volatility on Software Project Performance”, Proceedings of the Ninth Asia-Pacific Software Engineering Conference, 2002, pages Miscellaneous –Boehm, B., Klappholz, D., Colbert, E., et al., “Guidelines for Lean Model-Based (System) Architecting and Software Engineering (LeanMBASE)”, Center for Software Engineering, University of Southern California, –SE I (2008). Links to websites of all past semesters of Software Engineering I (CSCI 577A) course at USC, –SE II (2008). Links to websites of all past semesters of Software Engineering II (CSCI 577B) course at USC, –Standish Group (1995). “CHAOS”,