COCOMO III Brad Clark 31tst International Forum on COCOMO and System/Software Cost Modeling October 17, 2016

Topics COCOMO III Model Overview Discuss COCOMO III Cost Drivers 10/18/2016 Copyright © USC-CSSE

The COCOMO III Project COCOMO® (COnstructure COst MOdel) is the most widely used, free, open source software cost estimation model in the world. Registered Trademark for intellectual property protection COCOMO 81 and COCOMO II models are open and free for anyone to use Models have been commercialized It has been 16 years since the COCOMO II model has been updated and calibrated to new Software Engineering practices. 10/18/2016 Copyright © USC-CSSE

Commercialization – USC COCOMO vs. SystemStar 10/18/2016 Copyright © USC-CSSE

COCOMO III Project Purpose Broaden audiences of COCOMO® and address scope of modern projects: mobile devices, web/internet, big data, cloud-targeted, and multi-tenant software Modernize model size inputs Consider the impact of modern development processes (e.g. Agile) Improve the accuracy and realism of estimates Improve driver definitions New and updated software cost drivers and adjust their ratings as needed Quality estimation capability Point and range estimates based on risk Improve value of COCOMO® in decision-making 10/18/2016 Copyright © USC-CSSE

COCOMO III Project Objectives The project will: Broaden the audiences using COCOMO® Consider domains that use different size drivers (e.g. Function Points, Requirements, Use Cases, Story Points) Consider the impact of modern development processes (e.g. Agile) Address the scope of modern software projects: mobile devices, web/internet, big data, and cloud-targeted Improve the accuracy and realism of estimates Improve driver definitions New and updated software cost drivers and adjust their ratings as needed Quality estimation capability Point and range estimates based on risk 10/18/2016 Copyright © USC-CSSE

COCOMO III Project Objectives The project will: Estimate software cost that is complementary with a System Engineering cost estimate, COSYSMO Improve the value of COCOMO® in decision making Create a strategy for maintaining past COCOMO models The audience is invited to participate in the development of COCOMO III. 10/18/2016 Copyright © USC-CSSE

COCOMO III Project Scope COCOMO® III will product estimates for: Effort, Schedule, Cost, Quality Level (Defects) COCOMO® III can be applied at various moments in a project’s lifecycle: Early Estimation, Initial Project Estimation, Project Re-estimation COCOMO® III’s functional vision Single and Multiple component estimate Analysis of alternatives Analysis with Size-Effort-Schedule as independent variables Support for different lifecycle processes Lifecycle cost estimation Legacy system transformation 10/18/2016 Copyright © USC-CSSE

Historical Overview of COCOMO® Suite of Models COQUALMO 1998 COPROMO COSYSMO-SoS 2007 Legend: Model has been calibrated with historical project data and expert (Delphi) data Model is derived from COCOMO II Model has been calibrated with expert (Delphi) data COCOTS 2000 COSYSMO 2005 CORADMO 1999,2012 iDAVE 2004 COPLIMO 2003 COPSEMO COCOMO II DBA COCOMO COINCOMO 2004,2012 COSECMO 2008 Software Cost Models Software Extensions Dates indicate the time that the first paper was published for the model COTIPMO 2011 AGILE C II COCOMO 81 1981 10/18/2016 Copyright © USC-CSSE

COCOMO is an open and free model Software development and maintenance estimates for: Effort Cost & Schedule distributed by: Phase Activity Increment Quality Software product size estimate COCOMO III Model Software product, platform, personnel & project attributes Software reuse, maintenance, and increment parameters Defect removal profile levels Local calibration to organization’s data COCOMO is an open and free model 10/18/2016 Copyright © USC-CSSE

COCOMO III Model Concept Schedule Model Schedule (Months) Software product size estimate Staffing Levels Software product, platform, personal & project attributes Effort Model Effort (Person Months) Costs ($$) Labor Rates Number of est. non-trivial defects for Requirements, Design, & Code Defect Introduction Model Number of est. residual defects and the residual defect density Defect removal profile levels Defect Removal Model 10/18/2016 Copyright © USC-CSSE

COCOMO III Size Inputs Intent is to produce an estimation model that takes different software size inputs directly Current software size other than source lines of code (SLOC) is first converted to SLOC and use as “equivalent” size in the model Dependent on the data collected for calibration Software Requirements Function Point SNAP Points Fast Function Points COSMIC Points Automated Function Points Feature Points Use Case Points Story Points (Agile Development) 10/18/2016 Copyright © USC-CSSE

COCOMO III Cost Drivers -1 Product Attributes Impact of Software Failure (FAIL) (formerly RELY) Product Complexity (CPLX) Developed for Reusability (RUSE) Required Software Security (SECU) - New Dropped: Documentation Match to Lifecycle Needs Database Size Platform Attributes Platform Constraints (PLAT) – New Platform Volatility (PVOL) 10/18/2016 Copyright © USC-CSSE

COCOMO III Cost Drivers -2 Personnel Attributes Analyst Capability (ACAP) Programmer Capability (PCAP) Personnel Continuity (PCON) Applications Experience (APEX) Language and Tool Experience (LTEX) Platform Experience (PLEX) 10/18/2016 Copyright © USC-CSSE

COCOMO III Cost Drivers -3 Project Attributes Precedentedness (PREC) Development Flexibility (FLEX) Opportunity and Risk Resolution (RESL) Stakeholder Team Cohesion (TEAM) Process Capability & Usage (PCUS) (formerly PMAT) Use of Software Tools (TOOL) Multisite Development (SITE) Defect Removal Profile Automated Analysis People Reviews Execution Testing and Tools 10/18/2016 Copyright © USC-CSSE

COCOMO II Model Effort Distribution Product Design Detailed Design Code & Unit Test Integration & Test Requirements 45 40 Small project with low exponent 35 30 25 Percentage of Effort (PM) 20 15 Large project with high exponent 10 5 10/18/2016 Copyright © USC-CSSE

Incremental Development 10/18/2016 Copyright © USC-CSSE

COCOMO III Quality Model Predicts the number of residual defects in a software product Enables 'what-if' analyses that demonstrate the impact of various defect removal techniques effects of personnel, project, product and platform characteristics on software quality. Provides insights into Probable ship time Assessment of payoffs for quality investments Understanding of interactions amongst quality strategies 8/23/2016 Copyright © USC-CSSE

Defect Introduction Model Removal Defect removal profiles: Automation Reviews Testing Quality Model Defect Introduction Model Removal Number of est. non-trivial defects for Requirements, Design, & Code COCOMO II Cost Drivers (Size, Product, Platform, Personnel, & Project attributes) Number of est. residual defects and the residual defect density Defect removal profile levels 10/18/2016 Copyright © USC-CSSE

Defect Estimation Scope Estimates only “Nontrivial” defects Critical: causes a system to crash or unrecoverable data loss or jeopardizes personnel High: causes impairment of critical system function and no workaround solution exists Medium: causes impairment of critical system function, through a workaround solution does exist Defect estimates are for only 3 artifacts: Requirements Design Code 8/23/2016 Copyright © USC-CSSE

COCOMO® Model Websites cocomomodels.com cocomomodels.info cocomofamily.com cocomofamily.info cosysmo.com (future) cocomo81.com cocomo81.info cocomo2.com cocomo2.info cocomo3.com cocomo3.info 10/18/2016 Copyright © USC-CSSE

Invitation to Participate CSSE invites you to collaborate on model development Review model formulation Submit data for model calibration Actual Size Effort Schedule Defects Model Parameters Review of COCOMO III model If you contribute data for model calibration, you will receive: An advanced copy of the new model Comparison of your data with respect to other data points used to calibrate the model Please talk with me afterwards if you are interested Want to Participate? www.cocomo3.com JASanche@usc.edu Make suggestions and be a model reviewer! 10/18/2016 Copyright © USC-CSSE

Topics COCOMO III Model Overview Discuss COCOMO III Cost Drivers Cost driver definition refinement The New “Nominal” 10/18/2016 Copyright © USC-CSSE

The New “Nominal” Cost Driver definition refinement The New “Nominal Study of productivity trends over the past 40 years reveals a shift in “nominal” ratings Following slides show the shift in ratings for selected cost drivers For each driver, we will discuss the definition for a new “nominal” 10/18/2016 Copyright © USC-CSSE

Required Software Security (SECU)* Is this correlated to Process Maturity? Should it be an extension to FAIL Should it address the threat? Malicious hackers Foreign governments Should it address exposure On the network Under guard, separated from outside environment Where are the descriptors: Confidentiality Integrity Availability 10/18/2016 Copyright © USC-CSSE

Platform Definition “Platform” is used here to mean the complex of hardware and software (OS, DBMS, etc.) the software product calls on to perform its tasks, the target platform. If the software to be developed is an operating system then the platform is the computer hardware. If a database management system is to be developed then the platform is the hardware and the operating system. If a network web browser is to be developed then the platform is the network, computer hardware, the operating system, and the distributed information repositories. A target platform can include graphic user interfaces, databases, networks, distributed middleware, mobile computing, cloud computing, high security architectures, fault-tolerant computing, and software as a service (or “x” as a service). The “platform” is what the application is being developed to; the target platform. 10/18/2016 Copyright © USC-CSSE

Platform Constraints Execution Time Constraint Characteristic Should this be asking “How close is the application to the specification constraint” Relative term versus an absolute term, 85% of available execution time Same applies to Primary/Secondary storage constraint 10/18/2016 Copyright © USC-CSSE

Platform Volatility (PVOL) Concurrent development for the target platform and software application Change scales: Low: “Platform is stable” Extra High: Platform is “pic-in-the-sky”; haven't defined capability How does this map to different situations, e.g. system of systems Consider using Technology Readiness Levels (next slide) 10/18/2016 Copyright © USC-CSSE

Technology Readiness Levels Basic principles observed and reported Technology concept and/or application formulated Analytical and experimental critical function and/or characteristic proof of concept Component and/or breadboard validation in laboratory environment Component and/or breadboard validation in relevant environment System/subsystem model or prototype demonstration in a relevant environment System prototype demonstration in an operational environment. Actual system completed and qualified through test and demonstration. Actual system proven through successful mission operations. 10/18/2016 Copyright © USC-CSSE

Tester Capability There is Analyst Capability and Programmer Capability cost driver Should there be a Test Capability cost driver based on: Test design Test plan development Capability in test execution Documentation 10/18/2016 Copyright © USC-CSSE

Topics COCOMO III Model Overview Discuss COCOMO III Cost Drivers Cost driver definition refinement The New “Nominal” 10/18/2016 Copyright © USC-CSSE

Impact of Productivity Trends Kendall's Rank Correlation Coefficients between the Completion Year and COCOMO II Cost Drivers (sorted by degrees of correlation) Cost driver Kendall’s τ p-value TOOL Use of Software Tools -0.37 2.20E-16 PMAT Process Maturity (PCUS) -0.30 1.22E-13 STOR Main Storage Constraint -0.29 1.31E-11 TIME Execution Time Constraint -0.26 6.62E-10 PLEX Platform Experience -0.17 1.98E-05 PVOL Platform Volatility -0.18 2.04E-05 APEX Applications Experience 0.17 4.88E-05 LTEX Language and Tool Experience -0.15 2.84E-04 DATA Database Size 0.13 1.81E-03 RELY Required Software Reliability -0.10 1.42E-02 CPLX Product Complexity 1.58E-02 PREC Precedentedness of Application -0.09 2.13E-02 ACAP Analyst Capability 0.08 4.87E-02 10/18/2016 Copyright © USC-CSSE

Use of Software Tools-Tool 10/18/2016 Copyright © USC-CSSE

Process Maturity (Now PCUS) 10/18/2016 Copyright © USC-CSSE

Platform Experience-PLEX 10/18/2016 Copyright © USC-CSSE

Applications Experience-APEX 10/18/2016 Copyright © USC-CSSE

Language and Tool Experience-LTEX 10/18/2016 Copyright © USC-CSSE

Main Storage Constraint-STOR 10/18/2016 Copyright © USC-CSSE

Execution Time Constraint-TIME 10/18/2016 Copyright © USC-CSSE

Backup Charts 10/18/2016 Copyright © USC-CSSE

The Incremental Commitment Spiral Process: Phased View Anchor Point Milestones Synchronize, stabilize concurrency via FEDs The Incremental Commitment Life Cycle Process: Overview This slide shows how the ICSM spans the life cycle process from concept exploration to operations. Each phase culminates with an anchor point milestone review. At each anchor point, there are 4 options, based on the assessed risk of the proposed system. Some options involve go-backs. These options result in many possible process paths. The life cycle is divided into two stages: Stage I of the ICSM (Definition) has 3 decision nodes with 4 options/node, culminating with incremental development in Stage II (Development and Operations). Stage II has an additional 2 decision nodes, again with 4 options/node. One can use ICSM risk patterns to generate frequently-used processes with confidence that they fit the situation. Initial risk patterns can generally be determined in the Exploration phase. One then proceeds with development as a proposed plan with risk-based evidence at the VCR milestone, adjusting in later phases as necessary. For complex systems, a result of the Exploration phase would be the Prototyping and Competition Plan discussed above. Risks associated with the system drive the life cycle process. Information about the risk(s) (feasibility assessments) supports the decision to proceed, adjust scope or priorities, or cancel the program. Risk patterns determine life cycle process 10/18/2016 Copyright © USC-CSSE 41

The Incremental Commitment Spiral Model 10/18/2016 Copyright © USC-CSSE

The Basic COPLIMO Model - Constructive Product Line Investment Model Based on COCOMO II software cost model Statistically calibrated to 161 projects, representing 18 diverse organizations Based on standard software reuse economic terms RCR: Relative cost of reuse RCWR: Relative cost of writing for reuse Avoids overestimation Avoids RCWR for non-reused components Provides experience-based default parameter values Simple Excel spreadsheet model Easy to modify, extend, interoperate 10/18/2016 Copyright © USC-CSSE

10/18/2016 Copyright © USC-CSSE

Legacy Systems Patched, Highly Coupled Financial and Non-Financial Services Legacy Business Services Contract Services Project Services Budgeting Staffing Deliverables Management Billing Earned Value Management Subcontracting Change Tracking Progress Tracking Work Breakdown Structure Scheduling Reqs, Configuration Management 10/18/2016 Copyright © USC-CSSE

Result of Legacy Re-engineering Legacy Business Services Contract Services Project Services Contract Financial Services Billing Subcontract payments ... Contract Non-Financial Services Deliverables mgmt. Terms compliance ... General Financial Services Accounting Budgeting Earned value Payroll ... General Non-Financial Services Progress tracking Change tracking ... Project Financial Services WBS Expenditure categories ... Project Non-Financial Services Scheduling Staffing Reqs CM ... 10/18/2016 Copyright © USC-CSSE

USC-CSSE Modeling Methodology - concurrency and feedback implied Determine Model Needs Step 1 Analyze existing literature Step 2 Perform Behavioral analyses Step 3 Define relative significance,data, ratings Step 4 Perform expert-judgment Delphi assessment, formulate a priori model Step 5 Gather project data Step 6 Determine Bayesian A-Posteriori model Step 7 Gather more data; refine model Step 8 10/18/2016 Copyright © USC-CSSE

Step 6: Gather, Analyze Project Data Best to pilot data collection with early adopters Identifies data definition ambiguities Identifies data availability problems Identifies need for data conditioning Best to collect initial data via interviews Avoids misinterpretations Endpoint milestones; activities included/excluded; size definitions Uncovers hidden assumptions Schedule vs. cost minimization; overtime effort reported 10/18/2016 Copyright © USC-CSSE

Initial Data Analysis May Require Model Revision Initial COCOTS model adapted from COCOMO II, with different parameters Effort = A* (Size)B* Õ(Effort Multipliers) Amount of COTS integration glue code used for Size Data analysis showed some projects with no glue code, much effort Effort devoted to COTS assessment, tailoring 10/18/2016 Copyright © USC-CSSE

COCOTS Effort Distribution: 20 Projects Mean % of Total COTS Effort by Activity ( +/- 1 SD ) 70.00% 60.00% 61.25% 50.00% 50.99% 49.07% 40.00% 30.00% 31.06% % Person-months 20.00% 20.75% 21.76% 20.27% 10.00% 11.31% 0.00% 0.88% 2.35% -7.57% -7.48% -10.00% assessment tailoring glue code system volatility -20.00% 10/18/2016 Copyright © USC-CSSE