1. University of Southern California Center for Systems and Software Engineering COSATMO, COCOMO III, and COSYSMO: Developing Next-Generation Cost Models Jim Alstad, PhD Candidate USC Center for Systems and Software Engineering CS 510 Lecture September 17, 2014 09/16 1

2. University of Southern California Center for Systems and Software Engineering Next-Gen Cost Models 09/162 Learning objectives for CS 510 lecture: –To provide a quick view of how an estimating model is developed –To describe some of the current research in cost model development

3. University of Southern California Center for Systems and Software Engineering Next-Gen Cost Models Agenda 09/163 Agenda: Learning goals for 510 lecture A brief history of the COCOMO family of estimating models COCOMO III COSYSMO 3.0 COSATMO

4. University of Southern California Center for Systems and Software Engineering COCOMO II and Some Derivatives 1995: one-size-fits-all model for 21 st century software 1999: poor fit for schedule-optimized projects; CORADMO 2000: poor fit for COTS-intensive projects: COCOTS 2003: need model for product line investment: COPLIMO 2003: poor fit for agile projects: Agile COCOMO II (partial) 2012: poor fit for incremental development: COINCOMO 09/164

5. University of Southern California Center for Systems and Software Engineering COQUALMO 1998 COCOMO 81 1981 COPROMO 1998 COSoSIMO 2007 Legend: Model has been calibrated with historical project data and expert (Delphi) data Model is derived from COCOMO II Model has been calibrated with expert (Delphi) data COCOTS 2000 COSYSMO 2005 CORADMO 1999,2012 iDAVE 2004 COPLIMO 2003 COPSEMO 1998 COCOMO II 2000 DBA COCOMO 2004 COINCOMO 2004,2012 COSECMO 2004 Software Cost Models Software Extensions Other Independent Estimation Models Dates indicate the time that the first paper was published for the model COTIPMO 2011 AGILE C II 2003 COCOMO Family of Cost Models 09/165

6. University of Southern California Center for Systems and Software Engineering Determine Model Needs Step 1 USC-CSSE Modeling Methodology Analyze existing literature Step 2 Perform Behavioral analyses Step 3 Define relative significance,data, ratings Step 4 Perform expert- judgment Delphi assessment, formulate a priori model Step 5 Gather project data Step 6 Determine Bayesian A- Posteriori model Step 7 Gather more data; refine model Step 8 - concurrency and feedback implied 09/166

7. University of Southern California Center for Systems and Software Engineering Next-Gen Cost Models Agenda 09/167 Agenda: Learning goals for 510 lecture A brief history of the COCOMO family of estimating models COCOMO III COSYSMO 3.0 COSATMO Key slides provided by Barry Boehm, Vu Nguyen, and Brad Clark (CSSE 2014 Annual Research Review)

8. University of Southern California Center for Systems and Software Engineering COCOMO II Data by 5-Year Periods 09/168 COCOMO II Calibration Add in new data when calibrating

9. University of Southern California Center for Systems and Software Engineering COCOMO II Data: Productivity Trends 09/169 COCOMO II Calibration Calibrate to new productivity values

10. University of Southern California Center for Systems and Software Engineering Application Experience (APEX) Rating Trends 09/1610 COCOMO II Calibration Very High is >= 6 years Reduce VH (etc) definition to cover same % of projects

11. University of Southern California Center for Systems and Software Engineering Super-Domains and AFCAA Productivity Types 09/1611 Super DomainProductivity Types Real-Time (RT) 1 Sensor Control and Signal Processing 2 Vehicle Control 3 Vehicle Payload 4 Real Time Embedded-Other Engineering (ENG) 5 Mission Processing 6 Executive 7 Automation and Process Control 8 Scientific Systems 9 Telecommunications Mission Support (MS) 10 Planning Systems 11 Training 12 Software Tools 13 Test Software Automated Information System (AIS) 14 Intelligence and Information Systems Software Services Software Applications

12. University of Southern California Center for Systems and Software Engineering More on Application Domains COCOMO III plans to identify a set of application domains, and identify and calibrate a submodel to each domain Allows user to identify domain, and get a group of accordingly-set cost driver values off the shelf 09/1612

13. University of Southern California Center for Systems and Software Engineering COCOMO III Directions CSSE’s April Annual Research Review discussions set these directions for COCOMO development: –149 new data points have been added since the 161 data points used in COCOMO II.2000. Add these data points –The average level of productivity appears to have increased Therefore recalibrate the model’s parameters –The average level of several cost drivers appears to have changed Consider changing the definitions of the high/medium/low levels for these cost drivers –More data points might add to variability in the data Consider adding more cost drivers to improve the fit –Simplifying the model would make it easier to use Define (calibrated) submodels for certain important application domains that provide off-the-shelf values for some cost drivers Planned domains: Real-time, Engineering & Scientific, Command and Control, Automated Information Processing 09/1613

14. University of Southern California Center for Systems and Software Engineering Next-Gen Cost Models Agenda 09/1614 Agenda: Learning goals for 510 lecture A brief history of the COCOMO family of estimating models COCOMO III COSYSMO 3.0 COSATMO

15. University of Southern California Center for Systems and Software Engineering What Is Systems Engineering? 09/1615 Some clauses of a definition: –Considers the system as a whole –Starts from stakeholder needs, results in a technical approach meeting those needs Frequently develops system requirements from system needs Technical approach is commonly stated as the system architecture –Systems engineers typically need to have familiarity with all the specific kinds of engineering involved in their type of system –Systems engineers can usually be found working on systems that are large, critical, and/or important –Participation in identifying all major risks and reducing/managing them (ICSM) –Typically most system engineering is done in the Valuation phase, with major work also being done in the Exploration and Foundation phases (ICSM) –See also ICSM book section 0.2.1 The cost of system engineering needs an estimation model

16. University of Southern California Center for Systems and Software Engineering History of COSYSMO Models 09/1616 COSYSMO 1.0 Valerdi, 2005 Identifies form of model Identifies basic cost drivers Identifies Size measure Req’ts Volatile Pena, 2012 Adds scale factor based on requirements volatility With Reuse Fortune, 2009 Adds weights to Size elements, reducing net Size in the presence of reuse For Reuse Wang et al, 2014 Adds weights to Size elements, reducing net Size when artifacts are only partially completed Sys of Sys Lane et al, 2014 Adds effort multiplier when in the presence of system-of- systems COSYSMO 3.0 Alstad, 2015? Combines features of previous models

17. University of Southern California Center for Systems and Software Engineering Basic COSYSMO (1/2) COSYSMO [2] starts by computing the “size” of a system engineering project, in units of eReq (“equivalent nominal requirements”) These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios. Each artifact is evaluated as being easy, nominal, or difficult. Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size: 09/1617 Artifact TypeEasyNominalDifficult System Req’ts0.5 1.0 5.0 System Interfaces1.1 2.8 6.3 System Algs2.2 4.111.5 Op Scenarios6.214.430.0

18. University of Southern California Center for Systems and Software Engineering Basic COSYSMO (2/2) Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant. This results in the following equation for a COSYSMO estimate of effort in person-months: 09/1618

19. University of Southern California Center for Systems and Software Engineering What Is Systems Engineering for Reuse? Systems Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states: –Conceptualized for Reuse (e.g., Concept of Operations document) –Designed for Reuse (e.g., component detailed design) –Constructed for Reuse (e.g., integrated component) –Validated for Reuse (e.g., validated component) 09/1619

20. University of Southern California Center for Systems and Software Engineering COSYSMO For Reuse A SEFR estimate adjusts each artifact’s size contribution by considering its SEFR state according to this table: 09/1620 SEFR State (Degree of Development)SEFR State Factor Conceptualized for Reuse36.98% Designed for Reuse58.02% Constructed for Reuse79.15% Validated for Reuse94.74%

21. University of Southern California Center for Systems and Software Engineering What Is Systems Engineering with Reuse? Systems Engineering with Reuse is project development, with reusable artifacts being brought into the product –A special case: zero reusable artifacts Each reusable artifact is included in one of these SEWR states of maturity: –New (i.e., not reused) –Re-implemented (through requirements & architecture) –Adapted (through detailed design) –Adopted (through implementation) –Managed (through system verification & validation) 09/1621

22. University of Southern California Center for Systems and Software Engineering COSYSMO with Reuse A SEWR estimate adjusts each artifact’s size contribution by considering its SEWR state according to this table: 09/1622 SEWR State (Maturity)SEWR State Factor New100.00% Re-Implemented 66.73% Adapted 56.27% Adopted 38.80% Managed 21.70%

23. University of Southern California Center for Systems and Software Engineering Next-Gen Cost Models Agenda 09/1623 Agenda: Learning goals for 510 lecture A brief history of the COCOMO family of estimating models COCOMO III COSYSMO 3.0 COSATMO

24. University of Southern California Center for Systems and Software Engineering The Problem 09/1624 How much will the total system cost? Is one phase being optimized while increasing total cost? Is the system affordable? Does the acquisition comply with the Better Buying Power intiatives (DoD)?

25. University of Southern California Center for Systems and Software Engineering The Solution 09/1625 Example acquisition process (DoDI 5000.02) COSATMO assists acquirers and developers during these phases (highest payoff during early phases) COSATMO estimates the cost for these phases

26. University of Southern California Center for Systems and Software Engineering COSATMO Objective 09/1626 Context: –Current and future trends create challenges for full-system cost estimation Emergent requirements, rapid change, net-centric systems of systems, COTS, clouds, apps, widgets, high assurance with agility, multi-mission systems –Current development practices can minimize cost of one phase, such as development, while raising full-system cost The COSATMO project is developing a modern full- system cost model (first space systems, then other DoD domains) –“Constructive SATellite cost MOdel” –Current estimating models focus on one aspect, such as system engineering –COSATMO will enable: System-level trades to be handled within a single model Easy customer evaluation of full-system cost Modern technologies to be covered

27. University of Southern California Center for Systems and Software Engineering Segments of Satellite System Cost Total satellite system cost [tied to slide 25 phases] = System engineering cost [EMD] +Satellite software cost [EMD] +Satellite vehicle hardware development [EMD] and production [Prod] cost +Launch cost [Deploy] +Initial ground software cost [EMD] +Initial ground custom equipment cost [EMD] +Initial ground facility (buildings, communications, computers, COTS software) cost [EMD] +Operation & support cost [Deploy, O&S] Updated at GSAW (Feb 2014) Model as sum of submodels is new structure in COCOMO family 09/1627

28. University of Southern California Center for Systems and Software Engineering COSATMO Segment Tentative Models System engineering: COSYSMO, perhaps with add-ons Satellite vehicle hardware development and production: Current Aerospace hardware cost model(s); exploring extensions of COSYSMO for hardware cost estimation Satellite vehicle, ground system software development: COCOMO II, COCOTS, perhaps with add-ons Launch model: similarity model, based on vehicle mass, size, orbit Ground system equipment, supplies: construction, unit-cost, services cost models Operation & support: labor-grade-based cost models, software maintenance models 09/1628

29. University of Southern California Center for Systems and Software Engineering Key Overall Satellite System Cost Drivers Most Important: –Complexity, Architecture Understanding, Mass, Payload TRL level/Technology Risk, and Requirements Understanding. Important: –Reliability, Pointing Accuracy, Number of Deployables, Number of Key Sponsors, Data Rate, and Security Requirements for Communications. Determined at COCOMO Forum (Oct 2013) 09/1629

30. University of Southern California Center for Systems and Software Engineering Ground System Segment Development (1/2) Determined at GSAW (Feb 2014) Ground system-wide cost drivers –Most important: Accreditation (information assurance, etc), Required security –Also important: # satellites* Initial software cost drivers –Required data throughput –Generally handled by COCOMO II, COCOTS, COPLIMO 09/1630 *Indicates a size measure

31. University of Southern California Center for Systems and Software Engineering Ground System Segment Development (2/2) Ground custom equipment cost drivers –Most important: Amount of new development required, # of custom equipment sites*, Required site availability & reliability, Required site security –Also important: # driving requirements* Ground facility cost drivers –Most important: # facilities*, location of facilities (especially US vs foreign), # ground RF terminals* –Also important: Facility “reuse” Operation and support cost drivers –Most important: # years of operation*, # FTE staff (with labor mix)* –Also important: Size of software maintained*, Leased line cost*, level of automation 09/1631 *Indicates a size measure

32. University of Southern California Center for Systems and Software Engineering COSATMO as a Research Umbrella General direction: –Develop a full-coverage satellite system cost estimating model –Generalize that to additional applications 09/1632 Specific current research initiatives: –COSYSMO 3.0 –COCOMO III Research vehicles: –My thesis –Other theses –Other research

33. University of Southern California Center for Systems and Software Engineering Bibliography 1.“A Generalized Systems Engineering Reuse Framework and its Cost Estimating Relationship”, Gan Wang, Garry J Roedler, Mauricio Pena, and Ricardo Valerdi, submitted for publication. 2.“The Constructive Systems Engineering Cost Model (COSYSMO)”, Ricardo Valerdi (PhD Dissertation), 2005. 3.“Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0)”, Jared Fortune (PhD Dissertation), 2009. 4.“Quantifying the Impact of Requirements Volatility on Systems Engineering Effort”, Mauricio Pena (PhD Dissertation), 2012. 5.“Life Cycle Cost Modeling and Risk Assessment for 21st Century Enterprises”, Barry Boehm, Jo Ann Lane, Supannika Koolmanojwong, Richard Turner (presentation), April 29, 2014. 6."System Interoperability Influence on System of Systems Engineering Effort", Jo Ann Lane, Ricardo Valerdi, unpublished. 7.“COSYSMO Extension as a Proxy Systems Cost Estimation” (presentation), Reggie Cole, Garry Roedler, October 23, 2013. 09/1633