11/04/091 Some Topics Concerning The COSYSMOR Model/Tool John E. Gaffney, Jr Center For Process Improvement Excellence Corporate Engineering and Technology Lockheed Martin Corporation COCOMO Forum COSYSMO Workshop November 4, 2009 (c) Copyright Lockheed Martin Corporation 2009

Points Background: COSYSMOR Overview Some new features since original version: –New user-friendly front-end –Improved effort/schedule relationship---covered in Forum presentation on November 3 Relationship to Affordability –Affordability: Use of COSYSMOR estimates at successive points in project –Estimation Performance Indicator: Included in COSYSMOR 11/04/092(c) Copyright Lockheed Martin Corporation 2009

Background: COSYSMOR Overview 11/04/093(c) Copyright Lockheed Martin Corporation 2009

4 COSYSMOR or COSYSMO Risk and Reuse COSYSMOR evolved from the USC Academic COSYSMO systems engineering estimation model/tool (Dr. Ricardo Valerdi) and Lockheed Martin implemented these enhancements in a tool, COSYSMOR, or COSYSMO Risk and Reuse COSYSMOR is available on request from Lockheed Martin and at: 6b_COSYSMOR% a.xls 6b_COSYSMOR% a.xls A major driver for the development of COSYSMOR was to get away from “single point” cost estimates in order to better recognize the uncertainty associated with effort, schedule, and cost estimates 11/04/09

(c) Copyright Lockheed Martin Corporation Additional Functions Provided By COSYSMOR COSYSMOR provides four major additional functions beyond those provided by Academic COSYSMO: 1.Estimation of Cost/Effort and Schedule Uncertainties/Risk and Confidence: provides quantification of the impacts of uncertainties in the values of key model parameter values. Provides multiple cost and schedule values with associated probabilities. Risk=Prob [Actual Effort Will Be >Value] Confidence=100%-Risk 2.Representation of Multiple Types of Size Drivers: Provides for entering counts of: new, modified, reused, and deleted types for each of the four size driver categories. 3.Labor Scheduling: provides the spread of systems engineering labor for the five systems engineering activities and across four the development phases (time). 4.Labor Allocation: provides for the user to select the percentage allocations of the twenty activity/phase pairs or effort elements. 11/04/09

(c) Copyright Lockheed Martin Corporation Concerning The “Risk” Aspect of COSYSMOR COSYSMOR provides the ability to estimate Cost/Effort and Schedule Uncertainties/Risk and Confidence: 1) Provides quantification of the impacts of uncertainties in the values of key model parameter values. 2) Provides multiple cost and schedule values with associated probabilities. Risk=Prob [Actual Effort Will Be >Estimated Effort] Confidence=100%-Risk 11/04/09

Some new features since original version: New User-Friendly Front-end Improved effort/schedule relationship---Covered in Forum presentation on November 3 11/04/09(c) Copyright Lockheed Martin Corporation

New User-Friendly Front-End COSYSMOR has a flowchart that the user can employ to guide data entry The flowchart guides the user through the data entry process for the various selections, e.g., size driver values, cost driver values When the user clicks on a box in the flowchart, a corresponding sheet is opened that provides for the entry of data appropriate to that selection The flowchart/data entry approach is illustrated in the next two pages show the top portion of the flowchart and the data entry sheet corresponding to the first box on the flowchart 11/04/09(c) Copyright Lockheed Martin Corporation

9 999 Top Portion of Flowchart Click on button to bring up data entry form (see next chart) 11/04/09

10(c) Copyright Lockheed Martin Corporation Pressing Enter General Data Button Gets You To This Screen 11/04/09

11 Relationship to Affordability -Affordability: Use of COSYSMOR -Estimation Performance Indicator: Included in COSYSMOR (c) Copyright Lockheed Martin Corporation 2009

Affordability “Affordability” means that a given set of needs (performance requirements) can be met within stated cost and schedule constraints. “Affordability” can also be defined as the probability (confidence) of achieving a stated set of needs at a stated cost and schedule (effort). The associated “risk” is determined (estimated) on the basis of the capability of the organization to meet this set of needs. –“Risk” equals 100% minus “Confidence” 11/04/0912(c) Copyright Lockheed Martin Corporation 2009

1311/04/09(c) Copyright Lockheed Martin Corporation 2009 Example cost confidence plots (cumulative distributions) taken at successive times during a project until TF, the end of the project.

1411/04/09(c) Copyright Lockheed Martin Corporation 2009 Example cost confidence plots (probability densities) taken at successive times during a project until TF, the end of the project.

Estimation Performance Indicator (EPI) The EPI is a statistical measure, the coefficient of variation; it equals the standard deviation divided by the average or expected value. –It indicates the relative “fatness” of the distribution. Used in COSYSMOR as a measure of the goodness or quality of the various risk and confidence distributions, such as for the equivalent requirements or the systems engineering effort. The smaller the value of EPI, the less the relative dispersion. As successive estimates are made, the EPI is computed, successive values determined over the course of a project should be expected to decrease. Finally, when the project is complete, EPI is zero, as there is only one value, the final one; hence, there is no uncertainty as to the cost or any other measure relating to the project for which data is collected. The EPI is seen to be a measure of ignorance. The greater the value of the EPI, the more uncertain one is about what the actual value of the measure to which it refers, e.g., ultimate project cost, is going to be. Note that historical data on the EPI versus time in a project can be used a paradigm of estimation process performance, showing the expected relative reduction in the EPI with time. 11/04/0915(c) Copyright Lockheed Martin Corporation 2009

1611/04/09(c) Copyright Lockheed Martin Corporation 2009 Example of EPI versus time

Summary Lockheed Martin developed COSYSMOR for several reasons, including enabling uncertainties in key estimation parameters to be reflected in risk probability distributions of estimated size and effort and to be able to represent not only new size driver sizes, but also deleted, modified, and reused size values The original version of COSYSMOR, completed in 2007, has been augmented with several capabilities: new user-friendly, flowchart data entry, new effort/schedule relationship/tradeoff, estimation performance indicator. The ability of COSYSMOR to be used in affordability evaluations has been developed 11/04/09(c) Copyright Lockheed Martin Corporation