Harmonizing Systems and Software Estimation 23 rd International Forum on COCOMO and Systems/Software Cost Modeling and ICM Workshop USC Campus, Los Angeles, CA Oct. 27, 2008 Gan Wang, BAE Systems, Reston, VA Garry J. Roedler, Lockheed Martin, Philadelphia, PA Ricardo Valerdi, MIT, Cambridge, MA Aaron Ankrum, BAE Systems, Reston, VA John E. Gaffney, Jr., Lockheed Martin, Rockville, MD Jared Fortune, USC, Los Angeles, CA Don Reifer, Reifer Consultants, Neptune, CA

2 Acknowledgement This presentation uses material from the following: –PSM Workshop on Harmonizing COSYSMO and COCOMO Results Briefing, July 2008 –USC COSYSMO Workshop, March 2008 –BAE Briefing for COSYSMO Workshop, October 2008 –Lockheed Martin Project Briefings

3 Problem & Motivation Estimating tools available today (e.g., COSYSMO, COCOMO, PRICE, SEER) are functionally oriented –We assemble a total engineering bid from functional estimates Understand the issues related to integrating systems and software estimation for software-intensive projects –What are the exact estimate scopes of COSYSMO and COCOMO II? –How do we deal with potential gaps and overlaps?

4 Summary of Work So Far COSYSMO Workshop in conjunction with 2008 USC Annual Research Review –Scoped the problem –Prioritized with other COSYSMO projects – assessed as high priority –Identified that operational guidance may be as important as model constructs COSYSMO Workshop at 2008 PSM Users Group Conference (Mystic, CT) –Assigned WBS elements to functions –Assessed the coverage by COSYSMO and COCOMO –Identified potential gaps and overlaps Workshops within engineering communities at BAE Systems Project in Lockheed Martin to address Integrated Cost Estimation for Systems and Software Engineering Joint paper to INCOSE Symposium

5 Areas for Consideration in Analysis of Harmonization Overlap/Gaps of tasks –Per WBS, work products, and combined activities Analysis of Cost Drivers Commonality of Terminology, Constructs, Life Cycle Phases, Units, … Compatibility issues from any findings or recommendations Consideration of common size drivers Base assumptions of the models

6 Approach Determine estimate coverage in a common project framework – Engineering Work Breakdown Structure (WBS) vs. Organizational Breakdown Structure (OBS) –Defined based on MIL-HDBK-881A and EIA/ANSI 632 –Generic contract structure Two-step exercise conducted in roundtables/workshops: –Identify functional ownership of the tasks (leaf elements) –Determine estimate coverage by current COCOMO II and COSYSMO Additionally, address the other areas for consideration through analyses in the workshops Note: The use of COCOMO is incidental. It addresses similar issues with other models.

7 Contract Engineering WBS Based On Standards 1.0 – System/Project 1.1 – Integrated Project Management (IPM) 1.2 – Systems Engineering 1.3 – Prime Mission Product (PMP) – Subsystem / Configuration Item (CI) 1…n (Specify Names) – PMP Application Software – PMP System Software – PMP Integration, Assembly, Test & Checkout (IATC) – Operations/Production Support 1.4 – Platform Integration 1.5 – System Test & Evaluation (ST&E) 1.6 – Training 1.7 – Data Management 1.8 – Peculiar Support Equipment 1.9 – Common Support Equipment 1.10 – Operational / Site Activation 1.11 – Industrial Facilities Product-oriented construct, by tailoring MIL- HDBK 881A and EIA/ANSI 632 Six Functions: 1.Systems Engineering 2.Software Engineering 3.Electrical Engineering 4.Mechanical Engineering 5.Support Engineering 6.Project Engineering Management Six Functions: 1.Systems Engineering 2.Software Engineering 3.Electrical Engineering 4.Mechanical Engineering 5.Support Engineering 6.Project Engineering Management

8 The Exercise Y: COSYSMO; S: COCOMO; X: Ownership but no model coverage; U: Uncertainty

9 How Does It Work? We compare the systems and software columns. If any WBS element a mark (Y, S, or X), then it is “owned” by a function. Gaps: If an element has no mark or only “X” in either column, then we have a potential gap. That means neither COSYSMO nor COCOMO, as they stand today, covers that task. Overlaps: If an element has both a “Y” *and* an “S”, then we have a potential overlap. That means both models estimate that scope and we have to reconcile/deconflict. Uncertainty: “U” for an elements means it is not consistently estimated by the model A task may be “owned” by other functions, e.g., HW, in which case it should be covered by other functional models Important: –Task ownership not (necessarily) execution  You, the IPT lead, can use anyone you like to do the job –With this analysis, we attempt to cover nominal projects or majority behavior. There can be exceptions and it could be different from the last program you worked on, which is irrelevant!

10 Summary of Analysis WBS vs. OBS Cross-reference Matrix identifies areas of gaps and overlapsWBS vs. OBS Cross-reference Matrix Analysis identified more potential gaps than overlaps Potential gap areas: –Project engineering management  Variations in costs for types of life cycle models  Quality Management  Technical Process Strategy/Definition/Mgt (at SW level)  Accounting for subcontract or supplier mgt (at SW level) –Prime Mission Product (PMP) systems software  SRS Development  Development for Reusability (at system level) –ST&E test equipment, facility, and support  Accounting for SW support as needed  Test Database Size (at system level) –Ownership issues for the following major areas (often responsibility of Supportability)  Training  Data management  Site construction/conversion  Industry facilities

11 Summary of Analysis (Cont’d) Additional gap, not related to WBS –COSYSMO does not address duration or schedule Potential overlap areas: –PMP system design  Algorithm Development –Development test and evaluation (DT&E) Areas of Uncertainty (due to lack of explicit COSYSMO guidance) –Discrepancy Report (DR) work-off within PMP or support equipment (CI level) –ST&E Mock-ups / Prototypes / Simulations & Test Equipment –Contractor Technical Support – onsite during/after system activation/turnover

12 Results of Other Analysis Analysis of Cost Drivers –Most of the drivers have mappings between the models, albeit different in granularity or handling –Potential concerns covered in Gaps or Recommendations Commonality of Terminology, Constructs, Life Cycle Phases, Units, … –Additional commonality could improve concurrent usage, but is not essential Compatibility issues from any findings or recommendations –No apparent compatibility issues (backward compatibility or with other models in COCOMO Suite) from recommendations Consideration of common size drivers –No essential to harmonization, but may add utility to COCOMO Base assumptions of the models –Need to document assumption for COSYSMO

13 Additional Recommendations Standard phase alignment for both models –Per definitions used in ISO/IEC and Establish means to adequately account for recursion (at level of hands-off to SW) –Needed to resolve gaps Establish operational guidance to minimize variation in usage Add Guidance to COSYSMO Drivers TO: –Account for constraints (e.g. Time & storage) as requirements in the size. –Describe volatility covered in Requirements/Architecture understanding Look into ability to use COSYSMO size drivers in COCOMO for early estimates Add documented list of assumptions to COSYSMO

14 Next Steps Provide results and recommendations to USC team for these models Conduct 2-day workshop at USC in conjunction with COCOMO Forum –29-30 OCT 2008 –Workshop to be led by Ricardo Valerdi, Garry Roedler, and Jared Fortune –All PSM Workshop participants are invited

15 Backup

16 Gaps – Details

17 Gaps – Details (cont.)

18 Overlaps - Details

19 Uncertainties/Inconsistencies

20 Notes & Discussions