COnstructive SYStems Engineering Cost MOdel COSYSMO COnstructive SYStems Engineering Cost MOdel INCOSE IW Workshop Tampa, FL February 3, 2003 Dr. Barry Boehm Ricardo Valerdi University of Southern California Center for Software Engineering (CSE)

Outline Workshop Agenda Background on CSE, COSYSMO, and COCOMO II COSYSMO Overview Operational concept and scope Prototype demo Model Progress to Date Front end sizing and drivers Full life cycle sizing and drivers New proposed drivers Action items

Workshop Agenda Proposed drivers Process maturity # of recursive levels of design Technology risk (different ratings on the viewpoints) Length of life cycle Quality Attributes (ie., documentation) Manufacturability/Producibility Degree of Distribution Discontinuity of members ISO/IEC 15288 expertise in the working group Preparation for Delphi Round 2 Preliminary data collection

USC Center for Software Engineering (CSE) Researches, teaches, and practices CMMI-based Software engineering Systems and software engineering fully integrated Focuses on better models to guide integrated systems and software engineering Success models: stakeholder win-win, business cases Product models: requirements, architectures, COTS Process models: spiral extensions, value-based RUP extensions Property models: cost, schedule, quality Applies and extends research on major programs (DARPA/Army, FCS, FAA ERAM, NASA Missions)

USC-CSE Affiliates (34) Commercial Industry (15) Daimler Chrysler, Freshwater Partners, Galorath, Group Systems.Com, Hughes, IBM, Cost Xpert Group, Microsoft, Motorola, Price Systems, Rational, Reuters Consulting, Sun, Telcordia, Xerox Aerospace Industry (6) Boeing, Lockheed Martin, Northrop Grumman, Raytheon, SAIC, TRW Government (8) DARPA, DISA, FAA, NASA-Ames, NSF, OSD/ARA/SIS, US Army Research Labs, US Army TACOM FFRDC’s and Consortia (4) Aerospace, JPL, SEI, SPC International (1) Chung-Ang U. (Korea)

INCOSE Companies actively involved with COSYSMO Commercial Industry (1) Galorath Aerospace Industry (5) Lockheed Martin, Northrop Grumman*, Raytheon, SAIC, TRW Government (1) US Army Research Labs FFRDC’s and Consortia (2) Aerospace, SPC *Most recent addition

Key Members of the COSYSMO Working Group Aerospace Corp. Galorath LMCO Raytheon SAIC SPC US Army/PSSM USC Karen Owens, Marilee Wheaton Evin Stump Garry Roedler, Gary Hafen Gary Thomas, John Rieff Tony Jordano, Don Greenlee Chris Miller Cheryl Jones Barry Boehm, Elliot Axelband, Don Reifer, Ricardo Valerdi underline = will participate in Monday’s Workshop Italics = SE experience

USC-CSE Cost, Schedule, and Quality Models Build on experience with COCOMO 1981, COCOMO II Most widely used software cost models worldwide Developed with Affiliate funding, expertise, data support Collaborative efforts between Computer Science (CS) and Industrial Systems Engineering (ISE) Depts. 3 CS PhD’s, 2 ISE PhD’s to date Valerdi an ISE PhD student Boehm joint appointment in CS, ISE COCOMO Suite of models Cost, schedule: COCOMO II, CORADMO, COCOTS Quality: COQUALMO Systems Engineering: COSYSMO Uses mature 7-step model development methodology

7-step Modeling Methodology Analyze Existing literature 1 Perform Behavioral Analysis 2 Identify Relative Significance 3 Perform Expert- Judgement, Delphi Assessment 4 A-PRIORI MODEL + SAMPLING DATA = A-POSTERIORI MODEL Gather Project Data 5 Determine Bayesian A-Posteriori Update Gather more data; refine model 6 7

Parametric Cost Model Critical Path Usual # Months* Critical Path Task 6 Converge on cost drivers, WBS 6 Converge on detailed definitions and rating scales 12 Obtain initial exploratory dataset (5-10 projects) 6 Refine model based on data collection & analysis experience 12+ Obtain IOC calibration dataset (30 projects) 9 Refine IOC model and tool *Can be shortened and selectively overlapped

COSYSMO: Overview Parametric model to estimate system engineering costs Covers full system engineering lifecycle Focused on use for Investment Analysis, Concept Definition phases estimation and tradeoff analyses Input parameters can be determined in early phases

COSYSMO Operational Concept # Requirements # Interfaces # Scenarios # Algorithms + Volatility Factor Size Drivers COSYSMO Effort Effort Multipliers Duration Application factors 5 factors Team factors 7 factors Schedule driver Calibration WBS guided by EIA/ANSI 632 & ISO/IEC 15288

Previous COSYSMO Evolution Path Inception Elaboration Construction Transition Oper Test & Eval 1. COSYSMO-IP IP (Sub)system C4ISR System 2. COSYSMO-C4ISR Physical Machine System 3. COSYSMO-Machine System of Systems (SoS) 4. COSYSMO-SoS

Revised View of COSYSMO Evolution Path (Results from last week’s meeting) Operate, Maintain, or Enhance Oper Test & Eval Transition to Operation Replace or Dismantle Conceptualize Develop Global Command and Control System 1. COSYSMO-IP Include ISO/IEC 15288 Stages 2. COSYSMO-C4ISR Satellite Ground Station Joint Strike Fighter 3. COSYSMO-Machine Future Combat Systems 4. COSYSMO-SoS Initiate data collection for all and let the amount of data received determine what is included.

Breadth and Depth of Key SE Standards System life ISO/IEC 15288 Level of detail Conceptualize Develop Transition to Operation Operate, Maintain, or Enhance Replace or Dismantle Process description High level practices Detailed ISO/IEC 15288 - Establish a common framework for describing the life cycle of systems Purpose of the Standards: EIA/ANSI 632 EIA/ANSI 632 - Provide an integrated set of fundamental processes to aid a developer in the engineering or re-engineering of a system Input to 632/1220 IEEE 1220 IEEE 1220 - Provide a standard for managing systems engineering Source : Draft Report ISO Study Group May 2, 2000

ISO/IEC 15288 Key Terms System System-of-Interest System Element a combination of interacting elements organized to achieve one or more stated purposes System-of-Interest the system whose life cycle is under consideration in the context of this International Standard System Element a member of a set of elements that constitutes a system NOTE: A system element is a discrete part of a system that can be implemented to fulfill specified requirements Enabling System a system that complements a system-of-interest during its life cycle stages but does not necessarily contribute directly to its function during operation NOTE: For example, when a system-of-interest enters the production stage, an enabling production system is required Source: ISO/IEC 15288.

ISO/IEC 15288 System of Interest Structure Make or buy Source: ISO/IEC 15288.

Outline Workshop Agenda Background on CSE, COSYSMO, and COCOMO II COSYSMO Overview Operational concept and scope Prototype demo Model Progress to Date Front end sizing and drivers Full life cycle sizing and drivers New proposed drivers Action items

Recent developments Developed a project plan Reduced drivers from 24 to 18 Expanded the model to cover the later phases of the life cycle Replaced EIA 632 with ISO 15288 Introduced new drivers to reflect full life cycle scope Changed name from COSYSMO-IP (Information Processing) to COSYSMO Revised the evolution path to allow the data to drive the scope of the model

4 Size Drivers Number of System Requirements Number of Major Interfaces Number of Operational Scenarios Number of Unique Algorithms Each weighted by complexity, volatility, and degree of reuse

Number of System Requirements This driver represents the number of requirements that are typically taken from the system or marketing specification. A requirement is a statement of capability containing a normative verb such as “shall” or “will.” It may be functional, performance, feature, or service-oriented in nature depending on the methodology used for specification. System requirements can typically be quantified by counting the number of applicable “shall’s” or “will’s” in the system or marketing specification. Easy Nominal Difficult No. of System Requirements - Well specified - Loosely specified - Poorly specified - Traceable to source - Can be traced to source with some effort - Hard to trace to source - Simple to understand - Takes some effort to understand - Hard to understand - Little requirements overlap - Some overlap - High degree of requirements overlap - Familiar - Generally familiar - Unfamiliar - Good understanding of what’s needed to satisfy and verify requirements - General understanding of what’s needed to satisfy and verify requirements - Poor understanding of what’s needed to satisfy and verify requirements

Number of Major Interfaces This driver represents the number of shared major physical and logical boundaries between system components or functions (internal interfaces) and those external to the system (external interfaces). These interfaces typically can be quantified by counting the number of interfaces identified in either the system’s context diagram and/or by counting the significant interfaces in all applicable Interface Control Documents. Easy Nominal Difficult No. of Major Interfaces - Well defined - Loosely defined - Ill defined - Uncoupled - Loosely coupled - Highly coupled - Cohesive - Moderate cohesion - Low cohesion - Well behaved - Predictable behavior - Poorly behaved

Number of Operational Scenarios This driver represents the number of operational scenarios that a system must satisfy. Such threads typically result in end-to-end test scenarios that are developed to validate the system satisfies all of its requirements. The number of scenarios can typically be quantified by counting the number of end-to-end tests used to validate the system functionality and performance. They can also be calculated by counting the number of high-level use cases developed as part of the operational architecture. Easy Nominal Difficult No. of Operational Scenarios - Well defined - Loosely defined - Ill defined - Few end-to-end scenarios (< 10) - Modest no. of end-to-end scenarios (10 < OS < 30) - Many end-to-end scenarios (> 30) - Timelines not an issue - Timelines a constraint - Tight timelines through scenario network

Number of Unique Algorithms This driver represents the number of newly defined or significantly altered functions that require unique mathematical algorithms to be derived in order to achieve the system performance requirements. As an example, this could include a complex aircraft tracking algorithm like a Kalman Filter being derived using existing experience as the basis for the all aspect search function. Another example could be a brand new discrimination algorithm being derived to identify friend or foe function in space-based applications. The number can be quantified by counting the number of unique algorithms needed to support each of the mathematical functions specified in the system specification or mode description document (for sensor-based systems). Easy Nominal Difficult No. of Unique Algorithms - Existing algorithms - Some new algorithms - Many new algorithms - Basic math - Algebraic by nature - Difficult math (calculus) - Straightforward structure - Nested structure with decision logic - Recursive in structure with distributed control - Simple data - Relational data - Persistent data - Timing not an issue - Timing a constraint - Dynamic, with timing issues - Library-based solution - Some modeling involved - Simulation and modeling involved

12 Cost Drivers Application Factors (5) Requirements understanding Architecture complexity Level of service requirements Migration complexity Technology Risk

Requirements understanding This cost driver rates the level of understanding of the system requirements by all stakeholders including the systems, software, hardware, customers, team members, users, etc… Very low Low Nominal High Very High Poor, unprecedented system Minimal, many undefined areas Reasonable, some undefined areas Strong, few undefined areas Full understanding of requirements, familiar system

Architecture complexity This cost driver rates the relative difficulty of determining and managing the system architecture in terms of platforms, standards, components (COTS/GOTS/NDI/new), connectors (protocols), and constraints. This includes tasks like systems analysis, tradeoff analysis, modeling, simulation, case studies, etc… Very low Low Nominal High Very High Poor understanding of architecture and COTS, unprecedented system Minimal understanding of architecture and COTS, many undefined areas Reasonable understanding of architecture and COTS, some weak areas Strong understanding of architecture and COTS, few undefined areas Full understanding of architecture, familiar system and COTS 2 level WBS 3-4 level WBS 5-6 level WBS >6 level WBS

Level of service requirements This cost driver rates the difficulty and criticality of satisfying the Key Performance Parameters (KPP). For example: security, safety, response time, the “illities”, etc… Very low Low Nominal High Very High Difficulty Simple requirements Low difficulty Moderately complex Difficult requirements Really complex Criticality Slight inconvenience Easily recoverable losses Some loss High financial loss Risk to human life

Migration complexity This cost driver rates the complexity of migrating the system from previous system components, databases, workflows, etc, due to new technology introductions, planned upgrades, increased performance, business process reengineering etc… Very low Low Nominal High Very High Introduction of requirements is transparent Difficult to upgrade Very difficult to upgrade

Technology Risk The maturity, readiness, and obsolescence of the technology(ies) being implemented. This may include Viewpoint Rating VL L N H VH Technology Maturity Level Technology proven and widely used throughout industry Proven through actual use and ready for widespread adoption Proven on pilot projects and ready to roll-out for production jobs Ready for pilot use Still in the laboratory Technology Readiness Level Mission proven (TRL 9) Concept qualified (TRL 8) Concept has been demonstrated (TRL 7) Proof of concept validated (TRL 5 & 6) Concept defined (TRL 3 & 4) Technology Obsolescence Level - Technology is the state-of-the-practice - Emerging technology could compete in future - Technology is stale - New and better technology is on the horizon in the near-term - Technology is outdated and use should be avoided in new systems - Spare parts supply is scarce We need to supply guidelines on how to compose the readiness and obsolescence into a single factor.

12 Cost Drivers (cont.) Team Factors (7) Stakeholder team cohesion Personnel capability Personnel experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support

Stakeholder team cohesion Represents a multi-attribute parameter which includes leadership, shared vision, diversity of stakeholders, approval cycles, group dynamics, IPT framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in stakeholder community of the end users, customers, implementers, and development team. VL L N H VH Highly heterogeneous stakeholder communities with diverse objectives Multiple stakeholders with diverse expertise, task nature, language, culture, infrastructure System involving major changes in stakeholder roles & responsibilities Heterogeneous stakeholder community with converging organizational objectives System involves some changes in stakeholder roles & responsibilities Common shared organizational objectives Functional team Strong team cohesion High stakeholder trust level Clear roles & responsibilities Virtually homogeneous stakeholder communities Institutionalized, shared project culture Strong team dynamics

Personnel experience/continuity Personnel capability Systems Engineering’s ability to perform in their duties and the quality of human capital. Very Low Low Nominal High Very High 15th percentile 35th percentile 55th percentile 75th percentile 90th percentile Personnel experience/continuity The applicability and consistency of the staff over the life of the project with respect to the customer, user, technology, domain, etc… Very low Low Nominal High Very High Less than 2 months 1 year continuous experience, other technical experience in similar job 3 years of continuous experience 5 years of continuous experience 10 years of continuous experience

Revisit, should include more detail Under process levels, process improvement commitment Process maturity Maturity per EIA/IS 731, SE CMM or CMMI. Very low Low Nominal High Very High Extra High Level 1 (lower half) Level 1 (upper half) Level 2 Level 3 Level 4 Level 5 Multisite coordination Location of stakeholders, team members, resources (travel). Very low Low Nominal High Very High Extra High SITE: Collocation International Multi-city and multi-national Multi-city or multi-company Same city or metro area Same building or complex Fully located SITE: Communications Some phone, mail Individual phone, FAX Narrowband e-mail Wideband electronic communication Wideband electronic communication, occasional video conference Interactive multimedia Add Collaboration barriers row (time zone, security, export restrictions, language, culture, competition, Intellectual property, contractual)

Right size of documentation to the life cycle needs The Formality of deliverables The breadth and depth of documentation required to be formally delivered. Very low Low Nominal High Very High General goals Some conformity Some relaxation Partially streamlined process, occasional relaxation Rigorous, follows customer requirements Breadth depth Right size Reviews? Tool support Use of tools in the System Engineering environment. Very low Low Nominal High Very High No SE tools Simple SE tools, little integration Basic SE tools integrated (throughout the systems process) Strong, mature SE tools, integrated (integrated with other disciplines) Strong, mature proactive SE tools integrated with process (Model-based SE and management systems)

Outline Workshop Agenda Background on CSE, COSYSMO, and COCOMO II COSYSMO Overview Operational concept and scope Prototype demo Model Progress to Date Front end sizing and drivers Full life cycle sizing and drivers New proposed drivers Action items

# and diversity of installations/platforms The number of different platforms that the system will be hosted and installed on. The complexity in the operating environment (space, sea, land, fixed, mobile, portable, information assurance/security). For example, in a wireless network it could be the number of unique installation sites and the number of types of fixed clients, mobile clients, and servers. Include phase out Viewpoint N H VH Sites/installations Few # of installations or many similar installations Moderate # of installations or some amount of multiple types of installations Numerous # of installations with many unique aspects Operating environment Not a driving factor Moderate environmental constraints Multiple complexities/constraints caused by operating environment No. of Different Platforms Few types of platforms (< 5) Modest types of no. of platforms (5 < P <10) Many types of platforms (> 10) Homogeneous Mixed Heterogeneous Typically networked using a single protocol Typically networked using several consistent protocols Typically networked using different protocols

# of recursive levels in the design New cost driver. Can capture the number of boundaries in the SOI (System of Interest). Number of recursive levels that require SE. For example, a system with a single level of design may have lower numbers of recursive levels in the design. On the other hand, a system with a system integrator (system of systems level), a prime (system level), a subcontractor (segment level), second-tier subcontractor (subsystem level), and third-tier contractor (component/box level) would have 5 levels of recursive design. Includes internal design/subsystem levels. Note: this driver depends on what level your enabling system lies. Revision: Has system-wide multiplicative effect vs. incremental additive effect. Better handled in two cost drivers: Architecture Complexity (technical aspects) & Multisite Coordination (management aspects).

# and diversity of installations/platforms New size driver. Can capture the number of varied and multiple installations and/or platforms. This is in light of the discussion to cover the full system life cycle. This driver is especially important in later implementation/deployment stages. Revision: Its effect on effort is additive but the amount of added effort per type of installation/platform is a multiplier of the baseline effort. Probably best covered by a formula like: (# of diverse installation/platform types) * (Avg extra % of systems engineering effort per type)

The number of different processing platforms that the system will be hosted and installed on. For example, in a network it could be the number of unique installation sites and the number of workstations and servers. Viewpoint N H VH Sites/installations Few # of installations or many similar installations Moderate # of installations or some amount of multiple types of installations Numerous # of installations with many unique aspects Operating environment Not a driving factor Moderate environmental constraints Multiple complexities/constraints caused by operating environment # of Different Platforms Few platforms (< 5) Modest no. of platforms (5 < P <10) Many platforms (> 10) Homogeneous Mixed Heterogeneous Typically networked using a single protocol Typically networked using several consistent protocols Typically networked using different protocols

# of years in operational life cycle Revision: Another additive factor where the added effort is a multiplier of the baseline effort. Probably best covered by a formula like: (# of years in operational life cycle) * (Avg annual % of continuing systems engineering effort) Evolutionary acquisition # of independently evolving interoperating systems Upgrade to prior models

# and diversity of installations/platforms phased out Revision: Its effect on effort is additive but the amount of added effort per installation/platform being phased out is a multiplier of the baseline effort. Probably best covered by a formula like: (# of diverse installation/platform types) * (Avg extra % of systems engineering effort per type being phased out)

Quality Attributes Rationale: Reliability, availability and maintainability requirements for certain classes of systems often drive their costs. This is true for military as well as commercial systems. A cost driver aimed at assessing the impact of varying these requirements is therefore needed. Quality Attributes: Represents a measure of the extent to which the system must perform its intended requirements (reliability), be available (availability) and facilitate change, modification and repair (maintainability) over time. Viewpoint Rating VL L N H VH Reliability Errors led to slight inconvenience Errors led to easily recoverable loss Errors led to moderate recoverable loss Errors led to high financial loss Safety critical Availability < 70% availability Between 70% and 90% availability > 90% availability 24/7 with schedulable downtime (for PM) 24/7 with negligible downtime Maintainability Difficult to change, modify and/or repair Can change, modify and/or repair with effort Easily changed, modified and repaired Reconfigurable Self healing

Manufacturability/Producibility Rationale: The ability to manufacture/produce the system can drive the costs especially when advances in manufacturing technology are needed to field the system. Manufacturability/Producibility: Represents the ability of contractors to manufacture/produce the system in specific quantity to meet the market demand. Viewpoint Rating VL L N H VH Manufacturability New technology needed to manufacture system Facilities needed to manufacture systems Technology/facilities in place to manufacture system Can meet 100% of peak manufacturing demand Have excess manufacturing capability Producibility Can meet less than 70% of peak demand Can meet between 70 and 90% of peak demand Can meet 90% of peak demand Can meet 100% of peak demand Have excess production capability DFMA Design for “x” Design trades Revisit

Degree of Distribution Rationale: The degree of partitioning and distribution of the system impacts its cost. This cost driver how interoperable the system must be. Degree of Distribution: Characterizes how distributed the architecture is. Viewpoint Rating VL L N H VH Distribution Self-contained with limited distribution Distribution via LANs/direct links Distribution via LANs/WANs Network of networks System of systems Interoperability Self-contained with no interoperability Interoperates with fewer than 3 systems Interoperates with between 3 to 5 systems Interoperates with 5 to 10 systems Interoperates with more than 10 systems

Levels & complexity of V&V Revisit Might be under # of Requirements (under VH)

Size Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle Legend Bold = existing driver Italics = proposed addition

Cost Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle Legend Bold = existing driver Italics = proposed addition

Cost Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle Legend Bold = existing driver Italics = proposed addition

Action Items from Last WG Meeting Develop a project Plan Address technology maturity/obsolescence Refine driver definitions to incorporate ISO/IEC 15288 definitions Incorporate System and People idea Refine drivers applicability matrix Develop data collection strategy Generate Data Collection Form Update Stakeholder Cohesion to include diversity, identification and trust

Calendar of Activities: 2003 Paper & tutorial submitted USC CSE Annual Research Review INCOSE 2003 ISPA COCOMO Forum J F M A M J J A S O N D 2003 2004 INCOSE Fall Workshop Practical Software & Systems Measurement Workshop Conference on Systems Integration Paper submitted

Points of Contact Dr. Barry Boehm [boehm@sunset.usc.edu] Dr. Elliot Axelband [axelband@usc.edu] Don Reifer [dreifer@earthlink.net] Ricardo Valerdi [rvalerdi@sunset.usc.edu] Website http://sunset.usc.edu

Backup Charts

USC-CSE Research ($10M backlog) DARPA/Army: Model applications and extensions for Future Combat Systems DARPA: Architectures for mobile distributed systems (DASADA) FAA: Acquisition processes; COCOMO security extensions NASA: Empirical methods for High Dependability Computing NSF: Center for Empirically-Based Software Engineering (with U. of Maryland) NSF: Strategic Design (with CMU, Virginia, Washington) Industry Affiliates’ program

General Affiliate Benefits Affiliates-only Web portal Early access to tools, methods, papers, talks, student resumes Tools: COCOMO Suite, Architecture tools, WinWin Technical Report series Workshops on Affiliate-prioritized topics Annual Research Review and Steering Group meeting Annual one-day professor-visit Bilateral visit arrangements; internships Conferences and special workshops Monthly LA SPIN meetings Tutorials and eWorkshops

Collaboration Modes and Special Benefits Software architecting assistance -Aerospace, Hughes, JPL, Northrop Grumman, TACOM, TRW, Xerox Software process/cost/quality/cycle time assistance -Aerospace, Litton, Microsoft, Northrop Grumman, Raytheon, SAIC, Sun, TACOM, TRW, Xerox Management reviews of critical projects -Litton, Motorola, SAIC, SEI, TRW Reviews of corporate research programs -Daimler Chrysler, Draper Labs, Lockheed Martin, SAIC, SEI, SPC, Telcordia, TRW Joint research contracts -Aerospace, Lockheed Martin, Northrop Grumman, SEI, SPC, TRW Aid in commercializing USC-CSE research -C-Bridge, Galorath, Group Systems.com, Marotz, Price Systems, Rational

Collaboration Modes and Special Benefits - II Special Projects -Aerospace, Auto Club, FAA, Fidelity, IBM, JPL, Litton, Northrop Grumman, Telcordia Joint workshops on key topics -Aerospace, Motorola, Rational, DOD/SIS, SEI, SPC Focused working groups (COSYSMO) -Aerospace, Galorath, Lockheed Martin, Raytheon, SAIC, SPC, TRW Visiting collaborators -Aerospace, Chung-Ang, C-Bridge, IBM, JPL, Litton, Northrop Grumman, SEI, TRW Corporate State-of-the-art tutorials -Boeing, Chung-Ang, Daimler Chrysler, Draper, EDS, FAA, Fidelity, IBM, JPL, Litton, Lockheed Martin, Lucent, Motorola, Microsoft, Raytheon, SAIC, SEI, SPC, Sun, TRW, Xerox

Mapping of Old to New COSYSMO-IP Drivers Number of System Requirements Number of Major Interfaces Number of Technical Performance Measures Number of Operational Scenarios Number of Modes of Operation Number of Different Platforms Number of Unique Algorithms Number of System Requirements Number of Major Interfaces Number of Operational Scenarios Number of Unique Algorithms Size Factors Level of Service Requirements Architecture complexity

Delphi Round 1 Highlights Range of sensitivity for Size Drivers 6.48 5.57 6 Relative Effort 4 2.54 2.10 2.21 2.23 2 1 # Modes # TPM’s # Platforms # Scenarios # Interfaces # Algorithms # Requirements

Mapping of Old to New COSYSMO-IP Drivers # of TPMs # of Platforms Application Cost Factors Requirements understanding Architecture complexity Level of service requirements Legacy Transition complexity COTS assessment complexity Platform difficulty Required business process reengineering Technology Maturity Physical system/information subsystem tradeoff analysis complexity Requirements understanding Architecture complexity Level of service requirements Migration complexity Technology Maturity

Delphi Round 1 Highlights (cont.) Range of sensitivity for Cost Drivers (Application Factors) 4 EMR 2.81 2.43 2.24 2.13 2 1.13 1.74 1.93 COTS Legacy transition Platform difficulty Architecture und. Requirements und. Bus. process reeng. Level of service reqs.

Mapping of Old to New COSYSMO-IP Drivers Team Cost Factors Reqs Und Number and diversity of stakeholder communities Stakeholder team cohesion Personnel capability Personnel experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support Stakeholder team cohesion Personnel capability Personal experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support

Delphi Round 1 Highlights (cont.) Range of sensitivity for Cost Drivers (Team Factors) 4 EMR 2.46 2.16 1.91 1.94 1.78 1.84 2 1.25 1.28 Tool support Multisite coord. Formality of deliv. Process maturity Stakeholder comm. Stakeholder cohesion Personal experience Personnel capability

COSYSMO/COCOMO II Mapping Previous candidate starting point TBD When doing COSYSMO-IP and COCOMOII, Subtract grey areas prevent double counting. = COCOMOII = COSYSMO-IP