1. Constructive System of Systems Integration Cost Model (COSOSIMO)
Constructive System of Systems Integration Cost Model (COSOSIMO) ****************** Tutorial
COSOSIMO
Jo Ann Lane,
USC Center for Systems & Software Engineering
23 October 2006

2. Overview COSOSIMO Background
System of Systems (SoS) and SoS Engineering (SoSE) Environment
Current COSOSIMO Cost Estimation Approach
Conclusions
References

3. COCOMO Cost Model Suite Overview*
In the COCOMO estimating world, we first focused on software development estimation. As we got comfortable with that model, we began expanding the estimation models: COSYSMO, for system engineering, COCOTS for COTS integration, and so on. Then as we watched the complexity of software system development evolve in response to demands for more complex systems (often comprised of existing systems), we realized that a piece of the software effort was missing – this is the effort associated with the definition and integration of system of system architectures. To fill this hole, we began working on another model: COSOSIMO.
* Barry Boehm, Ricardo Valerdi, Jo Ann Lane, and Winsor Brown, “COCOMO Suite Methodology and Evolution”, CrossTalk, April 2005.

4. USC-CSE Modeling Methodology*
Analyze existing literature
Step 1
Concurrency and
feedback implied…
Perform Behavioral analyses
Step 2
Identify relative significance
Step 3
Perform expert-judgment Delphi assessment, formulate a-priori model
Step 4
Gather project data
Step 5
Step 1: Understand results of related efforts, current capabilities, potential parameters and parameter definition issues.
Step 2: Perform behavioral analyses to determine associated drivers and factors and relative ranges of values for these parameters.
Step 3: Identify relative significance of drivers and scale factors
Step 4: Obtain inputs from industry and CSE affiliates. This step of the methodology can often involve multiple rounds of the Delphi survey that provide model developers some insight into the effects of the model parameters on development effort. The Delphi surveys attempt to capture what the experts believe has an influence on development effort.
Step 5: Collect historical project data to validate the Cost Estimating Relationships in the model.
Step 6: Combine the project data with the expert judgment captured in the Delphi survey to produce a calibrated model. This is done using Bayesian statistical techniques that provide the ability to balance expert data and historical data.
Step 7: Continuous update/refinement process.
Determine Bayesian A-Posteriori model
Step 6
Gather more data; refine model
Step 7
* Boehm, et. al., Software Cost Estimation with COCOMOII, 2000.

5. Goal of Research Develop a cost model (COSOSIMO) to
Support the estimation of effort associated with System-of-System Engineering (SoSE)
May be performed by one or more Lead System Integrator (LSI) organizations
Complement the other USC CSE cost models for software development, system engineering (SE), and Commercial-Off-the-Shelf (COTS) integration, leading toward a more comprehensive and unified cost model to support the much broader system of interest life cycle
The goal is to support estimation activities for estimating the LSI effort in a way that allows users to develop initial estimates and then conduct tradeoffs based on alternatives.
The other goal is to minimize overlap with other cost models in the COCOMO suite.. And to use those other models to estimate the total effort associated with an SoS.
COSOSIMO will not estimate the total SoS development
costs, but rather just the SoSE costs at the SoS level…

6. History of COSOSIMO Model
Early 2003
Potential need for SoSE cost model identified
Fall/Winter 2003
Initial model developed based on software size
Fall 2004
Early design model based of SoS architecture characteristics (not software size)
Spring/Summer 2005
EIA 632-based survey conducted to determine SoSE differences from traditional systems engineering
Fall 2005
SoSE WBS analysis
Fall/Winter 2005
2-submodel version of COSOSIMO investigated
Spring/Summer 2006
SoSE-specific characteristics captured from SoSE conferences/workshops
Spring 2006
3-submodel version of COSOSIMO proposed

7. What is a “System-of-Systems”?
Very large systems developed by creating a framework or architecture to integrate component systems
SoS component systems independently developed and managed
New or existing systems
Have their own purpose
Can dynamically come and go from SoS
SoS exhibits emergent behavior not otherwise achievable by component systems
SoS activities often planned and coordinated by a Lead System Integrator (LSI)
Typical domains
Business: Enterprise-wide and cross-enterprise integration to support core business enterprise operations across functional and geographical areas
Military: Dynamic communications infrastructure to support operations in a constantly changing, sometimes adversarial, environment
INCOSE Handbook Definition: “Systems of Systems” are defined as an interoperating collection of component systems that produce results unachievable by the individual systems alone. (Krygiel 1999)

8. What is a “Lead System Integrator”?
Organization (or set of organizations) selected to accomplish the definition and acquisition of SoS components, and the continuing integration, test, and evolution of the components and SoS
Typical activities
Lead concurrent engineering of requirements, architecture, and plans
Identify and evaluate technologies to be integrated
Conduct source selection
Coordinate supplier activities and validate SoS architecture feasibility
Integrate and test SoS-level capabilities
Manage changes at the SoS level and across the SoS-related IPTs
Manage evolving interfaces to external systems
Typically do not develop system components to be integrated (possible exception: SoS infrastructure)

9. What is SoSE
USAF SAB Report on SoSE for Air Force Capability (USAF 2005): The process of planning, analyzing, organizing, and integrating the capabilities of a mix of existing and new systems into a system-of-systems capability that is greater than the sum of the capabilities of the constituent parts. This processes emphasizes the process of discovering, developing, and implementing standards that promote interoperability among systems developed via different sponsorship, management, and primary acquisition processes.
National Centers for Systems of Systems Engineering (NCOSOSE): The design, deployment, operation, and transformation of metasystems that must function as an integrated complex system to produce desirable results. These metasystems are themselves comprised of multiple autonomous embedded complex systems that can be diverse in technology, context, operation, geography, and conceptual frame. (

10. What is SoSE (continued)
Wikipedia ( SoSE is a set of developing processes and methods for designing and implementing solutions to System-of-Systems problems. SoSE is relatively new term being used in Department of Defense applications, but is increasingly being applied to non-military/security related problems (e.g. transportation, healthcare, internet, search and rescue, space exploration). SoSE is more than systems engineering of complex systems because design for System-of-Systems problems is performed under some level of uncertainty in the requirements and the constituent systems, and it involves considerations in multiple levels and domains.
SoSE and Systems Engineering are related but different fields of study. Where as systems engineering addresses the development and operations of products, SoSE addresses the development and operations of programs. In other words, traditional systems engineering seeks to optimize an individual system (i.e., the product), while SoSE seeks to optimize network of various systems brought together to meet specific program's (i.e., the SoS problem's) objectives. SoSE enables decision-makers to understand the implications of various choices; thus, SoSE methodology seeks to prepare the decision-makers for effective architecting of System-of-Systems problems.
Due to varied methodology and areas of applications in existing literature, there is no unified consensus for processes involved in System-of-Systems Engineering. One of the proposed SoSE frameworks, by Dr. Daniel A. DeLaurentis, recommends a three-phase method where a SoS problem is defined (understood), abstracted, modeled and analyzed for behavioral patterns.

11. SoSE Compared to Traditional SE Activities
Traditional SE Activities (EIA/ANSI 632)
Acquisition and supply
Product Supply
Product Acquisition
Supplier Performance
Technical management
Process Implementation Strategy
Technical Effort Definition
Schedule and Organization
Technical Plans
Work Directives
Progress Against Plans and Schedules
Progress Against Requirements
Technical Reviews
Outcomes Management
Information Dissemination
System design
Acquirer Requirements
Other Stakeholder Requirements
System Technical Requirements
Logical Solution Representations
Physical Solution Representations
Specified Requirements
Traditional SE Activities (continued)
Product realization
Implementation
Transition to Use
Technical evaluation
Effectiveness Analysis
Tradeoff Analysis
Risk Analysis
Requirements Statements Validation
Acquirer Requirements Validation
Other Stakeholder Requirements Validation
System Technical Requirements Validation
Logical Solution Representations Validation
Design Solution Verification
End Product Verification
Enabling Product Readiness
End Products Validation

12. SoSE Compared to Traditional SE Activities (continued)
Key Areas Where SoSE Activities Differ From Traditional Systems Engineering
Architecting composability vs. decomposition (Meilich 2006)
Added “ilities” such as flexibility, adaptability, composability (USAF 2005)
Net-friendly vs. hierarchical (Meilich 2006)
First order tradeoffs above the component systems level (e.g., optimization at the SoS level, instead of at the component system level) (Garber 2006)
Early tradeoffs/evaluations of alternatives (Finley 2006)
Human as part of the SoS (Siel 2006, Meilich 2006, USAF 2005)
Discovery and application of convergence protocols (USAF 2005)

13. SoSE Compared to Traditional SE Activities (continued)
Key Areas Where SoSE Activities Differ From Traditional Systems Engineering (continued)
Organizational scope defined at runtime instead of at system development time (Meilich 2006)
Dynamic reconfiguration of architecture as needs change (USAF 2005)
Modeling and simulation, in particular to better understand “emergent behaviors” (Finley 2006)
Component systems separately acquired and continue to be managed as independent systems (USAF 2005)
Intense concept phase analysis followed by continuous anticipation; aided by ongoing experimentation (USAF 2005)

14. SoSE Compared to Traditional SE Activities (continued)
Key Challenges for SoSE
Business model and incentives to encourage working together at the SoS level (Garber 2006)
Doing the necessary tradeoffs at the SoS level (Garber 2006)
Human-system integration (Siel 2006, Meilich 2006)
Commonality of data, architecture, and business strategies at the SoS level (Pair 2006)
Removing multiple decision making layers (Pair 2006)
Requiring accountability at the enterprise level (Pair 2006)
Evolution management (Meilich 2006)
Maturity of technology (Finley 2006)
For the most part, SoSE appears to be SE+

15. Sample Dynamic SoS: Metropolitan Area Crisis Management System
Net-Centric
Connectivity
Net -
Centric SoS

16. Sample “Steady-State” SoS: Enterprise Wide Integration of Core Business Applications
Supplier 1
Supplier n •
Net-Centric
Connectivity
• • •

17. System of Systems Cost Estimation
SOS Sm
S2 (SoS) S1
S11
S12
S1n
S21
S22
S2n
Sm1
Sm2
Smn ……
Level 0
Level 1
Level 2
Activity
Levels
Cost Model
SoS Lead System Integrator Effort (SoS scoping, planning, requirements, architecting; source selection; teambuilding, re-architecting, feasibility assurance with selected suppliers; incremental acquisition management; SoS integration and test; transition planning, preparation, and execution; and continuous change, risk, and opportunity management)
Level 0, and other levels if lower level systems components are also SoSs (e.g., S2)
COSOSIMO
Development of SoS Software-Intensive Infrastructure and Integration Tools
Level 0
COCOMO II
System Engineering for SoS Components
Levels 1-n
COSYSMO
Software Development for Software-Intensive Components
COTS Assessment and Integration for COTS-based Components
COCOTS

18. System of Systems Cost Model
Size Drivers
COSOSIMO
SoS
Definition and
Integration
Effort
Cost Drivers
Calibration
Characteristics of SoSs supported by cost model
Strategically-oriented stakeholders interested in tradeoffs and costs
Long-range architectural vision for SoS
Developed and integrated by an LSI
System component independence
Size drivers and cost drivers
Based on product characteristics, processes that impact LSI effort, and LSI personnel experience and capabilities

19. Proposed Size Drivers S2 S1 S4 S3 Number of SoS-related requirements
Number of of distinct interface protocols to be provided by the SoS framework
Number of independent system component organizations that are providing system components that will operate within the SoS framework
Number of SoS user scenarios
Number of unique component systems S2 S1
· Number of SoS Interface Protocols: The number of distinct interface protocols to be provided by the SoS framework.
· Number of Independent System Component Organizations: The number organizations that are providing system components that will operate within the SoS framework.
Each of these size inputs is weighted with respect to expected complexity. The interface protocol complexity is determined by factors such as number of protocol layers, desired security, and whether this is a new protocol being implemented for the first time or an existing protocol. The independent system component organization complexity is determined by the expected level of cooperation between organizations and the competing needs of the SoS versus the needs and schedules of the independent component system provider or sponsor. S4
Each weighted
by complexity… S3

20. Conceptual LSI Effort Profile
LSI activities focus on three somewhat independent activities, performed by relatively independent teams
A given LSI may be responsible for one, two, or all activity areas
Some SoS programs may have more than one organization performing LSI activities

21. COSOSIMO Reduced Parameter Sub-Model Overview
Planning,
Requirements
Management,
and Architecting
(PRA)
Size Drivers
SoS
Definition and
Integration
Effort
Source Selection
and Supplier
Oversight (SO)
Cost Drivers
SoS Integration
and Testing
(I&T)

22. COSOSIMO: PRA Sub-Model
Size Drivers
# SoS-related requirements
# SoS interface protocols
Planning,
Requirements
Management,
and Architecting
LSI PRA
Effort
Cost Drivers
Requirements understanding
Level of service requirements
Stakeholder team cohesion
SoS team capability
Maturity of LSI processes
Tool support
Cost/schedule compatibility
SoS risk resolution

23. COSOSIMO PRA Effort Estimation
m n
SoS PRAPM = APRA[ CREQi +  CIPj]BPRA
i= j=1
Where:
PRAPM LSI Planning, Requirements Management, and Analysis effort in person-months
APRA Constant derived from PRA historical data
CREQi Complexity factor associated with the ith SoS requirement
CIPj Complexity factor associated with the jth SoS interface protocol
m Number of SoS-related “sea-level” requirements
n Number of interface protocols supported by the SoS architecture
BPRA Effort exponent based on the PRA exponential scale factors. The geometric product of the scale factors results in an overall exponential effort adjustment factor to the nominal PRA effort
Basically, COSOSIMO is a model that sums together the weighed size drivers and exponentially adjusts the weighted sum by the product of the scale factors.
If follow-on surveys indicate that additional drivers or scale factors are required, they can be easily incorporated.
Some initial work has been done on determining values for the current set of candidate scale factors. However, not much more work is planned in this area until the actual set of scale factors stabilizes.

24. COSOSIMO: SO Sub-Model
Size Drivers
# independent component system organizations
Source Selection
and
Supplier
Oversight
LSI SO
Effort
Cost Drivers
Requirements understanding
Architecture maturity
Level of service requirements
SoS team capability
Maturity of LSI processes
Tool support
Cost/schedule compatibility
SoS risk resolution

25. COSOSIMO SO Effort Estimation
SoS SOPM = ASO[ CSCOj]BSO
j=1
Where:
SOPM LSI Source Selection and Supplier Oversight effort in person-months
ASO Constant derived from SO historical data
CSCOj Complexity factor associated with the jth SoS component system organization
n Number of organizations providing independently developed and maintained system components for the SoS
BSO Effort exponent based on the SoS SO exponential scale factors. The geometric product of the scale factors results in an overall exponential effort adjustment factor to the nominal SO effort
Basically, COSOSIMO is a model that sums together the weighed size drivers and exponentially adjusts the weighted sum by the product of the scale factors.
If follow-on surveys indicate that additional drivers or scale factors are required, they can be easily incorporated.
Some initial work has been done on determining values for the current set of candidate scale factors. However, not much more work is planned in this area until the actual set of scale factors stabilizes.

26. COSOSIMO: I&T Sub-Model
Size Drivers
# SoS interface protocols
# SoS scenarios
# unique component systems
SoS
Integration
and Testing
LSI I&T
Effort
Cost Drivers
Requirements understanding
Architecture maturity
Level of service requirements
SoS team capability
Maturity of LSI processes
Tool support
Cost/schedule compatibility
SoS risk resolution
Component system maturity and stability
Component system readiness

27. COSOSIMO I&T Effort Estimation
q r s
SoS I&TPM = AI&T[ CIPi +  CSCENj +  CSCOk]BI&T
i= j= k=1
Where:
I&TPM LSI Integration and Test effort in person-months
AI&T Constant derived from I&T historical data
CIPi Complexity factor associated with the ith SoS interface protocol
CSCENj Complexity factor associated with the jth SoS interface protocol
CSCOk Complexity factor associated with the kth SoS component system organization
q Number of interface protocols supported by the SoS architecture
r Number of SoS scenarios
s Number of organizations providing independently developed and maintained system components for the SoS
BI&T Effort exponent based on the I&T exponential scale factors. The geometric product of the scale factors results in an overall exponential effort adjustment factor to the nominal I&T effort
Basically, COSOSIMO is a model that sums together the weighed size drivers and exponentially adjusts the weighted sum by the product of the scale factors.
If follow-on surveys indicate that additional drivers or scale factors are required, they can be easily incorporated.
Some initial work has been done on determining values for the current set of candidate scale factors. However, not much more work is planned in this area until the actual set of scale factors stabilizes.

28. COSOSIMO Total SoSE Effort Estimation
SoSEPM = PRAPM + SOPM + I&TPM
Where:
PRAPM LSI Planning, Requirements Management, and Analysis effort in person-months
SOPM LSI Source Selection and Supplier Oversight effort in person-months
I&TPM LSI Integration and Test effort in person-months
Basically, COSOSIMO is a model that sums together the weighed size drivers and exponentially adjusts the weighted sum by the product of the scale factors.
If follow-on surveys indicate that additional drivers or scale factors are required, they can be easily incorporated.
Some initial work has been done on determining values for the current set of candidate scale factors. However, not much more work is planned in this area until the actual set of scale factors stabilizes.

29. SoS Schedule Estimation
Customer,
Users
LSI –
Agile
LSI IPTs –
Suppliers –
PD – V&V
Integrators
RFP, SOW, Evaluations, Contracting
Effort/Staff
Proposals
Similar, with
added change
traffic from
users…
Assess
compatibility, short-falls
Rework LCO  LCA
Packages at all levels
COSOSIMO
-like
Assess sources of change; Negotiate rebaselined LCA2 package at all levels
added re-
baselineing risks
and rework…
Inception
Elaboration
Source SoS Selection Architecting
Increment 1
Increments
2,… n
Develop to spec, V&V
CORADMO
Degree of Completeness
risks,
rework
Proposal Feasibility
LCO
LCA
LCA1
IOC1
Effort/staff
at all levels
Risk-manage slow-performer, completeness
Integrate
LCA2 shortfalls
Effort COSYSMO-like.
Schedule = Effort/Staff
Try to model
ideal staff size
LCA2

30. Conclusions
Traditional systems engineering takes too long and too much effort
LSIs are finding better ways to engineering SoSs (SoSE)
Many combine agile with traditional approaches
Increases concurrency
Reduces risk
Compresses schedules
Reduced-parameter set COSOSIMO captures effects of new processes in three key areas
Planning, requirements management, and architecting
Source selection and supplier oversight
SoS integration and testing
Sub-models have fewer parameters that are more tailored to associated SoSE activities
Allows LSIs to estimate areas of interest and conduct “what ifs” comparisons of different development strategies
COSOSIMO

31. Conclusions (continued)
With the addition of a new COSOSIMO cost model to existing cost model tools, it will be possible to get more complete estimates of the SoS development effort
Key to this process is
Having an SoS architecture sufficiently defined so that component system modifications to support operation in the SoS environment can be made with few dependencies on other SoS development efforts
Structuring the WBS so that
SoS and component system tasks can be decomposed into parts that can be estimated using the existing cost model tools
Parts not covered by cost models can be clearly identified and estimated using non-parametric methods
Expected COSOSIMO availability: Fall 2007
COSOSIMO
“All models are wrong, but some of them are useful”
(W. E. Deming)

32. What is Needed to Support Fall 2007 Availability
Participation in current SoSE surveys
Data from both SoS and SE programs
Process descriptions to help understand the differences between SoSE and SE
Effort data to calibrate COSOSIMO (either standalone model or special calibration of COSYSMO)
For those organizations that provide SoSE effort from at least 3 SoS projects, a local calibration will be provided…

33. COSOSIMO-Related References
Boehm, B., et al. (2000); Software Cost Estimation with COCOMO II; Prentice Hall
Boehm,B., Valerdi, R., Lane, J., and Brown, W. (2005); COCOMO Suite Methodology and Evolution; CrossTalk, Vol. 18, No. 5 (pp )
Boehm, B., and J. Lane (2006); “21st Century Processes for Acquiring 21st Century Systems of Systems; CrossTalk Vol. 19, No. 5 (pp. 4-9)
Lane, J. (2005); System of Systems Lead System Integrators: Where do They Spend Their Time and What Makes them More/Less Efficient; USC-CSE-TR
Lane, J. (2005); Factors Influencing System-of-Systems Architecting and Integration Costs; Conference on Systems Engineering Research
Lane, J (2006); COSOSIMO Parameter Definitions, USC-CSE-TR
Lane, J and Boehm, B. (2006); Synthesis of Existing Cost Models to Meet System of Systems Needs; Conference on Systems Engineering Research
Lane, J and Boehm, B. (2006); System-of-Systems Cost Estimation: Analysis of Lead System Integrator Engineering Activities; InterSymposium Symposium on Information Systems Research and Systems Approach
Lane, J and Valerdi, R (2005); Synthesizing SoS Concepts for Use in Cost Estimation; IEEE Systems, Man, and Cybernetics

34. SoSE-Related References
Carlock, P.G., and R.E. Fenton, "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp , 2001
DiMario, Mike (2006); “System of Systems Characteristics and Interoperability in Joint Command Control”, Proceedings of the 2nd Annual System of Systems Engineering Conference
Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System
Finley, James (2006); “Keynote Address”, Proceedings of the 2nd Annual System of Systems Engineering Conference
Garber, Vitalij (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference
INCOSE (2006); Systems Engineering Handbook, Version 3, INCOSE-TP
Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33
Maier, M. (1998); “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp )
Meilich, Abe (2006); “System of Systems Engineering (SoSE) and Architecture Challenges in a Net Centric Environment”, Proceedings of the 2nd Annual System of Systems Engineering Conference
Pair, Major General Carlos (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference
Proceedings of AFOSR SoSE Workshop, Sponsored by Purdue University, May 2006
Proceedings of Society for Design and Process Science 9th World Conference on Integrated Design and Process Technology, San Diego, CA, June 2006
Siel, Carl (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference
United States Air Force Scientific Advisory Board (2005); Report on System-of-Systems Engineering for Air Force Capability Development; Public Release SAB-TR-05-04