Towards COSYSMO 2.0: Update on Reuse USC ARR 2009 Los Angeles, CA Jared Fortune, USC Ricardo Valerdi, MIT

Outline COSYSMO 2.0 Introduction & Status Reuse Process Reuse of SE Artifacts Reuse Scenarios Reuse in Cost Drivers Organizational Impacts of Reuse Next Steps

Reuse Overview Systems engineering activities are support-focused Do not produce physical products (HW, SW, etc.) Produce architectures, requirements, test plans, and other technical documents Systems engineering products can be viewed as “artifacts” Encapsulation of systems engineering knowledge in a document or process Representative of systems engineering effort Reuse of an artifact should reduce the expected systems engineering effort for the development of a new system

Guidance for COSYSMO 2.0 COSYSMO introduced in 2005 Successful, implemented by both academia and industry Not without its limitations Relies on a “built from scratch” assumption Lack of reuse estimation capability cited as a potential source of error Need to develop reuse estimation capability for COSYSMO 2.0 Reuse Considerations Reuse Observations Revised Drivers Literature Review Industry Survey COSYSMO 2.0

Reuse Literature Observations Reuse is done for the purpose of economic benefit, intending to shorten schedule, reduce cost, and/or increase performance. Reuse is not free, upfront investment is required to understand the technical opportunities and limitations. Products, processes, and knowledge are all reusable artifacts. Reuse needs to be planned from the conceptualization phase of programs. Reuse is as much of an organizational issue as it is a technical one. Reuse is knowledge that must be deliberately captured in order to be beneficial. The benefits of reuse are limited to closely related domains. The benefits of reuse do not scale linearly

Reuse Survey Results Requirements reuse is only performed occasionally, but has the largest “benefit” associated with it. Reuse occurs more frequently early in the life cycle than later. Cost savings is the most promoted benefit for reuse, but benefits also exist in risk, schedule, and performance. The proposed five categories of reuse are reasonable in characterizing systems engineering reuse. Experienced personnel is a key factor for successful reuse.

COSYSMO Operational Concept # Requirements # Interfaces # Scenarios # Algorithms Size Drivers COSYSMO Effort Effort Multipliers Application factors 8 factors Team factors 6 factors Calibration

Proposed COSYSMO 2.0 Operational Concept 4+1 Reuse Categories # Requirements # Interfaces # Scenarios # Algorithms Size Drivers COSYSMO 2.0 Effort Effort Multipliers Application factors 8 factors (+1) Team factors 6 factors (+1) Calibration

Proposed Size Driver Extensions New: Artifacts that are completely new Modified: Artifacts that are inherited, but are tailored Adopted: Artifacts that are incorporated unmodified, also known as “black box” reuse Managed: Artifacts that are incorporated unmodified and untested Deleted: Artifacts that are removed from a system 4 or 5 categories

Which Methodology Provides Better Estimation Power? Methodology A Methodology B Reuse Extensions (4+1 categories) Reuse Extensions (4+1 categories) + Reuse Cost Drivers (Revised set)

COSYSMO 2.0 Status Dissertation proposal approved in October 2008 In data collection/model validation phase Aiming for COSYSMO 2.0 delivery in Spring 2010 Determine model updates needed to account for reuse 4+1 Reuse Categories (Size Drivers) 4+1 Reuse Categories & Revised Cost Drivers Mining the COSYSMO calibration data set 58% of projects have some reuse data Data provided as percentage of number of E/N/D value for each size driver

Reuse Process Objectives Improvements to the process Prescriptive guidance Illustrates a way for an organization to think about reuse from a larger perspective (strategy, organization, etc.) Helps an organization that wants to do reuse, establish a foundation Improvements to the process Is this the right sequence? Are these the right steps? What happens in each step? Are there any other feedback loops?

Reuse Process Identify reuse strategy Reuse intangibles Identify reuse artifacts Define classes of reuse Size drivers Cost drivers Risk assessment Document future reuse opportunities Will be discussed in detail Populate reuse repository

Reuse of SE Artifacts Survey results identified frequency of SE artifacts that are reused Requirements Architecture/design models Test data/procedures Documentation/templates What are other SE artifacts that are reused?

Reuse Scenarios What are some example scenarios of reusing systems engineering artifacts? Ex: Block II of a satellite program; many requirements already defined, interfaces are known, test procedures exist.

Reuse in the Cost Drivers Previous research/discussion focused on reuse in the size drivers 4+1 reuse categories, reuse continuum On-going research will provide additional insight on this issue What is the impact of reuse on the cost drivers? Survey revealed non-technical, personnel factors significantly influenced success of reuse Potential Definition changes? Rating scale changes? Modified set of drivers?

Capturing Reuse in the Cost Drivers Reuse Impact On? Impact On Reuse? Application Factors Requirements Understanding Architecture Understanding Level of Service Requirements Migration Complexity Technology Risk Documentation to Match Life Cycle Needs # and Diversity of Installations/Platforms # of Recursive Levels in the Design Team Factors Stakeholder Team Cohesion Personnel/Team Capability Personnel Experience/Continuity Process Capability Multisite Coordination Tool Support

Organizational Impacts of Reuse What do systems engineering departments need to do differently in order to support “strategic” reuse? Survey indicated “strategic” reuse is much more successful and effective than “ad hoc” reuse; reuse needs to be planned early in the lifecycle What mechanisms enable/hinder reuse? How do SE departments handle strategic vs. ad hoc reuse? To what degree does either strategy affect the SE department? What feedback loops exist in the reuse process?

Next Steps COSYSMO 2.0 data collection Expert survey Call for participation Expert survey Validate reuse size driver weights & cost driver adjustments

Call for Participation Background The USC Center for Systems and Software Engineering (CSSE) and Lean Advancement Initiative (LAI) at MIT in collaboration with the INCOSE Measurement Working Group have initiated the next phase of development of a Systems Engineering Cost Model called COSYSMO. Since the first version of the model was completed in 2005, COSYSMO has been widely accepted and adopted by over a dozen industrial and government organizations. To continually address the needs of the user community, an incremental update to the model is currently underway. This update, called COSYSMO 2.0, will improve the estimation power of the model by accounting for systems engineering reuse. To perform an industry calibration, we are seeking industry data in the form of labor actuals on various types of systems engineering projects that involved a significant amount of reuse. Benefits By providing data for this model your organization will: ensure that your particular application domain is addressed by COSYSMO 2.0 learn to tailor and calibrate the updated model for their specific application domain enable the quantification of varying degrees of systems engineering reuse on project estimates be able to claim in CMMI reviews that your systems engineering cost estimates are based on calibrated industry models Proven Methodology COSYSMO (Constructive Systems Engineering Cost Model) employs a proven methodology developed for the COCOMO (Constructive Cost Model), the most widely used software cost model in the world. Proven Process USC-CSSE and LAI at MIT have proven processes in place to ensure the confidentiality of the data with its Corporate Affiliates and Consortium Members. Successful data protection has enabled it to attract the participation of several organizations in this effort including Boeing, Raytheon, Northrop Grumman, Lockheed Martin, General Dynamics, SAIC, L-3 Communications, BAE Systems, and the US Air Force Space & Missile Systems Center. Contact Jared Fortune [fortune@usc.edu] Ricardo Valerdi [rvalerdi@mit.edu] 20