1. System-of-Systems Cost Modeling: COSOSIMO July 2005 Workshop Results Jo Ann Lane jolane@usc.edu University of Southern California Center for Software Engineering COCOMO Forum – October 2005 © USC CSE 2005

2. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20052 Overview COSOSIMO background July 2005 workshop objectives Summary of workshop outputs –Discussions –Surveys Going forward

3. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20053 Scope of Proposed SoS Cost Model Types of SoSs supported by cost model –Strategically-oriented stakeholders interested in tradeoffs and costs –Long-range architectural vision for SoS –Developed and integrated by a Lead System Integrator (LSI) –System component independence Size drivers and scale factors –Based on product characteristics, processes that impact LSI effort, and LSI personnel experience and capabilities Size Drivers Scale Factors SoS Definition and Integration Effort Calibration COSOSIMO

4. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20054 Size Drivers Scale Factors SoS Definition and Integration Effort Calibration Number of distinct protocols Number of independent system component organizations SoS architecture maturity Cost/schedule compression Integration risk resolution Component system maturity and stability Component readiness Integration team capability Integration process maturity COSOSIMO “Early Design*” COSOSIMO Concept * “Early design” refers to the COSOSIMO version that can be used in the early stages of concept development and elaboration.

5. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20055 July 2005 Workshop Objectives and Summary Objectives –Clarify the LSI activities to be estimated by the COSOSIMO model –Begin converging on a relevant and complete set of parameters for the COSOSIMO model that are easily discerned in the early stages of SoS development Summary of workshop activities –Attended by 20 people representing 12 organizations –Provided overview on research conducted to date on LSI activities –Discussed/updated list of key LSI activities –Conducted survey to better determine the differences between LSI activities and more traditional SE activities –Discussed size drivers and factors that impact LSI effort –Conducted survey to identify relevant LSI size drivers and scale factors

6. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20056 Workshop Recommendations and Results LSI Activities –Recommendation: LSI activities should be broken out into the following categories Management activities Technical activities –Results: Updated list of LSI activities No initial activities or issues dropped from list Additional activities added to list Additional issues added to list LSI Effort Management Effort Technical Effort Discussions to continue in this area …. Proposed scope for COSOSIMO if technical can be estimated using COSYSMO SoS calibration Investigate possibility of COSYSMO calibration to estimate this part

7. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20057 Workshop Recommendations and Results (continued) Suggested additional activities –Configuration management –Common infrastructure alignment/supply Development common vocabulary Management processes Technical processes/tools –Managing interoperability with external systems –Verification and validation of the total system Development of SoS test bed/integration lab SoS level requirements in addition to oversight of lower level V&V Suggested additional activities (continued) –Transition plans –Ensure communications between various SoS orgs –Provide logistics, support centers, other -ilities –Security approach (developmental and operational) –Safety plans –Training –Post implementation communications –Disaster recovery –Tradeoffs on level of service reqs –Development of SoS infrastructure? Discussions to continue in this area …. Activities to be analyzed with respect to size and cost drivers….

8. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20058 Workshop Results (continued) Suggested additional issues that may impact LSI effort –Conflict of interest –Sharing of proprietary info –Import/export concerns with international teams –Subcontractor process maturity –Supplier stabilization and synchronization –Diversity of supplier processes, methods, and tools –Synchronization of tools –Level of component independence Discussions to continue in this area …. Issues to be analyzed with respect to size and cost drivers….

9. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 20059 COSOSIMO Survey Overviews LSI activity survey –Description Typical LSI effort was compared to the typical systems engineering effort defined in COSYSMO Comparison was done for the 33 activities defined in EIA632 –12 responses received to date Size Driver and Scale Factor Survey –Description Survey contained a list of current COSYSMO size drivers and scale factors as well as the proposed COSOSIMO size drivers and scale factors Respondents were asked to indicate which parameters applied to LSI management activities (by entering an M) and which parameters applied to LSI technical activities (by entering a T) –12 responses received to date

10. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200510 Survey Results: LSI Activities General observations –Number of EIA/632 activities evaluated: 33 –Highest average rating: 0.67 Lowest average rating: 0.00 –Number of activities with average rating 0.5 or higher: 15 –Number of activities with average rating between 0.3 and 0.5: 9 –Number of activities with average rating 0.3 or lower: 9 -1 (less) 0 (same) 1 (more) Survey Scale:

11. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200511 Survey Results: LSI Activities (continued) Activities with average rating 0.5 or higher –Product supply– Product acquisition –Supplier performance– Process implementation strategy –Technical effort definition – Outcomes management –Information dissemination– Other stakeholder requirements –Transition to use– Effectiveness analysis –Risk analysis– Requirements statements validation –Logical solution validation– End product verification –End product validation Activities with average rating between 0.3 and 0.5 –Schedule and organization– Progress against plans and schedules –Technical reviews– System technical requirements –Implementation– Tradeoff analysis –Acquirer requirements validation– Other stakeholder requirements validation –Enabling product readiness Activities with average rating 0.3 or lower –Technical plans– Work directives – Progress against requirements– Acquirer requirements – Design: Logical solutions– Design: Physical solutions – Design: Specified requirements– System technical requirements validation –Design solution verification

12. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200512 Survey Results: LSI Activities (continued) LSI activity survey conclusions (so far) –All EIA 632 activities are applicable to the LSI effort (there were only two responses that indicated a specific activity was not applicable) –Most EIA 632 activities (75%) require the same or more effort in the LSI environment than in the more traditional SE projects –No survey identified additional activities to be included in this list – however, during workshop discussions, others were identified Interpretation: the other activities identified during discussions were either a clarification of the emphasis of the activity or a set of lower level activities

13. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200513 Survey Results: Size Drivers and Scale Factors Management Size Drivers –Average response “high”:# of independent system component orgs –Average response “medium:# of component systems –Average response “low: # system requirements # system interfaces # operational scenarios Subsystem software size # SoS interface protocols –Average response “N/A”:# algorithms Survey Rating Scale: High Applicability. Medium Applicability, Low Applicability, or Not Applicable (N/A)

14. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200514 Survey Results: Size Drivers and Scale Factors (continued) Technical Size Drivers –Average response “high”:# system interfaces # operational scenarios –Average response “medium”: # system requirements # of component systems # SoS interface protocols –Average response “low”: # algorithms Subsystem software size # of independent system component orgs –Average response “N/A”:(none) Survey Rating Scale: High Applicability. Medium Applicability, Low Applicability, or Not Applicable (N/A)

15. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200515 Survey Results: Size Drivers and Scale Factors (continued) Management Scale Factors –Average response “high”:Stakeholder team cohesion Multi-site coordination Cost/schedule compression –Average response “medium”: Requirements understanding Architecture understanding Migration complexity Technology risk # and diversity of installations/platforms Personnel/team capability Personnel experience/continuity Process capability Tool support Integration risk resolution Integration stability Component readiness Component system maturity/stability Survey Rating Scale: High Applicability Medium Applicability Low Applicability Not Applicable (N/A)

16. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200516 Survey Results: Size Drivers and Scale Factors (continued) Management Scale Factors –Average response “low”: Level of service requirements Documentation Recursive levels in the design Integration simplicity –Average response “N/A”:(none) Survey Rating Scale: High Applicability. Medium Applicability, Low Applicability, or Not Applicable (N/A)

17. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200517 Survey Results: Size Drivers and Scale Factors (continued) Technical Scale Factors –Average response “high”:Requirements understanding Personnel/team capability Personnel experience/continuity –Average response “medium”: everything else…. Survey Rating Scale: High Applicability. Medium Applicability, Low Applicability, or Not Applicable (N/A)

18. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200518 Survey Results: Size Drivers and Scale Factors (continued) Size driver survey conclusions (so far) –Management Size Drivers: Most important appear to be key drivers currently defined for COSOSIMO –Technical Size Drivers: Most of the high and medium applicability drivers are those currently defined for COSYSMO (not COSOSIMO) Exceptions –COSOSIMO “# of system components” included under medium applicability –COSOSIMO “# of SoS level interfaces” included under medium applicability (Possible interpretation: respondents may have thought this related to COSYSMO “# of system interfaces”) –COSYSMO “# of algorithms” thought to be low applicability –Rated low in both categories # of algorithms Sub-system software size –No significant suggestions for additional size drivers

19. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200519 Survey Results: Size Drivers and Scale Factors (continued) Scale factor survey conclusions (so far) –Management scale factors: Most (80%) of the COSYSMO and COSOSIMO scale factors were thought, on average, to be of high or medium applicability None of the listed scale factors had an average rating of “not applicable –Technical scale factors: ALL of the proposed scale factors were thought, on average, to be of either high or medium applicability No significant suggestions for additional scale factors Questions to consider –If COSYSMO is to be used to estimate LSI technical effort, are additional scale factors required? –Or is there embedded overlap in the current set of scale factors? –Is there a better, minimal set of scale factors that are sufficient for both COSYSMO and/or COSOSIMO?

20. COSOSIMO Workshop Results © USC CSE 2005 COCOMO Forum 200520 Going Forward Continue to –Collect survey data –Identify sources of actual SoS LSI effort data Compare COSYSMO outputs with COSOSIMO outputs for selected SoS programs Topics for discussion at this week’s workshop –Impact of July 205 survey results on current COSOSIMO model LSI activities Size drivers and scale factors –Using COSYSMO to estimate LSI technical effort –Management model ideas Overview of some management model parameters Sufficient for LSI management effort? –SoS/FoS WBS ideas For the most current information on COSOSIMO, see http://sunset.usc.edu/cse/pub/research/COSOSIMO/index.html