NDIA SoS SE Committee Topics of Interest May 25, 2009

Overview New NDIA SoS SE Committee formed with kickoff meeting in April Committee members submitted s identifying SoS issues which ‘keep them away at night” These ‘sleepless night’ issues were aggregated Members were asked to identify their top 5 issues and the questions they would like to see addressed with respect to these issues This presentation summarized the results

23 Sleepless Night Issues 1Lessons learned from past and on- going SoS programs 2Perspective of the SoS manager for SoS SE 3Defining SoS relationship in terms of an 'SoS Signature' 4Relationship among SE approaches for systems, SoS, FoS 5Difference in SoS definitions 6Developing validated SoS policy 7Relationship between SoS and acquisition process 8Addressing SoS from perspective of a new system acquisition 9Role of requirements in SoS 10SoS architecture 11SoS Integration 12SoS and Complexity 13Role of optimization in SoS SE 14Defining and measuring SoS performance 15SoS test and evaluation 16SoS SE competencies and training 17SoS governance 18Constructive approaches to addressing politics in an SoS 19SoS business models 20Relationship between SoS and net-centric services 21Science based theory to support SE of systems and SoS 22SoS modeling and simulation 23SoS SE Analyses of CBRN

Spreadsheet Soliciting Inputs Spreadsheet was ed to ~60 people indicating an interest in the SoS SE Committee Request for top 5 topics and questions of interest for these topics Received inputs from 14 members

Results 1 Issue 8 (of 12) expressed interest 1 Issues 7 (of 12) expressed interest 1 Issue 6 (of 12) expressed interest 2 Issues 5 (of 12) expressed interest 1 Issues 4 (of 12) expressed interest 2 Issues 3 (of 12) expressed interest 3 Issues 2 (of 12) expressed interest 10 Issues 1 (of 12) expressed interest 4 Issues 0 (of 12) expressed interest

Top Issues SoS Test and Evaluation (8 of 12) Relationship between SoS and acquisition process (7 of 12) Lessons Learned from past and on-going SoS programs (6 of 12) Defining and measuring SoS performance (5 of 12) [related questions were posed under SoS Test and Evaluation] SoS modeling and simulation ( 5 of 12 ) [Already a topic of NDIA M&S Committee] SoS architecture (4 of 12)

Top Issue: SoS Test and Evaluation (8 of 12) 1.How to evaluate SoS Performance metrics in T&E 2.Traditional techniques for T&E of individual systems do not always scale well to SoS. –What are some new constructs for SoS T&E? –How should model and simulation development progress, in step with SoS development, from concept to test and evaluation? 3.How to plan & leverage T&E throughout out SoS L/C? 4.How to assess implications of changes/new capabilities in constituent systems on SoS T&E? 5.Does this premise extend the underlying effectiveness, suitability, and survivability (ESS) test requirements at a system level? 6.What is the relationship of SoS test and evaluation requirements to the conceptual requirements of AEC's MBT&E process? 7.Should and how are SoS test and evaluation requirements defined in the TEMP and SEP? 8.How do you define success for a SoS? In operational measures such as mission success or by other more technical measures? 9.What are the means to determine the weak links and strong links within the SoS and to quantify the attributes that make sub-component strong?"

Relationship between SoS and the acquisition process (4 of 12) 1.What are the barriers/challenges or potential incentives to removing same when acquiring SoS (in non-Directed SoS)? 2.What specific milestones, decisions, or products is the SoS approach expected to inform along the JCIDS and DAS processes? 3.Are there any specific and unique products that the SoS approach is expected to produce and what are the relationships of these products to acquisition lifecycle products? 4.How does the SoS support the acquisition process? 5.What are the differences between acquiring a System and a SoS?

Lessons Learned from past and on-going SoS programs (6 of 12) 1.What is the recommended approach and what are the basic steps to design large integrated SoSs where the systems are independently and separately developed? 2.What is the recommended approach and what are the basic steps to design large integrated network centric SoSs where the systems are independently and separately developed? 3.What are the recommended suite of methods and tools that can be used to design large SoSs including integrated network SoSs? 4.What are best practices and lessons learned for IT-intensive SoS acqusitions? 5.As the Lead Systems Integrator for the FCS Program, how did Boeing organize for SoS management? What lessons did Boeing learn (5 "up" lessons and 5 "down" lessons)? 6.What are the best practices to surface issues with respect to achieving (software) interoperabilty? At which parts of life cycle are they most effective?

Next Steps Note small number of respondents and consider ways to get better input –Discuss with committee at next meeting –Are the top three an ok starting point? Or do we need to solicit more input? Begin addressing top issue: SoS and T&E –Circulate our SoS T&E paper to committee members now Paper raises almost all of the questions note in the input and provides one perspective to focus discussion –Solicit comments on it by 3 June –Make this a discussion topic at next meeting –Invite T&E committee to join us? Include AFEI SOA outbrief at next meeting –Partnership with AFEI (plus input from NSA) –Topic of IT intensive systems and net-centric systems raised in several of the lessons learned topic questions