System of Systems: What They Are and How to Engineer Them Jo Ann Lane jolane at usc.edu 12 April 2010

Overview SoS context and key challenges SoSE strategies SoS example Incremental commitment and evolution Lean principles Engineering cost estimation Engineering and management artifacts Test and evaluation SoS example Regional Area Crisis Management System (RACRS) Future plans Acknowledgements

What is a “System of Systems”? Very large systems using a framework or architecture to integrate constituent systems (CSs) Exhibits emergent behavior not otherwise achievable by CSs SoS CSs Independently developed and managed New or existing systems in various stages of development/evolution May include a significant number of COTS products Have their own purpose Can dynamically come and go from SoS Typical domains Military/Crisis Response: Dynamic communications infrastructure Business: Enterprise-wide and cross-enterprise integrations Net - Centric SoS Net-Centric Connectivity Laboratory System Imaging Management Pharmacy Patient Telemetry Health Care Network Based on Mark Maier’s SoS definition [Maier, 1998]

SoS Taxonomy Virtual [Maier, 1998] Collaborative [Maier, 1998] Lacks a central management authority and a clear SoS purpose Collaborative [Maier, 1998] CS engineering teams work together, but no overarching SoSE team to guide Acknowledged [Dahmann, 2008] Have recognized objectives, a designated manager, and resources at the SoS level (SoSE team) Directed [Maier, 2008] SoS centrally managed by a government, corporate, or Lead System Integrator (LSI) and built to fulfill specific purposes

Example: SoSE (Directed) Source Selection ● ● ● Valuation Exploration Architecting Develop Operation System A System B System C System x LCO-type Proposal & Feasibility Info Candidate Supplier/ Strategic Partner n Strategic Partner 1 SoS-Level FCR1 DCR1 OCR1 Rebaseline/ Adjustment FCR1 OCR2    OCRx1 FCRB DCRB OCRB1 FCRA DCRA FCRC DCRC OCRC1 OCRx2 OCRx3 OCRx4 OCRx5 OCRC2 OCRB2 OCRA1 5

Example: SoSE (Acknowledged) Source Selection? Existing collaborative SoS? ● ● ● Valuation Exploration Architecting Develop Operation System A System B System C System x Candidate Supplier/ Strategic Partner n Strategic Partner 1 SoS-Level FCR1 DCR1 OCR1 Rebaseline/ Adjustment FCR1 OCR2    OCRx1 FCRB DCRB OCRB1 FCRA DCRA FCRC DCRC OCRC1 OCRx2 OCRx3 OCRx4 OCRx5 OCRC2 OCRB2 OCRA1 6

Example: SoSE (Collaborative) Rebaseline/ Adjustment FCR1 FCR1 DCR1 OCR1 OCR2 SoS-Level Exploration Valuation Architecting Develop Operation    OCRx1 OCRx2 OCRx3 OCRx4 OCRx5 System x    Develop Operation Operation Operation Operation    ● ● ● FCRC DCRC OCRC1 OCRC2 System C    Exploration Valuation Architecting Develop Operation    FCRB DCRB OCRB1 OCRB2 System B    Exploration Valuation Architecting Develop Operation    FCRA DCRA OCRA1 System A    Exploration Valuation Architecting Develop Operation    7

SoSE Activities and Challenges for “Acknowledged” SoS Translating capability objectives Addressing new requirements & options Addressing & solution options Understanding systems & relationships (includes plans) External Environment Developing, evolving and maintaining SoS design/arch Developing & evolving SoS architecture Assessing (actual) performance to capability Orchestrating upgrades to SoS Monitoring & assessing changes SoSE Guidebook view based on interviews and analysis of 18 DoD SoSs in various stages: Communications systems Command and control systems Integrated combat systems Ballistic missile defense systems Intelligence information systems Space-related systems Key challenges Focusing CSs on SoS needs and capabilities Coordinating development of new capabilities across CSs Creating SoS roadmap to guide CS activities Testing SoS capabilities in an asynchronous development environment

SoSE Core Element Description Translating Capability Objectives Starts with an SoS need or new capability Works to understand new capability and alternatives for providing it Understanding Systems and Their Relationships Collects and maintains information about current state of the SoS and its CSs Assessing Performance to Capability Objectives Evaluation of current performance and how performance meets current and future needs Developing/Evolving SoS Architecture Evaluation of existing SoS architecture and identification of alternatives to mitigate limitations and improve performance Monitoring and Assessing Changes Monitoring of CS non-SoS changes Addressing Requirements and Solution Options Evaluation/prioritization of SoS reqs Evaluation of solution options and selection of option Orchestrating Upgrades Oversight activity to monitor progress of the CS SoS capability upgrades and mitigate obstacles

Current System Acquisition Methods Easy to misinterpret as one-size-fits-all V-Model1 Spiral Model2 High level guidance assumes that acquirers have extensive acquisition experience... Without experience, too easy to misinterpret and auger in with disastrous results... 1 http://en.wikipedia.org/wiki/V-Model 2 http://en.wikipedia.org/wiki/Spiral_model

Typical Acquisition Process Aircraft pilot coming off a cargo plane is assigned to manage/ oversee the acquisition of a new aircraft subsystem Excellent understanding of aircraft personnel needs No experience with system/software development Conditioned to plan the flight and fly the plan Tends to interpret V-model diagram sequentially Tends to interpret spiral diagram as one-size-fits-all Leading to Excessive complexity and confusion Missed budgets and schedules

Typical IT Acquisition/Legacy Replacement Process Organization has own IT department to handle maintenance and evolution of existing system New system requires additional resources Search for COTS solution Outsource development of new system Increased risks due to Lack of detailed knowledge about existing IT systems Little/no experience with COTS vendor or outsource supplier

Additional Systems and SoS Future Challenges Rapid pace of change In competition, mission priorities, technology, Commercial Off-the-Shelf (COTS), environment Need incremental development to avoid obsolescence Need concurrent vs. sequential processes Need both prescience and rapid adaptability Software important; humans more important Brownfield vs. Greenfield development Need to provide legacy continuity of service Need to accommodate legacy, OTS constraints Always-on, never-fail systems Need well-controlled, high-assurance processes Need to synchronize and stabilize concurrency Need to balance assurance and agility

Rapid Change Creates a Late Cone of Uncertainty – Need incremental vs Rapid Change Creates a Late Cone of Uncertainty – Need incremental vs. one-shot development Uncertainties in competition, technology, organizations, mission priorities There is Another Cone of Uncertainty: Shorter increments are better Uncertainties in competition and technology evolution and changes in organizations and mission priorities, can wreak havoc with the best of system development programs. In addition, the longer the development cycle, the more likely it will be that several of these uncertainties or changes will occur and make the originally-defined system obsolete. Therefore, planning to develop a system using short increments helps to ensure that early, high priority capabilities can be developed and fielded and changes can be more easily accommodated in future increments. 14

SoSE Synchronization Challenges SoS SE Level* Constituent System n (pre-existing)    Increment m Increment m+1    ● ● ● Constituent System B (pre-existing)    Increment n-1 Increment n Increment n+1    MS A MS B MS C New System A    MSA TD EMD PD O&S    15

SoSE Process Strategies: Incremental Commitment Model for SoS Clear “battle rhythm” for SoS incremental upgrades, driven by prioritized backlog of needed capabilities…. Constituent systems use their own lifecycle upgrade processes to integrate SoS requirements into their own incremental upgrade….

Initial Lean Indicators in SoSE Holistic view across SoS, focus on long term goals Guided by prioritized stakeholder capability needs Continuous learning organization Monitoring and assessing changes Assessing performance to capability objectives SoSE “battle rhythm” Make decisions slowly by consensus Consider all options Continuous process flow through concurrent engineering “Pull” CS engineering knowledge as needed Implement rapidly Level the workload—use systems with available “bandwidth” Minimize waste by Avoiding duplication of engineering efforts at CS level Eliminating none-valuing adding engineering activities Respect and challenge suppliers (CS engineering teams)

Objectives of Lean Enterprise Principles* Minimize waste Be responsive to change Right thing at the right place, at the right time, and in the right quantity Effective relationships (people and organizations) with the value stream Continuously improve processes and products Focus on quality from the beginning * E. Murman, et al., Lean Enterprise Value: Insights from MIT’s Lean Aerospace Initiative. New York , NY: Palgrave, 2002

SoSE Process Strategies: Incorporation of Lean Enterprise Principles SoSE guided by LAI Lean Enterprise 4 Grand Questions Lean Enterprise 4 Grand Questions mapped to DoD SoSE case studies…. Lean Enterprise 4 Grand Questions mapped to SoSE core elements…. SoSE Core Element Lean Enterprise Grand Questions Stakeholder Considerations Holistic Enterprise View Q1: Understand Current Q2: Future Possibilities Q3: Strategies and Tactics for Future Q4: Change Process Translating Capability Objectives X Understanding Systems and Relationships Assessing Performance to Capability Objectives Developing and Evolving an SoS Architecture Monitoring and Assessing Changes Addressing Requirements and Solution Options Orchestrating Upgrades to SoS

SoSE Process Strategies: Engineering Cost Estimation Calculations based on SoS characteristics/size and capability implementation approach using COSYSMO algorithm SoSE effort SoSE Effort Conversion to COSYSMO size units CS 1 SoSE contribution effort Equivalent set of “sea-level” requirements System Capability • • • CS n SoSE contribution effort Applies reuse factors, different cost factors for each engineering organization at each system level, and diseconomy of scale for SoS and CS-level requirements implemented in the same upgrade cycle….

SoSE Process Strategies: Artifacts Artifacts can be characterized as key boundary objects between SoS and constituent system levels…

SoSE Process Strategies: Test and Evaluation Key strategies SoSE team/framework responsible for SoS-level testing Build on CS-level testing Use of test ranges and operational testing Risk and evidence-based approach Focus on SoS capabilities Network Interoperability Emergent behaviors Performance assessment over time Feedback process for fielded SoS

Case Study: Regional Area Crisis Response SoS (RACRS) 23

Future SoSE Research Plans Conduct deeper dives into Lean lens analysis Test and evaluation analysis SoSE artifacts SoS architecture evolution RACRS model development and analysis techniques SoSE cost model Incorporate additional cost factors into COSYSMO to capture additional SoS characteristics and non-traditional SE effort Evaluate impacts of insufficient SE at SoS level on rework Break out SoSE effort Planning Implementation SoS-level testing

Acknowledgements DoD Director, Defense Research and Engineering (DDR&E) Stevens-USC Systems Engineering Research Center (SERC) support SoS case study work that has provided considerable engineering insights into SoSE LAI for their research into lean enterprise concepts Dr. Ricardo Valerdi’s COSYSMO cost model upon which the SoSE cost model is based

Questions Describe a key characteristic of an SoS. List the key SoSE activities for the SoS agile rebaselining team. Briefly describe 3 lean principles embedded in SoSE activities.

References Dahmann, J. and K. Baldwin. 2008. Understanding the current state of US defense systems of systems and the implications for systems engineering. Proceedings of the IEEE Systems Conference, April 7-10, in Montreal, Canada. Department of Defense. 2008. Systems engineering guide for system of systems, version 1.0. Maier, M. 1998. Architecting principles for systems-of-systems. Systems Engineering 1, no. 4: 267-284. Valerdi, R. 2005. Constructive systems engineering cost model. PhD. Dissertation, University of Southern California. Valerdi, R. and M. Wheaton. 2005. ANSI/EIA 632 as a standardized WBS for COSYSMO, AIAA-2005-7373, Proceedings of the AIAA 5th Aviation, Technology, Integration, and Operations Conference, Arlington, Virginia. Wang, G., R. Valerdi, A. Ankrum, C. Millar, and G. Roedler. 2008. COSYSMO reuse extension, Proceedings of the 18th Annual International Symposium of INCOSE, The Netherlands.