Systems of Systems: Characteristics and Challenges Barry Boehm, USC Center for Systems & Software Engineering October 25, 2006

2 Overview  Definitions, Examples & Motivation  SoS Characteristics and Challenges  Conclusions

3 What is a “System-of-Systems”?  Very large systems developed by creating a framework or architecture to integrate component systems  SoS component systems independently developed and managed –New or existing systems  Some outsourced  Some externally evolving –Have their own purpose –Can dynamically come and go from SoS  SoS exhibits emergent behavior not otherwise achievable by component systems  SoS activities often planned and coordinated by a Lead System Integrator (LSI)  Typical domains –Business: Enterprise-wide and cross-enterprise integration to support core business enterprise operations across functional and geographical areas –Military: Dynamic communications infrastructure to support operations in a constantly changing, sometimes adversarial, environment

4 What Does an SOS Look Like: Future Combat Systems

6 The Need for Net-Centric Systems of Systems (NCSOS)  Lack of integration among stove-piped systems causes –Unacceptable delays in service –Uncoordinated and conflicting plans –Ineffective or dangerous decisions –Inability to cope with fast-moving events  Increasing NCSOS benefits –See first; understand first; act first –Network-centric operations coordination –Transformation of business/mission potential –Interoperability via Integrated Enterprise Architectures

7 Overview  Definitions, Examples & Motivation  SoS Characteristics and Challenges  Conclusions

8 SoS Characteristics and Challenges - I  Complexity: Size and number of organizations, interfaces, suppliers, coordination groups –Scalability of processes, methods, and tools  Dynamism: Number of changes/month, average time to process change –Rapid change impact analysis, change synthesis, negotiation, development, validation, implementation  Emergence: requirements not pre-specifiable  Build-to-spec processes, contracts infeasible –C2ISR a better metaphor for SoS acquisition than purchasing-agent –Command, control, intelligence, surveillance, reconnaissance

9 Complexity of SoS Solution Spaces  Size: MLOC  Number of external interfaces:  Number of “Coopetitive” suppliers: –Even more separate work locations  Depth of supplier hierarchy: 6-12 levels  Number of coordination groups: –Reviews, changes, risks, requirements, architecture, standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,… –Key personnel spend 60 hours/week in meetings  Unprecedentedness  Emergence  Rapid change Necessarily software-intensive…

10 Average Change Processing Time: 2 SoSs  Average workdays to process changes

11 Acquisition C2ISR Via Spiral OODA Loops Life Cycle Architecture Milestone for Cycle Decide on next-cycle capabilities, architecture upgrades, plans Stable specifications, COTS upgrades Development, integration, V&V, risk management plans Feasibility rationale Act on plans, specifications Keep development stabilized Change impact analysis, preparation for next cycle (mini- OODA loop) Orient with respect to stakeholders priorities, feasibility, risks Risk/Opportunity analysis Business case/mission analysis Prototypes, models, simulations Observe new/updated objectives, constraints, alternatives Usage monitoring Competition, technology, marketplace ISR Operate as current system Accept new system Example: ARPANet/Internet Spiral

12 SoS Characteristics and Challenges - II  COTS/Reuse/Legacy diversity –Many sources of incompatibility, changes –COTS: average 8-10mo/release; unsupported after 3 releases  Multiple missions and stakeholders to support –Increment and change content requires negotiation  Independently evolving systems –Often with “coopetitive” suppliers, interoperators  More time needed for systems definition –Before and after source selection  More time needed for teambuilding, partner coordination, supplier management, change management, integration and test

13 How much Architecting is Enough: COCOMO II Analysis Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution Added Schedule Devoted to Rework (COCOMO II RESL factor) Total % Added Schedule KSLOC 100 KSLOC 10 KSLOC Sweet Spot Sweet Spot Drivers: Rapid Change: leftward High Assurance: rightward

14 SoS Integration and Testing Size Drivers # SoS interface protocols # SoS scenarios # unique component systems Cost Drivers Requirements understanding Architecture maturity Level of service requirements SoS team capability Maturity of LSI processes Tool support Cost/schedule compatibility SoS risk resolution Component system maturity and stability Component system readiness COSOSIMO: I&T Sub-Model LSI I&T Effort

15 Conclusions  Individual SoS attributes are highly challenging –Complexity, dynamism, emergence, uncontrollables, stakeholder diversity –Their combinations are even more challenging  Acquisition management and negotiation skills are at least as important as systems engineering skills –C2ISR a better metaphor for SoS acquisition than purchasing-agent  More time needed for systems definition –Before and after source selection

Backup Charts

17 Complexity of Solution Spaces - Breadth, Depth, and Length Platform N Platform 1 Infra C4ISR Command and Control Situation Assessment Info Fusion Sensor Data Management Sensor Data Integration Sensors Sensor Components : … Width Length Depth DOTMLPF Legend: DOTMLPF Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities C4ISR Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance

18 Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May Acquisition management and staffing 2.Requirements/architecture feasibility 3.Achievable software schedules 4.Supplier integration 5.Adaptation to rapid change 6.Quality factor achievability and tradeoffs 7.Product integration and electronic upgrade 8.Software COTS and reuse feasibility 9.External interoperability 10.Technology readiness

19 $100M $50M Arch. A: Custom many cache processors Arch. B: Modified Client-Server Response Time (sec) Original Spec After Prototyping Available budget Effect of Unvalidated Requirements -15 Month Architecture Rework Delay

20 Effect of Unvalidated Software Schedules  Original goal: 18,000 KSLOC in 7 years –Initial COCOMO II, SEER runs showed infeasibility –Estimated development schedule in months for closely coupled SW with size measured in equivalent KSLOC (thousands of source lines of code): –Months =~ 5 * 3√KSLOC - KSLOC ,000 - Months Solution approach: architect for decoupled parallel development; Schedule As Independent Variable (SAIV) process

21 The SAIV* Process Model 1.Shared vision and expectations management 2.Feature prioritization 3.Schedule range estimation and core-capability determination  Top-priority features achievable within fixed schedule with 90% confidence 4.Architecting for ease of adding or dropping borderline-priority features  And for accommodating past-IOC directions of growth 5.Incremental development  Core capability as increment 1 6.Change and progress monitoring and control  Add or drop borderline-priority features to meet schedule *Schedule As Independent Variable; Feature set as dependent variable. Also works for cost, schedule/cost/quality as independent variable.

22 Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May Acquisition management and staffing 2.Requirements/architecture feasibility 3.Achievable software schedules 4.Supplier integration 5.Adaptation to rapid change 6.Quality factor achievability and tradeoffs 7.Product integration and electronic upgrade 8.Software COTS and reuse feasibility 9.External interoperability 10.Technology readiness

23 COTS Upgrade Synchronization and Obsolescence  Many subcontractors means an increasing number of evolving COTS interfaces  Aggressively-bid subcontracts can deliver obsolete COTS –New COTS released every 8-9 months (GSAW) –COTS unsupported after 3 releases (GSAW) –An actual delivery: 120 COTS; 46% unsupported  Emphasize COTS interoperability in source selection process  Develop contract/subcontract provisions/incentives to ensure –Consistency and interoperability across contractor and subcontractor- delivered COTS software products –Such products are recent-release versions  Develop a management tracking scheme for all COTS software products in all NCSOS software elements  Develop a strategy for synchronizing COTS upgrades

24 Emerging Scalable Spiral Process - For 21st century systems engineering and acquisition  The C4ISR Metaphor for NCSOS Acquisition –Role of OODA loops –Example: Internet Spiral –Example: FCS Win Win Spiral Model  Risk-Driven Scalable Spiral Model –Increment view –Life-cycle view –Role of anchor point milestones  Acquisition management implications  People management implications

25 Risk-Driven Scalable Spiral Model: Increment View

26 Risk-Driven Scalable Spiral Model: Increment View

27 Risk-Driven Scalable Spiral Model: Life Cycle View System Inception System Elaboration Agile DI 2 (OO&D) Rebaselining Plan-Driven DI 1 Construction (A) DI 1 V&V Plan-Driven DI 2 Construction (A) DI 2 V&V System LCASystem, DI 1 LCADI 2 B/L LCA DI 2 LCA Changes LCA: Life Cycle Architecture IOC: Initial Operational Capability OO&D: Observe, Orient and Decide V&V: Verification and Validation DI: Development Increment B/L: Baselined

28 Risk-Driven Scalable Spiral Model: Life Cycle View System Inception System Elaboration Agile DI 2 (OO&D) Rebaselining Plan-Driven DI 1 Construction (A) DI 1 V&V Agile DI 3 (OO&D) Rebaselining Plan-Driven DI 2 Construction (A) DI 2 V&V Agile DI 4 (OO&D) Rebaselining Plan-Driven DI 3 Construction (A) DI 3 V&V DI 1 Trans’n DI 1 Usage DI 2 Trans’n DI 2 Usage DI 3 Trans’n DI 3 Usage System LCA DI 3 LCA System, DI 1 LCADI 2 B/L LCADI 3 B/L LCADI 4 B/L LCA Update DI 2 LCA Changes DI 4 LCA... DI 1 IOC DI 3 IOC DI 2 IOC LCA: Life Cycle Architecture IOC: Initial Operational Capability OO&D: Observe, Orient and Decide V&V: Verification and Validation DI: Development Increment B/L: Baselined

29 LCO (MS A) and LCA (MS B) Anchor Points Pass/Fail Criteria  A system built to the given architecture will –Support the operational concept –Satisfy the requirements –Be faithful to the prototype(s) –Be buildable within the budgets and schedules in the plan –Show a viable business case –Establish key stakeholders’ commitment to proceed LCO: True for at least one architecture LCA: True for the specific life cycle architecture; All major risks resolved or covered by a risk management plan

30 Spiral Feasibility Rationale Deliverable  LCO, LCA reviews not just UML/PowerPoint charts  Need to show evidence of product and process feasibility  Evidence provided by prototypes, production code, benchmarks, models, simulations, analysis –Sizing and cost/schedule model results for process feasibility  Evidence provided in advance to LCO/LCA review team –Key stakeholders, specialty experts  Lack of evidence risks destabilizing the process –Needs coverage by viable risk mitigation plan  Key new progress metric –Feasibility evidence progress vs. plans

31 Acronym Definitions B/LBaselined C4ISRCommand, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance CCDCore Capability Drive-Through COTSCommercial Off the Shelf CRACKCollaborative, Representative, Authorized, Committed, Knowledgeable DIDevelopment Increment DODAFDepartment of Defense Architectural Framework DOTMLPF Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities ERPEnterprise Resource Planning FEAFFederal Enterprise Architectural Framework GSAWGround System Architectures Workshop IESGInternet Engineering Steering Group IETFInternet Engineering Task Force IKIWISII’ll Know It When I See It IOCInitial Operational Capability IPTIntegrated Product Team IRRInception Readiness Review LCALife Cycle Architecture LCOLife Cycle Objectives LSILead System Integrator MLOCMillion Lines of Code MS Milestone NCSOSNet-Centric System of Systems OO&DObserve, Orient, and Decide OODAObserve, Orient, Decide, Act PMPerson-Month/Program Manager PRRProduct Release Review SAIVSchedule As Independent Variable SESystem Engineering SoSSystem of Systems SOWStatement of Work SWSoftware Sys EngrSystem Engineering V&VVerification and Validation WMIWar-fighter machine interface