SoS Process, Acquisition, Management Critical Success Factors Workshop Jo Ann Lane, Richard Turner, USC Center for Systems & Software Engineering October 26, 2006

2 Workshop Guidelines  Product: briefing, preferably with notes  Topics should include: –Most critical success factors in each area –Current best practices for addressing them –Areas for further research  Rated 0-10 on value and difficulty of research

3 SoS Process Critical Success Factors: Technical  Technical –Complexity management – including adequate and appropriate recording for ver/val methods –Scope as a variable and negotiable  Identify “good enough” early –Design for graceful degradation (architecture...move) –Plan and budget for evolution –Engineering processes that focus on capabilities/outcomes rather than requirements (research opportunity) –Avoidance of “False precision” - honesty about variance –Ensure that decisions support SoS, not just component system –Effective/efficient baseline management of SoS

4 SoS Process Critical Success Factors: Technical (continued)  Technical –Limit negative impact of changes to baselines out of locus of control (architecture)  Standards, guidelines, conventions – prevent lack of compatability –Budget and Plan for Continuous integration and degrees of continuous V&V –Proactive future interface management (goes along with requirements management) –C3ISR approach  Recon at boundaries  Awareness of environmental changes  Technical ISR of competition

5 SoS Process Critical Success Factors: Management  Management –Plan for evolution –Deliver capability quickly (plan for) –Continuous course correction  Steering verses plan-and-track leadership style –Managing velocity and acceleration of technical work  Rather than position (tracks instead of dots) –Identify “good enough” early –Avoidance of “False precision” - honesty about variance –Continuously “Right size” governance model –Effective/efficient baseline management of SoS –How to identify impact of proposed changes  Understanding business process re-engineering  Initial and revised/future

6 SoS Process Critical Success Factors: Management (continued)  Management –Ability to plan for unintended consequences  How to handle contingency  Add margin/empty spiral development cycles  Accept that not everything can be pre-planned and not fill schedule with known-knowns –Business ISR of competition –Metrics and measurement processes that support SoS objectives  Avoid too much detail or sub-optimization –Progress measured by operational capabilities –Managing compound risks  Compound risk management and mitigation  A “Risk of Risks”

7 SoS Process Critical Success Factors: People/Organizational  People –SoS Stakeholder processes that  Actively Balance Stakeholders  Maintain Stakeholder Involvement and Consensus  Identify, establish, and involve community  Identify the right stakeholders for each decision –How to do without CRACK representatives from all systems and stakeholders (use surrogate ones?)  Processes that still operate with lacking or inadequate levels of representation –Communicating and coordinating up and down supplier chain (robust and speedy) –Incentive structures

8 SoS Process Critical Success Factors: Acquisition and Point of View  Acquisition –Continuously “Right size” governance model –Scope as a variable and negotiable –Plan for evolution –Appropriate contracting vehicles  Point of View –Need to differentiate issues/process needs between a system of systems and addressing a collection of component systems –In SoS context, software properties may need to be recognized as more important than the hardware components –Acknowledge that process is a learning process (goes with scoping, governance models)

9 Best Practices  Technical –Architecture reviews of trade space results –WinWin to prioritize requirements –Spiral incremental commitment model to plan and budget for evolution [Two types of evolution(forseeable evolution, unanticipated changes)] –GQM or PSM –Baseline management automation –Continuous Integration

10 Best Practices (continued)  Management –Incremental Planning –Independent expert reviews –Add margin/empty spiral development cycles –C3ISR approach  Recon at boundaries  Awareness of environmental changes  ISR of competition  Test Driven Development / Scenario Based Testing –Risk identification templates  People –IPPD –Use surrogate representatives –Intranet portal technology –Collaboration tools –Incentive templates  Acquisition –Re-evaluation at the end of each “increment” (include in retrospective)

11 SoS Process Critical Success Factors: Areas for further research A.Engineering processes that focus on capabilities/outcomes rather than requirements B.How to fix acquisition process – current process not used right C.Risk identification templates that can succeed in a SoS context D.How to do without CRACK representatives from all systems and stakeholders E.Meta analysis of acquisition literature F.Methods for managing steering based approach G.Incentive structures H.Rapid change management techniques (acquisition and technical POV) I.Tools for impact analysis J.Fault tree /FEMA analysis to determine probability of success for complex programs K.Modelling and simulation for SoS programs L.Cost and schedule estimation tools M.Identification and evaluation in complex SoS of best practices in this environment Rate 0-10 on value and difficulty of research…

12 Research Priorities

Backup Charts/Additional Information

Success-Related Factors from Presentations  Boehm: Challenges of SoS –Complexity –Dynamism –Emergence –Build-to-spec processes, contracts infeasible  Dalrymple: Success Critical Factors for SoS –Baseline Management (processes) –Organizational Efficiency –Concurrency Management (processes) –Feasibility Assessment  Latimer: –Need case studies to support theory and process dev.

Success-Related Factors from Presentations (continued)  Nielsen: SoS Issues –Constituents (a.k.a. stakeholders) –Purpose –Control –Requirements –Ownership –Boundaries –Visibility  Pyster: Critical Success Factors –Team organization (integrated single team) –Communications –Standards Compliance –Simulation and Layered Architectures

Success-Related Factors from Presentations (continued)  Royce: Critical Success Factors (four patterns of success) –Scope management –Process management –Progress management –Quality management  Turner: Taglines –“Pay me now or pay me later” –“What’s my line?” –“Who’s on first?” –“Truth or Consequences?” –“What, me worry?”

17 Definitions  SoS Process –Risk management  Identify risks process is different?  Mitigation –Needs to deal with emergent stuff –SOS process needs to look like process we preach; ability to change is important –Don't know requirements of emergent systems  Acquisition  Management