1. University of Southern California Center for Systems and Software Engineering Incremental Commitment Model (ICM) Guidebook Status, Plans, Key Issues Barry Boehm and Jo Ann Lane, USC-CSSE ICM Guidebook Workshop 3 October 29, 2008 Most charts include explanatory notes

2. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE2 Outline Incremental Commitment Model (ICM) Overview –ICM distinguishing features and implications –Key ICM process views Proposed ICM Guidebook Organization –Executive Summary: Key Features, Implications, Case Study –Core Body: Phases and Activities; Case Study Illustrations –Appendices: Detailed Practices; Common Special Cases Guidebook Status and Plans Key Issues –Policies (5000.2, CP); Standards (IEEE-EIA, ISO) Conclusions; References; Acronyms; Backup charts

3. University of Southern California Center for Systems and Software Engineering ICM Distinguishing Features Not a one-size-fits-all model –Risk-driven activity sequences; common special cases Incremental vs. total resource commitment –System definition: compatible with competitive prototyping –System development: timeboxed increments; concurrent rebaselining; prioritized backlog Concurrent engineering –Of objectives and solutions; product and process; development and evolution; hardware, software, and human factors –Synchronized and stabilized at major milestones Evidence, risk, and commitment-based milestones –Feasibility evidence a first-class deliverable 15 July 2008©USC-CSSE3

4. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE4 2008 SOW: DoD ICM Guidebook Develop a draft Guidebook for next-generation DoD IS&SE based on the ICM –In collaboration with DoD and industry participants Collaborate with selected DoD and industry parties in experimental tailoring of draft Guidebook elements –Would like to explore collaboration opportunities here Hold a series of Guidebook workshops –Mar 19-20 (USC), July 15-17 (DC area), Oct 29-30 (USC) Coordinate Guidebook with other IS&SE initiatives –Systems of systems engineering, systemic analysis, acquisition, education, assessment, research and technology Research is finding better ways… and concepts are being evaluated and refined through OSD-sponsored workshops…

5. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE5 DoD ICM Guidebook Concept Easily digestible top-level ICM guide –Essentials of model and examples ICM guides for key new practices and areas –Evidence-based anchor point commitment milestones Developing Feasibility Evidence Descriptions –Using the ICM for systems of systems –Using the ICM for competitive prototyping –Using the ICM process decision table Detailed phase and activity guide –Draft being adapted from 200-page predecessor Potential follow-ons –Electronic Process Guide (some work underway with IBM) –Courseware (lectures, learning modules)

6. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE6 The Incremental Commitment Life Cycle Process: Overview Stage I: DefinitionStage II: Development and Operations Anchor Point Milestones Synchronize, stabilize concurrency via FEDs Risk patterns determine life cycle process

7. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE7 ICM HSI Levels of Activity for Complex Systems

8. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE8 Anchor Point Feasibility Evidence Description Evidence provided by developer and validated by independent experts that: If the system is built to the specified architecture, it will –Satisfy the requirements: capability, interfaces, level of service, and evolution –Support the operational concept –Be buildable within the budgets and schedules in the plan –Generate a viable return on investment –Generate satisfactory outcomes for all of the success-critical stakeholders All major risks resolved or covered by risk management plans Serves as basis for stakeholders’ commitment to proceed Can be used to strengthen current schedule- or event-based reviews

9. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE9 ICM integrates strengths of current process models But not their weaknesses V-Model: Emphasis on early verification and validation –But not ease of sequential, single-increment interpretation Spiral Model: Risk-driven activity prioritization –But not lack of well-defined in-process milestones RUP and MBASE: Concurrent engineering stabilized by anchor point milestones –But not software orientation Lean Development: Emphasis on value-adding activities –But not repeatable manufacturing orientation Agile Methods: Adaptability to unexpected change –But not software orientation, lack of scalability

10. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE10 Process Model Principles Principles trump diagrams 1.Commitment and accountability - Needed for high assurance systems 2.Success-critical stakeholder satisficing - Needed for multi-owner systems of systems 3.Incremental growth of system definition and stakeholder commitment - Needed for emergent requirements, rapid change 4, 5.Concurrent, iterative system definition and development cycles - Needed for rapid change, rapid OODA loops 6.Risk-based activity levels and anchor point commitment milestones Used by 60-80% of CrossTalk Top-5 projects, 2002-2005

11. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE11 Incremental Commitment in Gambling Total Commitment: Roulette –Put your chips on a number E.g., a value of a key performance parameter –Wait and see if you win or lose Incremental Commitment: Poker, Blackjack –Put some chips in –See your cards, some of others’ cards –Decide whether, how much to commit to proceed

12. University of Southern California Center for Systems and Software Engineering ©USC-CSSE 12 15 July 2008 Scalable remotely controlled operations

13. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE13 Total vs. Incremental Commitment – 4:1 RPV Total Commitment –Agent technology demo and PR: Can do 4:1 for $1B –Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months –PDR: many outstanding risks, undefined interfaces –$800M, 40 months: “halfway” through integration and test –1:1 IOC after $3B, 80 months Incremental Commitment [number of competing teams] –$25M, 6 mo. to VCR [4]: may beat 1:2 with agent technology, but not 4:1 –$75M, 8 mo. to ACR [3]: agent technology may do 1:1; some risks –$225M, 10 mo. to DCR [2]: validated architecture, high-risk elements –$675M, 18 mo. to IOC [1]: viable 1:1 capability –1:1 IOC after $1B, 42 months

14. University of Southern California Center for Systems and Software Engineering July 2008 ©USC-CSSE 14 The Incremental Commitment Life Cycle Process: Overview Stage I: DefinitionStage II: Development and Operations Anchor Point Milestones Concurrently engr. OpCon, rqts, arch, plans, prototypes Concurrently engr. Incr.N (ops), N+1 (devel), N+2 (arch)

15. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE15 Different Risk Patterns Yield Different Processes

16. University of Southern California Center for Systems and Software Engineering ©USC-CSSE 16 15 July 2008 Common Risk-Driven Special Cases of the ICM Special CaseExampleSize, Complexity Change Rate % /Month CriticalityNDI SupportOrg, Personnel Capability Key Stage I Activities : Incremental Definition Key Stage II Activities: Incremental Development, Operations Time per Build; per Increment 1Use NDISmall AccountingCompleteAcquire NDIUse NDI 2AgileE-servicesLow1 – 30Low-MedGood; in place Agile-ready Med-high Skip Valuation, Architecting phases Scrum plus agile methods of choice <= 1 day; 2-6 weeks 3Architected AgileBusiness data processing Med1 – 10Med-HighGood; most in place Agile-ready Med-high Combine Valuation, Architecting phases. Complete NDI preparation Architecture-based Scrum of Scrums 2-4 weeks; 2-6 months 4Formal MethodsSecurity kernel or safety-critical LSI chip Low0.3 – 1Extra high NoneStrong formal methods experience Precise formal specificationFormally-based programming language; formal verification 1-5 days; 1-4 weeks 5HW component with embedded SW Multi-sensor control device Low0.3 – 1Med-Very High Good; In place Experienced; med- high Concurrent HW/SW engineering. CDR-level ICM DCR IOC Development, LRIP, FRP. Concurrent Version N+1 engineering SW: 1-5 days; Market-driven 6Indivisible IOCComplete vehicle platform Med – High 0.3 – 1High-Very High Some in placeExperienced; med- high Determine minimum-IOC likely, conservative cost. Add deferrable SW features as risk reserve Drop deferrable features to meet conservative cost. Strong award fee for features not dropped SW: 2-6 weeks; Platform: 6-18 months 7NDI- IntensiveSupply Chain Management Med – High 0.3 – 3Med- Very High NDI-driven architecture NDI-experienced; Med-high Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements/ architecture definition Pro-active NDI evolution influencing, NDI upgrade synchronization SW: 1-4 weeks; System: 6-18 months 8Hybrid agile / plan-driven system C4ISRMed – Very High Mixed parts: 1 – 10 Mixed parts; Med-Very High Mixed parts Full ICM; encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces) Full ICM,three-team incremental development, concurrent V&V, next- increment rebaselining 1-2 months; 9-18 months 9Multi-owner system of systems Net-centric military operations Very HighMixed parts: 1 – 10 Very HighMany NDIs; some in place Related experience, med- high Full ICM; extensive multi-owner team building, negotiation Full ICM; large ongoing system/software engineering effort 2-4 months; 18-24 months 10Family of systems Medical Device Product Line Med – Very High 1 – 3Med – Very High Some in placeRelated experience, med – high Full ICM; Full stakeholder participation in product line scoping. Strong business case Full ICM. Extra resources for first system, version control, multi-stakeholder support 1-2 months; 9-18 months C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review. DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware. IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software

17. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE17 Outline Incremental Commitment Model (ICM) Overview –ICM distinguishing features and implications –Key ICM process views Proposed ICM Guidebook Organization –Executive Summary: Key Features, Implications, Case Study –Core Body: Phases and Activities; Case Study Illustrations –Appendices: Detailed Practices; Common Special Cases Guidebook Status and Plans Key Issues –Policies (5000.2, CP); Standards (IEEE-EIA, ISO) Conclusions; References; Acronyms; Backup charts

18. University of Southern California Center for Systems and Software Engineering Executive Summary Content Motivation and context –Current DoD needs and future challenges –How ICM addresses needs and challenges Key ICM distinguishing features and principles Relationships to DoD-relevant policies and standards –5000 series, competitive prototyping, ISO 15288/12207, others Structure and content of chapters –Phase guidelines follow Firesmith metamodel Goals and Objectives; Entry Conditions; Inputs; Steps; Exit Conditions; Work Products; Pitfalls 15 July 2008©USC-CSSE18

19. University of Southern California Center for Systems and Software Engineering Core Body Content Getting Started: The Exploration Phase –Case study: remotely piloted vehicle control –Common special cases Stage I: Valuation and Foundations Phases –Illustrated by case study –Artifact guidelines: Ops Concept, Requirements, Architecture, Life Cycle Plan, Feasibility Evidence Stage II: Incremental Development and Operations –Concurrent use of Increment N, development of Increment N+1, rebaselining of Increment N+2 –Illustrated by case study –Artifact guidelines: Plans for test phases, quality management, implementation, life cycle support 15 July 2008©USC-CSSE19

20. University of Southern California Center for Systems and Software Engineering Exploration Phase Content 15 July 2008©USC-CSSE20 Exploration

21. University of Southern California Center for Systems and Software Engineering Top-Level VCR/CD Package Operations/ Life Cycle Concept - Top-level system boundary and environment elements - Benefits chain or equivalent - Links initiatives to desired benefits and identifies associated SCSs - Including production and life cycle support SCSs - Representative operational and support scenarios Prototypes (focused on top development and operational risks) Objectives, Constraints, and Priorities - Initial Capabilities Document Leading Solution Alternatives - Top-level physical, logical, capability and behavioral views Life Cycle Plan Key Elements - Top-level phases, capability increments, roles, responsibilities, required resources Feasibility Evidence Description - Evidence of ability to meet objectives within budget and schedule constraints - Evidence of ability to provide desired benefits to stakeholders - Mission effectiveness evidence 15 July 2008©USC-CSSE21

22. University of Southern California Center for Systems and Software Engineering ICM Guidebook Content: Artifacts Stage I Artifacts –Operational Concept Definition –Requirements Definition –Architecture Definition –Life Cycle Plans Systems Engineering, Quality Management, … –Feasibility Evidence Description Stage II Artifacts –Detailed Increment Development Plans –Test and Assurance Plans –Implementation Plans –Life Cycle Support Plans 15 July 2008©USC-CSSE22

23. University of Southern California Center for Systems and Software Engineering

24. University of Southern California Center for Systems and Software Engineering

25. University of Southern California Center for Systems and Software Engineering

26. University of Southern California Center for Systems and Software Engineering ICM Guidebook Content: Appendices Using the ICM for Competitive Prototyping Incremental Commitment Milestones and Feasibility Evidence Preparation Stakeholder Satisficing Techniques and Tools Risk Assessment and Management ICM-Oriented Estimation and Management Metrics Using the ICM for systems of systems Using the ICM process decision table Applying Selected ICM Practices to Ongoing Projects Other special needs as appropriate 15 July 2008©USC-CSSE26

27. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE27 ICM Transition Paths Existing programs may benefit from some ICM principles and practices, but not others Problem programs may find some ICM practices helpful in recovering viability Primary opportunities for incremental adoption of ICM principles and practices –Supplementing traditional requirements and design reviews with development and review of feasibility evidence –Stabilized incremental development and concurrent architecture rebaselining –Using schedule as independent variable and prioritizing features to be delivered –Continuous verification and validation –Using the process decision table

28. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE28 Outline Incremental Commitment Model (ICM) Overview –ICM distinguishing features and implications –Key ICM process views Proposed ICM Guidebook Organization –Executive Summary: Key Features, Implications, Case Study –Core Body: Phases and Activities; Case Study Illustrations –Appendices: Detailed Practices; Common Special Cases Guidebook Status and Plans Key Issues –Policies (5000.2, CP); Standards (IEEE-EIA, ISO) Conclusions; References; Acronyms; Backup charts

29. University of Southern California Center for Systems and Software Engineering Guidebook Status and Plans Concepts worked out for most elements –A few outstanding issues; see below Critical mass of Electronic Process Guide in use –On 16 real-client MS-student team projects Critical mass of content available as PowerPoint charts with notes –See Workshop Materials on this Workshop’s web site –http://csse.usc.edu/csse/event/2008/cocomoicm08/pages/ma terial.htmlhttp://csse.usc.edu/csse/event/2008/cocomoicm08/pages/ma terial.html –Plan to convert Executive Summary into text by end of 2008 Looking for pilot users and authoring collaborators 15 July 2008©USC-CSSE29

30. University of Southern California Center for Systems and Software Engineering Some Outstanding ICM-for-DoD Issues Final versions of 5000 series and related policies Degree of compatibility with ISO standards Sequencing of DoD milestones and source selection Incremental Development vs. Operations and Maintenance Support of competitive prototyping Impacts on funding, EVMS, estimation, measurement 15 July 2008©USC-CSSE30

31. University of Southern California Center for Systems and Software Engineering Conclusions ICM proving robust, versatile on 16 test projects –Architected Agile; Hybrid Agile/Plan-Driven; NDI-Intensive ICM approaches being applied on some major programs –Future Combat Systems, Missile Defense Agency Several issues remain to resolve Critical mass of guidance available for review and experimentation Looking for pilot users and authoring collaborators 15 July 2008©USC-CSSE31

32. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE32 References - I Beck, K., Extreme Programming Explained, Addison Wesley, 1999. Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”, Systems Engineering 9(1), pp. 1-19, 2006. Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004. Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006. Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and Software Engineering,” CrossTalk, October 2007, pp. 4-9. Boehm, B., and Lane, J., “A Process Decision Table for Integrated Systems and Software Engineering,” Proceedings, CSER 2008, April 2008. Boehm, B., “Future Challenges and Rewards for Software Engineers,” DoD Software Tech News, October 2007, pp. 6-12. Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000. Boehm, B., Software Engineering Economics, Prentice Hall, 2000. Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-26, 2001. Carlock, P., and J. Lane, “System of Systems Enterprise Systems Engineering, the Enterprise Architecture Management Framework, and System of Systems Cost Estimation”, 21st International Forum on COCOMO and Systems/Software Cost Modeling, 2006. Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2 nd ed., 1999). Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.http://akss.dau.mil/dag/ Department of Defense (DoD), Instruction 5000.2, Operation of the Defense Acquisition System, May 2003. Department of Defense (DoD), Systems Engineering Plan Preparation Guide, USD(AT&L), 2004. Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System Galorath, D., and Evans, M., Software Sizing, Estimation, and Risk Management, Auerbach, 2006. Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976.

33. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE33 References -II Highsmith, J., Adaptive Software Development, Dorset House, 2000. International Standards Organization, Information Technology Software Life Cycle Processes, ISO/IEC 12207, 1995 ISO, Systems Engineering – System Life Cycle Processes, ISO/IEC 15288, 2002. Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,” Proceedings, ISPA 5, April 1983, pp. 88-92. Krygiel, A., Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33 Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007. Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE Systems, Man, and Cybernetics Conference, 2005. Lientz, B., and Swanson, E.B., Software Maintenance Management, Addison Wesley, 1980. Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC CSSE Technical Report USC-CSSE-2006-623, 2006. Maier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267- 284). Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp. 146-159. Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software Engineering Institute, 2006. Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look, National Academy Press, 2007. Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,” IEEE Trans SW Engr., July 1978, pp. 345-361. Rechtin, E. Systems Architecting, Prentice Hall, 1991. Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC Integrating Systems and Software Engineering Workshop, http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007. http://csse.usc.edu/events/2007/CIIForum/pages/program.html

34. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE34 List of Acronyms B/LBaselined C4ISRCommand, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance CDConcept Development CDRCritical Design Review COTSCommercial Off-the-Shelf DCRDevelopment Commitment Review DIDevelopment Increment DoDDepartment of Defense ECRExploration Commitment Review EVMSEarned Value Management System FCRFoundations Commitment Review FDNFoundations, as in FDN Package FEDFeasibility Evidence Description FMEAFailure Modes and Effects Analysis FRPFull-Rate Production GAOGovernment Accountability Office GUIGraphical User Interface

35. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE35 List of Acronyms (continued) HMIHuman-Machine Interface HSIHuman-System Interface HWHardware ICMIncremental Commitment Model IOCInitial Operational Capability IRRInception Readiness Review IS&SEIntegrating Systems and Software Engineering LCALife Cycle Architecture LCOLife Cycle Objectives LRIPLow-Rate Initial Production MBASEModel-Based Architecting and Software Engineering NDINon-Developmental Item NRCNational Research Council OCOperational Capability OCROperations Commitment Review OO&DObserve, Orient and Decide OODAObserve, Orient, Decide, Act O&MOperations and Maintenance

36. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE36 List of Acronyms (continued) PDRPreliminary Design Review PMProgram Manager PRPublic Relations PRRProduct Release Review RUPRational Unified Process SoSSystem of Systems SoSESystem of Systems Engineering SSESystems and Software Engineering SWSoftware SwESoftware Engineering SysESystems Engineering Sys EngrSystems Engineer S&SESystems and Software Engineering USD (AT&L)Under Secretary of Defense for Acquisition, Technology, and Logistics VCRValidation Commitment Review V&VVerification and Validation WBSWork Breakdown Structure WMIWarfighter-Machine Interface

37. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE37 Backup Charts

38. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE38 ICM Approach to Systems of Systems SoSs provide examples of how systems and software engineering can be better integrated when evolving existing systems to meet new needs Net-centricity and collaboration-intensiveness of SoSs have created more emphasis on integrating hardware, software, and human factors engineering Focus is on –Flexibility and adaptability –Use of creative approaches, experimentation, and tradeoffs –Consideration of non-optimal approaches that are satisfactory to key stakeholders Key focus for SoS engineering is to guide the evolution of constituent systems within SoS to Perform cross-cutting capabilities While supporting existing single-system stakeholders SoS process adaptations have much in common with the ICM –Supports evolution of constituent systems using the appropriate special case for each system –Guides the evolution of the SoS through long range increments that fit with the incremental evolution of the constituent systems –ICM fits well with the OSD SoS SE Guidebook description of SoSE core elements

39. University of Southern California Center for Systems and Software Engineering ©USC-CSSE 39 15 July 2008 Example: SoSE Coordination Challenge

40. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE40 Young Memo: Prototyping and Competition Discover issues before costly SDD phase –Producing detailed designs in SDD –Not solving myriad technical issues Services and Agencies to produce competitive prototypes through Milestone B –Reduce technical risk, validate designs and cost estimates, evaluate manufacturing processes, refine requirements Will reduce time to fielding –And enhance govt.-industry teambuilding, SysE skills, attractiveness to next generation of technologists Applies to all programs requiring USD(AT&L) approval –Should be extended to appropriate programs below ACAT I

41. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE41 Implementing the Young Memo Need way of assuring continuity of funding, but with off-ramps –Prototyping and Competition Plan –Incremental off-ramp milestones Need criteria for assessing feasibility to proceed –Avoid overfocus on prototyping Feasible, scalable technology and management infrastructure Adequate budget, schedule, critical skills Key stakeholders committed to proceed –Shortfalls in evidence are risks, need risk management plans ICM provides desired processes and criteria –Anchor Point milestones and Feasibility Evidence deliverable –Incremental phase entry and exit criteria

42. University of Southern California Center for Systems and Software Engineering ©USC-CSSE 42 15 July 2008 Some of these are root causes of technology immaturity Can address these via evidence-based Milestone B exit criteria Technology Development Strategy Capability Development Document Evidence of affordability, KPP satisfaction, program achievability Milestone B Focus on Technology Maturity Misses Many OSD/AT&L Systemic Root Causes 1 Technical process (35 instances) 6 Lack of appropriate staff (23) - V&V, integration, modeling&sim. 2 Management process (31) 7 Ineffective organization (22) 3 Acquisition practices (26) 8 Ineffective communication (21) 4 Requirements process (25) 9 Program realism (21) 5 Competing priorities (23) 10 Contract structure (20)

43. University of Southern California Center for Systems and Software Engineering ©USC-CSSE 43 15 July 2008 Acquiring Organization’s ICM-Based CP Plan Addresses issues discussed above –Risk-driven prototyping rounds, concurrent definition and development, continuity of support, stakeholder involvement, off-ramps Organized around key management questions –Objectives (why?): concept feasibility, best system solution –Milestones and Schedules (what? when?): Number and timing of competitive rounds; entry and exit criteria, including off-ramps –Responsibilities (who? where?): Success-critical stakeholder roles and responsibilities for activities and artifacts –Approach (how?): Management approach or evaluation guidelines, technical approach or evaluation methods, facilities, tools, and concurrent engineering –Resources (how much?): Necessary resources for acquirers, competitors, evaluators, other stakeholders across full range of prototyping and evaluation rounds –Assumptions (whereas?): Conditions for exercise of off-ramps, rebaselining of priorities and criteria Provides a stable framework for pursuing CP

44. University of Southern California Center for Systems and Software Engineering 15 July 2008©USC-CSSE44 Conclusions: IS&SE with ICM Current DoD SysE guidance seriously inhibits SwE best practices –Largely sequential definition, design, development, integration, and test –Slows agility, ability to turn inside adversaries’ OODA loop –Functional “part-of” vs. layered “served by” product hierarchy –Static vs. dynamic interfaces: messages vs. protocols –One-size-fits-all process guidance inhibits balancing agility and assurance Incremental Commitment Model (ICM) enables better IS&SE –Integrates hardware, software, human factors aspects of SysE –Concurrent exploration of needs, opportunities, solutions –Successfully addresses issues above in commercial practice –Only partially proven in DoD practice (examples: CrossTalk Top-5 projects) –Key practices applied to help major programs (example: Future Combat Systems) –Being adopted by major programs (example: Missile Defense Agency) –Transition paths available for partial adoption

45. University of Southern California Center for Systems and Software Engineering ICM Assumptions and Critical Success Factors The project can establish milestones at which it can synchronize and stabilize its elements –May exclude some classes of systems of systems Ad-hoc, collaborative, some acknowledged –But ICM principles can be applied to tailor variants The project can invest enough up front to generate and evaluate feasibility evidence and address risks The project can acknowledge and manage its risks –Risk-taking essential for many classes of projects Unprecedentedness, emergence, rapid change, immature technology DARPA: 100% success rate is not doing your job Commercial: 40% failure rate acceptable for new ventures 15 July 2008©USC-CSSE45

46. University of Southern California Center for Systems and Software Engineering ICM Assumptions and Critical Success Factors - II The program office has enough expertise to evaluate feasibility evidence and assess risks –If not in-house, need to contract for it Feasibility evidence is a first-class deliverable –Needs plans, EVMS monitoring of progress vs. plans Appropriate contracting mechanisms and incentive structures are available for executing ICM approach –Concurrent stabilized development, change rebaselining, V&V –Collaborative systems of systems –Competitive prototyping Representative examples are needed to clarify concepts –Ideally, success stories with excursions to show pitfalls 15 July 2008©USC-CSSE46

47. University of Southern California Center for Systems and Software Engineering July 2008©USC-CSSE47 Implications for funding, contracting, career paths Incremental vs. total funding –Often with evidence-based competitive downselect No one-size-fits all contracting –Separate instruments for build-to-spec, agile rebaselining, V&V teams With funding and award fees for collaboration, risk management Compatible regulations, specifications, and standards Compatible acquisition corps education and training –Generally, schedule/cost/quality as independent variable Prioritized feature set as dependent variable Multiple career paths –For people good at build-to-spec, agile rebaselining, V&V –For people good at all three Future program managers and chief engineers