1. Ideas & Products Enabling Your Success! Dayton Aerospace 1 Why Revitalize Systems Engineering? or How We Got To This Point Today July 21, 2004 Tom Hoog Dayton Aerospace, Inc. © 2004 Dayton Aerospace, Inc.

2. Ideas & Products Enabling Your Success! Dayton Aerospace 2 Systems Engineering Eulogy SE born in the 1930s as one of RCA Broadcasting Services & Standards Military development of Ops Research in the 1940s was instrumental in shaping its youth First formal education attempted at MIT in 1950 SE became the darling of NASA & military systems in the 1960s; processes imposed on prime contractors Its many offspring were Chief Systems Engineers, Directors of Engineering, VPs of Systems Engineering as well as many Systems Engineers It became bloated & unwieldy in late 1960s; became known as AFSCM 375-5 Attended fat farm & reappeared as Mil-Std-499A; grew in stature & acceptance SE became check & balance in massive engineering efforts It suffered an identity crisis when universities taught industrial engineering as systems engineering In the 1980s SE attempted to re-identify itself as Mil-Std-499B Later cashiered out of the military, it found civilian employment as an IEEE standard It wasted away with an illness brought on by rejection of its processes & bastardizing of its techniques & methods Design reviews became re-design meetings SE died a lingering death in the 1990s & was replace by IPTs, Design Teams & Group Gropes SE is survived by & remembered by a cult of old systems engineers who still practice their now meaningless rituals SE was almost 70 years old at the time of its death No services are planned Full text of this eulogy can be found at http://acc.dau.mil/simplify/ev.php?ID=6174_201&ID2=DO_TOPIC

3. Ideas & Products Enabling Your Success! Dayton Aerospace 3 Outline Background DoD Transformation –5000 –JCIDS –Scope of SE Systems Engineering Revitalization Initiatives –USD (AT&L) Imperatives –SAF/AQ Initiatives –USD (AT&L) Initiatives Health of SE – DAI’s View (Jul 03)

4. Ideas & Products Enabling Your Success! Dayton Aerospace 4 Outline Background DoD Transformation –5000 –JCIDS –Scope of SE Systems Engineering Revitalization Initiatives –USD (AT&L) Imperatives –SAF/AQ Initiatives –USD (AT&L) Initiatives Health of SE – DAI’s View (Jul 03)

5. Ideas & Products Enabling Your Success! Dayton Aerospace 5 Background DoD Decision Support Systems Three parts to DoD Decision Support Systems Requirements Generation –Governed by CJCSI 3170.01 –reissued 24 June 2003 Defense Acquisition –Governed by 5000 series –Reissued 12 May 2003 Financial Management –Governed by multiple sources –Being revised Financial Management System Requirements Generation System Defense Acquisition System

6. Ideas & Products Enabling Your Success! Dayton Aerospace 6 Functional Area Analysis ICD CDD CPD JCIDS Analysis Joint Operating Concepts Joint Functional Concepts Integrated Architectures Strategic Policy Guidance Joint Operations Concepts DOTMLPF Changes CJCSI 3180 Process Functional Solution Analysis DOTMLPF Analysis Materiel Changes CJCSI 3170 process Ideas for Materiel Approaches Analysis of Materiel Approaches Alternative N Alternative 2 Alternative 1 Post Independent Analysis DOTMLPF Change Recommendation Functional Needs Analysis Background

7. Ideas & Products Enabling Your Success! Dayton Aerospace 7 ADAPT INTEGRATE THE “New” 5000 MODEL User Needs & Technology Opportunities Process entry at Milestones A, B, or C Start into Increments where appropriate Increments are interrelated Continues for development life of system Adapted From DoDI 5000.2, 12 May 03 IOC BA Concept Refinement Technology Development System Development & Demonstration Production & Deployment Systems Acquisition Operations & Support C Sustainment FRP Decision Review FOC LRIP/OT&E Design Review Pre-Systems Acquisition (Program Initiation) Concept Decision System Integration System Demonstration TEST AND FIELD DEV DEMO DEVDEMO MATURE ADAPT INTEGRATE TEST AND FIELD B B B C C C Spiral Development - Insert technology into program when it is mature Readiness

8. Ideas & Products Enabling Your Success! Dayton Aerospace 8 SE in Evolutionary Acquisition Environment Multiple concurrent increments lead to increased need for disciplined SE process (vs. single step process) Continuous requirements definition & technology maturation tasks –Complete definition of requirements Plan to mature requirements Traceable from top level operational requirements to design implementation Inclusion of joint requirements (FoS, SoS, net–centric) Allocation of requirements to increments –Mature producibility, affordability, supportability, of technology as well as performance Need to cover both government & industry aspects © 2004 Dayton Aerospace, Inc.

9. Ideas & Products Enabling Your Success! Dayton Aerospace 9 5000 Series Extracts DoDD 5000.1 –Para E 1. 27 Acquisition programs managed through application of SE approach that optimizes total system performance & minimizes total ownership cost. A modular open systems approach shall be used DoDI 5000.2 –Para 3.9.2.2 Sustainment uses SE methodology –Para E 7.2 Human factors is part of SE –Para 7.7 Include ESH in SE DoD 5000.2-R (Interim Defense Acquisition Guidebook) Para C2.3.2 Acquisition strategy includes SE Para C2.8.2 Support strategy is part of SE Para C2.8.5.5 Human factors is part of SE Para C2.8.6 ESH is part of SE Para C6.7.2.7 Critical Program Information is part of SE Para C5.2 18 pages on SE

10. Ideas & Products Enabling Your Success! Dayton Aerospace 10 DoD 5000.2-R, Para C5.2 Defines what SE does Defines the SE activities –Requirements analysis –Functional analysis/allocation –Design synthesis & verification –System analysis & control –Manufacturing & production –Modeling & simulation –Quality –Acquisition logistics –Open systems design –Software management Other Design Considerations –WBS –Performance specifications –Metrics –Insensitive munitions COTS Reliability, availability & maintainability Human systems integration Environment, Safety & operational health Interoperability Survivability Mission assuredness Information assurance Anti-tamper provisions Other Design considerations WBS Performance Specifications Metric system Insensitive munitions Value engineering Precise time & time interval Accessibility Corrosion prevention & control

11. Ideas & Products Enabling Your Success! Dayton Aerospace 11 Scope of Systems Engineering Systems Engineering Adapted from DoDI 5000.2 12 May 2003 Acquisition Requirements Acquisition community participation in early requirements process to assist in understanding

12. Ideas & Products Enabling Your Success! Dayton Aerospace 12 SE Definitions Systems Engineering (SE) A comprehensive, iterative Technical Management (TM) process that includes translating operational requirements into configured systems, integrating the technical inputs of the entire design team, managing interfaces, characterizing and managing technical risk, transitioning technology from the technology base into program specific efforts, and verifying that designs meet operational needs. It is a life cycle activity that demands a concurrent approach to both product and process development. (11 th Glossary of DoD Acqu Terms) Systems Engineering - An inter-disciplinary approach and means to enable the realization of successful systems. (EIA 731.1)

13. Ideas & Products Enabling Your Success! Dayton Aerospace 13 Some Systems Engineering Characteristics Some Systems Engineering Characteristics Encompasses government and contractor efforts –Either may lead –Some joint Iterative process at multiple levels –System level –Subsystem/component level Specific tasks depend on program phase –Pre-Acquisition, Acquisition, Sustainment Specific focus of tasks/trades depends on type of program –New, modification –Prime or support equipment; system or subsystem –Aircraft, space, electronics, armament –Complete system, subsystem, System of Systems / Family of Systems © 2004 Dayton Aerospace, Inc.

14. Ideas & Products Enabling Your Success! Dayton Aerospace 14 System Engineering Focus Shifts over Program Life Cycle System Engineering Focus Shifts over Program Life Cycle Pre-Acquisition –Operational & system performance requirements –Life cycle affordability requirements Acquisition –Design requirements & implementation –Manufacturability, supportability, upgradeability, etc. –System integrity, Sustainment –Supportability –System integrity Modification/Upgrades: Repeat the Cycle © 2004 Dayton Aerospace, Inc.

15. Ideas & Products Enabling Your Success! Dayton Aerospace 15 Requirements Analysis Analyze missions & environments Identify functional requirements Define/refine performance & design constraint requirements Functional Analysis/Allocation Decompose to lower level functions Allocate performance & other limiting requirements to all functional levels Define/refine functional interfaces Define/refine/integrate functional architecture Synthesis Transform architectures Define alternatives system concepts, configuration items & system elements Define/refine physical interfaces Select preferred product & process solutions System Analysis & Control Requirements Loop Design Loop Verification Phase dependent Decision support data System architecture Specifications & baselines Input Customer needs/objectives/ requirements Technology base Outputs from prior phase Program decision requirements Requirements applied through specifications & standards Output Trade-off studies Effectiveness analyses Risk management Configuration management Interface Definition Performance based progress measurements IMP IMS Technical reviews Adapted from Mil-Std-499B (Draft)

16. Ideas & Products Enabling Your Success! Dayton Aerospace 16

17. Ideas & Products Enabling Your Success! Dayton Aerospace 17 Outline Background DoD Transformation –5000 –JCIDS –Scope of SE Systems Engineering Revitalization Initiatives –USD (AT&L) Imperatives –SAF/AQ Initiatives –USD (AT&L) Initiatives Health of SE – DAI’s View (Jul 03)

18. Ideas & Products Enabling Your Success! Dayton Aerospace 18 USD (AT&L) Imperatives “Provide a context within which I can make decisions about individual programs.” “Achieve credibility and effectivness in the acquisition and logistics support processes.” “Help drive good systems engineering practice back into the way we do business.” From Feb 04 OSD (AT&L) - NDIA SE Division Meeting

19. Ideas & Products Enabling Your Success! Dayton Aerospace 19 SAF/AQ SE Revitalization SAF/AQ Dr. Sambur Memo, Apr 9, 2003 –Focus attention on application of SE & elevate these disciplines commensurate with cost & schedule –Develop SE performance incentives –Include status of SE in future program reviews –Identify key SE processes & practices in acquisition documentation –Acquisition Strategy Panel briefings must address significant SE areas SAF/AQ Memo, Increment 2, Jan 7, 2004 –Intended to institutionalize key attributes of an acceptable SE approach –Insert the appropriate language into the following acquisition documents Solicitations Award Fee Plan/Incentive Fee Contract Contracts

20. Ideas & Products Enabling Your Success! Dayton Aerospace 20 USD (AT&L) Initiatives AT&L SE Policy Memo, Feb 20, 2004 –Effective immediately; included in next DoD 5000 revision –All programs, all ACATs shall apply a robust SE approach –SEP for MDA approval –Drive good SE back into the way we do business AT&L SEP Guidance Memo, Mar 30, 2004 –Purpose of SEP is to lay out a plan to guide the technical aspects of an acquisition program –Living document, tailored to the program; roadmap to support program management –No prescribed format; address integration of technical aspects to include: The systems engineering processes to be applied in the program The systems technical baseline approach Event-driven timing, conduct, success criteria, and expected products of technical reviews The integration of systems engineering into the program’s IPTs

21. Ideas & Products Enabling Your Success! Dayton Aerospace 21 Outline Background DoD Transformation –5000 –JCIDS –Scope of SE Systems Engineering Revitalization Initiatives –USD (AT&L) Imperatives –SAF/AQ Initiatives –USD (AT&L) Initiatives Health of SE –Briefing to AT&L (SE OPR) –Quick Study

22. Ideas & Products Enabling Your Success! Dayton Aerospace 22 Health of SE Tasks DoD 5000 & JCIDS reissued in May & June 03 Briefing in Jul 03 DAI personnel experiences –Lack of execution discipline on programs –18 problem areas cited –Common Denominators –List of Issues –Candidate emphasis areas listed DAI asked to provide supporting data Quick Study Report/Briefing in Sep 03

23. Ideas & Products Enabling Your Success! Dayton Aerospace 23 Common Denominators Common Denominators Program Teams are incentivized by cost and/or schedule - not execution of disciplined SE practices Products and Processes are getting out of balance Inadequate depth in applied SE capability © 2004 Dayton Aerospace, Inc.

24. Ideas & Products Enabling Your Success! Dayton Aerospace 24 Issues Lack of SE Execution Incentives Contracts reward cost and schedule performance Product and Process Imbalance Emphasis on speedy delivery of product can De-emphasize “processes” Misapplication of spiral / evolutionary development intent is used to justify program shortcomings – “fix it in the next spiral” Lack of true multifunctional activities in IPTs © 2004 Dayton Aerospace, Inc.

25. Ideas & Products Enabling Your Success! Dayton Aerospace 25 Issues (cont) Inadequate Depth in Applied SE Capability Sound SE execution requires many trade studies Government and industry “de facto” rely on their personnel to pass along “lessons learned” Government and their contractors are losing “critical mass” in some areas Systems Engineering processes are adequately defined BUT not followed, not understood, or are short circuited Advertised CMMI ratings do not assure effective program implementation © 2004 Dayton Aerospace, Inc.

26. Ideas & Products Enabling Your Success! Dayton Aerospace 26 Summary of Observed Problems Observed Problems - Approach Three Categories of Problems –Requirements Management (8) –Systems Engineering Process (7) –Engineering Management (6)

27. Ideas & Products Enabling Your Success! Dayton Aerospace 27 Observed Problems - Approach Based on analysis of many programs Each problem area included –Observed Problem/Issue Statement of problem or issue Program examples –Impact What happened because of problem –Recommendations What can be done to avoid the problem Focus was on systems engineering - recognized that systems engineering execution is not independent of overall program management Objective was to improve overall program performance © 2004 Dayton Aerospace, Inc.

28. Ideas & Products Enabling Your Success! Dayton Aerospace 28 Observed Problems Requirements Management Incomplete Definition of Requirements Requirements Creep Requirements Volatility Lack of a Specification Tree Incomplete/Weak Verification requirements Late requirements Forced Into Software Interface requirements and Interface Management Failure to Implement and Maintain Requirements Traceability Tools © 2004 Dayton Aerospace, Inc.

29. Ideas & Products Enabling Your Success! Dayton Aerospace 29 Observed Problems (cont) Requirements Management Example Inadequate definition of requirements –Statement of operational requirements –Statement of contractual performance requirements Problems –ORD too detailed; includes solution –SRD restates ORD –Contract spec paraphrases SRD; includes design solution –Requirements not refined at lower tiers; incomplete requirements to subcontractors –Incomplete requirements analysis Impacts –Design & implementation cannot proceed effectively –Incomplete design; scrap & rework; expectations not met Recommendations –Enforce requirements reviews at Milestone B Review –Establish process to assure the SRD is complete & accurate –Reinforce requirement that IMP have well defined accomplishments with entrance & exit criteria; SRR must be key event for “opening the gate” to proceed with design –Develop templates, guides checklists to aid in review processes © 2004 Dayton Aerospace, Inc.

30. Ideas & Products Enabling Your Success! Dayton Aerospace 30 Observed Problems (cont) Systems Engineering Process Systems Engineering Process Not Defined on the Program Systems Engineering Process Defined But Not Applied Lack of Flow Down of Systems Engineering Process Requirements to Development Subcontractors Lack of Robust Systems Engineering Applied to Top Level System Design Lack of Timely System Integration and Test (I&T) Planning Inadequate Planning for Obsolescence and Sustainment Design Incorporated into Performance Specifications © 2004 Dayton Aerospace, Inc.

31. Ideas & Products Enabling Your Success! Dayton Aerospace 31 Industry Standard Corporate Process Division Process Program Process Without a Tailored & Documented Systems Engineering Plan, Program Execution Suffers Very Generic Company Approach Top Level Applicable to all Products Applicable to Product lines Applicable to Specific Program Well Documented Rarely Documented

32. Ideas & Products Enabling Your Success! Dayton Aerospace 32 Why a Program Specific SEP Directly tied to rest of program –Consistency across program plans & documentation (SOW, WBS, EVMS, etc.) –Readily correlated to IMP & IMS for execution Establishes technical approach for stakeholders Includes program unique nuances –Established by program requirements –Provides for explanation of unique tailoring © 2004 Dayton Aerospace, Inc.

33. Ideas & Products Enabling Your Success! Dayton Aerospace 33 Observed Problems (cont) Engineering Management Inadequate Schedules and Budget to Accomplish Development Effort Assumed Reuse Not Confirmed Lack of Active Engineering Management of Development Subcontractor Integrated Master Plan (IMP) / Integrated Master Schedule (IMS) Not Adequately Addressed and/or Not Followed on the Program No Single Technical focal Point (Chief Engineer) © 2004 Dayton Aerospace, Inc.

34. Ideas & Products Enabling Your Success! Dayton Aerospace 34 Conclusions Execution of sound SE processes is more important than ever Need forcing functions that drive execution of disciplined SE processes Sound SE execution needs to be key ingredient in contract incentives © 2004 Dayton Aerospace, Inc.

35. Ideas & Products Enabling Your Success! Dayton Aerospace 35 Questions