1. 1 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Strengthening Systems Engineering Execution at Boeing ….. And Meeting Customer Needs Dev A. Banerjee Division Director – Systems and Flight Engineering (SAFE) Boeing St. Louis 6 th Annual Systems Engineering Conference October 23, 2003

2. 2 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Outline Assessment of USAF for better SE –Incentivizing Contractors (Excerpts from Dr. Sambur guidance) –Mapping of USAF SE Elements to PMBP, CMMI and SEPM –Joint DoD/Industry Suggested Actions Strengthening SE Execution at Boeing –Establish Management Commitment –Provide Infrastructure –Monitor and Support Execution

3. 3 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Incentivizing Contractors for Better SE Excerpts from Dr. Sambur Memo “An immediate transformation imperative for all our programs is to focus more attention on the application of Systems Engineering (SE) principles and practices throughout the system life cycle.” “A more robust SE environment can only be achieved through joint cooperative efforts with our contractors.” “ SAF/AQ and all other MDAs will not sign out any future ASPs that lack the necessary and sufficient attention to SE.” “Additionally, we are identifying ways to improve SE throughout the acquisition process, including workforce issues such as education and training,; tools such as policies, instructions, and guidance; and institutions such as the proposed Center of Excellence for SE under the Air Force Institute of Technology (AFIT).” “---the Air Force leadership will take a strong and renewed interest in how our acquisition community addresses SE disciplines and practices.” Marvin Sambur Assistant Secretary of the Air Force (Acquisition) Policy Memo, dated April 09, 2003 USAF Committed to Robust SE Practice for Future Acquisitions

4. 4 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Sambur SE Elements (1) - Examples - Program Management Best Practices CMMI Process AreasCMMI Process Area to SEPM Section Systems Requirements DocumentRequirements Management 2.2 Requirements Development1.2, 1.3, 1.4 Conduct and thoroughness of reviewsUse of Independent ReviewsVerification1.2, 1.5, 1.6 Validation1.5, 1.6 Test and Evaluation Master Plan (TEMP) Provides fully integrated engineering solutionsSystems Integration (Proposed)Product Integration1.5, 2.6 Effective SE interface control and managementInterface Definition / Management (Proposed) Trades Studies (Proposed)Decision Analysis and Resolution1.4 AffordabilityTechnical Solution1.2, 1.3, 1.4, 1.5, 1.6 System Safety PlanSystem Safety Risk Management PlanRisk Management 1.4, 2.5 Configuration Management PlanBaseline ManagementConfiguration Management1.6, 2.6, 2.7 Systems Engineering Management PlanOrganizationOrganizational Process Focus3.1 Organizational Process Performance Organizational Innovation and Deployment3.3 Integrated Project Management (IPPD)2.1, 2.2, 2.3 Integrated Teaming2.1 Life cycle management w/production and retrofit strategic plans Integrated Master PlanIntegrated PlanningProject Planning2.1, 2.2, 2.6 Integrated Master ScheduleIntegrated Schedules Earned Value ManagementProject Monitoring and Control2.2, 2.5 Source Selection PlanSupplier ManagementSupplier Agreement Management2.4 Integrated Supplier Management System Maturity MatrixTechnical Performance MeasurementMeasurement and Analysis2.2 Closed Loop Corrective ActionCausal Analysis and Resolution Contractor Performance Assessment Rating SystemManagement Information SystemQuantitative Project Management2.1 Help Needed All Program Management Best Practices ApplyOrganizational Process Definition3.1 Organizational Training3.2 Program Quality PlanProcess and Product Quality Assurance2.8 Logistics Management PlanProvide SupportOrganizational Environment for Integration3.4 Customer informed of configuration changes and impactsCustomer Communication/Contact PlanGeneral Goals and Processes Program Management Best Practice Suggested new Program Management Best Practice Proposed new PMBP awaiting approval Quality and Mission Assurance Best Practice (1) Source Memo: "Incentivizing Contractors for Better Systems Engineering", dated 6 Jan 2003 from Dr. Marvin Sambur, Assistant Secretary of the Air Force for Acquisition Mapping of Air Force SE Elements to PMBP, CMMI, and SEPM – existing pieces of our SE Infrastructure System(s) Requirements Development

5. 5 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Suggested Actions from the Joint DoD / Industry Meeting on SE – 11 April 2003 SE needs to be considered across the Program Life Cycle Develop case studies on SE, with both good and bad program examples Make use of disciplined assessment process to monitor rate and level of SE implementation Recommend minimum or normal set of SE products or deliverables Create a consistent set of SE measures/metrics Define an appeal process and take corrective action if a program deviates from the minimum level of SE needed for sound engineering practice

6. 6 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 I. Establish Management Commitment II. Provide Infrastructure III. Monitor and Support Execution Command Media VP Engineering guidance to Chief Engineers EIB & PM Council direction to Bus Goals/Metrics Strengthening Systems Engineering Execution at Boeing Oversight by SE HILT - SE review of proposals and programs Disciplined assessment process Metrics: - Monitoring Rate - Compliance Rate Comprehensive SE process assets (processes, tools, templates, etc) Training, SMEs Management tools SE HILT activity across sites Corrective Action

7. 7 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 I. Establish Management Commitment Command Media –“Systems Engineering Process Requirements”- in-work –Systems Engineering Process Documents –Systems Engineering Process Manual Leadership Endorsement / Direction –VP Engineering advisory to site Chief Engineers emphasizing commitment to SE –PM Council endorsement Goal –Initiate effort to define goal/metric addressing SE execution Leadership Advisories and Goals will Reinforce Expectations for SE Implementation across Boeing Leadership Advisories and Goals will Reinforce Expectations for SE Implementation across Boeing

8. 8 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 II. Systems Engineering Infrastructure SE Process Assets, Management Tools –The SEPM outlines detailed requirements for effective SE process –CMMI to provide detailed set of process assets (process documentation, templates, etc) –Program Management Best Practices (PMBPs) address some key aspects of systems engineering –Gaps in Process Assets (e.g. handbooks in specialty areas) to be filled by SE Horizontal Integration Leadership Team (HILT) Training, SMEs –Considerable training in SE process exists throughout Boeing –The SE HILT to consolidate SE Training –CMMI compliant process assets in development SE HILT Leadership, coupled with Site CMMI activities, will ensure a comprehensive SE infrastructure SE HILT Leadership, coupled with Site CMMI activities, will ensure a comprehensive SE infrastructure

9. 9 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 AnaheimCanoga ParkEl SegundoFloridaHouston Huntington & Seal Beach HuntsvilleLong BeachMesa / PhillyPuget SoundSt. LouisWichita Boeing Systems Engineering HILT / Process Action Teams (PATs) Strengthen/Integrate SE Across Boeing Systems Engineering HILT Process Action Teams (PATs) Network Centric Info. & Comm. Systems Engineering AffordabilityR,M&T Human Factors Safety Operations Analysis System of Systems Survivability / Vulnerability

10. 10 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 SE Training Database Internal Domain/ Technical Training Internal Process- Related Training External Training Basic Intermediate Advanced Training Level Source of Training Info & Comm Arch Networks Info Operations Adv Tech Courses SE Processes SoSE Process DoD Sys Acq Process Sys Arch Devel UCLA Ext UC Irvine AFCEA U of KS Available SE Training for Network Centric-Info & Comm (Sample) Training Topic By SE Discipline Typical Guidance SE Specialists and Technical Fellows more in-depth training in a selected domain. IPT Leaders and Program Managers more general training across the SE disciplines. Basic Intermediate Advanced Formal Edu - Graduate - Workshops - Cross Trng SE Leader Exper (yrs) Training Certificate Level Everyone Project Teams; SE Specialists Sr. SEs; IPT Lds; PMs; CEs Candidates BasicInterAdv 4242 641641 7+ 4 2 4-88-1313-18 # of Courses (Processes & Tools) Network Centric - Info & Comm SE Analysis & Control Affordability R, M&T Human System Interface System Safety Ops Analysis System of Systems (SoS) Survivability/ Vulnerability

11. Industry Seminars Cisco - Networking Alta Vista - Search/MetaData Topia-Intelligent Agents Entrust-Security Network Centric Communications Training Program External Training Universities / Associations / Gov. Labs / Agencies / Commercial Companies UCLA Extension–Short Course Programs Communications; Network Technology; Space Mission Analysis Satellite Communications Payloads & Systems Design High Frequency Satellite Communications UC Irvine–Certificate Program Communications & Network Engineering (18 units required) Advanced Communications Systems Engineering (18 units required) Armed Forces Communications & Electronics Association (AFCEA) Website courses: http://www.afcea.org/education/index.htm Aviation & Electronics Schools of America – individual short courses (3 - 6 days) Fiber Optics Systems for RF Telecommunications Network Technologies and Practices Defense Information Systems Agency (DISA) offers COE course University of Kansas-the Information and Telecommunication Technology Center: Lawrence Livermore National Laboratory Software/Commercial Companies offering related technology/software training: Robelle, Sun Microsystems, Borland USA, Object Management Group, etc. Network-Centric Communications Training Program: Topics  Internet  Interoperability  Performance  Protocols/controls  Processing Techniques  Vulnerabilities  Network protection Information Operations  Information assurance  Information Warfare  C4ISR  Information Protection  Processing Techniques  Advanced/evolving technologies  Information/data management   Comm Spectrum  SATCOM  Wireless Comm/ Terrestrial/ Cellular  Laser Com  Low Probability of Intercept  Evolving Technologies  Anti jam Techniques  Performance NetworksArchitectures Analog vs. Digital Comm - In-house training plans focused in 2 areas: domain/technology training and process training Network-Centric Communications Training Program: Internal Process-Related Training SE Process Requirements Management Risk Management Trade Study Methodology System-of-Systems Systems Engineering Process Detailed System-of-Systems Process Systems OO/UML Process OO Modeling in UML OO Analysis and Design SoSE Tool Kit High Level Architecture for Simulation The DoD Systems Acquisition Process System Architecture Development - DII COE C4ISR Architecture Framework Principles and Applications Information & Communication Architectures Satellite Communications Systems Open Architectures Phased Array Antennas Laser Communications Software Architecture RF Subsystem Design Networks Network - Centric Warfare Web Services Information Operations Government Information Technology Thrust Modulation Techniques Working with C4ISR Fundamentals of Digital Signal Processing Time and Frequency Reference Concepts Advanced Technology Courses Advanced Satellite Communications Payloads Satellite Network Architectures Advanced Fiber Optics. Network-Centric Communications Training Program: Internal Domain / Technology-Related Training Available Network Centric Training (Sample) -

12. 12 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 1.Sites provide resources to execute SE and monitor execution. 2.SE HILT provide ‘integration mechanism across Boeing’ Two Key Metrics to be monitored: - “Monitoring Rate” Metric - “Implementation Level” Metric SE Evaluation Templates required for program reviews: - proposal coverage - kickoff planning - program in-process reviews 3. SE HILT analyze metrics with sites; Jointly conduct root cause analyses and develop Corrective Action Plans 4. SE HILT monitor implementation & results of corrective actions III. Monitor and Support Execution on Proposals / Programs Effective monitoring Root cause analysis Implementation of corrective action Effective monitoring Root cause analysis Implementation of corrective action will strengthen the execution of Systems Engineering… consistent with customer desires. will strengthen the execution of Systems Engineering… consistent with customer desires.

13. 13 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Ensure SE Integrity Through Tailored Reviews during Major Program Milestones Ensure SE Integrity Through Tailored Reviews during Major Program Milestones Pre-Concept / Requirements Development Concept Definition Preliminary Definition Detail Definition Production Support Proposal Start Program Start SRRPDRCDR RFP System Life Cycle Conduct In-Progress Reviews on Programs

14. 14 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 - “Monitoring Rate” Metric: Was the SE Process Monitoring Activity conducted? - Measure instances as % of opportunities. Example: At each site, measure the number of proposals reviewed for SE compliance at Gate C as a percentage of the total number of Gate C proposals reviewed by the site. - “Implementation Level” Metric: When the SE Process Monitoring Activity was conducted, what was the level of execution implementation? Example: For each proposal reviewed, evaluate the level of coverage of SE process execution. The Metrics:

15. 15 / April 27, 2015 Systems Engineering Execution - Boeing \MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 Boeing is Committed to SE Execution across the Enterprise