In order of Appearance: Bob Scheurer Bill Jennings Shawn Simmons How to Start a Successful Program A Panel Discussion for Midwest Gateway INCOSE Chapter May 25, 2011 In order of Appearance: Bob Scheurer Bill Jennings Shawn Simmons

“Successful Programs Begin with Good Planning” …But what is good planning???

Good Planning: Combines the management approach with the technical solution definition/derivation Encompasses the holistic view of planning and engineering for the complete system lifecycle Continuous across pre-contract, post-award planning, and program execution

Good Planning Drives Execution Successful Program Execution is dependent on Clarity of program objectives and requirements Clarity of activities, schedules, deliverables, resources Early identification of needs Viewing requirements in terms of the entire system life cycle Integration of top-down needs and bottom-up solutions Flexibility in response to changing conditions - updates All of these depend on good planning!

But Good Planning has Been Missing… “A root cause of failures on acquisition programs includes …lack of technical planning and oversight ….” Mark Schaeffer, OUSD AT&L. “The Value of Systems Engineering to Program Management.” INCOSE. DoD-directed Studies/Reviews, 2005. “Reason for Weapon Systems Cost Growth is a poor foundations at Milestone B go ahead, including …. artificially low cost estimates and optimistic schedules and assumptions ….” Memo from John J. Young (DoD USD AT&L) to DoD Secretary Gates, “Reasons for Cost Change for Selected Major Defense Acquisition Programs” (Feb 2009).

Could something else be missing, too?: Integration Localized Technical Planning is Not Enough Too Many Disconnected Program Visions Inefficiencies from Plan Redundancies and Inconsistencies Gaps in Planning Details Integrated Technical Planning is the Gateway to Program Success Organized and Disciplined Aligned Planning Processes Instituted at the Very Beginning of an Initiative Describes Transformation from Problem (Requirements) to Design On-Going Updates and Coordination

Integrated Planning Over the Acquisition Lifecycle Defense Acquisition Management System, May 22, 2009 extrapolated The Materiel Development Decision precedes entry into any phase of the acquisition management system Entrance criteria met before entering phase Evolutionary Acquisition or Single Step to Full Capability User Needs Technology Opportunities & Resources A Technology Development B C IOC FOC (Program Initiation) Technological Maturity and Integration Risk Assessment Materiel Solution Analysis Engineering and Manufacturing Development Production and Deployment Operations and Support FRP Decision Review Competitive Prototyping PDR CDR LRIP/IOT&E Pre-Systems Acquisition Systems Acquisition Sustainment Materiel Development Decision Post-PDR Assessment Post-CDR Assessment = Decision Point = Milestone Review = Decision Point

Outcome of Int. Tech. Planning Outcome is an Integrated Technical View 1.0 WBS 2.0 3.0 2.1 2.2 2.3 WBS / PBS SOW Contract SRR SDR PDR. … Lifecycle Phases Work Product 1 Final Update Update Work Product 2 Prelim Final Update Work Product N - - - Work Products IMP/IMS Events Significant Accomplishments Accomplishment Criteria - - CDRLs SEIT PM IPT Flt EE SW Organization RAM SEMP PEP TEP RM Pgm Plans Plans SOO SRR SFR PDR. Lifecycle Phases / Events Standard Deck Deck + Metrics Sig Accom (SA) - Acc Crit (AC) Sig Accom (SA) - Acc Crit (AC) Airframe Wing SW Integration Lab System Integration Lab Program Customized Plan (Basis for Proposal Estimate & Schedule) Detailed / Refined Technical Plan Detailed Functional Work Flows Gate C Gate D ATP IBR Plan Maturation SRR PDR CDR TRR Work Product Repository SFR

Elements of the Integrated Planning Integrated & Traceable Statement of Objectives (SOO) / Statement of Work (SOW) (what to do) Deliverables (evidence of work done) Work Breakdown Structure (WBS) (how work is organized) Program Technical Specification (technical requirements) Specification / Document Tree (decomposition) Program Plans (how work will be done) Integrated Master Plan (IMP) – major events Reviews and Audits Systems Engineering Management Plan (SEMP) (what the supplier has planned) and Systems Engineering Plan (SEP) (what the acquirer/customer has planned) Technical Performance Measures Subordinate Plans (risk mgt, reqts mgt, trade studies, etc.) Test and Evaluation Master Plan (TEMP) Integrated Logistics Plan (for Operations and Support) Use of Suppliers Resource Requirements and Staffing Organization (how people are organized) Metrics for Technical Progress, Technical Maturity, and Quality/Productivity (measurement and feedback mechanism) Risks & Opportunities Inputs to Earned Value Management System Integrated Master Schedule (IMS) with Activity Networks (when) Cost Estimates (resource management)

Engineering and Manufacturing Development Production and Deployment And the Planning Needs to Continue: Plan the Work / Work the Plan … for Each Phase A C CBA B ICD Technology Development Engineering and Manufacturing Development Production and Deployment Materiel Solution Analysis MDD CPD CDD O&S FRP DR PDR CDR Planning Executing

Overview Summary of Integrated Technical Planning Performed Regularly Across the Total System Lifecycle Encompasses a Diverse Set of Views to Form the Complete Planning Picture Receives Guidance for Constructing the Various Plans from: Customer Documents Systems Engineering Process Product Architecture (Product Components via Modeling) IMP/IMS

Getting Started – Identifying the Planning Elements RFP/BAA/ECP CLINs/ CDRLs/SDRLs Customer SEP SOO/SOW Customer Documents Technical Plan Documents Contract SOW Planning Packages PEP/TEPs/ Organizational Breakdown Structure SEMP & Supporting Technical Plans IMP & IMS Contract WBS Methods & Tools Questions Frameworks Process Guides Process Networks Planning Checklists DAG BAA – Broad Agency Announcement BPI – Business Process Instruction BPG – Business Process Guide CLINs – Contract Line Item Numbers DAG – Defense Acquisition Guidebook ECP – Engineering Change Proposal IMP – Integrated Master Plan IMS – Integrated Master Schedule PEP – Program Execution Plan RFP – Request for Proposal SDRL – Supplier Data Requirements List SEMP – Systems Engineering Management Plan SEP – System Engineering Plan SOO – Statement of Objectives SOW – Statement of Work TEP – Team Execution Plan WBS – Work Breakdown Structure CDRL: Contract Data Requirements List Contract Deliverable Requirements List Contract Document Requirements List Contractor Data Requirements List Customer Data Requirements List

Program Specific Risks Technical Plan Documents – The Logical Connectivity Flow and Interfaces Between Planning Elements Technical Plan Documents Contract SOW Planning Packages PEP/TEPs/ Organizational Breakdown Structure SEMP & Supporting Technical Plans IMP & IMS Contract WBS Customer/Acquirer SOO WBS SOW Contractor CSOW CWBS Questions Program Plans OBS  RAM Checklists IMP Processes SEMP Iterate to align & add detail Industry Guidance SA’s and AC’s Program Specific Risks IMS Planning Packages

Summary of Program Planning Building Blocks BAA – Broad Agency Announcement CLINs – Contract Line Item (Numbers) CSOW – Contract Statement of Work CWBS – Contract Work Breakdown Structure DAG – Defense Acquisition Guidance EVM – Earned Value Management PBS – Product Breakdown Structure PEP – Program Execution Plan RFP – Request for Proposal SEMP – Systems Engineering Management Plan SEPMQC – SE Proposal Maturity Quality Checklist SOO – Statement of Objectives SOW – Statement of Work WBS – Work Breakdown Structure Work Packages Work Packages Build-up of Details Financials Financials Operating Plan Operating Plan Budget Budget EVM EVM Labor/ Skills Labor/ Skills Estimates Estimates Contracts Contracts Facilities Facilities Planning Packages Planning Packages Integrated Master Schedule Tiers 3 and Below Integrated Master Schedule Tiers 3 and Below Other Supporting Other Supporting PEP PEP SEMP SEMP Manuf Manuf , Support, Test Plans , Support, Test Plans Iterate these Plans Plans to add detail CSOW CSOW CWBS CWBS Organization Organization Processes Processes Integrated Master Schedule Tiers 1 & 2 Integrated Master Schedule Tiers 1 & 2 Foundation/ Inputs Integrated Master Plan Integrated Master Plan Planning Guidance Planning Guidance Major Major RFP/ BAA/ SOO/ RFP/ BAA/ SOO/ SOW/ SOW/ WBS/ WBS/ Success Success DAG/ Questions/Checklists Questions Questions PBS PBS CLINs/ Referenced Documents CLINs/ Referenced Documents Criteria Criteria Command Media Command Media & Products & Products 14 - Customer - P. Mgmt/ Finance - P. Mgmt & Engr - Engineering 14

Program Definition Documents To identify Customer documents that indicate the program requirements for each program phase, examine the related Program Definition Documents. Example: DoD Instruction 5000.02 ‘Operation of the Defense Acquisition System’ http://www.dtic.mil/whs/directives/ Defense Acquisition Management System Material Solution Analysis, Technology Development, Engineering & Manufacturing Development, etc. phases

Focus On Technical Baselines The Defense Acquisition Guidebook Focuses on the Technical Baselines and Maturing the Technologies/Designs Purpose A B C Materiel Solution Analysis Materiel Development Decision Post-PDR Assessment Engineering and Manufacturing Development Technology Development Operations and Support Production and Deployment Prototyping System Design IBR OTRR AOTR TRA Program Reviews with a technical basis Technical Reviews Post-CDR Assessment FRP Decision Review Sustainment Disposal ITR Baseline ASR Preferred System Concept SRR Prototype System Design Specification SFR Functional Baseline PDR may occur after MS B for less than MDAP Allocated Baseline PDR CDR Initial Product Baseline TRR SVR /FCA /PRR ISR PCA Phases Focus On Technical Baselines Technology Readiness Assessment Technical Reviews Program Reviews Decision Point Milestone Reviews Decision Point if PDR is not conducted before MS B AOTR - Assessment of Operational Test Readiness ASR - Alternative Systems Review CDR - Critical Design Review FCA - Functional Configuration Audit FRP – Full Rate Production IBR - Integrated Baseline Review ISR - In-Service Review ITR - Initial Technical Review OTRR - Operational Test Readiness Review PCA - Physical Configuration Audit PDR - Preliminary Design Review PRR - Production Readiness Review SFR - System Functional Review SRR - System Requirements Review SVR - System Verification Review TRA - Technology Readiness Assessment TRR - Test Readiness Review

Questions Framework Formal Guidance Question-based Framework By Stage Question-Based Construct By Stage Program Reviews JCIDS & DoDI 5000.02 Questions Sub-questions Lower Level Tasks & I/O JFCOM Q # Questions Sub-Questions WHEN Entry, Early, During and Sub-question in Task Format (Use Case names) Inputs Lower Level Tasks Outputs 2 Does the program have documented capability needs at the system level (Capability Description Document or equivalent) ? 1. Documented resolution of conflicts, uncertainties, and ambiguities within the program and with the customer 2. Updated capability needs 2.1 Does the program have documented capability needs at the system level? Entry Collect the documented capability needs 1. Possible sources -- ORD/O&O, CDD, SRD, SPS, SOW, RFP (Sections L&M and SOW) 1. Identify the inputs that may have 2. Search the inputs for capability needs Consolidated list of 2.2 Have these capability needs been reviewed for understanding by the program? Early Review the for understanding by the program 1. Review the needs 2. Identify conflicts 3. Identify ambiguities 4. Identify Description of conflicts, ambiguities and uncertainties 2.3 Have areas of potential misunderstanding or lack of clarity been identified and resolved for consensus within the program and with the customer? Resolve the conflicts, uncertainties with the customer 1. Consolidated list of 2. Description of conflicts, ambiguities and uncertainties 1. Hold discussions with the stakeholders to resolve the 2. Update the DoD Questions IMP Significant Accomplishments & Accomplishment Criteria IMS Tasks by IPT Program Reviews/ Events Program Reviews/ Events Critical Tasks and Work Products Significant Accomplishments Accomplishment Criteria 110 050 TPMs Process and Measures Updated Beyer / Shannon Strand / MAJ Byers 1 SoS/FoS TPMs updated & approved to reflect current program management needs SoS/FoS TPMs updated and approved to reflect current program management monitoring as of SoSFR . Review and concurrence , subject to required changes, of the approved TPMs by appropriate Tier 2 & 3 One Team Management. Required changes implemented in a timely manner. SA AC Event LSI Owner(s) / LSI Manager PMUA / TRADOC Closure Criteria IMS IMP 17 17

Planning information availability by phase Available Information Available Information Concepts of Operation Initial system-level capabilities Operations analysis studies Technology readiness information Lifecycle cost estimates Finalized system-level capabilities Requirements Functional architecture views Physical architecture views Preliminary design Key supplier data Substantiated lifecycle cost estimates Materiel Solution Analysis Engineering & Manufacturing Development Viable set of system-level capabilities & requirements Conceptual architecture views & design concepts Trade studies supporting draft capabilities & requirements Key supplier data Detailed lifecycle cost estimates Completed design Complete product assembled and tested Manufacturing and assembly process in place Fully mature lifecycle cost estimates Technology Development Production & Deployment 18 18 18

Purpose of the Integrated Master Plan Serves as a contractual agreement with the customer on what will be done Documents program events (PEs) Identifies significant accomplishments (SAs) for each event Specify accomplishment criteria (ACs) for each Significant Accomplishment Provides traceability to WBS, SOW, deliverables, risks, and opportunities Provides a framework for planning integration across the program Represents the integrated product development encompassing all functional disciplines Provides a common framework for developing the IMS and detailed team schedules Event-based IMP 19 19

IMS Calendar based Purpose of the IMS Documented schedule of program events Specify tasks to satisfy accomplishment criteria for each significant accomplishment Task traceability to WBS, SOW, deliverables, & program milestones Time-phased tasks and show predecessor and successor relationships Time-phased integrated product development encompassing all program teams Common framework for developing detailed team schedules Calendar based Framework for planning packages and ultimately work packages Facilitate resource needs estimating and use planning Track progress and completion status Basis for determining progress against planned efforts including start/finish/duration Key supplier events 20 20

Translating IMP into IMS content Lesson 3 – Develop an IMS using PMA Translating IMP into IMS content IMP IMS Sequential Flow on Vertical Axis Sequential Flow on Horizontal Axis Tier 1 – Events and tasks Tier 2 – Events and tasks Tier 3 – Events and tasks Tier 4 – Events and tasks Program Events (PEs) SAs and ACs Lower level tasks 21 21

Example: CWBS Dictionary Lesson 3: Relationships between Planning Elements Example: CWBS Dictionary 22 22

Systems Engineering Management Plan (SEMP) Program Overview & Objectives Document Scope Applicable and Reference Documents SE Organizational Integration and Technical Authority Systems Engineering Technical Processes SE Management Processes Integration with Program Management Efforts Transitioning Critical Technologies Integrated Environment for Systems Engineering Additional and Specialty SE Activities Supporting Plans Systems Engineering Management Plan (SEMP) describes the contractor's technical approach and proposed plan for the conduct, management, and control of the integrated systems engineering effort. 23 23

The structure of Integrated Technical Planning Program Events Define the availability of a Capability at a point in time. Accomplishments Represent requirements that enable Capabilities. Criteria Represent Work packages that fulfill Requirements. Narration: Integrated Technical Planning describes the increasing maturing of a product or service through Events or Milestones, Accomplishments, Criteria, and Work Packages. Each Event or Milestone represents the availability of one or more capabilities. The presence of these capabilities is measured by the Accomplishments and their Criteria. Accomplishments are the pre-conditions for the maturity assessments of the product or service at each Event or Milestone. This hierarchy decomposes the System Capabilities into Requirements, Work Packages, and the activities the produce the deliverables. This hierarchy also describes increasing program maturity resulting from the activities contained in the Work Packages. Performance of the work activities, Work Packages, Criteria, Accomplishments, and Events or Milestones is measured in units of “physical percent complete” by connecting Earned Value with Technical Performance Measures. We will learn more about the lower level planning details and plans in course 5. Work Package Work Package Work Package Work Package Work Package Work Package Work Package Work Package 24

Backups

Assessing Completeness Integrated Program Planning Achieve program technical objective Discover and demonstrate related findings of the assessment Apply the Questions Framework to get from the incomplete state to the complete state Correct deficiencies SOW CDRLs Requirements Processes Risks = Executable Questions IMP/IMS WBS Exit EVM Criteria File name.ppt | 1 Assess adequacy of integration of technical plans to ensure completeness Identifying conflicts between plan elements Identifying intermediate work products late or out of sequence Identifying incompatible levels of detail Identifying conflicts in the IMP/IMS

Steps 1 & 2: Determine the intended program state Purpose Step 1: Collect customer provided descriptions of their needs and concepts RFP/ BAA/ RFA etc. JCIDS Functional Needs Analysis (FNA) JCIDS Functional Solutions Analysis (FSA) Initial Capabilities Document (ICD) Analysis of Alternatives (AoA) Capabilities Development Document (CDD) Step 2: determine where your program is in the acquisition life cycle Identify the Program Events (PEs), e.g. ITR, ASR, SRR, etc. Identify technical objectives associated with events 27 27

Questions & Product Flow Technical guidance is available for major program stages “Questions focus” Questions & Product Flow Use Product Flow but add Questions philosophy if trying new methods & technologies A top-Down, question-based construct drives Integrated Planning by establishing a framework for robust, successful program execution & RFP responses

Good Planning is Valuable From Blanchard and Fabrycky, “Systems Engineering and Analysis”, 4th Edition 29 29

Engineering and Manufacturing Development Production and Deployment Integrated Planning Uses the Development Planning, Systems Engineering, and Developmental Test & Evaluation Framework A C CBA B ICD Technology Development Engineering and Manufacturing Development Production and Deployment Materiel Solution Analysis MDD CPD CDD O&S FRP DR PDR CDR Development Development Planning SE DP (Early SE) LCS DT&E DP: Development Planning DT&E: Developmental Test and Evaluation LCS: Life Cycle Sustainment SE: System Engineering CBA: Capabilities Based Assessment CDD: Capability Development Document CPD: Capability Production Document ICD: Initial Capabilities Document MDD: Materiel Development Decision 30 30

The DoD Customer is Trying to Help… Weapon System Acquisition Reform Law (2009) Defense Acquisition Management System - 2008 Systems Engineering Plan Preparation Guide - 2008 …With Law, Policy, and Guidance!!! 31 31