Post-MDD: Analysis of Alternatives (AoA)

Post-MDD: Analysis of Alternatives (AoA)

Lesson Objectives Explain how the Analysis of Alternatives (AoA) supports DoD’s three decision support systems Explain statute and policy Recognize the perspective what an ideal AoA provides Recognize some Best Practices Describe an AoA Study Team 2

Statutes and Policies Sections 2366a and 2366b of Title 10, USC
NDAA 2006 Weapons Systems Acquisition Reform Act of 2009 Public Law May 22, NDAA FY09 Director of Cost Assessment and Program Evaluation DoDD May 11, DA&M Operation of the Defense Acquisition System Interim DoDI January 7, USD(AT&L) Joint Capabilities Integration and Development System CJCSI H January 10, 2012 CJCS

A major defense acquisition program must have an AoA*
2366a and 2366b Certification § 2366a. Major defense acquisition programs: certification required before Milestone A (a) Certification—A major defense acquisition program may not receive Milestone A approval…. or otherwise be initiated prior to Milestone B approval…until the Milestone Decision Authority certifies… (4) that an analysis of alternatives has been performed consistent with study guidance developed by the Director of Cost Assessment and Program Evaluation; § 2366b. Major defense acquisition programs: certification required before Milestone B (a) Certification—A major defense acquisition program may not receive Milestone B approval…, until the milestone decision authority— (1) has received a business case analysis and certifies on the basis of the analysis that— (A) the program is affordable when considering the ability of the Department of Defense to accomplish the program's mission using alternative systems;… (3) further certifies that— … (B) the Department of Defense has completed an analysis of alternatives with respect to the program; A major defense acquisition program must have an AoA* Note: U.S. Code, Current through Public Law , enacted July 1, TITLE 10, SUBTITLE A, PART IV, CHAPTER 139 *Excludes current ops rapid acquisitions / MS C and beyond (Also includes MAIS programs)

Example documents for 2366b Business Case Analysis
ARA memo AoA ORD CDD ASR SCP ICE MNS PB TRA report and memo PDR report CDD JROCM Class Justification & Approval Service Full Funding Memos Example for SDB II MS B Certification: Memo for USD(AT&L) from AT&L/ARA: “Submittal of Business Case Analysis (BCA) and Other Substantiating Information to Support the Section 2366b Certification for the Small Diameter Bomb Increment II (SDB II) Program,” dated Aug 4, 2010. ARA: Acquisition Resources & Analysis ORD: Operational Requirements Document ASR: Acquisition Strategy Report ICE: Independent Cost Estimate SCP: Service Cost Position PB: President’s Budget TRA: Technology Readiness Assessment MNS: Mission Need Statement

WASRA “Responsibilities.—
The Director of Cost Assessment and Program Evaluation shall serve as the principal official within the senior management of the Department of Defense for the following: … (4) Formulation of study guidance for analyses of alternatives for major defense acquisition programs and performance of such analyses, as directed by the Secretary of Defense”1 “Study Guidance for Analysis of Alternatives.— The Director of Cost Assessment and Program Evaluation shall take the lead in the development of study guidance for an analysis of alternatives for each joint military requirement for which the Chairman of the Joint Requirements Oversight Council is the validation authority. In developing the guidance, the Director shall solicit the advice of appropriate officials within the Department of Defense and ensure that the guidance requires, at a minimum— full consideration of possible trade-offs among cost, schedule, and performance objectives for each alternative considered; and an assessment of whether or not the joint military requirement can be met in a manner that is consistent with the cost and schedule objectives recommended by the Joint Requirements Oversight Council.”2 1. Para (a), section 101, Title I, Public Law ; 2. ibid., para (d), section 201, Title II

DoDD 5105.84 Policy Requirements
DCAPE is the principal authority to guide and assess AoAs DCAPE ensures that AoAs consider trade-offs among effectiveness, suitability, and life-cycle costs (or total ownership costs, if applicable) of alternatives that satisfy established capability needs among a comprehensive set of potential material solutions DCAPE shall: Develop AoA Study guidance for MDAPs and MAIS programs. Approve AoA Study plans for each MDAP and MAIS program. Evaluate the adequacy of each AoA. Upon completion of an AoA, the DCAPE shall provide the Head of the DoD Component or PSA and to the Milestone Decision Authority an assessment of the AoA, to include a determination on whether the analysis is consistent with the AoA guidance. Note: The text above is a list of direct extracts from DoDD dated May 11, 2012.

DoDI 5000.02 Policy Requirements
The validated ICD and the AoA Study Plan will guide the AoA and Materiel Solution Analysis Phase activity The AoA will inform and be informed by affordability analysis, cost analysis, sustainment considerations, early systems engineering analysis, threat projections, and market research In developing the guidance, the DCAPE solicits the advice of other DoD officials and ensures that the guidance requires, at a minimum: Full consideration of possible tradeoffs among life-cycle cost, schedule, and performance objectives (including mandatory key performance parameters) for each alternative considered An assessment of whether the joint military requirement can be met in a manner consistent with the cost and schedule objectives recommended by the JROC or other requirements validation authority Consideration of affordability analysis results and affordability goals if established by the MDA Note: The text above is a list of direct extracts from Interim DoDI dated November 25, Greater detail is provided in Enclosure 9 of that document.

DoDI 5000.02 Policy Requirements (CONT)
The study plan will be coordinated with the MDA and approved by the DCAPE prior to the Materiel Development Decision. The final AoA shall be provided to the DCAPE not later than 60 calendar days prior to the Milestone A review (or the next decision point or milestone as designated by the MDA) Not later than 15 business days prior to the Milestone A review, DCAPE evaluates the AoA and provides a memorandum to the MDA, with copies to the head of the DoD Component or other organization or principal staff assistant assessing whether the analysis was completed consistent with DCAPE study guidance and the DCAPE-approved study plan. The final AoA will also be provided to and reviewed by the requirements validation authority prior to the Milestone A decisions or release of the request for proposals of the Technology Maturation and Risk Reduction Phase activities. Note: The text above is a list of direct extracts from Interim DoDI dated November 25, Greater detail is provided in Enclosure 9 of that document.

What Does and Ideal AoA Provide?
An ideal AoA provides essential information on the decision space available to make executive decisions Prioritized, quantified, contextualized capability gaps Feasible alternatives that are affordable and mitigate the capability gaps The trades within and across alternatives among cost, schedule, and performance Transparent analysis of recognized quality Sensitivity analysis—how robust are the solutions? AoA provides well-described and understood “options” for the decision-maker

Stand-up a Service Executive Steering Body
AoA Best Practices Stand-up a Service Executive Steering Body Good Practice: Service Executive Steering Body is formed and includes key stakeholders (e.g. requirements, acquisition, resources) Result: Ensures balanced approach needed for success Stakeholder integration throughout process mitigates risk of “stovepiped” analysis and solutions Provides a link to the CAPE-led “Study Advisory Group” which monitors execution of the AoA Poor Practice: Service Executive Steering Body not formed Result: Service more likely to proceed down path unaware of issues that may delay or kill effort SAG: Study Advisory Group—key stakeholders (SES level), AT&L, ASDs, Service reps, chaired by the CAPE (SES level). They review the guidance and monitor the execution of the AoA.

AoA Best Practices (CONT)
Cost and affordability considered upfront Good Practice: Cost and affordability boundaries established early in the analysis Result: Facilitates exploration of viable trade space Analytical resources not expended on infeasible solutions Includes O&S Includes impact on schedule—how long to build within cost limits Poor Practice: Cost and affordability not considered or not addressed until end of AoA Result: Exploration of viable trade space may be minimized or not accomplished Key aspects of viable trade space may be missed because analytical resources are applied toward exploring infeasible solutions Screen out infeasible candidates early in the effort

Transparent Analysis Good Practice: Site visits to centers of analysis and open analyst interaction Result: Facilitates understanding of analysis enabling leadership to focus on the implication of results Fosters sound analysis which “co-opts” CAPE and AT&L Service can address OSD concerns before study time and effort invested Poor Practice: No or limited analyst interaction Result: Understanding of analysis impeded Confidence in analysis diminished Leadership time diverted to analyst reviews Re-work more likely Recent site visits to Army Materiel Systems Analysis Activity proved useful in understanding methodology of on-going AoAs

Obtaining information from contractors / allies Good Practice: Use request for information (RFI) Result: Facilitates discussion of trade space to inform AoA Faster access to information May provide data as input to modeling and simulation Poor Practice: Use request for proposal (RFP) Result: May prejudge AoA Encumbers process with open competition constraints Use of contractor information is legal and desirable but must be well-protected to avoid compromise that puts SESB at risk

AoA Study Team Structure (example)
The study director establishes the study team to plan and execute the AoA. Study team membership is determined by the needs of the AoA; members with appropriate skills are usually drawn from many organizations. Members often include contractors who provide critical skills and resources. The team focuses on defining alternatives, then assessing and comparing their operational effectiveness, life cycle costs and risks. Organizations who typically contribute members to an AoA study team include: Operating Command (OC) • Financial Management/Comptroller (FM) • Manpower and Personnel (A1) • Intelligence, Surveillance, and Reconnaissance (A2) • Air and Space Operations (A3) • Maintenance and Logistics (A4) • Plans and Programs (A5) • Communications and Information (A6) • Installations and Mission Support (A7) • Requirements (A5/A8) • Analysis, Assessments, and Lessons Learned (A9) • Security (A3) • Weather (A3/5) • Engineering (A7) Implementing Command (IC) • AFMC/A3/FM • OAS (AFMC/A9) • Product Centers • Laboratories • Air Logistics Centers (ALCs) • System Program Offices (SPOs) Other AF Organizations • AF/A2 • AF/A5XW • SAF/AQ/FM • AF Cost Analysis Agency (AFCAA) • MAJCOMs • AF Operational Test & Evaluation Center (AFOTEC) • AF/A9 • AF Flight Standards Agency (AFFSA) • Global Cyberspace Integration Center (GCIC) • Air Force Intelligence, Surveillance, Reconnaissance Agency (AFISRA) • AF Global Weather Center (AFGWC) Other DOD Organizations • USA, USN, USMC • Combatant Commanders (COCOMs) • Defense Intelligence Agency (DIA) • Defense Threat Reduction Agency (DTRA) • Defense Logistics Agency (DLA) • National Geospatial Intelligence Agency (NGA) Non-DoD Organizations • Department of Homeland Security (DHS) • Department of State (DoS) • Department of Energy (DoE) • Department of Interior (DoI) • National Aeronautics and Space Administration (NASA) • Contractors (KTRs) • Federal Aeronautics Administration (FAA) • Department of Transportation (DoT) • National Imagery Mapping Agency (NIMA) Oversight/Advisory Organizations • OSD-level integrated product teams (IPTs) • AF Council (AFC) • AF Requirements and Operational Capabilities Council (AFROCC) • Technical Review Group (TRG) • OAS (AF CoE) • OSD/PA&E Office of Aerospace Studies:

References OMB Circular A-94: Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs: DoD Instruction , "Economic Analysis for Decision-making: DoD Product Support BCA Guidebook: DA Economic Analysis Manual: DA Cost Analysis Manual: US Army Cost-Benefit Analysis Guide: US Army Cost-Benefit Analysis Portal (must request access): DON Enterprise IT BCA Template: DON Enterprise IT Abbreviated BCA Template: DON PBL BCA Guidebook: AFI (Economic Analysis): AFI (Business Case Analysis): AFM (Economic Analysis): AFM (BCA Procedures): DAU CLL 040 BCA Tools: DAU CLL 015 Product Support BCA: DAU CLR 151 Analysis of Alternatives: The policy of both the Office of Management and Budget (OMB) and the United States Department of Defense (DoD) is to ensure that at least three feasible alternatives are analyzed prior to making costly investment decisions. OMB Circular A_11_2010 Preparation, Submission, and Execution of the Budget: Section A: Alternatives Analysis An Alternatives Analysis must evaluate the costs and the benefits of at least three alternatives and the status quo. The details of the analysis must be available to OMB upon request

RQM 310 MDD to Milestone A June 9, 2014
Presenter: Tom Fritz June 9, 2014 LPD-17 San Antonio, SBIRS, F-35B, XM-25

Lesson Objectives Determine the key requirements management activities and the role of the ICD in support of the Materiel Development Decision (MDD), during the Materiel Solution Analysis (MSA) phase, and Milestone A. Explain the inputs to the MDD Explain the outputs from MDD -- decisions and approvals Summarize the purpose and major activities that take place during the MSA phase and the links between those activities Explain the Milestone A decisions and approvals

Materiel Development Decision (MDD)
AoA review and Portfolio Assessment Full Rate Production / Full Deployment CDD Validation Joint Concepts President SECDEF CJCS Joint Community Strategic Guidance Disposal Capabilities - Based Assessment Technology Maturation & Risk Reduction Engineering & Manufacturing Development Production & Deployment Operations & Support A MDD Draft CDD ICD AoA report B C CPD Materiel Solution Analysis Sustainment RFP Release MDD is mandatory for all programs including Defense Business Systems For DAB oversight programs (ACAT ID & ACAT IAM) DCAPE presents the AoA Study Guidance AoA lead organization presents DCAPE approved AoA Study Plan Component provides the plan to staff and fund follow-on actions Components use similar process for Component-level programs (ACATs IC, IAC, II, III and IV)

Quick Review: Pre-MDD Activities
ICD validated ICD recommends type of materiel approach(es) preferred Analysis of Alternatives (AoA) AoA Study Guidance & AoA Study Plan approved prior to MDD Sponsor works with Director, Cost Assessment and Program Evaluation (DCAPE) on AoA Study Guidance & AoA Study Plan (JROC Interest programs only – Component process for others) Sponsor works with MDA’s staff to ensure other additional information is there to enable the desired MDD outcomes for entry into the acquisition process – Determines next phase and overall acquisition Model.

Materiel Development Decision (MDD) Inputs and Outputs
AoA review and Portfolio Assessment Full Rate Production / Full Deployment CDD Validation Joint Concepts President SECDEF CJCS Joint Community Strategic Guidance Disposal Capabilities - Based Assessment Technology Maturation & Risk Reduction Engineering & Manufacturing Development Production & Deployment Operations & Support A MDD Draft CDD ICD AoA report B C CPD Materiel Solution Analysis Sustainment RFP Release Milestone Decision Authority Inputs: Validated ICD – Validation Authority Initial Threat Environment Assessment (I, IA) Approved AoA Study Guidance - DCAPE Approved AoA Study Plan – AoA Lead Org Additional Analysis supporting preliminary conclusions to narrow range of alternatives – DoD Component Affordability Analysis: Tentative affordability cost goals and inventory goals to scope AoA Outputs: Entry into the acquisition process MDA’s Acquisition Decision Memorandum Designates lead DoD Component Determines the acquisition phase of entry Identifies initial review milestone Approved AoA Study Guidance and Study Plan (attached)

MDD to Milestone A -- Entry at Materiel Solution Analysis
AoA review and Portfolio Assessment Full Rate Production / Full Deployment CDD Validation RFP Release A B C MDD ICD Draft CDD CDD CPD Materiel Solution Analysis Strategic Guidance Joint Concepts Capabilities - Based Assessment Technology Maturation & Risk Reduction Engineering & Manufacturing Development Production & Deployment Operations & Support President SECDEF CJCS Joint Community AoA report Sustainment Disposal

Materiel Solution Analysis Requirements Management Activities
Contract(s) A Draft CDD Final RFP TMRR Phase B MDD ICD AoA Report AoA and AoA Report Purpose: Conduct analysis and other activities to support a decision on needed product PM appointed; program office established Capability gaps translated into system specific requirements (KPP and KSA) AoA conducted Acquisition strategy (AS), TEMP and other milestone documents prepared Requirements Management Activities Develop: CONOPS update, Operational Mode Summary/Mission Profile (OMS/MP) and Draft CDD – not submitted to JCIDS Help ensure AoA reflects solutions to satisfy the validated ICD Support conduct of AoA (AoA report briefing for FCB) Support writing of: AS, TEMP, SEP, RAM-C Report, Cybersecurity Strategy Support initial estimates for: Affordability Goals and Life Cycle Cost DODI : Regulatory. The CONOPS/OMS/MP is a Component approved acquisition document that is derived from and consistent with the validated/approved capability requirements document. The CONOPS/OMS/MP describes the operational tasks, events, durations, frequency and environment in which the materiel solution is expected to perform each mission and each phase of the mission. The CONOPS/OMS/MP will be provided to the MDA at the specified decision events and normally provided to industry as part of the RFP. JCIDS Manual: Data submission. Updates to CONOPS and/or OMS/MP documentation will be provided with the submission of CDDs and CPDs unless already available on the KM/DS system.

Reliability Analysis Planning, Tracking, and Reporting
PM formulates a comprehensive R&M program to ensure reliability and maintainability requirements are achieved. PM prepares a Reliability, Availability, Maintainability and Cost (RAM-C) Report in support of Milestone A Attached to the Systems Engineering Plan (SEP), updated in support of the Development RFP Release Decision Point, Milestone B, and Milestone C Reliability growth curves included in SEP at MS A, updated in the TEMP beginning at MS B. Programs other than JROC Interest develop RAM-C Reports as determined by DoD Component or PM Current practice? Is RAM-C Report Executive Summary being attached to (or Referenced in) CDD & CPD? See, DoD Reliability, Availability, Maintainability, and Cost Rationale Report Manual, June 1, 2009 and Encl. 3, interim DoDI

Milestone A Component Presents
Acquisition Strategy (AS), business approach, risk assessment, and “should cost management” targets Affordability analysis and proposed affordability goals Quantitative assessment within the prospective portfolio – unit production and sustainment cost Component Cost Estimate (CCE) Evidence of Full funding in FYDP MDA approves materiel solution; signs ADM approving: Acquisition Strategy Release of RFP and source selection for TMRR phase contract Exit criteria required to complete TMRR and enter EMD Additional MDA activity – Title 10, Section 2366a Certification A favorable Milestone A decision does not mean a new acquisition program has been initiated; except that ship programs may be initiated at Milestone A.

2366a Certification at MS A RM interest areas are in blue bold
MEMORANDUM FOR THE RECORD SUBJECT: Milestone A Program Certification As required by Section 2366a of Title 10, United States Code, I have consulted with the Joint Requirements Oversight Council (JROC) on matters related to program requirements and military needs for the (name of program) and certify that: (1) the program fulfills an approved initial capabilities document; (2) the program is being executed by an entity with a relevant core function as identified by the Secretary of Defense; (3) a determination of applicability of core depot-level maintenance and repair capabilities requirements has been made; (4) an analysis of alternatives has been performed consistent with the study guidance developed by the Director of Cost Assessment and Program Evaluation (DCAPE); (5) a cost estimate for the program has been submitted, with the concurrence of the DCAPE, and the level of resources required to develop, procure, and sustain the program is consistent with the priority level assigned by the JROC; and (6) [only include if the system duplicates a capability already provided by an existing system] the duplication provided by this system and (name of existing system) program is necessary and appropriate.

2366a Certification Notification
2366a certification is not submitted to Congress; however, If the projected cost of the program or subprogram, at any time prior to MB B approval, exceeds the cost estimate submitted at the time of certification by at least 25 percent, or the PM determines the time required to achieve initial operational capability is likely to exceed the schedule objective by more than 25 percent, the PM must notify the MDA. The MDA, in consultation with the JROC, shall determine whether the level of resources required to develop and procure the program remains consistent with the priority level assigned by the JROC. The MDA may withdraw the certification concerned or rescind MS A approval if the MDA determines that such action is in the interest of national defense.

Linkage: ICD, AoA, draft CDD, & Contract(s) for TMRR Phase
Materiel Development Decision Milestone A Strategic Direction Joint Concepts CONOPS Feedback from field Cost Estimate & $$ Full Funding in FYDP Final RFP for TMRR Phase Component approved Contract(s) Draft CDD ICD SPECs Preliminary CBA AoA Capability attributes MOEs/ MOPs KPPs/ KSAs TMRR Materiel Solution Analysis Phase 28

DAB Schedule

RQM 310 Planning, Programming, Budgeting and Execution (PPBE) Process and Related Topics I’m __________ and I’ll be discussing PPBE with you today. First, a little about my background……. PPBE is one of the three major decision support systems in the weapon system acquisition business. The other two, JCIDS and the acquisition management system, have already been discussed. Roberta Tomasini Defense Systems Management College Defense Acquisition University July 2015

PPBE Outline PPBE Overview Building Blocks PPBE Process and Schedule
FYDP, MFP, Program Elements PPBE Process and Schedule Resource Allocation Process Overlap Related Topics This is the outline for the PPBE session. There are couple of overview charts… There are a few charts on the PPBE building blocks…..namely, FYDP (the Future Years Defense Program, MFP (Major Force Programs), and program elements. This will be followed by the timing, key players, and key documents in the PPBE process. Lastly, there are charts of the schedule for this PPBE cycle, which is FY10-15.

FM Scope: From Requirement to Capability
Planning, Programming, Budgeting and Execution Life Cycle Cost Annual Funding Cost Analysis Estimate ICE Funding Policies Full Funding (Exceptions) FYDP DPG POM RMD MFP PE BES CAIV CCA Fiscal Environment Incremental Funding POE AoA Congressional Enactment HAC HASC HBC SAC SASC SBC President’s Budget Budget Resolution Authorization & Appropriation Laws Acquisition Program Baseline Operational Concept Force Structure Modernization Operational Capability Readiness Sustainability Feedback Commitment Reprogramming This chart is intended to show how we get from “Requirement” to “Capability” from a Financial Management perspective….. Starting with the lower left cloud, “Operational Concept”, as this is where it all starts for any discipline in acquisition. Operational Concept is the same as JCIDS plus. It includes the capability documents generated through the JCIDS process as well as a capabilities- based analysis and all acquisition programs have an acquisition program baseline (APB), which is a signed document between the program manager and the milestone decision authority in the acquisition chain of command. The agreement is on cost, schedule and performance parameters for a specific program. Operational concept leads to the first major area of Financial Management (FM), “Cost Analysis”. This includes doing a cost estimate, which is the foundation for the rest of the FM process. The second major area of FM is “Funding Policies”. This involves taking the cost estimate and turning it into a budget. Understanding funding policies is an important part of developing the budget. The third major area of FM is “PPBE”. This is the DoD process for allocating resources. We take a budget and submit it up the chain to get buy-in from our service/agency and OSD. This session will focus on this area. The end of the PPBE process is the president’s budget submission to Congress. This leads to the fourth area of FM, “Congressional Enactment”. The end of “Congressional Enactment” is the Authorization and Appropriation laws, which leads to the fifth and final area of FM, “Budget Execution”. Budget execution relates to actual appropriated funds. Now let’s focus on PPBE. Capabilities-Based Assessment Capability Docs Budget Authority Obligation Expenditure Outlay Budget Execution

Three Major DoD Management Systems
Planning, Programming, Budgeting and Execution QDR Joint Capabilities Integration and Development System (JCIDS) Defense Acquisition System -DoD has three Decision-making Support Systems -1. AMS (Acquisition Management System) -- how we acquire weapon systems -- primary document is DoDI -- also referred to as little “a” acquisition -- event driven process 2. JCIDS --determines what is needed -- primary document is CJCSI 3170, which is capabilities-based with more emphasis on jointness -3. PPBE -- resource allocation in DoD – providing warfighter with best mix of forces, equipment and support attainable under fiscal constraints -- increased emphasis on using performance metrics to focus on output, return on investment -- Calendar driven -QDR: the major statement of defense strategy and business policy --MID 913 states that QDR is the single hierarchical link integrating all internal decision processes -JCIDS and AMS are event-driven; PPBE is calendar-driven, backing off from the PB -All three circles form “Big “A” Acquisition

FY 16 PB (cont.) Modernization Accounts (Procurement and RDT&E)
Service Shares (PB 16) Navy/MC 30% Air Force 29% Army % Modernization Accounts (Procurement and RDT&E) Procurement $108B (PB16) vs $90B (PB15) vs $107B (PB10) For the fight 10 years from now RDT&E $70B (PB16) vs $63B (PB15) vs $79B (PB10) For the fight 20 years from now (Acq programs in development) For the fight 30 years from now (Science and Technology efforts) $12B floor

How many phases are in the PPBE process?
PPBE Phases Planning (OSD Policy) Assess capabilities / review threat Develop guidance Programming (OSD CAPE) Turn guidance into achievable, affordable packages Five-year program (Future Years Defense Program) Budgeting (OSD Comptroller) Test for efficient funds execution Scrub budget year Prepare defensible budget Execution Review (concurrent with program/budget review) Develop performance metrics Assess actual output against planned performance Adjust resources to achieve desired performance goals How many phases are in the PPBE process? Brief Overview of PPBE phases: There are three phases: planning, programming and budgeting. The major difference is that we now have a review of program execution/program performance concurrent with the program and budget review (NOT Army’s PPBES) Planning: QDR 2001 shifted the basis of defense planning from a "threat-based" model that dominated thinking in the past to a "capabilities-based" model Capabilities-based model: – Focuses on how an adversary might fight rather than specifically who the adversary might be or where a war might occur. – Recognizes that it is not enough to plan for large conventional wars in distant theaters. Instead, the United States must identify the capabilities required to deter and defeat adversaries who will rely on surprise, deception, and asymmetric warfare to achieve their objectives. Strategic Planning Guidance (SPG)/Guidance for the Development of the Forces (GDF) and Joint Programming Guidance (JPG): Marching Orders for next phase Programming: Components develop Program submissions based on guidance developed in the Planning phase OSD, OMB & Joint Staff review for priority, affordability Info captured in the FYDP Budgeting: Components develop budget submissions OSD(C) reviews budget inputs, with emphasis on funds execution Ultimate aim, to submit a defensible President’s Budget thru OMB to Congress Execution Review Overlays both Program Review and Budget Review processes In past emphasis on input, “how much to spend on each program”; now emphasis is on output, “what are we getting for our money?” Using performance metrics to examine program execution Let’s look at the years reviewed in the PPBE process…

Future Years Defense Program (FYDP)
Computer Database Maintained by OSD CAPE Contains Approved Force Structure and Resources for all Defense Programs Expected to be updated 3 Times this PPBE Cycle: Program Objectives Memorandum (POM) – Jul Budget Estimate Submission (BES) Sep President’s Budget (PB) - Feb Reflects PY, CY, BY, + 4 Out-Years 3 additional years for force structure only Component PPBE databases Navy: PBIS Air Force : ABIDES Army: PROBE 15 16 17 18 Computer database maintained by Dir (PA&E) Summarizes forces, equipment, and resources Comprises one prior year (PY), the current year (CY), the “biennial” budget years (BY1 and BY2), four additional years for resources (POM) and three additional years for force structure only It is a “living” record reflecting the decisions made in the various phases of PPBE Updated two times per year to reflect the POM/BES (Aug/Sep), and President’s Budget (Jan/Feb) submissions Why two budget years? The process was intended to produce a two year budget. However, Congress appropriates one year at a time. So, we have “off-years/odd-years”. Originally, OSD and the Components were expected to “tweak” the 2nd year of the President’s Budget/POM as part of an off year effort APOM by the Air Force Mini-POM by the Army Program Review by the Navy However, there have been changes in this off-year/odd-year process, starting in calendar year 2003. TRANSITION: Let’s take a look at how the FYDP is structured

Future Years Defense Program (FYDP)
DOD APPROPRIATIONS Military Personnel RDT&E Procurement Military Construction Ops & Maintenance Other D E F A O Strategic Forces (1) T I A General Purpose Forces (2) R G H E E Command, Control, Comm, Intell & Space (3) N F N R MAJOR FORCE PROGRAMS Mobility Forces (4) A A O C R V R I Guard & Reserve Forces (5) M Y C E Research and Development (6) Y E S Central Supply & Maintenance (7) Training, Medical, & Other Personnel Activities (8) Teaching Points: Data is divided into 11 “Major Force Programs” (MFP) for DoD internal management uses during the planning and programming phases of PPBE. Data is also divided into five Appropriation categories (plus one catch-all “other” category, which simply represents the various other appropriations that exist -- although we in the acquisition community are not normally very interested in their details) for use by Congress when reviewing budget requests and enacting budget authority. Data is also divided into service or component categories. Important to note here that the data in any single category (for example: RDT&E, Army or Strategic Forces) from any one of the three major divisions (i.e., Appropriation, Component or MFP) can contain data from any or all of the remaining two major divisions (for example: Strategic Forces could contain data from all five appropriations and all four Components). The lowest level of detail in the FYDP database is the program element, which is explained on the next chart….. Administration and Associated Activities (9) DOD COMPONENTS Support of Other Nations (10) Special Operations Forces (11)

Program Elements A - ARMY N - NAVY M - MARINE F - AF D - OSD C - MDA E - DARPA J - JCS S - DLA BB - SOCOM DBD - DFAS PROGRAM ELEMENT (PE): Smallest aggregation of resources normally controlled by OSD - PE NUMBER: Used to track and identify resources; seven digit number followed by an alphabetic suffix PROGRAM 2 ( GENERAL PURPOSE FORCES ) A - Aircraft Engine CIP N - F/A-18 Squadrons F - AMRAAM PROGRAM 3 ( C3, INTEL & Space) A - Joint Command and Control Program (JC2) N - Aerial Common Sensor (ACS) F - NAVSTAR GPS (User Equipment) Program Elements are the lowest level of detail in the FYDP database. Program Elements are a tool OSD uses to track resources by program, system, or function so they can see how much is being spent on them. PEs can be very broad, containing several programs. They also can be very specific, covering a single component of a weapon system. It all depends on the visibility OSD wants to maintain. These are some examples of Program Elements (PEs), use F-22. Seven digit code followed by a letter suffix designating the applicable component or agency. The first two digits indicate the MFP to which the PE belongs. Although a PE is limited to a specific MFP, it is not in most cases, limited to a single Appropriation. All programs and all resources have PEs. The entire FYDP database can be listed by PE. There are a few thousands of them. Most acquisition programs have two program elements: one that begins with “06” for the development effort and one that begins with something other than “06” for the production and O&S effort. Ref: DoD H

PPBE – Planning Phase Lead by USD (P)
YEAR 1 YEAR 2 Elapsed Time: Months FEB/MAR APR/MAY President National Security Council CIA/DIA/JCS/OSD CIA – Central Intelligence Agency COCOM – Combatant Commander CPR – Chairman’s Program Recommendation DIA – Defense Intelligence Agency DPG – Defense Planning Guidance JCS – Joint Chiefs of Staff NDS – National Defense Strategy NMS – National Military Strategy NSS – National Security Strategy OSD – Office of the Secretary of Defense QDR – Quadrennial Defense Review SCMR – Strategic Choices and Management Review USD(P) – Undersecretary of Defense (Policy) Planning Phase focus: Threat vs. Capabilities Update strategy Guidance for programming & budgeting NSS Every 4 Years Guidance for POM/BES submission and OSD Integrated Program/Budget Review OSD NDS QDR DPG Level New slide - Removed reference to SPRs JCS With one exception, this slide depicts the “new” annual Planning phase that is to be used for the PPBE cycle starting in Calendar Year 2010 and which will, ultimately produce the Defense portion of the President’s Budget for FY2012. The one exception to the annual process is the QDR which is conducted in the 1st year of an administration and submitted to Congress in the 2nd year. The DPPG replaces the GDF (Guidance for Development of the Force) and the JPG (Joint Programming Guidance) and is signed by the SECDEF. The FEAs are intended to address major issues for the SECDEF early in the POM/BES review process. For the Calendar Year 2010 process the SECDEF has selected eight major issues. The Director, CAPE is the Executive Secretary for the FEAs and tri-chairs the process with the VCJCS and the USD(P). Issues nominated but not selected for the FEA review process will be addressed by the DEPSECDEF through the Deputy’s Advisory Working Group (DAWG). Level NMS CPR (JCS, COCOMs, SERVICES)

OSD/OMB Budget Hearings
PPBE – Program/Budget Submission/Review Yr 2 JUL SEP OCT NOV DEC Year 3 JAN/FEB DoD “Large Group” BES – Budget Estimate Submission CAPE – Cost Assessment & Prgm Evaluation COCOM – Combatant Commander CPA – Chairman’s Program Assessment DMAG – Deputy’s Management Action Group FYDP – Future Years Defense Program MBI – Major Budget Issues OMB – Office of Management and Budget PB – President’s Budget POM – Program Objectives Memorandum RMD – Resource Management Decision SLRG – Senior Leader Review Group 3-Star Group OSD/ CAPE Issue Resolution POM Issue Teams SECDEF/DEPSECDEF SLRG & DMAG CPA JCS Components/ Defense Agencies Program Review focus: Compliance with DPG RMDs BES USD(C) & OMB Adv Questions/ OSD/OMB Budget Hearings MBI PB FYDP Update FYDP Update This slide shows the FEAs, initiated in the previous chart (Planning Phase) continuing to be conducted into the Integrated Program/Budget Review time period. The issues included in the FEAs are to be resolved through the Large Group (similar in make-up to the Defense Senior Leadership Conference, as listed in DoD D , DoD Senior Governance Councils). The output of the Large Group, as well as that of the DAWG-addressed issues that were not selected by the SECDEF for FEAs, is provided to the SECDEF/DEPSECDEF. All other POM issues are addressed under the existing process led by the CAPE. Asterisks (TBD) indicate that the organizations may be re-named or realigned, or that activities (MBI and reclama process) may or may not be conducted. Asterisks were placed based on feedback from OSD POCs. The results of the review process for both the BES and the POM will be documented, as appropriate in RMDs. It is expected that an RMD will have two annexes – one for program-related decisions and the other for budget-related decisions. DoDD specifically addresses the membership of the Defense Senior Leadership Conference, the Senior Leader Review Group, and the Deputy’s Advisory Working Group. The membership of the “Small Group”, the “Large Group”, and the “Large Group Plus” has been referred to in the media but not codified as of May 2010. Services / PEO / PM Answer / Reclama Budget Review focus: - Pricing Phasing - Funding policies - Budget execution

FY 17–21 Program/Budget Review Schedule
Mar Fiscal Guidance Issued 6-10 Jul Component POM Briefs to 3-Stars 13 Jul Component POM Submissions Due (Line item/PE level of detail) 5 Aug 15* Issue Nominations Due 17 Aug Issue Nomination Disposition 17 Aug – 2 Nov 15 Program Review 8 Sep BES due to OSD 11 Sep Budget Justification Material due to OSD 11 Sep – 23 Nov 15 Budget Review 2 Nov Programmatic RMDs 23 Nov Budget RMDs 18 Dec Budget Lock 1 Feb President’s Budget Released Source: Joint CAPE & USD(C) memo dated 23 Mar 2015; *modified by 14 May 2015 joint memo

Resource Allocation Process Overlap
CY14 CY15 CY16 J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D FY14 Cycle Execution 2nd Yr 3rd Yr FY 14 and prior FY15 Cycle Enactment Execution 2nd Yr 3rd Yr FY 15 FY 15 and prior PB FY16 Cycle Planning Program/Budgeting Enactment Execution 2nd FY 16-20 DPG FY POM FY 16 BES PB FY 16 FY 16 and prior FY17 Cycle Planning Program/Budgeting Enactment Exec FY DPG FY POM FY 17 BES FY 17 FY17 & prior PB FY18 Cycle Planning Program/Budgeting FY DPG FY POM FY 18 BES DPG – Defense Planning Guidance PB – President’s Budget POM – Program Objectives Memorandum BES – Budget Estimate Submission

Related Topics How do you get funding for an AoA? O&M? RDT&E?
Other Affordability Topics: When should you start becoming concerned about affordability? APB trade space (Cost, Schedule, Performance) What performance are you willing to trade off? At what point should we relook the AoA? Should Objective equal Threshold (O=T)? How firm is the threshold value? Cost as an Independent Variable (CAIV) How do you get funding for an AoA? O&M? RDT&E? Who is responsible for the AoA? How long does it take to get an approved capability document? How can it be done faster? How long does it take to get funding for a program? How fast do you really have to have the capability? What do you really need now versus what can wait? PPBE timing adequacy? Time Now versus desired IOC date versus time for each acq phase

Backups

Defense Topline Budget

Better Buying Power 2.0 (Apr 2013)
Achieve Affordable Programs Mandate affordability as a requirement Institute a system of investment planning to derive affordability caps Enforce affordability caps Control Costs Throughout the Product Lifecycle Implement “should cost” based management Eliminate redundancy within warfighter portfolios Institute a system to measure the cost performance of programs and institutions and to assess the effectiveness of acquisition policies Build stronger partnerships with the requirements community to control costs Increase the incorporation of defense exportability features in initial designs Incentivize Productivity & Innovation in Industry and Government Align profitability more tightly with Department goals Employ appropriate contract types Increase use of Fixed Price Incentive contracts in Low Rate Initial Production Better define value in “best value” competitions Only use LPTA when able to clearly define Technical Acceptability Institute a superior supplier incentive program Increase effective use of Performance-Based Logistics Reduce backlog of DCAA Audits without compromising effectiveness Expand programs to leverage industry’s IR&D Eliminate Unproductive Processes and Bureaucracy Reduce frequency of higher headquarters level reviews Re-emphasize AE, PEO and PM responsibility, authority, and accountability Eliminate requirements imposed on industry where costs outweigh benefits (deleted) Reduce cycle times while ensuring sound investment decisions Promote Effective Competition Emphasize competition strategies and creating and maintaining competitive environments Enforce open system architectures and effectively manage technical data rights Increase small business roles and opportunities Use the Technology Development phase for true risk reduction Improve Tradecraft in Acquisition of Services Assign senior managers for acquisition of services Adopt uniform services market segmentation Improve requirements definition/prevent requirements creep Increase small business participation, including through more effective use of market research (combined SB and MR initiatives) Strengthen contract management outside the normal acquisition chain – installations, etc. Expand use of requirements review boards and tripwires Improve the Professionalism of the Total Acquisition Workforce Establish higher standards for key leadership positions Establish stronger professional qualification requirements for all acquisition specialties Increase the recognition of excellence in acquisition management Continue to increase the cost consciousness of the acquisition workforce – change the culture

“Will Cost” vs “Should Cost” USD (AT&L) and USD(C) 22 Apr 11 Memo
Used for programming and budgeting Used for acquisition program baselines (APBs) Used for all reporting requirements external to DoD Should Cost Scrutinize every element of govt and contractor costs 3 ways to develop should cost estimates: Bottoms –Up estimate Determine specific discrete and measurable items Use competitive contracting and contract negotiations to identify should cost savings (old FAR definition)

Resource Management System
PPBE is the Primary Resource Management System for DoD: Articulates strategy Identifies size, structure and equipment for military forces Sets programming priorities Allocates resources Evaluates actual output against planned performance and adjusts resources as appropriate PPBE is the primary resource management system for DoD for all appropriated funding – not unique to weapon system acquisition. Only minor changes since developed by McNamara in 1961 Allows for DoD to review it’s mission/strategies bottom up every two years What do we do in PPBE? Strategy - How are we going to defend the nation? National Goals Forces - What are we going to use to defend the nation? to execute that Strategy? Prioritize - Which ones, which programs will be funded now? Resources - How much will be spent on each program? Output - Where are we getting our best return? There are three phases in PPBE as shown on the next chart.

Senior Leader Review Group
Secretary of Defense Deputy Secretary of Defense Secretary or Under Secretary of the Army Secretary or Under Secretary of the Navy Secretary or Under Secretary of Air Force Chairman Joint Chiefs of Staff Vice Chairman Joint Chiefs of Staff Under Secretary of Defense (AT&L)* Under Secretary of Defense (Policy)* Under Secretary of Defense (Comptroller) /Chief Financial Officer* Under Secretary of Defense (Intelligence)* Under Secretary of Defense (P&R)* Commandant or Assistant Commandant of the Marine Corps Director of Administration and Management Chief or Vice Chief of Staff of the Army Chief or Vice Chief of Naval Operations Chief of Staff or Vice Chief of the Air Force General Counsel ASD (Legislative Affairs) ASD (Networks & Information Integration/Chief Information Officer ASD for Public Affairs Director, Cost Assessment and Program Evaluation Director, Joint Staff Deputy Chief Management Officer Chief, National Guard Bureau This is the membership of the Senior Leader Review Group, chaired by SecDef. * Or Principal Deputy

Defense Appropriations “Colors of Money”
Military Personnel (MILPERS) Active & Reserve Forces Operation & Maintenance (O&M) (civilian Salaries, supplies, spares, fuels, travel, etc…) Environmental Restoration Former Soviet Union Threat Reduction Overseas Humanitarian, Disaster, & Civic Aid Procurement Aircraft Missiles Weapons Weapons & Tracked Combat Vehicles Ammunition Other Procurement Shipbuilding & Conversion Marine Corps Defense wide procurement National Guard & Reserves Research, Development, Test & Evaluation (RDT&E) Basic Research Applied Research Advanced Technology Development Advanced Component Development & Prototypes System Development & Demonstration RDT&E Management Support Operational Systems Development Military Construction (MILCON) Facilities Family Housing Base Realignment & Closure (BRAC) Other Defense Health Program Chemical Agents & Munitions Destruction Drug Interdiction & Counter-Drug Activities Joint Improvised Explosive Device Defeat Fund Rapid Acquisition Fund Office of the Inspector General RDT&E PROC MILPERS O&M MILCON Teaching Point: In the unlikely case the term “colors of money” is not familiar. This also shows a break out of the major appropriations. The different RDT&E budget categories are also referred to as different colors of money by some.

PPBE – Planning Phase President NSS NDS QDR DPG * NMS CPR FEB/MAR
APR/SEP CIA – Central Intelligence Agency COCOM – Combatant Commander CPR – Chairman’s Program Recommendation D, CAPE – Director, Cost Assessment & Program Eval DIA – Defense Intelligence Agency DPG – Defense Planning Guidance JCS – Joint Chiefs of Staff NDS – National Defense Strategy NMS – National Military Strategy NSS – National Security Strategy OSD – Office of the Secretary of Defense QDR – Quadrennial Defense Review USD(P) – Undersecretary of Defense (Policy) VCJCS – Vice Chairman, Joint Chiefs of Staff President National Security Council CIA/DIA/JCS/OSD Planning Phase focus: Threat vs. Capabilities Update strategy Guidance for programming & budgeting NSS Every 4 Years OSD NDS QDR DPG * Level Guidance for POM/BES submission and Integrated Program/Budget Review With one exception, this slide depicts the “new” annual Planning phase that is to be used for the PPBE cycle starting in Calendar Year 2010 and which will, ultimately produce the Defense portion of the President’s Budget for FY2012. The one exception to the annual process is the QDR which is conducted in the 1st year of an administration and submitted to Congress in the 2nd year. The DPPG replaces the GDF (Guidance for Development of the Force) and the JPG (Joint Programming Guidance) and is signed by the SECDEF. The FEAs are intended to address major issues for the SECDEF early in the POM/BES review process. For the Calendar Year 2010 process the SECDEF has selected eight major issues. The Director, CAPE is the Executive Secretary for the FEAs and tri-chairs the process with the VCJCS and the USD(P). Issues nominated but not selected for the FEA review process will be addressed by the DEPSECDEF through the Deputy’s Advisory Working Group (DAWG). JCS * Note: DPG is the current name of this document but is subject to change from one cycle to the next. Other names used previously include Defense Planning & Programming Guidance (DPPG) and Joint Programming Guidance (JPG). Level NMS CPR (JCS, COCOMs, SERVICES)

PPBE – Integrated Program/Budget Review OSD/OMB Budget Hearings
JUL OCT NOV DEC JAN/FEB DoD “Large Group” QDR BES – Budget Estimate Submission CAPE – Cost Assessment & Prgm Evaluation COCOM – Combatant Commander CPA – Chairman’s Program Assessment CPM – Capability Portfolio Manager DMAG – Deputy’s Management Action Group FYDP – Future Years Defense Program MBI – Major Budget Issues OMB – Office of Management and Budget PB – President’s Budget POM – Program Objectives Memorandum RMD – Resource Management Decision SLRG – Senior Leader Review Group 3-Star Group OSD/ CAPE Issue Resolution POM SECDEF/DEPSECDEF SLRG & DMAG CPA CPMs JCS Components/ Defense Agencies Program Review focus: Compliance with DPG RMDs BES USD(C) & OMB Adv Questions/ OSD/OMB Budget Hearings MBI PB This slide shows the FEAs, initiated in the previous chart (Planning Phase) continuing to be conducted into the Integrated Program/Budget Review time period. The issues included in the FEAs are to be resolved through the Large Group (similar in make-up to the Defense Senior Leadership Conference, as listed in DoD D , DoD Senior Governance Councils). The output of the Large Group, as well as that of the DAWG-addressed issues that were not selected by the SECDEF for FEAs, is provided to the SECDEF/DEPSECDEF. All other POM issues are addressed under the existing process led by the CAPE. Asterisks (TBD) indicate that the organizations may be re-named or realigned, or that activities (MBI and reclama process) may or may not be conducted. Asterisks were placed based on feedback from OSD POCs. The results of the review process for both the BES and the POM will be documented, as appropriate in RMDs. It is expected that an RMD will have two annexes – one for program-related decisions and the other for budget-related decisions. DoDD specifically addresses the membership of the Defense Senior Leadership Conference, the Senior Leader Review Group, and the Deputy’s Advisory Working Group. The membership of the “Small Group”, the “Large Group”, and the “Large Group Plus” has been referred to in the media but not codified as of May 2010. FYDP Update FYDP Update Services / PEO / PM Answer / Reclama Budget Review focus: - Pricing Phasing - Funding policies - Budget execution

Affordability Defense Acquisition Guide (DAG), Oct 2012
Chapter Affordability in the DoD Decision Support System JCIDS – Balances cost vs performance in establishing KPPs Milestone A – 10 U.S.C. 2366a – Congressional certification (for MDAPs) MDA certifies to Congress that a cost estimate has been submitted After consultation with JROC With Concurrence from D,CAPE PM must notify MDA between MS A and MS B, if 1. Projected cost of program exceeds the cost estimate by 25% OR 2. The time to reach IOC exceeds the scheduled duration by 25% The DAB considers affordability in the context of the capability portfolio and from a unit cost standpoint Portfolio: Done by lead service in the context of all other weapons systems in the portfolio or mission area Unit cost: Nominally the average unit acquisition cost* and average annual O&S cost per unit (* could be PAUC or APUC) Affordability targets are recorded in the MS A ADM

Affordability DAG (2/3) Milestone B – 10 U.S.C. 2366b – Congressional certification Based on a business case analysis, MDA certifies to Congress: The program is affordable Trade-offs between C/S/P have been made D,CAPE concurs with reasonable cost and schedule estimates Funding is available in the FYDP

Affordability DAG (3/3) 3.2.2 Affordability Assessments
Projected annual funding – 4-step process Address acq and O&S funding, force structure, and manpower requirements FYDP +/- 12 years Unit cost approach Compare unit cost of current program to unit cost of legacy system Use base year dollars

Affordability DAG (4/4) 3.2.3 Full Funding
At MS B, the most likely cost dollars and manpower needed to carry out the acquisition strategy are in the FYDP For MDAPs, the MDA must certify this in the 2366b certification and D,CAPE must concur MDAs should assess this for all major decision points for all acq programs

OSD/OMB Budget Hearings
PPBE – Integrated Program/Budget Review Year 2 JUL/AUG OCT NOV DEC Year 3 JAN/FEB DoD “Large Group” BES – Budget Estimate Submission CAPE – Cost Assessment & Prgm Evaluation COCOM – Combatant Commander CPA – Chairman’s Program Assessment DMAG – Deputy’s Management Action Group FYDP – Future Years Defense Program MBI – Major Budget Issues OMB – Office of Management and Budget PB – President’s Budget POM – Program Objectives Memorandum RMD – Resource Management Decision SLRG – Senior Leader Review Group 3-Star Group OSD/ CAPE Issue Resolution POM Issue Teams SECDEF/DEPSECDEF SLRG & DMAG CPA JCS Components/ Defense Agencies Program Review focus: Compliance with DPG RMDs (?) BES USD(C) & OMB Adv Questions/ OSD/OMB Budget Hearings MBI PB FYDP Update FYDP Update This slide shows the FEAs, initiated in the previous chart (Planning Phase) continuing to be conducted into the Integrated Program/Budget Review time period. The issues included in the FEAs are to be resolved through the Large Group (similar in make-up to the Defense Senior Leadership Conference, as listed in DoD D , DoD Senior Governance Councils). The output of the Large Group, as well as that of the DAWG-addressed issues that were not selected by the SECDEF for FEAs, is provided to the SECDEF/DEPSECDEF. All other POM issues are addressed under the existing process led by the CAPE. Asterisks (TBD) indicate that the organizations may be re-named or realigned, or that activities (MBI and reclama process) may or may not be conducted. Asterisks were placed based on feedback from OSD POCs. The results of the review process for both the BES and the POM will be documented, as appropriate in RMDs. It is expected that an RMD will have two annexes – one for program-related decisions and the other for budget-related decisions. DoDD specifically addresses the membership of the Defense Senior Leadership Conference, the Senior Leader Review Group, and the Deputy’s Advisory Working Group. The membership of the “Small Group”, the “Large Group”, and the “Large Group Plus” has been referred to in the media but not codified as of May 2010. Services / PEO / PM Answer / Reclama Budget Review focus: - Pricing Phasing - Funding policies - Budget execution

FY 16–20 Program/Budget Review Schedule
3 Apr 14 Fiscal Guidance Issued 2-12 Sep 14 Component POM Briefs to 3-Star/DMAG 2 Sep 14 Component POM Submissions Due (Databases Lock) 15 Sep 14 Budget Justification Material Due to OSD/OMB (POM) 18 Sep 14 Issue Nominations Due 29 Sep 14 Issue Nomination Disposition 3 Oct Refresh of BCA Cap (FY 16 Submission Due) Oct - Nov 14 Program/Budget Review and Issue Team Analysis 19 Dec 14 Budget Lock 2 Feb 15 President’s Budget Released NOTE: This chart reflects the official OSD planned dates for these events Source: Joint CAPE & USD(C) memo dated 22 May 2014

RQM 310 Getting from the AoA to KPPs

Lesson Objectives Given a real-world program with an ICD, AoA Study Guidance and an AoA--take the example from the ICD through KPP development 62

AoA to KPP

AoA An ideal AoA provides essential information on the decision space available to make executive decisions Prioritized, quantified, contextualized capability gaps Feasible alternatives that are affordable and mitigate the capability gaps The trades within and across alternatives among cost, schedule, and performance Transparent analysis of recognized quality Sensitivity analysis—how robust are the solutions? Do AoAs cover “knee-in-the-curve” analysis? This is different than sensitivity analysis….but it does kind of get into trade space.

KPPs & KSAs Key Performance Parameters (KPP) – are performance attributes of a system considered critical or essential to the development of an effective military capability. They are measurable, testable and quantifiable. Key System Attribute (KSA) – attributes or characteristics considered essential to achieving a balanced solution / approach to a system, but not critical enough to be designated a KPP. KSAs must be measurable, testable and quantifiable

Original Capability Gaps in ICD
The GCV ICD identified several gaps in current capability-We will concentrate on: Inability to detect mines and standoff (Survivability). Light vehicle protection against kinetic, chemical and tandem blast warheads (Protection). Maneuver for positional advantage in all terrain (Road and cross country speed, range and urban mobility)

AoA to KPP

Guidance on Structure of the Study
Per the guidance provided by the CAPE (Dec 2011 memo) The AMPV AoA – “ Should provide full consideration to the trades among cost, schedule and performance objectives within and across alternatives to identify a cost effective acquisition strategy.” So -- 5 main areas evolved: Assess performance, Identify sustainment impacts, Conduct a Schedule Assessment, Determine Lifecycle Costs, Determine Affordability

Study Guidance 5 Alternatives Base Case – M113 FoV
Mixed solution to include M113, Bradleys, MRAPs and Strykers Other Non-developmental solutions not included above New start including JLTV and GCV variants Mixed solution with specialized vehicles by mission The AoA focused on closing capability gaps two through four using the current Command, Control, Communications, Computers, Information, Surveillance, and Reconnaissance (C4ISR) capabilities. Per OUSD CAPE guidance, the AoA did not examine gap one due to time constraints.

M113 Family of Vehicles Medical Treatment Mission C2 General Purpose
• AMPV is the acquisition program to replace the Army's M113 family of vehicles in the Armored Brigade Combat Teams (ABCTs) (114 vehicles per ABCT) •Scope: procure -3,200 vehicles (number may vary depending on solution or potential mixed solution) • Current M113 family of vehicles lacks force protection for future threat environments and the size, weight, and power to support upgrades for protection, survivability, and the future network • Current M113 family of vehicles lacks mobility for future ABCT operational employment Mortar Carrier Medical Evacuation

M113 CONOPS The general purpose (GP), medical evacuation (ME) conduct maneuver with the forward line of troops to conduct medical evacuation tasks. The mortar carrier (MC) and battalion mission command (MCmd) roles maneuver within km of the forward line of troops and utilize terrain for protection The medical treatment (MT) and brigade MCmd mission roles maneuver km back), utilizing terrain to maneuver when possible.

MTLV – Mobile Tactical Light Vehicle CMTV – Caiman Multi-Terrain Veh
AoA Alternatives MTLV – Mobile Tactical Light Vehicle M113 – APC (Base Case) CMTV – Caiman Multi-Terrain Veh The JEF AoA examined a list of weapon systems paired with a variety of surface combatants. 71 different combinations of weapons and platforms were analyzed. As a replacement for the M113, GD Land Systems proposed a “tracked” version of the Stryker….as shown in the pics. The wheeled variant was not forwarded as an M113 replacement option because of the maneuverability that was required from a future M113. Notice all of the other pics are tracked also. Additionally, the stryker is the “double-v hull” for underbody protection. M2A3-T – Bradley minus turret SDVH – Stryker w/Double “V” Hull

AoA Alternatives The JEF AoA examined a list of weapon systems paired with a variety of surface combatants. 71 different combinations of weapons and platforms were analyzed. M113 – APC M2A3T – Bradley minus turret CMTV-Caiman Multi-Terrain Veh SDVH- Stryker with Double “V” Hull MTVL+FP-Mobile Tactical Vehicle Light (Enhanced FP)

Performance Requirements Considerations

Gap Analysis Gap 1 : Survivability – Concerned about mine and IED standoff detection and SWAP-C for the unit The systems mounted on the current vehicles lack the ability to detect, identify and prioritize both ground and aerial targets, at ranges in excess of the threat's detection and weapon systems effective ranges, and inside the threat's detection and response time.

Gap Analysis Gap 2: Protection – Armored and light vehicle protection against kinetic, chemical and tandem blast warheads Light tracked vehicles and Tactical Wheeled Vehicles (TWV) have insufficient protection against mines and underbelly IEDs. These Light tracked vehicles and TWVs have insufficient armor to stop EFPs, kinetic projectiles, as well as tandem chemical warheads. There is no "one-size fits all" vehicle that meets all likely threats. Current vehicles lack the capability to protect the occupants from the shock effects of blast.

Gap Analysis Gap 3: Mobility– Road speed, cross country speed, range and urban area maneuver Current vehicles lack the ability to negotiate the confined spaces presented in urban terrain. Niche force protection vehicles, such as MRAP, lack sufficient cross country and off road mobility. Non-maneuver elements lack the mobility, survivability and connectivity to immediately follow and support combat forces.

Gaps to KPPs (Survivability)

Gaps to KPPs (Force Protection)

Gaps to KPPs (Mobility)

AoA Results Only tracked alternatives provide sufficient mobility to maneuver with combat vehicles across a range of operational environments. M2A3-T and MTLV+FP achieve similar levels of force protection through use of add-on armor. While SDVH and CMTV excel at under body protection, these vehicles have limited direct fire protection. MT and MCmd (CTCP and BDE TOC) vehicles have lower exposure to direct fire threats and primarily travel with wheeled vehicles; opportunity to decrease force protection and mobility requirements.

Cost Estimation Methods
Analogy - Compare the new system with a similar system Parametric - Statistical analysis of a database of a similar system (Common for software) Engineering - “Bottom-up” analysis of all materials, parts and labor Actual - Extrapolated future estimates from previous nearly identical systems or sub-systems

Types of Costs

Types of Costs (CONT) Development Cost - RDT&E (R)
Flyaway/Rollaway Cost – Procurement (P) Weapon System Cost – Flyaway cost plus support equipment (P) Procurement Cost – Weapon system cost plus initial spares (P) Program Acquisition Cost – All costs including developing, procuring and housing the system (P)

Types of Costs (CONT) Operations and Support Cost – Costs for personnel, equipment, supplies, software, services needed to operate, maintain, supply, train and support a system (O&M) Life-Cycle Cost – Total cost of a program beginning to end. Includes program acquisition cost, operations and support cost and disposal costs. Also called Total Ownership Cost.

Affordability/Results
The Gap analysis and AoA results showed the M2A3-T met the Force Protection, Mobility and Survivability performance targets at a very high level to affordability and increased sustainment costs. Many AoA cost decisions use Procurement Unit Cost—the total Life-Cycle Cost divided by the number of units. -Multiple Solution for variants? -Is there a cost to using existing system?

Parametric Example

USAF Specialized Undergraduate Pilot Training
SUPT: 58 Week Program in a minimum of 2 different aircraft. 90 hrs of flight training within 26 weeks Students pick follow-on track based on performance Initial SUPT Tracks Helo Airlift/Tanker Fighter/Bomber For Aircraft: For Aircraft: For Aircraft: UH-1 HH-60 MH-53 CV-22 C-5/C17 C-130 KC-10/135 C-21/U-28 F-22 F-15/16 A-10 B-1/2/52 115 hr / 28 weeks Ft Rucker, AL 90 hr / 28 weeks All SUPT bases 100 hr / 27 weeks All SUPT bases

SUPT Fighter/Bomber CONOPS
Overall concept: Take students from complete novice status to combat-ready pilots able to employ in both legacy and 5th generation aircraft. 4th Gen Bridge Current Legacy Systems SUPT Advanced Fighter/Bomber Bridge Combat (FTU) 5th Gen Bridge Complex Systems Future

T-X AoA CAPE Guidance (March 2010)
CAPE Directed the following in their AoA Guidance: Feasibility of developing & fielding replacement aircraft When is it absolutely necessary to have a replacement for both the USAF and DoN trainer aircrafts? Can the T-38C FoS (and F-16 for F-22 bridge) be either cost effectively maintained/upgraded to meet ICD requirements? Can existing trainer or other FoS meet capability requirements? What are the risks with each APT FoS alternative? What are the contributions/impacts of various levels of maneuverability and propulsion on cost and effectiveness? To what degree are any of the new FoS suitable for DoN use? What degree of commonality can be achieved between USAF and DoN future advanced trainers/training systems? Assumption: USAF IOC: (Update: 2023) DoN Replacement Need: 2024

T-X AoA Results There were seven alternatives evaluated. Each alternative is a FoS made up of aircraft and Ground Based Training Simulators (GBTS) Sustain Current Fleet Modernize Current Fleet Use the Navy T-45 Aircraft 3 Single Engine Options (1 Subsonic & 2 Supersonic) One Developmental 2 Twin Engine Options (1 Subsonic & 1 Supersonic) Both use the F-16 bridge

T-X AoA Options

T-X AoA Results (1st AoA)
NOTE: IOC 2017 Prototype 2013 (T-45) Note: Last Navy T-45 Delivery Nov 2009 Most Cost Effective Can Meet IOC AoA Recommended Non-Development options due to IOC Not Cost Effective Not Viable Capability Shortfalls

T-X AoA Results (2nd AoA)
NOTE: IOC 2023 FOC 2031 (T-45) Not Examined Ruled Out in 1st AoA Performance limitations put viability in question Performance & development timeline now make these acceptable Vendor Pulled Proposal Includes performance upgrades but still has capability issues

T-X AoA Results (2nd AoA)
NOTE: IOC 2023 FOC 2031 (T-45) As a result of relative parity in costs, the AoA study team focused on capabilities being delivered by a new start and whether these capabilities were worth the cost, informed by risk to mission and program success. Alternatives 1 and 2, while gaining in cost-effectiveness, were assessed to have capability and risk deficiencies rendering them non-viable. Based on these factors, the team assessed the new start alternatives significantly increased mission effectiveness while reducing risk to the AETC mission, all at an acceptable level of risk for program success. The team recommended the T-38 FoS be replaced with a viable alternative that will meet an IOC date of 2023. Shortly after the AoA Update concluded, the F-16 Bridge course was eliminated based on ACC and the F-22 FTU assuming the risk created by the T-38C gaps. Total costs of Alt 1 & 2 were more closely aligned with the other alternatives

Video – “Pentagon Wars”
Subject – Bradley Fighting Vehicle Timeline – Seventeen Years, at a Cost of $14 Billion Video Shows Series of ‘Flashbacks’ into the Development Process Pictures on Wall(s) Uniforms Cast – Main Characters Lieutenant Colonel Robert L. Smith, Army (1970’s) Lieutenant Colonel James Burton, Air Force (Modern Day) Comedy Satirical Representation of Pentagon’s Bureaucracy Some Bad Language Intention is not to Point Fingers at Any Particular Military Service or Make-Fun of GO/FO/SES’s

Pentagon Wars -- Considerations
During Video, Please Consider the Following: Analysis Conducted on the Original “Armored Personnel Carrier” Cost/Budget Considerations Yesterday’s Priorities vs. Today’s Realities Requirements Creep Senior Leadership Direction Budget Factors

RQM 310 Milestone A to B LPD-17 San Antonio, SBIRS, F-35B, XM-25
Presenter: Thomas Fritz June 9, 2015 LPD-17 San Antonio, SBIRS, F-35B, XM-25

Lesson Objectives Determine the major requirements management activities during the Technology Maturation & Risk Reduction (TMRR) Phase, including the CDD Validation Decision, Development RFP Release Decision, and Milestone B Explain the purpose and major activities of the TMRR phase and the links between those activities Describe the decisions the Milestone Decision Authority (MDA) makes and the items the MDA approves at Milestone B

Milestone A to Milestone B Technology Maturation & Risk Reduction
AoA review and Portfolio Assessment CDD Validation Joint Concepts President SECDEF CJCS Joint Community Strategic Guidance Disposal CDR & CDR Assessment PDR & PDR Assessment Capabilities - Based Assessment Technology Maturation & Risk Reduction Engineering & Manufacturing Development Production & Deployment Operations & Support A MDD Draft CDD ICD AoA report B C CPD Materiel Solution Analysis Sustainment RFP Release Jump

TMRR thru Milestone B Requirements Management Activities
Purpose: Reduce risk Major activity: Competitive prototyping CDD Validation Development RFP Release A B Draft CDD Source Selection Contract(s) Final RFP TMRR Phase CDD Final RFP EMD Phase Source Selection Contract(s) Technology Maturation & Risk Reduction SRR SFR PDR TRA Requirements Management Activities Participate in IPTs and technical reviews MDA Reviews: Development RFP Decision Review, MS B Configuration Steering Board (CSB) Monitor/provide input Develop Revise CDD if necessary, CONOPS/OMS/MP update ; ensure CDD is validated and updates are accomplished before Development RFP Release decision Acquisition Strategy update PPBE Process TEMP update Affordability & Cost Estimate update Acquisition Program Baseline (APB) Technology Readiness Assessment (TRA) Life Cycle Sustainment Plan (LCSP) update AoA Update, RAM-C report update

CDD Validation Precedes the Development RFP Release Decision and provides a basis for preliminary design activities and the PDR that will occur prior to Milestone B unless waived by the MDA. The MDA (and CAE when the MDA is the DAE) participates in the review and staffing of the CDD (or equivalent requirements document), to ensure that requirements are technically achievable, affordable, and testable, and that requirements trades are fully informed by a systems engineering trade-off analyses completed by the PM or DoD Component.

Configuration Steering Board (CSB)
For ACAT I and IA programs*, after CDD Validation, the CAE forms and chairs a CSB Members include senior representatives from the military department, office of the USD(AT&L), the Joint Staff (DJ8), and the DoD CIO Program requirements fall under the cognizance of the CSB upon receipt of a validated CDD or other validated requirements document, before the Development RFP Release Decision Point (cognizance described on pg 22 of DODI ) The PM, in consultation with the PEO, will, at least annually, identify and propose to the CSB a set of descoping options that reduce program cost and/or moderate requirements Decisions on descoping options will be coordinated with capability requirements officials *DoD Components should also form CSBs for lower ACAT programs.

Preliminary Design Review (PDR)
Unless waived by the MDA, a PDR will be conducted before Milestone B and prior to contract award for EMD. Purpose: Provides the acquisition community, end user, and other stakeholders with an opportunity to understand the trade studies conducted during the preliminary design, and confirm that design decisions are consistent with the user’s performance and schedule needs prior to formal validation of the CDD. Outcome: Confirms that the preliminary design satisfies the operational and suitability requirements of the draft CDD.

Development RFP Release Decision (DRFPRD)
Purpose: Ensure that prior to release of final RFPs for EMD that an executable an affordable program has been planned. The Critical decision point in an acquisition program. MDA signs ADM that: Approves the Acquisition Strategy Authorizes release of the RFP and source selection for EMD; may include options for LRIP Determines preliminary LRIP quantities for ACAT I & II programs Documents criteria required for MS C (needed testing; affordability; FYDP funding) Requires validated CDD (to include ICD elements if ICD waived)

Technology Readiness Assessment (TRA)
TRA assesses maturity of, and risk associated with Critical technologies to be used in an MDAP TRA plan prepared by PM; approved by PEO, CAE and ASD(R&E). Conducted by Subject Matter Experts independent of PMO, but assigned by PM. Required for MDAPs for Sec. 2366b cert at Milestone B TRA results are included in CDD/CPD ASD(R&E) provides MDA an independent assessment and review of the TRA final report as to whether technology was demonstrated in relevant environment Technology Readiness Levels (TRLs) are knowledge-based standard but not a substitute for professional judgment F-22 OBOGS example Critical technologies are those that may pose major technological risk during development, particularly during the EMD phase of acquisition. (DoD TRA Guidance, April 2011, revised 13 May 2011)

Technology Maturity Technology Readiness Levels (TRLs)
System Test, Launch & Operations System/Subsystem Development Technology Demonstration Technology Development Research to Prove Feasibility Basic Technology Research TRL 9 TRL 8 TRL 7 TRL 6 TRL 5 TRL 4 TRL 3 TRL 2 TRL 1 Actual system proven through successful mission operations Actual system completed and qualified through test and demonstration (Full-Rate Production Decision) System prototype demonstration in a operational environment (Milestone C) System/subsystem model or prototype demonstration in a “relevant environment” (2366b certification item - Milestone B) Component and/or breadboard validation in relevant environment Component and/or breadboard validation in laboratory environment Analytical and experimental critical function and/or characteristic proof-of-concept Technology concept and/or application formulated Basic principles observed and reported Public law requires MDAPs be at TRL 6 for Milestone B

Milestone B Formal Program Initiation (most programs) with approval of the Acquisition Program Baseline (APB) Authorizes entry into EMD phase and award of contracts for EMD Most Milestone B documentation requirements will have been satisfied at the Development RFP Decision (DRFPRD) Any significant changes since DRFPRD must be provided Commits required investment resources to program MDA finalizes, if not already completed at DRFPRD: LRIP Quantity or limited fielding scope Exit criteria for leaving EMD and entering next phase MDA approves: APB (includes affordability caps) 2366b Certification ADM to document decisions Requires validated CDD (to include ICD elements if ICD waived), and Full Funding in the Future Years Defense Program (FYDP)

ACAT I Section 2366b Certification
RM interest areas are in blue bold MEMORANDUM FOR THE RECORD SUBJECT: Milestone B Program Certification As required by Section 2366b of Title 10, United States Code, (1) I have received a business case analysis for the (name of program) and certify on the basis of the analysis that: (A) the program is affordable when considering the ability of the DOD to accomplish the program's mission using alternative systems; (B) appropriate trade-offs among cost, schedule, and performance objectives have been made to ensure that the program is affordable when considering the per unit cost and the total acquisition cost in the context of the total resources available during the period covered by the future-years defense program submitted during the fiscal year in which the certification is made; (C) reasonable cost and schedule estimates have been developed to execute, with the concurrence of the Director of Cost Assessment and Program Evaluation, the product development and production plan under the program; and (D) funding is available to execute the product development and production plan under the program, through the period covered by the future-years defense program submitted during the fiscal year in which the certification is made, consistent with the estimates described in paragraph (C) for the program; (2) I have received the results of the preliminary design review and conducted a formal post-preliminary design review assessment, and certify on the basis of such assessment that the program demonstrates a high likelihood of accomplishing its intended mission;

2366b Certification continued..
RM interest areas are in blue bold (3) I further certify that: (A) appropriate market research has been conducted prior to technology development to reduce duplication of existing technology and products; (B) the DOD has completed an analysis of alternatives with respect to the program; (C) the JROC has accomplished its duties with respect to the program pursuant to section 181(b) of Title 10, including an analysis of the operational requirements for the program; (D) the technology in the program has been demonstrated in a relevant environment, as determined by the MDA on the basis of an independent review and assessment by the ASD for Research and Engineering in consultation with the DASD for Development Test and Evaluation; (E) life-cycle sustainment planning, including corrosion prevention and mitigation planning, has identified and evaluated relevant sustainment costs throughout development, production, operation, sustainment, and disposal of the program, and any alternatives, and that such costs are reasonable and have been accurately estimated; (F) an estimate has been made of the requirements for core logistics capabilities and the associated sustaining workloads required to support such requirements; (G) there is a plan to mitigate and account for any costs in connection with any anticipated de-certification of cryptographic systems and components during the production and procurement of the major defense acquisition program to be acquired; and (G) the program complies with all relevant policies, regulations, and directives of the DoD.

Linkage: CDD, APB, TEMP & Contracts for TMRR and EMD
Milestone A Milestone B Cost Estimate & $$ Competitive Designs & Prototypes DT TEMP COIs, MOEs, MOPs, CTPs EMD Contract(s) Draft CDD Contract(s) Validated CDD AoA Update SPECs “design-to” TMRR KPPs KSAs Final RFP TMRR Final RFP EMD MOEs MOPs KPPs KSAs APB KPPs Source Selection Source Selection Configuration Steering Board Established Technology Maturation & Risk Reduction Phase 115

Back-Up

Resources Thomas Fritz thomas.fritz@dau.mil JCIDS CAC enabled
DoD Generic Acquisition Process (Pre-Tailoring) Acquisition Decision Points and Phases Acquisition Waterfall Chart with color enhancements 17 Dec final (3).pdf Technical Reviews for Requirements Managers Link Jump Back

Technology Maturation and Risk Reduction
JCIDS Document Flow Initial Capabilities Document (ICD) Identifies capability gaps with recommendations for material solutions Information for an Analysis of Alternatives (AOA) of possible material solutions Sets the acquisition stage Only 10 pages max… MS “B” MS “C” MS “A” Evolutionary Acquisition Materiel Solution Analysis Engineering & Manufacturing Development Production & Deployment CDD CPD Analysis of Alternatives (AoA) Competitive Prototyping ICD Materiel Decision CA Technology Maturation and Risk Reduction

JCIDS Document Flow Capability Development Document (CDD)
Draft (Component-Approved) CDD at or before MS A Informs TMRR contracts Validated CDD for Requirements Decision Point prior to MS B Identifies KPPs, KSAs, & APAs Attributes should be Authoritative, Measureable, & Testable Describes DOTmLPF-P constraints associated with the solution Could describe multiple increments Supports AS & APB Insert all CDD KPPs and Sustainment KSAs verbatim into the APB 45 page limit… MS “B” MS “C” MS “A” Evolutionary Acquisition Materiel Solution Analysis Engineering & Manufacturing Development Production & Deployment CDD CPD Analysis of Alternatives (AoA) Competitive Prototyping ICD Materiel Decision CA CDD Validation Technology Maturation and Risk Reduction

Capability Production Document (CPD)
JCIDS Document Flow Capability Production Document (CPD) Supports one increment Production, Development Testing, and Deployment Documents authoritative & testable capabilities No new requirements! Must meet operational performance attributes 40 page limit… MS “B” MS “C” MS “A” Evolutionary Acquisition Materiel Solution Analysis Engineering & Manufacturing Development Production & Deployment CDD CPD Analysis of Alternatives (AoA) Competitive Prototyping ICD Materiel Decision CA

Requirements/Acquisition Document Traceability
Program Mgmt Contract Test & Evaluation CJCS/JROC CJCSI JCIDS Manual USD (AT&L) DoDD DoDI USD (AT&L) FAR/DFARS DOT&E DoDI DoDI CDD/ CPD KPPs KSAs APAs Threshold-Objective APB KPPs Cost & schedule SPEC Derived Technical Requirements TEMP COIs MOEs MOSs CTPs Service Chief Service Requirements Office/User CAE Program Executive Officer Program Office HCA/SPE Contracting Office Service Chief Operational Test Command Requirements Manager Program Manager Contracting Officer Operational Test Director -CDD performance attributes (KPP, KSA and others) are derived from the capability metrics in the ICD that are used in the AoA. They apply to the materiel solution approved at MS A, are in the draft CDD, then refined during the Technology Maturation and Risk Reduction (TMRR) phase. The CDD provides guidance for engineering and manufacturing development (EMD), and may be updated (if appropriate) based on experience gained during EMD to the CPD that describes the performance required from the production systems. -The APB contains verbatim the KPPs from the CDD/CPD. -The TEMP contains Critical Operational Issues (questions that need to be answered to reflect achievement of the KPPs), Measures of Effectiveness (MOE), Measures of Supportability (MOS) and Critical Technical Parameters (CTPs) derived from the KPP/KSA and other performance attributes in the CDD. Not shown are Measures of Performance (MOP), which are not in the TEMP, but are in the detailed test plans. -Specifications are the technical requirements needed to design/build the system to provide the needed capability. Technical requirements are derived from the MOE/MOS/MOP and CTPs. -Because the derived technical requirements are what the system is designed to, the Requirements Manager needs to be involved in the systems engineering process from KPP/KSA development, development of the MOE/MOS/CTP for the TEMP, and the translation of these attributes to design specs. Participation in the Test IPT, and attendance at technical reviews is the best way to participate in the process. APA: Additional Performance Attribute CDD: Capability Development Document CPD: Capability Production Document KPP: Key Performance Parameter KSA: Key System Attribute HCA: Head of Contracting Activity SPE: Senior Procurement Executive COI: Critical Operational Issues MOE: Measure of Effectiveness MOS: Measure of Suitability CAE: Component Acquisition Executive CTP: Critical Technical Parameter APB: Acquisition Program Baseline Spec: Technical Specification

RQM 310 Developing Requirements KPPs and Performance Attributes

Lesson Objectives Demonstrate the proper format of performance attributes and KPPs: Identify Mandatory KPPs Identify the characteristics of well and poorly written KPPs Critique KPPs for clarity, achievability and resourcing Explain the relationships of the KPPs and KSAs to Measures Effectiveness (MOEs), Measures of Performance (MOPs), and Measures of Suitability (MOSs) Identify solutions to requirements creep Explain Configuration Steering Boards (CSB) 123

KPP and KSA Development

DoD Definitions Requirement (Capability Requirement)— Capability—
A capability which is required to meet an organization’s roles, functions, and missions in current or future operations. Note: a requirement is considered ‘draft’ or ‘proposed’ until validated by the appropriate authority. Capability— The ability to execute a specific course of action. Capability Gap— The inability to execute a specified course of action.

DoD Definitions and Explanations
Requirement A capability which is required to meet an organization’s roles functions, and missions in current or future operations. Note: a requirement is considered ‘draft’ or ‘proposed’ until validated by the appropriate authority. Capability The ability to execute a specific course of action. Capability Gap The inability to execute a specified course of action. Example: USAF Tanker Fuel Load & Passenger Capabilities KC-XX Fuel Load: DoD Defined KPP Passengers: DoD Defined KPP KC-135 Fuel Load: 200,000 lbs. Passengers: 35 This slide provides an understanding of the ‘bread & butter’ of what JCIDS is all about. 1) Requirement -- this is the actual ‘capability’ required (or requested) to execute the specified course of action (BLACK LINE ABOVE) Capability -- this is the current ability to execute the specified course of action (BLUE LINE ABOVE). Sometimes the requirement & capability are co-equal. In most cases, however, they are not co-equal and thus the JCIDS process works to fill that ‘capability gap’  the difference btw the current capability and the necessary requirement. Capability Gap -- this is the inability to execute the specific course of action (GREEN LINE ABOVE). (Currently) KC-46 Fuel Load: 212,672 lbs. Passengers: 114

Key Performance Parameter (KPP) Definition
Key Performance Parameters (KPP) – Performance attributes of a system considered critical to the development of an effective military capability. The number of KPPs identified by a Sponsor should be kept to a minimum to maintain program flexibility. KPPs must be measurable, testable and quantifiable. Failure of a system to meet a validated KPP threshold/initial minimum rescinds the validation, brings the military utility of the associated system into question and may result in a reevaluation of the program or modification to production increments…

The difference between Threshold and Objective =
JCIDS Definitions of Threshold vs Objective Threshold — A minimum acceptable value considered achievable within the available cost, schedule, and technology at low-to-moderate risk. Performance below the threshold value is not operationally effective or suitable or may not provide an improvement over current capabilities. Objective— Is the desired operational goal achievable but at higher risk in cost, schedule and technology. Performance above the objective does not justify additional expense. The difference between Threshold and Objective = “Trade Space”

Key System Attribute (KSA) Definition
Key System Attribute (KSA) – attributes or characteristics considered essential to achieving a balanced solution / approach to a system, but not critical enough to be designated a KPP. KSAs must be measurable, testable and quantifiable.

Types and Attributes of Requirements
Types of Requirements Requirement should be Customer/Operational Functional Performance Design Derived Long Range, High Spd Light Allocated 100lbs Item 70lbs 30lbs Achievable Measurable/Testable/Quantifiable Unambiguous Complete Necessary Prioritized

Key Performance Parameters (KPPs) and Key System Attributes (KSAs)
Force Protection KPP (all manned systems) System Survivability KPP (all manned; may be applicable to unmanned ) Sustainment KPP (all ACAT I* ) Materiel Availability Operational Availability Supporting KSAs Materiel Reliability Operation & Support Costs Net Ready KPP (all IS & NSS) Training KPP (all ACAT I) Energy KPP (all where provisions of energy impact operational reach, or protection of energy infrastructure or energy resources is required) Page: D-A-2 to D-A-4 JCIDS Manual Operation & Support Cost KSA: Previously called “Ownership Cost” KSA. Did not include all elements of O&S costs – specifically, did not include cost of system specific training, or any indirect costs. Now includes all CAPE specified elements of O&S costs. The Manual now includes specific instructions for development of the Training and Energy KPPs. Mandatory where applicable *ACAT II and below programs, must include the Sustainment KPP or Sponsor defined sustainment metrics.

Key Performance Parameters (KPPs)
Intended to ensure protection of occupants, users, or other personnel (other than the adversary) who may be adversely affected by the system or threats to the system. …. Addresses protection of the system operator or other personnel against kinetic and non-kinetic fires, CBRN, and environmental effects, rather then protection of the system itself and its capabilities. Force Protection KPP

…Intended to ensure the system maintains its critical capabilities under applicable threat environments. …may include reducing a system’s likelihood of being engaged by hostile fire, through attributes such as speed, maneuverability, detectability, and countermeasures; reducing the systems vulnerability if hit by hostile fire, through attributes such as armor, redundancy, enabling operation in degraded EM, space, or cyber environments… and allowing the system to survive and continue to operate in, or after exposure to, a CBRN environment, if required. In SoS approaches, it may also include resiliency attributes pertaining to the ability of the broader architecture to complete the mission despite the loss of individual systems. System Survivability KPP

Intended to ensure an adequate quantity of the capability solution will be ready for tasking to support operational missions. The supporting Reliability KSA and O&S Cost KSA, ensure that the Sustainment KPP is achievable and affordable in its operational environment. Note: Cost Assessment and Program Evaluation (CAPE) will be significantly involved with all cost analysis ie. fully burdened cost of energy. Together, the KPP and supporting KSAs ensure early sustainment planning, enabling the requirements and acquisition communities to provide a capability solution with optimal availability and reliability to the warfighter at an affordable life cycle cost. $ustainment KPP

Intended to ensure combat capability of the force by balancing the xxxxx performance of systems and the provisioning of xxxxx to sustain systems/forces required by the operational commander under applicable threat environments. Energy KPP Note: The JCIDS Manual does not attempt to direct and/or describe the methodologies or rationale for energy usage within an environment or asset. However, sufficient analysis must be included and must withstand critical review. Operational energy demand (ie. lbs per hour), energy resupply risks (ie. overall logistical footprint), etc, are priorities that must be addresses w/in analysis The Energy KPP includes, but is not limited to, optimizing fuel and electric power demand in capability solutions, in the context of the logistical supply of energy to the warfighter, as it directly affects the burden on the force to provide and protect critical energy supplies. The Energy KPP includes both fuel and electric power demand considerations in systems, including those for operating “off grid” for extended periods when necessary, consistent with SSA products. In cases where energy demand reduction is impractical or insufficient to align with projected energy supply, complementary DOTmLPF-P changes to the energy supply chain must be addressed in the document to accommodate the increased energy demands and satisfy the Energy KPP.

Intended to ensure new and modified IS fits into DOD architectures and infrastructure to the maximum extent practicable. Net Ready KPP To mitigate the risk of delivering non-interoperable capabilities to the Warfighter The NR KPP is applicable to IS-ICDs, and all CDDs and CPDs addressing IS, regardless of classification or sensitivity of the data handled by the IS, unless defined as non-DODIN IT by reference qqqq. Net-ready attributes determine specific criteria for interoperability, and operationally effective end-to-end information exchanges which are traceable to their associated operational context, and are measurable, testable, and support efficient and effective T&E. (D-E-2)

Intended to ensure that materiel aspects of training capabilities, when applicable, are addressed as part of the development of the capability solution outlined in the CDD or CPD. Non-materiel aspects of training are to be captured as part of the DOTmLPF-P section of the CDD or CPD. Training KPP For example, the long mission durations of submarine operations may necessitate that the warfighter use the Training KPP to specify certain training and simulation capabilities be integrated into the weapon system. Other weapon systems with shorter mission durations may have greater flexibility so the specific training approaches do not need to be dictated by the warfighter, leaving the most effective training approach to be determined by the training experts. The Training KPP is applicable to all CDDs and CPDs with materiel training requirements which dictate specific operational performance characteristics of the capability solution.

KPP Relief Requesting KPP Relief JCIDS Man D-A-11
To fulfill the JROC’s Title 10 responsibilities for tradeoffs between cost, schedule, and Performance when validating military requirements, the JROC encourages the requirements Sponsor, in coordination with the MDA, to request requirements relief from the validation authority when cost-benefit analyses indicate previously validated KPPs may drive costs out of proportion with the capability delivered to the operational user.

KPP Relief Changing context over time
While the KPPs (and KSAs and APAs) documented and validated in capability requirement documents represent the validation authority’s best military advice at an instant in time, knowledge gained through acquisition activities, changes to strategic guidance, external threats, mission requirements, or budgetary realities may make relief from previously validated KPPs appropriate. Thesis: Energy logistics & infrastructure are becoming more contested in future battlespace Opportunity cost - delivering/protecting more fuel logistics means doing less of something else - allocation of scarce DoD resources Direct cost of energy is growing - long-term budget risk set mostly at program design Applies to systems where the provision of energy (fuel or electricity): Impacts operational reach (to execute missions), or… Requires protection of energy infrastructure or energy logistics forces What are we measuring? - Can be expressed as units of energy used per period of time (e.g. gallons per hour), or as number of refuelings required per period of time (e.g. tankings per hour/day) Ties energy demand back to mission performance (i.e. effectiveness in scenarios) Same approach as other KPPs - employment scenarios help set lethality, force protection & mobility requirements and metrics Who oversees? - Logistics FCB (JS J-4), advised by the Defense Energy Board for JROC or JCB Interest Programs Assesses the Energy KPP, or Assesses sponsor justification for why the Energy KPP is not applicable Logistics is an independent variable in the fight - Intent is to optimize fuel and electric power demand in solutions that meet capability requirements. Includes fuel and electric power demand considerations in systems consistent with future force plans and Integrated Security Constructs (ISCs - a.k.a., Defense Planning Scenarios). Analysis for this KPP must be derived from approved planning scenarios that include operation of the system and the energy-related logistics and force protection required in contested operational domains. All ISCs used for analysis must be of sufficient duration (multiple days to weeks) to demonstrate the effect of realistic opponent effects on logistics forces. Improved energy efficiency of a design or technology can be used in multiple ways in a system development program: Reduced logistics footprint - in operations and in peacetime Improve the range, loiter and ability to operate more quickly/effectively in small dispersed units Add new sub-systems with higher power demands - ISR, C2, DE weapons, etc. Add weight - for force protection, mitigate cost of lighter materials, etc. Question is, for the scenarios we’re designing this thing for, what’s the optimal mix?

KPP Relief Budgetary considerations
While there are no limitations for requesting requirements relief, KPP relief should be considered especially appropriate in cases where significant cost savings may be achieved with marginal impact to operational capability. i.e. – spending 15 percent of a program’s budget to get the last 3 percent of a KPP threshold, if the operational risk involved with a reduced threshold is minimal. Thesis: Energy logistics & infrastructure are becoming more contested in future battlespace Opportunity cost - delivering/protecting more fuel logistics means doing less of something else - allocation of scarce DoD resources Direct cost of energy is growing - long-term budget risk set mostly at program design Applies to systems where the provision of energy (fuel or electricity): Impacts operational reach (to execute missions), or… Requires protection of energy infrastructure or energy logistics forces What are we measuring? - Can be expressed as units of energy used per period of time (e.g. gallons per hour), or as number of refuelings required per period of time (e.g. tankings per hour/day) Ties energy demand back to mission performance (i.e. effectiveness in scenarios) Same approach as other KPPs - employment scenarios help set lethality, force protection & mobility requirements and metrics Who oversees? - Logistics FCB (JS J-4), advised by the Defense Energy Board for JROC or JCB Interest Programs Assesses the Energy KPP, or Assesses sponsor justification for why the Energy KPP is not applicable Logistics is an independent variable in the fight - Intent is to optimize fuel and electric power demand in solutions that meet capability requirements. Includes fuel and electric power demand considerations in systems consistent with future force plans and Integrated Security Constructs (ISCs - a.k.a., Defense Planning Scenarios). Analysis for this KPP must be derived from approved planning scenarios that include operation of the system and the energy-related logistics and force protection required in contested operational domains. All ISCs used for analysis must be of sufficient duration (multiple days to weeks) to demonstrate the effect of realistic opponent effects on logistics forces. Improved energy efficiency of a design or technology can be used in multiple ways in a system development program: Reduced logistics footprint - in operations and in peacetime Improve the range, loiter and ability to operate more quickly/effectively in small dispersed units Add new sub-systems with higher power demands - ISR, C2, DE weapons, etc. Add weight - for force protection, mitigate cost of lighter materials, etc. Question is, for the scenarios we’re designing this thing for, what’s the optimal mix?

Navy’s New “Key Parameter” Changes
Navy’s New “Key Parameters” (effective 11 Aug 14) Navy adding two new key parameters: Key Cost Parameter Key Schedule Parameter Navy’s goal is to place cost and schedule on par with performance and encourage better trades between cost, schedule, and performance to remain within fiscal constraints Navy is also adding another KPP – SWaP-C Navy’s goal for a “Space, Weight, Power, and Cooling” KPP is to ensure programs have sufficient and clearly defined design margins at the outset

Navy’s New “Key Parameter” Changes
CONT N8 Capability Review “Tripwire” If “actual or projected” cost or schedule parameters, as captured in the JCIDS docs, is exceeded, N8 will convene either an NCB or R3B Trades will be considered among cost, schedule and performance to get program back in line Cost and Schedule will also have corresponding KSA-like attributes Key Cost Attributes Key Schedule Attributes Operational test authorities will not test to the new KCPs or KSPs (nor their key attributes)

Navy’s “Key Parameter” Implementation
Set KCP “unit cost” at least 5% below threshold value JCIDS docs must explain what a “unit” is for the program and how KCP was derived KSP will use IOC date as threshold and IOC minus 6 months for objective New Navy ICDs will now have “not-to-exceed” cost cap and IOC need date captured in doc Resource sponsor must consider the amount of funding projected to be available for the program SWaP-C will only be considered for Navy “platforms” (ships, aircraft, UAVs, etc)

AoA Results Littoral Combat Ship (LCS) KPPs : Mission Package Payload
Sprint Speed Endurance Range Core Ship Crew Size Draft The littoral combat ship (LCS) is the first of a new family of surface ships for the US Navy. The LCS is a fast, highly maneuverable, networked surface combat ship, which is a specialized variant of the family of US future surface combat ships known as DD(X). LCS is designed to satisfy the urgent requirement for shallow draft vessels to operate in the littoral (coastal waters) to counter growing potential 'asymmetric' threats of coastal mines, quiet diesel submarines and the potential to carry explosives and terrorists on small, fast, armed boats. In May 2004, the United States Department of Defense and the US Navy announced the selection of two separate defense contracting teams led by Lockheed Martin and General Dynamics to each carry out system design and options for the detailed design and construction of two flight 0, or first-generation, LCS ships.

Congressional Research Service – LCS Tradeoffs
In Web Browser: Type: Congressional Research Service Navy Littoral Combat Ship (LCS) Program Click on: [PDF] Navy Littoral Combat Ship (LCS) Program – Federation of American …

KPP MOS MOE MOP KSA Measure of Effectiveness (MOE) Measure designed to correspond to accomplishment of mission objectives and achievement of desired results. MOEs can be assessed using two subsets; Measures of Performance (MOP) and Measures of Suitability (MOS) “What do we want the system to be able to do?”

Measures of Performance (MOP) and Suitability (MOS) – JSF Example
MOP – Quantifiable Performance Attributes ie. Speed, Payload MOS – Item’s Ability to be Supported in Intended Op. Environ. Operational capabilities Question: Are the JSF G-Limits Below MOPs or MOSs? USAF 9.0g USMC 7.0g USN 7.5g

GAO Report (Recent) – Configuration Steering Boards
In Web Browser: Type: GAO R Click on: [PDF] GAO R, Defense Acquisitions: Military Services …

Configuration Steering Boards (CSBs)
The Acquisition Executive of each DoD Component shall establish and chair a CSB with broad executive membership . . . DoDI per Section 814, FY ’09 National Defense Authorization Act CSBs meet at least annually Reviews all requirements changes and significant technical configuration changes Changes generally rejected, deferred to future blocks or increments If approved, changes require identified funding and mitigated schedule impacts The Program Manager (PM) (with the Program Executive Officer) identifies descoping options to reduce program cost or moderate requirements The CSB recommends which options should be implemented Final decisions coordinated with the Joint Staff and military department requirements officials -- Mrs. McFarland further emphasized.

NOTE: Only the JROC can initially validate and/or revalidate KPPs
Requirements Creep Creep is/are: New Requirements Being Levied on a Program Discovered during testing and development Budgetary Matters/Issues Creeping into Program’s Overall Cost; Thereby Causing KPP Re-Evaluation (For Cost) Is Requirements Creep, In One Form or Another, Inevitable? NOTE: Only the JROC can initially validate and/or revalidate KPPs

Remedies to Requirements Creep
Regular dialogue between RM and PM Well-Written Requirements Process Discipline Configuration Steering Boards (CSB)

Lesson Review The Requirements Managers represent the warfighters’ interests in the Acquisition process. The RMs’ major purpose in the JCIDS and Acquisition process is to prevent unnecessary requirement changes (creep) and minimize negative influences (delays, cost increases) on their program. Failure will result in delay and increased costs and may result in overall failure to complete the acquisition successfully…thereby failing the warfighter.

Lesson Review Requirements Managers and Acquisition Professionals must both remain aware of the requirements to gain approval for every change to any KPP. They both have a duty to inform the other, as well as their leadership, of changes (required or proposed) and establish a timeline to gain approval for these changes (may include a requirement to send an impact statement to decision making bodies in the chain of command).

Back-Ups

Recent Examples of Evaluation of Performance Parameters
Joint Strike Fighter During the recent requirements review, JROC avoided additional cost growth by de-scoping the requirements for combat radius, short take-off distance, and maximum carrier approach speed. JROC provided input for the dual path helmet development to the MDA. NOTE: To provide timely KPP relief (per VCJCS direction), the JROC receives an annual requirements review to evaluate program cost versus capability.

Recent Examples of Evaluation of Performance Parameters
UCLASS JROC re-examined operational concept for employment. JROC reviewed the ICD and revised requirements to provide maximum capability and capacity within the program funding baseline. Specifically, Increased the requirements for UCLASS' payload versatility. Increased the number of units procured while reducing survivability requirements. (JROCM ) JAGM: Joint Air-Ground Missile Affected interoperability with platforms and other KPPs in a three step process...increments 1, 2, and 3 thresholds and objectives. Addressing which aircraft were capable. Adjusting threshold aircraft.

Program and KPP Examples
Weather Satellite System Mission: to collect and disseminate other critical air, land, sea, and space environment data to support a broad range of national security users. KPPs Include: Ocean Color: accuracy and precision for ocean color, with values of 0.1 W m-2 μm-1 sr-1 or 10% (whichever is greater) and 0.05 W m-2 μm-1 sr-1 or 5% (whichever is greater), respectively Sensing depth: must be at least 0.2 cm (into the soil) based on a horizontal cell size (HCS) of 40 km Sea Surface Wind Speed: measurement accuracy which is required to be less than the “greater of 3 m/s or 15%” (of the measured value)

Medium Armored Vehicle (MAV) Mission: to provide a family of vehicles (FOV) that were air transportable anywhere in the world and support infantry operations. Source: Equipment Sustainment Requirements for the Transforming Army, RAND, 2009

Medium Armored Vehicle Resulted with Four KPPs Definition: Interoperability – All Services Transportability – C-130 Carry an Infantry Squad USMC vs. USA Destroy a standard infantry bunker Source: Equipment Sustainment Requirements for the Transforming Army, RAND, 2009

Requirements Creep Requirements creep refers to significant additions or modifications to the requirements of a new system throughout the lifecycle, resulting in extensions to and alteration of the system’s functionality and scope

KPP Development Guidelines for Writing Quality KPPs
Questions to ask before a performance attribute is selected as a KPP If an item does not meet the KPP definition, consider it for inclusion as a Key System Attribute (those issues that do not rise to the level of a Key Performance Parameter) Key Performance Parameter (KPP) Steps

Post-MDD: Analysis of Alternatives (AoA)

Similar presentations

Presentation on theme: "Post-MDD: Analysis of Alternatives (AoA)"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Post-MDD: Analysis of Alternatives (AoA)

Similar presentations

Presentation on theme: "Post-MDD: Analysis of Alternatives (AoA)"— Presentation transcript:

Similar presentations

About project

Feedback