Presentation is loading. Please wait.

Presentation is loading. Please wait.

Test and Evaluation Master Plan

Similar presentations


Presentation on theme: "Test and Evaluation Master Plan"— Presentation transcript:

1 Test and Evaluation Master Plan
(TEMP) Overview NOTE: Take away notes have been annotated on most of the slides in this module. April 2010 TEMAC T&E Refresher Course

2 What is a TEMP ? The ‘Contract’ Between All Stakeholders!
Executive level strategy and basic planning document for all life cycle T&E for a particular system acquisition Overall T&E structure, major elements, and objectives Consistent with Acquisition Strategy, TDS, CDD/CPD, STA, & ISP Consistent and complementary to the Systems Engineering Plan Sufficient detail to permit planning for timely availability of test resources required to support the T&E program Road Map for: Evaluation framework Schedules and Decisions supported Resource requirements The ‘Contract’ Between All Stakeholders!

3 Policy/Guidance 10 U.S.C. Section 139 – Director of Operational Test and Evaluation DODI : Operation of the Defense Acquisition System ( 8 Dec 2008) Interim Defense Acquisition Guidebook (7 July 2008) AR 73-1: Test and Evaluation Policy (1 August 2006) Chapter 8: T&E Working-level Integrated Product Team (WIPT) Chapter 10: T&E Review and Reporting Requirements Para Test and Evaluation Master Plan DA PAM 73-1: T&E in Support of Systems Acquisition (30 May 2003) Several authoritative documents exist that affect the Test and Evaluation Master plan (TEMP).

4 Materiel and Information Technology
Applicable Programs Every Army acquisition program will have a TEMP Only exceptions are investigational drugs, biologicals, & medical devices All Materiel and Information Technology Programs The message should be clear – with only rare exceptions, every system being acquired will have a TEMP.

5 Preparation Responsibility: Program / Project Manager (PM)
Execution: T&E WIPT

6 TEMP Format & Content Approval Page T&E WIPT Coordination Sheet
Part I - Introduction Part II - Test Program Management and Schedule Part III - Test and Evaluation Strategy Part IV - Resource Summary Annex A - Bibliography Annex B - Acronyms Annex C - Points of Contact Attachment 1 - Requirements/Test Crosswalk Matrix Additional appendices/annexes/attachments as needed TEMP format and content is per: Defense Acquisition Guidebook, AR 73-1 and DA Pamphlet 73-1. TEMP format mandatory for all new program starts, programs that are being restructured, and any other program at their discretion. Programs beyond Milestone B using another TEMP format may continue that format or convert to this format at the discretion of the PM (& T&E WIPT). Inclusion of all information shown is required for programs under OSD Test and Evaluation oversight.

7 TEMP Part I Introduction
1.1 PURPOSE 1.2 MISSION DESCRIPTION 1.3 SYSTEM DESCRIPTION System Threat Assessment Program Background Previous Testing Key Capabilities Key Interfaces Special test or certification requirements Systems Engineering (SE) Requirements

8 TEMP Part II - Test Program Management and Schedule
2.1 T&E MANAGEMENT T&E Organizational Construct 2.2 Common T&E Data Base Requirements 2.3 DEFICIENCY REPORTING 2.4 TEMP UPDATES 2.5 INTEGRATED TEST PROGRAM SCHEDULE Figure 2.1 – Integrated Test Program Schedule

9 TEMP Part III Test and Evaluation Strategy
1 of 3 3.1 T&E STRATEGY 3.2 EVALUATION FRAMEWORK Figure 3.1 – Top-Level Evaluation Framework Matrix 3.3 Developmental Evaluation Approach Mission-Oriented Approach Developmental Test Objectives Modeling and Simulation Test Limitations

10 TEMP Part III Test and Evaluation Strategy
2 of 3 3.4 Live Fire Evaluation Approach Live Fire Test Objectives Modeling and Simulation Test Limitations 3.5 Certification for IOT&E Assessment of Operational Test Readiness 3.6 Operational Evaluation Approach Operational Test Objectives Modeling and Simulation Test Limitations

11 TEMP Part III Test and Evaluation Strategy
3 of 3 3.7 OTHER CERTIFICATIONS 3.8 RELIABILITY GROWTH 3.9 FUTURE TEST AND EVALUATION

12 TEMP Part IV Resource Summary
1 of 2 4.1 Introduction Test Articles Test Sites and Instrumentation Test Support Equipment Threat Representation Test Targets and Expendables Operational Force Test Support Models, Simulations, and Test-beds Joint Operational Test Environment Special Requirements

13 TEMP Part IV Resource Summary
2 of 2 4.2 Federal, State, Local Requirements 4.3 Manpower/Personnel Training 4.4 Test Funding Summary Table 4.1 Resource Summary Matrix

14 TEMP - SEP CONTRAST TEMP SEP Integrated T&E Strategy SEP
T&E WIPT produces MDA (or designee) approves Serves as a T&E ‘Contract’ Issue level of detail Can include all events ATEC System Team (AST) produces ATEC Commander (or designee) approves Outlines HOW evaluation will be done Measure level of detail Only events necessary for evaluation Addresses Test Plan(s) SEP TEMP T&E Strategy Integrated While ATEC’s SEP and the T&E WIPT’s TEMP are DEVELOPED CONCURRENTLY, it is IMPORTANT to know the SALIENT DIFFERENCES between the two documents. GO OVER EACH ITEM. Developed concurrently, BOTH documents are CRUCIAL to a program’s successful Integrated T&E process. Some day you may deal with a PM (or T&E WIPT Chair) named ‘Bucky.’ AR 73-1 (para 1-4h) – ‘The detailed information contained in the SEP supports concurrent development of the TEMP.’ 14

15 TEMP Approval Process With HQDA Involvement
T&E WIPT develops TEMP (all members empowered; includes DA Staff) Final T&E WIPT TEMP ‘Signing Party’ meeting (members sign TEMP Coordination Sheet) PM submits TEMP Approval Page; obtains PEO, TRADOC, & ATEC simultaneous concurrence signatures (Goal: Complete at T&E WIPT via empowered members) PM delivers TEMP with all concurrence signatures to TEO HQDA(TEO) staffs &delivers TEMP to Army T&E Executive for approval Army T&E Executive provides sole HQDA approval signature within 10 days HQDA(TEO) sends TEMP to OSD (Army goal is within 30 calendar days from PM signature) OSD staffing and approval objective (45 calendar days from receipt to return to HQDA(TEO)) Note: TEMPs are required to be updated at Milestone C and Full Rate Production Decision Review, but should also be updated when the program baseline has been breached, when the associated JCIDS document or Information Support Plan has been significantly modified, or on other occasions when the program is significantly changed or restructured. Evolutionary acquisition programs may require additional updates to ensure that the TEMP reflects the currently defined program. When a program changes significantly, the TEMP due date will be negotiated between the PM and the DoD Component TEMP approval authority. In the case of programs under OSD T&E oversight, the negotiations will take place between the PM, DoD Component TEMP approval authority, USD(AT&L), and DOT&E In either case, the goal should be to update the TEMP within 120 days (Source: Interim Defense Acquisition Guidebook, 7 July 2009, Paragraph )

16 Without HQDA Involvement
TEMP Approval Process Without HQDA Involvement T&E WIPT develops TEMP (all members empowered) Final T&E WIPT TEMP ‘Signing Party’ meeting (members sign TEMP Coordination Sheet) PM delivers TEMP with all concurrence signatures to MDA or Designated TEMP Approval Authority for approval PM distributes TEMP after MDA approval Note: TEMPs are required to be updated at Milestone C and Full Rate Production Decision Review, but should also be updated when the program baseline has been breached, when the associated JCIDS document or Information Support Plan has been significantly modified, or on other occasions when the program is significantly changed or restructured. Evolutionary acquisition programs may require additional updates to ensure that the TEMP reflects the currently defined program. When a program changes significantly, the TEMP due date will be negotiated between the PM and the DoD Component TEMP approval authority. The goal should be to update the TEMP within 120 days (Source: Interim Defense Acquisition Guidebook, 7 July 2009, Paragraph )

17 Army TEMP T&E WIPT Coordination Process
Acquisition Strategy Requirements Documents (CDD/CPD) STAR, TDS, and ISP Approved Critical Operational Issues and Criteria Combat Developer PM T&E Manager Training Developer T&E WIPT Meetings* T&E WIPT Meeting Input Draft TEMP Sent to T&E WIPT Other Agencies OSD Action Officers SMDC ATEC HQDA TEO ASA(ALT) ASA(ALT) ILS DCS, G-1 DCS, G-2 DCS, G-3 DCS, G-8 CIO/G-6 All issues will be resolved before the T&E WIPT ‘Signing Party!’ T&E WIPT TEMP Concerns/Feedback Final T&E WIPT ‘Signing Party’ DTC OTC AEC Meetings repeated until TEMP coordination complete T&E WIPT Coordinated TEMP

18 TEMP Staffing and Approval Process OSD T&E Oversight Programs
T&E WIPT & HQDA Staff Coordinated TEMP PEO or Developing Agency DCS, Combat Developments, TRADOC (or FP) ATEC PM signs & submits PM forwards to TEO TEO Staffs (if necessary) Army T&E Executive Approval TEO Forwards Goal: Simultaneous concurrence at T&E WIPT ACAT I or OSD T&E Oversight PM and T&E WIPT No MDA/PEO TEO Distributes TIMELINES PM Signature to Army T&E Executive Approval Signature – 30 Calendar Days or less OSD Signature Objective – 45 Calendar Days or less Yes OSD Approval - Dir, DT&E - Dir, OT&E

19 TEMP Staffing and Approval Process
Multi-Service OSD T&E Oversight Programs – Army Lead ACAT I or OSD T&E Oversight Signatures PEO or Developing Agency TRADOC ATEC Participating Service: OTA PEO or Dev Agency User Rep T&E WIPT & HQDA Staff Coordinated TEMP PM Submits Yes TEO Forwards Other Service Approvals No Army T&E Executive Approval MDA* Approval TRADOC ATEC T&E WIPT Other Services TEO Forwards OSD Approval Dir, DT&E - Dir, OT&E Program Manager TEO Distributes TIMELINES PM Signature to Army T&E Executive Approval Signature – 30 Calendar Days or less OSD Signature Objective – 45 Calendar Days or less * ACAT II programs having MDA delegation also have TEMP delegated approval authority

20 TEMP Staffing and Approval Process
Multi-Service OSD T&E Oversight Programs – Army Participant ACAT I or OSD T&E Oversight T&E WIPT Equivalent Coordinated TEMP Army PM, PEO or Developing Agency Signs TRADOC Signs ATEC Signs Yes Lead Service Acquisition Executive Army T&E Executive Signs TEO Staffs No HQDA Staff Coordination Lead Service Acquisition Executive TRADOC ATEC T&E WIPT Other Services OSD Approval Dir, DT&E - Dir, OT&E Lead Service Acquisition Executive Army PM, PEO or MDA TEO Distributes TIMELINES TEMA Receipt to Army T&E Executive Approval Signature – 20 Calendar Days or less OSD Signature Objective – 45 Calendar Days or less

21 TEMP Staffing and Approval Process
Non-OSD T&E Oversight – ACAT II, III, & Army Special Interest Programs T&E WIPT (& HQDA Staff) Coordinated TEMP Program Manager Obtains Required Concurrences MDA (or Designee) Approves TEMP Program Manager Distributes to T&E WIPT

22 TEMP “Do’s” and “Don’ts”
1. DO ensure Core T&E WIPT members agree with any change(s) required to obtain TEMP approval after providing their coordination signature. 2. DO enforce configuration control over the TEMP. 3. DO have Part IV (Resources) based upon Parts II and III. 4. DO have the Requirements / Test Crosswalk Matrix as Attachment 1. 5. DO know the TEMP Approval Authority before developing the TEMP. 6. DO be very aware of COIC & CDD / CPD staffing approval timelines. 7. DO treat the TEMP as a CONTRACT among all Stakeholders!!! 8. DON’T attempt to gain TEMP approval before obtaining COIC approval. 9. DON’T “Concur With Comment” on TEMP’s T&E WIPT Coordination Sheet. 10. DON’T have TEMP simply refer to the System Evaluation Plan. Leave you with these points. Keep each in mind as you execute your ATEC System Team (AST) member RESPONSIBILITIES. Discuss each [if time permits.] Eight HQDA Offices: TEMA for the Army T&E Executive ASA(ALT) has 2 (ILS Division & Sys Mgmnt Integration) CIO/G-6; G-1, G-2, G-3/5/7, G-8 22

23 T&E WIPT “Do’s” and “Don’ts”
Do invite HQDA offices to participate in the T&E WIPT Do obtain appropriate OSD office(s) involvement early Do know T&E WIPT issues & be empowered Do hold program to the rigorous T&E process Do base positions on facts, vice emotions Do elevate an issue if unable to resolve within the T&E WIPT Don’t dismiss T&E WIPT meetings as unimportant Don’t dismiss T&E WIPT members’ comments without reasoned debate Don't allow unresolved issues to fester within the T&E WIPT (that is, take extra effort to resolve one-on-one and face-to-face) Don’t disband the T&E WIPT after TEMP approval Can apply to ASTs, too! 23

24 “Deadly Sins” Affecting TEMP Approval
Premature Submission: Submitting a TEMP for approval when an issue(s) precluding TEMP approval exists Failure to Pre-Coordinate: HQDA: T&E Office and HQDA Staff OSD: DT&E and DOT&E Incomplete: Requirements / Test Crosswalk Matrix missing Classified data not integrated - submitted separately Too many TBDs in Part IV (Resources) COIC: Not HQDA approved and incorporated Size: Body of plan, excluding annexes, not kept to a manageable length Annexes: Not kept to a minimum Reference Documents: ICD / CDD / CPD & STAR not approved / unavailable 24

25 OSD TEMP Process

26 OSD T&E Oversight Programs TEMP Signature Process
SECDEF As of October 2008 USD(AT&L) NII DOT&E (OT TEMP CO- APPROVAL W/ DS) DDR&E DOT&E AO TEO Forwards HQDA Approved TEMP NOTES: - Cognizant OIPT Chair signs for DT&E ASD(NII) approves C3I, IT, & Space Programs Dir, DT&E approves for weapon programs 26

27 OSD TEMP Process Develop the TEMP through the T&E WIPT process
1 of 4 Develop the TEMP through the T&E WIPT process Add the DT&E/DOT&E action officers to your T&E WIPT roster Maintain informal dialog with the DT&E and DOT&E action officers, as well as their support contractor Use Defense Acquisition Guidebook to prepare TEMP Provide sufficient details about your future T&E activities (Part III) Be specific, but eliminate the “fluff” TEMP is a master strategy document, not a detailed test plan. This is the section that answers the “how,” not the “why” Failure to include OSD AO’s in your TEMP development & staffing or to resolve their issues will guarantee a TEMP that is ‘Dead On Arrival’ at OSD. 27

28 OSD TEMP Process 2 of 4 Work with the Defense Intelligence Agency action officers Plan to test the system against the threats in the System Threat Assessment To reduce the size of TEMPs, use pointers to other documents where more details are present Software development plans Modeling and simulation plans Logistic support plans Include Modeling & Simulation in your T&E planning. Reduces cycle time & costs 28

29 OSD TEMP Process 3 of 4 Work to combine/integrate DT with OT to achieve mutual objectives at reduced costs: Successfully completing independent OT&E (Title 10 statute requirement) is the final exam in T&E planning Be careful to not give up [PM] control of your development test and evaluation plan to the operational test and evaluation side of the house There will still be a requirement for a dedicated and controlled OT&E event The DT&E is part of your risk reduction measures that are needed to ensure the system is ready for operational testing If the size and scope of the dedicated OT&E can be reduced without jeopardizing your DT&E objectives, then combined DT and OT makes sense NOTE: Early user involvement in DT. Make greater use of modeling and simulation . Make DT more rigorous to ensure success during IOT&E. 29

30 OSD TEMP Process 4 of 4 Submit electronic version of draft to OSD for staff review Identifies “showstoppers” before TEMP enters the Service signature chain Let OSD know early if you need an extension to your TEMP Due Date 45-day window for OSD signatures 30

31 Acknowledgement & Feedback
This module is a part of the T&E Managers Committee (TEMAC) T&E Refresher Course. Request your feedback. Please contact - T&E Office Deputy Under Secretary of the Army 102 Army Pentagon Washington, D.C (703) / DSN: 332


Download ppt "Test and Evaluation Master Plan"

Similar presentations


Ads by Google