1. EMIS 7307 T&E Part 2 1 Documents in flux. MNS - Mission need statement –Non system specific, a needed capability. Being replaced by Initial Capabilities Document (ICD). –ICD identifies the capability gap, describes the evaluation of different approaches and proposes an approach to pursue. - Supports the MS A decision. DOD 5000 Changes

2. EMIS 7307 T&E Part 2 2 ORD - Operational requirements document Minimum acceptable capability of possible system. ORD is being replaced by Capabilities Development Document (CDD) at MS B and Capabilities Production Document (CPD) for MS C. CDD- Contains operational performance parameters to design and build an affordable and useful increment includes KPPs (Key performance parameters). CPD - Contains production attributes of a single increment. DOD 5000 Changes

3. EMIS 7307 T&E Part 2 3 Revised DoD 5000 ICDCDDCPD

4. EMIS 7307 T&E Part 2 4 DoD 5000 with planned upgraded capabilities called increments. A spiral is used when upgrades are expected but are TBD. AoA Analysis of Alternatives DAB Defense Acquisition Board JROC Joint Requirements Oversight Council

5. EMIS 7307 T&E Part 2 5 A significant number of documents have relevancy to T&E.

6. EMIS 7307 T&E Part 2 6 Test Documentation Relationships Analogous to A, B and C spec relationships

7. EMIS 7307 T&E Part 2 7

8. EMIS 7307 T&E Part 2 8 Test Program Documentation

9. EMIS 7307 T&E Part 2 9 NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) Test and Evaluation Master Plan (TEMP)

10. EMIS 7307 T&E Part 2 10 NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) TEMP

11. EMIS 7307 T&E Part 2 11 NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) TEMP

12. EMIS 7307 T&E Part 2 12 Test Program Documentation System Test Plan –Describes ‘big picture’ of all the testing for the whole system, not a management document. –This is the foundational technical test document from which subservient plans are derived. –Includes laboratory, component, subsystem, system, software etc. Detail Test Plan –In a large system, each of these topics (laboratory, component, subsystem, system, software etc.) is likely to be a volume or a book of its own.

13. EMIS 7307 T&E Part 2 13

14. EMIS 7307 T&E Part 2 14 Other Documents Test Procedures –Implements a specific test with carefully constructed step by step directions. –Completeness is paramount as unplanned steps may invalidate the data being collected. –May be augmented by test cards or scripts. –Formality in execution may require QC and/or customer sign-off. –Dry run - find problems in procedure or UUT and fix. –Document changes during execution.

15. EMIS 7307 T&E Part 2 15 Other Documents Test Reports –Quick look - very important for management Based on observation and limited, if any, data analysis May affect next day’s testing - consider email? –Final report includes:

16. EMIS 7307 T&E Part 2 16 Test/Program Interrelationships

17. EMIS 7307 T&E Part 2 17 More about types of testing. DT&E again. Contractor perspective. –Demonstrates design and development and in the end-game, completeness i.e. meets spec. –Reduce risks. High-risk components. Intra-subsystems. Inter-subsystem.

18. EMIS 7307 T&E Part 2 18 DT&E government perspective. –Demonstrates spec compliance. –Used for acceptance of the EDM(s). –Estimate the military utility. –Confirms ready for field testing (OT&E).

19. EMIS 7307 T&E Part 2 19 DT&E by phase. –Prior to program start. Determine or at least estimate that the technology is feasible. –Concept and technology development. Assist in selecting between possible alternative system concepts. Info for SRR and program start decision.

20. EMIS 7307 T&E Part 2 20 DT&E by phase. –During System Development and Demonstration. Risks can be reduced to acceptable levels. From here on engineering efforts instead of experimental efforts. Supports decision to build an EDM. System I&T is a part of this. Includes EDM acceptance testing. Includes the testing of the EDM to provide data for the LRIP decision.

21. EMIS 7307 T&E Part 2 21 DT&E by phase. –During production and deployment phase. Prelude to IOT&E. Navy calls TECHEVAL. –During full rate production and operations. To verify corrections and/or improvements. In terms of calendar time and manhours consumed, DT&E is larger than OT&E and has mostly contractor involvement.

22. EMIS 7307 T&E Part 2 22 Another view of DT&E

23. EMIS 7307 T&E Part 2 23 DT&E For the MSs

24. EMIS 7307 T&E Part 2 24 OT&E is complex in it’s simplicity!

25. EMIS 7307 T&E Part 2 25 Types of OT&E Early Operational Assessment (EOA). –Performed for the Program Office and other decision makers as a indicator of how operational things are “shaping up”. –Done by operational testers but may be mostly analytical based on DT&E provided info. Initial OT&E –Supports full rate production decision. –Called OPEVAL by Navy.

26. EMIS 7307 T&E Part 2 26 Types of OT&E cont’d. Follow-on OT&E (FOT&E) –After full rate production. –Conducted during fielding. –Checks on correction of IOT&E deficiencies.

27. EMIS 7307 T&E Part 2 27 More comparisons

28. EMIS 7307 T&E Part 2 28 Lessons for the PM that you should know too.