A few DT&E Areas of focus EMIS 7307 T&E Part 3 A few DT&E Areas of focus Life testing. To make sure that the system will not fail prematurely due to fatigue, aging, long term environmental exposure etc. Time consuming and expensive. Consider accelerated life testing.

A few DT&E Areas of focus EMIS 7307 T&E Part 3 A few DT&E Areas of focus Design evaluation/verification. See if critical characteristics are achievable Provide data making the hardware or software more rugged in order to meet specs. Eliminate risks. Assist design evolution. Ensure component and system meet spec.

A few DT&E Areas of focus EMIS 7307 T&E Part 3 A few DT&E Areas of focus Design limit testing. Based on mission profile. Operated at the limits. May be destructive. Interpolation and extrapolation may be required.

A few DT&E Areas of focus EMIS 7307 T&E Part 3 A few DT&E Areas of focus Reliability Development testing. Also called reliability growth testing, test- analyze-fix-test (TAFT), of test analyze and fix (TAAF). Test under mission profile environments. Things fail. Determine and make fix. Commence testing again. Data allows updating reliability model for current predictions.

Built-In Test Difficult part of design Difficult part of I&T EMIS 7307 T&E Part 3 Built-In Test Difficult part of design Difficult part of I&T Difficult part of T&E Issues include: Level of fault isolation Level of automatic fault detection Induce failures for the T&E?

Contractor perspective. EMIS 7307 T&E Part 3 OT&E Contractor perspective. Usually involvement is limited to being the depot - a common approach. If the program/system has contractor logistics support (CLS) then their performance is tested here too.

OT&E government program office perspective. EMIS 7307 T&E Part 3 OT&E OT&E government program office perspective. ‘Make or break’ time. This is the program office’s final exam! Feels like you’re sending your kid to the army! Testing is done at arms length from the program office. No program office involvement except to pay for it!

OT&E government operational tester perspective. EMIS 7307 T&E Part 3 OT&E OT&E government operational tester perspective. An agent of the highest authority. One phrase in the final report and the system does not go to FRP. Not operationally suitable, not operationally effective.

OT&E Supports Reviews and MSs Too EMIS 7307 T&E Part 3 OT&E Supports Reviews and MSs Too

Critical Operational Issues (COI) EMIS 7307 T&E Part 3 OT&E Critical Operational Issues (COI) A key operational effectiveness or operational suitability issue that must be examined in operational test and evaluation to determine the system’s capability to perform its mission. A critical operational issue is normally phrased as a question to be answered in evaluating a system’s operational effectiveness and/or operational suitability. Identifying COIs Ask: “How well does the system perform a particular aspect of its mission?” “Can the system be supported logistically in the field?” Other issues arise from questions asked about system performance or how it will affect other systems with which it must operate. Critical issues provide focus and direction for the operational test. Identifying the issues is analogous to the first step in the system engineering process — that is — defining the problem. When critical operational issues are properly addressed, deficiencies in the system can be uncovered.

EMIS 7307 T&E Part 3 OT&E COIs are the basis for sub-objectives known as Measures of Effectiveness (MOEs) which in turn yield actual test measurements called Measures of Performance (MOPs).

EMIS 7307 T&E Part 3 OT&E

EMIS 7307 T&E Part 3 OT&E Test Realism Key issue, if it isn’t realistic, it’s not good OT&E. Realistic production representative version Realistic environment. Realistic tactics and threats. Realistic logistics. Realistic training. Realistic operators - not the ‘golden crew’.

Live Fire Testing (LFT) EMIS 7307 T&E Part 3 Live Fire Testing (LFT) A lot like OT&E. Mandated by congress. Prior to full rate production. Appropriate threat weapons used against our. systems, configured for combat. Or: Our munitions are fired against a threat target.

Before IOT&E, data are gleaned wherever possible. EMIS 7307 T&E Part 3 Before IOT&E, data are gleaned wherever possible.

Remember the definitions of test, evaluation! EMIS 7307 T&E Part 3 Evaluation Remember the definitions of test, evaluation! Establishing and maintaining a clear audit trail is essential from: System requirements through critical issues. Evaluation criteria. Test objectives. Measures of effectiveness. Data are used from all sources, not just tests Design reviews, inspection, simulations, etc.

Evaluation Issues: Criteria: EMIS 7307 T&E Part 3 Evaluation Issues: Questions that require answers during the acquisition process to support. Development of acquisition strategy. Refining performance requirements and designs. MS decisions. Criteria: Standards of assessment for: Technical and operational effectiveness/suitability characteristics. Resolution of operational issues.

Evaluation Key Performance Parameters (KPPs) from the ORD yield: EMIS 7307 T&E Part 3 Evaluation Key Performance Parameters (KPPs) from the ORD yield: Critical issues. Must be answered before the systems overall worth can be estimated or evaluated. Of primary importance to the milestone decision authority. Evaluation issues. Technical implies DT, if possible. Operational implies analysis, modeling, simulation. inspection, demo or test.

Evaluation Test issues. EMIS 7307 T&E Part 3 Evaluation Test issues. Issues that cannot or should not be addressed by any other method. Criteria – Required level of technical performance and operational effectiveness, suitability or supportability. Objective: Goal, a preferred level of capability. Not addressed by OT&E. Threshold: Minimum acceptable level of acceptability. System is not worth having below this level. One function of T&E is to verify meeting the threshold.

Generic Evaluation Process EMIS 7307 T&E Part 3 Generic Evaluation Process

Is the real thing always better? EMIS 7307 T&E Part 3 Is the real thing always better?

EMIS 7307 T&E Part 3 How to decide?

Modeling and Simulation Support Testing EMIS 7307 T&E Part 3 Modeling and Simulation Support Testing Help discover sensitive areas of special interest. Can go where real testing cannot. Can be updated to improve validity based. on selected test data. Can be used for design, test and operations.

The final consists of two sections. EMIS 7307 T&E Part 3 The final consists of two sections. An in-class section (25 %) 25 questions similar to midterms Mostly concerning topics since the 2nd midterm A four question essay section which you prepare and bring to the final (75 %) Use MSWord and size 12 or 14 Times New Roman font. Both substance and grammar will be graded. Each person’s offering must be unique. Number of words is approximate.

Essay portion of final exam EMIS 7307 T&E Part 3 Essay portion of final exam (1)What is the role of simulation and modeling in the I&T process? (250 words) (2)What is the role of simulation and modeling in the T&E process? (250 words)

Essay portion of final exam EMIS 7307 T&E Part 3 Essay portion of final exam (3)Pretend I’m your boss. Convince me to change our company’s approach to I&T. Tell me what’s wrong with our I&T. Provide pros and cons of how you would revise it, and then explain why the pros outweigh the cons. Show how this revised approach will improve our overall systems engineering flow and ultimately provide a better product. Include suggested organizational changes that will be required to implement this change. (250 words, plus a table and /or chart)

Essay portion of final exam EMIS 7307 T&E Part 3 Essay portion of final exam (4)Discuss the pitfalls, dangers and risks of the “we’ll make up lost time” and “we’ll fix the problems during I&T” mentality. (250 words) The remaining portion of the final will be administered in the same fashion as the midterms. After completing the remainder of the final, attach your essays to it for one combined submittal.