Work Train Assess E2E Lean Process Review - Evaluate Continuous Evaluation Cycle Continuous Cycle Begins with the “Work” Naval Education and Training Command (NETC) Maintain (Life-Cycle Maintenance) Evaluate Plan Draft NETC N7 Training Effectiveness Evaluation Plan (TEEP) (Program Management) Assess E2E Lean Process Review - Evaluate In a Systems Approach to Training A TEEP is Essential Perform Training Content Planning/Development/ Implementation Evaluate/Select Training Intervention Alternatives Content Surveillance/ Feedback / Maintenance A well defined Training Effectiveness Evaluation Plan (TEEP) allows us to: A TEEP is a quantitative evaluation strategy that identifies measureable performance outcomes and will identify Reaction, Summative, Behavior, and Result Evaluation criteria. Reaction Evaluations Level 1 Summative Evaluations Level 2 Behavior Evaluations Level 3 Result Evaluations Level 4 Data Collection Plan Comm Plan SWOT TEEP SIPOC Point 1 – Draft NETC N7 position regarding TEEP Policy by 1 Dec.

Work Train Assess E2E Lean Process Review – Evaluate Continuous Evaluation Cycle Continuous Cycle Begins with the “Work” Naval Education and Training Command (NETC Maintain (Life-Cycle Maintenance) Evaluate Plan Draft NETC N7 Training Effectiveness Evaluation Plan (TEEP) (Program Management) E2E Lean Process Review – Evaluate Assess In a Systems Approach to Training A TEEP is Essential Perform Training Content Planning/Development/ Implementation Evaluate/Select Training Intervention Alternatives Content Surveillance/ Feedback / Maintenance A well defined Training Effectiveness Evaluation Plan (TEEP) allows us to: 10/05/18 Strategic IPR Update Collaborating with N75 to review TRAINING EFFECTIVENESS EVALUATION for the NAVAL CONSTRUCTION FORCE CRANE TRAINING SYSTEM N95-TEE-F-80-1702 as a representative model standard Prototype TEEP checklist job aide validating data element requirements Drafted NETC Training Effectiveness Evaluation Plan Instruction posted to APAN Developing flowcharts and job aides for TEEP processes: TEEP Part 1 - Front Matter (e.g., Program data, POCs, RFT, etc.) TEEP Part 2 - Planned Evaluation (e.g., TSD current state, IPRD, TD desc.) TEEP Part 3 - Evaluation Strategy (e.g., data col. plan, empirical basis and controls) TEEP Part 4 - TEE Agent (TEEA) selection TEEP Part 5 - TEEP Approval process TEEP Part 6 - TEEP Results (e.g., findings, feedback, POA&M) A TEEP is a quantitative evaluation strategy that identifies measureable performance outcomes and will identify Reaction, Summative, Behavior, and Result Evaluation criteria. Reaction Evaluations Level 1 Summative Evaluations Level 2 Behavior Evaluations Level 3 Result Evaluations Level 4 Data Collection Plan Comm Plan SWOT TEEP SIPOC NETC N5 Learning Assessment Framework LAF Cooperative Research and Development Agreement (CRADA) focus on improving life cycle management of technical training data that can be configured to the systems it supports OPNAV N12 TEEP development

Background: OPNAVINST 1500 Background: OPNAVINST 1500.76D draft incorporates a discretionary requirement for Training Effectiveness Evaluation Plans, additionally, NETC EAP governance is under review and revision Problem: NETC COI’s currently lack standardized, comprehensive governance wrt Evaluation and assessment programs and processes resulting in: An inability to determine if NETC COI content assessments align with requirement/resource sponsor program/end user activity performance requirements Standardized external feedback from end user activities wrt trainee performance An inability to use curriculum surveillance feedback to assess health of the curriculum and improve COI’s Solution: Develop standardized evaluation/assessment feedback programs that will : Quantitatively - reduce takt time to 1collect, receive, analyze end user feedback, 2plan/execute content change/revision processes, 3achieve end user performance metric requirements, 4reduce # of formal assessment time Qualitatively – 1align content assessment processes and standards with end user performance metric requirements, 2identify authentic assessment resources takt time: The total available work time per day (or shift) divided by customer-demand requirements per day (or shift). Takt time sets the pace of production to match the rate of customer demand. Examples: If your customers demand 480 spark plugs per day and your production line operates 960 minutes per day, takt time is two minutes If customers want two new contracts written per month, takt time is two weeks. Takt time increases if customer buys fewer products. Takt time decreases if customer buys more products. Ideally cycle time should equal takt time.

NCF TEE Review Three phase process Is there a TEEP/with template data elements completed Are planned evaluation, evaluation strategy (including analytics) specifically defined Does NETC training output meet: Requirements sponsor Kirk. 4 org goals/objectives, TFOM, readiness DRRS, #NECs, Sup/Grad survey feedback wrt LO’s, training solution treatments Kirk. 3

TEEP Representative Model Sample N95-TEE-F-80-1702 TRAINING EFFECTIVENESS EVALUATION for the NAVAL CONSTRUCTION FORCE CRANE TRAINING SYSTEM PREPARED BY: NAVAL FACILITIES ENGINEERING COMMAND TRAINING SUPPORT AGENCY (EX8) PREPARED FOR: DEPUTY CHIEF OF NAVAL OPERATIONS FOR WARFARE SYSTEMS NAVAL EXPEDITIONARY WARFARE DIVISION (N95) MANPOWER AND TRAINING BRANCH (N959) DISTRIBUTION STATEMENT: Approved for public release. Distribution is unlimited. MARCH 2018