1. NA-40 Training Overview January 31, 2014

2. EOTA Qualification Standards

3. Qualification Process Analysis EvaluationTasks Curricula Exercise Objectives 3

4. ADDIE Model Systematic instructional design model used globally: Analysis Design Development Implementation Evaluation

5. Analysis Performs Feasibility Study Identifies learning problem, the goals, and objectives The audience’s needs Existing knowledge Learning Environment Any constraints Delivery options Timeline

6. Design A systematic process of specifying learning objectives based off of tasks Design Document – outlining details of course or exercise, including types of evaluation Exercise Design Document

7. Development The actual creation (production) of the content and learning materials based on the Design Phase/Document Exercise Design Document

8. Implementation Training is executed utilizing ISO registered processes for:  Web-Based/Computer-Based  Instructor-Led  Self-Study  Blended  Webinars  Mobile Training Teams  SharePoint and Learning Management System (Defensibility)  Repository  Version Control  Easy access from remote locations

9. Evaluation Formative Is present in each stage of the ADDIE process Monitors student learning to provide ongoing feedback throughout curriculum Summative Evaluates student learning at the end of the course Tests Feedback from students

10. Kirkpatrick’s Levels Four Levels of Evaluation 1.Reaction - immediate satisfaction 2.Learning - results in increase of knowledge/skills/attitudes 3.Behavior - transfer from classroom to job (3-6 months post training) 4.Results - can be measured in reduced cost, improved quality, increased production, efficiency, etc.

11. Learning Management System Handles course distribution and course management Facilitates course exams and allows item analysis Course registration/repository Creates custom reports Generates certificates

12. NA-40 Training EOTA DOE MANAGEMENT RESPONSIBILITY - Provide Direction/Set Standards - Quarterly Reviews - Reports/Metrics - Provide SMEs - Monitor Students/Customers Satisfaction Ratings RESOURCES - Experienced Staff - Objective Assessments - Standardization - Formal Review of Courseware - Partnerships - Training Technology - Tracking/Reporting Capabilities - Document Repositories - Collaboration Tool (SharePoint) QUALITY - Maintain ISO Registration - Corrective/Preventive Action - Audits/Reviews (internal/external) INPUT - Management Support - Feedback - NA-40 Program Knowledge - Subject Matter Expertise - Lessons Learned from Training, Exercises and Real World Events OUTPUT - Defensibility - Clearinghouse - Repository for all NA-40 Courses - Credibility - Customer Trust - Consistent Products - Standardized Methods - Reliability NA-40.2 Technical Training Liaison

13. Exercise Planner Support EOTA offers exercise capabilities and a performance-based exercise program for exercise design, development, conduct, evaluation, and improvement planning Standardized policies Methodologies Terminologies “Reachback” to Subject Matter Expertise On staff controllers, evaluators, and facilitators

14. Exercise Types Discussions-based Seminar Workshop Tabletop Operations-based Drill Functional Exercises Full-Scale Exercises

15. Implementation Development of resource documents Situation Manual Exercise Plan Controller/Evaluator Handbook MSEL Player Handout Exercise Evaluation Guides (EEGs) After Action Report

16. After Action Reports After Action Report (2 sections) Capture observations/recommendations (process driven) What was accomplished What worked well What requires improvement (tactics/techniques) Participant Reaction Form Summary Qualitative Quantitative Observations Recommendations After Action Report Participant Reaction Form Summary

17. Corrective Action Plan ISO registration includes: Corrective Actions Preventative Actions Improvement Opportunities Assistance with CAP for Watch Office Discussion Path forward Outline responsibilities Tracking Corrective Action Plan

18. EOTA Exercise Progress Conducted a Feasibility Assessment Analysis Completed Needs Analysis ( helped identify if training was truly the answer) Training Requirements Analysis (focused on what the regulatory requirements are that pertain to training and or areas of responsibility) Resulted in Standard Curriculum  Note: Review sample DIF surveys

19. Exercise Curriculum Web-Based Training EXR101DW: Intro to the DOE/NNSA Exercise Program EXR102DW: Exercise Design EXR103DW: Exercise Scenarios EXR104DW: Completing the Exercise Plan EXR105DW: Post Exercise Activities EXR106DW: Optimizing Your Site Exercise Program EXR107DW: Controlling Security Exercises EXR108DW: Designing Exercise Evaluations Basic Exercise Controller and Evaluator Tutorial (ORISE) Instructor-Led Training EXR 207: Controlling Security Exercises Practicum EXR 231: Exercise Design Laboratory (Pilot course January 14-16, 2014)

20. Where we are going… Clearinghouse – Validate content and format for consistent results across all vested customers Repository for All NA-40 courses – One area to look for all NA-40 available training as well as direction to locate other training offerings available (FEMA, DHS) Defensible Training