Thar’s Gold in Them Thar AAR/IPs

Welcome and Agenda Purpose and Value of After Action Reports How to develop an Improvement Plan Esther Corwin, MEP Exercise Training Officer, Texas Division of Emergency Management Esther.Corwin@dps.texas.gov 512-424-2198 Contact Info is on handout

What is an AAR A consolidation of information gathered during the testing and evaluation of a community’s emergency operations plan through an exercise Provides feedback to executive leadership, participating entities and governing agencies in the achievement of the exercise objectives and overall capabilities of the community Information gleaned from this process, identifies and guides future improvement actions

Why Write an AAR Creates a multi-agency process to review an exercise or incident General exercise identification information, but also includes evaluation information Facilitates honest and critical appraisal of actions based on plans and policies

Where Do We Look OR Exercises to simulate a stressful realistic scene Exercises to evaluate a new or seldom used equipment or skill Incidents presenting unexpected challenges Incidents having “less than expected” results OR Incidents employing innovative actions Incidents with unexpected positive results

Exercise Evaluation is Based On Community policies, Emergency Operations Plan, agency SOPs and SOGs, MOUs, etc. Exercise Evaluation Methodology S.MA.R.T. objectives Exercise Evaluation Guides Observational Data Hotwash Participant Feedback

Incident Evaluation is Based On Community policies, Emergency Operations Plan, agency SOPs and SOGs, MOUs, etc. Incident Action Plans Briefings, De-Briefings Observational Data Participant Feedback

Types of Observational Data Descriptive reporting Typically yields reliable data. Inferential reporting Includes terms like “adequate” or “timely” Yields inconsistent data Evaluative reporting Includes terms like “efficient” or “successful” Difficult to reliably to collect

Three Levels of Analysis

Three Levels of Analysis Capability Being Evaluated Critical Resource Logistics & Distribution Onsite Incident Command Level of Evaluation Capability-Level Analysis Resource Management – Identify, dispatch, mobilize, track, demobilize and pay for resources Direct/Control incident management activities. Activity-Level Analysis Identify the resources based on incident need Develop all necessary components of an IAP and obtain approvals Task-Level Analysis Inventory resources by type/ category that are available to support this incident Establish incident objectives and priorities and operational periods

Exercise Evaluation Guides Identify the activities, tasks, and performance measures to be observed Developed and customized before the exercise Evaluators complete the EEG by: Logging times and actions accurately Decision making processes and participants Player roles and responsibilities Coordination and cooperation How actions were performed, resources involved Documenting issues and recommendations

Data Analysis The goal is to evaluate the ability of involved functions to perform target capabilities

Preliminary Analysis Evaluators organize observations into key issues and a chronological narrative At a minimum, should include: Description of the assigned function or operation, analyzed by capability, activity, and task Documented record of significant observed actions

Review Objectives Review exercise objectives What was the intent of the objective? What would demonstrate the successful performance of the objective? If the objective was not met, what factors contributed to this result? Review incident operational objectives Review exercise objectives What was the intent of the objective? What would demonstrate the successful performance of the objective? If the objective was not met, what factors contributed to this result? Review incident operational objectives

4 Steps of Data Analysis Identify issues Determine root cause Compare performance data to standards Identify differences Determine consequences Determine root cause Discuss conditions leading to observed behavior The “Why” staircase Fool’s Gold

Why Staircase Each step should explain the step above … Why did it happen? Each step should explain the step above … Why did that happen? Why was that? And why was that? And why was that? ROOT CAUSE … down to the underlying root cause.

4 Steps of Data Analysis Develop recommendations Sustain and improve Short and long term What needs to be changed and how Give references Capture lessons learned Innovative practice or a piece of knowledge gained from experience Provides guidance for approaching a similar problem in the future Save time, conserve money, and accelerate preparedness

Draft AAR Provides feedback to participating entities and governing agencies regarding the achievement of objectives and overall capabilities Records recommendations for improvement Establish consensus and buy-in on next steps

The AAR Suggested AAR format includes: Executive Summary Overview Exercise Goals and Objectives Analysis of Capabilities Demonstrated Conclusion

Observations and Recommendations References Analysis For improvements, list consequences of action or inaction Recommendations Observations References Analysis For improvements, list consequences of action or inaction Recommendations Observations References Analysis For improvements, list consequences of action or inaction Recommendations Observations References Analysis For improvements, list consequences of action or inaction Recommendations

After Action Conference Planning Team, evaluators, and stakeholders meet to review and refine draft AAR Evaluation leads to a disciplined process for implementing improvement actions and continually strengthening preparedness

After Action Conference Address identified issues Develop specific improvement actions Prioritizes action items Assigns responsibility to track implementation

Developing Improvement Actions These questions aid development: What changes need to be made to plans and procedures to improve performance? What changes need to be made to organizational structures? What changes need to be made to leadership and management processes? ?

Developing Improvement Actions More questions to aid development: What training is needed? What changes to, or additional equipment are needed? What lessons can be learned that will direct how to approach a similar problem in the future?

Improvement Benchmarks Must be clearly defined and attainable Examples include: Number of personnel trained in a task Percentage of equipment that is up-to-date Finalization of an interagency agreement within a given amount of time Include concrete deadlines to track progress toward full implementation

Activity – Improvement Development Older elementary school in a rural area just before dismissal on a spring day, parents are arriving to take students home NWS issues tornado warning estimating less than 10 minutes before impact Principal activates “bell” for sheltering, but students/parents confuse it with dismissal Some parents enter school to take child and “run” from the storm As a large group, given a scenario and issue, discuss possible recommendations and develop a simple improvement action.

Activity – Develop Improvement Issue: Student Safety Basic Recommendation: Develop something to assist the principle in communicating emergency information Changes to plans and procedures Changes organizational structures Changes to leadership/ management processes What training is needed Changes or additional equipment Lessons learned for the future Jarrell Tornado May 27, 1997

Improvements make things better than before Summary Objectives and Evaluation Methodology are the foundation of Improvement Planning Root Cause analysis helps find the key policy, procedure, training or equipment to which an improvement could be made Applies to Exercises and actual Incidents Improvements make things better than before

Any question can lead to a good idea. Synergy is the Energy of People. Questions ??? Any question can lead to a good idea. Synergy is the Energy of People. Esther Corwin, MEP Exercise Training Officer, TDEM Esther.Corwin@dps.texas.gov 512-424-2198