CARE International: After Action Reviews Rapid use of evidence in the humanitarian context for efficient learning and forward accountability by Uwe Korus CARE International Emergency Group (CEG) Monitoring, Evaluation and Accountability Coordinator emergencyQA@careinternational.org November, 2014

Humanitarian Response review processes overview 1. Internal monitoring & self-assessment : Regular response level monitoring Participatory accountability reviews (Rapid Accountability Review (RAR), scorecard) After Action Reviews (AAR) 2. External Reviews : Real Time Evaluations (RTE) Interagency peer review (e.g. DEC, clusters) Independent evaluations Specific case studies – good practice reviews November 29, 2018

CONTEXT After Action reviews (AARs) introduced in the 1990s to humanitarian actors emerged as appropriate mechanisms to emhance action and wider practice in the sector. Real Time Evaluations (RTEs) became a more common approach to action oriented learning in the humanitarian sector in the early 2000s. Meta-evaluation in 2007 revealed the findings and recommendations from CARE‘s evaluations and other reviews of humanitarian projects remained largely unknown, unanalysed, and therefore did not adequately contribute to learning and improved practice. November 29, 2018

Concepts After Action Review (AAR): Originating in military in 70s AARs structure questions that help reflect and learn after undertaking an activity. According to Baird et al (1999) AARs work best if: Focused on the few critical issues Done immediately after the action Inclusive of the whole group In accordance with a structured process Leading back to action quickly Real Time Evaluation (RTE) - Oxfam 2007: RTE is an internal rapid review carried out early on in the response in order to gauge effectiveness and to adjust or correct the manner in which the response is being carried out. An RTE is a rapid appraisal of program performance, a learning exercise and qualitative in nature using standard benchmarks that can be modified to suit the situation. November 29, 2018

Typical AAR in CARE: purpose and objectives CARE adopted and refined AAR between 2008 and 2010: An AAR is an internal emergency response learning exercise towards the end of an action or at a significant juncture of a response allowing for a professional discussion with a focus on performance. It captures and applies learning as quickly as possible back into action and allows participants to get on with the tasks. In general an AAR should at least: enable participants to discover what happened and why, and how to sustain strengths and improve on weaknesses, draw both positive and negative lessons (including from Pre-AAR review and monitoring exercises), prioritize lessons with regards to relevance for the ongoing response and importance for the wider organisation, and formulate recommendations to CARE management for emergency practice now (ongoing action) and in the future, for adapting policy (protocols and guidance), and for (organisational environment). Therefore in CARE an AAR is the central in translating lessons and evidence from a particular response into improved action, enhanced organisational efficiency and strengthened relevance. November 29, 2018

Typical AAR in CARE – workshop outline Typical AAR in CARE 2-4 months after start of response: Participants: Response team + representatives of CARE entities involved (+ partners as appropriate), Duration: 2-3 days in workshop setting DAY 1: Timeline: what happened? What went well? What went not so well? Drivers? Synthesis of findings, recommendations, management response from response reviews (RAR, RTE, external evaluation) as well as additional structured feedback (questionnaire); Performance assessment against HAF and Gender Marker Prioritization of lessons for immediate application in response as well as for wider practice Day 2: Analyse drivers of performance: enablers, obstacles, opportunities, risks Formulate response relevant tactical, organisational and strategic recommendations as well as related action points. Day 3: Action planning with Response team   November 29, 2018

Process flow: Humanitarian Performance & Quality Management in CARE Humanitarian Strategy Humanitarian Policies Response Design Response Action Response Results Management Response & Accountability Progress monitoring HAF & related frameworks CHS Gender Marker Sphere & companions Feedback & complaints Recommendations & Actions Response Quality Mgmt (Action) Typically, an After Action Review focuses on a core set of questions: What was supposed to happen? What actually happened? What was successful and why? What didn’t work and why? What did we learn?   Humanitarian Performance Mgmt (Organisation) Outcomes & Lessons

Performance Review Frameworks for CARE AARs - 1 November 29, 2018

NEW CORE HUMANITARIAN STANDARD - CHS GLOBAL IMPLICATIONS merger of HAP & PiA integration of CHS in SPHERE standards global roll out establish verification mechanisms IMPLICATIONS for CARE review HAF promote gender sensitivity strengthen people focus intensify collaborative management of accountability mechanisms November 29, 2018

Performance Review Frameworks for CARE AARs - 2 HAF score card (for regular FGD + RAR):   (BASIC) (INTERMEDIATE) (MATURE) (HAF COMPLIANT) How much information you have about CARE and the project (INFORMATION SHARING) I know nothing about CARE or the project I know little about I know a lot about CARE and I have good knowledge about the project CARE and about the project including activities, plans and budgets The ways in which you have been involved in the different steps of CARE’s project (PARTICIPATION) I’m informed but not involved, CARE tells me what the project will do and how this affects me I’m consulted – CARE discusses options with me but I’m not part of the final decision made by CARE I’m involved – we sit together and take the decision together I’m leading the decision which is then implemented by CARE The way(s) in which you can provide feedback to CARE about the project (COMPLAINTS AND FEEDBACK HANDLING) I do not know how to provide feedback about CARE and do not use any complaint mechanism I know and am able to provide feedback about CARE but am not sure how CARE uses that feedback – there has been no response to my feedback I know and have used CARE’s feedback or complaint mechanism and I know that feedback has influenced some decisions taken by CARE I use CARE’s feedback mechanism regularly to influence decision making and make changes to the project November 29, 2018

Use of the accountability score card in the Philippines, in Jordan and in Bosnia November 29, 2018

Performance Review Frameworks for CARE AARs - 3 The Gender Marker Vetting Form is a step-by-step flow chart that identifies where projects are graded (0-4) along the Gender Continuum. The current Gender Marker Guidance Form is for Country Offices. More Guidance for CMPs will be available soon. November 29, 2018

Use of evidence for efficient learning & accountability in humanitarian response review processes – lessons 1 Enablers: Leadership at all levels: polices & ways of working – transfer into local frameworks and practices Transparent framework (HAF, gender marker) aligned with international standards Inclusive processes (staff, partners, beneficiaries / participants) - accountability Pool of trained staff and suitable facilitators (internal and external) – roster / standing team Increasing emphasis and investment in analysis and follow-up across CARE Back-up with deployable capacities to support implementation of recommendations (sub-) regional exchange and joint analysis with involvement of partners and peers November 29, 2018

Use of evidence for efficient learning & accountability in humanitarian response review processes – lessons 2 Obstacles: Number of emergency responses vs. capacity to analyse and follow-up at different CARE levels Response team capacity to generate information and support review processes including documentation of practice and lessons Timing and sequencing of different review processes – maintaining logical linkages of knowledge management and learning Balancing flexibility and adaptation learning process with need to develop and agree on core elements of performance reviews that facilitate comparability, accountability and triple loop learning (esp. influencing polices and ways of working) Capacity to sustain MEAL process over longer period of time in complex protracted emergencies Complex decision making systems and authority in CARE confederation for quick implementation of recommendations and action points November 29, 2018

Areas for improvement in response review processes - the experience of CARE International Streamline AAR and other review processes while keeping balance between participation of field/response staff and externals Higher emphasis on follow up / management response / accountability – synthesizing and reconciling action plans Balance emphasis on assessment of quality (approaches) and of effectiveness (outputs & outcomes) Stronger involvement of peers esp. with regards to assessment of comparative advantage Explicitly include in evaluation TOR: Gender marker Identification of good practices for scale up / replication / dissemination HR Management aspects (Safety & Security, wellbeing) November 29, 2018

From humanitarian action to learning and improved practice: the full learning cycle(s) of humanitarian MEAL in CARE Information Management Response Rationale Response Policies Response Design Response Action Response Results Dissemination & Sharing Management Response & Accountability in/output tracking FRAMEWORKS CHS Gender Marker Sphere & companions PDM Feedback & complaints Recommendations & Actions M&E RAR Learning & Knowledge Management case studies RTE Analysis Synthesis Joint Eval. After Action Review Lessons example Philippines: from Typhon Bopha to Typhon Haiyan November 29, 2018

The final question at the end of every MEAL process November 29, 2018