1. Planning Protocols and Procedures (Monitoring and Evaluation Updating Plan) First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman

2. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Learning objectives By the end of this session, you should be able to:  Discuss basic principles of evaluation  Identify key lessons learned for health emergency managers from recent disasters  Identify areas for action in HEM capacity development

3. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman How can we Define Evaluation? (1)  The classic perspective: o Concerned with the achievement of objectives  The broad perspective: o Achievement of objectives is a key, but it is only part of what an evaluation might be concerned with

4. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman How can we Define Evaluation? (2) Key elements are:  The need for systematic collection of information  The wide range of topics to which evaluation can be applied  To be effective, the evaluation results has to be used by someone  The wide variety of purposes of evaluations

5. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman What’s the Difference between Research & Evaluation?  Research & evaluations use the same toolbox  However, for a different purpose Research aims to prove… Evaluation aims to improve…

6. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Why do we do evaluations? It’s all about interventions or programs to:  Inform planning  Define progress  Examine efficiency  Examine effectiveness or achievement  Inform decision-making

7. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman How Could you Find out the Purpose of an Evaluation?  Who asked for evaluation?  Who pays for the evaluation?  Why do these people want an evaluation?  What are the decisions that need to be made?  What information is required to facilitate decision making?  Who is going to be affected by evaluation outcomes?

8. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman  Needs assessments  Before planning to understand context & needs  Program monitoring  Compliance with policy/plan  Validity of assumptions & pre-conditions  Formative evaluation  Efficiency  Summative evaluation  Effectiveness  Decision-making Classifying Evaluation Purposes

9. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Evaluation Logic PurposeWhy do we do this? QuestionsWhat do we want to know? ObjectivesWhat information is needed? MethodsThe way we collect/analyze information ResultsProcessed data or information ConclusionsAnswers to questions RecommendationsSuggestions for decision-making

10. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman What Methods can you Think of to Collect Information?  Documentation research  Observations  Surveys interviews  Individual interviews  In-depth interviews  Focus group discussions

11. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Quantitative Data Sources & Methods Survey interviews Indirect measures Characteristics of samples of a community or subgroup Observation Conditions within samples of a subgroup Documented information on population groups

12. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Qualitative Data Sources & Methods Individual interviews Focus groups In-depth interviews Key informants In-depth knowledge Individual perspective intra-cultural variation Community perspective normative view Observation Behaviour Indirect measures Documented information

13. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman About Methods  Questions will tell you what information you need  Information needed will tell you what methods could be used  Local context & resource limits will tell you what methods are feasible  Keep it as simple as possible

14. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman In Summary To find out something useful about an intervention or program, use whatever you have in your toolbox that will get the job done…

15. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Small Group Discussions  Based on personal experience, what are the lessons learned in terms of HEM?  Compare experiences & build consensus on common lessons learned  List on a flipchart your group results

16. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman SYNOPSIS OF EVALUATIONS ON THE HEALTH EMERGENCY RESPONSE TO SEA REGION TSUNAMI KEY LESSONS LEARNED FOR HEM

17. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Sources  The Phuket Conference Papers (2005)  Tsunami Evaluation Coalition (2005)  The WHO-SEARO Tsunami evaluation (2006)

18. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Lessons Learned in Context of HEM Early warning systems Health info system Capacity assessment Risk reduction Policy Needs assessment Emergency planning Incident command system Resource management Risk communication Donation management Operations management Recovery & reconstruction Capacity development

19. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Early Warning Systems  Coordination structures need to be developed  Flexible surveillance systems need to be promoted  Surveillance systems need further development in terms of local capacity building:  Personnel  Infrastructure

20. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Health Information System Management  Surveillance systems focused too exclusively on CD  Need to develop surveillance systems that are capable of covering all aspects of health  Surveillance and monitoring must be strengthened

21. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Capacity Assessment  Private & other sectors should be involved in disaster reduction  Preparedness, mitigation measures & capacity building are more cost-effective than enhancing damage response  Vulnerability indicators need to be defined  I.e. safe hospitals

22. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Risk Reduction  The importance of risk reduction need to be advocated  Health sector & facilities’ ability to recover from disaster stresses need to be improved  There is a need to develop vulnerability indicators  Integrate mitigation activities into the recovery process  PHC staff should receive training in disaster risk reduction

23. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Policy  Countries that did relatively well in HEM in terms of national & international coordination had:  Well-developed policies & systems  Strong health systems  Clear lines of authority  Problems encountered post tsunami are not new:  Policy without enforcement is powerless  Coordination policy-practice too often doesn’t happen  Relevant indicators on effectiveness, efficiency, costs, & benefits of both preparedness & response are needed

24. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Needs Assessment-1  A pragmatic approach to needs assessment  Assessment must be relevant to decision making  Assessments must be timely  Invest in national assessment capacity  Strengthen the capacity of national and local authorities to carry out cross-sector needs assessments

25. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Needs Assessment-2  The need to streamline the international assessment mechanism  Dysfunctional  Competitive  Not shared  The need for a common information system of beneficiaries  Mainly statistics  It does not tell who needs what  It does not facilitate matching needs & offers

26. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Emergency Planning & Response-1  Disaster preparedness is crucial  The quality of early responses is affected by the degree of preparedness  Lack of effective coordination: most common complaint  Internal  External  Between policy & practice  Across sectors & agencies  Information management  Reporting formats should facilitate decision-making  Media relationships including pre-disaster

27. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Emergency Planning & Response-2  Flash appeals: a solution causing problems…  Disaster preparedness must include training on Flash Appeal  Timing of funding was a problem  Imbalanced demands: curative care & public health  Funding worsening inequities  Consider psychosocial support mechanisms  Health system deficiencies  Alien expertise  Community-based mechanisms

28. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Incident Command System Operations  A 24-hour contingency response system should be in place  WHO internal relationships in response to the Tsunami raised questions:  Coordination  Sharing of information  Chains of decision making  First teams in the field should always be composed of experienced staff and not consultants

29. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Resource Management-1  Recruitment & briefing of consultants calls for care  There is scope to strengthen regional & national financial resource mobilization capacity including:  Disaster preparedness  Mitigation  Response

30. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Resource Management-2  WHO should assure its leadership in public health in its relations with UN agencies and NGOs  MOH should assure its leadership in public health in its relations with other ministries and international donors  Collaboration with the military is important and should be fostered

31. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Risk Communication  Working with the media is important so that reporting is:  Technically sound  Epidemiologically sound  We need to enhance the relationship with the media through:  Training of HEM managers  Regular briefing of media  Sharing some of the lessons learned

32. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Donation Management  Make international response more relevant to needs assessment  Humanitarian response is an unregulated industry  Setting priorities and distribution of tasks among actors requires significant improvement  Assessing needs is pointless if decision making is not reformed

33. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Operations Management  Standard Operating Procedures (SOPs) are essential and need to be developed & reviewed  Situation reports need to be standardized and regular  Logistics, supplies & donations can be troublesome

34. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Recovery & Reconstruction-1  Planning & preparedness can reduce the impact of disasters  There is a need for a strong public health infrastructure  Planning and coordination of recovery & reconstruction is essential

35. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Recovery & Reconstruction-2  Early responses do affect long term system functioning with consequences for health services  Researchers & implementers need to speak the same language  Relationships between national & local governments and international agencies are crucial for the course of development

36. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Capacity Development-1  Provide leadership  Strengthening organizational & management capacities  Human resource development:  International & national level staff are the sole barrier to improved health sector capacity  Not by quick fix short courses  Incorporate “Health Management in Disasters” in curricula of all relevant disciplines  Not HEM but HM knowing emergencies

37. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Capacity Development-2  Develop partnerships & networks  Professional associations  Multi-sectoral  National  Regional  Inter-regional  Develop & share knowledge  Case studies  Research  Continuous learning  HEM integrated in health curricula

38. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Putting it together-1  Build capacity in risk management & vulnerability reduction  Policy & legislation development  Develop disaster management lines of authority & control  Allocate resources for risk mgt & vulnerability reduction  Address the need for improved needs assessments  Relevant to decision-making  Multi-sectoral  Population-based  Inclusive

39. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Putting it together-2  Enforce public health practice:  Psycho-social support  Gender equity  Child health  Management of the dead & missing  Involvement of health volunteers  IEC for watsan  Develop benchmarks & standards of practice  Vulnerability indicators  Surveillance systems  Needs assessments  Indicators on effectiveness, efficiency, costs, & benefits of both preparedness & response

40. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Putting it together-3  Improve management & coordination of responses  Internal  External assistance  Policy-practice  Develop logistics of supplies  Legislation  Relevance  Capacity building

41. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Putting it together-4  Improve donation management  Timely  Sustainable  Relevant  Flexible  Foster contribution of other sectors  Civil-military liaisons  Public-private sector liaisons  NGOs’ inclusion not marginalization

42. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Putting it together-5  Develop media relations  IEC prior and during disasters  Combat disaster myths  Guidance on media relations  Developing capacity  HEM leadership  HEM capacity  Human resources  Networking & partnerships  Knowledge development

43. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Capacity Development in HEM Evaluation & Research Awareness Advocacy Capacity Building Strengthenin g Development

44. First National Course on Public Health Emergency Management 12 – 23 March 2011. Muscat, Oman Plenary Discussion At the end of the day…considering:  The roles of HEM in disasters and  The lessons learned from disasters How would describe the strengths & weaknesses in terms of HEM for your country?