A proposal of framework to evaluate community-based obesity prevention interventions in LAC Marcia Erazo, Margarita Safdie, Mariana Lazo, Angel Caballero, Richard Visser, Armando Barriguete PACO III Workshop: EPODE Population based interventions

Background In Latin America and the Caribbean (LAC) the obesity prevalence has raised at alarmingly pace. Community-based interventions are the most promising interventions because they can reach the entire population. These interventions are recommended to address the obesity causes rooted in the society and special needs of the individuals.

Background Ideal community interventions combines specific health promotion and evaluation theory. In LAC, there is no community interventions probably due to lack of evaluation frameworks. Limited data are available to guide the implementation and evaluation of these interventions in LAC.

Rationale Standardized and comprehensive evaluation frameworks are key for policy-makers, scientists and stakeholders to evaluate community-based programs

Objective To thoroughly examine already available evaluation frameworks (WHO, EPODE, and Victoria proposals) and related literature, and to provide a specific framework to evaluate community-based interventions in LAC.

Policy Makers PractitionersPublic Other Stakeholders External Groups Key stakeholders

WHO Steps Evaluation Health promotion programs 1.Describing the proposed program, policy or initiative 1.Program logic 2.Identify group to undertake the evaluation 3.Baseline data 2.Identify key issues and concerns 1.Key outcomes indicators 2.Stakeholders interests 3.Design: study, data collection 4.Data collection as planned 5.Data analysis 6.Recommendations 7.Dissemination of results 8.Take action Page 55 (28)-56(29)

Wimbush and Watson. Evaluation framework for health promotion Common themes in evaluation (Policy and strategic planning) – Outcomes and effectiveness Other important components (other stakeholders): – Improvement in quality of program – Improvement in program design Health Education Board Scotland Tip of Iceberg

Wimbush and Watson. Evaluation framework for health promotion Perspectives: – Policy Makers and Strategic Planners: (make decisions and are held accountable) What works? What are the best buys? – Program Managers: (role: budget holders, responsible for the delivery) Success, based on achievement of goals, extent of reacheness and sustainability – Practicioners: (role: operation and running of programs) Feedback from people involved in the collaborative action – Community/Users: Avenue to provide feedback, involvement: express concerns, satisfaction Extent to which program meets their perceived needs

Wimbush and Watson. Evaluation framework for health promotion Stages  Evaluation – Planning – Design and Pilot – Implementation –early start up – Implementation –establishment – Implementation –fully operational – Dissemination

Evaluation Framework for Health Promotion and Disease Prevention Programs. Victoria, Australia Describe the program – Program logic Evaluation preview: – Process – Outcomes – Implications for Future programs and policy Focus evaluation design: – Agreed or common indicators Collect Data Analyse and Interpret Data Dissemination of the results

EPODE By level: – Central – Local – Settings – Child To date, all EPODE programs have initiated continuous monitoring and evaluation practices at central, local and child levels, as well as in certain settings such as schools. Developing consistent practices and collecting comparable data across communities and programs remains an important challenge. Obesity Rev 2011

Proposal of a framework to evaluate community-based interventions to prevent child obesity LAC Public Policy Policy design Policy process Conceptual framework Logic model Objectives to achieve Conceptual framework Logic model Objectives to achieve Political concern Focused on the electorate Objectives not declared Political concern Focused on the electorate Objectives not declared Effectiveness Rating HIGH QUALITY Citizen’s feelings and technically correct HIGH QUALITY Citizen’s feelings and technically correct

Good governance Open: Active communication, accessible language, trust between citizens and the institutions. Participation: Wide community participation during development, Implementation and evaluation stages to achieve a public policy with high quality, pertinence, efficacy. Efficacy: Actions must achieve results as planned, evaluating their future impact on the community and considering cumulative experience. Coherence: Policies and actions must be coherent and easy to understand.

We need to evaluate both, Policy design and its effectiveness and Policy Process Policy design and its effectiveness Policy process Through rigorous scientific methods Through rigorous scientific methods Through key questions That reflect the political Environment and support And Population satisfaction Through key questions That reflect the political Environment and support And Population satisfaction

Why to evaluate political process? Politically strategic and democratic: Responsibility and accountability (social, economic and technical). Managerial: It helps to improve public management and the quality of the program Transparence: Strong mechanism to create “public value”. Society needs to know the way the money is spent, the results and the impact of the policies. Participation: It strengthens the communities and individuals. It also improves control and efficiency of public administration.

Public Policy Plans Programs Projects/Interventions Public services Complexity Political: Decisions Agreements Actions Political: Decisions Agreements Actions Problem definition Stakeholders Problem definition Stakeholders Objetives Logic model Objetives Logic model Nationwide Monitoring and Surveillance Investigation Nationwide Monitoring and Surveillance Investigation

Policy design WHOIrelandAustraliaCDCEPODE Describe proposed program Common themes in evaluation Describe program Engage stakeholdersMultilevel Identify key issues and concerns Engage other stakeholders Evaluation preview Describe program Multi stakeholders Design study and data collection Consider different perspectives Evaluation design Continuos monitoring and evaluation Collect data Consider different stages of evaluation Collect data Gather credible evidence Collecting comparable data accroos the levels, settings Data analysisAnalyse dataJustify conclusionsUse logic model RecomendationsDissemination Ensure use and share lessons learned Disemination Take action

Key common issues

Proposal of a Comprehensive framework LAC Planning Implementing Results Impact Need / Problem Intervention Objectives Actions Intermediate results Final results Health impact Diagnose and design Evaluation Diagnose and design Evaluation Implementation Evaluation Implementation Evaluation Results evaluation Impact evaluation Ex Ante Intermedia Ex post Beneficiaries/Society Normative existance Normative existance Stakeholders Participation Political commitment Stakeholders Participation Political commitment Population feeling Community believes Population feeling Community believes

Ex Ante Intermedia Ex post To analyse and determine if the intervention is adequate to solve the problem It pretends to guarantee that the intervention will achieve the goals It must evaluate: Needs Program theory Coverage Potencial effects, results and impact To analyse and determine if the intervention is adequate to solve the problem It pretends to guarantee that the intervention will achieve the goals It must evaluate: Needs Program theory Coverage Potencial effects, results and impact It is a critical analysis about the implementation and early results. It analyses the intervention validity, the pertinence of the objectives, the quality of the managerial and monitoring system. It encourages to improve the implementation in order to achieve the goals It is a critical analysis about the implementation and early results. It analyses the intervention validity, the pertinence of the objectives, the quality of the managerial and monitoring system. It encourages to improve the implementation in order to achieve the goals It evaluates: The successful of the strategy. Its flexibility and its capability to adapt the actions according to reality. Its efficacy and efficiency Managerial mechanisms It evaluates: The successful of the strategy. Its flexibility and its capability to adapt the actions according to reality. Its efficacy and efficiency Managerial mechanisms

1. “The circumstances external to the implementing agency do not impose crippling constraints” 2.“That adequate time and sufficient resources are made available to the programme” 3. That the required combination of resources is actually available” 4.”That the policy to be implemented is based upon a valid theory of cause and effect” 5.“That the relationship between cause and effect is direct and that there are few if any, intervening links” 6.“That dependency relationships are minimal” 7.“That there is understanding of, and agreement on, objectives” 8.“That tasks are fully specified in correct sequence” 9.“That there is perfect communication and coordination” 10.“That those in authority can demand and obtain perfect compliance” Key questions