1. KAROLINSKA INSTITUTET Institutionen för folkhälsovetenskap Avd för socialmedicin Introduction Fall 2007 Bo J A Haglund professor (http://www.phs.ki.se/socmed/postdoc/OH-presentations.html)

2. http://www.phs.ki.se/socmed/postdoc/OH- presentations.html

3. Slides on evaluation

4. http://www.phs.ki.se/socmed/postdoc/links.html

5. Theoretical distribution over time of outcomes from health promotion interventions Proportion of objective attained Units of Time 1 2 345 Raised awareness and participation Short-term program impact Social and behavioural risk changes Health status and disease changes

6. Approaches in managment HEALTH SERVICES PARTICIPATION THE ROLE OF THE PROFESSIONAL ROLE AND TRAINING OF THE CHW EVALUATIONFINANCEISSUES APROACHES MEDICAL EXTERNAL APPROACH HEALTH PLANNING MULTIPLE APPROACH COMMUNITY DEVELOPMENT INTERNAL APPROACH Major programme component For provid- ing better services Key Service Extender Health Status Statistics From whatever source available Necessary but not sufficient for health improvement For maximising resources Component Service Extender/ Change Agent Efficiency/ Effective- ness From outside working toward self- reliance Means for community mobiliza- tion For creating improved social structures Resource Change Agent Educative Process Self reliance as primary goal

7. Comparison of approaches for health promotion Noreen 1998

8. Five models of Health Promotion in terms of implications for evaluating effectiveness The medical model The educational model The behavioral or lifestyle model The social model The empowerment model Naidoo J, Wills J. Practicing Health Promotion. Dilemmas and Challenges. London: Baillière Tindall, 1998

9. Significant actors for evaluation in health promotion, and their needs for information Program funders: Program staff: Program beneficiaries: Other local stakeholders: The evaluation team: Accountability Professional and deontological Problem-solution dyad Contigent and variable Contingent and variable

10. Manpower Competence Methods materials Invest in Money and goodwill Theory and evidence Organisation network Leadership Healthy lifestyle Societal Context History The news Trends Demography Eonomy Cluture Politics etc Gain and respect Knowleldge and insight Policy strategy Social cohesion (Soc. Capital) Services Healthy environment and climate Health Literacy Human Biology Genetics Vitality Policy: Law, Rules, finance Equity = Equal access And opportunity Community Setting development Organisation Development Advocacy Lobby Policy-devel. Intersectoral action Information education Quality of life somatic psychic social Interventions Intervention-results DeterminantsHealthOrganise Health Promotion Framework After Hans Saan and Willy de Haes, NIGZ Social support

11. Health Promotion Improved Health status Health education -for behavioural change -for structural change _______________ Organisational actions _______________ Economic actions _______________ Political actions (including advocacy) Attitudinal, behavioural, Social and environmental changes conducive to health Howat et al. HPJA 2003;14(6):2-5

12. Community Coalitions, Advisory Groups, Networks Economic Sector External Context Social Welfare Sector Voluntary / Grassroots Agencies Religious Sector Communication Sector Recreational Sector Political Sector Health Sector Other Community Groups Indiviuals in the Community Education Sector Schematic of Community as System After Bracht 1999

13. The use of theory in programme planning and evaluation After Nutbeam D, Harris E. Theory in a Nutshell. A practttioner´s guide to commonly used theories and models in health promotion. University of Sydney 1998

14. Definition of Evaluation Evaluation is the systematic examination and assessment of features of a programme or other intervention in order to produce knowledge that different stakeholders can use for a variety of purposes. Rootman et al 2001

15. Analyse Plan ImplementEvaluate The Basic Planning Model

16. PlanningImplementationSustainability Activities or Services Funding Sources Needs analysis Partenarship development Elaboration of activities Search for funding Feasibility studies Define objectives Define theory of treatment Estimate resources Client services Personnal monitoring and supervision Activities for targeted clienteles Resources monitoring Combining activities and services Securing resources Enshrine activities in rules and procedures Adaption of activities to environment Evaluation Activities A Stage Model of Program Development Developed with Pluye, 2002

17. 1 2 3 4 5 6 7 Health Promotion Planning and Evaluation Cycle Theory helps identify what are targets for intervention Theory helps to clarify how and when change can be achieved in targets for intervention Theory indicates how to achieve organisation change, and raise community awareness Theory provides a benchmark against which actual ca be compared with ideal program. Stage five, six and seven: Theory defines outcomes and measurements for use in evaluation

18. Mortality, morbidity disability Qualtiy of life, functional independece, equity Social influence and action Healthy public policy and organisational practice Health literacy FacilitationAdvocacyEducation Effective health services Healthy environments Healthy lifestyles Intermediate Health Outcomes Health promotion Outcomes Health promotion Actions Health and Social Outcomes An outcome model for health promotion Nutbeam. Health Prom Int 1998;13:27-44

19. Fig 1. Outcome model for health promotion Nutbeam Social and Health Outcomes Social Outcomes Measures include: quality of life, functional independence, social capital, equity Health Outcomes Measures include: reduced morbidity, disability, avoidable mortality Intermediate Health Outcomes (modifiable determinants of health) Healthy Lifestyles Measures include: tobacco use, physical activity, food choices alcohol and illicit drug use Effective health service Measures include: provision of preventive services, access to and appropriateness of health services Healthy Environments Measures include: safe physical environment, supportive economic and social conditions, good food supply, restricted access to tobacco, alcohol Health Promotion Outcomes (intervention impact measures) Health Literacy Measures include: health-related knowledge, attitude, motivation, behavioural intentions, personal skills, self-efficacy Social action & influence Measures include: community participation, community empowerment, social norms, public opinion Healthy public policy & organisational practice Measures include: policy statements, legislation, regulation, resource allocation organisational practices Health Promotion Actions Education Examples include: patient education, school education, broadcast media communication Social mobilisation Examples include: community development, group facilitation, technical advice Advocacy Examples include: lobbying, political organisation and activism, overcoming bureaucratic inertia

20. Stages of Research and Evaluation for Health Promotion Programs Problem Solution definition Generation What is the How might it problem? be solved? Key Research Questions Epidemiology and demography Social, behavioural and organisational research Community needs analysis Intervention theory development Intervention literature search, meta- analysis Pre- Testing methods and materials

21. Stages of Research and Evaluation for Health Promotion Programs Problem Solution Innovation Intervention Intervention definition Generation Testing Demonstration Dissemination What is the How might it Did the Can the Can the problem? be solved? solution program beprogram be work? repeated/widely refined?reproduced? Key Research Questions Epidemiology and demography Social, behavioural and organisational research Community needs analysis Intervention theory development Intervention literature search, meta- analysis Assessment of outcome Understanding of Process Pre- Testing methods and materials

22. Stages of Research and Evaluation for Health Promotion Programs Problem Solution Innovation Intervention InterventionProgram definition Generation Testing Demonstration DisseminationMonitoring What is the How might it Did the Can the Can the Can the problem? be solved? solution program beprogram be program be work? repeated/widely sustained? refined?reproduced? Key Research Questions Epidemiology and demography Social, behavioural and organisational research Community needs analysis Intervention theory development Intervention literature search, meta- analysis Assessment of outcome Understanding of Process Assessment of cost and benefits (financial, social., political) Performance monitoring Pre- Testing methods and material s

23. Implementation Sustainability Planning Evaluation Processes in a Program Space Developed with Pluye, 2002

24. Program Effects Stakeholder Issues/Questions Purpose of evaluation Communi- cation Type of Evaluation e.g. formative summative Impact/Effects To be measured Measurment Methods Timeframe For Data Collection Nature/ Degrees of Stakholder Participation Education Enhanced Organizational Capacity Short-Term Implications of finding for Program Long-term Implication of finding for Program Cost/Benefits of Implementing Recommen- dations Cost/Benefits Of not Implementing Recommen- dations Enhanced Coordination of Efforts & Resources Improved Skills, Decison Making & Behaviours Improved Awareness & Knowledge Enhanced Organi- zational Capacity Increased Community Capacity & Participants Enhanced Health Promoting Public Policies Increased Health Promoting Public Policies Enhanced Coordination Of Efforts & Resources Improved Skills, Decision Making & Behaviours Organi- zational Development Community Development Policy Development Advocacy Intersectoral Collaboration Information & Materials Improved Organizational Structure & Climate Coordination of Efforts, Expanded Resources & Coalition Building Laws, Regulations & Policy Statements Public Dialogue re. Decision Marketing Coordination of efforts, Expanded Resources & Coalition Buliding Programs & Materials Enhanced Health Promoting Public Policies Improved Awareness & Knowledge Increased Community Capacity & Participation Increasaed Health Promoting Public Policies 1. Summarize and review current understanding of program/initiatives 2. Identify key stakeholder groups, their interests, expectations and desired modes of participation 3. Identify the purpose(s) of the Evaluation, evaluation questions (relative to program objectives) and Address issues of concern to stakeholders Evaluation Questions 4. Describe proposed program or initiative (objectives, activities and outputs) Activities Objectives 8. Develop and review recomenendations Based on the interpretation of data 5. Design the process of obtaining the Required information for each evaluation question 6. Collect the data using agreed-upon Methods and procedures Outputs 9. Disseminate evaluation findings in Formats appropriate for all stakeholder groups 7. Analyze and interpret the data with stakeholders Evaluation framework for Understanding the Effectiveness of Health Promotion - after Michael Goodstadt, 2001 bjah 2004

26. Review CollectJudge Evidence-process Research integrated Do EvaluatePlan Work-process PublishImplement