1. Health and Well-Being in Schools Steering Group 18 May 2009 Neil Craig Policy Evaluation & Appraisal Team NHS Health Scotland Logic Modelling for Outcomes- Based Approaches

2. Outline What is an “outcomes-focused approach”? What are logic models, how do you create them how can they help? Discussion

3. What is an outcome-based approach? “encourages us all to focus on the difference we make and not just the inputs or processes over which we have control.” “should clearly demonstrate how the activities of public bodies are aligned with the Govt’s overarching purpose through the National Performance Framework”... [and are] “supporting the achievement of outcomes” (Outcome based approach: Working guidance for Scottish public bodies, Sept 2008)

4. Why now? Part of wider performance management, outcome based planning system: “We are moving the whole of government to an outcomes- focused approach to performance” (Scottish Budget Spending Review, 2007)

5. National Performance Framework, HEAT & SOAs NHS Performance Management: 30 HEAT Targets SOA: Menu of 67 local outcome indicators PURPOSE PURPOSE TARGETS STRATEGIC OBJECTIVES NATIONAL OUTCOMES NATIONAL INDICATORS AND TARGETS Greener Wealthier & Fairer Healthier Smarter Safer & Stronger e.g. CHD mortality Life expectancy Child healthy weight Mental health- related outcomes Completion of child healthy weight programmes e.g. CHD mortality Life expectancy Child healthy weight Mental health-related outcomes We live longer, healthier lives We have tackled the significant inequalities in Scottish society We have strong, resilient and supportive communities… We give children the best start in life We live in well-designed, sustainable places We value and protect the natural environment and cut our environmental impact

6. SOAs: Guidance for 2009/10 SOAs are the “means by which CPPs agree their strategic priorities for their local area and express those as outcomes to be delivered by partners, … while showing how they contribute to the Scottish Gov’t’s relevant National Outcomes” The SOA “must be a strategic document … underpinned by robust perf. management arrangements… [and] a very clear line of sight from the SOA document.”

7. SOAs: Accountability Letter from SG to Chief Execs Signing up to an SOA means that partners “are corporately committed to the agreed outcomes … and … will take every opportunity to promote and support the achievement of outcomes.” “doing this is likely to include each partner looking at how they individually can contribute to outcomes” and “each partner will need to show that a ‘golden thread’ runs from the SOA through their planning, resourcing and performance management processes.”

8. Tools for the job? Logic models A convincing picture of what you are trying to achieve that shows the links between your intended inputs, activities, outputs and outcomes. Not an attempt to describe reality but your theory about the key things that need to happen to get to your intended outcomes A process and a way of thinking as well as an output

9. Basic model Resources / Inputs ActivitiesOutputs Short-term Outcomes Long term Impact Medium term Outcomes ASSUMPTIONS That affect delivery EXTERNAL FACTORS That affect results/impacts

10. Define your terms Inputs –What you invest –E.g. Resources, staff, funding, equipment, facilities supplies Activities –Key things you will have to do –E.g. marketing, recruitment, training, running programmes Outputs –Countable products –E.g. Number of people trained, information packs, courses run, number recruited Outcomes (short, interim and long-term) –The changes achieved in your target groups –E.g. Increased knowledge, increased confidence, improved risk factors, better health

11. An example: child healthy weight LONG-TERM HIGH LEVEL OUTCOMES INTERMEDIATE OUTCOMES SHORT-TERM OUTCOMES Outputs? Inputs? Activities/ Processes? Reduce the rate of increase in the proportion of children with BMI outwith healthy range by 2018 National Indicators Menu of Local OIs (Proportion and no. of obese children in P1) Increased healthy life expectancy Reductions in morbidity and mortality from obesity-related causes Achieve agreed completion rates for child healthy weight intervention programme by 2010/11 HEAT: H3 ?

13. Healthy weight – Cross-sector Contributions? Inputs Activities Outputs Reach Short-term outcomes Intermediate outcomes High level outcomes NHS What NHS does to reach target population Child healthy weight intervention Overweight children & families Completed healthy weight programmes Planners ? ? ? ? Schools ? ? ? ? Reduced increase in children with BMI outwith healthy range Improved mental wellbeing Reduced inequalities in healthy life expectancy Reduced prevalence of and inequalities in obesity-related ill-health Environments Physical: More opportunities to lead active lives? Economic: Less availability/affordability of energy-rich food? Social: Changing attitudes to food and exercise? SG, UK govts, EU ? ? ? ? Scottish Govt ? ? ? ? Vol orgs ? ? ? ? ?

14. INTERMEDIATE OUTCOMES Strategic Healthy Weight Outcomes Logic Model (Simplified) LONG-TERM OUTCOMES NESTED LOGIC MODELS Reduced Type 2 Diabetes Reduced CHD and stroke Reduced cancer NATIONAL OUTCOMES Longer, healthier lives Tackled significant inequalities Other non- health outcomes? Reduced prevalence of overweight and obesity Improved energy balance Model a Model b Model c Model d Model e Increased food industry corporate and social responsibility Changes in availability and affordability of energy-dense/low energy dense food Improved early years nutrition Increased individual empowerment and food literacy Improved media and social environment Increased physical activity

15. Nested model for child healthy weight programmes (simplified) Activities Short-term Outcomes (by end of programme) Intermediate Outcome Reach Reduced increase in children with BMI outwith healthy range ? ? Other NHS? ? ? ? Other non- NHS? ? Outputs % Target group completed programme Improved psychosocial outcomes e.g. self-esteem, body image Increased physical activity Improved nutrition Improved knowledge and skills Healthy weight programme: identification, contact, assessment, interventions 5–15 year olds, >91 centile No. sessions Type of sessions and range of content

16. How do you develop a model? Look for logic in plans and draft a model Agree intended use(s) of model(s) Engage stakeholders and adapt and refine Share model more widely Use, revisit and refine it regularly Workshops, discussions, interviews, focus groups

17. Process Engage people Set boundaries Draw on evidence, logic and ethics Interrogate and revisit the model

18. Is it plausible? Is it doable? Is it testable? Have external constraints/enablers and unintended consequences (+/-) been considered?

19. Learning and reflections No standardised approach – to the process or the product The value is in the process not just the product Models should be fit for purpose, not ‘perfect’ Logic models should be embedded in part of wider performance management system and learning culture Logic modelling is limited by, but may also contribute to, the existing evidence base

20. Conclusions: How can LMs help? Help you know what and when resources are needed Provides a focus for evaluation Helps identify what to measure + when In complex interventions, provides some credibility when attribution impossible Links with bigger picture - e.g. how programme activities support achievement of national objectives Logic models provide a framework for integrating planning, delivery, evaluation and performance management

21. Questions and Discussion