Contribution analysis Anita Morrison and Jackie Horne, Office of the Chief Researcher, Scottish Government

Contribution analysis, results chains, logic modelling…. It’s all about outcomes Clarify the language – activities, inputs, outputs and outcomes! Examine how we can map interventions to outcomes (both singly and multiply) Examine how we can assess progress and performance towards outcomes Get working on some useful stuff!

Contribution analysis Holy grail of attribution… “Builds a reasonably credible case about the difference the program is making.” (John Mayne) Good contribution analysis is, in effect, theory-based evaluation Use of logic models and results chains strengthens programme planning, design and review Clearly sets out assumptions and external factors Also strengthens approach to research / analytical planning

The Six Steps….. 1: Set out the attribution problem to be addressed 2: Develop a theory of change and ‘model’ or ‘map’ it (including external factors and underlying assumptions) 3: Assemble the contribution story 4: Assess the performance story 5: Seek out additional evidence 6: Revise and strengthen the contribution story

Understanding what we mean by outcomes Not straightforward….. Distinguishing between inputs, activities, outputs and outcomes…………

Understanding the relationship between policies and outcomes – single intervention

A simple results chain – example - exercise classes for over 65s and HLE Over 65s, especially those not doing enough exercise Staff, equipment, leisure centres, etc. Invest in exercise facilities and classes for over 65s More accessible exercise classes and facilities for over 65s. Increased levels of physical activity in over 65s Lower blood pressure, improved mobility among over 65s Increase in healthy life expectancy

Results chain-Twilight basketball funded through Cashback National Outcome We live our lives safe from crime, disorder and danger (NO9) Intermediate Outcomes Less anti-social behaviour Less time spent bored and on streets Constructive use of leisure time Reach16-21 year olds in areas hardest hit by crime OutputCoaching, games, leagues, role models, education during time-out ActivitiesTwilight basketball (funded through Cashback) InputsStaff, funding, etc.

Results chain-Twilight basketball funded through Cashback National Outcome Our young people are successful learners, confident individuals, effective contributors and responsible citizens (NO 4) Intermediate Outcomes Aspirations and confidence increase Young people recognise that they have more opportunities Young people learn a new skill Reach16-21 year olds in areas hardest hit by crime OutputCoaching, games, leagues, role models, education during time-out ActivitiesTwilight basketball (funded through Cashback) InputsStaff, funding, etc.

National outcomeWe live our lives safe from crime, disorder and danger (NO 9) Our young people are successful learners, confident individuals, effective contributors and responsible citizens. (NO 4) Intermediate outcomes →Less antisocial behaviour. → Less time spent bored and on streets. → Constructive use of leisure time. → Aspirations and confidence increase. → Young people recognise that they have more opportunities. → Young people learn a new skill. Reach16 – 21 year old in areas hardest hit by crime. OutputCoaching, games, leagues, role models, education during time out. ActivitiesTwilight basket ball (funded through Cashback) Inputs

A simple results chain National outcome/purpose target Intermediate outcomes →→→→→→ →→→→→→ Reach Output Activities Inputs

Multiple policies - logic models and contribution analysis Move beyond results chains to consider inter- linkages between policies and outcomes Consider strength (and weaknesses) of the contribution of different policies to outcomes Assess and measure progress towards purpose targets and national outcomes Inform cost-effectiveness and future resource allocation Highlight strengths and weaknesses in evidence base

Alcohol – Partner Contributions to Outcomes Inputs Activities Outputs Reach Short-term outcomes Intermediate outcomes High level outcomes NHS Brief advice Preventive services Hazardous and harmful drinkers Service uptake & engagement Police Enforcement of drink driving laws Random breath testing Drivers Increased detection rate Local authorities Enforcement of planning controls & licensing laws Enforcement actions Licensed trade Compliance with laws Behaviour Reduced alcohol consumption levels Less drunkenness; less drink-driving Improved mental wellbeing Reduced inequalities in healthy life expectancy Reduced inequalities in alcohol-related deaths and hospital admissions Environments Physical: Reduced exposure to alcohol-related hazards Economic: Reduced availability/affordability of alcohol Social: Drunkenness less attractive; sensible drinking the norm SG, UK govts, EU Industry regulation Taxation,displays, promotions, advertising Alcohol industry Increased price Reduced incentives Scottish Govt Media campaigns Sensible drinking messages General public - targeted Understanding risks, attitudes to drinking Vol orgs Detox, Intensive support Addiction services Adults with alcohol problems Increased sobriety & stability

The actual and potential uses of this work? Is this the same for you? Generating a shared understanding of an outcome; and what government interventions might be best to deliver it Organisational capacity building in ‘outcomes working’ Systematically assessing our performance /progress Reporting to our Board on progress against outcomes A basis for business planning and business improvement Possibly contributing to outcomes budgeting processes

Producing a logic model…

Contact details Anita Morrison Jackie Horne