1. Social capital & Spirit’s ToC Applying Understanding Society data to test a third sector theory of change on social capital and sports participation Pat McGinn, PMG Consulting, pmcginn@john-lewis.com pmcginn@john-lewis.com

2. Background - qualitative Depth case studies Context + mechanism = outcome configuration Modelling prospects QCA: Qualitative Comparative Analysis Status: year 1 complete Evidence issue: replication

3. Background - quantitative 'Spirit 60' Longitudinal tracking Status: in design Evidence issue: depth v breadth Efficiency issue: evaluation costs Does secondary data analysis of UKHLS offer a partial solution? Hypothesis testing v data mining

4. Theory of Change (ToC) From logframes to ToCs Depth: testing investments' logics (QCA) Breadth: testing macro hypotheses UKHLS solution: efficient & replicatable evidence?

5. Understanding Society (UKHLS) Panel design, quality surveying & sampling Waves, ~ years! Modules, revolving, only 1 Olympics module Files, indresp is used here, ‘wide-file’ format Variables, from different waves Weights, 2012 self-completion, cross- sectional used here

6. A small but important part of the ToC Inspiring events - ‘people inspired by events are more likely to get involved in social action’ Controlling for background factors (context), engaging with the Olympic Games (mechanism) is associated with more volunteering (outcome) For Spirit this is an assumption it is trying to test

7. What questions might UKHLS help us to answer? Is there evidence for the assumption? Is the evidence spread widely or narrowly? How much of the wider volunteering people do, may the Games ‘account for’? How did engagement with the Games compare to sports & arts participation (in accounting for volunteering)? What difference does social capital make to this?

8. Technical considerations Sample size Binary logistic regression models Odds ratios & ‘base cases’ ‘Pseudo R-squared’ Cross-sectional weights Data from multiple waves

9. What does Understanding Society have on the Games? Data on before, during & after the Games (for different people) Passive & active engagement Multiple choice, showcard, code all Passive engagement questions cover –Watching on TV at home –Listening to the radio at home –Watching or listening on the internet at home –Reading the newspaper online or offline –Watching live events on a public big screen

10. Active engagement questions Attending a free Olympic or Paralympic event Attending a ticketed Olympic or Paralympic event Taking part in a Games-related sports or physical activity Using a new or improved sports facility linked to Games Games-related employment or training Taking part in a Games-related cultural event or activity Volunteering during the Games Taking part in a Games-related community event / activity

11. Other data used Sports questions cover extensive list of indoor & outdoor, team & individual sports activities during reference period Arts questions cover participation in & event attendance at variety of settings Social capital as personal trust people, trust neighbourhood, number of friends GHQ, Caseness, “a scale running from 0 (the least distressed) to 12 (the most distressed)” (UKHLS) Controls were age, gender, marital & socio- economic status

12. Models ModelNPseudo R- squared Wave base case20,2270.02174 + Games11,7240.04274 + sports10,6560.07402 + arts10,6550.11492 + social capital 7,3780.12481, 3 + GHQ7,3700.12514 (all models are statistically significant, not all variables, not all odds ratios are)

13. Headline findings The base case is a poor model (by design) The + Games model is much better The + sports model is a further improvement The + arts model is a further improvement The + social capital model additional contribution is modest The + GHQ model contribution is weak (caution!)

14. What do the (short) models ‘look like’? + GHQ model for volunteeringOdds Ratio (relative to base v)P>z age1.010.00 female1.120.20 Married0.890.26 Semi-routine, routine, etc0.700.00 Attending a free Olympic or Paralympic event (eg marathon, cycling, etc) mentioned1.630.00 [sports & arts items not included for reason of space] can't be too careful0.760.00 depends1.160.10 Agree trust neighbours0.820.07 N of friends1.020.00 GHQ1.000.77

15. Limitations Wave 4! Crudity of the models Partiality of the models (only volunteering) Validity of the measures Social capital’s operationalisation Use of Pseudo R-squared Sample size near-guarantees ‘significance’ Cross-sectional, not longitudinal Too ‘cheap & cheerful’?

16. Conclusions Controlling for background factors (context), engaging with the Olympic Games (mechanism) is associated with more volunteering (outcome). There is evidence for the assumption but scale issue Evidence is spread widely across background factors, modest importance The quantum of the wider volunteering people do, that is associated with the Games, is small but far from negligible The relative contribution of the Games to sports & arts participation (in accounting for volunteering) is similar Social capital makes surprisingly little difference!

17. Thank you All Stata commands & outputs available by email