1. Online survey analysis tools Paul Lambert, University of Stirling Presentation to the Scottish Civil Society Data Partnership Project (S-CSDP), Webinar 2 on ‘Popular tools of data analysis: MS Excel and online data analysis resources’ www.thinkdata.org.ukwww.thinkdata.org.uk, 4 Mar 2016

2. What are ‘online survey analysis tools’? Typically commercial: 1)Online systems for survey data collection and its analysis Typically non-commercial: 2)Academic/Public sector online systems for analysis of nominated secondary data 3)Online software supporting analysis of local data 4)Online organisations offering data analysis support 5)Online access to downloadable freeware for processing local data S-CSDP, 4 Mar 20162 This talk provides a non-comprehensive review of online survey analysis tools, highlighting selectively some of those that are most likely to be useful for research on the third sector in Scotland

3. 1) {Typically commercial} online systems for survey data collection and its analysis Examples often used for low cost small scale surveys inlcude: ‘Surveymonkey’ ‘Kwiksurveys’ ‘Bristol online surveys’ S-CSDP, 4 Mar 20163 Facilities for questionnaire design; data collection, storage and export; and simple data analysis for the online survey data

4. Some commercial online survey tools include facilities for uploading and analysing other survey data S-CSDP, 4 Mar 20164

5. 2) Academic sector online systems for analysis of nominated secondary data S-CSDP, 4 Mar 20165 Image from NESSTAR, a tool for online analysis of large-scale archived surveys Image from OECD ‘gender data portal’ (http://www.oecd.org/gender/data/), access to bespoke secondary macrodatahttp://www.oecd.org/gender/data/

6. E.g. IPUMS-I online analysis of census datasets S-CSDP, 4 Mar 20166

7. 3) Online software supporting analysis of local data (non commercial) Typically projects with origins in computer science, developing distributed data processing facilities S-CSDP, 4 Mar 20167 SDA for analysis of archived data via online tools UK ‘e-Social Science’ & ‘Digital Social Research’ programmes explored distributed data storage & tools for online data analysis

8. 4) Online non-commercial organisations offering data analysis support ‘Datakind’ is a project putting volunteer analysts in touch with not-for-profit research data holders Academic training projects of potential value to non- academic research users, e.g.: S-CSDP, 4 Mar 20168 UK Data Service, https://www.ukdataservice.ac.uk/ https://www.ukdataservice.ac.uk/ UCLA statistics, http://www.ats.ucla.edu/stat/ http://www.ats.ucla.edu/stat/ Lemma training projects on statistical modelling, http://www.bristol.ac.uk/cmm/learning/online-course/index.html http://www.bristol.ac.uk/cmm/learning/online-course/index.html

9. 5) Online access to downloadable freeware for processing local data S-CSDP, 4 Mar 20169 ‘R’ is an impressive freeware with considerable statistical analytical capability – but also a very challenging software to make use of (https://www.r-project.org/ )https://www.r-project.org/ ‘gretl’ is a more accessible freeware offering a number of useful analytical options in a more intuitive format (http://gretl.sourceforge.net/ )http://gretl.sourceforge.net/

10. S-CSDP, 4 Mar 201610 ‘Stat-JR’ offers dowloadable integration between software, including freeware, through locally installed copies (http://www.bristol.ac.uk/cm m/software/statjr/ )http://www.bristol.ac.uk/cm m/software/statjr/

11. Aside: The tension between ‘simpler’ & ‘more complex’ statistical analysis ‘Complex’ analytical methods E.g. statistical models; sampling weights and survey design factors; sensitivity analysis for data permutations; ‘multivariate’ and ‘multiprocess’ systems Can be thought of as featuring a substantial element of ‘control’ for other factors relevant to the social mechanisms, e.g. ‘statistical’ models with many parameters expressing influences of ‘background variables’ and complex data structures ‘Simpler’ analytical methods E.g. univariate distributions, bivariate comparisons, accessible graphical summaries and headline percentages Can be appealing to communicate and still have important strengths, e.g. statistically representative patterns Introduce risks in summarising social mechanisms: spurious and unduly simplified trends and associations (e.g. interactions); incorrect point estimates and/or incorrect representation of uncertainty; encourages view that ‘statistics equal lies’ S-CSDP, 4 Mar 201611 -> Most online data analysis tools are using ‘simpler’ methods and moreover cannot readily be adapted to more complex analytical methods

12. Summary: Online survey analysis tools …The above is not comprehensive! Postings/reflections invited to this channel from Civil Society researchers in Scotland… Tools for online survey data collection and analysis can be handy and free/low cost There are ongoing innovations in tools for analysis of locally held statistical data via online engines (in development) There are some academic and public sector research tools for online analysis of existing datasets Some challenges: – Can we achieve effective documentation and clarity over analtical methods working in these modes? – Can we ensure that the analysis goes as far as it ought to in terms of its statistical complexity? S-CSDP, 4 Mar 201612