So, did it work? Academic Development Officer, HEA 1 April 2014 Jenni Carr Academic Development Officer, HEA 1 April 2014

Evaluation and/or impact assessment? Evaluation concentrates on processes and outcomes. It seeks to find out what is happening or has happened as a result of the activity, and what is (or is not) working and what did (or did not) work. the objectives (the intended achievements) of the activity being undertaken; the inputs (resources) into the activity (including money and people’s time); the outputs (activities and products) of the activity; the outcomes of the activity (what happens as a result of the activity). The importance of distinguishing between evaluation and impact assessment. Assessing impact (highlighted in bold) requires a focus on ‘measuring’ change and so when planning activities you need a model that is designed to do this.

Impact is about change What do we want to achieve? What needs to change in order to achieve this? What activity is needed in order to bring about those changes? Theory of change model. Sometimes the planning process, especially when related to bids for funding, leads to a focus on designing activity first. This model is designed to make planning processes focus on impact first. Useful both at the beginning of the project, but also for reviewing process in light of interim evaluations. Always starts with ‘what do we want to achieve?’

Participants in change (In)visible witnesses Children and young people Educators Funding bodies Programme-makers HEA Social Sciences strategic priorities Project teams The wider higher education community Lecturers Deans PVCs Firstly you need to identify who will be the participants in change. They will participate at different levels e.g. HEA strategic priorities – project teams/lecturers, we aim to change their practice, but Deans/PVCs, we aim to increase their understanding. So how do we plan activities for impact at these different levels?

Generic learning outcomes (GLO) All impact/change can be related to generic learning outcomes model (GLO) This version is particularly useful as developed as part of the work of Museums, Libraries and Archives Council and would be relevant to the SUPI projects. URL takes you to an interactive version of the model with more detailed description of each category. Further useful resources on this website include: Guidance on recording and analysing both qualitative and quantitative data; activities for practising your GLO coding and further advice on measuring outcomes. http://www.inspiringlearningforall.gov.uk/toolstemplates/genericlearning/

Some thoughts on methods The value of quantitative – but remember to measure change Social networking Games, including competitions Simulations “Making is doing” creative activities If you are going to use surveys/questionnaires to generate quantitative data, do remember to design one section of questions that focuses on measuring change. This is generally more effective if you combine with opportunities for free text answers, so you do need to think about how you will handle the qualitative data. Some level of co-ordination across the different projects that are part of the initiative could be useful. Sharing questionnaires that can then be adapted if needed not only allows for some comparison, but helps with workload! There’s also a question bank available on the MLA website: http://www.inspiringlearningforall.gov.uk/resources/research.html There are numerous tools for capturing data from social networking – see LSE Impact blog: http://blogs.lse.ac.uk/impactofsocialsciences/ including excellent ‘basics’ guide for Twitter. Game-based learning – remember to have maximum impact participants need to have some idea of what the game is designed to achieve otherwise it is ‘just a game’ – and you need to think about how you measure the outcomes/changes. Alex Moseley – University of Leicester - has useful blog and has always been helpful in responding to queries. http://moerg.wordpress.com/contact/ Young people in particular seem to enjoy simulation activities. If scenarios and activity sheets are well-written these activities give them a high degree of autonomy. Combine with reflective activity to help ‘focus down’ on outcomes. David Gauntlett’s Art Lab website has some useful ideas http://www.artlab.org.uk/ and MLA case studies also have useful ideas: http://www.inspiringlearningforall.gov.uk/successstories/ and download section has good resources for measuring outcomes: http://www.inspiringlearningforall.gov.uk/resources/research.html, particularly useful for interpreting visual images.