Talk to Us Approach to Evaluation Lindsay Wager 1 st April 2014

2 Questionnaires written by PhD student (member of Outreach team). Pupils- pre involvement questionnaire - post visit questionnaire - focus groups - celebration event questionnaire Teachers- pre involvement questionnaire - interviews - post project questionnaire Researchers- pre involvement questionnaire - interviews - post project questionnaire Year 1 Purpose: to evidence impact of project in terms of intended outcomes and to inform practice regarding content, structure, level etc. of programmes.

3 School StudentsTeachersResearchers Senior Managers in Schools Main aims and objectives -Understanding of purpose of research at University -Understanding about the research topic relating to the sub-project -Understanding about intentions to study at HE and engage in future research -Understanding of purpose of research at University -Confidence in leading their own research projects within school -Views on students’ interest in attending HE, engagement with research and researchers -Views on benefits for students, school and personal being involved in project -Challenges and barriers of involvement -Perceived impact of project on students, school and personal -Views on working with school students -View on personal barriers and concerns with engagement to project -Views on impact of interactions to students and teachers -Personal impact of interactions -Views on how student and teacher feedback is influencing development/review of activities -Views on benefits for school, teachers and students being involved -Perceived impact of project on students, school and teachers involved Methodologies -Questionnaires to all participating students (pre and post interactions) -Focus group of 5-6 randomly selected students from each participating school -Questionnaires to all lead contact teachers from each school (pre and post interaction) -Interview with each contact teacher -Interview with teachers who withdraw from project -Questionnaires to all researchers -Focus group -Exit interviews for any researchers who withdraw from project -Interviews (towards end of project 2017) Timeline of evaluation interaction -Pre-interaction questionnaire to be completed before first interaction with researchers -Post-event interaction questionnaire -Focus group session up to 2months after the first interaction -End of year/full interaction celebration event questionnaire to be completed at end of year celebration event -Pre-interaction questionnaire to be completed on agreeing to be part of project -Interview session up to 2 months after first interaction -Post-interaction questionnaire to be completed at end of year celebration event -Exit interview should school withdraw from project -Pre-interaction questionnaire -Focus group session up to 2 months after first interaction -Post-interaction questionnaire to be completed at end of year celebration event -Exit interview should researcher withdraw from project -Initial interview to coincide with first interaction/school signing up to project -Final interview to take place after last interaction in year 3 of project

4 Focus of questionnaires/ nature of questions Lost PhD student Programmes didn’t follow planned structure due to planning process. logistics involved with paper questionnaires Pupil/ parental consent Difficulty getting data from researchers Celebration event questionnaire Useful data on impact of project on academic aspirations and enjoyment of science at school. Useful feedback and positive comments from pupils. Focus groups particularly useful in lieu of questionnaires. Issues encountered Successes

5 Original pupil questionnaireNew pupil questionnaire Have you visited the University of Southampton before? If yes, please describe the activity. Does anyone in your family have a degree? If yes, please state who? How interested are you in attending university? How do you feel about learning science at school? How interested are you in studying science subjects after your GCSEs? How would you rate your confidence in leading your own scientific research in school? Do you think scientific research is used by companies or the general public? What do you hope to find out during the visit today? Have you visited the University of Southampton with your school before? What do you think the term ‘research’ means? Why do you think universities carry out research? Do you think the scientific research that happens at the University of Southampton is directly relevant to your everyday life? Please explain your answer. How confident are you at carrying out scientific investigations at school? Use focus groups to unpick ideas they have related to themselves as researchers.

6 Questions to focus on information required Questionnaires and consent forms online (will still need parental consent forms for pupils). Nature of questions? (focus for researcher and teacher questionnaires). Parental consent? Long term impact? Senior management? Getting full data sets (particularly researchers). Moving forward… Feedback…