Can survey data be used as an indicator of learning outcomes? Karl Molden, Senior Planning Analyst Veronika Hulikova, Student Analyst

| University of Greenwich Context 38,372 Students 67% Full time 82% Undergraduate In 2014 the University Student Survey was issued to: On-campus, first year undergraduates Non final-year undergraduates UK partner colleges Wholly overseas based students In 2015 we introduced surveys at module level – every student on every module is asked to answer a standard set of questions The university is developing its systems and policy with regards to analytics and their use

| Motivation Surveys are used across the sector and typically well understood by staff and students – potential to develop something clear and understandable for both groups which could be adapted easily by other institutions We know that other work carried out has found a small but significant relationship between measures of engagement and academic achievement* - this project would build on that to create “real-time” analytics from module level surveys Systems for collection of data already exist and are well supported We are "drowning in a sea of data“ – why add more if we can leverage what’s already there?"drowning in a sea of data“ * Sheffield Hallam University, Using UKES results and institutional award marks to explore the relationship between student engagement and academic achievement

| Method Match University Student Survey (USS) data with academic outcomes Statistical analysis to determine if there is a correlation between responses to USS questions and outcomes. Where/if a correlation is found, how strong is it? Process developed using R, open source statistical software. Code will be made available online, to allow other institutions to repeat and/or develop analysis. USS 2014 contained the UKES questions as a subset of the whole survey which means we can try to replicate the results from Sheffield Hallam. This material and its contents [UKES] is developed by The Higher Education Academy (“HEA”). Some of the questions in this survey are used with permission from The College Student Report, National Survey of Student Engagement, Copyright The Trustees of Indiana University. With the exception of the aforementioned, the copyright in such material and content belongs to the HEA. No reproduction, modification, or adaptation is permitted without the prior written consent of the HEA © The Higher Education Academy 2012, Amended 2015, All Rights Reserved.

| UKES Results Sheffield Hallam identified 6 UKES questions which had a statistically significant correlation with academic outcomes. Our analysis has identified 4 of the same 6 questions as having a similar correlation. One further USS question also found to have statistically significant correlation. Three of these five questions are thematically linked – around working with/explaining material to other students.

| Results

| Course Evaluation Survey Results Further to the results with the UKES questions, two questions from the course evaluation surveys were found to have a correlation with academic outcome. As these surveys are conducted every term this could be useful in developing “real-time” interventions. Further work is needed to look at the potential of adding further data to the mix to develop more sophisticated models.

| Course evaluations

| Conclusion The evidence seems to support a small, positive correlation between certain survey question responses and academic outcomes. By identifying activities which may lead to improved responses to certain questions it may therefore be possible to improve these outcomes. Similar relationships in “in-session” course level surveys have been identified which mean we may be able to develop other processes to improve academic achievement of students. | Questions?