Conducting Research on Student Learning in Higher Education Developing research questions and workable methods Gary Poole Institute for the Scholarship of Teaching and Learning UBC Deakin University February 15, 2007
Two Fundamental Assumptions: Educational decisions based on good evidence are more likely to make a constructive difference The determination of SoTL impact must extend beyond traditional publication and grant counts
Session Objectives To make our reflection on teaching and learning more evidence-based To generate researchable questions from our teaching To categorize types of scholarly evidence in SOTL To link available evidence to questions
Session Overview Identify the sources and nature of our inferences about student learning Identify the sources and nature of our initial evidence about student learning Apply a 2x2 taxonomy to help locate, generate and use good data for the scholarship of teaching and learning
Inference My students are: Learning more Writing better Thinking more critically Thinking like Engineers
Initial Evidence Intuitive Anecdotal Observational
Scholarly evidence Forms Qualitative Quantitative Sources Course activity Added data collection
Data Forms and Sources Course Activity Added data collection Qualitative Quantitative
Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Quantitative
Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Quantitative Exam scores Assignment scores
Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Focus Groups Open-ended survey Quantitative Exam scores Assignment scores
Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Focus Groups Open-ended survey Quantitative Exam scores Assignment scores Likert surveys Enrolment figures
How to best measure impact — RQF Statements THE OLD “The current research block funding scheme is based on quantitative measures. That is, numbers of publications, external research income and Higher Degree by Research (HDR) student load and completions” (RQF, 2004). THE NEW “Research impact is defined as the social, economic, environmental, and/or cultural benefit of research to end-users in the wider community regionally, nationally, and/or internationally” (RQF, 2004).
How to best measure impact — Implications for SoTL FROM Counting number of publications and subsequent citations Grant funding TO Impact on practice Evidence of student learning Evidence of use Evidence of impact on the way the discipline is taught
Applying RQF Impact Criteria to SoTL A 10-page impact statement, required of all Research Groups: Verifiable, evidence-based claims against specific impact criteria; Up to four case studies that illustrate examples of those claims; Details of end-users who can be contacted as referees.