1. Conducting Research on Student Learning in Higher Education Developing research questions and workable methods Gary Poole Institute for the Scholarship of Teaching and Learning UBC Deakin University February 15, 2007

2. Two Fundamental Assumptions: Educational decisions based on good evidence are more likely to make a constructive difference The determination of SoTL impact must extend beyond traditional publication and grant counts

3. Session Objectives To make our reflection on teaching and learning more evidence-based To generate researchable questions from our teaching To categorize types of scholarly evidence in SOTL To link available evidence to questions

4. Session Overview Identify the sources and nature of our inferences about student learning Identify the sources and nature of our initial evidence about student learning Apply a 2x2 taxonomy to help locate, generate and use good data for the scholarship of teaching and learning

5. Inference My students are: Learning more Writing better Thinking more critically Thinking like Engineers

6. Initial Evidence Intuitive Anecdotal Observational

7. Scholarly evidence Forms Qualitative Quantitative Sources Course activity Added data collection

8. Data Forms and Sources Course Activity Added data collection Qualitative Quantitative

9. Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Quantitative

10. Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Quantitative Exam scores Assignment scores

11. Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Focus Groups Open-ended survey Quantitative Exam scores Assignment scores

12. Data Forms and Sources Course Activity Added data collection Qualitative Journal Group process analysis Focus Groups Open-ended survey Quantitative Exam scores Assignment scores Likert surveys Enrolment figures

13. How to best measure impact — RQF Statements THE OLD “The current research block funding scheme is based on quantitative measures. That is, numbers of publications, external research income and Higher Degree by Research (HDR) student load and completions” (RQF, 2004). THE NEW “Research impact is defined as the social, economic, environmental, and/or cultural benefit of research to end-users in the wider community regionally, nationally, and/or internationally” (RQF, 2004).

14. How to best measure impact — Implications for SoTL FROM Counting number of publications and subsequent citations Grant funding TO Impact on practice Evidence of student learning Evidence of use Evidence of impact on the way the discipline is taught

15. Applying RQF Impact Criteria to SoTL A 10-page impact statement, required of all Research Groups: Verifiable, evidence-based claims against specific impact criteria; Up to four case studies that illustrate examples of those claims; Details of end-users who can be contacted as referees.