Learning Analytics (LA’s), Student Engagement and Retention

Overview of session Lisa – LAs, engagement & retention (15 mins) Brett – LA project at Deakin (15 mins) Questions? (5 mins) Your turn! (25 mins)

The connections…LAs, engagement & retention LAs, like any data, tell us nothing on their own… What LA’s do we collect, to answer what questions? For what purposes?

LAs a part of strategic response to retention ‘Retention, completion and success: what do we know?’ (Deakin- 2018) The 20 Course Retention Project (2017) http://dteach.deakin.edu.au/wp- content/uploads/sites/103/2019/05/Retention- What-do-we-know- 2018.pdf?_ga=2.161257213.1978603766.1558911 225-2073488728.1538950765

Deakin’s current context Retention steady for 5 years (79-81%) Cloud campus rate significantly lower Deakin University CRICOS Provider Code: 00113B

Engagement & retention Early engagement (first 6 weeks) is key to retention Timely, targeted with supports embedded in curriculum Best approach – coordinated whole-of institution response

Measurement & metrics Is engagement time on task?, reflecting on task (cognitive), investment in the task (affective)? Is it a product or a process? Measured at Unit, Course, uni level?

Measuring types of individual student engagement Behavioral Cognitive Affective Frequency questions asking Proportion of coursework emphasizing higher order thinking strategies Effort to work harder to meet instructor’s expectations Frequency of group / collaborative work Time spent on projects requiring integration and synthesis of ideas Investment to better understand someone else’s perspective Frequency of tutoring others Amount of coursework requiring practical application of knowledge or skills Time investment in studying Frequency of attending events in the community related to course material Tendency to be prepared (or lack preparation) for class Frequency of discussing course material outside of class-time Butler (2011) differentiates typical assessment indicators along each of the typically studied dimensions of student engagement. Gives a good sense of the sophistication required for LAs around engagement.

Academic Analytics “Academic Analytics reflects the role of data analysis at an institutional level, whereas learning analytics centers on the learning process (which includes analyzing the relationship between learner, content, institution, and educator)” – (Long, Siemans et al. 2011) Deakin University CRICOS Provider Code: 00113B

Learning Analytics “Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs” – (Long, Siemans et al. 2011) More critically, in todays context, it enables users of learning analytics (students, academics, administrators) to take action based on the data – as the learning occurs. Deakin University CRICOS Provider Code: 00113B

Three Levels of Analytics Student The Student is categorised as an end user. Their focus is on how LA dashboard can enable them to study and engage better. The dashboard provides insight not only to their own actions but also the broader cohort and cross unit interactions. Undergraduate/Postgraduate Full/Part Time Part of an equity/diversity cohort Cloud/Located Domestic /International Teaching Academic The Teaching Academic is at the coal face. They are delivering one or more unit. Their focus is on how LA can be used to deliver interventions or gain insight in how their students are engaging with their units and to take action. Delivering multiple units High Volume of students Is Sessional or Contract staff member Managing Sessional or Contract staff Part of an equity/diversity cohort Cloud/Located Administration The Administration layer is looking at more than one unit and multiple staff. This role is more interested in a organisational view/trends rather than direct interactions with single or groups of students. Educational/Instructional Designers Course Directors Unit chair leading multiple staff in one or more units Exploring trends Managing moderation of assessments Deakin University CRICOS Provider Code: 00113B

If I had asked people what they wanted, they would have said faster horses. apocryphally attributed to Henry Ford Eadweard Muybridge: The Horse in Motion 1878 Deakin University CRICOS Provider Code: 00113B

Design and Consistency Context is Critical while data does not lie it does rely heavily on context to move from a generated data point to actionable insight. Consistency is Key for learning analytics to pull and action meaningful outputs it must have a consistency of approach within the LMS and related systems. Consistency does not mean the same and LA should not stifle innovation. Clear Documented Design Processes are crucial as they provide not only a means to interrogate the data coming out of the system but a tool to map and align outcomes (unit, course and graduate) to assessment and regulatory standards. Dashboards provide a single location to access high level data, drill down to more granular levels, customise user reports and interventions. Deakin University CRICOS Provider Code: 00113B

Over to you! We will now spend 30 minutes workshopping your responses to a set of questions we have posed? Your responses will be used to…? Deakin University CRICOS Provider Code: 00113B

Feedback on todays session & next steps Provide feedback on the project now or email: Brett.McLennan@deakin.edu.au   Deakin University CRICOS Provider Code: 00113B