Outcomes of the ACODE 60 workshop on Learning Analytics Associate Professor Rob Phillips Murdoch University, Perth

ACODE: the peak Australasian organisation for universities engaged or interested in open, distance, flexible and e-learning. Mission: to enhance policy and practice in open, distance, flexible and e-learning in Australasian higher education

Aims What interesting things are being done with analytics? What is the question to which analytics is the answer? What are the ethical issues involved in analytics? What is ACODE’s role in developing a national agenda around analytics?

Academic and Learning Analytics Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in Learning and Education. EDUCAUSE Review, September/October, pp Maree Gosper For students For teachers

Part 1 What interesting things are being done?

Academic Analytics Central Queensland data – Beer et al. – 300M Blackboard records – 110M Moodle records – 80k student records Queensland University of Technology data – Harper et al. Outcomes – Grade Point Average -> better results – Online engagement -> better results

Academic Analytics Open Universities Australia data – Lynch – 33k students – 66k units Outcomes – Success in one unit predicts future success – Previous university study predicts success – Exam in first unit predicts failure

Big Data Assume homogeneity in: – course/ unit design – assessment – the role of the teacher Individual variability is hidden by the volume of data More complex

Curriculum Analytics

Learning Analytics for Students Felicia Zhang – Early detection of students at risk – alert system Wendy Harper – Individual patterns of behaviour affect results

Learning Analytics for Teachers Correlation between teacher engagement and student engagement??? Student behaviour is driven by: – course/ unit design – assessment – what the teacher does How can teachers use learning analytics to improve course design? Theme from meeting that we don’t do this well yet

Learning Analytics for Understanding Low 10% student located in network Students with a grade >75% < 90% Social Network Analysis Shane Dawson

My Work How do students study and learn in technology- enhanced tertiary learning environments? Use Lectopia learning analytics to select students with diverse behaviours

When listened to?

Categorisation

My Work How do students study and learn in technology- enhanced tertiary learning environments? Use Lectopia learning analytics to select students with diverse behaviours Interview them about their study behaviour Get their marks Tell their stories – rich descriptions All participants exhibited thoughtful study behaviour

Part 2 What issues arise? Some discussion starters for the panel session…

Ethics The ethical professional – Respecting the rights of students – Stepping in to provide pastoral support advise about risks of failure advise about increasing chances of success

Ethics The ethical professional – Respecting the rights of students – Stepping in to provide pastoral support advise about risks of failure advise about increasing chances of success Research ethics – Risk minimisation – Needed for publication – Issues with: accessing ‘databanks’ anonymity

Privacy Issues The Greater Good vs Big Brother Teacher: – “It’s unethical not to tell a student they are at risk of failing” Student: – “I don’t want you to be looking over my shoulder. I can make my own choices about my study.” Reports to staff vs dashboards for students

Analytics at Institutions Role description: Learning Analyst Skill set – Pedagogy – Programming – Statistics/ visualisation – Teams? Structures – Institutional silos

Usefulness of analytics What is the question to which analytics is the answer? – Don’t just buy a product Learning analytics are just indicators of behaviour – They don’t explain behaviour A single source of analytic data is probably insufficient – Combine data into a data warehouse

Policy Inform students on enrolment that system data is being collected, and it can be used for quality improvement

Summary Analytics can tell us lots of interesting stuff Analytics data is complex and insights can be lost through aggregation - learning design and teacher role Tension between the ethical professional and research ethics Skills of a ‘Learning Analyst’

What’s Next? ACODE Special Interest Group Written workshop report Annotated bibliography Executive briefing