LEARNING ANALYTICS iTeach Seminar Series, 2016

WHAT IS LEARNING ANALYTICS? “This multiplicity of definitions should not be taken to indicate a field in confusion, but rather indicate the richness of the domain and the multiplicity of approaches to it.” JISC – UK

HOW CAN LEARNING ANALYTICS BE USED? Understand learners' behaviours and adapt to a personalised learning path. Identifying student and class engagement patterns online, i.e. are students logging in and accessing items/tools, and if so, during what time periods. Identifying resource usage patterns i.e. what resources are students finding valuable. Clarifying access patterns, both timing and duration, for individual students and the class. This can be particularly useful in the context of assessment submission.

THE ELEPHANT IN THE ROOM

Ethical principles – beneficence – doing good – Non-maleficence – do no harm – Respect for autonomy – self determination – Justice – concept of fairness Cost/benefit Mitigating the risk Using a moral compass

LEARNING ANALYTICS AT CDU Analytics Reports Combines data from LMS with student and unit/course attributes from Student Management System Key metrics: Accesses, Time, Interactions & Clicks, Submissions & Posts, LMS Grade Performance, Student Demographics and Unit/Course attributes Learnline Analytics Reports

LEARNING ANALYTICS AT CDU

Provides a view of information about sites and activity Can be used to develop a high-level understanding of how the Learnline is being used How is the use of Learnline is changing over time Offer self-service reporting options Gain insight into user activity, site design and student grades

IS IT BEING CDU? Analytics Reports

JOSEPHINE STUDENT (#####) LEARNING ANALYTICS AT CDU

HOW IS LEARNING ANALYTICS BEING USED? Student Snapshot Survey results (September 2015) Overall how useful do you find the Student Snapshot report?

HOW IS LEARNING ANALYTICS BEING USED? Student Snapshot Survey results (September 2015) What has the report has been used for? Results Familiarise myself with the class for general preparation for teaching 40% rated as very useful Identify issues to discuss with colleagues 20.8 % rated as vey helpful Identify services to support students37.5 rated as very helpful Identify attributes which may impact learning & teaching approaches 41.7% rated as very helpful In ranking the usefulness of the report variables: Student NESB attribute and Number of Unit attempts # 1 Number of Unit enrolments # 2 ATSI indicator & INT/EXT indicator # 3

HOW IS IT BEING USED? Learnline Staff support scenario “Learnline support, can I help you ” “Hi do I need to use Tracking on content items to show me that a student has viewed the item? ” The Analytics Reports Unit at A Glance and Student at a Glance shows clicks and interactions across the class and for each student enrolled in the class. The Items accessed by Student table in the Student at a Glance report shows the item name, access count, date of first access & last and time measures. “You could use the Analytics Reports to show you what items have been viewed”

HOW IS IT BEING USED? Dr James Valentine, Lecturer - Tertiary Enabling Program - Maths / Bioscience

HOW IS IT BEING USED? Monitoring student engagement First and last student accesses to TEP025 Bioscience Learnline site during S115

Analytics – Understanding your teaching style and your students learning style Dr Hana Morrissey

Units I taught in S PHA411 – 19 students PHA306 – 10 students SCL200 – 14 students

Who I will be teaching? PHA411

Who I will be teaching? PHA306

Who I will be teaching? SCL200

Do I need to change anything in my approach? No, but I need to keep an eye on 4 students with GPA <4 in PHA411, 3 in PHA306 and 2 in SCL200, as potentially at risk

During Semester, I keep eyes on..

All weeks will start purple, then…

More details……keep in mind that….. These units are offered only internal mode, also heavy in PBL, i.e. most interaction done in class, most content is either lectures or resources.

Placement, 1 st lecture Monday W3 Oral presentations and report due and revisions

Placement for 3 weeks Oral presentations and report due and revisions Compulsory module 5 W/s – aseptic and cytotoxic and PharmStock game due

Series of workshops Preparation of Mini Interviews End of unit interactions, W12

Most classes and access the LL during class Very steady every day, 4 th year unit

Heavy on in-class work during lectures and tutorials, but also son online activities

Heavy on in-class work during lectures and tutorials, increased towards end of semester

90 min

110 min

280 min

Steady…

Dropped during the placement time weeks 6-9

Dropped after week 6 where the intensive class practice started up to week 12….

No online assessment items

Only PharmStock

No online assessment items

No LL tools, students used PCC4U webpage to complete their 30% assessment items

Compulsory module 5 W/s – aseptic and cytotoxic

After the first term, students used the textbook interactive learning page

Who access LL more, the top student or the students at risk? Are all top students’ Learnline use is the same?

Top student

Very low GPA

Middle 50/50

All contact done in class

Average student

top student

Results vs assessments Out of the three units no student failed – I had intensive remediation with 6 students

Was I worried? I saw my students every week Understanding student by looking at the full picture, not just how many clicks

Thank you

KEY FINDINGS “ Learning Analytics: Assisting Universities with Student Retention” (2015) Charles Darwin University, Batchelor Institute of Indigenous Tertiary Education, Griffith University, Murdoch University and the University of Newcastle 1. The sector in Australia is at an early stage of development, implementation and understanding. 2. Context is important. 3. There is great variability across the sector – in relation to preparedness and how institutions are thinking about and implementing analytics. 4. There is tension around the extent to which learning analytics can drive actions and behaviours or take the functions of people. 5. There is tension between the ‘business’ needs, wants and limitations (costs) verses ‘academic” staff needs and wants (e.g. academic freedom, and innovation in learning and teaching) 6. People across institutions have a key role to play in leveraging the opportunities of learning analytics, which must take account of the relationships between strategy, planning, policy and action.

KEY FINDINGS “ Learning Analytics: Assisting Universities with Student Retention” (2015) Charles Darwin University, Batchelor Institute of Indigenous Tertiary Education, Griffith University, Murdoch University and the University of Newcastle Curriculum /course design and teaching improvement Institutional resource planning Improving retention Student behaviours Academic actions and behaviours Student background and demographics Student performance KEY DATA THAT ACADEMIC STAFF WANT USE CASES/ APPLICATIONS DATA SOURCES OUTCOME

KEY ANALYTICS QUESTIONS Theme 1:Curriculum/Course design & improvement – Is unit/course design effective? – Which unit/course(s) are performing well? (within a school, faculty, university) For example – How active are students in unit/course content overall and at an individual content level? – Analysis of unit/course responsiveness to shape future courses to meet user preferences

KEY ANALYTICS QUESTIONS Theme 2:Student behaviour When and how are students engaged with the LMS and associated tools? – what times/lengths of day – spread of access throughout academic calendar, particularly key milestone dates? When did a student last access ? When did a student last access Wi-Fi? Is there correlation between messaging and test attempts?

KEY ANALYTICS QUESTIONS Theme 3:Improve Retention Which students are at risk? Can we predict/anticipate which students are likely not to engage/stay in course? How do we predict which student will perform well? Or vice versa? Analytics for identifying, monitoring, managing and responding to students at risk. What are the indicators? Who are the most at risk? What interventions are more likely to be effective? What trends can we see in terms of at risk students?

KEY ANALYTICS QUESTIONS Theme 4:Resourcing/planning at institutional level What does the institution need to do to assist the student in their learning? Gauging impact of new initiatives/introduction of new technologies Where are students connecting with “the university” in their learning journey? E.g. Where in Bb? Library? Face to face? Services? Other unexpected places? Can we identify potential marketing opportunities to recruit new cohorts of students? Is the time and/or money/effect that has been put into something worthwhile? Are there particular things which are over or underrepresented?

OTHER APPLICATIONS EVALUATION CURRICULUM REVIEW SOTL - AWARDS

WHAT COULD YOU DO WITH IT? In your context/role – how can you use analytics? omap out the purpose otiming, oreflection/ action Share from your table the key ideas/plans

FOLLOW UP ACTIVITY Bb Analytics Consultant, Nicole Wall at Casuarina Campus PYRAMID ANALYTICS SCHOOL DASHBOARDS – OPEN SESSION TUESDAY 15 MARCH, OLT TRAINING ROOM, ORANGE (12:30PM-1:30PM)