1. Links in the Chain: turning learning analytics data into actions
ABLE Project BE-EPPKA3-PI-FORWARD STELA Project: EPP BE-EPPKA3-PI-FORWARD

2. Development of the NTU Student Dashboard

3. Learning Analytics
"Analytics is a term used in business and science to refer to computational support for capturing digital data to help inform decision-making … Learning Analytics appropriates this concept for education.”
(Buckingham Shum, 2012, p.1)
(Clow, 2012)

4. Why NTU is interested in learning analytics?
Retention
Belonging
Attainment
Research & improving the student experience
What works? 1
Internal audit
Entrepreneurial IS team

5. Developmental Cycle Pilot Phase Phase One Phase Two Phase Three
Sept 2013
Sept 2014
Sept 2015
Feb
2016
Pilot Phase
Phase One
Phase Two
Phase Three
4 courses
40 staff
500 1st years
Willing participants: very positive staff feedback, limited student awareness
8 of 9 Schools
Governance
Problem solving
Ethics
Near to whole University roll out: increased awareness
All Schools
New data sources
Assessment view
Whole University roll out: increased awareness
Personalisation
Embedding into University systems
Increasing resources (e.g. how to guides), communication, guidance
Staff and student consultation ongoing throughout developments
Further details of projects at and

6. The NTU Student Dashboard
Can be viewed as two products:
Physical
Dashboard
Algorithm
Staff and students interact with physical dashboard
Algorithm is the behind the scenes, learning analytics element

7. What does the Dashboard do?
Student biographical info, e.g. enrolment status
Can make comments in free text box
Staff
view
Student
view
Evidence of student engagement
Door swipes (where appropriate)
Library books
NOW use
Dropbox submissions
Attendance data
Access to e-books and journals through Shibboleth authentication
NTU Student Dashboard
Compares student engagement across the cohort & gives rating
Raises alerts!!

8. Potential benefits of learning analytics for personal tutors

9. Data accuracy for algorithm spotting students at risk
Two big questions:
Can the algorithm correctly identify at risk students?
Can it do so on a timescale that allows intervention?

10. Relationship between yearly average engagement & progression
Low average (mode) engagement for the year is an indicator of risk
Please do not share progression data
The graph relates to the impact of average engagement upon progression. Without explanation, it would be easy for a student to misinterpret any current low engagement meaning that they are highly at risk of early withdrawal. Relatively few students had an low average engagement for the whole year.

11. Relationship between term one average engagement & progression
Low average (mode) engagement for the 1st term is an indicator of risk
Please do not share progression data
The graph relates to the impact of average engagement upon progression. Without explanation, it would be easy for a student to misinterpret any current low engagement meaning that they are highly at risk of early withdrawal. Relatively few students had an low average engagement for the whole year.

12. No engagement alerts Any one alert is an indicator of risk
Students with multiple alerts had lower incidence of progression

13. Class view Designed so staff have easy access to student data.
Allows staff to quickly identify potentially at risk students
Links to student’s dashboard
Able to sort on headings

14. Individual student view
Not Fully Enrolled
Staff and students can benchmark engagement – springboard for conversation

15. Notes and referrals Notes inputted by staff only Time and date stamp
Referral to:
Library Academic Skills
Student Support Services
Employability (planned)
Personal tutors can track interactions with students
Personal tutors can make referral whilst in room with student

16. Student profile Basic information (ability to report if this is wrong)
Engagement summary
Entry qualification details
Engagement history (for previous years)
Details to help early tutorials

17. Attendance Data drawn automatically from the attendance app
The wheel shows overall attendance
The table below provides more detailed information for recent weeks
Springboard for conversation (potential context of School attendance policy)

18. Assessment & Feedback View
Only show assessments and feedback submitted through NOW (the University’s VLE)
Shows assessments and feedback for multiple modules
Better sight of student performance than only seeing own module results so tutor can make more informed recommendations

19. The challenges of converting student information to actionable intelligence

20. How are we using the Dashboard to address disparities in attainment?
Two change agents
Developing learning analytics is challenging
Institutional change based on learning analytics may be a different order of magnitude
Institutions already have data on students at risk, attendance, non-submission of coursework
Yet it is often extremely difficult to change student outcomes
Students
Staff

21. Exploring the NTU Student Dashboard
Checking their engagement scores and increasing the amount of time they spend studying were the top explorations of the Dashboard amongst students.
Have you logged into the NTU student Dashboard?
When using the Dashboard, how often have you explored the following?
Base: 515 (2016), 469 (2015)

22. Relationship between number of Dashboard log-ins and engagement classification
Some groups are more likely to log in more frequently, e.g. males, A level students

23. How are we engaging with Staff as Change Agents?
Changing personal tutoring policy
Providing evidence through research
Induction activity
Student surveys and interviews
iPad trial
Making the systems better
Notes and referrals function
Staff communication
Briefings and drop-in sessions
Newsletter term 1 – low engagement
Newsletter term 2 – alerts

24. Thank you for listening. Any questions?