2. MOOC Observatory Dashboard
Slides author: Eugen Patrascu
Presenter: Manuel León
Principal Investigator:​ Su White (UoS)
Co­Investigators​:​ Manuel León (UoS), Adriana Wilde (UoS), Lisa Harris (UoS), Ruth Cobos (Universidad Autónoma de Madrid, UAM), Graeme Earl (UoS)
Interns: Eugen Patrascu & Raluca Ispas
01 September 2017

3. MOOC DASHBOARD - OVERVIEW
Dashboard: analysis and visualisation of learner activity data in MOOCs
Uses the University of Southampton courses data from the FutureLearn platform
The dashboard features a set of visualisations that range from demographic data, aggregate data, charts and heatmaps of activity data, comment viewing tools, a series of “scatterplots” that show different correlations regarding different aspects of learner activity and many others
The tool is aimed at all tutors, educators, instructors, facilitators, and teaching assistants involved in the delivery of MOOCs.

4. MOOC DASHBOARD - OVERVIEW
FutureLearn
University of Southampton courses
MOOC dashboard – can select a number of courses and render various visualisations of data
platform

5. MOOC DASHBOARD – previous work Demographic data
Demographic details of the learners: age, gender, education, status, degree level, geographical region, HDI index.
Same visualisations for the statement purchasers
Details displayed using various charts and maps
Useful to understand the learners’ background and make the courses more suitable for target audiences

6. MOOC DASHBOARD – previous work Activity data
The dashboard contains details about the course activities of the learners: the number of times each step was visited or completed, the number of steps visited/completed per day or even more complex visualisations
Details displayed using line charts, bar charts, or heatmaps
Useful to analyse the activity of users and understand their behaviour, in order to improve the quality of the courses

7. MOOC DASHBOARD – previous work Comments data
Details about the comments activity of the learners: comment viewer tool, visualisations displaying the number of comments and replies by course step, day or week
Heatmaps, bar charts, line charts and tables were used
Provides a good analysis of comments behaviour and engagement in discussions

8. MOOC DASHBOARD – previous work Enrolment data, correlations and more
Enrolment data for each course and run (e.g. number of joiners, leavers, active learners, statements sold etc.)
Sign ups and statements sold by day
Different correlations between metrics (e.g. likes vs replies)
And many more!

9. MOOC dashboard – our recent work Engagement trends in MOOCs
Using bar charts and regression lines we show the trend of learners’ engagement in a course run (based on the number of times steps were completed/visited)
‘Average regression line’ to see how the involvement in a particular course run compares to the average involvement
Statistical significance test to check if the difference in values is significant

10. MOOC dashboard – our recent work Further comments analysis
Further work has been done towards the classification of comments for easier analysis: lone/initiating posts, first/further/initiator’s replies.
Tables and various charts used to observe this categorisation by days, activity steps or total comments

11. MOOC dashboard – our recent work Understanding learners: comment behaviour
Classification of learners based on their commenting behaviour – loners, repliers, active social learners etc
Tables which show details for every learner: category, type and number of comments, comment text
Useful for research in social learning (how the people interact with each other in learning environments), topics of discussion or understanding the different types of students involved in MOOCs

12. MOOC dashboard – our recent work Understanding learners: background & expectations
Another step forward towards understanding learners is looking at their background and expectations of the course
Used their comment in step 1.2 (where they manifest their interest in the course) and the pre-course survey they submitted (what they want to get out of the course, ways to learn, experience with MOOCs, place to take the course)
These visualisations and data can be used to improve the courses considering where and how students learn, see if the course meets people’s expectations and check its suitability for its target audiences

13. MOOC dashboard – our recent work Understanding learners: learning paths
Visualising learning paths: do the students go through the course step by step or do they jump ahead?
Sankey diagrams used to show the number of students going from one course week to another, or from one step to another. It shows the transitions in both directions: forward and backwards
There is important research going on about the adherence of students to the designed learning paths – useful for improving the retention and engagement rates

14. MOOC dashboard – our recent work Educators’ involvement in MOOCs
Educators/Mentors have an important role in MOOCs: they help improve the learning experience by clarifying concepts, questions and encouraging engagement in discussions
Developed tools to track their activity in online courses (their comments and the correspondent dates and steps), as well as mark the comment threads which already have educators involved
Useful to ensure the students benefit from the mentors’ involvement and find ways to improve the overall experience

15. MOOC dashboard – our recent work The fine details and other work
Some of our work included basic software development tasks, such as fixing bugs or inconsistencies (links, heatmaps, download buttons, progress bars, lack of data) or implementing other basic features (e.g. visualising charts by percentages or by values)
Other work towards helping MOOC research include:
contribution to reports using new data
reading papers and implementing features explained there
creating excel files with various details about courses (e.g. type and number of comments of learners and mentors for each course run) for further comparisons or analyses of courses

16. MOOC dashboard – future work ideas and improvements
Back-end:
some speed improvements would be necessary, as some charts/tables take very long to compute
need to always ensure the code is well commented and follows good practices
Front-end:
user experience improvements needed (e.g. charts resizing depending on the screen size)
some research into tool usability would be good (e.g. usability testing)
MOOC research ideas:
Individual reports for learners (e.g. including trajectory, quizzes responses, pre and post course surveys)
analysis based on the types of steps (article/video/quiz/etc), such as measuring the engagement of learners with every type, or comparing course popularity/success based on the activities they use
measuring the effectiveness of video learning using the recent video statistical data