MOOC Observatory Dashboard Slides author: Eugen Patrascu Presenter: Manuel León Principal Investigator: Su White (UoS) CoInvestigators: Manuel León (UoS), Adriana Wilde (UoS), Lisa Harris (UoS), Ruth Cobos (Universidad Autónoma de Madrid, UAM), Graeme Earl (UoS) Interns: Eugen Patrascu & Raluca Ispas 01 September 2017
MOOC DASHBOARD - OVERVIEW Dashboard: analysis and visualisation of learner activity data in MOOCs Uses the University of Southampton courses data from the FutureLearn platform The dashboard features a set of visualisations that range from demographic data, aggregate data, charts and heatmaps of activity data, comment viewing tools, a series of “scatterplots” that show different correlations regarding different aspects of learner activity and many others The tool is aimed at all tutors, educators, instructors, facilitators, and teaching assistants involved in the delivery of MOOCs.
MOOC DASHBOARD - OVERVIEW FutureLearn University of Southampton courses MOOC dashboard – can select a number of courses and render various visualisations of data platform
MOOC DASHBOARD – previous work Demographic data Demographic details of the learners: age, gender, education, status, degree level, geographical region, HDI index. Same visualisations for the statement purchasers Details displayed using various charts and maps Useful to understand the learners’ background and make the courses more suitable for target audiences
MOOC DASHBOARD – previous work Activity data The dashboard contains details about the course activities of the learners: the number of times each step was visited or completed, the number of steps visited/completed per day or even more complex visualisations Details displayed using line charts, bar charts, or heatmaps Useful to analyse the activity of users and understand their behaviour, in order to improve the quality of the courses
MOOC DASHBOARD – previous work Comments data Details about the comments activity of the learners: comment viewer tool, visualisations displaying the number of comments and replies by course step, day or week Heatmaps, bar charts, line charts and tables were used Provides a good analysis of comments behaviour and engagement in discussions
MOOC DASHBOARD – previous work Enrolment data, correlations and more Enrolment data for each course and run (e.g. number of joiners, leavers, active learners, statements sold etc.) Sign ups and statements sold by day Different correlations between metrics (e.g. likes vs replies) And many more!
MOOC dashboard – our recent work Engagement trends in MOOCs Using bar charts and regression lines we show the trend of learners’ engagement in a course run (based on the number of times steps were completed/visited) ‘Average regression line’ to see how the involvement in a particular course run compares to the average involvement Statistical significance test to check if the difference in values is significant
MOOC dashboard – our recent work Further comments analysis Further work has been done towards the classification of comments for easier analysis: lone/initiating posts, first/further/initiator’s replies. Tables and various charts used to observe this categorisation by days, activity steps or total comments
MOOC dashboard – our recent work Understanding learners: comment behaviour Classification of learners based on their commenting behaviour – loners, repliers, active social learners etc Tables which show details for every learner: category, type and number of comments, comment text Useful for research in social learning (how the people interact with each other in learning environments), topics of discussion or understanding the different types of students involved in MOOCs
MOOC dashboard – our recent work Understanding learners: background & expectations Another step forward towards understanding learners is looking at their background and expectations of the course Used their comment in step 1.2 (where they manifest their interest in the course) and the pre-course survey they submitted (what they want to get out of the course, ways to learn, experience with MOOCs, place to take the course) These visualisations and data can be used to improve the courses considering where and how students learn, see if the course meets people’s expectations and check its suitability for its target audiences
MOOC dashboard – our recent work Understanding learners: learning paths Visualising learning paths: do the students go through the course step by step or do they jump ahead? Sankey diagrams used to show the number of students going from one course week to another, or from one step to another. It shows the transitions in both directions: forward and backwards There is important research going on about the adherence of students to the designed learning paths – useful for improving the retention and engagement rates
MOOC dashboard – our recent work Educators’ involvement in MOOCs Educators/Mentors have an important role in MOOCs: they help improve the learning experience by clarifying concepts, questions and encouraging engagement in discussions Developed tools to track their activity in online courses (their comments and the correspondent dates and steps), as well as mark the comment threads which already have educators involved Useful to ensure the students benefit from the mentors’ involvement and find ways to improve the overall experience
MOOC dashboard – our recent work The fine details and other work Some of our work included basic software development tasks, such as fixing bugs or inconsistencies (links, heatmaps, download buttons, progress bars, lack of data) or implementing other basic features (e.g. visualising charts by percentages or by values) Other work towards helping MOOC research include: contribution to reports using new data reading papers and implementing features explained there creating excel files with various details about courses (e.g. type and number of comments of learners and mentors for each course run) for further comparisons or analyses of courses
MOOC dashboard – future work ideas and improvements Back-end: some speed improvements would be necessary, as some charts/tables take very long to compute need to always ensure the code is well commented and follows good practices Front-end: user experience improvements needed (e.g. charts resizing depending on the screen size) some research into tool usability would be good (e.g. usability testing) MOOC research ideas: Individual reports for learners (e.g. including trajectory, quizzes responses, pre and post course surveys) analysis based on the types of steps (article/video/quiz/etc), such as measuring the engagement of learners with every type, or comparing course popularity/success based on the activities they use measuring the effectiveness of video learning using the recent video statistical data