1. Innovation in Learning Analytics and Learning Scenarios in Clinical Psychiatry WEDNESDAY APRIL 13 2016│ K1.28 Carolyn Hodgman Leading Education Initiatives, TEL Manager, IoPPN Dr Patrick Davey Lecturer IoPPN, Junior Doctor SLAM

2. Overview Scenario Branching Learning Analytics Case Studies Instructional Design & Subject Specialism Design Process Data Visualisation

3. Learning Analytics ‘the use of intelligent data, learner- produced data, and analysis models to discover information and social connections, and to predict and advise on learning’ Siemens, 2010 Debated Definitions ‘the process of developing actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data’. Cooper, 2012 ‘collecting traces that learners leave behind and using those traces to improve learning’ Duval 2012 ‘uses data about students and their activities to help institutions understand and improve educational processes, and provide better support to learners’ JISC, 2015

4. Learning Analytics Wider view: ‘the development of methods that harness educational data sets to support the learning process … through a reference model for LA based on four dimensions, namely data, environments, context (what?), stakeholders (who?), objectives (why?), and methods (how?)’ Chatti, et al, 2014

5. Learning Analytics This Context The 2016 Horizon Report describes learning analytics as "an educational application of web analytics aimed at learner profiling, a process of gathering and analysing details of individual student interactions in online learning activities." Mental Health Case Studies Framed as exam revision and formative assessment they focus on behaviours over ideas and provide an opportunity to reflect on students’ preparedness for clinical practice by recording decision pathways taken by students and measuring these against the optimal, or ‘golden path’ recommended by subject specialists.

6. Scenario Branching Conditional branching (or Skip logic) is a feature that changes what question or page a respondent sees next based on how they answer the current question. Conditional branching creates a custom path through the survey that varies based on a respondent’s answers. Moodle.org 2014 Simulation Based Learning

7. Mental Health Case Studies Opportunity to explore real world challenges and engage in the decision making process, without opening students, and patients, to risks In Real Life. The online clinical case scenarios from the IoPPN’s MSc in Mental Health Studies, of various mental health conditions, give students the opportunity to further develop skills, understanding and confidence in: Asking appropriate questions in a psychiatric history Recognising symptoms of mental illnesses Distinguishing between different types of mental illnesses Managing mental illnesses

8. Demonstration https://keats.kcl.ac.uk/course/view.php?id=27625 http://moodle2.iop.kcl.ac.uk/course/view.php?id=185 8 Case Studies, 300 pages of content, 200 short videos, 300 Qs, As & Feedback each What skills/people involved 55 Students in pilot. 130 students on MSc per year. Averaging 40 mins – 1 hrs learning time each case study

9. INSTRUCTIONAL DESIGN & SUBJECT SPECIALISM

10. Process Writing Script & Content Decision Trees Questions, Answers & Feedback Coding the Content Filming Technical Design & VLE Data and Analysis

11. Decision Tree Process

13. Coding Qs, As & Feedback Feedback sheets Scoring for optimal path Demonstration

14. Coding & Scoring

15. Moodle Lesson & Extracting Data Technologies used were the Moodle lesson tool, SQL queries to mine data, ‘R’ for data analytics and visualisations. Expertise in learning scenarios in clinical psychiatry as well as statistical analysis were also used. Demonstration

16. Raw Data

17. Pilot Stage 55 students in total accessed the scenarios. The numbers for each are as below : 53 - Scenario 1 Alcohol 46 - Scenario 2 Borderline 31 - Scenario 3 BPAD 26 - Scenario 4 Drugs 23 - Scenario 5 Eating Disorder 18 - Scenario 6 Memory 21 - Scenario 7 OCD 18 - Scenario 8 PTS

18. Learning Analytics and Data Visualisation Interpreting raw data and using ‘R’ (Dr Dan Joyce) Demonstration

19. Learning Analytics and Data Visualisation Case Study 8 Example

20. Ongoing work Finalise data visualisations Lay optimal paths over actual results Improving feedback Evaluate skills and time spent on project Publish research paper

21. Lessons Learned Importance of instructional design and planning Subject specialist input Appropriate budget and resourcing Awareness of technological limitations Collaboration is key Think big, start small

22. Thank You

23. Working with Legacy Content Hosted in various places and bespoke build. http://mentalhealthstudies.iop.kcl.ac.uk/MHS/i ndex_extra.html http://mentalhealthstudies.iop.kcl.ac.uk/MHS/i ndex_extra.html Please feel free to use (nothing is logged in this version): http://mentalhealthstudies.iop.kcl.ac.uk/Alcoh ol/scenario_1.html http://mentalhealthstudies.iop.kcl.ac.uk/Alcoh ol/scenario_1.html

24. Legacy Content