1. http://learning-maps.ncl.ac.uk Simon Cotterill Dynamic Learning Maps – fusing curriculum maps with personal learning

2. Introduction Dynamic Learning Maps – JISC Curriculum Delivery Programme 2 year project Newcastle University Medicine Psychology Speech Therapy Embedding Benefits – DLM Project 12 month project Partners: Bangor University Bradford University University of Cumbria Aims: Evaluate in other contexts Enhance wider uptake Improve training materials Enhance documentation Implementation lessons Wider dissemination XCRI project – incl. embed course information feeds in DLM Innovations project – Geography DLM Dentistry OER Rapid Innovation projects - with DLM integration

3. Workshop Overview http://learning-maps.ncl.ac.uk Structure Presentation:What are curriculum maps?15 min Group work: What do you want from curriculum maps ?25 min Presentation: Dynamic Learning Maps / Demos50 min Tea/Coffee Group work: Your views – feedback15 min Presentation: Mapping with DLM and future directions 30 min

4. Intended Outcomes Develop a common understanding of curriculum maps in the School Understand the role of curriculum maps for different stakeholders Be able to identify key challenges for developing curriculum maps Understand Dynamic Learning Maps and how these could be applied to your curricula Become familiar with ways in which technology can enhance curriculum maps Understand conceptual and practical approaches for constructing online curriculum maps Understand key challenges and opportunities in using technology to enhance curric. maps http://learning-maps.ncl.ac.uk

5. What are Curriculum Maps? Who are they for? What do they do? http://learning-maps.ncl.ac.uk

6. What are Curriculum Maps? “a diagrammatic representation of the curriculum” Different ‘windows’ onto the curriculum e.g. Intended outcomes Curriculum content /subject areas Learning opportunities Assessment Learning resources People (students / staff) “a diagrammatic representation of the curriculum” Different ‘windows’ onto the curriculum e.g. Intended outcomes Curriculum content /subject areas Learning opportunities Assessment Learning resources People (students / staff)

7. Curriculum maps elsewhere Design – questionnaire-based survey Canadian & UK Medical Schools Results – 31 responses – 20% completed maps on-going upgrading – majority in progress Outcomes – significant variation software used in construction elements included – challenges complexity human resource demands medical ontologies, faculty development interface design Willett TG, 2008. Current status of curriculum mapping in Canada and the UK. Medical Education 42: 786-93

8. Maps: levels of detail Curriculum strands Module maps Session maps

9. Maps as a Metaphor Where have I been? Where am I now? Where am I going? Stakeholders Learners Teachers (incl. occasional teachers) Curriculum Managers External Regulators Administrators Reflection Revision Contextualisation Preparation What should the students already know? Where is topic X taught in the curriculum ? Career choices Curriculum choices Where is my specialty covered in the curriculum ? http://learning-maps.ncl.ac.uk Synthesis / Metacognition Planning Does the curriculum cover all the professional standards?

10. Curriculum Maps: Potential role in Monitoring & QA Declared curriculum Taught curriculum Learning & development Assessed Curriculum ‘Constructive Alignment’ (curriculum – T&L – assessment) Better insight into learning outside the curriculum External resourcesPrior learning‘Life-wide’ learning Identify popular external resources (QA + peer review) Map to other Curricula (widens learning opportunities) Identify ‘gaps’ in teaching Identify duplication Monitor access & equality of learning opportunities

11. What should be on a curriculum map? Example of different interpretations / implementations Delgaty L.. Curriculum Mapping: are you thinking what I’m Thinking? A VISUAL COMPARISON OF STANDARDIZED, PRESCRIPTIVE PROGRAMMES. ARECLS, 2009, Vol.6, 35-58. 5 clinical teachers mapped the ALS (Advanced Life Support) course. Short duration, highly specified courses - Yet great diversity in mapping

12. Group Work 1 Who are the main stakeholders and what are their needs in relation to curriculum maps ? What elements should be included in curriculum maps? How are/could curriculum maps be useful in your programme? How detailed should curriculum maps be? http://learning-maps.ncl.ac.uk Common themesField specific Bilingual Support

13. Key curriculum drivers for the project: 1: Modular courses - promoting cross-modular learning - mapping to transferable graduate skills / careers 2: Medicine: Communicating Complex Curricula Dynamic Learning Maps Understanding linkages within the spiral curriculum (students & teachers) Occasional Teachers -some unintended duplication in teaching Part of the Curriculum Delivery Programme

14. Overview of Dynamic Learning maps Interactive ‘Web 2.0 Sharing, rating and reviews Harvesting multiple sources (‘Mashups’ ) Facilitating communities of interest Curriculum Maps Overview, Prior learning, Current & Future learning Personal Learning Personalised, sharing, reflective notes and evidencing outcomes Linking Learning Resources Curriculum & External Resources

15. Aims to be an interactive & participative environment changing experience & expectations of learners students and staff can add and connect topics reflection / portfolio learning directly embedded in curriculum add and rate external learning resources Personal and Social Learning Share, rate, discuss Extend maps & connect topics Integrates with portfolio (Leap2A) Clear differentiation between ‘core curriculum’ and community +ability to filter

16. Choice of views Text-based interface Mind-map style interface n=193 (student response system) I Prefer: Diversity and strong polarisation of personal preferences for visual, text or other views Further evaluation results: https://learning-maps.ncl.ac.uk/

17. ‘List view’ (focus groups saw relevance for revision) Choice of other views Tag Cloud

18. A demonstration https://learning-maps.ncl.ac.uk

19. e- Learning Technical approach: breaking boundaries Curricula databases Library databases ePortfolio / blog Repositories External Feeds Learning Resources Life-long Learning Record ID-MAPs project Student Information Systems reflection evidencing discussion adding resources, rating & reviewing Learning Maps (topic-specific) Curriculum map Personal learning Community http://learning-maps.ncl.ac.uk

20. Network structure Topic Types (extensible) e.g. – Learning Outcomes – Modules – Session – Assessment – Skill – Specialty Connections between topics (curriculum, community or personal) – Parent / child – General connection – Pre-requisite / Post-requisite – Co-requisite – Equivalence Meta data (optional) Maps provide a mix formal and common ‘folksonony’ – Formal taxonomies e.g. MeSH – Module / session codes These provide ‘hooks’ into external data sources. E.g. Library reading lists +automated updating with data sources e.g. timetable & module catalogue Adding a new topic: May include curriculum structures, syllabus, careers, skills… Scope and level of detail vary according to programme requirements Simplified ‘neural network’ of connected nodes (aka ‘topics’)

21. utilises ‘semantic’ connections made between topics and with resources Search over the curriculum

22. Plant your trees, watch the forest grow! Start simple - hierarchical taxonomies Add formal ‘meta data’ -draw in relevant content from your existing data sources Taxonomy Ontology Experts / Community - extend the map - make linkages between the topics Resources / Tagging -links informal terms to taxonomy -learns the strength of relationships between topics

24. The map would benefit my learning n=193 (student response system) 80% of students agreed “Excellent way of linking learning and thought processes”. “Hopefully minimises learning occurring in isolated chunks.” “Potential for more joined-up thinking for patient care”  “If assessment is not linked to it how much will it be used?” Clinical teachers (focus group)

25. Knowing how a teaching session relates to the rest of the curriculum is important to me n=193 (student response system) “Will increase relevance of Phase I lectures to clinical presentations/experience” From a students point of view, one could be much clearer on ‘the big picture’, as you have a curriculum map laid out in front of you...” 77% Agreed Stage 4 Medical Students - Focus groups

26. Having the map will be useful for revision n=193 (student response system) ? “Any way of ticking off lectures revised?” “Excellent revision tool” “Helpful revision tool” Stage 4 Medical Students Clinical Teacher 91% Agreed

27. It would be useful to add notes and reflections to teaching sessions and other parts of the map “[liked] linking the portfolio (which may appear otherwise abstract) with the rest of a student’s education”. “link to portfolio may increase its usage!” Stage 4 Medical Students (Focus Group) 73% of students agree But clinical teaching staff skeptical: n=193 (student response system) “Can this really be used for portfolio (if voluntary) ?“ “How to engage student with e-portfolios – won’t work without their engagement”

28. I frequently supplement my learning with external resources on the web n=193 (student response system) I only want information and resources provided by teaching staff n=193 (student response system) 72% agreed 26% agreed

29. Using Dynamic Learning Maps “Learning maps are an excellent tool for reminding students where they’ve studied curricula content. I found the mapping function especially useful …. learning maps allowed me to quickly and easily search for content and collate links to previous sessions into one ‘toolkit’ of relevant SSC resources for our students.” (Senior Lecturer, Anatomy and Clinical Skills) Clarity of purpose: Staff Students Think of the bigger picture not just my module topics should indicate context e.g. in paediatrics: ‘perscribing for children’ rather than just ‘perscribing’

30. Getting the right balance AutomationSpecificity & Granularity Initially reliant on manually making connections (curriculum & community) Maintenance cost as curriculum changes Search – High volume of results mixed relevance Saturation (too many Connections – ‘hairball’) e.g. MBBS: 60+ learning outcomes per module. High-level outcomes present in virtually every module. Refine relevance scoring Data on connected topics used to improve future automation/specificity (related keywords / strength of connections)

31. Challenges for Curriculum Mapping ♯ 1 Stage 5  Stage 4  Stage 3  Stage 2  Stage 1  2010/112011/122012/132013/142014/15 Student journey through the curriculum ‘here and now’ teaching focus Between major restructuring of the MBBS curriculum (aprox. every 5-7yrs) :  stable: units (modules), programme outcomes  minor adjustments: sessions, cases, unit outcomes (responsive to evaluation / QA)  more variation in assessment & differences in delivery by 4 ‘Base Units’ (stages 3 & 5) The curriculum changes over time

32. Challenges for Curriculum Mapping ♯ 2 Stepped availability of study guides, cases and timetable data Semester 2 Available  Semester 1 Available  Sept 2010Jan 2011Sept 2011Jan 2012 i.e. a fully detailed / data-driven curriculum map for the current academic year would not be available until Semester 2. A partial map would be no good at all! (Needs to be a semi-persistent map but drawing on latest information as it becomes available). Resources (presentations etc) are uploaded into the VLE on a ‘just in time’ basis.

33. Challenges for Curriculum Mapping ♯ 3 MBBS study guide databases (baseline)  designed to support complex curricula with large number of contributors  using familiar Word documents (well formatted ‘portal documents’)  these populate databases and structure the VLE when they are uploaded  supports changing curriculum with views by multiple academic years Module Database (baseline) outcomes are blocks of free text: variable formats and amount of detail Fit for purpose, but raise challenges for online curriculum maps: Key data is in the form of non-standardised text  e.g. learning outcomes, core presentations / conditions etc.  language is inconsistent between study guides (modified for context)  hard to differentiate between unit-specific and programme outcomes & content Codes used in VLE and timetable are not persistent  e.g. ‘PPD2.15’ may referrer to completely different teaching sessions from one academic year to the next  Problematic as resources are linked to these non-persistent codes Existing data may not be in a readily usable

34. Recap: Summary of DLM Interactive ‘Web 2.0 Sharing, rating and reviews Harvesting multiple sources (‘Mashups’ ) Facilitating communities of interest Curriculum Maps Overview, Prior learning, Current & Future learning Personal Learning Personalised, sharing, reflective notes and evidencing outcomes Linking Learning Resources Curriculum & External Resources

35. Next Steps Setting up a DLM site for you Mapping the high level curriculum – Pathways – Modules ‘Hands-on’ training session? Mapping the modules Evaluation / refinements: feeding back into the DLM roadmap Longer-term plan re: integration with University data

36. What we can offer Hosting Setting up a DLM site for you Option to host locally later Consultancy Advice / help with mapping your curriculum How to implement / integrate with your curriculum (paedegogy) How to integrate with your other systems (technology) Training / Orientation Tailored training - theoretical & ‘Hands-on’ Collaboration Evaluation / refinements: feeding back into the DLM roadmap Building/updating shared taxonomies (e.g. MeSH, NHS Evidence/BNF) Longer-term plan re: integration with University data

37. Curriculum Managers Analytical Tool (under development) Cross-reference with: Display as: We need a generic tool to answer questions such as: -Are all of the GMC/GDC outcomes covered in the curriculum? -What forms of assessment are used for module X outcomes? -How do session outcomes for provider-1 map to outcomes for provider-2? For the current topic (and all topics below it in the hierarchy) cross-ref with a TYPE: GMC Outcome 1 -Session Outcome 2 -Session Outcome 3 GMC Outcome 2 GMC Outcome 3 -Session Outcome 1 List view (ordered): > GMC Outcomes 12345 00125 10242 24610 00001 STAGE-  GMC Outcome 1 GMC Outcome 2 GMC Outcome 3 GMC Outcome 4 Heat Map by Stage: (Based on Meta data for Stage + Details on mouse-over number)

38. Project funded by Further information & Public Demonstrator: http://learning-maps.ncl.ac.uk Thank You