Diana Laurillard and Dejan Ljubojevic London Knowledge Lab Institute of Education, London Evaluating learning designs through the formal representation.

Slides:



Advertisements
Similar presentations
IB Portfolio Tasks 20% of final grade
Advertisements

Our vision for a pedagogic planning tool Practical assistance for lecturers designing blended learning activities Both literature-based and adaptable in.
Completing the cycle: an investigation of structured reflection as a tool to encourage student engagement with feedback Jackie Pates Lancaster Environment.
Building online support for HE teachers as digital innovators Diana Laurillard London Knowledge Lab Institute of Education OER in the disciplines 26 October.
What The Research Says: Can Apps for Teachers Drive Learning Innovation? 21 March 2014 Teachers building community knowledge of learning innovation Diana.
Digital technology for enhanced and efficient learning and teaching
TEA Science Workshop #7 October 25, 2012 Kim Lott Utah State University.
Assessment matters: What guides might we use as individuals, teams and institutions to help our assessment endeavours? A presentation to Wolverhampton.
The use of a computerized automated feedback system Trevor Barker Dept. Computer Science.
ANY A HIGGINS Senior Lecturer Sport Studies USING FLIP CAMERA IN ASSESSMENT.
Phoebe A wiki-based pedagogic planner to promote innovative practice in Design for Learning Liz Masterman Oxford University Computing Services with Marion.
Outline: Can technology enrich quality, improve productivity
No Longer the Wave of the Future. No Longer the Wave of the Future How do you know if you’ve caught the wave?
Teaching literacy in TECHNOLOGY (MANDATORY) Stage 4 - Year 7
Interactive Science Notebooks: Putting the Next Generation Practices into Action
Jude Carroll, author of Tools for Teaching in an Educationally Mobile World (Routledge 2015) Supporting teaching across cultures: the role of good practice.
The draft NSW English K-10 syllabus Version 2 February, 2012.
The Learning Design Revolution: Implementing the Learning Activity Management System (LAMS) Professor James Dalziel Director, Macquarie University E-learning.
Enhancing Student Success: Teaching Well With Technology October 23, 2009.
Marion Webb January  By the end of the session, participants will be able to:  Discuss the role of assessment for learning  Describe key assessment.
Jan 2013 cc: by-nc-sa Learning design for online courses The Pedagogical Patterns Collector, the Learning Designer, and MOOCs Diana Laurillard.
Interstate New Teacher Assessment and Support Consortium (INTASC)
Feedback and Next Step Marking
1 Pedagogical implications of mobile technologies Diana Laurillard WLE Symposium on M-Learning 9 February 2007.
Learning design as a foundation for the future success of e-learning Diana Laurillard 2007 European LAMS Conference University of Greenwich 5 July 2007.
Pedagogic design for active learning Workshop Diana Laurillard, IOE.
Innovative Schools toolkit Strategic Workshop 3 - Exploring good practice case studies.
As a Teaching Strategy Ellen Plummer, RN Melissa Lewis, RN.
Guidelines for Designing Inquiry-Based Learning Environments on the Web: Professional Development of Educators Byung-Ro Lim IST, Indiana Univ. July 20,
Centre for Academic Practice University of Strathclyde
Aims of Workshop Introduce more effective school/University partnerships for the initial training of teachers through developing mentorship training Encourage.
Teaching in a Web-Based Distance Learning Environment: An Evaluation Summary Based on Four Courses Charles Graham, Joni M. Craner, Byung-ro Lim, & Kursat.
Presenting Evidence to meet the Graduating Teacher Standards at the conclusion of Edprac 608 The use of MyPortfolio.
Group work – why do it? Rachel Horn – Civil & Structural Engineering.
1 Introduction to eLearning Pedagogy Lecture-4: week 1- Semester-1/ 2009 Dr. Anwar Mousa University of Palestine Faculty of Information Technology.
It’s difficult, but it’s worth a try, because… Teachers need much more support than they get to make the most of learning technologies If they can learn.
Post-Graduate students’ perceptions of feedback Dr Helen Boulton Nottingham Trent University. Dr Alison Hramiak Sheffield Hallam University.
1 Ideas of Problem-based Learning As a learner-centred process, problem- based learning meets the learners' interests and as such gives room for developing.
Do you have your PaSsPorT? Rethinking teaching spaces for student-centred learning Wendy A. Crocker, PhD & Stephanie Oliver, PhD Teaching Support Centre.
1 Using Feedback as a Teaching Tool in the Online Classroom.
Formative and Summative Evaluation. Formative Evaluation The goal of formative assessment is to Monitor student learning Provide ongoing feedback Improve.
EToolbox: Mapping Technology to Learning Getting Started.
1 Techniques for Online Retention Dr. Andrea Henne, Dean, Online and Distributed Learning.
Session Objectives Analyze the key components and process of PBL Evaluate the potential benefits and limitations of using PBL Prepare a draft plan for.
1 Faculty Development Opportunities Weekly events – Institute on High Impact Pedagogical Practices – Faculty Development Workshop Series – The Pre-Tenure.
Teaching in teams: lessons from systematic review training NCRM Training the Trainers Event 4 th June 2007 Angela Harden and Karen Bird MRS Node EPPI Centre,
Programme design and student assessment David Baume 1.
Re-engineering Assessment Practices in Scottish Higher Education [REAP] Dr David Nicol, Project Director Centre for Academic Practice and Learning Enhancement.
Learning type: Acquisition Learning through acquisition is what learners are doing when they are listening to a lecture or podcast, reading from books.
Diana Laurillard London Knowledge Lab Institute of Education, London Introduction to designing MOOCs: Theory, practice and evidence What The Research Says:
Teaching with CHRONOS Data and Tools A Framework for Design Cathy Manduca Science Education Resource Center Carleton College June 13, 2006.
This resource has been released by the University of Bath as an Open Educational Resource. The materials are licensed under a Creative Commons Attribution-Noncommercial-ShareAlike.
Observation System Kidderminster College January 2012.
true potential An Introduction to the Middle Manager Programme’s CMI Qualifications.
Supporting the teacher as innovative learning designer March 2012 Diana Laurillard London Knowledge Lab Institute of Education.
Pedagogical aspects in assuring quality in virtual education environments University of Gothenburg, Sweden.
true potential An Introduction to the First Line Manager Programme’s CMI Qualifications.
UDL & DIFFERENTIATION (DI) Melody Murphy Week 4 Discussion.
Designs and testing for a ‘pedagogic planning tool’ A User-oriented Planner for Learning Analysis and Design - Overview Review meeting 23 January 2007.
Learning type: Acquisition
Learning type: Acquisition
Assessment and Feedback – Module 1
CTE 2010 Summer Institute on Teaching and Learning
Our Research Into Assessment
Mark Russell King’s College
Quality assurance and curriculum development
Learning design as a foundation for the future success of e-learning
Our Research Into Assessment
Learning design tools for institutional change
Providing feedback to learners
Presentation transcript:

Diana Laurillard and Dejan Ljubojevic London Knowledge Lab Institute of Education, London Evaluating learning designs through the formal representation of learning patterns

A Learning Design Support Environment (LDSE) IOE/LKL Brock Craft (Technical) Sarah Gelcich (Admin) Diana Laurillard (PI) Dejan Ljubojevic (RF) Javier Calzada-Prado (Intern) Oxford Liz Masterman (CoPI) Marion Manton (CoPI) Birkbeck/LKL George Magooulas (CoPI) Patricia Charlton LondonMet Tom Boyle (CoPI) LSE Steve Ryan (CoPI) Ed Whitley Roser Pujadas (Studentship) RVC Kim Whittlestone (CoPI) Stephen May Carrie Driver (Studentship)

OUTLINE Teachers are interested in learning design and the use of technology but lack the tools they need  LDSE Planning, advice, exemplars – could it also evaluate a design? Design principles in the literature Learning design patterns – to capture pedagogic principles? Representing learning theories as an evaluative framework Evaluating the pedagogy in a learning design Testing the evaluative capability of the framework Evaluating learning designs

Design principles in the literature Good feedback practice: 1. helps clarify what good performance is - goals, criteria, expected standards 2. facilitates the development of self-assessment (reflection) in learning 3. delivers high quality information to students about their learning 4. encourages teacher and peer dialogue around learning 5. encourages positive motivational beliefs and self-esteem 6. provides opportunities to close the gap between current and desired performance 7. provides information to teachers that can be used to help shape the teaching. (Nicol & MacFarlane-Dick, 2006)

Design principles in the literature 1. Encourage contacts between students and faculty 2. Develop reciprocity and cooperation among students 3. Use active learning techniques 4. Give prompt feedback 5. Emphasize time on task 6. Communicate high expectations 7. Respect diverse talents and ways of learning. (Chickering and Gamson, 1991) Where is the pedagogy that aids learning design? What active learning techniques? What kind of feedback? Can pedagogical patterns assist with designing and evaluating the quality of a learning design?

Learning design patterns The LDSE( learning design support environment) aims to evaluate the pedagogy in a learning design in terms of theory We look for pattern templates that will help to evaluate pedagogy in a design A learning design pattern as a learning activity sequence designed to lead to a specific learning outcome

Comparing pedagogy pattern templates iCOPERPlanetTELL Author & CopyrightCredits Summary/ ThumbnailContext Rationale Subject/ DisciplineContext Learning outcomesProblem Forces Group size Duration (part) Learner CharacteristicsAudience Type of Setting Graphical RepresentationDiagram Sequence of Activities Solution Roles Type of Assessment Where is the effective pedagogy captured?

We need categories within the ‘solution’ or ‘sequence of activities’ to help teacher: Develop reciprocity and cooperation among students Use active learning techniques Give prompt feedback Some accounts of learning activity sequences are quite elaborate Categories do not sufficiently discriminate the form of pedagogy that will make the critical difference to learners Comparing pedagogy pattern templates iCOPERPlanetTELL Rationale Learning outcomesProblem Sequence of Activities Solution Roles Type of Assessment Needs further detailed analysis to support learning design Pattern for understanding authentic practicePattern for helping whole class benefit from individual experiences

Representing learning patterns Structured text-based accounts, e.g. iCoper, Learning Designs Text cannot be easily interpreted by a program. Computationally defined activities can be (e.g. LAMS). Therefore can be evaluated against theory. Box-and-arrow diagrams for a sequence of activities, e.g. LAMS, iCoper, ISiS-ECTEL

Representing learning theories The Conversational Framework: developed to provide a representation of what it takes to learn in education based on the main design principles for learning and pedagogy drawn from the literature can test a learning design in terms of which aspects of the framework it covers.

The learner learning LC TC LP TP Thoughts Action plans Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Investigating Listening/ Reading Reflection Producing Revising Working to a goal Feedback LC LP Adaptation being supported,actingthinking,

Learning theories represented in the Conversational Framework LC TC LP TP Thoughts Action plans Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Investigating Listening/ Reading Reflection Producing Revising Working to a goal Feedback LC LP Adaptation 1. Encourage contacts between students and faculty 4. Give prompt feedback 2. Develop reciprocity and cooperation among students 3. Use active learning techniques

Learning theories represented in the Conversational Framework LC TC LP TP Thoughts Action plans Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Acting Listening/ Reading Reflection Producing Revising Working to a goal Information LC LP Adaptation Reflection Adaptation

Learning theories represented in the Conversational Framework LC TC LP TP Thoughts Action plans Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Acting Listening/ Reading Reflection Producing Revising Working to a goal Information LC LP Adaptation Reflection Adaptation

Learning theories represented in the Conversational Framework LC TC LP TP Thoughts Action plans Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Acting Listening/ Reading Reflection Producing Revising Working to a goal Feedback LC LP Adaptation Reflection Adaptation

Representing technologies in support of learning LC TC LP TP Thoughts Action plans OC OP Reflection LC LP Adaptation

Evaluating a learning design The Conversational Framework: can test a learning design in terms of which aspects of the framework it covers - IF we can link each LAMS activity (or similar computationally defined activity) in the sequence to part of the framework

LAMS – representing learning design as a temporal sequence of computationally defined activities Chat Chat and Scribe Forum Forum and Scribe MCQ Notebook Noticeboard Q&A Resources & Forum Share resources Submit Voting To understand the processes within a system

Chat and Scribe Mapping LAMS activities to theory via CF LC TC LP TP Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Investigating Listening/ Reading Producing Revising Working to a goal Feedback LC LP Forum and Scribe MCQ Notebook Q&A Share resources Submit Voting Chat and Scribe Notebook Noticeboar d Q&A Share resources Submit Forum and Scribe MCQ Voting

LAMS – representing learning design as a temporal sequence of computationally defined activities To understand the processes within a system

Interpreting the quality of the design LC TC LP TP Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Investigating Listening/ Reading Producing Revising Working to a goal Feedback LC LP Notebook Chat and Scribe Noticeboar d Q&A Share resources Submit Forum and Scribe MCQ Voting System can interpret design as scoring: 4 out of 10 activities; No iterations; Time for each task is good; Sequence is good for ‘awareness’; less good for ‘understanding’ Evaluating the quality of the design

Improving the quality of the design LC TC LP TP Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Investigating Listening/ Reading Producing Revising Working to a goal Feedback LC LP Chat and Scribe Noticeboar d Q&A Voting System can propose: Adding in designs or patterns that complete the other cycles; Checking that the sequence follows the motivating cycles of iteration

Testing the framework The Conversational Framework: can differentiate between sequences in terms of coverage of the framework, amount of iteration, and effective ordering of activities But to what extent can it recognise the relative effectiveness of learning designs identified in the literature?

Pattern 1 Contrasting cases text Analyse data S analyses data S produces graphs S listens to lecture S selects and predicts Very accurate MCQs High transfer to prediction Pattern 2 Theory and examples text Read text S reads text S produces summary S listens to lecture S selects and predicts Very accurate MCQs Much lower transfer to prediction Comparing learning design patterns Intended learning outcome: To predict accurately the outcomes for a hypothetical experiment on memory [Schwartz and Bransford,1998] Learning activities Reading T sets a task goal Investigating Producing output Listening Working to achieve goal Actual outcomes Pattern 1 Contrasting cases text Analyse data S analyses data S produces graphs S listens to lecture S selects and predicts Very accurate MCQs High transfer to prediction Pattern 2 Theory and examples text Read text S reads text S produces summary S listens to lecture S selects and predicts Very accurate MCQs Much lower transfer to prediction Critical differences between patterns are not just the nature of the activity, but the internal relations between contents of activities in the sequence

Interpreting the quality of the designs LC TC LP Text Data Guidance 3. Summarising 3. Investigating 1. Reading 4. Producing Revising 2. Working to a goal Feedback LC LP System must be able to differentiate between producing graphs and summaries; And between providing Text and Data in the practice environment 5. Listening 4. Producing Reflecting Adapting Asking Questions

Representing the range of good pedagogy LC TC LP TP Guidance OC OP Articulating ideas Others’ ideas Preparing Outputs Others’ Outputs Asking Questions Investigating Listening/ Reading Producing Revising Working to a goal Feedback LC LP Notebook Chat and Scribe Noticeboar d Q&A Share resources Submit Forum and Scribe MCQ Voting A System like LAMS can represent good pedagogy in terms of type and range of learning activity Cannot represent good pedagogy within the practice environment – simulation, model, digital library, data repositories, type of task and type of feedback

Summary Design principles in the literature Learning design patterns Representing learning theories as a framework for design Evaluating the pedagogy in a learning design Testing the evaluative capability of the framework Too general to be helpful for teacher-designers May miss categorising the critical pedagogy Makes it possible to relate theory to computational activities Needs mapping from framework to LAMS or similar system May not capture all the key differences in effectiveness

Pedagogy operates both across and within activities We need user-editable patterns for activity sequences – One role for learning technologists is to create the means for teachers to adopt and adapt pedagogic patterns for activity sequences – e.g. LAMS – One role for teachers is to these to adopt, adapt and share pedagogic patterns for activity sequences We need customisable programmed patterns for activities – Another role for the learning technologist is to discover and create the means for teachers to adopt and adapt pedagogic patterns for practice environments – e.g. GLO tool, NetLogo Evaluating learning designs through the formal representation of learning patterns Made publicly available by ALT under a Creative Commons Attribution-Non-Commercial 2.0 UK: England & Wales license