1. Blended Learning Workshop STRATEGIES FOR ASSURING THE QUALITY OF A BLENDED COURSE Richard Walker E-Learning Development Team University of York

2. 2 1.What steps can we take to ensure that a blended course really meets its objectives in supporting student learning? 2.How can we support a continuous process of improvement in the way that we design and support student learning activities. Underpinning these questions: Key questions for this workshop 3.How can we engage course instructors in QA & continuous improvement processes?

3. 3 1. ‘Designing in’ good practice -QA & pre-testing frameworks 2. Developing an evaluation plan -Principles & practical considerations 3. Course delivery & the development cycle -Building a continuous process of improvement Workshop Outline

4. 4  Pedagogic aims for online delivery  Design models: what’s possible?  E-tools: best fit for pedagogic objectives  Develop site: reflecting guidelines & standards  Test: peer review & student testing  Induction  Supporting / sustaining student activity  Evaluating student learning experience  Lessons learned, informing course design, task design & instructional responsibilities. Virtuous development cycle

5. 5 Design phase: clear objectives for student learning  Learning outcomes informing activity design; technology; structure of the blend Development phase: embedding QA principles  Implement in development of learning space  Informing design and presentation of learning resources, tasks & activities  Supported through training & quality frameworks Pre-testing phase: review of learning space  Fitness for purpose ‘Designing in’ good practice

6. 6 8 general standards: 1. Course overview & introduction 2. Learning objectives 3. Assessment & measurement 4. Resources & materials 5. Learner engagement 6. Course technology 7. Learner support 8. Accessibility Quality Matters Rubric http://www.qmprogram.org/rubric

7. 7 A.Course Overview & Introduction  Statement of purpose; objectives; orientation B.Course Design  Course structure, usability, guidance & support C.Presentation of Resources  Layout, format, instructions D.Site Interaction  Communication channels, standards for participation Blended Module Checklist Online interactive version of the checklist

8. 8 Self-directed testing Peer review Pre-testing your site

9. 9 Health check template

10. 10 Self-directed testing Peer review Student testing Pre-testing your site

11. 11 How can you be sure that the learning objectives for the blended course are realised? How will you track student participation and engagement? How will you evaluate the overall effectiveness of the course design & delivery processes? Reflection point

12. 12  Plan before course starts  Embed in overall design of course (reflecting learning objectives)  Inform students about evaluation (if participation required) Your plan should consider: i.Aims & focus of evaluation ii.Key questions iii.Stakeholders iv.Time scales & dependencies v.Instruments & methods Developing your evaluation plan Adapted from Jara et al. (2008) Evaluation of E-Learning Courses

13. 13 Principles for course evaluation Outcome-based: focusing on measurable & objective standards –Were the course objectives met (e.g. levels of engagement & patterns of use of online resources)? –Did learners reach the targeted learning outcomes (e.g. approaches to learning; levels of understanding)?  Outcome-based: focusing on measurable & objective standards –Were the course objectives met (e.g. levels of engagement & patterns of use of online resources)? –Did learners reach the targeted learning outcomes (e.g. approaches to learning; levels of understanding)?  Interpretive: focusing on context (perceptions of the learning experience) – What were the students’ affective and attitudinal responses to the blended course experience? – How were the e-learning tools used by students to support their learning in formal & informal study activities? – How did the lecturer/tutors perceive students’ learning relative to previous performance? (What actions should be taken for future course development?)

14. 14 Data collection methods (Informal progress checks) Entry & exit surveys Contribution statistics Focus group interviews Tools for reflection Course statistics

15. 15 Evaluation Pathway Class Sessions Class Sessions Class Sessions Class Sessions Feedback on performance Online Activity Feedback on performance Online Activity Feedback on performance Online Activity RoleStartCourse DeliveryEndPost Course InstructorEntry Survey Feedback on performanceExit Survey StudentsTask performance and self reflection SystemCourse statistics & contribution histories ResearcherContent analysisFocus Group

16. 16 Evaluation Pathway Class Sessions Class Sessions Class Sessions Class Sessions Feedback on performance Online Activity Feedback on performance Online Activity Feedback on performance Online Activity RoleStartCourse DeliveryEndPost Course InstructorEntry Survey Feedback on performanceExit Survey StudentsTask performance and self reflection SystemCourse statistics & contribution histories ResearcherContent analysisFocus Group

17. 17 Evolutionary Ecology

18. 18 CHD Case Study

19. 19 Wiki participation proxy indicator

20. 20 Output and work patterns : Evolutionary Ecology ModuleOutput & participationComments Evolutionary Ecology 8 x 8 reports > 50% of modifications by 1 student in 3 groups Allocation of report writing to individuals Collaborative research Mix of communication methods (Facebook/f2f/blog)

21. 21 Evolutionary Ecology Case Study

22. 22 Characteristic of cognitive skill Example from blog posts Offering resourcesThis case relates to cases of master and servant, these principles apply equally to directors serving the company under express or implied contracts of service, and who are therefore also employees (Dranez Anstalt v. Zamir Hayek,) Making declarative statements I cannot understand the reason, you mentioned, that the UCTA may not apply to this case. LC is not of course a consumer, but M is a relevant consumer. Supporting positions on issues Once Ackerman heard from the inside information from his father in law, he would be as insider under s. 118B (e) of FSMA because he has information “which he has obtained by other means which he could be reasonable expected to know is inside information”. Therefore his action to sell his share of SAH would be dealt with as insider dealing. Adding examplesThe offence of insider dealing can be committed in 3 ways. If an insider: deals in price-affected securities, when in possession of inside information, s.52(1) CJA 1993 encourages another to deal in price-affected securities, when in possession of inside information, s.52(2)(a) CJA 1993, or discloses inside information other than in the proper performance of his employment or profession, s.52(2)(b) CJA 1993. Categories of cognitive skills Framework based on Fox and MacKeogh’s 16 categories of cognitive thinking: Fox, S. and MacKeogh, K. (2003) 'Can eLearning Promote Higher-order Learning Without Tutor Overload?', Open Learning: The Journal of Open and Distance Learning, 18: 2, 121 — 134 and examples from the weekly blogs

23. 23 Student engagement  Survey fatigue Reliability: halo/horns effect Validity  Visibility of student learning  Context of student learning Challenges in interpreting your data

24. 24 Was the course design fit for purpose?  Usefulness / engagement patterns for online components of module  Complementary nature of class-based & online activities  Relevance of assessment plan  Sequencing of tasks Were the course materials suited for the online tasks?  Levels of learning / differentiation & accessibility Was instructional support adequate, enabling & timely?  Instructions, feedback and support Reflection on action : Defining next steps

25. 25 Design : Summary Course delivery as a development cycle Deliver : Evaluate : Review : Pedagogic aims; design model; course testing; delivery & evaluation plans Socialise; support; sustain; sum up student learning. Evidence collection as a feature of course delivery Establish holistic view of student learning – employing outcome focused & interpretive research methods Reflection on action – defining next steps

26. 26 References and recommended reading Fox, S. and MacKeogh, K. (2003) 'Can eLearning Promote Higher-order Learning Without Tutor Overload?' Open Learning: The Journal of Open and Distance Learning, 18: 2, 121 — 134 Gunawardena, C., Lowe, C. & Carabajal, K. (2000). Evaluating Online Learning: models and methods. In D. Willis et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2000 (pp. 1677-1684). Chesapeake, VA: AACE. Jara, M., Mohamad, F., & Cranmer, S. (2008). Evaluation of E-Learning Courses. WLE Centre Occasional Paper 4. Institute of Education, University of London. http://www.wlecentre.ac.uk/cms/files/occasionalpapers/evaluation_of_online_cou rses_25th.pdf Quality Matters Program Rubric http://www.qmprogram.org/rubrichttp://www.qmprogram.org/rubric

27. Thank You Richard Walker The University of York richard.walker@york.ac.uk Questions & comments