1. Supplementing lectures with additional online materials Matthew Juniper, CUED June 2007

2. Introduction Lecture

3. Introduction Lecture Work on examples papers

4. Introduction LectureSupervision Work on examples papers

5. Introduction LectureSupervision Work on examples papers Additional materials available on-line

6. Effectiveness study 2005-06

7. Effectiveness Study Work on examples papers Additional materials available on-line Textual concept descriptions Hint 1 (usually text) Hint 2 (usually text) Audio-visual concept descriptions

8. STUDENTS' PERCEPTION OF THE USEFULNESS OF THE ADDITIONAL MATERIALS Source :Survey data Incompressible Flow Overall, how useful were the additional materials for each question? Boundary Layers Not at all Not much A bit Fairly Very Question Number mean Student feedback

9. Source :Survey data Audio-visual A-V Textual Incompressible FlowBoundary Layers Not at all Not much A bit Fairly Very Textual mean STUDENTS' PERCEPTION OF THE USEFULNESS OF THE ADDITIONAL MATERIALS Question Number Overall, how useful were the additional materials for each question? Student feedback

10. Source :Survey data Audio-visual A-V Textual Incompressible FlowBoundary Layers Not at all Not much A bit Fairly Very Textual mean STUDENTS' PERCEPTION OF THE USEFULNESS OF THE ADDITIONAL MATERIALS Question Number Overall, how useful were the additional materials for each question? Student feedback new concepts introduced

11. STUDENTS' AND SUPERVISORS' OPINIONS ON THE EFFECT OF THE AMs Source :Survey data (37 student responses, 18 supervisor responses) Do the AMs hinder or aid students' ability to answer the question? Strongly aid students Strongly hinder Do the AMs hinder or aid discussion in the supervision? Do the AMs hinder or aid students' understanding of the concepts? supervisors students supervisors students supervisors Strongly aid Strongly hinder Strongly aid Strongly hinder No effect Student and supervisor feedback

12. COMMENTS ON THE EFFECT OF AMs ON STUDENTS' ABILITY TO ANSWER THE QUESTIONS Source :Survey data "Visualization of the problem in Q16 was slightly easier." (IF) StudentsSupervisors "The student that used the AMs was able to get through the material faster than the other student" (IF) "Particularly useful for Q12" (IF) "I looked at the AMs after completing the questions in order to consolidate." (IF) "The hints were generally at a good level – they helped but didn't do it all for you." (IF) "Helped to get started on the subject." (IF) "Mostly very well thought out, with the level of help given just right. I still got stuck a couple of times but that is what supervisions are for." (IF) "The AMs usually helped because students often seemed to get stuck at the first hurdle." (IF) Getting started: "I would not even know where to begin were it not for the hints" (BL) Directing thought processes: Getting further: "The AMs helped direct my thinking when the questions were written vaguely." (BL) "I would have got nowhere in Q4 without the AMs but in the end I still got a lot of it wrong. Half way answers would really help." (BL) How to use the AMs "I found it useful to look at the hints after I had got stuck on a question." (BL) Getting started: Directing thought processes: Getting further: "The students found the AMs useful, especially when they didn't know how to start the question." (BL) "The AMs for Q11 have been very useful as they clarify how students should work out this question." (BL) "One student submitted attempts with and without the AMs, getting much further through the questions by using them." (BL) "The students who used the AMs could get a lot further in the questions. Those who did not use them despaired and gave up, expecting to be told all in the supervision." (BL) "One of the students had used the AMs and was able to attempt all the questions, unlike the other student." (BL) "Both students specifically said that they would not have been able to answer the questions without the AMs" (BL) "It was especially useful for Q4" (BL) "The AMs are most useful for the latter half of the BL questions [these are the hardest questions]" (BL) Specific comments Student and supervisor feedback

13. Source :Survey data "The AMs gave me a better understanding so I was better prepared for the supervision and could go into more depth." (IF) "On one question they didn't need me to point out the challenging issue. However, since they had figured out the question themselves, I couldn't use it to reinforce the concept." (IF) "It is slightly more difficult to start discussion because students get there without the supervisor's help." (IF) "For students who put in less time the AMs can complete my explanations – very useful." (IF) "Very helpful for discussion in supervision and understanding principles but I still got stuck on the maths." (IF) "Less time was required for going over answers so more time was available for discussion." (IF) "In supervisions I conduct some spot checks to check that students understand. These are similar to the AMs so now I don't know whether they really understand or have simply seen it already on the AMs." (IF) COMMENTS ON THE EFFECT OF AMs ON DISCUSSION IN THE SUPERVISION Focus discussion: StudentsSupervisors "For Q4 the video was useful for targeting discussion towards at an area that I had not understood." (BL) "In the supervision the AMs provided a discussion point and we referred to them a lot when going through Q4" (BL) Focus discussion: "Having all looked at the materials we could generally take a more unified approach." (BL) "The students who were fuzzy on the Coriolis force had a better understanding of Q11 having used the AMs. This made discussion easier." (BL) Save time: "We were starting from a point further along than we otherwise would have done." (BL) "We spent less time grinding through the questions" (BL) "They didn't ask me anything related to what they saw on the additional materials." (BL) Problems: "Only one student had used the AMs, which resulted in an unbalanced supervision." (BL) Student and supervisor feedback

14. Source :Survey data "They seemed to understand the concepts even without using the materials." (IF) "The explanation of complex transformations was useful so that this concept could be fully clarified by the supervisor." (IF) "The AMs were generally good at explaining the concepts behind the questions but on some questions I still could not see how to apply the concepts, even after reference to the AMs." (IF) "The audio-visual concept description for Q9 was very helpful." (IF) "For Q12 there was a good visualization of the transformation from one plane to another." (IF) "The AMs helped because I could replay the audio-visual presentations if I didn't get the concepts right in lectures." (IF) StudentsSupervisors General: COMMENTS ON THE EFFECT OF AMs ON STUDENTS' CONCEPTUAL UNDERSTANDING "Audio-visual aids very helpful for understanding concepts" (BL) "The AMs provided extra insight into particular issues that may not have been covered in lectures." (BL) Specific: "Conceptual descriptions for Q4 and Q7 were particularly helpful [these are audio-visual]." (BL) Process: "The BL topic is one of the most difficult and the AMs act like a tutor for students while they are doing the problems." (BL) "The AMs aid with conceptual understanding because the students do not get as bogged down in trying to solve questions and so have time to ponder the concepts." (BL) "The concepts come from discussion in class. The AMs tend to focus on mathematical developments." (BL) "The excellent AV concept descriptions are a good remedy for the difficulty of the BL course." (BL) Student and supervisor feedback

15. Source :Survey data StudentsSupervisors Audio-visual AMs: GENERAL COMMENTS ON THE AMs Supervisors' use of AMs: "I found the audio-visual concept description the most helpful part of the AMs. It was good that I could replay the presentation over and over again." (IF) "Audio-visual descriptions very helpful" (BL) "Audio-visual hints very useful indeed" (BL) "I found the audio-visual presentation particularly helpful because I could replay it on demand." (IF) "Hints for Q7 confused me. My supervisor's explanation was much simpler" "I try to avoid using the AMs and treat them as a last resort. It was difficult to avoid being tempted to use them more than necessary. From what I have seen so far [first 11 questions of IF course] I would not invest in expanding the system." Comments on the adverse effects of AMs "The hints were only useful for one question. They could do the rest without any difficulty." (IF) "I use them to look for useful ways to teach the concepts" "I take a look to make sure that my explanations and answers are fully correct." "I used the AMs to obtain better understanding of question 4" (BL) "I always use the AMs for (a) a good route through the questions (b) good diagrams and ways to teach the theory and (c) extra understanding of any questions I found hard." (BL) Student and supervisor feedback

16. Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Conclusions and implications Implications - Reduce AV production time so that all additional materials can be presented audio- visually - Reduce reliance on other people to produce the additional materials

17. Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Conclusions and implications Implications - Reduce AV production time so that all additional materials can be presented audio- visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced- Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures

18. Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Conclusions and implications Implications - Reduce AV production time so that all additional materials can be presented audio- visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced- Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures - a question is particularly difficult- Create challenging questions and back them up with additional materials

19. Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Conclusions and implications Implications - Reduce AV production time so that all additional materials can be presented audio- visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced- Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures - a question is particularly difficult- Create challenging questions and back them up with additional materials Additional materials are not useful when: - questions are on familiar concepts - Don't use additional materials for simple questions

20. Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Conclusions and implications Implications - Reduce AV production time so that all additional materials can be presented audio- visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced- Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures - a question is particularly difficult- Create challenging questions and back them up with additional materials Additional materials are not useful when: - questions are on familiar concepts - Don't use additional materials for simple questions - Do worked examples that are similar, but not identical to, the questions