Group demonstrations and reports in engineering John Mitchell UCL Electronic and Electrical Engineering With thanks to: Sally Day, Rosalind Duhs, Lewis Elton, Tony Kenyon, Oleg Mitrofanov, Cyril Renaud, Miguel Rio, Ed Romans, Seb Savory, Jan Smith, Benn Thomsen. Innovations in assessment: the student learning experience

Developing scenario-based learning for formative and summative assessment It is not in fact teaching but assessment that plays the largest part in guiding students learning. 1 it can be argued that giving learners feedback is just about the most important dimension of the work of teachers in post-compulsory education … but perhaps all told, formative feedback is the vital dimension, as given at the right time and in the best possible way it can lead learners steadily towards successful achievement in summative assessment contexts. 2 1 Snyder B.R, (1971) The Hidden Curriculum. New York; Alfred A Knopf 2 Race P. (2005) Making Learning Happen, Chapter 5, Learning through Feedback, London, Sage

Rational for assessment developments Students didnt perceive that they were receiving feedback Highlighted in the national student survey Feedback on my work has been prompt. Agree – 41% I have received detailed comments on my work. Agree – 49% Feedback on my work has helped me clarify things I did not understand. Agree – 53% The NSS has highlighted that students are notably less positive about assessment and feedback on their assignments than about other aspects of their learning experience. Exploring the National Student Survey, The HEA NSS - Engineering and Technology, UCL perceive

Scenarios Building on the PBL in E&EE Following Civil Engineering model One week, self contained, group work scenarios Supporting learning not teaching new material 3 first year scenarios and 2 second year scenarios Currently in third year STUDENT–CENTRED LEARNING: WHAT DOES IT MEAN FOR STUDENTS AND LECTURERS? Geraldine ONeill and Tim McMahonIn: Emerging Issues in the Practice of University Learning and Teaching

Problem Based Learning Iterative Assessment Cycle –Report submitted at end of problem for summative assessment –Opportunity to use feedback to improve report –At end portfolio of reports submitted for formative assessment Narrative Commentary introduced Peer Assessment

Opportunities for Feedback in Scenario Assessment Formative Summative

Scenario Assessment A mix of instruments used –Milestones –Presentations –Competitions –Individual technical reports Traditional reports Critical Assessments –Group technical output User manual Due diligence document Presentations –Individual Narratives SUMMATIVE FORMATIVE

Milestones

Encouraging Brainstorming

Competitions

Presentations Presentations mid-way as guides rather than later as assessment Student dedicate significant time to presentations Too much focus on information not enough on message Opportunity to learn from other groups and provide mid-point feedback

Reflective Commentary advice given Consider how they tackled the problem, both as a team and individually, and how they would approach such problems in the future Merely documenting the steps you have gone through in the problem solving process will not earn you a very high mark – you will need to demonstrate what you have learned from the process. But often ignored!

Narratives Oh yeah, the narrative [wry chuckle]. The narrative, I didnt like the way the narrative was 50% of the marks because we spent a lot of time on those reports, a lot of time doing research into suitable system diagrams and everything else and I didnt like the way half the marks just went to the narrative…

Reports Attempted to get away from usual reports –User Manuals/Documentation –Very specific guidance given

Marking Grid Marking Proforma1 st Grade /52 nd Grade /5 Problem Definition This could have been improved, You need a lot more details 2 Design description It is not clear why some choices have been made. And a bullet point time line does not give any idea on some of the design. Your design is based on an error in your data-collection. 2 Theoretical Analysis The theory is adequate but you have not used it to its full potential. 3 Results and Conclusion There is almost no analysis in the conclusion, and you do not compare your results with the theory. 1 Individual input You claim to have done your part of the work over 1 day. You should have spend more time trying to understand the rest of the issues. 2 Reflective commentary More comments on what you have learned are needed 3 Total Mark20

Written Feedback

Student Debrief and Feedback Session Not yet fully utilised General comments to the whole group Citing good and bad practice

Assessment Weighting Course marking scheme (1-5 range) Currently a low percentage of final grade All aspects rewarded but none too heavily Group vs Individual? Peer Assessment?

Conclusions Student always appreciate feedback Even if they then go on to ignore it! Use assessment to guide learning rather than just to judge learning Linking feedback to further assessment seems vital Scenario Brief MilestonesFeedbackAssessment Summative Feedback