Keele University TBL Group Team Based Learning in Chemistry: Inspiration, Adaptation and Integration Keele University TBL Group Natalie Brown , Laura Hancock, Chloe Howe, Graeme Jones, Tess Phillips & Daniela Plana https://www.keele.ac.uk/tbl/

Summer 2014

First Attempts 14/15 Jones, G. R. and Hancock L. M. (2015) Team-Based Learning: A Scratch Approach to Large Group Problem Classes. Journal of Academic Development and Education, 4, 98-103.

Science Foundation Year A one year transition course onto degree programmes Two 20 credit Chemistry modules Chemistry modules a gateway to 6 degree courses including Chemistry, Pharmacy and Medicine Wide variety of backgrounds Range of abilities Range of learning difficulties

Module Structure 14/15 Three 1 hour lectures per week One 1.5 hour problem class One 2 hour laboratory session Assessment - Labs (30%) Class Test (20%) Exam (50%) Class Test has formative feedback but marks are recorded. Cohort of 91 students

Problem Solving Pathway 14/15 Lectures Problem classes Class Test Individual problem solving under exam conditions Formative feedback written on answers Basic knowledge Exemplar instructor problem solving Test basic knowledge Individual & group formative problem solving Pre-class Preparation About 50% of a Chemistry exam is problem solving Exam Individual summative problem solving

By the end of 1.5 hours everyone wants the problem sessions to stop! Problem classes 14/15 Attendance poor and difficult to monitor Students unprepared Large ability range Students inhibited about asking questions Not enough instructors Poor student and staff feedback 50 students, 4 staff By the end of 1.5 hours everyone wants the problem sessions to stop!

Student feedback 14/15 All answers were negative Not enough staff to help sometimes.  A few more people to help and give 1:1 advice.  Staff are not always at hand to help students who are struggling, always occupied with others.  Waiting a long time to talk to teachers about my mistakes, I prefer one on one sessions.  Sometimes can run out of time. Not enough time to go round all students.  Not enough time to get full marked answers.  Longer problems class needed or chemistry tutorial as an extra lesson (2).  Needs more hands on, maybe an extra workshop to go through answers.  Need more lecturers or postgrads to help.   They should go through methods as my questions didn’t get marked.  Small ratio of staff to students could remain in questions remaining unanswered or having to wait a long time to go through answers with staff. Not always enough staff to check answers. All answers were negative

Changes for 15/16 Wanted to see if TBL could improve performance Attempted to keep all variables the same. Hadn’t factored in cohort increase from 91 to 154 students with no additional staffing. TBL sessions of 50 students, 2 staff.

Problem Solving Pathway 15/16 Lectures Readiness Assurance Process Application Activities Basic knowledge & exemplar problem solving Test basic knowledge with iFA-AT and instructor feedback Formative team problem solving Pre-class Preparation Exam Class Test Individual problem solving under exam conditions Formative feedback written on answers Summative individual problem solving

TBL Adaptation Chemistry students have to be able to draw chemical structures and sketch diagrams. Application activity problems based on exam style questions but broken down into sections. Groups write and draw answers on A4 paper. Groups are selected randomly to reveal and present their answer using a visualiser. Other groups then challenge and question the presenting group and the instructor facilitates discussion. We have compromised on the simultaneous reveal

Team v individual performance in the RAT Individual average score Highest individual score Team score

Feedback

Student Feedback 15/16 Positive comments Negative comments Particularly effective and innovative aspect of module (x17). They have been the most helpful part for me. TBL was my favourite part of the course. I got very close to my group and the majority of us got very involved. The problem classes allowed me to see what will be required from me in tests It was fun to go to TBL workshops. Problem classes forced me to study regularly which increased my understanding. TBL classes have benefitted me greatly. Negative comments People that are struggling in the team don't tend to speak up. Not all contribute to the activity. Sometimes long & tedious due to different abilities. My TBL team had all done chemistry before. I'm the thick one. Not a nice feeling. They rush through and I'm constantly asking for clarification.

Results 14/15, 15/16 Science Students Health Students Year (student no.) Class Test (20%) Exam (50%) 2014/15 (60) 58 48 2015/16 (125) 59 55 Health Students Year (student no.) Class Test (20%) Exam (50%) 2014/15 (31) 73 56 2015/16 (29) 62 61

Changes for 16/17 Improved the introduction to TBL to explain how it works and the benefits of it. Introduction of purple pens feedback session after class test to improve interaction with formative feedback. New member of staff new to TBL joined team to cover maternity leave.

Problem solving pathway 16/17 Lectures Readiness Assurance Process Application Activities Basic knowledge & exemplar problem solving Test basic knowledge with iFA-AT and instructor feedback Formative team problem solving Pre-class Preparation Exam Purple Pens Class Test Summative individual problem solving Formative class and individual feedback on problem solving Individual problem solving under exam conditions

What is Purple Pens Students sit class test under exam conditions. Instructor reviews scripts and decides to find out where students have gone wrong. Instructor presents answers to class test, explains where marks were allocated. Students write down their own feedback and mark their answers directly on their scripts using purple pens. Scripts returned to instructor who reviews marking and feedback and records adjusted marks.

Purple Pens McGarvey, D. J. and Hancock, L. M. (2017) PURPLE PENS: Enhancing Assessment Literacy and Student Engagement with Feedback https://rscposter.tumblr.com/tagged/RSCEdu [accessed 05/04/17]

Feedback for 16/17 Scale 1 - poor 2 3 4 5 - excellent

Feedback for 16/17 Positive comments (70 in total 37 TBL, 6 purple pens) TBL sessions as they allowed problems to be explained and allowed me to help others, which helped my understanding. I enjoyed the group sessions as we're able to accumulate knowledge and learn from each other - sometimes a student is better able to explain something than a lecturer. The TBL sessions, as they allow for any problems to be worked through and explained in a way by team mates that is sometimes easier to understand. The purple pens feedback - gives us an opportunity to see where we went wrong before the final marks are given. Negative comments (51 in total 4 TBL, 1 purple pens) TBL sessions- generally quite stigmatising for people with firstly no previous chemistry knowledge and secondly a good way to undermine confidence, as it was essentially confidence destroying to watch your team get everything right and claim it was easy whilst you were either confused or had to work like hell to grasp the same concepts.

Results 14/15, 15/16, 16/17 Science Students Year (student no.) Class Test (20%) Exam (50%) 2014/15 (60) 58 48 2015/16 (125) 59 55 2016/17 (85) 61 Health Students Year (student no.) Class Test (20%) Exam (50%) 2014/15 (31) 73 56 2015/16 (29) 62 61 2016/17 (35) 63 67

Conclusions TBL and Purple Pens feedback has had a beneficial impact on Chemistry FY teaching. TBL and Purple Pens have improved exam performance. TBL has not improved Class Test performance. Application activities have been adapted using visualisers. TBL has reduced staffing levels of problem classes. TBL can be quickly adopted by new staff. Hancock, L. M., Howe C., Plana, D., Jones, G.R. and Phillips, T. R. (2017) Scratching the Surface of Team-Based Learning. Education in Chemistry, 5, 22-25.

Acknowledgements Mark Paetkau (Thompson Rivers) TBL Consortium Keele Teaching Innovation Fund Keele chemistry Foundation Year students Keele TBL Group Dave McGarvey