Assessing Teaching and Learning of Medical Statistics: Teachers and Learners Perspectives Dr. Salma Ayis and UMS Stat team King’s College London Division of Health and Social Care Research Department of Primary Care and Public Health Sciences Burwalls Meeting Ist July 2015 King’s College London

Outlines Teaching and learning (statistics) Challenges in medical schools Assessing the process as perceived by learners and teachers: reflection Taking actions to improve Assessing the change?

Why Do we Teach Medical Statistics? What are the intended learning outcomes? o Statistics is arguably an integral component of scientific methods, particularly biologic and health sciences. o Medical students are taught statistics to allow them understand the conceptual basis and usage of common statistical methods, and their applications in clinical medicine and public health. o These are important for the critical reading, understanding and assessment of research findings, hence for the progress of evidence based medicine (EBM).

Teaching and Learning o Teaching is as much about setting the context or climate for learning. o The process of teaching and learning is affected by a range of factors: Student specific, such as motivation External(extrinsic), such as the environment. Most widely acknowledged were: o Differences in cognitive and learning styles o Approaches to learning o Learning environments Teaching statistics in medical schools has some additional factors that are often challenging.

Challenges ?? Aim: to meet the intended learning outcomes from a statistics course to medical students o A subject that seem peripheral to the main thrust of the medicine course o Considered dry o Students tend to give a much lower approval to parallel courses than to the main teaching o Time constraints and the heavy programme o The proportion of time and resources allocated to teaching o The proportion of the subject in the final assessment, among other reasons

6 Alignment of teaching and intended learning outcomes o Clearly specified outcomes of teaching in terms of topic and content, in the level of understanding that students were expected to achieve. o Set up an environment that maximises the likelihood that students will engage in the activities designed to achieve the intended outcomes(ILOs); and o Choose an assessment tasks that will tell how well students have attained these [Biggs J. Constructive Alignment- and why it is important to the learning process. Higher Education 2010]

Teachers and Learners Perspectives

Assessing Teaching and Learning o Students: 276 male and female, MBBS1 students took part in a pilot survey study. Questions 1 and 2, grades background knowledge and knowledge following tutorials. Question 3, grades the link between lectures and tutorials. A space was provided for a free text / comment. Data were analysed using quantitative and qualitative methods. o Teachers: two focus groups including all available teachers (10-20) convene twice a year to share experiences and to reflect on their practice of teaching medical statistics

King’s College London, School of Medicine, MBBS1 Teaching Statistics Evaluation form (A Pilot) Date: 27 September, 2012 We would like to know your views on this practical / tutorial and its link to the preceding lectures. These will help us improve the teaching and learning of statistics for MBBS1 students at King’s. (1) Please rate the extent to which today’s practical / tutorial has improved on your knowledge and understanding of Statistics applied to medicine by circling one option for each of the items below, where: 1 = poor and 5 = excellent (2) To what extent do you see this practical linked (aligned with / matched) with the preceding lecture; 1 = poor alignment and 5= excellent alignment (3) Please add any comment or suggestion/s Thank you very much for your response! This is anonymous Extent of alignment Pre tutorial knowledge Post tutorial knowledge 12345

Example: Teachers Meeting End of Term Agenda Aim: Share our experiences this year and plan further improvements focusing on:  How the teaching went  How do we think it could be further improved  Tips for new people who will be joining the team  Feedback on tutorials you taught Feedback structured as: o Environmental Factors o Students o Materials

Findings

Knowledge gained via tutorials Pre tutorial knowledge * post tutorial knowledge Cross tabulations Post tutorial knowledge Total Pre tutorial knowledge %21.2%54.5%21.2%3.0% %3.3%41.1%53.3%1.1% %2.6%16.7%71.1%9.6% %.0%3.0%39.4%57.6% %.0%16.7%.0%83.3% %4.7%27.5%54.0%13.4%

Extent of link between lectures and tutorials 1 = Poor2345 = Excellent Total Frequency Percent2.9%3.6%32.0%38.9%17.8%

What Students Think

Too much / too fast “I found taking to fellow students more useful. Sometimes tutor gives so much help. That it is a bit overwhelming” “the MCQs were ambiguous. Quote, use the word certain, suggest stats is an absolute, a 95% CI is not certain it is 95%” “Do work before hand and go through in more detail in tutorials. Maybe more difficult questions and problems “ “Please talk slower, for international student like me, it could be hard to get the meaning at instant” “Classes should be optional for students with previous degrees etc with a statistical component”

Is stat relevant to being a doctor? “Don't really see the practical side to this in being a doctor” “it would be nice to highlight the relevance of these more with how it would help us in the future, because it would make it easier to put in context” “Classes should be optional for students with previous degrees etc with a statistical component.” “lecturer was enthusiastic & tried very hard to make this type of statistics interesting to a group of medical students. This is not easy!. Effort appreciated”...

Everything seems fine “the tutorial was helpful, the facilitator was nice and knowledgeable” “well explained. Answered questions”

What Teachers Think

Environmental Factors o Availability of pens, marker boards, flip chart: Back up: take yours check room the day before contact AV 24 hours before your session o Difficulty using overhead systems Training is proposed and in progress o Virtual Campus – difficult to navigate VC, and some abbreviations not clear- Provide direct links in circulations

Students’ Factors o Attendance: varies greatly, much fewer students around exam times. Head of year 1 & 2 re-inforce the importance of attendance We take attendance (signed sheet) for each session o Preparations: varies greatly. A short standardised overview of core information is given Head of year 1 & 2 reinforces the importance of preparation pre-all tutorials including stat tutorials in a circulation to all students o Students that ask for further assistance are not necessarily the ones that are struggling with the core material. We provide further assistance on demand for all who are interested: A revision lecture FAQ online service

Material related Varied preparation and different knowledge background o May need to go over key points of topic. o Streamlining of core knowledge - suggestion for a front page (overview of material) for tutors, to address main points. o This could be given to student but may lead to them not reading wider notes. (??) o It could highlight the essential knowledge – similar to ‘key messages’? o Put students into groups based on how they have prepared (or not)? Specific tutorials may benefit from shortening

Points that have arisen/tips for other o Prepare minutes of material beforehand in case students are not prepared. o There are different styles of teaching – and don't need to stick to a particular format. o Helpful to see others teach – to understand different teaching styles.

Further Actions We launched a revision to lectures and tutorials, including simplifying /shortening. Re-inforce the use of statistics and why students learn statistics formally in the first lecture (JP) and informally afterwards

Evidence-based medicine o Why is it important for medicine? o Do we need to do research to practice evidence-based medicine? o What research skills are needed?

Tomorrow's Doctors Document that describes medical curriculum in broad terms

...quotes from section on the Doctor as a scholar and scientist

Conclusions o Reflection on practise is the way to improve o Challenges include, time, background knowledge, the perception of the subject among others: can be addressed o Assess changes as you go along in a formative way o Beyond our reach is the percentage of the subject in the final assessment