e-Bug Pack Evaluation Senior Schools Warsaw, June 2008 e-Bug is funded by the European Commission DG-SANCO communicating science edinburgh Mar 08 Senior school pack evaluation ~ Donna Lecky June 2008 1 1

Evaluation Process The evaluation was carried out in 3 countries France Czech Republic United Kingdom Control and Intervention Schools Junior and Senior Schools Minimum of 151 students in each group

Evaluation Process Quantitative Data Qualitative Data Baseline knowledge Knowledge change Knowledge retention Teacher questionnaires Teacher focus groups Student questionnaires

Quantitative data Overview Topics % Correct before % Correct after % Improvement score p value Micro organisms 50 63 17 (14, 21) <0.001 Spread of Infection 68 83 19 (15, 23) Treatment and Prevention 29 51 27 (23, 31)

Introduction to Microbes Quantitative Data Significant increase in student knowledge (<0.001) Improvement scores (baseline vs change) by statement and topic for senior schools Micro organisms: % Improvement (95% CI) p Value Fungi can be used to make antibiotics 14 (2, 25) 0.02 Microbes are the cause of all illnesses 12 (1, 23) 0.04 Microbes can cause food poisoning 11 (1, 22) 0.03

Introduction to Microbes Teacher Comments 7 10 Fact sheets Images did not photocopy well Too much information and too advanced Main Activity Quite childish although students seemed to like it Suggestions More electron microscope images Classroom demonstration to explain relative sizes – string around classroom, glitter, balloons, etc Extension activity look at microbe types under microscope

Introduction to Microbes Student Comments How enjoyable did students find the activity? 72% of students questioned rated the activity ≥3 on a scale of 1 – 5 Average rating of 3 in France Student Likes Practical activity Interesting facts Whiteboard presentations and images Game Student Dislikes Confusing Too easy Childish Complicated names What were the main things they learned? There are 3 different types of microbes Some can be useful and some harmful

Our suggestions Make more ID cards of microbe Make game of the microbe names Fact sheets Highlight key words Make outline diagrams in pack and 3D on web Condense to 1 student handout Make the ‘size’ presentation into a practical

Good Microbes Teacher Comments 7 10 The activity Fun and interesting Met all the learning outcomes Suggestions Make smaller quantities of yogurt Remove the ‘add fruits etc to taste’, students cannot eat it The sterile milk is NOT a proper control step, change to sterile yogurt and add a boiling step Add the question – what is the purpose of a control?

Good Microbes Student Comments How enjoyable did students find the activity? 90% of students questioned rated the activity ≥3 on a scale of 1 – 5 Average ranking of 4 in France Student Likes Practical activity helped understanding Interesting and fun Microscope work Memorable Student Dislikes Teacher made the yogurt while the students watched Smell No images What were the main things they learned Microbes can be useful There are good and bad microbes Actual size of bacteria

Our Suggestions Suggest teacher sterilizes equipment and makes a batch of yogurt in advance Include images of extension activity Add an alternative to view mould in blue cheese Highlight the benefits of good microbes Include ‘what’s the purpose of a control?’ question

Bad Microbes Teacher Comments 5 10 Main Activity Liked the teacher sheets Time-consuming Suggestions Use the map as a starting point Extension activity Too much reading, replace with research activity into MRSA or C. difficile

Bad Microbes Student Comments How enjoyable did students find the activity? 67% of students questioned rated the activity ≥3 on a scale of 1 – 5 Average ranking of 4 in France Student Likes Researching the topics Group work Interesting and fun Videos and images Designing factsheets Student Dislikes Too much copying from board No practical Homework Not enough time What were the main things they learned Geographic location and quantity of different diseases How microbes can infect people Diarrhoea can kill

Our Suggestions Make student questionnaires shorter Suggest activity as a group work activity Include more images Use examples more relevant to teenagers i.e. holiday destinations, STIs Replace extension activity with research into MRSA or C. difficile

Transfer of Infection Quantitative Data Significant increase in student knowledge (<0.001) Improvement scores (baseline vs change) by statement and topic for senior schools Micro organisms: % Improvement (95% CI) P Value Fungi can be used to make antibiotics 14 (2, 25) 0.02 Microbes are the cause of all illnesses 12 (1, 23) 0.04 Microbes can cause food poisoning 11 (1, 22) 0.03

Hand Hygiene Teacher Comments 6 10 Main activity Students enjoyed the activity and the teachers enjoyed teaching this lesson Didn’t work very well in the ‘all girls’ school as their hands were unbelievably clean to begin with (girls also unwilling to take part) Suggestions Prompt the students to touch surfaces etc prior to starting the activity Glitter glue worked better than the Glo gel Stronger UV light works better or darken the room before showing the results

Hand Hygiene Student Comments How enjoyable did students find the activity? 78% of students questioned rated the activity ≥3 on a scale of 1 – 5 Average rank of 4 in France Student Likes Visualise the microbes Seeing how microbes grow on the agar Interactive Seeing how and why to wash hands properly Student Dislikes Seeing how dirty our hands are Too much class discussion What were the main things they learned How to stop germs spreading How and why to wash hands Microbes are always there

Our Suggestions Replace photos with coloured agar to make it easier to visualise Simplify the questions Take out graph paper and condense to 1 worksheet Suggest activity begins with Glo gel and use photos of colonies for graphing work

Respiratory Hygiene Teacher Comments 6 10 Main lesson Popular activity with students Chain of Infection sheets did not photocopy well Presentation needs to be more interactive Suggestions Extra prompt to make sure squeezy bottle works Use of specific terms requested in the UK curriculum Dependant and Independent variable

Respiratory Hygiene Student Comments How enjoyable did students find the activity? 94% of students questioned rated the activity ≥3 on a scale of 1 - 5 Student Likes Fun practical activity Visualise the spread of infection Situation that was relevant to them as young people Got to learn the causes AND prevention of infection Student Dislikes Teacher did everything Repetitive Graphs were boring Copying from board What were the main things they learned How infection can spread How to prevent infection from spreading Chain of Infection

Our Suggestions Delete diagram on SW1 as teachers didn’t feel there was any need for it Introduce a global activity aspect? Take out graph paper from pack as this is generally supplied by schools Senior school pack evaluation ~ Donna Lecky June 2008

Sexual Transmission Teacher Comments 7 10 General comments Students enjoyed the activity and were genuinely interested Area that is difficult to teach but very relevant Main activity Prompt teachers to choose someone who won’t be picked on if they are the initial carrier Add extra prompt to ensure students mix outside their own group Extension Activity Cartoons very childish and almost spoilt the activity Student discussions were interesting and informative Dialogues can be changed on the web – not good!!!

Sexual Transmission Student Comments How enjoyable did students find the activity? 100% of students questioned rated the activity ≥3 on a scale of 1 – 5 Activity rank of 5 in France Student Likes Talked to as grown ups Finally explained WHY its important to have safe sex Showed HOW to be safe Fun and memorable Student Dislikes Slightly embarrassing Stories were too basic Practical was disturbing What were the main things they learned How STIs can be transmitted How to protect yourself To get regular testing if sexually active

Our Suggestions Suggest activity be carried out in smaller groups of 15 students Modify the online images so dialogues cannot be changed Suggest teachers may wish to bring in other teacher or a doctor to carry out the test to make it more realistic Demonstrate how to put on a condom?

Treatment and Infection Quantitative Data Significant increase in student knowledge (<0.001) Improvement scores (baseline vs change) by statement and topic for senior schools Micro organisms: % Improvement (95% CI) p Value Vaccines help protect people against viral infections only 8 (-1, 18) 0.09

Treatment of Infection Teacher Comments 6 10 Activity Disappointed that there was no natural defence system diagrams or images Too advanced, dry and dull Some teachers thought it was interesting Suggestions Request activity, possibly similar to the junior school pack activity

Treatment of Infection Student Comments How enjoyable did students find the activity? 83% of students questioned rated the activity ≥3 on a scale of 1 – 5 Activity rank 3 from France Student Likes Story Problem solving How resistance works Student Dislikes Too much writing No practical activity Boring Difficult (but challenging) What were the main things they learned Overuse leads to resistance Drugs become resistant Antibiotics should only be used on bacteria!

Our Suggestions Use antibiotic discs to demonstrate resistance Make the activity into an investigation or drama style activity

Prevention of Infection Teacher Comments 6 10 Start with travel vaccines to make it more relevant to the students Add Jenner story as an introduction and then data analysis Include success rate facts Picture of rare or eliminated diseases due to vaccines Factsheet too static and too much information More information on natural defences – request video clip of white cells engulfing bacteria cells More ICT – have students draw graphs on computers

Prevention of Infection Student Comments How enjoyable did students find the activity? 94% of students questioned rated the activity ≥3 on a scale of 1 – 5 Activity rank of 2 in France Student Likes Interesting Story Computer work Student Dislikes No practical Got boring Seeing pox pictures Complicated What was the main thing they learned How a vaccine works

Our Suggestions Make shorter Rethink entire activity Ask students to research vaccines they would need for holidays How do they work, etc Add a section on the success of vaccines Add a cartoon or series of images on how vaccines work instead of fact sheet

How to improve the activities Make the activities more interactive Links to junior pack preferable for lower ability classes Include more video clips and images Suggest work be done in groups or pairs Some activities too easy More explanations – WHY do things happen! Less worksheets and more ICT Use songs, drama, etc to help us remember activities Teachers should be more involved Better ways of presenting practical