1 e-Bug Pack Evaluation

2 Evaluation Process IrelandEngland France Spain Portugal Finland Italy Greece Poland Denmark Belgium Czech Rep. Slovenia Latvia Lithuania Hungary Slovakia Croatia England France Czech Rep. Control and Intervention Schools Junior and Senior Schools Minimum of 151 students in each group Evaluation carried out in 3 countries

3 Regions Evaluated England Czech Republic France Gloucester London Bordeaux Nice Prague Ostrava

4 Primary Outcome Knowledge Change –Baseline knowledge –Knowledge change –Knowledge retention Variables –Teacher knowledge of microbiology –Class size –Region –Learning disability register

5 Secondary Outcome Satisfaction –Teacher Questionnaires –Student Questionnaires –Focus groups

6 Knowledge Change Results Variable did not have any effect on knowledge change at any of the time points Exception Region

7 Introduction to Microbes An Introduction –Make microbes out of paper cut outs and Arts & Crafts materials Good Microbes –Yogurt Making Activity Bad Microbes –Interactive Discussion

8 Introduction to Microbes ControlIntervention BaselineChangeRetentionBaselineChangeRetention Gloucester 6.8 (-4.4, 18.0) 33.5 (22.3, 44.7) 33.7 (22.6, 44.9) (30.3, 34.7) 29.8 (27.0, 32.6) London Nice 8.1 (-0.7, 16.9) 24.3 (14.5, 34.1) 26.5 (15.5, 37.4) 7.0 (-0.8, 14.7) 30.0 (22.3, 37.8) 32.5 (24.3, 40.6) Bordeaux 9.3 (-0.6, 19.1) 29.1 (18.8, 39.4) 31.5 (21.3, 41.8) 13.2 (2.2, 24.2) 36.4 (24.2, 48.5) 36.2 (23.8, 48.6) Prague 3.5 (-4.7, 11.7) 14.1 (5.3, 22.9) 13.4 (4.7, 22.0) 6.0 (-2.4, 14.4) 26.4 (18.0, 34.8) 28.4 (19.9, 36.9) Ostrava -0.7 (-8.7, 7.3) 2.1 (-6.2, 10.4) 11.6 (3.6, 19.6) 4.7 (-3.5, 12.8) 30.8 (22.7, 39.0) 29.1 (21.0, 37.3)

9 Introduction to Microbes Control Change Control Retention e-Bug Change e-Bug Retention Gloucester   London Nice   Bordeaux   Prague   Ostrava  Significant improvement in knowledge across all regions No loss in student knowledge over time except Gloucester *

10 Spread of Infection Hand Hygiene –Hand washing activity using UV gel/powder to represent microbes Respiratory Hygiene –Sneezing Runway Food Hygiene –Chicken salad activity using UV gel/powder to represent microbes on raw chicken

11 Spread of Infection ControlIntervention BaselineChangeRetentionBaselineChangeRetention Gloucester 5.7 (-3.5, 14.9) 10.5 (1.3, 19.8) 12.2 (3.0, 21.4) (8.6, 12.6) 12.2 (9.6, 14.8) London Nice 2.8 (-4.7, 10.3) -7.7 (-16.2, 0.8) -2.1 (-11.7, 7.5) -3.0 (-9.5, 3.4) 6.2 (-0.3, 12.6) 8.1 (1.2, 14.9) Bordeaux 1.8 (-6.3, 9.9) 11.5 (3.0, 20.1) 12.4 (3.9, 21.0) 5.0 (-4.0, 14.0) 15.5 (5.3, 25.7) 11.9 (1.5, 22.3) Prague 3.3 (-3.5, 10.1) 3.4 (-4.1, 10.9) 2.1 (-5.2, 9.4) -3.8 (-10.9, 3.3) 9.4 (2.4, 16.5) 6.9 (-0.2, 14.1) Ostrava -4.4 (-11.1, 2.2) -5.3 (-12.3, 1.7) -4.3 (-11.0, 2.4) -2.7 (-9.5, 4.1) 8.6 (1.8, 15.4) 10.7 (3.9, 17.5)

12 Spread of Infection Control Change Control Retention e-Bug Change e-Bug Retention Gloucester  London---- Nice  Bordeaux  Prague  Ostrava  Significant improvement in knowledge across all regions No loss in student knowledge over time

13 Treatment / Prevention of Infection Vaccinations –Reading comprehension ‘The story of Edward Jenner’ Antibiotic Use and Medicines –Comic Strip and classroom discussion

14 ControlIntervention BaselineChangeRetentionBaselineChange Retention Gloucester 2.8 (-8.4, 14.1) 16.8 (5.5, 28.1) 14.7 (3.5, 25.9) (12.6, 18.1) 14.6 (11.1, 18.1) London Nice 15.9 (6.7, 25.0) 24.1 (13.6, 34.6) 12.1 (0.0, 24.3) 13.3 (5.5, 21.1) 22.5 (14.6, 30.3) 27.4 (19.0, 35.9) Bordeaux 19.0 (9.1, 28.9) 26.9 (16.4, 37.5) 32.2 (21.6, 42.7) 18.7 (7.8, 29.7) 30.8 (18.1, 43.5) 22.3 (9.3, 35.3) Prague 14.0 (5.7, 22.3) 19.8 (10.5, 29.2) 24.4 (15.4, 33.4) 16.9 (8.2, 25.5) 29.8 (21.2, 38.4) 26.0 (17.3, 34.7) Ostrava 12.4 (4.3, 20.5) 10.1 (1.4, 18.7) 10.3 (2.1, 18.5) 9.5 (1.2, 17.8) 28.3 (19.9, 36.6) 20.0 (11.7, 28.3) Treatment / Prevention of Infection

15 Control Change Control Retention e-Bug Change e-Bug Retention Gloucester   London Nice   Bordeaux   Prague   Ostrava  Significant improvement in knowledge across all regions No loss in student knowledge over time except Gloucester Treatment / Prevention of Infection

16 Knowledge Change Summary Significant Improvement in all pack teaching areas over time No reduction in student knowledge after a 6 week period Control schools performed equally as well

17 Student Satisfaction

18 Student Satisfaction Likes –Interesting Facts –Interactive, fun activities –Team work –Seeing and naming the microbes –Characters Dislikes –Waiting too long to take your turn –Some activities too difficult –When the teacher did the work –Finishing the activity!

19 Teacher Satisfaction Junior Schools –100% satisfaction in England and Czech Republic –3/11 teachers in France said they would NOT use e-Bug in the future

20 Teacher Satisfaction Teacher Likes –Interactive –Engaged students Teacher dislikes –Too much photocopying –Some activities too advanced –Yogurt activity not suitable for practical reasons –Pages didn’t photocopy well –Not enough multimedia support

21 General Changes Photocopying issues –Merged student handouts –Created handouts in MS PowerPoint –Changed the colour of the resource

22 General Changes Multimedia Support –Films of activities –Added alternative activities to website –Included more posters etc on the website –Pack available in MS Word and PDF

23 General Changes Other –Provide more alternative materials –Include more prompts and suggestions for teachers –Provide more animations –Introduce an Immunity Section

24 Specific Sectional Changes Introduction –Changed arts and craft materials to playdough Good and bad –Changed titles to USEFUL and HARMFUL microbes –Created a new ‘Yeast Races’ activity Food Hygiene –Food Hygiene: Condensed main activity Vaccinations –Vaccinations: Added a role play element