1. Center for Biofilm Engineering CBE workshop – July 2009 Al Parker Statistician and Research Engineer Montana State University Ruggedness Assessment and Experimental Design in the Biofilm Laboratory

2. Acknowledgments  Colleagues in the CBE; esp., the SBML team  Funding  US EPA  Montana Board of Research, Commercialization and Technology  Industrial Associates of the CBE  Big Sky Statistical Analysts LLC

3. Statistical thinking  Data  Ruggedness Testing  2 p Factorial versus One-at-a-time design  Uncertainty assessment

4. Statistical thinking  Data  Ruggedness Testing  2 p Factorial versus One-at-a-time design  Uncertainty assessment

5. Grow: CDC biofilm reactor

6. Sampling 1. Rod is removed from reactor 2. Coupon is removed from the rod 3. Rinse

7. Treat: disinfecting an established biofilm

8. Sample: harvest from coupon, disaggregate harvest biofilm by scraping with a wooden applicator stick K. Moll 2008 homogenize to disaggregate clumps K. Moll 2008

9. Sampling and analysis Biofilm is scraped & rinsed into the dilution tube; the suspension is disaggregated Dilution series; plate in duplicate or triplicate Treated coupon A. Hilyard, 2008 Drop plate: Viable cell density (cfu/cm 2 )

10. Statistical thinking  Data  Ruggedness Testing  2 p Factorial versus One-at-a-time design  Uncertainty assessment

11. A standard laboratory method is said to be rugged if the outcome is unaffected by slight departures from the protocol. Ruggedness

12.  Disaggregation: sonicated or homogenized  Nutrient (TSB, continuous flow): 50, 100, 200 mg/l  Rotation (stir plate): 125, 225, 325 rpm  Temperature: 20, 23, 26 o C  Time in batch mode: 3, 18, 24 hr Parameters in the protocol

13. Is efficacy testing using the CDC reactor rugged with respect to changes in the batch time over which the biofilm was grown? Ruggedness

14. Ruggedness with respect to batch time > 18 hours Time (h) Viable cell density (log scale) 0 10 7 10 5 18 10 3 © 2002 CBE

15. 1. Conduct a minimal number of experiments to:  Identify unimportant parameters: Is the biofilm significantly influenced by all 5 parameters?  Check for interactions among parameters 2. Conduct another series of experiments using only the influential parameters and interactions. Performing a ruggedness test

16. Is efficacy testing using the CDC reactor rugged with respect to changes in stir plate rotation and temperature? Ruggedness

17. Full factorial design: 2 factors, each at 3 levels © 2002 CBE

18. Statistical thinking  Data  Ruggedness Testing  2 p Factorial versus One-at-a-time design  Uncertainty assessment

19. One-at-a-time: Study temperature © 2002 CBE

20. One-at-a-time: Study rpm © 2002 CBE

21. One-at-a-time design for 2 factors © 2002 CBE

22. 2 2 factorial design © 2002 CBE

23. A factorial design is superior to a one-at-a- time design:  Factorial design can detect an important interaction between the two factors; the one-at-a-time design can’t  Factorial design has greater precision when estimating the main effects of each factor than the one-at-a-time design

24. True mean log(density) increases with temperature; slope depends on RPM © 2002 CBE

25. One-at-a-time design can estimate these four points only © 2002 CBE

26. One-at-a-time design cannot detect the fact that the slope depends on RPM © 2002 CBE

27. Factorial design can estimate these four points only © 2002 CBE

28. Factorial design can detect the fact that the slope depends on RPM © 2002 CBE

29. Temperature Batch time RPM One-at-a-time However … the factorial approach requires more experimental effort © 2002 CBE

30. Temperature Batch time RPM One-at-a-time Factorial However … the factorial approach requires more experimental effort © 2002 CBE

31. Temperature Batch time RPM 2 3 Factorial ½ Fraction of Can use fewer factorial runs; however, can’t estimate all interactions © 2002 CBE

32. 2 5 factorial design: 2 5 = 32 One-at-a-time design: 2*5 = 10 ¼ fraction of the 2 5 factorial design: 32* ¼ = 8 For Five factors, instead of three: how many experimental runs?

33. Statistical thinking  Data  Ruggedness Testing  2 p Factorial versus One-at-a-time design  Uncertainty assessment

34. Differences among experiments is the major source of variation  67% attributable to between experiments  33% attributable to within experiments

35. S n c m c 2 + Formula for the SE of the mean LR, averaged over experiments S c = within-experiment variance of control coupon LD S d = within-experiment variance of disinfected coupon LD S E = between-experiments variance of LR n c = number of control coupons n d = number of disinfected coupons m = number of experiments 2 2 2 S n d m d 2 + S m E 2 SE of mean LR =

36. Where to invest effort to get the most precision?  2 experiments; 6 coupons each; SE of mean log density (12 coupons) = 0.24  4 experiments; 2 coupons each; SE of mean log density (8 coupons) = 0.19  The precision is increased by running more experiments with less effort per experiment

37. Summary  It is important to do an arm-chair experiment first  Use 2 p (fractional) factorial design to determine ruggedness of the protocol to changes in parameters  Rely on multiple, independent experiments, each with few samples, in contrast to one experiment with many samples

38. Fin

39. Reliable laboratory methods