1. Analyzing the Power and Error of Listeria monocytogenes Growth Challenge Studies Mark Powell U.S. Department of Agriculture Washington, DC IAFP 2009, Grapevine, TX, 12-15 July

2. Introduction For ready-to-eat (RTE) foods that do not support growth of L. monocytogenes, food safety criteria limit of 100 colony forming units (cfu)/g. –EC Regulation 2073/2005 –FDA (2008) draft compliance policy guide –Codex (2009) microbiological criteria For RTE foods that do support growth of L. monocytogenes, “zero tolerance” (i.e., not detected in a regulatory sample). Design and interpretation of challenge studies to determine whether RTE are unable to support growth of L. monocytogenes.

3. Introduction Type I (F+) error (α): probability that H 0 is rejected when true. Type II (F-) error (β): probability that H 0 is not rejected when H a is true. Power = (1-β). By convention, α ≤ 0.05 and (1-β) ≥ 0.8

4. Fixed Exceedance Values To distinguish real growth from measurement uncertainty, L. monocytogenes challenge study protocols apply a fixed exceedance value: difference (δ) < M. EU/CRL (2008): difference between the initial and final sample median concentrations < 0.5 log 10 cfu/g for all batches tested (M m = 0.5 log). CCFH (2009): ≤ (on average) 0.5 log 10 cfu/g increase for at least the expected shelf life (M xbar = 0.5 log). FDA (2008): < 1 log 10 increase during replicate trials (assume M xbar = 1 log).

5. Fixed Exceedance Values M ~ ISO “expanded uncertainty” (U) x ± U = x ± 2σ x –where σ x = std. error of meas. uncertainty 2 (k factor) ≈ z (1-0.05/2) = 1.96 –α = 0.05; 2-sided interval If σ x = 0.25 log, → M = 0.5 log If σ x = 0.50 log, → M = 1.0 log

6. Variance of a difference If two means are independent: –where: –Assuming equal σ x and n:

7. Quantitative Measurement Uncertainty σ x ≠ 0.25 or 0.5 or any other fixed value. EC, FDA, and CCFH reference ISO Method 11290-2 for enumerating L. monocytogenes in RTE foods. Scotter et al (2001): std dev reproducibility (s R ) = 0.17 - 0.45 log cfu/g in food samples. s R : an intra-laboratory measure of quantitative measurement uncertainty.

8. Challenge Study Designs Differ Number of sampling times Number of batches Experiment-wise α depends on: –Number of comparisons –Whether multiple comparisons are independent or dependent. Independent: (μ final – μ initial ) X multiple batches Dependent: μ(t) – μ(t 0 ) within a batch

9. Challenge Study Designs Differ EU/CRL (2008): k = 2 sampling times (initial and final), b ≥ 3 batches, sample size (n) = 3 samples per sampling time. –c ≥ 3 multiple, independent pair-wise comparisons. –std dev w/in batch < 0.3 log at t 0. FDA cites Scott et al. (2005): k = 5-7 sampling times, sample size (n) = 2-3 samples per sampling time. –c = k-1 dependent pair-wise comparisons per trial (μ(t) – μ(t 0 )). –No minimum number of batches.

10. Type I error for fixed exceedance value (M xbar ) For a single comparison test of H 0 : δ ≤ 0: For multiple independent comparisons: For multiple dependent comparisons, Monte Carlo simulation, with α = proportion (F+) Based on Scotter et al (2001), consider σ x from 0.15 log cfu/g to 0.50 log cfu/g

11. Type I error for difference in means fixed exceedance value (M xbar ) = 0.5 log cfu/g ** p<0.01 std. dev. (log cfu/g) sample size (n) = 2sample size (n) = 3 independent comparisons (c) 123456123456 p(type I error) ≤ α 0.15** 0.200.01 0.02 0.030.04** 0.01 0.250.020.040.070.090.110.130.01 0.020.030.04 0.300.050.090.140.180.220.250.020.040.060.080.100.12 0.350.080.150.210.270.330.380.040.080.120.150.190.22 0.400.110.200.280.360.430.490.060.120.180.230.280.32 0.450.130.250.350.440.510.580.090.170.240.300.360.42 0.500.160.290.400.500.580.650.110.210.300.370.440.50

12. Type I error for difference in means fixed exceedance value (M xbar ) = 0.5 log cfu/g ** p<0.01 std. dev. (log cfu/g) sample size (n) = 2sample size (n) = 3 dependent comparisons (c) 123456123456 p(type I error) ≤ α 0.15** 0.200.01 0.02 0.03 ** 0.01 0.250.020.040.060.070.080.100.01 0.02 0.03 0.300.050.090.110.140.160.180.020.040.050.060.080.09 0.350.080.130.170.210.230.260.040.070.090.120.140.15 0.400.110.180.230.270.310.330.060.110.140.170.200.22 0.450.130.220.280.330.360.390.090.140.190.230.260.29 0.500.160.250.320.370.410.450.110.180.240.280.320.34

13. Power of F-test for One-Way ANOVA SAS © PROC Power –where: –F ω = non-central F dist –F crit = critical value of the F dist with k-1 and k(n-1) df –ω (non-centrality parameter) = –H0: μ i = μ for all i –Ha: μ max – μ min = δ –Power depends on δ and growth pattern under H a

14. Pattern that maximizes power for δ = 1 log

15. Pattern that minimizes power for δ = 1 log

16. Power curves for one-way ANOVA F-test (α = 0.05, δ = 1 log cfu/g) with sample size n = 2 and sampling times k = 2-7 max min

17. Power curves for one-way ANOVA F-test (α = 0.05, δ = 1 log cfu/g) with sample size n = 3 and sampling times k = 2-7 max min

18. Conclusions Applying any fixed acceptance criteria exceedance value (e.g., less than a 0.5 log or 1 log increase) to distinguish real growth from quantitative measurement uncertainty over different experimental designs and/or measurement uncertainty values implies highly inconsistent type I error probabilities.

19. Conclusions None of the L. monocytogenes growth challenge study designs currently being considered are likely to provide an F-test with α = 0.05 and power ≥ 0.8 to detect a 1 log increase in mean concentration over the entire range of measurement uncertainty values for enumeration of L. monocytogenes reported in food samples in a validation study of ISO Method 11290- 2.

20. Conclusions Satisfying these conventional experimental design criteria would require a larger sample size, lower measurement uncertainty, or both.

21. Thank you