Analyzing the Power and Error of Listeria monocytogenes Growth Challenge Studies Mark Powell U.S. Department of Agriculture Washington, DC IAFP 2009, Grapevine, TX, July

Introduction For ready-to-eat (RTE) foods that do not support growth of L. monocytogenes, food safety criteria limit of 100 colony forming units (cfu)/g. –EC Regulation 2073/2005 –FDA (2008) draft compliance policy guide –Codex (2009) microbiological criteria For RTE foods that do support growth of L. monocytogenes, “zero tolerance” (i.e., not detected in a regulatory sample). Design and interpretation of challenge studies to determine whether RTE are unable to support growth of L. monocytogenes.

Introduction Type I (F+) error (α): probability that H 0 is rejected when true. Type II (F-) error (β): probability that H 0 is not rejected when H a is true. Power = (1-β). By convention, α ≤ 0.05 and (1-β) ≥ 0.8

Fixed Exceedance Values To distinguish real growth from measurement uncertainty, L. monocytogenes challenge study protocols apply a fixed exceedance value: difference (δ) < M. EU/CRL (2008): difference between the initial and final sample median concentrations < 0.5 log 10 cfu/g for all batches tested (M m = 0.5 log). CCFH (2009): ≤ (on average) 0.5 log 10 cfu/g increase for at least the expected shelf life (M xbar = 0.5 log). FDA (2008): < 1 log 10 increase during replicate trials (assume M xbar = 1 log).

Fixed Exceedance Values M ~ ISO “expanded uncertainty” (U) x ± U = x ± 2σ x –where σ x = std. error of meas. uncertainty 2 (k factor) ≈ z (1-0.05/2) = 1.96 –α = 0.05; 2-sided interval If σ x = 0.25 log, → M = 0.5 log If σ x = 0.50 log, → M = 1.0 log

Variance of a difference If two means are independent: –where: –Assuming equal σ x and n:

Quantitative Measurement Uncertainty σ x ≠ 0.25 or 0.5 or any other fixed value. EC, FDA, and CCFH reference ISO Method for enumerating L. monocytogenes in RTE foods. Scotter et al (2001): std dev reproducibility (s R ) = log cfu/g in food samples. s R : an intra-laboratory measure of quantitative measurement uncertainty.

Challenge Study Designs Differ Number of sampling times Number of batches Experiment-wise α depends on: –Number of comparisons –Whether multiple comparisons are independent or dependent. Independent: (μ final – μ initial ) X multiple batches Dependent: μ(t) – μ(t 0 ) within a batch

Challenge Study Designs Differ EU/CRL (2008): k = 2 sampling times (initial and final), b ≥ 3 batches, sample size (n) = 3 samples per sampling time. –c ≥ 3 multiple, independent pair-wise comparisons. –std dev w/in batch < 0.3 log at t 0. FDA cites Scott et al. (2005): k = 5-7 sampling times, sample size (n) = 2-3 samples per sampling time. –c = k-1 dependent pair-wise comparisons per trial (μ(t) – μ(t 0 )). –No minimum number of batches.

Type I error for fixed exceedance value (M xbar ) For a single comparison test of H 0 : δ ≤ 0: For multiple independent comparisons: For multiple dependent comparisons, Monte Carlo simulation, with α = proportion (F+) Based on Scotter et al (2001), consider σ x from 0.15 log cfu/g to 0.50 log cfu/g

Type I error for difference in means fixed exceedance value (M xbar ) = 0.5 log cfu/g ** p<0.01 std. dev. (log cfu/g) sample size (n) = 2sample size (n) = 3 independent comparisons (c) p(type I error) ≤ α 0.15** **

Type I error for difference in means fixed exceedance value (M xbar ) = 0.5 log cfu/g ** p<0.01 std. dev. (log cfu/g) sample size (n) = 2sample size (n) = 3 dependent comparisons (c) p(type I error) ≤ α 0.15** **

Power of F-test for One-Way ANOVA SAS © PROC Power –where: –F ω = non-central F dist –F crit = critical value of the F dist with k-1 and k(n-1) df –ω (non-centrality parameter) = –H0: μ i = μ for all i –Ha: μ max – μ min = δ –Power depends on δ and growth pattern under H a

Pattern that maximizes power for δ = 1 log

Pattern that minimizes power for δ = 1 log

Power curves for one-way ANOVA F-test (α = 0.05, δ = 1 log cfu/g) with sample size n = 2 and sampling times k = 2-7 max min

Power curves for one-way ANOVA F-test (α = 0.05, δ = 1 log cfu/g) with sample size n = 3 and sampling times k = 2-7 max min

Conclusions Applying any fixed acceptance criteria exceedance value (e.g., less than a 0.5 log or 1 log increase) to distinguish real growth from quantitative measurement uncertainty over different experimental designs and/or measurement uncertainty values implies highly inconsistent type I error probabilities.

Conclusions None of the L. monocytogenes growth challenge study designs currently being considered are likely to provide an F-test with α = 0.05 and power ≥ 0.8 to detect a 1 log increase in mean concentration over the entire range of measurement uncertainty values for enumeration of L. monocytogenes reported in food samples in a validation study of ISO Method

Conclusions Satisfying these conventional experimental design criteria would require a larger sample size, lower measurement uncertainty, or both.

Thank you