1. Statistical Analysis of IC50s Nick Andrews, Statistics Unit CFI, HPA

2. Outline Curve fitting to estimate IC50 Determining cut-offs for outliers (high IC50) Monitoring trends in IC50

3. Curve fitting The data in each run consists of measuring the RFU of a sample against the antiviral at different concentrations (ten 4-fold dilutions) as well as with no-antiviral (virus-control) Estimate IC50: the concentration at which the rfu is 50% of that from the virus control.

4. Sources of variation in IC50 Curve fitting Within a run (2 – replicates) Between runs (Reference samples)

5. Point to Point (Excel)

6. Smooth Curves (S-shaped Graph-pad)

7. Comparison-curve fitting SampleP-PS-shape Ratio (Hi/Low) 292R0.690.691.00 292K>40008551- 119V40.340.71.01 A/Lisbon22/22070.590.561.05 A/Latvia/685/20070.680.691.01 A/Denmark/1/20070.430.411.05 A/Denmark/2/20070.540.511.06 A/Denmark/3/20070.540.481.12

8. Comments (Curve fitting) For most samples curve will make no more than about 5% difference to results. Must be clear exactly how the curve fitting and calculation of IC50 works – e.g. is IC50 based on 50% of OD of virus control or 50% of fitted upper asymptote. Need to ensure problems in curve fitting are flagged.

9. Between replicate variation (Point to point) SampleRep1Rep2Ratio(High/Low) 292R0.750.631.19 292K3769>4000- 119V42.038.61.09 A/Lisbon22/22070.570.611.07 A/Latvia/685/20070.700.661.06 A/Denmark/1/20070.400.461.15 A/Denmark/2/20070.580.491.18 A/Denmark/3/20070.540.531.02

10. Comments on replicates Between replicates in a run variation is greater than curve fitting with most results within about 15%-20%. Using replicates and taking the average reduces this effect. Large differences between replicates should be flagged (e.g. >30%).

11. Between run variation

12. Comparison runs (Reference Virus) SampleRun1Run2Ratio(High/Low) 274H Osel0.420.361.17 274H Zan0.170.221.29 274Y Osel3003031.01 274Y Zan0.310.371.19 119V Osel43.734.01.29 199V Zan1.411.321.07 DB Osel92013161.43 DB Zan2393511.47

13. Comments Between runs variation is higher still at about 40%+. Hence retest those with high values and use controls in runs to identify problems in a run.

14. Determining Cut-offs The aim is to pick up outliers with high IC50 results as they may indicate resistant strains. Need a method to determine the ‘normal range’ in the absence of outliers. Various statistical methods may be used – the important thing is to ensure the outliers do not unduly affect the cut-off.

15. Methods used Need sufficient data points for initial estimate (about 30-50). These can be refined later with more data. Advisable to transform data to approximate normality (log-transform). Obtain a robust estimation of the Standard deviation (1.48*Median absolute deviation or using 0.75*Inter-quartile range). Use median + say 1.65, 3 SD.

16. Example: Initial 30 results Median=0.47 Robust SD (log-scale) = 0.17 Cut1 =1.07 Cut2=1.81

17. Example: Final results 2006/07 Median=0.48 Robust SD (log-scale) = 0.12 Cut1 =0.93 Cut2=1.35

18. Do we need to log-transform? Most results from dilution assays produce ‘geometric results’ so likely to be sensible Sometimes doesn’t seem to matter but sometimes data are skewed. (e.g. lower quartile much closer to median than upper quartile) – important to transform as robust methods assume data are normal once outliers are removed.

19. Non-log scale 1.6

20. Log-scale (y-axis) 2.7

21. Alternative presentation – Box and Whisker Plot Uses median + 1.5 * IQR (+2SD) Could be adjusted to match the scatter plots 3.0

22. Monitoring ICD50 (outliers) over time First look at the data (scatter plots/box-whisker) Calculate statistics for different time periods –Median –%Outliers –Geometric means with 95% CI with outliers removed –Kruskal Wallis test for medians –Calculate Geometeric Means with 95% CI and t-test/ anova (outliers removed) –regression for time trend (outliers removed).

23. Monitoring ICD50 (outliers) over time Perform statistical tests –Kruskal Wallis test for medians –Chi-squared or Fishers exact test for % outliers –Anova or t-tests for comparing means (oultiers removed). Alternatively look for non- overlapping 95% CI which is a conservative method (approx p<0.007)

24. Graphical Presentation – Box and Whisker

25. Scatter Plot

26. Scatter plot

27. Statistics H3N2 Zanamivir NIMR Year Samples>3SD (%)medianGeomean (95% CI) 2005/06881 (1.1%)0.670.69 (0.62-0.76) 2006/071734 (2.3%)0.590.58 (0.56-0.61) P-value comparing %>3SD = 0.67 P-value comparing medians = 0.0002 – significant although actual difference small. 95% CI for Geometric means do not overlap Note of caution: Changes may occur within a year so this comparison maybe too simplistic. For example there appears to be some evidence of a change within 2006/07

28. Summary Good use of statistical methods can help interpret the IC50 results and ensure assay results are reliable. Many appropriate methods already in place but more could be incorporated