1. 1 Is it potent? Can these results tell me? Statistics for assays Ann Yellowlees PhD Quantics Consulting Limited

2. 2 Contents Role of statistics in bioassay  Regulations Estimating Relative Potency  Choice of model for RP estimation  Parallelism Case study

3. 3 Role of statistics in bioassay Design / optimisation Analysis Validation

4. 4 Why use statistics - regulations Validation of routine and custom assays:  ICH Q6B SPECIFICATIONS: TEST PROCEDURES AND ACCEPTANCE CRITERIA FOR BIOTECHNOLOGICAL/BIOLOGICAL PRODUCTS “Assessment of biological properties constitutes an....essential step in establishing a complete characterisation profile” “Appropriate statistical analysis should be applied” “Methods of analysis, including justification and rationale, should be described fully” “A relevant, validated potency assay should be part of the specifications for a biotechnological or biological drug substance and/or drug product”

5. 5 Why use statistics - regulations Stability testing, more defined by ICH:  ICH Q5C QUALITY OF BIOTECHNOLOGICAL PRODUCTS “At time of submission, applicants should have validated the methods that comprise the stability-indicating profile and the data should be available for review”  ICH Q1A (R2) STABILITY TESTING “An approach for analyzing data of quantitative attribute that is expected to change with time is to determine the time at which the 95% one-sided confidence limit for the mean curve intersects the acceptance criterion”

6. 6 Estimating Relative Potency Analysis of dose – response data  Choosing the best model for estimating RP  Checks for parallelism  Estimating RP  Calculations: RP and its precision  Improving precision

7. 7 Data types Response per unit (animal, welI, etc):  Binary Dead / alive at a given time point Diseased / disease free at a given time point Summarised as % or proportion  Continuous Antibody level Time of death Optical density

8. 8 Continuous response Note:  Log concentration  S shape  Noise level varies

9. 9 Continuous response – means

10. 10 What is Relative Potency? Potency of the sample in comparison with the reference Mathematically this is the same as:

11. 11 Estimating RP

12. 12 When is it valid to calculate RP? When the bioassay is a dilution assay “the unknown preparation to be assayed is supposed to contain the same active principle as the standard preparation, but in a different ratio of active and inert compounds” ** Implies RP constant across concentrations  One curve is a horizontal shift of the other  i.e. ‘parallel’ curves ** European Pharmacopoeia 3.1.1

13. 13 Check for parallelism 1.Ph. Eur approach:  Residual sum of squares (RSS) and the F-test Arbitrary p value Penalises ‘good’ data 2.USP  Confidence intervals on differences between parameters Arbitrary confidence level Arbitrary limits on width Penalises ‘bad’ data 3.Others  Chi squared test Similar to (1)

14. 14 Check for parallelism EP, USP are guidance only ‘ No simple, generally applicable statistical solution exists to overcome these fundamental problems. The appropriate action has to be decided on a case-by-case basis’ USP Workshop 2008

15. 15 Estimating RP from data 1.Choose a model; fit to each material 2.Check system suitability reference is behaving as expected parallel models are appropriate 3.Calculate RP and 95% confidence interval

16. 16 Choose a model

17. 17 Choose a model

18. 18 Linear model (4 concentrations) Assume: Middle 4 concentrations of interest Parallel when β the same for both materials

19. 19 Four parameter logistic model Note: If A = 0 and B = 1 this is a simple logistic model: proportions.

20. 20 Four parameter logistic model Parallel: when A, B and scale are the same for both materials

21. 21 Five parameter logistic model Parallel: when A, B scale and asym are the same for both materials

22. 22 Which model to use?

23. 23 Which model to use? Consider the relevant range of concentrations  How much do you need to know about the ends? Pros and cons –  Need more data to fit curves  More data = more precision? Weighting / variability at ends Formal statistical tests for fit ‘Is a 5PL model really necessary or is it a statistical remedy for a bad assay?’ R Capen Chair, USP workshop 2008

24. 24 4 PL parallel model chosen

25. 25 Calculate RP and 95% CI Parallel model provides an estimate of log e RP  Horizontal distance between the curves  log e RP = 0.233 Back-transform for RP  RP = e 0.233 = 1.26 95% confidence interval for RP (1.26)  (0.84, 1.90)

26. 26 Assay development / optimisation Choose statistical model Design assay  Number of replicates per concentration  Operators, days etc  to achieve required precision for RP Set suitability criteria for assay  Reference behaviour  Parallelism

27. 27 Model selection for an assay Model must:  Fit the data  Allow RP calculation most of the time i.e. the curves are parallel  Provide precise estimates of RP

28. 28 Example with 12 plates 12 development plates run  Wide range of concentrations 0.001 – 2000 IU/ml Reference and sample 3 replicates 4 statistical models examined  Linear (4), Linear (6), 4PL, 5PL Parallelism Precision Fit

29. 29

30. 30 Summary: Parallelism test (F) ModelN failing (P < 0.05)% parallel * LM 40100% LM 6187% 4PL467% 5PL375% * Denominator = 12

31. 31 Summary: Precision

32. 32 Linear model: 4 points, parallel

33. 33 Summary: Model selection Linear model based on 4 concentrations  All 12 pairs passed linearity test  All 12 pairs passed parallelism test  Provided the best precision  No apparent bias

34. 34 Improving precision If the linear model can be justified:  Allows extra replication  Better precision within plate  ? fewer plates required  How low can you go? 2 doses: test for linearity cannot be done 3 doses: test for linearity has low power

35. 35 Summary System software provides most of the required statistics per plate When do you need a statistician? Choosing model –“Appropriate statistical analysis should be applied” –“Methods of analysis, including justification and rationale, should be described fully” Designing and validating assay –Assessing sources of variation –Simulation –Setting suitability criteria –“A relevant, validated potency assay should be part of the specifications for a biotechnological or biological drug substance and/or drug product”

36. 36 Thank you Quantics staff  Kelly Fleetwood, Catriona Keerie  Analysis and graphics

37. 37