1. A Bayesian Approach to Parallelism Testing in Bioassay
Steven Novick, GlaxoSmithKline
Harry Yang, MedImmune LLC

2. John Peterson, Director of statistics, GlaxoSmithKline
Manuscript co-author
John Peterson, Director of statistics, GlaxoSmithKline

4. Warm up exercise

6. When are two lines parallel?
Parallel: Being everywhere equidistant and not intersecting
Slope
Horizontal shift places one line on top of the other.

7. When are two curves parallel?
Parallel: Being everywhere equidistant and not intersecting?
Can you tell by checking model parameters?
Horizontal shift places one curve on top of the other.

8. Where is parallelism important?
Gottschalk and Dunn (2005)
Determine if biological response(s) to two substances are similar
Determine if two different biological environments will give similar dose–response curves to the same substance.
Compound screening
Assay development / optimization
Bioassay standard curve

9. Screening for compound similar to “gold-standard”
E.g., seeking new HIV compound with AZT-like efficacy, but different viral-mutation profile.
Desire for dose-response curves to be parallel.
Parallel indicates a change in potency only.

10. Change to assay procedure
E.g., change from fresh to frozen cells.
Want to provide same assay signal window.
Desire for control curves to be parallel.
Fresh
Frozen
Day Day Day 10
CD8 T cells, HIV ELISPOT assay

11. Assess validity of bioassay used for relative potency
Dilution must be parallel to original.
Callahan and Sajjadi (2003)
Original
Dilution

12. Replacing biological materials used in standard curve
E.g., ELISA (enzyme-linked immunosorbent) assay to measure protein expression.
Recombinant proteins used to make standard curve.
Testing clinical sample
New lot of recombinant proteins for standards.
Check curves are parallel
Calibrate new curve to match the old curve.

13. Potency often determined relative to a reference standard such as ratio of EC50
Only meaningful if test sample behaves as a dilution or concentration of reference standard
Testing parallelism is required by revised USP Chapter <111> and European Pharmacopeia

14. Linear: Two lots of Protein “A”
Estimated Concentrations
Lot 2 is 1.4-fold higher than Lot 1
Log10 Signal
=0.14

15. If the lines are parallel…
Shift “Lot 2” line to the left by a calibration constant .
 is log relative potency of Lot 2.
USP = United States Pharmacopeia
USP sets standards for the quality, purity, strength, and consistency of products–critical to the public health.
Draft USP <1034> 2010

16. Testing for Parallelism in bioassay

17. Typical experimental design
Serial dilutions of each lot
Several replicates
Fit on single plate (no plate effect)
Log10 Signal

18. Tests for parallel curves Linear model
Hauck et al, 2005; Gottschalk and Dunn, 2005
H0: b1 = b2
H1: b1 ≠ b2
ANOVA:
T-test
F goodness of fit test
2 goodness of fit test
May lack power with small sample size
Might be too powerful for large sample size

19. A better idea Callahan and Sajjadi 2003; Hauck et al. 2005
Slopes are equivalent
H0: | b1 − b2 |≥ 
H1: | b1 − b2 |< 

20. Nonlinear: Two lots of protein “B”
Estimated Concentrations
Lot 2 is 1.6-fold higher than Lot 1
Depending on the response to back-calculate, the fold-change between lots 1 and 2 can range from as little as 1.5-fold to as much as 5.5-fold.
=0.21

21. Tests for parallel curves 4-parameter logistic (FPL) model
Jonkman and Sidik 2009
F-test goodness of fit statistic
H0: A1 = A2 and B1 = B2 and D1 = D2
H1: At least one parameter not equal
May lack power with small sample size
Might be too powerful for large sample size

22. Calahan and Sajjadi 2003; Hauck et al. 2005; Jonkman and Sidik (2009)
Equivalence test for each parameter = intersection-union test
H0: |1 / 2|  1 or |1 / 2|  2
H1: 1 <|1 / 2| < 2
i = Ai, Bi, Di , i=1,2
Equiv. of params does not provide assurance of parallelism (except for linear)
May lack power with small sample size
Forces a hyper-rectangular acceptance region

23. “Parallel Equivalence”
Our proposal
“Parallel Equivalence”

24. Definition of Parallel
Two curves and are parallel if there exists a real number ρ such that for all x.

25. Definition of Parallel Equivalence
Two curves and are parallel equivalent if there exists a real number ρ such that
for all x  [xL, xU].

26. It follows that two curves are parallel equivalent if there exists a real number ρ such that
It also follows that

27. Are these two lines parallel enough when xL < x < xU ?
< ?

28. Linear-model solution
Just check the endpoints

29. Parallel equivalence = slope equivalence
wlog
Same as testing: | b1 − b2 |< 

30. Parallel Equivalence FPL model: No closed-form solution.
Simple two-dimensional minimax procedure.

31. Are these two curves parallel enough when xL < x < xU ?
< ?

32. Testing for parallel equivalence
H0:
H1:
Proposed metric (Bayesian posterior probability):

33. Computing the Bayesian posterior probability
For each curve, assume
Data distribution:
i =1, 2 = reference or sample
j = 1, 2, …, N = observations
Prior distribution:
Posterior distribution proportional to:

34. Draw a random sample of the i of size K from the posterior distribution (e.g., using WinBugs).
The posterior probability
is estimated by the proportion (out of K) that the posterior distribution of:

35. = 0.07  90% probability to call parallel equivalent
If we examine the difference in estimated concentration for lot 2 from the x-range, we get a delta.x = > 1.17-fold maximum shift in estimated conc. Frequentist model point-estimate maximum shift = 1.07-fold.
= (100.14=1.4-fold shift)
= 0.07  90% probability to call parallel equivalent

36. = 0.52  90% probability to call parallel equivalent
If we examine the difference in estimated concentration for lot 2 from the x-range, we get a delta.x = > 2.15-fold maximum shift in estimated conc. Frequentist model point-estimate maximum shift = 1.67-fold.
= 0.33 (100.33=2.15-fold shift)
= 0.52  90% probability to call parallel equivalent

37. Simulation: FPL Model Based on protein “B” data

38. Simulation: FPL model Similar to Protein “B” protein-chip data.
Concentrations (9-point curve + 0):
0, 102(=100), , , …, 106.5(=3,200,000)
Three replicates
xL = 3.5=log10(3162) & xU = 5=log10(100,000)
δ = 0.2.
 = 0.02, 0.04, 0.11, and (%CV = 5, 10, 25, 50)
For each Monte Carlo run, I computed:

39. Scenario
Original
New
Max Diff A1 B1 C1 D1 A2 B2 C2 D2 1 2
4.5 4 -1
4.7 3
2.445
4.945
0.15
-1.312 5
2.61
5.11
=0.20 6
2.25
4.75
-1.5
0.30
5,000 Monte Carlo Replicates

40. Example data (CV=10%)
Diff:

41. Diff:

42. Frequentist statistical power
Scenario
Frequentist statistical power
Max Diff
%CV=5
%CV=10
%CV=25
%CV=50 1
1.00
0.50 2 3
0.15 (shape 1)
0.99
0.28
< 0.01 4
0.15 (shape 2)
0.88
0.36
0.14 5
δ = 0.20
0.10
0.02 6
0.30
0.00

43. Summary Straight-forward and simple test method to assess parallelism.
Yields the log-relative potency factor.
Easily extended.

44. Extensions Instead of f(, x), could use
f(, x) /  x = instantaneous slope
f-1(, y) = estimated concentrations

45. What’s next? Head-to-head comparison with existing methods
Choosing test level and , possibly based on ROC curve? – Harry Yang paper
Guidance for prior distribution of 1 and 2.

46. References
Callahan, J. D. and Sajjadi, N. C. (2003), “Testing the Null Hypothesis for a Specified Difference - The Right Way to Test for Parallelism”, Bioprocessing Journal Mar/Apr 1-6.
Gottschalk P.J. and Dunn J.R. (2005), “Measuring Parallelism, Linearity, and Relative Potency in Bioassay and Immunoassay Data”, Journal of Biopharmaceutical Statistics, 15: 3,
Hauck W.W., Capen R.C., Callahan J.D., Muth J.E.D., Hsu H., Lansky D., Sajjadi N.C., Seaver S.S., Singer R.R. and Weisman D. (2005), “Assessing parallelism prior to determining relative potency”, Journal of pharmaceutical science and technology, 59,
Jonkman J and Sidik K (2009), “Equivalence Testing for Parallelism in the Four- Parameter Logistic Model”, Journal of Biopharmaceutical Statistics, 19: 5,

47. Thank you!
Questions?