1. Statistical Methods in the Evaluation of Red Blood Cell Products (In vivo study) Jessica Kim, Ph.D. Mathematical Statistician FDA/CBER/OBE/DB Blood Products Advisory Committee 91st Meeting, May 1, 2008 Rockville Pike, Rockville, MD

2. May 2008 BPAC2 Outline Background Current recommendations for the RBC products: Statistical methods –Emphasis on the individual viability –Statistical power of a study BEST (Biomedical Excellence for Safer Transfusion) Collaborative study on RBC recovery FDA’s analysis of BEST/FDA data Summary

3. May 2008 BPAC3 Background PeriodFDA acceptance criteria for RBC products in vivo study ~199775% RBC survival 1998~ 2003 Mean RBC survival ≥ 75% Standard deviation ≤ 9% At least 20 units at 2 sites 2004~One sided lower 95% CI for the population proportion of successes > 70% (A success: RBC survival ≥75%) CI: Confidence Interval

4. May 2008 BPAC4 FDA Current Acceptance Criteria: in vivo 24-hour RBC survival study A one-sided 95% lower confidence limit for the population proportion of successes is > 70% (95-70 Rule) “Success” : in vivo RBC survival ≥ 75% The sample mean survival is ≥ 75% The sample standard deviation is ≤ 9% At least 20 units in total/At least 2 sites

5. May 2008 BPAC5 In Vivo Study: Study Hypothesis A one-sided 95% lower confidence limit for the population proportion of successes is > 70% (“Success” : in vivo RBC survival ≥ 75% ) Null hypothesis: Population proportion of successes  70% Alternative hypothesis: Population proportion of successes > 70%

6. May 2008 BPAC6 Study Hypothesis (cont.) Based on the BEST data In vivo RBC Survival % Proportion of successes BEST: Biomedical Excellence for Safer Transfusion

7. May 2008 BPAC7 Sample Size to Meet Minimum Acceptable Proportion of Successes Observed proportion of successes # of allowable failures One-sided lower 95% CI for the population proportion of successes 21/24 (87.5%)370.8% 24/28 (85.7%)470.2% 28/33 (84.9%)570.8% 31/37 (83.8%)670.5% 35/42 (83.3%)771.0%

8. May 2008 BPAC8 Sample Size and Statistical Power Statistical Power –The likelihood of achieving a statistically significant result if your research hypothesis is actually true. –If the population proportion of successes is greater than 70%, what is the probability that a RBC recovery study will meet the acceptance criteria? Sample Size and Statistical Power –If the likelihood is good (e.g., at least 80%), then your sample size would be considered adequate.

9. May 2008 BPAC9 Power Consideration True Rate Sample Size (# of allowable failures) 9 (0)14 (1)19 (2)24 (3)28 (4)33 (5) 0.750.0750.1010.1110.1150.1350.132 0.80.1340.1980.2370.2640.3150.329 0.850.2320.3570.4410.5050.5870.626 0.90.3870.5850.7050.7860.8580.894 0.950.6300.8470.9330.9700.9890.995

10. May 2008 BPAC10 Power Consideration (cont.) P: true rate N=24 N=28 N=33

11. May 2008 BPAC11 BEST Study Results of RBC Recovery RBC Type: 42d Liquid Stored RBC Type NRBC Recovery (%) Frequency of RBC Recovery  75% Frequency of RBC Recovery  70% Frequency of RBC Recovery  67% Mean (%) SD (%) Mini mum Lower 5% Quantile Med- ian 42d Liquid Stored 6413670.782.488.3%95.5%98.1%82.16.71 Source: This table is quoted from Dr. Dumont’s paper

12. May 2008 BPAC12 BEST Study Results of RBC Recovery RBC Type: 42d Liquid Stored (cont.) Recovery Success Threshold Estimated “true” rate from the BEST data Probability 1 meet FDA current criteria (95-70 rule): n=24  75% 88.3%0.693  70% 95.5%0.979  67% 98.1%0.999 1 Binomial probability with at most 3 failures out of 24 Source: This table is quoted from Dr. Dumont’s paper

13. May 2008 BPAC13 BEST Data # of studies meeting old criteria vs. current criteria Yearold criteria current criteria 199010 199110 199210 199411 199521 199622 199932 Yearold criteria current criteria 200022 200121 200222 200322 200411 200522 200611 Note: Four additional approved studies (2004~2007) will be added in the FDA’s analysis

14. May 2008 BPAC14 BEST and FDA Data RBC Survivals by Year-Study FDA data: 2004.83, 2007.57, 2007.61, and 2007.74 Horizontal line indicates cutting value for an individual unit to be considered as a success:75%

15. May 2008 BPAC15 BEST and FDA Data Proportion of Successes Studies didn’t meet the current criteria

16. May 2008 BPAC16 Statistical Power Consideration n=735 (BEST:641 + FDA:94) combined BEST & FDA data set analyzed by FDA True Rate Using BEST /FDA data Sample Size (# of allowable failures) 20 (2)24 (3)28 (4)33 (5) 0.888 (≥ 75%) 0.608 0.7210.801 0.842 0.901 (≥ 74%) 0.683 0.791 0.862 0.898 0.927 (≥ 73%) 0.824 0.906 0.950 0.970

17. May 2008 BPAC17 1990~1997 (n=140) 1998~2007 (n=549) success rate power n=24 success rate power n=24 RBC%≥75%0.8360.430.8980.78 RBC%≥74%0.8640.580.9070.82 RBC%≥73%0.8860.710.9330.93 # of studies meeting current criteria 4 out of 817 out of 19 BEST and FDA Data combined data set analyzed in different time periods Note: study 40 and 41 were not included (46 data points) due to unknown approval year

18. May 2008 BPAC18 BEST and FDA Data combined data set analyzed in different time periods (cont.)

19. May 2008 BPAC19 1990~2003 (n=516) 2004~2007 (n=173) success rate power n=24 success rate power n=24 RBC%≥75%0.8700.620.9310.92 RBC%≥74%0.8840.700.9420.95 RBC%≥73%0.9150.860.9480.97 # of studies meeting current criteria 13 out of 198 out of 8 BEST and FDA Data combined data set analyzed in different time periods (cont.) Note: study 40 and 41 were not included (46 data points) due to unknown approval year

20. May 2008 BPAC20 BEST and FDA Data combined data set analyzed in different time periods (cont.)

21. May 2008 BPAC21 Threshold value of RBC recovery 1990~1997 (n = 140) 1998~2003 (n = 376) 2004~2007 (n = 173) Success Rate Power N = 24 Success Rate Power N = 24 Success Rate Power N = 24 ≥ 75%0.8360.430.8830.690.9310.92 ≥ 74%0.8640.580.8910.740.9420.95 ≥ 73%0.8860.710.9260.900.9480.97 # of studies meeting current criteria 4 of 89 of 118 of 8 Statistical Power Consideration n=689 (595 + 94) from 1990 to 2007 combined data set analyzed in different time periods Note: study 40 and 41 were not included (46 data points) due to unknown approval year

22. May 2008 BPAC22 BEST and FDA Data combined data set analyzed in different time periods (cont.)

23. May 2008 BPAC23 Summary Clinical significance to ensure that each recipient receives a high viable RBC product should be emphasized The power to meet FDA’s “95-70 rule” depends on the study size as well as an estimate for the population proportion –Increase study size –Estimate “true” rate under current acceptance criteria /or under more recent time period 80% power has been used for a typical clinical trial

24. May 2008 BPAC24 Summary (cont.) The data show that manufacturers are able to produce better products over time Current acceptance criteria serve well for the purpose of regulating high viable RBC products