1 Issues in Selection of Deltas in Non-Inferiority Trials : Acute Bacterial Meningitis and Hospital- Acquired Pneumonia John H. Powers, M.D. Medical Officer Division of Special Pathogen and Immunologic Drug Products Center for Drug Evaluation and Research U.S. Food and Drug Administration

2 Introduction u Clinical perspective on delta –definition of delta and components –impact of deltas in clinical setting u Delta 1 issues in acute bacterial meningitis and HAP –data from pre-antibiotic and antibiotic eras –confounders in determining efficacy of control regimens u Delta 2 issues with acute bacterial meningitis and HAP –consequences of less effective therapy –practical issues in selecting delta

3 Clinical Trials u Purpose of clinical trials –Distinguish effects of drug from other influences »spontaneous change in course of disease »placebo effect »biased observations –difficult for clinicians to make judgments on drug efficacy/safety outside of setting of clinical trial »high spontaneous resolution rate in less serious diseases »confounding factors for lack of patient improvement in serious diseases »lack of direct comparison of safety of two drugs in similar patient population

4 Non-Inferiority Trials u Non-inferiority trials attempt to prove test drug is not inferior to control drug by some margin »cannot statistically prove two drugs are identical in efficacy »need some way to estimate the variability around the difference between two treatments u Non-inferiority margin (delta) = maximum degree of inferiority of test drug compared to control drug that trial will attempt to exclude statistically »specified prior to initiation of trial

5 Non-Inferiority Margins u After completion of trial: 1) calculate difference in point estimates of efficacy of test agent minus control agent 2) calculate 95% confidence interval around difference in point estimates »gives some idea of variability around the estimate in the differences 3) compare lower bound of 95% CI to pre-specified non-inferiority margin -20% +20% -8%-15%

6 Components of Delta u Delta 1 –conservative estimate of advantage of active control over placebo »data-based u Delta 2 –largest clinically acceptable difference between active control and experimental drug »judgement based on consequences to patients of treatment failure u overall delta for clinical trial smaller of the two values –if delta 1 is large, overall delta set by delta 2

7 Components of Delta - Delta 1 u Historically-based data –Do we really know what we think we know? »lack of data from pre-antibiotic era »change in resistance patterns and epidemiology of organisms »differing response rates in sub- populations »changes in practice of medicine »problems with defining patients with bacterial infection vs. non-bacterial/non-infectious causes »different definitions of success and failure in current trials compared to previous mortality- based trials

8 Components of Delta - Delta 2 u Judgement based “acceptable loss” relative to current therapy –ideal situation »smaller delta for more severe disease u less loss relative to current therapy given potential for greater overall morbidity/mortality »larger delta for less severe disease u greater loss relative to current therapy may not translate into as great a consequence for patients –BUT we don’t live in an ideal world »practicalities of performing clinical trials

9 Components of Delta - various diseases u Acute bacterial meningitis –  1 = magnitude of advantage over placebo well- known AND large –  2 = decision on “acceptable loss” u Hospital-acquired pneumonia –  1 = magnitude of advantage over placebo not as clear –  2 = decision on “acceptable loss” u Acute exacerbations of chronic bronchitis –  1 = advantage over placebo unclear (and small?) –  2 = decision on acceptable loss not as critical

10 Components of Delta Meningitis and HAP u Delta 1 - important questions Q1: What is the magnitude of benefit of any antibiotic therapy over placebo? Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit? Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial?

11 Components of Delta Meningitis and HAP u Delta 2 Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease? –Scientific considerations »consequences of treatment failure in various patient subsets with meningitis or HAP –Practical considerations »effect of changes in delta on sample size as efficacy rate changes

12 Historical Data - Meningitis u Acute bacterial meningitis highly lethal in pre- antibiotic era –meningococcal disease most common and occurred in previously healthy young people –overall mortality 70-90% without specific therapy –mortality decreased to 30% with introduction of antimeningococcal serum u Flexner S. J Exp Med 1913;17: –sulfanilamide treatment reduced mortality to 10% (9/11 patients survived in original series) u Schwenker F et al. JAMA 1937;108:1407-8

13 Historical Data - Meningitis u Problems with historical data –different endpoints in current trials »developmental, neurologic, audiologic sequelae as well as mortality –different epidemiology »pneumococcal meningitis most common now in U.S. –different populations »proportionately more older adults with meningitis since introduction of HIB vaccine u Schuchat A et al. N Engl J Med 1997;337:970-6.

14 Historical Data - HAP u Clinical entity of HAP not described in pre-antibiotic era –only 2 spontaneous cures out of 151 cases in military recruits in S. aureus outbreaks in 1918 –few reports of gram-negative pneumonias »How certain is diagnosis in these case reports? u No way to compare antibiotic therapy to placebo

15 u Celis R. Chest 93; –30.5% (33/108) all-cause mortality with “appropriate” antibiotics –91.6% (11/12) all-cause mortality with“inappropriate” antibiotics u Alvarez-Lerma et al. Intensive Care Med 1996;22: –16.2% (36 /146)attributable mortality with “appropriate” antibiotics »all-cause mortality 34.9% (51/146) –24.7% ( 46/284) attributable mortality with “inappropriate” antibiotics »all-cause mortality 32.4% (92/284) Historical Data - HAP

16 Historical Data - HAP u Problems with historical data –Difficulty in clinical diagnosis of HAP »patients in study who do not have disease –Change in nosocomial organisms over time »changes in resistance patterns –Different outcomes in various patient populations »mechanically ventilated pts. Vs. others –Death attributable to pneumonia vs. all-cause mortality –Clinical endpoints other than mortality in current trials

17 Components of Delta Meningitis u Delta 1 - important questions Q1: What is the magnitude of benefit of any antibiotic therapy over placebo? Appears as large as 60%-80% mortality benefit but magnitude of benefit on clinical parameters not as clear Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit? Yes and No Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial? Yes

18 Components of Delta HAP u Delta 1 - important questions Q1: What is the magnitude of benefit of any antibiotic therapy over placebo? May be anywhere from 8.5%-60% depending on how and in whom it is measured. Unclear benefit on clinical parameters Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit? Yes and No Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial? Point for committee discussion

19 Components of Delta Meningitis and HAP u Delta 2 Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease? –Scientific considerations »consequences of treatment failure in various patient subsets with HAP –Practical considerations »effect of changes in delta on sample size as efficacy rate changes

20 Consequences of Failure u Meningitis –clear mortality benefit of antibiotic therapy –morbidity is developmental, neurological and audiological sequelae »what is magnitude of benefit of antibiotics? u HAP –mortality »magnitude of benefit varies depending on how and in whom it is measured –morbidity »increased costs and hospital stay »effect on rate of clinical resolution?

21 Practical Issues u Effect of success rate and delta selection on sample size –Selection of a smaller delta in more severe diseases with relatively lower success rates would increase sample size –Is larger sample size practical given: 1) epidemiology of the disease 2) limitations of inclusion and exclusion criteria 3) inability to continue on randomized therapy in studies of severe disease

22 Clinical Trial Implications: Sample size per arm to achieve 80% power 

23 Epidemiology of Meningitis* *Based on 248 cases in 1995 from Schuchat et al. N Engl J Med. 1997;337:970-6.

24 Epidemiology of Meningitis u Case fatality rates and incidence vary by organism –H. influenzae lower case fatality rates than S. disease caused by S. pneumoniae –S. pneumoniae now more common overall –mortality rates in future trials may be higher than those in past given shift in epidemiology u Number of cases in U.S. declining since introduction of HIB vaccine –estimated 12,920 cases in 1986 –estimated 5,755 cases in 1995 »Schuchat et al. N Engl J Med 1997;337:

25 Epidemiology of HAP u Actual incidence of HAP unclear (not a reportable illness) –NNIS data estimates 250,000 cases/year in U.S. »uses clinical definition of HAP –estimated 1% of all patients entering hospital develop pneumonia –15-18% of all hospital acquired infections »2nd most common after UTI »most common infection in ICU setting u ICARE report. Am J Infect Control 1999;27:

26 Epidemiology u Estimated U.S. cases per year (1994) –acute otitis media26,000,000 –acute sinusitis23,000,000 –tonsillitis/pharyngitis21,000,00 –pneumonia (community) 4,000,000 –hospital-acquired pneumonia 250,000 –acute bacterial meningitis <10,000 –acute bacterial endocarditis 10,000

27 Recent Trials u Practical Points –success rates in HAP trials in 50% - 70% range »much larger sample size with smaller delta –recent approvals with 20% delta based on 1992 guidance in all recent HAP trials »theoretically a new drug could be as much as 20% less effective than comparator –almost half of patients do not complete trial »must take into account when planning sample size

28 Clinical Trial Implications: Sample size per arm to achieve 80% power 

29 Components of Delta Meningitis and HAP u Delta 2 Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease? »serious nature of meningitis and HAP would seem to call for selection of smaller deltas »smaller deltas would result in larger sample size of clinical trials - is this practical? »balance with risk of accepting drugs which may be 20% less effective than currently approved therapy u could be success rate of 40% for new drug for HAP

30 The Balance u Risk to patients of accepting larger deltas, especially in more severe disease versus u Realities of performing clinical trials