Study Designs for Acute Otitis Media: What can each design tell us? C. George Rochester, Ph.D. Anti-Infective Advisory Committee Meeting, July 11, 2002 Mathematical Statistician, Division of Biometrics III

Anti-Infective AC Meeting 7/11/022 Outline The role of tympanocentesis (TAP) in AOM trials Advantages and disadvantages of each design Superiority design: placebo-controlled (PCT) Non-inferiority (NI)

Anti-Infective AC Meeting 7/11/023 Population Studied AOM represents a spectrum of illness Proof of efficacy include Clinical proof Microbiologic proof Cautions: Guard against post-hoc subset analyses as proof Extrapolation to populations not directly studied

Anti-Infective AC Meeting 7/11/024 Current state of affairs Clinical only, comparative, non-inferiority study: new versus standard Reference to Non-inferiority design (R. Dagan, Nov 2001, AIAC): “…most of the acute otitis media trials with clinical outcome as currently conducted are virtually guaranteed to show no differences between agents, dosing, or duration of treatment.” Baseline bacteriology and clinical outcome - often non-comparative

Anti-Infective AC Meeting 7/11/025 Why do some trials fail to detect differences among treatments? Differences among arms may truly not exist “Noise” usually makes arms converge Sources of noise in AOM studies Enrollment of subjects without bacterial infection at baseline (viral infection), loose case definition Spontaneous resolution even with bacterial infection and varies with different organisms Determination of treatment response includes subjective components which may be subject to inter-rater variability Strategies for handling noise: Placebo controlled trials: if difference is observed a treatment benefit is established Non-Inferiority trial: Baseline TAP reduce noise in diagnosis; Repeat TAPS reduce noise in outcome assessment

Anti-Infective AC Meeting 7/11/026 Should TAPS be performed? Placebo-controlled trials Clear evidence of clinical benefit TAPS add efficiency Baseline TAP Improves NI design where “noise” may lead to false proof of efficacy Follow-up TAP Bacteriological outcome is more objective The optimal time and number of TAPS to perform may need further research On-therapy (e.g. Day 3-5, or 4-6) TAP all failures

Anti-Infective AC Meeting 7/11/027 Single TAP at baseline Bacteriologic diagnosis and clinical outcome assessment Baseline TAP ensures that patients in the primary analysis have baseline pathogens Better than no TAPS but bacteriologic outcome is presumptive In practice failures do not usually get follow-up TAPS regardless of protocol specification NI with baseline TAP May allow a wider non-inferiority margin (smaller sample size) than NI without baseline TAP

Anti-Infective AC Meeting 7/11/028 Repeat TAPS Provides objective bacteriologic outcome Blinding not as critical for bacteriologic endpoint but is essential to reduce bias if the clinical outcome is the ultimate goal Study is successful if efficacy is shown at microbiologic and clinical assessment time points

Anti-Infective AC Meeting 7/11/029 Fundamental question regarding the utility of a microbiologic endpoint Bacteriologic endpoint is a “surrogate” and correlation with clinical endpoint may be less than satisfactory given current data Much uncertainty about the microbiologic endpoint remains

Anti-Infective AC Meeting 7/11/0210 Agency for Health Research Quality Evidence Report (2001): Management of Acute Otitis Media “There is still a need to adequately address the role of antibiotics in the initial treatment of AOM in children compared to placebo or observational treatment especially in terms of various influencing factors such as age and otitis-prone status. Close monitoring of patients in these studies with a priori plans for appropriate intervention should allay any concerns about suppurative complications and should also be a focus of research.”

Anti-Infective AC Meeting 7/11/0211 Randomized, double-blind, placebo-controlled trials PCT is the GOLD STANDARD Efficient and easy to interpret: direct evidence Three arm trials - New drug, standard, placebo Blinding (double): reduce experimental bias during study conduct Randomization: ensure balance in treatment groups for known and unknown factors Placebo: direct estimate of treatment benefit Provides scientific foundation on which to plan future active-controlled trials

Anti-Infective AC Meeting 7/11/0212 Advantages and disadvantages of placebo-controlled trials AdvantagesDisadvantages Provide clear evidence of clinical benefit One treatment group is untreated* If no TAPS additional microbiologic data would be necessary TAPS improve efficiency and provide direct bacteriologic information Smaller sample size than Non-inferiority design

Anti-Infective AC Meeting 7/11/0213 Non-inferiority trials Compare new drug against a “standard” Estimate of benefit of new drug depends intricately upon knowing the benefit of the standard over placebo Efficacy is indirect (relative) and relies on choice of an appropriate non-inferiority margin (delta) Efficacy is demonstrated if we are certain that benefit of active control over placebo on primary endpoint is greater than the non-inferiority margin  in the population studied Choice of non-inferiority margin also depends on microbiologic rigor

Anti-Infective AC Meeting 7/11/0214 Non-inferiority design without TAPS AdvantagesDisadvantages Acceptability to patients: all get treated Bacteriologic infection not clearly established at baseline Provides comparative clinical information Over time standard may not retain the magnitude of initial benefit No assurance new drug could beat placebo Does not provide bacteriologic information

Anti-Infective AC Meeting 7/11/0215 Non-inferiority design with baseline TAP AdvantagesDisadvantages Microbiologic diagnosis is certain Delta is easier to define - reduce noise Microbiologic outcome is presumptive Determination of efficacy is indirect- relies on clinical judgment (subjective) Smaller sample size than NI without TAP

Anti-Infective AC Meeting 7/11/0216 Non-inferiority design with repeat TAPS Microbiologic endpoint is a surrogate for the ultimate clinical outcome and may not be highly predictive There may be ethical and practical limitations Assess both endpoints Delta micro (DM) Delta clinical (DC) Micro endpoint objective DisadvantagesAdvantages

Anti-Infective AC Meeting 7/11/0217 Which design to use? To demonstrate absolute efficacy: PCT To demonstrate absolute and relative efficacy: PCT with 3 arms If the magnitude of the advantage of the active control over placebo is known for the primary endpoint: NI design (ICH E-10) NI margin must be less than this advantage Be conservative if historical data is poor Be conservative if prior information is not relevant to current population/endpoint

Anti-Infective AC Meeting 7/11/0218 What does each design tell us?

Anti-Infective AC Meeting 7/11/0219 Summary TAPS improve the efficiency of AOM trials Repeat TAPS provide objective microbiologic information Placebo-controlled trials Efficient and easy to interpret NI design should be considered when The benefit of the standard over placebo is known Microbiologic rigor can improve the quality of these trials Is the microbiologic or the clinical endpoint more desirable to patients?

Anti-Infective AC Meeting 7/11/0220 Credits Erica Brittain, Ph.D. Daphne Lin, Ph.D. Ruthanna Davi, M.S. Karen Higgins, Sc.D. C. George Rochester, Ph.D. Clinical reviewing divisions : Division of Anti-Infective Drug Products- HFD-520 Division of Special Pathogen & Immunologic Drug Products- HFD-590