CAT 2: Therapy Maribeth Chitkara, MD Rachel Boykan, MD Stony Brook Long Island Children’s Hospital

Where Do We Get Evidence? Your patients Your patients Colleagues Colleagues Published anecdotal cases Published anecdotal cases Case series Case series Cohort studies Cohort studies Randomized control trials Randomized control trials Systematic review Systematic review Meta-analyses Meta-analyses

Scenario 4 month old infant w/ RSV bronchiolitis 4 month old infant w/ RSV bronchiolitis Hospitalized for 2.5 days for O2 and IVF Hospitalized for 2.5 days for O2 and IVF Family h/o asthma Family h/o asthma Parents wonder if there’s “anything out there” to prevent the infant from developing asthma Parents wonder if there’s “anything out there” to prevent the infant from developing asthma

Answerable Clinical Question PICO Patient – Infant w/ RSV bronchiolitis Patient – Infant w/ RSV bronchiolitis Intervention – Given a course of montelukast Intervention – Given a course of montelukast Comparison – Compared with infants without montelukast Comparison – Compared with infants without montelukast Outcome – Decrease the subsequent asthma illnesses? Outcome – Decrease the subsequent asthma illnesses?

The Search Pubmed search MeSH database Pubmed search MeSH database Terms “Respiratory Syncytial Virus” and “montelukast” Terms “Respiratory Syncytial Virus” and “montelukast” You find a double-blind, placebo-controlled study of infants hospitalized with RSV bronchiolitis evaluating efficacy of montelukast in reducing subsequent asthma exacerbations You find a double-blind, placebo-controlled study of infants hospitalized with RSV bronchiolitis evaluating efficacy of montelukast in reducing subsequent asthma exacerbations Bisgaard J for the Study Group on Montelukast and RSV: A Randomized Trial of Montelukast in RSV Postbronchiolitis. Am J of Respir Crit Care Med 2003; 167: Bisgaard J for the Study Group on Montelukast and RSV: A Randomized Trial of Montelukast in RSV Postbronchiolitis. Am J of Respir Crit Care Med 2003; 167:

Is the study VALID? Is the study VALID? What are the RESULTS? What are the RESULTS? Can I APPLY the results to my patient? Can I APPLY the results to my patient?

Validity Randomized trial? Randomized trial? Was Randomization concealed? Was Randomization concealed? Follow up long enough? Follow up long enough? Patients accounted for at end of trial? Patients accounted for at end of trial? Were patients, health care workers, data collectors, and/or data analysts blinded to treatment? Were patients, health care workers, data collectors, and/or data analysts blinded to treatment?

Validity Were both groups similar at start of trial? Were both groups similar at start of trial? Aside from the treatment itself, were both groups treated equally? Aside from the treatment itself, were both groups treated equally? Did authors utilize an “Intention to Treat” analysis? Did authors utilize an “Intention to Treat” analysis?

Intention to Treat The principle of attributing all patients to the group to which they were randomized The principle of attributing all patients to the group to which they were randomized Preserves the value of randomization Preserves the value of randomization Considers the “worst case scenario” Considers the “worst case scenario” Scenario 1: Lost experimental subjects fare like control subjects, lost control subjects fare like experimental subjects Scenario 1: Lost experimental subjects fare like control subjects, lost control subjects fare like experimental subjects Scenario 2: Lost experimental subjects do poorly, lost control subjects do well Scenario 2: Lost experimental subjects do poorly, lost control subjects do well Scenario 3: Lost subjects in both groups do poorly Scenario 3: Lost subjects in both groups do poorly

Per Protocol Analyses Only patients who completed research protocol in entirety are included in final data analysis Only patients who completed research protocol in entirety are included in final data analysis Excludes all non-compliant patients Excludes all non-compliant patients Leaves behind those who may be destined to have a better outcome Leaves behind those who may be destined to have a better outcome Has the potential to severely bias the findings in the study or negate the value of randomization Has the potential to severely bias the findings in the study or negate the value of randomization

Therapy – Results (per protocol) Outcome Present Outcome Absent Totals New Drug/tx ABA + B PlaceboCDC + D TotalsA + CB + DA + B + C + D Control Event Rate (CER) C / C + D Experimental Event Rate (EER) A / A + B Absolute Risk Reduction (ARR) CER - EER Relative Risk (RR) EER / CER Relative Risk Reduction (RRR) RRR = [1-RR] x 100 RRR = [1-RR] x 100 Number Needed to Treat (NNT) 1 / ARR

Scenario RADNo RADTotalLost to Follow UP Montelukast Placebo Total CER = 10/55= 0.18 CER = 10/55= 0.18 EER = 4/61 = 0.07 EER = 4/61 = 0.07 ARR = = 0.11 ARR = = 0.11 RR = 0.07/0.18 = 0.38 RR = 0.07/0.18 = 0.38 RRR = [1-0.38] x 100=62% RRR = [1-0.38] x 100=62% NNT = 1/0.11= 9 NNT = 1/0.11= 9 Bisgaard J for the Study Group on Montelukast and RSV: A Randomized Trial of Montelukast in RSV Postbronchiolitis. Am J of Respir Crit Care Med 2003; 167:

Per Protocol vs ITT Per protocol Per protocol ARR = 0.11 ARR = 0.11 NNT = 9 NNT = 9 ITT (Scenario 1) ARR = 0.9 NNT = 11 ITT (Scenario 2) ARR = 0.3 NNT = 32 ITT (Scenario 3) ARR = 0.18 NNT = 5

Number Needed to Treat (NNT) Your personal treatment threshold Your personal treatment threshold I would be willing to treat… I would be willing to treat… 3 patients to see benefit in one 3 patients to see benefit in one 5 patients to see benefit in one 5 patients to see benefit in one 10 patients to see benefit in one 10 patients to see benefit in one 100 patients to see benefit in one 100 patients to see benefit in one

Results How precise was the estimate of treatment effect? How precise was the estimate of treatment effect? Calculated results are point estimates Calculated results are point estimates True result is somewhere within the 95% CONFIDENCE INTERVAL (CI) True result is somewhere within the 95% CONFIDENCE INTERVAL (CI) Tighter the CI, the more likely the calculated numbers are near the truth Tighter the CI, the more likely the calculated numbers are near the truth Larger sample size  larger the number of outcome events  greater confidence that the true risk is close to what we have observed Larger sample size  larger the number of outcome events  greater confidence that the true risk is close to what we have observed

95% Confidence Intervals for the ARR If the value “O” lies in your 95% CI for your ARR, your result is NOT significant…there may be no difference between treatment and placebo If the value “O” lies in your 95% CI for your ARR, your result is NOT significant…there may be no difference between treatment and placebo If your 95% CI includes a negative number, your patient may get worse with treatment…STOP If your 95% CI includes a negative number, your patient may get worse with treatment…STOP

Applicability Applicability Are the results applicable to my patient? Are the results applicable to my patient? Is our patient so different from those in the study that its results cannot apply? Is our patient so different from those in the study that its results cannot apply? Is the treatment feasible in our setting? Is the treatment feasible in our setting? What are our patient’s potential benefits and harms from the therapy? What are our patient’s potential benefits and harms from the therapy? What are our patient’s values and expectations for both the outcome we are trying to prevent, and the treatment we are offering? What are our patient’s values and expectations for both the outcome we are trying to prevent, and the treatment we are offering?

Applicability-Scenario Do the parents in the scenario have a personal experience with terrible asthma and would be willing to try anything? Do the parents in the scenario have a personal experience with terrible asthma and would be willing to try anything? Is this therapy too expensive for them to consider? Is this therapy too expensive for them to consider? Does the NNT of 9 influence your or the parents’ decision about this medication? Does the NNT of 9 influence your or the parents’ decision about this medication? What if the NNT were 2? What if the NNT were 2?

Applicability- Bottom Line Is this research highly valid, the best there is, and should influence practice? Is this research highly valid, the best there is, and should influence practice? Are there some issues with validity, but the best we have for now and thus should not dismiss altogether? Are there some issues with validity, but the best we have for now and thus should not dismiss altogether? Is it so flawed that we should not consider this new treatment, or practice without evidence beyond experience? Is it so flawed that we should not consider this new treatment, or practice without evidence beyond experience?

Practice Case References Case 1: Case 1: Cohen HA, Varsano I, Kahan E, Sarrell EM, Uziel Y. Effectiveness of an Herbal Preparation Containing Echinacea, Propolis, and Vitamin C in Preventing Respiratory Tract Infections in Children. Arch Pediatr Adolesc Med, 2004; 158: Case 2: Freedman SB, Adler M, Seshadri R, Powell EC. Oral Ondansetram for Gastroenteritis in a Pediatric Emergency Department. N Engl J Med, 2006; 354: Case 3: Bauchner H, Vinci R, Bak S, Pearson C, Corwin M. Parents and Procedures: A Randomized Controlled Trial. Pediatrics, 1996; 98:

EBM References/Resources Guyatt, G., Rennie, D., Meade, M.O., & Cook, D.J. (2008) Users' guides to the medical literature: A manual for evidence-based clinical practice (2nd ed.). New York: McGraw-Hill Medical. Online EBM Calculator: Online EBM Calculator: