Trial Data Must Be Accessible: Why Collecting the Same Outcomes Isn’t Enough Outcomes in Trials and Systematic Reviews: Why We Should be Paying More Attention Nicole Fusco Society for Clinical Trials & International Clinical Trials Methods Conference Liverpool, UK May 8, 2017

Declaration of interests May 2014-Oct. 2016: PCORI ME-1303-5785 Nov. 2016-Present: Thomas Greene fund for research scholarship

Trial information and data are available from multiple sources Public Sources Non-Public Sources Journal Articles Clinical Study Reports Conference Abstracts Information from Regulators Individual Patient Data Trial Registries

Five elements of a completely specified outcome Time-Points Domain e.g., 1 month e.g., Pain intensity Method of Aggregation Specific measurement IV II e.g., 0-10 scale e.g., Mean Specific Metric e.g., Value at a time-point III Saldanha IJ, et al. PLoS One 2014

Few domains, large number of outcomes 4 domains  64 defined outcomes Mayo-Wilson E, et al. Submitted 2017

Example of multiple outcome definitions With just 4 pre-specified outcome domains, we observed 214 defined effectiveness outcomes across 21 trials 21 Trials Mayo-Wilson E, et al. Submitted 2017

Trials reported different defined outcomes No outcomes reported in all trials 116/214 (54%) outcomes reported in only one trial

Effectiveness outcomes in public vs. non-public sources We had non-public sources for 6/21 trials More than half (110/214) of effectiveness outcomes appeared only in non-public sources

Adverse events in clinical trials

Adverse events in public vs. non-public sources We had non-public sources for 6/21 trials Most adverse events (83%) appeared only in non-public sources

Few AEs are reported in public sources, compared with non-public sources This shows only 6 trials – we assume that the other trials are similar. Public sources for other trials report about the same number of AEs, so we assume that there are AEs unreported for those trials * *Trial not published

Few trials report number of patients experiencing any AE 3 trials: AE data only in public sources Using only public sources, we have data from only 5 trials 21 trials testing gabapentin for neuropathic pain 10 trials: reported AE data 9 trials: reported meta-analyzable AE data 2 trials: AE data in both public and non-public sources Make sure to talk about the so what!!!! Before MA box, remind that we want to combine trials to inform policy 11 trials: did not report AE data 1 trial: did not report meta-analyzable AE data 4 trials: AE data only in non-public sources

Conclusions Effectiveness outcomes Hundreds of different outcomes reported 4 domains  214 defined outcomes Trials don’t report overlapping outcomes Many outcomes were not reported in public sources Adverse events Hundreds of different adverse events reported Most adverse events were not reported in public sources This is a problem in general – not just for this case example

Multiple Data Sources (MUDS) Team Steering Committee Dickersin, Kay - PI (KD) Fusco, Nicole (NF) Li, Tianjing (TL) Mayo-Wilson, Evan (EMW) Tolbert, Elizabeth (ET) Conception & design, funding Doshi, Peter (PD) Vedula, Swaroop (SV) KD, TL Protocol development, study implementation Cowley, Terrie (TC) Haythornthwaite, Jennifer (JH) Hong, Hwanhee Payne, Jennifer (JP) Singh, Sonal (SS) Stuart, Elizabeth (ES) EMW, KD, TL, NF, ET, JE Data acquisition Bertizzolo, Lorenzo (LB) Ehmsen, Jeffery (JE) Gresham, Gillian (GG) Heyward, James (JHe) Lock, Diana (DL) Rosman, Lori (LR) Suarez-Cuervo, Catalina (CS) Twose, Claire (CT) KD, NF, EMW, TL, SV Analysis and interpretation of data Canner, Joseph (JC) Guo, Nan (NG) Hong Hwanhee (HH) Stuart, Elizabeth (ES) NF, EMW, KD, TL Systematic Review Data Repository Jap, Jens (JJ) Lau, Joseph (JL) Smith, Bryant (BS) Ancillary studies Golozar, Asieh (AG) Hutfless, Susie (SH) EMW, KD, TC

Thank you!

References Saldanha, Ian J., et al. "Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability." PloS one 9.10 (2014): e109400.