CAMARADES: Bringing evidence to translational medicine Optimizing the Predictive Value of Pre-Clinical Research Session 3: Reviewer Perspective Malcolm Macleod Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies and Centre for Clinical Brain Sciences, University of Edinburgh

CAMARADES: Bringing evidence to translational medicine Quality is important EAE PD AD

CAMARADES: Bringing evidence to translational medicine Prevalence of reporting of some measures to improve validity Randomisation Blinded Outcome Assessment Sample Size calculation Stroke36%29%3% MND31%20%<1% AD15%25%0% PD12%15%0% EAE8%15%<1% Glioma14%0% Pain14%25%0%

CAMARADES: Bringing evidence to translational medicine Quality by Journal 4584 full publications curated on CAMARADES Reporting the efficacy of an intervention in an animal disease model Journals contributing more than 100 publications –Brain Research –Experimental Neurology –JCBFM* –Journal of Immunology –Journal of Neuroimmunology –Journal of Neuroscience –Neuroscience –PNAS –Stroke

CAMARADES: Bringing evidence to translational medicine Quality by Journal

CAMARADES: Bringing evidence to translational medicine Does high Impact Factor reflect high quality research? 563 publications in focal cerebral ischaemia Weak association between Impact Factor and quality (adjusted r 2 = 0.06) Weaker association between number of citations received by that publication and quality (adjusted r 2 = 0.004)

CAMARADES: Bringing evidence to translational medicine Change over time EAE: some improvement over time: 26 years per point increment in quality AD: some improvement over time: 24 years per point improvement in study quality Study quality of in vivo studies selected from random sample of 1000 publications from PubMed Randomisation Blinded outcome assessment Blinded conduct of experiment Concealment of allocation sequence Sample size calculation Conflict of Interest statement

CAMARADES: Bringing evidence to translational medicine

Publication bias n expts Estimated unpublished Reported efficacy Corrected efficacy Stroke – infarct volume %23.8% EAE - neurobehaviour %15.0% EAE – inflammation %37.5% EAE – demyelination %30.5% EAE – axon loss %41.7% AD – Water Maze sd0.498 sd AD – plaque burden sd0.610 sd - 32%20%

CAMARADES: Bringing evidence to translational medicine Dimensions of assessment Internal validity Efficacy Generalisability Validity Efficacy Exemplar heat map of 30 experiments testing an intervention in EAE

CAMARADES: Bringing evidence to translational medicine Evidence based translational medicine Experimental Studies Systematic review and meta-analysis how powerful is the treatment? what is the quality of evidence? what is the range of evidence? is there evidence of a publication bias? What are the conditions of maximum efficacy? Multi Centre Animal Studies confirm efficacy robust and monitored conduct of experiments transparent analysis and reporting deliberate heterogeneity Clinical trial