M.Res Programme CRITICAL APPRAISAL Bob Lightowlers, WT Centre for Mitochondrial Research

NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!!

‘The statistical error that keeps on coming’ Ben Goldacre – The Guardian 9 th Sept 2011

Numbers of mice not likely to generate significance No randomisation No blinded evaluations Bias in reporting (no reporting of negative data) Variable genetic background Broader endpoints such as life expectancy/behaviour too crude Problem transferring ‘effective’ drugs from mouse model to clinic Nonhuman primate models ?

Sept 2011 Bayer Pharmaceuticals assessed reproducibility of 67 published projects Only 20-25% of published data in line with in-house findings

Jan 2012 Amgen tried to confirm research from 53 published ‘landmark’ studies

Jan 2012 Amgen tried to confirm research from 53 published ‘landmark’ studies ONLY 6 (11%) could be confirmed

Jan 2012 Amgen tried to confirm research from 53 published ‘landmark’ studies ONLY 6 (11%) could be confirmed

What causes this lack of reproducibility ?

NOT EVERYTHING THAT IS PUBLISHED IS CORRECT!! ONLY 15% OF PUBLICATIONS ARE TRUSTWORTHY Be critical Guilty until proven innocent

The peer review process

What should we be looking for ? Incorrect statistical analysis Power of study Absence of essential controls Incorrect methodology Over/mis interpretation of data Lack of reference to any conflicting data

What should we be looking for ? Incorrect statistical analysis Power of study Absence of essential controls Incorrect methodology Over/mis interpretation of data Lack of reference to any conflicting data Falsification of data

Somatic Cell Nuclear Transfer

Peer review cannot guarantee scientific integrity

Critical Evaluation - a worked example

Human mtDNA An autosomally replicating genome Found in mitochondrial matrix Comprises app. 0.1% of total cell DNA Varies enormously in copy number/cell Approx. 700 in fibroblasts to >200,000 in some mammalian oocytes Maternally inherited Often heteroplasmic in the diseased state 16,569 bp Encodes 13 proteins, all of which are OXPHOS components

Hypothesis: 1. Alzheimers Disease could be caused by defects in activity of the respiratory chain complex cytochrome c oxidase 2. Alzheimers Disease is due to mutations in the mitochondrial genome

Why ? Lack of FH is a negative risk factor

Why ? Lack of FH is a negative risk factor Risk of AD increases with affected maternal relative (mtDNA?)

Why ? Lack of FH is a negative risk factor Risk of AD increases with affected maternal relative (mtDNA?) Mutations in mtDNA can lead to defective OXPHOS

Why ? Lack of FH is a negative risk factor Risk of AD increases with affected maternal relative (mtDNA?) Mutations in mtDNA can lead to defective OXPHOS Neurons may be particularly susceptible to such defects

Diabetes Thyroid Disease Myopathy Peripheral Neuropathy Deafness CVA / Seizures / Developmental delay Respiratory Failure Optic Atrophy / Retinitis Pigmentosa Cardiomyopathy Short Stature Marrow Failure Liver Failure Neurological Non-Neurological

Why ? Lack of FH is a negative risk factor Risk of AD increases with affected maternal relative (mtDNA?) Mutations in mtDNA can lead to defective OXPHOS Neurons may be particularly susceptible to such defects COX activity reported to decrease in brain of AD patients

Methods used MtDNA isolation and sequencing in patients, asymptomatic relatives and controls

Methods used MtDNA isolation and sequencing in patients, asymptomatic relatives and controls All three COX genes sequenced

Methods used MtDNA isolation and sequencing in patients, asymptomatic relatives and controls All three COX genes sequenced Platelet fusion from AD patients to neuronal cells lacking mtDNA (rho0)

Generation of transmitochondrial cybrids Biopsy EthBr Enucleation

Methods used MtDNA isolation and sequencing in patients, asymptomatic relatives and controls All three COX genes sequenced Platelet fusion from AD patients to neuronal cells lacking mtDNA (rho0) Analysis of respiratory enzyme activity in the cybrids

Methods used MtDNA isolation and sequencing in patients, asymptomatic relatives and controls All three COX genes sequenced Platelet fusion from AD patients to neuronal cells lacking mtDNA (rho0) Analysis of respiratory enzyme activity in the cybrids Analysis of ROS production in cybrids

Results 506 Patients and 95 controls

Results 506 Patients and 95 controls 10 clones of all three COX genes sequence

Results 506 Patients and 95 controls 10 clones of all three COX genes sequence 6 mutations found in COI and COII

Results 506 Patients and 95 controls 10 clones of all three COX genes sequence 6 mutations found in COI and COII Different levels of heteroplasmy but levels significantly greater in the AD cohort

Results 506 Patients and 95 controls 10 clones of all three COX genes sequence 6 mutations found in COI and COII Different levels of heteroplasmy but levels significantly greater in the AD cohort No disease-associated mutations in COIII gene

Results 506 Patients and 95 controls 10 clones of all three COX genes sequence 6 mutations found in COI and COII Different levels of heteroplasmy but levels significantly greater in the AD cohort No disease-associated mutations in COIII gene AD cybrids but not controls had low COX activity

Results 506 Patients and 95 controls 10 clones of all three COX genes sequence 6 mutations found in COI and COII Different levels of heteroplasmy but levels significantly greater in the AD cohort No disease-associated mutations in COIII gene AD cybrids but not controls had low COX activity Increased production of ROS in AD cybrids

Critical evaluation: How appropriate and robust are the methods ? Is the data and evaluation robust ? Are the conclusions valid, based on the reported data ? Has the paper been validated by other labs and do the authors self-cite excessively ? How does the paper stand the test of time ? Is there any conflict of interest ?

Why has this not been reported before ? mtDNA is destroyed by boiling We’re looking at pseudogenes!

How can an apparent increase in nuclear pseudogene mutations predispose to Alzheimers Disease and cause respiratory chain deficiency in transmitochondrial cybrids ?

Critical evaluation: How appropriate and robust are the methods ? Is the data and evaluation robust ? Are the conclusions valid, based on the reported data ? Has the paper been validated by other labs and do the authors self-cite excessively ? How does the paper stand the test of time ? Is there any conflict of interest ?

How can an apparent increase in nuclear pseudogene mutations predispose to Alzheimers Disease and cause respiratory chain deficiency in transmitochondrial cybrids ?