Summarising what we already know – the pivotal role of systematic reviews Malcolm Macleod

Is there an agreed and accepted definition of systematic reviews? Characteristics –A pre-defined question –A pre-defined search strategy –Pre-defined inclusion and exclusion criteria May include a meta-analysis, but doesnt have to

[Are systematic reviews only about the application of a methodology (bias-reducing principles), or do systematic reviews require a clearly defined topic and specific question, as well as rigorous inclusion and exclusion criteria?]

Why bother – what does it add? 1.Less biased research summary –Positive findings are more likely in Papers compared with Abstracts High impact journals compared with low impact journals English Language verses non-English language reports Published versus unpublished findings

The File Drawer problem 0 worsebetter

Characteristics of narrative reviews Research summaries based on what comes through your inbox in eTOC alerts do not adequately describe the whole literature Data supporting the opinion of the reviewer are more likely to be cited

Why bother – what does it add? 2.Has the work been done before? –Replicating the work of others is an important part of science –Unknowing replication is a waste of time (and your money) –Some things may already be known with sufficient certainty that further experiments are not required (but e.g. positive control)

When and where are systematic reviews most likely to be carried out and published from now on?

Are they common? Pubmed search systematic review limited to animals conducted 22/1/09

When? In drug selection for clinical trial In decisions to proceed to clinical trial In preparation of grant and ethics applications In the planning of research In the refinement of the methodologies used in animal experiments

Where, and by whom? By people like me interested in developing new treatments By basic scientists summarising existing knowledge and planning new experiments By people studying the in-use characteristics of animal models of their disease

Publication Systematic reviews of clinical trials are widely published – sometimes in specialist journals Rigorous systematic reviews are cited about twice as frequently as narrative reviews The publishers of the Cochrane Library are exploring the possibility of publishing preclinical reviews

Do we have some examples of systematic reviews in basic animal research? Different purposes … –To survey an area of research –To summarise the effectiveness of a specific intervention –To examine the impact of potential sources of bias

To survey an area of research

To summarise the effectiveness of a specific intervention

Hypothermia 101 publications 277 experiments 3353 animals Better Worse

To examine the impact of potential sources of bias

NXY publications 29 experiments 408 animals Improved outcome by 44% (35-53%)

Animal Studies Clinical Trial Systematic Review And Meta-analysis

Should the approach of systematic reviews be applied to all areas of preclinical research? It probably could be, but methodologies may need some adaptation according to the specifics of the field Horizontal extension –from stroke to related diseases –Parkinsons, MS, Alzheimers, Neuro-oncology, Depression Vertical extension –Transgenic studies –Pathophysiological studies –In vitro studies Its wider application is likely to lead to substantial insights into the characteristics of the models used

Does the application of systematic reviews to animal research require additional funding, and if so where should this come from? Yes – but not much –Edinburgh group ~ £6k per publication This funding should come from those wishing to improve the usefulness of animal models of human disease

Are many reviews currently being published which are not systematic -- and if so when, where and why?

Who is responsible for ensuring that reviews are systematic, and how can we encourage this to happen?

What questions remain about the validity of research using animals as models of human disease, and whether animal research has predictive value for human health?