Why are systematic reviews important? Iain Chalmers Editor, James Lind Library Cochrane Workshop Independent University, Bangladesh

Good intentions are not enough to protect patients from harm and to avoid waste of resources

Fund raising advert for the United Nations Association

Five points 1.Systematic reviews of research should inform decisions in healthcare 2.Biased under-reporting of research should be outlawed 3.Research should be done for the right reasons 4.New research should begin with systematic reviews 5.New research should end with systematic reviews

Five points 1.Systematic reviews of research should inform decisions in healthcare 2.Biased under-reporting of research should be outlawed 3.Research should be done for the right reasons 4.New research should begin with systematic reviews 5.New research should end with systematic reviews

An example from child health

UNRWA Clinic, Khan Younis Refugee Camp, Gaza Strip, 1969/70.

Severe malnutrition following measles

I could have served my Palestinian patients and their community better if I had had access to systematic reviews of reliable, relevant research.

A systematic review of relevant research reported between 1939 and 1967 shows that: antibiotics prescribed for children with measles can reduce their risk of developing pneumonia

BMJ 23 October 2006

An example from maternal health

Which elements of routine antenatal care help women and use limited resources efficiently?

Published on paper in The Lancet and electronically in The Cochrane Database of Systematic Reviews, so that it can be updated and improved, as new information becomes available

SYSTEMATIC, UP-TO-DATE REVIEWS OF RESEARCH ON THE EFFECTS OF HEALTH CARE Essential, although not sufficient, for improving policies and decisions in health care & research Systematic reviews of other necessary evidence Cochrane Database of Systematic Reviews Needs MAKING POLICIES & MAKING DECISIONS professionals & providers service users & purchasers researchers & funders Priorities Evidence on effects of care Resources

Setting systematic reviews in context: The WHO Reproductive Health Library

If systematic reviews are to inform practice and policy reliably they must try to take account of all the relevant evidence from research, wherever the research was conducted and whatever its results.

Five points 1.Systematic reviews of research should inform decisions in healthcare 2.Biased under-reporting of research should be outlawed 3.Research should be done for the right reasons 4.New research should begin with systematic reviews 5.New research should end with systematic reviews

People have suffered and resources have been wasted because disappointing results of research have not been reported.

Publication bias can kill! Cowley et al. 1993: “When we carried out our study in 1980 we thought that the increased death rate that occurred in the drug group was an effect of chance…The development of the drug was abandoned for commercial reasons, and this study was therefore never published; it is now a good example of ‘publication bias’. The results described here…might have provided an early warning of trouble ahead.”

At the peak of their use in the late 1980s, it has been estimated that these drugs killed as many Americans every year as were killed during the whole of the Vietnam war. Moore 1995.

Biased under-reporting of research is ethically and scientifically unacceptable, and it should be outlawed. Information about ongoing research should be registered prospectively and publicly.

WHO is showing international leadership in calling for registration of clinical trials and assignment of unique study identifiers (ISRCTNs)

Five points 1.Systematic reviews of research should inform decisions in healthcare 2.Biased under-reporting of research should be outlawed 3.Research should be done for the right reasons 4.New research should begin with systematic reviews 5.New research should end with systematic reviews

The current research agenda does not serve the interests of patients and the public efficiently because of perverse commercial and academic influences on the research agenda

Researchers and users of research should work together to identify unanswered questions of importance.

Inspiring examples

The evaluation of Oral Rehydration Therapy has been pioneered in Bangladesh

A randomised trial in rural Nepal to assess the effects on birth outcomes of a participatory intervention involving women’s groups [HMG MOH Nepal, DfID, WHO, UNICEF Nepal, UNFPA Nepal]

Five points 1.Systematic reviews of research should inform decisions in healthcare 2.Biased under-reporting of research should be outlawed 3.Research should be done for the right reasons 4.New research should begin with systematic reviews 5.New research should end with systematic reviews

People have suffered and resources have been wasted because existing research evidence has not been reviewed systematically before doing additional research.

Preventing complications after premature birth Babies have suffered unnecessarily and resources have been wasted

Five points 1.Systematic reviews of research should inform decisions in healthcare 2.Biased under-reporting of research should be outlawed 3.Research should be done for the right reasons 4.New research should begin with systematic reviews 5.New research should end with systematic reviews

People have also suffered and resources have been wasted because new evidence has not been set in the context of up-to-date, systematic reviews of all other relevant evidence.

People reading research reports want researchers to provide answers to four questions: Why did you start? What did you do? What answer did you get? What does it mean anyway? Hill 1965

Reports of new research should make clear what contribution the new evidence has made to the totality of relevant evidence.

Classification of Discussion sections in RCT reports published in May issues of Ann Int Med, BMJ, JAMA, Lancet, and N Eng J Med 1997 n= n= n=18 First trial addressing the question 133 Contained an updated systematic review integrating the new results 200 Discussed a previous review but did not attempt to integrate new results 435 No apparent systematic attempt to set new results in context of other trials

Failure to prepare systematic reviews of existing research evidence has meant that: “Advice on some life-saving therapies has been delayed for more than a decade, while other treatments have been recommended long after controlled research has shown them to be harmful.” Antman et al. 1992

“Advice to put infants to sleep on the front for nearly half a century was contrary to evidence available from 1970 that this was likely to be harmful. Systematic review of preventable risk factors for SIDS from 1970 would have led to earlier recognition of the risks of sleeping on the front and might have prevented over infant deaths in the UK and at least in Europe the USA and Australasia.”

Suggested action points 1.Encourage admission of uncertainty about the effects of health practices and policies. 2.Increase the capacity for preparing, maintaining and disseminating systematic reviews of research evidence. 3.Outlaw biased under-reporting of health research and require public registration of controlled trials. 4.Promote research relevant to people planning, working in, and using the health services. 5.Refuse to support new research unless systematic reviews of existing evidence show that it is ethical and likely to be worthwhile.

Thank you for listening!