The role of peer reviewed journals in providing information for doctors and patients Richard Smith Editor, BMJ

What I want to talk about The information supply of doctors: what’s wrong now What’s wrong with journals? Drivers of change How might general journals look in the future A word on patients

The information supply for doctors: what’s wrong now?

Current problems A picture that captures in one image how doctors feel about information

Current problems One man’s view

Current problems “Our current information policy resembles the worst aspects of our old agricultural policy, which left grain rotting in thousands of storage files while people were starving. We have warehouses of unused information rotting while critical questions are left unanswered and critical problems are left unresolved.” Al Gore

View of an ordinary doctor

Current problems “On my desk I have accumulated journals and books as information sources, and I assume that I use them. But in some respects they are not as useful as they might be. Many of my textbooks are out of date; I would like to purchase new ones, but they are expensive. My journals are not organised so that I can quickly find answers to questions that arise, and so I don’t have print sources that will answer some questions. On the other hand, there is likely to be a human source who can answer nearly all of the questions that arise, albeit with another set of barriers.” An ordinary doctor

Current problems Think of all the information that you might read to help you do your job better. How much of it do you read?

Less than1% 1%- 10% 11%- 50% 51%- 90% More than90% Amount read Percentage Series2 Series1

Current problems Do you feel guilty about how much or how little you read?

How much time did you spend reading around your patients in the past week?

Reading of Bristol general practitioners

How far behind are you with your reading? Number of journals New articles a week Time to read article30 mins A doctor spends all day reading; after six weeks how far behind is he or she with his or her reading? A century

A reason to love economists Rational ignorance

Words used by 41 doctors to describe their information supply Impossible Impossible Impossible Overwhelming Overwhelming Overwhelming Difficult Difficult Daunting Daunting Daunting Pissed off Choked Depressed Despairing Worrisome Saturation Vast Help Exhausted Frustrated Time consuming Dreadful Awesome Struggle Mindboggling Unrealistic Stress Challenging Challenging Challenging Excited Vital importance

The information paradox “Doctors are overwhelmed with information but cannot find information when they need it” Muir Gray “Water water everywhere, nor any drop to drink” Samuel Taylor Coleridge

Introducing the Gerald Ratner of medical journals

What’s wrong with medical journals Don’t meet information needs Too many of them Too much rubbish Too hard work Not relevant Too boring Too expensive

What’s wrong with medical journals Don’t add value Slow every thing down Too biased Anti-innovatory Too awful to look at Too pompous Too establishment

What’s wrong with medical journals Don’t reach the developing world Can’t cope with fraud Nobody reads them Too much duplication Too concerned with authors rather than readers

A comment on medical journals from Drummond Rennie, deputy editor (west), JAMA There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.

Problems with peer review Slow Expensive A lottery Ineffective Biased Easily abused Can’t detect fraud Works for improving studies not selecting which to publish Can’t detect fraud

Peer review: detecting errors 600 word study with 8 errors Sent to 400 reviewers 60% response rate Median number of errors detected 2 Maximum number detected5 Percentage not detecting any 20

The power of peer review Forgive me if I return it without formal review, but I am totally unqualified to comment. You need someone with a postgraduate training in epidemiology, not an unlearned professor of neurology. Having said that, the paper is clearly rubbish…

The power of peer review Reviewer A “I found this paper an extremely muddled paper with a large number of deficits.” Reviewer B “It is written in a clear style and would be understood by any reader.”

Medical journals: where’s the meat?

Utility of information Utility=relevance x validity x interactivity work to access

Utility of different sources of information

Predictions of Lord Kelvin, president of the Royal Society, Radio has no future X rays will prove to be a hoax Heavier than air flying machines are impossible

What are the drivers of a new form of publishing? Failures of the present system A vision of something better Money Balkanisation of the literature Slowness

A vision of something better: for researchers "It's easy to say what would be the ideal online resource for scholars and scientists: all papers in all fields, systematically interconnected, effortlessly accessible and rationally navigable, from any researcher's desk, worldwide for free.” Stevan Harnad

A vision of a better information tool for clinicians 1 Able to answer highly complex questions Connected to a large valid database Electronic Fast Easy to use Portable

A vision of a better information tool for clinicians 2 Prompts doctors in a way that’s helpful not demeaning Connected to the patient record A servant of patients as well as doctors Provides psychological support

Future of scientific papers Will be “published” on the world wide web--perhaps Pubmed Central or an open archive They will be multimedia and include raw data and the software used to manipulate it They will be live not dead documents

Journals in the new world The future is not paper or electronic but paper and electronic, using the strengths of each medium Not “business as usual” but “reinventing ourselves” Probably far fewer Concentrate on meeting the needs of readers/ a community rather than authors “The long march from Brain to GQ”

Journals in the new world Rather than peer reviewing whatever is sent to them they would select relevant material from Pubmed Central (or whatever) and present it in an attractive way. (What the BMJ has always done). All the rest - education, debate, reviews, what’s on, obituaries Forum for debate

Journals in the new world “Be the glue that holds a community together” ELPS (electronic long, paper short) Online open review Copyright back to authors - each does what they want, payment to authors for reprints Benign publishers - low profit professional societies

ELPS (Electronic long, paper short) Paper - easier, shorter, brighter, more fun, more readable Electronic full data, software, video, sound extra material links interactive updating immediate posting

Towards online peer review Reviewers identity revealed to authors (RCT) Reviewers’ comments posted on the web of accepted papers (RCT) Reviewers’ comments posted as available Training reviewers (RCT started)

Vision of peer review “Peer review is changed from being an arbitrary decision made in a closed box to an open scientific discourse.”

A word on patients The impact of medical journals on patients and the public is mainly via the mass media Most mass media coverage is reasonable, but some can be way off beam, sometimes causing major problems The public are often very confused by conflicting reports from scientific journals

A word on patients Patients increasingly have access to the same information as doctors, including journals The BMJ has just appointed a patient editor Ten years from now journals might be for doctors and patients

Conclusions There are many problems currently with the information supply to doctors There are many problems with journals Peer review is a flawed process Original articles will be posted on the internet

Conclusions Clinicians will have their information needs met in other, far more effective ways The future for journals is paper and electronic, using the strength of both media But we need to reinvent ourselves, becoming more like GQ than Brain