If it works in Edmonton, will it work in Edinburgh ? Professor Helen Roberts DPhil FFPH General and Adolescent Paediatrics Unit, UCL Institute of Child Health, London

... might systematic reviewing help?.. Well up to a point Lord Copper. SRs can offer A synthesis of studies in a particular field which no policy maker or practitioner, however diligent, could hope to read themselves and sometimes A challenge to received wisdom

What we say…What non-systematic reviewers are really thinking… We carried out a comprehensive search of the literature, including Felonbase, Quackline, NERD alerts, etc They searched the one or two databases where 95% of the literature is published, plus an unnecessarily large number of oddly-named databases that no-one has ever heard of Mark Petticrews Cochrane lecture 2007

What systematic reviewers sayWhat non-systematic reviewers are really thinking… We found 1,000,000 studies, of which 1 nearly met the inclusion criteria, but not quite They excluded all the really useful stuff We concluded that more research is needed …and then they concluded that more money is needed Mark Petticrews Cochrane lecture 2007

Meaning well and doing good Nowhere is wisdom more necessary than in the guidance of charitable impulses. Meaning well is only half our duty; thinking right is the other and equally important half Samuel Gridley Howe, 1866

Tabloid solutions Licences for smokers Separate queues in the supermarket for alcohol Letters to parents of fat kids Exercise hours for large firms

Traffic management in Athens(of the south) Attempts to reduce traffic Odd and even numbered number plates on different days Tax breaks for greener cars The consequences of interventions with strong face validity arent always what the experts think they might be

Mentoring to reduce offending Anti-social behaviour in childhood and adolescence is a problem for young people, their families, the police, communities and politicians In 2003, the Government announced funding for mentoring schemes in England, arguing that mentors can make a real difference to... some of the most vulnerable people....There are... excellent examples of schemes which really work. Mentoring is non-invasive and does not require drug treatment. It is easy to see why it might work, and why it might be attractive to politicians and policy makers. Unfortunately it can have the reverse of the desired effect

What might help in working out what works? Reversing the lack of a rigorous outcome evaluation culture Recognition that quick fixes are rare and that social and educational interventions have as much or even more potential to do good or harm as health interventions

What else might work ? Include high quality qualitative and ethnographic studies Paying greater attention to the experiences of people on the receiving end of interventions, policies and practices (c) Helen Roberts

Horses for courses Effectiveness Does this work? Does it work better than doing something else, or nothing? Effectiveness of service delivery How does it work ? Safety Does it do more good than harm ? Cost effectiveness Is it worth buying? Acceptability Will children or parents want to (at best) or be willing to (at worst) take it up ? Appropriateness Is this the right service for these children ? Quality How good is the service? Salience Does it matter? Evidence, hierarchies and typologies: Horses for Courses, Petticrew M. and Roberts H. Journal of Epidemiology and Community Health, 57: , 2003

The Road to Wisdom The road to wisdom? Well, it's plain And simple to express: Err and err and err again, but less and less and less. Piet Hein