1. Evidence-Based Websites Douglas Badenoch Minervation Ltd

2. About me Strathclyde University –Lecturer, 1990-95 Oxford CEBM –Programme Manager, 1996-2002 Minervation Ltd –Founding Director, 2002-

3. About Minervation Evidence-based health care consultancy Information science Usability testing Design, hosting and development

4. Our Work Content J&J, NLH Evaluation CRUK, NHS Evidence Design, hosting and development CEBM, EQUATOR Network, ScottishDental

5. EBW could mean… 1.Websites supporting EBHC 2.Websites about EBHC 3.Embedding evidence in websites 4.Evidence about (the effectiveness of) websites

6. This session mostly about… 1.Websites supporting EBHC 2.Websites about EBHC 3.Embedding evidence in websites 4.Evidence about (the effectiveness of) websites

7. “We can have more impact on health by doing what we already know than by any device or technology likely to become available in the next decade.” Muir Gray, 2005

8. We know what they should look like 1There’s good evidence that we are not getting the evidence to where it’s needed, but that when we do, it has a positive impact upon practice. –Accessible, user-friendly –Full text, evidence-based summaries Clinicians have about 5 minutes to look for evidence –About half their questions go unanswered –They perform better when they have access to evidence-based summaries

9. Websites about EBHC 4.The “maverick” traditions of EBHC innovation show no signs of abating Reporting guidelines Clinical research online Uncertainties

10. www.equator-network.org Worldwide initiative to bring together all that we know about how to report the results of research. By study type By clinical specialty By publication type By document structure

11. www.equator-network.org Audience: Researchers Editors RG developers Current developments: Implementing a database of RGs Linking RGs to the underlying evidence base Supporting the implementation of RGs “Take me to your Reporting Guidelines”

12. We’re starting to do more about… 3.Embedding evidence-based methods into content production User involvement in requirements, comprehensive literature searching, critical appraisal and transparent evidence synthesis Review and updating protocols

13. www.scottishdental.org Set up in 2003 Relaunched 2007 Taxonomy- based User centred design

14. UK Prostate Link www.prostate-link.org.uk Quality-assessed information about prostate cancer Regular updates and alerts Popular with users, not with providers!

15. Instruments No lack of them (over 270 in our 2005 review)! –Searched health and information science databases –Very few instruments assessed all of the elements that mattered to our users –Of those that did, none had been validated –“Levels of evidence” impractical –Evidence of problems with policing

16. Implementation The assessment method had to: Address what matters to end-users Encompass validated, evidence-based methods of evaluating information Be valid and transparent Produce a quantitative score for use in search ranking

17. Implementation Usability Questions: 1. Is the site accessible without a login? 2. Does the site conform to web accessibility standards? 3. Is the site design clear and transparent? 4. Is the site design consistent from one page to another? 5. Can users find what they need on the site? 6. Is the format of information clear and appropriate for the audience? Reliability Questions: 7. Is it clear who has developed the web site and what their objectives are? 8. Does the site report a robust quality control procedure? 9. Is the page content checked by an expert? 10. Is the page updated regularly? 11. Does the page cite relevant sources where appropriate?

18. Validation Blind, independent comparison of two trained assessors: Information specialists Ranked 42 websites Measured the rank order correlation P < 0.0001

19. Prostate Cancer Support Federation http://prostatecancerfederation.org.uk/ A small user-led charity which provides unique information on a minimal budget. Usability Questions:Score out of 3 1. Is the site accessible without a login?3 2. Does the site conform to web Accessibility standards?2 3. Is the site design clear and transparent?3 4. Is the site design consistent from one page to another?2 5. Can users find what they need on the site?1 6. Is the format of information clear and appropriate for the audience?2 Reliability Questions: 1. Is it clear who has developed the web site and what their objectives are?3 2. Does the site report a robust quality control procedure?0 3. Is the page content checked by an expert?0 4. Is the page updated regularly?1 5. Does the page cite relevant sources where appropriate?2 Total19/33

20. Embedding quality Greater public awareness –e.g Bad Science Various quality initiatives must be joined up Role of the IS must be central to quality Will people start looking at the About Us pages?

21. From prevalence to incidence Having identified the existing best available evidence, the challenge is to communicate it effectively and keep it up to date.

22. We have a real problem with.. 4.Working out whether they work or not –Standards of proof are lower than normal NHS interventions –EBHC websites (type 1) don’t even use the traditional methods well –User modelling is only now taking centre stage in evaluating website performance –Political imperatives will override all of that anyway