Alison Day Lead Librarian alison.day@poole.nhs.uk Grand Round Introduction to Critical Appraisal 20th March 2014

What is Critical Appraisal? It is about weighing up whether the research evidence is robust and relevant to your situation (for drs this could mean relevance of recommending a particular treatment, for managers it could mean if introducing a service change is appropriate, for librarians it could mean the adoption of different training approaches or resources) It is not: finding fault, inaccuracies or finding the problems with a piece of research, however this may be part of the process. Think “critique” Weighing up evidence to see how useful it is in decision making

Evidence Based Practice

5 steps of EBP Question Evidence Critical Appraisal Application Implementation and monitoring

Step 1 – The Question Broad questions are easy to understand e.g. What is the best treatment for depression? But return too many search results

Framework to Patient/Problem Intervention Comparison Outcome In a man with depressive symptoms… …Is cognitive behavioural therapy… …compared with fluoxetine… …better at improving depressive symptoms? Patient/Problem Intervention Comparison Outcome

Alternative frameworks ECLIPSE, Wildridge & Bell (2002) Expectation, Client group, Location, Impact, Professionals, Service, Evaluation SPICE, Booth (2004) Setting, Perspective, Intervention, Comparison, Outcomes

Step 2 Choose a database Broaden your search with thesaurus terms Use boolean operators (+ OR NOT)to combine terms together Use clinical query limits to locate best evidence

Great you have identified some abstracts of “best evidence” research articles what next...

Step 3 Validity: How trustworthy is it as a piece of research? Results: What is it telling us? Relevance: How useful will the research be to my situation (to my patients)

First steps Title: Focus. Does it clearly indicate what it is about? Authors: Credibility. What are their qualifications, experience, affiliations? Abstract: Summary. Does it give a clear overview? Does it look relevant to your practice? Is it worth reading? If NOT then STOP Initial critical appraisal as we would do for any source including information from the internet – e.g. HON standards http://www.healthonnet.org/HONcode/Conduct.html From Portsmouth NHS Library Services Quick Guide to Critical Appraisal. http://www.porthosp.nhs.uk/Library-Downloads/Guides/CriticalAppraisal2012.pdf

Next step… Assess the kind of study /research you are looking at Use the most appropriate tool/checklist to help you ask the right questions Is the study valid? What are the results? Will the results help locally?

Levels of Evidence Pyramid Quality and quantity of main research designs. Gold standard is a systematic review – plus a cherry if it is a Cochrane review. For more information describing features of different research study designs see Gosall(2012) From Evidence Based Nursing website http://ebp.lib.uic.edu/nursing/node/12

Quantitative or qualitative? Quantitative - numerical data Methods used include Randomised Controlled Trials (RCTs) Cohort studies Case control studies Case series

Qualitative Examines people’s opinions, beliefs, experiences and behaviour. Methods used include Focus groups Interviews Observation

Which checklist? Annotated checklists for reviews, RCTS, qualitative studies and more http://www.casp-uk.net CEBM Oxford checklists for reviews and RCTs http://www.cebm.net

Systematic Reviews A thorough search Synthesised and summarised Meta-analysis For definitions and descriptions of features of other research study designs please see Gosall(2012)

Homework http://www.nutritionandmetabolism.com/content/pdf/1743-7075-3-2.pdf

http://www.casp-uk.net/wp-content/uploads/2011/11/CASP-Systematic-Review-Checklist-31.05.13.pdf

Ready-made critical appraisal Reviews in Clinical Evidence www.clinicalevidence.org Cochrane reviews and DARE abstracts www.thecochranelibrary.com Summaries in Bandolier www.medicine.ox.ac.uk/bandolier Critique of media stories in Behind the Headlines www.nhs.uk/News/Pages/NewsArticles.aspx Good critical appraisal can take a long time so there are some sources that have done the hard work for you

Archie Cochrane, British epidemiologist 1992 set up the Cochrane Collaboration International organisation producing and disseminating systematic reviews of healthcare interventions

6 databases in one Keyword or MeSH search Combine search terms Detailed abstracts or full text to see the meta-analysis Plain language summaries

“Practice doesn't make perfect. Practice reduces the imperfection.” ― Toba Beta, Master of Stupidity

Want to know more? Check out www.swims.nhs.uk for details of books about CA View the library blog to link to resources http://eastdorsetnhslibrary.wordpress.com For help with searching or facilitation of a journal club/CA ask a librarian library@poole.nhs.uk