Critically appraising research in pain management Making sense of the evidence M.Sc. The nature of pain and its’ management, 2006

Why do we need to appraise the literature on pain ? What is the current state of the art ? How do we go about it ? What are good sources of information ?

Critical Appraisal The process of “weighing up” the evidence to assess how useful it is in making decisions about clinical care

Some “drivers” promoting critical review of research literature Evidence based practice Clinical effectiveness > guidelines, standards and pathways Continuing Professional Development (CPD) Modernisation Agenda eg. patient choice Research – asking the right questions

Types of Evidence Primary Generated from first hand experience eg. diaries, letters, research reports Secondary (synthesis of others work) eg. narrative and systematic literature reviews, scholarly papers

Information explosion c journals, 2 million papers p.a.

Pain Management – The Evidence Meta-analyses, RCT’s Systematic reviews Cohort studies Case studies Narrative reviews Updates Reports, editorials, working papers, product updates Conference proceedings, abstracts Dissertations etc.

Traditional Hierarchy of Evidence Quantitative Research Which research has most “weight” ? Meta-analysis, systematic review High quality RCT Controlled study without randomisation Observational study eg. case control study Descriptive study Expert consensus Clinicians and clients/patients views

Systematic review - The “sharp end” of critical appraisal -Used for development of clinical guidelines Many stages including: Development of a protocol Systematic selection of studies Data Extraction Quality assessment Statistical or other synthesis of findings Rating of the overall body of evidence

NICE Cox II selective inhibitors are not recommended for routine (regular) use in patients with rheumatoid arthritis or osteoarthritis (2001, Guidance 27) Current evidence on the safety and efficacy of percutaneous disc decompression using coblation for lower back pain does not appear adequate to support the use of this procedure without special arrangements for consent and for audit or research (Consultation Document, 25 May 2004) Adalimumab, etanercept and infliximab for the treatment of ankylosing spondylitis (in progress, due Feb. 2007)

Assessing the quality “critical appraisal” a) Quality of individual publications Checklists Guidelines/reading guides Questionnaires Frameworks, Criterion based scoring tools b) Quality of bodies of evidence Grading schemes hierarchies

Quality Evaluation Tools Many “off-the-shelf” tools available for different types of studies eg. Jadad, Delphi, Maastricht, SIGN Debate on scales v scores v weighting Few have been properly validated and the criteria for validation are unknown Move towards customised checklists

Critical Appraisal Skills Programme Scottish Intercollegiate Guidelines Network Jadad et. al. (1996) Control Clin Trials 17:1–12 Newcastle Ottawa Observational Study checklist _epidemiology/oxford.htm _epidemiology/oxford.htm

Quality in quantitative research eg. RCT’s Emphasis on issues such as: Random allocation of subjects, “blinding” Allocation concealment Similarity at baseline Standard, reliable and valid outcome assessment Drop-outs Intention to treat analysis (ITT) Validity Reliability

Quality in qualitative research Dependability (reliability) eg. - justification of methods - audit trail - providing information about the researcher - member checking, inter-researcher comparisons, triangulation Credibility eg. - providing raw quotes - reference to accepted procedures for analysis - discussion of how researcher may have effected outcomes - clear distinction between data and interpretation Transferability eg. - detail about participants and context - identifying differences between individual participants and participants and researcher

CASP – Qualitative Studies Rigour: has a thorough and appropriate approach been applied to key research methods in the study? Credibility: are the findings well presented and meaningful? Relevance: how useful are the findings to you and your organisation?

Structure of a research paper Title, author Abstract: summary of what the paper is about Introduction: Background including previous research, aims, research question/hypothesis, ethics Methods: patients, methods, equipment, data analysis Results Discussion (may include results in qualitative research) Conclusion, recommendations for clinical practice and further research

General questions for any study Are the aims clearly stated ? Was it ethical ? Does the design match the aims ? Was the sample size justified ? Sufficient methodological detail eg. drop outs ? Missing data ? Are the measurements/outcomes valid and reliable ? (NB. study population) Are statistical methods/data analysis described ? Are the main findings explicit ? Are important effects considered ? Are results/findings compared with previous work? Are implications for practice discussed ?

Papers for review: Effects of pool-based and land-based aerobic exercise on women with fibromyalgia/ chronic widespread muscle pain Jentoft et al Arthritis Care and Research, 2001, 45:42-47 In the system: the lived experience of chronic back pain from the perspectives of those seeking help from pain clinics Walker et al Pain, 1999,