Division of Population Health Sciences Royal College of Surgeons in Ireland Coláiste Ríoga na Máinleá in Éirinn Register of Clinical Prediction Rules, Methodological Quality Assessment and Implementation Strategies Emma Wallace, Claire Keogh, Susan Smith, Borislav D. Dimitrov, Tom Fahey

Division of Population Health Sciences Outline Section 1 -Development of register of Clinical Prediction Rules (CPRs) -Search strategies to identify CPRs relevant to primary care from the MEDLINE database Section 2 - Methodological quality assessment of CPRs Section 3 - Implementation strategies

Division of Population Health Sciences SECTION 1 Development of Register of Clinical Prediction Rules (CPRs)

Division of Population Health Sciences Definitions Clinical Prediction Rule –Clinical tool that quantifies the contribution of Patient History Physical Examination Diagnostic Tests –Stratify patients diagnosis Probability of having target disorder. –Outcome can be in terms of diagnosis, prognosis, referral or treatment

Division of Population Health Sciences CPR Examples 1 ) Ottawa ankle rule

2 ) The Centor Score

Division of Population Health Sciences Stages of development of CPR

Aim Develop a web-based register of CPRs for use in primary care as part of Cochrane Primary Healthcare Field

Division of Population Health Sciences CPR register development Development of an electronic search filter-identify CPRs relevant to primary care Haynes filter in Clinical Queries Broad and narrow search filters

Academic Emergency MedicineFamily Medicine American Family PhysicianFamily Practice American Journal of MedicineJournal of American Medical Association Annals of Emergency MedicineJournal of the American Board of Family Medicine Annals of Family MedicineJournal of Clinical Epidemiology Annals of Internal MedicineJournal of Family Practice Annals of MedicineJournal of Internal Medicine Annual Review of MedicineLancet Archives of Internal MedicineMedical Care BMC Family PracticeMedical Decision Making British Medical JournalMedicine British Journal of General PracticeNew England Journal of Medicine Canadian Family PhysicianPublic Library of Science Medicine Canadian Medical Association JournalPrimary Care Cochrane Database Systematic ReviewsScandinavian Journal of Primary Health Care 30 journals included on the register

Division of Population Health Sciences Methods Manually searched 30 journals relevant to primary care for the year 2008 (‘reference standard’) 7 individual electronic searches of the 30 journals (each filter treated as ‘diagnostic tests’) Test accuracy analysis: Sensitivity and specificity Aim: to maximise sensitivity

DatabaseFilter nameFilter search string PubMed Haynes Broad Filter (HBF) (predict*[tiab] OR predictive value of tests[mh] OR scor*[tiab] OR observ*[tiab] OR observer variation[mh]) PubMed Haynes Narrow Filter (HNF) (validation[tiab] OR validate[tiab]) EBSCO host McGrath/Murphy Broad Filter (MMBF) ((predict* N3 rule* OR predict* N3 model OR predict* N3 models) OR (decision* N3 rule*) OR (TX validat*)) EBSCO host McGrath/Murphy Narrow Filter (MMNF) ((predict* N3 rule* OR predict* N3 model OR predict* N3 models) OR (decision* N3 rule*)) PubMed Teljeur/Murphy Inclusion Filter 26 item (TMIF-26) "clinical prediction" OR "clinical model*" OR "clinical score*" OR "decision rule*" OR "diagnostic accuracy" OR "diagnostic rule*" OR "diagnostic score*" OR "diagnostic value" OR "predictive outcome*" OR "predictive rule*" OR "predictive score*" OR "predictive value" OR "predictive risk*" OR "prediction outcome*" OR "prediction rule*" OR "prediction score*" OR "prediction value*" OR "prediction risk*" OR "risk assessment" OR "risk score*" OR "validation decision*" OR "validation rule*" OR "validation score*" OR (derivation AND validation) OR (sensitivity AND specificity) OR (symptoms AND signs) PubMed Teljeur/Murphy Inclusion Filter 22 item (TMIF-22) (clinical[tiab] AND predict*[tiab]) OR (clinical[tiab] AND model*[tiab] ) OR (clinical[tiab] AND score*[tiab]) OR (decision [tiab] AND rule*[tiab]) OR (derive*[tiab] AND validat*[tiab]) OR (diagnos*[tiab] AND accura*[tiab]) OR (diagnos*[tiab] AND rule*[tiab]) OR (diagnos*[tiab] AND score*[tiab]) OR (diagnos*[tiab] AND value[tiab]) OR (predict*[tiab] AND outcome*[tiab]) OR (predict*[tiab] AND rule*[tiab] OR (predict*[tiab] AND score*[tiab] ) OR (predict*[tiab] AND validat*[tiab]) OR (predict*[tiab] AND value*[tiab]) OR (risk*[tiab] AND assessment*[tiab]) OR (risk[tiab] AND score*[tiab]) OR (sensitivity[tiab] AND specificity[tiab]) OR (symptoms[tiab] AND signs[tiab]) OR (validat*[tiab] AND decision*[tiab]) OR (validat*[tiab] AND rule*[tiab]) OR (validat*[tiab] AND score*[tiab]) OR (predict*[tiab] AND risk*[tiab]) PubMedTeljeur/Murphy Exclusion Filter (TMEF) (allele OR amino OR animal OR apoptosis OR chromosome OR congenital OR dental OR dna OR endogenous OR endothelial OR epithelial OR mammalian OR mice OR molecule OR molecular OR mouse OR mutate OR mutation OR necrosis OR pathogenesis OR phosphorylation OR polymorphism OR receptor OR signal OR species OR tissue OR tumor OR tumour OR tyrosine OR vitro)

Division of Population Health Sciences Results Filter name N articles retrieved N CPRs retrieved Sensitivity (%)Specificity (%) Haynes Broad Filter Haynes Narrow Filter McGrath/Murphy Broad Filter McGrath/Murphy Narrow Filter Teljeur Murphy Inclusion Filter-26 item Teljeur/Murphy Inclusion Filter-22 item Teljeur/Murphy Exclusion Filter Haynes Narrow Filter OR Teljeur Murphy Inclusion Filter-26 item Manual ‘reference standard’ search retrieved 6344 articles, 41 of which were CPRs

Division of Population Health Sciences MEDLINE versus the final search filter applied to 30 primary care journals ( 1966 – 2008 ) Number of articles in PubMEd Year of publication

Division of Population Health Sciences Creating the International Register: Preliminary findings YearsAverage articles retrieved per year Average CPRs for primary care identified* Average questionnaires for primary care identified* 2008 – * Preliminary findings resulted in 383 CPRs and 141 Questionnaires relevant to primary care

Division of Population Health Sciences Overview of classified CPRs Type of articlesNumber of articles CPRs114 (out of 383) Original articles86 Review articles19 Systematic reviews9 Age groupsOlder adults (64), adults (82), adolescents (11), children (7), infants (4), not specified (23) Setting32 (28%) from primary care – with the remaining set in Emergency department, in- patients, nursing home, clinical trials data

Preliminary results: Original articles (n=86) ICPC2 Clinical areaLevel 4Level 3Level 2Level 1 KCardiovascular17847 RRespiratory11823 AGeneral and unspecified9103- UUrological44-- TEndocrine/Metabolic and Nutritional432- LMusculoskeletal4213 DDigestive4111 NNeurological32-- -Process codes33-- SSkin1--1 BBlood and Immune1---

Division of Population Health Sciences SECTION 2 Methodological Quality assessment of CPR register

Division of Population Health Sciences Health Research Reporting

Division of Population Health Sciences Health Research Reporting International initiative ‘Equator’ set up in 2008 –Aim to provide resources & education enabling improved reporting and monitor progress –Develop network of reporting guidelines e.g.  CONSORT- RCTs  STARD- Diagnostic Accuracy studies  STROBE- Observational Studies  PRISMA- Systematic reviews

Division of Population Health Sciences Methodological Quality assessment of CPRs To date Equator have not developed a reporting guideline for CPR Quality assessment Wasson et al, 1985, Laupacis et al, 1997, Stiell et al 1999, McGinn et al, 2000, Cook et al, 2010 Different quality markers for derivation and validation CPR studies Consensus to use McGinn quality checklist due to content and utility

Division of Population Health Sciences Results-preliminary Quality assessment of a small representative sample of CPR register assessed to date as an example

Division of Population Health Sciences Results Derivation CPRs n= 18 -Validation CPRs n= 20

Results CPRs (derivation), 2008, n= 18

Results CPRs (validation), 2008, n= 20

Division of Population Health Sciences SECTION 3 Implementation strategies

Division of Population Health Sciences Implementation 1. Register of CPRs Cochrane Primary Health Care Field 2. Clinical Decision Support Systems (CDSSs)

Division of Population Health Sciences Register of CPRs Cochrane Primary Health Care Field Implement International Register of CPRs that is; –Publicly available, web based, user friendly –Searchable across clinical domains –Maintained and updated

Division of Population Health Sciences Registry

Division of Population Health Sciences Implementation 1. Register of CPRs Cochrane Primary Health Care Field 2. Clinical Decision Support Systems (CDSSs)

Division of Population Health Sciences Clinical Decision Support Systems

Division of Population Health Sciences Clinical Decision Support Systems Clinical Decision Support Systems (CDSSs) –Systems that are designed to improve clinical decision making Key points –Integrated with the electronic patient record –Available at the point of care –Computerised knowledge base –Provide patient-specific content

Search other databases Embase Cochrane Medion LILACS Expert contact Author’s databases Published Unpublished Secondary sources Evidence Based Medicine (BMJ ) Rational Clinical Examination (JAMA) Register Completing evidence base/CPR register Systematic reviews