"Improving decision-making and workflow in patient care: a review of PROforma technology and its current evidence base" Cancer Research UK* has been developing the PROforma language for modelling clinical processes and associated decision support and workflow technologies for about ten years. During this period the foundations of the language have been formally established, clinical applications and development tools have become increasingly flexible, and a range of applications have been built and clinically tested. Applications to date range from suport fpr prescribing and referral decisions in general practice to management of cancer and HIV+ patients. This talk will summarise the concepts underlying PROforma and review seven empirical trials which have provided quantitative data. These results strongly suggest that appropriate technologies can yield major benefits in consistency, quality and safety of patient care, together with improved resource management and good clinical acceptability. More information about PROforma and other related technologies can be found at and PROforma in particular at *previously Imperial Cancer Research Fund

Improving decision-making and workflow in patient care: a review of PROforma technology and its current evidence base" BCS 2005

“Medicine is a humanly impossible task ” Up to 97,000 unnecessary deaths p.a. in the US are due to medical error. Total national costs … between $17B and $29B Institute of Medicine, To Err is Human 1999 In NHS hospitals “overall rate of preventable adverse events of 11.7%. [A third of which] led to …disability or death, … others are frequent, minor events … but together have massive economic consequences” Vincent et al, BMJ 2001

… and in oncology … “Perhaps 16,000 lives could be saved if all current knowledge of cancer were properly applied” ICRF Vision for Cancer, 1995 “There have been undoubted improvements in service delivery but there is still a sense that progress has been patchy and that much has yet to be achieved.” National Service Framework Assessment of NHS Cancer Care 2001.

Promoting best practice

… but  Busy clinicians have little time to read  Even if there is time, memories are unreliable, working pressures acute  Conventional guidelines address general principles of care, not the needs of individual patients

Support at the point of care R Steele et al, Proc. AI in Medicine Europe,

The PROforma method

What is PROforma? Before musical notation, every singer had to memorize the entire repertoire. Those singers then went on to teach the next generation. Small errors in memory or differences of taste caused the chants to change over the years and no two singers would learn a chant precisely the same way. Notation made it possible to record a chant in a definitive form for easier and more reliable communication. Guido d'Arezzo Benedictine monk, musical theorist and teacher.

The PROforma language is based on an ontology of general tasks Generic task “Keystone” Enquiries Plans DecisionsActions

The Tallis toolset Composing and publishing clinical guidelines, protocols and pathways plan :: 'plan2' ; caption ::"Chemotherapy"; description ::"Care pathway for chemotherapy"; precondition :: result_of( decision2) = Chemotherapy ; component :: 'action2' ; number_of_cycles ::1; component :: 'plan5' ; schedule_constraint :: completed('action2') ; number_of_cycles :: 3 ; component :: 'action4' ; schedule_constraint :: completed('plan5') ; number_of_cycles ::1; end plan. decision :: 'decision1' ; caption ::"Diagnosis?"; description ::"Differential diagnosis between cancer and peptic ulcer."; candidate :: 'peptic_ulcer' ; argument :: for, ( biopsy = negative ) argument :: for, ( pain_time = delayed ) argument :: for, ( age = young or age = adult ) argument :: for, ( pain_site = epigastric ) recommendation :: Netsupport( decision1, peptic_ulcer ) >= 1 ; candidate :: 'cancer' ; argument :: for, ( biopsy = positive ) argument :: for, ( pain_site = epigastric ) argument :: for, ( age = elderly ) argument :: for, ( smoker = yes ) argument :: for, ( pain_time = immediate ) recommendation :: Netsupport( decision1, cancer ) >= 1 ; end decision. action :: 'action5' ; caption ::"Medication"; description ::"This (dummy) action is carried out if the diagnosis is peptic ulcer"; procedure ::'Medication'; end action. action :: 'action1' ; caption ::"Refer to surgeon"; procedure ::'Refer to surgeon'; end action. enquiry :: 'enquiry2' ; caption ::"'Measure wbc'"; description ::"Chemotherapy: record white blood count"; source :: 'wbc' ; end enquiry.

Repertoire: knowledge bases of standard reusable components

Protocols, guidelines, care pathways Formalised in PROforma Tested in silico (“on the bench”) Routine use Feedback into research and policy Trials Cost-benefit analysis Cost-benefit analysis The “figure of eight” model

Applications and evidence

Prescribing in general practice (CAPSULE) Walton et al British Medical Journal 1997

Automated image interpretation Paul Taylor, Andrew Todd-Pokropek, Medical Image Analysis (2000) BrCa pathway

RAGs: Risk Assessment in Genetics Andrew Coulson, Jon Emery, David Glasspool BMJ 1999; 2000; Meth. Inf. Med 2001

ERA: cutting waiting times Jon Bury, Michael Humber BrCa pathway

ALL Dose adjustment study J Bury, C Hurt, A Roy, L Cheesman, M Bradburn, S Cross, J Fox, V Saha (submitted) Objectives: – To assess the clinical value of a decision support system designed to assist with dosage adjustments during maintenance therapy for childhood Acute Lymphoblastic Leukaemia. Decision model: – one PROforma decision task, 8 options. Each has between 1 and 5 criteria associated with it, each referring to different clinical situations, expressed in terms of 5 parameters. Materials and methods – Balanced-block crossover experiment, in which 36 clinicians with varying degrees of experience were asked to decide on oral chemotherapy dosages for 8 simulated cases: 4 using decision support and 4 without. – Outcome measures were number of protocol consistent dosage decisions; time to manage each case; accuracy of dosage calculations and clinicians' opinions about the value and usability of the system.

Dose adjustment in chemotherapy with CRUK Paediatric Oncology Group, London Hospital Bury, Hurt et al, Proc. American Medical Informatics Association, 2002 Hurt et al, Proc. AI in Medicine Europe, 2003 Bury et al, British Journal of Haematology (in submission)

ALL Dose adjustment study – results J Bury, C Hurt, A Roy, L Cheesman, M Bradburn, S Cross, J Fox, V Saha (submitted) MeasureWithout DSS With DSS Number of erroneous prescriptions 54/1440/144p<0.001 Number of times users deliberately overrode the protocol 6/1447/144 Time taken to reach a Novices decision for each case Experts 156.5s 110.8s 125.4s 133.6s p = 0.02 P = /36 subjects said they would be likely to use the system if it were available MeasureWithout DSS With DSS Number of erroneous prescriptions 54/1440/144p<0.001 Number of times users deliberately overrode the protocol 6/1447/144 Time taken to reach a Novices decision for each case Experts 156.5s 110.8s 125.4s 133.6s p = 0.02 P = 0.02

Triple Assessment study C Hurt, V Patkar, R Steele, T Rose, M Williams, J Fox (report in preparation) Objective: – To evaluate the potential effect of PROforma decision support on clinical decision making with respect to national guidelines for Triple Assessment. Decision model – Pathway included 4 decisions (familial risk, type of imaging, type of biopsy, and management). Materials and methods: – 15 hypothetical paper cases covering range of clinical scenarios developed by an expert panel of five judges (2 breast surgeons, 1 breast pathologist, 1 radiologist and 1 geneticist) and optimal management for each case was agreed by consensus. – 24 doctors asked to manage 5 cases with and 5 without computer support. A balanced block design used to allocate cases.

Triple assessment of breast cancer Hurt C, Patkar V, Steele R, Rose T, Fox J (in preparation) Steele R, Fox J Proceedings of European conference on AI in Medicine, 2003

24 participants (17 consultants, 5 specialist registrars, 1 nurse practitioner) Average number of years in speciality = 9.3 (range ) Deviations / errors Without DSSWith DSSTotal decisions All deviations60 (50%) 16 (13%) 120 In each arm Critical errors16 (13%) 1 (0.8%) Critical errors that escaped peer-check 10 (8.3%) 1 (0.8%) Triple Assessment Study – Results C Hurt, V Patkar, R Steele, T Rose, M Williams, J Fox (report in preparation) Would patient care improve with TA decision support? In favour 16Undecided 1Against 7 Would they personally wish to work with TA decision support? In favour 12Undecided 3Against 9 Deviations / errors Without DSSWith DSSTotal decisions All deviations60 (50%) 16 (13%) 120 In each arm Critical errors16 (13%) 1 (0.8%) Critical errors that escaped peer-check 10 (8.3%) 1 (0.8%)

65 Decisions If compliance with best practice is 99% then 50% of women will get “perfect care” ( ) If compliance is 95% then 3% of women will get perfect care ( ) Our results from several studies suggest actual deviations from quality standards are between 10 and 30%

Other projects (InferMed ) Hoffman-La Roche –Retrogram® –Trans-national study (ORAMA) on Acute Renal Anaemia Brown University Pain Management, Long term care of the elderly. Mater Misericordiae Dublin – 2 electronic guidelines New Zealand Ministry of Health – Diabetes Management in General Practice Eclipsys Inc. – Integration with HIS Pfizer – 3 Post-Op pain management guidelines (PROSPECT)

Management of HIV+ patients Tural et al, AIDS, 2002, 16, Retrogram® for Roche Arezzo (3)

A standard format for guidelines and care pathways? Research/ centres of excellence Specialist services General hospitals Primary care Home and self care

More information