Benchmarking Clinicians Farrokh Alemi, Ph.D.

Why should it be done? Hiring, promotion, and management decisions Help clinicians improve

Intrusion in clinician’s practice? Managers understand patient outcomes Practice profiles are constructed after the fact, when the patient is gone. Both the patient and managers can use benchmarked data  Poor clinicians are bad for the patient as well as for the organization

How should the analysis be done? Compare clinician to average peer Compare clinician to average peer taking care of same kinds of patients Compare clinician to expectations on admission Compare clinician and peers on patients matched on certain features In benchmarking, a clinician's performance is compared to an expected value.

Compare Clinician to Average Peer Calculate peer providers and the clinician’s average and standard deviation Compare using test of hypothesis with unequal means Problem:  Maybe misleading as providers see different kinds of patients and the clinician with more severely ill patients will naturally have worst outcomes

Example Data 123 internal medicine residents at the New York-Presbyterian Hospital in New York City. The outcomes examined included following outcomes:  Patients' satisfaction measured by telephone interviews of at least 10 patients  Disease-management profiles for average of 7 diabetes and 11 hypertensive patients. Patient's condition Frequency of use of various medications  Faculty-evaluations on seven dimensions: History taking Physical examination Differential diagnosis Diagnostic and/or treatment plan Health care maintenance Act compassionately Team player

Sample Report

Compare Clinician to Average Peer Caring for Same Kinds of Patients

Example Data

Comparing Clinicians to Expected Prognosis at Admission Assess patients severity Predict prognosis Calculate pair-wise student-t comparison of observed and expected values

Example Data

Comparing Clinicians When Patient's Severity of Illness Is Not Known

Example Data

Event Tree for Clinician’s Patients Is Kept, Outcomes Change

Is it reasonable to benchmark clinicians? Measurement distorts goals Measurement leads to defensive behavior No adequate measure of severity maybe available Too much time spent on measurement and too little on improvement

How Should Benchmarked Data Be Presented? Before the meeting  Schedule a feedback time and date as soon as possible.  Check your data to make sure there are no errors.  Add text, charts or graphics. Supplement numeric data with anecdotal information and the customer's voice (e.g. a short audio from a patient).  Distribute handouts ahead of meeting to participants

How Should Benchmarked Data Be Presented? At the meeting  Make it clear that the evaluation is confidential  Make a brief introduction of the purpose of the session  Acknowledge the limitation of the practice profiling method  Present the data and not the conclusions  Explicitly ask for clinician's evaluation of the data after each section of the report is presented  DO NOT defend the practice profiling method, the benchmarking effort or any aspect of your work  Thank the clinicians for their time and describe next steps

How Should Benchmarked Data Be Presented? After the meeting  Summarize the comments and append it to the report.  Describe resources available.  Send a written report to each clinician.  Ask the clinicians to comment on: What worked well and what needs improvement? Do they plan to change their practice and in what way? Was it worthwhile?  Set the time of next benchmarking report. .

Take Home Lesson Expected Outcomes Can Be Benchmarked Using Severity of Patients’ Illness