1. Department of Health and Human Services Measuring Clinical Lab Ordering Quality: Theory and Practice Steven M. Asch MD MPH VA, RAND, UCLA April 29, 2005

3. INSTITUTE OF MEDICINE DEFINITION OF QUALITY The degree to which health services for individuals and populations * increase the likelihood of desired health outcomes and * are consistent with current professional knowledge

4. Lundberg, 1981 Were results used properly to improve care? Has the right test been ordered? Action The 9 steps in the performance of any laboratory test. The brain-to-brain turnaround time loop. Interpretation Reporting Analysis PreparationTransportation Identification Collection Ordering

5. WHAT IS POOR QUALITY ? Too little care – underuse –Failure to provide an effective service when it could have produced a good outcome Too much care – overuse –Providing care when its risks of harm greater than potential benefit The wrong care – misuse –Avoidable complications of appropriate care

7. CONCEPTUAL FRAMEWORK STRUCTUREPROCESSOUTCOMES Technical Excellence Right choices Effective/skillful Interpersonal Excellence Patient-centered Responsive Functional Status Satisfaction Mortality Biological Status Health Care Organization Characteristics Provider Characteristics Community Characteristics Population Characteristics

8. EXAMPLES OF STRUCTURAL MEASURES Health care organization characteristics -Weekend and night hours and convenient locations of laboratories -Volume Provider characteristics –Number of pathologists –Training of laboratory staff

9. CONCEPTUAL FRAMEWORK STRUCTUREPROCESSOUTCOMES Technical Excellence Right choices Effective/skillful Interpersonal Excellence Patient-centered Responsive Functional Status Satisfaction Mortality Biological Status Health Care Organization Characteristics Provider Characteristics Community Characteristics Population Characteristics

10. HTN NEW DIAGNOSIS LABS Asch et. al. BMC CV, 2005

11. QATOOL SCORES BY MODE Visit73% Medication69% Immunization66% Physical Exam63% Laboratory/Radiology62% Surgery57% History43% Education18% McGlynn, Asch et. al. NEJM 2003

12. CONCEPTUAL FRAMEWORK STRUCTUREPROCESSOUTCOMES Technical Excellence Right choices Effective/skillful Interpersonal Excellence Patient-centered Responsive Functional Status Satisfaction Mortality Biological Status Health Care Organization Characteristics Provider Characteristics Community Characteristics Population Characteristics

13. WHY MEASURE OUTCOMES? –Allow innovation in process –People care about outcomes directly

14. SAMPLE SIZE PROBLEMS –For mortality, need huge samples: CHF patients: 12% vs 16%, need 957 patients at each hospital. –Rarer outcomes People care, but statistical comparison is impossible.

15. DOES SICKNESS OR QUALITY DETERMINE CHF MORTALITY? Sickness at Process Admission Poor Medium Good Total Least Sick 1/4 4 7 4 5 Middle 1/2 11 8 8 9 Most Sick 1/4 37 32 26 32 Total 16 14 12 14

16. ACCOUNTABILITY: IS PROVIDER RESPONSIBLE FOR PROBLEM? –Current treatment must have big impact relative to other factors. –Do not want providers avoiding those who: have a bigger chance of problems are less likely to adhere to treatment

17. CHOOSING MEASURES: PRACTICAL CONSIDERATIONS –Choosing areas to measure –Selecting indicators –Designing specifications –Testing the measure

18. CHOOSING AREAS: ASSESSING HEALTH IMPORTANCE –Mortality –Morbidity –Utilization –Cost

19. PREVALENCE OF SELECTED ACUTE CONDITIONS AMONG WORKING ADULTS Condition Work Loss Days/100 Persons Injuries Influenza Infections and parasitic disease Common cold Digestive system conditions Other upper respiratory Acute ear infections 85.5 53.1 20.6 15.4 12.3 9.3 3.2

20. CHOOSING AREAS: POTENTIAL FOR IMPROVEMENT –What are the key outcomes of interest? –What processes produce those outcomes? –How well are key elements of care delivered today? –How variable is care delivery?

21. CHOOSING MEASURES: DEGREE OF PROVIDER CONTROL –How might the measure be affected by characteristics of the enrolled population? –What actions can providers or clinical laboratories take to improve performance?

22. STRENGTH OF SCIENTIFIC EVIDENCE I: Randomized controlled trial II-1: Nonrandomized controlled trial I-2: Cohort or case control studies II-3: Multiple time series III: Opinions or descriptive studies

23. COST-EFFECTIVENESS OF PROCESS MMR Immunization$14 saved/$1 spent Cervical cancer screening$21,000 spent/year (ages 20-28) Cervical cancer screening$11,000 spent/year (ages 29-50)

24. DESIGNING MEASURE SPECIFICATIONS –Define indicator –Identify target population –Define eligible population –Determine need for risk adjustment –Identify data sources –Write data collection instructions –Develop scoring rules

25. Example measure Men with a new diagnosis of prostate cancer, who have not had a serum PSA in the prior three months, should have serum PSA checked within one month after diagnosis or prior to any treatment, whichever comes first.

26. EVALUATING DATA SOURCES DATA SOURCE STRENGTHS WEAKNESSES Medical Record Clinical Detail Expense Missing links Administrative Use of services Clinical detail Patient Surveys General Health Expense Interpersonal Clinical detail

27. TESTING THE MEASURE –Reliability: The proportion of times that repeated use of measure in same population gives the same result –Validity: The extent to which the measure accurately represents the concept being assessed –Interpretability: Ease with which target audience can understand and use information generated by measure

28. WHY SHOULD CLINICIANS CARE ABOUT MEASURING QUALITY? –Internal quality improvement –External monitoring and evaluation –Consumer/purchaser decision-making

31. ADEQUACY OF CASE-MIX CONTROL –Severity of disease –Incidence and prevalence by demographics age race gender