AHRQ Quality Indicators

AHRQ Quality Indicators
Developed by Stanford-UCSF Evidence Based Practice Center Funded by the Agency for Healthcare Research and Quality EPC Team (PSI Development) PI: Kathryn McDonald, M.M., Stanford Patrick Romano, M.D., M.P.H, UC Davis Jeffrey Geppert, J.D., Ed.M., Stanford Sheryl Davies, M.A., Stanford Bradford Duncan, M.D., M.A., Stanford Kaveh G. Shojania, M.D., UCSF Support of Quality Indicators PI: Kathryn McDonald, M.M., Stanford Sheryl Davies, M.A., Stanford Patrick Romano, M.D., M.P.H, UC Davis Jeffrey Geppert, J.D. Ed.M., Stanford Mark Gritz, PhD, Battelle Greg Hubert, Battelle Denise Remus, RN PhD, AHRQ Project Officer

Acknowledgements Funded by AHRQ Contract No. 290-97-0013
Support of Quality Indicators Contract No Presentation funded by AHRQ Data used for analyses: Nationwide Inpatient Sample (NIS), Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality State Inpatient Databases (SID), 1997 (19 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality

Acknowledgements We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health and Development; Colorado Health and Hospital Association; CHIME, Inc. (Connecticut); Florida Agency for Health Care Administration; Georgia Hospital Association; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Maryland Health Services Cost Review Commission; Massachusetts Division of Health Care Finance and Policy; Missouri Hospital Industry Data Institute; New Jersey Department of Health and Senior Services; New York State Department of Health; Oregon Association of Hospitals and Health Systems; Pennsylvania Health Care Cost Containment Council; South Carolina State Budget and Control Board; Tennessee Hospital Association; Utah Department of Health; Washington State Department of Health; and Wisconsin Department of Health and Family Service.

Outline Administrative data and quality indicators
AHRQ Quality Indicators (QI) Development of AHRQ QIs Risk adjustment & MSX smoothing methods Application of QIs to research and quality

History of AHRQ QIs/PSIs
Healthcare Cost and Utilization Project (HCUP) HCUP discharge data collection (1988) HCUP Quality Indicators Mortality for Inpatient Procedures Complication Rates Potentially Inappropriate Utilization Potentially Avoidable Hospital Admissions HCUP – began as a data collection project in 1988. QIs developed by AHRQ in the early 1990s in response to States' requests for a quality assessment tool that could be used with hospital administrative data. 33 Quality indicators. Intended to be used a screening tools. Based on existing indicators reported in the literature at the time. Limitations of QIs: 1. Limited Risk Adjustment. 2. Low frequencies in some 3. Primarily surgical indicators. 4. Some less appropriate denominators (not area level).

Refinement of HCUP QIs Refinement commissioned by AHRQ in 1999
Completed by UCSF-Stanford EPC Two related projects Two technical reviews Refinement of the HCUP Quality Indicators Measures of Patient Safety Based on Administrative Data Three indicator sets, AHRQ QIs Inpatient Quality Indicators (IQIs) Prevention Quality Indicators (PQIs) Patient Safety Indicators (PSIs) Refinement of HCUP QIs completed between 1999 and 2002 by UCSF-Stanford EPC. The first project examined mortality, utilization and ambulatory care sensitive conditions, later split into the PQIs and IQIs. Defined at both an area level, and hospital level. Used literature review and empirical analyses to identify and summarize the evidence for use of these quality indicators as screening tools, including face validity, precision of measurement, unbiasedness, construct validity, and fostering true quality improvement. Novel empirical methods used mathematical techniques to extract true variations and discard noise or random variation. Second project identified complications and patient safety indicators. Methods will be explained in detail later, similar to QI project, with additional clinical review. Result of the two projects was two separate technical reviews, available for download or from AHRQ. Three indicators sets and software – IQIs, PQIs, and PSIs.

Administrative Data & Quality Improvement
Opportunities Coding practices improving Data availability improving (e.g., less truncation) More specific codes Large data sets improve precision Comprehensive: all hospitals Quality screening feasible Obstacles Coding errors introduce noise Lack of information on timing, comorbidity vs. adverse events Varying number of secondary diagnoses fields can cause bias Heterogeneous severity within single code

Administrative Data State Inpatient Databases
Includes ICD-9-CM dx and procedure codes, DRG, dates, age, sex, payer, race, discharge disposition, hospital and/or patient zip codes 33 States 80% + of all U.S. hospital discharges 18 states available for purchase In 27 state sample, approximately 3200 hospitals Ranges from $20-$1600 per state per year for academic organizations.

Administrative Data Nationwide Inpatient Sample (NIS)
Sampling of State Inpatient Databases 7.5 million discharges/1,000 hospitals/33 States Approximates 20% sample of nonfederal acute care hospitals Discharge level weights applied for national estimates Available for purchase NIS releases for data years are available from the HCUP Central Distributor. NIS 2001 may be purchased for $200 in a set of two CD-ROMs with accompanying documentation from the HCUP Central Distributor

HCUPnet http:///hcupnet.ahrq.gov/
Web-based tool to query NIS and KIDS databases, Pre-run tables for Query based on ICD-9-CM, DRG or CCS Information on hospitalizations, charges, length of stay, mortality, discharge status Stratification by age, sex, race, income, insurance, hospital characteristics Rank order hospitalizations

Sample AHRQ QI definition
DECUBITUS ULCER Relationship to quality: Identifies cases of decubitus ulcers that develop during hospitalization. Indicator definition: Number of patients with decubitus ulcer (see definition and exclusions below) per 100 eligible admissions (population at risk). Definition of decubitus ulcer: Definition of population at risk: Patients eligible to be included in this indicator: Secondary diagnosis code of decubitus ulcer: decubitus ulcer [707.0] a. All patients (medical and surgical) c. Patient must have a length of stay of more than 4 days. d. Patient must not be in MDC 9 (Diseases and Disorders of the Skin Subcutaneous Tissue and Breast), or have any diagnosis of hemiplegia, paraplagia, or quadriplagia. The indicator definition summarizes the operationalization, but doesn’t include ICD-9-CM codes. The specific ICD-9-CM codes used in the definition of the complication are listed, as well as the denominator (the population at risk). Specific ICd-9-CM codes for the denominator were listed in an appendix. This indicator has several examples of exclusion criteria. Some of these exclusions exist because patients in these categories were more likely to have decubiti present on admission (which cannot be discerned with most administrative data), or are more likely to develop a non-preventable decubitus ulcer. The clinical rationale briefly explains the face validity assertion of the indicator, the source, and the reasoning for all the details of the operationalization of the indicator.

Prevention Quality Indicators (PQIs)
Defined using area population as denominator Potentially avoidable hospitalizations or ambulatory care sensitive conditions Conditions for which good outpatient care can potentially prevent the need for hospitalization or for which early intervention can prevent complications or more severe disease Public health, comprehensive health care systems Based on existing, validated indicators set, but modified and updated conditions for which good outpatient care can potentially prevent the need for hospitalization or for which early intervention can prevent complications or more severe disease. Potentially of interest to public health, researchers or comprehensive health care systems. Examine the health care system as a whole, access to quality outpatient care. Most indicators, but not all, based on Billings et al, and Weissman et al. indicators sets, but have been modified and updated.

Prevention Quality Indicators (PQIs)
Bacterial pneumonia Dehydration Pediatric gastroenteritis Urinary tract infection Perforated appendix Low birth weight Angina without procedure Congestive heart failure Hypertension Adult asthma Pediatric asthma Chronic obstructive pulmonary disease Diabetes short-term complication Diabetes long-term complication Uncontrolled diabetes Lower-extremity amputation among patients with diabetes 16 measures included in PQI set, several measures for diabetes. Covers acute infection, asthma, and other chronic conditions. These measures are all defined as number of admissions per 10,000 (or 1000) population, except perforated appendix (per admissions for appendicitis in an area) and low birth weight (per births in an area).

Inpatient Quality Indicators (IQIs)
Defined using both hospital admissions and area population as denominator Inpatient mortality for certain procedures and medical conditions Utilization of procedures for which there are questions of overuse, underuse, and misuse Volume of certain procedures Risk-adjusted using APR-DRGs Potential for internal quality improvement purposes Based on existing, validated indicators Most indicators are defined using hospital level admissions as the denominator (e.g. mortality after CABG, denominator is all CABG surgeries). Included in the IQIs are some area level utilization indicators for potentially overused procedures (e.g., hysterectomy rate). Appropriate denominators cannot be ascertained using administrative data, so population rates are best alternative. All mortality measures are at hospital level, and reflect mortality for both surgical cases and medical cases (e.g., CHF, hip fx, or AMI). Mortality measures come from literature, previous HCUP QIs, and Clinical Classification System (CCS) developed by AHRQ. Utilization measures defined at hospital level have appropriate denominators (e.g., bilateral cath, C-section, VBAC) and which hospital have more impact over rates. Most do not have “right rates” with the exception of bilateral catheterization and incidental appendectomy in the elderly, which are likely to be very rarely indicated. Other procedures may be over or under-utilized. Volume indicators reflect hospital level volumes for procedures for which volume has a strong association with better outcomes. Proxy measure, not direct, but in absence of other data, the Leap Frog group has recommended these types of measures. Volume definitions taken from the literature where the volume-outcome relationship has been established (Ethan Halm did a large literature review). Risk adjusted using APR-DRGs from 3M. Difference between APR-DRGs and DRGs: Although both APR-DRGs and DRGs are based on a patient’s International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic and procedure codes, they are different patient classification schemes. The 3M Corporation developed and sells a widely used software product to define risk adjustment categories called APR-DRGs, which has the advantage of subclass groupings for Severity of Illness and Risk of Mortality. DRGs are designated by the Centers for Medicare and Medicaid Services (CMS) and software groupers are available from various vendors to assign DRGs.

Inpatient Quality Indicators (IQIs)
Mortality Rates for Conditions Acute myocardial infarction (2 versions) Congestive heart failure Gastrointestinal hemorrhage Hip fracture Pneumonia Stroke Mortality Rates for Procedures Abdominal aortic aneurysm repair Coronary artery bypass graft Craniotomy Esophageal resection Hip replacement Pancreatic resection Pediatric heart surgery Hospital-level Procedure Utilization Rates Cesarean section delivery (primary and total) Incidental appendectomy in the elderly Bi-lateral cardiac catheterization Vaginal birth after Cesarean section (2 versions) Laparoscopic cholecystectomy Area-level Utilization Rates Coronary artery bypass graft Hysterectomy Laminectomy or spinal fusion PTCA Volume of Procedures Abdominal aortic aneurysm repair Carotid endarterectomy Esophageal resection Pancreatic resection Pediatric heart surgery See previous slide for information on classes and sources. A few indicators are paired indicators (and you may want to say less strong). Volume indicators tend to have associated mortality indicators, although some of the associated mortality indicators are not widely used, nor will be applicable to many hospitals (e.g., the resections).

Patient Safety Indicators (PSIs)
Defined using hospital admissions as denominator Inpatient complications of care and potential patient safety events Potential for internal quality improvement purposes, monitoring of patient safety events Novel indicators, based on concepts reported in the literature Patient safety indicators are also hospital based, but cover complications of care (e.g. deep vein thrombosis after surgery) and potential patient safety events (e.g., in-hospital hip fracture).

Patient Safety Indicators (PSIs)
Provider-level Patient Safety Indicators Accidental puncture or laceration during procedure Complications of anesthesia Death in low mortality DRGs Decubitus ulcer Failure to rescue Foreign body left in during procedure Iatrogenic pneumothorax Selected infection due to medical care Postoperative hemorrhage or hematoma Postoperative hip fracture Postoperative physiologic and metabolic derangements Obstetric trauma – vaginal delivery with instrument Obstetric trauma – vaginal delivery without instrument Obstetric trauma – cesarean section delivery Postoperative pulmonary embolism or deep vein thrombosis Postoperative respiratory failure Postoperative sepsis Transfusion reaction Postoperative wound dehiscence in abdominopelvic surgical patients Birth trauma – injury to neonate Area-level Patient Safety Indicators Foreign body left in during procedure Iatrogenic pneumothorax Infection due to medical care Technical difficulty with medical care These indicators were selected using a structured selection process. Covers post-surgical complications, obstetric complications and general complications. Heavy on the surgical. Not the way it was designed, but surgical are often easier to operationalize (less present on admission/case mix concerns, more homogeneous). For PSIs the denominators are specific to the complication, not generic. Specific inclusion and exclusion criteria are used to refine denominator to those that are truly at risk for a potentially preventable complication (as opposed to not at risk for the complication at all, or likely to have a non-preventable complication). Area indicators are analogs of the provider-level indicators, but include complications which appear in the principal diagnosis. This means that the preceding condition or procedure occurred in an outpatient setting or in another hospitalization. Unlike other area indicators, the denominator is not population, but it is the same denominator as the provider-level indicators taken at an area level.

PQI Rates PQI Per 100,000 Area SD DIABETES SHORT TRM COMPLICATION 43.2
33.3 PERFORATED APPENDIX 311.6 157 DIABETES LONG TERM COMPLICATION 100.4 83.1 PEDIATRIC ASTHMA 166.4 191.2 CHRONIC OBSTRUCTIVE PULMONARY DISEASE 371.6 342.0 PEDIATRIC GASTROENTERITIS 128.8 189.6 HYPERTENSION 50.2 67.0 CONGESTIVE HEART FAILURE 526.6 488.3 LOW BIRTH WEIGHT 38.0 39.0 DEHYDRATION 158.9 143.6 BACTERIAL PNEUMONIA 506.8 360.0 URINARY INFECTION 169.0 142.1 ANGINA 108.8 128.2 DIABETES UNCONTROLLED 33.7 39.3 ADULT ASTHMA 102.3 95.6 LOWER EXTREMITY AMPUTATION 27.9 32.1 Source: SID, AHRQ Prevention Quality Indicators SAS Software Version 2.1 Revision 3.

IQI Rates IQI Per 100 Provider SD IN-HOSP MORT ESOPHAGEAL RESECTION
11.51 28.88 IN-HOSP MORT PANCREATIC RESECTION 10.53 25.11 IN-HOSP MORT PEDIATRIC HEART SURG 4.90 11.87 IN-HOSP MORT AAA REPAIR 16.87 22.97 IN-HOSP MORT CABG 3.91 4.35 IN-HOSP MORT CRANIOTOMY 9.74 12.35 IN-HOSP MORT HIP REPLACEMENT 0.38 2.32 IN-HOSP MORT AMI 15.41 13.16 IN-HOSP MORT CHF 5.03 4.39 IN-HOSP MORT STROKE 11.19 8.35 IN-HOSP MORT GI HEMORRHAGE 3.46 5.27 IN-HOSP MORT HIP FRACTURE 3.45 6.52 IN-HOSP MORT PNEUMONIA 8.11 4.86 CESAREAN SECTION DELIVERY 20.33 8.59 PRIMARY CESAREAN SECTION 13.12 7.45 VAGINAL BIRTH AFTER C-SECTION 26.54 15.28 LAPAROSCOPIC CHOLECYSTECTOMY 73.25 18.65 INCIDENTAL APPENDECTOMY 2.83 5.08 BI-LATERAL CATHETERIZATION 13.96 Ranges from 0.38% (hip replacement) to (AAA repair) for mortality. Source: SID, AHRQ Inpatient Quality Indicators SAS Software Version 2.1, Revision 3. Release pending.

PSI Rates PSI Rate per 1000 Complications Of Anesthesia 0.71
Death In Low Mortality Drgs 0.60 Decubitus Ulcer 21.96 Failure To Rescue 143.82 Foreign Body Left In During Proc 0.08 Iatrogenic Pneumothorax 0.81 Infection Due To Medical Care 2.00 Postoperative Hip Fracture 0.31 Postop Hemorrhage Or Hematoma 2.24 Postop Physio Metabol Derangmnt 0.87 Postop Respiratory Failure 3.48 Postoperative Pe Or Dvt 7.29 Postoperative Sepsis 10.75 Postoperative Wound Dehiscence 1.89 Accidental Puncture/Laceration 3.25 Transfusion Reaction 0.005 Birth Trauma 6.58 Ob Trauma - Vaginal W Instrument 222.63 Ob Trauma - Vaginal Wo Instrument 83.69 Ob Trauma - C-Section 5.95 Source: NIS, AHRQ Patient Safety Indicators SAS Software Version 2.1 Revision 2. Release pending.

Methods Evaluation framework Literature review
Identification of indicators Gray literature/interviews Evidence for indicators Empirical analyses ICD-9-CM coding review (PSI only) Clinical panel reviews (PSI only) 4 pronged approach, intended to provide information regarding the indicators. All were used to identify the indicators included in the final PSI set. Each will be described in details. Standard empirical analyses were conducted for descriptive purposes only, however, specified empirical analyses were done to help operationalize indicators (example – iatrogenic complications – try to figure out what complications it is capturing and whether they are meaningful).

Evaluation Framework Face validity: does the indicator capture an aspect of quality that is widely regarded as important and subject to provider or public health system control? Precision: is there a substantial amount of provider or community level variation that is not attributable to random variation? Minimum Bias: is there either little effect on the indicator of variations in patient disease severity and comorbidities, or is it possible to apply risk adjustment and statistical methods to remove most or all bias? Construct validity: does the indicator perform well in identifying true (or actual) quality of care problems? Fosters real quality improvement: Is the indicator insulated from perverse incentives for providers to improve their reported performance by avoiding difficult or complex cases, or by other responses that do not improve quality of care? Application: Has the measure been used effectively in practice? Does it have potential for working well with other indicators?

Literature Review Identification of Indicators
Systematic review to identify indicators Thousands of articles screened Over 200 abstracted Only 20 + articles actually described indicators, most of which had overlapping indicators Grey literature searched to identify over 200 indicators

Empirical Analyses Used novel statistical methods to measure
Precision/Reliability Bias Inter-relatedness of indicators Precision criteria of 1.0% or more systematic variation among providers Then, literature review conducted

Literature Review Evidence for Each Indicator
Identified and reported evidence for: Face validity Precision and reliability Potential bias Construct validity Fosters true quality improvement (gaming) Current use

PSIs Methods Development of Candidate Indicator List
Background literature review Little evidence in peer reviewed journals Complications Screening Program Miller et al. Patient Safety Indicators Review of ICD-9-CM code book Codes from above sources grouped into indicators and assigned denominators Review of CSP evidence to retain indicators Final refinements of indicators Complications Screening Program consists of 28 complications, developed by Lisa Iezzoni et al. for the purpose of identifying potentially preventable complications of adult medical and surgical hospital care, using administrative data. Uses ICD-9-CM diagnosis and procedure codes, patient age, sex, DRG, and date of procedure. CSP was developed using clinical judgement of developers and review of ICD-9-CM codebook. CSP uses “risk pools” instead of standard denominators – “major surgery, minor surgery, medical patients, etc.” Miller et al. PSIs – researchers reviewed ICD-9-CM code book, including CSP utilized codes. Selected indicators based on team judgement. Strict definition of patient safety event. 75% positive predictive value (3/4 patients identified by indicators actually had complication) 46% PPV for “process failure” (average for non-flagged cases) Final refinements of indicators. Changes to the denominator definitions intended to reduce the bias due to the inclusion or exclusion criteria Elimination of selected ICD-9-CM codes from numerator definitions, intended to focus attention on more clinically significant complications, or complications more likely to result from medical errors. Addition of selected ICD-9-CM codes to numerator definitions, intended to capture related complications that could result from the same or similar medical errors. Division of a single indicator into two or more related indicators, intended to create more clinically meaningful and conceptually coherent indicators. Stratification or adjustment by relevant patient characteristics (e.g. by procedure type for delivery, or high-low risk patients for decubitus ulcer).

PSIs Methods Review of Candidate Indicators
Literature review of potential indicators Coding validity/consistency Construct validity ICD-9-CM coding review Clinical panel review (face validity) Results used to define final set of indicators Literature review on candidate indicators identified reported validity of similar indicators – specificity and sensitivity of codes in identifying complications, specificity and sensitivity of codes in identifying complications caused by medical errors. ICD-9-CM coding review by coding specialist at AHIMA. Reviewed indicators to ensure appropriate use of ICD-9-CM codes, suggested additions and deletions and responded to team inquiries. Clinical panel review assessed face validity of the indicators, more detailed methods next slide.

PSIs Methods Clinical Panel Review
Intended to establish consensual validity Modified RAND/UCLA Appropriateness Method Doctors of various specialties/subspecialties, nurses, specialized (e.g., midwife, pharmacist) Initial rating, followed by conference call, followed by final rating Rated indicator on: Overall usefulness Present on admission Preventability of complication Likelihood due to medical error Extent indicator subject to bias Eight multispecialty panels, three surgical panels (5-9 members on each panel) Consensual validity – “extends face validity from one expert to a panel of experts who examine and rate the appropriateness of each item” RAND/UCLA appropriateness method designed to assess the appropriateness of a procedure given various indications. We modified the method substantially, but the idea is the same – use a structured process and a panel of experts to assess the appropriateness of the indicators for use for screening tools. Each panel had 5-9 members of various specialties. All included and internist, surgeon, at least 1 nurse (surgical or critical care) and critical care doc (except OB panels). Surgical panels included only surgical subspecialists. For some indicators specific specialties were required, such as a pharmacist for dosage complications, or cardiologists for CABG after PTCA. Each panel looked at 4-6 indicators. Surgical panels followed multispecialty panels.

Example reviews Multispecialty Panels
(5) (7) (4) (2) (6) (3) (8) Postop Pneumonia Decubitus Ulcer Overall rating Not present on admission Preventability (4) Due to medical error (2) Charting by physicians (6) Not biased (3) Ratings shown are for multispecialty panel and are on a 9 point scale, with 9 being more favorable (I flipped bias) Postoperative pneumonia – Interesting concept to panelists, but operationalization not doable. Preventability somewhat shakey given current definition, definitely not likely due to medical error, varied definition of “pneumonia” and case mix lead to bias. Overall wanted to use concept, but couldn’t do it well with data. In contrast… Decubitus ulcer was very attractive. It was an error felt to be very preventable, and given definition was not likely to be present on admission. Did change a few aspects of the indicator to include more patients in the denominator (removed age limit).

PSIs Methods Final Selection of Indicators
Indicators for which “overall usefulness” rating was high Some changes in indicator set based on coding review and operationalization concerns (e.g., reopening of surgical site) Empirical analyses of nationwide rates, variation, impact of risk adjustment, and relationship between indicators Final indicator set consists of indicators rated as a 6.5/8 or higher for overall usefullness. Changes were made due to coding review or other concerns that we believed would have changed the panels rating had they known the information at the time of rating – e.g. reopening of surgical site: 1) coding only includes certain reopenings, when not involving another procedure, including repairs 2) impossible to operationalize planned reopenings. Empirical analyses intended as descriptive analyses.

Risk-Adjustment Criteria
User-specified criteria for evaluating risk-adjustment systems 1) “Open” systems preferred 2) Data collection costs minimized and well-justified 3) Multiple-use coding system 4) Official recognition Also conducted interviews with stakeholders and found they preferred the following:

Evidence on DRG-based Systems
Open systems Widely adopted by state agencies Based on existing data collection systems Use for reimbursement ensures improved data quality Evidence suggests at least equivalent performance across broad spectrum of conditions Studies underway to examine alternatives

3M APR-DRG All-patient refined (956 categories in version 15.0, including pediatrics) Severity of illness subclass that reflect presence of co morbidity/complication and level Risk of mortality subclass Differential impact of secondary diagnosis by condition

Evidence on 3M APR-DRG All-patient refined (956 categories in version 15.0, including pediatrics) Severity of illness subclasses that reflect presence of co morbidity/complication and level Risk of mortality subclasses Differential impact of secondary diagnosis by condition

Evidence on 3M APR-DRG Better empirical performance than DRG-based alternatives on predicting mortality (especially for surgical patients; patients at large, urban, teaching hospitals) Better empirical performance than DRG-based alternatives on predicting resource use (especially for medical patients; patients over 65, at children, teaching hospitals) Better at reflecting the distribution of patient severity at the extremes

Risk-Adjustment Conclusions
No single system based on administrative or clinical data is clearly superior DRG-based systems perform as well, and often better, than alternatives Data enhancements may improve performance (e.g., condition present on admission, key clinical variables)

Risk-Adjustment Model Inpatient Quality Indicators
Direct standardization Indirect standardization RA = (OR / ER) * PR (RA – risk adjusted; OR – observed; ER – expected; PR – population)

Risk-Adjustment Model
Expected rate – Assuming the hospital’s case-mix and the population rates Risk-adjusted rate – Assuming the population’s case-mix and the hospital’s rates WILL BE DOING A CRASH COURSE THIS MONTH TO GET MORE FAMILIAR WITH THE MODELS

Linear regression model: observed rate = hospital effect + demographic effect + condition effect + error Model estimated on the SID, 2000 (25 million discharges)

IQI – Age, sex, APR-DRG (with risk of mortality or severity of illness subclass) (linear with hospital fixed effects) PQI – Age and sex (linear with area fixed effects) PSI – Age, sex, modified CMS DRG and AHRQ comorbidity (logistic)

How it Works: CABG Mortality
Covariate Freq. Effect Intercept 0.000 1652 0.240 -0.001 Sex 0.279 0.005 1653 0.131 0.020 Age1 0.204 0.001 1654 0.036 0.301 Age2 0.179 0.009 1661 0.127 Age3 0.195 0.013 1662 0.134 Age4 0.245 0.031 1663 0.086 0.014 Sex*Age1 0.049 0.010 1664 0.350 Sex*Age2 0.011 1631 Sex*Age3 0.060 0.018 1632 Sex*Age4 0.085 0.034 1633 0.016 0.051 1634 0.004 0.478 Other 0.077 0.163

How it Works: CABG Mortality
SID Effect Freq. Hosp Diff Effect* Sex 0.013 0.279 0.262 -0.016 0.000 56 to 63 0.005 0.204 0.256 0.052 64 to 68 0.016 0.179 0.161 -0.018 69 to 73 0.027 0.195 0.155 -0.039 -0.001 74 to high 0.054 0.245 0.165 -0.079 -0.004 Demographic Effect -0.005 Observed Rate 0.047 Risk-adjusted Rate

MSX Smoothing Model Observed quality measure = true quality (signal) + error (noise) Smaller hospitals and/or less frequent conditions have more noise Difficult to compare hospitals, trend over time, and identify best practices Confidence intervals reflect but do not address the problem

Key Features of MSX Approach
Removes noise – uses redundancy over time and among measures Improves forecasts – predicting current quality based on past performance Reduces dimensionality appropriately - allows meaningful summary measures Reveals and helps reduce biases, identify best practices

Caveats of Use Validity of data Incomplete risk adjustment
Validity of coding Present on admission Outpatient care Linking of admissions and impact of LOS Incomplete risk adjustment

Using the AHRQ QI State monitoring of rates
Hospital quality improvement National Healthcare Quality Report PQIs and PSIs CMS Pay for Performance Demonstration Project Postoperative Hemorrhage or Hematoma Postoperative Metabolic and Physiologic Derangement Romano et al PSI National trends, (HA, Mar/Apr ’03)

Using the AHRQ QI Kovner Miller Alexander/Shortell Baker Rosen Volpp
QI and nurse staffing Miller PSI and Costs and LOS Alexander/Shortell PSI and Quality improvement culture Baker PSI and Patient safety culture; hospital characteristics Rosen VA hospitals, QI and other measures (NSQuIP) Volpp QIs and new resident work hours

Technical Reports Development of Quality Indicators, risk adjustment and MSX methods documented in: Davies S, Geppert J, McClellan M, McDonald KM, Romano PS, Shojania KG. Refinement of the HCUP Quality Indicators. Technical Review Number 4. Rockville, MD: (Prepared by the UCSF-Stanford Evidence-based Practice Center under Contract No ) Agency for Healthcare Research and Quality; Report No.: McDonald KM, Romano PS, Geppert J, Davies S, Shojania KG. Measures of Patient Safety Based on Hospital Administrative Data: The Patient Safety Indicators. Technical Review Number 5. Rockville, MD: (Prepared by the UCSF-Stanford Evidence-based Practice Center under Contract No ) Agency for Healthcare Research and Quality; August Report No.:

For More Information on AHRQ QIs
Quality Indicators: Additional information and assistance Website: QI technical reports, documentation and software is available on the website User Support is provided under contract by Battelle Memorial Institute, Stanford University and University of California at Davis

AHRQ Quality Indicators

Similar presentations

Presentation on theme: "AHRQ Quality Indicators"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

AHRQ Quality Indicators

Similar presentations

Presentation on theme: "AHRQ Quality Indicators"— Presentation transcript:

Similar presentations

About project

Feedback