1 Informational Strategies in Policy Design Organizational Report Cards By Dr. David L. Weimer

2 Information as Public Policy Problem Information as public good –If exclusion not possible, then too little produced –If exclusion possible, then too little consumed Informational asymmetry –One party to a transaction has information that would alter action of other party were it shared

3 Information as Public Policy Solution Direct provision by government –Product warnings –Public information campaigns Standards Labeling requirements Liability rules Reporting requirements

4 Comparing Services Provided by Organizations Comparative reports on many types of organizations are now common in the U.S. –Schools, colleges, and graduate programs –Hospitals and specific hospital services –Health Maintenance Organizations –Airlines –Insurance companies –Many more…

5 Organizational Report Cards Various ratings and ranking for comparing organizations William Gormley and I call these “organizational report cards” Today I will provide a brief overview of our research into organizational report cards as policy instruments

6 Organizational Report Cards: Definition “… a regular effort by an organization to collect data on two or more other organizations, transform the data into information relevant to assessing performance, and transmit the information to some audience external to the organizations themselves.”

7 NYS Coronary Artery Bypass Graft Surgery Report (NY-CSR) Since 1989, NYS has collected clinical data on all candidates for cardiac artery graft bypass surgery (approximately 20 thousand per year) Since 1990, makes public risk-adjusted mortality rates for all hospitals (currently 33 hospitals) Since 1991, makes public risk-adjusted mortality rates for all surgeons who performed more that 200 surgeries in last three years, or at least one surgery in each of these years

8 Service Accountability Top-down accountability – regulations, budgets, grants, oversight Bottom-up accountability – purchases by individuals, businesses, governments

9 Values for Assessing Report Cards Validity – measures performance Comprehensiveness – covers important outcomes Comprehensibility – facilitates understanding Relevance – informs decisions Reasonableness – minimizes compliance costs Functionality – induces appropriate behavior

10 Assessing Outcomes Outcomes achieved with available inputs Outcomes often cannot be measured: –For example, with respect to education Outcome: preparation for economic & political life Output: test scores Process: courses completed Inputs: teacher/student ratios

11 Measuring Organizational Performance

12 NY-CSR: Performance Measurement Outcome: Survival of surgery and subsequent hospital stay Risk-adjustment: probability of mortality for each patient based on statistical model with clinical risk factors as explanatory variables Performance: risk-adjusted mortality rates for hospitals and individual surgeons

13 Effective Communication Information must be available –News stories alert consumers to availability –Increasingly, the Internet provides easy access Information must be accessible –Includes non-technical interpretations –Provides simple summaries: ratings, ranks

14 Supply Effects: Organizational Responses Informed consumer choice (or oversight) may improve quality of services –CS mortality rates in NYS fell by 35% between 1989 (3.5%) and 1997 (2.2%) –Lowest reported rate in U.S. Impacts on supply depend on organizational responses

Organizational Responses

16 The Bright Side: Functional Responses Process improvement: Observe practices in better performing organizations Input reallocation: Move resources to more valued uses – perhaps terminate supply Managerial focusing: Direct attention to service quality Mission enhancement: Comparisons reinforce organizational-wide efforts

17 The Dark Side: Dysfunctional Responses Self-selection: Avoid being compared if possible Cream skimming: Take the easy cases, avoid the hard Teaching to the test: Emphasize only the outputs and outcomes being measured Deception: Cheat! Blaming the messenger: Divert effort toward attacking the report card

18 Government Roles in Report Card Production Data gathering and verification –Mandate participation or reporting –Support private efforts to develop data Data analysis and presentation –Supply when market demand is weak –Contract-out where appropriate Information dissemination –Provide information to mass media

19 Conclusion Well-designed organizational report cards can be a useful policy instrument for improving the quality of services –Sometimes alternative to direct regulation Poorly-designed organizational report cards can make things worse! Design and assessment must be done on a case-by- case basis –Attention to relevant values –Attention to appropriate public roles