State Health Access Reform Evaluation Lynn A. Blewett, Ph.D. State Health Access Data Assistance Center AcademyHealth State Coverage Initiatives (SCI) State Coverage Institute Chicago, Illinois September 26, 2007 Funded by a grant from the Robert Wood Johnson Foundation

2 Overview of Presentation Quick Review-State Health Reform Evaluation 101 Examples: Performance Outcomes for Health Reform Data sources – State vs. National Technical Assistance –State Health Access Reform Evaluation –State Health Access Data Assistance Center

3 Presentation Objectives To get you to think about measurable outcomes during your discussions of health reform To be specific about your expectations for reform initiatives –How you will know if the program achieved its objectives? What data will you need? To talk to each other and reach consensus on expectations….. Think about state-specific data needs and build those into your legislation

Quick Overview of State Health Reform

5 Drivers of State Health Reform Activity Increasing number of uninsured –Drop in employer-sponsored coverage –Kids impact moderated by SCHIP –No safety net for adults Increasing number of underinsured –Higher out-of-pocket costs Relatively positive state revenue climate Lack of national efforts for reform –Iraq, immigration, etc. dominating Congress

6 Source: U.S. Census Bureau, Current Population Surveys (March), New verification question Continued Increase in Uninsured Millions of Uninsured, all ages 15.8% of Population

7 Drop in Employer-Sponsored Coverage (U.S.) Source: US. Census Bureau, Income, Poverty, and Health Insurance Coverage in the United States: 2006.

8 Increase in Uninsured Children, ,000 New Uninsured Children <200% FPL % FPL 400%+ FPL 220, , , % 47.5% 21.2% Note: 200% to 399% of the federal poverty level (FPL) is roughly $40,000- $80,000 in annual income for a family of four in Source: KCMU/Urban Institute analysis of the March 2007 CPS.

9

10 Source: Quarterly Summary of State and Local Government Tax Revenue, U. S Census Bureau. Annual Change in 2nd Quarter State and Local Tax Revenue Increase in State Revenues

11 Source: Congressional Budget Office Federal Relief?

12 States Are in Different Phases of Reform Process In Discussion Legislation Introduced Legislation Passed/ Implementation Post – Implementation Colorado Minnesota Illinois (all) New Jersey New Mexico PA (all) WI (all) California Oregon WI Massachusetts PA (kids) Vermont WI (kids) Illinois (kids) Maine

13 Framework for Reform Public Sector Initiatives Employer- Based Initiatives Insurance Reform Tax credits HIFA expansion waivers SCHIP premium assistance Small employer purchasing pools

Health Reform Evaluation -101

15 Ideal Policy Making Process Informed Decision-Making Process –Made based on best available data and analysis –Decision maker is knowledgeable, interested and has best information and advice –Science and methods are sound –Thoughtful discussion of alternatives –Public is informed

16 POLICY FORMULATION -Agenda Setting -Assessment of Options -Political Setting POLICY IMPLEMENTATION POLICY Adoption POLICY MODIFICATION Program Evaluation, Assess impact of policies Influence future policy formulation INPUTS Preferences of individuals, organizations, interest groups, along with cultural Demographics, ethical, legal, social and technological inputs RulemakingOperation Source: Longest, 1998 Longest, 1988.Policy Making Process in the US

17 Performance Assessment/Evaluation OBJECTIVE –to provide information/data/evidence… –to support resource allocation and other policy decisions…. –to improve service delivery and program effectiveness.

18

19 Performance Assessment or Program Evaluation? Performance Measurement –ongoing assessment –use of performance standards –early warning system to management –vehicle for improving accountability to the public Program Evaluation –typically more in-depth examination of program –broader context of environment and alternatives –overall assessment of whether the program works –identification of adjustments that may improve its results Source: U.S. Government Accountability Office, 1998.

20 The Importance of Reform Evaluation Inform future state policy Justify program budget/financing Satisfy legislative requirements Identify successful initiatives Change course if program not meeting benchmarks Inform state and national debate on health reform –What works at the state level? Source: CMS Evaluating Demonstrations: A technical assistance guide for states.

21 Program Objectives Measurable results that the program sets out to achieve SMART Objective S – Specific M – Measurable A – Appropriate R – Realistic T – Time bound Focus of objective  What will change? Not what the program will do

22 Levels of Measurement Population-based –Data collected from sample of target population –Survey data, focus groups Program-based –Data collected from program recipients, clients, or participants –Administrative data

23 Program Evaluation Data Quantitative Data –Surveys, secondary data –Measures activities carried out –Use to assess extent of program utilization, behavior change, or cost per unit of change Qualitative Data –Interviews, focus groups, observations, document review –Use to understand attitudes, beliefs, reactions, to assess quality, to explain perceptions

24 Performance Assessment How do you know if your health reform was successful? Key Areas to Assess: –Coverage –Cost and Efficiency –Fairness and Equity –Choice and Autonomy Develop goals that are measurable, assessable and realistic. Consider short-term and long-term outcomes.

Examples: Performance Outcomes for Health Reform

26 Goals of Vermont Reform Increase Access to Affordable Health Care Coverage –More than 96% of Vermonters will have coverage. Improve Quality of Care –100% of Medicaid, Dr. Dynasaur, and VHAP beneficiaries living with a chronic condition will receive health care services that are based on nationally recognized clinical best-practice guidelines for health treatments and self care. Contain Costs –The cost shift resulting from insufficient Medicaid reimbursement rates will be reduced. Source: Vermont Health Care Reform: 5 Year Strategic Plan

27 Goals of Maine Reform Increase Access to Affordable Health Care Coverage –Aim to insure up to 31,000 individuals during its first year and provide access to coverage for at least another estimated 110,000 individuals by Improve Quality of Care Contain Costs through Voluntary Caps –Dirigo Health requests hospitals and other providers to limit their cost growth to 3% per year and their operating margins to 3.5%. Insurers are asked to limit their operating margin to 3.5%.

28 Additional Examples Process Measurement –Time from first contact to enrollment –Waiting list, backup list –# of contacts per outreach effort –# of contacts that result in enrollment Outcomes Measurement –Increase coverage for low-income children by 30% –Increase the number of plans offering insurance –Reduce growth in public program spending by 10% –Increase immunization rates for non-white populations to meet 2010 goals –Estimates of direct substitution of public for employer- based plan (i.e., Crowd Out)

29 Key Dimensions of Reform Increase access and coverage plus demonstrating that reform initiative is: Affordable Sustainable Efficient

Data Sources – State vs. National

31 Assumption “Good policy needs good data and collecting good data is good policy.” Source: Ernie Ingles, Chief Librarian, University of Alberta

32 Current Population Survey (CPS) Annual estimate of distribution of health insurance coverage Released by the Census Bureau every August Only federal survey with uninsurance estimates for all 50 states and DC Used to compare coverage and change in coverage across states and for nation

33 CPS Also Key to SCHIP Funding Formula inputs: estimates of low-income children and low-income uninsured children in each state –Pooled years of CPS data –Still small sample size, especially for small states –Bias in data skews distribution of funds State administrative data is not used in SCHIP funding formula –More kids enrolled, less uninsured, less allotment Is there a better state-level data source to inform formula inputs?

34 SCHIP Evaluation Requirements States required to monitor changes in levels of uninsurance for low-income kids –Back to CPS? –State survey? –Some state using administrative counts to show increase in public program enrollment New- Demonstrate that public program expansion does not lead to drop in employer sponsored coverage –Enrollee surveys? –Employer surveys?

35 HRSA-SPG Funded Survey Activity Household and Employer Survey Household Survey

36 Why States Conduct Surveys Typically more sample than CPS Ability to drill down to subpopulations –Children –Geographic Units –Race/ethnicity Analysts have data in hand –Better knowledge of data –Ability to do analysis in-house

37 On Average Uninsurance Estimates from State Surveys are 22% Lower than CPS Uninsurance Estimates for Select States

38 Increased Sample Size in State Surveys

39 States Use of Data Four primary areas of policy development: (1) To develop reliable and accurate estimates of the number of uninsured –To facilitate state-specific policy discussions (2) To identify the demographic, economic and health-related characteristics of the uninsured –To help target programs and outreach

40 Focus Groups (3) To collect information about opinions, attitudes, and values -- To facilitate the design policy options -- To assess political feasibility (4) To understand the motivations of uninsured populations and employers -- To design approach and incentives States Use of Data (2)

Technical Assistance - State Health Access Reform Evaluation - State Health Access Data Assistance Center

42 State Health Access Data Assistance Center Funded by the Robert Wood Johnson Foundation –University of Minnesota, School of Public Health Goals: –Help states monitor rates of health insurance coverage using state and federal data –Research factors associated with access and coverage –Provide targeted policy analysis and technical assistance to states Bridge between state and federal agencies; between survey data and state health policy Technical assistance on use of federal survey data and best practices on state surveys

43 State Health Access Reform Evaluation National Program Office of the RWJF –Co-located with SHADAC at University of MN, SPH Goals: –Fund and coordinate evaluations of state health reform –Identify and fill gaps on research to identify what works and why –Organize and disseminate findings in a manner that is meaningful and user-friendly –Inform state and national policy on health care access and coverage Technical assistance on program evaluation and assessment

44 State Health Reform If States are the Laboratories….. Where are the lab reports?

45 Concluding Comments Staff - Ask key legislative leaders what they will want to know next year? Discuss long- and short-term program objectives and identify the monitoring process up front Make sure staff understand legislative/policy objectives Make sure policy makers understand limitations of data/measurement Build performance assessment into legislative initiatives

46 More Concluding Comments Don’t forget to think about…. Performance assessment and –Monitoring along the way –Likely within state government Program evaluation –How are we going to know in 2-3 years if the program achieved its objectives? –Possible outside vendor/contractor Think now about what you will want to know later –It’s more difficult to go back and evaluate a program without good baseline data and information

47 Federal Data (e.g., CPS) Better Understanding of the Characteristics of the Uninsured State Data (e.g., State Surveys) The SHADAC Vision Increase Coverage and Access THANK YOU!

48 Contact Information University of Minnesota School of Public Health Division of Health Policy and Management 2221 University Avenue, Suite 345 Minneapolis Minnesota (612) Principal Investigator: Lynn A. Blewett, Ph.D.