Hospital Ownership and Performance: An Integrative Research Review Research-in-Progress Seminar Stanford, May 11, 2005 Yu-Chu Shen Naval Postgraduate School and NBER Karen Eggleston, Joseph Lau, Christopher Schmid Tufts University Funded by grant # under the Robert Wood Johnson Foundation’s Changes in Health Care Financing and Organization (HCFO) Initiative Preliminary: work in progress Comments welcome

Presentation Outline Research objective Brief theory background State of the empirical literature Scope of our integrative review Analytical methods to synthesize literature Results Discussions

Mixed Ownership Is an Abiding Feature of Healthcare Delivery in the US Source: Eggleston (2004), based on Rorem (1930); Hayes (1954); American Hospital Association Hospital Statistics (various years).

Research Objective Does ownership affect hospital performance (quality, finance, or provision of uncompensated care)? Competing theories with contrasting predictions Hundreds of empirical studies to date with conflicting findings policymakers have little clear evidence economics of ownership and behavior imperfectly understood

Taxonomy of Theories of Nonprofits Complete Information Incomplete Information Objectives Differ Altruism / Quality- Quantity Maximization (Newhouse 1970; Lakdawalla & Philipson 1998) Physicians’ Cooperative (Pauly & Redisch 1973) NPs help government fulfill demand for collective goods (Weisbrod 1975) NPs as “for-profits in disguise” (Weisbrod 1988) Government favoritism of NPs because ‘trustworthy’ (James 1985) Choice (objectives need not differ) Regulation and tax treatment: Firms differ in ability to benefit from a given ownership form (David 2004; Lakdawalla & Philipson 1998) Trust signal and concontractible quality (Arrow 1963; Hansmann 1980; Glaeser and Shleifer 1998) Mechanism for consumer control (Ben-Ner and Gui 1993) (modified from Guy David 2004)

Empirical Predictions Consistent With Some Ownership Theories

Mixed Empirical Evidence  Studies differ widely in analytic methods  Mixed and inconclusive evidence on whether ownership differs and the magnitude of differences in quality, cost, and social benefits  We use meta analytical methods to combine quantitative evidence from different studies

Scope of the Integrative Review  Synthesize the main findings of the empirical literature between 1990 and July 2004 on hospital ownership and performance (published or unpublished)  Examine multivariate empirical studies of US acute general short stay hospitals;  Examine studies that compare differences between for-profits and nonprofits, between nonprofits and government, or both.

 Focus on four broad categories of performance measures: financial performance (efficiency, cost, revenue, profit) quality / patient outcomes uncompensated care or community benefits Staffing  Presenting only findings from financial performance measures Scope of the Integrative Review

Literature Selection Process  1434 potentially relevant studies from 1990 to 2004 were identified and screened for retrieval through search engines (EconLit, MedLine, Proquest, ABI) contacting all corresponding authors of initially included studies

Defining Study Population 1434 potentially relevant studies Non-hospital studies (n=529) Non-US hospital studies (n=83) Non-acute general short stay hospitals studies (n=36) Acute general short stay hospital studies (n=786)

Inclusion and Study Design Criteria Acute general short stay hospital studies (n=786) Not ownership comparison studies (n=385) Non-empirical studies and case studies (n=61) Empirical studies on ownership (n=340)

Outcome Criteria and Other Exclusions 141 empirical studies fit the selection criteria.

Number of Studies By Category of Hospital Performance Performance Measure Number of studies with this as “primary” measure* Number of studies that analyze this measure Cost, revenue, profits 4547 Efficiency 2122 Cost shifting 34 Staffing 813 Patient outcomes 4459 Uncompensated care/community benefits 1623 Other misc outcomes 44 TOTAL

Detailed Financial Performance Categories # of articles Total or operating cost 21 Profit margin (total or operating) 14 Patient revenue or returns on assets 13 Overall efficiency 12

Detailed Financial Performance Categories Technical efficiency--efficient levels of inputs 6 cost of a specific disease 5 Medicare cost 5 scale efficiency--efficient output/input mix 3 misc efficiency measures 3 labor or personnel cost 3 allocative inefficiency--efficient mix of inputs 2 debt/asset ratio 2 payroll/labor as a share of revenue or expense 2 changes in total or operating cost 7 changes in total or patient revenue 3 changes in other financial measures 3 changes in profit or margin 1 misc financial outcomes 14

How much work is a systematic review? Allen and Olkin (1999) analyze 37 meta-analyses Average hours were 1138 per study Based on their formula, it implied 1,044 hours for our review

Analytical Methods  A typical study estimates the impact of ownership on performance as follows:  The coefficients β1 and β2 capture the effect on Y of for-profit and public ownership, respectively, relative to nonprofit ownership

Defining Effect Size of Ownership Studies (1)  The goal of our integrative analysis is to answer the following questions: 1.What is the magnitude of the relationship between ownership and performance—what is the effect size? 2.How precise or reliable is this estimated effect size? 3.How do differences in analytic methods and other study features affect the estimates of effect size?

Defining Effect Size of Ownership Studies (2)  Problems with using β1 and β2 directly from studies Heterogeneous dependent variables Effect size can be measured in actual dollars or in percentage.

Defining Effect Size of Ownership Studies (3)  Partial correlation coefficient as a measure of effect size: Y*=the residuals in a regression of Y on a set of X FP*=the residuals in a regression of FP ownership on a set of X The partial correlation r* is the simple correlation between Y* and FP*.  r* measures the correlation between a given ownership and Y controlling for the effect of X

Partial Correlation Coefficient As Effect Size  Partial correlation coefficient, r, measures the correlation between a given ownership and Y controlling for the effect of X  It can be derived from commonly reported statistics: r=  It’s unit free, so comparable across a heterogeneous set of studies  Unlike t-statistics, magnitude of r does not depend on sample size

Adjusting Effect Size Estimates  The distribution of r* becomes more skewed as the population value of r* gets further and further away from zero.  We apply Fisher (1928) transformation that is distributed nearly normally:

Estimating Confidence Intervals Around the Effect Size  Adjusted effect size =  Variance(Z r )=  95% Confidence interval of the adjusted effect size=

Combining Effect Sizes Across Studies (1)  A common way to combine study results is to compute a weighted average effect size:  The weight that minimizes the variance of this measure is the inverse of the effect size variance from each study:

Combining Effect Sizes Across Studies (2)  Variance of weighted average effect size:  Confidence interval of the combined effect size measures:

Issues In Combining Effect Size For Hospital Ownership Literature  Research questions are not homogeneous.  Studies vary widely in analytical methods. Need to categorize the methods in some ways.  Overlapping hospitals and data sources Unlike randomized clinical trials with independent samples, there are fewer than 5000 general acute hospitals in the US. Many studies analyze almost the entire population of hospitals Furthermore, most studies use one of two common data sources.

Research Questions Vary  Fixed- or random-effects models? When the combined studies are a homogeneous set designed to answer the same question in the same population, a fixed-effects model is appropriate. When heterogeneity is detected, random-effects models are used, which assume that there is no single truth, but a distribution of such truths.

Random Effects Model  “True effect size” is not fixed. The variance of effect size from each study is assumed to have two components: Between-studies variance Within-study variance  Because of the additional between- studies variance, random effects model tend to be more conservative than fixed-effects model.

Categorizing Analytical Methods  Three types of methodology rigor  Type 3: if a study meets both of the following conditions: (a) uses panel estimation or explicitly accounts for potential selection problem (b) includes two of the following three sets of controls: patient level, hospital level, market level  Type 2: if meets EITHER (a) or (b)  Type 1: if meets NEITHER (a) nor (b)

Overlapping Sample and Data Sources  Partial correlation coefficients are valid effect size measures when observations are correlated.  Meta regression has been suggested as a possible imperfect solution by including dummies of the common data sources.  No satisfactory solutions to date.

Meta Regression  The regression approach allows us to examine whether differences in effect sizes across studies can be explained by analytical methods, region studied, years covered, or other study features.  The dependent variable is the effect size from each study.  The explanatory variables are the empirical features of each study (differ across financial measures)  The model is necessarily parsimonious due to sample size issues.

Integrative Review of Hospital Overall Efficiency  Overall efficiency is usually defined as least cost production or least amount of input for a given level of output.  Two common ways to estimate overall efficiency: Stochastic frontier approach Data envelopment analysis (DEA) Both are controversial (e.g. Newhouse 1994)  10 studies contain N-F comparison  7 studies contain N-G comparison

Efficiency: Summary of Effect Size By Methods (N-F)

Efficiency: Summary of Effect Size By Decade (N-F)

Efficiency: Summary of Effect Size By Covered Region (N-F)

Efficiency: Potential Publication Bias? N-F Comparison

Integrative Review of Hospital Cost: N-F Differences  Studies assume different functional forms for the cost model: Log (total cost):  6 studies/10 observations Log (average cost per admission):  5 studies/11 observations Average cost per admission/discharge:  3 studies/6 observations Others:  4 studies/4 observations

Cost: Summary of N-F Effect Size By Cost Definition Log(Total Cost) Log(Avg Cost) Avg Cost Other Cost Def

Cost: Summary of N-F Effect Size By Decades Data from 1980s Data from 1990s

Cost: Summary of N-F Effect Size By Covered Years 1 year of data Multiple year of data

Cost: Summary of N-F Effect Size By Method Types Method Type 1 Method Type 2 Method Type 3

Cost: Potential Publication Bias? N-F Comparison

Integrative Review of Hospital Revenue: N-F Differences  Studies assume different functional forms for the revenue model: Log (average revenue):  3 studies/4 observations Average cost per admission):  3 studies/3 observations Returns on assets:  5 studies/5 observations Others:  1 study/1 observations

Revenue: Summary of N-F Effect Size By Revenue Definition Log(average revenue) Average revenue Returns on assets

Revenue: Summary of N-F Effect Size By Covered Region

Revenue: Summary of N-F Effect Size By Method Types

Revenue: Potential Publication Bias? N-F Comparison

Integrative Review of Profit Margin: N-F Differences  Profit margins are usually defined in the form of (revenue-cost)/revenue.

Profit Margin: Summary of N-F Effect Size By Covered Region CA FL Urban National VA

Profit Margin: Summary of N-F Effect Size By Method Types

Profit margin: Potential Publication Bias? N-F Comparison

What Do We Learn? (1)  Evidence is pretty conclusive regarding revenue and profit margins For-Profits tend to earn more revenue (per admission) and have higher profit margins  There is little evidence of any difference in cost between FP and NP hospitals  Evidence is inconclusive regarding efficiency. Although almost all individual studies report significant findings, collectively their results are not consistent.

What Do We Learn? (2)  Functional forms and analytical methods matter Weaker methods and functional forms tend to predict larger differences between nonprofits and for-profits  National samples tend to produce more conservative estimates of effect size  No evidence of publication bias