Patsi Sinnott, PT, PhD March 2011 Instrumental Variables Models

Outline 1. Causation Review 2. The IV approach 3. Examples 4. Testing the Instruments 5. Limitations 2

Causality Review Randomized trial provides the structure for understanding causation –Does daily dark chocolate affect health? –Does rehab reduce long term costs of care in patients with THR/TKR? 3

Randomization 4 Recruit Participants Random Sorting Treatment Group (A) Comparison Group (B) Outcome (Y)

Randomization In OLS (y i = α + βx i + ε i ) –The x’s explain the variation in y –ε = the random error –Randomization assumes a high probability that the two groups are similar 5

OLS Assumptions Right hand side variables are measured without noise (i.e., considered fixed in repeated samples) There is no correlation between the right hand side variables and the error term E(x i u i )=0 6

However Randomized trials may be –Unethical –Infeasible –Impractical –Not scientifically justified 7

Observational Studies Natural Experiment Administrative Data Observable characteristics (e.g. age, gender, smoking status) can be included in the model, BUT……. Treatment (sorting) is not randomly assigned All characteristics are not likely to be “generally” evenly distributed 8

Potential violation of OLS assumptions –Explanatory variables are not fixed in repeated samples –Regressors are correlated with the error term (endogeneity) 9

Violations occur ….. Measurement error in the explanatory variables Autoregression with autocorrelated errors Simultaneity (and reverse causation) Omitted variables Sample selection (in the population because of some unmeasured characteristic) 10

The IV approach Assumptions for the instrument –Causes substantial variation in the problematic variable –Independent  No direct or indirect effect on the outcome 11

Instrumental Variable Cause variation in the variable of interest No direct or indirect effect on the outcome 12 Instrumental variable Treatment Outcome Other variables

Example Wage = fn(years of education) 13

Whiteboard What’s missing (wage=fn (years of education) 14

Example School performance = fn (class size) 15

Whiteboard What’s missing: School performance = fn (class size) 16

Example Mortality after AMI = fn (cardiac cath.) 17

Whiteboard Mortality after AMI = fn (cardiac cath.) + 18

The IV approach School performance = fn (class size) + parent involvement Wage = fn (years of education) + ability Mort. after AMI = fn (cardiac cath) + other var. 19

IV (Omitted variable) Wage = fn (years of education) + ability –Instrument influence years of education but not ability (error term) –Distance to nearest college – related to years of college (not ability) 20

IV (Sample selection) School performance = fn (class size) + parent involvement –Class size  Dummy variable (split class due to enrollment changes)  Captures effect of smaller classes (without other unmeasured characteristics of students/parents associated with smaller class size) 21

IV Mortality after AMI = fn (cardiac cath.) + other var. –Instrument affects likelihood of cath. –No direct effect on mortality –Differential distance to nearest cath. hospital 22

Example Does more intensive treatment of AMI in the elderly reduce mortality (McClellan; McNeil; Newhouse. JAMA. 9/21/94) 23 No CathCath w/in 90 days Female53.5%39.7% Age in years Admit to cath hospital40.9%62.9% One day mortality10.3%.9% 7-day Mortality22.0%3.3% 30-day mortality26.6%7.4% 1-year mortality47.1%16.6% 2-year mortality55.3%21.3% 4-year mortality66.7%29.9% No. of observations158,26146,760

McClellan, McNeil and Newhouse Preliminary results –23% w/AMI received catheterization –Year 4 – 2/3 rd of those who did NOT receive cath had died (compared to 30% of cath group) 24

Are the groups different? No CathCath w/in 90 days Female53.5%39.7% Age in years Admit to cath hospital40.9%62.9% One day mortality10.3%.9% 7-day Mortality22.0%3.3% 30-day mortality26.6%7.4% 1-year mortality47.1%16.6% 2-year mortality55.3%21.3% 4-year mortality66.7%29.9% No. of observations158,26146,760 25

Results 37% points difference in mortality at 4 years in unadjusted 28% points difference with observable characteristics (age, gender)at 4 years (OLS) –7 percentage point difference in 1-day mort –13 percentage point difference at 7 days –18 percentage points at 30 days 26

IV for cath rate Differential distance between nearest hospital and nearest catheterization facility/hospital –Pts w/AMI will go to nearest hospital –Distance from nearest hospital to nearest cath hospital will be independently predictive of catheterization for similar patients. 27

IV for cath rate Zip code center to Zip code center If nearest hospital = cath hospital then DD = 0 Differential distance to cath hospital –≤2.5 mi. and > 2.5 miles 28

Characteristics by DD DD ≤ 2.5 milesDD > 2.5 miles Female51.3%49.5% Age76.1 Admit to cath hospital45.4%5.0% 90 day cath26.2%19.5% 1-day mortality7.5%8.88% 7-day mortality16.80%18.59% 30-day mortality24.86%26.35% 1-year mortality39.79%40.54% 2-year mortality47.20%47.89% 4-year mortality58.06%58.52% No. of observations102,516102,505 29

Results w/DD IV That variation in IV causes variation in the treatment variable (cath) is satisfied –( = 6.75% point greater chance of getting cath within 90 days following AMI) That DD affects outcomes only through likelihood of receiving treatment –reasonability 30

Results w/DD IV Patient characteristics appear balanced Hospital characteristics not 31 DD ≤ 2.5 milesDD > 2.5 miles Female51.3%49.5% Age76.1 Admit to cath hospital45.4%5.0% Admit to revasc. hospital41.7%10.7% Admit to high volume hospital 67.1%36.5% 90 day cath26.2%19.5%

Multiple regression results 32 Patient characteristics Hospital IVs including high volume hospital (1,0), rural residence (1,0) DD IV

Multiple regression w/IV results Rec’d cathAdmit hi-volumeRural residence 1-day mortality-5.0(1.1)*-.88 (0.24)0.57 (0.19) 7-day mortality-8.0(1.8)-1.23 (0.33)0.49(0.26) 30-day mortality-6.8(2.6)-1.45(0.38)0.50 (0.30) 1-year mortality-4.8(3.2)-1.07(0.88)-0.15 (0.33) 2-year mortality-5.4(3.3)-.88 (0.43)-0.02 (0.33) 4-year mortality-5.1(3.2)-.75 (0.42)0.14 (0.32) 33 * In percentage points, standard errors in parens.

Summary of results Unadjusted – 37 % age pt effect on four year mortality Adjusted w/out IV – 28 % age pt effect Adjusted w/DD IV – 6.9 % age pt effect Adjusted with DD, high volume hospital and rural IVs – approx 5 percentage pts. –The 5% age pt. difference at day 1 34

Inference RCT provides the average effect for the population eligible for the study Simple IV estimator –Effect of cath = ∆mortality/ ∆ cath rate = –=( )/( ) = –-0.46/6.7 = –That the 6.7% of patients who rec’d cath at nearby hospital had on average 6.9% percentage point additional chance of 4 year survival 35

Inference Wide angle view of the effects of treatment –More general population than RCT –External validity Marginal effect on a selection of the population: –Problematic for clinicians –Policy implications – incremental effects 36

Testing for strength of the instrument Tells a good story Validity of overidentification Does the instrument have the expected sign and is significant Compare to alternative instruments (if available) 37

Testing Defend assumption that instrument is NOT an explanatory variable Explain why instrument is not correlated with omitted explanatory variable 38

Testing Test if errors are correlated with regressors –Hausman test Test if instrument is uncorrelated with the error –Sargan test 39

Testing Test if the correlation between the instrument and the troublesome variable is strong enough –F statistic, regressing troublesome variable on all instruments – to test the null that the slopes of all instruments equal zero Staiger Stock test 40

Conclusion Instrumental variables mimic randomization But good instruments are hard to find. An estimate of the marginal effect/influence on outcome 41

References 1.McClellan M, McNeil BJ, Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA 1994;272: Newhouse JP, McClellan M. Econometrics in outcomes research: the use of instrumental variables. Annu Rev Public Health 1998;19: Rassen JA, Brookhart MA, Glynn RJ, Mittleman MA, Schneeweiss S. Instrumental variables I: instrumental variables exploit natural variation in nonexperimental data to estimate causal relationships. J Clin Epidemiol 2009;62: Rassen JA, Brookhart MA, Glynn RJ, Mittleman MA, Schneeweiss S. Instrumental variables II: instrumental variable application-in 25 variations, the physician prescribing preference generally was strong and reduced covariate imbalance. J Clin Epidemiol 2009;62:

References 5.Humphreys K, Phibbs CS, Moos RH. Addressing self-selection effects in evaluations of mutual help groups and professional mental health services: an introduction to two- stage sample selection models. Evaluation and Program Planning 1996;19: Kennedy P. A Guide to Econometrics. Sixth ed. Malden, MA: Blackwell Publishing;

Other IV methods Randomization 2SLS 44