Almost-Exact Matching for Causal Inference Cynthia Rudin Associate Professor Departments of Computer Science, Electrical and Computer Engineering and Statistical Science Duke University I’m not talking about decision trees, I talk about them too much so I decided to change at the last minute to causal inference. When you’re working with observational data you ideally want to find identical twins in the treatment and control groups. But you can’t, because very few individuals are really alike in high dimensions. So we want to match them on as many relevant covariates as possible and that’s what my talk is about.

FLAME: Fast Large-scale Almost Matching Exactly (with Sudeepa Roy, Alexander Volfovsky, Tianyu Wang, Awa Dieng, Yameng Liu)

Matching in Potential Outcomes Framework n x p n x 1 n x 1 {0,1} X, Y, T observational data, SUTVA, strong ignorability I always say that causal inference is half supervised. For each treatment unit, you know their outcome, for each control unit, you know their outcome, but you never get the *treatment effect* for any individual. You never know what would happen if you gave the treatment to that control unit. Matching is a popular way to handle causal inference problems, because can be really interpretable. Causal Inference is ”half supervised” Matching is interpretable. “Identical” twins. Most matching methods don’t try to match exactly.

covariates: age, gender, heart conditions, blood pressure, toenail length, eyeball width, etc treated patient Marietta [ 50 F 1 0 1 1 68 1.5cm 2cm 1 0 3 0 ..... ] control patient 1 Lee Ann [ 50 F 1 0 1 1 68 14cm 1cm 4 1 5 6 ..... ]

FLAME: Fast Large-scale Almost Matching Exactly Goal: Match treatment and control units using as many important covariates as possible Handle large data sets Work fast BasicExactMatch subroutine uses an efficient database query:

FLAME: Fast Large-scale Almost Matching Exactly Use a holdout training set to determine How important a variable is for prediction error Whether a subset of variables predicts sufficiently well

FLAME: Fast Large Almost Matching Exactly Algorithm: Start with exact matching on all covariates. Find as many matched groups as possible (using BasicExactMatch). Eliminate the least important covariate Repeat, find as many matched groups as possible each time we eliminate a covariate. Match Quality on training set: MQ = -PredictionError + C*BalancingFactor Prediction error constraint: Always keep enough covariates to be able to predict the outcome well. Balance constraint: Do not ever eliminate too many points from either treatment or control groups.

Some (Insightful) Experiments No ground truth so need simulated data U=5 (no noise) 20K units, 10K treatment, 10K control

Regression cannot handle model misspecification

Some (Insightful) Experiments (no noise)

On the dataset I will show next

Collapsing FLAME Try to match each treatment unit to at least one control on as many variables as possible. Match on all variables. Temporarily remove variable 1, get matches, put it back. Temporarily remove variable 2, get matches, put it back. : Temporarily remove variables 1 AND 2, get matches, put them back. Temporarily remove variables 1 AND 3, get matches, put them back

Breaking the Cycle of Drugs and Crime in the United States First covariate eliminated Alabama, Florida, and Washington Participants were chosen to receive screening shortly after arrest and participate in a drug intervention under supervision, control group from same population. Last covariate eliminated

CORELS Rule List for treatment effect: Positive estimated treatment effect Negative estimated treatment effect Siong Thye Goh

Takeaway Most matching methods can't handle irrelevant variables. FLAME leverages ideas from ML + databases scalable fast accurate passes sanity checks FLAME's code is here: https://github.com/ty-w/FLAME

Thanks FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference (with Sudeepa Roy, Alexander Volfovsky, Tianyu Wang) Code: https://github.com/ty-w/FLAME