Judea Pearl Computer Science Department UCLA DIRECT AND INDIRECT EFFECTS

QUESTIONS ASKED Why decompose effects? What is the semantics of direct and indirect effects? What are the policy implications of direct and indirect effects? Can path-analytic techniques be extended to nonlinear and nonparametric models? When can direct and indirect effect be estimated consistently from experimental and nonexperimental data.

1.Direct (or indirect) effect may be more transportable. 2.Indirect effects may be prevented or controlled. 3.Direct (or indirect) effect may be forbidden WHY DECOMPOSE EFFECTS? Pill Thrombosis Pregnancy +  + Gender Hiring Qualification

EFFECT-DECOMPOSITION IN LINEAR MODELS XZ Y c a b abc Definition:

COUNTERFACTUALS: STRUCTURAL SEMANTICS Notation: Y x (u) = y Abbreviation: y x Formal: Y has the value y in the solution to a mutilated system of equations, where the equation for X is replaced by a constant X=x. u Y x (u)=y Z W X=x u Y Z W X Probability of Counterfactuals: Functional Bayes Net

TOTAL, DIRECT, AND INDIRECT EFFECTS HAVE SIMPLE SEMANTICS IN LINEAR MODELS XZ Y c a b z = bx +  1 y = ax + cz +  2 a + bc bc a

z = f (x,  1 ) y = g (x, z,  2 ) XZ Y SEMANTICS BECOMES NONTRIVIAL IN NONLINEAR MODELS (even when the model is completely specified) Dependent on z ? Void of operational meaning?

Indirect Effect? NEED OF FORMALIZATION XZ AND Y = What is the direct effect of X on Y?

Starting from X=0, (and Z=0 and Y=0) Total Effect: Change X from 0 to 1, and test the change in Y. Controlled DE: Keep Z constant at Z=0, or Z=1, and change X=0 to X=1. Controlled IE: None. Natural DE: Keep Z constant at its current value, and change X to 1. Natural IE: Keep X at 0, but set Z to what it would be if X were 1. TWO CONCEPTIONS OF DIRECT AND INDIRECT EFFECTS: Controlled vs. Natural XZ AND Y =

``The central question in any employment-discrimination case is whether the employer would have taken the same action had the employee been of different race (age, sex, religion, national origin etc.) and everything else had been the same’’ [ Carson versus Bethlehem Steel Corp. (70 FEP Cases 921, 7 th Cir. (1996))] x = male, x = female y = hire, y = not hire z = applicant’s qualifications LEGAL DEFINITIONS TAKE THE NATURAL CONCEPTION (FORMALIZING DISCRIMINATION) NO DIRECT EFFECT

Starting from X=x *, (and Z=Z x* (u) and Y= Y x* (u)) Total Effect: TE(x,x * ;Y) = E(Y x ) – E(Y x* ) Controlled DE: CDE Z (x,x * ;Y) = E(Y xz ) – E(Y x*z ) Controlled IE: None. Natural DE: NDE(x,x * ;Y) = E(Y xZ x* ) – E(Y x* ) Natural IE: NIE(x,x * ;Y) = E(Y x*Z x ) – E(Y x* ) TWO CONCEPTIONS OF AVERAGE DIRECT AND INDIRECT EFFECTS: POPULATION-LEVEL DEFINITIONS XZ y = f (x,z,u) Y u2u2 u3u3 u1u1 Probabilistic causal model:  P(u)M, (all other parents of Y)

z = f (x,  1 ) y = g (x, z,  2 ) XZ Y THE OPERATIONAL MEANING OF AVERAGE DIRECT EFFECTS “Natural” Direct Effect of X on Y: The expected change in Y per unit change of X, when we keep Z constant at whatever value it attains before the change. In linear models, NDE = Controlled Direct Effect

POLICY IMPLICATIONS (Who cares?) f GENDERQUALIFICATION HIRING What is the direct effect of X on Y? The effect of Gender on Hiring if sex discrimination is eliminated. indirect XZ Y IGNORE

z = f (x,  1 ) y = g (x, z,  2 ) XZ Y THE OPERATIONAL MEANING OF INDIRECT EFFECTS “Natural” Indirect Effect of X on Y: The expected change in Y when we keep X constant, say at x 0, and let Z change to whatever value it would have under a unit change in X. In linear models, NIE = TE - DE

Example: Theorem: If there exists a set W such that GRAPHICAL CONDITION FOR EXPERIMENTAL IDENTIFICATION OF AVERAGE NATURAL DIRECT EFFECTS

HOW THE PROOF GOES? Proof: Each factor is identifiable by experimentation.

GRAPHICAL CRITERION FOR COUNTERFACTUAL INDEPENDENCE U3U3 U1U1 X Y Z U2U2 U3U3 U1U1 XZ Y U2U2 U3U3 U1U1 X Y U2U2 Z

GRAPHICAL CONDITION FOR NONEXPERIMENTAL IDENTIFICATION OF AVERAGE NATURAL DIRECT EFFECTS Identification conditions 1.There exists a W such that (Y Z | W) G XZ 2.There exist additional covariates that render all counterfactual terms identifiable.

Corollary 3: The average natural direct effect in Markovian models is identifiable from nonexperimental data, and it is given by where S stands for all parents of X (or another sufficient set). IDENTIFICATION IN MARKOVIAN MODELS X Z Example: S =  Y

How effective would the drug be if we eliminate its side-effect (Headache)? POLICY QUESTION ANSWERED BY NATURAL DIRECT EFFECT Outcome Drug Aspirin X Y Z W Headache

NIE(x,x * ;Y) = Expected increase in sales, if we bluff the competitor into believing that X is about to change from x * to x. For Markovian models: POLICY-BASED INTERPRETATION OF INDIRECT EFFECTS X Y Z (Sales) (Advertisement Budget) (Competitor’s Budget)

Theorem 5: The total, direct and indirect effects obey The following equality In words, the total effect (on Y) associated with the transition from x * to x is equal to the difference between the direct effect associated with this transition and the indirect effect associated with the reverse transition, from x to x *. RELATIONS BETWEEN TOTAL, DIRECT, AND INDIRECT EFFECTS

Y Z X W x*x* z * = Z x* (u) Nonidentifiable even in Markovian models GENERAL PATH-SPECIFIC EFFECTS (Def.) Y Z X W Form a new model,, specific to active subgraph g Definition: g -specific effect

SUMMARY OF RESULTS 1.New formulation of path-specific effects, based on signal blocking, instead of value fixing. 2.Path-analytic techniques extended to nonlinear and nonparametric models. 3.Conditions for estimating direct and indirect effects from experimental and nonexperimental data. 4.Estimability conditions hold in Markovian models. 5.Graphical techniques of inferring effects of nonstandard policies, involving signal blocking.