1. 1 Day 2: Search June 9, 2015 Carnegie Mellon University Center for Causal Discovery

2. Outline Day 2: Search 1.Bridge Principles: Causation  Probability 2.D-separation 3.Model Equivalence 4.Search Basics (PC, GES) 5.Latent Variable Model Search (FCI) 6.Examples 2

3. 3 Causal Structure Testable Statistical Predictions Causal Graphs e.g., Conditional Independence X _||_ Z | Y  x,y,z P(X = x, Z=z | Y=y) = P(X = x | Y=y) P(Z=z | Y=y)

4. 4 Bridge Principles: Acyclic Causal Graph over V  Constraints on P(V) Weak Causal Markov Assumption V 1,V 2 causally disconnected  V 1 _||_ V 2 V 1,V 2 causally disconnected  i.V 1 not a cause of V 2, and ii.V 1 not an effect of V 2, and iii.No common cause Z of V 1 and V 2

5. 5 Bridge Principles: Acyclic Causal Graph over V  Constraints on P(V) Weak Causal Markov Assumption V 1,V 2 causally disconnected  V 1 _||_ V 2 Causal Markov Axiom If G is a causal graph, and P a probability distribution over the variables in G, then in satisfy the Markov Axiom iff: every variable V is independent of its non-effects, conditional on its immediate causes. Determinism (Structural Equations)

6. 6 Causal Markov Axiom Acyclicity d-separation criterion Graphical Independence Oracle Causal Graph Z X Y1Y1 Z _||_ Y 1 | XZ _||_ Y 2 | X Z _||_ Y 1 | X,Y 2 Z _||_ Y 2 | X,Y 1 Y 1 _||_ Y 2 | XY 1 _||_ Y 2 | X,Z Y2Y2 Bridge Principles: Acyclic Causal Graph over V  Constraints on P(V)

7. 7 D-separation Undirected Paths Colliders vs. Non-Colliders

8. 8 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y:

9. 9 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y: 1) X  V  Y

10. 10 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y: 1) X  V  Y 2) X  Y

11. 11 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y: 1) X  V  Y 2) X  Y 3) X  Z1  W  Y

12. 12 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y: 1) X  V  Y 2) X  Y 3) X  Z1  W  Y 4) X  Z1  W  U  Y

13. 13 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y: 1) X  V  Y 2) X  Y 3) X  Z1  W  Y 4) X  Z1  W  U  Y 5) X  Z1  Z2  U  Y

14. 14 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Paths from X to Y: 1) X  V  Y 2) X  Y 3) X  Z1  W  Y 4) X  Z1  W  U  Y 5) X  Z1  Z2  U  Y 6) X  Z1  Z2  U  W  Y

15. 15 D-separation: Undirected Paths X Y Z1Z1 Z2Z2 V W U Undirected Path from X to Y: any sequence of edges beginning with X and ending at Y in which no edge repeats Illlegal Path from X to Y: 1) X  Z1  Z2  U  W  Z1  Z2  U  Y

16. 16 Colliders Y: Collider Shielded Unshielded X Y Z X Y Z X Y Z Y: Non-Collider X Y Z X Y Z X Y Z

17. 17 A variable is or is not a collider on a path X Y Z1Z1 Z2Z2 V W U X  Z1  W  U  Y Variable: U Paths from X to Y Paths on which U is a non-collider:

18. 18 Colliders – a variable on a path X Y Z1Z1 Z2Z2 V W U Variable: U Paths from X to Y Paths on which U is a non-collider: Path on which U is a collider: X  Z1  W  U  Y X  Z1  Z2  U  Y

19. 19 Colliders – a variable on a path X Y Z1Z1 Z2Z2 V W U Variable: U Paths from X to Y Paths on which U is a non-collider: Path on which U is a collider: X  Z1  W  U  Y X  Z1  Z2  U  Y X  Z1  Z2  U  W  Y

20. 20 Conditioning on Colliders induce Association Gas [y,n] Battery [live, dead] Car Starts [y,n] Gas _||_ Battery Gas _||_ Battery | Car starts = noExp _||_ Symptoms | Infection Exp_||_ Symptoms Exp [y,n] Symptoms [yes, no] Infection [y,n] Conditioning on Non-Colliders screen-off Association

21. 21 D-separation X is d-separated from Y by Z in G iff Every undirected path between X and Y in G is inactive relative to Z An undirected path is inactive relative to Z iff any node on the path is inactive relative to Z A node N (on a path) is inactive relative to Z iff a) N is a non-collider in Z, or b) N is a collider that is not in Z, and has no descendant in Z X Y Z1Z1 Z2Z2 V W Undirected Paths between X, Y: 1) X  Z 1  W  Y 2) X  V  Y A node N (on a path) is active relative to Z iff a) N is a non-collider not in Z, or b) N is a collider that is in Z, or has a descendant in Z X d-sep Y relative to Z =  ?

22. 22 D-separation X is d-separated from Y by Z in G iff Every undirected path between X and Y in G is inactive relative to Z An undirected path is inactive relative to Z iff any node on the path is inactive relative to Z A node N (on a path) is inactive relative to Z iff a) N is a non-collider in Z, or b) N is a collider that is not in Z, and has no descendant in Z X Y Z1Z1 Z2Z2 V W X  Z 1  W  Y active? 1)Z1 active? 2)W active? A node N (on a path) is active relative to Z iff a) N is a non-collider not in Z, or b) N is a collider that is in Z, or has a descendant in Z X d-sep Y relative to Z =  ? No Yes No

23. 23 D-separation X is d-separated from Y by Z in G iff Every undirected path between X and Y in G is inactive relative to Z An undirected path is inactive relative to Z iff any node on the path is inactive relative to Z A node N (on a path) is inactive relative to Z iff a) N is a non-collider in Z, or b) N is a collider that is not in Z, and has no descendant in Z X Y Z1Z1 Z2Z2 V W X  V  Y active? 1)V active? A node N (on a path) is active relative to Z iff a) N is a non-collider not in Z, or b) N is a collider that is in Z, or has a descendant in Z X d-sep Y relative to Z =  ? Yes No

24. D-separation X is d-separated from Y by Z in G iff Every undirected path between X and Y in G is inactive relative to Z An undirected path is inactive relative to Z iff any node on the path is inactive relative to Z A node N is inactive relative to Z iff a) N is a non-collider in Z, or b) N is a collider that is not in Z, and has no descendant in Z X Y Z1Z1 Z2Z2 V W X d-sep Y relative to Z = {W, Z 2 } ? Undirected Paths between X, Y: 1) X  Z 1  W  Y 2) X  V  Y A node N (on a path) is active relative to Z iff a) N is a non-collider not in Z, or b) N is a collider that is in Z, or has a descendant in Z

25. D-separation X is d-separated from Y by Z in G iff Every undirected path between X and Y in G is inactive relative to Z An undirected path is inactive relative to Z iff any node on the path is inactive relative to Z A node N is inactive relative to Z iff a) N is a non-collider in Z, or b) N is a collider that is not in Z, and has no descendant in Z X Y Z1Z1 Z2Z2 V W X d-sep Y relative to Z = {W, Z 2 } ? 1) X  Z 1  W  Y Z1 active? A node N (on a path) is active relative to Z iff a) N is a non-collider not in Z, or b) N is a collider that is in Z, or has a descendant in Z W active? Yes No

26. 26 D-separation X Y Z1Z1 Z2Z2 X d-sep Z 2 given  ? X d-sep Z 2 given {Z 1 } ? X YZ1Z1 Z2Z2 X d-sep Y given  ? X d-sep Y given {Z 1 } ? No

27. 27 Statistical Control ≠ Experimental Control Experimentally control for X 2 D-separation + Intervention: I Question: Does X 1 directly cause X 3 ? Truth: No, X 2 mediates How to find out?

28. 28 Statistical Control ≠ Experimental Control No! X 3 _||_ X 1 | X 2 Yes: X 3 _||_ X 1 | X 2 (set) Statistically control for X 2 Experimentally control for X 2 D-separation + Intervention: I X 3 d-sep X 1 by {X 2 } ??? X 3 d-sep X 1 by {X 2 set} ???

29. 29 Break

30. 30 Equivalence Classes Independence (d-separation equivalence) DAGs : Patterns PAGs : Partial Ancestral Graphs Intervention Equivalence Classes Measurement Model Equivalence Classes Linear Non-Gaussian Model Equivalence Classes Etc. Equivalence: Independence Equivalence: M 1 ╞ (X _||_ Y | Z)  M 2 ╞ (X _||_ Y | Z) Distribution Equivalence:  1  2 M 1 (  1 ) = M 2 (  2 ), and vice versa)

31. 31 D-separation Equivalence Theorem (Verma and Pearl, 1988) Two acyclic graphs over the same set of variables are d-separation equivalent iff they have: the same adjacencies the same unshielded colliders d-separation/Independence Equivalence

32. 32 Colliders Y: Collider Shielded Unshielded X Y Z X Y Z X Y Z Y: Non-Collider X Y Z X Y Z X Y Z

33. 33 D-separation Equivalence Theorem (Verma and Pearl, 1988) Two acyclic graphs over the same set of variables are d-separation equivalent iff they have: the same adjacencies the same unshielded colliders d-separation/Independence Equivalence Exercises 1)Create a 4-variable DAG 2)Specify a 1-edge variant that is equivalent 3)Specify a 1-edge variant that is not 4)Show with IM and Estimators that you have succeeded in steps 2 and 3

34. 34 Independence Equivalence Classes: Patterns & PAGs Patterns (Verma and Pearl, 1990): graphical representation of d-separation equivalence class (among models with no latent common causes) PAGs: (Richardson 1994) graphical representation of a d- separation equivalence class that includes models with latent common causes and sample selection bias that are d-separation equivalent over a set of measured variables X

35. 35 Patterns

36. 36 Patterns: What the Edges Mean

37. 37 Patterns

38. 38 Patterns Specify all the causal graphs represented by the Pattern: 1) 2) ??

39. 39 Patterns Specify all the causal graphs represented by the Pattern: 1) 2)

40. 40 Tetrad Demo: Generating Patterns

41. 41 Break

42. 42 Causal Search Spaces are Large

43. 43 Experimental Setup(V) V = {Obs, Manip} P(Manip) P Manip (V) Data Discovery Algorithm Causal Knowledge e.g., Markov Equivalence Class of Causal Graphs Causal Search as a Method General Assumptions -Markov, -Faithfulness -Linearity -Gaussianity -Acyclicity Background Knowledge -Salary  Gender -Infection  Symptoms Statistical Inference

44. 44 For Example Statistical Inference Background Knowledge X 2 prior in time to X 3 Passive Observation General Assumptions Markov, Faithfulness, No latents, no cycles,

45. 45 Faithfulness Constraints on a probability distribution P generated by a causal structure G hold for all parameterizations of G. Revenues :=  1 Rate +  2 Economy +  Rev Economy :=  3 Rate +  Econ Faithfulness:  1 ≠ -  3  2  2 ≠ -  3  1 Tax Rate Economy Tax Revenues 11 33 22

46. 46 Faithfulness Constraints on a probability distribution P generated by a causal structure G hold for all parameterizations of G. All and only the constraints that hold in P(V) are entailed by the causal structure G(V), rather than lower dimensional surfaces in the parameter space. Causal Markov Axiom: X and Y causally disconnected ╞ X _|| _ Y Faithfulness: X and Y causally disconnected ╡ X _|| _ Y

47. 47 Challenges to Faithfulness Gene A Gene B Protein 24 + + - By evolutionary design: Gene A _||_ Protein 24 Air Temp Core Body Temp Homeostatic Regulator By evolutionary design: Air temp _||_ Core Body Temp Sampling Rate vs. Equilibration rate

48. 48 Search Methods Constraint Based Searches PC, FCI Pointwise, but not uniformly consistent Scoring Searches Scores: BIC, AIC, etc. Search: Hill Climb, Genetic Alg., Simulated Annealing Difficult to extend to latent variable models Meek and Chickering Greedy Equivalence Class (GES) Pointwise, but not uniformly consistent Latent Variable Psychometric Model Search BPC, MIMbuild, etc. Linear non-Gaussian models (Lingam) Models with cycles And more!!!

49. 49 Score Based Search Background Knowledge e.g., X 2 prior in time to X 3 Model Score Model Scores: AIC, BIC, etc.