1. Learning Causal Models of Multivariate Systems and the Value of it for the Performance Modeling of Computer Programs Jan Lemeire December 19 th 2007 Supervisor: Prof. dr. ir. Erik Dirkx

2. Pag. Jan Lemeire / 49 2 Causal Inference & Performance Analysis Learning causal models for the performance analysis of programs executed on various computer systems. Intermezzo I: Causal inference. Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference. Intermezzo II: Kolmogorov Minimal Sufficient Statistics. The importance of qualitative properties.

3. Pag. Jan Lemeire / 49 3 Causal Inference & Performance Analysis Learning causal models for the performance analysis of programs executed on various computer systems. Intermezzo I: Causal Inference. Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference. Intermezzo II: Kolmogorov Minimal Sufficient Statistics The importance of qualitative properties.

4. Pag. Jan Lemeire / 49 4 Causal Inference & Performance Analysis What is Parallel Processing? Ideally: Speedup = number of processors Computational work: Parallel system

5. Pag. Jan Lemeire / 49 5 Causal Inference & Performance Analysis Parallel Overhead Speedup = 2.55 Overhead = time the processors are not spending on useful work = lost processor cycles

6. Pag. Jan Lemeire / 49 6 Causal Inference & Performance Analysis Overhead Analysis Impact of overhead on speedup

7. Pag. Jan Lemeire / 49 7 Causal Inference & Performance Analysis Experimental Parallel Performance Analysis: Data Acquisition

8. Pag. Jan Lemeire / 49 8 Causal Inference & Performance Analysis EPDA: Multivariate Analysis

9. Pag. Jan Lemeire / 49 9 Causal Inference & Performance Analysis Intermezzo I: Causal Inference EVT Experimenten in animatie tonen (zonder (a) en (b)

10. Pag. Jan Lemeire / 49 10 Causal Inference & Performance Analysis Causal Inference for Performance Analysis Utility based on the following properties: 1.Dependency analysis: how variables relate. 2.Markov property. 3.A causal model corresponds to a decomposition.

11. Pag. Jan Lemeire / 49 11 Causal Inference & Performance Analysis Execution of program gives cache misses x? datatype (integer, float, double,…) data size in Bytes 4 4

12. Pag. Jan Lemeire / 49 12 Causal Inference & Performance Analysis Markov Property Provides explanations Differentiate direct from indirect relations Correlated With information about the data size:

13. Pag. Jan Lemeire / 49 13 Causal Inference & Performance Analysis Can we Observe Causal Relations? ~ OK, but:or ???

14. Pag. Jan Lemeire / 49 14 Causal Inference & Performance Analysis What is Causality? A causal relation denotes a mechanism, that a variable is `produced’ by its causes. However… not directly observable. Causality is a relic of a bygone age Mmmh Bertrand Russell Judea Pearl But: we want to learn something about underlying system (goal of statistics)

15. Pag. Jan Lemeire / 49 15 Causal Inference & Performance Analysis Second Cause ~

16. Pag. Jan Lemeire / 49 16 Causal Inference & Performance Analysis V-structure Property angle independent from gunpowder but dependent when distance is known

17. Pag. Jan Lemeire / 49 17 Causal Inference & Performance Analysis Conditional Independencies Make Causal Inference Possible From a causal structure follow conditional independencies, irrespective of the mechanisms. –Markov –V-structure

18. Pag. Jan Lemeire / 49 18 Causal Inference & Performance Analysis Graph is a Description of Independencies Graphical criterion: d-separation –Intuitive Faithfulness property: independencies independencies in graph in reality

19. Pag. Jan Lemeire / 49 19 Causal Inference & Performance Analysis Causal Structure Learning In two steps: 1.Undirected graph 2.Orientation

20. Pag. Jan Lemeire / 49 20 Causal Inference & Performance Analysis Result Partially directed acyclic graph “We know what parts are unknown.” Faithfulness assumption: all independencies follow from the causal structure Dit kan ook pas verder, bij bespreking van unique

21. Pag. Jan Lemeire / 49 21 Causal Inference & Performance Analysis Experimental Results (1) Automatic learning of accurate performance models (2) Model validation (3) Identification of unexpected dependencies (4) Explanations for outliers Contribution 1 Figuur opnieuw in png, zonder losless compression

22. Pag. Jan Lemeire / 49 22 Causal Inference & Performance Analysis Learning causal models for the performance analysis of programs executed on various computer systems. Intermezzo I: Causal Inference. Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference. Intermezzo II: Kolmogorov Minimal Sufficient Statistics The importance of qualitative properties.

23. Pag. Jan Lemeire / 49 23 Causal Inference & Performance Analysis Practical Causal Inference The following limitations had to be overcome: Non-linear relations: form-free independence test Mixture of continuous, discrete and categorical data: general independence test Deterministic relations: augmented causal model and extended learning algorithms

24. Pag. Jan Lemeire / 49 24 Causal Inference & Performance Analysis Form-Free and General Dependency Test Mutual information Example Kernel density estimation Pearson: R xy =0.083 => X and Y linearly independent I(X;Y)=0.90 bits => dependent X Y X Y P(X, Y)

25. Pag. Jan Lemeire / 49 25 Causal Inference & Performance Analysis Deterministic Relations Data size and data type are information equivalent with respect to cache misses During learning connect least complex relation

26. Pag. Jan Lemeire / 49 26 Causal Inference & Performance Analysis Complexity Criterion Correct models are learned under the Complexity Increase Assumption Contribution 2a

27. Pag. Jan Lemeire / 49 27 Causal Inference & Performance Analysis Reestablishment of Faithfulness Consequences are considered Information equivalences Independence and simplicity D-separation extension Faithful model: represents all independencies Contribution 2b Information is added to the model Basic information equivalences Dit moet erbij!! Details misschien niet?

28. Pag. Jan Lemeire / 49 28 Causal Inference & Performance Analysis Extension of PC Learning Algorithm Detection of information equivalences Among information equivalent relations, the simplest one is chosen Orientation rules remain the same Correct models are learned from data containing deterministic relations. Contribution 2c

29. Pag. Jan Lemeire / 49 29 Causal Inference & Performance Analysis Learning causal models for the performance analysis of programs executed on various computer systems. Intermezzo I: Causal Inference. Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference. Intermezzo II: Kolmogorov Minimal Sufficient Statistics The importance of qualitative properties.

30. Pag. Jan Lemeire / 49 30 Causal Inference & Performance Analysis

31. Pag. Jan Lemeire / 49 31 Causal Inference & Performance Analysis Inductive Inference Occam’s Razor “Among equivalent models choose the simplest one.” William of Ockham Jaartallen van scientists erbij zetten BUT: Objective measure of complexity?

32. Pag. Jan Lemeire / 49 32 Causal Inference & Performance Analysis Kolmogorov Complexity Andrey Kolmogorov Kolmogorov Complexity of a binary string: the length of the shortest program that computes the string and halts Applied to Occam’s Razor: “Select model that describes the observations minimally”

33. Pag. Jan Lemeire / 49 33 Causal Inference & Performance Analysis Shortest Programs 001001001001001001001001001001001 regularity of repetition allows compression 011000110101101010111001001101000 random information = incompressible

34. Pag. Jan Lemeire / 49 34 Causal Inference & Performance Analysis Randomness versus Regularity 001001001001001001001001001001001 011000110101101010111001001101000 Only random information (incompressible) Kolmogorov Minimal Sufficient Statistics (KMSS): formal separation Meaningful information regularities Accidental information randomness repetition 11 times, 001

35. Pag. Jan Lemeire / 49 35 Causal Inference & Performance Analysis Learning = finding regularities = maximal compression regularities random Structure of a diamond Exact size random

36. Pag. Jan Lemeire / 49 36 Causal Inference & Performance Analysis Meaningful Information of Probability Distributions meaningful information (Theorem 1) Kolmogorov Minimal Sufficient Statistic if graph and CPDs are incompressible (Theorem 2) Contribution 3a a graph with random CPDs is faithful (Theorem 4)

37. Pag. Jan Lemeire / 49 37 Causal Inference & Performance Analysis Causal Aspect of Causal Models = Decomposition Canonical decomposition: quasi-unique and minimal decomposition into atomic and independent components (the CPDs) Corresponds to reality (mechanisms)

38. Pag. Jan Lemeire / 49 38 Causal Inference & Performance Analysis Causal Component Relies on Reductionism When DAG of Bayesian network is a complete graph no meaningful information holism The world can be studied in parts. Or, even more: The world is made up of indivisible parts. Figuurtje toevoegen van holisme en reductionisme Even more

39. Pag. Jan Lemeire / 49 39 Causal Inference & Performance Analysis Validity of Causal Inference Do CPD components correspond to physical mechanisms? Contribution 3b Minimal model? Faithful? Other regularities? How OK is the learned causal model?

40. Pag. Jan Lemeire / 49 40 Causal Inference & Performance Analysis Well-known Example of Unfaithfulness ’Normally’: A and D correlate A and D get independent if influences along paths 1 and 2 cancel each other out Mechanisms are related Regularity among them

41. Pag. Jan Lemeire / 49 41 Causal Inference & Performance Analysis Learning causal models for the performance analysis of programs executed on various computer systems. Intermezzo I: Causal Inference. Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference. Intermezzo II: Kolmogorov Minimal Sufficient Statistics The importance of qualitative properties.

42. Pag. Jan Lemeire / 49 42 Causal Inference & Performance Analysis Regularities are Qualitative Properties Different from quantitative information. Allow for qualitative reasoning. Qualitative properties determine behavior.

43. Pag. Jan Lemeire / 49 43 Causal Inference & Performance Analysis Communication Schemes on Network Topologies Communication time?

44. Pag. Jan Lemeire / 49 44 Causal Inference & Performance Analysis Generic Performance Model Good predictions for combinations of random schemes and random topologies Contribution 4a

45. Pag. Jan Lemeire / 49 45 Causal Inference & Performance Analysis Combinations of Patterns Performance depends on match! Met minder voordehandliggende figuurtjes tonen Broadcast niet in stervorm, shift in lijnvorm, torus toevoegen Contribution 4b

46. Pag. Jan Lemeire / 49 46 Causal Inference & Performance Analysis Qualitative Properties Faithfulness: ” graph should describe all independencies ” KMSS: ”model should describe all regularities” Qualitative informationQuantitative information contains no more regularities explicitly describe regularities

47. Pag. Jan Lemeire / 49 47 Causal Inference & Performance Analysis Explicitly Mention Qualitative Properties!

48. Pag. Jan Lemeire / 49 48 Causal Inference & Performance Analysis Conclusions Contribution to performance analysis. Automatic causal analysis. Useful add-on in combination with other techniques. The value of causal inference is underlined. The importance of regularities or qualitative properties.

49. Pag. Jan Lemeire / 49 49 Causal Inference & Performance Analysis Future Work Application of the learned performance models for optimization. Is the failure of generic performance models only due to regularities? Augment models with qualitative properties. But: how define, recognize and reason with regularities?