1. Automating Configuration Troubleshooting with Dynamic Information Flow Analysis Mona Attariyan Jason Flinn University of Michigan

2. Mona Attariyan - University of Michigan2 Configuration Troubleshooting Is Difficult Software systems difficult to configure Users make mistakes Misconfigurations happen

3. Mona Attariyan - University of Michigan3 Configuration Troubleshooting Is Difficult

4. Mona Attariyan - University of Michigan4 What To Do With Misconfiguration? …… &$%#! ….. config file Ask colleagues Search manual, FAQ, online forums Look at the code if available A tool that automatically finds the root cause of the misconfiguration in applications?

5. Mona Attariyan - University of Michigan5 ConfAid Application code has enough information to lead us to the root cause Insight Dynamic information flow analysis on application binaries How?

6. Mona Attariyan - University of Michigan6 How to Use ConfAid? error …… config file likely root causes 1)… 2)… 3)… …… Application ConfAid

7. Motivation How ConfAid runs Information flow analysis algorithms Embracing imprecise analysis Evaluation Conclusion Mona Attariyan - University of Michigan7 Outline

8. Mona Attariyan - University of Michigan8 How Developers Find Root Cause ExecCGI Config file file = open(config file) token = read_token(file) if (token equals “ExecCGI”) execute_cgi = 1 … if (execute_cgi == 1) ERROR() Application

9. Mona Attariyan - University of Michigan9 How ConfAid Finds Root Cause Config file file = open(config file) token = read_token(file) if (token equals “ExecCGI”) execute_cgi = 1 … if (execute_cgi == 1) ERROR() ConfAid uses taint tracking ExecCGI

10. 10 How to Avoid Error? if (b) if (c) This path leads to some other error likely root cause if (a) This path ends before the error happens This path successfully avoids the error

11. Motivation How ConfAid runs Information flow analysis algorithms Embracing imprecise analysis Evaluation Conclusion Mona Attariyan - University of Michigan11 Outline

12. Mona Attariyan - University of Michigan12 Data Flow Analysis x = y + z, T y = {, } T z = {, } T x = {,, } T y  T z value of x might change, if tokens or change T x = {, } Taint propagates via data flow and control flow

13. Mona Attariyan - University of Michigan13 Control Flow Analysis /* c = 0 */ /* x is read from file*/ if (c == 0) { x = a } T a = { } T x = { T c = { } T x = { } What could cause x to be different ? }, Data flow Control flow, ()

14. Mona Attariyan - University of Michigan14 Alternate Path Exploration y depends on c if(c) y = a if(!c) ckpt /* c = 1*/ /* y is read from file*/ if (c) { /*taken path*/ … } else { y = a }

15. Mona Attariyan - University of Michigan15 Effect of Alternate Path Exploration /* c = 1*/ /* y is from file*/ if (c) { … } else { y = a } What could cause y to be different? T a = { } T y = { T c = { } T y = { } }, Alternate path exploration, () Alternate path + Data flow

16. Motivation How ConfAid runs Information flow analysis algorithms Embracing imprecise analysis Evaluation Conclusion Mona Attariyan - University of Michigan16 Outline

17. Mona Attariyan - University of Michigan17 Embracing Imprecise Analysis Complete and sound analysis leads to: –poor performance –high false positive rate To improve performance To reduce false positives Bounded horizon heuristic Single mistake heuristic Weighting heuristic

18. Bounded horizon prevents path explosion Alternate path runs a fixed # of instructions 18 Bounded Horizon Heuristic if (b) if (c) max reached, abort exploration likely root causes

19. Configuration file contains a single mistake Reduces amount of taint and # of explored paths Mona Attariyan - University of Michigan19 Single Mistake Heuristic /* x=1, c=0*/ if (c == 0) { x = a } T a = { } T x = {,, ( )} T c = { } T x = { }

20. Mona Attariyan - University of Michigan20 Weighting Heuristic Insufficient to treat all taint propagations equally –Data flow introduces stronger dependency than ctrl flow –Branches closer to error stronger than farther branches Assign weights to taints to represent strength level –Data flow taint gets a higher weight than ctrl flow taint –Branches closer to error get higher weight than farther

21. Mona Attariyan - University of Michigan21 Example of Weighting Heuristic if (x) { … if (y) { … if (z) { ERROR() } likely root causes

22. 22 Heuristics: Pros and Cons Bounded horizon Single mistakeWeighting Simplify control flow analysis Improve performance Reduce FP Increase FP Increase FN FP = False Positive, FN = False Negative

23. Mona Attariyan - University of Michigan23 ConfAid and Multi-process Apps ConfAid propagates taints between processes –Intercepts IPC system calls –Sends taint along with the data ConfAid currently supports communication via: –Unix sockets, pipes, TCP and UDP sockets –Regular files

24. Motivation How ConfAid runs Information flow analysis algorithms Embracing imprecise analysis Evaluation Conclusion Mona Attariyan - University of Michigan24 Outline

25. ConfAid debugs misconfiguration in: –OpenSSH 5.1 (2 processes) –Apache HTTP server 2.2.14 (1 process) –Postfix mail transfer agent 2.7 (up to 6 processes) Manually inject errors to configuration files Evaluation metrics: –The ranking of the correct root cause –The time to execute the application with ConfAid Mona Attariyan - University of Michigan25 Evaluation

26. Real-world misconfigurations: –total of 18 bugs from manuals, forums and FAQs Randomly generated bugs: –60 bugs using ConfErr [Keller et al. DSN 08] Mona Attariyan - University of Michigan26 Data Sets

27. Mona Attariyan - University of Michigan27 How Effective is ConfAid ? Total tokens First tied w/1 Second tied w/1 Worse than second OpenSSH47-4922210 Apache88-9331020 Postfix27-2955000 Correct root caused ranked first or second for all 18 real-world bugs 72%28% 0%

28. Mona Attariyan - University of Michigan28 How Effective is ConfAid ? Total tokens First tied w/1 Second tied w/1 Worse than second OpenSSH47171101 Apache88171011 Postfix27150203 Correct root caused ranked first or second for 55 out of 60 randomly-generated bugs 85% 7% 8%

29. Mona Attariyan - University of Michigan29 How Fast is ConfAid? Average Execution Time OpenSSH52 seconds Apache2 minutes 48 seconds Postfix57 seconds OpenSSH7 seconds Apache24 seconds Postfix38 seconds Average execution time for real-world bugs: 1m 32s Average time for randomly-generated bugs: 23s

30. ConfAid automatically finds root cause of problems ConfAid uses dynamic information flow analysis ConfAid ranks the correct root cause as first or second in: –18 out of 18 real-world bugs –55 out of 60 random bugs ConfAid takes only a few minutes to run Mona Attariyan - University of Michigan30 Conclusion

31. Mona Attariyan - University of Michigan31 Questions?

32. ConAid may or may not report all For independent mistakes, ConfAid first finds the one that led to the first failure For dependent mistakes, ConfAid may report all based on their effect on program Mona Attariyan - University of Michigan32 What if there are multiple mistakes?

33. Mona Attariyan - University of Michigan33 Effect of Bounded Horizon Heuristic

34. Mona Attariyan - University of Michigan34 Effect of Weighting Heuristic Max # tokens: 49 Max # tokens: 93 Max # tokens: 5