1 Validation of Security Protocols Joint work with Gul Agha, Michael Greenwald, Carl Gunter, Sanjeev Khanna, Darko Marinov, Jose Meseguer, Prasanna Thati, Mahesh Viswanathan Koushik Sen

2/32 Formal Analysis of Cryptographic Protocols Integrity and Confidentiality  Recipient not fooled or leaks information  algebraic techniques assumes idealized cryptographic primitives  complexity-theoretic techniques based on complexity assumptions

3/32 Availability Attack Availability threats  whether recipient available to valid sender  algebraic and/or complexity theoretic methods are not suitable for finding availability threats assumes adversary can insert, delete, or replay messages  availability attack is assured as the adversary can delete any valid packet

4/32 Availability Attack Availability threats  whether recipient available to valid sender  algebraic and/or complexity theoretic methods are not suitable for finding availability threats assumes adversary can insert, delete, or replay messages  availability attack is assured as the adversary can delete any valid packet  How to model and analyze availability formally?

5/32 Our Goal Given a protocol P, let properties T hold for P  P is a traditional non-deterministic specification  T is a set of integrity and confidentiality properties Extend P to P * and T to T *  P * is DoS hardened P  T * includes availability properties in addition to T Goal  Prove that T * hold for P * without re-proving that T hold for P

6/32 Our Results Given a protocol P, let properties T hold for P  P is a traditional non-deterministic specification  T is a set of integrity and confidentiality properties Extend P to P * and T to T *  P * is DoS hardened P  T * includes availability properties in addition to T Goal  Prove that T * hold for P * without re-proving that T hold for P ?

7/32 Modeling and Analysis Probabilistic Rewrite Theories  PMaude  Unified Algebraic Model  Probabilistic Object Model Properties in Continuous stochastic logic (CSL)  Statistical Model-checking [Sen et al. CAV’04, CAV’05, QEST’05] using VESTA using Monte Carlo simulation and statistical hypothesis testing QuaTEx  Quantitative Temporal Expressions  Query language to gain quantitative insight about a model  Statistical computation of QuaTEx [QAPL’05] in VESTA

8/32 Achievements In the last meeting we presented our preliminary ideas  On formal modeling and analysis of DoS resistant security protocols This year we have worked out the details  Specified TCP/IP 3-way handshaking protocol in PMaude  Model-checked availability properties using the VESTA tool  Designed a language to query more informative quantitative information about a model QuaTEx  Published our results at FCS 05, QAPL 05, CAV 05, and QEST 05

9/32 TCP/IP: A case study Common Susceptible to DoS attacks:  SYN flood and others Existing solutions as benchmark:  Increase size of SYN cache, random drop, SYN cookies

10/32 Selective Sequential Verification SYN flood attack: the adversary can devote his entire channel to fake SYN packets Countermeasure :  Valid sender sends multiple copies of the SYN packet  receiver checks each incoming SYN packet with some probability (say, 25% or 1%) Specified DoS hardened TCP/IP 3-way handshaking protocol in PMaude C Gunter, S Khanna, K Tan, S Venkatesh 2004

11/32 Availability Property Property: The probability that eventually the attacker X successfully fills up the SYN cache of B is less than P <0.01 [§(sucessful_attack())] Statistical Model-checking using Vesta model-checker K Sen, M Viswanathan, G Agha (CAV 2005)

12/32 Results Cache-size = 10,000 timeout = 10 seconds number of valid senders = 100 G. Agha, M. Greenwald, C. Gunter, S. Khanna, J. Meseguer, P. Thati (FCS 05)

13/32 Quantitative Queries Using QuaTEx What is the expected number of clients that successfully connect to S? CountConnected() = if completed() then count() else ° (CountConnected()) fi; eval E[CountConnected()] G Agha, J Meseguer, K Sen (QAPL 2005)

14/32 Quantitative Queries Using QuaTEx What is the probability that a client connected to S within 10 seconds after it initiated the connection request? Prob() = if globaltime()>10 then 0.0 else if connected() then 1.0 else ° (Prob()) fi fi; eval E[Prob()] G Agha, J Meseguer, K Sen (QAPL 2005)

15/32 Linux Kernel Test Attack rate in SYNs/sec received at server Graph shows successful connections per 450 threads Defenseless kernel: >6 SYNs/sec shuts out client Aggregate connections Attack rate Model predicts cliff M Delap, M Greenwald, C Gunter, S Khanna, Y Xu 2004

16/32 Results Expected number of clients out of 100 clients that get connected with the server under DoS attack G. Agha, M. Greenwald, C. Gunter, S. Khanna, J. Meseguer, P. Thati (FCS 05)

17/32 Results Expected number of clients out of 100 clients that get connected with the server under DoS attack Experiments with Linux kernel involved 30 days of work Experiments with PMaude model involved only 1 day work G. Agha, M. Greenwald, C. Gunter, S. Khanna, J. Meseguer, P. Thati (FCS 05)

18/32 Discussion A general framework for modeling and verifying DoS properties of communication protocols. Capable of expressing and proving key availability properties. Performance limitations require us to use scaled down version of parameters. Future Work  Addressing efficiency limitations  Verifying the properties for general systems

19/32 Testing Security Protocol Implementations Integrity and Confidentiality properties  Verification techniques

20/32 Testing Security Protocol Implementations Integrity and Confidentiality properties  Verification techniques  How about verifying real implementations? Intractable

21/32 Testing Security Protocol Implementations Integrity and Confidentiality properties  Verification techniques  How about verifying real implementations? Intractable  How about trying to find security bugs? Concolic testing

22/32 Our Testing Approach CUTE: A Concolic Unit Testing Engine for e  A scalable automated unit testing tool  Combines concrete and symbolic execution  Can test real-world C programs Tested C implementation of security protocols  Needham-Schroeder, TMN, …  Discovered known security attacks in a few seconds K Sen, D Marinov, G Agha (ACM SIGSOFT ESEC/FSE 2005)

23/32 Goal of Concolic Testing Automated Scalable Unit Testing of real- world C Programs  Generate test inputs  Execute unit under test on generated test inputs so that all reachable statements are executed  Any assertion violation gets caught

24/32 Goal of Concolic Testing Automated Scalable Unit Testing of real- world C Programs  Generate test inputs  Execute unit under test on generated test inputs so that all reachable statements are executed  Any assertion violation gets caught Our Approach:  Explore all execution paths of an Unit for all possible inputs Exploring all execution paths ensure that all reachable statements are executed

25/32 Execution Paths of a Program Can be seen as a binary tree with possibly infinite depth  Computation tree Each node represents the execution of a “if then else” statement Each edge represents the execution of a sequence of non-conditional statements Each path in the tree represents an equivalence class of inputs

26/32 Execution Paths of a Program Can be seen as a binary tree with possibly infinite depth  Computation tree Each node represents the execution of a “if then else” statement Each edge represents the execution of a sequence of non-conditional statements Each path in the tree represents an equivalence class of inputs Explicit Path Model Checking

27/32 Existing Approach I Random testing  generate random inputs  execute the program on generated inputs Probability of reaching an error can be astronomically less test_me(int x){ if(x==94389){ ERROR; } Probability of hitting ERROR = 1/2 32

28/32 Existing Approach II Symbolic Execution  use symbolic values for input variables  execute the program symbolically on symbolic input values  collect symbolic path constraints  use theorem prover to check if a branch can be taken Does not scale for large programs test_me(int x){ if((x%10)*4!=17){ ERROR; } else { ERROR; } Symbolic execution will say both branches are reachable: False positive

29/32 Approach Combine concrete and symbolic execution for unit testing  Concrete + Symbolic = Concolic In a nutshell  Use concrete execution over a concrete input to guide symbolic execution  Concrete execution helps Symbolic execution to simplify complex and unmanageable symbolic expressions by replacing symbolic values by concrete values Achieves Scalability  Higher branch coverage than random testing  No false positives or scalability issue like in symbolic execution based testing

30/32 Experiments: Needham-Schroeder Protocol Tested a C implementation of a security protocol (Needham-Schroeder) with a known attack  270 lines of code  Took less than 8 seconds on a machine with 1.8GHz processor to discover middle-man attack In contrast, a software model-checker (VeriSoft) took 8 hours on a similar implementation

31/32 Experiments: TMN Protocol Tested a C implementation of TMN protocol  Tatebayashi, Matsuzaki, and Newman (TMN) Protocol: a protocol for distribution of a fresh symmetric key.  329 lines of code  Took less than 6 seconds on a machine with 1.8GHz processor to discover an attack an intruder can establish a parallel session through eavesdropping and obtain the secret key

32/32 Discussion CUTE is  light-weight  dynamic analysis (compare with static analysis) ensures no false alarms  concrete execution and symbolic execution run simultaneously symbolic execution consults concrete execution whenever dynamic analysis becomes intractable  real tool that works on all C programs completely automatic Requires actual code that can be fully compiled Can sometime reduce to Random Testing Complementary to Static Analysis Tools