1. Institut Mines-Télécom Symbolic Passive Testing - Application to an industrial case study (Diamonds project) Pramila Mouttappa, Stephane Maag and Ana Cavalli Institut Mines-Télécom/Télécom SudParis RCIS 2013 Industrial Day - 31st May, 2013 Paris, France Work supported by DIAMONDS (ITEA 2) Project

2. Institut Mines-Télécom Overview  Motivation  Our approach  IOSTS based models  Parametric trace slicing  Formal property evaluation  Prototype results  Conclusions Symbolic Passive Testing

3. Institut Mines-Télécom Conformance Testing Télécom SudParis Modèle de présentation Active Tester Verdict Test Cases Specification / Requirements IUT Test generation ACTIVE TESTING IUT Specification / Requirements Passive Tester Verdict User PASSIVE TESTING Trace P.O.

4. Institut Mines-Télécom Problem Statement  Limitation of Passive Testing - For checking both the data and control part of the protocol. - Enumeration of data values. Symbolic Passive Testing (i)ONLY CONTROL PART Invariant : Req / Ack Verdict = True

5. Institut Mines-Télécom Problem Statement (contd.) Symbolic Passive Testing (ii) CONTROL + DATA PART Invariant : Req(A) / Ack(B) Verdict = False or Inconclusive

6. Institut Mines-Télécom Motivations ■ To reduce false positive verdicts ■ To avoid data enumeration ■ To model eventual behavior deviations ■ To study the scalability of very long traces (i.e., if the traffic collection is continuous) Télécom SudParis Modèle de présentation

7. Institut Mines-Télécom Our approach... ■ Functional and Security attacks can be modeled – IOSTS ■ Enumeration of data values is not required – SYMBOLIC ■ Data relationship between messages – REDUCES FALSE POSITIVE VERDICTS ■ New Passive Testing Technique - INTEGRATION of Parametric Trace Slicing and Symbolic Execution Symbolic Passive Testing

8. Institut Mines-Télécom Framework of our approach – Automotive case study Symbolic Passive Testing 1 2 3 5 6 Specification / Requirement’s Symbolic Passive Testing Tool : TestSym-P   Verdicts Pass / Fail / Attack – Pass / Inconclusive 7 ? 4

9. Institut Mines-Télécom Sequence diagram - Bluetooth Connectivity & Bluestabbing attack Symbolic Passive Testing [1] “Bluetooth specification version 2.0 + edr [vol 0],” 1999. [2] D. Browning and G. Kessler, “Bluetooth hacking: A case study,” in Proceedings of the Conference on Digital Forensics, Security and Law, 2009, pp. 20–22. [1] “Bluetooth specification version 2.0 + edr [vol 0],” 1999. [2] D. Browning and G. Kessler, “Bluetooth hacking: A case study,” in Proceedings of the Conference on Digital Forensics, Security and Law, 2009, pp. 20–22.

10. Institut Mines-Télécom IOSTS Model Symbolic Passive Testing S3 S3.1 ! hci-change-local-name (loc_name0) G2:{loc_name ≠vloc_name} A3: {vloc_name:=loc_name0}

11. Institut Mines-Télécom Symbolic Execution (SE) of IOSTS Symbolic Passive Testing

12. Institut Mines-Télécom Parametric Trace Slicing  Trace Monitoring technique  Trace slices – Each parameter instance observed in the trace.  The events corresponding to a particular parameter instance are grouped in the order they appear in the trace in a particular trace slice. Symbolic Passive Testing

13. Institut Mines-Télécom Parametric Trace Slicing - Example Consider a sample Bluetooth trace, ■ !hci-inquiry ?hci-inquiry-complete !hci-create-connection(bd addr1 ) !hci-create-connection(bd addr2 ) ?hci-connect-complete(bd addr1 ) Symbolic Passive Testing SlicesData PortionControl Portion Slice 1bdaddr1!hci-inquiry?hci-inquiry- complete!hci-create-connection ?hci- connect-complete Slice 2bdaddr2!hci-inquiry?hci-inquiry- complete!hci-create-connection

14. Institut Mines-Télécom Integration of SE and Parametric Trace Slicing ■ Symbolic Execution (SE):  The path of the symbolic execution tree – represents the property to be tested on the trace.  Valuation of the parameters – symbolic values. ■ Parametric Trace Slicing:  Trace analysis  Valuation of the parameters – concrete values.  Trace slices put together constitutes the implementation trace.  Path of the SE tree – checked against each trace slice to obtain the Verdict. Symbolic Passive Testing

15. Institut Mines-Télécom Evaluation - IOSTS property on the trace slices 1. Control portion : Control portion(SE) = Control portion(slice) goto 2. 2. Data portion: Substitute concrete data values in place of symbolic values. Satisfiability of each state in path of the SE - Guard condition check. AttackSeq : 0 (Property Sequence), 1 (Attack Sequence) Symbolic Passive Testing AttackSeqControl PortionData PortionVerdictsComments 0YYPassProperty satisfied. 0YNFailProperty not satisfied. 1YYAttack PassAttack sequence satisfied. 0 or 1N-InconclusiveTrace length insufficient to decide. else ≠

16. Institut Mines-Télécom Final Evaluation – Property on the Implementation trace ■ PASS ●if (Every (Verdict(tsi) = Pass)) ■ Attack-PASS ●if (Exists(Verdict(tsi) = Attack-Pass)) ■ FAIL ●if [(Exists(Verdict(tsi) = Fail)) ^ (Exists(Verdict(tsi)≠ Attack- Pass))] ■ INCONCLUSIVE, otherwise. Symbolic Passive Testing

17. Institut Mines-Télécom Prototype Model – TestSym-P Symbolic Passive Testing

18. Institut Mines-Télécom Prototype – Trace Parsing Symbolic Passive Testing

19. Institut Mines-Télécom Prototype - Trace Slicing Symbolic Passive Testing

20. Institut Mines-Télécom Prototype – Evaluation Output Symbolic Passive Testing

21. Institut Mines-Télécom Evaluation Results Symbolic Passive Testing [3] Pramila Mouttappa, Stephane Maag and Ana Cavalli, "IOSTS based Passive Testing approach for the Validation of data-centric Protocols", 12th International Conference on Quality Software (QSIC 2012), Xi’an, China, 27-29 August 2012. [4] Pramila Mouttappa, Stephane Maag and Ana Cavalli, “Improving Protocol Validation by an IOSTS-based Passive Testing approach”, 9th Workshop on System Testing and Validation (STV 2012), Paris, France, 24 th October 2012. [5] Pramila Mouttappa and Stephane Maag and Ana Cavalli, “Monitoring based on IOSTS for testing functional and security properties: Application to an Automotive case study”. To be published, 37 th Annual International Conference on Computers, Software and Applications (COMPSAC 2013), 22-26 July 2013, Kyoto, Japan [3] Pramila Mouttappa, Stephane Maag and Ana Cavalli, "IOSTS based Passive Testing approach for the Validation of data-centric Protocols", 12th International Conference on Quality Software (QSIC 2012), Xi’an, China, 27-29 August 2012. [4] Pramila Mouttappa, Stephane Maag and Ana Cavalli, “Improving Protocol Validation by an IOSTS-based Passive Testing approach”, 9th Workshop on System Testing and Validation (STV 2012), Paris, France, 24 th October 2012. [5] Pramila Mouttappa and Stephane Maag and Ana Cavalli, “Monitoring based on IOSTS for testing functional and security properties: Application to an Automotive case study”. To be published, 37 th Annual International Conference on Computers, Software and Applications (COMPSAC 2013), 22-26 July 2013, Kyoto, Japan TraceNo. MessagesNo. SlicesTrace output without errorsTrace output with errors & attacks PFIVerdictPFIAPVerdict 18121-1I-11-F 28931-2I--21AP 38121-1I-11-F 4 21-1I--11AP 58121-1I-11-F 6 21-1I--11AP 78121-1I--11AP 88121-1I--11AP 98121-1I-11-F 108121-1I--11AP P - Pass F - Fail AP - Attack Pass I - Inconclusive P - Pass F - Fail AP - Attack Pass I - Inconclusive

22. Institut Mines-Télécom Innovations with respect to STOA ■ From our knowledge, there are currently no works tackling Passive testing/Monitoring based on IOSTS without any awareness on the states of the execution traces, moreover,  the integration of symbolic execution of IOSTS and Slicing technique for Passive Testing,  dealing with symbolic values eliminates the necessity of enumeration of all data values,  the approach enables testing functional and vulnerability / attack patterns by passive testing. Symbolic Passive Testing

23. Institut Mines-Télécom Future Works ■ Online testing - i.e., evaluation of properties as the implementation is being run. ■ Time constraints - that are needed in the definition of several properties. Symbolic Passive Testing