Model Based Conformance Testing for Extensible Internet Protocols Anastasia Tugaenko Scientific Adviser: Nikolay Pakulin, PhD

2/19 Internet Protocols Features  Internet protocols are underspecified  Internet protocols are nondeterministic  Internet protocols requirements differ in the level of obligations (MUST, SHOULD, MAY, …)  Protocol’s architecture is extensible

3/19 Architecture of Extensible Protocols Protocol Functions Basic are mandatory for all protocol implementations; specifies the basic protocol functionality, functions that are the basis for the protocol Extensions are optional, may be unsupported by implementations; add new functions without changing the basic protocol’s model; alter format of the protocol’s messages; alter basic protocol model, alter functionality of other extensions; alter protocol’s modus operandi

4/19 Conformance Testing Many implementations of protocols from different developers are functioning in the contemporary Internet The total reliability of data transfer substantially depends on implementations compatibility The basic method of attesting implementations compatibility is conformance testing (testing to conform to the standard) Protocol’s extensions may alter protocol’s standards

5/19 Traditional Methodology of Conformance Testing  Test suite consists of formal given tests which are not connected to the implementation  Test purposes describe situations to be tested. Test purpose is realized in one or several tests  Connections between requirements and test purposes and between test purposes and their tests are informal  Implementation is considered to conform to the standard if all test purposes have been passed

6/19 A Problem with Test Suites Extensions Basic Standard + few extensions Test Basic Standard + few extensions + one more extension How can we get a new test ? assume, that we already got it manually change the test suite constructing new test from a set of test units

7/19 A Problem with Adapting Test Suites to Implementations IUT1 extension 1 extension 2 IUT2 extension 1 extension 3 IUT3 extension 4 How can we compose tests for each implementation? Manually adapt the tests Set a number of options in the configuration file if (extension1) {…} else {… } Use specification and test units and automatically compose test suites for target set of extensions from that units

8/19 Requirements for Test Suite for Extensible Protocols Testing  For basic standards:  formal relation between requirements and tests  automated verdict assignment about the correctness of IUT behavior  automated tests generation depending on IUT responses  For extensions:  easily changing of a protocol model  the ability to combine specifications and tests from separate units

9/19 A Method for Extensible Protocols Testing  Constructing specification from units  Specification for implementation is a collection of specifications units (basic specification, extensions specifications).  Developing tests as units  The comprehensive test is a composition of units of tests for basic functionality and supported implementations.  Automated construction of model and tests for an arbitrary set of extensions

10/19 A Method for Extensible Protocols Testing. Features  Each specification unit do  check the postcondition  change the test state  assign a verdict concerning the correctness of IUT behavior  Test sequence iterator  test sequence generates automatically from test model and contract specifications  Automated coverage calculation  states and transitions may be labeled with marks and branches, in this case they will be represented in the test report.  Easily changing of a protocol model

11/19 Case Study. SMTP Protocol  SMTP – Simple Mail Transfer Protocol  SMTP protocol has 7 states:  DISCONNECTED  CONNECTED  AFTER HELLO  AFTER MAIL FROM  AFTER RCPT TO  AFTER DOT  AFTER DATA Transaction commands are allowed only from these states

12/19 Case Study. Changing SMTP Protocol Model  If AUTH extension is supported SMTP protocol has 8 states:  DISCONNECTED  CONNECTED  AFTER HELLO  AFTER AUTH  AFTER MAIL FROM  AFTER RCPT TO  AFTER DOT  AFTER DATA Several commands are allowed only after AUTH command was successfully handled

13/19 A Method for Extensible Protocols Testing. Protocol Model Model is an extended finite state machine command addition state addition a с t i o n s statesstates

14/19 A Method for Extensible Protocols Testing. Protocol Model Case Study Define 2 mappings :  allowed  denied Got 2 types of testing:  conformance (unknown transitions are considered as denied)  robustness (unknown transitions are considered as allowed)

15/19 A Method for Extensible Protocols Testing. Main Steps  Creation of requirements catalogues  Designing of extensible model for the protocol  Developing units with specifications and units with tests  Formalization of requirements  Developing test scenarios  Constructing a comprehensive test for target implementation, executing it and analyzing the results

16/19 Test Suite for SMTP Protocol  Basic Specification  model ~1250 strings  test scenario ~250 strings  DSN extension (Delivery Status Notifications)  Adds new functionality  Brings in new parameters into existing commands MAIL and RCPT, doesn’t change the protocol model.  model ~100 strings  test scenario ~ 50 strings  AUTH Extension (Authentication)  Alter basic functionality  Adds new state into the protocol model, adds new AUTH command or parameter for existing command MAIL, brings in new response codes.  model ~250 strings  test scenario ~120 strings

17/19 Future Work  Extend the method to develop tests for other types of protocols extensions  Dynamically construct the model of target implementation taking into account the profile of supported extensions  Extend test traces and improve test report generator

18/19 Conclusion  The method for model based conformance test suite development for extensible protocols is presented  method is model based  uses unit approach for modeling and specifying tests for protocols extensions  uses pure Java without any extensions  The prototype of test suite for SMTP protocol with extensions was developed using presented method. The developed test suite allows to find the following types of incompliances:  protocol rules violation  absence of supporting of required commands  wrong reply codes to protocol commands

19/19 Thank You!