1. IHA præsentation1 Protocol Testing Lesson 5

2. IHA præsentation2 Outline for today Guidelines for testing protocols TTCN – A Standard Language For Testing Protocols Motivation The Language The usage

3. IHA præsentation3 Guidelines for protocol testing

4. IHA præsentation4 Guidelines Unit Testing tests if individual units of source code are fit for use. A unit is the smallest testable part of an application Integration Testing individual software modules are combined and tested as a group Occurs after unit testing Coverage Testing describes the degree to which the source code has been tested

5. IHA præsentation5 Guidelines Smoke Testing Collection of written tests that are performed on a system prior to being accepted for further testing Regression Testing seeks to uncover software regressions previously working software functionality stops working as intended Conformance Testing to determine whether a system meets some specified standard

6. IHA præsentation6 Conformance Testing Framework Conformance testing is the process of testing the extent to which implementations adhere to the requirements stated in relevant standard or specification Conformance Testing is functional black-box testing Functional refers to the correct functional behavior of an Implementation Under Test (IUT) Black-box means that the internal structure of the IUT remains hidden

7. IHA præsentation7 Conformance Testing Framework IUT – Implementation Under Test UT – Upper Tester function LT – Lower Tester function PCO – Point of Control & Observation TCP – Test Coordination Procedures ASP – Abstract Service Primitive

8. IHA præsentation8 Conformance Testing Framework UT plays the role of a user that makes use of the service provided by the IUT LT plays the role of the peer entity of the IUT, i.e. the LT and the IUT communicate in order to provide the service to the IUT

9. IHA præsentation9 Conformance Testing Framework PCO is a standardized interface. Typically the lower interface of an IUT is accessible only from the remote Communication is always meant to be asynchronous => PCO is modeled as two FIFO queues

10. IHA præsentation10 Conformance Testing Framework TCP can be used to coordinate the actions of LT and UT. LT and UT can be on separate testers. IUT and LT communicates by means of ASP. PDUs are encoded in ASPs of the underlying service.

11. IHA præsentation11 Conformance Testing Framework Test case development starts with: Identification of a test purpose Description of a single requirement or set of related requirements A test case is an implementation of a test purpose

12. IHA præsentation12 Conformance Testing Framework Test Case Scheme A test case starts and ends in stable testing states. Preamble to take the IUT from stable testing state to test state. Test body is performed in order to check the test purpose. Verification to check test body Postamble to drive the IUT into a stable testing state again

13. IHA præsentation13 Protocol Testing Until now we have: Developed our protocol layer including: –Timers –State - event model –PDUs to be exchanged Interfaces to upper and lower layers HOW DO WE TEST OUR PROTOCOL LAYER?

14. IHA præsentation14 Protocol Testing Protocol Testing is mainly about: Testing the state – event model. PDU (message) Our protocol layer Test System ?? Observe response

15. IHA præsentation15 Protocol Testing Testing the state – event model Difficult part is when we send an unexpected message into our layer Unexpected PDU (message) Our protocol layer Test System ?? Ignore – take no action Reject – return an explicit denial to the sender Allow – take action to process the event Standard testing languages and tools available

16. IHA præsentation16 Test & Testing Control Notation Edition 3 (TTCN-3)

17. IHA præsentation17 Introduction What is TTCN? Why use TTCN? The TTCN-3 Language Background Example TTCN-3 Usage Where is it used? Future Outlook

18. IHA præsentation18 What is TTCN? Internationally standardized testing language for formally defining test scenarios and their implementation. Designed purely for testing Tool independent Flexible and powerful language Combined use with other languages like ASN.1 Well-defined syntax, interchange format Different presentation formats (e.g. tabular and graphical) Support text-based protocols Support for IP protocols

19. IHA præsentation19 What is TTCN? In TTCN we can…. Define Test Suites Define Test Cases Define Test steps Declare variables (test suite & test case variables) Declare timers Create PDUs Etc..

20. IHA præsentation20 What is TTCN? TTCN Edition 3 is documented in a serie of ETSI Specification ES 201 873-1 (TTCN-3 Core Language) ES 201 873-2 (Tabular presentation format) ES 201 873-3 (Graphical presentation format) etc.

21. IHA præsentation21 Why Use TTCN? Quality a well-tested product increases confidence in the product’s quality Time/Cost Improvement TTCN is a language designed for protocol conformance testing TTCN is standardized TTCN is fairly simple Formal strict rules, complete BNF and operational semantics Abstract TTCN test suites are independent of an implementation

22. IHA præsentation22 Application Areas New application areas Software Testing Text-based protocols… Additional communication paradigm Message-based communication Procedure-based communication Different kinds of testing Functional testing Conformance testing (behavioral testing) Scalability testing…

23. IHA præsentation23 SUT TTCN Basics PASS, FAIL or INCONCLUSIVE TTCN Test Suite Result IUT Implementation Under Test Observe Responses Send Stimuli Behavioral testing send stimulus, observe response was response received within specified time? Test System

24. IHA præsentation24 TTCN – Example testcase MyTestCase() runs on DNSTester { P.send(query); P.receive(answer); setverdict(pass); stop; } Minimal Test Configuration

25. IHA præsentation25 TTCN-3 test systems in a nutshell TTCN-3 specifies a test but a test system is needed for test execution TRI and TCI standards define test system architecture TTCN- 3 tools are required to support internal interfaces Allows reuse of test platforms with different tools but also for different SUTs A test system requires A TTCN-3 tool = TTCN-3 compiler and execution environment A test platform for a specific device under test Note: Tools come with default Test Control & Logging Test System Executor SUT AdapterPlatform Adapter Test Control System Under Test (SUT) [compiled] TTCN-3 Executable CodecsLogging TTCN-3 Test System TCI TRI ( s + s) ( ) TCI = TTCN-3 Control Interface TRI = TTCN-3 Runtime Interface

26. IHA præsentation26 An example adaptation: A IPv6 test system Test System Executor Ethernet AdapterReal-time Adapter Parameter File Open Source FreeBSD Router [compiled] IPv6 ATS TTCN-3 Executable IPv6 Codecs Tool Logging TTCN-3 Test System TCI TRI

27. IHA præsentation27 User’s view of TTCN-3

28. IHA præsentation28 The Core Language and Other Presentation Formats TTCN-3 Core Language Text format Presentation Format 3 Presentation Format n Graphical Format Tabular Format Core format is text based (most popular) TTCN-3 can be edited or viewed in other formats Tabular format (for TTCN-2 people) Graphical format (good for visual overview) Other standardized formats in the future? Proprietary formats possible

29. IHA præsentation29 Example Core (Text) Format testcase TC_resolveEtsiWww() runs on DnsClient { timer t_ack; serverPort.send(m_dnsQuestion("www.etsi.org")); t_ack.start(1.0); alt { [] serverPort.receive(mw_dnsAnswer("172.26.1.17")) { setverdict (pass); } [] serverPort.receive { // any other message setverdict(fail); } [] t_ack.timeout { setverdict(inconc); } } t_ack.stop; }

30. IHA præsentation30 testcase TC_resolveEtsiWww() runs on DnsClient Example Graphical Format DnsClient mtc DnsPort serverPort m_dnsQuestion("www.etsi.org") timer t_ack t_ack alt mw_dnsAnswer("172.26.1.17") ? t_ack inconc fail pass t_ack

31. IHA præsentation31 Example Tabular Format Testcase NameTC_resolveEtsiWww() Group Purpose System Interface MTC TypeDnsClient Comments Local Def NameTypeInitial valueComments t_acktimer Behavior serverPort.send(m_dnsQuestion("www.etsi.org")); t_ack.start(1.0); alt { [] serverPort.receive(mw_dnsAnswer("172.26.1.17")) {setverdict (pass); } [] serverPort.receive // any other message { setverdict(fail); } [] t_ack.timeout { setverdict(inconc); } } t_ack.stop; Detailed Comments:

32. IHA præsentation32 Use of TTCN-3 With Other Languages TTCN can be integrated with types systems of other languages Fully harmonized with ASN.1 (1997) Harmonized with other languages IDL, XML, C/C++ TTCN-3 Core Language IDL Types & Values Other types & Values n XML Types & Values ASN.1 Types & Values

33. IHA præsentation33 ASN.1 Abstract Syntax Notation One language for describing structured information standardized internationally (ISO/IEC 8824, ITU-T X.680) TTCN includes ASN.1 used for creating data descriptions and constraints can be part of TTCN script or imported as a module SDL also includes ASN.1 ITU-T Z.105 ASN.1 can be re-used across design and test! ASN.1 SDLTTCN

34. IHA præsentation34 TTCN – Example testcase MyTestCase() runs on DNSTester { P.send(query); P.receive(answer); setverdict(pass); stop; } P.receive( answer ) blocks until it receives a message that matches answer Any other message does not unblock the tester, which then blocks forever If no message is received, the tester will also block forever

35. IHA præsentation35 TTCN Test Case Example testcase MyTestCase2() runs on DNSTester { timer t := 5.0; P.send(query); t.start; alt { [] P.receive(answer); { setverdict(pass); } [] P.receive { // any message setverdict(fail); } [] t.timeout { setverdict(inconc); } stop; }

36. IHA præsentation36 Test Suite Building blocks of a TTCN-3 Test Suite Test Data Types Data types which specify Structure of messages or calls and their information elements (fields, parameters) Internal data structures (e.g., for computation) Possibly encoding or display information Built-in basic types integer, boolean, float, bitstring, hexstring, octetstring, charstring, universal charstring... and structured types record, record of, set, set of union, enumerated... and special types such as component, port, verdicttype, default, etc

37. IHA præsentation37 Test Suite Building blocks of a TTCN-3 Test Suite Actual Test Data Test Data Types Actual test data (values) used during testing Constants or Templates for specific message or call parameter values Matching expressions for allowing multiple message or call parameter values value range, value list, wildcards, presence, length, size, permutation regular expressions Using also template decomposition, parameterization and modification

38. IHA præsentation38 Test Suite Building blocks of a TTCN-3 Test Suite Test Configuration Actual Test Data Test Data Types Static aspects Test component and port types Dynamic aspects Dynamic instantiation and management of test components Mappings of test components to abstract test system interfaces Connections between test component interfaces Management of test components

39. IHA præsentation39 Test Suite Building blocks of a TTCN-3 Test Suite Test Behaviour Test Configuration Actual Test Data Test Data Types test cases specify sending/receiving messages, computation (e.g., checksums), and verdict assignment can be decomposed with functions and altsteps can (re)use default behaviour can use timers and timeouts test execution control (optional) order, repetition, conditions, etc

40. IHA præsentation40 TTCN-3 Module module EtsiDnsTests { // Test definition part control { // Test execution part // (optional) } Module (…) Module Control Module Definitions

41. IHA præsentation41 Definitions Part module EtsiDnsTests { // Message structure // Actual test data // Test configuration // Test Case definitions }

42. IHA præsentation42 module EtsiDnsTests { group MessageStructure { // Defintions of message types } group TestData { // Templates for messages instances } group TestSystemConfiguration { // Port and component types and mappings } group TestCases { // Test case definitions } Structured Definitions Part

43. IHA præsentation43 Message Structure and Test Data type record DnsMsg // simplified message structure! { DnsMsgKind kind, charstring question, charstring answer optional } type enumerated DnsMsgKind {e_query, e_response} template DnsMsg m_dnsQuestion( charstring p_question ) { kind := e_query, question := p_question, answer := omit // no answer } template DnsMsg mw_dnsAnswer( charstring p_answer ) { kind := e_answer, question := ?, // any question ok answer := p_answer }

44. IHA præsentation44 Test Configuration type port DnsPort message { inout DnsMsg } // Note: port types may also allow multiple different // message types or restrict the direction type component DnsClient { port DnsPort serverPort } // Note: component types can also define multiple port // instances of the same or different port type and // declare timers, constants or variables

45. IHA præsentation45 Test Behaviour query response serverPort testcase TC_resolveEtsiWww() runs on DnsClient { timer t_ack; serverPort.send(m_dnsQuestion("www.etsi.org")); t_ack.start(1.0); alt { [] serverPort.receive(mw_dnsAnswer("172.26.1.17")) { setverdict(pass); } [] serverPort.receive { // any other message setverdict(fail); } [] t_ack.timeout { setverdict(inconc); } } t_ack.stop; } mtc

46. IHA præsentation46 The Control Part Module (…) Module Control Module Definitions module EtsiDnsTests { // Test definition part modulepar boolean mp_example; testcase TC_resolveEtsiWww() runs on DnsClient { //.. as in previous slide } // Test execution part control { if (mp_example) { execute(TC_resolveEtsiWww()); }

47. IHA præsentation47 TTCN-3 Usage

48. IHA præsentation48 TTCN in 3GPP Lessons from GSM TTCN ATSs contribute to interoperability powerful competitive tool for entry to the global GSM mobile market TTCN ATSs widely used within members’ development processes 3GPP TTCN in Protocol Conformance Specs 34.123-1Prose description of tests 34.123-2Implementation Conformance Statement (ICS) 34.123-3TTCN Abstract Test Suite TTCN will be the normative specification

49. IHA præsentation49 TTCN and Bluetooth Bluetooth SIG selected TTCN for... definition of Protocol Requirements tests interoperability definition of Profile Requirements tests application interoperability end-user expectations RF Baseband Audio Link Manager L2CAP TCP/IPHIDRFCOMM Applications Data Control