An overview on the Testing and Test Control Notation version 3 (TTCN-3) Helmut Neukirchen Háskóli Íslands

Helmut Neukirchen: TTCN-3 2 Outline 1.About me 2.What is TTCN-3? 3.TTCN-3 example 4.Test automation with TTCN-3 5.Conclusions

Helmut Neukirchen: TTCN-3 3 Helmut (í. Helmút) Neukirchen: Born and educated in Germany: University of Aachen, University of Lübeck, University of Göttingen. PhD in Computer Science. Since 2008: Associate professor for Computer Science and Software Engineering at University of Iceland. Working in the field of software quality and distributed systems, in particular testing distributed systems and quality of tests. Test methodology, test languages, test tools, test metrics, test “smells”, test refactoring, test patterns, standardisation of tests. If you have interesting testing problems for students to work on as part of their BSc, MSc, PhD project: contact me! Office in Tæknigarður, best contacted via 1. About me

Helmut Neukirchen: TTCN What is TTCN-3? Testing and Test Control Notation version 3 Test specification and test implementation language. Test cases specified at an abstract level. Hardware and operating system independent. Can be turned into automated test cases by compiling into executable code, adding an adaptation layer. Standardised technology: European Telecommunication Standards Institute (ETSI). International Telecommunication Union (ITU). Strength in functional black-box testing of distributed systems. However, applicable to various other domains, levels, and types of tests: Embedded systems, distributed systems, pure software systems. From unit test to acceptance test level. From functional to non-functional (real-time, load, performance) tests.

Helmut Neukirchen: TTCN TTCN-3 example: Scenario Black-box test of a message-based weather service: Purpose of a very simple example test case: Send weather request for Reykjavik to SUT. Check that weather response is received containing: location “Reykjavik” any date, temperature within a reasonable range ( °C) wind speed within a reasonable range (0..100m/s). System Under Test (SUT)Test System Weather serviceTest case 1. Weather request(“Reykjavik”) 2. Weather response (“Reykjavik”, , 3°C, 9m/s)

Helmut Neukirchen: TTCN-3 6 TTCN-3 example: Test architecture Weather service example: SUT=Weather service, Simple example test case: only one test component needed. However, for some load test, we could, e.g., run the same test case in parallel on multiple test components to generate some load. System Under Test (SUT) Real Test System Interface Test System Abstract Test System Interface Test component InOut Specified and implemented using TTCN-3 language Test adapter (implemented using Java, C, C++ etc.) Black-box Ports Test component Test behaviour is executed by a test component. TTCN-3 supports distributed, parallel test components.

Helmut Neukirchen: TTCN-3 7 TTCN-3 example: Definition of data types and test data module WeatherServiceTestSuite { type record weatherRequest { charstring location } template weatherRequest ReykjavikWeatherRequest := { location := "Reykjavik" } type record weatherResponse { charstring location, charstring date, integer temperature, integer windVelocity } template weatherResponse ReykjavikWeatherResponse := { location := "Reykjavik", date := *, temperature := ( ), windVelocity := (0..100) } Definition of data types. Templates= Definition of test data. Definition of test data: powerful matching mechanism allows fuzziness (wildcards, ranges, regular expressions).

Helmut Neukirchen: TTCN-3 8 TTCN-3 example: Definition of ports and interfaces type port weatherPort message { in weatherResponse; out weatherRequest; } type component TestComponentType { port weatherPort weatherOffice; } Definition of a port used for communication: allowed messages (=data types). Definition of a test component type: ports that may be used by test behaviour running on an instance of that component type.

Helmut Neukirchen: TTCN-3 9 TTCN-3 example: Definition of test behaviour testcase testWeather() runs on TestComponentType { weatherOffice.send(ReykjavikWeatherRequest); alt { [] weatherOffice.receive(ReykjavikWeatherResponse) { setverdict(pass) } [] weatherOffice.receive { setverdict(fail) } } // End of testcase control { execute(testWeather()) } } // End of module Test component type used to run test case behaviour. Send message specified by template via port. Two expected alternatives: Either expected response (i.e. matched by template) is received (PASS) or any other response is receive (FAIL). Specification of order (and, e.g., conditional execution) of test cases.

Helmut Neukirchen: TTCN-3 10 TTCN-3: Further concepts Not presented in the previous example: Procedure-based communication, Timer and catching timeouts, Distributed testing using parallel test components, Concepts from general-purpose programming languages: Conditional statements (if-then-else, for, while, …), Local variables, Functions, Parametrisation.

Helmut Neukirchen: TTCN Test automation with TTCN-3 TTCN-3 test cases are abstract: weatherOffice.send(ReykjavikWeatherRequest) makes no statement how the actual communication shall be implemented: UDP datagram? SOAP over http? SMS over GSM? …? how the data to be communicated shall be encoded: XML? Proprietary ASCII-based format? Proprietary binary format? …?

Helmut Neukirchen: TTCN-3 12 Test adaptation Automatically executable TTCN-3 test suite = Compiled TTCN-3 test cases + CoDecs (encode data to be sent to SUT / decode data received from SUT) + SUT adapter (communication with SUT) + Platform adapter (operating system specific) Test tool user interface SUT AdapterPlatform Adapter Test Control System Under Test (SUT) [compiled] TTCN-3 Executable CodecsLogging TTCN-3 Test System Note: TTCN-3 execution tools come with default test control, logging, and platform adapter.

Helmut Neukirchen: TTCN-3 13 TTCN-3 & Re-usability Separation of concerns (abstract test case description vs. concrete adaptors): Re-use TTCN-3 test cases for different SUTs: Same, unmodified weather service test case can be used for SOAP/http-based system under test, for SMS/GSM-based system under test, etc. Re-use adaptors for different test cases: A SOAP/http SUT adaptor can be used for testing all kinds of Web services, an SMS/GSM SUT adaptor can be used for testing all kinds of mobile services, etc.

Helmut Neukirchen: TTCN-3 14 TTCN-3 tools Tools for, e.g., editing, compiling, deploying, executing, debugging TTCN-3: Telelogic Tester (IBM/Rational/Telelogic, Sweden), TTworkbench (Testing Technologies, Germany), OpenTTCN Tester for TTCN-3 (OpenTTCN, Finland) TTCN-3 toolbox (Danet, Germany), Exhaustif/TTCN (Métodos y Tecnología, Spain), MessageMagic (ELVIOR, Estonia), TTCN-3 Express (Fraunhofer FIRST/Metarga, Germany). TRex TTCN-3 Refactoring and Metrics tool (University of Göttingen/University of Iceland, Germany/Iceland) Standardised language and test-tool interfaces: No vendor lock-in!

Helmut Neukirchen: TTCN Conclusions TTCN-3 benefits: Standardised test technology supporting automated tests. Well documented, many tools, no vendor lock-in. Compatible with any test methodology and test management approach. Fosters re-use. Mature technology used by industry and standardisation. Nokia, Ericsson, Siemens, Motorola, Huawei, ETSI, WiMax forum, Open Mobile Alliance, Automotive Open System Architecture, … TTCN-3 standard freely available (via ETSI European Standard , ITU-T Recommendation Z.140 TTCN-3 risks: Heavyweight approach (may only pay off for big, distributed systems): Compilers are expensive (>1000€). Another language and technology to learn. While Unit Testing is possible (and done) with TTCN-3, I would in most cases recommend a xUnit framework for Unit Testing.

Helmut Neukirchen: TTCN-3 16 Thank you for your attention! Make sure to use the right tools!