1 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Test Implementation using TTCN-3 Libraries TTCN-3 User Conference, 7 th of June 2005 Stephan Schulz Validation & Testing Technologies Computing Architectures Lab Nokia Research Center Helsinki Sebastian Müller FSCOM Sophia Antipolis France

2 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Presentation Outline About the ETSI IPv6 Library Motivation TTCN-3 Libraries Library-based Test Suite Implementation An Example TTCN-3 Library: ETSI Common Conclusions

3 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller About IPv6 Library Implemented at ETSI by STF 276 Conformance and interoperability test specs for IPv6 Joint e-Europe/ETSI project Promote deployment of IPv6 technology within Europe Establish European player in IPv6 testing Developed to support IPv6 Logo program by IPv6 Forum Provide faster access to IPv6 market Test implementations are publicly ( open+free ) available Insure high deployment Make tests transparent

4 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Motivation Common problem in TTCN coding is the “copy & paste” mentality of test engineers  Lots of duplicated code + large code size = hard to maintain Another problem: “The reinvention of the wheel“ We desire a test suite implementation which allows as much reuse of TTCN-3 code within as well as between different test suites as possible  Much of the code developed for the IPv6 Logo certification should be reusable also in a IPv6 test suite for, e.g., 3GPP! A test suite should also be useful for additional test case development by other, external parties

5 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller The Idea Create a test suite using TTCN-3 libraries which can then be reused to compose also other related test suites  So just use TTCN-3 like a regular programming language! Ability to reuse depends on making important TTCN-3 code structuring decisions early on in test suite development  Strong need for a sound methodology!

6 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Benefits of TTCN-3 libraries Less test suite specific TTCN-3 code Less room for errors in test implementations Use of validated TTCN-3 code segments “More readable” test implementations Less time needed for test case implementation and validation Avoids the “reinvention of the wheel”  Summary: Higher quality test suite implementations!

7 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Keys to Reusable TTCN-3 Code Follow the TTCN-3 Framework Methodology for Test Suite Implementation Modularization Rules on how to implement TTCN-3 behavior Naming conventions Specify well defined interfaces Use component type compatibilitytype compatibility Document TTCN-3 code Standard practice in any programming language Free documentation tools are available

8 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Libraries in TTCN-3 A TTCN-3 library consists of modules separating behavior(s) types and constants templates Example: Libraries in the IPv6 test suite Common IPv6 independent code (e.g, Synchronization) Common IPv6 code (e.g., Echo Procedure) Common IPv6 test suite code (e.g., Ipv6 Host Preamble ) But then (not reusable): Specific IPv6 test suite code (e.g., IPv6 Core Test Cases)

9 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Basic Structuring of IPv6 Test Cases Decompose test case behavior based on well established methodology Configuration (CF) function for test configuration set up Preamble (PR) function get IUT into testing state Test body (TP) function perform test as specified by test purpose Postamble (PO) function return IUT to its initial state  All of the above implemented by invoking test suite, IPv6, or common library functions! Implement test case or test case (TC) functions by simply invoking PR, TP, PO functions

10 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller testcase (PR) function (TP) function (PO) function (MTC) (TC) function (PR) function (TP) function (PO) function (PR) function (TP) function (PO) function (TC) function (PTC1)(PTCn) testcase (MTC) create/start preamble sync TP sync TC functions in Single and Multiple Interfaces Specify test case (TC) functions for each test component Use MTC in concurrent test cases only to spawn and synchronize PTCs No Test case (TC) functions needed

11 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Rules for reusable TTCN-3 Behavior Function should not contain any stop statement since it prevents the execution of postambles! Test suite independent functions should avoid to use the setverdict operation Instead use the return value of a function ! Verdict setting harms reuse, e.g., testing of valid versus invalid behavior A module which defines reusable behavior should contain a test component type definition tailored its needs

12 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller About TTCN-3 Code Documentation Without proper documentation reuse is questionable Documentation of library code is well established standard practice in other programming languages (e.g., man pages) It should be possible to use a TTCN-3 library or modules without ever reading source files Any TTCN-3 code should be documented regardless of its reusability! Guidelines part of framework document A free documentation tool t3doc is available: Follows ideas of similar tools for C and Java, e.g., doxygen Generates HTML pages from TTCN-3 code comments WWW tool download:

13 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller A Library Example: ETSI Common Three modules with common type and value definitions Common basic, data string and text string constant and type definitions, e.g., c_CRLF, UInt, etc. Module for verdict control handling Definition of function return code and verdict setting functions Module for timing related functions, e.g., f_sleep Modules which implement a generic test component synchronization as well as sync examples Our classic “reinvention of the wheel”

14 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller The Synchronization Library Module Implements an easy-to-use synchronization handling for both single as well as multi component test cases Key concepts Named synchronization points TTCN-3 default used for handling shutdown Uses extensively TTCN-3 log statement Results in more readable execution traces! Current main user API: f_selfSync(syncPointName, retCode) f_serverSyncClients(#, syncPointName) f_serverSyncClientsTimed(#, syncPointName, time) f_serverWaitForAllClientsToStop() f_clientSync(syncPointName, retCode) [a_dummyShutDown()] // shutdown example

15 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Example of Nice Multi-Component Sync f_tc1_ptc2() (PTC2) f_tc1_ptc1() (PTC1) f_PR_tc1_ptc1() f_TP_tc1_ptc1() f_PO_tc1_ptc1() tc1() (MTC) create/start PTC1 and PTC2 f_serverSyncClients(2,“prDone”) f_serverSyncClients(2,“tpDone”) f_serverWaitForAllClientsToStop() f_clientSync(“prDone”, e_success); f_clientSync(“tpDone”, e_success); f_PR_tc1_ptc2() f_TP_tc1_ptc2() f_PO_tc1_ptc2() f_clientSync(“prDone”, e_success); f_clientSync(“tpDone”, e_success); Ready! Ready. Ready! Ready.

16 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Example of Sync Problem Handling f_tc1_ptc2() (PTC2) f_tc1_ptc1() (PTC1) f_PR_tc1_ptc1() tc1() (MTC) create/start PTC1 and PTC2 f_serverSyncClients(2,“prDone”) f_clientSync(“prDone”, e_error); f_PR_tc1_ptc2() f_clientSync(“prDone”, e_success); STOP! Ready! STOP! f_wait4AllClientsToShutDown() a_shutDown_if1() (calls stop!) a_shutDown_if2() (calls stop!)

17 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Example of Self Synchronization Note that shutdown defaults are neither test case nor component but only interface specific (e.g, IPv6, SIP, etc.) Therefore the same default can be used to shutdown any test component (which uses that interface) in any test case. tc2() f_PR_tc2() f_TP_tc2() (MTC) f_selfSync(“prDone”, e_success); f_selfSync(“tpDone”, e_error); STOP! a_shutDown_if1() (calls stop!)

18 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller Conclusions This really works in practice The ETSI common TTCN-3 library has already been successfully deployed within STF 276 as well as a number of different protocol test suites within Nokia First versions of the generic IPv6 and IPv6 Core libraries have been successfully used in the execution of the ETSI IPv6 test suite prototype against a Linux Ipv6 SUT At the end the real key to implementing truly reusable TTCN-3 code are TTCN-3 framework Well-defined interfaces TTCN-3 code documentation

19 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller References More info about this STF and documents Test suite and executable download ETSI DTS/MTS-IPT-0001, “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); IPv6 Testing: Methodology and Framework”, Sophia-Antipolis, S. Moseley et al., “IPv6 Test Specifications from ETSI”, Global Ipv6 Summit, Barcelona, (to appear), June, 2005.

20 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ /S. Schulz and S. Müller About Component Type Compatibility type component Ipv6Comp { IpPort pt_ip; SyncPort pt_sync; timer t_sync; } type component SyncComp { SyncPort pt_sync; timer t_sync; } function f_clientSync(…) runs on SyncComp { // some code } function f_PR_EchoProcUp(…) runs on Ipv6Comp { var FncRetCode v_ret; v_ret := f_echoProc(); f_clientSync(“prDone”, v_ret); } Key to create self contained libraries is the runs on clause with TTCN-3 functions An IPv6 preamble may invoke, e.g., a synchronization function, if its component type definition is compatible