TTCN-3 Test Case Generation from arbitrary traces Capture & Replay Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies)

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Agenda The Motivation The Technical Challenges Capture & Replay for TTworkbench Demo Q & A 2

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Motivation(s) TTCN-3 Users We have a TTCN-3 framework, where do the test cases come from? Issues Manual test case creation tedious task Error prone Domain Knowledge required Non-TTCN-3 Users We see a scenario and we would like to replay it later Issues What means replay w.r.t. testing? What is the test system? What is the system under test? 3 The desire to capture observed communication in test scripts suitable for test automation

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © The Idea 4

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © What is a Trace? 5

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © What is a Trace? The observed communication is captured (stored) in a trace (file) Trace contains Communication between two or more entities Message exchange One or more protocols 6

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © A Generic Communication Setup 7 A1 A2 B1 B

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © The Setup But Who is Who? A1 B1 A1 SUT B1 IOP1 IOP2 SIM1 SIM2

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © The Setup From Message to Templates A1 B1 INVITE SIP/2.0 Call-ID: Contact: Content-Disposition: session Content-Length: 150 Content-Type: application/sdp CSeq: 2 INVITE From: "TESTER1" ;tag=1169 Max-Forwards: 70 To: Via: SIP/2.0/UDP :5061;branch=z9hG4bK template INVITE_Request T1 := { requestLine := { method := “ INVITE”, requestUri := “ sipVersion := “ SIP/2.0” }, msgHeader := { callId := “ contact := …, cSeq := “ 2 INVITE”, fromField := "TESTER1 …", toField :=, via := …., maxForwards := { fieldName := MAX_FORWARDS_E, forwards := 70 },.... allow := omit,...

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © The Technical Challenge It is somehow easy to generate TTCN-3 templates from each message, but the challenge is to obtain usable and maintainable test data and test scripts Variable information From run to run, perhaps unpredictable From message to message, perhaps constant perhaps dependend Variable but constant information Should be configurable before execution White noise Lot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution 10

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © The Technical Challenge It is somehow easy to generate TTCN-3 templates from each message, but the challenge is to obtain usable and maintainable test data and test scripts Variable information From run to run, perhaps unpredictable From message to message, perhaps constant perhaps dependend Variable but constant information Should be configurable before execution White noise Lot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution 11 Protocol and/or domain knowledge has to be included!

The Solution Automated TTCN-3 Test Generation and Execution 12

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Example Scenario 13 SIP User A calls User B User B accept call

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Example Scenario 14 SIP Use a common Monitor Tool to capture messages e.g. Wireshark Trace File

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Example Scenario 15 Trace FileTTworkbench Execution Put the Trace File into TTworkbench …. Generate the captured Test Scenarios… Replay your previously captured Test Scenarios full automatically

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Example Scenario 16 Trace FileTTworkbench Execution Put the Trace File into TTworkbench …. Generate the captured Test Scenarios… Replay your previously captured Test Scenarios full automatically

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Know-How Integration Manual definitions in a configuration file, to Extract existing data and use the same values in the message flow (sequence numbers, call IDs, …) Define uninteresting fields for ignoring at matching time Automatically extract similar data from templates to create “better” templates and structures Reuse existing Type systems and codecs Test adapters and component definitions Test functions and Alt-Steps

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © DEMO! 18

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Key Benefits Speed up functional and interoperability testing By reducing manual task Automate testing Generate your test, instead of implementing it manually Execute them full automatically Get your detailed test reports Quality Optimization Test case analyzer reduces errors in test cases Fast reusability and easy extension of already defined tests 19

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Supported Capturing Sources & Requirements Wireshark The most popular network analysis and capturing tool Jpcap Service The remote capturing service provides capturing on remote machines Pcap Files Trace files created beforehand TTworkbench logs (TLZ) Analyze the logs without an SUT Capturing Console Input and Output Capturing of data via serial connections (RS232, USB) SSH, TELNET Local Console Windows WinPCAP Linux/Unix Libpcap at least version 1.0 Recommendation: Install Wireshark Includes the installation of WinPCAP and LibPCAP 20

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Further information Testing Technologies testingtech.com facebook.com -> Testing-Technologies Capture & Replay testingtech.com/products/ttworkbench.php testingtech.com/download/datasheets/CaptureAndReplay.pd f 21

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Contact 22 Bogdan Stanca-Kaposta Testing Technologies Theo Vassiliou Testing Technologies

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © The process 23

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Features Designed to generate test cases from existing traces Start your own capture with the Capture & Replay tool Or import trace files created beforehand with other capturing tools, e.g. Wireshark Capture from multiple sources at the same time 24

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © Benefits Fast and easy test authoring No programming skills required Support of multiple capturing sources Post-hoc analysis of real world situations via simulation mode Flexible creation of test configurations out of selected scenarios Decide which communication peers shall be tested Decide which ones shall be simulated 25

Copyright Testing Technologies Confidential Information. All Rights Reserved. More Information at © On the second thought... User: Wait! How about the variable data? The Manager: Sell consultancy $$$ Technical guy: Let‘s integrate the protocol knowledge Extract existing data and use the same values in the message flow (e.g. sequence numbers, call IDs) Ignore the uninteresting fields for matching Extract similar data from templates User: I would like to reuse my Test Structure The Manager: Reuse existing Adaptation, Codecs, Type systems, Components, Functions, Alt Steps 26