1 Nikolay Pakulin/ ISPRAS Bogdan Stanca-Kaposta/ Testing Technologies Bostjan Pintar / Sintesio Tomas Urban / Elvior TTCN-XML CONFORMANCE TEST SUITE STF 475 MILESTONE C

Contents 2 Introduction Results Technical details Proposed Method ATS

Objectives 3 TTCN-XML conformance test specification STF: Cataloguing of requirements. Adapting existing framework used by the Core Language TTCN-3 reference test specification Develop tools for the validation of XML encoding/decoding Develop the implementation conformance statement (ICS). Develop the test suite structure and test purposes (TSS & TP). Develop the abstract test suite (ATS) and IXIT

Objectives (2) Cover all XML features used by the 3GPP UE tests Cover overall 80% of all sections of the part 9 “TTCN-3: Using XML schema with TTCN-3” with conformance test cases. 4

Results The STF has defined the ATS structure, the testing method and developed ATS total 347 tests (79 negative, 268 positive) 266 XML Schema documents Coverage 97 clauses of part 9 directly 21 clauses indirectly (covered by tests from other sections) 85% of the Part 9 specification. 63 clauses are covered out of 63 clauses that were identified as priority for TF 160 (3GPP RAN): 62 are covered directly and 1 is covered indirectly. The STF has filed 27 CRs. Validation by 4 teams: Testing Tech, Elvior, IBM, Devoteam. 10 issues reported to Mantis. 6 issues were closed, 3 are open, one issue is acknowledged. Effort: 54 working days in the reporting period Total effort of the project comprised 120 working days, as contracted. 5

Coverage 97 sections covered directly Indirectly covered 20 sections: Normalized string, Token, NMTOKEN, NCName, ID, IDREF, ENTITY, Hexadecimal binary, Base 64 binary, Language, Duration, Time, Gregorian month and day, Gregorian day, Gregorian month, 6.6 Sequence types, NMTOKENS, IDREFS, ENTITIES, QName. These sections define data types included in XSD built-in module. Covered by tests for Annex A “TTCN-3 module XSD” 1 indirectly covered section for 3GPP UE: “Abstract”. Covered by tests for section 8.1 6

Validation 4 teams Team members: Testing Technologies, Elvior External: IBM, Devoteam Issues: 10 6 bugs in TCs (TTCN/XML/XSD) – 1 open 1 incorrect sample in Part 9 (CR filed) 1 question (open) 2 bugs in XmlDiff – 1 fixed and 1 acknowledged Caused by incorrect handling of xsi:type attribute in Xerces XML processor – bur report filed, not fixed yet Triggers on 2 TCs for union type Hampered 2 vendors 7

Technical details 8

Overview: using XML Schema with TTCN3 9 Direct import of definitions of types, elements, attributes as TTCN3 types from XML Schema Definition (XSD) file Automagic transformation Inbound XML message -> TTCN3 template Sending TTCN3 template -> Outbound XML message ES “TTCN-3: Using XML schema with TTCN-3” XSDTTCN-3 importsuses references ATS Test case XML produces

Proposed Method 10 ATS = ∑ test cases Test case = TTCN3 script + XSD to import + reference XML Reference XML to validate encoding result SUT = TTCN-3 tool + XSD import facility + XML codec SUT executes TTCN3 script and Produces some output: verdict, log Output is validated against expected SUT launch, execution of tests, collecting outputs is vendor specific.

11 XSDTTCN-3 importsuses references ATS Test case TTCN-3 Tool (SUT) Evaluation Execution results Execution error Invalid XSD,TTCN-3 FAILPASS _ _ _ _ _ _ _ _ _ _ _ _ XML Expected output Proposed Method

Proposed method 12 Expected SUT output is encoded in the header comment of TTCN-3 script: E. pass accept, ttcn3verdict:pass Same as TTCN-3 Conformance Test Suite Reused tools to execute SUT & validate outputs validation framework from TTCN-3 Conformance TS

Test case construction 13 Positive test: valid TTCN-3, valid XSD, must compile & execute with verdict PASS Import definitions from XSD Construct a template; encode template into string Compare string with the reference XML document Decode string Check whether decoded value matches original template referencesencode Types TTCN-3 Template XSD XML Data Decoded Value decode If matches test case PASS, otherwise FAIL XML references XML Compare

Test case construction 14 Negative test: valid TTCN-3, valid XSD, must compile & execute with verdict ERROR Import definitions from XSD Construct a template; encode template into string Encoding must fail -> TTCN-3 verdict error If encoding succeeds, it is an SUT error referencesencode Invalid XSD and TTCN-3 Template not compatible Test Case is PASS Test Case FAIL Types TTCN-3 Template XSD

Sample TTCN-3 Script 15 module Pos_060101_length_001 { import from schema_Pos_060101_length_001 language "XSD" all; /* target NS: “schema:Pos_060101_length_001” */ template E1 m_msg := "length_010"; /* Aux. definitions */ testcase TC_Pos_060101_length_001() runs on C system C { /* loopback: sends back string with encoded XML*/ map(self:p, system:p); p.send(m_msg); /* encode */ alt { /* The actual matching happens here */ }

Sample TTCN-3 Script: alt 16 alt { // compare the encoded message with the XML file []p.check(receive(universal charstring:?) -> value v_rcv) { if (matchFile(v_rcv, "Pos_060101_length_001.xml", { "Pos_060101_length_001.xsd" }, v_matchError) { alt { // match decoded value to pass test [] p.receive(m_msg) { setverdict(pass, "Decoded value matches"); } [] p.receive { setverdict(fail, "XML decoding failure"); } } else { setverdict(fail, v_matchError); } } [] p.receive { setverdict(fail, “Failed to decode XML"); } }

Sample XSD 17 <schema xmlns= targetNamespace="schema:Pos_050101_namespaces_001“ xmlns:ns="schema:Pos_050101_namespaces_001">

XML Comparison 18 Two XML files are considered equal if: Tree of element nodes are equal, and qualified names of elements are the same for corresponding nodes. NOTE: take care about for unordered subtrees corresponding to choice Set of attributes for each element node are equal. Qualified names of attributes must match. Literal values of attributes must match. float type are compared by their numerical values. Textual contents of elements must match, whitespaces are ignored. CDATA is considered equal to plain text with equal contents. float type are compared by their numerical values. Comments and processing instructions are ignored. Invoked as external function Reference implementation is based on XmlUnit toolkit, written in Java.