oneM2M Testing and Dev. Support TST-2016-0185R01-testing_and_dev_support oneM2M Testing and Dev. Support Group Name: TST Source: JaeSeung Song (TST Chair), KETI, jssong@sejong.ac.kr Meeting Date: 2016-10-17 Agenda Item: TBD

Contributors Testing oneM2M with POSTMAN (Lionel Florit, Cisco) Conformance Testing Tool – “oneM2M Tester” (Martin Miao, KETI, EGM, Ericsson, Sejong Univ.) oneM2M Testing with TTWorkBench (Richard, Spirent) Tools for oneM2M developers and SIs (Dale, InterDigital) TTA verified testing tool (Keebum, TTA) Web-based AE testing tool (JaeSeung, KETI & Sejong Univ.) ontology validator (Franck, EGM) F-Interop as a oneM2M interop testing tool (Miguel, ETSI)

Testing oneM2M with POSTMAN Lionel Florit lflorit@cisco.com

Why Postman? It is free, free and it is free  A Google Chrome App, easy to install on any system as Easy to collaborate, exchange and share test libraries Use it to aid in the CSE development process The test libraries provide real queries, ready to edit/use/augment Limitations HTTP only Does not do the server side: good to test a CSE but can’t emulate one Not a compliance tool: does not check if the response is compliant! Only sends compliant requests.

Test your CSE in 5 minutes Install Postman: www.getpostman.com Import the oneM2M TS13 test collection (JSON) using this link Test! https://www.getpostman.com/collections/d00200a7484683f09298

Martin MIAO (novamartin@keti.re.kr) Jaeho KIM (jhkim@keti.re.kr ) oneM2MTester Martin MIAO (novamartin@keti.re.kr) Jaeho KIM (jhkim@keti.re.kr )

Conformance Testing Tool oneM2M Tester In parallel with the test code (ATS), a test tool development is also ongoing System adapter and codecs User Guide Based on the open source TTCN-3 tool Evaluation HTTP binding and XML serialization at the 2nd oneM2M plugtest event HTTP binding and JSON serialization at the 3rd oneM2M plugtest event Demos and dissemination Presentations and live demos at international software testing conferences and events (HUSTEF, UCAAT, ECS 2016) etc. Tutorials via the oneM2M webinar series oneM2M Tester will be open sourced Open Source

oneM2M Tester Capture oneM2MTester Testcase Executor capture

oneM2M Tester Video Demo

Testing oneM2M with TTworkbench and TTCN-3 October 2016

Ttworkbench and TTCN-3 for oneM2M oneM2M Functional Architecture – TTworkbench Test Access TTworkbench TTCN-3 Conformance test tool Sensors, Actuators, Gateways Field Domain Infrastructure Domain #1 Application (AE) Mca Application (AE) Mca Mca Mca Infrastructure Domain Provider #2 Common Service (CSE) Common Service (CSE) Mcc Mcc Mcc Mcn Mcn Network (NSE) Network (NSE) Optionally available via non-standard test API Standard Test API

oneM2M Testing with TTworkbench TTworkbench uses existing oneM2M TTCN-3 Conformance Test Cases for the CSE Registration Data management and repository Location Group management Future ….. Future AE Conformance …. Extendable TTCN-3 Framework Possibility to reuse existing TTCN-3 code to design custom test scenarios Possibility to use existing Test Adaptation Plugins to access Sensors, Actuators or other entities Possibility to integrate custom Test Access as required Also possibilities to use for Interoperability testing if required

TTworkbench Support Latest versions of both Release 1 & Release 2 of the oneM2M standard Bindings CoAP HTTP MQTT WebSockets (added by oneM2M v2) Serialization XML JSON CBOR (added by oneM2M v2)

Dale Seed InterDigital Dev. Support Dale Seed InterDigital

Tools for Developers and SIs InterDigital has several useful tools for oneM2M developers oneM2M Wireshark Dissector oneM2M Primitive Exerciser oneM2M curl scripts oneM2M test tools

Publically Available IN-CSE, MN-CSE and AEs IN-CSE is hosted in the Cloud (No SW Install Required!) Ideal for folks that want to quickly test without installing a CSE locally Submit a request for AE-ID or CSE-ID and PSK Credentials via InterDigital’s Website Register your AE or CSE to IN-CSE via PSK-based secured Mca or Mcc (HTTPs or CoAPs) MN-CSE is available for download from InterDigital’s Website Ideal for folks that want to run locally in their own environments Simply download, configure and run MN-CSE Register your AE or CSE to MN-CSE via Mca or Mcc (HTTPs or CoAPs) InterDigital has sample AEs available for download from InterDigital’s Website These AEs are based on InterDigital’s oneM2M programmatic API library Simply download and try them out These AEs can be run with InterDigital’s IN-CSE, MN-CSE or your own CSE Go to InterDigital’s Developer Portal and sign up for a oneMPOWER account IN-CSE Contact Information: FQDN: cse.onem2mpub.wot.io IP Address: 137.116.117.137 HTTPs Port: 9013 CoAPs Port: 9012 https://app.interdigital.com/m2mconfig/users/sign_in

oneM2M Message Dissector View and decode oneM2M protocol in Wireshark! Dissector can be downloaded from http://www.interdigital.com/downloads/onem2m-wireshark-dissector Currently Supports HTTP and CoAP oneM2M bindings (MQTT under development) Selectively capture or filter on oneM2M traffic Dissects and displays oneM2M request and response parameters Dissector is implemented as a Wireshark plug-in that can be easily installed Dissects and displays oneM2M content

oneM2M Primitive Exerciser Tool is hosted in the Cloud No SW Install Required! URL – http://cse.onem2mpub.wot.io:3000/ Username: oneM2M Password: oneM2MForEveryone Currently supports HTTP and CoAP oneM2M bindings (MQTT under development) The oneM2M Primitive Exerciser is a handy tool to generate oneM2M requests (e.g. to send to a CSE) Think of it as Postman on steroids  Tool provides a history of issued requests that can be re-issued (similar to Postman) The tool is oneM2M aware and provides a nice GUI for developers to build oneM2M requests and decode responses Tool supports a sequence generator that can be used to create sequences of requests that can be issued sequentially The tool also allows developers to see protocol binding and serialization level details for requests it generates and responses it receives Tool supports saving requests to file that can be re-loaded into the tool and re-issued

(Yes, you know who you are ) oneM2M curl scripts #################################################### # Create AE resource using HTTP binding and JSON serialization curl -v -X POST -H 'X-M2M-RI: xyz2' -H 'X-M2M-Origin: CAE001' -H 'Content-Type: application/vnd.onem2m-res+json; ty=2' -H 'Accept: application/vnd.onem2m-res+json' -d @payloadAe.json http://127.0.0.1:9011/oneMPOWER-IN-CSE > POST /oneMPOWER-IN-CSE HTTP/1.1 > User-Agent: curl/7.32.0 > Host: 127.0.0.1:9011 > X-M2M-RI: xyz2 > X-M2M-Origin: CAE001 > Content-Type: application/vnd.onem2m-res+json; ty=2 > Accept: application/vnd.onem2m-res+json > Content-Length: 96 > { "m2m:ae": { "rn": "TestAE_JSON", "api": "myAppId", "rr": "true" } < HTTP/1.1 201 Created < Content-Type: application/vnd.onem2m-res+json < X-M2M-RI: xyz2 < X-M2M-RSC: 2001 < Content-Location: /ID-CSE-01/CAE001 < Content-Length: 154 < "m2m:ae":{ "aei":"CAE001", "ct":"20160608T103559", "et":"20160611T215559", "lt":"20160608T103559", "pi":"ID-CSE-01", "ri":"CAE001", "rn":"TestAE_JSON", "ty":2 curl is a popular open source command line tool used for issuing HTTP client requests. It ships with most operating systems curl is used by many hard core developers preferring a command line interface (Yes, you know who you are ) InterDigital maintains a library of curl scripts that provide working examples of different types oneM2M requests Curl scripts can be downloaded from: https://app.interdigital.com/m2mconfig/downloads

oneM2M test tools InterDigital has developed several test tools and test suites for the purposes of interop and conformance testing These tools and test suites have been used to test InterDigital’s oneMPOWER solution as well as oneM2M solutions from other parties. At this time, InterDigital is not making these tools publically available. For more information regarding these tools please contact us

TTA Testing Tool for oneM2M Device (AND-AE) Keebum Kim, TTA, keebum.kim@tta.or.kr

TTA Testing Tool for oneM2M TTA started “oneM2M TTA Verified” Program in April 2016. To run this program, TTA prepared two kind of Testing Tool. Testing Tool for originators (e.g. ADN, ASN, MN) Testing Tool for receivers (e.g. IN, MN) For receivers, many references are existing. (like postman), but it was hard to find for originators. So, TTA decide to develop oneM2M Device (originator) Testing Tool. Further slides are for introducing TTA oneM2M Device Testing Tool.

Testing Scenario Test Scenario Flow ③ ⑦ ① ④ ② ⑤ ⑥ oneM2M Platform oneM2M Based Device Tester Control CRUD RESTful Request Result View Test-Case Configuration & Launch ③ Test Scenario Flow ⑦ ① ④ Verifying ② Real-Time Ajax Request oneM2M Platform Mongo DB Tester (WAS) ⑤ Logging Resource Data Response ⑥

Testing Tool Structure ADN ASN MN AE(1…N) CSE(1) AE(0…N) IN MQTT Broker CoAP Server HTTP Mongo DB oneM2M Testing Tool App User Common Platform Testing Tool Create raw packet file real-time Robot Framework Rebot Result View MQTT Client HTTP Log MQTT Log CoAP Log WAS Packet sniffer

Testing Tool Application Test Application Test Environments Test Result Test Launcher Device Config URI Config Test Case Check-Up List Initial Config Test Result Display Packet Sniffing Dump Output HTML Output Test Result Statistics Status Display Test Information Log Display

Select Binding Protocol Testing Tool Test Environment Configuration Step. Select Binding Protocol (HTTP, CoAP, MQTT) Select Node Type (ADN, ASN, MN)

Testing Tool (Cont.) Test Running Step. Summary of all Test Results Detail information of each test History logs during testing

Web-based oneM2M Self-Testing Tool for Dev. JaeSeung Song, Sejong University & KETI, jssong@sejong.ac.kr

python oneM2M pcap library (as a testing tool) View an oneM2M packet via the webpage Interface based on the webpage Information of a selected packet http://goiot-forum.org/~~~/View packet The process to select a packet that you want 3 Select the pcap file in my computer 1 Select the pcap file … Selected file : ??.pcap reset Select a packet in pcap file 2 Select a packet A packet B packet C packet … OK reset

python oneM2M pcap library (as a testing tool) Compare the two oneM2M packets via the webpage Interface based on the webpage Information of two selected packets http://goiot-forum.org/~~~/Comapre packet The process to select two packets that you want Select the first pcap file in my computer 1 Fisrt packet secnod packet Select the pcap file … Selected file : ??.pcap reset Select a packet in pcap file 2 Select a packet A packet B packet … OK reset Select the second pcap file in my computer 3 Select the pcap file … Selected file : ??.pcap 4 reset Two pcap files does not matter the same or different Select a packet Select a packet in pcap file A packet B packet … OK reset

oneM2M AE code oneM2M Testing code oneM2M AE code will send the specific user request to TestSystem (In this document we will use the AE registration case) oneM2M AE Code is constructed for processing the specific user request such as CRUDN Testable AE is uploaded via web interface Cloud AE TE Android App (AE) Send oneM2M AE code oneM2M AE code Testcase (TTCN-3) Build Upload oneM2M Testing code Testable AE oneM2M Testing code Triggering User has to implement the oneM2M Testing code. It is used to handle the Test System(TE) request (Test case) oneM2M Testing code is implemented for processing the test case receive the test case Value. Testcase will check the oneM2M Payload for user specific command such as AE registration and issue the report about testing result

Cloud AE TE Send oneM2M AE code Testcase (TTCN-3) oneM2M Testing code Triggering After User upload the testable AE and select the specific test cases, Test System processes it.

Insert “testing code” Start “testing server”

Check a received primitive Execute the corresponding testcase. For example, execute Registration

Semantic ontology validation. Mengxuan Zhao, Franck Le Gall, Easy Global Market Mengxuan.zhao@eglobalmark.com, Franck.le-gall@eglobalmark.com

Validation in semantic data life cycle From data production to consumption Semantic data production: Ontology validation Annotation validation Semantic data consumption Reasoning rule validation Query validation Our focus

Basic aspects to be validated on semantic information Lexical validation Ill-formed RDF serialization format (XML, JSON, etc.) Syntactic validation Untyped resources and literal Ill-formed URIs and language tags on literals Problematic prefix namespaces Unknown properties and classes Semantic validation Inheritance relationships for classes and properties Consistency of classes and individuals

Self-ontology selection Popular reference ontologies are integrated in the tool Format Filename extension N-Quads .nq N-Triples .nt N3 .n3 Turtle .ttl TriG .trig TriX .trix RDF/JSON .rj JSON-LD .jsonld BinaryRDF .brf RDF/XML .rdf OWL .owl

Results See running at ETSI M2M Workshop 2016 demo showcase Errors categorized according to checked validation option by user See running at ETSI M2M Workshop 2016 demo showcase

F-Interop as yet another interop testing tool Miguel ETSI

F-Interop H2020 Project www.f-interop.eu 1 November 2015 – 31 October 2018 develop and provide online interoperability and performance test tools to support emerging technologies from research to standardization and market launch 9 partners

Various IoT Testing Configurations Tested Device  F-Interop test server Deported test with downloaded resource Remote interop with 2 participants Interop against testbed Local interop Remote interop with N participants Remote interop with N participants and testbeds

Online IoT Testing Framework

Targeted Standards Initially standards of the IoT realm Take, as a starting point, the ETSI plugtests specifications and build an architecture that allows those to be done remotely CoAP oneM2M 6TiSCH etc.