1. oneM2M Testing and Dev. Support
TST R01-testing_and_dev_support
oneM2M Testing and Dev. Support
Group Name: TST
Source: JaeSeung Song (TST Chair), KETI,
Meeting Date:
Agenda Item: TBD

2. Contributors Testing oneM2M with POSTMAN (Lionel Florit, Cisco)
Conformance Testing Tool – “oneM2M Tester” (Martin Miao, KETI, EGM, Ericsson, Sejong Univ.)
oneM2M Testing with TTWorkBench (Richard, Spirent)
Tools for oneM2M developers and SIs (Dale, InterDigital)
TTA verified testing tool (Keebum, TTA)
Web-based AE testing tool (JaeSeung, KETI & Sejong Univ.)
ontology validator (Franck, EGM)
F-Interop as a oneM2M interop testing tool (Miguel, ETSI)

3. Testing oneM2M with POSTMAN
Lionel Florit

4. Why Postman? It is free, free and it is free 
A Google Chrome App, easy to install on any system as
Easy to collaborate, exchange and share test libraries
Use it to aid in the CSE development process
The test libraries provide real queries, ready to edit/use/augment
Limitations
HTTP only
Does not do the server side: good to test a CSE but can’t emulate one
Not a compliance tool: does not check if the response is compliant! Only sends compliant requests.

5. Test your CSE in 5 minutes
Install Postman:
Import the oneM2M TS13 test collection (JSON) using this link
Test!

6. Martin MIAO (novamartin@keti.re.kr) Jaeho KIM (jhkim@keti.re.kr )
oneM2MTester
Martin MIAO
Jaeho KIM )

7. Conformance Testing Tool oneM2M Tester
In parallel with the test code (ATS), a test tool development is also ongoing
System adapter and codecs
User Guide
Based on the open source TTCN-3 tool
Evaluation
HTTP binding and XML serialization at the 2nd oneM2M plugtest event
HTTP binding and JSON serialization at the 3rd oneM2M plugtest event
Demos and dissemination
Presentations and live demos at international software testing conferences and events (HUSTEF, UCAAT, ECS 2016) etc.
Tutorials via the oneM2M webinar series
oneM2M Tester will be open sourced
Open Source

8. oneM2M Tester Capture
oneM2MTester Testcase Executor capture

9. oneM2M Tester Video Demo

10. Testing oneM2M with TTworkbench and TTCN-3
October 2016

11. Ttworkbench and TTCN-3 for oneM2M
oneM2M Functional Architecture – TTworkbench Test Access
TTworkbench TTCN-3 Conformance test tool
Sensors, Actuators, Gateways
Field
Domain
Infrastructure Domain #1
Application (AE)
Mca
Application (AE)
Mca
Mca
Mca
Infrastructure Domain
Provider #2
Common Service (CSE)
Common Service (CSE)
Mcc
Mcc
Mcc
Mcn
Mcn
Network (NSE)
Network (NSE)
Optionally available via non-standard test API
Standard Test API

12. oneM2M Testing with TTworkbench
TTworkbench uses existing oneM2M TTCN-3 Conformance Test Cases for the CSE
Registration
Data management and repository
Location
Group management
Future …..
Future AE Conformance ….
Extendable TTCN-3 Framework
Possibility to reuse existing TTCN-3 code to design custom test scenarios
Possibility to use existing Test Adaptation Plugins to access Sensors, Actuators or other entities
Possibility to integrate custom Test Access as required
Also possibilities to use for Interoperability testing if required

13. TTworkbench Support
Latest versions of both Release 1 & Release 2 of the oneM2M standard
Bindings
CoAP
HTTP
MQTT
WebSockets (added by oneM2M v2)
Serialization
XML
JSON
CBOR (added by oneM2M v2)

14. Dale Seed InterDigital
Dev. Support
Dale Seed
InterDigital

15. Tools for Developers and SIs
InterDigital has several useful tools for oneM2M developers
oneM2M Wireshark Dissector
oneM2M Primitive Exerciser
oneM2M curl scripts
oneM2M test tools

16. Publically Available IN-CSE, MN-CSE and AEs
IN-CSE is hosted in the Cloud (No SW Install Required!)
Ideal for folks that want to quickly test without installing a CSE locally
Submit a request for AE-ID or CSE-ID and PSK Credentials via InterDigital’s Website
Register your AE or CSE to IN-CSE via PSK-based secured Mca or Mcc (HTTPs or CoAPs)
MN-CSE is available for download from InterDigital’s Website
Ideal for folks that want to run locally in their own environments
Simply download, configure and run MN-CSE
Register your AE or CSE to MN-CSE via Mca or Mcc (HTTPs or CoAPs)
InterDigital has sample AEs available for download from InterDigital’s Website
These AEs are based on InterDigital’s oneM2M programmatic API library
Simply download and try them out
These AEs can be run with InterDigital’s IN-CSE, MN-CSE or your own CSE
Go to InterDigital’s Developer Portal and sign up for a oneMPOWER account
IN-CSE Contact Information:
FQDN: cse.onem2mpub.wot.io
IP Address:
HTTPs Port: 9013
CoAPs Port: 9012

17. oneM2M Message Dissector
View and decode oneM2M protocol in Wireshark!
Dissector can be downloaded from
Currently Supports
HTTP and CoAP oneM2M bindings
(MQTT under development)
Selectively capture or filter on oneM2M traffic
Dissects and displays oneM2M request and response parameters
Dissector is implemented as a Wireshark plug-in that can be easily installed
Dissects and displays oneM2M content

18. oneM2M Primitive Exerciser
Tool is hosted in the Cloud
No SW Install Required!
URL –
Username: oneM2M
Password: oneM2MForEveryone
Currently supports
HTTP and CoAP oneM2M bindings
(MQTT under development)
The oneM2M Primitive Exerciser is a handy tool to generate oneM2M requests (e.g. to send to a CSE)
Think of it as Postman on steroids 
Tool provides a history of issued requests that can be re-issued
(similar to Postman)
The tool is oneM2M aware and provides a nice GUI for developers to build oneM2M requests and decode responses
Tool supports a sequence generator that can be used to create sequences of requests that can be issued sequentially
The tool also allows developers to see protocol binding and serialization level details for requests it generates and responses it receives
Tool supports saving requests to file that can be re-loaded into the tool and re-issued

19. (Yes, you know who you are )
oneM2M curl scripts
####################################################
# Create AE resource using HTTP binding and JSON serialization
curl -v -X POST -H 'X-M2M-RI: xyz2' -H 'X-M2M-Origin: CAE001' -H 'Content-Type: application/vnd.onem2m-res+json; ty=2' -H 'Accept: application/vnd.onem2m-res+json'
> POST /oneMPOWER-IN-CSE HTTP/1.1
> User-Agent: curl/7.32.0
> Host: :9011
> X-M2M-RI: xyz2
> X-M2M-Origin: CAE001
> Content-Type: application/vnd.onem2m-res+json; ty=2
> Accept: application/vnd.onem2m-res+json
> Content-Length: 96
> {
"m2m:ae": {
"rn": "TestAE_JSON",
"api": "myAppId",
"rr": "true" }
< HTTP/ Created
< Content-Type: application/vnd.onem2m-res+json
< X-M2M-RI: xyz2
< X-M2M-RSC: 2001
< Content-Location: /ID-CSE-01/CAE001
< Content-Length: 154
<
"m2m:ae":{
"aei":"CAE001",
"ct":" T103559",
"et":" T215559",
"lt":" T103559",
"pi":"ID-CSE-01",
"ri":"CAE001",
"rn":"TestAE_JSON",
"ty":2
curl is a popular open source command line tool used for issuing HTTP client requests. It ships with most operating systems
curl is used by many hard core developers preferring a command line interface
(Yes, you know who you are )
InterDigital maintains a library of curl scripts that provide working examples of different types oneM2M requests
Curl scripts can be downloaded from:

20. oneM2M test tools
InterDigital has developed several test tools and test suites for the purposes of interop and conformance testing
These tools and test suites have been used to test InterDigital’s oneMPOWER solution as well as oneM2M solutions from other parties.
At this time, InterDigital is not making these tools publically available. For more information regarding these tools please contact us

21. TTA Testing Tool for oneM2M Device (AND-AE)
Keebum Kim, TTA,

22. TTA Testing Tool for oneM2M
TTA started “oneM2M TTA Verified” Program in April 2016.
To run this program, TTA prepared two kind of Testing Tool.
Testing Tool for originators (e.g. ADN, ASN, MN)
Testing Tool for receivers (e.g. IN, MN)
For receivers, many references are existing. (like postman), but it was hard to find for originators.
So, TTA decide to develop oneM2M Device (originator) Testing Tool.
Further slides are for introducing TTA oneM2M Device Testing Tool.

23. Testing Scenario Test Scenario Flow ③ ⑦ ① ④ ② ⑤ ⑥ oneM2M Platform
oneM2M Based Device
Tester
Control
CRUD RESTful
Request
Result
View
Test-Case
Configuration &
Launch ③
Test Scenario Flow ⑦ ① ④
Verifying ②
Real-Time
Ajax Request
oneM2M
Platform
Mongo DB
Tester
(WAS) ⑤
Logging
Resource Data
Response ⑥

24. Testing Tool Structure
ADN
ASN MN
AE(1…N)
CSE(1)
AE(0…N) IN
MQTT
Broker
CoAP
Server
HTTP
Mongo DB
oneM2M
Testing Tool App
User
Common
Platform
Testing Tool
Create raw packet file
real-time
Robot Framework Rebot Result View
MQTT Client
HTTP Log
MQTT Log
CoAP Log
WAS
Packet sniffer

25. Testing Tool Application
Test Application
Test Environments
Test Result
Test Launcher
Device Config
URI Config
Test Case
Check-Up List
Initial Config
Test Result Display
Packet Sniffing
Dump Output
HTML Output
Test Result Statistics
Status Display
Test Information
Log Display

26. Select Binding Protocol
Testing Tool
Test Environment Configuration Step.
Select Binding Protocol
(HTTP, CoAP, MQTT)
Select Node Type
(ADN, ASN, MN)

27. Testing Tool (Cont.) Test Running Step. Summary of all Test Results
Detail information
of each test
History logs during testing

28. Web-based oneM2M Self-Testing Tool for Dev.
JaeSeung Song, Sejong University & KETI,

29. python oneM2M pcap library
(as a testing tool)
View an oneM2M packet via the webpage
Interface based on the webpage
Information of a selected packet
packet
The process to select a packet that you want 3
Select the pcap file in my computer 1
Select the pcap file …
Selected file : ??.pcap
reset
Select a packet in pcap file 2
Select a packet
A packet
B packet
C packet … OK
reset

30. python oneM2M pcap library
(as a testing tool)
Compare the two oneM2M packets via the webpage
Interface based on the webpage
Information of two selected packets
packet
The process to select two packets that you want
Select the first pcap file in my computer 1
Fisrt packet
secnod packet
Select the pcap file …
Selected file : ??.pcap
reset
Select a packet in pcap file 2
Select a packet
A packet
B packet … OK
reset
Select the second pcap file in my computer 3
Select the pcap file …
Selected file : ??.pcap 4
reset
Two pcap files does not matter the same or different
Select a packet
Select a packet in pcap file
A packet
B packet … OK
reset

31. oneM2M AE code oneM2M Testing code
oneM2M AE code will send the specific user request to TestSystem (In this document we will use the AE registration case)
oneM2M AE Code is constructed for processing the specific user request such as CRUDN
Testable AE is uploaded via web interface
Cloud AE TE
Android App (AE)
Send
oneM2M
AE code
oneM2M
AE code
Testcase
(TTCN-3)
Build
Upload
oneM2M
Testing code
Testable AE
oneM2M
Testing code
Triggering
User has to implement the oneM2M Testing code. It is used to handle the Test System(TE) request (Test case)
oneM2M Testing code is implemented for processing the test case receive the test case Value.
Testcase will check the oneM2M Payload for user specific command such as AE registration and issue the report about testing result

32. Cloud AE TE
Send
oneM2M
AE code
Testcase
(TTCN-3)
oneM2M
Testing code
Triggering
After User upload the testable AE and select the specific test cases, Test System processes it.

33. Insert “testing code”
Start “testing server”

34. Check a received primitive
Execute the corresponding testcase.
For example, execute Registration

35. Semantic ontology validation.
Mengxuan Zhao, Franck Le Gall, Easy Global Market

36. Validation in semantic data life cycle
From data production to consumption
Semantic data production:
Ontology validation
Annotation validation
Semantic data consumption
Reasoning rule validation
Query validation
Our focus

37. Basic aspects to be validated on semantic information
Lexical validation
Ill-formed RDF serialization format (XML, JSON, etc.)
Syntactic validation
Untyped resources and literal
Ill-formed URIs and language tags on literals
Problematic prefix namespaces
Unknown properties and classes
Semantic validation
Inheritance relationships for classes and properties
Consistency of classes and individuals

38. Self-ontology selection
Popular reference ontologies are integrated in the tool
Format
Filename extension
N-Quads
.nq
N-Triples
.nt N3
.n3
Turtle
.ttl
TriG
.trig
TriX
.trix
RDF/JSON
.rj
JSON-LD
.jsonld
BinaryRDF
.brf
RDF/XML
.rdf
OWL
.owl

39. Results See running at ETSI M2M Workshop 2016 demo showcase
Errors categorized according to checked validation option by user
See running at ETSI M2M Workshop 2016 demo showcase

40. F-Interop as yet another interop testing tool
Miguel
ETSI

41. F-Interop H2020 Project www.f-interop.eu
1 November 2015 – 31 October 2018
develop and provide online interoperability and performance test tools to support emerging technologies from research to standardization and market launch
9 partners

42. Various IoT Testing Configurations
Tested Device  F-Interop test server
Deported test with downloaded resource
Remote interop with 2 participants
Interop against testbed
Local interop
Remote interop with N participants
Remote interop with N participants and testbeds

43. Online IoT Testing Framework

44. Targeted Standards Initially standards of the IoT realm
Take, as a starting point, the ETSI plugtests specifications and build an architecture that allows those to be done remotely
CoAP
oneM2M
6TiSCH
etc.