1. Proposals on Test Events
Group Name: TST
Source: Hiroyuki Maeomichi – Kei Harada –
Meeting Date:

2. Contents Operation of Test Events Improvement of TS-0013
Preliminary Matches concept
Statement Sheets
Awards/Recognition
Improvement of TS-0013
Test cases specifying ID types to be used.

3. Preliminary matches concept
3-partners-tests are more complex and have more pit-falls than 2-partner-tests .
It is time consuming to wait a problem resolution for the 3rd partners, who is not directly involved with the problem of other 2 partners. In concrete, when partner A and B are troubleshooting, waiting partner C is consuming time.
So the proposal is introducing preliminary matches concept like sports games.
qualified
implementations
2nd round
(3-partners-test)
preliminary matches
(only 2-partners-test)
preliminary matches
(only 2-partners-test)
remaining
implementations

4. Comparison Table     Existing way
Introduction of preliminary matches
qualified implementations
many waiting occasion in 3-partners-test
effective execution of 3-partners-tests.
developing implementations
could introduce trouble, but they also could suffer from other developing implementations
possible to focus their own problems (not disturbed by unrelated problems)
Organizer
Can execute with fixed plans
Need more arrangement.
as oneM2M community
running rate on 3-partners-test ↑
Finding more hidden problems.    

5. Preparation of Statement sheets
Configuration sheets
summarize necessary configurations to connect to participant’s CSE.
Less mistakes than hand writing or talked info.
If put them on WIKI, we could copy and paste it. (less and less mistakes)
Implemented feature sheets
summarize implemented feature,
help reducing unnecessary try and error procedures. (E.g. After analyzing logs for several minutes, “Oops, we don’t implement it.”),
potentially help reducing unimplemented features, which are introduced misunderstanding of the specifications. (E.g. various ID types, which are confirmed to be implemented in the Interop3 again.)

6. Sample: Implemented feature sheets
Resource ID types:
receiving capability (note: ALL must be checked to be compliant.)
□ Absolute-Resource-ID
□ SP-relative-Resource-ID
□ Structured-CSE-relative-Resource-ID
□ Unstructured-CSE-relative-Resource-ID
sending capability
Protocol/ Serialization/Version
HTTP/XML □R1 □R2
HTTP/JSON □R1 □R2
HTTP/CBOR □R2
MQTT/XML □R1 □R2.....
Supported Resources
□CSEBase, □RemoteCSE, □Application Entity, □Container, □ContentInstance, □Subscription, □AccessControlPolicy, □Group, □Node, □PollingChannel ...
Supported Test Cases
□TD_M2M_NH_01
□TD_M2M_NH_02
... ・
This may remind participants to get ready before event.
With this information, we could avoid uncommunicable combinations with uncomplete implementation on ID type support.
Difference from current Checklist:
■HTTP ■ MQTT ■ XML ■ JSON
could support all variation or just supporting 2 combinations like HTTP/XML and MQTT/JSON.
or checking unsupported test cases

7. Awards / Recognition
It is important to give motivation to participants for participating more, bringing more sophisticated implementations, executing tests more effective manners. As the result of them, primary purpose (improvement oneM2M specs) will be achieved in effectively.
It is difficult to know the best implementation in interop event.
Proposal:
Giving award to sessions. (2 or 3 partners who got most OKs)
brings cooperation atmosphere to participants, instead of competition. (inspired by Jogo concept of Capoeira (Brazilian Martial arts))
Providing competition categories with protocol bindings and serializations could motivate participants to bring new/minor combinations. As a result we could debug specs wildly.
Proposal 2:Awarding the 2nd round implementation of preliminary match concept.

8. TS-0013 Improvement (specifying ID)
Current TS-0013 doesn’t specify ID types to be used. They are allowed to participants’ preference.
We propose to add test cases that clearly specify ID type to be used for very small number of resources (CSEBase or AE).
It could be enough just retrieving CSEBase with each ID types. (Absolute-Resource-ID, SP-relative-Resource-ID, Structured-CSE-relative-Resource-ID, and Unstructured-CSE-relative-Resource-ID
Question: Do you think useful?

9. Summary Operation of Test Events
Preliminary Matches concept: Do 3-partners-tests with qualified implementations.
Statement Sheets could help effective event.
Configuration Test : print or put on WIKI about configurations
Implemented feature sheet: avoid “Oops, we don’t implement it”
Awards/Recognition
Best Session Award
qualified implementations on preliminary matches
Improvement of TS-0013
Test cases specifying ID types to be used