1. © 2006 Open Grid Forum GridRPC Interoperability Test Response to comments Yusuke Tanimura

2. © 2006 Open Grid Forum 2 Comment #1 Response: Use terminology test purpose and test case in this document. The definition of test case follows http://en.wikipedia.org/wiki/Test_case. http://en.wikipedia.org/wiki/Test_case Use test suites instead of test code. GridRPC is giving high level descriptions essentially of what to test. PTCC calls that test purposes. What GridRPC calls "test code" comes closer to our idea of a test case. Something which is executable.. a test script if you will.

3. © 2006 Open Grid Forum 3 Comment #2 Response: Describe WHAT in 1. Introduction Describe HOW in 2. Proposed process In the same way we feel it is beneficial to separate WHAT is to be tested (=purpose) and HOW to test (= description).

4. © 2006 Open Grid Forum 4 Comment #3 Both documents position themselves as being a basis for interoperability testing. From PTCC point of view both are really specifying conformance tests. I am not sure why the term conformance is avoided. Or maybe there is just a mix-up of words here which is (achievement of) interoperability versus interoperability testing. GridRPC states for example that "As such, the requirement is only to test client interoperability: that multiple implementations of a specification provide consistent results to a suite of tests". In my opinion this just falls short of saying that these implementations conform to the specification which has been selected to be the basis of these so called interoperability tests, i.e., GFD-R.52. In both specs it seems to me that the system under test consists of one entity, i.e., the server, being tested in isolation. I could be wrong. In ETSI (formal) interoperability testing is defined as operating different implementations of different entities against each other, e.g., a NIFT client implementation against a GridSolve server. Therefore an interoperability test would describe how to stimulate and observe client as well as server implementations not the interface betweeen them.

5. © 2006 Open Grid Forum 5 Comment #3 (contd.) Response: There is a statement in GFD-R.52, "Interoperability between Implementations. Since this document focuses on the GridRPC API, it says nothing about the protocols used to communicate between clients, servers, and registries. Hence, it does not address interoperability." In our document, the interoperability test is to prove the specification is mature and useful enough to share the client program among GridRPC-compliant systems. In our document, the interoperability test is equal to a conformance test.

6. © 2006 Open Grid Forum 6 Comment #4 Response: Only C API is defined in GFD-R.52. GridRPC more at what I would consider software module testing and ByteIO more as protocol/Unit testing. From our point of view that is fine. However I wonder in the case of GridRPC why this specification is limited to C implementations only. It seems there is a lot of Grid middle ware based on Java out there. Should this test specification not be more implementation agnostic?

7. © 2006 Open Grid Forum 7 Comment #5 Response: Add more quotation of GFD-R.52, specially if the behavior is different among each implementation. Both documents are lacking references in the test purpose/description to the requirement(s) they pertain to in the specification, e.g., document clause or quotation of specification text. Such reference are very important to trace back test results to the specification or to ensure the test itself is a valid one.

8. © 2006 Open Grid Forum 8 Comment #6 Response: Add more quotation of GFD-R.52. Already have While these undefined behaviors are a minor problem, they should be clearly defined in the future version of the specification according to the use cases. in Appendix A. Another point is clearer identification and separation of mandatory versus optional features of the specification / implementation under test. That means entire tests or test purposes should be optional not only some of multiple validation criteria within the same test purpose. This categorization is also helpful for identifying the minimum set of tests that a implementation must pass before being compliant or shall we say "considered interoperable". GridRPC has some discussion of optional responses based on about observed differences with various applications but it does not clarify what the response should be, i.e., what the specification actually states.

9. © 2006 Open Grid Forum 9

10. 10 OGF IPR Policies Apply I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy. Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to: the OGF plenary session, any OGF working group or portion thereof, the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, the ADCOM, or any member thereof on behalf of the ADCOM, any OGF mailing list, including any group list, or any other list functioning under OGF auspices, the OGF Editor or the document authoring and review process Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions. Excerpt from Appendix B of GFD-C.1: Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non- discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification. OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.

11. © 2006 Open Grid Forum 11 Full Copyright Notice Copyright (C) Open Grid Forum (applicable years). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. The limited permissions granted above are perpetual and will not be revoked by the OGF or its successors or assignees.