1. National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008

2. National Institute of Standards and Technology 2 Outline Naming and Design Rules Quality of Design Tool (QOD) OAGi NDR v9.0 Encoding –Testability –Observations –Demo NDR Authoring/Sharing What’s Next

3. National Institute of Standards and Technology 3 NDR: Naming and Design Rules Rules applied when creating standards for the exchange of information –Any of a series of rules documents since 2003 that restrict XML Schema flexibility –Indicate features which should, may, may not, should not, must, or must not be used.

4. National Institute of Standards and Technology 4 What’s in an NDR? Naming Rules: –Names (or tags) represent information to be exchanged –Subject names to consistent naming practices for clarity and consistency in an XML schema. Design Rules –Address how information is structured Superstructures: namespaces Microstructures: global vs. local elements

5. National Institute of Standards and Technology 5 Benefits of Encoding Rules NDRs written in English –Easy to misinterpret –Schema testing and validation not automated Encoded NDRs –Result in better rules Verifies rule is clear Validates rule is being used as intended –Can be computationally enforced –Cycle time from requirements for a schema to production XML is reduced

6. National Institute of Standards and Technology 6 QOD – Apply rules directly Interface to write Test Requirements (rules) and associate with Test Cases (encoded rules) –Schematron –Java-Expert System Shell (JESS) Automate testing and validation against schema(s) –Displays and stores results from execution against Schematron/JESS Repository to store rules –Easy to reuse –Group rules to create Test Profile

7. 7 QOD – Testing Environment Common Rule Database Schema Compliance Report Schema to Check Rule Input & Test GUI

8. National Institute of Standards and Technology 8 QOD Tool Overview Test Requirements Page Test Case Page Test Profile Page * Results Page (Updated) * Import an NDR profile (New) * Export a QOD test profile (New)

9. National Institute of Standards and Technology 9 OAGi NDR v9.0 Encoding Rule Testability Additional observations Demo

10. National Institute of Standards and Technology 10 Rule Testability Fully Testable –Rule can be encoded without any assumptions Partially Testable –Some assumptions are needed Not Testable –unable to encode the rule

11. National Institute of Standards and Technology 11 Testability of OAGi Rules

12. National Institute of Standards and Technology 12 Not Testable Rules – From Our Perspective QOD Limitations –Related to XML instance Lack of clarity Nondeterministic –Lack of information in the OAGi schemas Documentation or Guidance –Supplementary information

13. National Institute of Standards and Technology 13 Examples: OAGi-76, OAGi-70 - Ambiguous : OAGi-76 : “xsd:notation MUST NOT be used.” –Which notation must not be used? xsd:notation element or xsd:NOTATION type ? - Nondeterministic: OAGi-70 “ Minor version schema MUST incorporate all XML constructs from the immediately preceding major or minor version schema” - No minor version information in OAGi schemas

14. National Institute of Standards and Technology 14 Additional Observations Overlaps –“Fully Overlapped” Generalization vs. Restriction Generic rule may be removed for same results –“Partially Overlapped” Occasionally can combine rules if same type of schema Links (one rule refers to another) –For testing purposes, all referenced rules have to be included A B Rule A Rule B

15. National Institute of Standards and Technology 15 Overlaps between rules Example : OAGi-11 “XML element, attribute and type names constructed from dictionary entry names MUST NOT include periods, spaces, or other separators; or characters not allowed by W3C XML 1.0 for XML names.” OAGi-10 “Element, attribute and type names MUST be drawn from the following set: a – z and A – Z.” OAGi-10 is restriction of OAGi-11 OAGi-11 can be deleted from the NDR without affecting the original intention

16. National Institute of Standards and Technology 16 OAGi Profiles in QOD General Profile –Rules that can be applied to all OAGi schemas BOD Profile –Rules specific to BOD schemas

17. National Institute of Standards and Technology 17 Demo Overview of the QOD main functionalities (Public vs Private) Demo Test Profile : - Rules related to the BOD - Rules related to the usage of annotation and documentation elements Schema to be tested: - OAGi Developer BOD version 9.0 Full Test Results of the execution of OAGi Developer BOD schemas version 9.0, 9.1 and 9.2 against the rules that we have encoded.

18. National Institute of Standards and Technology 18 OAGi Developer BOD Testing : Materials Schemas tested - Developer BOD schemas version 9.0 : 434 files - Developer BOD schemas version 9.1 : 434 files - Developer BOD schemas version 9.2 : 494 files Profiles tested - General Profile : 25 rules - BOD Profile : 5 rules

19. National Institute of Standards and Technology 19 General Profile BOD Profile Total Developer BOD v9.0 errors Developer BOD v9.1 errors Developer BOD v9.2 errors Rule OAGi 90 : 1 error found - GetReceiveItem.xsd Rule OAGi 98 : 6 errors found - ShowCustomerPartyMaster.xsd - ShowPartyMaster.xsd - ShowPayable.xsd - ShowSupplierPartyMaster.xsd - SyncCatalog.xsd - SyncSupplierPartyMaster.xsd Rule OAGi 100 : 6 errors found - GetLocationService.xsd - ShowLocationService.xsd Error details Rule OAGi 98 : 6 errors found - ShowCustomerPartyMaster.xsd - ShowPartyMaster.xsd - ShowPayable.xsd - ShowSupplierPartyMaster.xsd - SyncCatalog.xsd - SyncSupplierPartyMaster.xsd Error details 13 6 0 2 1 0 Rule OAGi 31 : 2 errors found - GetReceiveItem.xsd - GetRouting.xsd Rule OAGi 31 : 1 error found - GetRouting.xsd 15 7 0 OAGi Developer BOD Testing : Results

20. National Institute of Standards and Technology 20 NDR Sharing Environment Facilitate reuse of rules –Different groups have different requirements but some overlap –Group relevant tests to create customized test profile –Upload new tests to share with others

21. National Institute of Standards and Technology 21 NDR Authoring Environment Capability to export/import rules –Edit rules offline –Display in multiple ways HTML, XML, PDF, Spreadsheet, DocBook, etc –Customize NDR to fit your needs –Based on NDR Profile Schema Standard structured vocabulary for describing NDR rules Available: http://ecc02.eccnet.com/ndr/ http://ecc02.eccnet.com/ndr/

22. National Institute of Standards and Technology 22 QOD – Authoring and Sharing Common Rule Database Schema Compliance Report Schema to Check Rule Selection Custom NDRs Sharing GUI Rule Input & Test GUI

23. National Institute of Standards and Technology 23 OAGi NDR Sharing Based on UN/CEFACT 26 totally reusable from UN/CEFACT 59 modifications of UN/CEFACT rules and can be partially reuse 17 rules are unique

24. National Institute of Standards and Technology 24 NDR Profile Schema Purpose of NDR Schema –Standard set of rules for describing NDR rules –Use of open standards when developing NDRs –Standard structured vocabulary for describing NDR rules Benefits of NDR Schema –Enables exchange of NDR profiles between parties. –Facilitates import/export of NDR rules with QOD. –Can be developed using standard-based tools –Is machine (computer) readable –Multiple presentation methods for human visualization of rules HTML/PDF/Excel, etc.

25. National Institute of Standards and Technology 25 NDR Profile Authoring Ways to create NDR Profile XML Exchange Documents conforming to the NDR Profile Schema –Export from QOD –XForms creation –Excel Spreadsheet Template XSLT transform from Excel to NDR Profile –Any Validating XML Editor

26. National Institute of Standards and Technology 26 What’s next Create stand alone system Enhance support for rule sharing by including rule traceability Pursue similar support for developing tests for constraints on instance data Incorporate feedback into our tools!

27. National Institute of Standards and Technology 27 Discussion Points Authoring/Sharing Environment –Do OAGi members see it useful for their needs? Instance testing Stand alone vs. web based system

28. National Institute of Standards and Technology 28 Contacts KC Morris, NIST – kcm@nist.govkcm@nist.gov Simon Frechette, NIST – simon.frechette@nist.govsimon.frechette@nist.gov Josh Lubell, NIST – lubell@cme.nist.govlubell@cme.nist.gov Puja Goyal, NIST –puja.goyal@nist.gov–puja.goyal@nist.gov Salifou Sidi, NIST – salifou.sidi@nist.govsalifou.sidi@nist.gov Betty Harvey, ECC, Inc. -- harvey@eccnet.comharvey@eccnet.com Websites QOD online http://syseng.nist.gov/b2bTestbed/projects/QOD/ Developer’s site for QOD tool and executable rules: http://qod.sourceforge.net/ http://qod.sourceforge.net/ XML Tools available from MSID: http://www.mel.nist.gov/msid/xml_related.htm http://www.mel.nist.gov/msid/xml_related.htm DAS XSI Working Group: https://www.collab.core.gov/CommunityBrowser.aspx?id=2234 https://www.collab.core.gov/CommunityBrowser.aspx?id=2234

29. National Institute of Standards and Technology 29 Welcome page

30. National Institute of Standards and Technology 30 Testing as Guest

31. National Institute of Standards and Technology 31 My account

32. National Institute of Standards and Technology 32 OAGi demo profile

33. National Institute of Standards and Technology 33 TP results

34. National Institute of Standards and Technology 34 OAGi 31 results with schema

35. National Institute of Standards and Technology 35 NDR Authoring Environment Based on XML Schema – NDR ProfileNDR Profile User’s Guide Prototype Authoring and Rendering tools Display NDR Document Edit NDR Profile The Bridge to Testing: QOD Import/ExportQOD Import/Export

36. National Institute of Standards and Technology 36 Mark-Up of Guidance Check the usage of empty elements Fully-Testable Empty elements MUST NOT be used. [OAGi 90] Error : Empty elements MUST NOT be used. [OAGi 90] Error : Empty elements MUST NOT be used. ]]>

37. National Institute of Standards and Technology 37 Demonstration of Editing http://www.eccnet.com:8080/NDRProfile/ndr/

38. National Institute of Standards and Technology 38 Editing (cont.)

39. National Institute of Standards and Technology 39 Demonstration of Rendering: NDR in PDF

40. National Institute of Standards and Technology 40 NDR in HTML with links to Profile Data and QOD

41. National Institute of Standards and Technology 41 Profile Depicted as HTML