1. Improvements on the benchmark suites. October 10th 2005 1 © Raúl García-Castro Improvements on the benchmark suites Raúl García-Castro October 10th, 2005 Interoperability Working Days October 10th-11th, 2005

2. Improvements on the benchmark suites. October 10th 2005 2 © Raúl García-Castro Import/Export Confusion in class and property benchmarks (I13-I18, E13-E18): These benchmarks describe classes that are related to each other by properties, and these properties are defined with a domain and range of some metaclass of the classes (as rdfs:Class): Some participants considered that the benchmarks described properties that had as domain and range the classes: Action: To redefine the benchmarks in order to state clearly the domain and the range of the properties. C1C2 property

3. Improvements on the benchmark suites. October 10th 2005 3 © Raúl García-Castro Import/Export Possible redundant benchmarks Should they be removed? Action: Leave them all, as up to now we don’t have any way of ensuring that a tool that passes the first three benchmarks will pass the fourth. I26One property that has as domain a class I28Several properties that have as domain the same class I29One property that has as domain rdfs:Class I30Several properties that have as domain rdfs:Class I31One property that has as range a class I33Several properties that have as range the same class I34One property that has as range rdfs:Class I35Several properties that have as range rdfs:Class I31One property that has as range a class I33Several properties that have as range the same class I36One property that has as range rdfs:Literal I37Several properties that have as range rdfs:Literal E19One datatype property E20Several datatype properties E24One datatype property that has as range “string” E25Several datatype properties that have as range “string”

4. Improvements on the benchmark suites. October 10th 2005 4 © Raúl García-Castro Import/Export New Import Benchmarks One property that has as domain a class and as range another class, with all the classes defined in the ontology One property that has as domain a class and as range several classes, with all the classes defined in the ontology One property that has as domain several classes and as range a class, with all the classes defined in the ontology One property that has as domain several classes and as range several classes, with all the classes defined in the ontology One property that has as domain and range the same class, defined in the ontology One property that has as domain a class and as range rdfs:Literal, with the class defined in the ontology One property that has as domain several classes and as range rdfs:Literal, with all the classes defined in the ontology Export of datatype and object properties In the tools where properties can’t be modelled without their domain and range classes, the different combinations of properties could not be evaluated. Action: Define new benchmarks to evaluate the different combinations of properties with domain and range in all the tools. New Export Benchmarks One datatype property that has as domain a class and as range String, with the class defined in the ontology One datatype property that has as domain several classes and as range String, with all the classes defined in the ontology One object property that has as domain a class and as range another class, with all the classes defined in the ontology One object property that has as domain a class and as range several classes, with all the classes defined in the ontology One object property that has as domain several classes and as range a class, with all the classes defined in the ontology One object property that has as domain several classes and as range several classes, with all the classes defined in the ontology One object property that has as domain and range the same class, defined in the ontology

5. Improvements on the benchmark suites. October 10th 2005 5 © Raúl García-Castro Import/Export Instance benchmarks (I47-I58, E41-E52) In some tools, instance benchmarks can’t be modeled and imported, as the definition of the class must be present in order to define the instances. Action: Redefine the benchmarks to include in the ontologies the definition of classes. Define new benchmarks to evaluate the import and export of instances with undefined classes. I47Just one instance of a class I47One class and one instance of the class

6. Improvements on the benchmark suites. October 10th 2005 6 © Raúl García-Castro Import/Export Instance and property benchmarks (I50-I58, E44-E52) In some tools, instance and property benchmarks can’t be modeled and imported, as the definition of the property must be present in order to define it in the instances. Action: Redefine the benchmarks to include in the ontologies the definition of properties. Define new benchmarks to evaluate the import and export of instances and undefined properties (object and datatype). I50One instance that has a property with another instance of the same class I50One class and a property with domain and range the class, and one instance of the class that has the property with another instance of the same class

7. Improvements on the benchmark suites. October 10th 2005 7 © Raúl García-Castro Import Some RDF/XML syntax benchmarks not executed by some tools (I65, I66, I69-I72) As WebODE and Protégé do not import classes with properties, we had to change the RDF(S) files of some benchmarks in order to check the import of some ontologies that deal with the RDF/XML syntax. The new ontologies use instances with properties instead of classes with properties, and check the same variants of the RDF/XML syntax than the previous ones. Action: Change the ontologies, as the same variants of the syntax are checked and every tool imports correctly instances with properties Literal 1 Literal 2 Literal 3 Literal 4 Literal 1 Literal 2 I65 OLDI65 NEW

8. Improvements on the benchmark suites. October 10th 2005 8 © Raúl García-Castro Import Specialization of the metamodel Define new benchmarks to evaluate the specialization of the RDF(S) metamodel: Action: Right now it is not clear whether current ontology development tools are used (or able) to model specialised RDF(S) metamodels. Also, the number and variety of these benchmarks is undefined. These kind of benchmarks will not be included for now.

9. Improvements on the benchmark suites. October 10th 2005 9 © Raúl García-Castro Import Literals with language attributes Define another benchmark to evaluate the import of literals with language attributes: Import a graph containing literals with xml:lang attributes: class1 clase1 Action: Define the benchmark and include it in the benchmark suite.

10. Improvements on the benchmark suites. October 10th 2005 10 © Raúl García-Castro Import/Export XML Schema datatypes Define new benchmarks to evaluate the export and import of properties with XML Schema datatypes: Import a graph containing a class and a property with domain the class and range the XML Schema datatype “string”, and an instance of the class with the property with a value: g1:class1 rdf:type rdfs:Class. g1:property1 rdf:type rdf:Property. g1:property1 rdfs:domain g1:class1. g1:property1 rdfs:range xsd:string. g1:instance1 rdf:type g1:class1. g1:instance1 g1:property1 “String”^^xsd:string. Import a graph containing a class and several properties with domain the class and range the XML Schema datatypes “boolean”, “float” and “dateTime”, and an instance of the class with the properties with values: g1:class1 rdf:type rdfs:Class. g1:property1 rdf:type rdf:Property. g1:property1 rdfs:domain g1:class1. g1:property1 rdfs:range xsd:boolean. … g1:instance1 rdf:type g1:class1. g1:instance1 g1:property1 “true”^^xsd:boolean. g1:instance1 g1:property2 “10.0”^^xsd:float. g1:instance1 g1:property3 “1999-05-31T13:20:00-05:00”^^xsd:dateTime. Export a graph containing a class and a datatype property with domain the class and range the XML Schema datatype “string”, and an instance of the class with the datatype property with a value. Export a graph containing a class and several datatype properties with domain the class and range the XML Schema datatypes “boolean”, “float” and “dateTime”, and an instance of the class with the properties with values. Action: Define the benchmark and include it in the benchmark suite.

11. Improvements on the benchmark suites. October 10th 2005 11 © Raúl García-Castro Export URI character restrictions Define new benchmarks to evaluate the export of concepts and properties with invalid characters in their names: Concepts and properties whose names start with a character that is not a letter or ‘_’ Export an ontology containing two concepts named “1concept” and “2concept”, each with one dataype property of type String named “-datatypeProperty1” and “-datatypeProperty2” respectively, and an object property between the concepts named “.objectProperty”. Concepts and properties with spaces in their names Export an ontology containing two concepts named “concept 1” and “concept 2”, each with one dataype property of type String named “datatype property 1” and “datatype property 2” respectively, and an object property between the concepts named “object property”. Concepts and properties with URI reserved characters in their names (‘;’, ‘/’, ‘?’, ‘:’, ‘@’, ‘&’, ‘=‘, ‘+’, ‘$’, ‘,’) Export an ontology containing two concepts named “concept/1” and “concept:1”, each with one dataype property of type String named “datatype/property/1” and “datatype=property=2” respectively, and an object property between the concepts named “object$property”. Concepts and properties with XML delimiter characters in their names (“ ”, “#”, “%”, “””) Export an ontology containing two concepts named “concept 1”, each with one dataype property of type String named “datatype#property#1” and “datatype%property%2” respectively, and an object property between the concepts named “object”property”. Action: Define the benchmarks and include them in the benchmark suite.

12. Improvements on the benchmark suites. October 10th 2005 12 © Raúl García-Castro Import/Export More Improvements?

13. Improvements on the benchmark suites. October 10th 2005 13 © Raúl García-Castro Improvements on the benchmark suites Raúl García-Castro October 10th, 2005 Interoperability Working Days October 10th-11th, 2005