1. Wrap up – Day 1. October 10th 2005 1 © Raúl García-Castro Wrap up – Day 1 Raúl García-Castro October 10th, 2005 Interoperability Working Days October 10th-11th, 2005

2. Wrap up – Day 1. October 10th 2005 2 © Raúl García-Castro Index Comments Export experiments Import experiments

3. Wrap up – Day 1. October 10th 2005 3 © Raúl García-Castro When do benchmarks fail? Two options: A benchmark fails when the tool does not produce its expected result. A benchmark fails when there is any problem while executing it or the result does not correspond to the ontology decribed in the benchmark. IdDescriptionWebODE instatiation Expected ontology Knowledge Lost Result E03Export one class instance of another class that is instance of a third one ---- IdDescriptionWebODE instatiation Expected ontology Knowledge Lost Result E03Export one class instance of another class that is instance of a third one ---FAIL IdDescriptionExpected ontology Knowledge Lost Result I03Import one class instance of another class that is instance of a third one Only classes rdfs:type properties OK IdDescriptionExpected ontology Knowledge Lost Result I03Import one class instance of another class that is instance of a third one Only classes rdfs:type properties FAIL

4. Wrap up – Day 1. October 10th 2005 4 © Raúl García-Castro Benchmark results in two dimensions Execution (OK/FAIL/N.E.*). A benchmark fails when there is any problem while executing it or the tool does not produce its expected result. Modeling (Y/N). Modeling is “YES” when the tool is able to model the ontology components described in the benchmark. * In the export benchmarks, if the tool cannot model the ontology (Modeling=NO), the benchmark can’t be executed and the execution result is NON EXECUTED (N.E.).

5. Wrap up – Day 1. October 10th 2005 5 © Raúl García-Castro Export benchmark results Modeling = YES. The tool is able to model the ontology components described in the benchmark. –Execution = OK. There is no problem while executing the benchmark and the tool produces its expected result (importing the ontology components). –Execution = FAIL. There is any problem while executing the benchmark or the tool does not produce its expected result (not importing the ontology components). Modeling = NO. The tool is not able to model the ontology components described in the benchmark. –Execution = N.E. The tool is unable to model the ontology and, therefore, to execute the benchmark.

6. Wrap up – Day 1. October 10th 2005 6 © Raúl García-Castro Import benchmark results Modeling = YES. The tool is able to model the ontology components described in the benchmark. –Execution = OK. There is no problem while executing the benchmark and the tool produces its expected result (importing the ontology components). –Execution = FAIL. There is any problem while executing the benchmark or the tool does not produce its expected result (not importing the ontology components). Modeling = NO. The tool is not able to model the ontology components described in the benchmark. –Execution = OK. There is no problem while executing the benchmark and the tool produces its expected result. –Execution = FAIL. There is any problem while executing the benchmark or the tool does not produce its expected result.

7. Wrap up – Day 1. October 10th 2005 7 © Raúl García-Castro Index Comments Export experiments Import experiments

8. Wrap up – Day 1. October 10th 2005 8 © Raúl García-Castro Export results IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Class benchmarks E01One classOKY Y Y Y Y E02Several classesOKY Y Y Y Y Metaclass benchmarks E03One class instance of another class that is instance of a third one -NOKY Y Y Y E04One class instance of several classes-NFAILYOKY Y Y E05Several classes instance of the same class-NOKY Y Y Y E06One class instance of another class and viceversa -NOKY Y Y Y E07One class that is instance of itself-NOKY Y Y Y Subclass benchmarks E08One class subclass of another class that is subclass of a third one OKY Y Y Y Y E09One class subclass of several classesOKY Y Y Y Y E10Several classes subclass of the same classOKY Y Y Y Y E11One class subclass of another class and viceversa -N-N-N-NOKY E12One class that is subclass of itself-N-N-N-NOKY

9. Wrap up – Day 1. October 10th 2005 9 © Raúl García-Castro Export results 2 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Class and object property benchmarks E13One class that has an object property with another -NOKY Y Y Y E14One class that has the same object property with several classes -NOKY Y Y Y E15Several classes that have the same object property with the same class -NOKY Y Y Y E16One class that has an object property with itself -NOKY Y Y Y Class and datatype property benchmarks E17One class that has a datatype property with a literal -NOKY-N-N Y E18One class that has the same datatype property with several literals -NOKY-N-N Y

10. Wrap up – Day 1. October 10th 2005 10 © Raúl García-Castro Export results 3 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Datatype property benchmarks E19One datatype property-NOKY Y-N Y E20Several datatype properties-NOKY Y-N Y E21One datatype property that has as domain a class -NOKY Y-N Y E22One datatype property that has as domain several classes -NFAILYOKY-N Y E23Several datatype properties that have as domain the same class -NOKY Y-N Y E24One datatype property that has as range “string” -NOKY Y-N Y E25Several datatype properties that have as range “string” -NOKY Y-N Y E26One datatype property that has as domain a class and as range “string” OKY Y Y Y Y E27One datatype property that has as domain several classes and as range “string” -NFAILYOKY Y Y

11. Wrap up – Day 1. October 10th 2005 11 © Raúl García-Castro Export results 4 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Object property benchmarks E28One object property-NOKY Y-N Y E29Several object properties-NOKY Y-N Y E30One object property that has as domain a class-NOKY Y-N Y E31One object property that has as domain several classes -NFAILYOKY-N Y E32Several object properties that have as domain the same class -NOKY Y-N Y E33One object property that has as range a class-NOKY Y-N Y E34One object property that has as range several classes-NOKY Y-N Y E35Several object properties that have as range the same class -NOKY Y-N Y E36One object property that has as domain a class and as range another class OKY Y Y Y Y E37One object property that has as domain a class and as range several classes -NOKY Y-N Y E38One object property that has as domain several classes and as range a class -NFAILYOKY-N Y E39One object property that has as domain several classes and as range several classes -NFAILYOKY-N Y E40One object property that has as domain and as range the same class OKY Y Y Y Y

12. Wrap up – Day 1. October 10th 2005 12 © Raúl García-Castro Export results 5 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Instance benchmarks E41One instance of a classOKY Y Y Y Y E42One instance of several classes-NFAILYOKY Y Y E43Several instances of a classOKY Y Y Y Y Instance and object property benchmarks E44One instance that has an object property with another instance of the same class OKY Y Y Y Y E45One instance that has the same object property with several instances of the same class OKY Y Y Y Y E46Several instances that have the same object property with the same instance of the same class OKY Y Y Y Y E47One instance that has an object property with himselfOKY Y Y Y Y E48One instance that has an object property with another instance of another class OKY Y Y Y Y E49One instance that has the same object property with several instances of another class OKY Y Y Y Y E50Several instances that have the same object property with the same instance of another class OKY Y Y Y Y Instance and datatype property benchmarks E51One instance that has a datatype property with a literalOKY Y Y Y Y E52One instance that has the same datatype property with several literals OKY Y Y Y Y

13. Wrap up – Day 1. October 10th 2005 13 © Raúl García-Castro Index Comments Export experiments Import experiments

14. Wrap up – Day 1. October 10th 2005 14 © Raúl García-Castro Import results IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Class benchmarks I01One classOKY Y Y Y Y I02Several classesOKY Y Y Y Y Metaclass benchmarks I03One class instance of another class that is instance of a third one OKN N Y Y Y I04One class instance of several classesOKN N Y Y Y I05Several classes instance of the same classOKN N Y Y Y I06One class instance of another class and viceversa OKN N Y Y Y I07One class that is instance of itselfOKN N Y Y Y Subclass benchmarks I08One class subclass of another class that is subclass of a third one OKY Y Y Y Y I09One class subclass of several classesOKY Y Y Y Y I10Several classes subclass of the same classOKY Y Y Y Y I11One class subclass of another class and viceversa OKNFAILNOKN NFAILN I12One class that is subclass of itselfOKNFAILNOKN N Y

15. Wrap up – Day 1. October 10th 2005 15 © Raúl García-Castro Import results 2 IdDescriptionWebODEProtégéKAONOntoSt.Corese EMEMEMEMEM Class and property benchmarks I13One class that has a property with another classOKN NFAILYOKY Y I14One class that has the same property with several classes OKN NFAILYOKY Y I15Several classes that have the same property with the same class OKN NFAILYOKY Y I16One class that has a property with itselfOKN NFAILYOKY Y I17One class that has a property with a literalOKN NFAILYOKY Y I18One class that has the same property with several literals OKN NFAILYOKY Y Property benchmarks I19One propertyOKY Y Y N Y I20Several propertiesOKY Y Y N Y Subproperty benchmarks I21One property subproperty of another property that is subproperty of a third one OKN Y Y N Y I22One property that is subproperty of several propertiesOKN Y Y N Y I23Several properties that are subproperty of the same property OKN Y Y N Y I24One property that is subproperty of another property and viceversa OKNFAILNOKY NFAILN I25One property that is subproperty of himselfOKNFAILNOKY N Y

16. Wrap up – Day 1. October 10th 2005 16 © Raúl García-Castro Import results 3 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Property with domain and range benchmarks I26One property that has as domain a classOKY Y YFAILOKY I27One property that has as domain several classes FAILY NOKYFAILOKY I28Several properties that have as domain the same class OKY Y YFAILOKY I29One property that has as domain rdfs:ClassOKY N YFAILOKY I30Several properties that have as domain rdfs:Class OKY N YFAILOKY I31One property that has as range a classOKY Y YFAILOKY I32One property that has as range several classesOKY N YFAILOKY I33Several properties that have as range the same class OKY Y YFAILOKY I34One property that has as range rdfs:ClassOKY Y YFAILOKY I35Several properties that have as range rdfs:Class OKY Y YFAILOKY I36One property that has as range rdfs:LiteralFAILYOKY YFAILOKY I37Several properties that have as range rdfs:Literal FAILYOKY YFAILOKY

17. Wrap up – Day 1. October 10th 2005 17 © Raúl García-Castro Import results 4 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Property with domain and range benchmarks (2) I38One property that has as domain a class and as range another class OKY Y Y Y Y I39One property that has as domain a class and as range several classes OKY N Y Y Y I40Several properties that have as domain several classes and as range a class OKYFAILNOKYFAILOKY I41Several properties that have as domain several classes and as range several classes OKYFAILNOKYFAILOKY I42Several properties that have as domain and range the same class OKY Y Y Y Y I43One property that has as domain a class and as range rdfs:Literal OKY Y Y Y Y I44One property that has as domain several classes and as range rdfs:Literal OKYFAILNOKYFAILOKY I45One property that has as domain and range rdfs:Class OKY N YFAILOKY I46One property that has as domain rdfs:Class and as range rdfs:Literal FAILYOKN YFAILOKY

18. Wrap up – Day 1. October 10th 2005 18 © Raúl García-Castro Import results 5 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Instance benchmarks I47One instance of a classOKY Y Y Y Y I48One instance of several classesOKN N Y Y Y I49Several instances of a classOKY Y Y Y Y Instance and property benchmarks I50One instance that has a property with another instance of the same class OKN NFAILYOKY Y I51One instance that has the same property with several instances of the same class OKN NFAILYOKY Y I52Several instances that have the same property with the same instance of the same class OKN NFAILYOKY Y I53One instance that has a property with himselfOKN NFAILYOKY Y I54One instance that has a property with another instance of another class OKN NFAILYOKY Y I55One instance that has the same property with several instances of another class OKN NFAILYOKY Y I56Several instances that have the same property with the same instance of another class OKN NFAILYOKY Y I57One instance that has a property with a literalOKY NFAILYOKY Y I58One instance that has the same property with several literals OKY NFAILYOKY Y

19. Wrap up – Day 1. October 10th 2005 19 © Raúl García-Castro Import results 6 IdDescriptionWebODEProtégéKAONOntoStudioCorese EMEMEMEMEM Syntax and abbreviation benchmarks I59Several classes with absolute URI referencesOKY Y Y Y Y I61Several classes with URI references relative to a base URI OKY Y Y Y Y I61Several classes with URI references transformed from rdf:ID attribute values OKY Y Y Y Y I62Several classes with URI references relative to an ENTITY declaration OKY Y Y Y Y I63Several classes with empty nodesOKYFAILYOKY Y Y I64Several classes with empty nodes shortenedOKY Y Y Y Y I65Several classes with multiple propertiesOKY YFAILYOKY Y I66Several classes with multiple properties shortenedOKY YFAILYOKY Y I67Several classes with typed nodesOKY Y Y Y Y I68Several classes with typed nodes shortenedOKY Y Y Y Y I69Several classes with properties with string literalsOKY YFAILYOKY Y I70Several classes with properties with string literals as XML attributes OKY YFAILYOKY Y I71Several classes with blank nodes with identifierOKYFAILY Y YOKY I72Several classes with blank nodes shortenedOKY YFAILY YOKY

20. Wrap up – Day 1. October 10th 2005 20 © Raúl García-Castro Wrap up – Day 1 Raúl García-Castro October 10th, 2005 Interoperability Working Days October 10th-11th, 2005