Utilizing a Compositional System Knowledge Framework for Ontology Evaluation: A Case Study on BioSTORM H.Hlomani, M.G.Gillespie, D.Kotowski, D. A. Stacey School of Computer Science University of Guelph Guelph, Ontario, Canada

Slide 2 of 28 Who Are We? Guelph Ontology Team (GOT) School of Computer Science, University of Guelph Website: Research Foci: Compositional Systems Workflow Planning Ontology Discovery and Reuse Knowledge Engineering and Ontology Development 2011

Slide 3 of 28 Goal of this Presentation This paper is a case study based upon a framework established in the paper: “A Knowledge Identification Framework for the Engineering of Ontologies in System Composition Processes” (IRI 2011) To introduce aspects of an ODCS that needs to be considered when designing ontologies Explain the checklist developed from KIFEO Application of checklist to BioSTORM Knowledge Engineering and Ontology Development 2011

Slide 4 of 28 Ontology Driven Compositional System (ODCS) An Ontology Driven Compositional System is reasons with ontological representations to construct a resultant Knowledge Engineering and Ontology Development 2011 Source Giliepse et. al. (2011)

Slide 5 of 28 ODCS Examples: Semantic Web Services Automatic Composition of Web Services Arpinar et al. (2005) WebService.owl Process.owl Domain.owl Knowledge Engineering and Ontology Development 2011

Slide 6 of 28 ODCS Examples: BioSTORM Agent Composition Automatic composition of syndromic surveillance software agents DataSource.owl SurveillanceMethods.owl SurveillanceEvaluation.owl Knowledge Engineering and Ontology Development 2011

Slide 7 of 28 ODCS Examples: Algorithm Composition Semi-automatic composition of Algorithms Hlomani & Stacey (2009) Algorithm.owl - Timeline.owl Gillespie et al. (2011) StatisticalModelling.owl PopulationModelling.owl Knowledge Engineering and Ontology Development 2011

Slide 8 of 28 Let’s Not Reinvent the Wheel Each system defines there own way to share knowledge. Often this method is unique to each system. However all these systems are trying to accomplish the same thing (even though they may be named different things) – Define Data Architecture – Describe Compositional Units – Define a Workflow Knowledge Engineering and Ontology Development 2011

Slide 9 of 28 Wouldn’t it be Nice Method for understanding what knowledge to capture. To have a basis for evaluating our knowledge bases. Identifying elements not captured but which may be important as the system evolves. Knowledge Engineering and Ontology Development 2011

Slide 10 of 28 Knowledge Identification Framework Purpose Generalize knowledge entities within any type of ODCS Propose collaborative vocabulary Assist with Merging and Mapping between ODCS ontologies Enhance adaptability of future ontologies for ODCS’s 10

Slide 11 of 28 Knowledge Identification Framework Five Categories of Knowledge Compositional Units Workflow Data Architecture Human Actors Physical Resources 11

Slide 12 of 28 Knowledge Identification Framework Internal vs. External Compositional Units Workflow Data Architecture Human Actors Physical Resources 12

Slide 13 of 28 Knowledge Identification Framework Internal vs. External Compositional Units Work-flow Data Architecture Human Actors Physical Resources 13

Slide 14 of 28 Knowledge Identification Framework Syntactic vs Semantic Knowledge Entities Syntactic entities represent actual objects Semantic entities represent the realization of those actual objects Like “Information Realization” ontology design pattern (Gangemi & Prescutti, 2009) 14

Slide 15 of 28 Knowledge Identification Framework Semantic Knowledge Entity Sub-Types Function Data Execution Quality Trust 15

Slide 16 of 28 Knowledge Identification Framework Relationships between Knowledge Categories Syntactic Relationships Semantic Relationships 16

Slide 17 of 28 Relationships between Knowledge Categories Syntactic Relationship Example Algorithm Input Specification has_input Compositional UnitData ArchitectureCompositional UnitData ArchitectureHuman Actor ---- Input Specification Data Source Datum requires sameAs contains Person owns can_use

Slide 18 of 28 Relationships between Knowledge Categories Semantic Relationship Example (Function & Trust) Algorithm Input Specification has_feature Compositional UnitHuman Actor SpaceTime Dimension Person works_in trusts_ using ---- Organizational Role trusts recommends 18

Slide 19 of 28 Let’s Apply This Framework Develop an evaluation tool: Identifies areas of knowledge or relations which may be missing in key systems ontologies. In the paper we focused on using the framework to identify whether or not the ontologies designed are adaptable. We apply our framework to an existing ODCS system: BioSTORM ontologies Knowledge Engineering and Ontology Development 2011

Slide 20 of 28 What is being Evaluated There are different aspects of ontology evaluation (Brank 2005, Vrandecic 2009) – Context – considering the aspects of the ontology in relation to other variables in its environment How well a given aspect satisfies certain criteria – Adaptability – extent to which ontology can be extended without breaking axioms Knowledge Engineering and Ontology Development 2011

Slide 21 of 28 The Tool Evaluation Checklist A tool often used within software quality assurance. Enables the quantitative analysis of ontologies as well as allows for repeated use. Used by many industries to ensure the highest quality of there products. Knowledge Engineering and Ontology Development 2011

Slide 22 of 28 The Tool The sections of our checklist Part A: ODCS & Ontology Overview Part B: ODCS & Categories of Knowledge (Syntax and Semantics) Part C: Internal Relation Ship Part D: Human Actor Relationships Part E: Physical Resource Relationships Part F: Overall Assessment Part G: Extra Space for Comments Knowledge Engineering and Ontology Development 2011

Slide 23 of 28 Methodology: Steps 1.Review ontologies and supporting documentation (and publications if they exist). 2.Understand system-specific domain and the domain-specific application. 3.Run a preliminary overview (Part A of the checklist: ODCS & Ontology Overview). Knowledge Engineering and Ontology Development 2011

Slide 24 of 28 Methodology: Steps cont’d 4.For each category of knowledge that exists with the ontologies, document it (Part B). 5.Consider all possible relationships that could exist between the categories of knowledge (Part C,D, and E). 6.Provide an overall assessment (Part F) utilizing the evaluation within the checklist. 7.Additional comments and important points discovered during the review (Part G). Knowledge Engineering and Ontology Development 2011

Slide 25 of 28 Knowledge Engineering and Ontology Development 2011

Slide 26 of 28 DISCLAMER! It is important to note that the ontologies used in BioSTORM work as intended within that system This evaluation is not meant to comment about the adaptability of the ontologies within the bioSTORM project but the adaptability of the ontologies if they were to be used within another generic ODCS. Knowledge Engineering and Ontology Development 2011

Slide 27 of 28 Results No knowledge of chronological ordering captured within workflow ontologies. The JADE-CLASS is difficult to adapt, as most ODCS would not use this multi-agent system, thus contextually this CU syntax is difficult to adopt. The data source ontology have non- domain-specific descriptions and thus this ontology is highly adaptable and can be used for any type of system. Knowledge Engineering and Ontology Development 2011

Slide 28 of 28 Summary Knowledge Identification Framework assists in: Detailing relationships between the categories of knowledge Both syntactic and semantic Merging and mapping between ODCS ontologies Can be used to develop tools to evaluate ontologies. From our case study we were able to evaluate the ontologies of an existing ODCS and gauge adaptability. Knowledge Engineering and Ontology Development 2011

Thank you Knowledge Engineering and Ontology Development 2011

Slide 30 of 28 Examples of Knowledge Entities Compositional Unit Examples Syntactic:  Algorithm, Web Service, System Library Function, Input/Output Specification Semantic: subType:: Function (i.e. Domain-specific actions) Data aggregation/conversion/plotting/analysis, Statistical model, Aberrancy detection, etc. subType:: Execution subType:: Quality Operating system Average Runtime 30

Slide 31 of 28 Examples of Knowledge Entities Data Architecture Examples Syntactic:  Single Datum, Structured Data, Data Source, Data Set Semantic:  subType: Data Data Context, Data Context Component DataSource Structure, DataSource FileFormat Data Structure (i.e., Matrix, Vector, Variable) Data Type Units of Measure 31

Slide 32 of 28 Examples of Knowledge Entities Human Actor Examples Syntactic:  Person, Organization, Recommendation Semantic: subType: Trust Role (i.e., software developer, domain-expert, novice-user) Recommendation Context Organization Type Organization Governance 32