December 2, 2013 Thessaloniki, Greece GNORASI WORKSHOP Charalampos Doulaverakis CERTH/ITI Knowledge and processing algorithms for remote sensing data Reasoning and semantic interpretation of visual data in GNORASI

Goals of semantic interpretation  Reasoning and representation of knowledge for semantic-enabled image analysis  Expert knowledge and visual information processing data are represented through ontologies  Development of a reasoning process for the knowledge assisted interpretation of images Reasoning methods Fuzzy inference support 2

 Definition on ontology  Representation and querying  Land use/Land cover ontologies Ontologies and reasoning 3

Ontology definition 4  An ontology defines a set of representational primitives with which to model a domain of knowledge or discourse (Gruber, 1995)  Classes: which represent a set of objects  Properties: which express attributes and relations between classes and objects  Constraints: for expressing logical consistency  Individuals: atoms (objects) which are members of a class  Abstraction level of data models  Analogous to hierarchical and relational models  Ontologies, through inference, can provide us with implicitly defined information  Reasoning engines Thomas R. Gruber. Toward Principles for the Design of Ontologies Used for Knowledge Sharing. International Journal of Human-Computer Studies, vol. 43, no. 5-6, pages , 1995

Ontology languages 5  Several languages have been proposed  RDF(S) RDF: Uses URI for expressing relationships between objects (triples) RDF: Allows structured and semi-structured data to be mixed, exposed, and shared across different applications. RDFS: Allows the definition of classes, the relations between them and semantic constraints on RDF  OWL Based on Description Logics. Designed to represent rich and complex knowledge 3 types of increasing expressivity Lite, DL, Full  OWL2 Logical extension of OWL Deals with weaknesses in OWL expression and integrates features requested by users 3 variations EL, QL, RL

Rule languages 6  They add expressive extensions to ontology languages  E.g: SandArea(?x), SeaArea(?y), isAdjacent(?x,?y)-> Beach(?x)  Standard languages have been defined  RuleML, SWRL  Other rule languages are offered by reasoning engines  Such as Jena, OWLIM, Pellet, Hermit, …

Query languages 7  Query languages have been proposed for retrieving information from ontology repositories  SPARQL, SeRQL, RDQL  Most have similarities with SQL  SELECT * WHERE {?X rdf:type gn:LandCover}  Retrieve all instances of class gn:LandCover (SPARQL)  Query languages can be used for deriving new facts  CONSTRUCT {?region gn:depicts gn:Vegetation} WHERE { ?region gn:hasNDVI ?value. FILTER (?value > 0.5) } A type of rule expression  Such approaches are proposed e.g. in SPIN (SPARQL Inference Notation)

Land Use/Land Cover systems 8  Available land classifications correspond to specific applications, e.g. crop or vegetation characterization  As such, they cannot be used for generic telesensing applications  Most important of them are  CORINE Organized in a 3 level hierarchy: 5 categories of the 1 st level are broken down to 15 categories on the 2 nd level which in turn are broken down to 44 3 rd level categories  Land Cover Classification System (LCCS) It doesn’t specify predefined land cover categories, it rather defines general classification criteria for characterizing land covers

Knowledge-based analysis in GNORASI 9  Facts: large number of objects and fuzzy inference support  Solution  Ontology classification processor for rule definition  Membership values are sent to the ontology  Knowledge web service (java). Demonstrates the external use of processors  Classification strategy (objects are always assigned to subclasses)  Iterative execution of SPARQL UPDATE, GeoSPARQL  Objects are classified to the class with highest membership value  Example

GNORASI classification 10  Challenges  Solutions  Usage examples  Development details

Challenges 11  Object-based image analysis produces a large number of objects (thousands)  Probabilistic inference for class membership  Classification is based on user-defined rules  Feature-based  Geospatial restrictions  Classes can appear as premises in rules

Solutions 12  Numerical computations are executed outside the ontological framework  Fuzzy membership values are computed using membership functions  Ontological inference is used for the assignment of objects to classes  According to user rules  SPARQL Update and GeoSPARQL are used to define the rules in ontological terms  Development decision  The ontological classification is implemented as an external web service

Rule definition UI 13 Hierarchy Class ruleset Rule definition

Fuzzy values 14  The outcome of the rule definition processor are the fuzzy values of all objects for the features present in the rules  These arithmetic values have to be assigned to semantic entities, i.e. the defined classes Object idFuzzy Band1 Mean Fuzzy Roundness Fuzzy NDVI Fuzzy Band3 Kurtosis

Ontology data 15  The fuzzy values along with the class hierarchy and rule definitions are sent to the ontology classification service  The following are performed by the service  The class hierarchy is added to the core ontology  The fuzzy values are transformed to ontological data properties  Rules are translated to SPARQL Update queries  Rules are iteratively executed

Example SPARQL Update 16  Example queries for assigning an object to classes Sidewalk and Vegetation with confidences ?conf1 and ?conf2 Sidewalk INSERT {?object gn:depicts gn:depiction1. gn:depiction1 gn:depictsClass gn:Sidewalk. gn:depiction1 gn:withConfidence ?conf. } WHERE {?object rdf:type gn:Object. ?object gn:fuzzyNDVIMean ?conf. } Vegetation INSERT {?object gn:depicts gn:depiction2. gn:depiction2 gn:depictsClass gn:Vegetation. gn:depiction2 gn:withConfidence ?conf2. } WHERE {?object rdf:type gn:Object. ?object gn:fuzzyNDVIMean ?conf2. } INSERT {?object gn:depicts gn:depiction1. gn:depiction1 gn:depictsClass gn:Sidewalk. gn:depiction1 gn:withConfidence ?conf. } WHERE {?object rdf:type gn:Object. ?object gn:fuzzyNDVIMean ?conf. ?filterObject rdf:type gn:Object. ?filterObject gn:depicts ?gn:filterDepiction. ?filterDepiction gn:depictsClass gn:Road FILTER (geof:sfTouches(?object, ?filterObject)) } Sidewalk rule with geospatial restriction (adjacent to Road )

Classification 17  In the example, Sidewalk depends on Road definition. Objects assigned to Road must exist  Iterative rule execution until convergence (no changes in the repository)  In the end, every object will be assigned to classes with different membership values INSERT {?object gn:depicts gn:depiction1. gn:depiction1 gn:depictsClass gn:Sidewalk. gn:depiction1 gn:withConfidence ?conf. } WHERE {?object rdf:type gn:Object. ?object gn:fuzzyNDVIMean ?conf. ?filterObject rdf:type gn:Object. ?filterObject gn:depicts ?gn:filterDepiction. ?filterDepiction gn:depictsClass gn:Road FILTER (geof:touches(?object, ?filterObject)) }

Classification 18  In a hierarchy, objects will try to match the deepest classes

Classification 19  Object are assigned to classes with the highest membership value  A minimum threshold is applied  The service returns a list [ ]*  This is the classification result

Development 20  Java based  REST communication  Server  Grizzly project 1  Reasoner employed  OpenRDF Sesame 2 backend, OWLIM-lite 3 reasoner  Geospatial library  uSeekM IndexingSail 4 1 Grizzy, 2 OpenRDF Sesame, 3 OWLIM Lite, 4 IndexingSail,

Concluding 21  Efficient ontology-based classification  Employ both feature-based classification and geospatial restrictions  Handling of fuzzy membership values  Built as a web service

22 Thank you! Questions?