Bigscholar 2014, April 8, Seoul, South Korea1 Trust and Hybrid Reasoning for Ontological Knowledge Bases Hui Shi, Kurt Maly, and Steven Zeil Contact:

2 Outline Problem –Semantic web subject to changes –How to scale a reasoner to big data? Background –Knowledge base using ontologies –Inference strategies –Benchmarks –Optimized backward chaining Hybrid Reasoner –Materialization with search and retrieval –Change and marking trusted/untrusted areas –Reason over untrusted goals Conservative trust assessment –Property based trust –Pattern based trust Evaluation Conclusions Bigscholar 2014, April 8, Seoul, South Korea

3 Problem Efficiency of reasoning in the face of large scale and frequent changes within a question/answer system over a semantic web Issue –Forward chaining scales well for fixed knowledge bases –Backward chaining can handle changes in knowledge base but does not scale Bigscholar 2014, April 8, Seoul, South Korea

Background Existing semantic application: question/answer systems –Libra, Cimple, Arnetminer Semantic Web –Resource Description Framework(RDF) –Web Ontology Language (OWL) for specific knowledge domains –SPARQL query language for RDF –SWRL rule language Reasoning systems –Jena proprietary Jena rules –Pellet and KANON –ORACLE 11g –OWLIM Bigscholar 2014, April 8, Seoul, South Korea 4

5 Background Knowledge base (KB) –Ontologies –Representation formalism: Description Logic (DL) Inference methods for First Order Logic –Materialization and forward chaining pre-computes inferred truths and starts with the known data suitable for frequent computation of answers with data that are relatively static Owlim and Oracle –Query-rewriting and backward chaining expands the queries and starts with goals suitable for efficient computation of answers with data that are dynamic and infrequent queries Virtuoso Bigscholar 2014, April 8, Seoul, South Korea

Background Benchmarks evaluate and compare the performances of different reasoning systems –The Lehigh University Benchmark (LUBM) –The University Ontology Benchmark (UOBM) 6Bigscholar 2014, April 8, Seoul, South Korea

Background Optimized backward-chaining algorithm –generate a query response for a given query pattern based on a specific rule set (RDFS, Horst, custom) Ordered Selection Function Switching between Binding Propagation and Free Variable Resolution Avoid Repetition and Non-Termination (OLDT) owl:sameAs Optimization Bigscholar 2014, April 8, Seoul, South Korea 7

Hybrid reasoner Motivation example Assume fully materialized KB Harvester adds new fact: student0 enrolled course0 Query ‘Who is enrolled in course 0?’ ok Assume fact Porf0 teaches course0 in KB Query “Who is being taught by Prof0?” not ok as simple lookup; needs reasoning with rule such as: enrolledIn(?Student,?Course?), teaches(?Faculty,?Course) :- isTaughtBy(?Student,?faculty) Bigscholar 2014, April 8, Seoul, South Korea 8

Hybrid reasoner Mark region of KB ‘trusted’ that is not affected by change Hybrid algorithm: –If a goal is in trusted region then return substitutions from KB –else for each rule R and substitution σ1 such that the head of R σ1 matches goal proveTheRuleBody (R.body, σ1) proveTheRuleBody: prove each goal in the rule body one by one recursively 9Bigscholar 2014, April 8, Seoul, South Korea

Trustworthy Goals proof goal p(?X,?Y) is trustworthy if all instances of that goal derivable from facts and rules in the knowledge base are present in that knowledge base as instances –In practice, we will need to approximate set of trustworthy goals A partition into trusted and untrusted sets is called conservative if no untrustworthy goals are trusted 10Bigscholar 2014, April 8, Seoul, South Korea

Approximation 1: Trusted Properties property-based trust: –assume that any property P that was involved in a change is itself untrusted –take the closure of the “is used as a premise of” relation, P occurs in the body of a rule used to prove R …, P(x,y), … :- R(w,z) –then R is also untrusted. Bigscholar 2014, April 8, Seoul, South Korea 11

Approximation 1: Trusted Properties property-based trust breaks down in the face of “meta- rules” in the knowledge base, rules that permit reasoning about properties themselves, e.g., inverse rule special handling of the meta-rules common to RDF and OWL result in significant fractions of the knowledge base being marked as untrusted unnecessarily 12Bigscholar 2014, April 8, Seoul, South Korea

Approximation 2: Trusted Patterns –Pattern-based trust: a pattern P(X,Y) (where X and Y could be ground instances or free variables) is untrusted if it matches a change to the knowledge base or if it can be derived from a rule with an untrusted pattern as a premise –Offers finer discrimination than property-based Bigscholar 2014, April 8, Seoul, South Korea 13

Computing Untrusted Pattern Marking algorithm Add change Check each rule in the rule set to see if we can propagate the “untrust” forward by a limited, specialized analogue of forward chaining Add untrusted set produced from the above one change to the existing untrusted set, discarding any patterns that are specializations of other elements 14Bigscholar 2014, April 8, Seoul, South Korea

Untrusted Pattern Example –Harvester adds ‘worksFor(Fullprofessor0, University0)’ –Marking algorithm discovers as untrusted worksFor (Fullprofessor0, University0) member (University0, Fullprofessor0) memberOf (Fullprofessor0, University0) –Query: “Who are members of University0?” needs reasoning –Query: “Who are members of University1” ok for direct retrieval as memberOf(?x, University1) is trusted Bigscholar 2014, April 8, Seoul, South Korea 15

Evaluation: property-based 16 Bigscholar 2014, April 8, Seoul, South Korea Changes Actual # new properties Actual # new facts # untrusted properties Adding a new class2312 Add a subclass relationship between two new classes 2612 Add new Class as subClass of existing class 2512 Adding a new Property2212 Add a new Property as subPropertyOf of another new Property 2412 Add new Property as subPropertyOf of existing Property 2312 Add new Class as domain to a new Property 3513 Add new Class as range to a new Property 3513

Evaluation: pattern-based Produces the same number of properties as the ‘actual’ columns show comparison of performance of our hybrid pattern-based proof algorithm against our regular, optimized backward chaining algorithm and against the OWLIM using LUBM1, LUBM10, and LUBM40, of size 100,839, 1,272,871, and 5,307,754 objects Query response time (ms) after adding student 17Bigscholar 2014, April 8, Seoul, South Korea

Evaluation Query response time (ms) after adding undergraduate student percentage of untrusted facts in KB ranges 0 to a high of 10% percentage of untrusted patterns in KB ranges 0 to a high of 5% 18Bigscholar 2014, April 8, Seoul, South Korea

19 Conclusions We reported on our efforts to use ‘ trust’ in backward- chaining reasoners to accommodate the changing knowledge base. We have shown that a pattern-based marking algorithm errs on the conservative side at an acceptable level and We show that compared to a forward chaining algorithm and a pure backward chaining algorithm that our hybrid algorithm is better in almost all cases tested Bigscholar 2014, April 8, Seoul, South Korea

Future Work Explore the performance of the trust marking algorithm and of the hybrid reasoner as a function of the fraction of the knowledge base that is untrusted Explore the impact of long sequences of individual changes on the marking algorithm time and subsequently on the hybrid reasoner Explore performance of the hybrid reasoner as a function of the overall degree of inter-connection within the knowledge base semantics as a loosely connected network will lead to faster termination of the trust marking algorithm 20Bigscholar 2014, April 8, Seoul, South Korea