Berendt: Advanced databases, winter term 2007/08, 1 Advanced databases – Inferring implicit/new knowledge from data(bases): Deductive Databases Bettina Berendt Katholieke Universiteit Leuven, Department of Computer Science Last update: 21 November 2007

Berendt: Advanced databases, winter term 2007/08, 2 Agenda Types of reasoning Deduction in Prolog and Datalog Reasoning in OWL

Berendt: Advanced databases, winter term 2007/08, 3 Deductive database languages / Datalog: Motivation SQL-92 (= SQL2) cannot express some queries: n Are we running low on any parts needed to build a ZX600 sports car? n What is the total component and assembly cost to build a ZX600 at today's part prices? NB: SQL saw a new version (SQL3) in 1999 and further developments since then. Some DDB concepts are used to support the advanced features of more recent SQL standards.SQL

Berendt: Advanced databases, winter term 2007/08, 4 What is a deductive database (system)? A deductive database system is a database system which can make deductions (ie: conclude additional facts) based on rules and facts stored in the (deductive) database.

Berendt: Advanced databases, winter term 2007/08, 5 Styles of reasoning: „All swans are white“ n Deductive: towards the consequences l All swans are white. l Tessa is a swan. l  Tessa is white. n Inductive: towards a generalisation of observations l Joe and Lisa and Tex and Wili and... (all observed swans) are swans. l Joe and Lisa and Tex and Wili and... (all observed swans) are white. l  All swans are white. n Abductive: towards the (most likely) explanation of an observation. l Tessa is white. l All swans are white. l  Tessa is a swan.

Berendt: Advanced databases, winter term 2007/08, 6 What about truth? n Deductive: l Given the truth of the assumptions, a valid deduction guarantees the truth of the conclusion n Inductive: l the premises of an argument (are believed to) support the conclusion but do not ensure it l has been attacked several times by logicians and philosophers n Abductive: l formally equivalent to the logical fallacy affirming the consequent

Berendt: Advanced databases, winter term 2007/08, 7 What about new knowledge? C.S. Peirce: n Introduced „abduction“ to modern logic n (after 1900): used „abduction“ to mean: creating new rules to explain new observations (this meaning is actually closest to induction) n >  essential for scientific discovery

Berendt: Advanced databases, winter term 2007/08, 8 Knowledge discovery and styles of reasoning 1. Business understanding  Evaluation n Learn a model from the data (observed instances) n Generally involves induction (during Modelling) 2. Deployment n Apply the model to new instances n Corresponds to deduction l (if one assumes that the model is true)

Berendt: Advanced databases, winter term 2007/08, 9 Agenda Types of reasoning Deduction in Prolog and Datalog Reasoning in OWL

Berendt: Advanced databases, winter term 2007/08, 10 Deductive databases in a Computer Science context n Deductive databases have grown out of the desire to combine logic programming with relational databases to construct systems that support a powerful formalism and are still fast and able to deal with very large datasets.logic programmingrelational databases n Deductive databases are more expressive than relational databases but less expressive than logic programming systems.relational databaseslogic programming n Deductive databases have not found widespread adoptions outside academia, but some of their concepts are used in todays relational databases to support the advanced features of more recent SQL standards.relational databasesSQL

Berendt: Advanced databases, winter term 2007/08, 11 Datalog n a query and rule language for deductive databases that syntactically is a subset of Prolog.querydeductive databasesProlog n Roots in 1970s; the term Datalog was coined in the mid 1980s by a group of researchers interested in database theory. n Query evaluation is sound and complete and can be done efficiently even for large databases. n Query evaluation is usually done using bottom up strategies. n In contrast to Prolog, Datalog l disallows complex terms as arguments of predicates, e.g. P(1, 2) is admissible but not P(f1(1), 2), l imposes certain stratification restrictions on the use of negation and recursion, andstratification l only allows range restricted variables, i.e. each variable in the conclusion of a rule must also appear in a not negated clause in the premise of this rule.

Berendt: Advanced databases, winter term 2007/08, 12 Further differences between deductive DBs & logical programming n Order sensitivity and procedurality: In Prolog program execution depends on the order of rules in the program and on the order of parts of rules; these properties are used by programmers to build effective programs. In database languages (like SQL or Datalog), however, program execution is independent of the order or rules and facts.PrologSQLDatalog n Special predicates: In Prolog programmers can directly influence the procedural evaluation of the program with special predicates such as the cut, this has no correspondence in deductive databases.Prologcut n Function symbols: Logic Programming languages allow function symbols to build up complex symbols. This is not allowed in deductive databases. function symbols n Tuple oriented processing: Deductive databases use set oriented processing while logic programming languages concentrate on one tuple at a time.

Berendt: Advanced databases, winter term 2007/08, 13 What is Prolog? n Prolog is the most widely used language to have been inspired by logic programming research. Some features: n Prolog uses logical variables. These are not the same as variables in other languages. Programmers can use them as ‘holes’ in data structures that are gradually filled in as computation proceeds. n A Prolog program can also be seen as a relational database containing rules as well as facts.

Berendt: Advanced databases, winter term 2007/08, 14 What a program looks like /* At the Zoo */ elephant(george). elephant(mary). panda(chi_chi). panda(ming_ming). dangerous(X) :- big_teeth(X). dangerous(X) :- venomous(X). guess(X, tiger) :- stripey(X), big_teeth(X), isaCat(X). guess(X, koala) :- arboreal(X), sleepy(X). guess(X, zebra) :- stripey(X), isaHorse(X).

Berendt: Advanced databases, winter term 2007/08, 15 Prolog is a ‘declarative’ language Clauses are statements about what is true about a problem, instead of instructions how to accomplish the solution. The Prolog system uses the clauses to work out how to accomplish the solution by searching through the space of possible solutions. Not all problems have pure declarative specifications. Sometimes extralogical statements are needed.

Berendt: Advanced databases, winter term 2007/08, 16 Structure of Programs Programs consist of procedures. Procedures consist of clauses. Each clause is a fact or a rule. Programs are executed by posing queries. An example…

Berendt: Advanced databases, winter term 2007/08, 17 Example elephant(george). elephant(mary). elephant(X) :- grey(X), mammal(X), hasTrunk(X). Procedure for elephant Predicate Clauses Rule Facts

Berendt: Advanced databases, winter term 2007/08, 18 Example ?- elephant(george). yes ?- elephant(jane). no Queries Replies

Berendt: Advanced databases, winter term 2007/08, 19 Clauses: Facts and Rules Head :- Body.This is a rule. Head.This is a fact. ‘if’ ‘provided that’ ‘turnstile’ Full stop at the end.

Berendt: Advanced databases, winter term 2007/08, 20 Deductive database languages / Datalog: Motivation SQL-92 cannot express some queries: n Are we running low on any parts needed to build a ZX600 sports car? n What is the total component and assembly cost to build a ZX600 at today's part prices? Can we extend the query language to cover such queries? n Yes, by adding recursion.

Berendt: Advanced databases, winter term 2007/08, 21 Datalog SQL queries can be read as follows: “If some tuples exist in the From tables that satisfy the Where conditions, then the Select tuple is in the answer.” Datalog is a query language that has the same if-then flavor: n New: The answer table can appear in the From clause, i.e., be defined recursively. n Prolog style syntax is commonly used.

Berendt: Advanced databases, winter term 2007/08, 22 Example Find the components of a trike? We can write a relational algebra query to compute the answer on the given instance of Assembly. But there is no R.A. (or SQL-92) query that computes the answer on all Assembly instances. Assembly instance part subpart number trike wheel frame spoke tire seat pedal rim tube

Berendt: Advanced databases, winter term 2007/08, 23 The Problem with R.A. and SQL-92 Intuitively, we must join Assembly with itself to deduce that trike contains spoke and tire. n Takes us one level down Assembly hierarchy. n To find components that are one level deeper (e.g., rim), need another join. n To find all components, need as many joins as there are levels in the given instance! For any relational algebra expression, we can create an Assembly instance for which some answers are not computed by including more levels than the number of joins in the expression!

Berendt: Advanced databases, winter term 2007/08, 24 A Datalog Query that Does the Job Comp(Part, Subpt) :- Assembly(Part, Subpt, Qty). Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), Comp(Part2, Subpt). Can read the second rule as follows: “ For all values of Part, Subpt and Qty, if there is a tuple (Part, Part2, Qty) in Assembly and a tuple (Part2, Subpt) in Comp, then there must be a tuple (Part, Subpt) in Comp.” head of rulebody of ruleimplication

Berendt: Advanced databases, winter term 2007/08, 25 Using a Rule to Deduce New Tuples Each rule is a template: by assigning constants to the variables in such a way that each body “literal” is a tuple in the corresponding relation, we identify a tuple that must be in the head relation. n By setting Part=trike, Subpt=wheel, Qty=3 in the first rule, we can deduce that the tuple is in the relation Comp. n This is called an inference using the rule. n Given a set of tuples, we apply the rule by making all possible inferences with these tuples in the body.

Berendt: Advanced databases, winter term 2007/08, 26 Example For any instance of Assembly, we can compute all Comp tuples by repeatedly applying the two rules. (Actually, we can apply Rule 1 just once, then apply Rule 2 repeatedly.) Comp tuples got by applying Rule 2 twice Comp tuples got by applying Rule 2 once

Berendt: Advanced databases, winter term 2007/08, 27 Datalog vs. SQL Notation Don’t let the rule syntax of Datalog fool you: a collection of Datalog rules can be rewritten in SQL syntax, if recursion is allowed. WITH RECURSIVE Comp(Part, Subpt) AS ( SELECT A1.Part, A1.Subpt FROM Assembly A1) UNION ( SELECT A2.Part, C1.Subpt FROM Assembly A2, Comp C1 WHERE A2.Subpt=C1.Part) SELECT * FROM Comp C2

Berendt: Advanced databases, winter term 2007/08, 28 Fixpoints Let f be a function that takes values from domain D and returns values from D. A value v in D is a fixpoint of f if f(v)=v. Consider the fn double+, which is applied to a set of integers and returns a set of integers (I.e., D is the set of all sets of integers). n E.g., double+({1,2,5})={2,4,10} Union {1,2,5} n The set of all integers is a fixpoint of double+. n The set of all even integers is another fixpoint of double+; it is smaller than the first fixpoint.

Berendt: Advanced databases, winter term 2007/08, 29 Least Fixpoint Semantics for Datalog The least fixpoint of a function f is a fixpoint v of f such that every other fixpoint of f is larger than or equal to v. In general, there may be no least fixpoint (we could have two minimal fixpoints, neither of which is smaller than the other). If we think of a Datalog program as a function that is applied to a set of tuples and returns another set of tuples, this function (fortunately!) always has a least fixpoint.

Berendt: Advanced databases, winter term 2007/08, 30 Negation If rules contain not there may not be a least fixpoint. Consider the Assembly instance; trike is the only part that has 3 or more copies of some subpart. Intuitively, it should be in Big, and it will be if we apply Rule 1 first. n But we have Small(trike) if Rule 2 is applied first! n There are two minimal fixpoints for this program: Big is empty in one, and contains trike in the other (and all other parts are in Small in both fixpoints). Need a way to choose the intended fixpoint. Big(Part) :- Assembly(Part, Subpt, Qty), Qty >2, not Small(Part). Small(Part) :- Assembly(Part, Subpt, Qty), not Big(Part).

Berendt: Advanced databases, winter term 2007/08, 31 Stratification T depends on S if some rule with T in the head contains S or (recursively) some predicate that depends on S, in the body. Stratified program: If T depends on not S, then S cannot depend on T (or not T). If a program is stratified, the tables in the program can be partitioned into strata: n Stratum 0: All database tables. n Stratum I: Tables defined in terms of tables in Stratum I and lower strata. n If T depends on not S, S is in lower stratum than T.

Berendt: Advanced databases, winter term 2007/08, 32 Fixpoint Semantics for Stratified Pgms The semantics of a stratified program is given by one of the minimal fixpoints, which is identified by the following operational defn: n First, compute the least fixpoint of all tables in Stratum 1. (Stratum 0 tables are fixed.) n Then, compute the least fixpoint of tables in Stratum 2; then the lfp of tables in Stratum 3, and so on, stratum-by-stratum. Note that Big/Small program is not stratified.

Berendt: Advanced databases, winter term 2007/08, 33 Aggregate Operators The notation in the head indicates grouping; the remaining arguments (Part, in this example) are the GROUP BY fields. In order to apply such a rule, must have all of Assembly relation available. Stratification with respect to use of is the usual restriction to deal with this problem; similar to negation. NumParts(Part, SUM ( )) :- Assembly(Part, Subpt, Qty). SELECT A.Part, SUM (A.Qty) FROM Assembly A GROUP BY A.Part

Berendt: Advanced databases, winter term 2007/08, 34 Evaluation of Datalog Programs Repeated inferences: When recursive rules are repeatedly applied in the naïve way, we make the same inferences in several iterations. Unnecessary inferences: Also, if we just want to find the components of a particular part, say wheel, computing the fixpoint of the Comp program and then selecting tuples with wheel in the first column is wasteful, in that we compute many irrelevant facts.

Berendt: Advanced databases, winter term 2007/08, 35 Avoiding Repeated Inferences Seminaive Fixpoint Evaluation: Avoid repeated inferences by ensuring that when a rule is applied, at least one of the body facts was generated in the most recent iteration. (Which means this inference could not have been carried out in earlier iterations.) n For each recursive table P, use a table delta_P to store the P tuples generated in the previous iteration. n Rewrite the program to use the delta tables, and update the delta tables between iterations. Comp(Part, Subpt) :- Assembly(Part, Part2, Qty), delta_Comp(Part2, Subpt).

Berendt: Advanced databases, winter term 2007/08, 36 Avoiding Unnecessary Inferences There is a tuple (S1,S2) in SameLev if there is a path up from S1 to some node and down to S2 with the same number of up and down edges. SameLev(S1,S2) :- Assembly(P1,S1,Q1), Assembly(P2,S2,Q2). SameLev(S1,S2) :- Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2). trike wheel frame spoke tire seat pedal rim tube

Berendt: Advanced databases, winter term 2007/08, 37 Avoiding Unnecessary Inferences Suppose that we want to find all SameLev tuples with spoke in the first column. We should “push” this selection into the fixpoint computation to avoid unnecessary inferences. But we can’t just compute SameLev tuples with spoke in the first column, because some other SameLev tuples are needed to compute all such tuples: SameLev(spoke,seat) :- Assembly(wheel,spoke,2), SameLev(wheel,frame), Assembly(frame,seat,1).

Berendt: Advanced databases, winter term 2007/08, 38 “Magic Sets” Idea Idea: Define a “filter” table that computes all relevant values, and restrict the computation of SameLev to infer only tuples with a relevant value in the first column. Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1). Magic(spoke). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), Assembly(P2,S2,Q2). SameLev(S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev(P1,P2), Assembly(P2,S2,Q2).

Berendt: Advanced databases, winter term 2007/08, 39 The Magic Sets Algorithm Generate an “adorned” program n Program is rewritten to make the pattern of bound and free arguments in the query explicit; evaluating SameLevel with the first argument bound to a constant is quite different from evaluating it with the second argument bound n This step was omitted for simplicity in previous slide Add filters of the form “Magic_P” to each rule in the adorned program that defines a predicate P to restrict these rules Define new rules to define the filter tables of the form Magic_P

Berendt: Advanced databases, winter term 2007/08, 40 Generating Adorned Rules The adorned program for the query pattern SameLev bf, assuming a right-to-left order of rule evaluation : SameLev bf (S1,S2) :- Assembly(P1,S1,Q1), Assembly(P2,S2,Q2). SameLev bf (S1,S2) :- Assembly(P1,S1,Q1), SameLev bf (P1,P2), Assembly(P2,S2,Q2). An argument of (a given body occurrence of) SameLev is b if it appears to the left in the body, or in a b arg of the head of the rule. Assembly is not adorned because it is an explicitly stored table.

Berendt: Advanced databases, winter term 2007/08, 41 Defining Magic Tables After modifying each rule in the adorned program by adding filter “Magic” predicates, a rule for Magic_P is generated from each occurrence O of P in the body of such a rule: n Delete everything to the right of O n Add the prefix “Magic” and delete the free columns of O n Move O, with these changes, into the head of the rule SameLev bf (S1,S2) :- Magic_SL(S1), Assembly(P1,S1,Q1), SameLev bf (P1,P2), Assembly(P2,S2,Q2). Magic_SL(P1) :- Magic_SL(S1), Assembly(P1,S1,Q1).

Berendt: Advanced databases, winter term 2007/08, 42 Summary Adding recursion extends relational algebra and SQL-92 in a fundamental way; included in SQL:1999, though not the core subset. Semantics based on iterative fixpoint evaluation. Programs with negation are restricted to be stratified to ensure that semantics is intuitive and unambiguous. Evaluation must avoid repeated and unnecessary inferences. n “Seminaive” fixpoint evaluation n “Magic Sets” query transformation

Berendt: Advanced databases, winter term 2007/08, 43 Agenda Types of reasoning Deduction in Prolog and Datalog Reasoning in OWL

Berendt: Advanced databases, winter term 2007/08, 44 What style of reasoning have we seen in the OWL examples?

Berendt: Advanced databases, winter term 2007/08, 45 Ocean Lake BodyOfWater River Stream Sea NaturallyOccurringWaterSource Summary of Properties for he Water Taxonomy Tributary Brook Rivulet Properties: feedsFrom: River Properties: emptiesInto: BodyOfWater (Functional) (Inverse Functional) (Inverse) Properties: containedIn: BodyOfWater (Transitive) Properties: connectsTo: NaturallyOccurringWaterSource (Symmetric)

Berendt: Advanced databases, winter term 2007/08, 46 Inferences we can make now that we have characterized the properties Yangtze.rdf <River rdf:ID="Yangtze" xmlns:rdf=" xmlns=" We can infer that: 1. The EastChinaSea feedsFrom the Yangtze. (Since emptiesInto is the inverse of feedsFrom) 2. The Wu connectsTo the Yangtze. (Since connectsTo is symmetric) 3. The EastChinaSea is a BodyOfWater. (Since the range of emptiesInto is a BodyOfWater. 4. The Wu is a NaturallyOccurringWaterSource. (Since the range of connectsTo is NaturallyOccurringWaterSource)

Berendt: Advanced databases, winter term 2007/08, 47 RDF / OWL inference engines n Euler is an inference engine supporting logic based proofs. It is a backward-chaining reasoner enhanced with Euler path detection and will tell you whether a given set of facts and rules supports a given conclusion. To parse the manifests and the test documents Jena 2 is used. l n F-OWL an open source OWL inference engine, implemented in the Flora-2/XSB system. l n A comparison of different engines' reasoning correctness on test cases: l

Berendt: Advanced databases, winter term 2007/08, 48 Next lecture Types of reasoning Deduction in Prolog and Datalog Reasoning in OWL Single-table and multirelational data mining

Berendt: Advanced databases, winter term 2007/08, 49 References / background reading; acknowledgements n p. 4: n pp. 6, 7: n p. 10, 12: n p. 11: n pp taken from (with minor modification): l Clocksin, W.F. (2001). Prolog Programming. n pp taken from (with minor modification): l Ramakrishnan, R. & Gehrke, J. (2002?). Database Management Systems, 3rd Edition Instructor Slides. Ch Deductive Databases. s3ed-english/Ch25_DedDB-95.pdf s3ed-english/Ch25_DedDB-95.pdf n pp. 45, 46 taken from Costello, R.L. & Jacobs, D.B. (2003). OWL Web Ontology Language. n Picture credit p. 22: