1. A COURSE ON PROBABILISTIC DATABASES Dan Suciu University of Washington June, 2014Probabilistic Databases - Dan Suciu 1

2. Outline 1. Motivating Applications 2. The Probabilistic Data ModelChapter 2 3. Extensional Query PlansChapter 4.2 4. The Complexity of Query EvaluationChapter 3 5. Extensional EvaluationChapter 4.1 6. Intensional EvaluationChapter 5 7. Conclusions June, 2014Probabilistic Databases - Dan Suciu 2 Part 1 Part 2 Part 3 Part 4

3. Review: Model Counting Model Counting Problem: given F, compute #F Probability Computation Problem: given F, compute P(F) #P-complete Exact solvers – based on the DPLL procedure Approximate solvers – will not discuss in this tutorial Question: what performance guarantees offer DPLL- based algorithm for query evaluating on probabilistic databases? June, 2014Probabilistic Databases - Dan Suciu 3

4. Background: DPLL June, 2014Probabilistic Databases - Dan Suciu Based on Davis, Putnam, Logemann, Loveland, from the 60s; [Gomes et al., 2009] Modern model counting systems are based on DPLL c2d [Huang and Darwiche, 2007], Dsharp [Muise et al., 2012] Usually F is in CNF, then, add this simple heuristics: favor a variable X that occurs only (un-)negated. We will apply DPLL to positive DNF formulas F // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) 4

5. The trace of DPLL The trace of a DPLL algorithm on a Boolean function F is a Read-Once-Branching-Program (RBOC), also called a Free Binary Decision Diagram (FBDD) The trace is used in: Knowledge compilation: constructs the trace explicitly Lower bounds for DPLL-based algorithms: we do this next June, 2014Probabilistic Databases - Dan Suciu 5

6. The trace of DPLL June, 2014Probabilistic Databases - Dan Suciu // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) 6 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 X1 X2 X3 0 1 0 1 0 1 The trace is a tree FBDD 01 01 01 01 0 1

7. Background: DPLL with Caching June, 2014Probabilistic Databases - Dan Suciu All modern model counting systems extend DPLL with Caching 7 // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // DPLL with caching: Cache F and P(F); look it up before computing // DPLL with caching: Cache F and P(F); look it up before computing

8. The trace of DPLL with Caching June, 2014Probabilistic Databases - Dan Suciu 8 // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // DPLL with caching: Cache F and P(F); look it up before computing // DPLL with caching: Cache F and P(F); look it up before computing F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 X1 X2 X3 1 0 The trace is an FBDD 01 0 1 01 0 1

9. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once June, 2014Probabilistic Databases - Dan Suciu 9 X1 X2 X3 1 0 01 0 1 01 0 1 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3

10. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once Computing F with the FBDD: If X i = 0 follow the 0 edge If X i = 1 follow the 1 edge June, 2014Probabilistic Databases - Dan Suciu 10 X1 X2 X3 1 0 01 0 1 01 0 1 X1=0, X2=1, X3=0 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3

11. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once Computing F with the FBDD: If X i = 0 follow the 0 edge If X i = 1 follow the 1 edge Computing P(F) with the FBDD Dynamic programming bottom-up June, 2014Probabilistic Databases - Dan Suciu 11 X1 X2 X3 1 0 01 0 1 01 0 1 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 P(X1)=0.1 P(X2)=0.2 P(X3)=0.3

12. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once Computing F with the FBDD: If X i = 0 follow the 0 edge If X i = 1 follow the 1 edge Computing P(F) with the FBDD Dynamic programming bottom-up June, 2014Probabilistic Databases - Dan Suciu 12 X1 X2 X3 1 0 01 0 1 01 0 1 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 1.0 0.0 P(X1)=0.1 P(X2)=0.2 P(X3)=0.3

13. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once Computing F with the FBDD: If X i = 0 follow the 0 edge If X i = 1 follow the 1 edge Computing P(F) with the FBDD Dynamic programming bottom-up June, 2014Probabilistic Databases - Dan Suciu 13 X1 X2 X3 1 0 01 0 1 01 0 1 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 P(X1)=0.1 P(X2)=0.2 P(X3)=0.3 1.0 0.0 0.3

14. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once Computing F with the FBDD: If X i = 0 follow the 0 edge If X i = 1 follow the 1 edge Computing P(F) with the FBDD Dynamic programming bottom-up June, 2014Probabilistic Databases - Dan Suciu 14 X1 X2 X3 1 0 01 0 1 01 0 1 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 P(X1)=0.1 P(X2)=0.2 P(X3)=0.3 1.0 0.0 0.3 0.06 0.8*0.3+0.2*1 =0.44

15. Background: FBDD An FBDD (Free Binary Decision Diagram) is a graph s.t.: Sink node are labeled 0 or 1 Internal nodes are labeled with a variable X i, and have two outgoing edges, labeled 0 and 1 Every root-to-sink path visits each variable X i at most once Computing F with the FBDD: If X i = 0 follow the 0 edge If X i = 1 follow the 1 edge Computing P(F) with the FBDD Dynamic programming bottom-up June, 2014Probabilistic Databases - Dan Suciu 15 X1 X2 X3 1 0 01 0 1 01 0 1 F = X 1 X 2 ∨ X 1 X 3 ∨ X 2 X 3 P(X1)=0.1 P(X2)=0.2 P(X3)=0.3 1.0 0.0 0.3 0.06 0.8*0.3+0.2*1 =0.44 0.9*0.06+0.1*0.44 =0.098

16. Background:DPLL w Fixed Variable Order The order in which the DPLL procedure processes the Boolean variables X has dramatic impact on performance Heuristics: choose a fixed variable order Π: X 1, X 2, …; process variables always in this order An OBDD (Ordered Binary Decision Diagram) is an FBDD where every path from the root to a leaf node visits the variables in the same order Π June, 2014Probabilistic Databases - Dan Suciu 16

17. Background:DPLL w Fixed Variable Order June, 2014Probabilistic Databases - Dan Suciu 17 // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // DPLL with caching: Cache F and P(F); look it up before computing // DPLL with caching: Cache F and P(F); look it up before computing X1 X2 X3 1 0 OBDD Π=X1,X2,X3 01 0 1 01 0 1 Process variables in predefined order X 1, X 2, … X1 X2 X3 0 0 0 1 0 1 1 1 X2 0 1 0 1 FBDD

18. Background: DPLL w/ Components June, 2014Probabilistic Databases - Dan Suciu Most modern model counting systems implement components 18 // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // basic DPLL: Function P(F): if F = false then return 0 if F = true then return 1 select a variable X, return (1-P(X))×P(F X=0 ) + P(X)×P(F X=1 ) // DPLL with caching: Cache F and P(F); look it up before computing // DPLL with caching: Cache F and P(F); look it up before computing // DPLL with components: if F = F 1 ∧ F 2 and F 1, F 2 have no common variables then return P(F 1 ) × P(F 2 ) // DPLL with components: if F = F 1 ∧ F 2 and F 1, F 2 have no common variables then return P(F 1 ) × P(F 2 ) The trace of a DPLL w/ caching and components is called a “decision-DNNF”. Theorem: [Beame’13] Every decision-DNNF for a positive k-DNF formula can be converted into an FBDD of size O(N k )

19. Summary of DPLL June, 2014Probabilistic Databases - Dan Suciu 19 DPLL VariantTrace DPLL with caching and fixed variable orderOBDD DPLL with cachingFBDD DPLL with caching and components FBDD (for queries) … a different model counting formalismd-DNNF

20. Intentional Query Evaluation Query Q + database D  lineage expression F Q Compute P(F) using a general model counting system Question: for which Q does the system run in PTIME? The size of the trace gives a lower bound on the running time of DPLL-based algorithm June, 2014Probabilistic Databases - Dan Suciu 20

21. xy a1b1Y1 a1b2Y2 a2b3Y3 a2b4Y4 a2b5Y5 x a1X1 a2X2 a3X3 Example June, 2014Probabilistic Databases - Dan Suciu 21 S R SELECT DISTINCT ‘true’ FROM R, S WHERE R.x = S.x SELECT DISTINCT ‘true’ FROM R, S WHERE R.x = S.x Q = R(x), S(x,y) Study the size of OBDD, FBDD, etc for the formula F Q F Q = X1 Y1 ∨ X1 Y2 ∨ X2 Y3 ∨ X2 Y4 ∨ X2 Y5

22. 1. Read-Once Boolean Formulas A Boolean formula F is called read-once if it can be written such that every Boolean variable occurs only once June, 2014Probabilistic Databases - Dan Suciu 22 (X ∨ Z) ∧ (Y ∨ U)

23. 1. Read-Once Boolean Formulas A Boolean formula F is called read-once if it can be written such that every Boolean variable occurs only once P(F) can be computed in linear time: June, 2014Probabilistic Databases - Dan Suciu P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) 23 (X ∨ Z) ∧ (Y ∨ U)

24. 1. Read-Once Boolean Formulas A Boolean formula F is called read-once if it can be written such that every Boolean variable occurs only once P(F) can be computed in linear time: F has an OBDD with n nodes, where n = # of variables June, 2014Probabilistic Databases - Dan Suciu P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) 24 F1 ∧ F2F1 ∧ F2 OBDD for F 1 01 OBDD for F 2 01 F1 ∨ F2F1 ∨ F2 (X ∨ Z) ∧ (Y ∨ U)

25. 1. Read-Once Boolean Formulas A Boolean formula F is called read-once if it can be written such that every Boolean variable occurs only once P(F) can be computed in linear time: F has an OBDD with n nodes, where n = # of variables June, 2014Probabilistic Databases - Dan Suciu P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) 25 F1 ∧ F2F1 ∧ F2 OBDD for F 1 01 OBDD for F 2 01 OBDD for F 1 OBDD for F 2 F1 ∨ F2F1 ∨ F2 01 (X ∨ Z) ∧ (Y ∨ U)

26. 1. Read-Once Boolean Formulas A Boolean formula F is called read-once if it can be written such that every Boolean variable occurs only once P(F) can be computed in linear time: F has an OBDD with n nodes, where n = # of variables June, 2014Probabilistic Databases - Dan Suciu P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) P(F 1 ∧ F 2 ) = P(F 1 ) × P(F 2 ) P(F 1 ∨ F 2 ) = 1 – (1- P(F 1 )) × (1-P(F 2 )) 26 F1 ∧ F2F1 ∧ F2 OBDD for F 1 01 OBDD for F 2 01 OBDD for F 1 OBDD for F 2 F1 ∨ F2F1 ∨ F2 OBDD for F 1 OBDD for F 2 010 1 (X ∨ Z) ∧ (Y ∨ U)

27. xy a1b1Y1 a1b2Y2 a2b3Y3 a2b4Y4 a2b5Y5 x a1X1 a2X2 a3X3 1. Read-Once Example June, 2014Probabilistic Databases - Dan Suciu 27 S R SELECT DISTINCT ‘true’ FROM R, S WHERE R.x = S.x SELECT DISTINCT ‘true’ FROM R, S WHERE R.x = S.x Q = R(x), S(x,y) = X1 (Y1 ∨ Y2) ∨ X2 (Y3 ∨ Y4 ∨ Y5) Read-once F Q = X1 Y1 ∨ X1 Y2 ∨ X2 Y3 ∨ X2 Y4 ∨ X2 Y5

28. 1. Read-Once Example June, 2014Probabilistic Databases - Dan Suciu 28 X1 Y1 Y2 1X2 Y3 Y4 1 0 0 1 0 1 0 1 01 0 01 F Q = X1 (Y1 ∨ Y2) ∨ X2 (Y3 ∨ Y4 ∨ Y5) 1 Q = R(x), S(x,y) Y5 Note: every Boolean variable occurs only once in the OBDD The size of the OBDD is linear in the size of the database D Read-once to OBDD

29. 1. Read-Once Queries June, 2014Probabilistic Databases - Dan Suciu 29 Read-once: Q = R(x), S(x,y) Q U = R(x 1 ), S(x 1,y 1 ) ∨ T(x 2 ), S(x 2,y 2 ) … all hierarchical, conjunctive queries w/o self-joins … Not read-once (but still PTIME): Q J = R(x 1 ), S(x 1,y 1 ),T(x 2 ), S(x 2,y 2 ) … Theorem For any query Q, the following are equivalent: ∀ D, the lineage F Q is read-once Q can be written such that it is both hierarchical, and every symbols occurs once. Theorem For any query Q, the following are equivalent: ∀ D, the lineage F Q is read-once Q can be written such that it is both hierarchical, and every symbols occurs once. = ∃ x (R(x) ∨ T(x)) ∧ ∃ y S(x,y)

30. Landscape of Probabilistic Databases June, 2014Probabilistic Databases - Dan Suciu #P-hard PTIME Read Once Q J = R(x 1 ),S(x 1,y 1 ),T(x 2 ),S(x 2,y 2 ) is not read-once! [Jha&S.2011] QJQJ QUQU 30 H0H0 H1H1 H2H2 hierarchical H3H3 non-hierarchical

31. 2. OBDD OBDD Synthesis [Wegener’2000] Let Π be an order of the Boolean variables: X 1, X 2, …, X k, …, X n. If the Π-OBDD for F 1 has size N 1, and the Π-OBDD for F 2 has size N 2 then Π-OBDD for F 1 ∧ F 2 and for F 1 ∨ F 2 has has size ≤ N 1 N 2 June, 2014Probabilistic Databases - Dan Suciu 31

32. 2. OBDD OBDD Synthesis [Wegener’2000] Let Π be an order of the Boolean variables: X 1, X 2, …, X k, …, X n. If the Π-OBDD for F 1 has size N 1, and the Π-OBDD for F 2 has size N 2 then Π-OBDD for F 1 ∧ F 2 and for F 1 ∨ F 2 has has size ≤ N 1 N 2 June, 2014Probabilistic Databases - Dan Suciu 32 F1F1 X1X1 XkXk … XkXk m 1 nodes 01 F2F2 X1X1 XkXk … XkXk m 2 nodes 01

33. 2. OBDD OBDD Synthesis [Wegener’2000] Let Π be an order of the Boolean variables: X 1, X 2, …, X k, …, X n. If the Π-OBDD for F 1 has size N 1, and the Π-OBDD for F 2 has size N 2 then Π-OBDD for F 1 ∧ F 2 and for F 1 ∨ F 2 has has size ≤ N 1 N 2 June, 2014Probabilistic Databases - Dan Suciu 33 F1F1 X1X1 XkXk … XkXk m 1 nodes 01 F2F2 X1X1 XkXk … XkXk m 2 nodes 01 F1∧F2F1∧F2 X1X1 XkXk … XkXk m 1 m 2 nodes 0∧00∧01∧11∧10∧10∧11∧01∧0

34. 2. OBDD Example June, 2014Probabilistic Databases - Dan Suciu 34 Q J = R(x 1 ),S(x 1,y 1 ), T(x 2 ),S(x 2,y 2 )

35. 2. OBDD Example June, 2014Probabilistic Databases - Dan Suciu 35 Q J = R(x 1 ),S(x 1,y 1 ), T(x 2 ),S(x 2,y 2 ) Q 1 = R(x 1 ),S(x 1,y 1 ) Q 2 = T(x 2 ),S(x 2,y 2 ) = ∧

36. 2. OBDD Example June, 2014Probabilistic Databases - Dan Suciu 36 x1 y1 y2 1x2 y3 y4 10 0 10 1 0 1 01 0 01 F 1 = X 1 Y 1 ∨ X 1 Y 2 ∨ X 2 Y 3 ∨ X 2 Y 4 1 Q J = R(x 1 ),S(x 1,y 1 ), T(x 2 ),S(x 2,y 2 ) Q 1 = R(x 1 ),S(x 1,y 1 ) Q 2 = T(x 2 ),S(x 2,y 2 ) = ∧

37. 2. OBDD Example June, 2014Probabilistic Databases - Dan Suciu 37 x1 y1 y2 1x2 y3 y4 10 0 10 1 0 1 01 0 01 F 1 = X 1 Y 1 ∨ X 1 Y 2 ∨ X 2 Y 3 ∨ X 2 Y 4 1 F 2 = Z 1 Y 1 ∨ Z 1 Y 2 ∨ Z 2 Y 3 ∨ Z 2 Y 4 37 z1 y1 y2 1z2 y3 y4 10 0 10 1 0 1 01 0 1 01 ∧ Q J = R(x 1 ),S(x 1,y 1 ), T(x 2 ),S(x 2,y 2 ) Q 1 = R(x 1 ),S(x 1,y 1 ) Q 2 = T(x 2 ),S(x 2,y 2 ) =

38. 2. OBDD Example June, 2014Probabilistic Databases - Dan Suciu 38 x1 y1 y2 1x2 y3 y4 10 0 10 1 0 1 01 0 01 F 1 = X 1 Y 1 ∨ X 1 Y 2 ∨ X 2 Y 3 ∨ X 2 Y 4 1 F 2 = Z 1 Y 1 ∨ Z 1 Y 2 ∨ Z 2 Y 3 ∨ Z 2 Y 4 38 z1 y1 y2 1z2 y3 y4 10 0 10 1 0 1 01 0 1 01 ∧ Same variable order in both OBDDs! Q J = R(x 1 ),S(x 1,y 1 ), T(x 2 ),S(x 2,y 2 ) Q 1 = R(x 1 ),S(x 1,y 1 ) Q 2 = T(x 2 ),S(x 2,y 2 ) =

39. 2. OBDD Example June, 2014Probabilistic Databases - Dan Suciu 39 x1 y1 y2 1x2 y3 y4 10 0 10 1 0 1 01 0 01 F 1 = X 1 Y 1 ∨ X 1 Y 2 ∨ X 2 Y 3 ∨ X 2 Y 4 = Efficient OBDD for Q J = Q 1 ∧ Q 2 1 F 2 = Z 1 Y 1 ∨ Z 1 Y 2 ∨ Z 2 Y 3 ∨ Z 2 Y 4 39 z1 y1 y2 1z2 y3 y4 10 0 10 1 0 1 01 0 1 01 ∧ Same variable order in both OBDDs! Q J = R(x 1 ),S(x 1,y 1 ), T(x 2 ),S(x 2,y 2 ) Q 1 = R(x 1 ),S(x 1,y 1 ) Q 2 = T(x 2 ),S(x 2,y 2 ) =

40. 2. Queries with Efficient OBDD June, 2014Probabilistic Databases - Dan Suciu 40 Theorem For any query Q the following are equivalent: ∀ D, the lineage F Q has an OBDD of size |D| O(1) Q is inversion-free (hierarchical + same hierarchy order for each symbol; see book) Theorem For any query Q the following are equivalent: ∀ D, the lineage F Q has an OBDD of size |D| O(1) Q is inversion-free (hierarchical + same hierarchy order for each symbol; see book) Inversion-free … Has inversion (but still PTIME): Q J = R(x 1 ), S(x 1,y 1 ),T(x 2 ), S(x 2,y 2 ) … Q V = R(x 1 ),S(x 1,y 1 ) ∨ S(x 2,y 2 ),T(y 2 ) ∨ R(x 3 ),T(y 3 ) In S: at(x 1 ) ⊃ at(y 1 ) In S: at(x 2 ) ⊂ at(y 2 ) Same order in both S: at(x 1 ) ⊂ at(y 1 ) at(x 2 ) ⊂ at(y 2 )

41. Landscape of Probabilistic Databases June, 2014Probabilistic Databases - Dan Suciu #P-hard PTIME Poly size OBDD Read Once QJQJ Q V = R(x 1 ),S(x 1,y 1 ) ∨ S(x 2,y 2 ),T(y 2 ) ∨ R(x 3 ),T(y 3 ) has no poly-size OBDD! QVQV 41 H0H0 H1H1 H2H2 hierarchical H3H3 non-hierarchical QUQU

42. 3. FBDD Some queries have inversion (hence no efficient OBDD), but have an efficient FBDD We only know of examples of such queries, no complete characterization June, 2014Probabilistic Databases - Dan Suciu 42

43. 3. FBDD June, 2014Probabilistic Databases - Dan Suciu 43 Q V = R(x 1 ),S(x 1,y 1 ) ∨ S(x 2,y 2 ),T(y 2 ) ∨ R(x 3 ),T(y 3 ) Fact: (The lineage of ) Q V has an FBDD whose size is linear in |D| Q W = [R(x 0 ),S 1 (x 0,y 0 ) ∨ S 2 (x 2,y 2 ),S 3 (x 2,y 2 )] ∧ /* Q1 */ [R(x 0 ),S 1 (x 0,y 0 ) ∨ S 3 (x 3,y 3 ),T(y 3 )] ∧ /* Q2 */ [S 1 (x 1,y 1 ),S 2 (x 1,y 1 ) ∨ S 3 (x 3,y 3 ),T(y 3 )] /* Q3 */ Theorem: Any FBDD for (the lineage of ) Q W has size exponential in |D| See the book for both results.

44. Landscape of Probabilistic Databases June, 2014Probabilistic Databases - Dan Suciu #P-hard PTIME Poly-size FBDD Poly-size OBDD Read Once QJQJ QVQV QWQW Q9Q9 H0H0 H1H1 H2H2 QUQU H3H3 hierarchical Non-hierarchical 44 Open Q W has no poly-size FBDD!

45. 4. d-DNNF June, 2014Probabilistic Databases - Dan Suciu ∨ ∧∧ XYZ U ¬U X Y U ∨ X Y Z P(F) in PTIME in size(d-DNNF) P(F 1 ∧ F 2 ) = P(F 1 ) * P(F 2 ) P(F 1 ∨ F 2 ) = P(F 1 ) + P(F 2 ) [Darwiche’2000] 45 d-DNNF = expression DAG where ∧ are “Decomposable” ∨ are “Deterministic” All negations at the leaves d-DNNF = expression DAG where ∧ are “Decomposable” ∨ are “Deterministic” All negations at the leaves

46. 4. d-DNNF June, 2014Probabilistic Databases - Dan Suciu [Darwiche’2000] 46 Fact 1. Q W has an efficient d-DNNF Conjecture. Q 9 has as no efficient d-DNNF Fact 2. ¬Q W has an efficient d-DNNF Proof. Condition on h 30 : Q W = (h 30 ∨ h 32 ) ∧ (h 30 ∨ h 33 ) ∧ (h 31 ∨ h 33 ) = [(¬h 30 ) ∧ h 32 ∧ h 33 ∧ (h 31 ∨ h 33 )] ∨ [h 30 ∧ (h 31 ∨ h 33 )] = [(¬h 30 ) ∧ h 32 ∧ h 33 ] ∨ [h 30 ∧ (h 31 ∨ h 33 )] Inversion-free Deterministic See book

47. Landscape of Probabilistic Databases June, 2014Probabilistic Databases - Dan Suciu #P-hard PTIME Poly-size d-DNNF Poly-size FBDD Poly-size OBDD Read Once QJQJ QVQV QWQW Q9Q9 H0H0 H1H1 H2H2 ? QUQU Conjectured H3H3 hierarchical Non-hierarchical 47 Open