1. Representing and Querying Correlated Tuples in Probabilistic Databases
Prithviraj Sen Amol Deshpande

2. Independent tuples model Tuple correlations Representing Dependencies
outline
General Info
Introduction
Independent tuples model
Tuple correlations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

3. Issues with the use of probabilistic databases
General info
High demand for storing uncertain data
A framework that can represent not only probabilistic tuples but also correlations among them to tackle these limitations
Issues with the use of probabilistic databases
1) existent probabilistic databases make simplistic assumptions about the data that make it difficult to use them in applications that naturally produce correlated data
2) Most probabilistic databases can only answer a restricted subset of the queries that can be expressed using traditional query languages
Yparxei megalh zhthsh gia thn apo8ukefsh abaibewn dedomenwn
OI iparxouses baseis dedomenwn k;anoyn uperaplousteumenes upo8eseis gia ta dedomena (oti ta einai ane3arthta meta3h tous ) me apotelesma na mhn mporoume na tis xrisimopoihsoue ean ta dedomena dn einai ane3arthta meta3h tous
Episeis oi perissoteres pi8anotikes baseis dedomenwn mporoun na apantoun mono stis poio sunxes glwsses querry.
Sto paper afto proteinoun ena framework to opoio mporei na anaparhsta kai pi8anotikes pleiades alla kai sxeseis meta3i tous.

4. outline General Info Introduction Independent tuples model
Tuple correlations
Probabilistic graphical models & factored representations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

5. Introduction (1/2)
Database research has primarily concentrated on how to store and query exact data
Many real-world applications produce large amounts of uncertain data
Databases need to do more than simply store and retrieve; they have to help the user sift through the uncertainty and find the results most likely to be the answer.
1)H erebna stis baseis dedomenwn exei sigentrw8ei sthn epo8hkeush bebaiwn dedomenwn
2)Polles efarmoges paragoun abebaia dedomena kai se tetoies periptwseis oi baseis dedomenwn ektos tou na anaktoun kai na paragoun dedomena prepei na boi8oun ton xrhsth na briskei ta apotelesmata pou einai poio pi8ano na apanth8oun

6. Introduction (2/2)
Numerous approaches (models) proposed to handle uncertainty.
However, most models make assumptions about data uncertainty that restricts applicability (they cannot easily model or handle dependencies and correlations among tuples)

7. outline General Info Introduction Independent tuples model
Tuple correlations
Probabilistic graphical models & factored representations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

8. Independent tuples model(1/2)
One of the most commonly used tuple-level uncertainty models, associates existence probabilities with individual tuples and assumes that the tuples are independent of each other
Estw Dp mia bash dedomenwn me sxeseis Sp (pou periexei thn pleiada s1 me pi8aotita 0.6 kai thn pleiada s2 me pi8anotita 0.5) kai Tp (pou periexei thn pleiada t1 me pi8anotita 0.4)
Blepoume ston pinaka afto olous tous pi8anous kosmous kai h pi8anotita gia ka8e kosmo bgenei (me enwsh olwn twn pi8anotitwn) pollaplasiazontas apla tis pi8anotites ka8e pleiadas efoson einai ane3arthtes.
Ean twra ektelesoume to querry panw stous pia8ous kosmous

9. Independent tuples model (2/2)
Evaluating a query via the set of possible worlds is clearly intractable as the number of possible worlds is very big
Intensional semantics guarantee results in accordance
with possible words semantics but are computationally
expensive.
Extensional semantics are computationally cheaper but do not guarantee results in accordance with the possible worlds semantics.
Base tuples are independent of each other, the intermediate tuples that are generated during query evaluation are typically correlated

10. outline General Info Introduction Independent tuples model
Tuple correlations
Probabilistic graphical models & factored representations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

11. Tuple correlations (1/2)
As 8ewrisoume twra 4 set apo pi8anous kosmous pou proerxontai apo thn idia bash dedomenwn opws proigoumenos ala me diaforetikes e3arthseis pou endexomenos 8eloume na anaparasthsoume

12. Tuple correlations (2/2)
Although the tuple probabilities associated with s1, s2 and t1 are identical, the query results are drastically different across these four databases.
Since both intensional and extensional semantics assume base tuple independence neither can be directly used to do query evaluation in such cases.
->Parolou pou oi pi8anotites ton pleiadwn einai s1 s2 einai idies ta apotelesmata twn query einai teleiws diaforetika.
-> oi me8odoi extensional kai intentional pou eipame parapanw einai mono gia ane3artites pleiades kai giafto ton logo den boroun na xrhshmopoihs8oun gia thn a3iologhsh tou querry.

13. Independent tuples model Tuple correlations Query evaluation
outline
General Info
Introduction
Independent tuples model
Tuple correlations
Representing correlations
Query evaluation
Experiments
Conclusions & Work to be done

14. Representing correlations(1/3)
Associate every tuple t with a Boolean valued random variable Xt
f (X) is a function of a (small) set of random variables X, where <= f (X) <=1
Associate with each tuple in the probabilistic database a random variable
Define factors on (sub)sets of tuple-based random variables to
encode correlations.
5) The probability of an instantiation of the database is given by the product of all the factors.

15. Representing correlations(2/3)
Suppose we want to represent mutual exclusivity between tuples s1 and t1. In particular, let us try to represent the possible worlds:

16. Representing correlations(3/3)
Suppose we want to represent positive correlation between t1 and s1. In particular, let us try to represent the possible worlds:

17. Probabilistic graphical model representation
A probabilistic graphical model is graph whose nodes represent random variables and edges represent correlations Complete Ind. Mutual Exclusivity Positive Correlation
Xt1
Xs1
Xt1
Xs1
Xt1
Xs1
Xs2
Xs2
Xs2

18. Probabilistic graphical model representationX1 X2 X3

19. outline General Info Introduction Independent tuples model
Tuple correlations
Probabilistic graphical models & factored representations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

20. Query evaluation: basic idea
Treat intermediate tuples as regular tuples.
Carefully represent correlations between intermediate tuples, base tuples and result tuples to construct a probabilistic graphical model.
Cast the probability computations resulting from query evaluation to inference in probabilistic graphical models.
1)Oi endiameses pleiades prepei na xeiristoun san kanonikes
2)Prosektika prepei na anaparastountai oi sxeseis meta3i tous gia na dimiourgi8ei to pi8anotiko grafiko mondelo

21. Query evaluation: example

22. Opou to Fq einai to set twn paragontwn pou xreiazontai gia thn dimiourgeia ths pleiadas t epagogika sto querry

23. Query evaluation :example Probabilistic graphical model
Query evaluation problem in Prob. Databases: Compute the probability of the result tuple summed over all possible worlds of the database
Equivalent problem in prob. graph. models: marginal probability computation.
use inference algorithms
Xs2
Xs1
Xt1
Xi1
Xi2
Xr1

24. Xs2
Xt1
Xi1
Xi2
Xr1

25. Representing probabilistic relations
Mia pleida borei na iparxei se polles e3arthseis. Sthn dikia tous ilopoihsh oi pleiades apo8ikebontai san komati mia sxeshs. Gia thn apo8ukeush ths abebaibeotitas xrhshmopoioun partition
Ena partition apoteleite apo ena factor kai ena set apo anafores stis pleiades stis opoies oi tuxaies times einai ta orismata gia to sigekrimeno factor
Ektos apo ka8e pleiada t se mia sxesh iparxei episeis mia lista apo pointers sta partitions pou exoun anafores stis pleiades aftes
Sto sxhma: blepoume pou organonontai oi sxesies kai ta partitions gia thn bash me thn nxor e3arthsh
Oi diakekomenes grames einai oi pointesapo ths pleiades sta partitions kai oi kanonikes einai oi anafores

26. outline General Info Introduction Independent tuples model
Tuple correlations
Probabilistic graphical models & factored representations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

27. Experiments (1/3)
Database contains 860 publications from CiteSeer [GBL98].
Searched for publications for given (misspelt) author name.
Naturally involves mutual exclusivity correlations

28. Experiments (2/3)
Ran experiments on randomly generated TPC-H dataset of size 10MB.
The first bar on each query indicates the time it took to run the full query including all the database operations and the probabilistic computations.
The second one indicates the time it took to run only the database operations using our Java implementation.

29. Experiments(3/3)
The result of running an average query over a synthetically generated dataset containing tuples

30. outline General Info Introduction Independent tuples model
Tuple correlations
Probabilistic graphical models & factored representations
Representing Dependencies
Query evaluation
Experiments
Conclusions & Work to be done

31. conclusions
There is an increasing need for database solutions for efficiently managing and querying uncertain data exhibiting complex correlation patterns.
A simple and intuitive framework is presented, based on probabilistic graphical models, for explicitly modeling correlations among tuples in a probabilistic database

32. Work to be done
Problem: Although conceptually the approach presented allows for capturing arbitrary tuple correlations, exact query evaluation over large datasets exhibiting complex correlations may not always be feasible.
Future Considerations:
Development of approximate query evaluation techniques that can be used in such cases
Develop disk-based query evaluation algorithms so that their techniques can scale to very large datasets.