1. 1 iTrails: Pay-as-you-go Information Integration in Datasapces Authors: Salles, Dittrich et al. (ETH Zurich) Published in VLDB2007 Presenter: Jim 7 Dec 2007

2. 2 Background Dataspace –A step further than database –Model all kinds of files iDM: a DSMS

3. 3 Background cont. Information integration –Purpose: query multiple data sources Data source 2Data source 1Data source 3 Query interface Result Query Global schema Source schema ?

4. 4 Background cont. Schema matching/mapping –Enable the reformulation of query on the global schema to the query on source schema Global schema Source schema ? m1: for d in dept exists d' in Tdept where ( d'.dname=d.dname ^ d'.location=d.location) m2: for d in dept, e in d.emps exists d' in Tdept, e' in d'.emps where... A complete and complicate mapping is necessary for the query.

5. 5 text, links Background cont. Search engine –Easier way for query multiple data sources text, links text, links text, links Graph IR Search Engine Query semantics are NOT precise. global warming zurich

6. 6 Motivation of iTrail Find a integration solution in-between these two extremes? ? Dataspace System text, links text, links text, links text, links Graph IR Search Engine Data Integration System Temps Cities CO 2 Sunspots............... The more effort you pay, the more query power you have.

7. 7 iTrails Core Idea: Add Integration Hints Incrementally Step 1: Provide a search service over all the data –Use a general graph data model (iDM) –Works for unstructured documents, XML, and relations Step 2: Add integration semantics via hints (trails) on top of the graph –Works across data sources, not only between sources Step 3: If more semantics needed, go back to step 2 Impact: –Smooth transition between search and data integration –Semantics added incrementally improve precision / recall

8. 8 iTrails: Defining Trails Basic Form of a Trail Q L [.C L ] → Q R [.C R ] Intuition: –When I query for Q L [.C L ], you should also query for Q R [.C R ] Queries: keyword and path expressions Attribute projections Query example: global warming zurich or //Temperatures/*[celsius > 10]

9. 9 Trail examples: keyword query 20 15 14 BE ZH Trail for Implicit Meaning: “When I query for global warming, you should also query for Temperature data above 10 degrees” Trail for an Entity: “When I query for zurich, you should also query for references of zurich as a region ” global warming → //Temperatures/*[celsius > 10] DB Server Temperatures city celsius Bern Zurich zurich → //*[region = “ZH”] Uster region global warming zurich 9 ZH Zurich 26-Sep 25-Sep 24-Sep 23-Sep date

10. 10 Trail Example: Deep Web Bookmarks Trail for a Bookmark: “When I query for train home, you should also query for the TrainCompany’s website with origin at ETH Uni and destination at Seilbahn Rigiblick” train home train home → //trainCompany.com//*[origin=“ETH Uni” and dest =“Seilbahn Rigiblick”] Web Server

11. 11 Trail Examples: Thesauri, Dictionaries car auto car carro car → auto car → carro carro → car Laptop Email Server Trail for Thesauri: “When I query for car, you should also query for auto ” Trails for Dictionary: “When I query for car, you should also query for carro and vice-versa”

12. 12 Trail Examples: Schema Equivalences Trail for schema match on names: “When I query for Employee.empName, you should also query for Person.name ” Trail for schema match on salaries: “When I query for Employee.salary, you should also query for Person.income ” Employee empName salary Person name age income //Employee//*.tuple.empName → //Person//*.tuple.name //Employee//*.tuple.salary → //Person//*.tuple.income DB Server empId SSN

13. 13 How are Trails Created? Given by the user –Explicitly –Via Relevance Feedback (Semi-)Automatically –Information extraction techniques –Automatic schema matching –Ontologies and thesauri (e.g., wordnet) –User communities (e.g., trails on gene data, bookmarks)

14. 14 Uncertainty/certainty of trails Probabilistic Trails: –model uncertain trails –probabilities used to rank trails Q L [.C L ] → Q R [.C R ], 0 ≤ p ≤ 1 –Example: car → auto Scored Trails: –give higher value to certain trails –scoring factors used to boost scores of query results obtained by the trail QL [.CL] → QR [.CR], sf > 1 –Examples : T1: weather → //Temperatures/* T2: yesterday → //*[date = today()– 1] p p = 0.8 p = 1, sf = 3 p = 0.9, sf = 2

15. 15 Rewriting Queries with Trails Canonical form of the query –Decompse the query into location step seperators (‘/’ or ‘//’) and predicates –Construct a tree based on the decompsed query Query: “retrieve all information about the projects containing keyword Mike” //home/projects//*[“Mike”] U Mike U home projects (simplified form for presentation)

16. 16 Rewriting Queries with Trails cont. U weather yesterday (1) Matching T 2 : yesterday → //*[date = today() – 1] (2) Transformation U weather yesterday U //*[date = today() – 1] (3) Merging T 2 matches Query Weather yesterday Trail Rewriting method: Matching => Transformation => Merging “ yesterday ” is contained In the query

17. 17 Rewriting Queries on the canonical form of the query Trails: NOT the final result! The rewriting based on ψ 7 and ψ 9 are omitted.

18. 18 Problem: Recursive Matches... U U weather yesterday U //*[date = today() – 1] T 2 : yesterday → //*[date = today() – 1] New query still matches T 2, so T 2 could be applied again U weather U yesterday U //*[date = today() – 1] U... Infinite recursion T 2 matches

19. 19 Problem: Recursive Matches cont. U weather yesterday U //*[date = today() – 1] Trails may be mutually recursive T 3 : //*.tuple.date → //*.tuple.modified U weather U yesterday //*[date = today() – 1] T 10 : //*.tuple.modified → //*.tuple.date U //*[modified = today() – 1] U weather U yesterday //*[date = today() – 1] U //*[modified = today() – 1] U //*[date = today() – 1] We again match T 3 and enter an infinite loop T 3 matches T 10 matches

20. 20 U weather yesterday First Level U weather yesterday //Temperatures/* U U //*[date = today() – 1] U weather yesterday //Temperatures/* U U //*[modified = today() – 1] U U //*[received = today() – 1] //*[date = today() – 1] Second Level T 1 matches T 2 matches T 3, T 4 match Solution: Multiple Match Coloring Algorithm T1: weather → //Temperatures/* T2: yesterday → //*[date = today() – 1] T3: //*.tuple.date → //*.tuple.modified T4: //*.tuple.date → //*.tuple.received

21. 21 Multiple Match Coloring Algorithm Analysis Problem: – MMCA is exponential in number of levels Solution: Trail Pruning –Prune by number of levels –Prune by top-K trails matched in each level Ranking by probability/certainity/weight –Both Indexing techniques –Similar with inverted list indexing Given a query, materialize the query results (node ids) –Materialization options Left/right side of the trails

22. 22 Experiment setup Test base: iMeMex Dataspace System Criterias in evaluation –Quality: Top-K Precision and Recall –Performance: Use of Materialization –Scalability: Query-rewrite Time vs. Number of Trails iTrail scenario –Scenario 1: Few High-quality Trails –Scenario 2: Many Low-quality Trails Configured iMeMex to act in three modes –Baseline: Graph / IR search engine –iTrails: Rewrite search queries with trails –Perfect Query: Semantics-aware query

23. 23 Experiment setup cont. Trails and queries max original tree size: 14 max final tree size: 35 max # of trails applied: 5

24. 24 Quality: Top-K Precision and Recall Search Engine misses relevant results Q3: pdf yesterday Search Query is partially semantics-aware Q13: to = raimund.grube@ enron.com Scenario 1: few high-quality trails (18 trails) Queries perfect query Perfect Query always has precision and recall equal to 1 K = 20 Much improvement comparing with “baseline”.

25. 25 Performance: Use of Materialization Trail merging adds overhead to query execution Trail Materialization provides interactive times for all queries Scenario 1: few high-quality trails (18 trails)

26. 26 Scalability: Query-rewrite Time vs. Number of Trails Query-rewrite time can be controlled with pruning Scenario 2: many low-quality trails

27. 27 Conclusion: Pay-as-you-go Information Integration Dataspace System global warming zurich text, links Data Sources Approach –Step 1: Provide a search service over all the data –Step 2: Add integration semantics via trails –Step 3: If more semantics needed, go back to step 2 Their Contributions –iTrails: a generic method to model semantic relationships (e.g. implicit meaning, bookmarks, dictionaries, attribute matches,...) –Framework and algorithms for Pay-as-you-go Information Integration –Smooth transition between search and data integration