INDEXING DATASPACES by Xin Dong & Alon Halevy ITCS 6010 FALL 2008 Presented by: VISHAL SHETH

AGENDA Background Motivation Problem Definition Indexing Structure Experimental Evaluation Related Work Conclusion Future Work 2

Background Indexing – A technique used for faster execution of queries and result retrieval which can be created on one or more columns of DB table – More indexes means faster query performance, but also longer transformation/load times – Types of Indexes: B-Tree, Bitmap Dataspace – It is a data co-existence approach which forms a semantic web of inter-related / similar things. E.g. Music DataspaceMusic Dataspace DS Indexing v/s DB Indexing 3 DB INDEXINGDS INDEXING Indexing on tables of Relational DB of same source Indexing on dataspace having heterogeneous data sources Data is structuredData may be structured or unstructured Underlying DB Schema is very well defined (Relational) Underlying schema may/may not be known (DB, XML, Doc, PPT)

Motivation Indexing of data from disparate data sources is a big problem and challenging To answer queries with keyword and structure efficiently Faster execution of queries on semantically different data 4

Indexing Heterogeneous Data – Support queries over different “types” of data – Data may or may not be having semantic similarity – Data may be structured (XML/DB/Spreadsheet) or (un/partially)structured files (PPT/DOC/ /LaTex Files/WebPages) – To extract associations / relationships between either structured or unstructured or both 5 Problem Definition Inverted Lists Querying Heterogeneous Data

Solution to Indexing Heterogeneous Data Results of queries are typically from different sources (XML/tuples…) Index (an inverted list) is built whose leaves are references to data items in the individual sources 6

Solution Contd… 7 Data is modeled as a set of triples called as triple base which can take form of (instance, attribute, value) or (instance, association, instance) Instance is a real world object described by multi-valued attributes. Association is a directional relationship between two instances (two directions of a particular association are named differently)

Example of a Triple Base 8 Legends : a – Article Instance, p – Person Instance, c – Conference Instance a 1 is associated with p 1, p 2 and c 1

9 Querying Heterogeneous Data – Support queries over user independent data source structure – Support queries that enable users to specify structure, or none at all Problem Definition Inverted Lists Indexing Heterogeneous Data

Solution… Two types of query proposed Predicate Queries o Describes the desired instances by a set of predicates o Each predicate specifies an attribute value or an associated instance o Example – “Raghu’s Birch paper in Sigmod 1996” o Three predicates – (“title ‘Birch’”), (“author ‘Raghu’”), (“publishedIn ‘1996 Sigmod’”) o Definition of a predicate query :  Each predicate is of the form (v, {K 1,...,K n }). v (verb - attribute / association) and K 1,...,K n (keywords)  v = attribute  attribute predicate and v = association  association predicate  Returned instances need to satisfy at least one predicate in the query.  An instance satisfies an attribute predicate if it contains at least one of {K 1,...,K n } in the values of attribute v or sub-attributes of v.  An instance o satisfies an association predicate if there exists i, 1<=i<=n, such that o has an association v or sub-association of v with an instance o that has an attribute value K i. 10

Neighborhood Keyword Queries o Extends keyword search by considering association o A neighborhood keyword query is a set of keywords, K 1,...,K n o Definition of a Neighborhood Keyword query: An instance satisfies a neighborhood keyword query if:  It contains at least one of {K 1,...,K n } in attribute values. (relevant instance) OR  The instance is associated (in either direction) with a relevant instance (associated instance) 11

Inverted Lists It is a 2-D table with indexed keyword (as rows) and instances (as columns) Concept: – i th row represents indexed keyword K i – j th column represents instance I j – Cell (K i, I j ) records no. of occurrences (called as occurrence count) of keyword K i in the attributes of I j – Non zero cell value  Instance I j is indexed on K i – Keywords are sorted and arranged in an alphabetical order in the list – Instances are ordered by their identifiers – No structural information present – Stored as sorted array or a prefix B-Tree 12

13 Triple BaseCorresponding Inverted List Inverted Lists Contd…

Indexing Structure It is an extension to Inverted List addressing some of the issues (structural information). E.g. Tian = Last Name or First Name ? It describes how attributes and association are indexed to support predicate queries Two ways: – Indexing Attribute  ATtribute Inverted List (ATIL) – Indexing Associations  Attribute-Association Inverted List (AAIL) 14

Indexing Attribute Indexing each attribute (excessive overhead) Specify the attribute name in the cells of IL (complex query answering) ATIL (k-Keyword, a-attribute, I-Instance) – There is a row in IL for k//a//, when k appears in the value of a – The cell (k//a//, I) records occurrence count – E.g. Attribute Predicate = (“LastName, ‘Tian’”) Query converted to Keyword query as “Tian//LastName//” Search yields p 3 and not p 1 15

Indexing Association Perform keyword search on keywords, find a set of instances that contain these keywords and find associated instances for each instance (very expensive) AAIL (k-Keyword, r-association, I-Instance, a-attribute) – There is a row in IL for k//r//, when k appears in the value of a – The cell (k//r//, I) records occurrence count – E.g. Query = “Raghu’s Paper” It has an association predicate = “author ‘Raghu’” and keyword = “raghu//author//” Search yields a 1 – ATIL + association information  Slightly slow in answering attribute predicates but speeds up answering association predicates 16

Indexing Hierarchies Answering predicate queries having hierarchical structure E.g. Query = (“Name, ‘Tian’”) Results = p 1 and p 3 Find all the descendants of an attribute (FirstName, LastName and NickName) Expand the scope of query by adding above attributes E.g. “Tian//Name//” OR “Tian//FirstName//” and so on This incurs multiple index lookups and hence expensive Solution – Attribute IL with duplication (Dup-ATIL) – Attribute IL with Hierarchies (Hier-ATIL) – Hybrid Attribute IL (Hybrid-ATIL) 17

Index With Duplication Duplicate a row with attribute name for each of its ancestors Dup-ATIL (k-Keyword, a 0 -attribute, a-ancestor of a 0, I-Instance) – There is a row in IL for k//a// – The cell (k//a//, I) records occurrence count of k in values of a of I – E.g. Query = “Name ‘Tian’”  Results retrieved = p 1 and p 3 – Extensive index size (long hierarchy)  problem? – Appropriate when k occurs in many a 0 with common ancestors 18

Index with Hierarchy Path Keyword includes the hierarchy path Hier-ATIL (k-Keyword, a-attribute, I-Instance) – Hierarchy path = a 0 //…//a n // for attribute a n – There is a row for k//a 0 //…//a n // – The cell (k//a 0 //…//a n //, I) records occurrence count of k in I’s a n attributes – E.g. Query = “Name ‘Tian’”  Prefix Search = “Tian//Name//*”  Results = p 1 and p 3 – Answering query by converting into prefix search can be more expensive than a keyword search – Appropriate when k occurs in a few a with common ancestors 19

20 Hybrid Index Combination of Dup-ATIL and Hier-ATIL Hybrid-ATIL (k-Keyword, a 0 -attribute, a-ancestor of a 0, I-Instance) – Build an IL that answer’s prefix-search query with rows < threshold (t) – Hierarchy path = a 0 //…//a n // for attribute a n – p = k//a 0 //…//a n // is an indexed keyword – The cell (p//, I) records occurrence count of k in I’s a n attributes – E.g. Query = “Name ‘Jeff’”  Prefix Search = “Jeff//Name//*”  Result = p 3 – E.g. Query = “Name ‘Tian’”  Prefix Search = “Tian//Name//*”  Result = p 1 and p 3 20 t = 1

Neighborhood Keyword Queries Keyword Inverted List (KIL) – Equal to Hybrid-AAIL – Summarize prefixes ending with hierarchy path and also the one corresponding to keywords – Keywords (k 1,…,k n ) are transformed to a prefix search (k 1 //*,…, k n //*) – E.g. Query = “birch”  prefix-search = “birch//*”  results = a 1, c 1, p 1, p 2 21 t = 1

Experimental Evaluation Indexing structure + text  improves performance in answering both the type of queries Data set = personal data on desktop + some external sources Extracted associations and relationships from disparate items are stored in RDF file managed by Jena RDF : Resource Description Framework Jena : Java framework supporting Semantic Web applications RDF file had 105,320 object instances; 300,354 attribute values; 468,402 association instances; file size = 52.4 MB Four types of queries – – PQAS: Predicate Queries with Attribute (no sub-attributes) – PQAC: Predicate Queries with Attribute (with sub-attributes) – PQR: Predicate Queries with association – NKQ: Neighborhood Keyword Queries Hardware – 4 CPU’s (with 3.2 GHz Processor and 1 MB Cache memory) – 1 GB memory (RAM) 22

Performance 23 Alternative approaches – NAÏVE (Basic IL) and SEPIL (3 separate indexes (IL, structured index & relationship index) Both returned instances with no occurrence count and hence an extra overhead Clauses – Introducing some variation (E.g. change no. of keywords)

Performance Contd… Compare efficiency of ATIL with a technique that creates separate index for each attribute ATIL reduces indexing time by 63 % and keyword-lookup time by 33 % 24

Related Work Indexing XML – Indexing on Structure Schema-driven queries (list all book authors) Does not index text values – Indexing on Value Indexes text values and encodes parent-child/ancestor- descendant relation – Indexing on both Combines indexes on structure and on text Indexing keyword queries in R-DB – DISCOVER, DBXplorer and BANKS require join-network at run-time which is expensive 25

Conclusion Novel indexing approach to support flexible querying over dataspaces Inverted list are used for creating indexes IL captures the structure including attributes of instances, relationships between instances and hierarchies of schema elements. The experimental results shows that IL speeds up query answering 26

Future Work Extend indexes to support heterogeneous (attribute) values Appropriate ranking algorithms 27