Database Systems and XML David Wu CS 632 April 23, 2001

Researched Papers J. Shanmugasundaram, et al. "Efficiently Publishing Relational Data as XML Documents", VLDB Conference, September J. Shanmugasundaram, et al. "Relational Databases for Querying XML Documents: Limitations and Opportunities," VLDB Conference, September 1999.

Efficiently Publishing Relational Data as XML Documents

Motivation Relational database systems and XML are heavily used on the Web. Would like some way to publish relational data as XML.

What is Needed Language to specify the conversion from relational data to XML. Implementation to efficiently carry out the conversion.

SQL Based Language

Implementation Alternatives Main differences between relations and XML: XML docs have tags XML has nested structure

Early Tagging, Early Structuring Stored Procedure Approach (outside engine) –Performs a nested-loop join by issuing queries for each nested structure in the desired XML. –High overhead due to the number of queries. –Fixed join order.

Early Tagging, Early Structuring Correlated CLOB Approach (inside engine) –Have one large query with sub-queries is run within the engine. –Must add XML constructor support to the engine. –XML fragments from the constructors are stored as CLOBs (Character Long Objects). Costly to handle. De-Correlated CLOB Approach (inside) –Perform query de-correlation to give optimizer more flexibility.

Late Tagging, Late Structuring Two phases: 1)Content creation 2)Tagging and structuring

Late Tagging, Late Structuring Content Creation: Redundant Relation Approach –Join all source tables –Both content and process redundancy

Late Tagging, Late Structuring Content creation: Outer Union Approach –Separate the children of the same parent (e.g. one tuple should represent either account or purchaseOrder). –At the end outer union the results. –Still some data redundancy (e.g. parent info)

Late Tagging, Late Structuring Outer Union Plan:

Late Tagging, Late Structuring Structuring/Tagging: Hashed-based Tagger Group by hashing Extract tuples and tag them.

Late Tagging, Early Structuring Late Tagging, Late Structuring requires much memory for the hash table. Fix by creating “structured content” and then tag.

Late Tagging, Early Structuring Structured content: Sorted Outer Union Approach –Desired format 1.Parent information comes before or with its child 2.All info of a node and its descendants occur together 3.Relative order of the tuples matches user-specified order –Achieve by performing a sort on ids on the result of the outer union.

Late Tagging, Early Structuring Tagging Sorted Data:ConstantSpaceTagger –Can append tags as soon as data is seen. –Only need to remember the parent ids of the last tuple seen to know when to append closing tags.

Experiement Inside Engine Outside Engine

Breakdown of Construction

Summary of Results Constructing inside the relational engine is more efficient. When processing can be done in main mem, the Unsorted Outer Union approach wins. When main mem is not enough, the Sorted Outer Union approach is best.

Relational Databases for Querying XML Documents

Why Bother? XML is becoming the standard for data representation in WWW. A query engine designed to tap information from XML documents is valuable. Relational database system is a mature technology and could be used to support XML querying.

Basic Idea Step 1: Generate a relational schema from the DTD Step 2: Parse the XML document and load the data into tuples of the relational table. Step 3: Translate the semi-structured XML queries into SQL corresponding to the relational data. Step 4: Convert the result back to XML.

Translating XML to Relational Schema Main Issues: 1.DTDs complexity 2.Arbitrary nesting of XML DTDs vs. two- level nature of relational schemas. 3.Set-valued attributes and recursion

1)Flattening transformation 2)Simplification transformation of unary operations 3)Grouping transformation

Techniques to translate XML DTD to relations. Basic Inlining Technique Shared Inlining Technique Hybrid Inlining technique

Basic Inlining Technique Inlining as many descendants of an element into a relation. (author:firstname,lastname,address) Every element will have a relation corresponding to it. (firstname, lastname, and address will all have elements)

Basic Inlining Technique (cont.) Complications: 1)Set-valued attributes (eg. Article) Solve by using foreign keys and other tables. 2)Recursion Solve with relational keys and relational recursive processing to retrieve the relationship.

Tools used in creating relations DTD Graph –Nodes are elements, attributes,operators –Each element appears once –Attributes and operators appear as many times as they do in the DTD –Cycles in the graph indicates recursion

Tools used in creating relations Element Graphs –Generated from the DTD graph –Created by doing a DFS from an element node

Creating a Relation Given an element graph, the root it made into a relation with all descendents inlined into it, except: 1)Children directly below a “*” are made into separate relations; 2)Each node with a backpointer edge are made into separate relations. These additional relations are named by their path from the root and have parentID fields that serve as foreign keys (e.g. Article.author has the attribute article.author.parentID)

Problems with Basic Large number of relations it creates Not efficient for certain queries –Good: “list all authors of books” –Bad: “list all authors having first name Jack”

Shared Inlining Technique Idea: Identify commonly used element nodes and share them by creating separate relations for them.

Shared Inlining Technique Rules for creating relations: –Nodes with in-degree>1 have relations made –Nodes with in-degree=1 are inlined –Nodes with in-degree=0 have relations made –Nodes following “*” have relations made –Nodes with in-degree=1 AND mutually recurive, one of them is made into a relation

Shared Inlining Technique Rules for designing the schema: –Relation X inlines all nodes Y that it an reach such that the path from X to Y does not contain a node that is to be made a separate relation. –Inlined elements are flagged as being a root with the isRoot field.

Problems with Shared Too many joins required!

Hybrid Inlining Technique Same as Shared except Hybrid also inlines elements that… –have in-degree>1 AND –are not recursive AND –are not reached through a “*” node.

Evaluation Metric For path expressions of length N, data was gathered on: The avg number of SQL queries generated The avg number of joins in each SQL query The total average number of joins in order to process the path expression

Results for N=3 For Basic, 1/3 of the DTDs tests didn’t run to completion due to lack of virtual memory. Basic is thus ignored.

Results for N=3

Group 1: Hybrid reduce join/query, increases a smaller amount of queries => Hybrid requires fewer joins than shared. Group 2: Hybrid reduces join/query, increases a comparable amount of queries=> Hybrid and Shared are the same.

Results for N=3 Group 3: Hybrid reduces some joins/query, but increased the queries by a lot => Hybrid generates more joins than Shared. Hybrid and Shared performed similarly in both joins/query and # of queries => Hybrid and Shared are about the same.

Semi-Structured Queries to SQL Semi-structured query languages –Allow path expressions with various operators and wildcards. XML-QL QueryLorel

Simple Path to SQL 1.The relations corresponding to the start of the root path is added to the FROM clause. 2.If needed, the path expressions are translated to joins.

Simple Recursive Path to SQL 1.Find initialization of the recursion (e.g. *.monograph.editor with condition monograph.title= “Subclass Cirripedia”) 2.Find the actual recursive path expression (e.g. monograph.editor) 3.Union the two

Arbitrary Path to Simple Recursive Path Use a general technique to translate path expressions to many simple (recursive) path expressions.

Relational Results to XML: Simple Structuring Requires only attaching appropriate tags to each tuple.

Relational Results to XML: Tag Variables Have the relational query contain the tag value in the result tuple. Then just covert it to a tag during XML generation.

Grouping a)Could sort the result tuples by the group- by field and and scan through it in order when generating the XML. b)Could do a grouping operation.

Other Cases Complex Element Construction –e.g. asking for all article elements and assume that may be multiple elements (e.g. author & title) –Difficult to do in traditional relational model. Heterogeneous Results –e.g. asking for either title or author of article. –Could be done in two queries and then merged.

Other Cases Nested Queries –Could be rewritten in terms of SQL queries using outer joins.

Conclusion Suggested modifications to relational systems: Untyped/variable-typed references. Information retrieval style indices Flexible comparison operators Multiple-query optimization/execution More powerful recursion support.