1. 1 XQuery to XAT Xin Zhang

2. 2 Outline XAT Data Model. XAT Operator Design. XQuery Block Identification. Equivalent Rewriting Rules. Computation Pushdown Navigation Pushdown Groupby Operator Simplification

3. 3 Data Model An Ordered Table in two dimensions Tuple order Column order. Every cell has its own domain, e.g.: SQL domains. XML Fragment. Can be a list of XML elements. Every column binds to one variable. Comparison are done by values Note: When values are “handles”, the comparison are done by deference of handles.

4. 4 Data Model Examples Table of XML Fragments. Table Types: Regular Relations. Table with XML Elements. Table with XML Fragments. Table with Variable Binding. Table with Path Navigation. $carrier </carrier invoice_idcarrier carrier_entry carriers $carrier </carrier $carrier ………. //invoice/invoice/account_number $rate

5. 5 Column Names A String: “name” A Variable Binding: “$var” Operators with their parameters: “op(p 1, p 2,..., p n )” A XPath with Entry Point Notation. “/”, “/invoice”, “/invoice/book” “invoice:/”, “book:invoice:/”

6. 6 Operators SQL like (9): Project, Select, Join (Theta, Outer, Semi), Groupby, Orderby, Union (Node, Outer), CO XML like (3): Tagger Navigate Aggregate: Groupby without by-column. Special (5): SQL, Function, Source, Name, FOR

7. 7 SQL like Operators (9) OperatorSyntaxDescription ProjectPi(col + )[s]Project out multiple columns from source s. SelectTheta(c)[s]Filter source s by condition c. Theta JoinJoin(c)[l, r]Join two sources l and r under condition c. Outer JoinLOJ(c)[l, r] ROJ(c)[l, r] Left (right) outer join sources l and r by condition c. Semi JoinLSJ(c)[l, r] RSJ(c)[l, r] Left (right) semi join sources l and r by condition c. GroupbyGB(col +, F (col) + )[s]Groupby multiple columns by multiple aggregation functions F() of columns over source s. OrderbyOB(col + )[s]Sort source s by multiple columns. UnionU[s + ]Union multiple sources together. Outer UnionOU[s + ]Outer union multiple sources together. COpCOp(col+, Op)[s 1, s 2 ]Correlated Operator on columns col+. s 1 is outer query, s 2 is inner query.

8. 8 XML like Operators OperatorSyntaxDescription TaggerTag(p)[s]Taggering source s by pattern p. NavigateNav(path)[s]Navigate from source s through a XPath. AggregateAgg(F (col) + )[s] Aggregate source s by multiple aggregate functions F() of columsn over source s.

9. 9 Special Operators OperatorParametersDescription SQLSQL(stmt)[s + ]One SQL query statement stmt over multiple sources. FunctionF(param + )[s + ]User defined function over multiple sources with multiple parameters. Sources(desc)Identify a data source by description desc. NameRho(col 1, col 2 )[s] Rho(s 2 )[s 1 ] Rename column col 1 of source s into name col 2. Rename source s 1 into s 2. FORFOR(col + )[s 1, s 2 ]FOR operator iterate over sources s 1 and execute subquery s 2 with variable binding columns col 1..n.

10. 10 Project Pi(col 1..n )[s] Input: table s Output: table s Logic: Same as SQL. Order Handling: Keep original tuple order, the schema order is reordered as the col 1..n in the project operator. Requirement: The col 1..n should be in source s.

11. 11 Select Theta(c)[s] Input: table s Output: table s Logic: Same as SQL. Order Handling: Keep original tuple order, keep original schema order. Requirement: Condition c should be only reference to the source s.

12. 12 Theta Join Join(c)[l, r] Input: table l, and table r. Output: One table (with temporary table name) Logic: Same as SQL. Order Handling: The schema order of the output table is columns of table l followed by the columns of table r. The tuple order of the output table is iteration of tuples in r over the iteration of tuples in l, e.g., {,,, } Requirement: Condition c should be relates to both tables l and r.

13. 13 Outer Join LOJ(c)[l, r] Input: table l, and table r. Output: One table (with temporary table name) Logic: Same as SQL. Order Handling: The schema order of the output table is columns of table l followed by the columns of table r. The tuple order of the output table is iteration of tuples in r over the iteration of tuples in l, e.g., {,,,, } Requirement: Condition c should be relates to both tables l and r.

14. 14 Outer Join ROJ(c)[l, r] Input: table l, and table r. Output: One table (with temporary table name) Logic: Same as SQL. (Similar to LOJ) Order Handling: The schema order of the output table is columns of table l followed by the columns of table r. The tuple order of the output table is iteration of tuples in l over the iteration of tuples in r, e.g.,{,,,,, }, “null” is at the beginning of the output. Requirement: Condition c should be relates to both tables l and r.

15. 15 Semi Join LSJ(c)[l, r] Input: table l, and table r. Output: table l. Logic: Same as SQL. Order Handling: The schema order of the output table same as table l. The tuple order of the output table is same as table l. Requirement: Condition c should be relates to both tables l and r.

16. 16 Semi Join RSJ(c)[l, r] Input: table l, and table r. Output: table r. Logic: Same as SQL. Order Handling: The schema order of the output table same as table r. The tuple order of the output table is same as table r. Requirement: Condition c should be relates to both tables l and r.

17. 17 Groupby GB(col 1..n, F 1..m (col))[s] Input: table s. Output: table s. Logic: Same as SQL. Order Handling: The schema order of the output table is col 1..n followed by F 1..m (col). F 1..m (col) can be nested operators, e.g., a subquery. The tuple order of the output table is same as table s. Requirement: col 1..n and all the col in the F 1..m should be in table s.

18. 18 Groupby Example Input: S (a, b, c) Operator: GB (b, a, avg(c), count(c)) Output: S (b, a, “avg(c)”, “count(c)”)

19. 19 Orderby OB(col 1..n )[s] Input: table s. Output: table s. Logic: Same as SQL. Order Handling: The schema order of the output table is same as table s. The tuple order of the output table is as specified. Requirement: col 1..n should be in table s.

20. 20 Union U[s 1..n ] Input: Multiple tables s 1..n. Output: One table (with temporary name). Logic: Same as SQL. Order Handling: The schema order of the output table is same as table s 1. The tuple order of the output table is in the order of table s 1..n. Requirement: All tables s 1..n have same schema.

21. 21 Outer Union OU[s 1..n ] Input: Multiple tables s 1..n. Output: One table (with temporary name). Logic: Same as SQL. Order Handling: The schema order of the output table is un-decidable, it depends on implementation. The schema order should be ensured by another projection node. The tuple order of the output table is in the order of table s 1..n. Requirement: N/A.

22. 22 Tagger Tag(p)[s] Input: Table s. Output: Table s. Logic: One additional column is added with tagged information. Pattern p is only one level. Order Handling: The tagged column is added to the end. The tuple order of the output table is same as table s. Requirement: The columns used in pattern p should be in table s.

23. 23 Navigate Nav(path)[s] Input: Table s. Output: Table s. Logic: One additional column is added with navigation information. Tuples are multiplied if there are more than one results in the navigation. If the navigation result is empty, put NULL in the new column. Order Handling: The navigation column is added to the end. The tuple order of the output table is same as table s and the navigation order. Requirement: N/A

24. 24 Aggregate Agg(F 1..m (col))[s] Input: table s. Output: table s. Logic: Merge all tuples in that table into one, and apply functions on those columns. If there is no functions, then just merge all the content. Order Handling: The schema order of the output table is F 1..m (col). There is only one tuple. Requirement: All the col in the F 1..m should be in table s.

25. 25 SQL SQL(stmt)[s 1..n ] Input: Multiple tables s 1..n. Output: Temporary table. Logic: Execute stmt over the multiple tables and output the result. It is assumed to be executed by a RDB engine. Usually, it’s the operator right above the source (e.g., table) operator. Order Handling: The schema order of the output table is depends on the underlying implementation. The schema order can be reconfirmed by additional projection node. The tuple order is un-decidable. The tuple order can be reconfirmed by additional orderby node. Requirement: N/A.

26. 26 Function F(param 1..m )[s 1..n ] Input: Multiple tables s 1..n. Output: Temporary table. Logic: Execute some user defined function on the data sources. Or used to represent a recursive query. Order Handling: Schema and tuple orders are depends on the implementation. They can be reconfirmed by projection and orderby nodes. Requirement: N/A.

27. 27 Source s(desc) Input: N/A Output: A table with a given name. Logic: Identify following sources: view, xml document, or a table. Order Handling: Depends on the implementation. Keep original schema and tuple order as much as possible. Requirement: N/A.

28. 28 Name Rho(col 1, col 2 )[s] Input: Table s. Output: Table s. Logic: Rename col 1 in table s into col 2. Order Handling: Keep all the schema and tuple orders. Requirement: Col 1 in table s.

29. 29 Name Rho(s 2 )[s 1 ] Input: Table s 1. Output: Table s 2. Logic: Rename table s 1 to table s 2. Order Handling: Keep all the schema and tuple orders. Requirement: N/A.

30. 30 Correlated Ouput FOR(col + )[s 1, s 2 ] Input: Tables s 1 and s 2. Output: Evaluation of subquery s 2 for each tuple in subquery s 1.. Logic: It’s a FOR iteration operator. For value in the columns col + of table s 1, evaluate the sub-query that generates the table s 2. Order Handling: Schema order is output table s 2. Tuple order is similar to the join operator without the left part. Requirement: N/A.

31. 31 Steps in Translation XQuery  XML Algebra Tree User View  XML Algebra Tree View Composition Computation Pushdown Optimization Execution

32. 32 <!DOCTYPE invoice [ <!ELEMENT invoice (account_number, bill_period, carrier+, itemized_call*, total)> <!ATTLIST itemized_call no ID #REQUIRED date CDATA #REQUIRED number_called CDATA #REQUIRED time CDATA #REQUIRED rate (NIGHT|DAY) #REQUIRED min CDATA #REQUIRED amount CDATA #REQUIRED> ]> 555 777-3158 573 234 3 Jun 9 - Jul 8, 2000 Sprint $0.35 Example of Telephone Bill

33. 33 Example XQuery User XQuery: { FOR $rate IN distinct(document(“invoice”)/invoice/itemized_call@rate) LET $itemized_call := document(“invoice”)/invoice/itemized_call[@rate=$rate] WHERE $itemized_call/@number_called LIKE ‘973%’ RETURN $rate count($itemized_call) } Count number of itemized_calls in calling area 973 grouped by the calling rate.

34. 34 XQuery  XML Algebra Tree Translate XQuery into XAT by grammar. Convert each query block into XAT. Identify correlated operators. Identify query blocks. Query decorrelation.

35. 35 XAT Graph Notation Unordered Graph. Nodes: Operators with its parameters. If there is only one source name, we ignore it. Blocks (subqueries) We can use block name as the alias of the table name out of that block. Terminals V3:=Tagger( [V2] ) B2

36. 36 XAT Example Select(count(“$itemized_call”)) Navigate(“$itemized_call”, @number_called) Select(“@number_called:$itemized_call” like ‘973%’) T2 := Source(“invoice.xml”) $itemized_call := Navigate(“/”, invoice/itemized_call) Select(“@rate:$itemized_call” = “$rate”) V2 := Tagger( [V1] ) $rate := Select(distinct(“invoice/itemized_call/@rate:/”)) T1 := Source(“invoice.xml”) Navigate(“/”, invoice/itemized_call/@rate) Navigate(“$itemized_call”, @rate) FOR($rate) Aggregate V1:=Tagger( [$rate] [count($itemized_call)] )

37. 37 XQuery Block Identification Every query block has only one input point and one output point. Potential Query Block Separation Point: Independent sources. Correlated Operators. Block is used for query optimization, e.g., cutting.

38. 38 Identification of Blocks Select(count(“$itemized_call”)) Navigate(“$itemized_call”, @number_called) Select(“@number_called:$itemized_call” like ‘973%’) T2 := Source(“invoice.xml”) $itemized_call := Navigate(“/”, invoice/itemized_call) Select(“@rate:$itemized_call” = “$rate”) V3 := Tagger( [V1] ) $rate := Select(distinct(“invoice/itemized_call/@rate:/”)) T1 := Source(“invoice.xml”) Navigate(“/”, invoice/itemized_call/@rate) Navigate(“$itemized_call”, @rate) B1 B2 B3 FOR($rate) Aggregate V1:=Tagger( [$rate] [count($itemized_call)] ) B4

39. 39 XAT Block Tree B1 B2 B3 B4

40. 40 Equivalent Rewriting Rules Navigation Pushdown Swap navigation operator down. Computation Pushdown Swap SQL operator down. Groupby Operator Simplification Pull functions (subqueries) out of Groupby function.