1. XML Data Management 8. XQuery Werner Nutt

2. Requirements for an XML Query Language David Maier, W3C XML Query Requirements: Closedness: output must be XML Composability: wherever a set of XML elements is required, a subquery is allowed as well Support for key operations: –selection –extraction, projection –restructuring –combination, join –fusion of elements

3. Requirements for an XML Query Language Can benefit from a schema, but should also be applicable without Retains the order of nodes Formal semantics: –structure of results should be derivable from query –defines equivalence of queries Queries should be representable in XML documents can have embedded queries

4. How Does One Design a Query Language? In most query languages, there are two aspects to a query: –Retrieving data (e.g., from … where … in SQL) –Creating output (e.g., select … in SQL) Retrieval consists of –Pattern matching (e.g., from … ) –Filtering (e.g., where … ) … although these cannot always be clearly distinguished

5. XQuery Principles Data Model identical with the XPath data model –documents are ordered, labeled trees –nodes have identity –nodes can have simple or complex types (defined in XML Schema) A query result is an ordered list/sequence of items (nodes, values, attributes, etc., but not lists) –special case: the empty list ()

6. XQuery Principles (cntd) XQuery can be used without schemas, but can be checked against DTDs and XML schemas XQuery is a functional language –no statements –evaluation of expressions –function definitions –modules

7. The Recipes DTD (Reminder) <!ATTLIST ingredient name CDATA #REQUIRED amount CDATA #IMPLIED unit CDATA #IMPLIED> <!ATTLIST nutrition calories CDATA #REQUIRED fat CDATA #REQUIRED>

8. {for $r in doc("recipes.xml")//recipe return $r/title} returns Beef Parmesan with Garlic Angel Hair Pasta Ricotta Pie … A Query over the Recipes Document

9. Query Features XPath {for $r in doc("recipes.xml")//recipe return $r/title} doc(String) returns input document Part to be returned as it is given{To be evaluated} Iteration $var - variables Sequence of results, one for each variable binding

10. An Equivalent Stylesheet Template

11. Features: Summary The result is a new XML document A query consists of parts that are returned as is... and others that are evaluated (everything in {...} ) Calling the function doc( String ) returns an input document XPath is used to retrieve node sets and values Iteration over node sets: for binds a variable to all nodes in a node set Variables can be used in XPath expressions return returns a sequence of results, one for each binding of a variable

12. XPath is a Fragment of XQuery doc("recipes.xml")//recipe[1]/title returns Beef Parmesan with Garlic Angel Hair Pasta doc("recipes.xml")//recipe[position()<=3] /title returns Beef Parmesan with Garlic Angel Hair Pasta, Ricotta Pie, Linguine alla Pescadora an element a list of elements

13. Beware: Attributes in XPath doc("recipes.xml")//recipe[1]/ingredient[1] /@name attribute name {"beef cube steak"} string(doc("recipes.xml")//recipe[1] /ingredient[1]/@name) "beef cube steak" an attribute, represented as a constructor for an attribute node (not in Saxon) a value of type string

14. Beware: Attributes in XPath (cntd.) {string(doc("recipes.xml")//recipe[1] /ingredient[1]/@name)} beef cube steak an element with string content

15. Beware: Attributes in XPath (cntd.) {doc("recipes.xml")//recipe[1] /ingredient[1]/@name} an element with an attribute Note: The XML that we write down is only the surface structure of the data model that is underlying XQuery

16. Beware: Attributes in XPath (cntd.) <first-ingredient oldName="{doc("recipes.xml")//recipe[1] /ingredient[1]/@name}"> Beef Beef An attribute is cast as a string

17. Constructor Syntax For all constituents of documents, there are constructors element first-ingredient { attribute oldName {string(doc("recipes.xml")//recipe[1] /ingredient[1]/@name)}, "Beef" } equivalent to the notation on the previous slide attribute constructorelement constructor

18. Iteration with the For-Clause Syntax: for $ var in xpath-expr Example: for $r in doc("recipes.xml")//recipe return string($r) The expression creates a list of bindings for a variable $ var If $ var occurs in an expression exp, then exp is evaluated for each binding For-clauses can be nested: for $r in doc("recipes.xml")//recipe for $v in doc("vegetables.xml")//vegetable return...

19. What Does This Return? for $i in (1,2,3) for $j in (1,2,3) return element {concat("x",$i * $j)} {$i * $j}

20. Nested For-clauses: Example {for $r in doc("recipes.xml")//recipe return {for $i in $r//ingredient return {string($i/@name)} } } Returns my-recipes with titles as attributes and my-ingredients with names as text content

21. The Equivalent Stylesheet Template

22. The Let Clause Syntax: let $ var := xpath-expr binds variable $ var to a list of nodes, with the nodes in document order does not iterate over the list allows one to keep intermediate results for reuse (not possible in SQL) Example: let $oorecps := doc("recipes.xml")//recipe [.//ingredient/@name="olive oil"]

23. Let Clause: Example {let $oorecps := doc("recipes.xml")//recipe [.//ingredient/@name="olive oil"] for $r in $oorecps return {$r/title/text()} {": "} {string($r/nutrition/@calories)} } Calories of recipes with olive oil Note the implicit string concatenation

24. Let Clause: Example (cntd.) The query returns: Beef Parmesan: 1167 Linguine alla Pescadora: 532

25. The Where Clause Syntax: where occurs before return clause similar to predicates in XPath comparisons on nodes: = for node equality > for document order Example: for $r in doc("recipes.xml")//recipe where $r//ingredient/@name="olive oil" return...

26. Quantifiers Syntax: some / every $ var in satisfies $ var is bound to all nodes in Test succeeds if is true for some/every binding Note: if is empty, then some is false and all is true

27. Quantifiers (Example) Recipes that have some compound ingredient Recipes where every top level ingredient is non-compound for $r in doc("recipes.xml")//recipe where some $i in $r/ingredient satisfies $i/ingredient return $r/title for $r in doc("recipes.xml")//recipe where every $i in $r/ingredient satisfies not($i/ingredient) return $r/title

28. Element Fusion To every recipe, add the attribute calories! {let $rs := doc("recipes.xml")//recipe for $r in $rs return {$r/nutrition/@calories} {$r/title} } an elementan attribute

29. Element Fusion (cntd.) The query result: Beef Parmesan with Garlic Angel Hair Pasta Ricotta Pie Linguine Pescadoro Zuppa Inglese Cailles en Sarcophages

30. Fusion with Mixed Syntax We mix constructor and XML–Syntax: element result {let $rs := doc("recipes.xml")//recipe for $r in $rs return {attribute calories {$r/nutrition/@calories}} {$r/title} }

31. The Same with Constructor Syntax Only element result {let $rs := doc("recipes.xml")//recipe for $r in $rs return element recipe { attribute calories {$r/nutrition/@calories}, $r/title }

32. Join condition Pair every ingredient with the recipes where it is used! let $rs := doc("recipes.xml")//recipe for $i in $rs//ingredient for $r in $rs where $r//ingredient/@name=$i/@name return {$i/@name} {$r/title} Join

33. Join (cntd.) The query result: Beef Parmesan with Garlic Angel Hair Pasta, Beef Parmesan with Garlic Angel Hair Pasta, Beef Parmesan with Garlic Angel Hair Pasta

34. Join Exercise Return all pairs of ingredients such that the ingredients have the same name, but occur with different amounts and return the recipes where each of them is used together with the amount being used in those recipes, while returning every pair only once. Could a query for these ingredients be expressed in XPath?

35. Join condition Document Inversion For every ingredient, return all the recipes where it is used! {let $rs := doc("recipes.xml")//recipe for $i in $rs//ingredient return {$i/@*} {$rs[.//ingredient/@name=$i/@name]/title} }

36. Document Inversion (cntd.) The query result: Zuppa Inglese … Beef Parmesan with Garlic Angel Hair Pasta Linguine Pescadoro …

37. Eliminating Duplicates The function distinct-values( Node Set ) –extracts the values of a sequence of nodes –creates a duplicate free list of values Note the coercion: nodes are cast as values! Example: let $rs := doc("recipes.xml")//recipe return distinct-values($rs//ingredient/@name) yields xdt:untypedAtomic("beef cube steak"), xdt:untypedAtomic("onion, sliced into thin rings"),... by the Galax engine

38. Avoiding Multiple Results in a Join We want that every ingredient is listed only once: Eliminate duplicates using distinct-values! {let $rs := doc("recipes.xml")//recipe for $in in distinct-values( $rs//ingredient/@name) return {$rs[.//ingredient/@name=$in]/title} }

39. Avoiding Multiple Results (cntd.) The query result: Beef Parmesan with Garlic Angel Hair Pasta Beef Parmesan with Garlic Angel Hair Pasta... Linguine Pescadoro Cailles en Sarcophages...

40. Syntax: order by expr [ ascending | descending ] for $iname in doc("recipes.xml")//@name order by $iname descending return string($iname) yields "whole peppercorns", "whole baby clams", "white sugar",... The Order By Clause

41. The Order By Clause (cntd.) let $rs := doc("recipes.xml")//@name for $r in $rs order by $r/nutrition/@calories return $r/title In which order will the titles come?

42. The Order By Clause (cntd.) The interpreter must be told whether the values should be regarded as numbers or as strings (alphanumerical sorting is default) for $r in $rs order by number($r/nutrition/@calories) return $r/title Note: –The query returns titles... –but the ordering is according to calories, which do not appear in the output Also possible in SQL! What if combined with distinct-values?

43. FLWOR Expresssions (pronounced flower) We have now seen the main ingredients of XQuery: For and Let clauses, which can be mixed a Where clause imposing conditions an Order by clause, which determines the order of results a Return clause, which constructs the output. Combining these yields FLWOR expressions.

44. Conditionals if (expr) then expr else expr Example let $is := doc("recipes.xml")//ingredient for $i in $is[not(ingredient)] let $u := if (not($i/@unit)) then attribute unit {"pieces"} else () creates an attribute unit="pieces" if none exists and an empty item list otherwise

45. We use the conditional to construct variants of ingredients: let $is := doc("recipes.xml")//ingredient for $i in $is[not(ingredient)] let $u := if (not($i/@unit)) then attribute {"unit"} {"pieces"} else () return {$i/@* | $u} Conditionals (cntd.) Collects all attributes in a list and adds a unit if needed

46. Conditionals (cntd.) The query result:,..., …

47. Exercises Write queries that produce A list, containing for every recipe the recipe's title element and an element with the number of calories The same, ordered according to calories The same, alphabetically ordered according to title The same, ordered according to the fat content The same, with title as attribute and calories as content. A list, containing for every recipe the top level ingredients, dropping the lower level ingredients

48. Sample Solution 1 A list, containing for every recipe the recipe's title element and an element with the number of calories {for $r in doc("recipes.xml")//recipe return ($r/title, {number($r//@calories)} ) } The results returned are 2-element lists. The list constructor is (.,. )

49. Sample Solution 6 {for $r in doc("recipes.xml")//recipe return {attribute title {$r/title}, for $i in $r/ingredient return if (not($i/ingredient)) then $i else {$i/@*} } }

50. Aggregation Aggregation functions count, sum, avg, min, max Example: The number of recipes with olive oil let $doc := doc("recipes.xml) return {count($doc//recipe [.//ingredient/@name = "olive oil"])}

51. Grouping and Aggregation For each recipe, the number of simple ingredients for $r in doc("recipes.xml")//recipe return {attribute title {$r/title/text()}} {count($r//ingredient[not(ingredient)])}

52. Grouping and Aggregation (cntd.) The query result: 11, 12, 15, 8, 30

53. Grouping and Aggregation (cntd.) A list, containing for every ingredient, the number of occurrences of that ingredient let $d := doc("recipes.xml") let $is := distinct-values($d//ingredient/@name) return {for $i in $is order by $i return {count($d//ingredient[@name=$i])} }

54. Nested Aggregation The recipe with the maximal number of calories! let $rs := doc("recipes.xml")//recipe let $maxCal := max($rs//@calories) for $r in $rs where $r//@calories = $maxCal return string($r/title) returns "Cailles en Sarcophages"

55. User-defined Functions declare function local:fac($n as xs:integer) as xs:integer { if ($n = 0) then 1 else $n * local:fac($n - 1) }; local:fac(10) Function Declaration Function Call

56. Example: Nested Ingredients declare function local:nest($n as xs:integer, content as xs:string) as element() { if ($n = 0) then element ingredient{$content} else element ingredient{local:nest($n - 1,$content)} }; local:nest(3,"Stuff")

57. What Does this Function Return? declare function local:depth($n as node()) as xs:integer { if (fn:empty($n/*)) then 1 else let $cdepths := for $c in $n/* return local:depth($c) return fn:max($cdepths) + 1 };

58. Exercise Write a function local:element-copy that takes as input a node (= XML tree) produces as output a copy of the tree, but without the attributes