XML Data Management 8. XQuery Werner Nutt

Requirements for an XML Query Language David Maier, W3C XML Query Requirements: Closedness: output must be XML Composability: wherever a set of XML elements is required, a subquery is allowed as well Support for key operations: –selection –extraction, projection –restructuring –combination, join –fusion of elements

Requirements for an XML Query Language Can benefit from a schema, but should also be applicable without Retains the order of nodes Formal semantics: –structure of results should be derivable from query –defines equivalence of queries Queries should be representable in XML documents can have embedded queries

How Does One Design a Query Language? In most query languages, there are two aspects to a query: –Retrieving data (e.g., from … where … in SQL) –Creating output (e.g., select … in SQL) Retrieval consists of –Pattern matching (e.g., from … ) –Filtering (e.g., where … ) … although these cannot always be clearly distinguished

XQuery Principles Data Model identical with the XPath data model –documents are ordered, labeled trees –nodes have identity –nodes can have simple or complex types (defined in XML Schema) A query result is an ordered list/sequence of items (nodes, values, attributes, etc., but not lists) –special case: the empty list ()

XQuery Principles (cntd) XQuery can be used without schemas, but can be checked against DTDs and XML schemas XQuery is a functional language –no statements –evaluation of expressions –function definitions –modules

The Recipes DTD (Reminder) <!ATTLIST ingredient name CDATA #REQUIRED amount CDATA #IMPLIED unit CDATA #IMPLIED> <!ATTLIST nutrition calories CDATA #REQUIRED fat CDATA #REQUIRED>

{for $r in doc("recipes.xml")//recipe return $r/title} returns Beef Parmesan with Garlic Angel Hair Pasta Ricotta Pie … A Query over the Recipes Document

Query Features XPath {for $r in doc("recipes.xml")//recipe return $r/title} doc(String) returns input document Part to be returned as it is given{To be evaluated} Iteration $var - variables Sequence of results, one for each variable binding

An Equivalent Stylesheet Template

Features: Summary The result is a new XML document A query consists of parts that are returned as is... and others that are evaluated (everything in {...} ) Calling the function doc( String ) returns an input document XPath is used to retrieve node sets and values Iteration over node sets: for binds a variable to all nodes in a node set Variables can be used in XPath expressions return returns a sequence of results, one for each binding of a variable

XPath is a Fragment of XQuery doc("recipes.xml")//recipe[1]/title returns Beef Parmesan with Garlic Angel Hair Pasta doc("recipes.xml")//recipe[position()<=3] /title returns Beef Parmesan with Garlic Angel Hair Pasta, Ricotta Pie, Linguine alla Pescadora an element a list of elements

Beware: Attributes in XPath doc("recipes.xml")//recipe[1]/ingredient[1] attribute name {"beef cube steak"} string(doc("recipes.xml")//recipe[1] "beef cube steak" an attribute, represented as a constructor for an attribute node (not in Saxon) a value of type string

Beware: Attributes in XPath (cntd.) {string(doc("recipes.xml")//recipe[1] beef cube steak an element with string content

Beware: Attributes in XPath (cntd.) {doc("recipes.xml")//recipe[1] an element with an attribute Note: The XML that we write down is only the surface structure of the data model that is underlying XQuery

Beware: Attributes in XPath (cntd.) <first-ingredient oldName="{doc("recipes.xml")//recipe[1] Beef Beef An attribute is cast as a string

Constructor Syntax For all constituents of documents, there are constructors element first-ingredient { attribute oldName {string(doc("recipes.xml")//recipe[1] "Beef" } equivalent to the notation on the previous slide attribute constructorelement constructor

Iteration with the For-Clause Syntax: for $ var in xpath-expr Example: for $r in doc("recipes.xml")//recipe return string($r) The expression creates a list of bindings for a variable $ var If $ var occurs in an expression exp, then exp is evaluated for each binding For-clauses can be nested: for $r in doc("recipes.xml")//recipe for $v in doc("vegetables.xml")//vegetable return...

What Does This Return? for $i in (1,2,3) for $j in (1,2,3) return element {concat("x",$i * $j)} {$i * $j}

Nested For-clauses: Example {for $r in doc("recipes.xml")//recipe return {for $i in $r//ingredient return } } Returns my-recipes with titles as attributes and my-ingredients with names as text content

The Equivalent Stylesheet Template

The Let Clause Syntax: let $ var := xpath-expr binds variable $ var to a list of nodes, with the nodes in document order does not iterate over the list allows one to keep intermediate results for reuse (not possible in SQL) Example: let $oorecps := doc("recipes.xml")//recipe oil"]

Let Clause: Example {let $oorecps := doc("recipes.xml")//recipe oil"] for $r in $oorecps return {$r/title/text()} {": "} } Calories of recipes with olive oil Note the implicit string concatenation

Let Clause: Example (cntd.) The query returns: Beef Parmesan: 1167 Linguine alla Pescadora: 532

The Where Clause Syntax: where occurs before return clause similar to predicates in XPath comparisons on nodes: = for node equality > for document order Example: for $r in doc("recipes.xml")//recipe where oil" return...

Quantifiers Syntax: some / every $ var in satisfies $ var is bound to all nodes in Test succeeds if is true for some/every binding Note: if is empty, then some is false and all is true

Quantifiers (Example) Recipes that have some compound ingredient Recipes where every top level ingredient is non-compound for $r in doc("recipes.xml")//recipe where some $i in $r/ingredient satisfies $i/ingredient return $r/title for $r in doc("recipes.xml")//recipe where every $i in $r/ingredient satisfies not($i/ingredient) return $r/title

Element Fusion To every recipe, add the attribute calories! {let $rs := doc("recipes.xml")//recipe for $r in $rs return {$r/title} } an elementan attribute

Element Fusion (cntd.) The query result: Beef Parmesan with Garlic Angel Hair Pasta Ricotta Pie Linguine Pescadoro Zuppa Inglese Cailles en Sarcophages

Fusion with Mixed Syntax We mix constructor and XML–Syntax: element result {let $rs := doc("recipes.xml")//recipe for $r in $rs return {attribute calories {$r/title} }

The Same with Constructor Syntax Only element result {let $rs := doc("recipes.xml")//recipe for $r in $rs return element recipe { attribute calories $r/title }

Join condition Pair every ingredient with the recipes where it is used! let $rs := doc("recipes.xml")//recipe for $i in $rs//ingredient for $r in $rs where return {$r/title} Join

Join (cntd.) The query result: Beef Parmesan with Garlic Angel Hair Pasta, Beef Parmesan with Garlic Angel Hair Pasta, Beef Parmesan with Garlic Angel Hair Pasta

Join Exercise Return all pairs of ingredients such that the ingredients have the same name, but occur with different amounts and return the recipes where each of them is used together with the amount being used in those recipes, while returning every pair only once. Could a query for these ingredients be expressed in XPath?

Join condition Document Inversion For every ingredient, return all the recipes where it is used! {let $rs := doc("recipes.xml")//recipe for $i in $rs//ingredient return }

Document Inversion (cntd.) The query result: Zuppa Inglese … Beef Parmesan with Garlic Angel Hair Pasta Linguine Pescadoro …

Eliminating Duplicates The function distinct-values( Node Set ) –extracts the values of a sequence of nodes –creates a duplicate free list of values Note the coercion: nodes are cast as values! Example: let $rs := doc("recipes.xml")//recipe return yields xdt:untypedAtomic("beef cube steak"), xdt:untypedAtomic("onion, sliced into thin rings"),... by the Galax engine

Avoiding Multiple Results in a Join We want that every ingredient is listed only once: Eliminate duplicates using distinct-values! {let $rs := doc("recipes.xml")//recipe for $in in distinct-values( return }

Avoiding Multiple Results (cntd.) The query result: Beef Parmesan with Garlic Angel Hair Pasta Beef Parmesan with Garlic Angel Hair Pasta... Linguine Pescadoro Cailles en Sarcophages...

Syntax: order by expr [ ascending | descending ] for $iname in order by $iname descending return string($iname) yields "whole peppercorns", "whole baby clams", "white sugar",... The Order By Clause

The Order By Clause (cntd.) let $rs := for $r in $rs order by return $r/title In which order will the titles come?

The Order By Clause (cntd.) The interpreter must be told whether the values should be regarded as numbers or as strings (alphanumerical sorting is default) for $r in $rs order by return $r/title Note: –The query returns titles... –but the ordering is according to calories, which do not appear in the output Also possible in SQL! What if combined with distinct-values?

FLWOR Expresssions (pronounced flower) We have now seen the main ingredients of XQuery: For and Let clauses, which can be mixed a Where clause imposing conditions an Order by clause, which determines the order of results a Return clause, which constructs the output. Combining these yields FLWOR expressions.

Conditionals if (expr) then expr else expr Example let $is := doc("recipes.xml")//ingredient for $i in $is[not(ingredient)] let $u := if then attribute unit {"pieces"} else () creates an attribute unit="pieces" if none exists and an empty item list otherwise

We use the conditional to construct variants of ingredients: let $is := doc("recipes.xml")//ingredient for $i in $is[not(ingredient)] let $u := if then attribute {"unit"} {"pieces"} else () return | $u} Conditionals (cntd.) Collects all attributes in a list and adds a unit if needed

Conditionals (cntd.) The query result:,..., …

Exercises Write queries that produce A list, containing for every recipe the recipe's title element and an element with the number of calories The same, ordered according to calories The same, alphabetically ordered according to title The same, ordered according to the fat content The same, with title as attribute and calories as content. A list, containing for every recipe the top level ingredients, dropping the lower level ingredients

Sample Solution 1 A list, containing for every recipe the recipe's title element and an element with the number of calories {for $r in doc("recipes.xml")//recipe return ($r/title, ) } The results returned are 2-element lists. The list constructor is (.,. )

Sample Solution 6 {for $r in doc("recipes.xml")//recipe return {attribute title {$r/title}, for $i in $r/ingredient return if (not($i/ingredient)) then $i else } }

Aggregation Aggregation functions count, sum, avg, min, max Example: The number of recipes with olive oil let $doc := doc("recipes.xml) return {count($doc//recipe = "olive oil"])}

Grouping and Aggregation For each recipe, the number of simple ingredients for $r in doc("recipes.xml")//recipe return {attribute title {$r/title/text()}} {count($r//ingredient[not(ingredient)])}

Grouping and Aggregation (cntd.) The query result: 11, 12, 15, 8, 30

Grouping and Aggregation (cntd.) A list, containing for every ingredient, the number of occurrences of that ingredient let $d := doc("recipes.xml") let $is := return {for $i in $is order by $i return }

Nested Aggregation The recipe with the maximal number of calories! let $rs := doc("recipes.xml")//recipe let $maxCal := for $r in $rs where = $maxCal return string($r/title) returns "Cailles en Sarcophages"

User-defined Functions declare function local:fac($n as xs:integer) as xs:integer { if ($n = 0) then 1 else $n * local:fac($n - 1) }; local:fac(10) Function Declaration Function Call

Example: Nested Ingredients declare function local:nest($n as xs:integer, content as xs:string) as element() { if ($n = 0) then element ingredient{$content} else element ingredient{local:nest($n - 1,$content)} }; local:nest(3,"Stuff")

What Does this Function Return? declare function local:depth($n as node()) as xs:integer { if (fn:empty($n/*)) then 1 else let $cdepths := for $c in $n/* return local:depth($c) return fn:max($cdepths) + 1 };

Exercise Write a function local:element-copy that takes as input a node (= XML tree) produces as output a copy of the tree, but without the attributes