Xquery Introduction by Examples

Sources XQuery 1.0: An XML Query LanguageW3C Working Draft 22 August 2003 Don Chamberlin’s Sigmod03 talk: sigmod03_xquery.pdf Xquery from the Experts. Katz et. Al. Definitive XML Schema. Priscilla Walmsley.

Example of an Input Expression This is “onebook.xml”: TCP/IP Illustrated Stevens W. Addison-Wesley 65.95

Expression and Result Expression: document("onebook.xml") Result: document { TCP/IP Illustrated Stevens W. Addison-Wesley }

Bibliography Database TCP/IP Illustrated Stevens W. Addison-Wesley Advanced Programming in the Unix environment Stevens W. Addison-Wesley 65.95

Bibliography Database Data on the Web Abiteboul Serge Buneman Peter Suciu Dan Morgan Kaufmann Publishers 39.95

Bibliography Database The Economics of Technology and Content for Digital TV Gerbarg Darcy CITI Kluwer Academic Publishers

Some Path Expressions Expression: document("xmpbib.xml")//book[2] Result Advanced Programming in the Unix environment Stevens W. Addison-Wesley 65.95

Node Creation Expression: Go West J. Cowen Smith Result: Go West J. Cowen Smith

Element Creation - Constructor Expression: element title { "Harold and the purple crayon" } Result: Harold and the purple crayon

Element Creation - Constructor Expression: element {concat("tit","le")} { "Harold and the purple crayon" } Result: Harold and the purple crayon

Document Creation Expression: document{ Go West J. Cowen Smith } Result: document { Go West J. Cowen Smith }

Node Creation - Computation Expression: here is a query document("b.xml")//book[1]/title { document("../docs/xmpbib.xml")//book[1]/title } Result: here is a query document("b.xml")//book[1]/title TCP/IP Illustrated

FLWOR for $b in document("../docs/xmpbib.xml")//book where = "2000" return $b/title Result: Data on the Web

FLWOR for $i in (1,2,3) return {$i} Result: 1, 2, 3

Let let $i := (1,2,3) return {$i} Result: 1 2 3

Tuple Stream Production for $i in (1,2,3), $j in (4,5,6) return {$i} {$j} Result:next slide

Result of Tuple Stream Production 1 4, 1 5, 1 6, 2 4, 2 5, 2 6, 3 4, 3 5, 3 6

Tuple Stream Production for $i in (1,2,3) let $j := (1,2,3) return {$i} {$j} Result: , ,

FLWOR + Let for $b in document("../docs/xmpbib.xml")//book let $c := $b/author return { $b/title, { count($c) } } Result: Next slide

Result: FLWOR + Let TCP/IP Illustrated 1, Advanced Programming in the Unix environment 1, Data on the Web 3, The Economics of Technology and Content for Digital TV 0

More FLWOR for $b in document("../docs/xmpbib.xml")//book where $b/price < 50 return $b/title Result: Data on the Web

More FLWOR – Let + Count for $b in document("../docs/xmpbib.xml")//book let $c := $b//author where count($c) > 2 return $b/title Result: Data on the Web

More FLWOR – Order for $t in document("../docs/xmpbib.xml")//title order by $t return $t Result: Advanced Programming in the Unix environment, Data on the Web, TCP/IP Illustrated, The Economics of Technology and Content for Digital TV

More FLWOR – Variable Binding for $b in document("../docs/xmpbib.xml")//book, $t in $b/title order by $t return $t Result: Advanced Programming in the Unix environment, Data on the Web, TCP/IP Illustrated, The Economics of Technology and Content for Digital TV

Order Specification for $b in document("../docs/xmpbib.xml")//book, $a in $b/author order by $a/last descending, $a/first ascending return $a Result: Suciu Dan, Stevens W., Buneman Peter, Abiteboul Serge

Ordering without Outputting let $b := document("../docs/xmpbib.xml")//book for $t in distinct-values($b/title) let $a1 := $b[title = $t]/author[1] order by $a1/last, $a1/first return $t Result: Data on the Web, TCP/IP Illustrated, Advanced Programming in the Unix environment, The Economics of Technology and Content for Digital TV

Order Specification – Empty sequences let $b := document("../docs/xmpbib.xml")//book for $t in distinct-values($b/title) let $a1 := $b[title = $t]/author[1] stable order by $a1/last empty least, $a1/first empty least return $t Result: The Economics of Technology and Content for Digital TV, Data on the Web, TCP/IP Illustrated, Advanced Programming in the Unix environment

Using FLWOR to Bind a Variable let $authors := for $a in document("../docs/xmpbib.xml")//author order by $a/last empty least, $a/first empty least return $a return $authors Result: Abiteboul Serge, Buneman Peter, Stevens W., Suciu Dan

Using FLWOR to Bind a Variable ( Abiteboul Serge, Buneman Peter, Stevens W., Suciu Dan ) /last Result: Abiteboul, Buneman, Stevens, Suciu

A Bug? let $authors := for $a in document("../docs/xmpbib.xml")//author order by $a/last empty least, $a/first empty least return $a return $authors/last Result: Stevens, Abiteboul, Buneman, Suciu

Another Ordering Example for $a in document("../docs/xmpbib.xml")//author order by $a/last empty least, $a/first empty least return $a/last Result: Abiteboul, Buneman, Stevens, Suciu

Predicates let $a := (1,2,3) [. > 2 or. = 1] return $a Result: 1, 3

Predicates let $a := (1,2,3) [. > 2 or. = 1] for $i in $a return { $i } Result: 1, 3

Computing Content for $t in (document("../docs/xmpbib.xml")//title) [count(../author) < 3 ] return { $t } Result: TCP/IP Illustrated, Advanced Programming in the Unix environment, The Economics of Technology and Content for Digital TV

Returning a Variable Content let $i := (1, 2, 3) return $i Result: 1, 2, 3

Computing Content for $b in document("../docs/xmpbib.xml")//book return {$b/title, $b/price} Result: TCP/IP Illustrated 65.95, Advanced Programming in the Unix environment 65.95, Data on the Web 39.95, The Economics of Technology and Content for Digital TV

Computing Content – Similar for $b in document("../docs/xmpbib.xml")//book return {$b/author, $b/price} Result: Next slide

Result: Computing Content Stevens W , Stevens W , Abiteboul Serge Buneman Peter Suciu Dan 39.95,

More Computing Content let $i := (1, 2, 3) return {for $a in $i return { $a } } Result: 1 2 3

Some Fine Points The part of a direct element constructor between the start tag and the end tag is called the content of the element constructor. This content may consist of literal text characters, nested element constructors, and expressions enclosed in curly braces. In general, the value of an enclosed expression may be any sequence of nodes and/or atomic values. Enclosed expressions can be used in the content of an element constructor to compute both the content and the attributes of the constructed node

Conceptually, the content of an element constructor is processed as follows: 1. The content is evaluated to produce a sequence of nodes called the content sequence, as follows: 1.Predefined entity references and character references are expanded into their referenced strings, as described in Literals Literals 2.Each consecutive sequence of literal characters evaluates to a single text node containing the characters. However, if the sequence consists entirely of boundary whitespace as defined in Whitespace in Element Content and the Prolog does not specify xmlspace = preserve, then no text node is generated Whitespace in Element Content 3.Each nested element constructor is evaluated according to the rules in this section, resulting in a new element node.

Cont. 4. Enclosed expressions are evaluated as follows: For each node returned by an enclosed expression, a new deep copy of the node is constructed, including all its children, attributes, and namespace nodes (if any). Each copied node has a new node identity. Copied element nodes are given the type annotation xs:anyType, and copied attribute nodes are given the type annotation xs:anySimpleType. For each adjacent sequence of one or more atomic values returned by an enclosed expression, a new text node is constructed, containing the result of casting each atomic value to a string, with a single blank character inserted between adjacent values.

Cont. 2.If the content sequence contains a document node, a type error is raised.[err:XQ0023]type errorerr:XQ If the content sequence contains an attribute node following a node that is not an attribute node, a type error is raised.[err:XQ0024] Attribute nodes occurring at the beginning of the content sequence become attributes of the new element node. If two or more attributes of the new element node have the same name, a dynamic error is raised.[err:XQ0025]type errorerr:XQ0024 dynamic errorerr:XQ0025

Cont. 4.Adjacent text nodes in the content sequence are coalesced into a single text node by concatenating their contents, with no intervening blanks. 5.The resulting sequence of nodes becomes the children and attributes of the new element node in the data model representation.data model 6.The new element node is automatically validated, as described in Type of a Constructed Element Type of a Constructed Element

Examples Example: {1} The constructed element node has one child, a text node containing the value " 1 ". Example: {1, 2, 3} The constructed element node has one child, a text node containing the value " ". Example: {1}{2}{3} The constructed element node has one child, a text node containing the value " 123 ".

Examples Example: {1, "2", "3"} The constructed element node has one child, a text node containing the value " ". Example: I saw 8 cats. The constructed element node has one child, a text node containing the value " I saw 8 cats. ". Example: I saw {5 + 3} cats. The constructed element node has one child, a text node containing the value " I saw 8 cats. ".

Examples Example: I saw {5 + 3} cats. The constructed element node has three children: a text node containing " I saw ", a child element node named howmany, and a text node containing " cats. ". The child element node in turn has a single text node child containing the value " 8 ".

Example for $a in document("../docs/xmpbib.xml")//author return {string($a/first), " ", $a/last} Result: W. Stevens, Serge Abiteboul, Peter Buneman, Dan Suciu

Example for $a in document("../docs/xmpbib.xml")//author return {string($a/first), " ", string($a/last)} Result: W. Stevens, W. Stevens, Serge Abiteboul, Peter Buneman, Dan Suciu

Example (XHTML) document { Title Publisher Price Year TCP/IP Addison Elements and Attributes Weston } TitlePublisherPriceYear TCP/IPAddison Elements and Attributes Weston let $t := document("bib.xhtml") return $t Result

Use “at” - Translate XHTML  “Obvious” XML let $t := document("bib.xhtml")//table for $r in $t/tbody/tr return { for $c at $i in $r/td return element { lower-case(data($t/thead/tr/td[$i])) } { string($c) } } Result: Next slide

Result TCP/IP Addison , Elements and Attributes Weston

distinct distinct-values( document("../docs/xmpbib.xml")//author/la st ) Result: Stevens, Abiteboul, Buneman, Suciu

distinct-values for $a in distinct-values( document("../docs/xmpbib.xml")//author ) return $a Result: Stevens W., Abiteboul Serge, Buneman Peter, Suciu Dan

Distinct let $a := ( Elements Are Redundant, Elements Are Redundant ) return { distinct-nodes($a) } but { distinct-values($a) }

Result Elements Are Redundant Elements Are Redundant but Elements Are Redundant Larger font : Elements Are Redundant Elements Are Redundant but Elements Are Redundant

distinct-nodes fn:distinct-nodes($srcval as node*) as node* Returns the sequence that results from removing from $srcval all but one of a set of nodes that have the same identity as one another, based on node identity (that is, using op:node-equal()). The order in which the distinct nodes are returned is ·implementation dependent·. If $srcval is the empty sequence, returns the empty sequence. For detailed semantics see section of [XQuery 1.0 and XPath 2.0 Formal Semantics].op:node-equal()·implementation dependent·[XQuery 1.0 and XPath 2.0 Formal Semantics]

distinct-values fn:distinct- values($srcval as xs:anyAtomicType*) as xs:anyAtomicType* fn:distinct- values($srcval as xs:anyAtomicType*,$collationLiteral as xs:string) as xs:anyAtomicType* Returns the sequence that results from removing from $srcval all but one of a set of values that are eq to one other. All the values must be of a single type or one of its subtypes (for numeric values, the numeric promotion rules defined in 6.2 Operators on Numeric Values are used to promote all values to a single common type).6.2 Operators on Numeric Values The type returned is a sequence of values of the same type as $srcval. The type must have a total order. If this condition is not satisfied, an error is raised ("Type does not have total order"). Equality must also be defined for the type. If this condition is not satisfied, an error is raised ("Type does not have equality defined"). For detailed semantics see section of [XQuery 1.0 and XPath 2.0 Formal Semantics].[XQuery 1.0 and XPath 2.0 Formal Semantics]

distinct-values If $collationLiteral is not in the lexical space of xs:anyURI an error is raised ("Invalid collationURI"). If $srcval is the empty sequence, the empty sequence is returned. For xs:float and xs:double values, NaN is considered to be equal to itself and 0.0 is equal to If an xs:dateTime, xs:date or xs:time value does not have a timezone, an implicit timezone is provided by the evaluation context. The normalized value is adjusted using this implicit timezone if necessary. The adjusted normalized value is used to compute distinctness. If multiple adjusted normalized values compare equal but the accompanying timezones are different, it is ·implementation dependent· which value is returned. ·implementation dependent·

distinct-values Equality of string values is determined according to the collation that is used. The order of the values returned is ·implementation dependent·. The collation used by the invocation of this function is determined according to the rules in 7.3 Equality and Comparison of Strings. If the type of the items in $srcval is not xs:string and $collationLiteral is specified, the collation is ignored. ·implementation dependent·7.3 Equality and Comparison of Strings Examples fn:distinct-values(1, 2.0, 3, 2) returns (1, 3, 2.0). So, what about semantics for node sequence argument?

List by Author let $books := document("../docs/xmpbib.xml")//bib return { for $a in distinct-values($books//author) order by $a return { $a/last/text() } { for $b in $books//book [author = $a] order by $b/title return $b/title } } Could we use distinct- nodes?

Result Abiteboul Data on the Web Buneman Data on the Web Stevens Advanced Programming in the Unix environment TCP/IP Illustrated Suciu Data on the Web

More Examples with Distinct for $a in distinct-values( document("../docs/xmpbib.xml")//author ) for $l in distinct-values($a/last), $f in distinct-values($a [ last = $l]/first) return { $l } { $f } Result: Next slide

Result Stevens W., Abiteboul Serge, Buneman Peter, Suciu Dan

Concatenate let $a := ( Elements Are Redundant, Elements Are Redundant ) return op:concatenate(distinct-values( $a ), distinct-nodes($a))

Result Elements Are Redundant, Elements Are Redundant

Concatenate op:concatenate($seq1 as item*, $seq2 as item*) as item* Returns a sequence consisting of the items in $seq1 followed by the items in $seq2. This function backs up the infix operator ",". If either sequence is the empty sequence, the other operand is returned Examples op:concatenate((1 2 3), (4 5)) returns ( ). op:concatenate((), ()) returns ().

Data let $a := ( Elements Are Redundant, Elements Are Redundant ) return data($a) Result: Elements Are Redundant, Elements Are Redundant

Data data( Attributes are Redundant ) Result: Attributes are Redundant12.30

Data fn:data($srcval as item*) as xdt:anyAtomicType* fn:data takes a sequence of items and returns a sequence of atomic values. The result of fn:data is the sequence of atomic values produced by applying the following rules to each item in $srcval: If the item is an atomic value, it is returned. If the item is a node, fn:data() returns the typed value of the node as defined by the accessor function dm:typed-value in [XQuery 1.0 and XPath 2.0 Data Model].[XQuery 1.0 and XPath 2.0 Data Model]

dm:typed-value dm:typed-value($n as Node) as xdt:anyAtomicType*Node The dm:typed-value accessor returns the typed-value of the node, which is a sequence of zero or more atomic values derived from the string-value of the node and its type in such a way as to be consistent with validation.dm:typed-value If the node is a comment, document, namespace, processing- instruction, or text node, then its typed value is equal to its string value as an instance of xdt:untypedAtomic. If the node is an attribute node with type xs:anySimpleType, then its typed value is equal to its string value as an instance of xdt:untypedAtomic. The typed value of an attribute node with any other type is derived from its string value and type annotation in a way that is consistent with XML Schema validation. If the node is an element node with type xs:anyType, then its typed value is equal to its string value, as an instance of xdt:untypedAtomic.

dm:typed-value (Cont.) If the node is an element node with a simple type or with a complex type of simple content, then its typed value is derived from its string value and type in a way that is consistent with XML Schema validation. If the item is an element node with complex type of empty content, then its typed value is the empty sequence. If the node is an element node with a complex type of mixed content, then its typed value is its string value as an instance of xdt:untypedAtomic. Recall dm:string-value: The concatenation of the string-values of all the text node descendants of the element in document order.dm:string-value If the item is an element node with complex type of complex content, then its typed value is undefined and dm:typed-value raises a type error, which may be handled by the host language.dm:typed-value For detailed semantics see [XQuery 1.0 Formal Semantics].[XQuery 1.0 Formal Semantics]

A note on Understanding the Spec The basics are: –The Xpath 2.0 and Xquery 1.0 data model. –XML Schema. –Sequence types (Xquery). Next come built-in functions and operators. Next come Xquery expressions.

Join – File xmpreviews1.xml TCP/IP Illustrated One of the best books on TCP/IP.

Join for $t in document("../docs/xmpbib.xml")//title, $e in document("../docs/xmpreviews.xml")//entry where $t = $e/title return { $t, $e/review } Result: Next slide

Result TCP/IP Illustrated One of the best books on TCP/IP., Advanced Programming in the Unix environment A clear and detailed discussion of UNIX programming., Data on the Web A very good discussion of semi-structured database systems and XML.

Join – Another way for $t in document("../docs/xmpbib.xml")//title return { $t } {for $e in document("xmpreviews1.xml")//entry where $t = $e/title return $e/review } Result: Next slide

Result TCP/IP Illustrated One of the best books on TCP/IP., Advanced Programming in the Unix environment, Data on the Web, The Economics of Technology and Content for Digital TV

Hierarchy Inversion { for $p in distinct-values(document("../docs/xmpbib.xml")//publisher) order by $p return { $p } { for $b in document("../docs/xmpbib.xml")/bib/book where $b/publisher = $p order by $b/title return $b/title } } Result: Next slide

Result Addison-Wesley Advanced Programming in the Unix environment TCP/IP Illustrated Kluwer Academic Publishers The Economics of Technology and Content for Digital TV Morgan Kaufmann Publishers Data on the Web

Existential Quantifiers for $b in document("../docs/xmpbib.xml")//book where some $a in $b//author satisfies ($a/last="Stevens" and $a/first="W.") return $b/title Result: TCP/IP Illustrated, Advanced Programming in the Unix environment

Universal Quantifiers for $b in document("../docs/xmpbib.xml")//book where every $a in $b//author satisfies ($a/last="Stevens" and $a/first="W.") return $b/title Result: (note the empty satisfaction in the 3 rd title) TCP/IP Illustrated, Advanced Programming in the Unix environment, The Economics of Technology and Content for Digital TV

Example: Books per Author { let $a := document("../docs/xmpbib.xml")//author for $l in distinct-values($a/last), $f in distinct-values($a [ last = $l]/first) order by data($l), data($f) return { $l, " ", $f } { for $b in document("../docs/xmpbib.xml")//bib/book where some $ba in $b/author satisfies ($ba/last= $l and $ba/first = $f) order by $b/title return $b/title } }

Result Abiteboul Serge Data on the Web Buneman Peter Data on the Web Stevens W. Advanced Programming in the Unix environment TCP/IP Illustrated Suciu Dan Data on the Web

One More for $b in document("../docs/xmpbib.xml")//book return { $b/title } { for $a at $i in $b/author where $i <= 2 return { string($a/last), ",", string($a/first) } } { if (count($b/author) > 2) then et. al. else () }

Result TCP/IP Illustrated Stevens, W., Advanced Programming in the Unix environment Stevens, W., Data on the Web Abiteboul, Serge Buneman, Peter et. al., The Economics of Technology and Content for Digital TV

mybook.xml

Recursive Functions define function toc($book-or-section as element) as element* { for $s in $book-or-section/section return { toc($s) } } { for $a in document("mybook.xml")/book return toc($a) } Result: Next slide

Result

Trace define function toc($book-or-section as element) as element* { for $s in $book-or-section/section return { toc( trace($s, " ARG1 ") ) } } { for $a in document("mybook.xml")/book return toc( trace($a, " ARG ") ) } Result: Next slide

ARG Some background... Some papers... some defs... System design... Overview... The details... ARG1 Some background... Some papers... ARG1 Some background... ARG1 Some papers... ARG1 some defs... ARG1 System design... Overview... The details...

ARG1 System design... ARG1 Overview... The details... ARG1 Overview... ARG1 The details...

Assignment Modify the previous program so that the output in general is of the form such that under each section each first-level child section is indented by 5 spaces, For example: Intro Background Related Work Definitions The System Design Implementation Overview Detailed :