1. Introduction to XQuery Bun Yue Professor, CS/CIS UHCL

2. W3C Recommendations  http://www.w3.org/TR/xquery/: W3C XQuery http://www.w3.org/TR/xmlquery-use-cases: XQuery use cases. http://www.w3.org/TR/xquery/ http://www.w3.org/TR/xmlquery-use-cases  http://www.w3.org/TR/xquery-operators/: XQuery and XPath functions. http://www.w3.org/TR/xquery-operators/  http://www.w3.org/TR/xpath-datamodel/: XQuery 1.0 and XPath 2.0 Data Model. http://www.w3.org/TR/xpath-datamodel/  http://www.w3.org/TR/xpath20/: XPath 2.0. http://www.w3.org/TR/xpath20/  http://www.w3.org/TR/xmlschema-1/: XML Schema Part 1: Structures. http://www.w3.org/TR/xmlschema-1/  http://www.w3.org/TR/xmlschema-2/: XML Schema Part 2: datatypes. http://www.w3.org/TR/xmlschema-2/

3. Introduction  XQuery is designed for effectively query and retrieve information from a diversified XML sources.  The XML sources can be one or more XML documents.  XQuery is derived from Quilt, and has borrowed features from XPath, XQL, SQL, etc.

4. Introduction  It is a functional language where a query is an expression.  There are three faces of the XQuery languages: A "surface" syntax that programmers may probably use. An XML-based syntax that machine may probably use (XQueryX). A formal semantic that XQuery engine implementators use.

5. Introduction.  XQuery 1.0 extends XPath 2.0.  The type system of XQuery is based on XML Schema.  A limitation of XQuery: No update or insert.  The basic building block of XQuery is expressions. (In this sense, like SQL, XQuery is not a full programming language.)

6. Comparing to SQL Relational DB: SQL XML DB: XQuery Basic unitsrelationscollections Recordstuples or rows of schema documents of same schema SchemaRelational Schema DTD, XML Schema Query resultsRelations: unordered list of rows Ordered sequences of nodes.

7. Review of XPath 2.0  The value of an expression is a sequence, which is an ordered list of items.  An item can be a node or of atomic value.  There are 7 node types: Document Element Attribute Comment Text Processing Instruction Namespace

8. XQueryX For doc("census.xml")//person[@job="Athlete"] the corresponding XQueryX can be: census.xml person job Athlete

9. Data Types  XQuery is strongly typed.  XQuery types are based on XML Schema: using the namespace prefix xs and url: http://www.w3.org/2001/XMLSchema. XPath functions and operators: using the namespace prefix xdt and url: http://www.w3.org/2004/07/xpath- datatypes

10. Types

11.  xdt:untyped is used to denote element nodes not yet validated.  xdt:untypedAtomic is used to denote atomic types that has not been assigned a more specific type.

12. Query  A query in XQuery is an expression for reading XML documents or fragments and returning a sequence of well-formed XML fragments  Everything in XQuery is an expression that is evaluated to a value.

13. Query expressions  Some common forms of XQuery expressions are (these appear in most tutorials): path expressions element constructors FLWR or FLOWR (pronounced as "flower") expressions list expressions conditional expressions quantified expressions datatype expressions

14. More Queries  Examples of other expressions include: primary expressions sequence expressions arithmetic expressions logical expressions comparison expressions sorting expressions validate expressions

15. Comments  XQuery comments are embedded within (: and :).

16. Functions  Supports a collection of about 200 built-in operators and functions to be used within expressions.  Input functions in XQuery include doc() and collection(). They are used to identify the sources of the XML documents.

17. Input Functions  Input functions: doc() collection().

18. Prolog  XQuery may have prologs for declarations. Examples: Variable declarations Function declarations Base-URI declarations Version declarations Module import …

19. Variable Declarations  Format: declare variable $name = expression;  E.g. declare variable $a := doc("census.xml")//person ;

20. Path Expressions  XQuery 1.0 is a superset of XPath 2.0.  An XPath expression is also an XQuery expression

21. Editix  Use “View > Windows > XQuery Builder”  For XQ files, use “XSLT/XQuery > Transform using an XQuery Request…” Specify source xq file, xml file and output file. Use.xml extension. If you use.txt extension, only text node contents are output.

22. Examples declare base-uri "whatever-path"; doc("bib.xml")/* Return basically bib.xml.

23. Example doc("bib.xml")//* Return many nodes (in a sequence).  Results are not well-formed.

24. Examples doc("bib.xml")//book[@year] count(doc("census.xml")//person)

25. Element Constructors  Element constructors can be used to construct XML elements.  If the name, attributes, and content of the element are all constants, the element constructor is based on standard XML notation and is called a direct element constructor (W3C).

26. Example The XQuery Bun Yue returns Bun Yue

27. Element Constructors  XQuery expressions can be embedded in the direct element constructors within a pair of curly braces, {}.  For the characters '{' and '}', use '{{' and '}}' respectively.  XQuery expressions may be separated by commas.

28. Example Bun Yue { doc("bib.xml")//author } Adds Bun Yue to the authors of bib.xml.

29. Computed Constructors  Computed constructors can also be used to declare nodes: Use the keywords element, attribute, document, text, processing-instruction, comment, or namespace to declare the type of the nodes. Specify the node names for those node types with names (element, attribute, processing instruction, and namespace nodes) Use a pair of braces to define the content expressions. Note the use of commas to separate expressions in the context.

30. Example (from W3C) element book { attribute isbn {"isbn-0060229357" }, element title { "Harold and the Purple Crayon"}, element author { element first { "Crockett" }, element last {"Johnson" } }

31. Example (result) Harold and the Purple Crayon Crockett Johnson

32. Dynamic Element Names  Computed expressions can be used to create elements with dynamic names.

33. Example { for $author in doc("bib.xml")//author return element {$author/last/text()} { $author/first }

34. Example Result W. W. Serge Peter Dan

35. Example  Note that is a child element. See the difference of: { for $author in doc("bib.xml")//author return element {$author/last/text()} { $author/first/text() }

36. Example  This example may also result in a runtime error (as the value of may not be suitable for a QName.

37. FLWOR expressions  FLWOR expressions are one of the most important constructs in XQuery.  You may compare with the SELECT statement of SQL.

38. FLWOR (W3C) [42] FLWORExpr ::= (ForClause | LetClause)+ WhereClause? OrderByClause? "return" ExprSingle [43] ForClause ::= "for" "$" VarName TypeDeclaration? PositionalVar? "in" ExprSingle ("," "$" VarName TypeDeclaration? PositionalVar? "in" ExprSingle)* [45] LetClause ::= "let" "$" VarName TypeDeclaration? ":=" ExprSingle ("," "$" VarName TypeDeclaration? ":=" ExprSingle)* [123] TypeDeclaration ::= "as" SequenceType [44] PositionalVar ::= "at" "$" VarName [46] WhereClause ::= "where" Expr [47] OrderByClause ::= ("order" "by" | "stable" "order" "by") OrderSpecList [48] OrderSpecList ::= OrderSpec ("," OrderSpec)* [49] OrderSpec ::= ExprSingle OrderModifier [50] OrderModifier ::= ("ascending" | "descending")? (("empty" "greatest") | ("empty" "least"))? ("collation" StringLiteral)?

39. FLWOR  FLWOR expressions allow: For: Iteration through items in XPath 2.0 sequences. Create a tuple stream where each tuple contains a distinct binding for each variable to a distinct value. Let: Variables binding Where: Predicate application for inclusion in the iteration. Order by: Ordering data set for the iteration. Return: Constructing new result for returning.

40. For and Let  The for and let clauses produces a tuple stream.  A tuple consists of one or more bound variables.  A variable begins with the prefix $.  A bound variable is one that has been assigned a value.

41. Example declare base-uri “whatever”; let $a := doc("bib.xml")//author return { $a }

42. Example Results Stevens W. Stevens W. …

43. Example Note  In this example: The tuple stream is composed of only one tuple. The variable $b in this tuple is bound to the node sequence of 5 nodes.

44. Example for $a in doc("bib.xml")//author return { $a }

45. Example Result Stevens W. Stevens W. …

46. Example Notes  In this example: The tuple stream is composed of only five tuples. The variable $b in this tuple is bound to one node at a time.

47. Example for $a in doc("bib.xml")//author, $b in doc("bib.xml")//author return

48. Example Result … (: 25 counts :)

49. Example Note The tuple stream is composed of only 25 tuples. The 25 tuples are:  ($a: Stevens W., $b: Stevens W. )  ($a: Stevens W., $b: Abiteboul Serge )  …

50. Example for $a in doc("bib.xml")//author, $b in $a/last return

51. Example Result

52. Example Note The tuple stream is composed of only 5 tuples. The 5 tuples are:  ($a: Stevens W., $b: Stevens</last)  ($a: Abiteboul Serge, $b: Abiteboul</last)  …

53. Example for $a in doc("bib.xml")//author, $b in doc("bib.xml")//author where $a = $b return { $a/last/text() } { $b/last/text() }

54. Example Result Stevens … (: three more times. :) Abiteboul Buneman Suciu

55. Example Note The tuple stream is composed of only 7 tuples. The 7 tuples are:  ($a: Stevens W., $b: Stevens W. ) (: 4 times)  ($a: Abiteboul Serge, $b: Abiteboul Serge )  …

56. Example {for $f in doc("tree-data.xml")//figure return { $f/@* } { $f/title } }

57. Example Result Traditional client/server architecture Graph representations of structures Examples of Relations

58. Example Note  There are three tuples in the tuple stream of the for clause. Each tuple has one variable: $f, which is bounded to each of the three elements in the input xml contents respectively.  { $f/@* } returns the attributes of the original elements, which will be put as attributes of the output element.

59. Example { fn:string-join(for $a in doc("tree- data.xml")//author return $a/text(), ", ") }

60. Example Result Serge Abiteboul, Peter Buneman, Dan Suciu

61. Example Note  fn:string-join takes two arguments: A sequence of string, and A string join separator

62. Example {for $f in doc("tree-data.xml")//figure return { attribute size { $f/@width * $f/@height } } }

63. Example Result

64. Example { for $f in doc("tree-data.xml")//figure let $size := $f/@width * $f/@height order by $size return { attribute size { $size } } }

65. Example Result

66. Exercise #1  Use bib.xml, Show all books published by Addison Wesley. TCP/IP Illustrated Stevens W. Advanced Programming in the Unix environment Stevens W.

67. Exercise #2  All books by Addison-Wesley using different format: TCP/IP Illustrated Advanced Programming in the Unix environment

68. Exercise #3  All books written by W. Stevens ordered by years: Advanced Programming in the Unix environment TCP/IP Illustrated

69. Exercise #4  All books written by W. Stevens ordered by years in descending order: TCP/IP Illustrated Advanced Programming in the Unix environment

70. Exercise #5  Use ft2.xml, return every with its and child elements. Add a child element to include the number of email addresses. Boris Becker 2 …

71. Exercise #6  Return all elements with all attributes. The body of the element should be the name of the person in the format of first name and then last name. For ft2.xml, it returns: Boris Becker Valerie Becker Chris Becker Julie Becker John Becker Mary Becker

72. Exercise #7  Return all pairs of elements of persons with the same last name, not including pairing with oneself. Each pair of result is embedded in an element with the last name of the persons as the element name. For ft2.xml, it returns: Boris Valerie Boris Chris Boris Julie Boris John Boris Mary Valerie Boris …

73. Exercise #8  Convert all text nodes to and all elements with name x to. For ft2.xml, it returns: …

74. Function Declarations  XQuery allows user-defined functions in the prolog. [26] FunctionDecl ::= "declare" "function" QName "(" ParamList? ")" ("as" SequenceType)? (EnclosedExpr | "external")FunctionDecl QNameParamList SequenceTypeEnclosedExpr [27] ParamList ::= Param ("," Param)*ParamListParam [28] Param ::= "$" QName TypeDeclaration?ParamQName TypeDeclaration [118] TypeDeclaration ::= "as" SequenceTypeTypeDeclaration SequenceType

75. Example: factorial($i) declare function local:factorial($i as xs:integer) as xs:integer { if ($i < 0) then 0 else if ($i = 0) then 1 else $i * local:factorial($i - 1) }; local:factorial(6)

76. Functions  There is a ; after the function declaration.  The namespace prefix local is used for user-defined functions. XQuery predefines the namespace prefix local to the namespace http://www.w3.org/2004/07/xquery-local- functions, and reserves this namespace for use in defining local functions.  The types of the arguments and return values should be sequence types.

77. Types  Sequence type can be: empty(), or ItemType OccurrenceIndicator?  OccurrenceIndicator can be +, ? or *.  Item type can be: item() atomic type, or kind test.

78. Kind Tests  Important kind tests include node() text() comment() processing-instruction(): with optional name argument. element test attribute test

79. Element Tests  Example of element tests are: element(*) element(familytree) element(man, personType)

80. Functions  Writing XQuery functions: Functional programming. Many are recursive in nature. Beware of types of parameters and return values.

81. Example from W3C declare function local:depth($e as node()) as xs:integer { (: A node with no children has depth 1 :) (: Otherwise, add 1 to max depth of children :) if (fn:empty($e/*)) then 1 else fn:max(for $c in $e/* return local:depth($c)) + 1 }; { local:depth(doc("ft2.xml")) }

82. Exercise #9  Write an XQuery function to count the number of elements in an element node (including itself). Try to use a recursive solution.

83. Exercise #10 For XML document such as ft2.xml, write a function that returns all child person nodes with parent of social security number $ssn.

84. Questions