1. 1 Lecture 9: XQuery

2. 2 XQuery Motivation XPath expressivity insufficient –no join queries (as in SQL) –no changes to the XML structure possible –no quantifiers (as in SQL) –no aggregation and functions

3. 3 FLWR (“Flower”) Expressions FOR... LET... WHERE... RETURN... FOR... LET... WHERE... RETURN... XQuery uses XPath to express more complex queries

4. 4 Addison-Wesley Serge Abiteboul Rick Hull Victor Vianu Foundations of Databases 1995 Freeman Jeffrey D. Ullman Principles of Database and Knowledge Base Systems 1998 Addison-Wesley Serge Abiteboul Rick Hull Victor Vianu Foundations of Databases 1995 Freeman Jeffrey D. Ullman Principles of Database and Knowledge Base Systems 1998 Sample Data for Queries

5. 5 Basic FLWR Find all book titles published after 1995: FOR $x IN document("bib.xml")/bib/book WHERE $x/year/text() > 1995 RETURN $x/title FOR $x IN document("bib.xml")/bib/book WHERE $x/year/text() > 1995 RETURN $x/title Result: abc def ghi

6. 6 FLWR vs. XPath expressions Equivalently FOR $x IN document("bib.xml")/bib/book[year/text() > 1995] /title RETURN $x FOR $x IN document("bib.xml")/bib/book[year/text() > 1995] /title RETURN $x And even shorter: document("bib.xml")/bib/book[year/text() > 1995] /title

7. 7 Result Structuring Find all book titles and the year when they were published: FOR $x IN document("bib.xml")/ bib/book RETURN { $x/title/text() } { $x/year/text() } Braces { } denote evaluation of enclosed expression

8. 8 Result Structuring Notice the use of “{“ and “}” What is the result without them ? FOR $x IN document("bib.xml")/ bib/book RETURN $x/title/text() $x/year/text()

9. 9 XQuery Joins and Nesting For each author of a book by Morgan Kaufmann, list all books she published: FOR $b IN document(“bib.xml”)/bib, $a IN $b/book[publisher /text()=“Morgan Kaufmann”]/author RETURN { $a, FOR $t IN $b/book[author/text()=$a/text()]/title RETURN $t } FOR $b IN document(“bib.xml”)/bib, $a IN $b/book[publisher /text()=“Morgan Kaufmann”]/author RETURN { $a, FOR $t IN $b/book[author/text()=$a/text()]/title RETURN $t } In the RETURN clause comma concatenates XML fragments

10. 10 XQuery Nesting Jones abc def Smith ghi Jones abc def Smith ghi Result:

11. 11 Aggregates Find all books with more than 3 authors: count = a function that counts avg = computes the average sum = computes the sum distinct-values = eliminates duplicates FOR $x IN document("bib.xml")/bib/book WHERE count($x/author)>3 RETURN $x

12. 12 Aggregates Same thing: FOR $x IN document("bib.xml")/bib/book[count(author)>3] RETURN $x

13. 13 Aggregates Print all authors who published more than 3 books – be aware of duplicates ! FOR $b IN document("bib.xml")/bib, $a IN distinct-values($b/book/author/text()) WHERE count($b/book[author/text()=$a)>3 RETURN { $a }

14. 14 Aggregates Find books whose price is larger than average: FOR $b in document(“bib.xml”)/bib LET $a:=avg($b/book/price/text()) FOR $x in $b/book WHERE $x/price/text() > $a RETURN $x FOR $b in document(“bib.xml”)/bib LET $a:=avg($b/book/price/text()) FOR $x in $b/book WHERE $x/price/text() > $a RETURN $x

15. 15 Result Structure “Flatten” the authors, i.e. return a list of (author, title) pairs FOR $b IN document("bib.xml")/bib/book, $x IN $b/title/text(), $y IN $b/author/text() RETURN { $x } { $y } Result: abc efg abc hkj

16. 16 Result Structure For each author, return all titles of her/his books FOR $b IN document("bib.xml")/bib, $x IN $b/book/author/text() RETURN { $x } { FOR $y IN $b/book[author/text()=$x]/title RETURN $y } What about duplicate authors ? Result: efg abc klm....

17. 17 Result Structure Eliminate duplicates: FOR $b IN document("bib.xml")/bib, $x IN distinct-values($b/book/author/text()) RETURN $x { FOR $y IN $b/book[author/text()=$x]/title RETURN $y }

18. 18 SQL and XQuery Side-by-side Product(pid, name, maker) Company(cid, name, city) Find all products made in Seattle SELECT x.name FROM Product x, Company y WHERE x.maker=y.cid and y.city=“Seattle” FOR $r in document(“db.xml”)/db, $x in $r/Product/row, $y in $r/Company/row WHERE $x/maker/text()=$y/cid/text() and $y/city/text() = “Seattle” RETURN { $x/name } SQL XQuery FOR $y in /db/Company/row[city/text()=“Seattle”], $x in /db/Product/row[maker/text()=$y/cid/text()] RETURN { $x/name } Cool XQuery

19. 19 ??? …. …....

20. 20 XQuery Variables FOR $x in expr -- binds $x to each value in the list expr LET $x := expr -- binds $x to the entire list expr –Useful for common subexpressions and for aggregations

21. 21 XQuery: LET $b is a collection of elements, not a single element count = a (aggregate) function that returns the number of elms { FOR $p IN distinct-values(//publisher/text()) LET $b := /db/book[publisher/text() = $p] WHERE count($b) > 100 RETURN { $p } } { FOR $p IN distinct-values(//publisher/text()) LET $b := /db/book[publisher/text() = $p] WHERE count($b) > 100 RETURN { $p } } Find all publishers that published more than 100 books:

22. 22 FOR v.s. LET FOR Binds node variables  iteration LET Binds collection variables  one value

23. 23 FOR v.s. LET FOR $x IN /bib/book RETURN { $x } FOR $x IN /bib/book RETURN { $x } Returns:... LET $x := /bib/book RETURN { $x } LET $x := /bib/book RETURN { $x } Returns:...

24. 24 Collections in XQuery Ordered and unordered collections –/bib/book/author/text() = an ordered collection: result is in document order –distinct-values(/bib/book/author/text()) = an unordered collection: the output order is implementation dependent LET $a := /bib/book  $a is a collection $b/author  a collection (several authors...) RETURN { $b/author } Returns:...

25. 25 SQL and XQuery Side-by-side Product(pid, name, maker, price) Find all product names, prices, sort by price SELECT x.name, x.price FROM Product x ORDER BY x.price SQL FOR $x in document(“db.xml”)/db/Product/row ORDER BY $x/price/text() RETURN { $x/name, $x/price } XQuery

26. 26 abc 7 def 23.... XQuery’s Answer Notice: this is NOT a well-formed document ! (WHY ???)

27. 27 Producing a Well-Formed Answer { FOR $x in document(“db.xml”)/db/Product/row ORDER BY $x/price/text() RETURN { $x/name, $x/price } }

28. 28 abc 7 def 23.... XQuery’s Answer Now it is well-formed !

29. 29 SQL and XQuery Side-by-side For each company with revenues < 1M count the products over $100 SELECT y.name, count(*) FROM Product x, Company y WHERE x.price > 100 and x.maker=y.cid and y.revenue < 1000000 GROUP BY y.cid, y.name FOR $r in document(“db.xml”)/db, $y in $r/Company/row[revenue/text() { $y/name/text() } { count($r/Product/row[maker/text()=$y/cid/text()][price/text()>100]) }

30. 30 SQL and XQuery Side-by-side Find companies with at least 30 products, and their average price SELECT y.name, avg(x.price) FROM Product x, Company y WHERE x.maker=y.cid GROUP BY y.cid, y.name HAVING count(*) > 30 FOR $r in document(“db.xml”)/db, $y in $r/Company/row LET $p := $r/Product/row[maker/text()=$y/cid/text()] WHERE count($p) > 30 RETURN { $y/name/text() } avg($p/price/text()) A collection An element

31. 31 XQuery Summary: FOR-LET-WHERE-RETURN = FLWR

32. 32 XML from/to Relational Data XML publishing: –relational data  XML XML storage: –XML  relational data

33. 33 Client/server DB Apps Relational Database Application Network Tuple streams SQL

34. 34 XML Publishing Relational Database Application Web XML publishing Tuple streams XML SQL XPath/ XQuery

35. 35 XML Publishing Relational schema: Student(sid, name, address) Course(cid, title, room) Enroll(sid, cid, grade) StudentCourse Enroll

36. 36 XML Publishing Operating Systems MGH084 John Seattle 3.8 … Database EE045 Mary Shoreline 3.9 … … Operating Systems MGH084 John Seattle 3.8 … Database EE045 Mary Shoreline 3.9 … … Other representations possible too Group by courses: redundant representation of students

37. 37 XML Publishing First thing to do: design the DTD:

38. 38 { FOR $x IN /db/Course/row RETURN { $x/title/text() } { $x/room/text() } { FOR $y IN /db/Enroll/row[cid/text() = $x/cid/text()] $z IN /db/Student/row[sid/text() = $y/sid/text()] RETURN { $z/name/text() } { $z/address/text() } { $y/grade/text() } } { FOR $x IN /db/Course/row RETURN { $x/title/text() } { $x/room/text() } { FOR $y IN /db/Enroll/row[cid/text() = $x/cid/text()] $z IN /db/Student/row[sid/text() = $y/sid/text()] RETURN { $z/name/text() } { $z/address/text() } { $y/grade/text() } } Now we write an XQuery to export relational data  XML Note: result is in the right DTD

39. 39 XML Publishing Query: find Mary’s grade in Operating Systems FOR $x IN /xmlview/course[title/text()=“Operating Systems”], $y IN $x/student/[name/text()=“Mary”] RETURN $y/grade/text() FOR $x IN /xmlview/course[title/text()=“Operating Systems”], $y IN $x/student/[name/text()=“Mary”] RETURN $y/grade/text() XQuery SELECT Enroll.grade FROM Student, Enroll, Course WHERE Student.name=“Mary” and Course.title=“OS” and Student.sid = Enroll.sid and Enroll.cid = Course.cid SQL Can be done automatically

40. 40 XML Publishing How do we choose the output structure ? Determined by agreement with partners/users Or dictated by committees –XML dialects (called applications) = DTDs XML Data is often nested, irregular, etc No normal forms for XML