1. MonetDB/XQuery Technology Preview 1 Stefan Manegold Centrum voor Wiskunde en Informatica Amsterdam http://monetdb.cwi.nl/ - http://pathfinder-xquery.org/

2. European Pathfinder Team CWI, Amsterdam (Netherlands) – Peter Boncz, Stefan Manegold, Sjoerd Mullender University of Twente (Netherlands) – Maurice van Keulen, Jan Flokstra University of Konstanz (Germany) – Torsten Grust, Jens Teubner, Jan Rittinger Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

3. Results: Performance (1) Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery XMark benchmark, 110 MB: MonetDB/XQuery vs. X-Hive & Galax

4. Results: Performance (2) Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery XMark benchmark, 1.1 GB: MonetDB/XQuery vs. X-Hive

5. Story XQuery Example Relational XQuery –System Architecture –XML Encoding Science & Reseach Scalability Outlook –Conclusions –Roadmaps –Release & References Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

6. For each author, return number of books and receipts for books published in the past 2 years, ordered by name let $cat := fn:doc(“www.bn.com/catalog.xml”), (: Documents :) $sales := fn:doc(“www.publishersweekly.com/sales.xml”) for $author in distinct-values($cat//author) (: Grouping :) let $books := $cat//book[@year >= 2003 and author = $author],(: Sel. :) $receipts := $sales/book[@isbn = $books/@isbn]/receipts (: Join :) order by $author (: Ordering :) return (: XML Construction :) { $author } { fn:count($books) } (: Aggregation :) { fn:sum($receipts) } XQuery Example Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

7. For each author, return number of books and receipts for books published in the past 2 years, ordered by name let $cat := fn:doc(“www.bn.com/catalog.xml”), Documents $sales := fn:doc(“www.publishersweekly.com/sales.xml”) for $author in distinct-values($cat//author) Grouping let $books := $cat//book[@year >= 2003 and author = $author], Sel. $receipts := $sales/book[@isbn = $books/@isbn]/receipts Join order by $author Ordering return XML Construction { $author } { fn:count($books) } Aggregation { fn:sum($receipts) } XQuery Example Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

8. XQuery Systems: 2 Approaches Existing “native” XML/XQuery systems are built from scratch –Galax, Saxon, … –X-Hive, Tamino, … –(Still have to) re-invent optimization technology Our approach: –Build XQuery system on top of an RDBMS –Leverage mature relational technology to achieve efficient XQuery processing Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

9. Architecture Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

10.     xx XML in an RDBMS: XPath Accelerator Node-based relational encoding of XQuery's data model  f /following: SELECT * FROM pre_post WHERE pre > f.pre AND post > f.post  f /descendant: SELECT * FROM pre_post WHERE pre > f.pre AND post < f.post  f /preceeding: SELECT * FROM pre_post WHERE pre < f.pre AND post < f.post  f /ancester: SELECT * FROM pre_post WHERE pre f.post Similar queries for all 13 XPath axes 0 1 2 3 4 5 6 7 8 9 0 1 2 0 1 3 2 4 5 6 3 7 4 5 8 9 6 7 8 9 Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

11. Science & Research More research lead to more optimization –Join Recognition –Embedded XPath processing –Order Awareness Various scientific publications Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

12. Results: Scalability (3) Unsurpassed scalability Standard Opteron PC, 8GB RAM, 64-bit Linux Can process 11GB documents! Mostly linear scaling with document size Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

13. Conclusions Relational approach  Works  Is fast Is scalable Crucial Optimizations –Join recognition –Embedded XPath processing –Order awareness Research turned into open-source release Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

14. Roadmap 30.05.05: MonetDB/XQuery 4.8/0.8 “Mercurius” –Developers Release / Technology Preview 1 30.09.05: MonetDB/XQuery 4.10/0.10 “Venus” –Student Release / Technology Preview 2 –XUpdate, More Optimization 30.12.05: MonetDB/XQuery 4.12/1.12 “Mars” –Final Release –Application Programming Interfaces –End-User Front-Ends Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

15. Open Source Release & References MonetDB + Pathfinder on SourceForge –Mozilla-like License MonetDB homepage –http://monetdb.cwi.nl/ Pathfinder homepage –http://pathfinder-xquery.org/ Developers website –http://sf.net/projects/monetdb/ You are welcome to join the MonetDB/XQuery team! Stefan Manegold HollandOpen, Amsterdam 31.5.2005 MonetDB/XQuery

16. Stefan ManegoldHollandOpen, Amsterdam 31-5-2005MonetDB/XQuery Results: Performance (4) XMark performance in seconds: MonetDB/XQuery vs. Galax & X-Hive