1. 1 XQuery XML Databases Roger L. Costello 16 June 2010

2. 2 XQuery Databases XQuery is an XML query language. It can be used to efficiently and easily extract information from Native XML Databases (NXD). It can be used to query XML views of relational data.

3. 3 Database Systems supporting XQuery The following database systems offer XQuery support: –Native XML Databases: Berkeley DB XML eXist MarkLogic Software AG Tamino Raining Data TigerLogic Documentum xDb (X-Hive/DB) –Relational Databases: IBM DB2 Microsoft SQL Server Oracle

4. 4 Native XML Database (NXD) Native XML databases have an XML-based internal model, i.e., their fundamental unit of storage is XML. To enable Oxygen XML to use a database, you need to download the database's.jar files and then point Oxygen to them. For instructions on configuring each database, select the Oxygen Help menu and search for this: Native XML Database (NXD) Support

5. 5 Use Cases for Native XML Databases Table of contents 1.0 Got XML? 2.0 A very quick review of native XML databases 3.0 Storing and querying document-centric XML 3.1 Documents in the real world 3.2 Inside the applications 3.2.1 Managing documents 3.2.2 Finding documents 3.2.3 Retrieving information 3.2.4 Reusing content 3.3 Why you need a native XML database 3.4 A peek into the future 4.0 Data integration 4.1 Data integration in the real world 4.2 Inside the applications 4.2.1 Query architectures 4.2.2 Handling differences in schemas 4.3 Why you need a native XML database 4.4 A peek into the future 5.0 Working with semi-structured data 5.1 Semi-structured data in the real world 5.2 Inside the applications 5.3 Why you need a native XML database 5.4 A peek into the future 6.0 Schema evolution 6.1 Schema evolution in the real world 6.2 Inside the applications 6.3 Why you need a native XML database 6.4 A peek into the future 7.0 Long-running transactions 8.0 Handling large documents 9.0 Hierarchical data 10.0 Other uses 11.0 A final peek into the future 12.0 Conclusion 13.0 Thanks 14.0 Resources http://www.rpbourret.com/xml/UseCases.htm

6. 6 XML and Databases Table of Contents 1.0 Introduction 2.0 Is XML a Database? 3.0 Why Use a Database? 4.0 Data versus Documents 4.1 Data-Centric Documents 4.2 Document-Centric Documents 4.3 Data, Documents, and Databases 5.0 Storing and Retrieving Data 5.1 Mapping Document Schemas to Database Schemas 5.1.1 Table-Based Mapping 5.1.2 Object-Relational Mapping 5.2 Query Languages 5.2.1 Template-Based Query Languages 5.2.2 SQL-Based Query Languages 5.2.3 XML Query Languages 5.3 Storing Data in a Native XML Database 5.4 Data Types, Null Values, Character Sets, and All That Stuff 5.4.1 Data Types 5.4.2 Binary Data 5.4.3 Null Data 5.4.4 Character Sets 5.4.5 Processing Instructions and Comments 5.4.6 Storing Markup 5.5 Generating XML Schemas from Relational Schemas and Vice Versa 6.0 Storing and Retrieving Documents 6.1 Storing Documents in the File System 6.2 Storing Documents in BLOBs 6.3 Native XML Databases 6.3.1 What is a Native XML Database? 6.3.2 Native XML Database Architectures 6.3.2.1 Text-Based Native XML Databases 6.3.2.2 Model-Based Native XML Databases 6.3.3 Features of Native XML Databases 6.3.3.1 Document Collections 6.3.3.2 Query Languages 6.3.3.3 Updates and Deletes 6.3.3.4 Transactions, Locking, and Concurrency 6.3.3.5 Application Programming Interfaces (APIs) 6.3.3.6 Round-Tripping 6.3.3.7 Remote Data 6.3.3.8 Indexes http://www.rpbourret.com/xml/XMLAndDatabases.htm

7. 7 MarkMail -- An email search site built on XQuery http://markmail.org/

8. 8

9. 9 eXist It is a native XML database It is open source More info: http://exist-db.orghttp://exist-db.org Want to see some sample XQueries run against an eXist database? Check out the eXist sandbox: http://demo.exist-db.org/exist/sandbox/sandbox.xql http://demo.exist-db.org/exist/sandbox/sandbox.xql In the drop-down box labeled "Paste example" select "Find books by author". Examine the XQuery then press the Send button.

10. 10

11. 11 Installing eXist In your C: folder create a new folder, eXist C: eXist In the XQuery folder is a subfolder, eXist-exe-file. Open it. Double click on the.exe file. The second prompt will ask: Select the installation path. Enter: C:\eXist As the password for user admin enter: admin Select these packs to install: Core, Javadocs (we dont need Source)

12. 12 Running eXist After its finished installing, start the database: –Start All Programs eXist XML Database eXist Database Startup (alternatively, double click startup.bat in c:\eXist\bin) –Minimize the DOS window that is generated by starting the database (dont close it) –Open a browser and enter: http://localhost:8080/exist/index.xmlhttp://localhost:8080/exist/index.xml View the Admin page. The link to it is on the left side. You need to scroll down to see the link. Remember the password is: admin.

13. 13 How to configure an eXist datasource in Oxygen 1.Open Oxygen. Go to Options -> Preferences -> Data Sources. In the Data Sources panel click the New button. 2.Enter this name for this data source: eXist-DB Select eXist from the driver type combo box. 3.Press the Add button and add the following eXist specific files. They are located in the eXist installation root directory (c:\eXist): 1.exist.jar 2.lib/core/xmldb.jar 3.lib/core/xmlrpc-client-3.1.1.jar 4.lib/core/xmlrpc-common-3.1.1.jar 5.lib/core/ws-commons-util-1.0.2.jar 4.Click OK to finish the data source configuration.

14. 14 How to configure an eXist connection in Oxygen 1.Open Oxygen. Go to Options -> Preferences -> Data Sources. In the Connections panel click the New button. 2.Enter this name for this connection: eXist-Connection Select eXist-DB from the Data Source combo box. 3.Fill in the Connection Details: 1.XML DB URI: use this value: xmldb:exist://localhost:8080/exist/xmlrpc 2.User: use this value: guest 3.Password: use this value: guest 4.Collection: use this value: /db 4.Click OK to finish the connection configuration.

15. 15 eXist Collections eXist organizes all documents in hierarchical collections. Collections are like directories. They are used to group related documents together. On the last slide you set the default collection name to /db

16. 16 Add a new Collection to the Database Switch on Oxygen's Database Perspective: Perspective -> Database Right-click on db, and select Add a collection…

17. 17 Add a new Collection to the Database Type in example01

18. 18 Put a Resource (XML File) into the Collection Right-click on example01, and select Add Resource …

19. 19 Put a Resource (XML File) into the Collection Select FitnessCenter.xml in the xquery/eXist-examples/example01 folder

20. 20 Editing Resources Double clicking on FitnessCenter.xml allows you to view and edit it:

21. 21 Run an XQuery on the XML Drag and drop FitnessCenter.xq into Oxygen. Modify the XQuery to use the doc() function (i.e., use an explicit input):

22. 22 Specify the Transformer Click on the wrench icon then click on the edit button:

23. 23 Specify the Transformer Select eXist-Connection in the Transformer drop-down box:

24. 24 Execute the XQuery Click on the red triangle:

25. 25 View the Results

26. 26 Remember to Restart eXist Suppose that you shut down your computer for the day. Tomorrow you Open Oxygen and try to run an XQuery on FitnessCenter.xml in the eXist database. You get an error. Why? Answer: you need to restart eXist before you open Oxygen. –Start All Programs eXist XML Database eXist Database Startup (alternatively, double click startup.bat in c:\eXist\bin)

27. 27 XQuery Update in eXist eXist supports XQuery update in a slightly non- standard way. You must precede each update operation with update, e.g., for $i in doc('FitnessCenter.xml')//(*|@*) return update rename node $i as lower-case(name($i)) Do Lab1