1. Native XML Database for Information Systems Chris Wallace IS School Research Seminar Feb 2006

2. Chris Wallace, IS School Research Seminar, Feb 2006 2 Exploring the design space “design as a conversation with the materials in the situation” (Schon) Native XML database (NXD) –Storing, querying and updating XML documents without mapping into relations –Schema-free –Trees are to NXD what tables are to RDBMS –Tables are trees Information Systems –Focus on semi-structured data (mixture of simple data items, text and complex nested structures) –Searching, derived data, visualisation –Process support –Large problem space variously supported by spreadsheets, word documents, ad-hoc databases, increasingly web-integrated data.

3. Chris Wallace, IS School Research Seminar, Feb 2006 3 eXist Native XML Database Open source Java European team of developers led by Wolfgang Meier Documents (files) are organised in collections (folders) in a file store –XML Documents stored in an efficient, B+ tree structure with indexes –Non-XML resources (XQuery, CSS, JPEG..), etc can be stored as binary Deployable in different ways –Embedded in a Java application –Part of a Cocoon pipeline –As web application in Apache/Tomcat –With embedded Jetty HTTPserver (as on stocks) Multiple Interfaces –REST – to Java servlet –SOAP –XML:RPC

4. Chris Wallace, IS School Research Seminar, Feb 2006 4 NXD case studies FOLD –modules, programmes, scheme operations, staff, organisational structures, events Family photos and history –Integration of meta-data on family photos with family history (births, deaths and marriages)meta-databirths, deaths and marriages ISD3 Assignment –a web-based calculator –e.g. a currency convertercurrency converter The language patterns book

5. Chris Wallace, IS School Research Seminar, Feb 2006 5 ISD3 Coursework Develop a simple web-based calculator Not just a programming exercise –User interface design Users language, units, not raw data User interaction design –Data design choice of representation of domain facts –Veracity Relationship between data in database and domain being modelled How is veracity monitored and maintained –Process Examine some of the XP processes –Test-driven development

6. Chris Wallace, IS School Research Seminar, Feb 2006 6 Application Design Data model is one simple table: –Currency code, name and symbol –Latest conversion rates from GBP to currency X Currency Coding –Use ISO4217 e.g. XE.COM list ISO4217XE.COM list Core algorithm: –Conversion from N X to ? Y is N * rate(X to GBP) * rate(GBP to Y) i.e. N * (1/rate (GBP to X) * rate(GBP to Y) Currency rates to be updated by an administrator – (not via a web-service) Interface is to be a sticky form: –Input form and output result on one page –Input form default values from last input Veracity management –Rates must be dated and sourced –

7. Chris Wallace, IS School Research Seminar, Feb 2006 7 Technical Decisions Choice of platform: –PL/SQL and Oracle –ASP.NET and SQL Server –JSP (Java Servlet Page) and JDBC to Postgres(say) –PHP and MySQL –XML and Native XML Database (eXist) Calculation location: –client-side in ECMAScript (aka JavaScript) –server-side with/without Ajax

8. Chris Wallace, IS School Research Seminar, Feb 2006 8 Two approaches PHP-MySQL –Define and create MySQL table –Write PHP script to provide interface and access the database using SQL –Write editor for the Currency table XML –Create MS Excel spreadsheet of currencies –Convert to XML in Excel and save –Write XQuery script to provide interface

9. Chris Wallace, IS School Research Seminar, Feb 2006 9 Development Process for currency converter XP Practices: –‘Spike’ Simple end to end implementation –Incremental development Setup eXist databaseeXist database –Using the Admin interface: Create a directory for application Create a subdirectory for currency data Create XML dataset(s) in Excel Upload to eXist Write the XQuery script cur.xqlcur.xql Upload to eXist Execute in browser

10. Chris Wallace, IS School Research Seminar, Feb 2006 10 Currency Data in XML Start MS Excel 2003 Create the spreadsheet with column headings Convert to List (needs XML add-in) Save as XML dataXML data CodeNameRate GBPSterling1 USDUS Dollars1.7423 EUREuro1.46331 NZDNZ Dollars2.60665 JPYYen205.852

11. Chris Wallace, IS School Research Seminar, Feb 2006 11 XQuery W3C candidate recommendation –http://www.w3.org/TR/xquery/http://www.w3.org/TR/xquery/ Designed by, amongst others, Don Chamberlin –http://www.research.ibm.com/journal/sj/414/chambe rlin.pdfhttp://www.research.ibm.com/journal/sj/414/chambe rlin.pdf A functional programming Language –Based on XPath (tree-access language) –Integrate, select, update, compute and construct XML documents –cf PL/SQL http://www.w3.org/XML/Query/

12. Chris Wallace, IS School Research Seminar, Feb 2006 12 Write the XQuery script Use the admin interface to test simple queries Use a syntax aware editor if possible –JEdit –Dreamweaver –Java Client interface to eXist –PFE32

13. Chris Wallace, IS School Research Seminar, Feb 2006 13 Executing an XQuery eXist DB cur.xql XQuery Engine parameters html Client BrowsereXist: Server Get cur.xql + parameters servlet fetch cur.xql render User clicks link cur.xql?fromAmount=100&fromCode=USD&toCode=EUR XSLT

14. Chris Wallace, IS School Research Seminar, Feb 2006 14 XQuery Script (1) declare namespace request="http://exist-db.org/xquery/request"; let $fromAmount := request:request-parameter("fromAmount",“100"), $fromCode := request:request-parameter("fromCode","GBP"), $toCode := request:request-parameter("toCode","EUR"), $currencies := doc('/db/calculator/currencyTable.xml')//Currency, $fromCurrency := $currencies[Code=$fromCode], $toCurrency := $currencies[Code=$toCode], $toAmount := round(xs:decimal($fromAmount) * xs:decimal($toCurrency/Rate) div xs:decimal($fromCurrency/Rate) ) return

15. Chris Wallace, IS School Research Seminar, Feb 2006 15 XQuery Script (1) declare namespace request="http://exist-db.org/xquery/request"; let $fromAmount := request:request-parameter("fromAmount",“100"), $fromCode := request:request-parameter("fromCode","GBP"), $toCode := request:request-parameter("toCode","EUR"), $currencies := doc('/db/calculator/currencyTable.xml')//Currency, $fromCurrency := $currencies[Code=$fromCode], $toCurrency := $currencies[Code=$toCode], $toAmount := round(xs:decimal($fromAmount) * xs:decimal($toCurrency/Rate) div xs:decimal($fromCurrency/Rate) ) return … Default Return node sequence of all Currency elements in this doc Filter Condition Cast

16. Chris Wallace, IS School Research Seminar, Feb 2006 16 XQuery Script (2) return Currency Calculator Currency Calculator Amount to Convert <input type="text" name="fromAmount“ value="{$fromAmount}"/> Embedded XQuery XML Current script is called by default

17. Chris Wallace, IS School Research Seminar, Feb 2006 17 XQuery (3) From Currency {for $currency in $currencies let $code := data($currency/Code), $name := data($currency/Name) return if ($code = $fromCode) then {$name} else {$name} } FLWOR expression conditional

18. Chris Wallace, IS School Research Seminar, Feb 2006 18 XQuery (4) Converts to {$toAmount} Switch to XQuery again, In PHP this would be either $toAmount (if in PHP) or (if in HTML)

19. Chris Wallace, IS School Research Seminar, Feb 2006 19 Round two - enhancements Add another currency –ZAR Rand 10.4767 Add new columns –Meta data to convey the accuracy, timeliness and origin of the data itself –Source and date/time Update spreadsheet –Add columns and data Update XQuery script – cur2.xqlXQuery script cur2.xql –Add source – Sources and oldest date

20. Chris Wallace, IS School Research Seminar, Feb 2006 20 Round 3 – Currency Table Same document used for different purpose: –currency.xslcurrency.xsl –curtable.xqlcurtable.xql –Run it curtable.xqlcurtable.xql

21. Chris Wallace, IS School Research Seminar, Feb 2006 21 The Language Patterns book Local / Web-basedLocalWeb-based Work-in-progress Should be converted to XML –currently organised by Pattern –Language –But should be able to be viewed as Language –Pattern –An XML database would solve this problem As an exercise for next week, you will extend the number of languages with ActionScript –Get you familiar with the on-line book –Extend the book