Integrating a Heterogeneous Environment using XML Sandeepan Banerjee Director, Oracle Server Technologies

The Problem  Multiple domain-specific applications – Manufacturing, Inventory, Supply Chain, Financial, …  Information is trapped within these applications How does an organization get a consolidated view of its information – in real time ? CRM Application Financial Application Contract Management Application Synthesized Information Excel Files on Disk and Document Repositories

Technical Challenges  Domain-specific information – Replication does not make sense  Independent operation of applications – Access to information has to be in real-time  Different access method for each application – Each application has its own protocol and access method What architecture can best accommodate my present and future needs ? – Complexity: Avoid the n by m matrix – Flexibility: Add new sources easily – Time to market: Within days, not months

What is XML-based Data Integration? XML-based Data Integration or Enterprise Information Integration (EII):  Create aggregated views using XQuery  Get information from diverse sources in XML  Consume synthesized information

DatabaseWeb Service J2EE TM CA JDBCHTTP XQuery Engine Order Tracking Shipment Tracking XML Data Integration Example Synthesized Information EIS Parts Inventory

XML-based Data Integration J2EE TM Platform Technologies Involved  Why XML? – Different data formats  Why XQuery ? – Declarative way to query XML documents  Why J2EE TM ? – Standards-based infrastructure platform  Why XML Database ? – Native XML storage – XML data management – Performance optimizations XML Database XQuery Engine

Comparison with Existing Technologies DifferenceSimilarity Application Integration More about data pumping and synchronization among systems Involves data adapters, data translation and transformation Data Warehousing Explicit ETL steps required; large data volume; batch loading, not real-time Optional cache pre-population step similar to the “loading” step Traditional Report Generation SQL & relation-based vs. XQuery & XML-based Cannot handle non- relational sources Query-based

XQuery Status XQuery is emerging as the consensus ‘ native ’ query language for XML Expected W3C Recommendation in late 2005 Oracle 10gR2 is the first mainstream commercial database release to support XQuery Plan to release under an event in 10gR2

XQuery Example Assume a document – emp.xml To get the names of employees with salary > for $i in document(‘emp.xml’)/empset let $j = where > $j return Result (attribute node) JONES

Another Example Customer/Address/Zip ShipTo/Address/Zip Supplier/Location/AddressForTaxCalculation/Zip Customer/Address/Work/City/Zip Customer/Address/Home/City/Zip Shipper/DropOffLocation/US/California/SFO/Zip + any other zip nested to any unanticipated level count ( for $i in doc("contacts.xml")/Contact where $i//zip eq return $i)

Example III: Auction/Bids  Say we have 3 heterogeneous data sources that yield their contents as XML – Item Description comes from a Filesystem – User Information comes from LDAP/DB – Bid Information comes from an App Server -- For all bicycles, list the item number, description, highest bid (if any), ordered by item no. for $i in doc("items.xml")//item_tuple let $b := doc("bids.xml")//bid_tuple[itemno = $i/itemno] where contains($i/description, "Bicycle") order by $i/itemno return { $i/itemno} { $i/description } { max($b/bid) }

XQuery and SQL  Existing relational applications will continue to use SQL, and Oracle will remain the industry’s best implementation of SQL  New applications based on XML will use XQuery, and Oracle will be the industry’s best implementation of XQuery  Oracle will support XQuery both in the database and in the mid-tier – Use the mid-tier engine when you want to query non- database sources  Intelligent ‘query pushdown’ from mid-tier to db when possible  Can mix-and-match XQuery and SQL in same query

XQuery is different from SQL  Navigation-oriented (using Xpath expressions)  Different type system (XMLSchema based simple types)  Identity-based (XML Node identities and document order)  Namespace aware name-resolution (functions, variables, element creation)  XML-Item based vs Row-based  Results are heterogeneous sequences  Does not have all SQL extensions (e.g. data warehousing etc..)

XQuery Mid-tier Architecture XQuery Java Engine XQJ API Driver DB Drivers JDBC Driver Java XMLType XQuery XQueryX XPath SQL + XQuery or XQueryX Other Data sources

XQuery DB Architecture Mid-Tier XQuery Engine User XQuery Aware SQL Engine SQL Compiler Optimizer, Execution Engine Normalization XQuery Type check Compiler XQuery Parser XMLQuery, XMLTable SQL XQJ API XQuery SQLX/XPath

D E M O N S T R A T I O N XML Query

Oracle XML Database (XML DB)  Native XML storage – Available since Oracle Database Release 9.2  Inherits RDBMS features: Security, Transaction,…  XML-specific features – XML indexing, XPath & XSLT support, XML schema validation, XML partial update  Supports SQL/XML – Allows blending relational and XML data operations

Leveraging Oracle XML DB  XML DB can be an XQuery source – Can define XML views of relational data – XQuery engine can rewrite query into SQL/XML  XML DB could also be used for caching – Efficient storage & indexing for large data sets  Can leverage security framework of XML DB – For both source and cache

Datasources  Databases – Relational+ XML Views, Object-Relational, CLOB, Compact XML (future)  Mid-tier sources – files, cache, JCA datasources  Bind (an existing DOM)  xmldatasrc – Oracle language addition  Datasource API – initialize – describe – execute – Fetch

iAS XDS Architecture XDS Client API’s EJB JSP Tags Web Service XML Data Synthesis Cached XML Data Source Applications using XDS e.g. Portals, Reports etc Query Builder Tool Meta-data Repository Oracle Enterprise Manager XML DB JCache File system J2EE Security Framework XDS Cache Security XDS Caching Service XML Data source adaptors CCI-XML Web Services J2CA EAI SAP Oracle Apps JMS RDBMS Files HTTP Web Cache Java Functions XQuery Subsystem XQ4J/JXQI XQuery Engine XMLDataSource modules XQuery Result In- Mem ory Stored Query

Example – XDS usage  User registers webservice as datasource  XDS creates an XQuery module automatically  User Query for querying webservice import module namespace wss=“datasrc/stockws”; for $i in wss:getcompanies() return wss:get_stock_price($i/name) wss – namespace prefix for the loaded module JNDI lookup to get datasource implementation XDS adapters implement datasource

All Your Searches!  Search with any language – SQL/XQuery/XSL  Search anywhere - mid tier or backend  Search anything -any XML/relational content  Search everything – XML visualization of all data (backend) – XML based adapters provide XML content for all data in mid- tier (XDS)  Search any form – text based/structured  Search any size - Scalable solution  Search any time - Unbreakable solution

More Information  XML in general –  XML Query –  XML DB –

A Q & Q U E S T I O N S A N S W E R S