Copyright 2000, 2001, Ronald Bourret, Mapping DTDs to Databases Ronald Bourret

Copyright 2000, 2001, Ronald Bourret, Overview Table-based mappings Object-based mappings Generating relational schemas from DTDs Generating DTDs from relational schemas Mapping XML Schemas to databases

Copyright 2000, 2001, Ronald Bourret, Table-based Mappings

Copyright 2000, 2001, Ronald Bourret, There is an obvious mapping from this XML document... ccc ddd eee fff ggg hhh

Copyright 2000, 2001, Ronald Bourret, to this table ccc ddd eee fff ggg hhh Table A C D E ccc ddd eee fff ggg hhh

Copyright 2000, 2001, Ronald Bourret, How does the mapping work? Views the XML document as a single table

Copyright 2000, 2001, Ronald Bourret, How does the mapping work?... or a set of tables

Copyright 2000, 2001, Ronald Bourret, Table-based Mappings Pros: »Easy to understand »Easy to write code to transfer data »Efficient way to transfer data between relational databases Cons: »Only works for a small subset of XML documents Used by data transfer middleware such as ASP2XML, DatabaseDOM (IBM), DB2XML, DBIx::XML_RDB, ODBC Socket Server, and XML-DB Link

Copyright 2000, 2001, Ronald Bourret, Object-based Mappings

Copyright 2000, 2001, Ronald Bourret, There is also an obvious mapping from this XML document... bbb ccc ddd

Copyright 2000, 2001, Ronald Bourret, to this object... bbb ccc ddd object A { B = "bbb" C = "ccc" D = "ddd" }

Copyright 2000, 2001, Ronald Bourret, to a row in this table bbb ccc ddd object A { B = "bbb" C = "ccc" D = "ddd" } Table A B C D bbb ccc ddd

Copyright 2000, 2001, Ronald Bourret, And an obvious mapping from this element type definition...

Copyright 2000, 2001, Ronald Bourret, to this class... class A { String B; String C; String D; }

Copyright 2000, 2001, Ronald Bourret, to this table schema class A { String B; String C; String D; } CREATE TABLE A ( B VARCHAR(10) NOT NULL, C VARCHAR(10) NOT NULL, D VARCHAR(10) NOT NULL )

Copyright 2000, 2001, Ronald Bourret, A more complex example 1234 Gallagher Industries A B This XML document...

Copyright 2000, 2001, Ronald Bourret, A more complex example object SalesOrder { number = 1234; customer = "Gallagher Industries"; date = ; lines = {ptrs to Line objects}; } object Line { number = 1; part = "A-10"; quantity = 12; price = 10.95; } object Line { number = 2; part = "B-43"; quantity = 600; price = 3.95; }... maps to these objects...

Copyright 2000, 2001, Ronald Bourret, SaleOrders Number Customer Date 1234 Gallagher Industries Lines SONumber Line Part Quantity Price A B A more complex example... which map to these rows

Copyright 2000, 2001, Ronald Bourret, Objects are data-specific... Different for each DTD (schema) Model the content (data) of the document Orders SalesOrder Customer Line Part

Copyright 2000, 2001, Ronald Bourret, not the DOM Same for all XML documents Model the structure of the document Element (Orders) Element Attr (SalesOrder) (SONumber) Element Element Element (Customer) (OrderDate) (Line)

Copyright 2000, 2001, Ronald Bourret, How does the mapping work? 1. Map a DTD to an object schema 2. Map the object schema to a database schema »Direct mapping to object-oriented databases »Object-relational mapping to relational databases 3. (Optional) Combine steps (1) and (2) for a DTD-to-database mapping

Copyright 2000, 2001, Ronald Bourret, How does data transfer work? With intermediate objects: 1. Transfer data from an XML document to a tree of objects 2. Transfer data from objects to the database Without intermediate objects: 1. Transfer data directly from an XML document to the database Whether intermediate objects are useful depends on the application

Copyright 2000, 2001, Ronald Bourret, The Basic Mapping

Copyright 2000, 2001, Ronald Bourret, Element types are data types “Simple” element types have PCDATA-only content “Complex” element types have element or mixed content and/or attributes <!ATTLIST Customer CustNum CDATA #REQUIRED Name CDATA #REQUIRED Address CDATA #REQUIRED>

Copyright 2000, 2001, Ronald Bourret, Mapping “complex” element types Map complex element types to classes which are mapped to tables (class tables) class A {... } class A {... } Table A...

Copyright 2000, 2001, Ronald Bourret, Mapping content models Map references to simple element types to scalar properties which are mapped to data columns class A { String b;... } class A { String b;... } Table A Column b...

Copyright 2000, 2001, Ronald Bourret, Mapping content models (cont.) Map references to “complex” element types to pointer/reference properties which are mapped to primary / foreign key columns class A { String b; C c;... } Table A Column b Column c... Table C Column c... class C {... } class A { String b; C c;... } class C {... }

Copyright 2000, 2001, Ronald Bourret, Mapping attributes Map attributes to scalar properties which are mapped to data columns class A { String b; C c; String f; } class A { String b; C c; String f; } Table A Column b Column c Column f Table C Column c...

Copyright 2000, 2001, Ronald Bourret, The basic mapping: summary Map “complex” element types to classes, then to tables Map content models to properties, then to columns Map attributes to properties, then to columns Join class tables with primary key / foreign key pairs

Copyright 2000, 2001, Ronald Bourret, Some Important Points

Copyright 2000, 2001, Ronald Bourret, References are not definitions References must be mapped separately for each content model class Chapter { String title; String[] section; } class Appendix { String title; String[] section; }

Copyright 2000, 2001, Ronald Bourret, Data types Can map “simple” data types to any scalar data type class Part { String number; float price; } CREATE TABLE Part ( number CHAR(10) NOT NULL, price REAL NOT NULL }

Copyright 2000, 2001, Ronald Bourret, Names can be mapped, too DTD, object schema, and relational schema can use different names class PartClass { String numberProp; float priceProp; } CREATE TABLE PRT ( PRTNUM CHAR(10) NOT NULL, PRTPRICE REAL NOT NULL )

Copyright 2000, 2001, Ronald Bourret, Primary and foreign keys Primary key on “one” side of one-to-many relationship May be in table of parent or child /29/00 1 ABC Table Sales Number, Date Table Parts Number, Price Table Lines SONum, Num, Part, Qty

Copyright 2000, 2001, Ronald Bourret, Mapping Complex Content Models

Copyright 2000, 2001, Ronald Bourret, Complex content models How do you map the following? <!ELEMENT A (B?, (C | ((D | E | F | G)*, (H | I)+, J?)))>

Copyright 2000, 2001, Ronald Bourret, Sequences Map each reference in a sequence to a property......which map to tables and columns as appropriate class A { String b; String c; D d; } class A { String b; String c; D d; } Table A Column b Column c Column d Table D Column d...

Copyright 2000, 2001, Ronald Bourret, Choices Map each reference in a choice to a property which map to nullable columns class A { String b; String c; D d; } class A { String b; String c; D d; } CREATE TABLE A ( b VARCHAR(10), c VARCHAR(10), d INTEGER } No NOT NULL clause

Copyright 2000, 2001, Ronald Bourret, Choices (cont.) Property values can be null so column values can be NULL bbb object a { b = "bbb" c = null d = null } Table A b c d bbb NULL NULL object a { b = "bbb" c = null d = null }

Copyright 2000, 2001, Ronald Bourret, Repeated children Map to multi-valued properties Map repeated references to arrays of known size which map to multiple columns (shown) or separate tables class A { String[3] b; String c; } class A { String[3] b; String c; } Table A Column b1 Column b2 Column b3...

Copyright 2000, 2001, Ronald Bourret, Repeated children (cont.) Map references with * or + operator to arrays of unknown size which map to separate tables (property tables) class A { String[] b; String c; String d } class A { String[] b; String c; String d } Table A Column a Column c Column d Table B Column a Column b

Copyright 2000, 2001, Ronald Bourret, Optional children Map to nullable properties, then to nullable columns Applies to children in a choice and to children with ? or * operator class A { String b; // May be null String[] c; // May be null D d; } class A { String b; // May be null String c; // May be null D d; // May be null }

Copyright 2000, 2001, Ronald Bourret, Subgroups Map references in subgroup to properties of parent class... which map to columns in class table Table A Column b, Column c, Column d class A { String b; String c; D d; } class A { String b; String c; D d; }

Copyright 2000, 2001, Ronald Bourret, Subgroups (cont.) Works because elements in subgroup are still children of parent bbbbbb cccccc object a { b = "bbbbbb" c = "cccccc" d = null } bbbbbb eee fff object a { b = "bbbbbb" c = null d = ptr. to object d }

Copyright 2000, 2001, Ronald Bourret, Subgroups (cont.) Repeatability and optionality can be indirect class A { String b; String[] c; // May be null D[] d; // May be null String[] e; // May be null }

Copyright 2000, 2001, Ronald Bourret, Mapping Mixed Content

Copyright 2000, 2001, Ronald Bourret, Model PCDATA as elements This text cc makes bbbb no sense cccc except as bb an example. This text cc makes bbbb no sense cccc except as bb an example.

Copyright 2000, 2001, Ronald Bourret, Mapping mixed content Map PCDATA and element references to arrays of unknown size which are mapped to property tables class A { String[] pcdata; String[] b; String[] c; } class A { String[] pcdata; String[] b; String[] c; } Table A Column a Table B Column a Column b Table PCDATA Column a Column pcdata Table C Column a Column c

Copyright 2000, 2001, Ronald Bourret, Mixed content example This text cc makes bbbb no sense cccc except as bb an example. object a { pcdata = {"This text ", " makes ", " no sense ", " except as", " an example."} b = {"bbbb", "bb"} c = {"cc", "cccc"} }

Copyright 2000, 2001, Ronald Bourret, Mixed content example (cont.) object a { pcdata = {"This text ", " makes ", " no sense ", " except as", " an example."} b = {"bbbb", "bb"} c = {"cc", "cccc"} } Table A a 1 Table B a b 1 bbbb 1 bb Table C a c 1 cc 1 cccc Table PCDATA a pcdata 1 This text 1 makes 1 no sense 1 except as 1 an example.

Copyright 2000, 2001, Ronald Bourret, Mapping Sibling Order

Copyright 2000, 2001, Ronald Bourret, Siblings Sibling means “brother or sister” Sibling elements and text have the same parent A This text C makes B no sense C except as B an example cc bbbb cccc bb This text cc makes bbbb no sense cccc except as bb an example

Copyright 2000, 2001, Ronald Bourret, Sibling order Sibling order is order in which siblings occur Sibling order is different from hierarchical order... A This text C makes B no sense C except as B an example cc bbbb cccc bb A This text C makes B no sense C except as B an example cc bbbb cccc bb

Copyright 2000, 2001, Ronald Bourret, Sibling order (cont.)... and from document order A 1 This text C makes B no sense C except as B an example cc bbbb cccc bb

Copyright 2000, 2001, Ronald Bourret, Is sibling order important? Usually not important in data-centric applications Exception: Document validated against DTD 123 Turkey wrench object part { number = 123 desc = "Turkey wrench" price = } Turkey wrench 123

Copyright 2000, 2001, Ronald Bourret, Is sibling order important? (cont.) Very important in document-centric applications Ronald Bourret is an excellent writer. Only an idiot wouldn’t read his work. Ronald Bourret is an idiot. Only an execellent writer wouldn’t read his work.

Copyright 2000, 2001, Ronald Bourret, Mapping sibling order Store order values in: »Order properties/columns »Mapping

Copyright 2000, 2001, Ronald Bourret, Order properties/columns One order property per element or PCDATA... class A { String[] pcdata; int[] pcdataOrder; String[] b; int[] bOrder; String[] c; int[] cOrder }

Copyright 2000, 2001, Ronald Bourret, Order properties/columns (cont.)... each of which is mapped to a separate column Table A Column a Table C a, c, cOrder Table PCDATA a, pcdata, pcdataOrder Table B a, b, bOrder class A { String[] pcdata; int[] pcdataOrder; String[] b; int[] bOrder; String[] c; int[] cOrder }

Copyright 2000, 2001, Ronald Bourret, Mixed content example All sibling order properties share the same order space This text cc makes bbbb no sense cccc except as bb an example. object a { pcdata = {"This text ", " makes ", " no sense ", " except as", " an example."} pcdataOrder = {1, 3, 5, 7, 9} b = {"bbbb", "bb"} bOrder = {4, 8} c = {"cc", "cccc"} cOrder = {2, 6} }

Copyright 2000, 2001, Ronald Bourret, Mixed content example (cont.) Table A a 1 Table B a b order 1 bbbb 4 1 bb 8 Table C a c order 1 cc 2 1 cccc 6 Table PCDATA a pcdata order 1 This text 1 1 makes 3 1 no sense 5 1 except as 7 1 an example. 9 object a { pcdata={"This text ", " makes ", " no sense ", " except as", " an example."} pcdataOrder={1,3,5,7,9} b={"bbbb","bb"} bOrder={4,8} c={"cc","cccc"} cOrder={2,6} }

Copyright 2000, 2001, Ronald Bourret, Element content and order properties / columns Element content can use order properties / columns class A { String b; int bOrder; String c; int cOrder; D d; int dOrder; } Table D a, dOrder,... Table A a, b, bOrder, c, cOrder

Copyright 2000, 2001, Ronald Bourret, Storing order in the mapping Order values can be stored in the mapping Children of same type must be groupable No ordering within type (data-centric documents only) OK: Not OK: Element Type Order B 1 C 2 D 3

Copyright 2000, 2001, Ronald Bourret, Mapping Attributes

Copyright 2000, 2001, Ronald Bourret, Single-valued attributes CDATA, ID, IDREF, NMTOKEN, ENTITY, NOTATION, and enumerated Map to scalar-valued properties which map to columns class A { String b; String c; String d; } class A { String b; String c; String d; } Table A Column b Column c Column d

Copyright 2000, 2001, Ronald Bourret, Multi-valued attributes IDREFS, NMTOKENS, and ENTITIES Map to arrays of unknown size which map to property tables class A { String b; String c; String[] d; } class A { String b; String c; String[] d; } Table A Column a Column b Column c Table D Column a Column d

Copyright 2000, 2001, Ronald Bourret, Order of attributes Not significant according to XML Information Set No order properties needed <A B="bbb" C="ccc" D="ddd"/> <A C="ccc" D="ddd" B="bbb"/> =

Copyright 2000, 2001, Ronald Bourret, Order in multi-valued attributes One order property per multi-valued attribute Separate order space for each multi-valued attribute <!ATTLIST A B IDREFS #IMPLIED C NMTOKENS #IMPLIED> class A { String[] b; int[] bOrder; String[] c; int[] cOrder; } <A B="dd ee ff" C="gg hh"/> object a { b = {"dd", "ee", "ff"} bOrder = {1, 2, 3} c = {"gg", "hh"} cOrder = {1, 2} }

Copyright 2000, 2001, Ronald Bourret, ID/IDREF(S) Attributes Map to primary key / foreign key relationship “Decorate” IDs if not unique across all documents A B C D Table A a... Table B a ref_d Table C a ref_d... Table D id...

Copyright 2000, 2001, Ronald Bourret, Alternate Mappings

Copyright 2000, 2001, Ronald Bourret, Map complex element types to scalar types Map references to complex element types to scalar properties, then to columns Value is XML (e.g. XHTML) <!-- Use Inline entity from XHTML --> class Part { String num; String desc; } 127 A very big turkey wrench. Table Part Num Desc 127 A very big turkey wrench.

Copyright 2000, 2001, Ronald Bourret, Map scalar properties to tables Useful for storing BLOBs separately Table Parts Column num Table Descriptions Column num, Column desc class Part { String num; String desc; }

Copyright 2000, 2001, Ronald Bourret, Additional Comments

Copyright 2000, 2001, Ronald Bourret, What isn’t mapped? Physical structure and information: »Character and entity references »CDATA sections »Character encodings »Standalone declaration Document information: »Document type »DTD Other: »Comments »Processing instructions

Copyright 2000, 2001, Ronald Bourret, Pros and cons Pros: »Mapping model works for all XML documents »Preserves logical structure of data in XML document »Provides basis for easy-to-use, model-driven software »Round-tripping at the element/attribute/text level Cons: »Discards physical structure information »Inefficient for mixed content or deeply nested data Ideal for data-centric applications Poor for document-centric applications

Copyright 2000, 2001, Ronald Bourret, Widely used Relational databases such as Oracle 8i, IBM DB2, Informix, and Microsoft SQL Server Data transfer middleware such as ADO, XML SQL Utility for Java (Oracle), XML-DBMS, DB/XML Vision, and InterAccess Object servers such as Castor, Object Translator (Informix), and Total-e-Business (Bluestone) Varying levels of implementation

Copyright 2000, 2001, Ronald Bourret, Generating Database Schema from DTDs

Copyright 2000, 2001, Ronald Bourret, Example Part PartPK Order OrderPK Line LinePK 1) Generate class tables and prim. keys for complex element types

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Part PartPK, PartNum, Price Order OrderPK, OrderNum, Date, CustNum Line LinePK, LineNum, Quantity 2) Generate columns for single references to simple element types

Copyright 2000, 2001, Ronald Bourret, Example (cont.) 3) Generate foreign keys for references to complex element types Part PartPK, PartNum, Price, LineFK Order OrderPK, OrderNum, Date, CustNum Line LinePK, LineNum, Quantity, OrderFK

Copyright 2000, 2001, Ronald Bourret, Generating database schema from DTDs Process element types »Complex element types generate class tables and primary keys Process content models »Single references to simple element types generate columns »Repeated references to simple element types generate property tables with foreign keys »References to complex element types generate foreign keys in remote class tables »PCDATA in mixed content generates a property table with a foreign key

Copyright 2000, 2001, Ronald Bourret, Generating database schema from DTDs (cont.) Process attributes »Single-valued attributes generate columns »Multi-valued attributes generate property tables with foreign keys

Copyright 2000, 2001, Ronald Bourret, Generating DTDs from Database Schema

Copyright 2000, 2001, Ronald Bourret, Example Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 1) Generate element type for root table

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 2) Generate PCDATA-only elements for each data column

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 3) Generate PCDATA-only element for primary key

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 4) Add element for table to which primary key is exported

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 5) Process remote table (data columns and primary key column)

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 6) Add element for table to which foreign key corresponds

Copyright 2000, 2001, Ronald Bourret, Example (cont.) Parts PartNum, Price Orders OrderNum, Date, CustNum Lines OrderNum, LineNum, Quantity, PartNum 7) Process remote table (data columns and primary key column)

Copyright 2000, 2001, Ronald Bourret, Generating DTDs from database schema Process table »Generate element type with sequence content model Process data columns »Generate PCDATA-only elements »Add references to generated elements to sequence

Copyright 2000, 2001, Ronald Bourret, Generating DTDs from database schema (cont.) Process primary / foreign key columns »If key is primary key, optionally: Add PCDATA-only element types for columns in key Add references to element types to sequence »Add reference to remote element type to sequence If key is primary key, reference is optional/multiple (* operator) If key is foreign key and is nullable, reference is optional (? operator) »Process remote table

Copyright 2000, 2001, Ronald Bourret, Generation problems Naming problems (both directions) »Collisions »Illegal names »Names don’t necessarily make sense DTD => database schema »Can’t predict data types or lengths »Can’t recognize element types/attributes to use as keys »Can’t determine if primary key is in parent or child »DTD often has excess structure Database schema => DTD »Can’t recognize order columns or property tables Can’t round-trip

Copyright 2000, 2001, Ronald Bourret, Mapping XML Schemas to Databases

Copyright 2000, 2001, Ronald Bourret, Mapping XML Schemas to databases Map complex element types to classes (sort of) »Map complex type extension to inheritance Map simple data types to scalar data types »Many facets can’t be mapped »Map wildcards to Java Object, C++ pointer to void, etc. Treat “all” groups as unordered sequences Treat substitution groups as choices Map most identity constraints to keys

Copyright 2000, 2001, Ronald Bourret, Resources Ronald Bourret’s Papers Page » XML:DB.org’s Resources Page » XML:DB Mailing List »

Copyright 2000, 2001, Ronald Bourret, Questions? Ronald Bourret