1. Copyright 2000, Microsoft Corp. SQL Server 2000 XML Annotated Schemata Michael Rys Program Manager SQLServer XML Technologies Microsoft Corporation mrys@microsoft.com

2. Copyright 2000, Microsoft Corp. Scenarios Web applications with Dynamic Data Web applications with Dynamic Data  Browser based applications that require data from a database Business-to-business data processing; messaging Business-to-business data processing; messaging  Data interchange using XML as ubiquitous, extensible, platform independent transport format

3. Copyright 2000, Microsoft Corp. Loosely Coupled Systems Scalable. Many to Many. Scalable. Many to Many.  Changes in Implementation do not break each other AppLogic Data MappingMapping Application System Data System Object(XML)  Move data in a standardized format:  XML Syntax  XML Schemata for domain standard

4. Copyright 2000, Microsoft Corp. Challenges Providing XML views to relational tables Providing XML views to relational tables Allow efficient querying of XML views Allow efficient querying of XML views Allow updating of XML views Allow updating of XML views Provide other useful applications of XML views: Provide other useful applications of XML views:  Bulkload  Relational schema generation

5. Copyright 2000, Microsoft Corp. Orders ID 222 Cust 243 Order Lines Order ID 222 Line No 1 Item dt334 Quantity 1 View Example "" <order ordid= " 222 " "" cust= " 243 " > "" <line qty= " 1 " "" item= " dt334 " />

6. Copyright 2000, Microsoft Corp. XML Views Defines an XML View on the database Defines an XML View on the database Uses XML-Data Reduced (XDR) syntax with annotations (mapping schemas) Uses XML-Data Reduced (XDR) syntax with annotations (mapping schemas)  Similar to DTD but using XML grammar  Public schema grammar used by BizTalk Annotations specify the XML to relational database mapping Annotations specify the XML to relational database mapping  For column values  For relationships between contained tags XML View Mapper XML View Mapper XML View Mapper XML View Mapper

7. Copyright 2000, Microsoft Corp. "" "" <Schema name="Orders" "" xmlns="urn:schemas-microsoft-com:xml-data" "" xmlns:dt="urn:schemas-microsoft-com:datatypes"> """" "" "" Default Mapping

8. Copyright 2000, Microsoft Corp. Default Mapping Element name matches table name Element name matches table name Attribute and textOnly subelement names match column names Attribute and textOnly subelement names match column names No way to map hierarchies: resulting XML is flat No way to map hierarchies: resulting XML is flat

9. Copyright 2000, Microsoft Corp. Annotations sql:relation (map table-element name) sql:relation (map table-element name) sql:field (map column-element/attribute name) sql:field (map column-element/attribute name) sql:relationship (hierarchies, joins) sql:relationship (hierarchies, joins) sql:limit-field & sql:limit-value (horizontal partitions) sql:limit-field & sql:limit-value (horizontal partitions) sql:key-fields (keys) sql:key-fields (keys) sql:id-prefix (XML prefix for IDs) sql:id-prefix (XML prefix for IDs) sql:is-constant (exists in XML, do not map to DB) sql:is-constant (exists in XML, do not map to DB) sql:map-field (exists in schema, do not map at all) sql:map-field (exists in schema, do not map at all) sql:overflow (column for unmapped XML) sql:overflow (column for unmapped XML) sql:use-cdata (generate CDATA section) sql:use-cdata (generate CDATA section) sql:url-encode (contains dbobject reference) sql:url-encode (contains dbobject reference) sql:datatype (SQL datatype hint) sql:datatype (SQL datatype hint)xmlns:sql=“urn:schemas-microsoft-com:xml-sql"

10. Copyright 2000, Microsoft Corp. Annotations <Schema xmlns="urn:schemas-microsoft-com:xml-data" xmlns:sql="urn:schemas-microsoft-com:xml-sql"> xmlns:sql="urn:schemas-microsoft-com:xml-sql"> <sql:relationship key-relation="Customers" <sql:relationship key-relation="Customers" key="CustomerID" key="CustomerID" foreign-relation="Orders" foreign-relation="Orders" foreign-key="CustomerID"/> foreign-key="CustomerID"/> </ElementType> </ElementType></Schema>

11. Copyright 2000, Microsoft Corp. XPath to define XML Views W3C Recommendation W3C Recommendation Allows to navigate XML trees: Allows to navigate XML trees:  Specifies set of nodes  Allows specification of predicates SQLServer 2000 XPath: SQLServer 2000 XPath:  Subset of W3C XPath 1.0 Recommendation  Defines virtual XML view over relational database together with Annotated Schemas  Usable in URLs, Templates, and in ADO/OLE-DB NOTE: OpenXML uses MSXML XPath implementation NOTE: OpenXML uses MSXML XPath implementation

12. Copyright 2000, Microsoft Corp. Syntax General Syntax General Syntax  /a::n[p]/b::m[q] where a::n[p] location step; a, b axes; m, n nodetests; p, q predicates  Shortform for most common axes Example: Example:  /child::Customer[attribute::State="WA"]  Shortform: /Customer[@State="WA"]

13. Copyright 2000, Microsoft Corp. Semantics General Semantics: General Semantics:  Select all nodes with name n for which predicate p holds and can be reached via axis a  p is existentially quantified root Customer @state=WA@state=CA@state=WAOrder /root/Customer[@state="WA"]/Order

14. Copyright 2000, Microsoft Corp. SQLServer 2000 XPath Support Axes: child, attribute, parent, self Axes: child, attribute, parent, self Boolean-valued predicates Boolean-valued predicates Relational operators =, !=,, >= Relational operators =, !=,, >= Arithmetic operators +, -, *, div Arithmetic operators +, -, *, div Boolean operators/functions AND, OR, true(), false(), not() Boolean operators/functions AND, OR, true(), false(), not() Explicit conversion functions number(), string(), boolean() Explicit conversion functions number(), string(), boolean() Variables Variables Not supported: See Books-OnLine Not supported: See Books-OnLine

15. Copyright 2000, Microsoft Corp. XPath and Annotated Schema SQL DB XPath Query /A/B/C First location step A defines instance of XML view over DB Annotated Schema Defines set of potential XML views over DB

16. Copyright 2000, Microsoft Corp. Querying XPath Query XPath Query  Used with a reference to the mapping schema  /Customer[@ID=“ALFKI”] Results in: Results in: </Customer>

17. Copyright 2000, Microsoft Corp. Functional XPath Limitations Not a full Query Language, only navigation Not a full Query Language, only navigation XPath cannot project XPath cannot project  Tailor annotated schema to project only accessible data  Use XSLT postprocessing to project data needed in query but not exposed in view XPath cannot prune subtrees (e.g., give me all customers of state X and only their orders from today) XPath cannot prune subtrees (e.g., give me all customers of state X and only their orders from today)  Static pruning: map to SQL views  Dynamic pruning: XSLT XPath cannot transform XPath cannot transform  Tailor annotated schema  Use XSLT to transform

18. Copyright 2000, Microsoft Corp. URL Query http://server/vroot/vname/xpath?params http://server/vroot/vname/xpath?params Virtual name (vname) Virtual name (vname)  Direct reference to mapping schema  Path to directory  http://server/vroot/vname/nwind.xml/xpath/?params XPath Query (xpath) XPath Query (xpath)  Specifies set of nodes from virtual document to return  Encode + etc. into %xx in URL Parameters (params) Parameters (params)  xsl, outputencoding, contenttype, root, …  User defined

19. Copyright 2000, Microsoft Corp. Template http://server/vroot/vname?params http://server/vroot/vname?params Same as template query with embedded SQL Query except: Same as template query with embedded SQL Query except:  Embedded query references mapping schema file  Query is expressed as XPath Supports querying over namespace qualified names Supports querying over namespace qualified names

20. Copyright 2000, Microsoft Corp. Template (Example) <root xmlns:sql="urn:schemas-microsoft- com:xml-sql" sql:xsl="path to XSLT file" > sql:xsl="path to XSLT file" > WA WA /Customers[@Region=$state] /Customers[@Region=$state] </root>

21. Copyright 2000, Microsoft Corp. XML Views - Template </root>

22. Copyright 2000, Microsoft Corp. Template Processing ISAPI Data SQL FOR XML Middle-Tier SQL Server 2000 URL CustState.xml Computed FOR XML EXPLICIT XML Computes Request from CustState.xml Using nwind.xdr nwind.xdr TDS/ XML

23. Copyright 2000, Microsoft Corp. BizTalk Integration Uses same schema format as BizTalk framework and server Uses same schema format as BizTalk framework and server BizTalkSchema copy BizTalkSchema Annotations + = MappingSchema SQLServer + XPATH/Customers Query XMLinBizTalkGrammar Results

24. Copyright 2000, Microsoft Corp. UpdateGrams Expresses Modifications to an XML document Expresses Modifications to an XML document Uses optimistic concurrency control Uses optimistic concurrency control Captures inserts, deletes, and updates Captures inserts, deletes, and updates Supports hierarchical updates to multiple tables using annotated schemas Supports hierarchical updates to multiple tables using annotated schemas Allows specification of null image Allows specification of null image Values can be parameterized (uses $param, see DB202 for syntax) Values can be parameterized (uses $param, see DB202 for syntax) Currently ships in beta as web release Currently ships in beta as web release

25. Copyright 2000, Microsoft Corp. <Customer CustomerID="LAZYK" CompanyName="ISNULL" <Customer CustomerID="LAZYK" CompanyName="ISNULL" Address="12 Orchestra Terrace" > Address="12 Orchestra Terrace" > <Customer CustomerID="LAZYK" <Customer CustomerID="LAZYK" CompanyName="Lazy K Country Store" CompanyName="Lazy K Country Store" Address="12 Opera Court" > Address="12 Opera Court" > </r>UpdateGrams

26. Copyright 2000, Microsoft Corp. Bulkload Allows loading of large amount of XML into database via annotated schema. Allows loading of large amount of XML into database via annotated schema. Allows generation of relational schema Allows generation of relational schema Provided as COM object Provided as COM object Can be called from DTS Can be called from DTS Uses SAX to parse XML Uses SAX to parse XML Provides transacted and non-transacted modes Provides transacted and non-transacted modes Beta will ship Oct 2000 as web release Beta will ship Oct 2000 as web release

27. Copyright 2000, Microsoft Corp. Bulkload - Rules Only limited buffering while loading Only limited buffering while loading Record Lifetime Rule Record Lifetime Rule  Data for a record needs to be grouped in the same element subtree Basic Record Generation Rules Basic Record Generation Rules  A new record is generated for a new relation on node or if a sql:relationship exists for this node Subset Lifetime Rule Subset Lifetime Rule  FK of 1:n relationships is buffered only as long as parent that defines FK is in scope Key Ordering Rule Key Ordering Rule  FK of 1:n needs to appear before many children

28. Copyright 2000, Microsoft Corp. Bulkload - Interface ISQLXMLBulkLoad { ConnectionString(…); ConnectionString(…); ConnectionCommand(…); ConnectionCommand(…); BulkLoad(…); BulkLoad(…); KeepNulls(…); KeepNulls(…); KeepIdentity(…); KeepIdentity(…); CheckConstraints(…); CheckConstraints(…); ForceTableLock(…); ForceTableLock(…); Transaction(…); Transaction(…); XMLFragment(…); XMLFragment(…); SchemaGen(…); SchemaGen(…); SGDropTables(…); SGDropTables(…); SGUseID(…); SGUseID(…); ErrorLogFile(…); ErrorLogFile(…); Execute(…); } Execute(…); }

29. Copyright 2000, Microsoft Corp. Summary Use XML-centric application programming model Use XML-centric application programming model Queryable and updatable XML view over database Queryable and updatable XML view over database Provides: Provides:  Simple, declarative method for defining XML views on the database  NO CODING REQUIRED  Mechanism to retrieve XML conforming to Biztalk schema definitions  XSL for arbitrary output

30. Copyright 2000, Microsoft Corp. Future Web Release Model Web Release Model Updategrams (Aug-Nov 2000) Updategrams (Aug-Nov 2000) Bulk Load (Oct-Nov 2000) Bulk Load (Oct-Nov 2000) XML View Mapper (Sep-Nov 2000) XML View Mapper (Sep-Nov 2000) W3C Schemas (XSD) W3C Schemas (XSD) W3C Query Language W3C Query Language Performance Performance