19 – 21 MARCH 2006 Riyadh, Saudi Arabia

XML and Web Services support in SQL Server 2005 Michael Storey Xpertise Training Ltd (UK)

Outline The XML Data Type Schema validated and untyped xml XML type indexes FOR XML and OpenXML enhancements Using XQuery with the XML data type Updating the XML data type with XQuery DML SQL Server 2005 Web Service Support Accessing HTTP endpoints with.NET clients

XML in a relational database XML can be stored as text loses much of value of XML representation XML can decomposed into multiple relational tables allows use of relational technologies XML can be stored as an xml data type allows use of XML technologies

Using the XML data type CREATE TABLE xml_tab ( the_id INTEGER, xml_col XML) CREATE PROCEDURE transform XML OUTPUT) AS... CREATE FUNCTION simple NVARCHAR(max)) RETURNS XML AS XML

XML column XML column can store well-formed XML XML 1.0 recommendation documents or fragments

XML schema in database XML schemas used by XML data types must be in database Create XML SCHEMA COLLECTION collection name associated with XML instance Create XML SCHEMA COLLECTION requires literal schemas CREATE XML SCHEMA COLLECTION geocoll AS '<xs:schema... targetNamespace= urn:geo>... ' literal schema targetNamespace CREATE TABLE Locations ( location xml(geocoll)... ) reference to schema collection

Using strongly typed XML CREATE TABLE point_tab( id int IDENTITY primary key, -- geocoll include schema for 'urn:geo' namespace thepoint xml(CONTENT geocoll) GO -- this works, schema-valid Point INSERT INTO point_tab VALUES( ' ') -- this insert fails, value foo is not a dim (integer) INSERT INTO point_tab VALUES( ' 10 foo ')

XML indexes You can create XML INDEXes on an XML column optimizes XML Queries on the column primary XML index must be created first (node table) three specialized index types also available VALUE – optimizes content queries PATH – optimizes structure queries PROPERTY – optimizes name/value pair predicates CREATE TABLE xml_tab ( id integer primary key, doc xml) GO CREATE PRIMARY XML INDEX xml_idx on xml_tab (doc) GO

Enhancements to SELECT...FOR XML SELECT FOR XML is an extension to Transact-SQL usually produces a stream in SQL Server 2005, it can also produce an XML data type use TYPE keyword after FOR XML can be used with RAW, AUTO formats can be used to query a "collection" of documents xml = SELECT * FROM authors FOR XML AUTO, TYPE

FOR XML PATH SELECT FOR XML PATH allows shaping of output FOR XML AUTO, RAW allow little shaping FOR XML EXPLICIT is complex to write FOR XML PATH uses path-syntax in column aliases allows "creation" of elements simpler than XML EXPLICIT can specify namespaces with XMLNAMESPACES function

XML PATH example WITH XMLNAMESPACES('urn:authors' AS au) SELECT au_id as au_fname as [name/firstname], au_lname as [name/lastname] FROM authors FOR XML PATH <row xmlns:au="urn:authors" au:authorid=" "> Bob Smith...

Producing an XML Schema with FOR XML XMLSCHEMA keyword prepends XML Schema to FOR XML can be used with other keywords schema namespace can be specified xml = (SELECT * FROM authors FOR XML AUTO, TYPE, XMLSCHEMA('urn:authors')

More FOR XML Enhancements Many refinements and extensions to FOR XML element-centric XML using FOR XML RAW generate xsi:nil for NULL database values FOR XML AUTO/RAW, ELEMENTS XSINIL nested FOR XML queries can specify ROOT element can name element in FOR XML RAW changes to nesting algorithm in XML AUTO supports new data types varchar(max), nvarchar(max), varbinary(max) UDTs (must be cast/converted to XML in query)

Using XML Data Type with OpenXml OpenXml function can use XML data type originally used varchar or TEXT input still must be parsed with sp_xml_preparedocument permits user-defined fill into existing tables overflow column may be xml -- this assumes that the document -- looks exactly like the table CREATE PROCEDURE xml) AS int exec SELECT * FROM '/ROOT',2) WITH (mytable) AS A INTO mytable exec

Inserting XML Through Bulk Load New BULK Rowset Provider can insert XML inserts from file to XML column using SINGLE_BLOB option inserts only XML column SINGLE_CLOB, SINGLE_NCLOB also work SINGLE_BLOB avoids encoding problems format file required if inserting more than one row CREATE TABLE invoices ( rowid int primary key identity, invoice xml ) INSERT invoices SELECT * FROM OPENROWSET (BULK 'c:\invoice.txt', SINGLE_BLOB) as X

XML data type and FOR XML enhancements

SQL Server 2005 Support of XQuery XQuery is supported through methods on the XML type xml.exist - returns bool xml.value - returns scalar xml.query - returns XML data type instance xml.nodes - returns one column rowset w/XML column xml.modify - modifies an instance These methods can return XML data type (query) or SQL Scalar (exist, value) columns in rowsets - when used with SQL SELECT variables

XQuery Expressions XQuery is a superset of XPath can use XPath or FLWOR expressions almost all valid XPath statements are also XQuery statements XPath used to select values in FLWOR expressions for $p in /people/person where $p/age > 30 order by $p/age[1] return $p/name

FLWOR Expressions XML Queries are FLWOR expressions made up of five types of clause FOR LET (not supported by SQL Server 2005) WHERE ORDER BY RETURN

Constructors Constructors are used to construct nodes constructors can construct static content; XML Namespaces serialized form constructors can construct dynamic content dynamic content can be content of elements and/or attributes dynamic part of constructor enclosed in curly braces or use literal constructor

Martin Simon Dynamic constructors: Text return {$p/name[1]/givenName[1]/text()}

Literal constructors return element person { attribute name {data($p/name[1]/givenName[1]/text()[1])} }

value and exist XML data type methods value() data type method accesses scalar value in xml column XQuery or simple XPath can be used exist() data type method tests conditions in xml column XQuery or simple XPath can be used used with column name separated by "." select from Invoices... Invoices table invoice column XPath expression gets value attribute

xml.exist xml.exist uses XML type and XQuery expression returns false if query returns NULL sequence returns true otherwise Usable in XML type check constraints xml functions must be encapsulated as UDF

xml.value xml.value return a SQL Server scalar type SQL type, xml data type instance, XQuery as input returns scalar type or NULL cannot return XML type instance Usable within SQL Server in predicates as result values

Using xml.value -- insert some rows INSERT xml_tab VALUES(' ') INSERT xml_tab VALUES(' ') INSERT xml_tab VALUES(' ') -- this query SELECT id, AS name FROM xml_tab -- yields this resultset id name curly 2 larry 3 moe

xml.query xml.query returns an XML data type instance XML type instance and XQuery are inputs can return scalar type as XML fragment can use constructors to compose new data Soda Ice from invoice column in Invoices table SELECT invoice.query('Invoice/LineItem') FROM Invoices Soda Ice selects LineItem's from XML data -- yields this resultset Soda Ice

Using Relational Data in XQuery SQL Server XQuery can combine relational and XML sql:variable - uses TSQL variables sql:column - uses column value same row as XML type column

Using Relational Data With XML -- returns moe in record number x -- where x is the ID column, or blank column SELECT xml_col.query(' for $b in //person where return { } in record number {sql:column("xml_tab.id")} ') FROM xml_tab -- returns moe is a stooge VARCHAR(50) = ' is a stooge' SELECT xml_col.query(' for $b in //person where return { } } ') FROM xml_tab

nodes method nodes can decompose an XML data type similar to value(), but produces references to XML nodes many nodes rather than single scalar value result is one-column rowset T-SQL CROSS APPLY can be use to produce rowset similar to OpenXML better performance, XML column is already parsed -- xmlinvoices table contains custid, xmlinv columns -- return one row for each lineitem SELECT custid, tab.col.query('itemno') --get itemno subelement FROM xmlinvoices CROSS APPLY xmlinv.nodes('/invoice/lineitem') as tab(col) WHERE custid = 'ALFKI'

XQuery DML - Using xml.modify -- use modify to insert a subelement 'insert into /Invoice[1] ') -- or insert an attribute attribute status{"backorder"} into /Invoice[1] ') -- this deletes all LineItem elements /Invoice/LineItems/LineItem') -- change the value of the CustomerName element value of /Invoice[1]/CustomerName[1]/text()[1] with "John Smith" ')

XQuery

Why SQL Server and Web Services? Databases can deal with Web service clients directly Using TDS limits potential clients Web service check box requirement OLE DB, ODBC,.NET data provider required JDBC driver required free TDS - reduced functionality Web services from SQL Server HTTP, all clients have it XML, all clients support it

SQL Server Web Services descriptions SQL Server Web services configured in database just like other database features SQL Server Web service operations part of database assemblies, sprocs in database just like other database functionality SQL Server Web service security part of data base just like other database security SQL Server Web services fully described inside database just as database objects should be

SQL Server directly leverages HTTP Uses HTTP kernel protocol stack less layers, better scalability and thru-put requires WS2K3 or XP SP2 Can use SSL Inetinfo.exe not used

Web Service configuration Web service endpoint must be configured location, access, and security Web service operations must be configured specific operation names and/or ad hoc sql batches CREATE ENDPOINT for HTTP used to configure names endpoint before FOR SOAP configures endpoint after FOR SOAP configures soap operations CREATE ENDPOINT genled... AS HTTP (...) FOR SOAP (...) endpoint name endpoint configuration operations configuration

Endpoint authentication Basic authentication supported (w/Windows login) WS-Security supported w/SQL login (WS-I standard) but must specify an endpoint authentication type too Digest and integrated authentication supported integrated requires client in same or trusted domain Multiple authentication types supported Client must also be granted CONNECT ENDPOINT privilege Clear_port and ssl_port may be specified CREATE ENDPOINT genLed... AS HTTP (Authentication = (INTEGRATED), PORTS = (CLEAR), CLEAR_PORT = ) FOR SOAP CREATE ENDPOINT genLed... AS HTTP (Authentication = (BASIC), PORTS = (SSL)... ) FOR SOAP

Endpoint startup Endpoint must be in started state to accept request initial state can be started, stopped, disabled disabled state requires SQL Server restart to enable Endpoint state changed by using ALTER HTTP ENDPOINT CREATE ENDPOINT generalLedger STATE = STARTED AS HTTP (... ) FOR SOAP (... ) endpoint will accept requests CREATE ENDPOINT generalLedger STATE = STOPPED AS HTTP (... ) FOR SOAP (... ) endpoint will not accept requests

Operation configuration Operations supported must be configured operation name, function, schema defined after FOR SOAP Context for Web methods must be configured namespace, database used, etc. CREATE ENDPOINT genLed STATE = STARTED AS HTTP (... ) FOR SOAP ( WEBMETHOD a (...), WEBMETHOD b (...),... ) operations supported by endpoint context for operations

Endpoint operation Webmethod is used to define an endpoint operation namespace and name of operation name of stored proc or udf that implements it standard schema, none or default Multiple operations may be defined FOR SOAP WEBMETHOD ' (name='Pubs.dbo.GetCInfo', schema=STANDARD ), WEBMETHOD..., client sees a CustomerInfo operation operation is implemented by the GetCInfo sproc in the pubs database schema definition is available for this operation

Endpoint context Context operations execute in must be defined automatic WSDL generation ad hoc queries, disabled by default database used, default for login if not specified namespace for endpoint FOR SOAP ( WEBMETHOD..., WSDL = DEFAULT, BATCHES = ENABLED, DATABASE = 'pubs', NAMESPACE = 'urn:Accounting' )

Typical Web Service configuration CREATE ENDPOINT Accounting STATE = STARTED AS HTTP ( SITE = ' PATH = '/nwind', AUTHENTICATION = (INTEGRATED), PORTS = (CLEAR) FOR SOAP ( WEBMETHOD ' (name='Northwind.dbo.GetCustInfo', schema=STANDARD ), WSDL = DEFAULT, BATCHES = ENABLED, DATABASE = 'northwind', NAMESPACE = ' ) GO

Creating and calling a SQL Server 2005 web service

static void Main(string[] args) { // instantiate the proxy class myprog.localhost.soap s = new myprog.localhost.soap(); // use security of current client principal s.Credentials = System.Net.CredentialCache.DefaultCredentials; object[] oa; try { oa = s.AuthorsByRoyaltyAsDataSets(50); processResults(oa); } catch (Exception e) { Console.WriteLine("{0} {1}", e.GetType().ToString(), e.Message); if (oa) Console.WriteLine("error, {0} results", oa.Length); }.NET Client to Retrieve Results

void processResults(object oa[]) { for (int i=0;i<oa.Length;i++) { switch (oa[i].GetType().ToString()) { case "System.Data.DataSet": DataSet ds = (DataSet)oa[i]; Console.WriteLine(ds.GetXml()); break; case "myprog.localhost.SqlMessage": SqlMessage sqlm = (SqlMessage)oa[i]; Console.WriteLine("Error: {0} in proc {1}", sqlm.Message, sqlm.Procedure); break; case "System.Xml.XmlElement": XmlElement elem = (XmlElement)oa[i]; Console.WriteLine(elem.OuterXml); break; // more cases... default: Console.WriteLine("Unexpected type {0}:{1}", oa[i].GetType().ToString(), oa[i].ToString()); break; } Processing Results