XML and SQL Server

1

xml and sql server

2

sql server and xml

• The way to present data in a platform independent manner is through XML. The client-cursor, SQLOLEDB provider, and SQL Server internally all provide XML support.

3

why databases and xml?

• There are many reasons for integrating XML format and relational databases– Most data is stored in a database for administrative and

concurrency reasons– XML can be used as a method of data exchange– XML has become a popular data representation method– XML can represent hierarchical composition of multiple

relational tables– XML data can be decomposed into multiple relational tables

4

sql server and xml choices

• Many facets of SQL Server and XML integration– SQL Server 2000 has built-in XML functionality– SQLOLEDB 2.6 provider enables XML query dialects and

streamed input and output– ISAPI application enables HTTP access– XML for SQL Server Web Release 1 adds updategrams and

XML bulk load– SqlClient data provider supports extended XML function– XML for SQL Web Release 2 adds client-side XML, XSD

support and .NET support of SQLOLEDB 2.6 functionality

5

xml support in sql server 2000

• SQL Server 2000 includes internal XML support– SELECT…FOR XML queries return the resultset as a

stream of XML– sp_xml_preparedocument extended stored procedure

parses a VARCHAR variable containing XML, returns a document handle

– OpenXML function uses the document handle to produce a rectangular resultset

6

select … for xml

• SQL Server 2000 has some XML query capability built in– "FOR XML" on the SELECT statement is an extension to

Transact-SQL– Three different variations of XML output (RAW, AUTO and

EXPLICIT)– Produces streamed single-column XML as output– Element-normal form and XDR schema/DTD inclusion is

supported

7

for xml variations

• Different "FOR XML" variations server different purposes– RAW mode produces one set of <row> elements, even when

using JOIN– AUTO mode produces hierarchies with separate nested

elements for each JOINed table– EXPLICIT mode produces arbitrary shapes using UNION

queries and special columns and column names

8

elements, attributes and data values

• SQL column values can be stored in elements or attributes– All data in elements/attributes known as element/attribute

normal form XML– RAW always produces attribute-normal form– AUTO can produce element-normal or attribute-normal form– EXPLICIT can produce mixtures of elements and attributes

9

-- this query:SELECT Customers.CustomerID, Orders.OrderIDFROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY Customers.CustomerIDFOR XML RAW

-- produces this XML output document fragment<row CustomerID="ALFKI" OrderID="10643" /><row CustomerID="ALFKI" OrderID="10692" /><row CustomerID="ALFKI" OrderID="10703" /><row CustomerID="ALFKI" OrderID="10835" /><row CustomerID="ANATR" OrderID="10308" />

XML RAW query and output

10

-- this query:SELECT Customers.CustomerID, Orders.OrderIDFROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY Customers.CustomerIDFOR XML AUTO

-- produces the following XML document fragment<Customers CustomerID="ALFKI"><Orders OrderID="10643" /><Orders OrderID="10692" /><Orders OrderID="10702" /><Orders OrderID="10835" />

</Customers><Customers CustomerID="ANATR"><Orders OrderID="10308" />

</Customers>

XML AUTO query and output

11

-- this query:SELECT 1 as Tag, NULL as Parent,

Customers.CustomerID as [Customer!1!CustomerID],NULL as [Order!2!OrderID]

FROM CustomersUNION ALLSELECT 2, 1,

Customers.CustomerID,Orders.OrderID

FROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY [Customer!1!CustomerID]FOR XML EXPLICIT

-- produces this output document fragment<Customer CustomerID="ALFKI">

<Order OrderID="10643"/><Order OrderID="10692"/><Order OrderID="10702"/>

</Customer>

FOR XML Explicit format

12

storing xml in sql server

• Stored procedures and UDF can assist decomposition of XML documents– System stored procedure parse an XML document into a

"DOM handle"– OpenXML function produces a rectangular resultset from

XML– Nodesets of the XML document are mapped to columns with

XPath mappings– Can use INSERT...SELECT or UPDATE/DELETE to insert

document nodesets

13

declare @idoc intdeclare @doc varchar(1000)set @doc ='<ROOT><Customers CustomerID="VINET" ContactName="Paul Henriot">

<Orders CustomerID="VINET" EmployeeID="5"><OrderDetails OrderID="10248" ProductID="11" /><OrderDetails OrderID="10248" ProductID="42" />

</Orders></Customers><Customers CustomerID="LILAS" ContactName="Carlos Gonzlez">

<Orders CustomerID="LILAS" EmployeeID="3"><OrderDetails OrderID="10283" ProductID="72" Quantity="3"/>

</Orders></Customers></ROOT>'

exec sp_xml_preparedocument @idoc OUTPUT, @docSELECT *FROM OpenXML (@idoc, '/ROOT/Customers/Orders/OrderDetails',2)WITH (CustomerID varchar(10) '../@CustomerID',

ProdID int '@ProductID',Qty int '@Quantity')

exec sp_xml_removedocument @idoc

Using OpenXML and column patterns

14

DECLARE @h intDECLARE @xmldoc varchar(1000)

set @xmldoc ='<root><stores stor_id="8888" stor_name="Bob''s Books"

stor_address="111 Somewhere" city="Portland" state="OR"zip="97225">

<discounts discounttype="A Discount" stor_id="8888" discount="50.00"/></stores> </root>'

EXEC sp_xml_preparedocument @h OUTPUT, @xmldoc

INSERT INTO storesSELECT * FROM OpenXML(@h,'/root/stores')WITH stores

INSERT INTO discountsSELECT * FROM OpenXML(@h,'/root/stores/discounts')WITH discounts

EXEC sp_xml_removedocument @h

Decomposition using OpenXML

15

sqloledb and xml

• SQLOLEDB provider in OLE DB 2.6 enables XML features– XML functionality separated into SQLXMLX.DLL– Command coclass supports streamed input and output– Extra properties on Command enable stylesheets and

database/XML mappings• Two new query dialects enable XML format queries

– MSSQLXML wraps SQL or XPath queries in XML (requires streamed input)

– XPath queries are permitted as strings

16

<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"><sql:query>SELECT Customers.CustomerID, Orders.OrderIDFROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY Customers.CustomerIDFOR XML RAW</sql:query></ROOT>

<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"><sql:xpath-query mapping-schema="rtest.xdr">/Customers[@CustomerID="ALFKI"]

</sql:xpath-query></ROOT>

MSSQLXML input documents

17

using xpath queries

• Support of XML XPath query language requires two pieces– Mapping schema maps tables, columns, etc. to a "virtual

XML document"– SQLOLEDB 2.6 mapping schemas are XDR; WebRelease 2

adds support for XSD– XPath query is applied to virtual document to produce output– Templated and string based XPath queries supported

• Internally, XPath + Mapping Schema is translated to "SELECT FOR XML EXPLICIT" SQL query by SQLOLEDB

18

<Schema xmlns="urn:schemas-microsoft-com:xml-data"xmlns:sql="urn:schemas-microsoft-com:xml-sql">

<ElementType name="discounts" sql:relation="discounts" ><AttributeType name="discounttype" /><AttributeType name="stor_id" /><AttributeType name="discount" /><attribute type="discounttype" sql:field="discounttype" /><attribute type="stor_id" sql:field="stor_id" /><attribute type="discount" sql:field="discount" /></ElementType>

<ElementType name="stores" sql:relation="stores" ><AttributeType name="stor_id" /><AttributeType name="stor_name" /><attribute type="stor_id" sql:field="stor_id" /><attribute type="stor_name" sql:field="stor_name" /><element type="discounts" ><sql:relationshipkey-relation="stores“ key="stor_id"foreign-relation="discounts“ foreign-key="stor_id" />

</element></ElementType></Schema>

An XDR mapping schema

19

Sub DoXPath(xmlDoc as MSXML.DOMDocument)Dim conn as New ADODB.ConnectionDim cmd as New ADODB.CommandDim stmOut as New ADODB.Stream

conn.Open "provider=sqloledb;uid=sa;initial catalog=pubs"Set cmd.ActiveConnection = connstmOut.Open

' xmlDoc contains XPath query from next slideSet cmd.CommandStream = xmlDoc

' Set XPath dialect, output stream and mappingscmd.Dialect = "{ec2a4293-e898-11d2-b1b7-00c04f680c56}"cmd.Properties("Output Stream") = stmOutcmd.Properties("Base path") = "c:\schemas"cmd.Properties("Mapping schema") = "mappings.xdr"

' Return XML result in output streamcmd.Execute , , adExecuteStreamEnd Sub

Using an XML mapping schema (ADO)

20

-- query input<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"><sql:xpath-query mapping-schema="rtest.xdr">/stores</sql:xpath-query></ROOT>

-- returned document<root><stores stor_id="6380" stor_name="Eric the Read Books" /><stores stor_id="7066" stor_name="Barnum's" /><stores stor_id="7067" stor_name="News & Brews" /><stores stor_id="7131” stor_name="Doc-U-Mat: Quality" /><stores stor_id="7896" stor_name="Fricative Bookshop" /><stores stor_id="8042" stor_name="Bookbeat"><discounts discounttype=“Usual" stor_id="8042"

discount="5.00" /></stores>

</root>

Using an XML mapping schema - XPath query and results

21

sqlxmloledb provider

• Web Release 2 includes new OLE DB provider– SQL Server 2000 handles "FOR XML" inside SQL Server;

SQLXMLOLEDB provider exposes analogous functionality on client

– Preprocesses "FOR XML" queries so only raw SQL is submitted to SQL Server

– Post-processes SQL resultset in XML format on client– SQLXMLOLEDB is an OLE DB service provider - currently

only supports SQLOLEDB as underlying provider– Enables SQL Server 2000-like support for SQL Server 7

22

iis support for database driven xml

• Any version of SQL Server can directly export XML through IIS– SQL Server 2000 uses native "FOR XML" clause and

SQLOLEDB provider– HTTP GET and POST supported– Query templates enable fixed and parameterized queries

XPath and SQL supported

– Transactional XML Updategrams in WR1– Client-side XML is supported in WR2 through

SQLXMLOLEDB provider– Client-side XML enables using "FOR XML" queries with SQL

Server 7 or 2000

23

Using “FOR XML” and SQL Server 2000

Client SQLISAPI.DLL SQLServerHTTP

Submit through SQLOLEDB

Add stylesheet

TDS

XML

24

Using SQLXMLOLEDB provider

Client SQLISAPI.DLL SQLServerHTTP

Format SQL from XMLSubmit by SQLXMLOLEDB

Format output XMLAdd stylesheet, DTD, schema

TDS

25

updategrams

• Updategrams use XML format to update SQL Server– Supported in Web Release 1 through ISAPI program– Insert/Update/Delete consist of XML-format before and after

images– SQLOLEDB converts these XML format to SQL– Batching of updates and transactions are supported– Web Release 2 supports .NET Diffgrams

26

<ROOT xmlns:updg="urn:schemas-microsoft-com:xml-updategram"><updg:sync >

<updg:before></updg:before><updg:after>

<Employees><FirstName>Nancy</FirstName>

<LastName>Davolio</LastName></Employees>

</updg:after></updg:sync></ROOT>

Updategram in element-centric mode

27

<ROOT xmlns:updg="urn:schemas-microsoft-com:xml-updategram"><updg:sync >

<updg:before></updg:before><updg:after>

<Employees FirstName="Nancy" ><LastName>Davoliio</LastName>

</Employees></updg:after>

</updg:sync></ROOT>

Updategram in mixed mode

28

xml bulk loader

• Web Release 1 includes XML bulk load facility– OpenXML requires too much memory for bulk load of large

XML documents– Bulk load exposed through separate COM object– Bulk load analogous to import through DTS or BCP

29

set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkLoad")objBL.ConnectionString =

"provider=SQLOLEDB;uid=sa;pwd=;database=pubs"

objBL.ErrorLogFile = "c:\error.log"objBL.Execute "c:\SampleSchema.xml", "c:\SampleData.xml"set objBL=Nothing

XML Bulk Load through the bulk load class

30

sqlclient, datasets and diffgrams

• Three integration points in native .NET libraries– SQLISAPI program can be used directly as a web endpoint– SqlClient data provider exposes "FOR XML" results using

SqlCommand.ExecuteXmlReader– Resulting XmlReader can be used for DataSet input

producing multiple DataTables– DataSets can export two variations of Diffgram format

Entire DataSet and changes

Only changed rows

31

SqlCommand cmd = new SqlCommand("select * from authors for xml auto",new SqlConnection("server=localhost;uid=sa;database=pubs"));

cmd.Connection.Open();

XmlTextReader rdr;rdr = (XmlTextReader)cmd.ExecuteXmlReader();

DataSet ds = new DataSet();ds.ReadXml(rdr, XmlReadMode.Fragment);

Using SQLXML through a ExecuteXmlReader

32

DataSet ds = new DataSet();XmlTextReader rdr = new XmlTextReader(

"http://localhost/northwind/template/modeauto1.xml");

ds.ReadXml(rdr);

Using SQL Server 2000 ISAPI application

33

SqlDataAdapter da = new SqlDataAdapter("select * from sample","server=localhost;uid=sa;database=pubs");

DataSet dsAll = new DataSet("all");DataSet dsChanges = new DataSet("changes");

da.Fill(dsAll, "sample");DataTable t = dsAll.Tables["sample"];

DataRow r = t.NewRow();r[0] = 4;r[1] = "Shemp";t.Rows.Add(r);

t.Rows[1]["name"] = "CurleyJoe";t.Rows[2].Delete();

dsAll.WriteXml("c:\\all.xml", XmlWriteMode.DiffGram);dsChanges = dsAll.GetChanges();dsChanges.WriteXml("c:\\changes.xml", XmlWriteMode.DiffGram);

Producing DiffGrams

34

Microsoft.Data.SqlXml

• Web Release 2 includes SQLOLEDB-XML functionality– SqlXml data provider exposes Command, Parameter and

DataAdapter– All SQLOLEDB 2.6 and above functionality supported

MSSQLXML and XPath queries

XSL transforms

– DataAdapter enables integration with DataSet and DiffGrams

35

Using SqlXmlCommand and XSD mapping schema

void doXPath(){Stream strm;SqlXmlCommand cmd = new SqlXmlCommand("provider=sqloledb;uid=sa;database=northwind");cmd.CommandText = "x:Employee[@EID='1']";cmd.CommandType = SqlXmlCommandType.XPath;cmd.RootTag = "ROOT";cmd.Namespaces = "xmlns:x='urn:myschema:Employees'";cmd.SchemaPath = "c:\\xml_mappings\\MySchemaNS.xml";strm = cmd.ExecuteStream();StreamReader sw = new StreamReader(strm);Console.WriteLine(sw.ReadToEnd());}

36

Using SqlXmlCommand and XSD mapping schema

<schema xmlns="http://www.w3.org/2001/XMLSchema"

xmlns:msdata="urn:schemas-microsoft-com:mapping-schema"xmlns:emp="urn:myschema:Employees"targetNamespace="urn:myschema:Employees">

<complexType name="EmployeeType"><attribute name="EID" msdata:field="EmployeeID" type="ID"/><attribute name="FName" msdata:field="FirstName" type="string"/><attribute name="LName" msdata:field="LastName"/>

</complexType><element name="Employee" type="emp:EmployeeType"

msdata:relation="Employees"/></schema>

<?xml version="1.0" encoding="utf-8" ?><ROOT><y0:Employee xmlns:y0="urn:myschema:Employees"

LName="Davolio" EID="1" FName="Nancy"/></ROOT>

37

Using XSLT and SqlXmlCommand output

Stream strm;SqlXmlCommand cmd = new SqlXmlCommand(NorthwindConnString);cmd.CommandText ="select FirstName, LastName from Employees For XML Auto";

cmd.RootTag = "root";// get output Streamstrm = cmd.ExecuteStream();

// line up transform and documentXmlTextReader reader = new XmlTextReader(strm);XPathDocument xd = new XPathDocument(reader, XmlSpace.Preserve);XslTransform xslt = new XslTransform();xslt.Load("c:\\xml_mappings\\MyXSL.xsl", null);XmlTextWriter writer = new XmlTextWriter(

"c:\\xml_mappings\\xslt_output.html",System.Text.Encoding.UTF8);

// transform stream outputxslt.Transform(xd, null, writer);

38

summary

• SQL Server 2000 includes native XML support• SQLOLEDB provider contains additional support• ISAPI DLL exposes SQL/XML as HTTP endpoints• Updategrams and Bulk Load added in Web Release 1• Native .NET support in Web Release 2

Documents

XML and SQL Server