1. Database Management Systems, R. Ramakrishnan1 Introduction to Semistructured Data and XML Chapter 27

2. Database Management Systems, R. Ramakrishnan2 How the Web is Today  HTML documents often generated by applications consumed by humans only easy access: across platforms, across organizations  No application interoperability: HTML not understood by applications Database technology: client-server

3. Database Management Systems, R. Ramakrishnan3 New Universal Data Exchange Format: XML A recommendation from the W3C  XML = data  XML generated by applications  XML consumed by applications  Easy access: across platforms, organizations

4. Database Management Systems, R. Ramakrishnan4 Paradigm Shift on the Web  From documents (HTML) to data (XML)  From information retrieval to data management  For databases, also a paradigm shift: from relational model to semistructured data from data processing to data/query translation from storage to transport

5. Database Management Systems, R. Ramakrishnan5 HTML  HTML is widely used for formatting and structuring Web documents.  Designed to describe how a Web browser should arrange text, images and push-buttons on a page.  Easy to learn, but does not convey structure and meaning of data in the Web pages.  Fixed tag set. Welcome to the XML course Introduction Opening tag Text (PCDATA) Closing tag “Bachelor” tag Attribute nameAttribute value

6. Database Management Systems, R. Ramakrishnan6 Semistructure data 1. Information integration: important new application that motivates what follows. 2. Semistructured data: a new data model designed to cope with problems of information integration. 3. XML ( Extensible Markup Language ) : a new Web standard that is essentially semistructured data. 4. XQUERY: an emerging standard query language for XML data.

7. Database Management Systems, R. Ramakrishnan7 Information Integration Problem: related data exists in many places. They talk about the same things, but differ in model, schema, conventions ( e.g., terminology). Example: In the real world, every bar has its own database.  Some may have relations like beer-price; others have an Microsoft Word file from which the menu is printed.  Some keep phones of manufacturers but not addresses.  Some distinguish beers and ales; others do not.

8. Database Management Systems, R. Ramakrishnan8 The Semistructured Data Model &o1 &o12&o24&o29 &o43 &96 &243 &206 &25 “Serge” “Abiteboul” 1997 “Victor” “Vianu” 122133 paper book paper references author title year http author title publisher author title page firstname lastname firstnamelastnamefirst last Bib Object Exchange Model (OEM) complex object atomic object

9. Database Management Systems, R. Ramakrishnan9 Characteristics of Semistructured Data  Missing or additional attributes  Multiple attributes  Different types in different objects  Heterogeneous collections Self-describing, irregular data, no a priori structure

10. Database Management Systems, R. Ramakrishnan10 Comparison with Relational Data { row: { name: “John”, phone: 3634 }, row: { name: “Sue”, phone: 6343 }, row: { name: “Dick”, phone: 6363 } } row name phone “John”3634“Sue”“Dick”63436363

11. Database Management Systems, R. Ramakrishnan11 XML (Extensible Markup Language)  A W3C standard to complement HTML  Origins: Structured text SGML Large-scale electronic publishing Data exchange on the web  Motivation: HTML describes presentation XML describes content

12. Database Management Systems, R. Ramakrishnan12 From HTML to XML HTML describes the presentation

13. Database Management Systems, R. Ramakrishnan13 HTML Bibliography Foundations of Databases Abiteboul, Hull, Vianu Addison Wesley, 1995 Data on the Web Abiteboul, Buneman, Suciu Morgan Kaufmann, 1999

14. Database Management Systems, R. Ramakrishnan14 XML Foundations… Abiteboul Hull Vianu Addison Wesley 1995 … XML describes the content

15. Database Management Systems, R. Ramakrishnan15 Why are we DB’ers interested?  It’s data. That’s us.  Database issues: How are we going to model XML? (graphs). How are we going to query XML? (XQuery) How are we going to store XML (in a relational database? object-oriented? native?) How are we going to process XML efficiently? (many interesting research questions!)

16. Database Management Systems, R. Ramakrishnan16 XML Terminology  Tags: book, title, author, … start tag:, end tag:  Elements: …, … elements can be nested empty element: (Can be abbrv. )  XML document: Has a single root element  Well-formed XML document: Has matching tags  Valid XML document: conforms to a schema

17. Database Management Systems, R. Ramakrishnan17 Well-Formed XML 1. Declaration =. Normal declaration is “Standalone” means that there is no DTD specified. 2. Root tag surrounds the entire balance of the document.  is balanced by, as in HTML. 3. Any balanced structure of tags OK. Option of tags that don’t require balance, like in HTML.

18. Database Management Systems, R. Ramakrishnan18 XML: An Example Richard Feynman The Character of Physical Law 1980 R.K. Narayan Waiting for the Mahatma 1981 R.K. Narayan The English Teacher 1980

19. Database Management Systems, R. Ramakrishnan19 XML – Elements …  Xml is case and space sensitive  Element opening and closing tag names must be identical  Opening tags: “ ”  Closing tags: “ ”  Empty Elements have no data and no closing tag: They begin with a “ ” closing tag attribute attribute valuedata open tag element name

20. Database Management Systems, R. Ramakrishnan20 XML – Attributes …  Attributes provide additional information for element tags.  There can be zero or more attributes in every element; each one has the the form: attribute_name =‘ attribute_value ’ - There is no space between the name and the “=‘” - Attribute values must be surrounded by “ or ‘ characters  Multiple attributes are separated by white space (one or more spaces or tabs). closing tag attribute attribute valuedata open tag element name

21. Database Management Systems, R. Ramakrishnan21 Elements The segment of an XML document between an opening and a corresponding closing tag is called an element. Malcolm Atchison (215) 898 4321 mp@dcs.gla.ac.sc element not an element element, a sub-element of

22. Database Management Systems, R. Ramakrishnan22 XML – Data and Comments …  Xml data is any information between an opening and closing tag  Xml data must not contain the ‘ ’ characters  Comments: closing tag attribute attribute valuedata open tag element name

23. Database Management Systems, R. Ramakrishnan23 XML text XML has only one “basic” type -- text. It is bounded by tags, e.g. The Big Sleep 1935 --- 1935 is still text XML text is called PCDATA (for parsed character data). It uses a 16-bit encoding.

24. Database Management Systems, R. Ramakrishnan24 XML – Nesting & Hierarchy  Xml tags can be nested in a tree hierarchy  Xml documents can have only one root tag  Between an opening and closing tag you can insert: 1. Data 2. More Elements 3. A combination of data and elements Some Text More

25. Database Management Systems, R. Ramakrishnan25 Representing relational DBs: Two ways projects: title budget managedBy employees: name ssn age

26. Database Management Systems, R. Ramakrishnan26 Project and Employee relations in XML Pattern recognition 10000 Joe Joe 344556 34 Sandra 2234 35 Auto guided vehicle 70000 Sandra : Projects and employees are intermixed

27. Database Management Systems, R. Ramakrishnan27 Pattern recognition 10000 Joe Auto guided vehicles 70000 Sandra : Project and Employee relations in XML (cont’d) Joe 344556 34 Sandra 2234 35 : Employees follows projects

28. Database Management Systems, R. Ramakrishnan28 More XML: Oids and References Jane Mary John oids and references in XML are just syntax

29. Database Management Systems, R. Ramakrishnan29 XML Data Model (Graph)

30. Database Management Systems, R. Ramakrishnan30 Document Type Descriptors  Sort of like a schema but not really.  Inherited from SGML DTD standard  BNF grammar establishing constraints on element structure and content  Definitions of entities

31. Database Management Systems, R. Ramakrishnan31 DTD – An Example --------------------------------------------------------------------------------

32. Database Management Systems, R. Ramakrishnan32 DTD - !ELEMENT  !ELEMENT declares an element name, and what children elements it should have  Content types: Other elements #PCDATA (parsed character data) EMPTY (no content) ANY (no checking inside this structure) A regular expression NameChildren

33. Database Management Systems, R. Ramakrishnan33 DTD - !ELEMENT (Contd.)  A regular expression has the following structure: exp 1, exp 2, exp 3, …, exp k : A list of regular expressions exp*: An optional expression with zero or more occurrences exp+: An optional expression with one or more occurrences exp 1 | exp 2 | … | exp k : A disjunction of expressions

34. Database Management Systems, R. Ramakrishnan34 DTD - !ATTLIST <!ATTLIST Orange location CDATA #REQUIRED color ‘orange’>  !ATTLIST defines a list of attributes for an element  Attributes can be of different types, can be required or not required, and they can have default values. ElementAttributeTypeFlag

35. Database Management Systems, R. Ramakrishnan35 DTD – Well-Formed and Valid -------------------------------------------------------------------------------- Well-Formed and Valid Not Well-Formed Well-Formed but Invalid Home

36. Database Management Systems, R. Ramakrishnan36 Example: An Address Book MacNiel, John Dr. John MacNiel 1234 Huron Street Rome, OH 98765 (321) 786 2543 jm@abc.com Exactly one name At most one greeting As many address lines as needed (in order) Mixed telephones and faxes As many as needed

37. Database Management Systems, R. Ramakrishnan37 Specifying the structure  name to specify a name element  greet? to specify an optional (0 or 1) greet elements  name,greet? to specify a name followed by an optional greet

38. Database Management Systems, R. Ramakrishnan38 Specifying the structure (cont)  addr* to specify 0 or more address lines  tel | fax a tel or a fax element  (tel | fax)* 0 or more repeats of tel or fax  email* 0 or more email elements

39. Database Management Systems, R. Ramakrishnan39 A DTD for the address book <!DOCTYPE addressbook [ <!ELEMENT person (name, greet?, address*, (fax | tel)*, email*)> ]>

40. Database Management Systems, R. Ramakrishnan40 DTD for the example relational DB <!DOCTYPE db [... ]>

41. Database Management Systems, R. Ramakrishnan41 Summary of XML regular expressions  Each element name is a tag.  Its components are the tags that appear nested within, in the order specified.  AThe tag A occurs  e1,e2The expression e1 followed by e2  e*0 or more occurrences of e  e?Optional -- 0 or 1 occurrences  e+1 or more occurrences  e1 | e2either e1 or e2  (e)grouping

42. Database Management Systems, R. Ramakrishnan42 XML Querying Path Expressions :  Bib.paper  Bib.book.publisher  Bib.paper.author.lastname Given an OEM instance, the value of a path expression p is a set of objects

43. Database Management Systems, R. Ramakrishnan43 Path Expressions Examples: DB = &o1 &o12&o24&o29 &o43 &o70&o71 &96 &243 &206 &25 “Serge” “Abiteboul” 1997 “Victor” “Vianu” 122133 paper book paper references author title year http author title publisher author title page firstname lastname firstnamelastnamefirst last Bib &o44&o45&o46 &o47&o48 &o49 &o50 &o51 &o52 Bib.paper={&o12,&o29} Bib.book.publisher={&o51} Bib.paper.author.lastname={&o71,&206} Bib.paper={&o12,&o29} Bib.book.publisher={&o51} Bib.paper.author.lastname={&o71,&206}

44. Database Management Systems, R. Ramakrishnan44 XQuery Emerging standard for querying XML documents. Basic form: FOR WHERE RETURN ;  Sets of elements described by paths, consisting of: 1.URL, if necessary. 2.Element names forming a path in the semistructured data graph, e.g., //BAR/NAME = “start at any BAR node and go to a NAME child.” 3.Ending condition of the form [ ]

45. Database Management Systems, R. Ramakrishnan45 XQuery Overview:  FOR-LET-WHERE-ORDERBY-RETURN = FLWOR FOR/LET Clauses WHERE Clause ORDERBY/RETURN Clause List of tuples Instance of Xquery data model

46. Database Management Systems, R. Ramakrishnan46 XQuery  FOR $x in expr -- binds $x to each value in the list expr  LET $x = expr -- binds $x to the entire list expr Useful for common subexpressions and for aggregations

47. Database Management Systems, R. Ramakrishnan47 FOR v.s. LET FOR $x IN document("bib.xml") /bib/book RETURN $x FOR $x IN document("bib.xml") /bib/book RETURN $x Returns:... LET $x IN document("bib.xml") /bib/book RETURN $x LET $x IN document("bib.xml") /bib/book RETURN $x Returns:...

48. Database Management Systems, R. Ramakrishnan48 XQuery Find all book titles published after 1995: FOR $x IN document("bib.xml") /bib/book WHERE $x/year > 1995 RETURN $x/title FOR $x IN document("bib.xml") /bib/book WHERE $x/year > 1995 RETURN $x/title Result: abc def ghi

49. Database Management Systems, R. Ramakrishnan49 XQuery For each author of a book by Morgan Kaufmann, list all books s/he published: FOR $a IN distinct( document("bib.xml") /bib/book[publisher=“Morgan Kaufmann”]/author) RETURN $a, FOR $t IN /bib/book[author=$a]/title RETURN $t FOR $a IN distinct( document("bib.xml") /bib/book[publisher=“Morgan Kaufmann”]/author) RETURN $a, FOR $t IN /bib/book[author=$a]/title RETURN $t distinct = a function that eliminates duplicates

50. Database Management Systems, R. Ramakrishnan50 XQuery Result: Jones abc def Smith ghi

51. Database Management Systems, R. Ramakrishnan51 XQuery count = a (aggregate) function that returns the number of elms FOR $p IN distinct(document("bib.xml")//publisher) LET $b := document("bib.xml")/book[publisher = $p] WHERE count($b) > 100 RETURN $p FOR $p IN distinct(document("bib.xml")//publisher) LET $b := document("bib.xml")/book[publisher = $p] WHERE count($b) > 100 RETURN $p

52. Database Management Systems, R. Ramakrishnan52 XQuery Find books whose price is larger than average: LET $a=avg( document("bib.xml") /bib/book/price) FOR $b in document("bib.xml") /bib/book WHERE $b/price > $a RETURN $b LET $a=avg( document("bib.xml") /bib/book/price) FOR $b in document("bib.xml") /bib/book WHERE $b/price > $a RETURN $b

53. Database Management Systems, R. Ramakrishnan53 Examples for XQuery queries  FOR $x IN doc(www.company.com/info.xml) //employee [employeeSalary gt 70000]/employeeName RETURN $x/firstName, $x/lastName  FOR $x IN doc(www.company.com/info.xml)/company/employee WHERE $x/employeeSalary gt 70000 RETURN $x/employeeName/firstName, $x/employeeName/lastName  FOR $x IN doc(www.company.com/info.xml)/company /project [projectNumber = 5]/projectWorker, $y IN doc(www.company.com/info.xml)/company/employee WHERE $x/hours gt 20.0 AND $y.ssn = $x.ssn RETURN $x/EmployeeName/firstName, $y/employeeName/lastName, $x/hours