© Copyright 2008 STI INNSBRUCK Semantic Web Repositories and SPARQL Dieter Fensel Federico Facca

Where are we? 2 #DateTitle 1Introduction 2Semantic Web architecture 3RDF and RDFs 4Web of hypertext (RDFa, Microformats) and Web of data 5Semantic annotations 6Repositories and SPARQL 7OWL 8RIF 9Web-scale reasoning 10Social Semantic Web 11Ontologies and the Semantic Web 12SWS 13Tools 14Applications 15Exam

Agenda 1.Introduction and motivation 2.Technical Solution 1.RDF Repositories 2.SPARQL 3.Illustration by a large example 4.Extensions 5.Summary 6.References 3

Adapted from Semantic Web Stack 4

MOTIVATION 5

Motivation Having RDF data available is not enough –Need tools to process, transform, and reason with the information –Need a way to store the RDF data and interact with it Are existing storage systems appropriate to store RDF data? Are existing query languages appropriate to query RDF data? 6

Databases and RDF Relational database are a well established technology to store information and provide query support (SQL) How can we store the following RDF data in a relational database? Dieter Fensel University Professor 7

Databases and RDF Relational database are a well established technology to store information and provide query support (SQL) How can we store the following RDF data in a relational database? Dieter Fensel University Professor Relational “Traditional” approach 8 Lecturer idnametitle Dieter FenselUniversity Professor

Databases and RDF Relational database are a well established technology to store information and provide query support (SQL) How can we store the following RDF data in a relational database? Dieter Fensel University Professor Relational “Triple” based approach 9 Resources IdURI rdf:type 103uni:lecturer 104… Statement SubjectPredicateObjectURIObjectLiteral null ……null Literals IdValue 201Dieter Fensel 202University Professor 203… ……

Why Native RDF Repositories? What happens if I want to find the names of all the lecturers? Query is much more complex Relation “traditional” approach: SELECT NAME FROM LECTURER Relational “triple” based approach: SELECT L.Value FROM Literals AS L INNER JOIN Statement AS S ON S.ObjectLiteral=L.ID INNER JOIN Resources AS R ON R.ID=S.Predicate INNER JOIN Statement AS S1 ON S1.Predicate=S.Predicate INNER JOIN Resources AS R1 ON R1.ID=S1.Predicate INNER JOIN Resources AS R2 ON R2.ID=S1.ObjectURI WHERE R.URI = “uni:name” AND R1.URI = “rdf:type” AND R2.URI = “uni:lecturer” 10

Query Languages Querying and inferencing is the very purpose of information representation in a machine-accesible way A query language is a language that allows a user to retrieve information from a “data source” –E.g. data sources A simple text file XML file A database The “Web” Query languages usually includes insert and update operations 11

Example of Query Languages SQL –Query language for relational databases XQuery, XPointer and XPath –Query languages for XML data sources SPARQL –Query language for RDF graphs RDQL –Query language for RDF in Jena models 12

XPath: a simple query language for XML trees The basis for most XML query languages –Selection of document parts –Search context: ordered set of nodes Used extensively in XSLT –XPath itself has non-XML syntax Navigate through the XML Tree –Similar to a file system (“/“, “../“, “./“, etc.) –Query result is the final search context, usually a set of nodes –Filters can modify the search context –Selection of nodes by element names, attribute names, type, content, value, relations Several pre-defined functions Version 1.0, W3C Recommendation 16 November 1999 Version 2.0, W3C Recommendation 23 January

Other XML Query Languages XQuery –Building up on the same functions and data types as XPath –With XPath 2.0 these two languages get closer –XQuery is not XML based, but there is an XML notation (XQueryX) –XQuery 1.0, W3C Recommendation 23 January 2007 XLink 1.0, W3C Recommendation 27 June 2001 –Defines a standard way of creating hyperlinks in XML documents XPointer 1.0, W3C Candidate Recommendation –Allows the hyperlinks to point to more specific parts (fragments) in the XML document XSLT 2.0, W3C Recommendation 23 January

Why a New Language? RDF description (1): Dieter Fensel University Professor XPath query: /rdf:Description[rdf:type= " 15

Why a New Language? RDF description (2): Dieter Fensel University Professor XPath query: //uni:lecturer/uni:name 16

Why a New Language? RDF description (3): <uni:lecturer rdf:about="949318" uni:name=“Dieter Fensel" uni:title=“University Professor"/> XPath query: 17

Why a New Language? What is the difference between these three definitions? RDF description (1): Dieter Fensel University Professor RDF description (2): Dieter Fensel University Professor RDF description (3): <uni:lecturer rdf:about="949318" uni:name=“Dieter Fensel" uni:title=“University Professor"/> 18

Why a New Language? All three description denote the same thing:  #949318, rdf:type,   #949318,, “Dieter Fensel”   #949318,, “University Professor”  But the queries are different depending on a particular serialization: /rdf:Description[rdf:type= " name //uni:lecturer/uni: name name 19

TECHNICAL SOLUTION 20

RDF REPOSITORIES Efficient storage of RDF data 21

Different Architectures Based on their implementation, can be divided into 3 broad categories : In-memory, Native, Non-native Non- memory. In – Memory : RDF Graph is stored as triples in main – memory –E.g. Storing an RDF graph using Jena API/ Sesame API. Native : Persistent storage systems with their own implementation of databases. –E.g. Sesame Native, Virtuoso, AllegroGraph, Oracle 11g. Non-Native Non-Memory : Persistent storage systems set-up to run on third party DBs. –E.g. Jena SDB. 22

Implications Scalability Different query languages supported to varying degrees. –Sesame – SeRQL, Oracle 11g – Own query language. Different level of inferencing. –Sesame supports RDFS inference, AllegroGraph – RDFS++, –Oracle 11g – RDFS++, OWL Prime Lack of interoperability and portability. –More pronounced in Native stores. 23

What is Jena? Jena is a complete Java Framework for RDF, DAML and OWL Developed at HP Labs Jena supports –Triple storage –Reasoning Current Jena release: Available at 24

Jena’s Architecture 25 The RDF API ARP RDF Filter n- triples Readers XML n- triples Writers Mem RDB BDB Stores The DAML API RDQL Applications Jena 1 Architecture The RDF 2003 API Ontology API OWL Full OWL DL OWL Lite DAML +OIL RDFS Rule Systems RDQL Applications OWL RDFSOWL Lite External reasoner s Fast path query Reification Event handling The RDF 2003 API Jena SPI [

Jena SDB SDB basically is a Java Loader. Multiple stores supported: MySQL, PostgreSQL, Oracle, DB2. Takes incoming triples and breaks them down into components ready for the database. Multiple layouts Integration with the Joseki server. SPARQL supported. 26

What is Sesame? Sesame is an open source Java framework for storing, querying and reasoning with RDF and RDF Schema It can be used as: –Standalone Server: A database for RDF and RDF Schema –Java Library: For applications that need to work with RDF internally Sesame is similar to Jena –Supports triple storage –Supports reasoning –Supports Web services Available at 27

Sesame’s Architecture Repository Repository Abstraction Layer (RAL) Admin ModuleExport ModuleQuery Module HTTP Protocol HandlerSOAP Protocol Handler Sesame SOAP HTTP Clients 28

Sesame’s Repository DBMSs –Currently, Sesame is able to use PostgreSQL MySQL Oracle (9i or newer) Existing RDF stores (e.g. Jena) RDF files RDF network services –Using multiple sesame server to retrieve results for queries –This opens up the possibility of a highly distributed architecture for RDF storing and querying 29

Sesame’s Repository Abstraction Layer (RAL) RAL offers stable, high-level interface for talking to repositories It is defined by an API that offers these functionalities: –Add data –Retrieve data –Delete data Data is returned in streams (Scalability) –Only small amount of data is kept in memory –Suitable for use in highly constrained environments such as portable devices Caching data (Performance) –E.g. caching RDF schema data which is needed very frequently 30

Sesame’s Admin Module Allows inserting or deleting RDF data in repository Retrieves its information from an RDF(S) source, and parses it using an RDF parser Checks each (S, P, O) statement for consistency and infers implied information if necessary for instance: –If P equals type, it infers that O must be a class. –If P equals subClassOf, it infers that S and O must be classes. –If P equals subPropertyOf, then it infers that both S and O must be properties. –If P equals domain or range, then it infers that S must be a property and O must be a class 31

Sesame’s Query Module Evaluates RQL queries posed by the user It is independent of the underlying repository –Can not use optimizations and query evaluations offered by specific DBMSs RQL queries are translated into a set of calls to the RAL –E.g. when a query contains a join operation over two subqueries, each of the subqueries is evaluated, and the join operation is then executed by the query engine on the results 32

Sesame’s RDF Export Module This module allows for the extraction of the complete schema and/or data from a model in RDF format It supplies the basis for using Sesame with other RDF tools 33

Which RDF Store to Choose for an App? Frequency of loads that the application would perform. Single scaling factor and linear load times. Level of inferencing. Support for which query language. W3C recommendations. Special system needs. –E.g. Allegrograph needs 64 bit processor. 34

SPARQL A language to query RDF data 35

Querying RDF SPARQL –RDF Query language –Based on RDQL –Uses SQL-like syntax Example: PREFIX uni: SELECT ?name FROM WHERE { ?s uni:name ?name. ?s rdf:type uni:lecturer } 36

SPARQL Queries PREFIX uni: SELECT ?name FROM WHERE { ?s uni:name ?name. ?s rdf:type uni:lecturer } PREFIX –Prefix mechanism for abbreviating URIs SELECT –Identifies the variables to be returned in the query answer –SELECT DISTINCT –SELECT REDUCED FROM –Name of the graph to be queried –FROM NAMED WHERE –Query pattern as a list of triple patterns LIMIT OFFSET ORDER BY 37

Example RDF Graph "John Smith“ :_X1 _:X1 "John" _:X1 "Smith“ "32“ "Mary Smith“ :_X2 _:X2 "Mary" _:X2 "Smith" "29" 38

SPARQL Queries: All Full Names “Return the full names of all people in the graph” PREFIX vCard: SELECT ?fullName WHERE {?x vCard:FN ?fullName} result: fullName ================= "John Smith" "Mary Smith" vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Properties “Return the relation between John and Mary” PREFIX ex: SELECT ?p WHERE {ex:john ?p ex:mary} result: p ================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Complex Patterns “Return the spouse of a person by the name of John Smith” PREFIX vCard: PREFIX ex: SELECT ?y WHERE {?x vCard:FN "John Smith". ?x ex:marriedTo ?y} result: y ================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Complex Patterns “Return the spouse of a person by the name of John Smith” PREFIX vCard: PREFIX ex: SELECT ?y WHERE {?x vCard:FN "John Smith". ?x ex:marriedTo ?y} result: y ================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Blank Nodes “Return the first name of all people in the KB” PREFIX vCard: SELECT ?name, ?firstName WHERE {?x vCard:N ?name. ?name vCard:Given ?firstName} result: name firstName ================= _:a "John" _:b "Mary " vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Blank Nodes “Return the first name of all people in the KB” PREFIX vCard: SELECT ?name, ?firstName WHERE {?x vCard:N ?name. ?name vCard:Given ?firstName} result: name firstName ================= _:a "John" _:b "Mary " vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Building RDF Graph “Rewrite the naming information in original graph by using the foaf:name ” PREFIX vCard: PREFIX foaf: CONSTRUCT { ?x foaf:name ?name } WHERE { ?x vCard:FN ?name } result: #john foaf:name “John Smith" #marry foaf:name “Marry Smith" vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Building RDF Graph “Rewrite the naming information in original graph by using the foaf:name ” PREFIX vCard: PREFIX foaf: CONSTRUCT { ?x foaf:name ?name } WHERE { ?x vCard:FN ?name } result: #john foaf:name “John Smith" #marry foaf:name “Marry Smith" vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29. <rdf:RDF xmlns:rdf=" xmlns:foaf=" xmlns:ex=" John Smith Marry Smith

SPARQL Queries: Testing if the Solution Exists “Are there any married persons in the KB?” PREFIX ex: ASK { ?person ex:marriedTo ?spouse } result: yes ================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Constraints (Filters) “Return all people over 30 in the KB” PREFIX ex: SELECT ?x WHERE {?x hasAge ?age. FILTER(?age > 30)} result: x ================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Optional Patterns “Return all people and (optionally) their spouse” PREFIX ex: SELECT ?person, ?spouse WHERE {?person ex:hasAge ?age. OPTIONAL { ?person ex:marriedTo ?spouse } } result: ?person ?spouse ============================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Optional Patterns “Return all people and their spouse if the spouse has a name” PREFIX ex: SELECT ?person, ?spouse WHERE {?person ex:hasAge ?age. OPTIONAL { ?person ex:marriedTo ?spouse. ?spouse vCard:FN ?name} FILTER (bound(?name))} result: ?person ?spouse ============================= rg/#mary> vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Optional Patterns “Return all people and their spouse if the spouse has a name” PREFIX ex: SELECT ?person, ?spouse WHERE {?person ex:hasAge ?age. OPTIONAL { ?person ex:marriedTo ?spouse. ?spouse vCard:FN ?name} FILTER (bound(?name))} result: ?person ?spouse ============================= rg/#mary> vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Matching Alternatives “Return all people and their spouses” PREFIX ex: SELECT ?person, ?spouse WHERE { {?person ex:marriedTo ?spouse} UNION {?spouse ex:marriedTo ?person } } result: ?person ?spouse ============================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Queries: Matching Alternatives “Return all people and their spouses” PREFIX ex: SELECT ?person, ?spouse WHERE { {?person ex:marriedTo ?spouse} UNION {?spouse ex:marriedTo ?person } } result: ?person ?spouse ============================= vcard:. ex:john vcard:FN "John Smith" ; vcard:N [ vcard:Given "John" ; vcard:Family "Smith" ] ; ex:hasAge 32 ; ex:marriedTo :mary. ex:mary vcard:FN "Mary Smith" ; vcard:N [ vcard:Given "Mary" ; vcard:Family "Smith" ] ; ex:hasAge 29.

SPARQL Graph Patterns: WHERE Simple graph patterns –RDF triples where subject, predicate, object may be a variable –All triples must follow from graph for query answer ?x vCard:FN "John Smith". ?x ex:marriedTo ?y Optional graph patterns –Not required OPTIONAL {?person ex:marriedTo ?spouse} Filters in graph patterns –Conditions on query answer –E.g. string, numeric comparison FILTER(?age > 30) 54

SPARQL Query Answers Variable substitution –Assigning values to variables [?x=,?name= _:a, ?firstName=“John”] –Not all variables need to be bound [?person=,?spouse=] Query answer –Substitute variables in graph pattern vCard:N _:a. _:a vCard:Given "John“ –If resulting graph pattern is subset of original graph, the variable substitution is a query answer 55

SPARQL Unary Operators 56 [

SPARQL Binary Operators 57

More SPARQL Binary Operators 58

ILLUSTRATION BY A LARGER EXAMPLE An example of usage of SPARQL 59

A RDF Graph Modeling Movies movie1 movie:Movie “Edward ScissorHands” “1990” rdf:type movie:title movie:year movie:Genre movie:Romanc e movie:Comed y rdf:type movie:genre movie:Role “Edward ScissorHands” r1r1 actor1 movie:playedBy movie:characterName rdf:type movie:hasPart [ Sesame-tutorial-eswc2006.ppt] 60

Example Query 1 Select the movies that has a character called “Edward Scissorhands” PREFIX movie: SELECT DISTINCT ?x ?t WHERE { ?x movie:title ?t ; movie:hasPart ?y. ?y movie:characterName ?z. FILTER (?z = “Edward } 61

Example Query 2 Create a graph of actors and relate them to the movies they play in (through a new ‘playsInMovie’ relation) PREFIX movie: PREFIX foaf: CONSTRUCT { ?x foaf:firstName ?fname. ?x foaf:lastName ?lname. ?x movie:playInMovie ?m } WHERE { ?m movie:title ?t ; movie:hasPart ?y. ?y movie:playedBy ?x. ?x foaf:firstName ?fname. ?x foaf:lastName ?lname. } 62

Example Query 3 Find all movies which share at least one genre with “Gone with the Wind” PREFIX movie: SELECT DISTINCT ?x2 ?t2 WHERE { ?x1 movie:title ?t1. ?x1 movie:genre ?g1. ?x2 movie:genre ?g2. ?x2 movie:title ?t2. FILTER (?t1 = “Gone with the && ?x1!=?x2 && ?g1=?g2) } 63

EXTENSIONS 64

Massive Data Inference in RDF Repositories Present common implementations: –Make a number of small queries to propagate the effects of rule firing. –Each of these queries creates an interaction with the database. –Not very efficient Approaches –Snapshot the contents of the database-backed model into RAM for the duration of processing by the inference engine. –Performing inferencing in-stream. Precompute the inference closure of ontology and analyze the in-coming data-streams, add triples to it based on your inference closure. Assumes rigid separation of the RDF Data(A-box) and the Ontology data(T- box) Even this may not work for very large ontologies – BioMedical Ontologies 65

Extending SPARQL SPARQL is still under continuous development. Current investigate possible extensions includes: –Better support for OWL semantics –RDF data insert and update –Aggregate functions –Standards XPath functions –Manipulation of Composite Datasets –Access to RDF lists 66

SUMMARY 67

Summary RDF Repositories –Optimized solutions for storing RDF –Adopts different implementation techniques –Choice has to be lead by multiple factors In general there is no a solution better than another SPARQL –Query language for RDF represented data –More powerful than XPath based languages –Supported by a communication protocol 68

References Mandatory reading –Semantic Web Primer Chapter 3 (Sections 3.8) –SPARQL Query Language for RDF Chapters 1 to 11 Further reading –RDF SPARQL Protocol 69