SICoP Presentation A story about communication Michael Lang BEARevelytix April 25, 2007

Agenda ► Communication in Indonesia ► Schemas and Languages  Semantic Technology  ETL and data warehousing ► Roadmap  Transitioning from here to there ► Vocabulary Management ► Semantic Data Services

Indonesia

Indonesia in 1960 ► Indonesia is comprised of several thousand islands ► 340 different languages/dialects are spoken ► Indonesia is not an integrated country ► The President decided that Indonesia needed a common vocabulary so that the groups could communicate with each other

Indonesia Today ► The language/vocabulary was developed and is now in widespread use ► Indonesia is interoperable today

IT Systems ► Each system is an island and it has its own language  We call these languages schemas ► If we want to be able to speak among systems we need a common vocabulary ► Key concept #1  Language = Vocabulary = Schema

Languages ► Features  A way to represent concepts and the relationships between concepts  A way of representing facts about the world  A way of adding concepts, relationships and facts without breaking the language  A syntax or grammar that shows how the language is structured and functions

Why are there Data Warehouses ► The Sales community uses the schema of the CRM system to capture facts and concepts ► Manufacturing uses the schema of the ERP system ► Finance wants to forecast trends  Invent a new language that has some of the concepts and facts of both systems as well as additional concepts  Build a new system

XSD as a Language ► XSD is a great way to define the terms and syntax for a community  And it is a language, and it can be extended  But, systems processing XML documents containing the data are expecting the structure to be fixed according to an XSD  If the structure is different the system must be changed ► Address  Street  City  Zip code Address Street Apt number City Zip Address Street1 Street2 City Postal Code

XSD as a Language ► The purpose of XSD is to specify syntax  The representation of the data is determined by the schema  IT systems that process XML count on this ► XSD has serious limitations as the technology for achieving interoperability  Every time the Language is extended the systems must be recoded ► New communities cannot enter into the vocabulary by adding their own concepts

RDF OWL SPARQL Semantic Technology

► The first fundamental change in Information Management since the RDBMS was developed in the early 1980’s  25 years is a good run for a technology  It probably has 25 years still to go ► Key concept #2  The “data” exists independently of any schema  data = instances = facts

RDF RDF is uniquely positioned as a machine readable lingua franca for representing relational, XML, proprietary, and most other data formats ► BUT, it is not a language  There is no schema  It is a way of making data accessible to ANY schema defined using OWL or RDFS

RDF ► ► RDF uses a graph structure   Can more accurately represent the entities being modeled   Real objects, concepts, and processes often have a ragged shape   Can represent objects with complex structures directly without exposing implementation techniques, e.g. hierarchical structures ► ► Data schemas do not need to be determined a priori ► ► New data from disparate sources can be integrated seamlessly on the fly by merging two (or more) RDF graphs ► ► RDF models data’s content and meaning, rather than just its structure or serialization

RDF is Schemaless ► The schema can be defined using RDFS or OWL ► Since the data is independent of the schema any language can be developed and deployed   Seamlessly incorporate new data sources, regardless of schema   Business logic and user-interface logic far more resilient in the face of unexpected data and schema evolution

Creating Languages with OWL Structuring RDF with Ontologies ► ► The Web Ontology Language (OWL) allows the declaration of rules to define and describe RDF vocabularies. ► OWL can be used to define the language for any community ► OWL models can be extended without breaking the system that is consuming the data

SPARQL SPARQL: Web-aware semantic query ► ► SPARQL is a SQL-like language for querying distributed RDF graphs ► ► SPARQL queries emphasize:   Distributed data. Multiple data sources can be queried at once because SPARQL addresses graphs by URI.   Ragged data. The SPARQL OPTIONAL keyword lets users explore heterogeneous data in a single query.   Unpredictable data. The ability to query for predicates and information about predicates makes SPARQL ideal for exploring new and unexpected data.

Myth: None of this Works ► Not true  Many tools are capable of producing RDF  Some are good at representing relational data as RDF ► There are several good OWL modeling tools available ► Jena and Sesame are open source reasoner/SPARQL engines ► Many commercial products are incorporating support for semantic technology now

Roadmap

How to get from here to there ► Here  Currently SOA’s exchange data as XML where the document structure is determined by an XSD and the query language is XQuery ► There  SOA’s exchange data as RDF and query using SPARQL

Transition Plan ► Start developing Community of Interest based Vocabularies as OWL models immediately ► Generate XSD’s to facilitate the use of existing infrastructure directly from the OWL models ► Begin a program of representing service based information as RDF ► Build applications that access the RDF representation using the previously developed OWL models through SPARQL/reasoner engines

Conclusions ► Semantic Technology is the only natural integration technology available ► It is game changing in delivering capability ► The road ahead is apparent if not quite obvious yet ► Commercial tools are available and more are coming soon ► It is time to get started

Vocabulary Management

Designing Semantic Data Services Structured and Unstructured Data DB XML Customer Portal Partner App Employee Portal EAI / ESB BPM ?

Structured and Unstructured Data DB XML Customer Portal Partner App Employee Portal EAI / ESB BPM Semantic Data Services Designing Semantic Data Services

Vocabulary Management ► Step 1: Extract semantics from existing data  Import, tokenize, assign definitions, …

Vocabulary Management Step 1: Extract semantics from existing data DB XML

Vocabulary Management ► Step 1: Extract semantics from existing data  Import, tokenize, assign definitions, … ► Step 2: Create bootstrapped vocabulary  Discover terms, concepts, duplications, similarities, …

Vocabulary Management Step 2: Create bootstrapped vocabulary OWL

Vocabulary Management ► Step 1: Extract semantics from existing data  Import, tokenize, assign definitions, … ► Step 2: Create bootstrapped vocabulary  Discover terms, concepts, duplications, similarities, … ► Step 3: Evolve vocabulary collaboratively  Make meaningful for people using natural language  Use formalisms for machines via OWL  Web-based collaboration and sharing

Vocabulary Management Step 3: Evolve vocabulary collaboratively

Vocabulary Management ► Step 1: Extract semantics from existing data  Import, tokenize, assign definitions, … ► Step 2: Create bootstrapped vocabulary  Discover terms, concepts, duplications, similarities, … ► Step 3: Evolve vocabulary collaboratively  Make meaningful for people using natural language  Use formalisms for machines via OWL  Web-based collaboration and sharing ► Step 4: Use vocabulary  Used by people and tools

Vocabulary Management Step 4: Use vocabulary to understand

Structured and Unstructured Data DB XML Customer Portal Partner App Employee Portal EAI / ESB BPM Semantic Data Services Vocabulary Management ???? Step 4: Use vocabulary to design services OWL

Vocabulary Management Step 4: Use vocabulary to design services

Designing Semantic Data Services Vocabulary Management: ► Bootstrap vocabularies with existing data sources ► People collaborate on creation of vocabularies ► People can understand vocabularies ► Tools can use vocabularies (OWL artifacts) to produce semantic data services Next: ► Deploy and use semantic data services

Structured and Unstructured Data DB XML Customer Portal Partner App Employee Portal EAI / ESB BPM Designing Semantic Data Services BEA AquaLogic Data Services Platform (ALDSP) OWL Import BEA AquaLogic Workshop Semantic Data Services Mapping with MatchIT Knoodl

SICoP Knowledge Reference Model The point of this graph is that Increasing Metadata (from glossaries to ontologies) is highly correlated with Increasing Search Capability (from discovery to reasoning).