Exploiting large scale web semantics to build end user applications Enrico Motta Professor of Knowledge Technologies Knowledge Media Institute The Open University

Aims of the Talk What is the Semantic Web –Perspectives The SW as a ‘web of data’ The SW as a new context in which to build semantic applications and an unprecedented opportunity in which to address some classic AI problems –Typical misconceptions What the SW is not! Semantic Web for Users –Applications that do something interesting and useful to users, by exploiting available web semantics

The Semantic Web as a ‘Web of Data’ Making data available to SW-aware software

Enrico Motta Enrico Motta Knowledge Technologies Semantic Web Ontologies Problem Solving Methods Knowledge Modelling Knowledge Management London Luton Airport LTN Luton, United Kingdom AquaLog

The web of SW documents

Current status of the semantic web million semantic web documents –Expressed in RDF, OWL, DAML+OIL 7K-10K ontologies –These cover a variety of domains - music, multimedia, computing, management, bio-medical sciences, upper level concepts, etc… Hence: –To a significant extent the semantic web is already in place –However, domain coverage is very uneven –Still primarily a research enterprise, however interest is rapidly increasing in both governmental and business organizations “early adopters” phase The above figures refer to resources which are publicly accessible on the web

CS Dept Data AKT Reference Ontology RDF Data Bibliographic Data Geography

A ‘corporate ontology’ is used to provide a homogeneous view over heterogeneous data sources. Often tackle Enterprise Information Integration scenarios Hailed by Gartner as one of the key emerging strategic technology trends –E.g., Garlik is a multi-million startup recently set up in UK to support personal information management, which uses an ontology to integrate data mined from the web on a large scale “Corporate Semantic Webs”

AquaLog

Applications that exploit large scale semantic content

The web of data

Gateways to the SW Application Semantic Web

Sophisticated quality control mechanism –Detects duplications –Fixes obvious syntax problems E.g., duplicated ontology IDs, namespaces, etc.. Structures ontologies in a network –Using relations such as: extends, inconsistentWith, duplicates Provides interfaces for both human users and software programs Provides efficient API Supports formal queries (SPARQL) Variety of ontology ranking mechanisms Modularization/Combination support Plug-ins for Protégé and NeOn Toolkit Very cool logo!

Case Study 1: Automatic Alignment of Thesauri in the Agricultural/Fishery Domain

Method Concept_A (e.g., Supermarket) Concept_B (e.g., Building) Scarlet Semantic Web Semantic Relation ( ) Deduce Access -SCARLET - matching by Harvesting the SW -Automatically select and combine multiple online ontologies to derive a relation

Two strategies Supermarket Building Supermarket Shop PublicBuilding Building Scarlet CholesterolOrganicChemical Cholesterol Steroid Lipid OrganicChemical Scarlet Steroid Deriving relations from (A) one ontology and (B) across ontologies. Semantic Web (A)(B)

Matching: AGROVOC UN’s Food and Agriculture Organisation (FAO) thesaurus descriptor terms non-descriptor terms NALT US National Agricultural Library Thesaurus descriptor terms non-descriptor terms Experiment

226 Used Ontologies htechsight/Technologies.daml

Evaluation 1 - Precision Manual assessment of 1000 mappings (15%) Evaluators: –Researchers in the area of the Semantic Web –6 people split in two groups Results: –Comparable to best results for background knowledge based matchers.

Evaluation 2 – Error Analysis

Case Study 2: Folksonomy Tagspace Enrichment

Tagging as opposed to rigid classification Dynamic vocabulary does not require much annotation effort and evolves easily Shared vocabulary emerge over time –certain tags become particularly popular Features of Web2.0 sites

Limitations of tagging Different granularity of tagging –rome vs colosseum vs roman monument –Flower vs tulip –Etc.. Multilinguality Spelling errors, different terminology, plural vs singular, etc… This has a number of negative implications for the effective use of tagged resources –e.g., Search exhibits very poor recall

Giving meaning to tags

1. Mapping a tag to a SW element "japan" What does it mean to add semantics to tags? 2. Linking two "SW tags" using semantic relations {japan, asia}

Applications of the approach To improve recall in keyword search To support annotation by dynamically suggesting relevant tags or visualizing the structure of relevant tags To enable formal queries over a space of tags –Hence, going beyond keyword search To support new forms of intelligent navigation –i.e., using the 'semantic layer' to support navigation

Concept and relation identification No END Remaining tags? Clustering Google Folksonomy Cluster tags Cluster 1 Cluster 2 Cluster n … 2 “related” tags Find mappings & relation for pair of tags Yes Analyze co-occurrence of tags Co-occurence matrix Pre-processing Tags Group similar tags Filter infrequent tags Concise tags Clean tags Wikipedia SW search engine

participant innovation event developer activity creator planning example applica- tion user admin resource typeRangecomponent interface partici- patesIn in-event archive Information Object has-mention-of Examples Cluster_1 : { admin application archive collection component control developer dom example form innovation interface layout planning program repository resource sourcecode}

Examples Cluster_2 : { college commerce corporate course education high instructing learn learning lms school student} education training 1,4 qualification corporate 1 institution university 2,3 college 2 postSecondary School 2 school 2 student 3 studiesAt course 3 offersCoursetakesCourse activities 4 learning 4 teaching

Faceted Ontology Ontology creation and maintenance is automated Ontology evolution is driven by task features and by user changes Large scale integration of ontology elements from massively distributed online ontologies Very different from traditional top-down- designed ontologies

Case Study 3: Reviewing and Rating on the Web

Revyu.com

expertise the source has relevant expertise of the domain of the recommendation-seeking; this may be formally validated through qualifications or acquired over time. experience the source has experience of solving similar scenarios in this domain, but without extensive expertise. impartiality the source does not have vested interests in a particular resolution to the scenario. affinity the source has characteristics in common with the recommendation seeker, such as shared tastes, standards, values, viewpoints, interests, or expectations. track record the source has previously provided successful recommendations to the recommendation seeker. Trust Factors

subjective affinityexpertise experience objective solution factors emphasised

Applying the framework to revyu.com Affinity –Operationalised as the degree of overlap in items reviewed, and in ratings given Experience –Proxy metric: Usage of particular tags (as proxies for topics) Experience scores based on tagging data Integrates also data from del.icio.us for those users who have chosen to publish their del.icio.us account on FOAF Expertise –Proxy metric: Credibility –Captures the social aspect of expertise: endorsement

Using trust factors for ranking reviews

PowerAqua and PowerMagpie

How does the Semantic Web relate to Artificial Intelligence research?

AI as Heuristic Search

The knowledge-based paradigm in AI “Today there has been a shift in paradigm. The fundamental problem of understanding intelligence is not the identification of a few powerful techniques, but rather the question of how to represent large amounts of knowledge in a fashion that permits their effective use” Goldstein and Papert,1977

Knowledge Representation Hypothesis in AI Any mechanically embodied intelligent process will be comprised of structural ingredients that we as external observers naturally take to represent a propositional account of the knowledge that the overall process exhibits, and independent of such external semantic attribution, play a formal but causal and essential role in engendering the behaviour that manifests that knowledge Brian Smith, 1982

Knowledge-Based Systems Large Body of Knowledge Intelligent Behaviour

The Knowledge Acquisition Bottleneck Large Body of Knowledge Intelligent Behaviour KA Bottleneck Knowledge

The Cyc project

Problem Solving Method Generic Task Domain Model Mapping Knowledge Application-specific Problem-Solving Knowledge Application Configuration Parametric Design Library of PSMs Mapping Ontology Ontology Structured libraries of reusable components Classification Scheduling Etc…

The next knowledge medium “An information network with semi-automated services for the generation, distribution, and consumption of knowledge” However, our approach based on structured libraries of problem solving components only addressed the economic cost of KBS development…

SW as Enabler of Intelligent Behaviour Intelligent Behaviour Both a platform for knowledge publishing and a large scale source of knowledge

KBS vs SW Systems Classic KBSSW Systems ProvenanceCentralizedDistributed SizeSmall/MediumExtra Huge Repr. SchemaHomogeneousHeterogeneous QualityHighVery Variable Degree of trustHighVery Variable

Key Paradigm Shift Classic KBSSW Systems IntelligenceA function of sophisticated, logical, task- centric problem solving A side-effect of being able to integrate different types of reasoning to handle size and heterogeneous quality and representation

Conclusions

Typical misconceptions… “The SW is a long-term vision…” –Ehm…actually… it already exists… “The SW will never work because nobody is going to annotate their web pages” –The SW is not about annotating web pages, the SW is a web of data, most of which are generated from DBs, or from web mining software, or from applications which produce SW technology “The idea of a universal ontology has failed before and will fail again. Hence the SW is doomed” –The SW is not about a single universal ontology. Already there are around 10K ontologies and the number is growing… –SW applications may use 1, 2, 3, or even hundreds of ontologies.

Large Scale Distributed Semantics Widespread production of formalised knowledge models (ontologies and metadata), from a variety of different groups and individuals –E.g., legal, bio-medical, governmental, environmental, music, art, multimedia, computing, etc.. –“Knowledge modelling to become a new form of literacy?” Stutt and Motta, 1997 This large scale heterogenous resource will enable a new generation of semantic-aware technologies These developments may provide a new context in which to address the economic barriers to KBS development The SW already exists to some extent, however there is still a way to go, before it will reach the required degree of maturity

Large Scale Distributed Semantics Much like AI, the semantic web will only succeed if it becomes ubiquitous and hidden “There's this stupid myth out there that A.I. has failed, but A.I. is everywhere around you every second of the day. People just don't notice it. You've got A.I. systems in cars, tuning the parameters of the fuel injection systems. When you land in an airplane, your gate gets chosen by an A.I. scheduling system. Every time you use a piece of Microsoft software, you've got an A.I. system trying to figure out what you're doing, like writing a letter, and it does a pretty damned good job. Every time you see a movie with computer-generated characters, they're all little A.I. characters behaving as a group. Every time you play a video game, you're playing against an A.I. system.” Rodney Brooks