1. Machine Learning and the Semantic Web
Hendrik Blockeel
Katholieke Universiteit Leuven
Department of Computer Science
Thanks : Raymond Kosala, Nico Jacobs

2. Overview Machine learning and data mining
Relationship with semantic web
Synergy between both
Some concrete examples
Document classification
Information integration
Conclusions

3. Machine Learning & Data Mining
Related technology, different focus
Machine learning:
Programs that improve their performance on certain tasks
Focus on adaptive behaviour
Data mining:
Discovering implicit knowledge (regularities) in large amounts of data
Focus on handling large amounts of data
Very useful technology in the context of the Web

4. Learning Agents Programs that
Learn the user’s preferences
Make life for the user as simple as possible
E.g., intelligent mail reader
E.g., adaptive web pages
Move links, create “direct” links, ...
Index page synthesis (Perkowitz & Etzioni, IJCAI 1999)
Learn how to find reliable information
E.g., learn which other people have similar preferences to this user, use their opinions to make suggestions
(other applications: learning to play games, ...)

6. Mining the Web Analyze data that are available on the Web
Distinguish 3 types:
Web content mining
Look in contents of documents (text, ...)
Web structure mining
Look at links between documents
Web usage mining
Look at user logs (e.g. who accessed a web page, which links often used, ...)

7. Web Content Mining Relies on information extraction
E.g., in a text: find keywords, ...
Techniques from machine learning, statistics, ... used to guess from context
what a word means
what its function in the text is
...
Fill a schema with specific slots, based on analysis of text
Even more complicated: recognise objects in pictures, ...
I.E. is a complex matter

8. Mining for Genes
Jenssen et al. (2001), Nature Genetics 28, “A literature network of human genes”
Mining MEDLINE database of abstracts
Find names of genes occurring together
Construct similarity graph
Construct a database with this information
Database contains knowledge no single individual has, or could obtain without data mining
Similar techniques could be used on the web
One extra problem: uncertainty about reliability

9. Web Structure Mining Analyse structure of the web E.g., Google
Which sites have many incoming / outgoing links?
Identify “hubs”
Find clusters of sites that are strongly interconnected
Web communities
...
E.g., Google
Identifies important pages based on links that point to it (rather than contents of page itself)

10. Web Usage Mining Log user behaviour E.g., adaptive web sites
Which links are often followed, in which order, how long is a page looked at, ...
Possible at several levels:
General usage statistics
User-specific statistics
Relating behaviour to properties of user, insofar available
E.g., adaptive web sites
Adaplix project
automatic index page creation

11. Web Mining As It Currently Is
Machine learning / data mining strongly rely on
Data quantity
Data quality
Quantity is usually not a problem on the Web
Quality is!
Much data not in easily processable format
E.g. Inside text documents : need information extraction
Unstructured, poorly structured, heterogeneously structured
Lots of noise
...

12. How Is All This Related to the Semantic Web?
There can be a synergy :
Machine learning can help with building the Semantic Web
The Semantic Web will help mining the Web, making Web interfaces and agents more intelligent

13. What Machine Learning Can Do for the Semantic Web
Upgrading the current web to a semantic web involves a lot of work
Can partially be automated!
Examples:
Learning ontologies
Automatic document classification
Information integration
...

14. Learning Ontologies
Maedche & Staab (2001), “Ontology learning for the semantic web”
View:
Manually creating of ontologies is very labour-intensive
Fully automating creating of ontologies is not feasible
Hence: develop tool that helps building ontologies
Basic components:
Good graphical interface (interaction man-machine)
Powerful underlying machine learning techniques

15. Text-To-Onto Framework : Import / reuse existing ontologies
Extract ontology from documents
Identify new terms, map onto existing concepts or define new ones
Identify relationships between concepts
...
Many opportunities for general machine learning techniques
Prune ontology
Refine ontology

16. Some Useful Techniques for Learning Ontologies
Term extraction from texts
Identification of concepts
Hierarchical Clustering
Clustering: finding groups of “similar” things
Hierarchical clustering: clusters of clusters
Taxonomy can be constructed through hierarchical clustering of concepts
Association rules
Find sets of terms that often occur together
May indicate important relations
E.g., events in texts often co-occur with locations

17. Information Integration
Doan, Domingos, Halevy: “Reconciling Schemas of Disparate Data Sources”, ACM SIGMOD 2001
Context:
Given databases with different schemas:
Find similarities in schemas, guess how concepts map onto each other
Integrate the schemas
Essentially the same as mapping ontologies onto each other

18. Automated Document Classification
Mitchell et al.
Based on examples of web pages + what kind of page they are (course page, student page, ...),
Learn to classify new pages
Can be based on contents of page, links pointing to page, typical structure of certain kinds of web sites (e.g. universities), ...
Note: helps to relate objects to ontology
Problem: how to get labeled examples
Unlimited amount of unlabelled pages available
But labelling them manually is labour intensive!

19. Exploiting Unlabelled Data
A solution: co-training (Blum & Mitchell 1998)
Learn separate (imperfect) classifiers from disjoint sets of sufficient information
E.g. Learn to classify pages from
Content of page (“Home page of CS 101”)
Links pointing to page (“CS 101”)
Take classifications that classifier A is most certain of, add these labels to training set for B (and vice versa)
Repeat multiple times (kind of bootstrapping process)
Co-training allows to exploit large amounts of unlabelled data!

20. What the Semantic Web Can Do for Machine Learning
Will make mining the web much easier
Reason 1: removal of ambiguity
More precise knowledge of what is meant with certain terms
Reason 2: structured vs. unstructured data
Learning from structured data is much easier than from unstructured data
Reason 3: availability of background knowledge
Can be used to make better decisions when learning

21. Removal of Ambiguity Example: text document classification
E.g., given a text, tell in which newsgroups it belongs
Typical approaches: “bag of words”
Look only at which words occur, in the text, and how often
Each time a word occurs that occurs mainly in one particular class, increase probability for that class
But words are ambiguous!
Increased classification accuracy can be expected by removing ambiguity

22. Mining From (Un)structured Data
Mining data = intensively querying data
Answering a querying is
Easy in structured data
Relational database, XML, ...
Harder in semi-structured data (e.g., HTML)
Hard in unstructured data
Information exraction needed
Could do this by learning a “wrapper”
This involves one extra layer of learning
Relating this to our text example: taking into account function of words in text

23. Availability of Background Knowledge
Learning = finding relevant patterns in behaviour
Important to have the right context to describe these patterns
Example:
Making interesting offers to clients
“People who bought this book also bought ...”
= “Instance-based” learning
Estimate profile of user
Find users with similar profile
Look at behaviour of those users to help current user

24. Availability of Background Knowledge
Can work better if more background knowledge is available, e.g., type of book, author, ...
For instance, for books:
“similar profile” = users that up till now bought same books as this user
May not be many people
“similar” = often bought books by same author
Probably many more people, allows for more reasonable guess
“similar” = often bought books of same genre (fiction, ...)
May work even better
Ontologies (among other) provide such background knowledge

25. Web Mining Revisited Semantic Web will change
Content mining
Clearer view on contents and meaning of documents
Structure mining
More relevant structure
Usage mining
More relevant information on actions of user
Will in general improve intelligence of systems
E.g. mail filter gets a better view of contents of mails

26. Promising Learning Techniques
Many different learning techniques exist
Neural networks, support vector machines, instance-based learning, bayesian learning, association rules, ...
Not all equally suitable for any task
E.g. SVM for document classification works well
E.g. instance-based learning: find other users with same profile as this user to make predictions
Intelligent agents will use a mix of them
Relational learners seem interesting
Can handle explicit information on objects and relations between them
Classic example: Inductive logic programming

27. Inductive Logic Programming
Induces rules in first order logic from examples or other rules
Such rules can be used to reason with
The reasoning can be explained
Cf. example of mail program
Can use existing background knowledge
“knowledge intensive learning”
Currently: good background knowledge has to be engineered manually
Will become more easily available with semantic web
Example: mining in chemical domains

29. Mining in chemical domains
Example problem: relate activity of molecule to its properties
Useful for, e.g., drug development
Which properties are important?
Chemically relevant properties: functional groups, 3D structure, ... ?
Has to be encoded manually
Ideally: get relevant information from some trustworthy data source as and when needed
Intelligent agents will exploit (“tap”) the common intelligence of the Web

30. Conclusions Machine learning is an promising tool for the Semantic Web
For building it
For exploiting it
Clear synergy between Semantic Web efforts and Machine Learning efforts

31. Some References
Maedche, “A Machine Learning Perspective for the Semantic Web”, position paper
Maedche & Staab (2001): Ontology Learning for the Semantic Web, IEEE Intelligent Systems 16(2)
Jenssen et al., Nature Genetics 28
Doan et al. (2001), ACM SIGMOD conf.
Kosala & Blockeel (2000), SIGKDD Explorations 2(1)
Mitchell (1996), Machine Learning