1. Web Mining
By:-
Vineeta
8pgc18
M.Tech (II Semester)

2. Introduction Why we need ? What is it ?
How it is different from classical data mining ?
What are the problems ?
Role of web mining
Web mining Taxonomy
Applications

3. Why we need Web Mining?
Explosive growth of amount of content on the internet
Web search engines return thousands of results so difficult to browse
Online repositories are growing rapidly
Using web mining web documents can easily be BROWSED,
ORGANISED and CATALOGED with minimal human
intervention

4. What is it?
Web mining - data mining techniques to automatically discover and extract information from web documents/services
Knowledge
www

5. How does it differ from “classical” Data Mining?
The web is not a relation
Textual information and linkage structure
Usage data is huge and growing rapidly
Google’s usage logs are bigger than their web crawl
Data generated per day is comparable to largest conventional data warehouses
Ability to react in real-time to usage patterns
No human in the loop

6. Web Mining: Problems The “abundance” problem
Limited coverage of the Web
Limited query interface based on keyword-oriented search
Limited customization to individual users
Dynamic and semi structured

7. Role of web mining Finding Relevant Information
Creating knowledge from Information available
Personalization of the information
Learning about customers / individual users

8. Web Mining Taxonomy Web Mining Web Content Mining Web Structure Mining
Web Usage Mining
Identify information
within given web pages
Distinguish personal home pages from other web pages
Understand access
patterns and the trends
to improve structure
Uses interconnections
between web pages to
give weight to the
pages

9. Web Content Mining
Web Content Mining is the process of extracting useful information from the contents of Web documents.
Content data corresponds to the collection of facts a Web page was designed to convey to the users. It may consist of text, images, audio, video, or structured records such as lists and tables.
Research activities in this field also involve using techniques from other disciplines such as Information Retrieval (IR) and natural language processing (NLP).

10. Web Content Mining Web Content Mining Agent Based Approach
Database Approach
Intelligent
Search
Agent
Information
Filtering &
Categorization
Personalized
Web Agent
Multilevel
Databases
Web Query
Systems

11. Intelligent Search Agents
Concentrate on searching relevant information using the characteristics of a particular domain to interpret and organize the collected information.
It can be further classified into two types:
Interpretation Based on Pre-Specified Information:
Examples: Harvest, FAQFinder, Information Manifold, OCCAM
Interpretation Based on Unfamiliar Source:
Example: ShopBot

12. ShopBot
A ShopBot is an autonomous software agent that comb the internet providing users with low price product or product recommendations.
A ShopBot basically looks for product information from a variety of vendor sites using the general information about the product domain.
The following example displays a shopBot at

15. Information Filtering & Categorization
This makes use of various information retrieval techniques and characteristics of hypertext web documents to interpret and categorize data.
Examples: HyPursuit, BO (Bookmark Organizer).

16. Bookmark Organizer (BO)
Makes use of hierarchical clustering techniques and involves user interaction to organize a collection of web documents.
It operates in two modes:
Automatic
Manual
Frozen Nodes: In a hierarchical structure, if we freeze a node N, then the subtree rooted at N represents a coherent group of documents.

17. Personalized Web Agents
This category of Web agents learn user preferences and discover Web information sources based on these preferences, and those of other individuals with similar interests.
Examples:
WebWatcher
PAINT
Syskill&Webert
GroupLens
Firefly

18. Multilevel Databases Layer 0 :
Unstructured, massive and global information base.
Layer 1:
Derived from lower layers.
Relatively structured.
Obtained by data analysis, transformation & Generalization.
Higher Layers (Layer n):
Further generalization to form smaller, better structured databases for more efficient retrieval.

19. Web Query System These systems attempt to make use of:
Standard database query language – SQL
Structural information about web documents
Natural language processing for queries made in www searches.
Examples:
WebLog: Restructuring extracted information from Web sources.
W3QL: Combines structure query (organization of hypertext) and content query (information retrieval techniques).

20. Web Structure Mining
Web Structure Mining is the process of discovering structure information from the Web. This type of mining can be performed either at the (intra-page) document level or at the (inter-page) hyperlink level.The research at the hyperlink level is also called
HYPERLINK ANALYSIS

21. Web Structure Mining Different Algorithms for Web Structures:
Page-Rank Method
Sergey Brin and Lawrence Page: The anatomy of a large-scale hypertextual web search engine. In Proc. Of WWW, pages 107–117, Brisbane, Australia, 1998.
CLEVER Method

22. Page-Rank Method Introduced by Brin and Page (1998)
Used in Google Search Engine
Mine hyperlink structure of web to produce ‘global’ importance ranking of every web page
Web search result is returned in the rank order
Treats link as like academic citation
Assumption: Highly linked pages are more ‘important’ than pages with a few links
A page has a high rank if the sum of the ranks of its back-links is high

23. Backlink
Link Structure of the Web

24. CLEVER Method CLient–side EigenVector-Enhanced Retrieval
Developed by a team of IBM researchers at IBM Almaden Research Centre
Ranks pages primarily by measuring links between them
Continued refinements of HITS ( Hypertext Induced Topic Selection)
Basic Principles – Authorities, Hubs
Good hubs points to good authorities
Good authorities are referenced by good hubs

25. Web Usage Mining Web usage mining also known as Web log mining
mining techniques to discover interesting usage patterns from the data derived from the interactions of the users while surfing the web
mining Web log records to discover user access patterns of Web pages

26. Web Usage Mining – Three Phases

27. Web Usage Mining
Pre processing consists of converting the usage, content, and structure information contained in the various available data sources into the data abstractions necessary for pattern discovery
Pattern discovery draws upon methods and algorithms developed from several fields such as statistics, data mining, machine learning and pattern recognition.
The motivation behind pattern analysis is to filter out uninteresting rules or patterns from the set found in the pattern discovery phase. The exact analysis methodology is usually governed by the application for which Web mining is done.

28. Applications Personalized experience in B2C e-commerce –Amazon.com
Web search –Google
Web-wide user tracking –DoubleClick
Understanding user communities –AOL
Understanding auction behavior –eBay
Personalized web portal –MyYahoo

29. Conclusion
Web mining - data mining techniques to automatically discover and extract information from Web documents/services (Etzioni, 1996). Web mining research – integrate research from several research communities (Kosala and Blockeel, July 2000) such as:
Database (DB)
Information retrieval (IR)
The sub-areas of machine learning (ML)
Natural language processing (NLP)

30. References mandolin.cais.ntu.edu.sg/wise2002/web-mining-WISE-30
David Gibson, Jon Kleinberg, and Prabhakar Raghavan. Inferring web communities from link topology. In Conference on Hypertext and Hypermedia. ACM, 1998.

31. References http://en.wikipedia.org/wiki/Web_mining
Y. S. Mareek and I. Z. B. Shaul. Automatically organizing bookmarks per contents. Proc. Fifth International World Wide Web Conference, May
Cooley, R., B. Mobasher, et al. (1997). Web Mining: Information and Pattern Discovery on the World Wide Web, Proc. IEEE Intl. Conf. Tools with AI, Newport Beach, CA, pp , 1997.

32. References
R. Kosala. and H. Blockeel, Web Mining Research: A Survey, SIGKDD Explorations, 2(1):1-15, 2000.
R. Cooley, B. Mobasher, and J. Srivastava. Data preparation for mining world wide web browsing patterns. Journal of Knowledge and Information Systems 1, 5-32, 1999
S. Chakrabarti, Data mining for hypertext: A tutorial survey. ACM SIGKDD Explorations, 1(2):1-11, 2000System, 1(1), 1999

33. THANK YOU!!