1. Presented by: Apeksha Khabia Guided by: Dr. M. B. Chandak
A Combined Approach for Classification of Web Results based on Ranking & Clustering
Presented by:
Apeksha Khabia
Guided by:
Dr. M. B. Chandak

2. Contents Introduction Page rank, weighted page rank
Document clustering
Algorithm for clustering and ranking
Conclusion
Future scope

3. Introduction
The web is most precious place for Information retrieval and Knowledge Discovery
Retrieving information through queries from a search engine is tedious
Solution is Web Mining
web content mining
web structure mining
web uses mining

4. How to Generate Web Results?

5. Page Rank (PR)
Order the search results such that important documents move up and less important move down in the list
If a page has some important incoming links then its outgoing link also becomes important

6. Page Rank (PR)
Rank score of a page p is evenly divided among outgoing links
Modified PR in view Random Surfer Model – not all the users follow direct Links on WWW

7. Example of Page Rank (PR)
PR(A)= (1-d)+d((PR(B)/2+PR(C)/2 )
PR(B)= (1-d)+d( PR(A)/1+PR(C)/2 )
PR(C)= (1-d)+d( PR(B)/2)
IF d=0.5
PR(A)=1.2, PR(B)=1.2, PR(C)=0.8
Table : Iterative method of page rank

8. Weighted Page Rank (WPR)
Assign larger rank values to more important pages instead of evenly dividing among its outgoing links.
Outlink page gets value according to its popularity

9. Document Clustering
 Automatic document organization, topic extraction and fast information retrieval or filtering
Documents are grouped together based upon measure of similarity of content or of hyperlinked structure
Clustering divides the results of a search for "cell" into groups like "biology," "battery," and "prison."

10. Document Clustering Examples : K-means, hierarchical
Clustering may be based on content alone, or both on contents and links or only on links
Two ways to define content based similarity between the documents
Resemblance
Containment

11. Limitations of Ranking Approach
They give emphasis to links of the resultant pages
No algorithm exists to combine the link score and content score of the page into a single score
Existing approaches return millions of documents in an ordered format
Rank based approaches give equal emphasis to inlinks as well as outlinks of pages

12. Introduction of Combined Approach
This mechanism takes advantage of importance of inlinks over outlinks
With the use of this user can put search results into hierarchy of query related clusters
Also the documents in each cluster can be ranked to represent them according to their relevancy
Such organization enables the user to effectively limit his search area

13. Algorithm for Clustering and Ranking

14. Algorithm: Steps Step 1: Get the URLs of the pages
Step 2: Provide a similarity value sim(q, p) to each returned document
Step 3: Use sim(q, p) to cluster the documents
Step 4: Provide a rank score WSR(p) to the documents of each cluster:

15. Output
Clusters of web pages documents are formed based on the similarity
Also the documents in each cluster are ranked to represent them according to their relevancy

16. Similarity Calculation between Web Pages
Similarity of the document with the query means:
what query terms are present in the document?
where they are present?
how many times?
Calculated using cosine between vector of query terms and vector of documents

17. Rank Calculation of Web pages
WSR- Weight and Similarity based Rank
Back-links contribute more towards the importance of a page rather than forward links
WSR gives more importance to the inlinks of a page
Importance of the backlink page v of a page u, given by

18. Rank Calculation of Web pages
Redefined formula for rank is given by

19. Clustering of Web pages
The clustering is purely based on the similarity values of the pages with respect to the user query
The number of clusters is not predefined
The maximum number of pages that can be in a cluster should be decided

20. Clustering of Web pages
Lower and upper value of similarity is identified from range of similarity values
Complete page set is divided into number of sets according to the similarity values lying within the partitioned ranges.
Rank(p)= WSR(p) + sim(q, p)

21. Applications of Clustering and Ranking
Readability assessment - automatically determining the degree of readability of a text, either to find suitable materials for different age groups or reader types or as part of a larger text simplification system
Genre classification - automatically determining the genre of a text

22. Tools Available
Weka
Rapid Miner
KNIME
Orange

23. Conclusion
Ranking and clustering gives a way to organize the search results in the form of clusters, the pages in each cluster are further ranked to provide the most relevant and important pages on the top of the cluster
User search space decreases and he can get required content in short time

24. Future Ideas
Different data mining tools (KNIME, Rapid Miner, Weka) can be used to analyze the result for classification
Search query results for classification can be incorporated from multiple search engines

25. References
Parneet Kaur, Sawtantar Singh Khurmi and Gurpreet Singh Josan, " Analysis for Classification of Similar Documents among Various Websites using Rapid Miner“. In the proceedings of IEEE International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT), 2014
Neelam Duhan and A.K. Shanna, "A Novel Approach for Organizing Web Search Results using Ranking and Clustering". In the Proceedings of International Journal of Computer Applications, vol. 5, No. 10, pp. 1-9, August 2010.
O. Zamir, O. Etzioni. “Web document clustering: A feasibility demonstration”. Proceedings of the 19th International ACM SIGIR Conference on Research and Development of Information Retrieval (SIGIR'98), 46-54,1998.

26. References
Miguel Gomes da Costa Júnior, Zhiguo Gong, “Web Structure Mining: An introduction”. Proceedings of the IEEE International Conference on Information Acquisition, 2005, China.
Taher H. Haveliwala, Aristides Gionis, Dan Klein, Piotr Indyk, “Evaluating strategies for similarity search on the Web”. WWW2002, May, 2002, Honolulu, Hawaii, USA.ACM /02/0005.

27. Thank You!!