Presentation is loading. Please wait.

Presentation is loading. Please wait.

Comparing the accuracy of the semantic similarity provided by the Normalized Google Distance (NGD) and the Search Term Recommender (STR). Wilko van Hoek,

Published bySamson Wood Modified over 8 years ago

Similar presentations

Presentation on theme: "Comparing the accuracy of the semantic similarity provided by the Normalized Google Distance (NGD) and the Search Term Recommender (STR). Wilko van Hoek,"— Presentation transcript:

1 Comparing the accuracy of the semantic similarity provided by the Normalized Google Distance (NGD) and the Search Term Recommender (STR). Wilko van Hoek, Brigitte Mathiak, Philipp Mayr and Sascha Schüller 10 th NKOS Workshop at the TPDL 2011, Berlin

2 2 Comparing the accuracy of NGD and STRWilko van Hoek Overview  Motivation  Background  The Search Term Recommender (STR)  The Normalized Google Distance (NGD)  First research  Second research  Conclusion & Outlook

3 3 Comparing the accuracy of NGD and STRWilko van Hoek Motivation | Background | Research1 | Research2 | Outlook Motivation  The following use-case led to our research:  A documentation officer applying thesaurus terms to new documents  (Sometimes) it can be unclear which terms to apply  The STR can suggest eligible keywords  In some cases the STR has no recommendation to offer  NGD based on the web could still recommend TheSoz terms  Can the NGD be a solution in these cases? Is the NGD comparably accurate as the STR?

4 4 Comparing the accuracy of NGD and STRWilko van Hoek The Search Term Recommender  Recommends semantically similar TheSoz-terms  Based on Mindserver (proprietary software)  Co-Word analysis  Training Sets:  Social science database SOLIS (370.000 documents, title, abstract and controlled thesaurus terms)  Others: CSA-SA, CSA-PEI, SPOLIT, FES, … Motivation | Background | Research1 | Research2 | Outlook

5 5 Comparing the accuracy of NGD and STRWilko van Hoek An example recommended term (1.0): Environmental Education search term: Environmental Awareness Motivation | Background | Research1 | Research2 | Outlook

6 6 Comparing the accuracy of NGD and STRWilko van Hoek The Normalized Google Distance  Measures semantic similarity of two terms (x and y)  Bases on the number of webpages for either: x, y, x + y f(x) and f(x,y) denote the number of webpages found using the search terms x and x+y, N is the normalizing factor. According to [2] N should be greater than max(f(x),f(y)) and can be the total number of pages indexed by the search engine in use. Motivation | Background | Research1 | Research2 | Outlook

7 7 Comparing the accuracy of NGD and STRWilko van Hoek xyx+y f(x) 3,800,0006,680,000931,000 log f(x) 6.57986.82485.9689 x: environmental awareness y: environmental education Motivation | Background | Research1 | Research2 | Outlook An example

8 8 Comparing the accuracy of NGD and STRWilko van Hoek First research setup  88 random user search terms from Sowiport-logfile  Top 50 STR-recommendations per search term  pairwise calculation of NGD for STR-recommendation and search term Motivation | Background | Research1 | Research2 | Outlook

9 9 Comparing the accuracy of NGD and STRWilko van Hoek Top 10 of STR and NGD Search term: environmental awareness Number of different terms in top 10: 3 STR-recommendationNGD-resultsSTR - NGD environmental education1.0000 environmental policy0.91430.0857 environmental sociology1.0000 garbage removal0.90360.0964 environmental protection1.0000 environmental safety0.85250.1475 attitude1.0000 waste0.82200.1780 environmental behaviour1.0000 environmental behaviour0.81960.1804 environmental policy1.0000 public opinion0.81220.1878 ecology0.9999 environmental protection0.80600.1939 formation of consciousness0.9999 action0.80570.1943 environmental psychology0.9999 sustainable development0.80550.1943 everyday life0.9998 environmental pollution0.79320.2066 Motivation | Background | Research1 | Research2 | Outlook

10 10 Comparing the accuracy of NGD and STRWilko van Hoek Distribution of STR and NGD STR recommendationNGD result environmental education1.0000 environmental policy0.9143 environmental sociology1.0000 garbage removal0.9036 environmental protection1.0000 environmental safety0.8525 attitude1.0000 waste0.8220 environmental behaviour1.0000 environmental behaviour0.8196 environmental policy1.0000 public opinion0.8122 Motivation | Background | Research1 | Research2 | Outlook

11 11 Comparing the accuracy of NGD and STRWilko van Hoek Second research setup  4 significant user terms:  Environmental Awareness  Mobbing  Labour Market  Sustainability  All TheSoz terms (7,935) per search term  NGD pairwise for TheSoz term and search term  In total: 39680 queries processed Motivation | Background | Research1 | Research2 | Outlook

12 12 Comparing the accuracy of NGD and STRWilko van Hoek (new) Top10 of STR and NGD STR-recommendationNGD-resultsSTR - NGD environmental education1.0000 cost-benefit analysis0.98410.0159 environmental sociology1.0000 cultural program0.97590.0241 environmental protection1.0000 press0.97390.0261 attitude1.0000 marketing instrument0.97130.0287 environmental behaviour1.0000 manufacturing area0.97130.0287 environmental policy1.0000 school education097060.0294 ecology0.9999 instructor0.96820.0317 formation of consciousness0.9999 political administrative system0.96700.0329 environmental psychology0.9999 construction industry0.96550.0344 everyday life0.9998 assistance0.96560.0342 Motivation | Background | Research1 | Research2 | Outlook Search term: environmental awareness Number of different terms in top 10: 0

13 13 Comparing the accuracy of NGD and STRWilko van Hoek Results of the second research  Term overlap within top50:  Environmental Awareness:0  Mobbing:0  Labour Market:4 {(Un-)Employment, Measure, unemployed person}  Sustainability:1 {Ecology}  Human assessment Motivation | Background | Research1 | Research2 | Outlook equivalentpartly equ.topicsubtopicnonsense top 50 STR14%19%1%39%27% NGD1%2%0%29%69% top 10 STR33%38%5%20%5% NGD3% 0%30%65%

14 14 Comparing the accuracy of NGD and STRWilko van Hoek Conclusion  NGD not very accurate  This idea cannot be applied in our use-case Motivation | Background | Research1 | Research2 | Outlook Outlook  Examine other web based term suggestion possibilities  Apply and evaluation NGD to offline corpus

15 15 Comparing the accuracy of NGD and STRWilko van Hoek Thank you! Contact: Wilko van Hoek (wilko.vanhoek@gesis.org)wilko.vanhoek@gesis.org GESIS – Leibniz Institute for the Social Sciences Lennéstr. 30, 53113 Bonn www.gesis.orgwww.gesis.org Motivation | Background | Research1 | Research2 | Outlook

Download ppt "Comparing the accuracy of the semantic similarity provided by the Normalized Google Distance (NGD) and the Search Term Recommender (STR). Wilko van Hoek,"

Similar presentations

Documentation and Document Control

Documentation and Document Control
Assignments for the First and Second Year That Build Research and Information Literacy Skills PETAL Winter Workshop January 9, 2007 Marianne Simmons Lavery.

Assignments for the First and Second Year That Build Research and Information Literacy Skills PETAL Winter Workshop January 9, 2007 Marianne Simmons Lavery.
The Google Similarity Distance  We’ve been talking about Natural Language parsing  Understanding the meaning in a sentence requires knowing relationships.

The Google Similarity Distance  We’ve been talking about Natural Language parsing  Understanding the meaning in a sentence requires knowing relationships.
Independent Educational Evaluations Developed by Contra Costa SELPA As Recommended for LEA Board Policy 2001-2002.

Independent Educational Evaluations Developed by Contra Costa SELPA As Recommended for LEA Board Policy
Personalized Query Expansion for the Web Paul-Alexandru Chirita, Claudiu S. Firan, Wolfgang Nejdl Gabriel Barata.

Personalized Query Expansion for the Web Paul-Alexandru Chirita, Claudiu S. Firan, Wolfgang Nejdl Gabriel Barata.
Jean-Eudes Ranvier 17/05/2015Planet Data - Madrid Trustworthiness assessment (on web pages) Task 3.3.

Jean-Eudes Ranvier 17/05/2015Planet Data - Madrid Trustworthiness assessment (on web pages) Task 3.3.
GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.

GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.
Ao-Jan Su † Y. Charlie Hu ‡ Aleksandar Kuzmanovic † Cheng-Kok Koh ‡ † Northwestern University ‡ Purdue University How to Improve Your Google Ranking: Myths.

Ao-Jan Su † Y. Charlie Hu ‡ Aleksandar Kuzmanovic † Cheng-Kok Koh ‡ † Northwestern University ‡ Purdue University How to Improve Your Google Ranking: Myths.
Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)

Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)
A Virtual Organisation for e-Learning Nicola Capuano, Pierre Carrolaggi, Jerome Combaz, Fabio Crestani, Matteo Gaeta, Erich Herber, Enver Sangineto, Krassen.

A Virtual Organisation for e-Learning Nicola Capuano, Pierre Carrolaggi, Jerome Combaz, Fabio Crestani, Matteo Gaeta, Erich Herber, Enver Sangineto, Krassen.
Personalizing Search via Automated Analysis of Interests and Activities Jaime Teevan Susan T.Dumains Eric Horvitz MIT,CSAILMicrosoft Researcher Microsoft.

Personalizing Search via Automated Analysis of Interests and Activities Jaime Teevan Susan T.Dumains Eric Horvitz MIT,CSAILMicrosoft Researcher Microsoft.
Time-dependent Similarity Measure of Queries Using Historical Click- through Data Qiankun Zhao*, Steven C. H. Hoi*, Tie-Yan Liu, et al. Presented by: Tie-Yan.

Time-dependent Similarity Measure of Queries Using Historical Click- through Data Qiankun Zhao*, Steven C. H. Hoi*, Tie-Yan Liu, et al. Presented by: Tie-Yan.
Topic-Sensitive PageRank Taher H. Haveliwala. PageRank Importance is propagated A global ranking vector is pre-computed.

Topic-Sensitive PageRank Taher H. Haveliwala. PageRank Importance is propagated A global ranking vector is pre-computed.
University of Kansas Department of Electrical Engineering and Computer Science Dr. Susan Gauch April 2005 I T T C Dr. Susan Gauch Personalized Search Based.

University of Kansas Department of Electrical Engineering and Computer Science Dr. Susan Gauch April 2005 I T T C Dr. Susan Gauch Personalized Search Based.
Every Solution Consultancy ISO 9001:2008 Certification IMPLEMENTATION Web:

Every Solution Consultancy ISO 9001:2008 Certification IMPLEMENTATION Web:
New Ways of Mapping Knowledge Organization Systems Using a Semi-Automatic Matching- Procedure for Building Up Vocabulary Crosswalks Andreas Oskar Kempf.

New Ways of Mapping Knowledge Organization Systems Using a Semi-Automatic Matching- Procedure for Building Up Vocabulary Crosswalks Andreas Oskar Kempf.
‘ KNOWLEDGE MANAGEMENT ’ ACCORDING TO B USINESS S CIENCE Omwoyo Bosire Onyancha University of South Africa Department of Information Science.

‘ KNOWLEDGE MANAGEMENT ’ ACCORDING TO B USINESS S CIENCE Omwoyo Bosire Onyancha University of South Africa Department of Information Science.
OARE Module 5B: Searching for Scientific Research Using Environmental Issues and Policy Index (EBSCO)

OARE Module 5B: Searching for Scientific Research Using Environmental Issues and Policy Index (EBSCO)
Doing your literature review: an overview Katy Jordan Librarian, Economics & International Development Library & Learning Centre.

Doing your literature review: an overview Katy Jordan Librarian, Economics & International Development Library & Learning Centre.
Life Cycle Overview & Resources. Life Cycle Management What is it? Integrated concept for managing goods and services towards more sustainable production.

Life Cycle Overview & Resources. Life Cycle Management What is it? Integrated concept for managing goods and services towards more sustainable production.

Similar presentations

Ads by Google