Presentation is loading. Please wait.

Presentation is loading. Please wait.

Learning to Query: Focused Web Page Harvesting for Entity Aspects

Published byVivien Reynolds Modified over 8 years ago

Similar presentations

Presentation on theme: "Learning to Query: Focused Web Page Harvesting for Entity Aspects"— Presentation transcript:

1 Learning to Query: Focused Web Page Harvesting for Entity Aspects
Yuan Fang1, Vincent Zheng2, Kevin Chang23 ICDE Helsinki 1 Institute for Infocomm Research, Singapore 2 Advanced Digital Sciences Center, Singapore 3 University of Illinois at Urbana-Champaign, USA

2 In this talk Problem: L2Q Challenges and solution Experimental Study
Domain-awareness Context-awareness Experimental Study Conclusion

3 Entities and their aspects are abundant, but scattered, on the Web
People entities alone: 10% of Bing’s search volume Entity type Common aspects celebrity spouse, age, net worth, … car safety, cost, interior, … business address, opening hour, phone no., …

4 Motivation: Focused Web Page Harvesting for Entity Aspects
Business Analytics Vertical portal or search

5 High level problem: Learning to query (L2Q)
Keywords (uniquely) identifying the entity Seed query Target aspect A pre-trained classifier 𝑌, mapping each page to “relevant” or “not relevant” to the target aspect Utility (precision/recall) In each iteration, 𝑞 ∗ = arg max 𝑞 𝒰 𝑌 (𝑞)

6 In this talk Problem: L2Q Challenges and solution Experimental Study
Domain-awareness Context-awareness Experimental Study Conclusion

7 Subproblem #1: Domain-aware L2Q
Target Entity Marc Snir Peer Entities Philips Yu Andrew Ng …… entity pages 𝑃 𝐸 Gradually harvested in earlier iterations domain pages 𝑃 𝐷 Pre-collected / previously harvested pages 𝑞 ∗ = arg max 𝑞 𝒰 (𝑌) (𝑞| 𝑃 𝐸 , 𝑃 𝐷 )

8 Subproblem #1: Vocabulary variations
topics hpc data mining ai … journals ijhpca tkde jmlr … <topic> <journal> (template)

9 Subproblem #1: Bridging domain and entity phases
Domain Phase Entity Phase Utility regularization (i.e. supervision on target aspect)

10 In this talk Problem: L2Q Challenges and solution Experimental Study
Domain-awareness Context-awareness Experimental Study Conclusion

11 Subproblem #2: Context-aware L2Q
In iteration-i, a context of already fired queries Φ= 𝑞 0 , 𝑞 1 ,…, 𝑞 𝑖−1 . Queries can retrieve redundant pages Marc Snir HPC Marc Snir Parallel

12 Subproblem #2: Accounting for redundancy
pages for target aspect Ω(𝑌) pages harvested in previous iterations Ω(Φ) pages of candidate query Ω(𝑞) Collective Utilities Collective precision : Ω 𝑞 ⋃Ω Φ ⋂Ω 𝑌 Ω 𝑞 ⋃Ω(Φ) Collective recall : Ω 𝑞 ⋃Ω Φ ⋂Ω 𝑌 Ω 𝑌

13 In this talk Problem: L2Q Challenges and solution Experimental Study
Domain-awareness Context-awareness Experimental Study Conclusion

14 Experiment setup Datasets: two domains Search engine: language model
996 researchers & 143 car models Pre-collected pages to simulate the corpus Search engine: language model Dictionaries for templates Gathered from existing knowledge base Manually compiled Entity aspects 7 attributes for each domain Pre-trained aspect classifier with high accuracy

15 Experiment methodology
Utilities of two forms: precision & recall Evaluation metrics Precision, recall Combined F-score Metrics reported are normalized Against ideal precision/recall Ideal metrics computed by “peeking” at un- retrieved pages

16 Finding #1: Effect of domain and context-awareness
RND: select query randomly P/R: optimize precision/recall without domain and context-awareness P/R+q: with domain pages, but do not employ templates, and without context P/R+t: with domain pages and templates, without context L2QP/L2QR: full approaches optimizing precision/recall

17 Finding #2(a): Comparing precision with indep. baselines
LM: language feedback model AQ: adaptive querying for text databases HR: harvest rate for hidden structured databases MQ: manually designed queries

18 Finding #2(b): Comparing recall with indep. baselines
LM: language feedback model AQ: adaptive querying for text databases HR: harvest rate for hidden structured databases MQ: manually designed queries

19 Finding #2(c): Comparing F-score with indep. baselines
L2QBAL: optimize for F-score, balancing L2QP & L2QR LM: language feedback model AQ: adaptive querying for text databases HR: harvest rate for hidden structured databases MQ: manually designed queries

20 In this talk Problem: L2Q Challenges and solution Experimental Study
Domain-awareness Context-awareness Experimental Study Conclusion

21 Conclusion L2Q: a novel paradigm of crawling Domain-aware L2Q
Templates to handle vocabulary variations Context-aware L2Q Collective utilities to account for page redundancy

Download ppt "Learning to Query: Focused Web Page Harvesting for Entity Aspects"

Similar presentations

Chapter 5: Introduction to Information Retrieval

Chapter 5: Introduction to Information Retrieval
1.Accuracy of Agree/Disagree relation classification. 2.Accuracy of user opinion prediction. 1.Task extraction performance on Bing web search log with.

1.Accuracy of Agree/Disagree relation classification. 2.Accuracy of user opinion prediction. 1.Task extraction performance on Bing web search log with.
Querying for Information Integration: How to go from an Imprecise Intent to a Precise Query? Aditya Telang Sharma Chakravarthy, Chengkai Li.

Querying for Information Integration: How to go from an Imprecise Intent to a Precise Query? Aditya Telang Sharma Chakravarthy, Chengkai Li.
An Unsupervised Framework for Extracting and Normalizing Product Attributes from Multiple Web Sites Center for E-Business Technology Seoul National University.

An Unsupervised Framework for Extracting and Normalizing Product Attributes from Multiple Web Sites Center for E-Business Technology Seoul National University.
Jean-Eudes Ranvier 17/05/2015Planet Data - Madrid Trustworthiness assessment (on web pages) Task 3.3.

Jean-Eudes Ranvier 17/05/2015Planet Data - Madrid Trustworthiness assessment (on web pages) Task 3.3.
GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.

GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.
Dialogue – Driven Intranet Search Suma Adindla School of Computer Science & Electronic Engineering 8th LANGUAGE & COMPUTATION DAY 2009.

Dialogue – Driven Intranet Search Suma Adindla School of Computer Science & Electronic Engineering 8th LANGUAGE & COMPUTATION DAY 2009.
Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)

Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)
Domain-Independent Data Extraction: Person Names Carl Christensen and Deryle Lonsdale Brigham Young University

Domain-Independent Data Extraction: Person Names Carl Christensen and Deryle Lonsdale Brigham Young University
Automatic Discovery and Classification of search interface to the Hidden Web Dean Lee and Richard Sia Dec 2 nd 2003.

Automatic Discovery and Classification of search interface to the Hidden Web Dean Lee and Richard Sia Dec 2 nd 2003.
 Manmatha MetaSearch R. Manmatha, Center for Intelligent Information Retrieval, Computer Science Department, University of Massachusetts, Amherst.

 Manmatha MetaSearch R. Manmatha, Center for Intelligent Information Retrieval, Computer Science Department, University of Massachusetts, Amherst.
Presented by Zeehasham Rasheed

Presented by Zeehasham Rasheed
Information retrieval Finding relevant data using irrelevant keys Example: database of photographic images sorted by number, date. DBMS: Well structured.

Information retrieval Finding relevant data using irrelevant keys Example: database of photographic images sorted by number, date. DBMS: Well structured.
1 PageSim: A Link-based Similarity Measure for the World Wide Web Zhenjiang Lin, Irwin King, and Michael, R., Lyu Computer Science & Engineering, The Chinese.

1 PageSim: A Link-based Similarity Measure for the World Wide Web Zhenjiang Lin, Irwin King, and Michael, R., Lyu Computer Science & Engineering, The Chinese.
Online Learning for Web Query Generation: Finding Documents Matching a Minority Concept on the Web Rayid Ghani Accenture Technology Labs, USA Rosie Jones.

Online Learning for Web Query Generation: Finding Documents Matching a Minority Concept on the Web Rayid Ghani Accenture Technology Labs, USA Rosie Jones.
An investigation of query expansion terms Gheorghe Muresan Rutgers University, School of Communication, Information and Library Science 4 Huntington St.,

An investigation of query expansion terms Gheorghe Muresan Rutgers University, School of Communication, Information and Library Science 4 Huntington St.,
The Relevance Model  A distribution over terms, given information need I, (Lavrenko and Croft 2001). For term r, P(I) can be dropped w/o affecting the.

The Relevance Model  A distribution over terms, given information need I, (Lavrenko and Croft 2001). For term r, P(I) can be dropped w/o affecting the.
Chapter 5: Information Retrieval and Web Search

Chapter 5: Information Retrieval and Web Search
Databases & Data Warehouses Chapter 3 Database Processing.

Databases & Data Warehouses Chapter 3 Database Processing.
Supporting the Automatic Construction of Entity Aware Search Engines Lorenzo Blanco, Valter Crescenzi, Paolo Merialdo, Paolo Papotti Dipartimento di Informatica.

Supporting the Automatic Construction of Entity Aware Search Engines Lorenzo Blanco, Valter Crescenzi, Paolo Merialdo, Paolo Papotti Dipartimento di Informatica.

Similar presentations

Ads by Google