Crawling and Aligning Scholarly Presentations and Documents from the Web By SARAVANAN.S 09/09/2011 Under the guidance of A/P Min-Yen Kan 10/23/2015 1

Motivation More articles  more users Searching for documents is difficult Aim: Find pairs of presentations and documents automatically 10/23/2015 2

System Architecture Search Engine Wrapper Query “File type”( PDF, PPT or PS) operator is added with the user query Before sending it to Google. Re-Ranking Top results of Google Output:(3-way) 1.Exact URL 2.Message for No-free files 3.No result Google Used Yee Fan’s Search Engine Wrapper – just Google subsystem 10/23/2015 3

Methodology (1) Re-Ranking Computed similarity between user query and documents retrieved for re-ranking. Methods used for computing similarity are Jaccard co- efficient, Bilingual Evaluation Understudy (BLEU). Threshold value is used to restrict the system from considering low similarity scored documents. Google’s Top Results Similarity Score Computation Re-Ranking Results Based on Similarity Score Similarity is computer between Query Title and each Google’s result Title, Snippet, URL. 10/23/2015 4

Jaccard Measure Jaccard measure is used to compute similarity between Query Title and Google’s result Title, Snippet, and URL. Simple word by word matching. Problems are: Snippets have more words than title. Union in Jaccard increases while intersection remains same. Sentence1: Finding related pages in the world wide web. Sentence2: Finding Related pages using the Link structure of the WWW. 10/23/2015 5

BLEU metric Why BLEU?? n-gram similarity of words. Helps in accessing the sequential order of the words when finding similarity between two sets. Sequential order of words matters with snippet  query terms may appear in a random position. 10/23/2015 6

Rules Special rules are used for better matching: Rule1: Removing special symbols. (On/Off) Rule2: Stop-words removal (On/Off) Rule3: URL filter by.edu (On/Off) Rule4: Stemming (Porter stemming algorithm) (On/Off) All these rules are used with both the methodologies. 10/23/2015 7

Methodology MIME-types: To differentiate free PDF from subscription type, I used the MIME-types. It returns the content-type of the URL. Dataset collection: Queries from, Computer science. Medical science. Architecture. Mathematics. 10/23/2015 8

Experiment Experiments on – Jaccard Measure.(All special rules are tested with On/Off). – BLEU measure (All special rules are tested with On/Off). – Query set with about 50 queries. – Threshold is set from 0.1 to 1.0 range for all experiments. – Highest recall with high threshold is considered. Experiment results – Jaccard similarity. – BLEU similarity. 10/23/2015 9

Experiment result of Jaccard RuleGoogle Target ThresholdPrecisionRecallF-Score 1234 OnOff Title OffOnOff Title Off OnOffTitle Off OnTitle On Off Title OffOn OffTitle Off On Title OnOff OnTitle On OffTitle OffOn Title OnOffOn Title On OffOnTitle On Title Off Title OnOffOnOffTitle OffOnOffOnTitle Best F-score achieved 10/23/

Best F- score achieved Experiment result of BLEU 10/23/ RuleGoogle Target ThresholdPrecisionRecallF-Score OnOff Title OffOnOff Title Off OnOffTitle Off OnTitle On Off Title OffOn OffTitle Off On Title OnOff OnTitle On OffTitle OffOn Title OnOffOn Title On OffOnTitle On Title Off Title OnOffOnOffTitle OffOnOffOnTitle

Related Work Base Reference: – SlideSeer: a digital library of aligned document and presentation pairs, [Kan, JCDL’07]. – Learning to Rank for Information Retrieval. [Liu et al., WWW’09]. – Kairos: Proactive Harvesting of Research Paper Metadata from Scientific Conference Web Sites.[Hänse, ICADL’09] Approaches to Similarity Computation – BLEU: a Method for Automatic Evaluation of Machine Translation. [Papineni et al., ACL July’02]. – BLEU algorithm for evaluation machine translations implementation.[Payson et al.] 10/23/

Conclusion Matching documents based on similarity score Jaccard measure -- Jaccard similarity computed over Query title and Document title with rule special symbol removed retrieves best articles. -- Threshold: F-score: BLEU metric -- BLEU similarity computed over Query title and Document title with rule special symbol removed retrieves best articles. -- Threshold F-score: /23/

Thank you Comments are welcome 10/23/