1. CiteSight: Contextual Citation Recommendation with Differential Search Avishay Livne 1, Vivek Gokuladas 2, Jaime Teevan 3, Susan Dumais 3, Eytan Adar 1 1 University of Michigan, 2 Qualcom, 3 Microsoft

2. #SIGIR18 #JaimesBackyard

3. CiteSight: Contextual Citation Recommendation with Differential Search Avishay Livne 1, Vivek Gokuladas 2, Jaime Teevan 3, Susan Dumais 3, Eytan Adar 1 1 University of Michigan, 2 Qualcom, 3 Microsoft

4. Search Engines Focus on Speed

5. Why Do We Cite? Paying homage to pioneers Giving credit for related work Identifying methodology Providing background Correcting one’s work Correcting the work of others Substantiating claims … [Garfield, 1965]

6. How Do We Cite? Many resources –Search engines –Bibliographic tools –Colleagues Work practice –Papers we know –Papers we should know

7. Why × How = 2 Specs Spec 1 –I know what I want, give it to me now –Citation context: “… calculating the differences between blocks of text [“ Spec 2 –I don’t know or can’t remember what I want [cite] Complex, dynamic search space = slow –Inherent trade-off Can we build a system to support both?

9. The CiteSight User Interface

10. Split World Into Two Stuff I don’t know about Stuff I want fast = stuff I know about Microsoft Academic

11. Strategy Small, personalized index –Updated dynamically What you’ve cited before What you’ve cited now What other people have cited –Venue, co-citation, etc. Run a big index for everything else

12. Ranking Query: Citation context –“… calculating the differences between blocks of text [“ Dynamic recommendations –Immediately: Search the cache –In the background: Search the full index Rank retrieved papers: –Gradient boosted regression tree –Features: network + text Popularity, author similarity, textual similarity,…

13. Citation Context Citation context is really good at picking out “winners” People talk about a paper the same way as you! Not the same way the author talks about their work Paper text Bob et al. introduced ABC in […] XYZ is similar to ABC […] We utilize ABC to…[…]

14. That’s nice… (S. Redner, 1998) Citations

15. Context Coupling Popular paperLess-popular paper AB A and B related –Co-cited: When B is mentioned, A is “Borrow” contexts from A to B Borrowed context used as a feature in ranking papers

16. CiteSight Evaluation Can CiteSight predict existing citations? –1000 randomly selected CS papers (2011) Criteria: 20-40 citations –5-fold cross validation –Metric: NDCG Gain of 1 when guesses correct citation Gain related to # of co-citations for close guesses User feedback from 5 CS grad students

17. Results Large improvement –Context coupling –All features FeaturesNDCG@10 Text only40.8% Context coupling46.5% All features61.9%

18. Results Large improvement –Context coupling –All features –Citation-related features > text More info = better –Authors –Citations, to a point FeaturesNDCG@10 Text only40.8% Context coupling46.5% All features61.9% + keywords46.5% + title46.6% + authors similarity47.5% + abstract47.8% + citation count48.6% + venue relevancy49.2% + citations53.0% + co-citations56.7% + authors history57.6%

19. Cache v. Corpus Relevance –Cache accounts for 46% of NDCG@10 of the corpus –10% cache is better Speed –Cache: 6 ms Instantaneous! –Corpus: 450 ms

20. Summary Differential need for speed CiteSight – differential search –Two different use cases = two indices 1.Local index updated dynamically, contextually 2.Global index with full content –Context coupling improves relevance –Local index improves speed Able to provide instantaneous results Often relevant because contextually updated

21. Questions?