Download presentation
Presentation is loading. Please wait.
Published byBennett Homer Stanley Modified over 9 years ago
1
Personalizing Web Search Jaime Teevan, MIT with Susan T. Dumais and Eric Horvitz, MSR
3
Demo
4
Personalizing Web Search Motivation Algorithms Results Future Work
5
Personalizing Web Search Motivation Algorithms Results Future Work
6
Study of Personal Relevancy 15 SIS users x ~10 queries Evaluate 50 results Highly relevant / Relevant / Irrelevant Query selection Previously issued query Chose from 10 pre-selected queries Collected evaluations for 137 queries 53 of pre-selected queries (2-9/query)
7
Relevant Results Have Low Rank Highly Relevant Relevant Irrelevant
8
Same Query, Different Intent Different meanings “Information about the astronomical/astrological sign of cancer” “information about cancer treatments” Different intents “is there any new tests for cancer?” “information about cancer treatments”
9
Same Intent, Different Evaluation Query: Microsoft “information about microsoft, the company” “Things related to the Microsoft corporation” “Information on Microsoft Corp” 31/50 rated as not irrelevant Only 6/31 do more than one agree All three agree only for www.microsoft.com
10
More to Understand Do people cluster? Even if they can’t state their intention How are the differences reflected? Can they be seen from the information on a person’s computer? Can we do better than the ranking that would make everyone the most happy? Best common ranking: +38% Best personalized ranking: +55%
11
Personalizing Web Search Motivation Algorithms Results Future Work
12
Personalization Algorithms Standard IR Related to relevance feedback Query expansion Document Query User Server Client v. Result re-ranking
13
Result Re-Ranking Takes full advantage of SIS Ensures privacy Good evaluation framework Look at light weight user models Collected on server side Sent as query expansion
14
BM25 N nini NniNni w i = log riri R with Relevance Feedback Score = Σ tf i * w i
15
BM25 with Relevance Feedback N nini (r i +0.5)(N-n i -R+r i +0.5) (n i -r i +0.5)(R-r i +0.5) riri R w i = log Score = Σ tf i * w i
16
(r i +0.5)(N-n i -R+r i +0.5) (n i - r i +0.5)(R-r i +0.5) User Model as Relevance Feedback N nini R riri Score = Σ tf i * w i (r i +0.5)(N’-n i ’-R+r i +0.5) (n i ’- r i +0.5)(R-r i +0.5) w i = log N’ = N+R n i ’ = n i +ri
17
User Model as Relevance Feedback N nini R riri World User Score = Σ tf i * w i
18
User Model as Relevance Feedback R riri User N nini World World related to query N nini Score = Σ tf i * w i
19
User Model as Relevance Feedback N nini R riri World User World related to query User related to query R N nini riri Query Focused Matching Score = Σ tf i * w i
20
User Model as Relevance Feedback N nini R riri World User Web related to query User related to query R N riri Query Focused Matching nini World Focused Matching Score = Σ tf i * w i
21
Parameters Matching User representation World representation Query expansion
22
Parameters Matching User representation World representation Query expansion Query focused World focused
23
Parameters Matching User representation World representation Query expansion Query focused World focused
24
User Representation Stuff I’ve Seen (SIS) index Recently indexed documents Web documents in SIS index Query history Relevance judgments None
25
Parameters Matching User representation World representation Query expansion Query focused World focused All SIS Recent SIS Web SIS Query history Relevance feedback None
26
Parameters Matching User representation World representation Query expansion Query Focused World Focused All SIS Recent SIS Web SIS Query History Relevance Feedback None
27
World Representation Document Representation Full text Title and snippet Corpus Representation Web Result set – title and snippet Result set – full text
28
Parameters Matching User representation World representation Query expansion Query focused World focused All SIS Recent SIS Web SIS Query history Relevance feedback None Full text Title and snippet Web Result set – full text Result set – title and snippet
29
Parameters Matching User representation World representation Query expansion Query focused World focused All SIS Recent SIS Web SIS Query history Relevance feedback None Full text Title and snippet Web Result set – full text Result set – title and snippet
30
Query Expansion All words in document Query focused The American Cancer Society is dedicated to eliminating cancer as a major health problem by preventing cancer, saving lives, and diminishing suffering through...
31
Parameters Matching User representation World representation Query expansion Query focused World focused All SIS Recent SIS Web SIS Query history Relevance feedback None Full text Title and snippet Web Result set – full text Result set – title and snippet All words Query focused
32
Parameters Matching User representation World representation Query expansion Query focused World focused All SIS Recent SIS Web SIS Query history Relevance feedback None Full text Title and snippet Web Result set – full text Result set – title and snippet All words Query focused
33
Parameters Matching User representation World representation Query expansion Query focused World focused All SIS Recent SIS Web SIS Query history Relevance feedback None Full text Title and snippet Web Result set – full text Result set – title and snippet All words Query focused
34
Personalizing Web Search Motivation Algorithms Results Future Work
35
Baselines Best possible Random Text based ranking Web ranking URL Boost http://mail.yahoo.com/inbox/msg10 +1
36
Best Parameter Settings Richer user representation better SIS > Recent > Web > Query History > None Suggests rich client important Efficiency hacks don’t hurt Snippets query focused Length normalization not an issue Query focus good
37
Text Alone Not Enough Better than some baselines Better than random Better than no user representation Better than relevance feedback Worse than Web results Blend in other features Web ranking URL boost
38
Good, but Lots of Room to Grow Best combination: 9.1% improvement Best possible: 51.5% improvement Assumes best Web combination selected Only improves results 2/3 of the time
39
Personalizing Web Search Motivation Algorithms Results Future Work
40
Finding the Best Parameter Setting Almost always some parameter setting that improves results Use learning to select parameters Based on individual Based on query Based on results Give user control?
41
Further Exploration of Algorithms Larger parameter space to explore More complex user model subsets Different parsing (e.g., phrases) Tune BM25 parameters What is really helping? Generic user model or personal model Use different indices for the queries Deploy system
42
Practical Issues Efficiency issues Can interfaces mitigate some of the issues? Merging server and client Query expansion Get more relevant results in the set to be re-ranked Design snippets for personalization
43
Thank you!
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.