1. Evidence from Behavior
INST 734
Doug Oard
Module 7 1

2. Agenda
Explicit feedback
Implicit Feedback
Link analysis
Clickstreams

3. Recommender Systems
Source: Jon Herlocker, SIGIR 1999

4. Relevance Feedback x x x x o x x x x x x x o x o x x o x o o x x x x
Initial query x x x x o x x x  x x x x o x o x  x o x o o x x x x
Revised query
qm = modified query vector
q0 = original query vector
α,β,γ: weights (hand-chosen or set empirically)
Dr = set of known relevant doc vectors
Dnr = set of known irrelevant doc vectors
x non-relevant documents
o relevant documents

5. Rocchio Example (+) (-) 1 1 1 1 2 16 8 8 2 1 8 4 4 1 8 16 4 16 12 2 4
Typically,  < 
Computer
Apollo
Program
Theater
NASA
Navy
Original query 1 1 1 1
(+)
Positive Feedback 2 16 8 8 2 1 8 4 4 1
(-)
Negative feedback 8 16 4 16 12 2 4 1 4 3
New query -1 3 3 1 -2 16

6. Relevance Feedback Assumptions
The initial query was reasonable
The positive examples are representative
The user will give feedback

7. A1: Good Initial Query? Requires finding something on the first try!
Problems:
Knowledge gaps (“Anomalous state of knowledge”)
Vocabulary gaps (active vs. passive vocabulary)
Misspelling (e.g., Brittany Speers)

8. A2: Representative Examples?
Some relevant documents may be dissimilar
Examples:
Medical documents written for doctors or patients
Policy documents from different organizations
Documents written in different dialects

9. A3: Will People Use It? Efficiency Understandability Risk
Longer queries require more processing time
Understandability
Harder to see why subsequent documents retrieved
Risk
Users are reluctant to provide negative feedback

10. “Blind” Relevance Feedback
Goal: include terms user could have chosen
By choosing terms related to those they did choose
Three-stage process
Perform an initial search
Select new terms from the top results
e.g., highest “offer weight” terms in ≥ 2 top results
Expand (and reweight) the query
Improves MAP for about 2/3 of the queries
But increases the variance in AP

11. Agenda
Explicit feedback
Implicit Feedback
Link analysis
Clickstreams