An Adaptive User Profile for Filtering News Based on a User Interest Hierarchy Sarabdeep Singh, Michael Shepherd, Jack Duffy and Carolyn Watters Web Information Filtering Lab Faculty of Computer Science Dalhousie University

Overview News Reading Behaviour Related Research Our Approach Experiments Results Summary

News Reading Behaviour Uses and Gratification –An example of extrinsically motivated behaviour in that there is some reward to be gained by engaging in the activity –Based on the assumption that the reader has some underlying goal, outside the reading itself, that reading the news satisfies. Ludic or Play theory of news reading –An example of intrinsically motivated behavior in that the activity appears to be spontaneously initiated by the person in pursuit of no other goal than the activity itself. –This theory asserts that, “... the process of news reading is intrinsically pleasurable, … a more casual, spontaneous, and unstructured form of news reading.”

Reading News is a Social Phenomenon News has a social and context function in that it provides the information necessary to participate fully as a citizen in the local, national, and international community Several research projects have focused on fine- grained filtering of news articles. Results suggest that personal profiles need to be offset by community interests for ludic news reading behavior.

Knowledge Acquisition and Modeling There are many systems for user modeling and news reading described in the literature The key research issues for modeling for ludic behaviour include: –Implicit or explicit knowledge acquisition –Long term interests and/or short term interests –Drifting interests

Our Approach This research does not filter in the sense that it removes articles Rather it re-ranks the news articles bringing articles “of interest” closer to the front of the queue without eliminating articles that may be, serendipitously, of interest to the user

Research Questions Can we develop a system that learns a user profile? Can the system adapt to changes in the user’s interests?

User’s Interest Hierarchy Profile k, w category 1,1 Category 2,2 Category 2,3 Category 2,1 Category 1,2 Category 1,3 Category 2,4 Category 3,1 Category 3,21 k, w

Bigrams Bigram consists of two words that occur in the same news article A term may be part of many bigrams Strength of relationship between terms of a bigram is based on the Augmented Expected Mutual Information –Prob of both terms occurring in the same news article –Less prob of one term occurring without the other Modified by a specificity function that acts like the inverse document frequency to counter the effect of a term occurring in many news article

T1 T2 T5 T6 T3 T9 T7 T10 T12 T4 T8 T11 Bigram Graph

T1 T2 T5 T6 T3 T9 T7 T10 T12 T4 T8 T11 Removal of Edges weight < 0.35

Profile k, w Topics of Interest category 1,1 Category 2,2 Category 2,3 Category 2,1 Category 1,2 Category 1,3 Category 2,4 Category 3,1 Category 3,21 k, w

Process User evaluates 100 news articles in order to initialize profile 100 news articles Order news articles by profile Explicit feedback from user Update profile User profile

Initialize the Profile Create bigram graph from 100 news articles, with keyword weights = 0. Ask user to rate these news articles as being either “of interest” or “not of interest” Initialize weights of the keywords in the graph based on user evaluations

Adapting the Weights in the Interest Hierarchy For each article, i, in which term k occurs, the weight in the profile associated with k, is modified as follows: where a i is the learning rate associated with article k and is (-0.9, +0.9) is the weight of term k in the profile and is the weight of term k in the term vector representing news article i.

Ordering the News Articles According to the Profile Each leaf category in the profile is represented as a vector of weighted terms. Each news article is represented as weighted term vector where weights are tf.idf The cosine similarity is calculated between an article and every leaf category in the profile. The average of these similarity measures is then taken to be the closeness of that news article to the user’s profile.

Note Profile categories are not developed from individual articles. Rather, they are developed from categories of user interests developed from the bigram graph. As the terms from an article may occur in several different categories, the news articles themselves are not associated with a particular category, but are distributed over multiple categories in the profile.

Updating the Profile Existing User Profile Bigram of newest 100 news articles with user feedback Merge

Merging For each leaf category in the new hierarchy Calculate cosine similarity with each leaf category in existing profile Find profile leaf category with max similarity If max similarity > Threshold Merge new leaf category with profile leaf category Else Create new leaf category in profile with leaf category from new hierarchy Endif Endfor

Experiments 3 users with static user interests Each initialized a profile on those interests Each then iterated through 5 sets of 100 news articles, evaluating based on these static interests Each then created a new set of user interests and iterated through another 5 sets of 100 news articles, evaluating based on the new set of user interests

Processing and Measurement After each set of 100 news articles were evaluated, the Normalized Recall was determined for that set of 100 Normalized Recall measures how close the system was to being perfect, i.e., all the articles “of interest” would be ranked before all the articles evaluated as “not of interest” These 100 news articles and their evaluations were then used to update the user profile

Assume 5 news articles out of 10 are “of interest”

Number Relevant Random R Norm After Training Set Training + set 1 Training + sets 1-2 Training + sets 1-3 Training + sets 1-4 Set Set Set Set Set Normalized Recall – User 1

All Users over all Sets

Summary Results There were significant differences among the users The system did learn for all users, but not equally The system stopped learning after 3 iterations on first set of trials The system did adapt to the changed profiles The system appears to be sensitive to the amount of positive feedback (“of interest”) when learning a new set of interests

Conclusions and Discussion The system did learn the users’ interests and did adapt to changes in interests Although only 3 users, the results are significant for these users as there were 1000 data points for each users Cannot generalize to other users

Future Research Larger study with more users and dynamic news feeds Fine-grained learning rate based on Likert scale of user evaluations Collaborative interest hierarchy