BuzzTrack Topic Detection and Tracking in IUI – Intelligent User Interfaces January 2007 Keno Albrecht ETH Zurich Roger Wattenhofer ETH Zurich Gabor Cselle Google

2 Overload clients were not designed to handle volume and variety of messages users are dealing with today: Large volumes of Task Management Personal Archiving or Filing Keeping Context [Whittaker and Sidner, 1996]

3 Search vs. Inbox Browsing Fast full-text search is today's solution to finding past s. But the flat inbox view of newly incoming s hasn’t changed. In our work, we focus on the problem of sensibly structuring s in the inbox.

4 Today's Clients: The Three-Pane View No sense of context: unrelated messages are shown together Important s may drop off the “first screen” “Thread-based” tree views are unsophisticated, may not pull in all relevant messages.

5 BuzzTrack client extension for Mozilla Thunderbird for displaying grouped by topic.

6 Related Work

7 Visualizations: Conversations Gmail (Google) common conversation title one entry per , folds out on click

8 Automatic Foldering Using machine learning techniques to automatically move s into folders upon arrival Low accuracy rates [Bekkerman et al, 2005], conceptual problems: Users need to manually create folders and seed them with data.

9 People-Centered Clients Bifrost ContactMap [Bälter and Sidner, 2002] [Whittaker et al., 2004]

10 Task-based Example: TaskMaster thrasks thrask contents item contents ( s, documents, etc.) TaskMaster [Belotti et al., 2003]

11 BuzzTrack

12 BuzzTrack Mozilla Thunderbird extension to automatically group related s into topics. Will be distributed through website: Provides a view on the user’s inbox.

13 What’s a Topic? Topics are groups of s that relate to the same idea, action, event, task, or question. Examples: A conversation about buying a digital camera. Referring a candidate for a job. All s belonging to same newsgroup.

14 Clustering Process For every new incoming PreprocessingClustering Label generation Cluster store BuzzTrack View in Thunderbird

15 Preprocessing Tokenization (remove HTML tags, style sheets, punctuation, and numbers) Language detection Stemming For topic labelling: Identify Parts-of-speech Remember popular original word forms

16 Clustering Single-link clustering: Newly incoming s are compared to every in existing topics: Similarity value > threshold: assigned to topic Similarity value <= threshold: starts new topic

17 Features - 1 How do we generate similarity values between s? Via a linear combination of several similarity features. Examples: Text similarity (TFIDF Value, cosine similarity metric) People similarities (comparing sets of people in the From / To / Cc lines of headers) Thread membership

18 Features - 2 Other features for deriving similarities: Subject similarity Sender domain overlaps Sender rank and percentage % of from sender that is answered Time passed since last in topic People and reference count for Known people and reference % Cluster size Has attachment

19 Decision Score Similarities are combined into a decision score for each / cluster pair through a linear combination of feature values: dec i,j = w a *sim a (mi,Cj) + w b *sim b (mi,Cj) + … We tested two sets of weights w x, both trained on a development set of s: Empirical Linear SVM

20 Evaluation How do we evaluate clustering quality? Topic Detection and Tracking competitions by NIST. Aimed at clustering news articles. Corpus:

21 Clustering Tasks Clustering Task is split into subtasks: New Topic Detection (NTD): Given stream of s, which ones start new topics? Topic Tracking (TT): Given a fixed topic, which newly incoming s belong to it? DET Curves plot miss rate vs. false alarm rate for possible threshold for decision scores

22 Results NTD TDT New Topic Detection Task Miss: 3% False alarm: 30% better

23 Results TT TDT Topic Tracking Task Miss: 8% False alarm: 2% better

24 Comparison Comparable quality to TDT for news articles [NIST 2004] News has less metadata, has worse text quality. Wide body of work exists on improving clustering performance on news, we haven’t tapped into that yet.

25 BuzzTrack View Mozilla Thunderbird plugin that provides useful view on inbox data “for free” Topics contain from last 60 days We’re interested in current only Reduces initial clustering time Each is shown in one topic

26

27 Demo 1: BuzzTrack

28 BuzzTrack Panes Topic pane: Provides additional info Starred topics pane: Topics sorted by last incoming

29 Future Work Distribute plugin to Thunderbird users Input on possible UI improvements Input on clustering quality Different clustering styles People-based Thread-based We hope BuzzTrack will be valuable tool for real-world users

30 Questions? Contact: Gabor Cselle, Website: