Trains of Thought: Generating Information Maps Dafna Shahaf, Carlos Guestrin and Eric Horvitz

The abundance of books is a distraction ‘‘,, Lucius Annaeus Seneca 4 BC – 65 AD

So, you want to understand a complex topic… Now what?

Search Engines are Great But do not show how it all fits together

Timeline Systems e.g., NewsJunkie [Gabrilovich, Dumais, Horvitz]

Real Stories are not Linear

Metro Map A set of lines Each line follows a coherent narrative thread Structure + multiple aspects austerity bailout junk status Germany protests strike labor unions Merkel

Map Definition A map M is a pair (G,  ) where – G=(V,E) is a directed graph –  is a set of paths in G (metro lines) – Each e  E must belong to at least one metro line austerity bailout junk status protests strike Germany labor unions Merkel

Game Plan

Properties of a Good Map 1. Coherence ???

Greece Europe Italy Republican Protest Coherence: Main Idea Connecting the Dots [S, Guestrin, KDD’10] Debt default Coherence is not a property of local interactions: Incoherent: Each pair shares different words

Greece Austerity Italy Republican Protest Coherence: Main Idea Connecting the Dots [S, Guestrin, KDD’10] Debt default A more-coherent chain: Coherent: a small number of words captures the story

Properties of a Good Map 1. Coherence Is it enough?

Max-coherence Map Query: Clinton Clinton visits Belfast Clinton set for Dublin High hopes for Clinton visit Clinton, Religious Leaders Share Thoughts Church Leaders Praise Clinton's 'Spirituality' Religion Leaders Divided on Clinton Moral Issue Clinton Should Resign, 2 Religious Leaders Say

Properties of a Good Map 1. Coherence 2. Coverage Should cover diverse topics important to the user

Coverage Select a small set of diverse articles that covers the most important stories January 17, 2009 Turning Down the Noise [El-Arini, Veda, S, Guestrin, KDD’09]

Coverage: The Idea Documents cover concepts: Corpus Coverage

High-coverage, Coherent Map Greek Civil Servants Strike over Austerity Measures Greece Paralyzed by New Strike Greek Take to the Streets, but Lacing Earlier Zeal Infighting Adds to Merkel’s Woes It’s Germany that Matters UK Backs Germany’s Effort Germany says the IMF should Rescue Greece IMF more Likely to Lead Efforts IMF is Urged to Move Forward

Properties of a Good Map 1. Coherence 2. Coverage 3. Connectivity

Definition: Connectivity Experimented with formulations Users do not care about connection type Encourage connections between pairs of lines

Tying it all Together: Map Objective Coherence – Either coherent or not: Constraint Coverage – Must have! Connectivity – Nice to have Consider all coherent maps with maximum possible coverage. Find the most connected one.

Game Plan

Approach Overview Documents D … 1. Coherence graph G 2. Coverage function f f( ) = ? 3. Increase Connectivity

Coherence Graph: Main Idea Vertices correspond to short coherent chains Directed edges between chains which can be conjoined and remain coherent

Finding Vertices Vertices are short, coherent chains Can use [KDD’10] – Expensive – Solving many LPs Take advantage of simplicity of short stories – No topic drift – Sampling-based (fast) algorithm

Finding Edges Problem: Combining several strong chains may result in a much-weaker chain Discontinuity: Change of focus Discontinuity: Change of focus

A chain is m-coherent if each sub-chain (di, …, di+m) is coherent. m-Coherence Control discontinuity points: m: size of user's ‘history window‘ – m=length(chain) : standard coherence – m=1: optimize transitions without context

Observation If two chains are m-Coherent and have m-1 overlap, the conjoined chain is m-coherent:

Using the Observation If two chains are m-Coherent and have m-1 overlap, the conjoined chain is m-coherent: Useful for divide and conquer: – Add edge if m-1 overlap

Approach Overview Documents D … 1. Coherence graph G 2. Coverage function f f( ) = ? 3. Increase Connectivity

Finding High-Coverage Chains Paths correspond to coherent chains. Problem: find a path of length K maximizing coverage of underlying articles Cover( ) > Cover( ) ?

Reformulation Paths correspond to coherent chains. Problem: find a path of length K maximizing coverage of underlying articles Submodular orienteering – [Chekuri and Pal, 2005] – Quasipolynomial time recursive greedy – O(log OPT) approximation Orienteering a function of the nodes visited

Approach Overview: Recap Documents D … 1. Coherence graph G 2. Coverage function f f( ) = ? 3. Increase Connectivity Encodes all m-coherent chains as graph paths Submodular orienteering [Chekuri & Pal, 2005] Quasipoly time recursive greedy O(log OPT) approximation Submodular orienteering [Chekuri & Pal, 2005] Quasipoly time recursive greedy O(log OPT) approximation

Example Map: Greece Debt

Game Plan

Evaluation User study – Document selection: capturing important content? – Micro-knowledge: question-answering – Macro-knowledge: high-level summaries – Effect of structure New York Times ( ) – 18K+ articles – Chile, Haiti, Greece

Document Selection Experts compose a list of important events Subtopic recall (% of events in the map): # lines Subtopic recall

Micro-Knowledge (Question Answering) Mechanical Turk Competitors: – Google News – Event threading (TDT) [Nallapati et al, 04] – Structureless maps Results: minor gains – map structure helps Question 2: How many miners were trapped?

Macro-Knowledge (High-Level Summaries) Summarize complex story in a paragraph – Maps vs. Google News – ~15 paragraphs per task Mturk to evaluate paragraphs: – Which paragraph provided a more complete and coherent picture of the story? – Justification: Paragraph A is more… – ~300 evaluations per task

Macro-Knowledge: Results Greece: 72% prefer maps – Justifications: Haiti: 59% prefer maps – Map users mostly summarized one story line MapsGoogle News Bottom line: maps are more useful as high-level tools for stories without a single dominant storyline

Conclusions Formulated metrics characterizing good maps Efficient methods with theoretical guarantees User studies highlight the promise of the method Website on the way! Personalization Thank you!

Finding Coherent Chains Goal: represent all coherent chains Problem: intractable Divide and conquer: – Find short coherent chains – Concatenate to form longer coherent chains

Website