Connecting the Dots Between News Articles Dafna Shahaf and Carlos Guestrin

research papers

Well, we have Google…

Search Limitations InputOutput Interaction New query

Our Approach InputOutput Interaction Phrase complex information needs Structured, annotated output New queryRicher forms of interaction

Connecting the Dots: News Domain

Input: Pick two articles (start, goal) Output: Bridge the gap with a smooth chain of articles Input: Pick two articles (start, goal) Output: Bridge the gap with a smooth chain of articles Input: Pick two articles (start, goal) InputOutput Interaction InputOutput Interaction Bailout Housing Bubble

Keeping Borrowers Afloat A Mortgage Crisis Begins to Spiral,... Investors Grow Wary of Bank's Reliance on Debt Markets Can't Wait for Congress to Act Bailout Plan Wins Approval Housing Bubble Bailout InputOutput Interaction Input: Pick two articles (start, goal) Output: Bridge the gap with a smooth chain of articles Input: Pick two articles (start, goal) Output: Bridge the gap with a smooth chain of articles

Game Plan What is a good chain? Formalize objective Score a chain Find a good chain

What is a Good Chain? What’s wrong with shortest-path? Build a graph – Node for every article – Edges based on similarity Chronological order (DAG) – Run BFS s t

Shortest-path A1: alks Over Ex-Intern's Testimony On Clinton Appear to Bog A2: Judge Sides with the Government in Microsoft Antitrust Trial A3: Who will be the Next Microsoft? – trading at a market capitalization… A4: Palestinians Planning to Offer Bonds on Euro. Markets A5: Clinton Watches as Palestinians Vote to Rescind 1964 Provision A6: ontesting the Vote: The Overview; Gore asks Public For Lewinsky Florida recount Talks Over Ex-Intern's Testimony On Clinton Appear to Bog Down Contesting the Vote: The Overview; Gore asks Public For Patience;

Shortest-path A1: alks Over Ex-Intern's Testimony On Clinton Appear to Bog A2: Judge Sides with the Government in Microsoft Antitrust Trial A3: Who will be the Next Microsoft? – trading at a market capitalization… A4: Palestinians Planning to Offer Bonds on Euro. Markets A5: Clinton Watches as Palestinians Vote to Rescind 1964 Provision A6: ontesting the Vote: The Overview; Gore asks Public For Talks Over Ex-Intern's Testimony On Clinton Appear to Bog Down Contesting the Vote: The Overview; Gore asks Public For Patience;

Shortest-path A1: A2: Judge Sides with the Government in Microsoft Antitrust Trial A3: Who will be the Next Microsoft? – trading at a market capitalization… A4: Palestinians Planning to Offer Bonds on Euro. Markets A5: Clinton Watches as Palestinians Vote to Rescind 1964 Provision A6: Talks Over Ex-Intern's Testimony On Clinton Appear to Bog Down Contesting the Vote: The Overview; Gore asks Public For Patience; Stream of consciousness? - Each transition is strong - No global theme Stream of consciousness? - Each transition is strong - No global theme

More-Coherent Chain Lewinsky Florida recount Talks Over Ex-Intern's Testimony On Clinton Appear to Bog Down Contesting the Vote: The Overview; Gore asks Public For Patience; B1: B2: Clinton Admits Lewinsky Liaison to Jury B3: G.O.P. Vote Counter in House Predicts Impeachment of Clinton B4: Clinton Impeached; He Faces a Senate Trial B5: Clinton’s Acquittal; Senators Talk About Their Votes B6: Aides Say Clinton Is Angered As Gore Tries to Break Away B7: As Election Draws Near, the Race Turns Mean B8:

More-Coherent Chain Talks Over Ex-Intern's Testimony On Clinton Appear to Bog Down Contesting the Vote: The Overview; Gore asks Public For Patience; B1: B2: Clinton Admits Lewinsky Liaison to Jury B3: G.O.P. Vote Counter in House Predicts Impeachment of Clinton B4: Clinton Impeached; He Faces a Senate Trial B5: Clinton’s Acquittal; Senators Talk About Their Votes B6: Aides Say Clinton Is Angered As Gore Tries to Break Away B7: As Election Draws Near, the Race Turns Mean B8: What makes it coherent?

For Shortest Path Chain Topic changes every transition (jittery) Word Patterns

For Coherent Chain Topic consistent over transitions Use this intuition to estimate coherence of chains

What is a Good Chain? Every transition is strong Global theme No jitteriness (back-and-forth) Short (5-6 articles?)

What is a Good Chain? Every transition is strong Global theme No jitteriness (back-and-forth) Short (5-6 articles?)

Strong transitions between consecutive documents d1 w1: Lewinsky w2: Clinton w5: Microsoft w4: Intern w3: Oath min(4,3,1)=1 d2d3 d4

Strong transitions between consecutive documents min(4,3,1)=1 Too coarse – Word importance in transition – Missing words ??? Intuitively, high iff d i and d i+1 very related w plays an important role in the relationship Intuitively, high iff d i and d i+1 very related w plays an important role in the relationship

Influence min(4,3,1)=1 Intuitively, high iff d i and d i+1 very related w plays an important role in the relationship Intuitively, high iff d i and d i+1 very related w plays an important role in the relationship Most methods assume edges – Influence propagates through the edges No edges in our dataset

Computing Influence(d i, d j | w) Clinton Admits Lewinsky Contest the Vote Judge Sides with the Govmnt The Next Microsoft Clinton Judge Microsoft Gore didi didi djdj djdj w w

Computing Influence(d i, d j | w) Clinton Admits Lewinsky Contest the Vote Judge Sides with the Govmnt The Next Microsoft Clinton Judg e Microsof t Gore didi didi djdj djdj w w 1. Run random walks - Random restarts from d i - ε controls expected length 1. Run random walks - Random restarts from d i - ε controls expected length

Computing Influence(d i, d j | w) Clinton Admits Lewinsky Contest the Vote Judge Sides with the Govmnt The Next Microsoft Clinton Judge Microsoft Gore didi didi djdj djdj w w

Computing Influence(d i, d j | w) Clinton Admits Lewinsky Contest the Vote Judge Sides with the Govmnt The Next Microsoft Clinton Judge Microsoft Gore didi didi djdj djdj w w Calculate stationary distribution of dj - Intuitively, high if documents are related Calculate stationary distribution of dj - Intuitively, high if documents are related How important is w? -Check how many walks went through w How important is w? -Check how many walks went through w

Computing Influence(d i, d j | w) Clinton Admits Lewinsky Contest the Vote Judge Sides with the Govmnt The Next Microsoft Clinton Judge Microsoft Gore didi didi djdj djdj w w d j no longer reachable: All influence is due to w d j no longer reachable: All influence is due to w 2. Influence(d i, d j | w) = Stationary distribution(d j ) with w - Stationary distribution(d j ) without w 2. Influence(d i, d j | w) = Stationary distribution(d j ) with w - Stationary distribution(d j ) without w

Influence: Reality Check d i : OJ Simpson trial article – d j : DNA evidence in OJ trial – d j : Super Bowl 49ers

Coherence formulation No edges. Computed using random walks

What is a Good Chain? Every transition is strong Global theme No jitteriness (back-and-forth) Short (5-6 articles?)

Global Theme, No Jitter Jittery chain can score well! But need a lot of words… Good chains can often be represented by a small number of segments

Global Theme, No Jitter Choose 3 segments to be scored on Good score Score = 0

Coherence: New Objective Maximize over legal activations: – Limit total number of active words – Limit number of words per transition – Each word to be activated at most once

Game Plan What is a good chain? Formalize objective Score a chain Find a good chain

Scoring a Chain Problem is NP-Complete Softer notion of activation: [0,1] Natural formalization as a linear program (LP)

Scoring a chain – September 11 th to Daniel Pearl Example Activation levels weighted by influence (rescaled)

Game Plan What is a good chain? Formalize objective Score a chain Find a good chain

Finding a good chain Can’t brute-force – n d possible chains: >>10 20 after pruning Joint LP: optimize activation and chain New variables: – Is document d i a part of the chain? – Does document d j come after d i in the chain? New constraints: – Chain structure – Length = K s23t next(s,3) next(s,2) next(s,t)

Unlike previous LP, we need to round – Extract a chain Approximation guarantees – Chain length K in expectation – Objective: O(sqrt(ln(n/  )) with probability 1-  Rounding s23t

Game Plan What is a good chain? Formalize objective Score a chain Find a good chain How good is it?

Evaluation: Competitors Shortest path Google Timeline Enter a query Pick k equally-spaced articles Event threading (TDT) [Nallapati et al ‘04] Generate cluster graph Representative articles from clusters

Example Chain (1) Simpson Strategy: There Were Several Killers O.J. Simpson's book deal controversy CNN OJ Simpson Trial News: April Transcripts Tandoori murder case a rival for OJ Simpson case Google News Timeline Simpson trial Simpson verdict

Example Chain (2) Issue of Racism Erupts in Simpson Trial Ex-Detective's Tapes Fan Racial Tensions in LA Many Black Officers Say Bias Is Rampant in LA Police Force With Tale of Racism and Error, Lawyers Seek Acquittal Connect-the-Dots Simpson trial Simpson Verdict

Evaluation #1: Familiarity 18 users Show two articles – 5 news stories – Before and after reading the chain Do you know a coherent story linking these articles?

Average fraction of gap closed Better Base familiarity: Effectiveness (improvement in familiarity)

Average fraction of gap closed Better Base familiarity: Effectiveness (improvement in familiarity)

Average fraction of gap closed Better Base familiarity: Effectiveness (improvement in familiarity)

Average fraction of gap closed Better Base familiarity: Effectiveness (improvement in familiarity)

Average fraction of gap closed Base familiarity: Better We are better almost everywhere

Evaluation #2: Chain Quality Compare two chains for – Coherence – Relevance – Redundancy  

Relevance, Coherence, Non- redundancy Across complex stories Better CoherenceRelevanceNon-redundancy

Relevance, Coherence, Non- redundancy Across complex stories Better CoherenceRelevanceNon-redundancy

Relevance, Coherence, Non- redundancy Across complex stories Better CoherenceRelevanceNon-redundancy

Relevance, Coherence, Non- redundancy Across complex stories Better CoherenceRelevanceNon-redundancy

What’s left? Interaction Two documents Chain

Interaction Types 1.Refinement 2.User interests d1 d2 d3d4 ???

Interaction … Defense cross-examines state DNA expert With fiber evidence, prosecution … Simpson Defense Drops DNA Challenge Simpson Verdict … A day the country stood still In the joy of victory, defense team in discord … Many black officers say bias Is rampant in LA police force Racial split at the end … Verdict Race Blood, glove Algorithmic ideas from online learning

Interaction User Study Refinement – 72% prefer chains refined our way User Interests – 63.3% able to identify intruders 2 correct words out of 10

Conclusions Fight information overload – Provide a structured, easy way to navigate topics New task Explored desired properties – LP formalization – Efficient algorithm Evaluated over real news data – Demonstrate effectiveness – Interaction Complex information needs Structured, annotated output

Scaling Up LP has variables Polynomial, but D is large – Restricting number of documents – Sparsifying the graph Random walks