DEANNA: Natural Language Questions for the Web of Data Mohamed Yahya † Klaus Berberich † Shady Elbassiousni * Maya Ramanath ‡ Volker Tresp # Gerhard Weikum † † Max Planck Institute for Informatics * Qatar Computing Research Institute ‡ IIT Delhi # Siemens

“Who played in Casablanca and was married to a writer born in Rome?” “Classical” QA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.2

“Who played in Casablanca and was married to a writer born in Rome?” “Classical” QA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.3 Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant.

“Who played in Casablanca and was married to a writer born in Rome?” “Classical” QA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.4 Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant. In 1950, after a decade of stardom in American films, she starred in the Italian film Stromboli, which led to a love affair with director Roberto Rossellini while they were both already married. The affair and then marriage with Rossellini created a scandal that forced her to remain in Europe until 1956, when she made a successful Hollywood return in Anastasia, for which she won her second Academy Award, as well as the forgiveness of her fans. Many of her personal and film documents can be seen in the Wesleyan University Cinema Archives. [4] [4]

“Who played in Casablanca and was married to a writer born in Rome?” “Classical” QA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.5 Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant. In 1950, after a decade of stardom in American films, she starred in the Italian film Stromboli, which led to a love affair with director Roberto Rossellini while they were both already married. The affair and then marriage with Rossellini created a scandal that forced her to remain in Europe until 1956, when she made a successful Hollywood return in Anastasia, for which she won her second Academy Award, as well as the forgiveness of her fans. Many of her personal and film documents can be seen in the Wesleyan University Cinema Archives. [4] [4] Roberto Gastone Zeffiro Rossellini (8 May 1906 – 3 June 1977) was an Italian film director and screenwriter. Rossellini was one of the directors of the Italian neorealist cinema, contributing films such as Roma città aperta (Rome, Open City 1945) to the movement.

“Who played in Casablanca and was married to a writer born in Rome?” “Classical” QA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.6 Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant. In 1950, after a decade of stardom in American films, she starred in the Italian film Stromboli, which led to a love affair with director Roberto Rossellini while they were both already married. The affair and then marriage with Rossellini created a scandal that forced her to remain in Europe until 1956, when she made a successful Hollywood return in Anastasia, for which she won her second Academy Award, as well as the forgiveness of her fans. Many of her personal and film documents can be seen in the Wesleyan University Cinema Archives. [4] [4] Roberto Gastone Zeffiro Rossellini (8 May 1906 – 3 June 1977) was an Italian film director and screenwriter. Rossellini was one of the directors of the Italian neorealist cinema, contributing films such as Roma città aperta (Rome, Open City 1945) to the movement. Rossellini was born in Rome. His mother, Elettra (née Bellan), was a housewife, and his father, Angiolo Giuseppe "Beppino" Rossellini, owned a construction firm. His mother was of part French descent, from immigrants who had arrived in Italy during the Napoleonic Wars. He lived on the Via Ludovisi, whereBenito Mussolini had his first Roman hotel in 1922 when Fascism obtained power in Italy.

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 7

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 8

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 9

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 10 S ubject P redicate O bject Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman WordNet +

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 11 S ubject P redicate O bject Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman WordNet + YAGO2: 120 million facts on 10 million entities

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 12 S ubject P redicate O bject Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman WordNet + YAGO2: 120 million facts on 10 million entities

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 13 S ubject P redicate O bject Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman WordNet + YAGO2: 120 million facts on 10 million entities

QA, meet the (semantic) Web of Data EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. 14 S ubject P redicate O bject Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman WordNet + YAGO2: 120 million facts on 10 million entities LOD: > 31 BILLION triples

Semantics Types EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.15 U.S. Cities ITS LARGEST AIRPORT WAS NAMED FOR A WORLD WAR II HERO; ITS SECOND LARGEST, FOR A WORLD WAR II BATTLE What is Toronto???

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.16

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.17 SELECT ?p WHERE{ ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } SPARQL

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.18 SELECT ?p WHERE{ ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } SPARQL Projection Variable

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” Dot: conjunction EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.19 SELECT ?p WHERE{ ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } SPARQL Projection Variable

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” Dot: conjunction ?p, ?w: variables to be bound EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.20 SELECT ?p WHERE{ ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } SPARQL Projection Variable

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” Dot: conjunction ?p, ?w: variables to be bound Same variable  same binding (join) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.21 SELECT ?p WHERE{ ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } SPARQL Projection Variable

Crash Course: Querying the Web of Data “Who played in Casablanca and was married to a writer born in Rome?” Dot: conjunction ?p, ?w: variables to be bound Same variable  same binding (join) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.22 SELECT ?p WHERE{ ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } SPARQL ?p ?w Projection Variable

What is DEANNA? EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. DEANNA Question SPARQL KB Answers 23

What is DEANNA? EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 24

What is DEANNA? EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome. “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 25

What is DEANNA? EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome. “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 26

What is DEANNA? EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome. “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 27

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.28 DEANNA Question SPARQL KB Answers

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.29 DEANNA Question SPARQL KB Answers Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.30 DEANNA Question SPARQL KB Answers Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.31 DEANNA Question SPARQL KB Answers Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.32 DEANNA Question SPARQL KB Answers Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.33 DEANNA Question SPARQL KB Answers Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Inside DEANNA EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.34 DEANNA Question SPARQL KB Answers Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Structured Query Generation EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.35 SELECT ?p WHERE { ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome } q1q1 q2q2 q3q3 a writer Casablanca played in Who was married to was born Rome c:writer r:bornInPlace e:Rome e:Casablanca_(film) r:actedIn c:person r:isMarriedTo

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.36 5 Experiments & Results Disambiguation graph construction Disambiguation graph processing DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.37 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Phrase Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.38 Concepts: entities & classes: Dictionary-based Relations: Mainly use Reverb [Fader et al. EMNLP’11]: V | VP | VW*P … was/VBD married/VBN to/TO a/DT… ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca … …

Phrase Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.39 Casablanca played played in Who married married to was married to a writer Concepts: entities & classes: Dictionary-based Relations: Mainly use Reverb [Fader et al. EMNLP’11]: V | VP | VW*P … was/VBD married/VBN to/TO a/DT… ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca … …

Phrase Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.40 Casablanca played played in Who married married to was married to a writer Concepts: entities & classes: Dictionary-based Relations: Mainly use Reverb [Fader et al. EMNLP’11]: V | VP | VW*P … was/VBD married/VBN to/TO a/DT… ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca … …

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.41 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.42 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Phrase Mapping EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. Casablanca played played in 43

Phrase Mapping EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. Casablanca played played in 44 Concepts: entities & classes: Dictionary-based ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca Played_(film) Played

Phrase Mapping EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. Casablanca played played in e:White_House e:Casablanca e:Casablanca_(film) e:Played_(film) 45 Concepts: entities & classes: Dictionary-based ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca Played_(film) Played

Phrase Mapping EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. Casablanca played played in e:White_House e:Casablanca e:Casablanca_(film) e:Played_(film) 46 Concepts: entities & classes: Dictionary-based Relations: Dictionary -based ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca Played_(film) Played RelationPhrase actedIn acted in actedIn played in hasMusicalRole plays hasMusicalRole mastered

Phrase Mapping EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. Casablanca played played in e:White_House e:Casablanca e:Casablanca_(film) e:Played_(film) r:actedIn r:hasMusicalRole 47 Concepts: entities & classes: Dictionary-based Relations: Dictionary -based ConceptPhrase Casablanca Casablanca, Morocco Casablanca_(film) Casablanca the film Casablanca_(film) Casablanca Played_(film) Played RelationPhrase actedIn acted in actedIn played in hasMusicalRole plays hasMusicalRole mastered

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.48 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.49 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Dependency Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.50 Look for specific patterns in dependency parses [de Marneffe et al. LREC’06] a writer was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome

Dependency Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.51 Look for specific patterns in dependency parses [de Marneffe et al. LREC’06] a writer was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome

Dependency Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.52 Look for specific patterns in dependency parses [de Marneffe et al. LREC’06] a writer was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome

Dependency Detection EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.53 Look for specific patterns in dependency parses [de Marneffe et al. LREC’06] q1q1 a writer was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome

RESULT EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.54

Disambiguation Graph EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.55 q1q1 q2q2 q3q3 a writer Casablanca played played in Who married married to was married to was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome e:White_House e:Casablanca e:Casablanca_(film) e:Played_(film) r:actedIn r:hasMusicalRole c:person e:Married_(series) c: married_person r:isMarriedTo q-nodes Phrase-nodes Semantic nodes

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.56 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.57 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.58 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Joint Disambiguation - ILP ILP: Integer Linear Programming maximize α Σ i,j w i,j Y i,j + β Σ k,l v k,l Z k,l + … Subject to:  No token in multiple phrases,  Triples observe type constraints, … EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.59

Joint Disambiguation – Objective α Σ i,j w i,j Y i,j + β Σ k,l v k,l Z k,l EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.60 Semantic nodes q1q1 a writer was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome q-nodes Phrase nodes Coherence Edges Similarity Edges Prior

Joint Disambiguation – Objective α Σ i,j w i,j Y i,j + β Σ k,l v k,l Z k,l EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.61 Semantic nodes Coherence q1q1 a writer was born born Rome c:writer r:bornInPlace r:bornOnDate e:Max_Born e:Born_(film) e:Sydne_Rome e:Rome q-nodes Phrase nodes Similarity Edges Coherence Edges

Joint Disambiguation – Constraints A phrase node can be assigned to one semantic node: EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.62 Casablanca e:White_House e:Casablanca e:Casablanca_(film) Phrase nodes Semantic nodes a Y a,1 Y a,2 Y a,3 α Σ i,j w i,j Y i,j + β Σ k,l v k,l Z k,l

Joint Disambiguation – Constraints A phrase node can be assigned to one semantic node: EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.63 Casablanca e:White_House e:Casablanca e:Casablanca_(film) Phrase nodes Semantic nodes a Y a,1 Y a,2 Y a,3 α Σ i,j w i,j Y i,j + β Σ k,l v k,l Z k,l

Joint Disambiguation – Constraints Classes translate to type-constrained variables  Every semantic triple should have a class to join & project! person actedIn Casablanca_(film) ▼ ?x type person. ?x actedIn Casablanca_(film) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.64

Joint Disambiguation – Constraints Classes translate to type-constrained variables  Every semantic triple should have a class to join & project! person actedIn Casablanca_(film) ▼ ?x type person. ?x actedIn Casablanca_(film) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.65

Joint Disambiguation – Constraints Classes translate to type-constrained variables  Every semantic triple should have a class to join & project! person actedIn Casablanca_(film) ▼ ?x type person. ?x actedIn Casablanca_(film) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.66 q1q1 a writer was born Rome c:writer r:bornInPlace r:bornOnDate e:Sydne_Rome e:Rome q-nodes e:The_Writer (magazine) Phrase nodes Semantic nodes

Joint Disambiguation – Constraints Classes translate to type-constrained variables  Every semantic triple should have a class to join & project! person actedIn Casablanca_(film) ▼ ?x type person. ?x actedIn Casablanca_(film) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.67 q1q1 a writer was born Rome c:writer r:bornInPlace r:bornOnDate e:Sydne_Rome e:Rome q-nodes e:The_Writer (magazine) Phrase nodes Semantic nodes

Joint Disambiguation – Constraints Classes translate to type-constrained variables  Every semantic triple should have a class to join & project! person actedIn Casablanca_(film) ▼ ?x type person. ?x actedIn Casablanca_(film) EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.68 q1q1 a writer was born Rome c:writer r:bornInPlace r:bornOnDate e:Sydne_Rome e:Rome q-nodes e:The_Writer (magazine) Phrase nodes Semantic nodes

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.69 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.70 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Structured Query Generation EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.71 SELECT ?p WHERE { ?w type writer. ?w bornIn Rome. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w } q1q1 q2q2 q3q3 a writer Casablanca played in Who was married to was born Rome c:writer r:bornIn e:Rome e:Casablanca_(film) r:actedIn c:person r:isMarriedTo

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.72 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Outline EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.73 5 Experiments & Results DEANNA Question SPARQL KB Phrase detection Phrase mapping Phrase mapping Dependency detection Dependency detection Joint Disambig. Joint Disambig. Query Generation Query Generation

Evaluation Methodology 3-stage evaluation for more insight: 1.Disambiguation 2.Query generation 3.Query answering We rely on human judges EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.74

Datasets QALD-1  YAGO2  27/50 questions within scope  “Which software has been published by Mean Hamster Software?” NAGA [Elbassuoni et al. CIKM’09]  YAGO+IMDB  44/87 questions within scope  “Which director has won the Academy Award for Best director and is married to an actress that has won the Academy Award for Best Actress?” EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.75

Results 1/3: Disambiguation EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.76 BenchmarkQALD-1NAGA cov macro prec macro cov micro prec micro cov = correct/ideal prec = correct / retrieved Casablanca e:White_House e:Casablanca e:Casablanca_(film)

Results 2/3: Query Generation EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.77 BenchmarkQALD-1NAGA cov macro prec macro cov micro prec micro cov = #correct triples/#ideal triples prec = #correct triples/#retrieved triples Issues: Incorrect disambiguation Incorrect dependencies.

Results 3/3: Answering EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.78 BenchmarkQALD-1NAGA #questions 2744 #queries 2041 #satisfactory 1015 #relaxed +3

Results 3/3: Answering EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.79 BenchmarkQALD-1NAGA #questions 2744 #queries 2041 #satisfactory 1015 #relaxed +3 unsatisfactory: main problem is empty result set

Results 3/3: Answering EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.80 BenchmarkQALD-1NAGA #questions 2744 #queries 2041 #satisfactory 1015 #relaxed +3 unsatisfactory: main problem is empty result set Relaxation: Keep type constraints, everything else to keywords ?x type writer. ?x bornIn Rome ▼ ?x type writer. ?x bornIn ?y [“Rome”]

Wrap-up EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. DEANNA Question SPARQL KB Answers 81

Wrap-up EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 82

Wrap-up EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 83

Wrap-up EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 84

Wrap-up EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 85 Rome isA city city subclassOf location Rossellini mariedTo Bergman

Wrap-up EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al. ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer. ?w bornIn Rome “Who played in Casablanca and was married to a writer born in Rome?” DEANNA Question SPARQL KB Answers 86 Rome isA city city subclassOf location Rossellini mariedTo Bergman

QUESTIONS/COMMENTS? Thank you. EMNLP July 12, 2012 Natural Language Questions for the Web of Data - Yahya et al.87 bit.ly/mpi-deanna