1. iNAGO Project
Automatic Knowledge Base Generation from Text for Interactive Question Answering

2. Brief Introduction In collaboration with iNAGO Inc. YorkU Team
Elnaz Delpisheh (Post Doc)
Heidar Davoudi (Ph.D.)
Emad Gohari (Masters)

3. Automatic Q/A generation
iNago Project
Automatic Q/A generation

4. Steps and timeline Sentence Simplification Named Entity Information
Semantic Role Labeling
Generate Questions and Answers
Importance of Generated Questions
Context issues
Human Evaluations

5. Sentence Simplification
Sentences may have complex grammatical structure with multiple embedded clauses.
We simplify the complex sentences with the intention to generate more accurate questions.
Pre-processing and data cleaning is done.
Complex Sentence (s):
Apple’s first logo, designed by Jobs and Wayne, depicts Sir Isaac Newton sitting under an apple tree.
Simple Sentence:
Apples first logo depicts Sir Isaac Newton sitting under an apple tree.
Apples first logo is designed by Jobs and Wayne. 

6. Named Entity Information
NE tagger that tags a plain text with named entities (people, organizations, locations, things).
Once we tag the body of text, we use some general purpose rules to create some basic questions.
Example:
Apples first logo depicts Sir [PER Isaac Newton] sitting under an apple tree .
Apples first logo is designed by [PER Jobs] and [PER Wayne. ]
Questions:
Who is Isaac Newton?
Who is Jobs?
Who is Wayne?

7. Semantic Role Labeling
Semantic Role Labeling: Giving semantic labels to phrases.
Provides a Structured Representation for text’s meaning
Semantic Role Labeling Knowledge-bases
PropBank
FrameNet

8. Semantic Role Labeling
The NYSE is prepared to open tomorrow on generator power if necessary," the statement said.
0: [ARG0 The NYSE] is prepared to [TARGET open ] [ARGM-TMP tomorrow] on generator power [ARGM-ADV if necessary] the statement said
0: [ARG1 The NYSE is prepared to open tomorrow on generator power if necessary] [ARG0 the statement] [TARGET said ]

9. Q/A from Semantic Role Labeling

10. Generate Questions and Answers
Given the Named Entity Information and Semantic Role Labels, Questions/Answers are generated.

11. Importance of Generated Questions
Find the topic of each section.
Compute topic-question similarity and prune Q/A

12. CoReferencing
Coreference resolution is the task of finding all expressions that refer to the same entity in a text.

13. Problem: Vague noun phrases
Noun phrases can refer to previous information in the discourse, leading to potentially vague questions.
The show boosted the studio to the top of the TV cartoon field
Q: What boosted the studio to the top of the TV cartoon field?
A: The show.

14. Solution 1: Vague noun phrases (In progress)
Paragraph segmentation:
Assumption: The content within the same topic is interrelated.
Hearst’s TextTiling algorithm
Text clustering using Topic Modeling (Hierarchical LDA)

15. Solution 2: Vague noun phrases (In progress)
Identifying intents of sentences.
“Before starting your vehicle, adjust your seat, adjust the inside and outside mirrors, fasten your seat belt.”
Intent: Things to do before starting your car.
We propose to classify intent into six categories:
State (Internal or external state)
Parts (Part of a vehicle)
Feature (Specific mode of a vehicle)
Problem
Procedures

16. Human Evaluations(In progress)
We use some native English-speaking people to judge the quality of the top-ranked 20% questions using two criteria: topic relevance, clarity and syntactic correctness.

17. Criteria-Value Extraction
iNago Project
Criteria-Value Extraction

18. Criteria-value extraction
A semantic representation of Q/A dataset in form of Attribute-Value pairs
Goals:
Complete representation of questions' different aspects
Enabling interactive conversation for question answering

19. Steps and timeline Phrase mining and concept identification
Question clustering and question intent detection
Identifying frames from patterns
Evaluation of generated criteria-values

20. Phrase mining and concept identification
Phrase mining: finding topical phrases from large text corpus
Finding domain-specific phrases
Entity recognition
Enhancing parsing results
Concept identification
To identify set of terms representing a concept in questions
Detecting important terms from words and phrases in questions
Using clustering algorithm for finding concepts
Concept pruning and labeling

21. Phrase mining and concept identification
Concept identification process
Measuring similarity with word embedding

22. Question clustering and question intent detection
Clustering questions based on similar intent
Extracting features with Semantic and syntactic parsing
Heuristic question patterns
Entity recognition
Constituent and dependency parse trees
Semantic role labeling

23. Identifying frames from patterns (in progress)
Frame: Grouping criteria-values for questions with same intent into a generalized form
Finding semantic patterns in question clusters
Detecting patterns based on shallow semantic parsing and SRL
Using external resources like FrameNet semantic dictionary
Generalization of semantic patterns for frame identification

24. Evaluation of criteria-values (in progress)
Defining quality metrics for criteria-values
Completeness:
Possibility of reconstructing a unique question from criteria-values
Informativeness:
No redundancy in criteria-values
Consistency:
Criteria should be consistent across all Q/A dataset
Designing a user study for measuring above qualities