1. CRCV REU 2019
Week 8
Aaron Honculada

2. Current Approach Parsing Question into Subqueries
How to represent video to utilize the information

3. Literature Review Neural-Symbolic Concept Learner
Visual Perception Module
Mask R-CNN
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision (Tenenbaum, ICLR 2019)

4. Neural-Symbolic Concept Learner
Semantic Parsing
Question to program
Hierarchy of operations
concepts as parameters
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision (Tenenbaum, ICLR 2019)

5. Neural-Symbolic Concept Learner
Domain Specific Language
Requires unique referral object
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision (Tenenbaum, ICLR 2019)

6. Dependency Parsing Sentence to Dependency Tree
Dependency from Grammatical Relation
Possible Multi-task model
VQA
Generate Dependency Tree

7. Dependency Parsing Example
What could happen to the newborn , Emily , after Izzie administers the ECMO ?

8. Dependency Parsing Locate keywords in questions
With keywords formulate subqueries
Biaffine Classifier and BiLSTM
Deep Biaffine Attention for Neural Dependency Parsing (Dozat, ICLR 2017)

9. Video Representation Cannot fully answer from visual concepts alone
Contextualize objects in scene
Color
Belongs to person or scene
Places365
Scene Attributes
Indoor/ Outdoor