Survey of Annotation Work Joint session Thursday afternoon, April 14 Chair: Eduard Hovy, ISI.

Slides:

Advertisements

Similar presentations

KR-2002 Panel/Debate Are Upper-Level Ontologies worth the effort? Chris Welty, IBM Research.

Advertisements

INTRODUCTION TO ARTIFICIAL INTELLIGENCE Massimo Poesio Relation Extraction.

Layering Semantics (Putting meaning into trees) Treebank Workshop Martha Palmer April 26, 2007.

Multilinugual PennTools that capture parses and predicate-argument structures, and their use in Applications Martha Palmer, Aravind Joshi, Mitch Marcus,

FrameNet, PropBank, VerbNet Rich Pell. FrameNet, PropBank, VerbNet  When syntactic information is not enough  Lexical databases  Annotate a natural.

Omega Ontology: Supporting Annotation Eduard Hovy with Andrew Philpot, Jerry Hobbs, Michael Fleischman, and Patrick Pantel USC/ISI.

1 An Integrated Annotation DB in OntoNotes Sameer Pradhan, Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel

Semantic Role Labeling Abdul-Lateef Yussiff

PropBanks, 10/30/03 1 Penn Putting Meaning Into Your Trees Martha Palmer Paul Kingsbury, Olga Babko-Malaya, Scott Cotton, Nianwen Xue, Shijong Ryu, Ben.

Steven Schoonover.  What is VerbNet?  Levin Classification  In-depth look at VerbNet  Evolution of VerbNet  What is FrameNet?  Applications.

Semantic Annotation Meeting April 14, 2005 NomBank & the Down-to-Earth Parts of Pie-in-the-Sky Adam Meyers New York University April 14, 2004.

1 NSF-ULA Sense tagging and Eventive Nouns Martha Palmer, Miriam Eckert, Jena D. Hwang, Susan Windisch Brown, Dmitriy Dligach, Jinho Choi, Nianwen Xue.

1 Annotation Guidelines for the Penn Discourse Treebank Part B Eleni Miltsakaki, Rashmi Prasad, Aravind Joshi, Bonnie Webber.

April 26, 2007Workshop on Treebanking, NAACL-HTL 2007 Rochester1 Treebanks: Layering the Annotation Jan Hajič Institute of Formal and Applied Linguistics.

OntoNotes/PropBank Participants: BBN, Penn, Colorado, USC/ISI.

Semantic Annotation Evaluation and Utility Bonnie Dorr Saif Mohammad David Yarowsky Keith Hall.

NomBank 1.0: ULA08 Workshop March 18, 2007 NomBank 1.0 Released 12/2007 Unified Linguistic Annotation Workshop Adam Meyers New York University March 18,

Lance Ramshaw (with Ralph Weischedel) BBN. 2 Ontobank Coreference Part of the multi-site Ontobank effort –Intended to combine with word-sense and propositional.

LCS and Approximate Interlingua at UMD Semantic Annotation Planning Meeting April 14, 2004 Bonnie J. Dorr University of Maryland.

OntoNotes project Treebank Syntax Training Data Decoders Propositions Verb Senses and verbal ontology links Noun Senses and targeted nominalizations Coreference.

PropBank Martha Palmer University of Colorado. Unified Linguistic Annotation: Merging PropBank, NomBank, TimeBank, Penn Discourse Treebank, Coreference,

EMPOWER 2 Empirical Methods for Multilingual Processing, ‘Onoring Words, Enabling Rapid Ramp-up Martha Palmer, Aravind Joshi, Mitch Marcus, Mark Liberman,

Comments to Karin Kipper Christiane Fellbaum Princeton University and Berlin-Brandenburgische Akademie der Wissenschaften.

Learning Narrative Schemas Nate Chambers, Dan Jurafsky Stanford University IBM Watson Research Center Visit.

PropBank, VerbNet & SemLink Edward Loper. PropBank 1M words of WSJ annotated with predicate- argument structures for verbs. –The location & type of each.

September 17, : Grammars and Lexicons Lori Levin.

1 Interlingual Annotation of Multilingual Text Corpora (IAMTC) Project Overview for ITIC November 13, 2003 Carnegie Mellon University Lori Levin, Teruko.

SALSA The Saarbrücken Lexical Semantics Annotation & Acquisition Project Aljoscha Burchardt, Katrin Erk, Anette Frank, Andrea Kowalski, Sebastian Pado,

Jennie Ning Zheng Linda Melchor Ferhat Omur. Contents Introduction WordNet Application – WordNet Data Structure - WordNet FrameNet Application – FrameNet.

The Prague (Czech-)English Dependency Treebank Jan Hajič Charles University in Prague Computer Science School Institute of Formal and Applied Linguistics.

Copy right 2003 Adam Pease permission to copy granted so long as slides and this notice are not altered Language to Logic Translation.

Penn 1 Kindle: Knowledge and Inference via Description Logics for Natural Language Dan Roth University of Illinois, Urbana-Champaign Martha Palmer University.

AQUAINT Workshop – June 2003 Improved Semantic Role Parsing Kadri Hacioglu, Sameer Pradhan, Valerie Krugler, Steven Bethard, Ashley Thornton, Wayne Ward,

MASC The Manually Annotated Sub- Corpus of American English Nancy Ide, Collin Baker, Christiane Fellbaum, Charles Fillmore, Rebecca Passonneau.

Modelling Human Thematic Fit Judgments IGK Colloquium 3/2/2005 Ulrike Padó.

Discourse Connectives and Their Argument Structure: Annotating a discourse treebank ARAVIND K. JOSHI Department of Computer and Information Science October.

Ideas for 100K Word Data Set for Human and Machine Learning Lori Levin Alon Lavie Jaime Carbonell Language Technologies Institute Carnegie Mellon University.

Resemblances between Meaning-Text Theory and Functional Generative Description Zdeněk Žabokrtský Institute of Formal and Applied Linguistics Charles University,

1 Discourse Connectives and Their Argument Structure: Annotating a discourse treebank ARAVIND K. JOSHI Department of Computer and Information Science August.

Semantic Role Labeling. Introduction Semantic Role Labeling AgentThemePredicateLocation.

Combining Lexical Resources: Mapping Between PropBank and VerbNet Edward Loper,Szu-ting Yi, Martha Palmer September 2006.

Ontologies and Terminology and how they relate to lexicography Adam Kilgarriff Auckland 20121Kilgarriff: Ontologies and Terminology.

Semantic Annotation for Interlingual Representation of Mulilingual Texts Teruko Mitamura (CMU), Keith Miller (MITRE), Bonnie Dorr (Maryland), David Farwell.

ENGLISH BASICS Subject- predicate notes. Every complete sentence needs… A SUBJECT and A PREDICATE.

Semantic Annotation & Utility Evaluation Meeting: Feb 14, 2008 Project Organization: Who is here? Agenda Meaning Layers and Applications Ongoing work.

LING 6520: Comparative Topics in Linguistics (from a computational perspective) Martha Palmer Jan 15,

ARDA Visit 1 Penn Lexical Semantics at Penn: Proposition Bank and VerbNet Martha Palmer, Dan Gildea, Paul Kingsbury, Olga Babko-Malaya, Bert Xue, Karin.

DGP – S ENTENCE 1 Sentence Parts. S ENTENCE / W ORD B ANK What’s the brand of your sneakers, man? Word Bank: intransitive verb, noun of direct address,

NLP. Introduction to NLP Last week, Min broke the window with a hammer. The window was broken with a hammer by Min last week With a hammer, Min broke.

1 Fine-grained and Coarse-grained Word Sense Disambiguation Jinying Chen, Hoa Trang Dang, Martha Palmer August 22, 2003.

Knowledge Structure Vijay Meena ( ) Gaurav Meena ( )

SALSA-WS 09/05 Approximating Textual Entailment with LFG and FrameNet Frames Aljoscha Burchardt, Anette Frank Computational Linguistics Department Saarland.

Multilinugual PennTools that capture parses and predicate-argument structures, for use in Applications Martha Palmer, Aravind Joshi, Mitch Marcus, Mark.

General characteristics As any other part of speech, the noun can be characterized by three criteria:  Semantic (the meaning)  Morphological (the form.

Subject Predicate Subject Main verb (Nominative structure) Auxiliary (link) verb.

Overview of Statistical NLP IR Group Meeting March 7, 2006.

Syntax- the object study. What is syntax?  Syntax is the study of the structure  of sentences.  Syntax analyzes how words combine to form sentences.

A Database of Narrative Schemas A 2010 paper by Nathaniel Chambers and Dan Jurafsky Presentation by Julia Kelly.

FFW  TAKE OUT HOMEWORK: IO AND DO WORKSHEET  IN YOUR NOTEBOOK, ANSWER THE FOLLOWING QUESTION: 1.WHAT IS A PREDICATE? 2.WHAT IS A PREDICATE NOMINATIVE?

COSC 6336: Natural Language Processing

English Proposition Bank: Status Report

Question Answering & Reading Comprehension

DGP – Sentence 2 Parts of Speech.

PREPOSITIONAL PHRASES

Towards Semantics Generation

Noun: Owner’s Manual Congratulations on your wise purchase of a NOUN. Your NOUN may be used to fit into the following frame: The____________. Your NOUN.

Some thing about Nouns in WordNet

Lecture 19 Word Meanings II

CS224N Section 3: Corpora, etc.

CS224N Section 3: Project,Corpora

Presentation transcript:

Survey of Annotation Work Joint session Thursday afternoon, April 14 Chair: Eduard Hovy, ISI

Phenomena (from OntoBank) LevelWhoPhenomenon L1Penn Treebankbracketing/grouping of predications L1Propbankverb sense creation and annotation (including copula) L1 Propbank, Framenet, Verbnet, LCS, ILIT verb sense frames & predicate structure (what labels?) L1 Propbank+Omega, IAMTC+Omega, ILIT, Scone semantic term repository: conversion of senses to concepts(/clusters), axiom creation, insertion into ontology L1,L2NomBank, ACEnoun senses, NP structure, propositions, (genitives, …) L1Gazetteersrepository of instances (people, places, events…) L1BBN, (ACE)co-reference links (including events) L2 pronoun (and empty trace?) classification (ref, bound, event, generic, other)(proposition vs. event?) L2Propbank II, ILITevent identification

LevelWhoPhenomenon L1direct quotation and reported speech L1simple quantifier phrases and numerical exprs L1,L2TimeBank, TIMEX, ISI (Hobbs), ILIT inter-predicate relations: temporal, spatial, manner, etc. (incl. effects from discourse and aspect) L2+WordNetPlus, Pantel, CYCentailments L2+comparatives L2coordination L2/L3Penn Discourse Treebank, RST Treebank, ILIT discourse structure L2/L3U Pitt, ISIopinions L3identifying propositions and simple modality L3/L4other adverbials (epistemic modals, evidentials) L3/L4polarity (more advanced than plain “neg” in L1) L3+Steedman, Hajicova, Sgallinformation structure (theme/rheme), focus L4ILITpragmatics/speech acts, style L4presuppositions ?CYC, Sconeaxioms and reasoning ?Framenetmetaphor

Notional goal phenomenon annot annot functionality funder speed reliability need noun senses 25 wph 86/90% IE,MT,QA... high verb senses 70 wph ~87% MT,QA,WSD high verb frames 80 w/week 87% MT,QA,IE… high time exprs 18 wpm 96% QA,IR,Summ med-hi discourse 100K in 400h~90/80%Summ,QA med gazetteers ?~95/90% QA,IE high opinions 100K in 400h ~76%QA,Summ med-hi number exprs ? ? IE,QA,Summ med hypotheticals ? ? QA,Summ low?

Agenda I Predicate/verb level: –PropBank I and II: Martha Palmer, UPenn –OntoBank corefs: Lance Ramshaw, BBN –IAMTC consortium: Steve Helmreich, NMSU –FrameNet: Charles Fillmore, UC Berkeley –Extended LCS: Bonnie Dorr, U Maryland Nominal level: –NomBank: Adam Meyers, NYU –ACE: Ralph Grishman, NYU Terminology banks: –WordNet: Christiane Fellbaum, Princeton –Omega: Eduard Hovy, USC/ISI to PropBank to IAMTC to OntoBank coref to Framenet to LCS to NomBank and Pie-in-the-Sky to ACE to WordNetPlus to Omega

Agenda II Discourse level: –RST treebank: Lynn Carlson, DoD –Penn discourse treebank: Aravind Joshi, UPenn Specific semantic phenomena: –TIMEX: Lisa Ferro, MITRE & Beth Sundheim, SPAWAR –ILIT: Sergei Nirenburg, UMBC –Opinions: Jan Wiebe, U Pitt –Gazetteers: Beth Sundheim, SPAWAR Inference and reasoning: –WN Entailments: Christiane Fellbaum, Princeton –CYC: Dave Schneider –Scone: Scott Fahlman to Penn discourse to TIMEX to ILIT to opinions to gazetteers to WN entailments to CYC to Scone to RST

Summary of annot work