Multimedia Concepts and Applications Multimedia Concepts and Applications Affect Sensing in Speech: Studying Fusion of Linguistic and Acoustic Features.

Slides:

Advertisements

Similar presentations

1 Speech Sounds Introduction to Linguistics for Computational Linguists.

Advertisements

A Word at a Time Computing Word Relatedness using Temporal Semantic Analysis Kira Radinsky, Eugene Agichteiny, Evgeniy Gabrilovichz, Shaul Markovitch.

V-1 Part V: Collaborative Signal Processing Akbar Sayeed.

Knowing More than One Language: The Psycholinguistics of Bilingualism Marina Blekher Department of Linguistics.

Improved TF-IDF Ranker

Automatic Prosodic Event Detection Using Acoustic, Lexical, and Syntactic Evidence Sankaranarayanan Ananthakrishnan, Shrikanth S. Narayanan IEEE 2007 Min-Hsuan.

SUPER: Towards Real-time Event Recognition in Internet Videos Yu-Gang Jiang School of Computer Science Fudan University Shanghai, China

Linguistic Regularities in Sparse and Explicit Word Representations Omer LevyYoav Goldberg Bar-Ilan University Israel.

Multimodal User Interface Management Part of STIMULATE project at CAIP Center, Rutgers University PI: Dr. James L. Flanagan Research on natural human-computer.

Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition Thurid Vogt, Elisabeth André ICME 2005 Multimedia concepts.

Insert A tree starts with the dummy node D D 200 D 7 Insert D

Chapter 7 Network Flow Models.

تمرين شماره 1 درس NLP سيلابس درس NLP در دانشگاه هاي ديگر ___________________________ راحله مکي استاد درس: دکتر عبدالله زاده پاييز 85.

1 Complementarity of Lexical and Simple Syntactic Features: The SyntaLex Approach to S ENSEVAL -3 Saif Mohammad Ted Pedersen University of Toronto, Toronto.

Explain reflexivity in qualitative research. Reflexivity Pretty new concept to psychology. The researcher reflects (writes at the end of the study) on.

Producing Emotional Speech Thanks to Gabriel Schubiner.

Adding Common Sense into Artificial Intelligence Common Sense Computing Initiative Software Agents Group MIT Media Lab.

Natural Language Understanding

Some Advances in Transformation-Based Part of Speech Tagging

Empirical Methods in Information Extraction Claire Cardie Appeared in AI Magazine, 18:4, Summarized by Seong-Bae Park.

Lecture 12: 22/6/1435 Natural language processing Lecturer/ Kawther Abas 363CS – Artificial Intelligence.

Whither Linguistic Interpretation of Acoustic Pronunciation Variation Annika Hämäläinen, Yan Han, Lou Boves & Louis ten Bosch.

{ LOCATION The concept of location in British Sign Language.

CRICOS No J † CSIRO ICT Centre * Speech, Audio, Image and Video Research Laboratory An Examination of Audio-Visual Fused HMMs for Speaker Recognition.

Exploiting Ontologies for Automatic Image Annotation M. Srikanth, J. Varner, M. Bowden, D. Moldovan Language Computer Corporation

Comparing tv news programmes A framework for analysis.

Experiments on Building Language Resources for Multi-Modal Dialogue Systems Goals identification of a methodology for adapting linguistic resources for.

Break-out Group # D Research Issues in Multimodal Interaction.

Based on “Semi-Supervised Semantic Role Labeling via Structural Alignment” by Furstenau and Lapata, 2011 Advisors: Prof. Michael Elhadad and Mr. Avi Hayoun.

Hierarchical Dirichlet Process (HDP) A Dirichlet process (DP) is a discrete distribution that is composed of a weighted sum of impulse functions. Weights.

SCALE Workshop, Saarbrücken, January 12, 2010 Prof. Hervé Bourlard Idiap Research Institute EPFL Idiap Research Institute Centre du Parc P.O Box 592 CH.

This work is supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior National Business Center contract number.

Machine Translation  Machine translation is of one of the earliest uses of AI  Two approaches:  Traditional approach using grammars, rewrite rules,

Pragmatically-guided perceptual learning Tanya Kraljic, Arty Samuel, Susan Brennan Adaptation Project mini-Conference, May 7, 2007.

A Brief Review of Theory for Information Fusion in Sensor Networks Xiaoling Wang February 19, 2004.

Efficient Language Model Look-ahead Probabilities Generation Using Lower Order LM Look-ahead Information Langzhou Chen and K. K. Chin Toshiba Research.

LANGUAGE MODELS FOR RELEVANCE FEEDBACK Lee Won Hee.

AI on the Battlefield: an Experimental Exploration Alexander Kott BBN Technologies Robert Rasch US Army Battle Command Battle Lab Views expressed in this.

Referring to Objects with Spoken and Haptic Modalities Frédéric LANDRAGIN Nadia BELLALEM & Laurent ROMARY LORIA Laboratory Nancy, FRANCE.

Multimodality, universals, natural interaction… and some other stories… Kostas Karpouzis & Stefanos Kollias ICCS/NTUA HUMAINE WP4.

Lecture 1 Lec. Maha Alwasidi. Branches of Linguistics There are two main branches: Theoretical linguistics and applied linguistics Theoretical linguistics.

Wrapping Up Ling575 Spoken Dialog Systems June 5, 2013.

The meaning of Language Chapter 5 Semantics and Pragmatics Week10 Nov.19 th -23 rd.

Multimedia Concepts and Applications Multimedia Concepts and Applications Differentiated Semantic Analysis in Lexical Affect Sensing Alexander Osherenko,

Public Speaking: The Communication Model. Objectives: Define Communication List and explain the components of the communication process.

Hello, Who is Calling? Can Words Reveal the Social Nature of Conversations?

Levels of Linguistic Analysis

Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)

10.0 Latent Semantic Analysis for Linguistic Processing References : 1. “Exploiting Latent Semantic Information in Statistical Language Modeling”, Proceedings.

Humanities Computing Master of arts in F A C U L T Y O F T H E.

Text Categorization by Boosting Automatically Extracted Concepts Lijuan Cai and Tommas Hofmann Department of Computer Science, Brown University SIGIR 2003.

Lexical Affect Sensing: Are Affect Dictionaries Necessary to Analyze Affect? Alexander Osherenko, Elisabeth André University of Augsburg.

Graphs Definition: a graph is an abstract representation of a set of objects where some pairs of the objects are connected by links. The interconnected.

Software Architecture for Multimodal Interactive Systems : Voice-enabled Graphical Notebook.

Linguistic Regularities in Sparse and Explicit Word Representations Omer LevyYoav Goldberg Bar-Ilan University Israel.

Cross-modal Hashing Through Ranking Subspace Learning

Towards Semantic Affect Sensing in Sentences Alexander Osherenko.

Speech and multimodal Jesse Cirimele. papers “Multimodal interaction” Sharon Oviatt “Designing SpeechActs” Yankelovich et al.

课件名称：词义和语境制作人：孙红梅、张培成单位：曲阜师范大学外国语学院

Word2Vec CS246 Junghoo “John” Cho.

GTECH 709 GIS Data Formats GIS data formats

CH 3: Applying the Multimedia Principle

Studying Spoken Language Text 17, 18 and 19

CS4705 Natural Language Processing

Levels of Linguistic Analysis

Jewitt, C. (2014). The Routledge Handbook of Multimodal Analysis

Semantics Going beyond syntax.

Artificial Intelligence 2004 Speech & Natural Language Processing

Presentation transcript:

Multimedia Concepts and Applications Multimedia Concepts and Applications Affect Sensing in Speech: Studying Fusion of Linguistic and Acoustic Features Alexander Osherenko, Elisabeth André, Thurid Vogt University of Augsburg

Multimedia Concepts and Applications Multimedia Concepts and Applications Affect Sensing Acoustic information Linguistic information (lexical, stylometric, deictic)

Multimedia Concepts and Applications Multimedia Concepts and Applications Fusion Decision-level Feature-level

Multimedia Concepts and Applications Multimedia Concepts and Applications Research Questions Fusion Context Decision-level vs. feature-level

Multimedia Concepts and Applications Multimedia Concepts and Applications Experimental Setting SAL corpus, 574 turns, 5 classes Decision-level using majority, feature-level – fusing features Data: 2 stages (history 0 and history 7) – Acoustic modality - 2 (discrete/continuous) acoustic datasets (A) – Lingustic modality - 29 lexical (L), 31 stylometric (S), 63 deictic datasets (D)

Multimedia Concepts and Applications Multimedia Concepts and Applications Tree – Nodes – features from particular modalities (A, L, S, D) – Values Maximal recall value Maximal multimodality value Dotted arcs Results‘ representation

Multimedia Concepts and Applications Multimedia Concepts and Applications Best results: 64.2% (history 7) and 44.2% (history 0) Significant improvement through context Insignificant improvement through fusion (about 2%) Maximal multimodality value (76.5%) Decision-level Fusion Before Discretization

Multimedia Concepts and Applications Multimedia Concepts and Applications Best results: 66.0% (history 7) and 49.0% (history 0) Significant improvement through context Insignificant improvement through fusion (about 2%) Maximal multimodality value (77.8%) Decision-level Fusion After Discretization

Multimedia Concepts and Applications Multimedia Concepts and Applications Best results: 62.8% vs. 64.2% (history 7) and 46.7% vs. 44.2% (history 0) Significant improvement through context Insignificant improvement through fusion (about 2%) Feature-level Fusion Before Discretization

Multimedia Concepts and Applications Multimedia Concepts and Applications Feature-level Fusion After Discretization Best results: 67.5% vs. 64.9% (history 7) and 52.8% vs. 45.9% (history 0) Significant improvement through context Insignificant improvement through fusion (about 2%)

Multimedia Concepts and Applications Multimedia Concepts and Applications Discussion Role of context Role of discretization Fusion?

Multimedia Concepts and Applications Multimedia Concepts and Applications Future work New modalities Weighting