Unsupervised Learning for Speech Motion Editing Eurographics/SIGGRAPH Symposium on Computer Animation (2003) Yong Cao 1,2 Petros Faloutsos 1 Frederic.

Slides:

Advertisements

Similar presentations

Independent Component Analysis

Advertisements

FACIAL EMOTION RECOGNITION BY ADAPTIVE PROCESSING OF TREE STRUCTURES Jia-Jun Wong and Siu-Yeung Cho Forensic and Security Lab School of Computer Engineering.

Color Imaging Analysis of Spatio-chromatic Decorrelation for Colour Image Reconstruction Mark S. Drew and Steven Bergner

Perceptually Guided Expressive Facial Animation Zhigang Deng and Xiaohan Ma Computer Graphics and Interactive Media Lab Department of Computer Science.

Facial expression as an input annotation modality for affective speech-to-speech translation Éva Székely, Zeeshan Ahmed, Ingmar Steiner, Julie Carson-Berndsen.

Implicit Probabilistic Models of Human Motion for Synthesis and Tracking Hedvig Sidenbladh, KTH, Sweden (now FOI, Sweden) Michael J. Black, Brown University,

AUTOMATIC SPEECH CLASSIFICATION TO FIVE EMOTIONAL STATES BASED ON GENDER INFORMATION ABSTRACT We report on the statistics of global prosodic features of.

Retargeting Algorithms for Performance-Driven Animation J.P. Lewis Fred Pighin.

SIGGRAPH Course 30: Performance-Driven Facial Animation For Latest Version of Bregler’s Slides and Notes please go to:

Summary & Homework Jinxiang Chai. Outline Motion data process paper summary Presentation tips Homework Paper assignment.

A 4-WEEK PROJECT IN Active Shape and Appearance Models

Evaluation in Digital Media Graphics Basic Concepts.

Introduction to Data-driven Animation Jinxiang Chai Computer Science and Engineering Texas A&M University.

Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004.

Principal Component Analysis

Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Dimensional reduction, PCA

Principal Component Analysis IML Outline Max the variance of the output coordinates Optimal reconstruction Generating data Limitations of PCA.

Face Poser: Interactive Modeling of 3D Facial Expressions Using Model Priors Manfred Lau 1,3 Jinxiang Chai 2 Ying-Qing Xu 3 Heung-Yeung Shum 3 1 Carnegie.

Independent Component Analysis (ICA) and Factor Analysis (FA)

Subspace Representation for Face Recognition Presenters: Jian Li and Shaohua Zhou.

Dynamic Response for Motion Capture Animation Victor B. Zordan Anna Majkowska Bill Chiu Matthew Fast Riverside Graphics Lab University of California, Riverside.

Realistic Facial Modelling For Animation. Facial Modeling For Animation Building a general face mesh Building a general face mesh 3D digitization of the.

Vision-based Control of 3D Facial Animation Jin-xiang Chai Jing Xiao Jessica Hodgins Carnegie Mellon University.

ICA Alphan Altinok. Outline  PCA  ICA  Foundation  Ambiguities  Algorithms  Examples  Papers.

Sufficient Dimensionality Reduction with Irrelevance Statistics Amir Globerson 1 Gal Chechik 2 Naftali Tishby 1 1 Center for Neural Computation and School.

K-means Based Unsupervised Feature Learning for Image Recognition Ling Zheng.

Facial Type, Expression, and Viseme Generation Josh McCoy, James Skorupski, and Jerry Yee.

Comparing Kernel-based Learning Methods for Face Recognition Zhiguo Li

Faces: Analysis and Synthesis Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski.

Facial Type, Expression, and Viseme Generation Josh McCoy, James Skorupski, and Jerry Yee.

Recognizing Emotions in Facial Expressions

Motion Capture in 3D Animation Edward Tse. Motion Capture as a Tool Motion capture (MOCAP) is an effective 3D animation tool for realistically capturing.

Survey on ICA Technical Report, Aapo Hyvärinen, 1999.

Facial Feature Detection

Human Emotion Synthesis David Oziem, Lisa Gralewski, Neill Campbell, Colin Dalton, David Gibson, Barry Thomas University of Bristol, Motion Ripper, 3CR.

: Chapter 1: Introduction 1 Montri Karnjanadecha ac.th/~montri Principles of Pattern Recognition.

Project 10 Facial Emotion Recognition Based On Mouth Analysis SSIP 08, Vienna 1

GIP: Computer Graphics & Image Processing 1 1 Medical Image Processing & 3D Modeling.

Graphite 2004 Statistical Synthesis of Facial Expressions for the Portrayal of Emotion Lisa Gralewski Bristol University United Kingdom

Hongyan Li, Huakui Wang, Baojin Xiao College of Information Engineering of Taiyuan University of Technology 8th International Conference on Signal Processing.

Principal Component Analysis Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

110/20/ :06 Graphics II Paper Reviews Facial Animation Session 8.

University of Coimbra ISR – Institute of Systems and Robotics University of Coimbra - Portugal Institute of Systems and Robotics

2D Animation Techniques for 3D Animation Research - KCGS Conference. Spring, In-Kwon Lee Game Animation Center Division of Media Ajou University.

Intelligent Control and Automation, WCICA 2008.

A NOVEL METHOD FOR COLOR FACE RECOGNITION USING KNN CLASSIFIER

3D Face Recognition Using Range Images

PCA vs ICA vs LDA. How to represent images? Why representation methods are needed?? –Curse of dimensionality – width x height x channels –Noise reduction.

© 2009 Robert Hecht-Nielsen. All rights reserved. 1 Andrew Smith University of California, San Diego Building a Visual Hierarchy.

University of Washington v The Hebrew University * Microsoft Research Synthesizing Realistic Facial Expressions from Photographs Frederic Pighin Jamie.

Animation From Observation: Motion Editing Dan Kong CMPS 260 Final Project.

Facial Animation Wilson Chang Paul Salmon April 9, 1999 Computer Animation University of Wisconsin-Madison.

Introduction to Independent Component Analysis Math 285 project Fall 2015 Jingmei Lu Xixi Lu 12/10/2015.

Constrained Synthesis of Textural Motion for Animation Shmuel Moradoff Dani Lischinski The Hebrew University of Jerusalem.

3D Face Recognition Using Range Images Literature Survey Joonsoo Lee 3/10/05.

Interpreting Ambiguous Emotional Expressions Speech Analysis and Interpretation Laboratory ACII 2009.

Nataliya Nadtoka James Edge, Philip Jackson, Adrian Hilton CVSSP Centre for Vision, Speech & Signal Processing UNIVERSITY OF SURREY.

Crowds (and research in computer animation and games)

Sentiment analysis algorithms and applications: A survey

PRESENTED BY Yang Jiao Timo Ahonen, Matti Pietikainen

Majkowska University of California. Los Angeles

Machine Learning Dimensionality Reduction

Outline Multilinear Analysis

PCA vs ICA vs LDA.

Object Modeling with Layers

Outline S. C. Zhu, X. Liu, and Y. Wu, “Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo”, IEEE Transactions On Pattern Analysis And Machine.

Turning to the Masters: Motion Capturing Cartoons

Speech Prosody Conversion using Sequence Generative Adversarial Nets

End-to-End Speech-Driven Facial Animation with Temporal GANs

Presentation transcript:

Unsupervised Learning for Speech Motion Editing Eurographics/SIGGRAPH Symposium on Computer Animation (2003) Yong Cao 1,2 Petros Faloutsos 1 Frederic Pighin 2 University of California, Los Angeles 1 Institute for Creative Technologies, University of Southern California 2

Problem ■ Motion Capture is convenient but lacks flexibility ■ Problem: How to extract the semantics of the data for intuitive motion editing?

Related Work 1. Face motion synthesis ■ Physics-based face model Lee, Terzopoulos, Water ( SIGGRAPH 1995) Kähler, Haber, Seidel (Graphics Interface 2001) ■ Speech motion synthesis Bregler, Covell, Slaney (SIGGRAPH 1997) Brand (SIGGRAPH 1999) Ezzat, Pentland, Poggio (SIGGRAPH 2002) 2. Separation of style and content Brand, Hertzmann (SIGGRAPH 2000) Chuang, Deshpande, Bregler (Pacific Graphics 2002)

Our Contribution ■ New statistical representation of facial motion Decomposition into style and content Intuitive editing operations

Our Contribution Original Neutral MotionEdited Sad Motion

Roadmap ■ Independent Component Analysis (ICA) ■ Facial motion decomposition ■ Semantics of components ■ Motion editing

New representation or

Independent Component Analysis (ICA) ■ ICA Linear transformation Components are independent ■ Example: Blind Source Separation ICA From: “

Independent Component Analysis (ICA) ■ Statistical technique ■ Linear transformation ■ Components are maximally independent

■ Preprocessing (PCA) ■ Centering ■ Whitening Decomposition: Reconstruction: ■ ICA decomposition Steps of ICA

ICA vs. PCA ■ The components of PCA are uncorrelated ■ The components of ICA are independent

Components of PCA Can NOT separate Mouth motion and Eye-brow motion

Components of ICA Mouth motion and Eye-brow motion being separated

Roadmap ■ Independent Component Analysis (ICA) ■ Facial motion decomposition ■ Semantics of components ■ Motion editing

Speech motion of 113 sentences in 5 emotion moods: Frustrated 18 sentences Happy 18 sentences Neutral 17 sentences Sad 30 sentences Angry 30 sentences Speech motion Dataset Each motion: 109 motion capture markers 2 – 4 seconds

Facial Motion and ………… Facial motion Components in ICA space DecompositionReconstruction

Roadmap ■ Independent Component Analysis (ICA) ■ Facial motion decomposition ■ Semantics of components ■ Motion editing

Interpretation of independent components ■ Qualitatively ■ Quantitatively ■ Goal: Find the semantics of each component Classify each component into: Style (emotion) Content (speech) ■ Methodology

Qualitatively Style (emotion)Content (speech) changing

Style: Emotion Same speech, different emotion ………… HappyFrustrated Quantitatively

■ Eyelid motion■ Eyebrow motion■ Mouth motion Speech Content Grouping of motion markers

Content: speech related motion Step1: Using each independent component to reconstruct facial motion Reconstruct …………

Step2: Compare according to certain region Content: speech related motion

Roadmap ■ Independent Component Analysis (ICA) ■ Facial motion decomposition ■ Semantic meaning of components ■ Motion editing

Motion Editing with ICA ■ Edit the motion in intuitive ways ■ Translate ■ Copy and Replace ■ Copy and Add

Results ■ Changing emotional state by translating

Conclusion ■ New statistical representation of facial motion Decomposition into content and style Intuitive editing operations

The End Thanks to Wen Tien for his help on this paper, Christos Faloutsos for useful discussions, and Brian Carpenter for his excellent performance. Thanks to the USC School of Cinema – Television and House of Moves for motion capture.