Computer Vision in Graphics Production Adrian Hilton Visual Media Research Group Centre for Vision, Speech and Signal Processing University of Surrey

Slides:

Advertisements

Similar presentations

Free-viewpoint Immersive Networked Experience February 2010.

Advertisements

Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.

Motion Capture The process of recording movement and translating that movement onto a digital model Games Fast Animation Movies Bio Medical Analysis VR.

Vision Sensing. Multi-View Stereo for Community Photo Collections Michael Goesele, et al, ICCV 2007 Venus de Milo.

Computer Graphics CMSC 435/ Graphics Areas “ Core ” graphics areas – Modeling – Rendering – Animation Other areas which draw on computer graphics.

EVENTS: INRIA Work Review Nov 18 th, Madrid.

SIGGRAPH Course 30: Performance-Driven Facial Animation For Latest Version of Bregler’s Slides and Notes please go to:

Vision For Graphics ICCV 2005 Vision for Graphics Larry Zitnick, Sing Bing Kang, Rick Szeliski Interactive Visual Media Group Microsoft Research Steve.

Dana Cobzas-PhD thesis Image-Based Models with Applications in Robot Navigation Dana Cobzas Supervisor: Hong Zhang.

Image-Based Modeling, Rendering, and Lighting

Advanced Computer Graphics CSE 190 [Spring 2015], Lecture 14 Ravi Ramamoorthi

Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004.

HKIE Accreditation Visit Welcome to Signal Processing Lab! Introduction to Signal Processing Lab Project Demonstrations Lab Tour.

SIGGRAPH Course 30: Performance-Driven Facial Animation Section: Markerless Face Capture and Automatic Model Construction Part 2: Li Zhang, Columbia University.

Advanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light Fields Ravi Ramamoorthi

Representations of Visual Appearance COMS 6160 [Spring 2007], Lecture 4 Image-Based Modeling and Rendering

Image-Based Rendering Computer Vision CSE576, Spring 2005 Richard Szeliski.

High-Quality Video View Interpolation

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Vision-based Control of 3D Facial Animation Jin-xiang Chai Jing Xiao Jessica Hodgins Carnegie Mellon University.

8/16/99 Experiments in Motion (Capture). 8/16/99 Experiments in Motion Capture Project Course Week 3-5: Motion Capture Pipeline Assignments Then start.

CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.

Convergence of vision and graphics Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

David Luebke Modeling and Rendering Architecture from Photographs A hybrid geometry- and image-based approach Debevec, Taylor, and Malik SIGGRAPH.

Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Lighting.

© 2006 Autodesk1 Image-processing Technologies for Digital Content Creation

Research & Innovation 1 An Industry Perspective on VVG Research Oliver Grau BBC Research & Innovation VVG SUMMER SCHOOL '07.

Real-Time High Quality Rendering CSE 291 [Winter 2015], Lecture 6 Image-Based Rendering and Light Fields

Supervisor : Mikael Fernstrom.  Story Telling  Script writing  Visual Effects - Practical / Digital / Optical  Foley Sound  Problem Solving  Visuals.

Integration Of CG & Live-Action For Cinematic Visual Effects by Amarnath Director, Octopus Media School.

Digital Face Cloning David Bennett Christophe HerySteve Sullivan George Borshukov J.P. Lewis Lance Williams Paul Debevec Fred Pighin Li Zhang.

Electronic Visualization Laboratory University of Illinois at Chicago Interaction between Real and Virtual Humans: Playing Checkers R. Torre, S. Balcisoy.

Recovering High Dynamic Range Radiance Maps from Photographs [Debevec, Malik - SIGGRAPH’97] Presented by Sam Hasinoff CSC2522 – Advanced Image Synthesis.

1 Introduction to Multimedia What is Multimedia. 1

GIP: Computer Graphics & Image Processing 1 1 Medical Image Processing & 3D Modeling.

Advanced Computer Graphics (Spring 2013) CS 283, Lecture 15: Image-Based Rendering and Light Fields Ravi Ramamoorthi

Real-Time High Resolution Photogrammetry John Morris, Georgy Gimel’farb and Patrice Delmas CITR, Tamaki Campus, University of Auckland.

Facial animation retargeting framework using radial basis functions Tamás Umenhoffer, Balázs Tóth Introduction Realistic facial animation16 is a challenging.

3D COMPUTER GRAPHICS IMD Chapter 1: 3D Computer Graphics Chapter 1: 1 Lecturer: Norhayati Mohd Amin.

Computer Graphics Psychophysics Heinrich H. Bülthoff Max-Planck-Institute for Biological Cybernetics Tübingen, Germany Heinrich H. Bülthoff Max-Planck-Institute.

Image-based rendering Michael F. Cohen Microsoft Research.

Centre for Vision, Speech & Signal Processing University of Surrey Adrian Hilton, John Collomosse Peng Huang, Dan Casas, Ed Brookes, Chris Budd, Marco.

N n Debanga Raj Neog, Anurag Ranjan, João L. Cardoso, Dinesh K. Pai Sensorimotor Systems Lab, Department of Computer Science The University of British.

Grand Theft Auto 4 took more than three and half years to create With over 1000 people working on it. at a cost of almost $100 million.

Research Interests of Dr. Dennis J Bouvier Fall 2007.

Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Spring 2015 CSc 83020: 3D Photography Prof. Ioannis Stamos Mondays 4:15 – 6:15

12/7/10 Looking Back, Moving Forward Computational Photography Derek Hoiem, University of Illinois Photo Credit Lee Cullivan.

Efficient Visual Object Tracking with Online Nearest Neighbor Classifier Many slides adapt from Steve Gu.

MASKS © 2004 Invitation to 3D vision. MASKS © 2004 Invitation to 3D vision Lecture 1 Overview and Introduction.

CPSC : Data-driven Computer Graphics Jinxiang Chai.

Applications of 3D z Applications that are used and why we use them Product design Modelling TV and film Web games Education Reconstruction Illustration.

Computer Vision, CS766 Staff Instructor: Li Zhang TA: Yu-Chi Lai

DIGITAL CONTENT CREATION PROCESS fff PRE-PRODUCTION PRODUCTION POST-PRODUCTION Process ANIMATION PROCESS.

3DDI: 3D Direct Interaction John Canny Computer Science Division UC Berkeley.

MASKS © 2004 Invitation to 3D vision. MASKS © 2004 Invitation to 3D vision Lecture 1 Overview and Introduction.

CS559: Computer Graphics Lecture 36: Raytracing Li Zhang Spring 2008 Many Slides are from Hua Zhong at CUM, Paul Debevec at USC.

Fundamentals of Computer Animation Motion Synthesis.

Portable Camera-Based Assistive Text and Product Label Reading From Hand-Held Objects for Blind Persons.

3D Animation 1. Introduction Dr. Ashraf Y. Maghari Information Technology Islamic University of Gaza Ref. Book: The Art of Maya.

Nataliya Nadtoka James Edge, Philip Jackson, Adrian Hilton CVSSP Centre for Vision, Speech & Signal Processing UNIVERSITY OF SURREY.

Advanced Computer Graphics

Can computers match human perception?

N7 Graphic Communication

Joshua Barczak* CMSC 435 UMBC

Image Based Modeling and Rendering (PI: Malik)

Announcements Review on Thurs Project 4 Extra office hour: Friday 4-5

Assist. Lecturer Safeen H. Rasool Faculty of SCIENCE IT Dept.

Computer Graphics Lecture 15.

End-to-End Speech-Driven Facial Animation with Temporal GANs

Presentation transcript:

Computer Vision in Graphics Production Adrian Hilton Visual Media Research Group Centre for Vision, Speech and Signal Processing University of Surrey

Overview Where is computer vision useful in content production? Where is vision used? What can and can’t current computer vision do?

Computer Vision in Content Production Computer vision: analysis & interpretation of real images/video Match moving Matting Model building Human motion capture Studio Production Facial animation Image-based illumination

Match Moving Reconstruction of camera movement to composite CG Products: 2d3, RealViz … (semi-automatic camera tracking) … standard tool in film production Oxford University/2d3

Matting Separation of foreground and background objects - actor/background separation - wire/set removal Studio: chroma-key (solved) Post-production: Imagineer, RealViz, …. (open-problem)

Model Building Capture of real shape for CG models objects environments characters Shape capture active sensors (laser/structured-light) - accurate 3D surface measurements - static objects or environments structure from images - low-accuracy - static objects Problem: unstructured surface measurements

Model Building (2) Walking with Dinosaurs - FrameStore

Structured Models

Courtesy Stanford Computer Graphics Lab. Animation of David

Human Motion Capture Capture of real motion Marker based systems widely used in performance animation whole-body/face character animation (Golum) ‘realistic’ motion characteristics accurate real-time? widely used in film production (with post-production)

Human Motion Capture(2) Markerless human motion capture advantages: unintrusive; simultaneous appearance capture model-based visual tracking low-accuracy visual ambiguity (uniform apperance, non-rigid shape)

Human Motion Capture(3)

Studio Production Modelling actors 3D animated models animation re-illumination loss of visual realism to captured images Free-viewpoint video post-production of arbitrary camera views/paths ‘matrix’ flowmo shots aim: quality equivalent to captured video

Studio Production (2)

Limitations: - shape detail face/hands/hair - not video quality Studio Production (3)

Studio Production: Free-view video

Facial Animation Production of photo-realistic faces 2D video-based approaches video rewrite [Bregler’97] concatenative synthesis [Cossato’98,Ezzat’02] photo-realistic limited viewpoint, illumination 3D markers Performance animation [Phigin’98] shape only 3D video (shape+appearance) concatenative synthesis photo-realistic control of viewpoint, illumination

3D Video

3D Video Database 51 people: Expressions + Speech (short sentence)

Initial Face Synthesis from Speech Input: Output:

Image-based illumination Illumination of CG scenes with real illumination high-dynamic range capture of illumination environment mapping illumination to CG scene widely used in production Debevec SIGGRAPH’00 FiatLux

Computer Vision in Content Production Match moving - yes, automatic camera tracking available Matting - partial, many unsolved problems Model capture – yes, tools for semi-automatic restructuring Human motion capture – no, inaccurate Studio Production – partial, free viewpoint video Facial animation – yes, 3D video Image-based illumination – yes, widely used Other applications of computer vision….

Future Directions Video-based rendering photo-realistic, free-viewpoint rendering of dynamic scenes relighting required Video-based animation animation from captured video (face,whole-body…) control of motion, viewpoint, illumination

People Jon Starck, Joel Mitchelson, Gordon Collins, Eng-Jon Ong, Ioannis Ypsilos, Rob Dilks, Michael Kalkavouras Collaborators BBC, BT, Sony, Canon, Philips, Hensons, Framestore, Snell&Wilcox, 3D Scanners, AvatarMe EPSRC – DTI