Silhouette Segmentation in Multiple Views Wonwoo Lee, Woontack Woo, and Edmond Boyer PAMI, VOL. 33, NO. 7, JULY 2011 Donguk Seo

Slides:

Advertisements

Similar presentations

A Robust Super Resolution Method for Images of 3D Scenes Pablo L. Sala Department of Computer Science University of Toronto.

Advertisements

1 Approximated tracking of multiple non-rigid objects using adaptive quantization and resampling techniques. J. M. Sotoca 1, F.J. Ferri 1, J. Gutierrez.

Patch to the Future: Unsupervised Visual Prediction

Learning to estimate human pose with data driven belief propagation Gang Hua, Ming-Hsuan Yang, Ying Wu CVPR 05.

Robust Foreground Detection in Video Using Pixel Layers Kedar A. Patwardhan, Guillermoo Sapire, and Vassilios Morellas IEEE TRANSACTION ON PATTERN ANAYLSIS.

Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor ：王聖智教授 Student ：周節.

Multiple People Detection and Tracking with Occlusion Presenter: Feifei Huo Supervisor: Dr. Emile A. Hendriks Dr. A. H. J. Stijn Oomes Information and.

Texture Segmentation Based on Voting of Blocks, Bayesian Flooding and Region Merging C. Panagiotakis (1), I. Grinias (2) and G. Tziritas (3)

GrabCut Interactive Image (and Stereo) Segmentation Joon Jae Lee Keimyung University Welcome. I will present Grabcut – an Interactive tool for foreground.

Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Saad M. Khan and Mubarak Shah, PAMI, VOL. 31, NO. 3, MARCH 2009, Donguk Seo

Shape from Contours and Multiple Stereo A Hierarchical, Mesh-Based Approach Hendrik Kück, Wolfgang Heidrich, Christian Vogelgsang.

Learning to Detect A Salient Object Reporter: 鄭綱 (3/2)

Structure from motion.

Boundary matting for view synthesis Samuel W. Hasinoff Sing Bing Kang Richard Szeliski Computer Vision and Image Understanding 103 (2006) 22–32.

Segmentation and Tracking of Multiple Humans in Crowded Environments Tao Zhao, Ram Nevatia, Bo Wu IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,

A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University.

1 Image-Based Visual Hulls Paper by Wojciech Matusik, Chris Buehler, Ramesh Raskar, Steven J. Gortler and Leonard McMillan [

Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.

3D Hand Pose Estimation by Finding Appearance-Based Matches in a Large Database of Training Views

Synchronization and Calibration of Camera Networks from Silhouettes Sudipta N. Sinha Marc Pollefeys University of North Carolina at Chapel Hill, USA.

Tracking Video Objects in Cluttered Background

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 11, NOVEMBER 2011 Qian Zhang, King Ngi Ngan Department of Electronic Engineering, the Chinese university.

Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.

Structure Computation. How to compute the position of a point in 3- space given its image in two views and the camera matrices of those two views Use.

Manhattan-world Stereo Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.

Image-Based Visual Hulls Wojciech Matusik Chris Buehler Ramesh Raskar Steven Gortler Leonard McMillan Presentation by: Kenton McHenry.

What Does the Scene Look Like From a Scene Point? Donald Tanguay August 7, 2002 M. Irani, T. Hassner, and P. Anandan ECCV 2002.

Finish Adaptive Space Carving Anselmo A. Montenegro †, Marcelo Gattass ‡, Paulo Carvalho † and Luiz Velho † †

A Thousand Words in a Scene P. Quelhas, F. Monay, J. Odobez, D. Gatica-Perez and T. Tuytelaars PAMI, Sept

Prakash Chockalingam Clemson University Non-Rigid Multi-Modal Object Tracking Using Gaussian Mixture Models Committee Members Dr Stan Birchfield (chair)

Epipolar geometry The fundamental matrix and the tensor

1 Mean shift and feature selection ECE 738 course project Zhaozheng Yin Spring 2005 Note: Figures and ideas are copyrighted by original authors.

1. Introduction Motion Segmentation The Affine Motion Model Contour Extraction & Shape Estimation Recursive Shape Estimation & Motion Estimation Occlusion.

A General Framework for Tracking Multiple People from a Moving Camera

Object Stereo- Joint Stereo Matching and Object Segmentation Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on Michael Bleyer Vienna.

#MOTION ESTIMATION AND OCCLUSION DETECTION #BLURRED VIDEO WITH LAYERS

Experiments Test different parking lot images captured in different luminance conditions The test samples include 1300 available parking spaces and 1500.

18 th August 2006 International Conference on Pattern Recognition 2006 Epipolar Geometry from Two Correspondences Michal Perďoch, Jiří Matas, Ondřej Chum.

Chapter 5 Multi-Cue 3D Model- Based Object Tracking Geoffrey Taylor Lindsay Kleeman Intelligent Robotics Research Centre (IRRC) Department of Electrical.

CVPR2013 Poster Detecting and Naming Actors in Movies using Generative Appearance Models.

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.

Final Review Course web page: vision.cis.udel.edu/~cv May 21, 2003  Lecture 37.

A Dynamic Conditional Random Field Model for Object Segmentation in Image Sequences Duke University Machine Learning Group Presented by Qiuhua Liu March.

776 Computer Vision Jan-Michael Frahm & Enrique Dunn Spring 2013.

Fast Semi-Direct Monocular Visual Odometry

Journal of Visual Communication and Image Representation

MASKS © 2004 Invitation to 3D vision Uncalibrated Camera Chapter 6 Reconstruction from Two Uncalibrated Views Modified by L A Rønningen Oct 2008.

Reconstruction from Two Calibrated Views Two-View Geometry

Principal Axis-Based Correspondence between Multiple Cameras for People Tracking Dongwook Seo

Improving Image Matting using Comprehensive Sampling Sets CVPR2013 Oral.

DISCRIMINATIVELY TRAINED DENSE SURFACE NORMAL ESTIMATION ANDREW SHARP.

Media Processor Lab. Media Processor Lab. Trellis-based Parallel Stereo Matching Media Processor Lab. Sejong univ.

Learning and Removing Cast Shadows through a Multidistribution Approach Nicolas Martel-Brisson, Andre Zaccarin IEEE TRANSACTIONS ON PATTERN ANALYSIS AND.

Detecting Moving Objects, Ghosts, and Shadows in Video Streams

Robot Vision SS 2011 Matthias Rüther / Matthias Straka 1 ROBOT VISION Lesson 7: Volumetric Object Reconstruction Matthias Straka.

Student Gesture Recognition System in Classroom 2.0 Chiung-Yao Fang, Min-Han Kuo, Greg-C Lee, and Sei-Wang Chen Department of Computer Science and Information.

3D Single Image Scene Reconstruction For Video Surveillance Systems

C. Canton1, J.R. Casas1, A.M.Tekalp2, M.Pardàs1

A Forest of Sensors: Using adaptive tracking to classify and monitor activities in a site Eric Grimson AI Lab, Massachusetts Institute of Technology

René Vidal and Xiaodong Fan Center for Imaging Science

Nonparametric Semantic Segmentation

Mauricio Hess-Flores1, Mark A. Duchaineau2, Kenneth I. Joy3

Vehicle Segmentation and Tracking from a Low-Angle Off-Axis Camera

Eric Grimson, Chris Stauffer,

PRAKASH CHOCKALINGAM, NALIN PRADEEP, AND STAN BIRCHFIELD

Silhouette Intersection

Filtering Things to take away from this lecture An image as a function

Video Compass Jana Kosecka and Wei Zhang George Mason University

Filtering An image as a function Digital vs. continuous images

Presentation transcript:

Silhouette Segmentation in Multiple Views Wonwoo Lee, Woontack Woo, and Edmond Boyer PAMI, VOL. 33, NO. 7, JULY 2011 Donguk Seo

2 Intelligent Systems Lab. Introduction Extraction of consistent foreground regions from multi views without a priori knowledge of the background Two assumptions for proposed method The region of interest appears entirely in all images. Background colors are consistent in each image.

3 Intelligent Systems Lab. Probabilistic model

4 Intelligent Systems Lab. Joint probability (1)

5 Intelligent Systems Lab. Spatial consistency term(1/4)

6 Intelligent Systems Lab. Spatial consistency term(2/4)

7 Intelligent Systems Lab. Spatial consistency term(3/4) (2)

8 Intelligent Systems Lab. Spatial consistency term(4/4) (3)

9 Intelligent Systems Lab. Image likelihood term (4) (5)

10 Intelligent Systems Lab. Inference of the silhouettes (6)

11 Intelligent Systems Lab. Iterative silhouette estimation Two assumptions  Any foreground element has an appearance different from the background in most images so that color segmentation positively detects the element in most images.  The region of interest appears entirely in all of the images considered. Iterative opitmization  Silhouettes are estimated using foreground and background models. (spatial and color consistencies)  These model are updated with the new silhouettes.

12 Intelligent Systems Lab. Initialization Foreground scene: observed by all cameras Belongs to the 3D space region that is visible from all cameras The background color model

13 Intelligent Systems Lab. Iterative optimization via Graph cut(1/2)

14 Intelligent Systems Lab. Iterative optimization via Graph cut(2/2) (7)

15 Intelligent Systems Lab. Silhouette refinement (8) (a)Input image. (b) Silhouette after the iterative optimization (c) Silhouette after refinements

16 Intelligent Systems Lab. Experimental results Synthetic data Kung-fu girl data set:

17 Intelligent Systems Lab. Kung-fu girl data (1/2) Segmentation results with different numbers of views (top row) accounting for spatial consistencies (bottom row) Only using background color consistency Two viewsfour viewsSix viewsOne view

18 Intelligent Systems Lab. Kung-fu girl data (2/2) Proposed method

19 Intelligent Systems Lab. Real data(1/7) Basic calibration GPU-based SIFT

20 Intelligent Systems Lab. Real data(2/7) (10 views) (12 views) (5 views) (12 views) (8 views) (6 views) (8 views)

21 Intelligent Systems Lab. Real data(3/7) Table 1. Silhouette extraction performance measurements (9)

22 Intelligent Systems Lab. Real data(4/7) (a) Only one object is spatially consistent. (6 views) (b) All three objects are spatially consistent. (6 views)

23 Intelligent Systems Lab. Real data(5/7)

24 Intelligent Systems Lab. Real data(6/7)

25 Intelligent Systems Lab. Real data(7/7)

26 Intelligent Systems Lab. Conclusions A novel method for extracting spatially consistent silhouettes of foreground objects from several viewpoints Using spatial consistency and color consistency constraints in order to identify silhouettes with unknown backgrounds The assumption Foreground objects are seen by all images and they present color differences with the background regions.

27 Intelligent Systems Lab.

28 Intelligent Systems Lab. Silhouette calibration ratio

29 Intelligent Systems Lab. Visual hull

30 Intelligent Systems Lab. Epipolar geometry 3D point Center of projection of camera Projection of point X Epipole Epipolar plane Epipolar line