Tracking Hands with Distance Transforms Dave Bargeron Noah Snavely.

Slides:



Advertisements
Similar presentations
Distinctive Image Features from Scale-Invariant Keypoints
Advertisements

Shape Context and Chamfer Matching in Cluttered Scenes
1 Hierarchical Part-Based Human Body Pose Estimation * Ramanan Navaratnam * Arasanathan Thayananthan Prof. Phil Torr * Prof. Roberto Cipolla * University.
CSCE643: Computer Vision Bayesian Tracking & Particle Filtering Jinxiang Chai Some slides from Stephen Roth.
Chamfer Distance for Handshape Detection and Recognition CSE 6367 – Computer Vision Vassilis Athitsos University of Texas at Arlington.
Change Detection C. Stauffer and W.E.L. Grimson, “Learning patterns of activity using real time tracking,” IEEE Trans. On PAMI, 22(8): , Aug 2000.
Lecture 31: Modern object recognition
Database-Based Hand Pose Estimation CSE 6367 – Computer Vision Vassilis Athitsos University of Texas at Arlington.
IIIT Hyderabad Pose Invariant Palmprint Recognition Chhaya Methani and Anoop Namboodiri Centre for Visual Information Technology IIIT, Hyderabad, INDIA.
Cambridge, Massachusetts Pose Estimation in Heavy Clutter using a Multi-Flash Camera Ming-Yu Liu, Oncel Tuzel, Ashok Veeraraghavan, Rama Chellappa, Amit.
Lecture Pose Estimation – Gaussian Process Tae-Kyun Kim 1 EE4-62 MLCV.
3D Shape Representation Tianqiang 04/01/2014. Image/video understanding Content creation Why do we need 3D shapes?
Silhouette Lookup for Automatic Pose Tracking N ICK H OWE.
Foreground Modeling The Shape of Things that Came Nathan Jacobs Advisor: Robert Pless Computer Science Washington University in St. Louis.
Forward-Backward Correlation for Template-Based Tracking Xiao Wang ECE Dept. Clemson University.
Automatic in vivo Microscopy Video Mining for Leukocytes * Chengcui Zhang, Wei-Bang Chen, Lin Yang, Xin Chen, John K. Johnstone.
Segmentation-Free, Area-Based Articulated Object Tracking Daniel Mohr, Gabriel Zachmann Clausthal University, Germany ISVC.
Discrete-Continuous Optimization for Large-scale Structure from Motion David Crandall, Andrew Owens, Noah Snavely, Dan Huttenlocher Presented by: Rahul.
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
Deformable Contours Dr. E. Ribeiro.
Simultaneous Segmentation and 3D Pose Estimation of Humans or Detection + Segmentation = Tracking? Philip H.S. Torr Pawan Kumar, Pushmeet Kohli, Matt Bray.
Edge Detection CSE P 576 Larry Zitnick
1 Model Fitting Hao Jiang Computer Science Department Oct 6, 2009.
Lecture 6: Feature matching CS4670: Computer Vision Noah Snavely.
Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos Carnegie Mellon University June 16, 2007 Peter Barnum Takeo Kanade Srinivasa.
A Study of Approaches for Object Recognition
Feature matching and tracking Class 5 Read Section 4.1 of course notes Read Shi and Tomasi’s paper on.
Robust Lane Detection and Tracking
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
3D Hand Pose Estimation by Finding Appearance-Based Matches in a Large Database of Training Views
Chamfer Matching & Hausdorff Distance Presented by Ankur Datta Slides Courtesy Mark Bouts Arasanathan Thayananthan.
Lecture 11: Structure from motion CS6670: Computer Vision Noah Snavely.
Recognizing and Tracking Human Action Josephine Sullivan and Stefan Carlsson.
Object Recognition Using Geometric Hashing
Feature and object tracking algorithms for video tracking Student: Oren Shevach Instructor: Arie nakhmani.
3D Fingertip and Palm Tracking in Depth Image Sequences
Geodesic Minimal Paths Vida Movahedi Elder Lab, January 2010.
Multimodal Interaction Dr. Mike Spann
Human-Computer Interaction Human-Computer Interaction Tracking Hanyang University Jong-Il Park.
3D SLAM for Omni-directional Camera
International Conference on Computer Vision and Graphics, ICCVG ‘2002 Algorithm for Fusion of 3D Scene by Subgraph Isomorphism with Procrustes Analysis.
Detecting Curved Symmetric Parts using a Deformable Disc Model Tom Sie Ho Lee, University of Toronto Sanja Fidler, TTI Chicago Sven Dickinson, University.
Vision-based human motion analysis: An overview Computer Vision and Image Understanding(2007)
Learning the Appearance and Motion of People in Video Hedvig Sidenbladh, KTH Michael Black, Brown University.
Whiteboard Shape Recognition using Deformable Templates and Loopy Belief Propogation Noah Snavely David Bargeron April 2004.
MSRI workshop, January 2005 Object Recognition Collected databases of objects on uniform background (no occlusions, no clutter) Mostly focus on viewpoint.
Efficient Matching of Pictorial Structures By Pedro Felzenszwalb and Daniel Huttenlocher Presented by John Winn.
Sparse Bayesian Learning for Efficient Visual Tracking O. Williams, A. Blake & R. Cipolloa PAMI, Aug Presented by Yuting Qi Machine Learning Reading.
Looking at people and Image-based Localisation Roberto Cipolla Department of Engineering Research team
Pictorial Structures and Distance Transforms Computer Vision CS 543 / ECE 549 University of Illinois Ian Endres 03/31/11.
CSE 185 Introduction to Computer Vision Feature Matching.
Lecture 9 Feature Extraction and Motion Estimation Slides by: Michael Black Clark F. Olson Jean Ponce.
Announcements No midterm Project 3 will be done in pairs same partners as for project 2.
Strong Supervision From Weak Annotation Interactive Training of Deformable Part Models ICCV /05/23.
Level Set Segmentation ~ 9.37 Ki-Chang Kwak.
Fast Human Detection in Crowded Scenes by Contour Integration and Local Shape Estimation Csaba Beleznai, Horst Bischof Computer Vision and Pattern Recognition,
Learning Image Statistics for Bayesian Tracking Hedvig Sidenbladh KTH, Sweden Michael Black Brown University, RI, USA
Lecture 26 Hand Pose Estimation Using a Database of Hand Images
LOCUS: Learning Object Classes with Unsupervised Segmentation
Dynamical Statistical Shape Priors for Level Set Based Tracking
Structure from motion Input: Output: (Tomasi and Kanade)
Object recognition Prof. Graeme Bailey
PRAKASH CHOCKALINGAM, NALIN PRADEEP, AND STAN BIRCHFIELD
Structure from Motion with Non-linear Least Squares
Kiera Henning, Jiwon Kim,
CSE 185 Introduction to Computer Vision
Flexible templates, density estimation, mean shift
Lecture 5: Feature invariance
Structure from motion Input: Output: (Tomasi and Kanade)
Structure from Motion with Non-linear Least Squares
Presentation transcript:

Tracking Hands with Distance Transforms Dave Bargeron Noah Snavely

The problem Input: A video with a (rigid) hand Output: A sequence of hand locations and orientations

Approach 1.Generate hand templates for all possible orientations 2.Find the edges in each input image 3.For each edge image, find the template and location which minimizes the chamfer distance

Step 1: Generate Templates 1.Create 3D hand model 2.Render in a set of orientations 3.Use depth buffer to find silhouette and contours

Steps 2 + 3: Find the hand 1.Compute the distance transform of the edge image 2.Slide each template over the distance transform, compute the chamfer distance 3.Pick the template with the minimum chamfer distance

Problems Large number of templates –(3211 rotations) x (3072 translations) x (5 scales) = 49,320,960 templates In a cluttered image: –Chamfer distance has many local optima –Global optimum may not be correct Solving each frame separately not a good idea

Solution Part 1: Template Tree Coarse-to-fine search in parameter space

Solution Part 2: Tracking Detect the hand in frame 0 For each frame k > 0: –Compute the most likely transition from state in frame k-1 Use chamfer distance as a likelihood Use transition probability (assumed Gaussian) as a prior –Use transition probabilities to prune branches of the search tree

Results – Hand Detection InputEdge imageDistance transformOutput

Results – Video

Edge ImagesDistance Transforms

Results – Video

Extensions Better tracking –Use color in addition to shape –Use edge orientations –More templates; allow for on-line generation for refinement More flexible tracking –Track deformable hand –Automatically determine hand parameters (e.g. finger length)

References Björn Stenger’s Ph.D thesis – B. Stenger, et. al. “Filtering Using a Tree-Based Estimator.” ICCV Pedro Felzenszwalb and Dan Huttenlocher. “Distance Trasforms of Sampled Functions.”