Human Posture Recognition with Convex Programming Hao Jiang, Ze-Nian Li and Mark S. Drew School of Computing Science Simon Fraser University Burnaby, BC,

Slides:



Advertisements
Similar presentations
Shape Context and Chamfer Matching in Cluttered Scenes
Advertisements

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.
Solving LP Models Improving Search Special Form of Improving Search
Chamfer Distance for Handshape Detection and Recognition CSE 6367 – Computer Vision Vassilis Athitsos University of Texas at Arlington.
Presented by Xinyu Chang
Face Alignment with Part-Based Modeling
Database-Based Hand Pose Estimation CSE 6367 – Computer Vision Vassilis Athitsos University of Texas at Arlington.
3D Human Body Pose Estimation from Monocular Video Moin Nabi Computer Vision Group Institute for Research in Fundamental Sciences (IPM)
Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)
Intelligent Systems Lab. Recognizing Human actions from Still Images with Latent Poses Authors: Weilong Yang, Yang Wang, and Greg Mori Simon Fraser University,
Convex Quadratic Programming for Object Location Hao Jiang, Mark S. Drew and Ze-Nian Li School of Computing Science Simon Fraser University.
Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.
Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.
Robust Object Tracking via Sparsity-based Collaborative Model
Quadtrees, Octrees and their Applications in Digital Image Processing
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
Object Detection by Matching Longin Jan Latecki. Contour-based object detection Database shapes: …..
We propose a successive convex matching method to detect actions in videos. The proposed scheme does not need foreground/background separation, works in.
Shape from Contours and Multiple Stereo A Hierarchical, Mesh-Based Approach Hendrik Kück, Wolfgang Heidrich, Christian Vogelgsang.
Silhouettes in Multiview Stereo Ian Simon. Multiview Stereo Problem Input: – a collection of images of a rigid object (or scene) – camera parameters for.
Lecture 5 Template matching
Human Body Shape Estimation from Single Image Moin Nabi Computer Vision Lab. ©IPM - Dec
Tracking Objects with Dynamics Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 04/21/15 some slides from Amin Sadeghi, Lana Lazebnik,
1 Model Fitting Hao Jiang Computer Science Department Oct 6, 2009.
Real-time Embedded Face Recognition for Smart Home Fei Zuo, Student Member, IEEE, Peter H. N. de With, Senior Member, IEEE.
A Study of Approaches for Object Recognition
CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.
Processing Digital Images. Filtering Analysis –Recognition Transmission.
Segmentation Divide the image into segments. Each segment:
Quadtrees, Octrees and their Applications in Digital Image Processing
3D Hand Pose Estimation by Finding Appearance-Based Matches in a Large Database of Training Views
Chamfer Matching & Hausdorff Distance Presented by Ankur Datta Slides Courtesy Mark Bouts Arasanathan Thayananthan.
Object Detection and Tracking Mike Knowles 11 th January 2005
Recovering Articulated Object Models from 3D Range Data Dragomir Anguelov Daphne Koller Hoi-Cheung Pang Praveen Srinivasan Sebastian Thrun Computer Science.
CS335 Principles of Multimedia Systems Multimedia and Human Computer Interfaces Hao Jiang Computer Science Department Boston College Nov. 20, 2007.
Linear Solution to Scale and Rotation Invariant Object Matching Professor: 王聖智 教授 Student : 周 節.
כמה מהתעשייה? מבנה הקורס השתנה Computer vision.
Robust fitting Prof. Noah Snavely CS1114
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean Hall 5409 T-R 10:30am – 11:50am.
Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
3D Fingertip and Palm Tracking in Depth Image Sequences
Shape-Based Human Detection and Segmentation via Hierarchical Part- Template Matching Zhe Lin, Member, IEEE Larry S. Davis, Fellow, IEEE IEEE TRANSACTIONS.
Characterizing activity in video shots based on salient points Nicolas Moënne-Loccoz Viper group Computer vision & multimedia laboratory University of.
ALIGNMENT OF 3D ARTICULATE SHAPES. Articulated registration Input: Two or more 3d point clouds (possibly with connectivity information) of an articulated.
Project title : Automated Detection of Sign Language Patterns Faculty: Sudeep Sarkar, Barbara Loeding, Students: Sunita Nayak, Alan Yang Department of.
Processing Images and Video for an Impressionist Effect Author: Peter Litwinowicz Presented by Jing Yi Jin.
A New Fingertip Detection and Tracking Algorithm and Its Application on Writing-in-the-air System The th International Congress on Image and Signal.
Vision-based human motion analysis: An overview Computer Vision and Image Understanding(2007)
CS654: Digital Image Analysis Lecture 25: Hough Transform Slide credits: Guillermo Sapiro, Mubarak Shah, Derek Hoiem.
Efficient Visual Object Tracking with Online Nearest Neighbor Classifier Many slides adapt from Steve Gu.
Looking at people and Image-based Localisation Roberto Cipolla Department of Engineering Research team
DETECTING AND TRACKING TRACTOR-TRAILERS USING VIEW-BASED TEMPLATES Masters Thesis Defense by Vinay Gidla Apr 19,2010.
Supervisor: Nakhmani Arie Semester: Winter 2007 Target Recognition Harmatz Isca.
1 Scale and Rotation Invariant Matching Using Linearly Augmented Tree Hao Jiang Boston College Tai-peng Tian, Stan Sclaroff Boston University.
Linear Solution to Scale and Rotation Invariant Object Matching Hao Jiang and Stella X. Yu Computer Science Department Boston College.
Affine Registration in R m 5. The matching function allows to define tentative correspondences and a RANSAC-like algorithm can be used to estimate the.
Computer Vision Computer Vision based Hole Filling Chad Hantak COMP December 9, 2003.
Tracking Hands with Distance Transforms Dave Bargeron Noah Snavely.
Portable Camera-Based Assistive Text and Product Label Reading From Hand-Held Objects for Blind Persons.
Hough Transform CS 691 E Spring Outline Hough transform Homography Reading: FP Chapter 15.1 (text) Some slides from Lazebnik.
Deformation Modeling for Robust 3D Face Matching Xioguang Lu and Anil K. Jain Dept. of Computer Science & Engineering Michigan State University.
Traffic Sign Recognition Using Discriminative Local Features Andrzej Ruta, Yongmin Li, Xiaohui Liu School of Information Systems, Computing and Mathematics.
Tracking Objects with Dynamics
Object Matching Using a Locally Affine Invariant and Linear Programming Techniques - H. Li, X. Huang, L. He Ilchae Jung.
Real-Time Human Pose Recognition in Parts from Single Depth Image
Liyuan Li, Jerry Kah Eng Hoe, Xinguo Yu, Li Dong, and Xinqi Chu
Paper Reading Dalong Du April.08, 2011.
“Traditional” image segmentation
Presentation transcript:

Human Posture Recognition with Convex Programming Hao Jiang, Ze-Nian Li and Mark S. Drew School of Computing Science Simon Fraser University Burnaby, BC, V5A 1S6

July, 11 th, 2005Simon Fraser University Human Posture Recognition  Recognizing human postures is very important in vision and multimedia.  It has many applications in surveillance, human computer interaction, image and video database analysis and retrieval.  At the same time, recognizing human postures is a hard problem.

July, 11 th, 2005Simon Fraser University The Challenges of Human Posture Recognition  It is hard to recognize human postures because: –Articulated nature of a human body –No segmentation schemes are available for general images or videos. –Strong background clutters. –Large appearance changes because of clothing  Different schemes have been studied.

July, 11 th, 2005Simon Fraser University Methods for Posture Recognition  Methods having been studied: –Silhouette based method with background subtraction –Multi-camera based methods –Tracking body movement –Chamfer matching based schemes –Shape context based schemes  These methods are not sufficient to address the problem robustly.

July, 11 th, 2005Simon Fraser University The Proposed Method  We will present a matching based scheme that has the following properties: –Based on a robust convex (linear) programming matching scheme –Work for cases where no background subtraction is available –Able to deal with strong background clutters –Able to deal with large appearance changes

July, 11 th, 2005Simon Fraser University Matching Distance Transform Canny Edge Detection Distance Transform Feature Point Selection Delaunay Triangulation Template Generation Canny Edge Detection Distance Transform Matching With LP Template Image Target Image Object Recognition result

July, 11 th, 2005Simon Fraser University Matching as a Labeling Problem p q fpfp fqfq Target p’ Target q’ Clutter Target Template Mesh Target Image

July, 11 th, 2005Simon Fraser University The Labeling Problem  The matching problem can be formulated as the following optimization problem: Matching cost Smoothing term

July, 11 th, 2005Simon Fraser University Convex Relaxation  The original problem is a hard non-convex problem. We convert it to LP: |fp-fq| c’(s,j)

July, 11 th, 2005Simon Fraser University Properties of the Relaxation 1. For convex problems, LP exactly solves the continuous extension of the original problem. 2. For general non-convex problems, LP solves the problem where each matching surface is replaced by the lower convex hull. 3. The “cheapest” basis set for each site corresponds to the lower convex hull’s vertices

July, 11 th, 2005Simon Fraser University The Effect of Covexification For site 0 For site M-1 … c(0,j) c(M-1,j) Label :Basic Labels Convexification Label c(i,j) : Lower Convex Hull Vertices Label For non-convex problems, the relaxation replaces each c(m,j) by its lower convex hull surface:

July, 11 th, 2005Simon Fraser University Searching Scheme of Simplex Method 4. Using simplex method, there are at most three adjacent non-zero weight basis labels: : Zero-weight basis label : Non-zero-weight basis label : non basis label: Continuous label Searching for one site

July, 11 th, 2005Simon Fraser University Successive Relaxation Scheme  Single relaxation may miss the global optimum because of convexification effect  An intuitive scheme is to shrink the trust region and reconvexify the data in the smaller region  This scheme is found to be able to greatly improve the matching results

July, 11 th, 2005Simon Fraser University The Trust Region Shrinking

July, 11 th, 2005Simon Fraser University Successive Relaxation Scheme (An Example) min C(1,  1 )+ C(2,  2 )+0.5|  1 -  2 |

July, 11 th, 2005Simon Fraser University Shape Recognition  We have to define the goodness of matching –Matching cost (M): Average difference of the template and target image in the ROI. –Deformation (D): Affine transformation compensated pairwise distance changes –Shape Context in the ROI (C).  Finally, we use M + a*D+b*C to quantify the matching

July, 11 th, 2005Simon Fraser University Random Dots Experiment Noise: 50% Random Disturbance: 5 Noise: 100% Random Disturbance: 5 Noise: 50% Random Disturbance: 10 Noise: 100% Random Disturbance: 10

July, 11 th, 2005Simon Fraser University Matching Synthetic Images Results (a): Template model showing distance transform; (b): Matching result of proposed scheme; (c): Matching result by GC; (d): Matching result by ICM. (e): Matching result by BP. : GC : ICM : BP : LP

July, 11 th, 2005Simon Fraser University Matching Leaves

July, 11 th, 2005Simon Fraser University Experiment Results An example where traditional methods fail. (a): Template image; (b): Target image; (c): Edge map of template image; (d): Edge map of target image; (e): Template mesh; (f): Matching result of the proposed scheme; (g): ICM matching result; (h): Sliding template search result.

July, 11 th, 2005Simon Fraser University Gesture Recognition Results Template Top match Second match

July, 11 th, 2005Simon Fraser University Gesture Recognition Results

July, 11 th, 2005Simon Fraser University Video Browsing Result

July, 11 th, 2005Simon Fraser University Video Browsing Result

July, 11 th, 2005Simon Fraser University Multiple Target Matching Results

July, 11 th, 2005Simon Fraser University Conclusion and Future Directions  We present a robust matching framework for human posture recognition  The method can be applied to multimedia data retrieval in image or video database, or human computer interaction applications  In future work: –We will add tempera information for behavior recognition

July, 11 th, 2005Simon Fraser University Future work –The successive reconvexification is in fact very general. It can be used to increase the robustness of many other matching schemes, such as BP and GC –The proposed matching can be used for many other applications, such as tracking, object recongnition, motion estimation etc.

July, 11 th, 2005Simon Fraser University The End