Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA

Myself gesture recognition Masters thesis on gesture recognition at the University of Padova theory of evidence Ph.D. thesis on the theory of evidence Post-doc in Milan with the Image and Sound Processing group Post-doc at UCLA in the Vision Lab

My past work… geometric approach geometric approach to the theory of belief functions space of belief functions geometry of Dempsters rule

.. again.. algebra of compatible frames linear independence on lattices action recognition and object tracking metrics on the space of dynamical models

… and todays talk the pose estimation problem model-free pose estimation evidential model experimental results

Pose estimation pose estimating the pose (internal configuration) of a moving body from the available images features salient image measurements: features t=0t=T CAMERA t=0t=T

Model-based estimation a-priori model if you have an a-priori model of the object.... you can exploit it to help (or drive) the estimation example: kinematic model

Model-free estimation if you do not have any information about the body.. the only way to do inference is to learn a map learn a map between features and poses directly from the data training stage this can be done in a training stage

Collecting training data motion capture system 3D locations of markers = pose

Training data when the object performs some significant movements in front of the camera … … a finite collection of configuration values are provided by the motion capture system … while a sequence of features is computed from the image(s) q q yy 1 1 T T

Learning feature-pose maps Hidden Markov models Hidden Markov models provide a way to build feature-pose maps from the training data approximate feature space a Gaussian density for each state is set up on the feature space -> approximate feature space map map between each region and the set of training poses q k with feature value y k inside it

Evidential model approximate feature spaces.... and approximate parameter space.. family of compatible frames: the evidential model.. form a family of compatible frames: the evidential model

Estimation these belief functions are projected onto the approximate parameter space.... and combined through Dempsters rule a point-wise estimate of the pose is obtained by probabilistic approximation new features are represented as belief functions..

Human body tracking two experiments, two views four markers on the right arm six markers on both legs

Feature extraction three steps: original image, color segmentation, bounding box

Performances comparison of three models: left view only, right view only, both views pose estimation yielded by the overall model estimate associated with the right model left model ground truth

Estimation errors Euclidean distance between real and predicted marker position marker 4 3cm marker 2 8cm

Visual estimate compares the actual image with the weighted sum of the training images

Conclusions pose estimation of unknown objects is a difficult task a bottom-up model has to be built from the data in a training session the DS framework allows to formalize the idea of feature-pose maps in a natural way through the notion of compatible frames Dempsters combination provides a method to integrate features to increase robustness