01/18 Lab meeting Fabio Cuzzolin UCLA Vision Lab Department of Computer Science University of California at Los Angeles Los Angeles, January 18 2005

… past and present PhD student, University of Padova, Department of Computer Science (NAVLAB laboratory) with Ruggero Frezza Visiting student, ESSRL, Washington University in St. Louis Visiting student, UCLA, Los Angeles (VisionLab) Post-doc in Padova, Control and Systems Theory group Young researcher, Image and Sound Processing Group, Politecnico di Milano Post-doc, UCLA Vision Lab

… the research research Computer vision Discrete mathematics linear independence on lattices object and body tracking data association gesture and action recognition research Belief functions and imprecise probabilities geometric approach algebraic analysis combinatorial analysis

Upper and lower probabilities 1 Upper and lower probabilities

Past work Geometric approach to belief functions (ISIPTA’01, SMC-C-05) Algebra of families of frames (RSS’00, ISIPTA’01, AMAI’03) Geometry of Dempster’s rule (FSKD’02, SMC-B-04) Geometry of upper probabilities (ISIPTA’03, SMC-B-05) Simplicial complexes of fuzzy sets (IPMU’04)

The theory of belief functions

Uncertainty descriptions A number of theories have been proposed to extend or replace classical probability: possibilities, fuzzy sets, random sets, monotone capacities, etc. theory of evidence (A. Dempster, G. Shafer) belief functions Dempster’s rule families of frames

Motivations

Axioms and superadditivity probabilities additivity: if then belief functions 3. superadditivity

Example of b.f.

Belief functions belief functions s: 2Θ ->[0,1] A B1 ..where m is a mass function on 2Θ s.t. B2

Dempster’s rule b.f. are combined through Dempster’s rule AiÇBj=A Ai intersection of focal elements

Example of combination s1: m({a1})=0.7, m({a1 ,a2})=0.3 a1 a2 a3 a4 s2: m()=0.1, m({a2 ,a3 ,a4})=0.9 s1  s2 : m({a1})=0.19, m({a2})=0.73 m({a1 ,a2})=0.08

Bayes vs Dempster Belief functions generalize the Bayesian formalism as: 1- discrete probabilities are a special class of belief functions 2 - Bayes’ rule is a special case of Dempster’s rule 3 - a multi-domain representation of the evidence is contemplated

My research algebraic analysis geometric analysis Theory of evidence combinatorial analysis categorial? probabilistic analysis

Algebra of frames

.0 .1 .00 .01 .10 .11 .2 .3 .4 0.00 0.09 0.90 0.99 0.49 0.25 0.75 0.5 Family of frames refining Common refinement example: a function y Î [0,1] is quantized in three different ways 1

order relation: existence of a refining Lattice structure 1F maximal coarsening Q Å W Q W minimal refinement Q Ä W order relation: existence of a refining F is a locally Birkhoff (semimodular with finite length) lattice bounded below

Geometric approach to upper and lower probabilisties

Belief space the space of all the belief functions on a given frame each subset A  A-th coordinate s(A) in an Euclidean space it has the shape of a simplex

Geometry of Dempster’s rule constant mass loci foci of conditional subspaces Dempster’s rule can be studied in the geometric setup too

Geometry of upper probs the space of plausibilities is also a simplex

Belief and probabilities study of the geometric interplay of belief and probability

Consistent probabilities Each belief function is associated with a set of consistent probabilities, forming a simplex in the probabilistic subspace the vertices of the simplex are the probabilities assigning the mass of each focal element of s to one of its points the center of mass of P(s) coincides with Smets’ pignistic function

Possibilities in a geometric setup possibility measures are a class of belief functions they have the geometry of a simplicial complex

Combinatorial analysis

Total belief theorem generalization of the total probability theorem a-priori constraint conditional constraint

Existence candidate solution: linear system nn where the columns of A are the focal elements of stot problem: choosing n columns among m s.t. x has positive components method: replacing columns through

Solution graphs all the candidate solutions form a graph Edges = linear transformations

New goals... algebraic analysis geometric analysis Theory of evidence combinatorial analysis probabilistic analysis ?

Approximations problem: finding an approximation of s compositional criterion the approximation behaves like s when combined through Dempster probabilistic and fuzzy approximations

Indipendence and conflict s1,…, sn are not always combinable 1,…, n are indipendent if any s1,…, sn are combinable  are defined on independent frames

Pseudo Gram-Schmidt Vector spaces and frames are both semimodular lattices -> admit independence pseudo Gram-Schmidt new set of b.f. surely combinable

Canonical decomposition unique decomposition of s into simple b.f. convex geometry can be used to find it

Tracking of rigid bodies data association of points belonging to a rigid body m-1m past and present target association old estimates Kalman filters Am-1 past targets - model associations  m-1m Am-1  = Am-1  m-1m Am-1 () rigid motion constraints Am current targets – model association Am new estimates rigid motion constraints can be written as conditional belief functions  total belief needed

Total belief problem and combinatorics general proof, number of solutions, symmetries of the graph relation with positive linear systems homology of solution graphs matroidal interpretation

2 Computer vision

Vision problems HMM and size functions for gesture recognition (BMVC’97) object tracking and pose estimation (MTNS’98,SPIE’99, MTNS’00, PAMI’04) composition of HMMs (ASILOMAR’02) data association with shape info (CDC’02, CDC’04, PAMI’05) volumetric action recognition (ICIP’04,MMSP’04)

Size functions for gesture recognition

Size functions for gesture recognition Combination of HMMs (for dynamics) and size functions (for pose representation)

Size functions “Topological” representation of contours

Measuring functions Functions defined on the contour of the shape of interest real image family of lines measuring function

Feature vectors a family of measuring functions is chosen … the szfc are computed, and their means form the feature vector

Hidden Markov models Finite-state model of gestures as sequences of a small number of poses

Four-state HMM Gesture dynamics -> transition matrix A Object poses -> state-output matrix C

EM algorithm learning the model’s parameters through EM two instances of the same gesture feature matrices: collection of feature vectors along time A,C EM learning the model’s parameters through EM

Compositional behavior of Hidden Markov models

Composition of HMMs Compositional behavior of HMMS: the model of the action of interest is embedded in the overall model Example: “fly” gesture in clutter

State clustering Effect of clustering on HMM topology “Cluttered” model for the two overlapping motions Reduced model for the “fly” gesture extracted through clustering

Kullback-Leibler comparison We used the K-L distance to measure the similarity between models extracted from clutter and in absence of clutter

Model-free object pose estimation

Model-free pose estimation Pose estimation: inferring the configuration of a moving object from one or more image sequences Most approaches in the literature are model-based: they assume some knowledge about the nature of the body (articulated, deformable, etc) and some sort of model T=0 t=T

Model-free scenario Scenario: the configuration of an uknown object is desired, given no a-priori information about the nature itself of the body The only info available is carried by the images We need to build a map from image measurements to body poses This can be done by using a learning technique, based on training data

1 3 2 “Evidential” model approximate feature spaces feature-pose maps (refinings) 2 training set of sample poses The evidential model is built during the training stage, when the feature-pose maps are learned

2 Feature extraction 1 3 1 From the blurred image the region with color similar to the region of interest is selected, and the bounding box is detected. 2 3

Estimates from the combined model Ground truth versus estimates... ... for two components of the pose

JPDA with shape information for data association

JPDA with shape info JPDA model: independent targets Shape model: rigid links Dempster’s fusion robustness: clutter does not meet shape constraints occlusions: occluded targets can be estimated

Triangle simulation the clutter affects only the standard JPDA estimates

Body tracking Application: tracking of feature points on a moving human body

Volumetric action recognition

Volumetric action recognition problem: recognizing the action performed by a person viewed by a number of cameras step 1: modeling the dynamics of the motion step 2: extracting image features 2D approaches: features are extracted from single views -> viewpoint dependence volumetric approach: features are extracted from a volumetric reconstruction of the moving body

Multiple sequences synchronized views from different cameras, chromakeying

Volumetric intersection more views -> more details silhouette extraction of the moving object from all views 3D object shape reconstruction through intersection of occlusion cones

k-means clustering to separate bodyparts 3D feature extraction locations of torso, arms, and legs of the moving person k-means clustering to separate bodyparts

two instances of the action “walking” Feature matrices X Y TORSO COORDINATES Z ABDOMEN COORDINATES RIGHT LEG COORDINATES LEFT LEG COORDINATES two instances of the action “walking”

Modeling and recognition HMM 1 HMM 2 … HMM n model of the “walking” action classification: each new feature matrix is fed to all the learnt models, generating a set of likelihoods

3 Combinatorics

Independence on lattices three distinct independence relations LI3 LI2=LI3 LI1 semimodular lattice modular lattice LI2 modularity  equivalent formulations

Scheme of the proof

4 Conclusions

CONDITIONALCONSTRAINTS from real to abstract OBJECT TRACKING DATA ASSOCIATION MEASUREMENTCONFLICT POINTWISE ESTIMATE CONDITIONALCONSTRAINTS ALGEBRAICANALYSIS GEOMETRICAPPROACH TOTAL BELIEF the solution of real problems stimulates new theoretical issues

…concluding the ToE comes from a strong critics to the Bayesian framework useful for sensor fusion problems under incomplete information real problem solutions stimulate the extension of the formalism complex objects  mathematically rich Young theory  need completion

In the near future.. search for a metric on the space of dynamical systems – stochastic models sistematic description of the geometric approach to non-additive measures understand the intricate relations between probability and combinatorics