1. 01/18 Lab meeting Fabio Cuzzolin
UCLA Vision Lab
Department of Computer Science
University of California at Los Angeles
Los Angeles, January

2. … past and present
PhD student, University of Padova, Department of Computer
Science (NAVLAB laboratory) with Ruggero Frezza
Visiting student, ESSRL, Washington University in St. Louis
Visiting student, UCLA, Los Angeles (VisionLab)
Post-doc in Padova, Control and Systems Theory group
Young researcher, Image and Sound Processing Group,
Politecnico di Milano
Post-doc, UCLA Vision Lab

3. … the research research Computer vision Discrete mathematics
linear independence on lattices
object and body tracking
data association
gesture and action recognition
research
Belief functions and imprecise probabilities
geometric approach
algebraic analysis
combinatorial analysis

4. Upper and lower probabilities1
Upper and lower probabilities

5. Past work Geometric approach to belief functions (ISIPTA’01, SMC-C-05)
Algebra of families of frames (RSS’00, ISIPTA’01, AMAI’03)
Geometry of Dempster’s rule (FSKD’02, SMC-B-04)
Geometry of upper probabilities (ISIPTA’03, SMC-B-05)
Simplicial complexes of fuzzy sets (IPMU’04)

6. The theory of belief functions

7. Uncertainty descriptions
A number of theories have been proposed to extend or replace classical probability: possibilities, fuzzy sets, random sets, monotone capacities, etc.
theory of evidence (A. Dempster, G. Shafer)
belief functions
Dempster’s rule
families of frames

8. Motivations

9. Axioms and superadditivity
probabilities
additivity: if then
belief functions
3. superadditivity

10. Example of b.f.

11. Belief functions belief functions s: 2Θ ->[0,1]A B1
..where m is a mass function on 2Θ s.t. B2

12. Dempster’s rule b.f. are combined through Dempster’s rule AiÇBj=A Ai
intersection of focal elements

13. Example of combination
s1:
m({a1})=0.7, m({a1 ,a2})=0.3 a1 a2 a3 a4
s2:
m()=0.1, m({a2 ,a3 ,a4})=0.9
s1  s2 :
m({a1})=0.19, m({a2})=0.73
m({a1 ,a2})=0.08

14. Bayes vs Dempster
Belief functions generalize the Bayesian formalism as:
1- discrete probabilities are a special class of belief functions
2 - Bayes’ rule is a special case of Dempster’s rule
3 - a multi-domain representation of the evidence is contemplated

15. My research algebraic analysis geometric analysis Theory of evidence
combinatorial analysis
categorial?
probabilistic analysis

16. Algebra of frames

17. .0 .1
.00
.01
.10
.11 .2 .3 .4
0.49
0.25
0.75
0.5
Family of frames
refining
Common refinement
example: a function y Î [0,1] is quantized in three different ways 1

18. order relation: existence of a refining
Lattice structure 1F
maximal coarsening
Q Å W Q W
minimal refinement
Q Ä W
order relation: existence of a refining
F is a locally Birkhoff (semimodular with finite length) lattice bounded below

19. Geometric approach to upper and lower probabilisties

20. Belief space the space of all the belief functions on a given frame
each subset A  A-th coordinate s(A) in an Euclidean space
it has the shape of a simplex

21. Geometry of Dempster’s rule
constant mass loci
foci of conditional
subspaces
Dempster’s rule can be studied in the geometric setup too

22. Geometry of upper probs
the space of plausibilities
is also a simplex

23. Belief and probabilities
study of the geometric interplay of belief and probability

24. Consistent probabilities
Each belief function is associated with a set of consistent probabilities, forming a simplex in the probabilistic subspace
the vertices of the simplex are the probabilities assigning the mass of each focal element of s to one of its points
the center of mass of P(s) coincides with Smets’ pignistic function

25. Possibilities in a geometric setup
possibility measures are a class of belief functions
they have the geometry of a simplicial complex

26. Combinatorial analysis

27. Total belief theorem generalization of the total probability theorem
a-priori constraint
conditional constraint

28. Existence candidate solution: linear system nn
where the columns of A are the focal elements of stot
problem: choosing n columns among m s.t. x has positive components
method: replacing columns through

29. Solution graphs all the candidate solutions form a graph
Edges = linear transformations

30. New goals... algebraic analysis geometric analysis Theory of evidence
combinatorial analysis
probabilistic analysis ?

31. Approximations problem: finding an approximation of s
compositional criterion
the approximation behaves like s when combined through Dempster
probabilistic and fuzzy approximations

32. Indipendence and conflict
s1,…, sn are not always combinable
1,…, n are indipendent if
any s1,…, sn are combinable  are defined on independent frames

33. Pseudo Gram-Schmidt
Vector spaces and frames are both semimodular lattices -> admit independence
pseudo Gram-Schmidt
new set of b.f. surely combinable

34. Canonical decomposition
unique decomposition of s into simple b.f.
convex geometry can be used to find it

35. Tracking of rigid bodies
data association of points belonging to a rigid body
m-1m
past and present target association
old estimates
Kalman filters
Am-1
past targets - model associations 
m-1m
Am-1
 = Am-1  m-1m
Am-1 ()
rigid motion constraints
Am
current targets – model association
Am
new estimates
rigid motion constraints can be written as conditional belief functions  total belief needed

36. Total belief problem and combinatorics
general proof, number of solutions, symmetries of the graph
relation with positive linear systems
homology of solution graphs
matroidal interpretation

37. 2
Computer vision

38. Vision problems
HMM and size functions for gesture recognition (BMVC’97)
object tracking and pose estimation (MTNS’98,SPIE’99, MTNS’00, PAMI’04)
composition of HMMs (ASILOMAR’02)
data association with shape info (CDC’02, CDC’04, PAMI’05)
volumetric action recognition (ICIP’04,MMSP’04)

39. Size functions for gesture recognition

40. Size functions for gesture recognition
Combination of HMMs (for dynamics) and size functions (for pose representation)

41. Size functions
“Topological” representation of contours

42. Measuring functions
Functions defined on the contour of the shape of interest
real image
family of lines
measuring function

43. Feature vectors a family of measuring functions is chosen
… the szfc are computed, and their means form the feature vector

44. Hidden Markov models
Finite-state model of gestures as sequences of a small number of poses

45. Four-state HMM Gesture dynamics -> transition matrix A
Object poses -> state-output matrix C

46. EM algorithm learning the model’s parameters through EM
two instances of the same gesture
feature matrices: collection of feature vectors along time
A,C EM
learning the model’s parameters through EM

47. Compositional behavior of Hidden Markov models

48. Composition of HMMs
Compositional behavior of HMMS: the model of the action of interest is embedded in the overall model
Example: “fly” gesture in clutter

49. State clustering Effect of clustering on HMM topology
“Cluttered” model for the two overlapping motions
Reduced model for the “fly” gesture extracted through clustering

50. Kullback-Leibler comparison
We used the K-L distance to measure the similarity between models extracted from clutter and in absence of clutter

51. Model-free object pose estimation

52. Model-free pose estimation
Pose estimation: inferring the configuration of a moving object from one or more image sequences
Most approaches in the literature are model-based: they assume some knowledge about the nature of the body (articulated, deformable, etc) and some sort of model
T=0
t=T

53. Model-free scenario
Scenario: the configuration of an uknown object is desired, given no a-priori information about the nature itself of the body
The only info available is carried by the images
We need to build a map from image measurements to body poses
This can be done by using a learning technique, based on training data

54. 1 3 2 “Evidential” model approximate feature spaces
feature-pose maps (refinings) 2
training set of sample poses
The evidential model is built during the training stage, when the feature-pose maps are learned

55. 2
Feature extraction 1 3 1
From the blurred image the region with color similar to the region of interest is selected, and the bounding box is detected. 2 3

56. Estimates from the combined model
Ground truth versus estimates...
... for two components of the pose

57. JPDA with shape information for data association

58. JPDA with shape info JPDA model: independent targets
Shape model: rigid links
Dempster’s fusion
robustness: clutter does not meet shape constraints
occlusions: occluded targets can be estimated

59. Triangle simulation
the clutter affects only the standard JPDA estimates

60. Body tracking
Application: tracking of feature points on a moving human body

61. Volumetric action recognition

62. Volumetric action recognition
problem: recognizing the action performed by a person viewed by a number of cameras
step 1: modeling the dynamics of the motion
step 2: extracting image features
2D approaches: features are extracted from single views -> viewpoint dependence
volumetric approach: features are extracted from a volumetric reconstruction of the moving body

63. Multiple sequences
synchronized views from different cameras, chromakeying

64. Volumetric intersection
more views -> more details
silhouette extraction of the moving object from all views
3D object shape reconstruction through intersection of occlusion cones

65. k-means clustering to separate bodyparts
3D feature extraction
locations of torso, arms, and legs of the moving person
k-means clustering to separate bodyparts

66. two instances of the action “walking”
Feature matrices X Y
TORSO COORDINATES Z
ABDOMEN COORDINATES
RIGHT LEG COORDINATES
LEFT LEG COORDINATES
two instances of the action “walking”

67. Modeling and recognition
HMM 1
HMM 2 …
HMM n
model of the “walking” action
classification: each new feature matrix is fed to all the learnt models, generating a set of likelihoods

68. 3
Combinatorics

69. Independence on lattices
three distinct independence relations
LI3
LI2=LI3
LI1
semimodular lattice
modular lattice
LI2
modularity  equivalent formulations

70. Scheme of the proof

71. 4
Conclusions

72. CONDITIONALCONSTRAINTS
from real to abstract
OBJECT TRACKING
DATA ASSOCIATION
MEASUREMENTCONFLICT
POINTWISE ESTIMATE
CONDITIONALCONSTRAINTS
ALGEBRAICANALYSIS
GEOMETRICAPPROACH
TOTAL BELIEF
the solution of real problems stimulates new theoretical issues

73. …concluding
the ToE comes from a strong critics to the Bayesian framework
useful for sensor fusion problems under incomplete information
real problem solutions stimulate the extension of the formalism
complex objects  mathematically rich
Young theory  need completion

74. In the near future..
search for a metric on the space of dynamical systems – stochastic models
sistematic description of the geometric approach to non-additive measures
understand the intricate relations between probability and combinatorics