Shape and Dynamics in Human Movement Analysis Ashok Veeraraghavan

Outline Motivation What do we want to do? Shape Shape based methods for recognition Dynamics based methods for recognition Results

Motivation Human Perception Shape or Dynamics (or is it Both??)

Laurel and Hardy

Laurel ??? Hardy ???

Who is this ? ? ?

Introduction Psychophysics work indicates that dynamics is important for recognition in humans. Johansson: Light Display Moving dots Murray(1964) : 24 gait components Cutting:Familiarity;Static Vs Dynamic Kozlowski: dynamics speed, bounciness, rhythm. Cutting :Dynamic Invariant Gender Discrimination

Prior Work Image Correlation. Silhoutte Based Nearest Neighbour. Dynamic Time Warping Hidden Markov Model Model parts of human body and extract gait signature.(eg., Thigh)

Most gait recognition algorithms are shape based ! Relative importance of shape and dynamics

Definition of Shape “Shape is all the geometric information that remains when location, scale and rotational effects are filtered out from the object”. Kendall’s Statistical Shape Theory used for the characterization of shape. Pre-shape accounts for location and scale invariance alone.

Pre-Shape k landmark points (complex vector) Translational Invariance: Subtract mean Scale Invariance: Normalize the scale

Feature Extraction Silhoutte Landmarks Centered Landmarks Pre-shape vector

Distance between shapes Shape lies on a spherical manifold. Shape distance must incorporate the non-Euclidean nature of the shape space. 1)Full Procrustes distance. 2)Partial Procrustes distance. 3)Procrustes distance.

Full Procrustes Distance Procrustes Fit Full Procrustes Distance=Minimum Procrustes Fit.

Other shape distances Partial procrustes distance Procrustes distance (ρ): distance on the Great circle.

Tangent Space Linearization of spherical shape space around a particular pole. The Procrustes mean shape is usually chosen as the pole. If the shapes in the data are very close to each other then Euclidean distance in tangent space approximates shape distances.

Shape based methods for Recognition Stance Correlation. Dynamic time warping in shape space. Hidden Markov Model in shape space.

Stance Correlation Exemplars for 6 stances for each individual. The correlation between exemplars is used as the matching criterion. Performance comparable to Baseline.

Dynamic time warping in shape space. Enforce end-point constraint. Obtain best warping path. Cumulative error is computed using the shape distances described. Performance is better than baseline.

Hidden Markov Model in shape-space Exemplars are regarded as states. HMM built for each person in the gallery. Identity established by maximizing the probability that the observation came from the model in the gallery. Performance is better than baseline and comparable to DTW.

Dynamical Models Stance based AR model. Linear Dynamical System

Stance based AR model Video sequence is clustered into 3 distinct stances. Each frame is identified as belonging to one of these three stances. Parameters of an AR model learnt for each stance. Model parameters used for recognition. Performance is below baseline.

Linear Dynamical System(ARMA) Parameters (A,C) of a dynamical system learnt for each individual. Distance between models used as score for recognition.

Learning the model

Distance between models Subspace angles (θ i : i=1,2….n). Martin,Gap and Frobenius Distance.

Results on USF database Gallery 71 people. Probe varies from Gallery in view, shoe and surface. CMS curves shown.

Sample Sequences

Stance Correlation.

Dynamic time warping

Comparison of DTW with Baseline

Stance based AR model

Linear Dynamical system

Comparison of various methods on the USF database.

Results on the CMU database Gallery consists of 25 people. 3 different activities studied: Slow walk, Fast walk and walk with ball. Recognition performed within and across activities.

Percentage of Recognition using Stance correlation. Slow Walk Fast Walk Ball Slow Walk Fast Walk Ball684892

Similarity Matrix using Linear Dynamical system(ARMA)

Percentage of Recognition using Linear Dynamical system Slow WalkFast WalkBall Slow Walk Fast Walk Ball

Mocap Data

Mocap (Activity Recognition)

Mocap (Activity using ARMA)

Conclusions Shape is more important for recognition than dynamics. Shape also provides for speed change invariance. Dynamics can help to improve performance of shape based methods. Activity Recognition: Dynamics plays a important role. Dynamical models like ARMA can perform recognition across activities.

Thank You.