Shape and Dynamics in Human Movement Analysis Ashok Veeraraghavan

Outline Motivation What do we want to do? Shape Shape based methods for recognition Dynamics based methods for recognition Results Current Work

Motivation Human Perception Shape or Dynamics (or is it Both??)

Laurel and Hardy

Who is this ? ? ?

Introduction Psychophysics work indicates that dynamics is important for recognition in humans. Johansson: Light Display Moving dots Murray(1964) : 24 gait components Cutting:Familiarity;Static Vs Dynamic Kozlowski: dynamics speed, bounciness, rhythm. Cutting :Dynamic Invariant Gender Discrimination

Prior Work Image Correlation. Silhoutte Based Nearest Neighbour. Dynamic Time Warping Hidden Markov Model Model parts of human body and extract gait signature.(eg., Thigh)

Most gait recognition algorithms are shape based ! Relative importance of shape and dynamics

Definition of Shape “Shape is all the geometric information that remains when location, scale and rotational effects are filtered out from the object”. Kendall’s Statistical Shape Theory used for the characterization of shape. Pre-shape accounts for location and scale invariance alone.

Pre-Shape k landmark points (complex vector) Translational Invariance: Subtract mean Scale Invariance: Normalize the scale

Feature Extraction Silhoutte Landmarks Centered Landmarks Pre-shape vector

Distance between shapes Shape lies on a spherical manifold. Shape distance must incorporate the non-Euclidean nature of the shape space. 1)Full Procrustes distance. 2)Partial Procrustes distance. 3)Procrustes distance.

Full Procrustes Distance Procrustes Fit Full Procrustes Distance=Minimum Procrustes Fit.

Other shape distances Partial procrustes distance Procrustes distance (ρ): distance on the Great circle.

Tangent Space Linearization of spherical shape space around a particular pole. The Procrustes mean shape is usually chosen as the pole. If the shapes in the data are very close to each other then Euclidean distance in tangent space approximates shape distances.

Shape based methods for Recognition Stance Correlation. Dynamic time warping in shape space. Hidden Markov Model in shape space.

Stance Correlation Exemplars for 6 stances for each individual. The correlation between exemplars is used as the matching criterion. Performance comparable to Baseline.

Dynamic time warping in shape space. Enforce end-point constraint. Obtain best warping path. Cumulative error is computed using the shape distances described. Performance is better than baseline.

Hidden Markov Model in shape-space Exemplars are regarded as states. HMM built for each person in the gallery. Identity established by maximizing the probability that the observation came from the model in the gallery. Performance is better than baseline and comparable to DTW.

Dynamical Models Stance based AR model. Linear Dynamical System

Stance based AR model Video sequence is clustered into 3 distinct stances. Each frame is identified as belonging to one of these three stances. Parameters of an AR model learnt for each stance. Model parameters used for recognition. Performance is below baseline.

Linear Dynamical System(ARMA) Parameters (A,C) of a dynamical system learnt for each individual. Distance between models used as score for recognition.

Results on USF database Gallery 71 people. Probe varies from Gallery in view, shoe and surface. Average CMS curves shown.

Sample Sequences

Comparison of various methods on the USF database.

Results on the CMU database Gallery consists of 25 people. 3 different activities studied: Slow walk, Fast walk and walk with ball. Recognition performed within and across activities.

Percentage of Recognition using Stance correlation. Slow Walk Fast Walk Ball Slow Walk Fast Walk Ball684892

Similarity Matrix using Linear Dynamical system(ARMA)

Percentage of Recognition using Linear Dynamical system Slow WalkFast WalkBall Slow Walk Fast Walk Ball

Conclusions Shape is more important for recognition than dynamics. Shape also provides for speed change invariance. Dynamics can help to improve performance of shape based methods. Activity Recognition: Dynamics plays a important role. Dynamical models like ARMA can perform recognition across activities.

Current Work Experiments on Manually Segmented Gait data. An attempt to isolate the effects of kinematics. ( so far “kinematics” referred to the dynamics of the representation feature (shape)). Building a kinematic model for gait. This model will be used as a prior for human body tracking.

Manually Segmented Data Extract Joint Angles from manually segmented data. Each gait sequence is now a sequence of Joint angles.

Dynamic Time Warping on Joint Angles.

Effect of Joint angles alone on HMM/DTW recognition performance Impose the extracted joint angles on a 3-d Volumetric Model (Thanks to Aravind Sundaresan) and artificially generate a gait sequence. Each sequence corresponds to same body shape but different style of walking. HMM and DTW for recognition on these silhouttes.

Recognition Performance of Shape normalized silhoutte data

Gait Modeling Model the trajectories of the joint angles over the gait sequence. Exploring Vector Autoregressive model (Order atleast 2).

Model Based Tracking Dynamical Model of gait will serve as a guide during 3-d model based tracking of humans. Hope to use image cues such as edges, silhouettes etc. as observations for tracking.

Thank You!!