1. The Functional Space of an Activity Ashok Veeraraghavan , Rama Chellappa, Amit Roy-Chowdhury
Avinash Ravichandran

2. Motivation
Variability exists in activity across subjects and across instances.
The Variability can be regarded as 2 types
Global These include changes in the frame rate and the overall duration of the activity
Local These include changes in the actual gait , i.e the leg moves a little faster or slower than a previous instance of walking for the same or different person
Ignoring this variability can lead to identical models having large distances in the modeling framework

3. Need for temporal Warping

4. Time warping for activities( t ) ; T b e v c o r l u d f n i b ( t ) ; T e a i m w r p d f u n c o . = w ( t ) : [ ; T b ] ! a w ( t ) = T a f b f ( t ) : [ ; 1 ] !
Goal : Given multiple instances of a gait, to find a model that fits all of instances, and the time warping for each instance
Model a ( t ) - c n o i l r j e y W s - p a c e o f l r m i b ( t ) P ( f t ) - p r o b a i l y h e

5. Properties of the Function Space2 [ ; 1 ] , f : ! ( ) = f 2 A ! ` ( t )
> ; 8 1 f i s c o n t u a d ` ( ) 8 f 2 A , 1 e x i s t
A is convex
Physical constraints reduce the space A to a subset of functions W
Properties of W u ( t ) l , 8 2 ; 1 f ( t ) = 2 W u f l , 8 2 W
W is convex

6. Canonical Form b ( t ) = a f u ( t ) = f , l T h e n a ; W d b r q i v
The model representation is not unique
Let
This is over come by picking a symmetric u,l , as the symmetric representation is unique b ( t ) = a f u 1 ( t ) = f , l T h e n a ; W u l d b 1 r q i v t L e t s ( ) b u c h a = + ; l
Given a non symmetric representation, it can be converted into a symmetric form u 1 ( t ) = l f ( i ) t = 2 u 1 l g f 1 ( u t ) = l f 1 ( t ) u = l f 1 ( t ) = 2 u l f ( t ) = 2 u 1 l g

7. Overall Scheme

8. Learning Model Parametersv e n N r a l z t o s , f w u m k b ( ) d ^ a = 1 N P i b ( f t ) B u t i n p r a c e l h w k o s b ( ) A s u m e t h a b 1 ( ) i p l , w n d f c =
This is done using the dynamic time warping scheme (details on next slide) ^ b = 1 N P i ( f t ) ,
Symmetric Model ^ u = m a x i 1 : N f ( t ) , 8 2 ; f ^ b ; W s g ^ l = m i n 1 : N f ( t ) , 8 2 ;

9. Dynamic Time warping
Used in speech recognition systems to warp instances to a template to learn a model
Works with the different spectra components, creating a vector valued signal from scalar speech signals
Based on dynamic programming, searching over a finite grid

10. Dynamic Time Warping Constraints Á ( k + 1 ) ¸ Á ( k + 1 ) ¡ ·
Endpoint Constraint
Monotonicity Condition
Local Continuity Constraints
Global Path Constraint
Slope Weighting Á x ( k + 1 ) Á x ( k + 1 )

11. Features for Activity Recognition
Silhouette shape is used as a feature
Each silhouette has K landmark points, and the trajectory of these K points form a(t)
Invariant to scale and translation
Preshape lies on the unit sphere
Partial procrustes distance is used as a distance metric Z = C X k C = I k 1

12. Activity Recognition
Assumption : The test sequence is from one of the training classes
100 sequences for 10 Activities. Training set was 90 sequences of 10 activities, Testing was done on the remaining 10. ^ f i = a r g m n 2 W s d t ( h ) ;
Best Warping Function
Closest Fit to the model ^ I = a r g m i n 1 : N d s t ( h ) ; f

13. Recognition of individuals

14. Discussion The concept of using DTW for gait is not new.
Previously DTW was used in a template matching framework where all training data is aligned to a common time frame
This works best when the testing data is among the training datasets, for new sequences they still suffer from the issue of time warping
This algorithm is feature independent, silhouettes can be replaced by intensities, joint location etc
Although the algorithm defines the space of all transformation, this is still dependent on the training dataset

15. Other aspects
Clustering EM like framework to cluster the gaits, 92 % clustering accuracy with the n classes all representing different gaits
Database organization into a tree like structure to reduce the number of distance calculations