Gait Recognition Guy Bar-hen Tal Reis

Introduction Gait – is defined as a “manner of walking”. Gait recognition – –is the term typically used to refer to the automatic extraction of visual cues that characterize the motion of a walking person in video and is used for identification purposes.

Motivation Gait is particularly an attractive modality for passive surveillance since, unlike most biometrics, it can be measured at a distance, hence not requiring interaction with or cooperation of the subject.

Usage Examples Classify moving objects – person / non – person Biometric recognition of a person

Gait Recognition Methods Two main classes of gait recognition - – Holistic characterizes body movement by the statistics of the raw spatiotemporal patterns generated by the person’s motion [XYT]. – Feature-based recovers explicit features (or parameters) describing gait dynamics, such as stride dimensions and the kinematics, of joint angles.

Gait Recognition Using Image Self- Similarity Chiraz BenAbdelkader Ross G. Culter Larry S. Davis

Assumption People walk with constant velocity for about 3 – 4 seconds. People are located sufficiently far from the camera. The frame rate is greater than twice the frequency of the walking. The camera is stationary.

Method Outline The includes three main modules – – Preprocessing module - segmentation and tracking of the moving person in each frame of a given sequence. – Feature measurement – computing SSP from silhouette sequence,[scaling alignment] obtaining a set of normalized feature vectors. – Pattern classification – identity is determined with standard classification techniques.

Method Overview

Preprocessing Segmentation is achieved via nonparametric background modeling or subtraction technique. Using cadence-based technique to determine whether a foreground blob corresponds to a moving person. Creating from N frame N silhouette templates.

Preprocessing Extract the blob region enclose with its bounding box either from – – Original color / grayscale image – Foreground image – Binary image

Feature measurement silhouette template scaling The silhouette templates need to be first scaled to a standard size to normalize for depth variations.

Feature measurement silhouette template scaling According to gait analysis the width and height of a person can be approximated as sinusoidal function –

Feature measurement silhouette template scaling De-trend due to the changing camera depth and the differences between non-fronto / fronto parallel walking as – The templates have equal mean width and height. The width-to-height aspect ratio need to remain constant throughout the sequence.

Feature measurement silhouette template scaling Final scaling so that the mean is equal to some given constant. Where H 0 = 50 pixels

Feature measurement computing the self-similarity plot [SSP] In a periodic motion the SSP is also periodic, a good way to detect and characterize the periodic motion. Computation of SSP is done by – Ith scaled template with size w(I) * h(I) in pixels

Feature measurement computing the self-similarity plot [SSP] Intersection = local minima of S S encodes both the frequency and phase of gait cycle

Feature measurement Normalizing the SSP SSP is needed later to compare two different walking sequences that contain equal number of walking cycle starts at the same phase. SSU – self-similarity units. The SSP can be viewed as a contiguous of SSU. For gait recognition using only the SSU corresponding to the left and right double-support poses. Scaling the SSU to uniform size m*m to be able to compare. – Double-support – cycle corresponds to when the feet are max apart.

Feature measurement Normalizing the SSP SSU start at pose A and C

Feature measurement Computing the frequency and phase of the gait Frequency – – Apply autocorrelation method on the SSP. – Smoothes the matrix of SSP – Computes peaks – Find the best fitting regular 2D lattice and then the period obtained as the width of best fitting lattice

Feature measurement Computing the frequency and phase of the gait Phase – – Compute by locating the local minima of the SSP that corresponds to A and C poses.

Pattern classification Definition – – Gallery – labeled SSU. – Probe – determine the person corresponding to a set of novel SSU. Two steps – – Pattern matching using statistical pattern classification, between each pair of probe. – Determine the probe’s correct class using KNN.

Results The methods was tested on 4 data set with low resolution video with recognition rate of 100% for fronto-paralle data set of 6 people and 70% for data set of 54 people.

Gait Analysis for Recognition and Classification L. Lee W.E.L. Grimson

Gait appearance representation Canonical view - perpendicular to the direction of walk. The silhouette of the walker is segmented from the background Walker is located sufficiently far from the camera. The camera is stationary.

Gait Dynamic Feature Vector Ability to describe appearance at a level finer than whole body description without having to segment individual limbs. Robustness to noise in video foreground segmentation. And simplicity of representation.

Ellipses Representation

(x i, y i ) – The center of the ellipse l i – The aspect ratio of major and minor axis  i – The orientation of major axis

For each frame The frame feature vector of the j-th frame is In addition to these 28 features we also use h, the height of the center of mass.

For each Sequence The mean and standard deviation of region features across time. Magnitudes and phases of each region feature related to the dominant walking frequency.

Gait Average Appearance Feature Vector j = 1…last frame; s is 57-dimentional.

Gait Spectral Component Feature Vector Ω d is the dominant walking frequency of a given sequence. t is 56-dimentional.

Are all features equally useful? A good feature should minimize within class variance and maximize interclass variance. Assuming that the features are independent from each other. The features can be ranked by p-value generated by analysis of variance (ANOVA).

Person Identification Gait data was gathered on 4 separated days with different backgrounds. 10 women, 14 men 194 walking sequences. A minimum of 3 complete walking cycles at 15 frames per second for each walking sequence.

Recognition using Averaged Appearance Feature Two gait recognition tests were performed using each of the two gait feature vectors p <  41 feature parameters for gait averaged appearance feature set, and 32 parameters for gait spectral components feature set.

Any-day test 100% correct identification for the 1 st match, using the best 41 features. 97% for the 1 st match and 100% by the 3 rd match, using all 57 features.

Cross Day Test The gait recognition performance was highest when comparing data collected on day A against all other sequences.

Cross Day Test The average appearance gait features contain the mean shape of the 7 regions of the silhouettes. The shapes of these regions are very much affected by drastic changes in clothing style, such as pants vs. dress, shorts vs. pants.

Recognition using Spectral Components Features The gait spectral component features only contain the changes in shape and the time delay between the different regions, hence it is less affected by the change of clothing style.

Gender Classification Each of the 57 features was ranked based on the p-value of ANOVA in separating the genders and set a threshold of p < 10 −9, which resulted in the best 6 features

Gender Classification SVM’s were trained using the 57 and the 6 gender features. Experimented with the linear, Gaussian and the 2nd degree polynomial kernels Random-person vs. Random-sequence

Gender Classification

Questions

Backup

Mahalanobis distance It is based on correlations between variables by which different patterns can be identified and analyzed. It is a useful way of determining similarity of an unknown sample set to a known one. It differs from Euclidean distance in that it takes into account the correlations of the data set and is scale-invariant, i.e. not dependent on the scale of measurements.