Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline Rousseau IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 5, MAY 2011
Introduction Our System and data set Falls Characteristics Shape deformation ▪ mean matching cost ▪ full Procrustes distance Fall Detection Using GMM Experimental Results Conclusion
Establish new healthcare systems to ensure the safety of elderly people at home. Falls are one of the major risks for old people living alone. Fall detection wearable sensor: Accelerometers or help buttons Problem: -forget to wear -unconscious after the fall -recharged regularly
Computer vision systems has overcome these problems. A camera provides a vast amount of information on his/her environment ▪ Monocular Systems ▪ Bounding box[8] ▪ Only placed sideways ▪ Occluding objects ▪ Multi-Camera Systems ▪ Auvinet et al.[17] reconstructed 3-D silhouette of an elderly person ▪ Need to be calibrated ▪ The video sequences need to be synchronize
Uncalibrated multi-camera system Low-cost IP cameras, 30 frames/s, 720 × 480 pixels Wide angle to cover all the room
Total of 75 different events, more than 12 min
1. Lack of significative movement 2. A lying position 3. A person lying on the ground 4. Vertical speed 5. An impact shock 6. Body shape change
The silhouette is extracted by a background subtraction N = 250 landmarks * Canny edge detector[25]
Shape context[20] is a way of describing shapes. Matching cost for pair (p i, q j ):, K=5*12 bins
Minimizing the total matching cost given a permutation π (i) Use the Hungarian algorithm[27] for bipartite matching Time complexity: O(n^3) Bad landmarks due to segmentation errors or partial occlusions ▪ Add dummy points (not easy to choose). ▪ Match only the most reliable points in our implement (min i Cij = min j Cij) mean matching cost: ij bipartite graph N ∗ : the total number of best matching points.
Procrustes analysis [21] has been widely used to compare shapes. Detect abnormal shape deformation for fall detection ▪ Step1 : image registration(one translation, no rotation, no scaling) ▪ Step2: Compute full Procrustes distance for compare. centered landmarks Zc : Z Zc two centered vectors : v = (v1, · · ·, vk) w = (w1, · · ·,wk). full Procrustes distance :
mean matching cost full Procrustes distance Consider 2 feature (F1, F2) 1) F1 representing the fall : F1 will high in case of fall 1) F1 representing the fall : F1 will high in case of fall 2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low 2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low
Model normal activity data with a Gaussian Mixture Model(GMM). GMM: weighted sum of Gaussian(normal) distributions M : the number of components in the mixture P (j) : the mixing coefficients The jth Gaussian probability density function p (x | j) ▪ d is the dimensionality of the input space expectation-maximization (EM) algorithm by maximizing the data likelihood expectation-maximization (EM) algorithm by maximizing the data likelihood GMM Classifier : only tell normal or abnormal! GMM Classifier : only tell normal or abnormal!
Leave-One-Out Cross-Validation 1. Divided the dataset into N video sequences 2. One sequence is removed 3. Training using the N − 1 remaining sequences (falls are deleted) 4. This sequence is classified with the resulting GMM. 5. Repeat N times 6. Count the number of errors, classification error rate
1. True Positives (TP): falls correctly detected; 2. False Negatives (FN): falls not detected; 3. False Positives (FP): normal activities detected as a fall; 4. True Negatives (TN): normal activities not detected as a fall; 5. Sensitivity: Se = TP/ (TP + FN); 6. Specificity: Sp = TN/ (TN + FP); 7. Accuracy: Ac = (TP+TN) / (TP+TN+FP+FN) ; 8. Classification error rate: Er = (FN+FP) / (TP+TN+FP+FN).
Shape matching : C++ using the OpenCV library [33] Fall detection : MATLAB using the NETLAB toolbox [32] to perform the GMM classification. The original video sequences frame : 30 frames/s 5 frames/s was sufficient to detect a fall Intel Core 2 Duo processor (2.4 GHz) The computational time of the shape matching step is about 200 ms
train a GMM with three components for our experiment.
Normalize training data. Detection threshold depends on the sensitivity.
false positives true positives
Simply majority vote on all cameras (>= 3 vote) In fig. 9 : error rate 10% 2.7%
We presented a new GMM classification method to detect falls By analyzing human shape deformation Robust to large occlusions and other segmentation difficulties