A neural approach to extract foreground from human movement images S.Conforto, M.Schmid, A.Neri, T.D’Alessio Compute Method and Programs in Biomedicine 82(2006) Che-Wei Sung 2007/12/24

Outline Introduction Materials and methods  Subtraction techniques  Neural approach Qualitative evaluation of results Objective evaluation of results Conclusions

Introduction The capture of human movement is a hot topic for surveillance, control and analysis. In the framework of human movement analysis often consists of separating the moving subject (i.e. foreground) from the background by techniques based on temporal or spatial.

Introduction Temporal data can be used in two different ways, subtraction and flow, while spatial techniques is applying markers on foreground. Mixed approaches have been presented, but none can be considered as outperforming in general terms. The work in this paper is to development of a markerless capture system for movement analysis application by making the ANN “learn” the background.

Materials and methods The moving subject is detected by analyzing the differences between the background scene., corresponding to the background image,represents the generic s-th image frame extracted from the video sequence gathering the moving subject over the background scene.

Materials and methods - Subtraction 1. Compute the image difference 2. For each row of, calculate the vectors of mean value, and standard deviation 3. Determine the 3D-classification interval, if a pixel lies inside the domain, it is classified as background, vice versa as foreground. 4. Detect the largest connected area that considered as actual foreground.

Materials and methods - ANN Neural network makes use of a Kohonen map, composed of (8×8) neurons

Materials and methods - ANN In this work, background image is partitioned into blocks of (8×8) pixels, and arranged in a mono-dimensional vector composed of H = (64×3) =192 components for training data.

Materials and methods - ANN Assume the image is subdivided into B blocks of size (8×8), the training input vector V b ={b=1,2…B} and the size of each synaptic weight vector is randomly initialized in [0,1], where h=1,2…H

Materials and methods - Training 1. One input vector V b is randomly extracted from the training set, and feeds the network. 2. In each neuron n ij, the distance d ij,b (k) between V b and (k) is calculated: 3. The best match neuron n BM (k) is defined as the n ij whose corresponding vector (k) is at the minimum distance from V b.

Materials and methods - Training 4. The weight vectors are updated by using typical Kohonen neighborhood procedure. where

Materials and methods - Training The training has been considered as complete when, for the 98% of training samples, the association between each V b and the corresponding best match neuron is not altered

Materials and methods - Testing 1. undergoes Data Shaping, creating a set of vectors. 2. For each block, the best match neuron is identified by considering the minimum Euclidean distance criterion.

Materials and methods - Testing 3. is used to build up a distance matrix, whose elements are rearranged respecting the spatial of, where each element occupies the position of block. 4. For each row of distance matrix, the mean value and the standard deviation are calculated. Blocks with corresponding distance values outside the range are considered as foreground. 5. A segmentation mask is built up by marking pixels with 0 for background, 1 for foreground.

Qualitative evaluation of results The proposed algorithm have been applied to analyze human body movement during three motor tasks: gait, pitching a ball and standing up from a chair. The training of Kohonen’s map has met convergence after around presentations of background blocks.

Qualitative evaluation of results

Objective evaluation of results quality_index s = 0.3shape_reg s temp_stab s contrast s  shape_reg s : the regularity of segmented object shape.  temp_stab s : the stability along the video sequence of extracted object.  contrast s : the contrast between the inside and the outside of the object evaluated along the border.

Objective evaluation of results

Conclusions The work proposes a new unsupervised approach for foreground extraction in human movement images based on ANN and the presented results demonstrate it is suitable.