Detection, Tracking and Recognition in Video Sequences Supervised By: Dr. Ofer Hadar Mr. Uri Perets Project By: Sonia KanOra Gendler Ben-Gurion University of the Negev Department of Communication Systems Engineering

Outline Motivation Our System Phase 1: Detection Phase 2: Tracking Phase 3: Recognition Result Video

Motivation National security is a matter of high priority. Main method of security – cameras. – Disadvantage – requirement for man power. A better method is required.

Outline Motivation Our System Phase 1: Detection Phase 2: Tracking Phase 3: Recognition Result Video

Our System An automatic security system. –Stationary camera. Goal – detect, track and recognize a moving human object in a video sequence. –Each phase is implemented as a separate algorithm. Simulation in Matlab.

Hall Monitor

A General View on the System Detect Recognize Track State machine : Tracking successful? YesNo

Outline Motivation Our System Phase 1: Detection Phase 2: Tracking Phase 3: Recognition Result Video

Object Detection Find edge map of difference image Find moving object edge ME n for frame n Delay Previous gray level image Current gray level image ME n-1 Use Canny edge detector to find all edge maps. Use four images: –Background edge map E b. –Difference edge map DE n. –Current edge map E n. –Previous algorithm output ME n-1. EnEn EbEb ME n DE n

Object Detection (Cont.) Extracted background edge map Background frame

Object Detection (Cont.) Use four images: –Background edge map E b. –Difference edge map DE n. –Current edge map E n. –Previous algorithm output Me n-1. Difference image contains the moving parts of the frame. Find edge map of difference image Find moving object edge ME n for frame n Delay Previous gray level image Current gray level image ME n-1 EnEn EbEb ME n DE n

Object Detection (Cont.) Current frame Edges extracted from the difference image Edges extracted from the original frame

Object Detection (Cont.) Use four images: –Background edge map E b. –Difference edge map DE n. –Current edge map E n. –Previous algorithm output ME n-1. Find pixels that belong to the moving parts of the object. Find pixels that belong to the still parts of the object. Combine the two components.

Detection Result Final edge map

Outline Motivation Our System Phase 1: Detection Phase 2: Tracking Phase 3: Recognition Result Video

Object Tracking Input: –Current edge map –Previous edge map Goal - estimate object’s location in the next frame. The estimation is performed for several frames, until tracking fails. In that case, the object is re- detected.

Object Tracking (Cont.) Divide the previous edge map into square blocks. Use thresholding to determine which blocks contain parts of objects. Empty blockBlock containing object

Object Tracking (Cont.) Locate the blocks containing objects from the previous edge map in the current edge map: –To match the blocks, we use correlation. previous Current Matched x =

Correlation Statistics

Previous edge map Previous edge map with blocks Current edge map with matched blocks Matching Results

Object Tracking (Cont.) For each matched block, calculate the amount of pixels it moved in each axis. Calculate the average E and standard deviation, and divide into five ranges. E

Object Tracking (Cont.) Estimate the location of each block in the next edge map according to the corresponding speed. Verify true location of each block using correlation.

Tracking Statistics Threshold

Tracking Results Current, next and following frames, double estimation.

Outline Motivation Our System Phase 1: Detection Phase 2: Tracking Phase 3: Recognition Result Video

Environment Actors Equipment Static Dynamic Size Motion Interaction Static Dynamic Scenario Analysis

Recognition Monitor amount of objects Recognize behavior of human object –Size – based on detected edges Size indicates motion inwards or outwards –Motion – based on tracked blocks Recognize sudden changes in speed as suspicious –Interaction with environment – based on skeleton Recognize suspicious postures.

Monitoring Amount of Objects

Locate center of mass –Using: Find end points of limbs Mark spine and limbs location Calculate angles between spine and limbs to recognize the posture Skeleton Construction

Outline Motivation Our System Phase 1: Detection Phase 2: Tracking Phase 3: Recognition Result Video

Separated Objects Demo

Click to edit Master title style Click to edit Master subtitle style 32 Thank You!

Freq. Domain Correlation If there is a match between an image f and an object h, the correlation will be maximal at the location of h in f. Correlation is given by: When: denotes the complex conjugate of f.

Mathematical Background Difference edge map: Pixels that belong to the moving parts of the object : Pixels that belong to the still parts of the object : Result: