Mobile Model Car Navigation Based on Wireless Video Camera and Laser Sensors

Contents Team introduction Project Introduction Applicability The whole picture Discussing tracking algorithms A project development Results

Team introduction Supervisor: Professor Ehud Rivlin . Instructor: Igor Katsman. ISL Engineer: Yekutiel Pekar Students : Alexander Sherman Valentin Rozental

Introduction Mobile navigation on unknown environment is an interesting and challenging task. Consists of several computer vision based subtasks. Each subtask is a project itself.

Where it could be used Space Exploring Medical applications Military applications Security applications Civil Engineering Science …

Subtasks Range-exploring algorithm for a mobile model. Tools On-board wireless video camera Laser sensors. distance

Control interface between a mobile model and a PC. Tools RC CAR remote control software module

The most interesting task Automatic tracking algorithm for a mobile model which would utilize an on-board wireless video camera to follow another leading mobile model. Tools on-board wireless video camera.

Lets focus on tracking algorithm General Scheme server frame target Turn Left Lets focus on tracking algorithm

Tracking algorithm features Simple Its should be real-time algorithm Robust It should focus on mobile features recognition Correct It should minimize misclassifications, “loosing” target, etc.

First steps Matlab environment acknowledgement Edge detection Contour recognition Noises

Known Algorithms discussion Temporal Differencing Background Subtraction Motion Regions Detection Motion Region Classification Velocity Vector Computation

Advantages Simple Robust for static camera tracking Background subtraction eliminates all static background’s pixels and moving objects could be easily detected. No information is needed about a starting position BUT …

Disadvantage A camera is in significant motion all the time. NO STATIC BACKGROUND presents.

Solution Image stabilization algorithm Drawbacks It computationally expensive Hard to get real-time speed

Conclusion Template Correlation matching algorithm may be used Another question : What is good template feature ???

Contour Advantages: Disadvantages: depends lightly on light conditions good for the simple objects Disadvantages: Hard to detect if the noise level is high Complex forms in different visual angles A big variety of such forms

Color Advantages: Disadvantages: avoid the noise problem more effective use of filters Don’t depend on object form Easier to detect the object location center of mass Disadvantages: depends heavily on light conditions Color is a feature that is widely distributed

Size Advantages : Disadvantages: Easy to detect if the object is big Doesn’t depend on light conditions Disadvantages: Easy to misclassify the object if the object is small Very sensitive to angle of sight Depends heavily on the object distance

Template Advantages: Disadvantages: Easy to scan the image Could be done by partial match Less sensible to noises Disadvantages: A variety of templates must be provided and (sequent/parallel) diagnosed Not a real-time method

But still there are good choices ! NO “SILVER BULLET” ! But still there are good choices !

A choice Color Simplicity is desirable Accuracy is mandatory Real-time performance is crucial Color

Color Histogram As flexible as memory-based methods more compact representation Estimation is trivial Color histograms for an image are built from pixel values in one of color spaces. Different weight to different color ranges increases probability of quick and precise target detection.

Algorithm When new image becomes available A local search is performed The best match is found Velocity vector is calculated

Matching For each searched area Get a color histogram Normalize it. Find an intersection with previously stored template histogram minimize differences

Get a color histogram

Normalize and compare If result < best result best result = result

Making the tracking more robust and efficient Template updating Template history management Motion area prediction

Template updating Solves disadvantage of great sensibility to light conditions Ensures that current template accurately represents the new image of object A method merging the previous instance of template with current information we have

Template History Management Solves partially or full occlusion of the object Prevents from misclassification and “loosing” the target A method merging the previous instance of template with current one A history parameter is estimated empirically

R(n) = a * M(n) + (1 - a) * R(n – 1) For example: R(n) = a * M(n) + (1 - a) * R(n – 1) where R(n) – is a template at n-th stage M(n) – is a new motion detected a – is a history parameter

Motion Area Prediction Processing a whole picture is computationally very expensive and is inapplicable to real-time applications or requires specialized hardware to operate in the real-time domain. To overcome this obstacle we use a motion region prediction that reduces search area.

A Method The 2D image velocity vector of the target (u, v) (pixels/frame) is approximately determined Calculating the difference between centroid of the previous template R(n-1) and centroid of the new template R(n)

In the area of centroid exhaustive search is performed In the area of centroid exhaustive search is performed. In case this search fails then area is enlarged in such way that ensures object detection.

First results First we tried to track simple movements in slow motions

Back and Forward motions Diagonal motions

Car come across Parallel motion

Car is approaching Car is moves away

Making things more complicated Then we decided to add some obstacles to see how we deal with them

Car is partially occluded Car is occluded by the obstacle of the same color range

Final Exam  freestyle 

A pleasant surprise Although we concentrated on car tracking it still capable of tracking other objects A HELICOPTER

AND finally some FUN See anybody you know?   

Conclusions: Using template correlation matching has three main advantages: Continuous tracking despite occlusions and cessation of target motion. Prevents template drifting onto background texture. Provides robust tracking.

Although a module show good results it has several drawbacks Although a module show good results it has several drawbacks. Some of them are: Hard to detect if the noise level is high. The probability to misclassifications of the object is great, since the color is a feature that is widely distributed.

Suggestions: To overcome these drawbacks gradient-based algorithm might be implemented as secondary stage recognition. This will strongly decrease an probability for misclassification and “loosing” a target.

To be continued ...