Attila Kiss, Tamás Németh, Szabolcs Sergyán, Zoltán Vámossy, László Csink Budapest Tech Recognition of a Moving Object in a Stereo Environment Using a Content Based Image Database

SAMI 2005 (2/21) Contents Introduction System build-up Techniques Results Summary

SAMI 2005 (3/21) Project description Two-camera system that is able to detect and recognize a moving object in the workspace Sub goals Detection of moving objects Produce 3D model for the detected objects using disparity map Forward deepness and other features to a content based retrieval system for object recognition

SAMI 2005 (4/21) System build-up Camera handler Motion detection Model preparation Modeling system Model creation OpenGL - visualization Content-based image retrieval system Feature extraction Similarity measure Feedback

SAMI 2005 (5/21) Camera handler Object detection Difference from background Marking regions Connection with CBIRS Surround the found object with the smallest rectangle or with convex hull Model preparation Searching feature points with Harris type corner detection algorithm

SAMI 2005 (6/21) Modeling system Model creation Intensity cross correlation Finding correspondence between left and right picture with intensity cross correlation using feature points Get deepness information from the matched feature points disparity Correlation based stereo Runs on whole image Slow Visualization OpenGL Mubarak Shah - „Fundamentals Of Computer Vision” Computer Science Department University of Central Florida, Orlando, 1997.

SAMI 2005 (7/21) Content based retrieval – 1 Jose A. Lay, Ling Guan – „Image Retrieval Based On Energy Histograms Of The Low Frequency DCT Coefficients”,

SAMI 2005 (8/21) Content-based retrieval – 2 Preprocessing, noise filtering Gauss filter Color normalization (Finlayson) Low level features DCT coefficients Color histograms in 6 colorspaces (RGB, HSV, YIQ, XYZ, L*u*v*, L*a*b*) Cornerness Disparity map B. V. Funt, G. D. Finlayson – “Color Constant Color Indexing” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMIÖ, Bd. 17, Nr. 5, 1995, S

SAMI 2005 (9/21) Content-based retrieval – 3 Similarity measure Minkowski distance Histogram intersection Hierarchical search Multi dimensional similarity measure Feedback, Yong Rui technique Yong Rui, Thomas S. Huang, Michael Ortega and Sharad Mehrotra: Relevance Feedback - „A Power Tool for Interactive Content-Based Image Retrieval” IEEE Transactions on Circuits and Video Technology, Special Issue on Segmentation, Description, and Retrieval of Video Content, pp , Vol 8, No. 5, Sept, 1998

SAMI 2005 (10/21) Testing, results – 1 Motion detection Feature points Convex hull

SAMI 2005 (11/21) Testing, results – 2 Matching feature points Detected feature points Corresponding feature points

SAMI 2005 (12/21) Testing, results – 3 Pentagon satellite stereo images and their disparity map Own images and their disparity map

SAMI 2005 (13/21) Testing, results – 4 Contents of own test dataset

SAMI 2005 (14/21) Testing, results – 5

SAMI 2005 (15/21) Testing, results – 6 Some results of content based image retrieval group test

SAMI 2005 (16/21) Testing, results – 7 Precision with University of Washington collection Henning Müller, Wolfgang Müller, Stephane Marchand-Maillet, Thierry Pun: A web-based evaluation system for CBIR, Some examples System Without feedback Four feedback Own system 30,83%41,25% GIFT with special plugin 53,92%91,07%

SAMI 2005 (17/21) Summary – 1 Two-camera stereo environment Detect moving Model workspace Forward disparity map to a CBIR Content based image retrieval system Pluginable by indexing techniques Automatically produce indices Color based Texture based Depth based

SAMI 2005 (18/21) Summary – 2 Semantic interpretations Textual description Hierarchically build database Query type Nearest neighbors Threshold Relevance Feedback Automatic testing and evaluation Store Compare

SAMI 2005 (19/21) Future plans Implement other techniques Fasten existing modeling algorithms Camera calibration Using OODB with special indexing e.g. a type of B-tree

SAMI 2005 (20/21) References M.J. Swain and B.H. Ballard - “Color Indexing” Int’l J. Computer Vision, vol. 7, no. 1, pp , B. V. Funt, G. D. Finlayson – “Color Constant Color Indexing” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMIÖ, Bd. 17, Nr. 5, 1995, S Yong Rui, Thomas S. Huang, Michael Ortega and Sharad Mehrotra: Relevance Feedback - „A Power Tool for Interactive Content-Based Image Retrieval” IEEE Transactions on Circuits and Video Technology, Special Issue on Segmentation, Description, and Retrieval of Video Content, pp , Vol 8, No. 5, Sept, 1998 Jonathan Owens, Andrew Hunter & Eric Fletcher - „A Fast Model-Free Morphology- Based Object Tracking Algorithm” Marc Pollefeys - „3D Modelling from Images” C. Harris and M. Stephens – „A combined corner and edge detector” Fourth Alvey Vision Conference, pp , Henning Müller, Wolfgang Müller, Stephane Marchand-Maillet, Thierry Pun: A web- based evaluation system for CBIR, Mubarak Shah - „Fundamentals Of Computer Vision” Computer Science Department University of Central Florida, Orlando, 1997.

SAMI 2005 (21/21) Thanks for Your attention! Accessibility: Attila Kiss Tamás Németh Zoltán Vámossy Szabolcs Sergyán László Csink Homepage: