MACHINE VISION GROUP MOBILE FEATURE-CLOUD PANORAMA CONSTRUCTION FOR IMAGE RECOGNITION APPLICATIONS Miguel Bordallo, Jari Hannuksela, Olli silvén Machine Vision Group University of Oulu

MACHINE VISION GROUP Contents Introduction Image recognition applications –Comparison of image-based context retrieval methods Context retrieval from video analysis System design –Application flow –Automatic start –Image registration –Moving-objects detection –Quality assesment Performance analysis Conclusions

MACHINE VISION GROUP Image-based context retrieval applications Point Your Camera to an object (landmark, poster) Take a Picture Get context information and display it

MACHINE VISION GROUP Mobile context retrieval applications Google googles Snaptell Kooaba Nokia Point & Find

MACHINE VISION GROUP Image recognition approaches Videos contain lots of information –Most of it redundant Image registration is easy –Smaller motions between frames –some frames can be discarded without losing information Videos can capture wide angle scenes. –3D world is better represented Transmission of compressed still image Needs lots of storage in server Image size implies large amount of data transmitted Compression artifacts diminish quality Features extracted from still images Amount of features needed not know beforehand No feedback. Re-takes needed often Two dimensional representation Feature-cloud extracted from video frames

MACHINE VISION GROUP Still image vs. Video based

MACHINE VISION GROUP Constructing a feature-cloud Frame #1Frame #16Frame #31Frame #46Frame #61

MACHINE VISION GROUP System design

MACHINE VISION GROUP Application flow (client side)

MACHINE VISION GROUP Automatic start of the application Recognizing characteristic motion patterns Holding phone like a camera Panning back and forth Reduces perceived latencies

MACHINE VISION GROUP Interactive capture VGA video analysis Motion estimation system calculates shift, rotation and scale in real time When frame is suitable for recognition (high quality), the user receives feedback and instructions

MACHINE VISION GROUP Feature extraction & Image registration Feature extraction based on CHoG features Compressed Histogram of Gradients Block Matching Best Linear Unbiased Estimator Compute registration parameters in real time to send to the server: Shift, rotation and change of scale

MACHINE VISION GROUP Moving-objects detection Object-detection ONObject-detection OFF

MACHINE VISION GROUP Moving-objects detection The features corresponding to a moving object are not sent to the server Not-valid features are transmitted to the server Object-detection ONObject-detection OFF

MACHINE VISION GROUP Quality assesment Server receives only the features corresponding to high quality frames

MACHINE VISION GROUP Performance comparison

MACHINE VISION GROUP Summary Improve results in 3 dimensional environments Interactivity Detection of moving objects Image quality assesment Bigger field of view Reduce the communications need between clients and server Bandwidth reduction Reduce the workload of the servers