Parallel Integration of Video Modules T. Poggio, E.B. Gamble, J.J. Little 6.899 Paper Presentation Presenter: Brian Whitman

Overview Different cues make up a ‘reliable map’ Edge Stereo Color Motion How can we integrate these cues to find surface discontinuities?

Architecture

Physical Discontinuities Depth Orientation Albedo Edges Specular Edges Shadow Edges

Implementation The architecture was not fully implemented Results in integrating brightness with: Hue Texture Motion Stereo But separately – not together

Smoothness Physical processes behind cues change slowly over time: Two points adjacent are not vastly different depths Need a representation to capture this

Discontinuities Cues are assumed smooth everywhere except on discontinuities Each module needs to assume and interpolate smoothness detect edges and changes

Dual Lattices Circles are smooth, crosses are lines / discontinuities

Neighborhoods

Quickly, MRF (again) Prior probability of depth in the lattice is: Z: normalization, T is temperature, U is energy (sum of local contributions) If we know g (observation) use it

Membrane Prior Prior energy when surface is smooth:

Gaussian Process If we assume gaussian process generated the noise:

Line Process Where is the smoothness assumption broken? l: line between i and j? Vc: varying energies for different line configurations

Integrated Process Extend the energy function to tie together vision modules to brightness gradients Assumption: changes in brightness guide our belief of the source of surface discontinuities

High Brightness Gradients Instead of energy terms based on line configuration, use strengths of brightness edges

Low-level Modules Paper mentions: Edge detection Stereo Motion Color Texture But only has short detail on texture & color.

Texture Module Measures level density ‘Blobs’ are taken through a center-surround filter

Color Module Hue = R/(R+G) Should be independent of illumination MRF uses this to segment image into sections of ‘constant reflectance’

Original image + brightness edges

Stereo data, MRF generated depth

Motion data, MRF generated flow

Texture data, MRF generated texture regions

Hue, MRF hue segmentation

Parallelizing Many words about specialized architecture Small processes better for mass computation Specialized experts model

More Recent Recent Mohan, Papageorgiou, Poggio paper: “Example-Based Object Detection in Images by Components” Train an ‘ACC’ using different ‘experts’

Conclusions All extracted surface discontinuities can be used in later understanding “Do brightness edges aid human computation of surface discontinuities?” Parallelizing image analysis…