Compact Windows for Visual Correspondence via Minimum Ratio Cycle Algorithm Olga Veksler NEC Labs America

Local Approach Look at one image patch at at time Solve many small problems independently Fast, sufficient for some problems Global Approach Look at the whole image Solve one large problem Slower, more accurate

Local Approach Sufficient for some problems Central problem: window shape selection Efficiently solved using Minimum Ratio Cycle algorithm for graphs

stereo left imageright image disparity = x1-x2 Visual Correspondence (x1,y) (x2,y) motion first imagesecond image vertical motion horizontal motion (x1,y1) (x2,y2)

Local Approach [Levine’73] left image right image p = Common C = i which gives best

Fixed Window Shape Problems true disparities fixed small windowfixed large window left image need different window shapes

Two inefficient methods proposed previously 1.Local greedy search [Levine CGIP’73, Kanade’PAMI94] 2.Direct search [Intille ECCV94,Geiger IJCV95] Variable Window: Previous Work ……. Need efficient optimization algorithm over sizes and shapes

Minimum Ratio Cycle G(V,E) and w(e), t(e): E R image pixels Find cycle C which minimizes: = W t

From Area to Cycle blue edge red edge sum up terms inside using weights of edges

Window Cost C(W) = = size of W + … OK for any graphs not OK for any graph positive negative

Compact Windows p simple graph cycles C one-to-one correspondence compact windows W we construct graph s.t. only clockwise cycles

Cycle which is not Simple    C   C in this case: cycle C

Solving MRC search for smallest s.t. there is negative cycle on graph with edge weights: negative cycle detection takes time due to the special structure of our graphs find smallest s.t. for some cycle   ew   et  ew  et -   ew   et  -

there are are compact windows, if the largest allowed window is n by n Contains all possible rectangles but much more general than just rectangles Find optimal window in in theory, linear ( ) in practice Search over in time perimeter area examples of compact windows (small )

Sample Compact Windows

Speedup for pixel p, the algorithm extends windows over pixels which are likely to have the same disparity as p use the optimal window computed for p to approximate for pixels inside that window

Comparison to Fixed Window Compact windows:16% errors true disparities fixed small window: 33% errorsfixed large window: 30% errors

motion

Algorithm Tsukuba Venus Sawtooth Map Layered Graph cuts Belief prop GC+occl Graph cuts Multiw. Cut Comp. win Results 13 other algorithms, local and global all global Running time: 8 to 22 seconds

Future Work Generalize the window class Generalize objective function –mean? –variance?