Computer Vision Group University of California Berkeley Ecological Statistics of Good Continuation: Multi-scale Markov Models for Contours Xiaofeng Ren and Jitendra Malik

Computer Vision Group University of California Berkeley Good Continuation Wertheimer ’23 Kanizsa ’55 von der Heydt, Peterhans & Baumgartner ’84 Kellman & Shipley ’91 Field, Hayes & Hess ’93 Kapadia, Westheimer & Gilbert ’00 … Parent & Zucker ’89 Heitger & von der Heydt ’93 Mumford ’94 Williams & Jacobs ’95 … Wertheimer ’23 Kanizsa ’55 von der Heydt, Peterhans & Baumgartner ’84 Kellman & Shipley ’91 Field, Hayes & Hess ’93 Kapadia, Westheimer & Gilbert ’00 … Parent & Zucker ’89 Heitger & von der Heydt ’93 Mumford ’94 Williams & Jacobs ’95 …

Computer Vision Group University of California Berkeley Approach: Ecological Statistics Brunswick & Kamiya ’53 Ruderman ’94 Huang & Mumford ’99 Martin et. al. ’01 Brunswick & Kamiya ’53 Ruderman ’94 Huang & Mumford ’99 Martin et. al. ’01 E. Brunswick Ecological validity of perceptual cues: characteristics of perception match to underlying statistical properties of the environment Gibson ’66 Olshausen & Field ’96 Geisler et. al. ’01 … Gibson ’66 Olshausen & Field ’96 Geisler et. al. ’01 …

Computer Vision Group University of California Berkeley Human-Segmented Natural Images D. Martin et. al., ICCV ,000 images, >14,000 segmentations

Computer Vision Group University of California Berkeley More Examples D. Martin et. al. ICCV 2001

Computer Vision Group University of California Berkeley Segmentations are Consistent A BC A,C are refinements of B A,C are mutual refinements A,B,C represent the same percept Attention accounts for differences Image BGL-birdR-bird grass bush head eye beak far body head eye beak body Perceptual organization forms a tree: Two segmentations are consistent when they can be explained by the same segmentation tree (i.e. they could be derived from a single perceptual organization).

Computer Vision Group University of California Berkeley Outline of Experiments Prior model of contours in natural images –First-order Markov model Test of Markov property –Multi-scale Markov models Information-theoretic evaluation Contour synthesis Good continuation algorithm and results Prior model of contours in natural images –First-order Markov model Test of Markov property –Multi-scale Markov models Information-theoretic evaluation Contour synthesis Good continuation algorithm and results

Computer Vision Group University of California Berkeley Contour Geometry First-Order Markov Model ( Mumford ’94, Williams & Jacobs ’95 ) –Curvature: white noise ( independent from position to position ) –Tangent t(s): random walk –Markov property: the tangent at the next position, t(s+1), only depends on the current tangent t(s) First-Order Markov Model ( Mumford ’94, Williams & Jacobs ’95 ) –Curvature: white noise ( independent from position to position ) –Tangent t(s): random walk –Markov property: the tangent at the next position, t(s+1), only depends on the current tangent t(s) t(s) t(s+1) s s+1

Computer Vision Group University of California Berkeley Test of Markov Property Segment the contours at high-curvature positions

Computer Vision Group University of California Berkeley Prediction: Exponential Distribution If the first-order Markov property holds… At every step, there is a constant probability p that a high curvature event will occur High curvature events are independent from step to step  Then the probability of finding a segment of length k with no high curvature is (1-p) k If the first-order Markov property holds… At every step, there is a constant probability p that a high curvature event will occur High curvature events are independent from step to step  Then the probability of finding a segment of length k with no high curvature is (1-p) k

Computer Vision Group University of California Berkeley Exponential ? Empirical Distribution NO

Computer Vision Group University of California Berkeley Empirical Distribution: Power Law Contour segment length Probability density

Computer Vision Group University of California Berkeley Power Laws in Nature Power Law widely exists in nature –Brightness of stars –Magnitude of earthquakes –Population of cities –Word frequency in natural languages –Revenue of commercial corporations –Connectivity in Internet topology … Usually characterized by self-similarity and multi-scale phenomena Power Law widely exists in nature –Brightness of stars –Magnitude of earthquakes –Population of cities –Word frequency in natural languages –Revenue of commercial corporations –Connectivity in Internet topology … Usually characterized by self-similarity and multi-scale phenomena

Computer Vision Group University of California Berkeley Multi-scale Markov Models Assume knowledge of contour orientation at coarser scales t(s) t(s+1) 2 nd Order Markov: P( t(s+1) | t(s), t (1) (s+1) ) Higher Order Models: P( t(s+1) | t(s), t (1) (s+1), t (2) (s+1), … ) s+1 s t (1) (s+1) s+1

Computer Vision Group University of California Berkeley Information Gain in Multi-scale 14.6% of total entropy ( at order 5 ) H( t(s+1) | t(s), t (1) (s+1), t (2) (s+1), … )

Computer Vision Group University of California Berkeley Contour Synthesis Multi-scale Markov First-Order Markov

Computer Vision Group University of California Berkeley Multi-scale Contour Completion Coarse-to-Fine –Coarse-scale completes large gaps –Fine-scale detects details Completed contours at coarser scales are used in the higher-order Markov models of contour prior for finer scales P( t(s+1) | t(s), t (1) (s+1), … ) Coarse-to-Fine –Coarse-scale completes large gaps –Fine-scale detects details Completed contours at coarser scales are used in the higher-order Markov models of contour prior for finer scales P( t(s+1) | t(s), t (1) (s+1), … )

Computer Vision Group University of California Berkeley Multi-scale: Example input coarse scalefine scale w/o multi-scale fine scale w/ multi-scale

Computer Vision Group University of California Berkeley Our resultCanny Comparison: same number of edge pixels

Computer Vision Group University of California Berkeley Our resultCanny Comparison: same number of edge pixels

Computer Vision Group University of California Berkeley Conclusion Contours are multi-scale in nature; the first-order Markov property does not hold for contours in natural images. Higher-order Markov models explicitly model the multi-scale nature of contours. We have shown: –The information gain is significant –Synthesized contours are smooth and rich in structure –Efficient good continuation algorithm has produced promising results Contours are multi-scale in nature; the first-order Markov property does not hold for contours in natural images. Higher-order Markov models explicitly model the multi-scale nature of contours. We have shown: –The information gain is significant –Synthesized contours are smooth and rich in structure –Efficient good continuation algorithm has produced promising results Ren & Malik, ECCV 2002

Computer Vision Group University of California Berkeley Thank You