CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik

CVR05 University of California Berkeley 2 Introduction CRF Conditional Random Fields on triangulated images, trained to integrate low/mid/high-level grouping cues Approach:

CVR05 University of California Berkeley 3 Joint Contour/Region Inference Xe Yt Contour variables{Xe} Region variables{Yt} Object variables{Z} Z Integrating {Xe},{Yt} and{Z}: low/mid/high-level cues

CVR05 University of California Berkeley 4 Grouping Cues Low-level Cues –Edge energy along edge e –Brightness/texture similarity between two regions s and t Mid-level Cues –Edge collinearity and junction frequency at vertex V –Consistency between edge e and two adjoining regions s and t High-level Cues –Texture similarity of region t to exemplars –Compatibility of region shape with pose –Compatibility of local edge shape with pose Low-level Cues –Edge energy along edge e –Brightness/texture similarity between two regions s and t Mid-level Cues –Edge collinearity and junction frequency at vertex V –Consistency between edge e and two adjoining regions s and t High-level Cues –Texture similarity of region t to exemplars –Compatibility of region shape with pose –Compatibility of local edge shape with pose L 1 (X e |I) L 2 (Y s,Y t |I) M 1 (X V |I) M 2 (X e,Y s,Y t ) H 1 (Y t |I) H 2 (Y t,Z|I) H 3 (X e,Z|I)

CVR05 University of California Berkeley 5 Conditional Random Fields for Cue Integration Estimate the marginal posteriors of X, Y and Z

CVR05 University of California Berkeley 6 H 3 (X e,Z|I): local shape and pose shapeme i (horizontal line) distribution ON(x,y,i) shapeme j (vertical pairs) distribution ON(x,y,j) Let S(x,y) be the shapeme at image location (x,y); (x o,y o ) be the object location in Z. Compute average log likelihood S ON (e,Z) as: Then we have: S OFF (e,Z) is defined similarly.

CVR05 University of California Berkeley 7 Training/Testing Trained on half (172) of the grayscale horse images from the [Borenstein & Ullman 02] Horse Dataset. Use human-marked segmentations to construct groundtruth labels on both CDT edges and triangles. Uses loopy belief propagation for approximate inference; takes < 1 second to converge for a typical image. Parameter estimation with gradient descent for maximum likelihood; converges in 1000 iterations. Tested on the other half of the horse images in grayscale. Quantitative evaluation against groundtruth: precision- recall curves for both contours and regions. Trained on half (172) of the grayscale horse images from the [Borenstein & Ullman 02] Horse Dataset. Use human-marked segmentations to construct groundtruth labels on both CDT edges and triangles. Uses loopy belief propagation for approximate inference; takes < 1 second to converge for a typical image. Parameter estimation with gradient descent for maximum likelihood; converges in 1000 iterations. Tested on the other half of the horse images in grayscale. Quantitative evaluation against groundtruth: precision- recall curves for both contours and regions.

CVR05 University of California Berkeley 8

CVR05 University of California Berkeley 9

CVR05 University of California Berkeley 10 Results InputInput PbOutput ContourOutput Figure

CVR05 University of California Berkeley 11 InputInput PbOutput ContourOutput Figure

CVR05 University of California Berkeley 12 InputInput PbOutput ContourOutput Figure

CVR05 University of California Berkeley 13 Conclusion

CVR05 University of California Berkeley 14 Thank You

CVR05 University of California Berkeley 15