1. Detecting Curved Symmetric Parts using a Deformable Disc Model Tom Sie Ho Lee, University of Toronto Sanja Fidler, TTI Chicago Sven Dickinson, University of Toronto Overview Motivation Robustness to curvature Symmetric parts are often curved We capture curvature explicitly and recover the part in one piece 1. Representing symmetric parts Object partMaximal discs Superpixel approximation Results Deformation-invariant space Evaluate symmetry in a warped space invariant to bending and tapering deformations Determine warp by fitting a deformable ellipse to region Extract spatial histogram of boundary edgels Extract interior color and texture features Deformable discs Use compact superpixels as deformable disc hypotheses Superpixels from different scales compose a single part 2. Deformable Disc Affinity 3. Finding sequences Robustness to taper Tapered parts vary in scale along the axis We allow parts to be composed along the axis from disc hypotheses of different scales Symmetric part detection as sequence finding Good symmetry follows a curvilinear axis We find high-affinity sequences of disc hypotheses Optimal sequences are computed using dynamic programming Our sequence formulation avoids branching clusters 81 images of horses Manually annotated symmetric parts as groundtruth regions [Levinshtein et al.] Count a hit when IoU-overlap > 40% between groundtruth and detected regions Weizmann Horse Database (WHD) Symmetric part detector trained on horse images generalizes to diverse objects Symmetry is a powerful and ubiquitous shape regularity Conclusions Source of images of diverse objects on cluttered backgrounds We manually annotated symmetric parts on 36 selected images Berkeley Segmentation Database (BSDS) Qualitative results Perform best-first search using priority queue of candidate sequences Dequeue candidate sequences and consider possible extensions Repeat minimization to find multiple symmetric parts Finding the optimal sequence Cost of a disc sequence Organize deformable disc hypotheses into graph Place edges between adjacent or overlapping discs Find disc sequences in the graph with high affinity Disc hypothesis graph Overview of affinity Define affinity between adjacent disc hypotheses High affinity reflects non-accidental symmetry Adjacent discs occupy a region r on which to extract features Train affinity on region symmetry features Affinity training Learn to map a region r to its affinity σ(r) Generate positive and negative training regions from annotated dataset Extract features on each training region Fit logistic regressor σ(r) to training examples Recovering an object’s generic part structure is a key step in bottom-up object categorization Symmetry has formed the basis of many 2D and 3D generic part representations, e.g. skeletons, shock graphs, generalized cylinders, geons Our goal is to detect 2D symmetric parts in a cluttered image The medial axis transform [Blum] decomposes a shape into the locus of maximal inscribed discs We define a symmetric part as a sequence of deformable discs Discs deform to the shape’s boundary while remaining compact Superpixel approximation Multi-scale composition [Levinshtein et al.] [ours] [Levinshtein et al.] [ours] [Levinshtein et al.] [ours] Deformable ellipse Our ellipse is parameterized by bending and tapering, axis scaling, and rigid transformations Find parameters w that locally minimize non-linear least squares: Cost is normalized by number of discs in sequence Growth term A favors longer sequences Symmetric parts of objects detected by our method Related work Classical skeletons [Blum ‘67; Brady ‘84] are inapplicable to cluttered scenes Filter-based approaches require reliable templates Contour-based approaches require quadratic grouping complexity Region-based grouping [Levinshtein et al. ‘09] offers a good alternative We demonstrate an improvement on [Levinshtein et al.] by capturing more shape variability and applying an optimal algorithm cost(P) can be globally minimized using dynamic programming We use the algorithm of [Felzenszwalb & McAllester ‘06] to compute the global minimum P* warp W spatial histogram Method AP on BSDS AP on WHD baseline [Levinshtein et al.] 10.9%8.0% baseline, with dynamic programming 16.2%10.4% ours, without ternary cost 21.2%12.9% ours22.3%14.2% BSDS WHD boundary edgels Score a sequence P = (d 0, …, d n ) in terms of local affinities σ(d i- 1,d i ) and σ(d i-1,d i,d i+1 ) Affinities favor local grouping of adjacent discs Ternary affinities favor curvilinear axis (smoothness) Convert affinities into binary costs {s i-1,i = 1 - σ(d i-1,d i )} and ternary costs {t i-1,i,i+1 = 1 - σ(d i-1,d i,d i+1 )} Ellipse fitted to a region hypothesis disc hypothesis graph Candidate sequence extensions under consideration