1. Computer Vision Group University of California Berkeley Estimating Human Body Configurations using Shape Context Matching Greg Mori and Jitendra Malik

2. Computer Vision Group University of California Berkeley Problem

3. Computer Vision Group University of California Berkeley Approach: Exemplar-based Matching Set of stored exemplars with hand-labeled keypoints Obtain sample points Deformable matching to exemplars: –Shape context matching to get correspondences –Kinematic chain deformation model Estimate 3D body configuration

4. Computer Vision Group University of California Berkeley Comparing Pointsets

5. Computer Vision Group University of California Berkeley Shape Context Count the number of points inside each bin, e.g.: Count = 4 Count = 10... FCompact representation of distribution of points relative to each point

6. Computer Vision Group University of California Berkeley Shape Context

7. Computer Vision Group University of California Berkeley Comparing Shape Contexts Compute matching costs using Chi Squared distance: Recover correspondences by solving linear assignment problem with costs C ij [Jonker & Volgenant 1987]

8. Computer Vision Group University of California Berkeley Deformable Matching Kinematic chain-based deformation model Use iterations of correspondence and deformation Keypoints on exemplars are deformed to locations on query image

9. Computer Vision Group University of California Berkeley

10. Computer Vision Group University of California Berkeley Problem

11. Computer Vision Group University of California Berkeley Estimate 3D Body Configuration [Taylor ’00] Known: –Relative lengths of body segments –(x,y) Image locations of keypoints –“closer endpoint” labels for each segment –Scaled orthographic camera model Solve for 3D locations of keypoints up to some scale factor –Scale factor can be estimated automatically

12. Computer Vision Group University of California Berkeley Solving for Foreshortening

13. Computer Vision Group University of California Berkeley Choosing Scale

14. Computer Vision Group University of California Berkeley Results

15. Computer Vision Group University of California Berkeley

16. Computer Vision Group University of California Berkeley Multiple Exemplars Parts-based approach –Use a combination of keypoints/whole limbs from different exemplars –Reduces the number of exemplars needed Compute a matching cost for each limb from every exemplar Compute pairwise “consistency” costs for neighbouring limbs Use dynamic programming to find best K configurations

17. Computer Vision Group University of California Berkeley Parts-based Approach

18. Computer Vision Group University of California Berkeley Tracking by Repeated Finding