Download presentation
Presentation is loading. Please wait.
Published byAthena Delbridge Modified over 10 years ago
1
Putting Objects in Perspective Derek Hoiem Alexei A. Efros Martial Hebert Carnegie Mellon University Robotics Institute
2
Understanding an Image
3
Today: Local and Independent
4
What the Detector Sees
5
Local Object Detection True Detection True Detections Missed False Detections Local Detector: [Dalal-Triggs 2005]
6
Work in Context Image understanding in the 70’s Guzman (SEE) 1968 Hansen & Riseman (VISIONS) 1978 Barrow & Tenenbaum 1978 Yakimovsky & Feldman 1973 Brooks (ACRONYM) 1979 Marr 1982 Ohta & Kanade 1973 Recent work in 2D context Kumar & Hebert 2005 Torralba, Murphy, Freeman 2004 Fink & Perona 2003 He, Zemel, Cerreira-Perpiñán 2004 Carbonetto, Freitas, Banard 2004 Winn & Shotton 2006
7
Real Relationships are 3D Close Not Close
8
Recent Work in 3D [Torralba, Murphy & Freeman 2003] [Han & Zu 2003] [Oliva & Torralba 2001] [Han & Zu 2005]
9
Biederman’s Relations among Objects in a Well-Formed Scene (1981): – Support – Size – Position – Interposition – Likelihood of Appearance Objects and Scenes Hock, Romanski, Galie, & Williams 1978
10
Biederman’s Relations among Objects in a Well-Formed Scene (1981): Hock, Romanski, Galie, & Williams 1978 – Support – Size – Position – Interposition – Likelihood of Appearance Contribution of this Paper
11
Object Support
12
Surface Estimation Image SupportVerticalSky V-Left V-Center V-RightV-PorousV-Solid [Hoiem, Efros, Hebert ICCV 2005] Software available online Object Surface? Support?
13
Object Size in the Image Image World
14
Input Image Object Size ↔ Camera Viewpoint Loose Viewpoint Prior
15
Input Image Object Size ↔ Camera Viewpoint Loose Viewpoint Prior
16
Object Position/Sizes Viewpoint Object Size ↔ Camera Viewpoint
17
Object Position/Sizes Viewpoint Object Size ↔ Camera Viewpoint
18
Object Position/Sizes Viewpoint Object Size ↔ Camera Viewpoint
19
Object Position/Sizes Viewpoint
20
What does surface and viewpoint say about objects? Image P(object) P(object | surfaces) P(surfaces) P(viewpoint) P(object | viewpoint)
21
Image P(object | surfaces, viewpoint) What does surface and viewpoint say about objects? P(object) P(surfaces)P(viewpoint)
22
Scene Parts Are All Interconnected Objects 3D Surfaces Camera Viewpoint
23
Input to Our Algorithm Surface EstimatesViewpoint Prior Surfaces: [Hoiem-Efros-Hebert 2005] Local Car Detector Local Ped Detector Object Detection Local Detector: [Dalal-Triggs 2005]
24
Scene Parts Are All Interconnected Objects 3D Surfaces Viewpoint
25
Our Approximate Model Objects 3D Surfaces Viewpoint
26
s1s1 o1o1 θ onon... snsn … Local Object Evidence Local Surface Evidence Local Object Evidence Local Surface Evidence Viewpoint Objects Local Surfaces Inference over Tree Easy with BP
27
Viewpoint estimation Viewpoint Prior Horizon Height Horizon Likelihood Viewpoint Final
28
Object detection 4 TP / 2 FP 3 TP / 2 FP 4 TP / 1 FP Ped Detection Car Detection Local Detector: [Dalal-Triggs 2005] 4 TP / 0 FP Car: TP / FP Ped: TP / FP Initial (Local)Final (Global)
29
Experiments on LabelMe Dataset Testing with LabelMe dataset: 422 images – 923 Cars at least 14 pixels tall – 720 Peds at least 36 pixels tall
30
Each piece of evidence improves performance Local Detector from [Murphy-Torralba-Freeman 2003] Car Detection Pedestrian Detection
31
Can be used with any detector that outputs confidences Local Detector: [Dalal-Triggs 2005] (SVM-based) Car Detection Pedestrian Detection
32
Accurate Horizon Estimation Median Error: 8.5 % 4.5 % 3.0 % 90% Bound: [Murphy-Torralba- Freeman 2003] [Dalal- Triggs 2005] Horizon Prior
33
Qualitative Results Initial: 2 TP / 3 FPFinal: 7 TP / 4 FP Local Detector from [Murphy-Torralba-Freeman 2003] Car: TP / FP Ped: TP / FP
34
Qualitative Results Local Detector from [Murphy-Torralba-Freeman 2003] Car: TP / FP Ped: TP / FP Initial: 1 TP / 14 FPFinal: 3 TP / 5 FP
35
Qualitative Results Car: TP / FP Ped: TP / FP Local Detector from [Murphy-Torralba-Freeman 2003] Initial: 1 TP / 23 FPFinal: 0 TP / 10 FP
36
Qualitative Results Local Detector from [Murphy-Torralba-Freeman 2003] Car: TP / FP Ped: TP / FP Initial: 0 TP / 6 FPFinal: 4 TP / 3 FP
37
Summary & Future Work meters Ped Car Reasoning in 3D: Object to object Scene label Object segmentation
38
Conclusion Image understanding is a 3D problem – Must be solved jointly This paper is a small step – Much remains to be done
39
Thank you
41
A Return to Scene Understanding Guzman (SEE), 1968 Hansen & Riseman (VISIONS), 1978 Barrow & Tenenbaum 1978 Brooks (ACRONYM), 1979 Marr, 1982 Ohta & Kanade, 1978 Yakimovsky & Feldman, 1973 [Ohta & Kanade 1978]
42
Images
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.