Download presentation
Presentation is loading. Please wait.
Published byEthel Phelps Modified over 9 years ago
1
3D LayoutCRF Derek Hoiem Carsten Rother John Winn
2
2 Goal 1: Object Description Object Description: Bounding Box Viewpoint Color Pose Subclass
3
3 Goal 2: Object Segmentation
4
4 Combine object-level and pixel-level reasoning Key Idea
5
5 Recognition Requires Object-Level Reasoning Position Shape/Size Viewpoint/Pose Style/Color
6
6 Recognition Requires Object-Level Reasoning
7
7 Solution: Window Detector? 45 degree range of viewpoints Minor scale/position variation
8
8 What if we have a really good model?
9
9 Recognition Requires Part-Level Reasoning Propose good global model
10
10 Recognition Requires Part-Level Reasoning Propose good global model Occlusions
11
11 Context Requires Both Object and Part-Level Info Size relationships require object model
12
12 Context Requires Both Object and Part-Level Info Surface relationships require occlusion info Visibly sitting on ground Not visibly sitting on ground
13
13 Our Object/Part Model T i = { h j object parts bounding box, viewpoint, color model, instance cost } part consistency occlusions TmTm h1h1 h2h2 h3h3 h4h4 h5h5 h6h6 h7h7 h8h8 h9h9 h 10 h 11 hnhn x … … Extension from [Winn Shotton 2006] T1T1 …
14
14 Modeling Viewpoint Parameterized by Bounding Box and Corner
15
15 Assigning Parts from Model Training Image F L Training Annotation Assigned Parts 3D Parts Model
16
16 Part Assignment Consistency
17
17 Relabeling Allowing slight deformations, relabel training data Training Image Original Labels New Labels
18
18 Eight Viewpoint/Scale Ranges Height Range Appearance (but not location) constant within each range
19
19 Eight Viewpoint Ranges Left – Back 1Left - Back 2Front-Left 1Front - Left 2 Back - Right 1Back - Right 2Right – Front 1Right - Front 2 Mirrored
20
20 Modeling Part Appearance Template patches (normalized xcorr) Intensity / Color Image Edges (DT)
21
21 Modeling Part Appearance Randomized decision trees –25 trees, 250 leaf nodes Once: –Learn structure on 50,000 object / 50,000 background pixels For each appearance model: –Learn parameters on all pixels (850 LabelMe images)
22
22 Inference Input Image
23
23 Inference Input Image Proposals One per appearance model Objects proposed by connected components
24
24 Proposal Stage Model h i object parts part consistency occlusions h1h1 h2h2 h3h3 h4h4 h5h5 h6h6 h7h7 h8h8 h9h9 h 10 h 11 hnhn x … … CRF Inference (TRW-BP)
25
25 Inference Refinement One per proposal Incorporate viewpoint, size information Proposals Input Image
26
26 Refinement Stage Model T i = { h i object parts bounding box, viewpoint } part consistency occlusions T1T1 h1h1 h2h2 h3h3 h4h4 h5h5 h6h6 h7h7 h8h8 h9h9 h 10 h 11 hnhn x … …
27
27 Inference Refinement Proposals Arbitration Includes color model, instance penalty (graph cuts) Input Image
28
28 Preliminary Results on UIUC Trained on 20, tested on rest Quantitatively comparable to best
29
29 Preliminary Results on UIUC Without Instance Cost With Instance Cost T1T1 h1h1 h2h2 h3h3 h4h4 h5h5 h6h6 h7h7 h8h8 h9h9 h 10 h 11 hnhn x … …
30
30 Preliminary Results on PASCAL’06 25 images –One proposal (viewpoint within 45 degrees, scale of 26-38 pixels)
31
31 Preliminary Results on PASCAL’06
32
32 Preliminary Results on PASCAL’06
33
33 Preliminary Results on PASCAL’06 Without Color Model With Color Model
34
34 Conclusion Combined object-level and pixel-level reasoning –Object-level: Position/Size, Viewpoint, Color –Pixel-level: Part appearance, Occlusion reasoning Good preliminary results
35
35 To Do Further refinement of color model (e.g. could have color/texture-based instance cost) Obtain specific 3D model from detection and texture-map onto it (tbd by Carsten ) Quantitative evaluation on Pascal –Evaluate color model, instance penalty, etc. –Compare to John’s UIUC results
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.