Download presentation
Presentation is loading. Please wait.
1
1 Structure from Motion with Unknown Correspondence (aka: How to combine EM, MCMC, Nonlinear LS!) Frank Dellaert, Steve Seitz, Chuck Thorpe, and Sebastian Thrun slides courtesy of Frank Dellaert, adapted by Steve Seitz
2
2 Traditionally: 2 Problems 1.Correspondence 2.Compute 3D positions, camera motion
3
3 The Correspondence Problem
4
4 The Structure from Motion Problem Find the most likely structure and motion
5
5 j ik = Structure From Motion u ik mimi m i’ xjxj Image i Image i’ Non-linear Least-Squares (Rick Szeliski’s lecture)
6
6 Question How to recover structure and motion with unknown correspondence? Aside: related subproblems –known correspondence, unknown structure, motion structure from motion –known motion, unknown correspondence, structure stereo –known structure, unknown correspondence, motion 3D registration, ICP
7
7 Without Correspondence: Try Tracking Features Lucas-Kanade Tracker (figures from Tommasini et. al. 98)
8
8 New Idea: try all correspondences Likelihood: P( ; U, J) structure + motion image featurescorrespondence Marginalize over J: P( ; U) = J P( ; U, J)
9
9 Combinatorial Explosion 3 images, 4 features: 4! 3 =13,824 5 images, 30 features: 30! 5 =1.3131e+162 (number of stars:1e+20, atoms: 1e+79) Computing P( ; U, J) is intractable !
10
10 EM for Correspondence
11
11 Expectation Maximization
12
12 E-Step: Soft Correspondences View 1View 3View 2 P(J | U, t )
13
13 M-Step: SFM Compute expected SFM solution Problem: this requires solving an exponential number of SFM problems If we assume correspondences j ik are independent, we can simplify...
14
14 Virtual Measurements View 2 v A = p 1A u 1 + p 2A u 2 + p 3A u 3 + p 4A u 4
15
15 j ik M-Step: SFM v ij mimi m i’ xjxj Image i Image i’ one SFM problem on virtual measurements
16
16 Structure from Motion without Correspondence via EM: In each image, calculate the n 2 “soft correspondences”
17
17 Key Issue: Calculating Weights
18
18 Calculating Weights Weights = Marginal Probabilities
19
19 Mutual Exclusion
20
20 Monte Carlo EM
21
21 Monte Carlo Approximation
22
22 Sampling Assignments using the Metropolis Algorithm (1953) P( ) = -------------- P( )
23
23 Swap Proposals (inefficient) a b c d e f
24
24 ‘Chain Flipping’ Proposals
25
25 Efficient Sampling (Example)
26
26 a b c d e f MCMC to deal with mutex
27
27 Input Images Manual measurements Random order
28
28 Results: Cube
29
29 Soft Correspondences scene feature x j uiui EM iterations
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.