Download presentation
Presentation is loading. Please wait.
Published byNicholas Park Modified over 9 years ago
1
1 Long-term image-based motion estimation Dennis Strelow and Sanjiv Singh
2
2 On the web Related materials: these slides related papers movies VRML models at: http://www.cs.cmu.edu/~dstrelow/maryland
3
3 Introduction (1) micro air vehicle (MAV) navigation AeroVironment Black WidowAeroVironment Microbat
4
4 Introduction (2) mars rover navigation Mars Exploration Rovers (MER)Hyperion
5
5 Introduction (3) robotic search and rescue Rhex Center for Robot-Assisted Search and Rescue, U. of South Florida
6
6 Introduction (4) NASA ISS personal satellite assistant
7
7 Introduction (5) Each of these problems requires: 6 DOF motion in unknown environments without GPS or other absolute positioning over the long term …and some of the problems require: small, light, and cheap sensors
8
8 Introduction (6) Monocular, image-based motion estimation is a good candidate In particular, simultaneous estimation of: multiframe motion sparse scene structure is the most promising approach
9
9 Outline Image-based motion estimation Improving image-based motion estimation Improving feature tracking Reacquisition
10
10 Outline Image-based motion estimation refresher difficulties Improving image-based motion estimation Improving feature tracking Reacquisition
11
11 Image-based motion estimation: refresher (1) A two-step process is typical… First, sparse feature tracking: Inputs: raw images Outputs: projections
12
12 Image-based motion estimation: refresher (2)
13
13 Image-based motion estimation: refresher (3) Second, estimation: Input: Outputs: 6 DOF camera position at the time of each image 3D position of each tracked point projections from tracker
14
14 Image-based motion estimation: refresher (4)
15
15 Image-based motion estimation: refresher (5) Algorithms exist For tracking: Lucas-Kanade (Lucas and Kanade, 1981)
16
16 Image-based motion estimation: refresher (6) For estimation: SVD-based factorization (Tomasi and Kanade, 1992) bundle adjustment (various, 1950’s) Kalman filtering (Broida and Chellappa, 1990) variable state dimension filter (McLauchlan, 1996)
17
17 Image-based motion estimation: difficulties (1) So, the problem is solved?
18
18 Image-based motion estimation: difficulties (2) If so, where are the automatic systems for estimating the motion of: in unknown environments? from images in unknown environments?
19
19 Image-based motion estimation: difficulties (3) …and for automatically modeling rooms buildings cities from a handheld camera?
20
20 Image-based motion estimation: difficulties (4) Estimation step can be very sensitive to: incorrect or insufficient image feature tracking camera modeling and calibration errors outlier detection thresholds sequences with degenerate camera motions
21
21 Image-based motion estimation: difficulties (5) …and for recursive methods in particular: poor prior assumptions on the motion poor approximations in state error modeling
22
22 Image-based motion estimation: difficulties (6) 151 images, 23 points
23
23 Image-based motion estimation: difficulties (7)
24
24 Image-based motion estimation: difficulties (8) For long-term motion estimation, these errors accumulate
25
25 Outline Image-based motion estimation Improving image-based motion estimation overview image and inertial measurements Improving feature tracking Reacquisition
26
26 Improving image-based motion estimation: overview
27
27 Improving image-based motion estimation: overview
28
28 Improving image-based motion estimation: image and inertial (1) Image and inertial measurements are highly complimentary Inertial measurements can: resolve the ambiguities in image-only estimates establish the global scale
29
29 Improving image-based motion estimation: image and inertial (2) Images measurements can: reduce the drift in integrating inertial measurements distinguish between rotation, gravity, acceleration, bias, noise in accelerometer readings
30
30 Improving image-based motion estimation: image and inertial (3)
31
31 Improving image-based motion estimation: image and inertial (4)
32
32 Improving image-based motion estimation: image and inertial (5) Other examples: global scale typically within 5% better convergence than image-only estimation
33
33 Improving image-based motion estimation: image and inertial (6) Many more details in: Dennis Strelow and Sanjiv Singh. Motion estimation from image and inertial measurements. IJRR, September 2004.
34
34 Outline Image-based motion estimation Improving image-based motion estimation Improving feature tracking Lucas-Kanade and real sequences The “smalls” tracker Reacquisition
35
35 Improving feature tracking: Lucas- Kanade and real sequences (1) Lucas-Kanade is the “go to” sparse feature tracker: iterative minimization of the intensity matching error function applied at several image resolutions to handle large motions features extracted based on image texture feature death based on iteration convergence and correlation error
36
36 Improving feature tracking: Lucas- Kanade and real sequences (2) Advantages: fast subpixel resolution can handle some large motions well uses general minimization, so easily extendible
37
37 Improving feature tracking: Lucas- Kanade and real sequences (3) 0.1 average pixel reprojection error!
38
38 Improving feature tracking: Lucas- Kanade and real sequences (4) But, Lucas-Kanade has some flaws: does not exploit the rigid scene poor heuristics for: large motions extracting features detecting feature mistracking
39
39 Improving feature tracking: Lucas- Kanade and real sequences (5)
40
40 Improving feature tracking: Lucas- Kanade and real sequences (6)
41
41 Improving feature tracking: Lucas- Kanade and real sequences (7)
42
42 Improving feature tracking: Lucas- Kanade and real sequences (7)
43
43 Improving feature tracking: Lucas- Kanade and real sequences (8)
44
44 Improving feature tracking: the “smalls” tracker (1) smalls is a new sparse image feature tracker designed to address these issues i.e., designed for long-term motion estimation
45
45 Improving feature tracking: the “smalls” tracker (2) Leonard Smalls: tracker, lone biker of the apocalypse
46
46 Improving feature tracking: the “smalls” tracker (3) epipolar geometry 1-D correlation matching along epipolar lines geometric mistracking detection feature death and birth outputto 6 DOF SIFT featuresestimation features
47
47 Improving feature tracking: the “smalls” tracker (4) SIFT keypoints (Lowe, IJCV 2004): image interest points can be extracted despite of large changes in viewpoint to subpixel accuracy A keypoint’s feature vectors in two images usually match
48
48 Improving feature tracking: the “smalls” tracker (5) Epipolar geometry between adjacent images is determined using… SIFT extraction and matching two-frame bundle adjustment RANSAC epipolar geometry SIFT features
49
49 Improving feature tracking: the “smalls” tracker (6) initial search position from nearby SIFT matches discrete SSD search (e.g., 60 pixels) 1-D Lucas-Kanade refines the match 1-D correlation matching along epipolar lines
50
50 Improving feature tracking: the “smalls” tracker (7) To check for mistracking, use only three- frame geometric consistency… geometric mistracking detection …determined using: three-frame bundle adjustment RANSAC
51
51 Improving feature tracking: the “smalls” tracker (8) After tracking in each image: features are pruned to maintain a minimum separation new features are selected in those parts of the image not already covered feature death and birth outputto 6 DOF featuresestimation
52
52 Improving feature tracking: the “smalls” tracker (9)
53
53 Improving feature tracking: the “smalls” tracker (10)
54
54 Improving feature tracking: the “smalls” tracker (11)
55
55 Improving feature tracking: the “smalls” tracker (12)
56
56 Outline Image-based motion estimation Improving image-based motion estimation Improving feature tracking Reacquisition
57
57 Reacquisition (1) Closing the loop to deal with drift: (1) recognizing revisited features (2) exploiting revisited features in the estimation
58
58 Future work (2)
59
59 Reacquisition (3) Image 149 close to… …image 95? Yes, but closer to image 84.
60
60 Reacquisition (4) …image 84? Yes, but closer to image 83. Image 149 close to…
61
61 Reacquisition (5) …image 83? Yes, and 83 is closest. Image 149 close to…
62
62 Thanks! Related materials: these slides related papers movies VRML models at: http://www.cs.cmu.edu/~dstrelow/maryland
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.