1. for Vision-Based Navigation
Landmark Selection
for Vision-Based Navigation
Pablo L. Sala
Joint work with Robert Sim, Ali Shokoufandeh and Sven Dickinson
To be presented in IROS 2004
September 17th, 2004

2. Robot Navigation Where am I? Where am I going? How do I get there?
[Leonard and Durrant-Whyte]
Where am I?
Where am I going?
How do I get there?

3. Robot Navigation Where am I? Localization Where am I going?
[Leonard and Durrant-Whyte]
Where am I? Localization
Where am I going?
How do I get there?

4. Robot Navigation Where am I? Localization
[Leonard and Durrant-Whyte]
Where am I? Localization
Where am I going? Goal Identification
How do I get there?

5. Robot Navigation Where am I? Localization
[Leonard and Durrant-Whyte]
Where am I? Localization
Where am I going? Goal Identification
How do I get there? Path-planning

6. Robot Navigation Where am I? Localization
[Leonard and Durrant-Whyte]
Where am I? Localization
Where am I going? Goal Identification
How do I get there? Path-planning

7. Landmark-Based Navigation
What makes a good landmark?

8. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)

9. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility

10. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility
How do we select good landmarks?

11. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility
How do we select good landmarks?
Manually

12. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility
How do we select good landmarks?
Manually
Automatically

13. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility
How do we select good landmarks?
Manually
Automatically… but how?

14. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility
How do we select good landmarks?
Manually
Automatically
Store every landmark visible at each location (costly!)

15. Landmark-Based Navigation
What makes a good landmark?
Distinctiveness (does it tell me where I am?)
Wide Visibility
How do we select good landmarks?
Manually
Automatically
Store every landmark visible at each location (costly!)
Find smallest subset of landmarks that supports reliable navigation (optimal!)

16. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization

17. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Collection of images acquired at known discrete points in pose space. Pose recorded and image features extracted and stored in database.

18. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Collection of images acquired at known discrete points in pose space. Pose recorded and image features extracted and stored in database.

19. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Collection of images acquired at known discrete points in pose space. Pose recorded and image features extracted and stored in database.

20. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Collection of images acquired at known discrete points in pose space. Pose recorded and image features extracted and stored in database.

21. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Collection of images acquired at known discrete points in pose space. Pose recorded and image features extracted and stored in database.

22. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization

23. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Four features are needed in this set.

24. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Four features are needed in this set.
Only two features needed.
Our goal is to find this decomposition.

25. View-Based Robot Navigation
Off-line
Exploration
Landmark
Database
Construction
On-line
Localization
Current pose is estimated using the locations of a small number of features in the current image, matched against their locations in two model views.

26. View-Based Robot Navigation

27. View-Based Robot Navigation

28. View-Based Robot Navigation

29. View-Based Robot Navigation

30. View-Based Robot Navigation

31. View-Based Robot Navigation

32. View-Based Robot Navigation

33. View-Based Robot Navigation

34. View-Based Robot Navigation

35. View-Based Robot Navigation

36. View-Based Robot Navigation

37. Intuitive Problem Formulation

38. Outline Problem Formulation Complexity Heuristic Methods
Results on Synthetic and Real Images
Conclusions

39. A Graph Theoretic Formulation
Problem Definition:
The -Minimum Overlapping Region Decomposition Problem (-MOVRDP) for a world instance <G=(V,E), F, {v} vV> consists of finding a minimum size -overlapping decomposition D = {R1, …, Rd} of V into regions such that:

40. A Graph Theoretic Formulation
Problem Definition:
The -Minimum Overlapping Region Decomposition Problem (-MOVRDP) for a world instance <G=(V,E), F, {v} vV> consists of finding a minimum size -overlapping decomposition D = {R1, …, Rd} of V into regions such that:
Theorem 1: A -MOVRDP can be reduced to an equivalent 0-MOVRDP, and the solution to this latter problem can be extended to a solution for the original problem.

41. A Graph Theoretic Formulation
Problem Definition:
The -Minimum Overlapping Region Decomposition Problem (-MOVRDP) for a world instance <G=(V,E), F, {v} vV> consists of finding a minimum size -overlapping decomposition D = {R1, …, Rd} of V into regions such that:
Theorem 1: A -MOVRDP can be reduced to an equivalent 0-MOVRDP, and the solution to this latter problem can be extended to a solution for the original problem.
Theorem 2: The decision problem <0-MOVRDP, d> is NP-complete.
(Proof by reduction from the Minimum Set Cover Problem.)

42. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.

43. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region:

44. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 25

45. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 25

46. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 19

47. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 19

48. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 19

49. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 19

50. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 17

51. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 17

52. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 14

53. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 14

54. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 11

55. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 11

56. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 9

57. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 8

58. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 8

59. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 6

60. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 4

61. Heuristic Methods for 0-MOVRDP
0-MOVRDP is intractable.
Optimal decomposition not needed in practice.
We developed and tested six greedy approximation algorithms.
Algorithm A.x: O(|V|2|F|)
k = 4
Features commonly visible in region: 4

62. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region:

63. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 1

64. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 1

65. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 1

66. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 1

67. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 1

68. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 1

69. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 2

70. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 2

71. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 2

72. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 2

73. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 2

74. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 3

75. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 4

76. Heuristic Methods for 0-MOVRDP
Algorithms
B.x and C:
O(k|V|2|F|)
k = 5
Features commonly visible in region: 5

77. Results
Simulated Data

78. Results
Simulated Data (cont.)

79. Results Simulated Data (cont.) World Settings
Two types of Worlds: Irregular (Irreg) and Rectangular (Rect).
average diameter: 40m.
pose space sampled at 50 cm intervals.
average number of sides: 6.
average number of obstacles: 7.

80. Results Simulated Data (cont.) World Settings
Two types of Worlds: Irregular (Irreg) and Rectangular (Rect).
average diameter: 40m.
pose space sampled at 50 cm intervals.
average number of sides: 6.
average number of obstacles: 7.
Two types of Features: Short-Range and Long-Range.
visibility range N(0.65, 0.2) to N(12.5, 1) m,
and angular range N(25, 3) degrees.
Visibility range N(0.65, 0.2) to N(17.5, 2) m,
and angular range N(45, 4) degrees.

81. Results (cont.) Simulated Data (cont.) 300 randomly generated worlds
Runtime of few seconds
Avg. size of regions depends on stability of features in pose space.
Number of regions increases as avg. size of regions decreases.
Alg. B.2 achieved the best results.
Algorithms 
A.1
A.2
A.3
B.1
B.2 C
Setting
World
Feature 1
Rect
Short-Range 2
Long-Range 3
Irreg 4

82. Results (cont.) Real Data
We applied the best-performing algorithm (B.2) to real feature visibility data. 0
90
180
270

83. Results (cont.) Real Data Experiment 1 Data collected in 2m  2m area.
Sampled at 20 cm intervals.
Total of 46 visible features.
Camera at a fixed orientation (looking forward).
Features were extracted using the Kanade-Lucas-Tomasi operator.
Parameters used:  = 0, k = 4.

84. Results (cont.) Real Data Experiment 1 Data collected in 2m  2m area.
Sampled at 20 cm intervals.
Total of 46 visible features.
Camera at a fixed orientation (looking forward).
Features were extracted using the Kanade-Lucas-Tomasi operator.
Parameters used:  = 0, k = 4.

85. Results (cont.) Experiment 2 Data collected in 6m  3m area.
Sampled at 25 cm intervals.
Total of 897 visible features.
Camera at 0, 90, 180, and 270
degree orientations.
Lowe’s SIFT features.

86. Typical Feature Visibility Regions
Results (cont.)
Experiment 2
Data collected in 6m  3m area.
Sampled at 25 cm intervals.
Total of 897 visible features.
Camera at 0, 90, 180, and 270
degree orientations.
Lowe’s SIFT features.
Typical Feature Visibility Regions

87. Results (cont.)
Experiment 2 (cont.)
k=4, =0

88. Results (cont.)
Experiment 2 (cont.)
k=4, =1

89. Results (cont.)
Experiment 2 (cont.)
k=10, =0

90. Results (cont.)
Experiment 2 (cont.)
k=10, =1

91. Conclusions
We have introduced a novel graph theoretic formulation of the landmark acquisition problem, and have established its intractability.
We have explored a number of greedy approximation algorithms, systematically testing them on synthetic worlds and demonstrating them on two real worlds.
The resulting decompositions find large regions in the world in which a small number of features can be tracked to support efficient on-line localization.
The formulation and solution are general, and can accommodate other classes of image features.

92. Future Work
Integrate the image collection phase with the region decomposition stage to yield an on-line process for simultaneous exploration and localization (SLAMB).
Path planning through decomposition space, minimizing the number of region transitions in a path.
Detect and cope with environmental change.
Compute the performance guarantee of our heuristic methods and provide tight upper bounds on the quality of our solution compared to the optimal.
Use feature tracking during the image collection stage to achieve larger areas of visibility for each feature. (Maintain equivalence classes of features in the DB.)