1. Range Image Segmentation for Modeling and Object Detection in Urban Scenes
Cecilia Chen & Ioannis Stamos Computer Science Department Graduate Center, Hunter College The City University of New York

2. Motivations Extract geometrically meaningful segments
Model large-scale urban scenes
Provide high-level scene understanding

3. Step 1 – Range Segmentation
Range Scan
Planar/Smooth Regions
Non-smooth Regions +

4. Step 2 – Segment Merging

5. Planar/Smooth Regions
Range Segmentation
Range Scans
Planar/Smooth Regions
Non-smooth Regions +

6. Range Segmentation
Details
All segments

7. Range Segmentation Normal computation Planar/Smooth region extraction
Identify all locally-planar points
Compute surface normal for all points
Planar/Smooth region extraction
Region growing based on local plane continuity
Plane fitting to differentiate two region types
Non-smooth region extraction
Region growing based on spatial continuity

8. Range Segmentation Normal computation Planar points (a)(b)(c)
Local plane fitting
Fitting error < θfit
Planar edge points (d)(e)
Find planar neighbors
Select a best local plane from those of neighbors
Non-planar points (f) P
(d) N2 F S1 S2 d2 N1 P
(a) S F d1
(e) F S2 S1 P
(b) S F P
(c) S F
(f) F S

9. Range Segmentation Planar/Smooth region growing
N (a neighbor of P) belongs to P’s region if
Distance (N, P’s local plane) < θpoint2plane
Intensity differs by <θintensity
Angle (P’s local plane, N’s local plane) < θangle
Non-smooth region growing
N (a neighbor of P) belongs to P’s region if
Distance (N, P) < θpoint2point
Intensity differs by <θintensity
Angle (P’s local plane, N’s local plane) < θ’angle

10. Range Segmentation Cooper Union results Execution time < 2 minutes
Quality is shown in model generation results

11. Range Segmentation
Planar/smooth and non-smooth regions

12. Range Segmentation Previous approaches Our algorithm
Only planar and smooth regions
Inaccurate region boundaries
Non-smooth regions are discard or approximated with smaller planar segments
Our algorithm
Planar, smooth, non-smooth regions
Accurate region boundaries
Non-smooth regions are grown based on spatial closeness, preserving original surface shapes

13. Segment Merging and Modeling……
Segmented Scans
Segment Merging
Range Registration
Merged Segments
Surface mesh per segment

14. Merging Segments Same region Identify overlapping regions
Merge overlapping regions

15. Merging Segments Algorithm design Advantages
Only consider segments at area of overlap
Utilize grid structure and generate z-buffers
Use each range image as pivot image
Transform and project other range images
Identify and merge overlapping segment sets
Advantages
Time efficient, due to grid utilization
Memory efficient, as only overlapping scans need to be in memory
Ability to be parallelized

16. Merging Segments (2 scans)
Merging two range scans I1 and I2
I1 has n1 segments; I2 has n2 segments. I2 I1
(Segment label in transformed I2,
depth, surface normal)
(Segment label in I1,
depth, surface normal)
Z21
Z11

17. Merging Segments (2 scans)
Two scans merged

18. Merging Segments (all scans)
Merging range scans I1, I2, …, In
Each scan (Ii) as pivot; other scans tranformed Ik
Zki Ij Ii
(Segment label in transformed Ij,
depth, surface normal)
(Segment label in Ii,
depth, surface normal)
Zji
Zii

19. Experimental Results Cooper Union building, NYC
8 scans merged in 4 minutes
Average surface fitting error of planar segments is 3mm
1760 planar region
382 non-planar regions

20. Experimental Results Details of merged segments Merged points
Surface meshes

21. Experimental Results Grand Central Station, NYC
15 range scans merged in 10 minutes
Average surface fitting error of planar segments is 4mm
1393 planar regions
787 non-planar regions

22. Experimental Results Details of merged segments Merged points
Surface meshes

23. Experimental Results
Surface meshes per segment

24. Conclusions and Future Work
Achievements
Extracting geometrically meaningful segments
Providing Accurate region boundaries
Merging segments for modeling
Future Work
Analysis of segment spatial relationship for high level scene understanding
Feature extraction from each segment for object recognition

25. Thanks !