1. R I T Rochester Institute of Technology Geometric Scene Reconstruction Using 3-D Point Cloud Data Research Plan Feng Li and Steve Lach Advanced Digital Image Processing: SIMG786 April 6, 2006

2. R I T Rochester Institute of Technology 2 Overview LIDAR Basics and Terminology Purpose/Overview Research Overview Approach Schedule Summary

3. R I T Rochester Institute of Technology 3 LIDAR Basics LIDAR: LIght Detection And Ranging LIDAR works like a radar system except it uses light instead of radio frequencies  GPS: measure LIDAR sensor in the air  IMU (Inertial Measuring Unit): measure the roll, pitch and heading of the aircraft  LIDAR sensor: measure angular orientation of laser pulse; measure time- interval between light reflects off the object and returns to the sensor www.ncfloodmaps.com

4. R I T Rochester Institute of Technology 4 LIDAR Basics - Continued Time of flight system produces 3D spatial imagery – Range resolution: ability to resolve two separate objects in depth – More sophisticated techniques discussed elsewhere – First return, interpolated DEM (range-based) is most common data set produced by commercial vendors Intensity image yields additional information 281989.740360 4773791.856553 134.944699 187 281990.740358 4773791.854912 134.971924 187 281991.740357 4773791.853271 135.253379 187 281992.740356 4773791.851630 136.049035 187 281993.740354 4773791.849989 136.452911 187

5. R I T Rochester Institute of Technology 5 Some Additional Terminology Our Terminology Point Cloud – Set of irregularly spaced 3D points Range Image - 3D points at regular locations on an x, y grid Digital Elevation Model (DEM)/Digital Surface Model (DSM) – Blanket over scene Digital Terrain Model (DTM) – Bare Earth (remove trees, buildings, etc) Voids – Portions of a DEM where no elevation data is available Image courtesy of Ma, “BUILDING MODEL RECONSTRUCTION FROM LIDAR DATA AND AERIAL PHOTOGRAPHS”, PhD Dissertation, OSU, 2004.

6. R I T Rochester Institute of Technology 6 Underlying Problem Through courses, we have learned to process regularly- sampled images However, we have a much smaller toolset for processing point cloud data We will develop tools for this new “image” type, and apply to data from RIT LIDAR collect

7. R I T Rochester Institute of Technology 7 Three 3D Working Environments Range Image (Interpolate irregularly distributed points to a regular grid) – Traditional, Somewhat Simple – Cost: loss information or lead to biases (Mixture of points from different categories) Point Cloud – Accurate – Difficult to Process; Can not use standard image processing routines (median filter, FFT) Combination of the Two – Use Range Image to classify regions – Use Point Cloud to produce accurate results

8. R I T Rochester Institute of Technology 8 Purpose of Project Use 3-D Point Data to geometrically reconstruct a scene Goal: Determine Appropriate Ways to: 1. Efficiently handle the data (relationship between the points) 2. Operate on the data (filter, interpolate, etc) 3. Extract objects (buildings and trees) 4. Reconstruct objects (buildings and trees) Focus will primarily be on Item #4, Other items will be done as required to support this task (additional work if schedule permits)

9. R I T Rochester Institute of Technology 9 Research Overview Identify Ground Points Create Geometric Terrain Model Classify Non-Ground points Create Spectral Terrain Model Overlay Spectral Textures Create Geometric Object Models Learn to Process 3D Data

10. R I T Rochester Institute of Technology 10 Approach: Goal #1 (Data Handling) Use range image to isolate groupings of points belonging to a single object Pixelize or Voxelize the data for each object (unsure which is preferable) Add extra data fields (i.e. X, Y, Z, I, Obj, Vox, Sub Vox, Neighbors, Distances, Directions) Initial Processing Likely to be computationally intense

11. R I T Rochester Institute of Technology 11 Approach : Goal #2 (Point Cloud Filtering) Use sliding window approaches – functional relationships based on distances, not discrete kernel Anticipate creating Median, Mean (low-pass), Weighted Mean, Differencing (high-pass)

12. R I T Rochester Institute of Technology 12 Approach : Goal #3 (Object Extraction) 3 Steps in achieving this goal: First Step: Generate range image - may need to work on interpolation techniques, right now will use pre-packaged routine

13. R I T Rochester Institute of Technology 13 Approach : Goal #3 (Object Extraction) Second Step: Use range image to generate DTM via Median Filtering and High-Passs Filtering

14. R I T Rochester Institute of Technology 14 Generating DTM LIDAR Point Data Initial Data Filtering Remove Non- ground pixels Final Data Filtering Thresholding above Global Estimated Ground Polynomial Thresholding above Local Estimated Ground Polynomial Threshold along rows/columns Modified Median/Thresholding High-Pass Filtering (Gradient, Laplacian) Nearest Neighbor, Triangular, Bilinear… Weighted value techniques based on Delaunay triangulation, Natural Neighbor, etc… Kriging Identify Non- ground pixels Interpolate to Grid Interpolate Across Removed Points

15. R I T Rochester Institute of Technology 15 Result of Median Filter Baseline Range Image (2m Centers) Flagged Points with Modified Median Filter (Center 10m, Outer 40m)

16. R I T Rochester Institute of Technology 16 Result of Median Filter Plus HP Filter Baseline Range Image Flagged Points with Modified Median Filter and High Pass Filter

17. R I T Rochester Institute of Technology 17 High Points Removed Baseline Range Image Scatter Plot with Flagged Points Removed

18. R I T Rochester Institute of Technology 18 Interpolation and Low-Pass Filtering (Final DTM) Baseline Range Image (2m Centers) Final Terrain Model after re-interpolation and smoothing

19. R I T Rochester Institute of Technology 19 Object Extraction: Segmenting Buildings and Trees Building/Tree Map using only morphological techniques Third Step: Differentiate High Points Can continue to work with range image, or go back to Point Cloud A host of features available: Length of edges Homogeneity of height (HP Filtering) Plane matching Morphological Techniques Exploitation of co-registered Spectral Image

20. R I T Rochester Institute of Technology 20 Approach : Goal #4 (Object Reconstruction) Trees Use blurred Lidar Height Information Find local maxima to identify potential tree centers Use “Circle” functions at various scales to isolate features with high radial symmetry (Correlation Technique); confirm tree location and determine spread Use results to pick tree from library of objects This technique will also help extract cylindrical features for use in building reconstruction Tree Extraction Algorithm * Ref Gray et al: “Scene Construction Methodologies and Techniques for Simulating Forest Areas”, 11 th Annual Ground Targets Modeling and Validation Conference, 2000.

21. R I T Rochester Institute of Technology 21 Approach : Goal #4 (Object Reconstruction) Buildings Method 1: Use range image and corner detector to find critical vertices (fairly simple) Method 2: Planar Patch Extraction, find edges via plane intersections (More Accurate) – Anticipate significant effort here (Ref: Schenk, “Detecting Planes by Hough Transform”) Method 3: Rectangular (and 3D Primatives) Reconstruction: Images Courtesy of Morgan and Habib, “Interpolation of Lidar Data and Automatic Building Extraction”, ASPRS Annual Conf, 2002 and Haala, “Laser Data for Virtual Landscape Generation”, IAPRS, Vol 32, 1999

22. R I T Rochester Institute of Technology 22 Planar Feature Extraction/Reconstruction Several techniques can be used to detect planar determine parameters – Segment images of roof gradient directions – Hough-based techniques – Clustering (based on the meshes of a Delaunay triangulation) – we will explore this first

23. R I T Rochester Institute of Technology 23 Schedule Literature Search Data Handling Operating on Data Extracting Objects Simple Processing on Range Image Use of Height Features for Segmentation Reconstructing Objects Range Image Techniques Planar Extraction Techniques Rectangular Fitting Technique Reports 4/2 4/94/164/23 4/305/7 5/145/21

24. R I T Rochester Institute of Technology 24 Summary We will be using 3-D Point Data to geometrically reconstruct a desired scene Basic technique is to use range image to isolate points, then process point cloud to do the reconstruction Focus on feature extraction rather than point analysis