Best Practices for Managing and Serving Lidar and Elevation Data Cody Benkelman

Expectations… This demo theater presumes an advanced audience… Data structures - LAS & zLAS formats - LAS Dataset - Mosaic Dataset Longer presentations with further detail: - Managing Lidar and photogrammetric point clouds - Wednesday, Room 16 B3:15 to 4:30 (1845 / 2107) - Thursday, Room 15 A1:30 to 2:45 (1845 / 1347)

Outline Workflow - QA/QC of lidar data - Creation of elevation surfaces (DTM & DSM) - Management & sharing of elevation surfaces - Sharing of lidar point files Automated workflows

Lidar – 3D points and derived 2.5D surfaces

Lidar QC

Quality Control Checks Class codes # of Return values Z (height) values Is coverage of project area complete? Pulse density Point density

Quality Control Checks Class codes - Are they as requested? Accurate? # of Return values - Are First, Second, Third, Fourth returns present? Z (height) values - Is the range reasonable? (“Bird strikes”, points below ground) Is coverage of project area complete? Pulse density Point density POINT STATISTICS

Quality Control Checks Class codes - Are they as requested? Accurate? # of Return values - Are First, Second, Third, Fourth returns present? Z (height) values - Is the range reasonable? (“Bird strikes”, points below ground) Is coverage of project area complete? Pulse density - # laser pulses sent out, per square meter Point density - “echos” or “returns” received by the sensor SPATIAL PROCESSING

Quality Control Part 1 – Data Statistics LAS Dataset Properties

Statistical Reports LAS Dataset Properties

Statistical Reports LAS Dataset Properties

Statistical Reports LAS Dataset Properties

Quality Control Part 2 – Spatial Processing Pulse Density Summarized per square unit area (meter) Output product is a raster (image / pixels) Example: 4 outgoing pulses per square meter

Quality Control Part 2 – Spatial Processing Point Density Summarized per square unit area (meter) Output product is a raster (image / pixels) Example: 10 return pulses per square meter

Quality Control Part 2 – Spatial Processing Pulse Density – Loss due to water absorption Summarized per square unit area (meter) Output product is a raster (image / pixels) Example: Fewer return pulses over water

Best Practices Tiled LAS, v1.1 or higher Projected, rearranged, indexed - zLAS File size: 1 – 2 GB or less (<500 MB if not rearranged) Keep file I/O local, avoid network - Study area boundary included as constraint Airborne lidar - Classified (bare earth, non-ground) - Breaklines for hydro enforcement Terrestrial lidar - RGB & intensity values, classified * Also applies to photogrammetric point clouds

DTM & DSM Creation of Raster Surfaces

Export raster surfaces from LAS Dataset “Workflow A” Most scalable method Ensures consistent surfaces for all users 5x – 10x data reduction, LAS files do not need to remain on line Recommended method for best scalability Test before export to define best parameters Ensure tiles overlap Lidar data may be moved to offline storage

Tool: LAS Dataset to Tiled Rasters Download from

Addition of feature constraints “Workflow A” Constraints for Hydrology, Breaklines, Model Key points DTM only

Alternative – Rasters generated on-the-fly from lidar points “Workflow B” Fastest method to share data Can support alternative surfaces (e.g. buildings on DTM, minus trees) LAS files should remain on line*

Managed/Versioned DTM using Terrain Dataset “Workflow C” For versioned digital terrain model Model points within a geodatabase

Elevation Data Management

Source Imagery Source Mosaic Datasets SRTM LiDAR Project #N Source Mosaic Datasets – Elevation & Lidar example LiDAR Project #1 Additional notes: 1)Create overviews on Source Mosaics, then use SRTM (instead of overviews) to fill in “background” elevation values 2)Don’t calculate statistics – it takes too long and statistics for elevation datasets aren’t really meaningful – instead, use Set Raster Properties to manually insert approximate statistics.

Source Imagery Source Mosaic Datasets Derived Mosaic Dataset Combine into Derived Mosaic Dataset Use TABLE Raster Type Advantage: All data available in a single location Multi-source, multi-resolution collection of elevation data

Source Imagery Source Mosaic Datasets Derived Mosaic Dataset Orthometric Height f Single image service with multiple server functions Example – ArcGIS World Elevation – Server Raster Functions …many other functions Aspect Ellipsoidal Height Slope Contour Hillshade

Source Imagery Source Mosaic Datasets Derived Mosaic Dataset Orthometric Height f Single image service with multiple server functions Example – ArcGIS World Elevation – Update with new data …many other functions Aspect Ellipsoidal Height Slope Contour New data collections added to the central Derived Mosaic appear immediately in all services Hillshade

Lidar-derived surfaces in ArcGIS Online World Elevation Service Demos

Support for 3D analysis

LAS / zLAS files exposed for download – ArcGIS for Server Server must have local storage for LAS/zLAS files Client = ArcGIS Desktop or custom web client

LAS / zLAS files exposed for download – Simple download (S3 / FTP) Simple cloud storage for LAS/zLAS files, linked to AGOL Feature Service Client = browser

Sharing Geoprocessing Services – Data and Tools in the Cloud Move the Processing to the Data, not the Data to the Processing… Take advantage of storage and computing power in the cloud or on a private server Expose Geoprocessing Tools as services Viewshed, Line of sight, Volume calculations, etc. Accessible to Desktop, Web, and Mobile clients Geoprocessing Service

Geoprocessing Services: Volume calculation Lidar point download: Sonoma County GIS Demos

Automated Workflows – for Repeatability & Scalability Simplicity Improve Productivity - Repeatability, Maintainability, Scalability - Documentation  Facilitate QA & QC, Design Review Training/Examples - Encapsulate best practices - Reusable templates

Image Management Workflows – Landing page Overview of Workflows Guidebook - Part of Online Doc ArcGIS Online Group - Gallery of downloadable items

Resources Imagery Resource Center : Image Management Workflows: Guidebook in ArcGIS Help: ArcGIS Online Group: Recorded Webinar on lidar data management: Optimized LAS tool: Tools from 3D Team: Contact information: - Cody - Lindsay Managing Lidar