® Sponsored by Enterprise Access to Point Cloud Data 95th OGC Technical Committee Boulder, Colorado USA 1 June 2015 Keith Ryden Esri Software Development.

Slides:



Advertisements
Similar presentations
1 Projection Indexes in HDF5 Rishi Rakesh Sinha The HDF Group.
Advertisements

CS144: Spatial Index. Example Dataset Grid File (2 points per bucket)
©Silberschatz, Korth and Sudarshan1.1Database System Concepts - 6 th Edition Database System Concepts.
Streaming NetCDF John Caron July What does NetCDF do for you? Data Storage: machine-, OS-, compiler-independent Standard API (Application Programming.
Making earth science data more accessible: experience with chunking and compression Russ Rew January rd Annual AMS Meeting Austin, Texas.
Chapter 11: File System Implementation
InSAR Data and GeoServer IU QuakeSim Team October 26, 2011.
Esri UC 2014 | Technical Workshop | Achieving Interoperability Using Open Standards and Specifications Satish Sankaran Kevin Sigwart.
Multiple-key indexes Index on one attribute provides pointer to an index on the other. If V is a value of the first attribute, then the index we reach.
Lecture 6 – Google File System (GFS) CSE 490h – Introduction to Distributed Computing, Winter 2008 Except as otherwise noted, the content of this presentation.
File System Implementation
Oct 31, 2000Database Management -- Fall R. Larson Database Management: Introduction to Terms and Concepts University of California, Berkeley School.
The Google File System. Why? Google has lots of data –Cannot fit in traditional file system –Spans hundreds (thousands) of servers connected to (tens.
Planned Title: Review of Evaluation of Geospatial Search Allan Doyle.
Spatial Data Server for Mobile Environment EDBT 2004, Greece March 16, B.W. Oh, M.S. Kim, M.J. Kim, and E.K. Lee Spatial Information Technology Center,
File Organizations and Indexing Lecture 4 R&G Chapter 8 "If you don't find it in the index, look very carefully through the entire catalogue." -- Sears,
Nikolay Tomitov Technical Trainer SoftAcad.bg.  What are Amazon Web services (AWS) ?  What’s cool when developing with AWS ?  Architecture of AWS 
Jeremy Boyd Director – Mindscape MSDN Regional Director
Overview of Search Engines
Data Warehousing: Defined and Its Applications Pete Johnson April 2002.
TECHNIQUES FOR OPTIMIZING THE QUERY PERFORMANCE OF DISTRIBUTED XML DATABASE - NAHID NEGAR.
An Introduction to Infrastructure Ch 11. Issues Performance drain on the operating environment Technical skills of the data warehouse implementers Operational.
Technical Workshops | Esri International User Conference San Diego, California Advanced Image Management using the Mosaic Dataset Vinay Viswambharan,
Cloud Computing for the Enterprise November 18th, This work is licensed under a Creative Commons.
Database System Concepts and Architecture Lecture # 3 22 June 2012 National University of Computer and Emerging Sciences.
Spatial Data Management Chapter 28. Types of Spatial Data Point Data –Points in a multidimensional space E.g., Raster data such as satellite imagery,
July, 2001 High-dimensional indexing techniques Kesheng John Wu Ekow Otoo Arie Shoshani.
CS6530 Graduate-level Database Systems Prof. Feifei Li.
High Performance I/O and Data Management System Group Seminar Xiaosong Ma Department of Computer Science North Carolina State University September 12,
Future of GIS GIS & the Internet  Access spatial data interactively anywhere in the world
material assembled from the web pages at
Managing Lidar (and other point cloud) Data
Open Source GIS Web Mapping Solutions Sachindra Singh ICT Systems Developer 2010 Pacific Islands Geographical Information Systems and Remote Sensing User.
Or google LPI Web Services
Indices Tomasz Bartoszewski. Inverted Index Search Construction Compression.
Oracle Advanced Compression – Reduce Storage, Reduce Costs, Increase Performance Session: S Gregg Christman -- Senior Product Manager Vineet Marwah.
1 Chapter 1 Database Systems Database Systems: Design, Implementation, and Management, Fifth Edition, Rob and Coronel.
Multidimensional Indexes Applications: geographical databases, data cubes. Types of queries: –partial match (give only a subset of the dimensions) –range.
Pusan National University, Korea Joon-Seok Kim Taehoon Kim Ki-Joune Li.
Wenchuan Earthquake International EO Data Assistance Grid Prof. Guoqing Li, CEODE, CAS Present to WGISS26 , Boulder, 23rd, Spet, 2008.
Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 12: File System Implementation File System Structure File System Implementation.
May 2003National Coastal Data Development Center Brief Introduction Two components Data Exchange Infrastructure (DEI) Spatial Data Model (SDM) Together,
Non-Traditional Databases. Reading 1. Scientific data management at the Johns Hopkins institute for data intensive engineering and science Yanif Ahmad,
Lidar distribution system for OLC lidar Rudie Watzig, Senior GIS Analyst Oregon Dept. of Geology and Mineral Industries Framework Review Committee, April.
1 TOPIC 6 DATABASE 6.1 Introduction to Database 6.2 Basic Concept of Database 6.3 Database Object DATABASE.
1 Overview Finding and importing data sets –Searching for data –Importing data_.
CS525: Big Data Analytics MapReduce Computing Paradigm & Apache Hadoop Open Source Fall 2013 Elke A. Rundensteiner 1.
File Systems cs550 Operating Systems David Monismith.
ERDAS TITAN: Rapid, Secure & Versatile GIS Data Sharing Eddie Pickle & Angela Miele November 6, 2008.
Introduction to Core Database Concepts Getting started with Databases and Structure Query Language (SQL)
® Sponsored by Improving Access to Point Cloud Data 98th OGC Technical Committee Washington DC, USA 8 March 2016 Keith Ryden Esri Software Development.
Multidimensional Access Structures COMP3017 Advanced Databases Dr Nicholas Gibbins –
Best Practices for Managing and Serving Lidar and Elevation Data Cody Benkelman.
File-System Management
MOBILE AND DISCONNECTED FIELD DATA COLLECTION
Spatial Data Management
Future Data Architecture Cloud Hosting at USGS
Point Cloud ad hoc 95th OGC Technical Committee Boulder, Colorado USA
Electronic Data Collection at Statistics Canada
Point Cloud DWG Agenda 98th OGC Technical Committee Washington, DC USA
Multidimensional Indexes
Point Cloud ad hoc 95th OGC Technical Committee Boulder, Colorado USA
Outline Ganesan, D., Greenstein, B., Estrin, D., Heidemann, J., and Govindan, R. Multiresolution storage and search in sensor networks. Trans. Storage.
Query Optimization.
Geoprocessing Sample Tools for Lidar Data Management
Optimizing LAS for Easier Use of Lidar
JADDS- Jülich Atmospheric Data Distribution Service
Esri LAS Optimizer: An Introduction
Scaling Bathymetry: Data handling for large volumes
Esri LAS Optimizer An Introduction
Presentation transcript:

® Sponsored by Enterprise Access to Point Cloud Data 95th OGC Technical Committee Boulder, Colorado USA 1 June 2015 Keith Ryden Esri Software Development

OGC ® Point Cloud Data Multi-dimensional Scientific data LiDAR Data Elevation Data Seismic Data Bathymetric Data Meteorological Data Fixed/Mobile consumer sensors (IoT) It’s not just LiDAR

OGC ® There’s Lots of it… Point Cloud data is typically Big Data –LiDAR data in a collection of LAS datasets are one example –It’s big if you don’t want to move it…. –Bring the processing to the data …. The amount of data is so large that repeated conversion, import, transport, etc., can be painful

OGC ® Enterprise Imagery and Point Cloud Management – Access through Services MSI/HSl LIDAR EO CIR FMV Multiple Sensors & Formats OGC Services WCS, WMS, WPS, WCPS, etc Data Formats KML, LAS, GML Store it Once, Use it Many Times

OGC ® Point Cloud Services Enterprise Point Cloud data services need to support: –Standardized Service Interfaces –Overlapping collections –Collections over time –Arbitrary query areas –High performance access –Efficient transfer format/schema –Efficient storage, backup, recovery –Elastic deployment

OGC ® LAS Data The LAS format is a data transfer/exchange format –Well understood, and widely supported –Not originally designed for direct use/exploitation Issues when accessed directly include –Simple format (a plus) but becomes an I/O bottleneck –Lack of spatial index –Lack of dataset statistics –Uncompressed

OGC ® Improving LAS Data Access Exploitation of LAS data can be improved by importing the LAS exchange format into a local cache: –Building statistics –Building a spatial index –Rearranging point records for efficient localized access –Compressing blocks of contiguous records These concepts could also be considered for a future update to the LAS data format

OGC ® Spatial Index Spatial indexing for fast access to data by extent/location. There are Several indexing possibilities Grids Quad Trees RTrees

OGC ® Spatial distribution of points Physical location in file Rearranging Point Records

OGC ® Spatial distribution of points Physical location in file Rearranging Point Records

OGC ® Compress Blocks of Point Records Spatial distribution of points

OGC ® Compress Blocks of Point Records Spatial distribution of points

OGC ® Summary Point Cloud data is Big Data… –Access via well defined web services –OGC is well positioned to influence these service specifications The LAS data format is part of the picture –An exchange format for LiDAR and similar data –Can be optimized for exploitation