Location Powers: Big Data Webinar 22 November 2016

Agenda for Today’s Webinar Introduction to Location Powers and Webinar Denise McKenzie, OGC Summary of Sept 20th Workshop George Percivall, OGC Management and Dissemination of Earth Observation Data in a Big Data World Jeff Walter, NASA Exploring Strategies For Optimizing Knowledge Derivation From Imagery Dan Getman, DigitalGlobe Summary and next steps on Big Data

Summary of Sept 20th Workshop Location Powers: Big Data Copyright © 2016 Open Geospatial Consortium

Location Powers:Big Data, 20 Sep 2016

Use Cases for Big Geo Data High Velocity Ingest Geospatial Databases Entity-oriented Spatial-temporal analytics Grid-oriented Spatial-temporal analytics Feature Fusion GeoAnalytics, Machine Learning Remote sensed data processing Machine Learning Spatial Modeling IoT Message Streaming Built environment models Array databases Users and consuming apps Social Media Message Processing Observation Sources NoSQL databases Integrated environmental models ETL Stream processing using RDF Graph databases Modeling and simulation Wide Area Motion Imagery SQL databases

Sessions 1 Opening; 2 Obtaining Big Data

Population Distribution and Dynamics Modeling LandScan Global Ambient Population Distribution (~ 1km) Increase in spatial and temporal resolution Top Down and Bottom Up Approach Ambient and Day/Night Population Distribution (~ 90 m) LandScan HD Rapid Settlement Identification and Characterization Facility Level Population Density Population Density Tables Settlement Mapping Slide: Jibo Sanyal, ORNL

Inputs to Geospatial Big Data NIST Public Big Data Working Group with 5 working groups: Requirements and Use Cases, Definitions and Taxonomies, Reference Architecture, Security and Privacy and Technology Roadmap 30% of uses cases were geospatial 80% of use cases were streaming Follow up activities extending work and building exemplar use cases defined with DevOps so can be used on multiple infrastructures: HPC, Docker, OpenStack, AWS Two Streaming workshops at http://streamingsystems.org/ Many important streaming geospatial use cases NSF SPIDAL (Scalable Parallel Interoperable Data Analytics Library) project developing HPC-ABDS High Performance Computing enhanced Apache Big Data Stack Slide Source: Geoffrey Fox, Indiana Univ.

Session 3. Maintaining Big Data

Big Data: Driving Forces 09/20/2016 Copyright 2016, JCC Consulting, Inc. Big Data: Driving Forces Inexpensive storage of large volumes of data Inexpensive compute power Next Generation Analytics Moving from off-line to in-line embedded analytics Explaining what happened Predicting what will happen Operating on Data at rest – stored someplace Data in motion – streaming Multiple disparate data sources Look at available data and wonder what answers are hidden there Slide Source: Keith W. Hare, JCC Consulting, Inc.

Copyright 2016, JCC Consulting, Inc. 09/20/2016 Copyright 2016, JCC Consulting, Inc. “Big Data” Data Types Traditional Data Types Character Numerical Date/Time/Timestamp Large Objects – LOB/BLOB/CLOB “Big Data” Data Types Multi-dimensional arrays Images/video Documents  Loosely formatted data Objects Spatial Slide Source: Keith W. Hare, JCC Consulting, Inc.

Apache Projects Geospatially Enabled* *not exhaustive Slide Source: Rob Emanuele, Azavea #LPBigData

Slide Source: Rob Emanuele, Azavea #LPBigData

The Land Change Monitoring Assessment and Projection (LCMAP) information system Slide Source: Glenn Guempel, USGS #LPBigData

Where We Want To Be Download as Last Resort Mentality The Land Change Monitoring Assessment and Projection (LCMAP) information system Where We Want To Be Download as Last Resort Mentality Store data in unzipped, optimal formats ready for direct processing by standard services or custom processes. Provide basic visualization, analysis and extraction functions through services on an open platform. The platform additionally provides the potential processing capacity for building unforeseen custom workflows and processes against big data. Analysis Ready Data We believe over years the download mentality will diminish. Storing data in a ready-to-be-used format will allow users to access data without downloading. Service Functions will be available for basic visualization, analysis and extraction of data. Only download what is needed - perhaps the results rather than all the raw data. Virtual Platforms, like current commercial clouds, will mature and provide cost-effective, on-demand capacity to process big data. Custom processes and workflows can be supported by allowing users to spin up large infrastructure components, process the data, and shutdown without ongoing costs. Slide Source: Glenn Guempel, USGS #LPBigData

Session 4. Analyzing Big Data

Earth Server: Datacubes At Your Fingertips Intercontinental initiative: EU + US + AUS started 2011 Agile Analytics on 3D, 4D Earth & Planetary datacubes Rigorously standards: OGC WMS + WCS + WCPS EU rasdaman + US NASA WorldWind 100s of TB sites now, next: 1+ PB Uni Jacobs PML NCI Australia ESA MEEO MWF EC

Science & GIS Tool Interfacing General-purpose scientist tools: Java, C++ python, R (under work) Geo tools: MapServer, GDAL, QGIS, OpenLayers, Leaflet, NASA WorldWind, ... OGC WCS Core & INSPIRE WCS Reference Implementation Can interface to all tools supporting OGC‘s „Big Geo Data“ standards suite

Moving Features Location data (ID of object, latitude, longitude, time) is one of the typical bigdata: Moving Features Most people have cell-phone (smart phone) Vehicle navigation system measures car locations Vessels and aircrafts locations are managed due to security And so on. The demand for Moving Features is very rapidly increasing Many Applications Moving Features Mobile Objects Disaster management Road traffic control

Applications of Moving Features bigdata Many kinds of Moving Features are used…, “Data integration” is a key point to produce more value Disaster management Traffic management Integration of tsunami and evacuation simulation Aircrafts, vessels, vehicles, and pedestrians traffic Indoor pedestrian flow Sports Tracks of visitors to shopping-malls are useful for marketing Tracks of soccer players and a ball are useful for considering tactics

L O C A T E Geo-enrichment Allows a wide variety of datasets to be appended to a data record using a common spatial ID. - What are the property attributes of this insured property? - What demographic group does this customer belong to? - What businesses are in this area of poor network coverage? Analytics Reduce the complexity of billions of transactional records by assigning data to geographic bins and aggregating results. - Is average 4G network coverage in this area better than a competitor? - Is the accumulated exposure at risk of hurricane damage too high? - Is this data point inside or outside of a geofence?

Session 5. Big Data Applications Panel

Emergent Themes from Workshop Loosely-coupled PB archives based on open standards for rapid geospatial information product creation at any scale Analysis Ready Data We live in a download mentality. How do we move to answering questions Focus shifting from understanding what happened last week to being able to predict what will happen next week Take better advantage of developments in Big Data Proper, which is only tangentially interested in Big Geo Data Multiple Applications: Telecommunications, property casualty insurance, financial services, Energy Monitoring and prediction, Population Dynamics, Settlement Mapping Input to Testbed 13, e.g., SpaceNet

Agenda for Today’s Webinar Introduction to Location Powers and Webinar Denise McKenzie, OGC Summary of Sept 20th Workshop George Percivall, OGC Management and Dissemination of Earth Observation Data in a Big Data World Jeff Walter, NASA Exploring Strategies For Optimizing Knowledge Derivation From Imagery Dan Getman, DigitalGlobe Summary and next steps on Big Data

© 2016 Open Geospatial Consortium OGC Actions on Big Data Location Powers: Big Linked Data Workshop Delft, 22 March 2017 Publish “Big Geo Data - White Paper” Apply/extend OGC Standards to Big Data WMS/WMTS, WFS, WCS/WCPS, WPS Moving Features Encoding Discrete Global Grid Systems (DGGS) Conduct OGC Innovation Program Testbeds Engineering Reports into Best Practices OGC Testbed 13 Coordinate: JTC 1 Big Data, Apache, Location Tech OGC Big Data Domain Working Group © 2016 Open Geospatial Consortium 25

OGC Big Data Domain Working Group Public forum for geospatial Big Data interoperability, access, and especially analytics. Encourage collaborative development among participants representing many organizations and communities, Ensure appropriate liaisons to other Big Data relevant working groups, both inside and outside OGC. http://www.opengeospatial.org/projects/groups/bigdatadwg E-mail list - Open to public: https://lists.opengeospatial.org/mailman/listinfo/bigdata.dwg © 2016 Open Geospatial Consortium 26

Agenda for Today’s Webinar Introduction to Location Powers and Webinar Denise McKenzie, OGC Summary of Sept 20th Workshop George Percivall, OGC Management and Dissemination of Earth Observation Data in a Big Data World Jeff Walter, NASA Exploring Strategies For Optimizing Knowledge Derivation From Imagery Dan Getman, DigitalGlobe Summary and next steps on Big Data