1. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman - AIP-8 Results - GEO XII Plenary, Mexico City, 2015-11-09 Alexandru Mircea Dumitru Jacobs University | rasdaman GmbH adumitru@rasdaman.com Big Datacubes at Your Fingertips [gamingfeeds.com]

2. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman  „raster data manager“: SQL + n-D arrays -Scalable parallel “tile streaming” architecture  Part of GEOSS Common Infrastructure (GCI)  Makers of OGC Big Datacube standards, ref impl -Coverages, WCS, WCPS  Continuous contributor since many AIPs now rasdaman @ AIP-8

3. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Web Coverage Service (WCS)  OGC Coverages unifying regular & irregular grids, point clouds, meshes -OGC Coverage Implementation Schema  WCS Core: access to spatio-temporal coverages & subsets -subset = trim | slice  WCS Extensions: optional functionality facets Large, growing implementation basis: rasdaman, GDAL, QGIS, OpenLayers, OPeNDAP, MapServer, GeoServer, GMU, NASA WorldWind, EOx-Server; Pyxis, ERDAS, ArcGIS,...

4. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Scalability  Adaptive data partitioning & distribution -storage layout language - 130+ TB datacubes  Distributed query processing -Heterogeneous hardware, cloud, federation - 1 query  1,000+ cloud nodes select max((A.nir - A.red) / (A.nir + A.red)) - max((B.nir - B.red) / (B.nir + B.red)) from A, B Dataset A select max((B.nir - B.red) / (B.nir + B.red)) from B select max((A.nir - A.red) / (A.nir +A.red)) from A Dataset B

5. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Demo  Virtual Globe Virtual Globe  Apache vs rasdaman Apache vs rasdaman  Terabyte Aggregation Terabyte Aggregation 5 [gamingfeeds.com]

6. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Outlook  WCS-T(ransaction) adopted -Coverage maintenance via Web service (eg, reproducible ingestion) -Coverage service mashups  Coverage 1.1 under adoption -As before: encoding in XML (in future: + JSON) + any other suitable format (GeoTIFF; NetCDF, GRIB,...) -Regular + irregular grids (generalizing GML 3.3), SensorML -Partitioning, efficient „geometry/value pair“ representation -Interpolation  ISO, INSPIRE adopting Coverages + WCS backwards compatible with GMLCOV 1.0

7. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Wrap-Up  Open-standard Coverage Cubes -flexible, interoperable, format-independent -Arriving at consensus across OGC, ISO, INSPIRE  WCS: scalable access & manipulation on n-D coverages  During AIP-8, advanced interactive demo: http://standards.rasdaman.com http://standards.rasdaman.com 7 [screenshots: rasdaman-based services; cube: gamingfeeds.com]

8. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Wrap-Up http://standards.rasdaman.com 8 [screenshots: rasdaman-based services; cube: gamingfeeds.com] „A cube tells more than a million images“

9. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Outlook  WCS-T(ransaction) adopted -Coverage maintenance via Web service (eg, reproducible ingestion) -Coverage service mashups  Coverage 1.1 -Much confusion  May 2015: renamed from „GML 3.2.1 Application Schema – Coverages“ -As before: encoding in XML + any other format (GeoTIFF; NetCDF, GRIB,...) Prepared for JSON -Partitioned representation, efficient „geometry/value pair“ representation -Regular + irregular grids (generalizing GML 3.3), SensorML -Interpolation backwards compatible

10. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Use Case: Plymouth Marine Laboratory  “Avg chlorophyll concentration for given area & time period, from x/y/t cube” -10, 60,120, 240 days  Conclusions: -„we must minimise data transfer as well as [client] processing” -“standards such as WCPS provide the greatest benefit” [Oliver Clements, EGU 2014] rasdaman

11. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman sensor feeds simulation data Coverages as Unifying Paradigm 11 Coverage server

12. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman SWE, SOS: upstream sensor data capturing W*S: downstream download, processing, visualization SOS WMS WCS WCPS WPS... coverage server Coverages as Unifying Paradigm

13. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman WCS Extension – Processing [OGC 13-057]  WCS wrapper for OGC Web Coverage Processing Service (WCPS) -high-level spatio-temporal geo raster query language for $c in ( M1, M2, M3 ) where some( $c.nir > 127 ) return encode( $c.red - $c.nir, “image/tiff“ ) (tiff A, tiff C ) 13  "From MODIS scenes M1, M2, M3: difference between red & nir, as TIFF" …but only those where nir exceeds 127 somewhere

14. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Inset: WaterML 2.0 Time Axis Handling  WaterML 2.0: timeseries = time slices -Hardwired for generation, not comsumption  OGC Coverages: datacubes -Time, elevation just another axis -Implementation can choose efficient layout

15. AIP Deep Dive :: Coverage Cubes :: © 2015 rasdaman Visualization-as-a-Query for $s in (SatImage), $d in (DEM) where $s/metadata/@region = “Glasgow" return encode( struct { red: (char) $s.b7[x0:x1,x0:x1], green: (char) $s.b5[x0:x1,x0:x1], blue: (char) $s.b0[x0:x1,x0:x1], alpha: (char) scale( $d, 20 ) }, “image/png" ) [JacobsU, Fraunhofer; data courtesy BGS, ESA]