AOGEOSS Task 11. Develop Regional GEOSS Data set.

Slides:



Advertisements
Similar presentations
Spatial Ontology Community of Practice Workshop, USGS, Dec.2, Using Knowledge to Facilitate Better Data Discovery, Access, and Utilization for CloudGIS.
Advertisements

Standards and Interoperability Forum: Status, Issues & Plans GEO Architecture and Data Committee Kyoto, Japan 9 February 2009 Siri-Jodha Singh Khalsa.
An Australian Geoscience Data Cube
Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.
Marihebert Leal. Alteryx is the fastest analytics plataform that is purpose- built to empower data analysts & their productivity. It blend complex data,
CEOS Data Cube Concept and Prototype Project Plans
Internet GIS. A vast network connecting computers throughout the world Computers on the Internet are physically connected Computers on the Internet use.
Global Data Flows Study SDCG-8, Session 6 Frank Martin Seifert (ESA); Gene Fosnight (USGS) 24 Sep 2015, Bonn.
Element 2: Country-specific Space Data Services
Brian Killough / NASA, CEOS SEO SDCG-8 Session 7, Agenda Items 26 and 27 GFOI Space Data Services SDCG-8 DLR, Bonn, Germany September 23 rd -25 th 2015.
WGISS Technology Subgroup Report WGISS-29 Bonn, 2010 Nataliia Kussul WGISS.
Australian Geoscience Data Cube
CEOS Data Cube Open Source Software Status Brian Killough CEOS Systems Engineering Office (SEO) WGISS-40 Harwell, Oxfordshire, UK September 30, 2015 (remote.
Managing Enterprise GIS Geodatabases
CoGland- Collaborative Global Land Information Jun Chen 1,2,3 1 National Geomatics Center of China 2 GEO SB Int. Society of Photogrammetry and.
Axis AI Solves Challenges of Complex Data Extraction and Document Classification through Advanced Natural Language Processing and Machine Learning MICROSOFT.
Page 1 CSISS Center for Spatial Information Science and Systems 09/12/2006 Center for Spatial Information Science and Systems (CSISS) George Mason University.
CEOS Data Cubes A new need for Capacity Building Brian Killough CEOS Systems Engineering Office (SEO) WGCapD-5 Meeting March 30 – April 1, 2016.
SEO Report to WGISS Brian Killough CEOS Systems Engineering Office (SEO) WGISS-41 Meeting March 14-18, 2016.
CEOS Working Group on Information System and Services (WGISS) Data Access Infrastructure and Interoperability Standards Andrew Mitchell - NASA Goddard.
Contribution to the Implementation of the African Space Programme: EO Data and Infrastructure *9.a) space segment - available observations (space & in-situ.
CEOS Data Cubes A briefing for the GEO Secretariat Brian Killough CEOS Systems Engineering Office (SEO) April 1, 2016.
Common interoperability, best practices and strategic approach
CEOS Data Cube Initiative
Analysis Ready Data LSI-VC – Adam Lewis Co-chair
VCL Virtual Computing Laboratory An Opportunity to Lead
2nd GEO Data Providers workshop (20-21 April 2017, Florence, Italy)
Information, Communication & Technology Strategy
USGS EROS LCMAP System Status Briefing for CEOS
Infrastructure for Accessing & Processing EO Data Session 10a & 10b
Land Cover Side Event: A new path forward for generating products
CEOS Data Cube Report Agenda Item #7 September 13, 2017
SEO Capacity Building Agenda #15 Brian Killough
GEO-CRADLE: Fostering regional cooperation and roadmap for GEO and Copernicus implementation in North Africa, Middle East and Balkans Funded under H2020.
Geoscience Australia – data cube progress
Future Data Architectures (FDA) Pilot Project Summary
Presentation on Copernicus Dissemination
Analysis Ready Data ..
SEO Report to WGISS Brian Killough CEOS Systems Engineering Office (SEO) WGISS-42 Meeting September 19, 2016.
Space Data Services Session 2: Space Data Country Outreach and Delivery Agenda Item #3 Brian Killough CEOS Systems Engineering Office (SEO) February 25,
Open is not enough! Sustainability and Innovation in Scholarly Communication
USGS Status Frank Kelly, USGS EROS CEOS Plenary 2017 Agenda Item #4.14
Federal Land Manager Environmental Database (FED)
LSI-VC Jenn Lacey, USGS, LSI-VC Co-Lead CEOS SIT-33
FDA context Steven Hosford On behalf of FDA Co-chairs
Opening Remarks European Commission CEOS 2018 Chair
Land Imagery Data Architectures
Open Data Cubes Cloud Services Experiences and Lessons Learned
Analysis ready data: definition document
SDCG REPORT TO SIT TW 2016 Oxford, September, 2016.
FDA Objectives and Implementation Planning
Achieving universal access to, and use of space applications for sustainable development in Asia and the Pacific Keran Wang Chief, Space Applications.
Geospatial Data Use and sharing Concepts
Proposed 2019 CEOS chair initiative
Scanning the environment: The global perspective on the integration of non-traditional data sources, administrative data and geospatial information Sub-regional.
LSI-VC Work Plan Updates
Capacity (and Capability) Building at GA
GEO-XIII Plenary St. Petersburg Russian Federation
SESSION 2: GLOBAL DATA FLOWS STUDY
FDA-08 FDA Whitepaper Update
SEO Report to WGISS Brian Killough CEOS Systems Engineering Office (SEO) WGISS-46 Meeting October 22, 2018.
WGISS Connected Data Assets Oct 24, 2018 Yonsook Enloe
FDA Topics Going Forward…???
Uncertainties for Analysis Ready Data
MIS COURSE: CHAPTER 1 INFORMATION SYSTEM IN GLOBAL BUSINESS TODAY
Agency Reports – USGS Jenn Lacey LSI-VC-5 Agenda Item #2 February 2018
TOOLS & Projects overview
Reinhard Scholl, GTSC-7 Chairman
CEOS Work Plan LSI-VC Deliverables
Open Data Cube Demo and FDA Interfaces
Presentation transcript:

AOGEOSS Task 11. Develop Regional GEOSS Data set

The Data Problem Increased Volume Low Capacity Slow Internet A significant growth in imagery data will increase data volumes by >10x in the next few years. Many countries lack the knowledge, infrastructure, and resources to access and use space-based data. Data Cube architecture provides a solution that saves countries time and money and reduces technical complexity. Increased Volume Low Capacity Slow Internet

Expectations for On-Demand Integration On the other side of this deluge of spatial content is a growing expectation by decision-makers for on-demand access to data from a diversity of sources for use with other content without a reliance on time-consuming and costly integration processes. The expectation amongst decision makers is for geospatial data integration on-demand. To enable this, a common analytical platform is required that can link very large multi-resolution and multi-domain datasets together to enable application of the required analytic processes.   Community safety Information for decision support Sentinel 2 and the Australian Geoscience Data Cube

Traditional remote sensing process Traditional approaches to acquiring, managing, distributing and even using this data have made it challenging to realise the true value of this data As well as the points we just made about use of satellite data to produce simulated ‘photos’ … Traditional ‘pull based’ models, where acquired data is only prepared to the point of being useful when somebody asks for it, create a significant ‘barrier to entry’ for those businesses, agencies or researchers thinking of a new initiative And even when the interest is there, the sheer amount of time and cost involved in the preparation process can be prohibitive … often resulting in Potential data being ‘thrown away’ because ‘too much’ of a particular scene was cloudy Initiatives being unnecessarily scaled back to focus on smaller areas, or smaller time periods, Only use data from one satellite because it is too hard for a particular group to justify the technical work required to make data from different satellites comparable The use of approaches such as ‘mosaicing’, where the raw data is ‘merged’ from multiple passes of the satellite is ‘merged’ to produce intermediary products that manageable and usable on commodity ICT infrastructure … but with the effect that often valuable data is ignored, for example reducing a potential 45 observations of a spot on the ground from the Landsat series down to just 1 that has been judged the ‘best’ for that year The storage of much data on tape archives, whilst ensuring data is preserved for the long haul, simply do not work in a world where information is needed ‘anywhere’ and ‘now’ And such approaches often result in the product of a product that provides a ‘point in time’ perspective, with significant challenges to creating the sorts of dynamic information now essential to businesses and governments

What are Data Cubes? Open Source Software https://github.com/data-cube Data Cube = Time-series multi-dimensional (space, time, data type) stack of spatially aligned pixels ready for analysis Proven concept by Geoscience Australia (GA) and the Australian Space Agency (CSIRO) and planned for the future USGS Landsat archive. Shift in Paradigm ... Pixels vs Scenes Analysis Ready Data (ARD) ... Dependent on processed products to reduce processing burden on users Supports an infinite number of applications, reduces data preparation time, allows time series analyses, increases interoperability of multiple datasets. Open source software approach allows free access, promotes expanded capabilities, and increases data usage. Open Source Software https://github.com/data-cube TIME

Advantages of Data Cube Free and Open ... Open source software, CEOS Agency contribution No data loss ... No scene-based screening required, all pixels used No processing ... Based on analysis-ready data products Spatial consistency ... Allows for user-defined nested grids Data interoperability ... Multiple aligned datasets in one system Efficient ... Less input-output (I/O) and smaller storage Faster ... Faster temporal and spatial analyses Flexible deployment ... Local computer, data hub, or cloud Allows automation ... Data download and ingestion can be automated Flexible architecture ... APIs allow infinite applications and users Improved capacity building .. Simple architecture and methods

Continental Scale Water Observations from Space 27 Years of data 25m N ominal Pixel Resolution Approx. 300 000 individual scenes in approx. 20 000 passes Entire 27 years of 1 312 087 ARG25 tiles => 93x1012 pixels visited 0.75 PB of data 3 hrs at NCI But the Data Cube paradigm has enabled us to do not only undertake this analysis at a regional scale for the first time, it has in fact enabled us to scale this to produce a comprehensive national product We can now analyse 15 years of data, across the country, representing every <n> square metres … in under 1 day. Not only does this mean we can now for the first time produce this analysis once … It means that we can produce it in an ongoing basis, enabling the production of updated information as new data becomes available, and enabling researchers to interact with the data iteratively to improve quality and try new ideas And it means that, in future, industry and others can build value-adding products on top of this data by leveraging the services-based approach adopted

Big Data for a Big Planet: a global network of regional data cubes? Data Cubes for: Africa, South America, Asia, China, India, Europe, North America, … Connecting the EO, Spatial and Statistical world to support global programs

Task Description: Task 11. Develop Regional GEOSS Data set Leaders: David Hudson (GA, Australia), YU Tao (RADI, China). Subtask 11.1 Create dialogue to assess data format requirements among data-rich countries Datasets to be developed based on GCI tools and DAB protocols, Subtask 11.2 Consolidate operational users' needs and dynamic features of large volume EO data to support development of AO Data Cube (AODC) data set format: released in 2018. Australia Geoscience Data Cube (AGDC) and Chinese Geoscience Data Tile DEM (CNGDT) Subtask 11.3 Advocate and support incorporation of pilot research using standardized AODC format, with the help of activities in task 1 to task 9. final assessment report will be released around the middle of 2019. Subtask 2.4 Advocate and support incorporation of data catalogues among member countries and contributes to task end of 2019.