GEO-Amazon Cloud Credit Program and a Prototype Sentinel Data Pipeline

Slides:



Advertisements
Similar presentations
CEOS Plenary Meeting Lucca, Italy November 8-9, 2011 CEOS Systems Engineering Office (SEO) Annual Report Brian Killough NASA Agenda Item #18.
Advertisements

Global Data Flows Study SDCG-8, Session 6 Frank Martin Seifert (ESA); Gene Fosnight (USGS) 24 Sep 2015, Bonn.
Eric C. Wood, USGS Steven P. Neeck, NASA SIT Workshop Agenda Item #5 CEOS Action / Work Plan Reference CEOS SIT Technical Workshop EUMETSAT, Darmstadt,
CEO Team SEO SIT Technical Workshop 25 – Agenda #16 EUMETSAT 17 th to 18 th September 2015 Committee on Earth Observation Satellites CEOS in the future.
CEOS Data Cube Open Source Software Status Brian Killough CEOS Systems Engineering Office (SEO) WGISS-40 Harwell, Oxfordshire, UK September 30, 2015 (remote.
Committee on Earth Observation Satellites Brian Killough & Kim Holloway, CEOS SEO Plenary Agenda Item # th CEOS Plenary Kyoto, Japan 6 November 2015.
GFOI Space Data Services 3-year Work Plan Brian Killough NASA LaRC, CEOS SEO Presented at SDCG-6 Oslo, Norway October 22-24, 2014.
Committee on Earth Observation Satellites ESA F. Sarti Day 1 – point 7 CEOS WGCapD-5 CEOS Systems Engineering Office (SEO), Hampton, VA, USA March 30th.
Capacity Building in: GEO Strategic Plan 2016 – 2025 and Work Programme 2016 Andiswa Mlisa GEO Secretariat Workshop on Capacity Building and Developing.
CEOS Data Cubes A briefing for the GEO Secretariat Brian Killough CEOS Systems Engineering Office (SEO) April 1, 2016.
CEOS Response to GEOSS Water Strategy
Copernicus Data & Information Access Service → DIAS 28 September 2016.
CEOS Data Cube Report Agenda Item #7 September 13, 2017
SEO Capacity Building Agenda #15 Brian Killough
Colombia Data Cube Brian Killough CEOS Systems Engineering Office (SEO) WGISS-43 Meeting April 4, 2017.
Future Data Architectures (FDA) Pilot Project Summary
CEOS and GEO-XIV Agenda Item 4.4 CEOS Plenary 2017
Space Data Services Session 2: Space Data Country Outreach and Delivery Agenda Item #3 Brian Killough CEOS Systems Engineering Office (SEO) February 25,
Copernicus Sentinel Data Uptake and Application
SDCG Support to Colombia
GFOI Status & Issues CEOS Lead & SDCG EXEC SIT-31 Agenda Item 5
LSI-VC Jenn Lacey, USGS, LSI-VC Co-Lead CEOS SIT-33
Status Report on ARD Usage
Copernicus DIAS European Commission CEOS 2018 Chair team WGISS
Committee on Earth Observation Satellites
CEOS and GEO-XIV Agenda Item 14 September 13, 2017
Opening Remarks European Commission CEOS 2018 Chair
CEOS Workplan FDA actions
Exploitation Platforms and Common Reference Architecture
Land Imagery Data Architectures
Open Data Cubes Cloud Services Experiences and Lessons Learned
Agency Report Geoscience Australia
CODE-DE Copernicus Data and Exploitation Platform - Deutschland
SEO Report Brian Killough NASA, CEOS Systems Engineering Office
Committee on Earth Observation Satellites
Technology Exploration Cloud Hosting at USGS
ARD Needs and Plans for Thematic Pilots
FDA Objectives and Implementation Planning
Purpose and Objectives of the FDA Big Data Workshop
CEOS & Work Plan Status Steven Hosford WGISS-46 22nd October 2018
ESA inputs on SAR ARD Susanne Mecklenburg, Nuno Miranda Steven Hosford
Review of Chair Priorities
CEOS Chair FDA Big Data Workshop
Prepared at the Joint Meeting For SDCG for GFOI / GEOGLAM / LSI-VC
GFOI Space Data Services
SEO Report to WGISS Brian Killough CEOS Systems Engineering Office (SEO) WGISS-46 Meeting October 22, 2018.
Plenary Decision Items
CEOS GEO Reporting and GEO-XV Plenary Preparations
Engagement with GEO Kerry Ann Sawyer, NOAA, CEOS SIT Chair Team
CEOS Membership Issues
Data Cubes and FDA Pilot Status Report
Session Introduction: Organizational Matters – Part II
Observations for Forests
SIT-31 Sessions CEOS SIT Chair Team ESA/ESRIN, Frascati, Italy
CEOS Context LSI/SDCG/GEOGLAM Joint Meeting
SEO Report Brian Killough, NASA, CEOS SEO CEOS Plenary 2018
Support to GEO Steven Hosford, ESA/CNES, CEO CEOS SIT-33
Copernicus Sentinel Data Uptake and Application
Session Introduction: Partnerships
Analysis Ready Data Strategy for CEOS
Brian Killough (NASA, SEO), Mirko Albani (ESA, WGISS Chair)
A Revisit of CEOS Agency Data Policies
Status Report on the Open Data Cube and the use of ARD
CEOS Engagement with GEO
SEO Report to WGISS-47 Brian Killough
Open Data Cube Demo and FDA Interfaces
CEOS Analysis Ready Data Strategy
SEO Report to WGISS-48 Brian Killough
CEOS Analysis Ready Data Strategy
CEOS Work Plan Progress Report
Presentation transcript:

GEO-Amazon Cloud Credit Program and a Prototype Sentinel Data Pipeline CEOS SIT-34 Meeting Session 3.5 Coconut Grove, Miami, FL, USA April 3-4, 2019 Brian Killough CEOS Systems Engineering Office NASA Langley Research Center

GEO-Amazon Cloud Credit Program Amazon Web Services (commercial cloud provider) has offered $1.5M USD of cloud credits to GEO These credits will allow developing countries access to cloud services to support hosting, processing and analysis of Earth observation data for sustainable development. Project proposals are due to GEO by 12 April (recently extended). Successful proposals will be announced by 15 May, with the credits being awarded as of 1 June, for a 3-year period. Evaluation panel includes the GEO Secretariat Director, GEO Senior Scientist and 3 GEO Programme Board members.

CEOS Participation in the GEO-AWS Program The call for proposals does NOT require the use of Data Cubes, but leaves all options open for use of AWS cloud resources. GEO does not know who will propose to this call and what those proposals will include. Many proposers could ask for help with CEOS satellite data preparation and Open Data Cube (ODC) installation. At the annual CEOS-GEO meeting (February 2019), GEO agreed that CEOS would conduct a technical evaluation of the proposals prior to selection and offer an analysis of issues or concerns regarding satellite data provisions and Open Data Cube installation and operation. The SEO will take the lead on this task. Support from CEOS Agencies is desired and welcome. If the proposers ask for satellite data support and ODC support, we (CEOS Agencies) will need to evaluate whether we can meet those requests, as it will take resources.

Sentinel Data Pipeline The issue ... The SEO has been engaged in a number of global projects that require S1 and S2 analysis-ready data (ARD). These include the Africa Regional Data Cube (ARDC), numerous country-level data cube prototypes and the future DE-Africa. S1 and S2 level-1 archive downloads are challenging due to significant data volumes. We would prefer to use cloud computing for ARD creation and storage of data. The SEO will lead a task to develop a prototype Sentinel “data pipeline” to efficiently and effectively produce ARD and make it available in the Amazon cloud. The SEO has been in contact with ESA (Steven Hosford and Albrecht Schmidt) to discuss implementation options and connections with DIAS as well as European commercial options.

Sentinel Data Pipeline The proposed solution ... The SEO has begun discussions with e-geos and Sinergise (commercial companies in Europe) to explore approaches for improved S1 and S2 data preparation. Each of these companies is involved with DIAS projects. e-goes has developed a web-based prototype CUBEO tool that pre-processes Sentinel data to CARD4L format and pushes the products directly to the Amazon cloud ... no downloads. They currently use the Copernicus Open Access Hub for their input data. Sinergise has extensive experience creating S1 and S2 ARD products for cloud storage (CREODIAS, Amazon). These solutions could provide critical “data services” for the Africa Regional Data Cube (ARDC), several other global data cube projects, and may serve the data needs for the new continental DE-Africa initiative.