Chris Lynnes with contributions from Katie Baynes NASA/GSFC

Slides:

Advertisements

Similar presentations

LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (

Advertisements

1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.

BEDI -Big Earth Data Initiative

R and HDInsight in Microsoft Azure

Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.

Next Generation Application Platform (NGAP) Andrew Mitchell WGISS-39 Tsukuba, Japan Monday, May 11,

Big Data A big step towards innovation, competition and productivity.

1 Closing Report on WTF-CEOP Project Team Satoko Horiyama MIURA/JAXA.

Promising data analytics technologies Tiffany Mathews.

, Increasing Discoverability and Accessibility of NASA Atmospheric Science Data Center (ASDC) Data Products with GIS Technology ASDC Introduction The Atmospheric.

Updates from EOSDIS -- as they relate to LANCE Kevin Murphy LANCE UWG, 23rd September

U.S. Department of the Interior U.S. Geological Survey Diving into the Data Pool with DAAC2Disk Kelly Lemig ERT, Inc., contractor to the U.S. Geological.

Global Imagery Browse Services Overview AMSR Science Team Meeting Matt Cechini, GIBS Lead Helen Conover, AMSR-E SIPS.

Ethics of Big Data Eduardo Felipe Zecca da Cruz. What is Big Data? Stamford, Conn.-based IT research firm Gartner Inc. defines "big data" as "high-volume,

EOSDIS Status 9/29/2010 Dan Marinelli, NASA GSFC

, Key Components of a Successful Earth Science Subsetter Architecture ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.

WGISS 37 Cocoa Beach Florida April 18, 2014 Andrew Mitchell Satoko H. MIURA.

What is CWIC? Authors: Doug Newman Andrew Mitchell

EOSDIS Status 10/16/2008 Dan Marinelli, Science Systems Development Office.

Evolving toward a Coherent, Collaborative Framework for Earth Science Data, Tools and Services Christopher Lynnes, Kwo-Sen Kuo and Kevin Murphy Earth Science.

Evolving a Legacy System Evolution of the Earth Observing Data and Information System M. Esfandiari 1, H. Ramapriyan 1, J. Behnke 1, E. Sofinowski 2 1.

What is ECHO? ECHO Open Search ECHO Facts NASA’s Earth Observing System ClearingHOuse (ECHO) acts as the core metadata.

E ARTHCUBE C ONCEPTUAL D ESIGN A Scalable Community Driven Architecture Overview PI:

National Land Remote Sensing System Data Distribution Model.

E-Sensing: Big Earth observation data analytics for land use and land cover change information.

1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)

LP DAAC Overview – Land Processes Distributed Active Archive Center Chris Doescher LP DAAC Project Manager (605) Chris Torbert.

Earth Science Data and Information System (ESDIS) Project Update Jeanne Behnke, Deputy Project Manager for Operations NASA Earth Science Data & Information.

Snip2Code: Search, Share and Collect Code Snippets Faster, Easier, Efficiently with Power of Microsoft Azure Platform MICROSOFT AZURE ISV PROFILE: SNIP2CODE.

WMO WIS strategy – Life cycle data management WIS strategy – Life cycle data management Matteo Dell’Acqua.

Evolving the Management and Dissemination of NASA Earth Observation Data in a Big Data World Jeff Walter 1, Mark McInerney 2 (1)NASA Langley Research Center;

Deconstruction and Recovery Information Modelling

Future Data Access and Analysis Architectures – Discussion

2nd GEO Data Providers workshop (20-21 April 2017, Florence, Italy)

Advanced Topics in Concurrency and Reactive Programming: Case Study – Google Cluster Majeed Kassis.

Presented to the CEOS WGISS

EOS ClearingHOuse Robin Pfister Code 423, NASA/GSFC

Relevancy Ranking of Satellite Dataset Search Results

Global Precipitation Data Access, Value-added Services and Scientific Exploration Tools at NASA GES DISC Zhong Liu1,4, D. Ostrenga1,2, G. Leptoukh4, S.

Gather Valuable Customer Data

NASA Langley Research Center, Hampton VA

EOSDIS Data Preservation Archive (EDPA)

Technology Exploration Cloud Hosting at USGS

Azure Lays the Path to Achieving Holistic App Performance Management, Cloud Optimization MINI-CASE STUDY “Microsoft Azure has allowed us to build a unified,

GCMD’s New Keyword Search Interface ‘Alpha Version’

NASA Agency Report Andrew Mitchell Deputy Project Manager / Technical

OpenNebula Offers an Enterprise-Ready, Fully Open Management Solution for Private and Public Clouds – Try It Easily with an Azure Marketplace Sandbox MICROSOFT.

H3 Solutions and the Azure Government Cloud Team Up to Power Contextual Intelligence Platform – Where Big Data Meets Business Productivity MICROSOFT AZURE.

Bringing processing Close to the data Richard MORENO CEOS - WGISS

Open Data Cubes Cloud Services Experiences and Lessons Learned

Future Data Architectures Big Data Workshop – April 2018

SEO Report Brian Killough NASA, CEOS Systems Engineering Office

Andrew Mitchell São Paulo, Brazil 12th of April 2018

Voice Analytics on Microsoft Azure Allows Various Customers to Get the Most Out of Conversations with Clients Through Efficient Content Analysis MICROSOFT.

ResourceFirst Puts Emphasis on Communication, Uses Power of Azure to Bring Successful Resource and Portfolio Management to Companies Globally MICROSOFT.

Technology Exploration Cloud Hosting at USGS

Earth Data Search Tool Demo

NASA Earthdata Login Andrew Mitchell - NASA WGISS-44 Beijing, China

FDA Objectives and Implementation Planning

Tools for Processing Big Data Jinan Al Aridhee and Christian Bach

WIS Strategy – Toward WIS 2.0

One-Stop Shop Manages All Technical Vendor Data and Documentation and is Globally Deployed Using Microsoft Azure to Support Asset Owners/Operators MICROSOFT.

WIS Strategy – WIS 2.0 Submitted by: Matteo Dell’Acqua(CBS) (Doc 5b)

Improve Patient Experience with Saama and Microsoft Azure

Nuvolex and Microsoft Azure Combine to Deliver a Multitenant Office 365 Management Platform that Ranks Among Most Advanced in the Industry MICROSOFT AZURE.

About Thetus Thetus develops knowledge discovery and modeling infrastructure software for customers who: Have high value data that does not neatly fit.

Zendos Tecnologia Utilizes the Powerful, Scalable

GEO-Amazon Cloud Credit Program and a Prototype Sentinel Data Pipeline

SEO Report to WGISS-48 Brian Killough

Presentation transcript:

Migrating the Earth Observing System Data and Information System into the Cloud Chris Lynnes with contributions from Katie Baynes NASA/GSFC Earth Observing System Data and Information System

EOSDIS Functions EOSDIS Research distribute Applications Education data downlink subset Users archive capture and clean process

DAACs and users are supported by EOSDIS Common Services

EOSDIS Works with (pretty) Big Data Archive Volume Petabytes Distribution Volume

EOSDIS Cloud Plans

Planned Cloud Prototypes 3 1 2 Key Archive Mgmt Analytics Support Application Hosting

Archive and Distribution Prototype(s) 1 Archive and Distribution Prototype(s) Cloud-based data distribution of Sentinel radar data by the Alaska Satellite Facility DAAC

Cloud Analytics Prototypes 2 Cloud Analytics Prototypes Containerized tools: EOSDIS-specific: Search / access Analyze Open source R, SciPy, nco, IDV, … Analytics-optimized storage examples Hadoop, MapReduce, Cassandra, SciDB, Rasdaman, Google Earth Engine, ...

Application-Hosting Prototypes 3 Application-Hosting Prototypes Port existing services to cloud Common Metadata Repository Global Imagery Browse System Earthdata Search client Leveraging a reusable EOSDIS framework NGAP...

Yay, Cloud! ...but, just 1 or 2, or maybe 8, big questions to answer along the way...

But, just 1 or 2, or maybe 8, BIG questions to answer along the way... Yay, Cloud! But, just 1 or 2, or maybe 8, BIG questions to answer along the way...

How can we supply data to all users on a non-discriminatory basis? Big Questions How can we supply data to all users on a non-discriminatory basis?

2. How can we avoid storage lock-in? Big Questions 2. How can we avoid storage lock-in?

Big Questions 3. How can we attract end users to the cloud, and how will that alter usage patterns?

4. How can we predict pricing 2- 5 years out... Big Questions 4. How can we predict pricing 2- 5 years out... ...especially as usage patterns change

Big Questions 5. How can we migrate near- archive data services to Web Object Storage? Web Object Storage OPeNDAP w10n OGC W*S

6. What functionality or data should NOT go into the cloud? Big Questions 6. What functionality or data should NOT go into the cloud?

Big Questions 7. How do we handle provisioning and accounting of cycles and storage across the DAACs*? * 4 NASA Centers, 2 Other federal agencies, 4 Universities

8. Do we need new operations policies or procedures? Big Questions 8. Do we need new operations policies or procedures?