Bringing processing Close to the data Richard MORENO CEOS - WGISS

Slides:

Advertisements

Similar presentations

BEDI -Big Earth Data Initiative

Advertisements

UNCLASSIFIED: LA-UR Data Infrastructure for Massive Scientific Visualization and Analysis James Ahrens & Christopher Mitchell Los Alamos National.

Monetize Your Website Audience and Manage Digital Ad Campaigns with Admixer.Publisher, Built on the Powerful Microsoft Azure Platform MICROSOFT AZURE ISV.

Maximize Return on Engagement via Scalable Omni-Channel Online Services in the Cloud COMPANY PROFILE: XOMNI, INC. Founded in 2011 and headquartered in.

GoogleDictionary Paul Nepywoda Alla Rozovskaya. Goal Develop a tool for English that, given a word, will illustrate its usage.

The Helix Nebula Marketplace HNX The European cloud marketplace for scientists, researchers, developers & public organisations Marc-Elian Bégin, CEO, Co-founder,

WGISS Working Group on Information Systems and Services Richard MORENO CNES WGISS report, Agenda Item 14 Tromsø, Norway October 2014.

WGISS Richard Moreno CNES SIT Workshop Agenda Item #7 WGISS working group CEOS SIT Technical Workshop EUMETSAT, Darmstadt, Germany 17 th -18 th September.

RDA Data Support Section. Topics 1.What is it? 2.Who cares? 3.Why does the RDA need CISL? 4.What is on the horizon?

Review of technologies for developing geospatial applications with a focus on open source (FOSS4G) and their implementation of cloud computing application.

GFOI Space Data Services 3-year Work Plan Brian Killough NASA LaRC, CEOS SEO Presented at SDCG-6 Oslo, Norway October 22-24, 2014.

Axis AI Solves Challenges of Complex Data Extraction and Document Classification through Advanced Natural Language Processing and Machine Learning MICROSOFT.

Our Place in the Cloud DCIA P2P & Cloud Market Conference March 9, 2010.

WGISS Working Group for Information Systems & Services Andrew Mitchell NASA– WGISS chair WGISS-41 Canberra – Australia 14 th March – 18 th March 2016 Committee.

An Open Data Platform in the framework of the EGI-LifeWatch Competence Centre Fernando Aguilar Jesús Marco

ETERE A Cloud Archive System. Cloud Goals Create a distributed repository of AV content Allows distributed users to access.

CEOS Working Group on Information System and Services (WGISS) Data Access Infrastructure and Interoperability Standards Andrew Mitchell - NASA Goddard.

Technology Exploration -Discussion and Next Steps- Satoko H. MIURA Tech. Expo Group Lead JAXA.

EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No Aalto Data Repository.

EGI-InSPIRE RI An Introduction to European Grid Infrastructure (EGI) March An Introduction to the European Grid Infrastructure.

Sentinel Data Access for Africa 11 December 2014 Sentinels Data Access for Africa.

Future Data Access and Analysis Architectures – Discussion

Copernicus Data & Information Access Service → DIAS 28 September 2016.

Datacube projects in ESA

NextGEOSS data hub incl. alpha release

CNES DATA CUBE ERWANN poupart

Tokamak data mirror for JET and MAST Moving towards an open data repository for European nuclear fusion research.

SEXTANT PRODUCTS CATALOGUE

WGISS Working Group for Information Systems & Services

Data Analytics and CERN IT Hadoop Service

MMG: from proof-of-concept to production services at scale (part II)

INTAROS WP5 Data integration and management

Business in a Connected World

Presentation on Copernicus Dissemination

PROCESS - H2020 Project Work Package WP6 JRA3

CEOS OpenSearch Project II

FDA context Steven Hosford On behalf of FDA Co-chairs

Yellowfin: An Azure-Compatible Business Intelligence Platform That Connects People with Their Data for Better Decision Making MICROSOFT AZURE APP BUILDER.

CEOS Workplan FDA actions

Exploitation Platforms and Common Reference Architecture

Open Data Cubes Cloud Services Experiences and Lessons Learned

Future Data Architectures Big Data Workshop – April 2018

CODE-DE Copernicus Data and Exploitation Platform - Deutschland

DATA SPHINX & EUDAT Collaboration

WGISS Connected Data Assets April 5, 2017 Yonsook Enloe

The Only Digital Asset Management System on Microsoft Azure, MediaValet Is Uniquely Equipped to Meet Any Company’s Needs MICROSOFT AZURE ISV PROFILE: MEDIAVALET.

FDA Objectives and Implementation Planning

Introducing Qwory, a Business-to-Business Search Engine That’s Powered by Microsoft Azure and Detects Vital Contact Information for Businesses MICROSOFT.

EOSCpilot All Hands Meeting 8 March 2018 Pisa

XtremeData on the Microsoft Azure Cloud Platform:

Abiquo’s Hybrid Cloud Management Solution Helps Enterprises Maximise the Full Potential of the Microsoft Azure Platform MICROSOFT AZURE ISV PROFILE: ABIQUO.

A distributed data-centric infrastructure

Thales Alenia Space Competence Center Software Solutions

SEO Report to WGISS Brian Killough CEOS Systems Engineering Office (SEO) WGISS-46 Meeting October 22, 2018.

FDA Topics Going Forward…???

Chris Lynnes with contributions from Katie Baynes NASA/GSFC

M. Albani, P. Mougnaud, A. Della Vecchia (ESA)

School Districts Can Analyze and Report on Data Across Multiple Systems with EdWire, a Powerful Integration Solution that Utilizes Microsoft Azure MICROSOFT.

GEO-Amazon Cloud Credit Program and a Prototype Sentinel Data Pipeline

A. Della Vecchia, D. Guerrucci (ESA)

Food Security TEP - WGISS-47

COMPANY PROFILE: REELWAY

Who is Using your webSite?

A Survey of Interactive Execution Environments

Roadmap and short term activities on interoperability of Data Cubes

OGC Happenings: OGC19-020: Testbed-15 Service Discovery

ESA EO Thesauri Andrea Della Vecchia (Randstad), Yves Coene (Spacebel)

Federated Earth Observation (FedEO)

SEO Report to WGISS-48 Brian Killough

Presentation transcript:

Bringing processing Close to the data Richard MORENO CEOS - WGISS 15 march 2016

SUMMARY Bringing processing Close to the data Context Data downloading improvement Processing close to the data Big data and distributed architecture Is Cloud computing THE solution ? SUMMARY

Need to change the data usage Context Big data Big volume Big processing Need to change the data usage Limitations Cost of storage of several PB Bandwidth resources No more downloading the full archive or bulk extraction Bring the processing close to the data Copernicus / Datacube Integration of the Copernicus mirrors In Europe Worldwide ??? Need / possibility to federate datacube : cube of cubes

Data downloading improvement Tools / Standards / Interoperabilty – French Coperniccus CollGS Natural langage for searching data of interest Web services access - opensearch Bulk extraction Metalink Jdownloader Do not solve Bandwidth resource Duplication of storage

Processing close to the data Different types of processing Interactive processing via web services : WPS Interactive processing via MMI Google engine, GA Analytics Expression langage Notebook (eg. Jupyter) Mass processing on HPC / Cloud SandBox for algorithms / processing tuning

Big data and distributed architecture Is big data compatible with distributed architecture ? Examples OGC OWS-10 ESA and european agencies Federated pilot EUCLID project Can be generalized ? Is centralized platform / cloud the unique solution ?

Is Cloud computing THE solution ? Advantages and disadvantages of Cloud computing based archiecture ?