Nutzerworkshop 05./06.10.2015 Data Analysis and Visual Output Andrea Pozzer (MPIC) Wolfgang Mueller (MPIM)

Slides:

Advertisements

Similar presentations

EUFORIA FP7-INFRASTRUCTURES , Grant JRA4 Overview and plans M. Haefele, E. Sonnendrücker Euforia kick-off meeting 22 January 2008 Gothenburg.

Advertisements

Climate Analytics on Global Data Archives Aparna Radhakrishnan 1, Venkatramani Balaji 2 1 DRC/NOAA-GFDL, 2 Princeton University/NOAA-GFDL 2. Use-case 3.

Welcome to the Plant Breeding and Genomics Webinar Series Today’s Presenter: Dr. Heather Merk Presentation & Supplemental Files:

Strategies for solving scientific problems using computers.

Rhea Analysis & Post-processing Cluster Robert D. French NCCS User Assistance.

23/04/2008VLVnT08, Toulon, FR, April 2008, M. Stavrianakou, NESTOR-NOA 1 First thoughts for KM3Net on-shore data storage and distribution Facilities VLV.

UNCLASSIFIED: LA-UR Data Infrastructure for Massive Scientific Visualization and Analysis James Ahrens & Christopher Mitchell Los Alamos National.

CLIMATE SCIENTISTS’ BIG CHALLENGE: REPRODUCIBILITY USING BIG DATA Kyo Lee, Chris Mattmann, and RCMES team Jet Propulsion Laboratory (JPL), Caltech.

1 Exploring Data Reliability Tradeoffs in Replicated Storage Systems NetSysLab The University of British Columbia Abdullah Gharaibeh Matei Ripeanu.

IS-ENES [ees-enes] InfraStructure for the European Network for Earth System Modelling IS-ENES will develop a virtual Earth System Modelling Resource Centre.

K E Y : SW Service Use Big Data Information Flow SW Tools and Algorithms Transfer Application Provider Visualization Access Analytics Curation Collection.

Capacity analysis of complex materials handling systems.

Nick Draper Teswww.mantidproject.orgwww.mantidproject.org Instrument Independent Reduction and Analysis at ISIS and SNS.

Active Monitoring in GRID environments using Mobile Agent technology Orazio Tomarchio Andrea Calvagna Dipartimento di Ingegneria Informatica e delle Telecomunicazioni.

INFSO-RI Enabling Grids for E-sciencE EGEODE VO « Expanding GEosciences On DEmand » Geocluster©: Generic Seismic Processing Platform.

Tag line, tag line Power Management in Storage Systems Kaladhar Voruganti Technical Director CTO Office, Sunnyvale June 12, 2009.

An Introduction to Visual Basic.NET. What is.NET.Net is a new framework for developing windows-based and web-based applications within the Microsoft environment.

Weka: Experimenter and Knowledge Flow interfaces Neil Mac Parthaláin

Data Publication and Quality Control Procedure for CMIP5 / IPCC-AR5 Data WDC Climate / DKRZ:

And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR

K E Y : SW Service Use Big Data Information Flow SW Tools and Algorithms Transfer Transformation Provider Visualization Access Analytics Curation Collection.

A Data Access Framework for ESMF Model Outputs Roland Schweitzer Steve Hankin Jonathan Callahan Kevin O’Brien Ansley Manke.

VAPoR: A Discovery Environment for Terascale Scientific Data Sets Alan Norton & John Clyne National Center for Atmospheric Research Scientific Computing.

VICOMTECH VISIT AT CERN CERN 2013, October 3 rd & 4 th O.COUET CERN/PH/SFT DATA VISUALIZATION IN HIGH ENERGY PHYSICS THE ROOT SYSTEM.

Near Real-Time Verification At The Forecast Systems Laboratory: An Operational Perspective Michael P. Kay (CIRES/FSL/NOAA) Jennifer L. Mahoney (FSL/NOAA)

Introduction to R Introductions What is R? RStudio Layout Summary Statistics Your First R Graph 17 September 2014 Sherubtse Training.

PRISM WP4a Visualization Meeting DKRZ / MPI-MaD Hamburg 29th January 2002 PRISM WP4a Visualization Meeting DKRZ / MPI-MaD Hamburg 29 th January 2002.

6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.

MetaManager As We Use It. Who is Crop Production Services? Global Agricultural company focused on retail sales of Fertilizer, Chemicals and Seed Child.

Microsoft Azure Active Directory. AD Microsoft Azure Active Directory.

1 Adventures in Web Services for Large Geophysical Datasets Joe Sirott PMEL/NOAA.

1 Accomplishments. 2 Overview of Accomplishments  Sustaining the Production Earth System Grid Serving the current needs of the climate modeling community.

SCD Research Data Archives; Availability Through the CDP About 500 distinct datasets, 12 TB Diverse in type, size, and format Serving 900 different investigators.

An introduction to CDO, NCL and PRECIS utilities

National Center for Supercomputing Applications University of Illinois at Urbana–Champaign Visualization Support for XSEDE and Blue Waters DOE Graphics.

CISC 849 : Applications in Fintech Namami Shukla Dept of Computer & Information Sciences University of Delaware iCARE : A Framework for Big Data Based.

LAS and THREDDS: Partners for Education Roland Schweitzer Steve Hankin Jonathan Callahan Joe Mclean Kevin O’Brien Ansley Manke Yonghua Wei.

Clustering in R Xue li CS548 showcase. Source html project.org/web/packages/cluster/index.html.

Cal/Val for physics MED-MFC internal meeting CMCC-INGV-SOCIB Lecce E. Clementi, INGV.

Data and storage services on the NGS.

End-to-End Data Services A Few Personal Thoughts Unidata Staff Meeting 2 September 2009.

IPCC WG II + III Requirements for AR5 Data Management GO-ESSP Meeting, Paris, Michael Lautenschlager, Hans Luthardt World Data Center Climate.

K E Y : DATA SW Service Use Big Data Information Flow SW Tools and Algorithms Transfer Hardware (Storage, Networking, etc.) Big Data Framework Scalable.

Climate-SDM (1) Climate analysis use case –Described by: Marcia Branstetter Use case description –Data obtained from ESG –Using a sequence steps in analysis,

5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.

Joint Institute for Nuclear Research Synthesis of the simulation and monitoring processes for the data storage and big data processing development in physical.

LIOProf: Exposing Lustre File System Behavior for I/O Middleware

INFSO-RI Enabling Grids for E-sciencE ESR Database Access K. Ronneberger,DKRZ, Germany H. Schwichtenberg, SCAI, Germany S. Kindermann,

2016 DAF Survey Results.

ARIS Extension Pack TOGAF April 2016

Delivery of Science Components to NRCS Business Applications

Trusted Repository Systems Overview

Introduction to Geospatial Technology

Visualisation in climate modelling

INTAROS WP5 Data integration and management

Intro to CMIP, the WHOI CMIP5 community server, and planning for CMIP6

Cloud Computing & ANalytics

Data Ingestion in ENES and collaboration with RDA

CMIP6-DICAD – FU Berlin - Germany

Principles of Eurostat's dissemination strategy

TIGGE Archives and Access

Analytics and OR DP- summary.

The Six Parts of a GIS.

Computing Infrastructure for DAQ, DM and SC

(VIP-EDC) Point 6 of the agenda

Storing and Accessing G-OnRamp’s Assembly Hubs outside of Galaxy

how users and data producers interact on WIS

Enol Fernandez & Giuseppe La Rocca EGI Foundation

Statistical Process Control

Statistical Process Control

Presentation transcript:

Nutzerworkshop 05./06.10.2015 Data Analysis and Visual Output Andrea Pozzer (MPIC) Wolfgang Mueller (MPIM)

Data Analysis and Visualization Nutzerworkshop 05./06.10.2015 Data Analysis and Visualization What softwares/courses are necessary for a full usage of the DKRZ capabilities? Which services will be needed for the future (near term projects 1-3yr, long-term strategy >5yr)? What are the main need for our dissemination and public relation?

Data analysis and visualisation Nutzerworkshop 05./06.10.2015 Data analysis and visualisation Refer to pure data analysis : Other work packages: Data provision (Datenbereitstellung) working group for large dataset such as World Data Center for Climate) Data production (Datenproduktion) for storage and data model runs DKRZ is NOT only data producer Dedicated server for visualisation Set of tools for data analysis

Analysis and visualization: important part of the DKRZ services Nutzerworkshop 05./06.10.2015 Analysis and visualization: important part of the DKRZ services

Analysis and visualization: important part of the DKRZ services Nutzerworkshop 05./06.10.2015 Analysis and visualization: important part of the DKRZ services

Yearly ~ 10 PB data are produced and stored: Nutzerworkshop 05./06.10.2015 Yearly ~ 10 PB data are produced and stored: How to analyze them? Set of software normally available: NCL, CDO, R ... all allow statistical analysis Some are free (open source) some are not (e.g. matlab) Is anything missing that MUST be there? Is the software enough ?

Yearly ~ 10 PB data are produced and stored: Nutzerworkshop 05./06.10.2015 Yearly ~ 10 PB data are produced and stored: How to analyze them? Beside the software... Is any need for algorithm repository? Frameworks for data analysis: ESMVal Tool (DLR)1, MiKlip Evaluation System (FUB)2 ... BUT.... These are CMIP and Climate Prediction related analyses Offers standard analysis and should/must be extended How much could be standardized? Diversity vs. Applicability 1 http://www.geosci-model-dev-discuss.net/8/7541/2015/gmdd-8-7541-2015.html 2 https://www-miklip.dkrz.de/

Nutzerworkshop 05./06.10.2015 Visualization Mistral offers 12GPU node dedicated to 2D-3D visualization. LUSTRE already mounted there, i.e. no need of data transfer! Software available: Anything missing here? 3D Avizo Paraview Simvis Vapor IDL 2D Ferret NCL GRADS GMT

Nutzerworkshop 05./06.10.2015 Visualization Additional services: In situ visualisation and video lab (3D) + direkt support from DKRZ See movie example....

Data Analysis and Visualization Nutzerworkshop 05./06.10.2015 Data Analysis and Visualization What softwares/courses are necessary for a full usage of the DKRZ capabilities? Which services will be needed for the future (near term projects 1-3yr, long-term strategy >5yr)? What are the main need for our dissemination and public relation?