GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Construction of e-science Environment.

Slides:



Advertisements
Similar presentations
X-SIGMA (An XML based Simple data Integration system for Gathering, Managing and Accessing scientific experimental data in grid environments) Karpjoo
Advertisements

PRAGMA BioSciences Portal Raj Chhabra Susumu Date Junya Seo Yohei Sawai.
Building Portals to access Grid Middleware National Technical University of Athens Konstantinos Dolkas, On behalf of Andreas Menychtas.
W w w. h p c - e u r o p a. o r g HPC-Europa Portal: Uniform Access to European HPC Infrastructure Ariel Oleksiak Poznan Supercomputing.
Prof. Natalia Kussul, PhD. Andrey Shelestov, Lobunets A., Korbakov M., Kravchenko A.
FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.
C. Grimme, A. Papaspyrou Scheduling in C3-Grid AstroGrid-D Workshop Project: C3-Grid Collaborative Climate Community Data and Processing Grid Scheduling.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
15 Chapter 15 Web Database Development Database Systems: Design, Implementation, and Management, Fifth Edition, Rob and Coronel.
GridScape Ding Choon Hoong Grid Computing and Distributed Systems (GRIDS) Lab. The University of Melbourne Melbourne, Australia WW Grid.
Interpret Application Specifications
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
1 Introduction to Web Development. Web Basics The Web consists of computers on the Internet connected to each other in a specific way Used in all levels.
- 1 - Grid Programming Environment (GPE) Ralf Ratering Intel Parallel and Distributed Solutions Division (PDSD)
Beyond DHTML So far we have seen and used: CGI programs (using Perl ) and SSI on server side Java Script, VB Script, CSS and DOM on client side. For some.
ATLAS DQ2 Deletion Service D.A. Oleynik, A.S. Petrosyan, V. Garonne, S. Campana (on behalf of the ATLAS Collaboration)
 DATABASE DATABASE  DATABASE ENVIRONMENT DATABASE ENVIRONMENT  WHY STUDY DATABASE WHY STUDY DATABASE  DBMS & ITS FUNCTIONS DBMS & ITS FUNCTIONS 
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong
CHEP 2000, Giuseppe Andronico Grid portal based data management for Lattice QCD data ACAT03, Tsukuba, work in collaboration with A.
ESP workshop, Sept 2003 the Earth System Grid data portal presented by Luca Cinquini (NCAR/SCD/VETS) Acknowledgments: ESG.
F. Toussaint (WDCC, Hamburg) / / 1 CERA : Data Structure and User Interface Frank Toussaint Michael Lautenschlager World Data Center for Climate.
CSE 548 Advanced Computer Network Security Document Search in MobiCloud using Hadoop Framework Sayan Cole Jaya Chakladar Group No: 1.
Bulk Metadata Structures in CERA Frank Toussaint, Michael Lautenschlager Max-Planck-Institut für Meteorologie World Data Center for Climate.
1 DIRAC – LHCb MC production system A.Tsaregorodtsev, CPPM, Marseille For the LHCb Data Management team CHEP, La Jolla 25 March 2003.
The PROGRESS Grid Service Provider Maciej Bogdański Portals & Portlets 2003 Edinburgh, July 14th-17th.
Introduction to the Adapter Server Rob Mace June, 2008.
Bookkeeping Tutorial. Bookkeeping & Monitoring Tutorial2 Bookkeeping content  Contains records of all “jobs” and all “files” that are created by production.
ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
INFSO-RI Enabling Grids for E-sciencE Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal P. Kacsuk*,
Shannon Hastings Multiscale Computing Laboratory Department of Biomedical Informatics.
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Code Applications Tamas Kiss Centre for Parallel.
Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,
Framework for MDO Studies Amitay Isaacs Center for Aerospace System Design and Engineering IIT Bombay.
Jian Gui WANG New Implementation of Agriculture Models APAN19---Jan New Implementations of Agriculture Models Using Mediate Architecture.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
Presented by Scientific Annotation Middleware Software infrastructure to support rich scientific records and the processes that produce them Jens Schwidder.
What is SAM-Grid? Job Handling Data Handling Monitoring and Information.
EGEE-II INFSO-RI Enabling Grids for E-sciencE The GILDA training infrastructure.
GRIDS Center Middleware Overview Sandra Redman Information Technology and Systems Center and Information Technology Research Center National Space Science.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
Presented by Jens Schwidder Tara D. Gibson James D. Myers Computing & Computational Sciences Directorate Oak Ridge National Laboratory Scientific Annotation.
May 6, 2002Earth System Grid - Williams The Earth System Grid Presented by Dean N. Williams PI’s: Ian Foster (ANL); Don Middleton (NCAR); and Dean Williams.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Services for advanced workflow programming.
FRANEC and BaSTI grid integration Massimo Sponza INAF - Osservatorio Astronomico di Trieste.
Greenstone Internals How to Build a Digital Library Ian H. Witten and David Bainbridge.
AliEn AliEn at OSC The ALICE distributed computing environment by Bjørn S. Nilsen The Ohio State University.
H. Widmann (M&D) Data Discovery and Processing within C3Grid GO-ESSP/LLNL / June, 19 th 2006 / 1 Data Discovery and Basic Processing within the German.
1 THETIS:A DATA MANAGEMENT AND DATA VISUALIZATION SYSTEM FOR SUPPORTING COASTAL ZONE MANAGEMENT OF THE MEDITERRANEAN SEA (F0069: Telematics on Research)
Bookkeeping Tutorial. 2 Bookkeeping content  Contains records of all “jobs” and all “files” that are produced by production jobs  Job:  In fact technically.
Super Computing 2000 DOE SCIENCE ON THE GRID Storage Resource Management For the Earth Science Grid Scientific Data Management Research Group NERSC, LBNL.
PROGRESS: GEW'2003 Using Resources of Multiple Grids with the Grid Service Provider Michał Kosiedowski.
Copyright 2007, Information Builders. Slide 1 iWay Web Services and WebFOCUS Consumption Michael Florkowski Information Builders.
+ Support multiple virtual environment for Grid computing Dr. Lizhe Wang.
David Adams ATLAS ATLAS Distributed Analysis and proposal for ATLAS-LHCb system David Adams BNL March 22, 2004 ATLAS-LHCb-GANGA Meeting.
© Geodise Project, University of Southampton, Workflow Support for Advanced Grid-Enabled Computing Fenglian Xu *, M.
Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre.
MSF and MAGE: e-Science Middleware for BT Applications Sep 21, 2006 Jaeyoung Choi Soongsil University, Seoul Korea
Introduction to the SAM System at DØ Physics 5391 July 1, 2002 Mark Sosebee U.T. Arlington.
Simulation Production System Science Advisory Committee Meeting UW-Madison March 1 st -2 nd 2007 Juan Carlos Díaz Vélez.
The AstroGrid-D Information Service Stellaris A central grid component to store, manage and transform metadata - and connect to the VO!
V7 Foundation Series Vignette Education Services.
Joe Foster 1 Two questions about datasets: –How do you find datasets with the processes, cuts, conditions you need for your analysis? –How do.
Grid Services for Digital Archive Tao-Sheng Chen Academia Sinica Computing Centre
University of Technology
MIK 2.1 DBNS - introduction to WS-PGRADE, 2013
Data Management Components for a Research Data Archive
Introduction to the SHIWA Simulation Platform EGI User Forum,
Presentation transcript:

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Construction of e-science Environment for Meteorological Data in Korea Jai-Ho Oh Dept. Environmental Atmospheric Science Pukyong National University

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Contents Introduction of the meteorological data sharing system. –Background –Concept –Architecture of the meteorological data sharing system Meteorological Information Search/Display system. Conclusion and Future Works

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Introduction of the meteorological data sharing system Background –Too many types of meteorology data is produced everyday with big size. –Difficulties to find meteorology data that scientists required to model. Concepts –If we have a metadata of meteorology data in indexed DB from the SAM file stored in SAN, we can easily find meteorology data. –If we control sharing permission of each meteorology data, we can safely handle and share the meteorology data. –If we are able to display the meteorology data before download, we can decide if the searched data is really one we need.

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Development of Meteorological Data Sharing System Construction of Internal Data Grid Basis using Super Computer –Construction of Data Grid middleware Resource Broker Data management : Replica Service Information and Monitoring Service Large amount storage service Security –Preparation of User Interface for Conjugation of Data Grid Design and Preparation of Meta catalog for Numerical Simulations –Design Metadata of KMA NWP data following WMO metadata Standards –Construction of NWP data sharing prototype using Metadata catalog –Meta Catalog Metadata Presentation Metadata Update Data Formats Support of data transport protocols Support of data access protocols Authentication and Authorization Advanced World Meteorology Information Network V-GISC and Logical Harmony of Metadata

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Detail objectives of Meteorological Data Sharing System Data Management Service : Simple way to search and acquire Selected Meteorological data (sorted by Area and Time) - Development of platform for WMO data sharing - Compatible with Data Grid Construction of Data Pool - Shared Data server using Data broker - Development of Common Interface for Atmospheric data server as like LAS (Live Access Server) & GDS (GrADS Data Server) Metadata cataloging Service - Prototype design for Metadata of Meteorological data - DataBase of Metadata Output Display Service - Various Visualization techniques to interpret Meteorological model results - GridSphere based on GrADS Plug-in

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Structure of meteorological Data sharing system

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Architecture of the meteorological data sharing system Architecture of the meteorology data sharing system –Consist of Two systems. Meteorological Administration Center Meteorology information search/display system Meteorologica l Administratio n center Meteorology information display system

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Access control of meteorological data(1) Select meteorology data we want to change Permission and Information. Show all meteorology data

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Access control of meteorological data(2) Click Enable to approve sharing and add additional Info Click Disable to prevent sharing

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Manage the information of meteorological data Click Meta-Info to see description

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Search Agent configuration Search Configuration DB Setting the search condition keep the search condition in DB Search Agent Search meteorological data using search condition in DB Search Condition 1.File Type 2.Production time 3.Execution time

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Work flow of the search agent WGRIB grib2ctl Control File Search Agent Register Agent DBMS 1. Search meteorological data to register 2. Make a Control File for Register Agent 3. Register meta data of meteorological data to DBMS

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches History view of the search request

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Register data grid server

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Meteorological information display system Software used by the meteorology information display system –GridSphere –Globus Toolkit –GrADS Functions of the meteorology information display system –Seek meteorology data using search condition. –Show the list of results. –Setting the GrADS display options. –Show meteorology data by graphic. –Users can have their own DataBox so that they can keep the searched results for later use.

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Structure of the meteorological information display system

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Legacy System Integration Efforts GDS&LAS Data Broker Implemented Data Broker for GDS&LAS Search Agent invokes Data Broker to add new datasets into the configuration file Run ‘Restart’ script to affect recently added datasets

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Work Flow between GridSphere and USER Authentication using GT Connect to the web server of Gridsphere Job Creation

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Search Option Page Input the Search Condition and Push the Search button Show the search result

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Setting the GrADS option And push Show Graph button

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Image Output(1)

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Image Output(2)

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Conclusion & Future Work Efficient Way for Data Sharing for Administrator Agent Based Data Searching/Cataloging Integration with Grid Sphere (portlet) On-Demand Service Architecture Dynamic Meta Data Cataloging WSRF based Data Sharing Extension to various types of data Need more collaboration with other nations ’ weather centers Any contributions to the E-Community