JAXA GRID Activity JAXA/EORC Kengo Aizawa KEIO UNIVHiromichi Fukui.

Slides:

Advertisements

Similar presentations

Chapter 1: The Database Environment

Advertisements

An Overview of the Integration of the UCSF Dept. of Radiology Teaching File with MIRC Wyatt M. Tellis University of California San Francisco Departments.

A Communication Relay Mechanism toward Construction of Virtual Cluster on Orverlay Network PRAGMA14 Working Group March 2008 Yasuyuki Kusumoto Osaka.

NetSEC: metrology-based application for network security Jean-François SCARIOT Bernard MARTINET Centre Interuniversitaire de Calcul de Grenoble TNC 2002.

© 2007 Open Grid Forum Data Management Challenge - The View from OGF OGF22 – February 28, 2008 Cambridge, MA, USA Erwin Laure David E. Martin Data Area.

ASIAES Project Overview Satellite Image Network for Natural Hazard Management in ASEAN+3 region Pakorn Apaphant Geo-Informatics and Space Technology Development.

JAXA GRID Update September 2005 JAXA/EORC Kengo Aizawa KEIO UNIVHiromichi Fukui.

ESA Data Integration Application Open Grid Services for Earth Observation Luigi Fusco, Pedro Gonçalves.

1 Copyright © 2005, Oracle. All rights reserved. Introducing the Java and Oracle Platforms.

GridPP July 2003Stefan StonjekSlide 1 SAM middleware components Stefan Stonjek University of Oxford 7 th GridPP Meeting 02 nd July 2003 Oxford.

The Quantum Chromodynamics Grid James Perry, Andrew Jackson, Matthew Egbert, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

Eldas 1.0 Enterprise Level Data Access Services Design Issues, Implementation and Future Development Davy Virdee.

Heroix Longitude - multiplatform, automated application performance monitoring and management software.

Database Management Using Microsoft Access Xinhua Chen, Ph.D. Chinese Association of Professionals in Science and Technology March 23, 2003.

Managed by UT-Battelle for the Department of Energy Best Ever Archive Utility, Yet (BEAUtY) Kay Kasemir April 2013.

Kensington Oracle Edition: Open Discovery Workflow Meets Oracle 10g Professor Yike Guo.

4 Oracle Data Integrator First Project – Simple Transformations: One source, one target 3-1.

Retrieval of Information from Distributed Databases By Ananth Anandhakrishnan.

Database System Concepts and Architecture

ArrayExpress Query Interface Gonzalo Garc í a Lara January, / 24.

A Prototype Implementation of a Framework for Organising Virtual Exhibitions over the Web Ali Elbekai, Nick Rossiter School of Computing, Engineering and.

Enhancing Spotfire with the Power of R

LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.

GGF Toronto Spitfire A Relational DB Service for the Grid Peter Z. Kunszt European DataGrid Data Management CERN Database Group.

1 Introduction The Database Environment. 2 Web Links Google General Database Search Database News Access Forums Google Database Books O’Reilly Books Oracle.

SmartSQL AlfaTech Software Solutions Application Requirements Document  Radi Bekker  Vladimir Goldman  Marina Shaevich  Alexander Shapiro Team Members:

Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Technology Education Copyright © 2006 by The McGraw-Hill Companies,

Overview of the ODP Data Provider Sergey Sukhonosov National Oceanographic Data Centre, Russia Expert training on the Ocean Data Portal technology, Buenos.

SQL Power Injector Avadanei AlinBalan Robert. What is SQL Power Injector ?  A graphical application created in C#.Net 1.1 that helps the penetration.

The Japanese Virtual Observatory (JVO) Yuji Shirasaki National Astronomical Observatory of Japan.

NAREGI WP4 (Data Grid Environment) Hideo Matsuda Osaka University.

OracleAS Reports Services. Problem Statement To simplify the process of managing, creating and execution of Oracle Reports.

DBSQL 14-1 Copyright © Genetic Computer School 2009 Chapter 14 Microsoft SQL Server.

Caravan Business Server a viable alternative development platform niti telecom consultancy april 2002.

ES Metadata Management Enabling Grids for E-sciencE ES metadata OGSA-DAI NA4 GA Meeting, D. Weissenbach, IPSL, France.

Prototype system of the Japanese Virtual Observatory The Japanese Virtual Observatory (JVO) aims at providing easy access to federated astronomical databases.

QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

JVO JVO Portal Japanese Virtual Observatory (JVO) Prototype 2 Masahiro Tanaka, Yuji Shirasaki, Satoshi Honda, Yoshihiko Mizumoto, Masatoshi Ohishi (NAOJ),

Fundamentals of Information Systems, Seventh Edition 1 Chapter 3 Data Centers, and Business Intelligence.

Database A database is a collection of data organized to meet users’ needs. In this section: Database Structure Database Tools Industrial Databases Concepts.

Module 6: Implementing SQL Server Replication in an Enterprise Environment.

FEN Introduction to the database field:  Applications, concepts and terminology Seminar: Introduction to relational databases.

ICDL 2004 Improving Federated Service for Non-cooperating Digital Libraries R. Shi, K. Maly, M. Zubair Department of Computer Science Old Dominion University.

“curator” DB design Curator meeting, GFDL, Sep 20.

Middleware for Grid Computing and the relationship to Middleware at large ECE 1770 : Middleware Systems By: Sepehr (Sep) Seyedi Date: Thurs. January 23,

1 © 1999 Microsoft Corp.. Microsoft Repository Phil Bernstein Microsoft Corp.

1 Why Learn About Database Systems? Database systems process and organize large amounts of data Examples –Marketing manager can access customer data –Corporate.

ABSTRACT The JDBC (Java Database Connectivity) API is the industry standard for database- independent connectivity between the Java programming language.

IODE Ocean Data Portal - ODP  The objective of the IODE Ocean Data Portal (ODP) is to facilitate and promote the exchange and dissemination of marine.

Japanese Virtual Observatory Project Abstract : The National Astronomical Observatory of Japan (NAOJ) started the Japanese Virtual Observatory (JVO) project.

DATABASE CONNECTIVITY TO MYSQL. Introduction =>A real life application needs to manipulate data stored in a Database. =>A database is a collection of.

Multilingual prototype GCMD Portal JAXA/EORC Kengo Aizawa KEIO UNIVERSITY Hiromichi Fukui Kazuyoshi Kunisawa March ８, 2005.

Ohio State University Department of Computer Science and Engineering An Approach for Automatic Data Virtualization Li Weng, Gagan Agrawal et al.

Windows 7 WampServer 2.1 MySQL PHP 5.3 Script Apache Server User Record or Select Media Upload to Internet Return URL Forward URL Create.

1 GFDL Data Portal Current Status, Achievements and Future Development NOAATECH-2006 K.Dixon, V.Balaji, S.Nikonov GFDL, Princeton.

Feb 24-27, 2004ICDL 2004, New Dehli Improving Federated Service for Non-cooperating Digital Libraries R. Shi, K. Maly, M. Zubair Department of Computer.

Soon Joo Hyun Database Systems Research and Development Lab. US-KOREA Joint Workshop on Digital Library t Introduction ICU Information and Communication.

Consulting Services JobScheduler Architecture Decision Template Information for Consulting Parties Information for Consulting Parties.

ECMM6018 Enterprise Networking For Electronic Commerce Tutorial 6 CGI/Perl and databases.

Enterprise Database Systems Introduction to SQL Server Dr. Georgia Garani Dr. Theodoros Mitakos Technological.

GRIN-Global Suite of Software Components. Updater.

Christoph F. Eick: Final Words COSC Topics Covered in COSC 3480  Data models (ER, Relational, XML)  Using data models; learning how to store real.

Presented by: K.AMARNATH Ht.no:10841f0045 Guided by: T.Suneetha.

Job workflow Pre production operations:

Database Processing: David M. Kroenke’s Chapter One: Introduction

Database Processing: David M. Kroenke’s Chapter One: Introduction

Topics Covered in COSC 6340 Data models (ER, Relational, XML (short))

نرم افزار گزارش ساز کارنیک

Topics Covered in COSC 6340 Data models (ER, Relational, XML)

Introduction of Week 11 Return assignment 9-1 Collect assignment 10-1

Presentation transcript:

JAXA GRID Activity JAXA/EORC Kengo Aizawa KEIO UNIVHiromichi Fukui

Background JAXA and KEIO Univ are start to try establishing Digital Asia system In Digital Asia, it required to - search and access distributed data - semi-real time data processing JAXA had two of experiments

Purpose Extract problems to realize Digital Asia system by developing prototype. Accumulate knowledge, know-how for design of Digital Asia system.

Data GRID

Search satellite data to each node seamlessly Databases are geographically distributed Each node has different RDB PostgreSQL MySQL Oracle system Search satellite data to each node seamlessly Databases are geographically distributed Each node has different RDB PostgreSQL MySQL Oracle system USER node Nepal node Nepal Assumed scene for Data Grid Node Japan Node Japan PostgreSQL MySQL Oracle Node Chaina Node Chaina

GT3 + OGSA- DAI My SQL Metadata for Unified DB SQL Oracle FADGE XQuery Query engine Procesing metadata Unified data view virtual XML adopt XQuery Interface Translate XQuery to appropriate SQL XQuery results SQL PostgreSQL Apply GT3+OGSA-DAI) access DB over Firewall GT3 + OGSA- DAI JDBC GT3 + OGSA- DAI JDBC GT3 + OGSA- DAI JDBC SQL Unified data view Virtual XML Internet FW DataGrid For Digital Asia Access to hetero DBMS Distributed virtual unification Overview

metadata #3 metadata #2 unified dataview #2 unified dataview #3 XQuery XML XQuery XML XQuery XML System A System B System C metadata #1 metadata For unified DB unified dataview (virtual XML) #1 Create unified data view by Drag and drop Difference of each DB table colmn will absorbed in the editor Columns of each tables will be collected automatically Data Grid For Digital Asia Allows to built virtual unified system for distributed DB Metadata editor for Unified DB Distributed virtual unification Overview

Sample

Computing GRID

Computing node Management server Processing program sceine Systemwalker CyberGRIP Master Systemwalker CyberGRIP Master Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Systemwalker CyberGRIP Agent Processed image [Windows] Windows SV 2003SE/EE Windows 2000 SV/AS/Pro Windows XP Professional [Linux] Red Hat Enterprise Linux AS(v. 3)/ES(v. 3) Red Hat Linux 9 Disk MB memory MB [Windows] Windows SV 2003SE/EE Windows 2000 SV/AS/Pro Windows XP Professional [Linux] Red Hat Enterprise Linux AS(v. 3)/ES(v. 3) Red Hat Linux 9 Disk MB memory MB Red Hat Enterprise Linux AS(v. 3)/ES(v. 3) Disk MB memory MB + User X 13MB) Red Hat Enterprise Linux AS(v. 3)/ES(v. 3) Disk MB memory MB + User X 13MB) Required spec of node Required spec of server Computing GRID Overview image processing image processing Image processing Image processing Image processing Image processing image processing image processing

Computing GRID Experiment We estimated following two cases. - submitting 100 dummy* jobs with no data transfer - submitting 100 dummy* jobs with 400MB data transfer * dummy job takes 90sec

Results

Cyber GRIP OJC(Organic Job Controller) Allows you to automatically generate a great number of computing jobs with simple descriptions using a script language. It also allows you to do parameter search, parameter sweep, and parametric study. GJES(GRID Job Execution System) - Virtually manages dispersed computing resources (computing servers) and equalizes loads on servers. Not only that, if a computer downs during job execution, automatically determines the downed event and executes the relevant job another computer. - Realizing unified job execution in a computing environment with heterogeneous operating systems

200 /11

Fujitsu plans - To create interface compliant to Globus tool kit

Future Plan Computing GRID Data GRID ( ) Grid Portal Server Grid SearchSearch AnalizeAnalize User Data Grid Computing Grid Realize GRID from search to Analize