Current Status of Japanese Virtual Observatory

Slides:



Advertisements
Similar presentations
IVOA Interoperablity meeting KyotoMasahiro Tanaka (NAOJ) 1 JVO use of Globus Toolkit Masahiro Tanaka (NAOJ)
Advertisements

May 9, 2007Astro-RG, OGF201 Japanese Virtual Observatory and NaReGi Masatoshi Ohishi / NAOJ, Sokendai & NII /, &
JAXA GRID Activity JAXA/EORC Kengo Aizawa KEIO UNIVHiromichi Fukui.
Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.
Construction of the Japanese Virtual Observatory (JVO) Abstract : The National Astronomical Observatory of Japan (NAOJ) has been operating several large.
Design and Implementation of JVO SkyNode Yuji SHIRASAKI National Astronomical Observatory of Japan IVOA Small Projects Meeting, 2004 Oct 1.
Distributed Heterogeneous Data Warehouse For Grid Analysis
SXDS database and Japanese Virtual Observatory Yuji Shirasaki and JVO collaborations National Astronomical Observatory of Japan.
Jun Peng Stanford University – Department of Civil and Environmental Engineering Nov 17, 2000 DISSERTATION PROPOSAL A Software Framework for Collaborative.
Web-Enabling the Warehouse Chapter 16. Benefits of Web-Enabling a Data Warehouse Better-informed decision making Lower costs of deployment and management.
Web-based Software Development - An introduction.
A long tradition. e-science, Data Centres, and the Virtual Observatory why is e-science important ? what is the structure of the VO ? what then must we.
Zhonghua Qu and Ovidiu Daescu December 24, 2009 University of Texas at Dallas.
The Japanese Virtual Observatory (JVO) Yuji Shirasaki National Astronomical Observatory of Japan.
NAREGI WP4 (Data Grid Environment) Hideo Matsuda Osaka University.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
Astronomical Data Query Language Simple Query Protocol for the Virtual Observatory Naoki Yasuda 1, William O'Mullane 2, Tamas Budavari 2, Vivek Haridas.
Yuji SHIRASAKI National Astronomical Observatory of Japan Current Status of Japanese Virtual Observatory (JVO)
WSRF Supported Data Access Service (VO-DAS)‏ Chao Liu, Haijun Tian, Dan Gao, Yang Yang, Yong Lu China-VO National Astronomical Observatories, CAS, China.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.
Prototype system of the Japanese Virtual Observatory The Japanese Virtual Observatory (JVO) aims at providing easy access to federated astronomical databases.
An-Najah National University Software Graduation Project Remote Control System Working Group: Mohammad Yasin Mohammad Yasin Ala’a Deeb Ala’a DeebInstructor:
JVO JVO Portal Japanese Virtual Observatory (JVO) Prototype 2 Masahiro Tanaka, Yuji Shirasaki, Satoshi Honda, Yoshihiko Mizumoto, Masatoshi Ohishi (NAOJ),
May 10, 2006IVOA-GGF Astro-RG WS1 Welcome and Workshop Goal Masatoshi Ohishi / NAOJ & Sokendai Chairman, IVOA 大石雅寿 / 国立天文台 & 総合研究大学院大学
Archivists' Toolkit - CRADLE Presentation, 10 Feb The Archivists’ Toolkit CRADLE Presentation 10 Feb
Archivists' Toolkit - CDL Presentation, October 17, 2005 The Archivists’ Toolkit Lee Mandell Brad Westbrook.
P Structured Query Language for Virtual Observatory Yuji Shirasaki National Astronomical Observatory of Japan, and Masahiro Tanaka (NAOJ), Satoshi.
IVOA Interoperalibity JVO Query Language Naoki Yasuda (NAOJ/Japanese VO)
IODE Ocean Data Portal - ODP  The objective of the IODE Ocean Data Portal (ODP) is to facilitate and promote the exchange and dissemination of marine.
February 20, 2006JVO-AstroGrid Collaboration1 Current Status of JVO, and Goal of Collaboration between JVO and AstroGrid Masatoshi Ohishi / NAOJ & Sokendai.
Grid Security: Authentication Most Grids rely on a Public Key Infrastructure system for issuing credentials. Users are issued long term public and private.
Oct. 1, 2004IVOA Small Projects Meeting1 Development of JVO prototype system and its application to Astrophysics Portal System : M. Tanaka Data Service.
Japanese Virtual Observatory Project Abstract : The National Astronomical Observatory of Japan (NAOJ) started the Japanese Virtual Observatory (JVO) project.
SEE-GRID-SCI The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no.
2003/03/25天文学会春季年会 JVO の研究開発 (2) 全体進捗 大石雅寿 国立天文台 & 総研大
EGEE User Forum Data Management session Development of gLite Web Service Based Security Components for the ATLAS Metadata Interface Thomas Doherty GridPP.
Japanese Virtual Observatory Project Abstract : The National Astronomical Observatory of Japan (NAOJ) started the Japanese Virtual Observatory (JVO) project.
2003/11/27CVO Meeting in Beijing Current Status of Japanese Virtual Observatory Masatoshi Ohishi National Astronomical Observatory of Japan and Sokendai.
German Astrophysical Virtual Observatory Overview and Results So Far W. Voges, G. Lemson, H.-M. Adorf.
Some comments on Portals and Grid Computing Environments PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics,
Japanese Virtual Observatory (JVO) National Astronomical Observatory of Japan (NAOJ) Contact Address: Demo 1: GRID-based database federation.
Chapter 5 Introduction To Form Builder. Lesson A Objectives  Display Forms Builder forms in a Web browser  Use a data block form to view, insert, update,
JVO portal service Yuji Shirasaki National Astronomical Observatory of Japan.
Data Manipulation with Globus Toolkit Ivan Ivanovski TU München,
Three color composite image Color-Color diagram SED SED relative to R-band Download FITS Searching for a cosmic string through the gravitational lens effect:
IVOA Small Projects Meeting Application to the science S. Honda, Y. Shirasaki, M. Tanaka and JVO team National Astronomical Observatory of Japan.
2004/03/08Japan-France Grid Computing Workshop in Paris Japanese Virtual Observatory Project Masatoshi Ohishi National Astronomical Observatory of Japan.
A Presentation Presentation On JSP On JSP & Online Shopping Cart Online Shopping Cart.
April 29, 2005ISGC Taipei1 Recent Progress of the Japanese Virtual Observatory Project Masatoshi Ohishi / NAOJ 大石雅寿 / 国立天文台
SmartCode Brad Argue INLS /19/2001.
Applications Active Web Documents Active Web Documents.
Web-based Software Development - An introduction
GPIR GridPort Information Repository
MSSL Astrogrid Workshop
Peter Kacsuk – Sipos Gergely MTA SZTAKI
Outline SOAP and Web Services in relation to Distributed Objects
Web Portal Project.
Outline SOAP and Web Services in relation to Distributed Objects
PHP / MySQL Introduction
University of Technology
May 14 (Wed) Masatoshi Ohishi (Japan)
2018/11/23 An Application for the Data Grid : A Prototype of the Japanese Virtual Observatory System Masatoshi Ohishi National Astronomical Observatory.
2018/11/27 An Application for the Data Grid : A Prototype of the Japanese Virtual Observatory System Masatoshi Ohishi National Astronomical Observatory.
Progress Report of VOQL WG
Serpil TOK, Zeki BAYRAM. Eastern MediterraneanUniversity Famagusta
Google Sky.
Data Management Components for a Research Data Archive
Grid Computing Software Interface
Presentation transcript:

Current Status of Japanese Virtual Observatory 2018/11/23 Current Status of Japanese Virtual Observatory Masatoshi Ohishi National Astronomical Observatory of Japan       and Sokendai masatoshi.ohishi@nao.ac.jp 2003/11/27 CVO Meeting in Beijing

Collaborators Fujitsu Ltd. NAOJ K. Monzen Y. Mizumoto Ken Miura K. Kawarai Y. Ishihara H. Yanaka Y. Yamaguchi M. Ishida N. Saito K. Abe J. Tsutsumi H. Nakamoto Y. Morita NAOJ Y. Mizumoto N. Yasuda Y. Shirasaki M. Tanaka S. Honda Ochanomizu U. Y. Masunaga Osaka U. S. Shimojo & his team Titech S. Matsuoka & his team Ken Miura 2003/11/27 CVO Meeting in Beijing

Large Telescopes of NAOJ Subaru (optical & Infrared) @ Hawaii Nobeyama (Radio) single dish : f 45-m interferometer : 6 x f 10-m solar interferometer : 84 x f 75-cm HALCA – in orbit f 8-m 2003/11/27 CVO Meeting in Beijing

Subaru 2003/11/27 CVO Meeting in Beijing

Subaru Telescope 2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing

45m Radio Telescope 2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing

VSOP 2003/11/27 CVO Meeting in Beijing

JAXA is going to join JVO !! Infrared Satellite by JAXA Launch : 2005 ? 2003/11/27 CVO Meeting in Beijing

Data Productivity Nobeyama Radio :~1TB/yr Subaru@Hawaii :~20TB/yr ALMA(planned)  :~PB/yr Flood of excellent data (survey data) Digitized & Archived 2003/11/27 CVO Meeting in Beijing

10 Gbps 2003/11/27 CVO Meeting in Beijing

HPC + High Speed NETWORK +GRID Technology → it is possible to federate multi-wavelength-DBs and computing facilities for data analyses → Virtual Observatory 2003/11/27 CVO Meeting in Beijing

JVO prototype : Ver. 1 Establish seamless access to federated DBs by using the Grid technology Define JVO Query Language, and implement its parser Test environment is in a closed subnet 2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing

Refreshed automatically in every 15 seconds Monitor Window: Refreshed automatically in every 15 seconds Dynamical creation of a workflow 2003/11/27 CVO Meeting in Beijing

Sample Outputs 2003/11/27 CVO Meeting in Beijing

UDDI Registory / Servlet Engine JVO Prototype System Architecture Redhat Linux 2Mass DataBase Redhat Linux JVO Service JVO Controller Solaris8 (Possible at Linux) Gb Ether GSDL UDDI Registory / Servlet Engine UDDI Maintenance Tool Java2SE 1.4 Globus Toolkit (Server) Java2SE 1.4 Hypertext contents Apache Tomcat Maintenance Tool JVO Service (X match) soapuddi UDDI4J Netscape Communicator Java2 Plugin JVO Client (applet) Apache SOAP ① Applet DownLoad JDBC register update delete JVO Service (select) SQL library ③ Find Service ② Request Observation JVO Controller DB Service Image Data (copy) DB Service Java2SE 1.4 PostgreSQL (DBMS) PostgreSQL JVO Server Monitoring UDDI4J UDDI Data SUBARU SupCam z-band Image Data 2Mass DB get & parse GSDL pre condition check execute Service post condition check Redhat Linux SUBARU DataBase ⑦ issue SQL JVO Service ⑤ execute Service ③ get GSDL GSDL (copy) ④’ Get GSDL by GridFTP GSDL ⑤’ execute Service Check status Globus Toolkit (Client) Internet Globus Toolkit (Server) ⑥’ spawn Service ④ Get GSDL by GridFTP JVO Service (image) AVO JVO Service JVO Service (X match) SUBARU DataBase Globus Toolkit Globus Toolkit (Server) GridFTP GSDL JVO Service (select) SQL library NVO ⑥ spawn Service Globus Toolkit JVO Service (count) SQL library JVO Service (select) SQL library JVO Service (image) JVO Service (X match) JVO Service (count) SQL library ⑦’ issue SQL Image Data Image Data (copy) Copy imate data by GridFTP Image Data (copy) DB Service ORACLE 2003/11/27 CVO Meeting in Beijing The following free software is included by globus toolkit. OpenSSL (Secure Socket Library) OpenLDAP (LDAP Server) wu-ftpd (ftp server) Copy imate data by GridFTP SUBARU SupCam i-band Free Software In House Software Image Data Commercial Software Function

JVO Query Language 2003/11/27 CVO Meeting in Beijing

SkyQL & ADQL JVOQL has been adopted as one of the basis for SkyQL (standard query language in IVOA) XML description of SkyQL = ADQL Astronomical Data Query Language 2003/11/27 CVO Meeting in Beijing

Serious Issue Q) GT2 has too large overhead before starting actual jobs !! They are several seconds. A) For batch jobs, such as database accesses, the overhead could be negligible. However users can never wait for several second for interactive data analyses. Request) Overheads must be < 0.1 seconds excluding the network delays. 2003/11/27 CVO Meeting in Beijing

Partially solved !! Introduction of NMI -- authentication module : binary Analysis of the source codes of GTK2 -- polling interval : 30 sec -> 3 sec Elapsed time was halved ! But …, still very slow. 2003/11/27 CVO Meeting in Beijing

Improvement of U.I. Need to know about SQL to use JVOQL → An editor to create JVOQL from GUI Edittable on screen → more flexibility for advanced users 2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing

Search for Gravitational Lenses caused (?) by Cosmic Strings SXDS data observed by Subaru Query results were obtained less than 5 min., displaying SEDs It has been proven that VO can accelerate researches. 2003/11/27 CVO Meeting in Beijing

Cosmic Strings? Theoretical origin of the elementary particles which existed (?) at t ~ 10-35 s,T ~ 1015 GeV Not a POINT, has its SIZE Width < 10-22m, Length ~ size of the Universe. Mass of 10km string ~ the Earth 2003/11/27 CVO Meeting in Beijing

Gravitational Lens A phenomenon that the space-time is distorted by a huge mass object, and multiple images of another object behind the massive object are observed around it. 2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing

Development of Prototype2 For preparation of Operational System User registration : single sign-on User area over Firewall : GSI-SFS Adoption of OGSA Implementation of several analysis tools 2003/11/27 CVO Meeting in Beijing

2003/11/27 CVO Meeting in Beijing データベースサーバ1 分散情報管理 データベースサーバ2 データ検索処理 ORACLE Linux クロスマッチ処理 データ検索処理 データ数count PostgreSQL 画像管理   DB WSDL データベースサーバ2 SUBARU SupCam i-band Globus Toolkit V3 OGSA-DAI 解析処理 検索依頼 GWSDL セキュリティ 管理 OGSA -DAI Gb Ether データ数count 画像管理   DB Gb Ether Registry Globus Toolkit V3 OGSA-DAI 2MASS 利用者操作画面 資源管理 GT3 利用者 データ Local disk GSI-SFS DAISGR Globus Tk Linux ポータルサーバ 統合検索指示 統 合 検 索 実 行 制 御 機 能  ポータルサーブレット 登録情報検索機能 利用者管理機能 GT3 利用者 データ Local disk GSI-SFS 状況表示 条件指定画面 分散処理制御 解析サーバ 統合検索状況モニター ・DAISGR検索 ・GWDSL解釈 ・準備、実行、結果確認 解析処理 Linux データ保存 検索状況モニター画面 Web Server データベースサーバ N 統合検索結果表示機能 Globus Toolkit V3 データ検索処理 PostgreSQL WSDL スケジュール作成 Ⅰ スケジュール 作成 Ⅱ 画像管理   DB 解析指示 FW データ数count クロスマッチ処理 Globus Toolkit V3 検索結果画面 JVO-QL翻訳サーバ Globus Toolkit V3 OGSA-DAI SUBARU SupCam z-band Webブラウザ Solaris8 GT3 利用者 データ 結果サーバ GSI-SFS 解析処理 Windows, Solaris8, Linux GT3 利用者 データ Local disk GSI-SFS 検索依頼 2003/11/27 CVO Meeting in Beijing Solaris8

Road Map 2003 Prototype 2 → development of components for operation 2004 start to develop operational system  late 2004 ?? trial use  2005-2006 trial use & upgrade 2007 prepare for partial operation of ALMA 2003/11/27 CVO Meeting in Beijing

ALMA in Chile (sub-mm wave) 2003/11/27 CVO Meeting in Beijing