Scientific Data Grid and e-Science

Slides:



Advertisements
Similar presentations
Grid Computing and Applications in China Kai Nan Computer Network Information Center (CNIC) Chinese Academy of Sciences (CAS) 17 Mar 2008.
Advertisements

Kejun Dong, Yihua CNIC Elaine Liu, Jurgen UCSD CNIC Tiled Display Wall and Astronomical Data Visualization PRAGMA11 Workshop Oct.16/17.
Kejun Dong, Kai Nan CNIC/CAS CNIC Resources And Activities Update Resources Working Group PRAGMA11 Workshop, Oct.16/17 Osaka, Japan.
Grid-enabled Research Activities in CAS Kai Nan Computer Network Information Center (CNIC) Chinese Academy of Sciences (CAS) Shanghai, 21 Feb 2006.
Development of China-VO ZHAO Yongheng NAOC, Beijing Nov
CNIC Grid/SDG CA Updates 2 nd APGrid PMA meeting, October 15, 2006 Morrise Xu NTARL, CNIC, China.
CNIC Grid CA/SDG CA Self Audit Kejun (Kevin) Dong Computer Network Information Center (CNIC) Chinese Academy of Sciences APGridPMA F2F.
Xingfu Wu Xingfu Wu and Valerie Taylor Department of Computer Science Texas A&M University iGrid 2005, Calit2, UCSD, Sep. 29,
International Workshop APAN 24, Current State of Grid Computing Researches and Applications in Vietnam Nguyen Thanh Thuy 1, Nguyen Kim Khanh 1,
CSC Grid Activities Arto Teräs HIP Research Seminar February 18th 2005.
Introduction to Scientific Data Grid Kai Nan Computer Network Information Center, CAS
Scientific Data Infrastructure in CAS Dr. Jianhui Scientific Data Center Computer Network Information Center Chinese Academy of Sciences.
Public-resource computing for CEPC Simulation Wenxiao Kan Computing Center/Institute of High Physics Energy Chinese Academic of Science CEPC2014 Scientific.
CNGI Applications in CSTNET QingHua Zhang CSTNET January 2007.
Open Science Grid For CI-Days Internet2: Fall Member Meeting, 2007 John McGee – OSG Engagement Manager Renaissance Computing Institute.
Computational Scientometrics Studying science by scientific means Dr. Katy Börner Cyberinfrastructure for Network Science Center, Director Information.
Scientific Data Grid on NGI Kai Nan Computer Network Information Center Chinese Academy of Sciences CANS 2004, Miami.
Open Science Grid For CI-Days Elizabeth City State University Jan-2008 John McGee – OSG Engagement Manager Manager, Cyberinfrastructure.
Scientific Database and Virtual Museums
An Introduction to Scientific Data Grid LUO Ze Computer Network Information Centre, Chinese Academy of Sciences.
1 Computing Challenges for the Square Kilometre Array Mathai Joseph & Harrick Vin Tata Research Development & Design Centre Pune, India CHEP Mumbai 16.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
1 GRID Based Federated Digital Library K. Maly, M. Zubair, V. Chilukamarri, and P. Kothari Department of Computer Science Old Dominion University February,
Scientific Data Grid & China-VO Kai Nan Computer Network Information Center Chinese Academy of Sciences November 27, 2003.
Introducing Virtualization via an OpenStack “Cloud” System to SUNY Orange Applied Technology Students SUNY Innovative Instruction Technology Grant Christopher.
ESFRI & e-Infrastructure Collaborations, EGEE’09 Krzysztof Wrona September 21 st, 2009 European XFEL.
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
1 Grid Activity Summary » Grid Testbed » CFD Application » Virtualization » Information Grid » Grid CA.
A Collaborative Research Environment for Avian Flu Research Luo Ze Computer Network Information Center, CAS
Construction of Computational Segment at TSU HEPI Erekle Magradze Zurab Modebadze.
December 10, 2003Slide 1 International Networking and Cyberinfrastructure Douglas Gatchell Program Director International Networking National Science Foundation,
Background Computer System Architectures Computer System Software.
Cyberinfrastructure Overview of Demos Townsville, AU 28 – 31 March 2006 CREON/GLEON.
European and Chinese Cooperation on Grid Virtual Laboratory: Exploring e-Science in CAS CNIC,CAS Jianjun Yu
DutchGrid KNMI KUN Delft Leiden VU ASTRON WCW Utrecht Telin Amsterdam Many organizations in the Netherlands are very active in Grid usage and development,
ChinaGrid: National Education and Research Infrastructure Hai Jin Huazhong University of Science and Technology
EGI-InSPIRE RI EGI Compute and Data Services for Open Access in H2020 Tiziana Ferrari Technical Director, EGI.eu
Grid Computing Activities in PKU Asso. Prof. CHEN Ping Prof. QIAN Sijin Asso. Prof. YU Huashan Peking University
EGI-InSPIRE RI An Introduction to European Grid Infrastructure (EGI) March An Introduction to the European Grid Infrastructure.
Grid Activities in the Philippines Rey Vincent P. Babilonia Advanced Science and Technology Institute Department of Science and Technology PHILIPPINES.
INTRODUCTION TO WEB HOSTING
Bob Jones EGEE Technical Director
Accessing the VI-SEEM infrastructure
Sun Gongxing, IHEP, Beijing
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING
Regional Operations Centres Core infrastructure Centres
Clouds , Grids and Clusters
Tools and Services Workshop
Joslynn Lee – Data Science Educator
GWE Core Grid Wizard Enterprise (
Information Collection and Presentation Enriched by Remote Sensor Data
StratusLab Final Periodic Review
NGIs – Turkish Case : TR-Grid
StratusLab Final Periodic Review
Christos Markou Institute of Nuclear Physics NCSR ‘Demokritos’
Design and realization of Payload Operation and Application system of China’s Space Station Wang HongFei 首页.
Joseph JaJa, Mike Smorul, and Sangchul Song
Grid Computing.
University of Technology
Objective Understand the concepts of modern operating systems by investigating the most popular operating system in the current and future market Provide.
CLUSTER COMPUTING.
MWCN`03 Singapore 28 October 2003
eGY Planning Meeting Boulder, February 2005
BUILDING A DIGITAL REPOSITORY FOR LEARNING RESOURCES
Google Sky.
Brokering as a Core Element of EarthCube’s Cyberinfrastructure
Objective Understand the concepts of modern operating systems by investigating the most popular operating system in the current and future market Provide.
What is a Grid? Grid - describes many different models
Computer Network Information Center, Chinese Academy of Sciences
Presentation transcript:

Scientific Data Grid and e-Science Kai NAN nankai@cnic.ac.cn Computer Network Information Center (CNIC) Chinese Academy of Sciences (CAS) 14 Dec 2006

Outline Background Scientific Data Grid (SDG) Prospect Scientific Database (SDB) CSTNet & GLORIAD Scientific Data Grid (SDG) Middleware Visualization Prospect CAS e-Science

Background Scientific Data Grid (SDG) Built upon the mass scientific data resources of the Scientific Database (SDB) of CAS Led by CNIC; dozens of institutes of CAS participated The vision of SDG is to take valuable data resources into full play by benefiting from advanced information technologies, in particular, the Grid technology.

Scientific Database (SDB) SDB is a long-term project since 1983, in which there are multi-disciplinary scientific data accumulated through the course of science activities in CAS. many institutes involved long-term, large-scale collaboration data from research, for research

SDB status 45 institutes across 16 cities 503 databases 16.6TB total volume

CSTNET Bandwidths Backbone 10G MAN link 1G WAN link 2.5G/155M Cover more than 20 provinces, 100 institutes, 1,000,000 end users 12 Subcenter in China Now CSTNET has twelve sub center distributed in China, like shanghai, wuhan, shenyang, xian, and other palce This picture shows us all of the nodes that CSTNET can reach in China.

CSTNET Internet Connections 622M 2.5G 155M

CSTNET in GLORIAD GLORIAD CSTNET Global Ring 10G 11 partners GLORIAD_CN Build HKOEP

SDG Supported by Chinese Academy of Sciences (CAS) 10th Five-year Informatization Program (2001-2005) 11th Five-year Informatization Program (2006-2010) Ministry of Science & Technology of China (MOST) 863 Program (2002-2005) 863 Program (2007-2010) National Science Foundation of China (NSFC) Network-based Science and Research Environment (aka. NSFC e-Science) (2004-2007)

SDG Milestones In 2000, the Scientific Database (SDB) project renewed fund by CAS 10th Five-year Program In March 2001, proposed “Scientific Data Grid” In October 2002, SDG joined the China National Grid (fund from MOST) In Nov 2003, SDG Middleware v1.0 released In July 2004, SDG got fund from NSFC In Sep 2004, SDG renewed fund from MOST In Oct 2004, DeepComp 6800 for SDG installed In Nov 2004, SDG Middleware v2.0 released In Aug 2005, SDG Middleware v2.1 released Now, we’re working for SDG in 11th Five-year Program 2006-2010

Scientific Database (SDB) & Scientific Data Grid (SDG) 45 institutes participated 503 databases 16.6 TB 236-CPU Superserver (1TF) 20TB Disk Array 50TB Tape Library VizWall & Access Grid

Requirements and SDG How to FIND the data I want from hundreds or thousands of databases How to ACCESS large-scale, distributed and heterogeneous scientific data uniformly and conveniently How to make sure all this goes always in a SECURE and proper way

SDG Software Architecture

Data Access Service (DAS) Uniform Access Interface (read-only) Rich metadata Easy publishing on web flexible configuration and extensibility

DAS modules Data Access Interface Virtual Database Physical Database MappingBuilder DataView

SDG Services Chinese Ancient Astronomical Phenomena Database Emperor’s name: Kang Xi Keyword: 日食(solar eclipse) 天象(astronomical phenomena) Chinese Ancient Astronomical Phenomena Database DataView Service

MappingBuilder & Dataview

SDG Today www.sdg.ac.cn

sdb6800 Superserver 59 nodes /236 CPUs official service started in Apr. 2005 node usage 79.7% storage usage 87% (by Sep 2005)

SDG Storage System

Visualization System

portal.sdg.ac.cn

SDG CA SDG security infrastructure Accredited by ApGridPMA & IGTF The subordinate CA of CNIC Grid CA Accredited by ApGridPMA & IGTF SDG CA Repository http://ca.sdg.grid.cn/ CP/CPS, Introduction, Manual Type of certificates Person Certificate: E=morrise@cnic.ac.cn,CN=Morrise Xu,DC=SDG,DC=Grid,DC=CN Host Certificate: CN=sdg6800.sdg.ac.cn, DC=SDG,DC=Grid,DC=CN Service Certificate: CN=DAS/sdg6800.sdg.ac.cn, DC=SDG, DC=Grid, DC=CN

CA Software Suite SDG CA V3.1 Certificate Utility V2.0 Based on the OpenCA V0.9.2.5 Localization and Improvement Certificate Utility V2.0 i18n

Applications High Energy Physics Astronomy Biology Geosciences … Cosmic Ray Data Processing (YBJ) Astronomy China Virtual Observatory Biology Avian-flu Comprehensive Information Platform Geosciences …

Tiled Display Wall @ CNIC A Tiled Display Wall (TDW) is a large-scale high-resolution display CNIC’s TDW uses a five by four grid of LCDs and 21-nodes cluster running Rocks (with CentOS installed). 1+20 cluster; 1Gigabit network for each node. A Tiled Display Wall is a large-scale high-resolution display. It is composed of multiple projectors or LCDs whose images are projected side-by-side to create a tiled mosaic of imagery. For some situations, an application runs on a single machine and a software or hardware distribution system divides up the screen image and distributes it out to the individual projectors, so that the resulting image appears as a single cohesive display. In others, a copy of the application runs on each machine, generating the imagery for the corresponding tile. CNIC’s Tiled Display Wall is setup at 2005, with the support of Chinese Academy of Sciences. We use a five by four grid of LCDs and 21-nodes cluster. The Rocks is running on the cluster. As you may know, Rocks clusters soft is developed by SDSC rocks group and now is widely used in world-wide cluster and it is a convenient tool and software to setup a cluster. For the network, dual one gigabit network adaptors are equipped for each node and one Cisco switch is installed for the tiled display wall.

Visualization Software Package Rocks Viz Roll – SDSC DMX - Distributed Multihead X http://dmx.sourceforge.net/ SAGE - EVL/UIC For the visualization Software package, two visualization system is installed in CNIC tiled display wall. One is rocks viz roll which is developed by SDSC group. Mason Katz and other engineers are involved in the project. Another one is SAGE visualization software, which is developed by Electronic Visualization Lab, SAGE group in University of Illinois at Chicago. As I know, KISTI also has setup a tiled display wall with SAGE visualization environment.

PyMol This is a snapshot of Pymol application to show the avian flu molecule.

xnview This is a snapshot of vnview image show application.

JuxtaView This is a snapshot of JuxtaView. It is a image display tool which can piece lots of images together and give a large-resolution image showing. We also can use the client tool to control the image showing, including zoom in, zoom out, move left, right, down and up. In the snapshot, the total image size is more than six hundred Megabytes with more than 10000 by 8000 pixels. and just a part of image is showed on the tiled display wall. The most important one for JuxtaView is that JuxtaView uses MPI parallel environment to accomplish distributed and parallel rendering and showing.

Collaborations PRAGMA EUChinaGrid ApGrid PMA / IGTF … www.pragma-grid.net EUChinaGrid www.euchinagrid.org Interconnection and Interoperability of Grids between Europe & China ApGrid PMA / IGTF …

Prospect – CAS e-Science CAS Informatization Program (2001-2005) emphasis on Infrastructure Upgrade

Resources Lenovo 6800 Superserver Storage VizWall Scientific Data (SDB) Science Digital Lib (CSDL)

CAS e-Science Initiative 2006-2010 e-Science would be applications-driven focus on implementation of e-Science Virtual Labs, the way for scientists to use infrastructure may need refactoring

e-Science Virtual Labs special meanings in the e-Science context the key position in our e-Science framework the core component to make e-Science a reality

vLabs Requirements Infrastructure may be (almost) ready, but e-Science is not yet. so many existing resources in place, but just a few could be brought into full play even now, with an advanced infrastructure ready. bottleneck may be the gap between products by computer experts and end users of domain scientists much more effort than expected to bridge this gap Virtual Lab is proposed to be a basic unit of research activity in the e-Science environment the right user interface between scientists and their e-Science environment

Conclusion Remarks Serving researchers in CAS with cyberinfrastructure is CNIC’s mission. We have been building and running grid environments to support research activities better. SDG and vLabs will play important roles in the CAS Informatization Program 2006-2010 in which e-Science is the goal. We look forward to further development and collaboration.

Thank you!