Solar-B Global DataGrid 21 st September 2006 UK e-Science All Hands Meeting 2006 RALMSSL Tim Folkes Elizabeth Auden Jens Jensen Paul Lamb Matthew WildMatthew.

Slides:

Advertisements

Similar presentations

LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (

Advertisements

Remote Visualisation System (RVS) By: Anil Chandra.

The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.

30-31 Jan 2003J G Jensen, RAL/WP5 Storage Elephant Grid Access to Mass Storage.

Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities.

The Science of Solar B Transient phenomena – this aim covers the wide ranges of explosive phenomena observed on the Sun – from small scale flaring in the.

Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.

Peter Berrisford RAL – Data Management Group SRB Services.

Hinode/EIS Data Products and Archive Access Jian Sun (MSSL)

COSPAR E July 22, Paris revised for Nobeyama Symposium 2004 October 29, Kiyosato Takeo Kosugi (ISAS/JAXA, Japan)

Multi-Wavelength Studies of Flare Activities with Solar-B ASAI Ayumi Kwasan Observatory, Kyoto University Solar-B Science February 4, 2003.

1 CEOS/WGISS20 – Kyiv – September 13, 2005 Paul Kopp SIPAD New Generation: Dominique Heulet CNES 18, Avenue E.Belin Toulouse Cedex 9 France

ESDO Algorithms, Visualization and Data Access Elizabeth Auden 27 September 2006 HELAS Workshop Nice.

19-20 March 2003 IVOA Registry Workgroup LeSc Astrogrid Registry: Early Designs Elizabeth Auden Astrogrid Registry Workgroup Leader IVOA Registry Workgroup.

Bookshelf.EXE - BX A dynamic version of Bookshelf –Automatic submission of algorithm implementations, data and benchmarks into database Distributed computing.

000509EISPDR_SciInvGIs.1 EIS Science Goals: The First Three Months…. Louise Harra Mullard Space Science Laboratory University College London.

Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.

SWIM WEB PORTAL by Dipti Aswath SWIM Meeting ORNL Oct 15-17, 2007.

Application of GRID technologies for satellite data analysis Stepan G. Antushev, Andrey V. Golik and Vitaly K. Fischenko 2007.

XRT X-ray Observations of Solar Magnetic Reconnection Sites

Magnetic Helicity • Magnetic helicity measures

Magnetic Field Measurements from Solar-B Information shown here is from Solar-B team (including Drs Ichimoto, Kosugi, Shibata, Tarbell, and Tsuneta)

Leicester Database & Archive Service J. D. Law-Green, J. P. Osborne, R. S. Warwick X-Ray & Observational Astronomy Group, University of Leicester What.

25 September 2007eSDO and the VO, ADASS 2007Elizabeth Auden Accessing eSDO Solar Image Processing and Visualisation through AstroGrid Elizabeth Auden ADASS.

11 Dec 2000F Harris Datagrid Testbed meeting at Milan 1 LHCb ‘use-case’ - distributed MC production

Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.

990901EIS_RR_Science.1 Science Investigation Goals and Instrument Requirements Dr. George A. Doschek EIS US Principal Investigator Naval Research Laboratory.

Edinburgh University Experimental Particle Physics Alasdair Earl PPARC eScience Summer School September 2002.

October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.

Solar-B Mission Preparation Len Culhane – UK EIS Principal Investigator Louise Harra – UK Project Scientist David Williams – UK EIS Chief Observer Mullard.

ESDO Algorithms, Visualization and Data Access Elizabeth Auden 21 September 2006 AHM 2006 Nottingham.

EIS - MSSL/NRL EUV Imaging Spectrometer SOT - ISAS/NAOJ Solar Optical Telescope XRT - SAO/ISAS X-ray Telescope FPP - Lockheed/NAOJ Focal Plane Package.

By: Kiana and Meagan. Purpose  To measure solar magnetic fields  To understand how energy generated by magnetic-field changes in the lower solar atmosphere.

MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.

Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.

ESP workshop, Sept 2003 the Earth System Grid data portal presented by Luca Cinquini (NCAR/SCD/VETS) Acknowledgments: ESG.

QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

8th November 2002Tim Adye1 BaBar Grid Tim Adye Particle Physics Department Rutherford Appleton Laboratory PP Grid Team Coseners House 8 th November 2002.

LHCb and DataGRID - the workplan for 2001 Eric van Herwijnen Wednesday, 28 march 2001.

ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.

Science - Coronal heating - Coronal structure / dynamics - Elementary processes in Magnetic Reconnection Mission instruments - Optical Telescope / Vector.

Solar-B Global DataGrid Update for AstroGrid SAG 10 RALMSSL Tim Folkes Elizabeth Auden Jens Jensen Paul Lamb Matthew WildMatthew Whillock Len Culhane Elizabeth.

Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.

Grid Architecture William E. Johnston Lawrence Berkeley National Lab and NASA Ames Research Center (These slides are available at grid.lbl.gov/~wej/Grids)

3-Nov-00D.P.Kelsey, HEPiX, JLAB1 Certificates for DataGRID David Kelsey CLRC/RAL, UK

Solar and STP science with AstroGrid Silvia Dalla School of Physics & Astronomy, University of Manchester A PPARC funded project.

Solar and space physics datasets within a Virtual Observatory: the AstroGrid experience Silvia Dalla * and Nicholas A Walton  * School of Physics & Astronomy,

MTA SZTAKI Hungarian Academy of Sciences Introduction to Grid portals Gergely Sipos

Joint Planning of SOT/XRT/EIS Observations Outline of 90 Day Initial Observing Plans T. Shimizu, L Culhane.

Michael Doherty RAL UK e-Science AHM 2-4 September 2003 SRB in Action.

Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.

UCL DEPT. OF SPACE & CLIMATE PHYSICS SOLAR & STELLAR PHYSICS GROUP Atomic Data for Astrophysics VOTADA VO Tools and Atomic Data for Astrophysics Giulio.

The SOLAR-B Science Center in Japan International Workshop: The 6 th SOLAR-B Science Meeting Kyoto International Community House Masumi.

AstroGrid NAM 2001 Andy Lawrence Cambridge NAM 2001 Andy Lawrence Cambridge Belfast Cambridge Edinburgh Jodrell Leicester MSSL.

Algorithm Preparation and Data Availability 1. Mullard Space Science Laboratory, University College London. 2. Physics and Astronomy Department, University.

Solar and STP with AstroGrid Silvia Dalla School of Physics & Astronomy, University of Manchester A PPARC funded project.

1 SUZAKU HUG 12-13April, 2006 Suzaku archive Lorella Angelini/HEASARC.

Joint Planning of SOT/XRT/EIS Observations Outline of 90 Day Initial Observing Plans T. Shimizu, L Culhane.

ESDO: UK Access to the Solar Dynamics Observatory 1. Mullard Space Science Laboratory, University College London. Holmbury St. Mary, Dorking, Surrey SM2.

EIS Operation Planning H. Hara (NAOJ) 2007 Dec 10.

Spectroscopic Observations with SolarB-EIS Helen E. Mason DAMTP, Centre for Mathematical Sciences Giulio Del Zanna, MSSL.

E.C. Auden1, J.L. Culhane1, Y. P. Elsworth2, A. Fludra3, M. Thompson4

UK GridPP Tier-1/A Centre at CLRC

ISAS Solar Physics Yohkoh (1991- ) Hionotori ( )

A Web-Based Data Grid Chip Watson, Ian Bird, Jie Chen,

Data Management Components for a Research Data Archive

Slit and Slot Interchange

Overview of Computer system

Presentation transcript:

Solar-B Global DataGrid 21 st September 2006 UK e-Science All Hands Meeting 2006 RALMSSL Tim Folkes Elizabeth Auden Jens Jensen Paul Lamb Matthew WildMatthew Whillock Len Culhane Elizabeth Auden21 September 2006e-Science AHM 2006

Overview The Solar-B mission will be launched tomorrow, Friday 22 September 2006, from the Uchinoura Space Center in Japan. Mission duration: 3 years nominal, +10 years lifetime expected Scientists and software engineers at MSSL and RAL have designed the Solar-B global data grid to provide mission data to solar research communities in the UK and abroad. Solar-B science objectives Data transfer and processing Data storage User access to data Data security Elizabeth Auden21 September 2006e-Science AHM 2006

Solar-B Mission Overview Elizabeth Auden21 September 2006e-Science AHM 2006 EIS - MSSL/NRL EUV Imaging Spectrometer SOT - ISAS/NAOJ Solar Optical Telescope XRT - SAO/ISAS X-ray Telescope FPP - Lockheed/NAOJ Focal Plane Package

Solar-B Science Elizabeth Auden21 September 2006e-Science AHM 2006 Key Science Questions 1.How is the solar corona heated? Are active regions heated in a different way to the quiet Sun? 2.What causes solar flares and coronal mass ejections? 3.How is energy transferred in the quiet Sun? How are solar magnetic fields affected? Solar events like flares and coronal mass ejections can have a profound affect on the Earth’s magnetosphere. A geomagnetic storm in 1989 triggered a blackout of the Hydro Quebec power system.

Transfer of Solar-B Data to ADS Solar-B data transferred via GridFTP from ISAS and MSSL to an NFS mounted disk at ADS. Elizabeth Auden21 September 2006e-Science AHM 2006 Data Migration Facility Manages NFS area Produced by SGI User transparent file migration Licence for 500Tbytes storage Backend: STK SL8500 –10,000 tape slots –STK 9940B drives (200GB 30MB/sec) –STK T10000 drives (500GB 120MB/sec) ADS RAL ISAS Japan MSSL processing Satellite data LL data HL data HL LL

ADS Data Storage Elizabeth Auden21 September 2006e-Science AHM 2006 DMF STK SL8500 robot The DMF monitors the Solar-B NFS area and removes disk copies of data to keep free space at required level. Small files have primary copy on disk. (Speeds up read/writes.) ADS Interface NFS Tape Firesafe Larger files have primary copy on tape. All files have secondary copy on tape. These are periodically moved to the firesafe. Hardware Based on Altix 350 Numaflex architecture 40 Tbytes raid disk Network Local server: 1Gbit network Part of 10Gbit back-bone Coming soon: SJ5 10Gbit link

User Access to Solar-B Data Elizabeth Auden21 September 2006e-Science AHM 2006 Instrument Data SOT: Solar Optical Telescope Magnetic field images Optical images Magnetic field velocity measurements XRT: X-Ray Telescope X-ray coronal images EIS: EUV Imaging Spectrometer Coronal line strengths and profiles Plasma diagnostics DMF Users access data as URLs through the AstroGrid workbench, STAP web services, browsers, or wget. Files via URLs and DMF system at RAL’s UKSSDC

Accessing Solar-B data through AstroGrid Elizabeth Auden21 September 2006e-Science AHM 2006 MySQL AstroGrid DSA FITS keywords, Data URLs DSA: DataSet Access ADQL to SQL AstroGrid STAP service INPUT: ADQL queries STAP: Simple Time Access Protocol OUTPUT: FITS keywords, URLs INPUT: start / stop dates OUTPUT: URLs AstroGrid workbench AstroGrid HelioScope, STAP enabled tools ADS Web-visible file system AstroGRID PLASTIC tools, Solar-B website, wget, browser INPUT: URLs OUTPUT: Solar-B files

AstroGrid User Components Elizabeth Auden21 September 2006e-Science AHM 2006 HelioScope: solar data results from STAP services. AstroGrid workbench: STAP service access through HelioScope, DSA access through the Task Launcher. Task Launcher: locate DSAs, submit ADQL queries Workbench: STAP: Simple Time Access Protocol DSA: DataSet Access ADQL: Astronomical Data Query Language

Mission Data Flow Elizabeth Auden21 September 2006e-Science AHM 2006

Solar-B Studies / Observation Planning Elizabeth Auden21 September 2006e-Science AHM 2006 Study Proposal –All study proposals are tested on the Flight Spare model before acceptance into the master Study Database –The Study Database contains information about the science (target, pointing, rationale) as well as meta-data (author, joint ops id, etc) Database Distribution –This is done using the automatic Solarsoft update mechanism through Goddard Space Flight Centre Daily Planning –Performed at ISAS using the Study Database –Studies are entered into the Timeline (As-planned) Database

FITS File Reformatting Elizabeth Auden21 September 2006e-Science AHM 2006 Fits Reformatting (1) –Level-0 performed at ISAS –Accesses As-Planned Database –Populates As-Run Database –Transfers files to ADS and ISAS web system –Accesses other instruments’ Fits files and transfers them to the ADS for storage Fits Reformatting (2) –EIS Level-0 Fits files are used to generate Level-2 Fits files at MSSL –Updates the data catalogue –Transfers Level-2 Fits files back to the ADS

Data Security GridFTP used for file transfers Elizabeth Auden21 September 2006e-Science AHM 2006 ADS (RAL) ADS (RAL) ADS (RAL) GridFTP HTTP USERS GRIDFTP Secure transfers 3 rd party transfers Good performance Certificates from UK e- Science CA and AIST (Japan) STORAGE 10 TB per year 30 TB over nominal 3 year duration of mission GridFTP: Clients / hosts authenticate with X509 certificates

Conclusions Solar-B launch tomorrow. 22 September 2006 Data transferred via GridFTP to ADS at RAL ADS facility uses DMF to manage files on disk and tape Data searchable through AstroGrid DSA and STAP web service Solar community accesses data through AstroGrid workbench, HelioScope or a web browser LINKS: AstroGrid: Solar-B countdown: Elizabeth Auden21 September 2006e-Science AHM 2006 Questions?