Page 1JSOC Overview August 2007 HMI Status HMI is virtually done. –Virtually  similar to but not in fact –Front window issue resolved, flight window now.

Slides:



Advertisements
Similar presentations
HMI Data Analysis Software Plan for Phase-D. JSOC - HMI Pipeline HMI Data Analysis Pipeline Doppler Velocity Heliographic Doppler velocity maps Tracked.
Advertisements

ESDO Algorithms, Visualization and Data Access Elizabeth Auden 27 September 2006 HELAS Workshop Nice.
Online School Registration System Solomon Ng Pei-Yu Wang Evan Chiu Curtis Wong.
Page 1JSOC for SDO MOR October 2007 JSOC Status Review.
Rasmus Munk Larsen / Pipeline Processing 1 JSOC Pipeline Processing Overview Rasmus Munk Larsen, Stanford University
HMI - Page 1HMI Team Meeting – Jan 26, 2005 JSOC Implementation HMI Team Meeting 26 Jan 2005 Jim Aloise System Programmer
Page 1JSOC Peer Review – AIA Science Center – March 17, 2005 AIA Science Data Processing Infrastructure Neal Hurlburt AIA Data Scientist
Page 1HMI/AIA Joint Sciemce Meeting – AIA Science Center – Feb 2006 AIA Visualization Center N. Hurlburt.
200Mm Time-Distance Pipeline Status 11/5/ Mm Time-Distance Lead – Tom Duvall Lead – Tom Duvall Task – From Dopplergrams generate deep-focus synoptic.
JSOC Overview - 1HMI/AIA Team Meeting – Feb 2006 JSOC Summary- Phil Scherrer Pipeline Processing - Rasmus Larsen Data Access - Rick Bogart Data Visualization.
Rasmus Munk Larsen / Pipeline Processing 1HMI Science Team Meeting – January, 2005 JSOC Pipeline Processing Environment Rasmus Munk Larsen, Stanford University.
Page 1LWS Teams Day JSOC Overview HMI Data Products: Plan and Status.
HMI – Synoptic Data Sets HMI Team Meeting Jan. 26, 2005 Stanford, CA.
HMI/AIA Science Team Meeting, HMI Science Goals Alexander Kosovichev & HMI Team.
The Solar Group is ~30 people working to better understand the Sun and the sources of its variability. There are several projects including one on the.
Page 1JSOC for SDO MOR October 2007 CARL (6)Level-0 Processing (b) HK Processing i. Hk Processing on HSB ii. RT HK Extraction Iii Dayfile HK Processing.
Time-Distance Pipeline for the Upper 30Mm Convection Zone Status 11/5/2007.
HMI Science Analysis. Primary Goal: Origins of Solar Variability The primary goal of the Helioseismic and Magnetic Imager (HMI) investigation is to study.
1.B – Solar Dynamo 1.C – Global Circulation 1.D – Irradiance Sources 1.H – Far-side Imaging 1.F – Solar Subsurface Weather 1.E – Coronal Magnetic Field.
Investigation Overview, Scherrer, Page 1SDO Science Writers Workshop – 16 Dec 2009 HMI Investigation Overview Philip Scherrer HMI Principal Investigator.
Page 1HMI Team Meeting – January 26, 2005 HMI Mission Operations Rock Bush HMI Stanford Program Manager Stanford University
Page 1JSOC Review – 17 March 2005 DRMS Core System Karen Tian
HMI Science Analysis Plan Magnetic ShearTachocline Differential Rotation Meridional Circulation Near-Surface Shear LayerActivity ComplexesActive Regions.
25 September 2007eSDO and the VO, ADASS 2007Elizabeth Auden Accessing eSDO Solar Image Processing and Visualisation through AstroGrid Elizabeth Auden ADASS.
HMI Magnetic Products & Pipeline
Advanced Technology Center 1 HMI Rasmus Larsen / Processing Modules Stanford University HMI Team Meeting – May 2003 Processing Module Development Rasmus.
HMI Science Objectives Convection-zone dynamics and the solar dynamo  Structure and dynamics of the tachocline  Variations in differential rotation 
Science Data Products – HMI Magnetic Field Images Pipeline 45-second Magnetic line-of-sight velocity on full disk Continuum intensity on full disk Vlos.
Upcoming Enhancements to the HST Archive Mark Kyprianou Operations and Engineering Division Data System Branch.
ESDO Algorithms, Visualization and Data Access Elizabeth Auden 21 September 2006 AHM 2006 Nottingham.
An Introduction to Helioseismology (Local) 2008 Solar Physics Summer School June 16-20, Sacramento Peak Observatory, Sunspot, NM.
GLAST LAT ProjectDOE/NASA Baseline-Preliminary Design Review, January 8, 2002 K.Young 1 LAT Data Processing Facility Automatically process Level 0 data.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
GONG data and pipelines: Present & future. Present data products 800x800 full-disk images, one per minute, continuous (0.87 average duty cycle) Observables:
AIA Core-Team Meeting April 2009 JSOC Stuff Phil Scherrer.
Page 1LWS Teams Day JSOC Overview HMI-AIA Joint Science Operations Center Science Data Processing a.k.a. JSOC-SDP Overview.
Page 1GS CDR May 2005 JSOC SDP Agenda Significant Level 4 requirements SDP Architecture Decomposition –Datacapture System –Datacapture System Components.
Computer Science Lecture 19, page 1 CS677: Distributed OS Last Class: Fault tolerance Reliable communication –One-one communication –One-many communication.
Page 1JSOC Peer Review 17Mar2005 HMI & AIA JSOC Architecture Science Team Forecast Centers EPO Public Catalog Primary Archive HMI & AIA Operations House-
HMI Major Science Objectives The primary goal of the Helioseismic and Magnetic Imager (HMI) investigation is to study the origin of solar variability and.
Page 1JSOC Peer Review 17Mar2005 HMI & AIA JSOC Architecture Science Team Forecast Centers EPO Public Catalog Primary Archive HMI & AIA Operations House-
UK E-Science Initiative and its Application to SDO J.L. Culhane MSSL.
Algorithm Preparation and Data Availability 1. Mullard Space Science Laboratory, University College London. 2. Physics and Astronomy Department, University.
Heliophysics Projects Division SDO Flight Operations Readiness Review (FORR) July 21-22, 2009 SOC Operations Readiness HMI-AIA JSOC Science Data.
Recent Enhancements to Quality Assurance and Case Management within the Emissions Modeling Framework Alison Eyth, R. Partheepan, Q. He Carolina Environmental.
Page 1SDO Teams Meeting, March 2008 Status and Overview of HMI–AIA Joint Science Operations Center (JSOC) Science Data Processing (SDP) P. Scherrer Science.
Page 1JSOC Review – 17 March 2005 JSOC Pipeline Processing: Data organization, Infrastructure, and Data Products Rasmus Munk Larsen, Stanford University.
HMI Major Science Objectives The primary goal of the Helioseismic and Magnetic Imager (HMI) investigation is to study the origin of solar variability and.
8:30-9:10 AM: Philip Scherrer, What Can We Hope to Learn from SDO Overview of SDO HMI Investigation HMI Instrument.
Simulations of Solar Convection Zone Nagi N. Mansour.
Page 1JSOC Review – 17 March 2005 JSOC 17 March 2005 “Peer” Review Stanford SDP Development Plan Philip Scherrer
ESDO: UK Access to the Solar Dynamics Observatory 1. Mullard Space Science Laboratory, University College London. Holmbury St. Mary, Dorking, Surrey SM2.
Helioseismology for HMI Science objectives and tasks* Data analysis plan* Helioseismology working groups and meetings *HMI Concept Study Report, Appendix.
JSOC Status at SDO PSR-1 January 21-22, 2009 GSFC
E.C. Auden1, J.L. Culhane1, Y. P. Elsworth2, A. Fludra3, M. Thompson4
HMI Status P. Scherrer.
Existing Perl/Oracle Pipeline
HMI JSOC Science Data Processing Art Amezcua
The Helioseismic & Magnetic Imager – Magnetic Field Data Products
The Helioseismic & Magnetic Imager – Magnetic Investigations
HMI Science Investigation Overview
JDAT Production Hardware
HMI Investigation Overview
HMI Data Analysis Pipeline
Status and Overview of HMI–AIA Joint Science Operations Center (JSOC) Science Data Processing (SDP) April 22, 2008 P. Scherrer.
A Web-Based Data Grid Chip Watson, Ian Bird, Jie Chen,
JSOC Pipeline Processing System Components
Science Data Capture Jim Aloise Nov , 2004 System Programmer
HMI – AIA Joint Science Operations Center “Peer” Overview, 17 March AM HEPL Conference Room, Stanford Overview of JSOC Parts (Phil, 5 min.) JSOC.
HMI Data Analysis Pipeline
Presentation transcript:

Page 1JSOC Overview August 2007 HMI Status HMI is virtually done. –Virtually  similar to but not in fact –Front window issue resolved, flight window now at LMSAL will be installed next week –Light leaks to be fixed next week. –Some electronics boards need conformal coating –Some software issues remain Vibration and EMI/EMC testing done Thermal Balance and Thermal Vacuum testing starting week after next. In Air and Vacuum Calibration next few weeks. Pre-Ship Review set for 25 Sept. Ship date estimated 5 October. Launch date slip to probably 15 Jan 2009 pending review at HQ. Some serious implications from delay

Page 2JSOC Overview August 2007 JSOC-SDP Status Capture system complete, waiting testing with DDS at White Sands DRMS and SUMS virtually done Level-0 work progressing, to be done by delivery Level-1 work to start after delivery Getting serious about work on basic pipeline modules

Page 3JSOC Overview August 2007 HMI & AIA JSOC Architecture Science Team Forecast Centers EPO Public Catalog Primary Archive HMI & AIA Operations House- keeping Database MOC DDS Redundant Data Capture System 30-Day Archive Offsite Archiv e Offline Archiv e HMI JSOC Pipeline Processing System Data Export & Web Service Stanford LMSAL High-Level Data Import AIA Analysis System Local Archive Quicklook Viewing housekeeping GSFC White Sands World

Page 4JSOC Overview August 2007 JSOC Dataflow Rates

Page 5JSOC Overview August 2007 JSOC-SDP Major Components

Page 6JSOC Overview August 2007 JSOC Pipeline Processing System Components Database Server SUMS Storage Unit Management System DRMS Data Record Management System SUMS Tape Farm SUMS Disks Pipeline Program, “module” Record Manage ment Keyword Access Data Access DRMS Library Link Manage ment Utility Libraries JSOC Science Libraries Record Cache PUI Pipeline User Interface Pipeline processing plan Processing script, “mapfile” List of pipeline modules with needed datasets for input, output Pipeline Operato r Processing History Log

Page 7JSOC Overview August 2007 Data Record Management Service (DRMS) Data Record Management Service (DRMS) Pipeline client-server architecture JSOC Disks Analysis code C/Fortran/IDL/Matlab JSOC Library Record Cache (Keywords+Links+Data paths) OpenRecords CloseRecords GetKeyword, SetKeyword GetLink, SetLink OpenDataSegment CloseDataSegment Pipeline client process JSOC Disks Data Record Management Service (DRMS) Storage Unit Management Service (SUMS) Tape Archive Service Series Tables Record Catalogs Storage Unit Tables Record Catalogs Record Tables PostgreSQL Database Server AllocUnit GetUnit PutUnit SQL queries Storage unit transfer Data Segment I/O Storage unit transfer Generic file I/O DRMS socket protocol

Page 8JSOC Overview August 2007 Pipeline batch processing A pipeline batch is encapsulated in a single database transaction, “DRMS session”: –If no module fails all data records are commited and become visible to other clients of the JSOC catalog at the end of the session –If failure occurs all data records are deleted and the database rolled back –It is possible to commit data produced up to intermediate checkpoints during sessions DRMS Service = Session Master Input data records Output data records DRMS API Register session DRMS API Module 1 DRMS API Module 2.2 … DRMS API Module N DRMS API Commit Data & Deregister Pipeline batch = atomic transaction Record & Series Database SUMS DRMS API Module 2.1

Page 9JSOC Overview August 2007 HMI module status and MDI heritage Doppler Velocity Heliographic Doppler velocity maps Tracked Tiles Of Dopplergrams Stokes I,V Continuum Brightness Tracked full-disk 1-hour averaged Continuum maps Brightness feature maps Solar limb parameters Stokes I,Q,U,V Full-disk 10-min Averaged maps Tracked Tiles Line-of-sight Magnetograms Vector Magnetograms Fast algorithm Vector Magnetograms Inversion algorithm Egression and Ingression maps Time-distance Cross-covariance function Ring diagrams Wave phase shift maps Wave travel times Local wave frequency shifts Spherical Harmonic Time series Mode frequencies And splitting Brightness Images Line-of-Sight Magnetic Field Maps Coronal magnetic Field Extrapolations Coronal and Solar wind models Far-side activity index Deep-focus v and c s maps (0-200Mm) High-resolution v and c s maps (0-30Mm) Carrington synoptic v and c s maps (0-30Mm) Full-disk velocity, sound speed, Maps (0-30Mm) Internal sound speed Internal rotation Vector Magnetic Field Maps MDI pipeline modules exist Standalone production codes in use at Stanford Research codes in use by team Codes to be developed at HAO Codes being developed in the community Codes to be developed at Stanford Primary observables Intermediate and high level data products

Page 10JSOC Overview August 2007 AIA Level-2

Page 11JSOC Overview August 2007 JSOC Data Volumes from Proposal

Page 12JSOC Overview August 2007 Stanford JSOC effort plan