Science Data Capture Jim Aloise Nov 16 - 17, 2004 System Programmer jim@quake.stanford.edu Nov 16 - 17, 2004

My Documents\DMR_SOC_Req.ppt DMR – SOC Requirements 8000.2.4 Science Data Processing, Archiving and Distribution Each SOC shall provide the necessary facility, software, hardware and staff to receive, process, archive and distribute the science data generated by its instruments. Implementation is a Joint SOC (JSOC) for HMI and AIA Emphasis here is on Science Data Capture My Documents\DMR_SOC_Req.ppt

SDO Ground System Architecture HMI and AIA Joint Science Operations Center Acquisition Data Observatory Commands Housekeeping Telemetry Tracking Data Station Status Ka-Band: 150 Mbps Science Data S-Band: Cmd & HK Tlm Station Control Same Interfaces as Prime Ground Station DDS Control DDS Status Data Distribution System Flight Dynamics SDO Mission Operations Center Telemetry & Command Ground Station Control System DDS Control System Mission Planning Trending S band Ground System Ka band SDO Ground Site #2 White Sands SDO Ground Site #1 HMI Science Data 55 Mbps HMI & AIA Instrument Commands and Loads Ka Science Data Stanford University Science Data Processing Lockheed Martin Instrument Operations AIA Science Data 67 Mbps Science Planning and FDS Products HMI, AIA & S/C

Network Context Diagram SDOGS WSGT Ka-band STGT Ka-band WSGT S-band STGT S-band FEP FEP FEP FEP RRCP RRCP RRCP RRCP Legend OC3 fiber Ethernet Ethernet T3 T1 / FT1 EVE JSOC Stanford HMI AIA SDO Network Devices Draft v13e last updated 10/18/04 Fibre Channel SDOGSsr-3a SDOGSsr-3b SDOr14-2a SDOr14-2b SDOGSsr-1a SDOGSsr-1b SDOr14-1a SDOGSsr-6a SDOsr-6a SDOsr-6b SDODDSsr-4a SDODDSsr-4b SDODDSsr-5a SDODDSsr-5b ant. ctlr 30-day archive DDS SDODDSsr-3a SDODDSsr-3b SDOr14-1b SDOsr14-4a SDOsr14-4b SDOsr14-5a SDOsr14-5a R/T #1 MOC R/T #2 SDODDSr-1a firewall-a SDODDSr-2a Support segment DMZ SDOsr14-3a SDOsr14-3b firewall-b FlatSat Open IOnet SDOJSOCr-2b SDOJSOCr-2a SDOEVEr-2a SDOJSOCr-1b SDOJSOCr-1a SDOEVEr-1a SDOEVEr-1b HMI AIA EVE JSOC Lockheed SOC SDP SOC T&C

SOC SDP Data Flows SDO Network Devices Legend DDS SOC T&C SOC SDP MOC EVE JSOC Lockheed HMI AIA Open IOnet DMZ 30-day archive Legend OC3 fiber Ethernet Ethernet T3 T1 / FT1 JSOC Stanford SDO Network Devices Draft v13e last updated 10/18/04 Fibre Channel SDOGSsr-3a SDOGSsr-3b SDOr14-2a SDOr14-2b SDOGSsr-1a SDOGSsr-1b SDOr14-1a SDOGSsr-6a SDOsr-6a SDOsr-6b SDODDSsr-4a SDODDSsr-4b SDODDSsr-5a SDODDSsr-5b SDODDSsr-3a SDODDSsr-3b SDODDSr-1a SDODDSr-2a firewall-b firewall-a SDOJSOCr-1b SDOJSOCr-1a SDOEVEr-1a SDOEVEr-1b SDOEVEr-2a SDOJSOCr-2b SDOJSOCr-2a SDOsr14-3a SDOsr14-3b DDS SOC T&C R/T #2 R/T #1 SOC SDP MOC Support segment FlatSat ant. ctlr SDOr14-1b SDOsr14-5a SDOsr14-4a SDOsr14-4b RRCP FEP SDOGS STGT S-band WSGT Ka-band

Stanford/Lockheed Connections DDS JSOC Disk array NASA AMES LMSAL 1 Gb Private line MOC “White” Net

JSOC Data Capture Front End Front End Server Tape Robot Duplicate Front End Disk array 40TB (30days) Relational Database I D C Analysis Server Dedicated LAN OC3 HMI OC3 AIA DDS [etc.] Analysis Cluster jim\My Documents\datacapture_frontend Back End Data Base Server

JSOC Data Capture Front End Two OC3 lines from DDS for AIA and HMI science data Lines can be switched to either of two redundant Data Capture Front Ends Tape robot will create a copy of input tlm files for offsite storage Telemetry data is passed to the back end server where it is again archived to tape and cataloged in a database Telemetry data is not marked delete pending until an ack is received from both the offsite location and the backend archiving Disk array can retain 30 days of data online Relational data base tracks tlm file datasets and tape locations Separate dedicated LAN for front end machines A backend pipeline creates lev0 and higher data products

EGSE Configuration Screen

EGSE Run Screens

EGSE Run Screens The egse run screen shows the files as they are received from the SSIM (/egse/ssim2soc) These file are moved into the SOC for processing (/egse/soc2soc) The telemetry files are cataloged and decoded and the level 0 images are made and cataloged A third screen displays the status as images are complete or abnormal conditions occur The “Display log” button will put up a separate window showing all the logging activity of the program The “Readme” button is an extensive man page and background charts

EGSE Raw and Lev0 Datasets

EGSE Raw and Lev0 Datasets Two types of datasets are created in the EGSE. One is level:raw the other is level:lev0 The series name is “egse_[hostname]” The level:raw consists of the .tlm and .qac file that is sent by the SSIM every minute. Its series number is a unique index number of seconds since 2000 The level:lev0 is a fitz file for a filtergram and an empty file giving the dataset name. The series number is the unique filtergram serial number (fsn) associated with evey filtergram hmi0:/PDS1/D46470> ls -l total 136 -rw-r--r-- 1 production SOI 135168 Nov 1 12:16 000035.fitz -rw-rw-r-- 1 production SOI 0 Nov 1 12:16 prog:hmi_ground,level:lev0[42],series:egse_hmi0[35]

Decode Image

EGSE Archiver csh> egsearc Usage: egsearc [ - v] [ d] [ q] [ a archive_dir] database_name v = verbose mode d = run in debug mode q = query only to see what's available for archive a = give the dir to cp the archived ds to the default archive_dir is /hmi0/archive Use /dev/null to not archive but make del pend Use /dev/mt to write to tape csh>

EGSE S/W Components

JSOC Service Structure

Test Schedule Milestones Ground System Integration and Testing Tasks and Start Dates Delivery of First EGSE SDP 10/04/04 Testing with DDS Prototype 02/07/05 Delivery of Flight EGSE SDP 06/06/05 SOC Integration with MOC 03/27/06 Prototype Data Capture System Ready 06/05/06 GSRT#1- Telemetry and Command Test 06/19/06 AIA SOC White Sands Complex(WSC) 07/25/06 Production Data Capture System Ready 09/04/06 SOC Network Ready 12/01/06 DDS-SOC Testing 12/04/06 GSRT#2- Science Data Processing Test (Ka-band) 01/31/07 HMI Connectivity, Dataflow, Retransmissions Test 02/07/07 AIA Connectivity, Dataflow, Retransmissions Test 02/14/07 GSRT#3-Mission Operations& RF Communications Test 02/26/07 GSRT#4-Fully Integrate Ground System 02/26/07 Ground System Freeze 01/18/08 GSRT#4-Launch Readiness Test 02/01/08 Launch Site End-to-End Test 03/14/08