Stubbe F. Hviid OSIRIS Team Meeting November 2005 Archive Access Status Stubbe F. Hviid OSIRIS Team Meeting November 2005

Data Access Current: Future: External: MPS Internal: ftp://134.76.232.5:910 MPS Internal: http://osi-storage \\osi-storage\osiris_archive Future: https://134.76.232.5/osiris_archive (ftp://134.76.232.5:910) to be phased out

Archive Web Interface Current Future Server filtering of PDS images Server filtering of command history Server filtering of CSV HK history files … Future Database search engine

Radius Authentication Server Archive Architecture OSI-Storage Win 2003 Server MPS Internet SQL Database MPAE_HUK Domain MPS_Gate Free BSD Web Server Team Members Firewall SMB Shares FTP Server DDS + RSOC Radius Client Archive access over encrypted lines Except for DDS Radius Authentication Server FTP Proxy Write access From inside MPS only (CVS excluded) HTTPS Proxy CVS Server CVS Proxy 3.5 Tb RAID 5

Archive Structure Flight data data/spacecraft/pfm/flight/<session>[/<subsession] /index.syn Session synopsis /observations.txt Observation list HK Data in CSV format /log/<packet>.csv (MCB, PCM, NAC, WAC, Heater, SREM) Event History /log/MessageLog.txt Command History /log/DDSCommandLog.cmdpf

Archive Structure II Flight data PDS Images FITS Images XDR Images /pds/<level> [level1, level2, level2] FITS Images /fts/<level> XDR Images /uax TMI (level 0) Images /tmi

Archive Structure III Flight data Shutter optimization data /shutter (Summary CSV files) /shutter/intermediate (Individual optimization steps) Recorder Data (DDS packet files) /recorder Data array dumps /data_arrays (Binary files containing various parameter table dumps. Use FwBinFileBrowser.exe for decode) Uplink ITL /uplink

Archive Structure IV See RO-RIS-MPAE-ID-017 For further details

Data Formats

Osiris PDS Images All Images stored in “PDS” format Meaning files with embedded PDS labels containing a std. IMAGE object. The OSIRIS PDS files (level 1) also contains the shutter pulse arrays and the original telemetry TMI2 header as embedded binary objects The data is stored as: Level 1 – unsigned 16 bit integer data Level 2 – double precision float (currently) Level 3 – ? (float) Level 4 – ? (float) The OSIRIS PDS label in documented in the OSIRIS EDR/SIS (Osiris_EDRSIS_1d.doc)

Show EDR/SIS

Calibration Pipeline Status Stubbe F. Hviid OSIRIS Team Meeting November 2005

Osiris Data Flow - Overview RTU Osiris MMB/IM Osiris Packetizer S/C SSMM S/C 1355 VC0 + VC1 ESA Osiris GSEOS TM Decoder + Level 0 TMI generator ESOC SCOS 2000 archive DDS I/F TMI Level 0 to PDS level 1 converter (tmi2pds.exe) Level 1 to Level 2 converter (IDL Based) Level 2 to Level 3 converter (IDL Based) Science  Level 1 Archive Level 2 Archive Level 3 Archive

Pipeline Elements? Task Responsible Status Level 1 -> 2 Bad pixel Replacement Hviid / Kramm Bad column only Coherent noise removal Kramm Done Low/High ADC offset removal Cosmics removal Ground / Flight software PUT LAST MIDAS Something exists Ghost removal Ignore N/A Straylight Pinholes PSF Deconvolution Shutter motion correction Hviid / ? Missing Bias Subtraction Ground / Flight software Knollenberg / Kramm Something exists. Improvements needed CTE ? Dark current correction (hot pixels) NAC “blob” correction (primitive) Flatfielding Thomas / Küppers Flatfields need update DN to Wm-2sr-1nm-1 (as function of T and pixel coord) Level 2 -> 3 Geometric distortion correction

Current pipeline implementation ADC Ofset Coherent Noise Removal Frame separation Bias subtraction Exposure time correction (included binning) Dark current removal (NAC “blob”) Bad column correction Flatfield correction Absolute calibration scaling

What is missing? Shutter motion Bias subtraction needs improvement: (Improve low exposure-time handling) Bias subtraction needs improvement: Only amp B really supported currently Improved modeling from Kramm needs integration Dark current correction needs improvement Additional calibration data needed (active checkout) Flatfields need to be updated to post launch Geometric Distortion correction missing (level 3)

Stubbe F. Hviid OSIRIS Team Meeting November 2005 PDS Archive Status Stubbe F. Hviid OSIRIS Team Meeting November 2005

Conclusion Commissioning Data delivery due Ultimo 2005 Help!

CODMAC Levels Level 0: TMI formatted uncalibrated data (header un-calibrated – Non PDS - Decoded) Level 1: Raw PDS images (header calibrated – Image data in raw DN) Level 2: Radiometric calibrated PDS images (header calibrated – Image data in Wm-2sr-1nm-1) Level 3: Geometric calibrated PDS images (header calibrated – Image data in Wm-2sr-1nm-1 – Geometry Corrected) Level 4: Geometric calibrated PDS images in J2000 PDS?

PDS Label Status Level PDS label defined and documented with following open points: Geometric parameters (Waiting for project input) Problem with shutter pulse array definition Level 2-4 additional entries not yet defined

Documentation Status EDR/SIS Late draft  EAICD in early draft  Supporting documentation Missing

Team Taskforce needs to be formed! Conclusion Help! Tasks Supporting Documentation needs to be collected OSIRIS PDS label needs to be reviewed EAICD needs to be finalized Automatic OSIRIS archive to PDS archive converter needs to be written Schedule Commissioning Data delivery due Ultimo 2005 Team Taskforce needs to be formed!