Calibration Pipeline.

Slides:



Advertisements
Similar presentations
Lab 8 Ordered list. OVERVIEW In an ordered list the elements are maintained in ascending (or descending) order based on the data contained in the list.
Advertisements

Rachel Klima (on behalf of the MASCS team) JHU/APL MASCS/VIRS Data Users’ Workshop LPSC 2014, The Woodlands, TX March 17,2014 MASCS/VIRS Data Record Structure.
VMF Detailed Design Version 3 by Yehuda Afek Alexander Matveev 5/11/2015VMF Dettailed Dezign ver21.
SJC - 109/29/05IRAC Calibration Workshop Status of IRAC Artifact Corrector Sean Carey.
STScI TIPS 19 January 2006 Removing SAA-Persistent Cosmic Ray Flux from NICMOS Anton Koekemoer (INS) 1 Removing SAA-Persistent Cosmic Ray Flux from NICMOS.
FMOS Observations and Data 14 January 2004 FMOS Science Workshop.
CSC 160 Computer Programming for Non-Majors Lecture #9: Booleans Prof. Adam M. Wittenstein
CSC 160 Computer Programming for Non-Majors Chapter 4: Conditional Expressions and Functions Prof. Adam M. Wittenstein
HI 1A - Raw Image. HI 1A – Shutterless/Flatfield Corrected Image (secchi_prep)
Rachel Klima (on behalf of the MASCS team) JHU/APL MASCS/VIRS Data Users’ Workshop LPSC 2014, The Woodlands, TX March 17,2014 MASCS Instrument & VIRS Calibration.
Label production Solution with Label Gallery programs Label Gallery is used for general label design and print GalleryForm is used to create data entry.
Processing of exoplanet full field images Farid Karioty CoRoT Week 12/06/2005.
Memorandam of the discussion on FMOS observations and data kicked off by Ian Lewis Masayuki Akiyama 14 January 2004 FMOS Science Workshop.
Pipeline calibrations of ACS data Max Mutchler Hubble Space Telescope Calibration Workshop October 2005.
6e-1 Science Data Products Daryl Swade DMS Systems Engineer S&OC System Design Review #1.
CSC 213 – Large Scale Programming. Why Do We Test?
NGC 6217 in DSI mode with F658N Mosaic before new superbias why we don’t release images until after SMOV! ACS CCD monitoring and pipeline calibration review.
The Field Camera Unit Project definition, organization, planning S. Scuderi INAF – Catania.
General Computer Science for Engineers CISC 106 Lecture 02 Dr. John Cavazos Computer and Information Sciences 09/03/2010.
Formal Modeling of an Openflow Switch using Alloy Natali Ruchansky and Davide Proserpio.
CCD Detectors CCD=“charge coupled device” Readout method:
Washington WASHINGTON UNIVERSITY IN ST LOUIS Packet Routing Within MSR Fred Kuhns
Page 1 PACS NHSC Data Processing Workshop – Pasadena 26 th - 30 th Aug 2013 Overview of SPIRE Photometer Pipeline Kevin Xu NHSC/IPAC on behalf of the SPIRE.
VB Games: Preparing for Memory Brainstorm controls & events Parallel structures (again), Visibility, LoadPicture, User-defined procedures, Do While/Loop,busy.
Tutorial T-10 Amber Data Reduction (practical point of view) EuroSummer School Observation and data reduction with the Very Large Telescope Interferometer.
ACS Drizzling Overview J. Mack; DA Training 10/5/07 Distortion Dither Strategies MultiDrizzle ‘Fine-tuning’ Data Quality Photometry.
Radiometric Correction and Image Enhancement Modifying digital numbers.
JWST Calibration Error Budget Jerry Kriss. 15 March 20072/14 JWST Flux & Wavelength Calibration Requirements SR-20: JWST shall be capable of achieving.
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Pipeline Calibration Outline of CALCOS Processing Steps and Keywords Reference Files Associations.
Observational Astrophysics I
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Pipeline Calibration Goals of CALCOS Association Table Input and Output Files High Level.
Hinode / EIS Hinode Data Analysis Workshop, Paris, Nov 2007 Dr Peter Young, Rutherford Appleton Laboratory What does eis_prep do? Dr Peter Young Dr Peter.
HARPS Data Flow System Christophe Lovis Geneva Observatory HARPS-N PDR, 6-7 December 2007, Cambridge MA.
C2d Data flow diagram BCD from SSC Texas SAO Quality Analysis and Improved Calibrated Data Mapping team.
Beginning Fortran Introduction 13 October 2009 *Black text on white background provided for easy printing.
Corot:Ubatuba Nov The Ground Segment: Data Processing. Software development in laboratories. Brazilian contribution. Michel Auvergne Réza Samadi.
July 21, 2004 Natalia Kuznetsova NICMOS Analysis of ACS ● Data Inventory ● Calibration ● Bad pixel mask ● Extracting photometry ● Next steps 1.
CUPID (Bob Narron)GRITS - May 14, CUPID “Customizable User Pipeline for IRS Data” Abstract: The CUPID package will allow users to run the Spitzer.
PyRAF/Pipeline Removal of SAA Persistence from NICMOS Data Elizabeth A. Barker Vicki Laidler Eddie Bergeron Anton Koekemoer Elizabeth A. Barker Vicki Laidler.
New NICMOS Readout Sequences Chun Xu Space Telescope Science Institute Abstract On 12 June 2005, four new NICMOS multi-accum sequences (SPARS4, SPARS16,
The Field Camera Unit Results from technical meeting S. Scuderi INAF – Catania.
1 Notes about eis_prep Alessandro Gardini University of Oslo Oslo, July 9th, 2009.
1 German Aerospace Center HRSC Data Products Thomas Roatsch, DLR 22-May-2008.
IAS Hinode Data WorkshopThursday 15-November-2007 The Hinode Solar Optical Telescope Data & Calibration Dr. Thomas Berger Lockheed Martin Advanced Technology.
WEC meeting TED status and WEC timing.
SINFONI data reduction using the ESO pipeline. Instrument design and its impact on the data (I) integral field spectrometer using mirrors brickwall pattern.
PACS Hitchhiker’s Guide to Herschel Archive Workshop – Pasadena 6 th – 10 th Oct 2014 Overview of SPIRE Photometer Data Reduction Pipeline Kevin Xu NHSC/IPAC.
OSIRIS operation summary
Calibration meeting summary
MPS - Archive Structure
Stubbe F. Hviid OSIRIS Team Meeting November 2005
OSIRIS Calibration Report (Lab Calibration)
Post-Operation Status: Archiving, Calibration & Science
NAC flat fielding and intensity calibration
OSIRIS operation summary
OSIRIS flight calibrations so far
JWST Pipeline Overview
Chapter 7 Text Input/Output Objectives
PICAM Pipeline S/W development Milestones
Metis On-Board SW overview
Digital Artifacts.
Structure Chart Equipment Tracking & Inspection System 1 User Login 2
True or False: {image} is one-to-one function.
Mosaic code update: mapping the kronian magnetosphere
A Simple Two-Pass Assembler
Goteachmaths.co.uk True or False Maze – Converting Mixed Numbers and Improper Fractions Students need to find a route from the start to the finish through.
Single-view geometry Odilon Redon, Cyclops, 1914.
Robert Gilmore Pontius Jr Geography Professor at Clark University
General Computer Science for Engineers CISC 106 Lecture 03
True or False True or False
Presentation transcript:

Calibration Pipeline

Osiris Data Flow - Overview RTU Osiris MMB/IM Osiris Packetizer S/C SSMM S/C 1355 VC0 + VC1 ESA Osiris GSEOS TM Decoder + Level 0 TMI generator ESOC SCOS 2000 archive DDS I/F TMI Level 0 to PDS level 1 converter (tmi2pds.exe) Level 1 to Level 2 converter (IDL Based) Level 2 to Level 3 converter (IDL Based) Science  Level 1 Archive Level 2 Archive Level 3 Archive

Calibration Pipeline Purpose: To perform the Level 1 to Level 2 calibration (From DN To Wm-2sr-1nm-1) To perform the Level 2 to Level 3 calibration (Geometric distortion) Constraints: Must work with PDS images Must produce legal PDS labels Must maintain strict revision control (Both on source level, data file level and PDS label level) Must be open source (To be archived with the PDS) Must correctly deal with HW and SW binning

Pipeline Elements? Level 1 -> 2 Bad pixel Replacement Hviid / Kramm Coherent noise removal Kramm Low/High ADC offset removal Cosmics removal ? Ghost removal Straylight Pinholes PSF Deconvolution Shutter motion correction Hviid / ? Bias Subtraction Knollenberg / Kramm Readout noise correction Dark current correction Flatfielding Thomas / Küppers DN to Wm-2sr-1nm-1 (as function of T and pixel coord) Level 2 -> 3 Geometric distortion correction

Pipeline I/F #1 Datafile naming convention: PDS Labels: <camera>_<unit>_<file id>_V<version>_<creationdate>.<ext> Example: NAC_FM_FLAT-FFPVIS-ORANGE_V1_26102004.img PDS Labels: Each processing step must write a processing history entry to the PDS header (using p_approc). This entry must include name of function, version of function and a list of the data files used during the processing. Each processing step must set a processing boolean flag in the header indicating that the processing step has been performed.

Pipeline I/F #2 IDL I/F: Pipeline main entry points: Subroutine I/F: result = osiris_level1_to_level2(header_in, image_in, header_out, image_out) result = osiris_level2_to_level3(header_in, image_in, header_out, image_out) Subroutine I/F: result = osiris_calibrate_<process name>(header, image, _EXTRA=_EXTRA) Example: result = osiris_calibrate_badpixels(header, image, _EXTRA=_EXTRA) Return value as a boolean TRUE=success FALSE=error

Example FUNCTION osiris_calibrate_badpixels, header, image, VERBOSE=VERBOSE,_EXTRA=_EXTRA VERSION = '0.1' if keyword_set(VERBOSE) then print, 'osiris_calibrate_badpixels running' ; Verify the the correction has not already been performed flag = p_value(header, 'SR_PROCESSING_FLAGS.BAD_PIXEL_REPLACEMENT_FLAG') if flag eq 'TRUE' then begin if keyword_set(VERBOSE) then print, 'osiris_calibrate_badpixels skipped - Already performed' return, 1 endif ; do the calibration print, '******* Bad pixel replacement implementation missing' ; Change the processing flag p_setvalue, header, 'SR_PROCESSING_FLAGS.BAD_PIXEL_REPLACEMENT_FLAG', 'TRUE', /LABEL, /NO_QUOTES p_approc,header,header,'osiris_calibrate_badpixels','V'+VERSION, 'Using bad pixel table: NAC_FM_BADPIXEL_V1_20082004.dat' ; finish if keyword_set(VERBOSE) then print, 'osiris_calibrate_badpixels - Done' END

PDS Label Example LABEL_REVISION_NOTE = "V3.0 June 8 2004 SFH" /* PROCESSING HISTORY */ /* PH: Tue Oct 26 12:57:41 2004 UTC hviid V0.1 - osiris_calibrate_badpixels - Using bad pixel table: NAC_FM_BADPIXEL_V1_20082004.dat */ /* PH: Tue Oct 26 12:57:41 2004 UTC hviid V0.1 - osiris_calibrate_shutter_motion */ /* PH: Tue Oct 26 12:57:41 2004 UTC hviid V0.1 - osiris_calibrate_absolute - Using datafile: NAC_FM_ABSCAL_V1_26012004.img */ /* PH: Tue Oct 26 12:57:41 2004 UTC hviid V0.1 - osiris_level1_to_level2 */ /* FILE CHARACTERISTICS */ RECORD_TYPE = FIXED_LENGTH RECORD_BYTES = 512 < --- Snip --- > /* PROCESSING FLAGS */ GROUP = SR_PROCESSING_FLAGS ROSETTA:READOUT_CORRECTION_FLAG = TRUE BAD_PIXEL_REPLACEMENT_FLAG = TRUE DARK_CURRENT_CORRECTION_FLAG = FALSE SHUTTER_CORRECTION_FLAG = FALSE FLATFIELD_CORRECTION_FLAG = FALSE ROSETTA:GEOMETRY_CORRECTION_FLAG = FALSE ROSETTA:DISTORSION_CORRECTION_FLAG = FALSE ROSETTA:READOUT_NOISE_CORRECTION_FLAG = FALSE ROSETTA:ABSOLUTE_CALIBRATION_FLAG = FALSE ROSETTA:ADC_CORRECTION_FLAG = TRUE SHUTTER_CORRECTION_FLAG = TRUE ROSETTA:READOUT_NOISE_CORRECTION_FLAG = TRUE FLATFIELD_CORRECTION_FLAG = TRUE ROSETTA:ABSOLUTE_CALIBRATION_FLAG = TRUE END_GROUP = SR_PROCESSING_FLAGS