SOAR Data Reduction Pipelines Progress Report Simón Torres R. Data Analyst storres@ctio.noao.edu

The Team La Serena Software Team Bruno Quint César Briceño Simón Torres See final slide for other people involved

Summary Goodman HTS Pipeline Overview We have something that works but still requires some development Other SOAR Data Reduction Pipelines Just some info since I’m not personally involved

Goodman Pipeline: What is it? Python-based tools to: Reduce Goodman’s raw data Imaging Spectroscopy Perform Spectroscopic Reduction Identification Trace Extract Wavelength Calibration (future plans) Flux Calibration Full process is split in two sub-processes

Our Goals We we’ll need beta testers! Very well documented software Most processes transparent to the user Different levels of debug available Easy code maintenance We’ll see that but we keep that in mind as we work Group development in mind Keep in mind new developers We we’ll need beta testers! Users/Community input is required

How is it being made? We use Python2.7 (Check compat. w/P3.5) General Coding Standards PEP8 – Style Guide PEP257 – Docstrings Convention (in-code documentation) – Google Style GIT + GitHub PyCharm Development Platform Open Source Multiplatform Compatibility “Modular Design” In theory could be used as a library PEP: Python Enhancement Proposal

Development Platform Centos 7 64bits No benchmarking yet but… 32GB RAM i7 Processor Solid State Drive 6TB Total Space, RAID 6 No benchmarking yet but… CCD Reduction: 286 Images, 98 Sci/Comp ~ 7:30 Minutes Spectroscopic Reduction 69 Science Images ~ 5:00 Minutes Including user input Only one spectroscopic configuration 400m2

Documentation Status In-Code Documentation – Docstrings Updated version of ccd reduction is incomplete GitHub Hosted Wiki Not extensive but precise User Documentation Requires more work but working on it

PEP257 Example

Status and Schedule Works up to Wavelength Calibration Schedule Not ready for a release Important modules missing (Flux Calibration) Other important Features missing Automatic Wavelength Calibration User documentation Cross platform compatible GUI (Qt4/5?) Schedule Internal testing and quality controls June Start Beta Testing (ends August) First 1.0 Release by the end of year Complete and stable

Conditions to use our pipeline Be consistent in getting your data. Some initial pre- pipeline data cleaning & book keeping will be necessary. Example: delete focus sequences Header information must be correct: e.g. no wrong object types Don’t mix ROIs, Imaging/Spectroscopy data Basic CCD reduction can handle BLUE/RED Camera as well as Imaging/spectroscopy. But not together. Use one of the “Supported Obs. Modes”, e.g. flats taken in afternoon, or together with science frames during the night We seek user input in order to make a better decision

Problems Developing a pipeline for an instrument with: “Infinite possible configurations” Two different cameras Headers and Keywords differ from each other Automatic Wavelength Solution There are some non-linearities in the data Maybe a characterization of them might help Graphical Interface to find w/solution Interactively Non-linear w/solution FITS documentation It was easier to linearize the spectrum

Different Screen sizes

Slit Sizes (Line detection solved)

How to get the code https://github.com/simontorres/goodman/tree/development

Documentation Availability This is how we will present the documentation. Will not be up dated unless there is an official release.

Snapshots and Data

Snapshots and Data Compared to iraf you don’t need to know…

Screen Recording on Youtube https://youtu.be/Ly1Mnx4MSy4

Future Plans Add Automatic Wavelength Calibration Add Flux Calibration Module Improve Multi-Target Capability MOS mode Create a Dedicated GUI We will provide Data Reduction Computers Three i7 32GB RAM 6TB RAID 6 Storage Computers Pipeline will still be publicly available

GUI Concept

SOAR Pipelines Summary Instrument Developer Language Status SIFS Luciano Fraga Python Development STELES Eder Martioli C++/Python Development* Goodman Simón Torres SOI PyRAF Production SAMI SAM-FP Bruno Quint SPARTAN Patrick O’Brien C++/Others Development** * STELES Pipeline based on OPERA by Canadian-French-Hawaii Telescope which is done but needs to be adapted to STELES. Eder Martioli from LNA. He was a post-doc at CFHT and it is a member o OPERA developed team. ** THELI * Based on OPERA by CFHT. Eder is part of OPERA dev. team. ** THELI, Was developed by Mischa Schirmer