15 December 2008Science from UKIDSS II WFCAM Science Pipeline Update WFCAM Science Pipeline Update Jim Lewis, Mike Irwin & Marco Riello Cambridge Astronomy.

Slides:



Advertisements
Similar presentations
26 January 2007 CAL 07 Garching 1 The VISTA Data Flow System Jim Lewis, Mike Irwin, Peter Bunclark, Simon Hodgkin Cambridge Astronomy Survey Unit.
Advertisements

A NEW CTE PHOTOMETRIC CORRECTION FORMULA FOR ACS Marco Chiaberge TIPS meeting 05/16/2012.
ADASS XVII Sep 2007The NOAO Pipeline Applications Francisco Valdes (NOAO) Robert Swaters (UMd) Derec Scott (NOAO) Mark Dickinson (NOAO)
Panoramic near-Infrared Astronomy A first look at the UKIDSS data Steve Warren UKIDSS Survey Scientist with Steve Weatherley (Deteca Ltd), Paul Hewett.
SDSS and UKIDSS Jon Loveday University of Sussex.
VISTA/WFCAM pipelines summit pipeline: real time DQC verified raw product to Garching standard pipeline: instrumental signature removal, catalogue production,
ELT Stellar Populations Science Near IR photometry and spectroscopy of resolved stars in nearby galaxies provides a way to extract their entire star formation.
Star-Formation in Close Pairs Selected from the Sloan Digital Sky Survey Overview The effect of galaxy interactions on star formation has been investigated.
10Tb 100Tb 1000Tb. WFCAM processing summary semesters: 05A, 05B, 06A, 06B, 07A nights: 81, 166, 95, 157, 50 (68%) 50 Tbytes raw data 17 Tbytes compressed.
UKIDSS SciVer Products E. A. Gonzalez-Solares (IoA, Cambridge) (+CASU) UKIDSS SV.
Cosmos Data Products Version Peter Capak (Caltech) May, 23, 2005 Kyoto Cosmos Meeting.
VISTA pipelines summit pipeline: real time DQC verified raw product to Garching standard pipeline: instrumental signature removal, catalogue production,
Extracting the Mystery from the Red Rectangle Meghan Canning, Zoran Ninkov, and Robert Slawson Chester Carlson Center for Imaging Science Rochester Institute.
WFAU Tapes === Store 6 x dual PCs 3GHz Xeons 6 x 1 Tbyte local disks 2 Tbyte ingest buffer 10 Tbytes of raw 20 Tbytes of processed WFCAM archive storage.
Signal vs Noise: Image Calibration First… some terminology:  Light Frame: The individual pictures you take of your target.  Dark Frame: An image taken.
18 April 2007 Second Generation VLT Instruments 1 VIRCAM & CPL: Lessons Learned Jim Lewis and Peter Bunclark Cambridge Astronomy Survey Unit.
Data provenance in astronomy Bob Mann Wide-Field Astronomy Unit University of Edinburgh
Image credit: ESO/NASA/ESA/JPL-Caltech Unveiling the astrophysics of high-redshift galaxy evolution.
Study of NICMOS Non-linearity B. Mobasher, A. Riess, R.de Jong, S.Arribas, E.Bergeron, R.Bohlin, H. Ferguson, A. Koekemoer, K. Noll, S. Malhotra, T. Wiklind.
S. Baggett, J. Anderson, K. S. Long, J. W. MacKenty, K. Noeske, J. Biretta, and the WFC3 team (STScI) WFC3 : Understanding and mitigating UVIS charge.
Pipeline calibrations of ACS data Max Mutchler Hubble Space Telescope Calibration Workshop October 2005.
PPA Stack User Driven Image Stacking for ODI data via a Highly Customizable Web Interface Soichi Hayashi Indiana University - Pervasive Technology Institute.
Science Impact of Sensor Effects or How well do we need to understand our CCDs? Tony Tyson.
Distortion in the WFC Jay Anderson Rice University
Jim Lewis and Guy Rixon, CASU. 24 April, 2001 Data-reduction Pipeline for the INT WFC: slide 1 The Data-reduction Pipeline for the INT Wide Field Camera.
NGC 2506 – a try for a spectroscopic study Ekaterina Atanasova Petr Kabath Christine Oppegaard Mª Carmen Sánchez Gil Tutor: Frédéric Royer 2nd NEON Archive.
Large Binocular Telescope Interferometer LBTI/NOMIC data analysis B. Mennesson, D. Defrère, P. Hinz, B. Hoffmann, O. Absil, B. Danchi, R. Millan-Gabet,
1 DES Photometric Calibration Strategy Douglas L. Tucker (FNAL) Filter and Calibrations Strategy Workshop 4 April 2008.
WFCAM Science Archive Critical Design Review, April 2003 The SuperCOSMOS Science Archive (SSA) WFCAM Science Archive prototype Existing ad hoc flat file.
Page 1 PACS NHSC Data Processing Workshop – Pasadena 26 th - 30 th Aug 2013 Overview of SPIRE Photometer Pipeline Kevin Xu NHSC/IPAC on behalf of the SPIRE.
GSC-II Plate Processing Pipeline Status C. Loomis, C. Sturch, F. Guglielmetti.
AST3-1 photometry from Dome A Bin Ma, Peng Wei, Yi Hu, Zhaohui Shang NAOC AST3
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Pipeline Calibration Outline of CALCOS Processing Steps and Keywords Reference Files Associations.
1 NHSC PACS NHSC/PACS Web Tutorials Running PACS photometer pipelines PACS-402 (for Hipe 12.0) Level 1 to Level 2 processing: The JScanam pipeline Prepared.
1 CORE DATA PROCESSING SOFTWARE PLAN REVIEW | SEATTLE, WA | SEPTEMBER 19-20, 2013 Name of Meeting Location Date - Change in Slide Master Data Release Processing.
Counting individual galaxies from deep mid-IR Spitzer surveys Giulia Rodighiero University of Padova Carlo Lari IRA Bologna Francesca Pozzi University.
Observational Astrophysics I
Galaxies in the UKIDSS Large Area Survey Jon Loveday Anthony Smith Celine Eminian University of Sussex.
April 2001 OPTICON workshop in Nice 1 The PSF homogenization problem in large imaging surveys Emmanuel BERTIN (TERAPIX)
VST ATLAS: Requirements, Operations and Products Tom Shanks, Nigel Metcalfe, Jamie McMillan (Durham Univ.) +CASU.
NIRISS Calibration Working Group Activities NIRISS Science Team Meeting Montréal, 2015 October 20/21 Kevin Volk STScI / HIA.
PVPhotFlux PACS Photometer photometric calibration MPIA PACS Commissioning and PV Phase Plan Review 21 st – 22 nd January 2009, MPE Garching Markus Nielbock.
M. Pilar Esquej, XMM-Newton European Space Astronomy Centre Page 1 Status of the XMM-Newton Slew Survey P. Esquej, R. Saxton, B. Altieri, A. Read, M. Freyberg,
NICMOS Calibration Challenges in the Ultra Deep Field Rodger Thompson Steward Observatory University of Arizona.
The European Hubble Space Telescope Legacy Archive Wolfram Freudling.
14 January Observational Astronomy SPECTROSCOPIC data reduction Piskunov & Valenti 2002, A&A 385, 1095.
Web Image Simulator Stefano Gallozzi & LBC Team Copyright © 2003.
Basic Detector Measurements: Photon Transfer Curve, Read Noise, Dark Current, Intrapixel Capacitance, Nonlinearity, Reference Pixels MR – May 19, 2014.
IPHAS Early Data Release E. A. Gonzalez-Solares IPHAS Consortium AstroGrid National Astronomy Meeting, 2007.
PACS NHSC Data Processing Workshop Aug 26-30, 2013 Page 1 SPIRE Spectrometer Data: Calibration Updates, User Data Reprocessing, and Other Issues Nanyao.
1 NHSC PACS NHSC/PACS Web Tutorials Running PACS photometer pipelines PACS-402 (for Hipe 13.0) Level 1 to Level 2.5 processing: The JScanam pipeline Prepared.
CCD Image Processing: Issues & Solutions. CCDs: noise sources dark current –signal from unexposed CCD read noise –uncertainty in counting electrons in.
26th October 2005 HST Calibration Workshop 1 The New GSC-II and it’s Use for HST Brian McLean Archive Sciences Branch.
GSPC -II Program GOAL: extend GSPC-I photometry to B = V ˜ 20 add R band to calibrate red second-epoch surveys HOW: take B,V,R CCD exposures centered at.
S. Baggett, J. Anderson, J. MacKenty, J. Biretta, K. Noeske, and the WFC3 team (STScI) HST/WFC3 UVIS Detectors: Radiation Damage Effects and Mitigation.
A. Ealet Berkeley, december Spectrograph calibration Determination of specifications Calibration strategy Note in
SINFONI data reduction using the ESO pipeline. Instrument design and its impact on the data (I) integral field spectrometer using mirrors brickwall pattern.
CCD Calibrations Eliminating noise and other sources of error.
Pete Kuzma PhD student, Research School of Astronomy and Astrophysics
A.Zanichelli, B.Garilli, M.Scodeggio, D.Rizzo
Hiroko Watanabe (Kyoto Univ.)
JWST Pipeline Overview
Cosmology and Galaxies Level 3 Requirements
Imaging data obtained with the
WFCAM Photometric Calibration
Integral Field Spectroscopy
Basics of Photometry.
Karen Meech Institute for Astronomy TOPS 2003
Announcements No lab this week since we had an observing night Tuesday. Next week: 1st Quarter Nights Tuesday and Thursday. Set-up will start at 6:30pm.
First Public Data Releases from the VISTA Science Archive
Presentation transcript:

15 December 2008Science from UKIDSS II WFCAM Science Pipeline Update WFCAM Science Pipeline Update Jim Lewis, Mike Irwin & Marco Riello Cambridge Astronomy Survey Unit

15 December 2008Science from UKIDSS II WFCAM processing summary Semesters05A05B06A06B07A07B08A08BNights ► an average of 70% of telescope time ► 115 TB Raw Data (30 TB compressed) ► 200 TB processed Data (50 TB compressed) ► 4 million 2k x 2k images

15 December 2008Science from UKIDSS II Processing steps reminder ► Internet raw data transfer -> Cambridge ► Ingest and verify, record OMP MSB status -> tape backup ► Dark and reset anomaly correction <- master darks ► Flat field correction <- master twilight flats ► Decurtaining - remove variable pattern noise ► Detector crosstalk removal - channels within quadrants ► Sky background estimation and subtraction -> sky images ► Interleaving 1x1, 2x2 or 3x3 ► Dither/jitter stacking -> science product image ► Catalogue generation ► Astrometric and photometric calibration ► Quality control checks

15 December 2008Science from UKIDSS II Why Do Background Correction ? typical before and after background correction #1

15 December 2008Science from UKIDSS II Update since 07B  extra processing to remove parquet floor pattern  monthly photometric recalibration (Hodgkin et al. 2008)  MSBTid logs available immediately after stage I processing  nebulosity filtering post-processing option  new observing and reduction strategies for improved background subtraction for large extended objects  bug fixes/workarounds including a long-standing one

15 December 2008Science from UKIDSS II problems with detectors #2,3 K-band sky #3 stacked final image

15 December 2008Science from UKIDSS II Parquet floor processing LAS Y-band detector #2

15 December 2008Science from UKIDSS II Sky flipping/offsets ► extended objects observed on a given detector ► objects are then ‘flipped’ to a different detector ► background for ‘object’ detector in one jitter determined from compliment observation jitter ► objects of interest are observed at all times (advantage over offset sky strategy) ► offset skies only for large objects, e.g. M101, clusters, which cover more than 1 detector

15 December 2008Science from UKIDSS II Bugs:  indexing bug in catalogue Petrosian routine  only affects deblended objects and is now fixed  fundamental problem with circular apertures  workaround found that can be applied retrospectively  all catalogues will be repaired at end of 08B processing deblended objects Petrosian fluxes and radii

Photometry in regions of high nebulosity - M17 K-band WFCAM