GSC-II Plate Processing Pipeline Status C. Loomis, C. Sturch, F. Guglielmetti.

Slides:



Advertisements
Similar presentations
The Stromlo Southern Sky Survey Paul Francis, Brian Schmidt and Mike Bessell.
Advertisements

26 January 2007 CAL 07 Garching 1 The VISTA Data Flow System Jim Lewis, Mike Irwin, Peter Bunclark, Simon Hodgkin Cambridge Astronomy Survey Unit.
The Molonglo Cross Survey Virtual Observatory David F. Crawford
Nov 26, 2009 Chongqing NAOC and Steward Observatory The South Galactic Cap U-band Sky Survey with the Bok 2.3m Telescope Fan Zhou NAOC.
1 Preprocessing for JPEG Compression Elad Davidson & Lilach Schwartz Project Supervisor: Ari Shenhar SPRING 2000 TECHNION - ISRAEL INSTITUTE of TECHNOLOGY.
Growth of Structure Measurement from a Large Cluster Survey using Chandra and XMM-Newton John R. Peterson (Purdue), J. Garrett Jernigan (SSL, Berkeley),
15 December 2008Science from UKIDSS II WFCAM Science Pipeline Update WFCAM Science Pipeline Update Jim Lewis, Mike Irwin & Marco Riello Cambridge Astronomy.
ART: Augmented Reality Table for Interactive Trading Card Game Albert H.T. Lam, Kevin C. H. Chow, Edward H. H. Yau and Michael R. Lyu Department of Computer.
Slews: Fast and Slow. Fast Slew Status  XMMSL1: catalogue of slew sources released May 2006 Revs , 5180 detections, 2692 Revs , 5180 detections,
Cosmos Data Products Version Peter Capak (Caltech) May, 23, 2005 Kyoto Cosmos Meeting.
Source detection at Saclay Look for a fast method to find sources over the whole sky Provide list of positions, allowing to run maximum likelihood locally.
VISTA pipelines summit pipeline: real time DQC verified raw product to Garching standard pipeline: instrumental signature removal, catalogue production,
18 April 2007 Second Generation VLT Instruments 1 VIRCAM & CPL: Lessons Learned Jim Lewis and Peter Bunclark Cambridge Astronomy Survey Unit.
Bruno Altieri September # 1 PACS photometer pipeline overview Bruno Altieri (ESA/HSC)
Commissioning the NOAO Data Management System Howard H. Lanning, Rob Seaman, Chris Smith (National Optical Astronomy Observatory, Data Products Program)
6 th Annual GSC-II Project Meeting C B A S GSC-II and DSS-II Projects 2000 Annual Report Brian McLean 19 th October 2000.
Digitized Sky Survey Update Brian McLean : Archive Sciences Branch / Operations and Engineering Division.
Memorandam of the discussion on FMOS observations and data kicked off by Ian Lewis Masayuki Akiyama 14 January 2004 FMOS Science Workshop.
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
Erich Franz Stocker * and Yimin Ji + * NASA Goddard Space Flight Center, + Wyle Inc/PPS The Global Precipitation Measurement (GPM) Mission: GPM Near-realtime.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
ERP IMPLEMENTATION LIFE CYCLE KASHIF SHAMIM. Enterprise resource planning ERP covers the technique and concepts employed for the integrated management.
06/02/2006 M.Razzano - DC II Closeout Meeting Pulsars in DC2 preliminary results from an “optimized” analysis Gamma-ray Large Area Space Telescope Massimiliano.
Managing Your Lawson System MSPs and CTPs Ron Wright netASPx Corp.
IT 456 Seminar 5 Dr Jeffrey A Robinson. Overview of Course Week 1 – Introduction Week 2 – Installation of SQL and management Tools Week 3 - Creating and.
Page 1 PACS NHSC Data Processing Workshop – Pasadena 26 th - 30 th Aug 2013 Overview of SPIRE Photometer Pipeline Kevin Xu NHSC/IPAC on behalf of the SPIRE.
Seminars on formation and evolution of the Galaxy Feb 12, 2002 The construction of the GSC2.2 Catalog Mario G. Lattanzi Osservatorio Astronomico di Torino.
GSCII Project Gauging the quality of the GSCII DB with QA_2plates: First results R. Pannunzio, R. Morbidelli, A Spagna, M. G.Lattanzi.
LSST: Preparing for the Data Avalanche through Partitioning, Parallelization, and Provenance Kirk Borne (Perot Systems Corporation / NASA GSFC and George.
1 M. Tudruj, J. Borkowski, D. Kopanski Inter-Application Control Through Global States Monitoring On a Grid Polish-Japanese Institute of Information Technology,
6dF Galaxy Survey Workshop Sydney Ophiuchus Supercluster and its Surroundings K. Wakamatsu, M.Nishida, M. Malkan, Q. Parker, W. Saunders, F. Watson.
1 BILLION “STARS” Barolo 2001 Attilio Ferrari Istituto Nazionale di Astrofisica.
7 th Annual GSC-II Project Meeting C B A S GSC-II and DSS-II Projects 2001 Annual Report Brian McLean 22 nd October 2001.
GSC2 Maintenance GSC2 Annual meeting Database administrative tasks Database production tasks Identification and correction of errors Processing.
Data Analysis Software Development Hisanori Furusawa ADC, NAOJ For HSC analysis software team 1.
Swift HUG April Swift data archive Lorella Angelini HEASARC.
PACS NHSC Data Processing Workshop – Pasadena 10 th - 14 th Sep 2012 The SPIRE Destriper Bernhard Schulz NHSC/IPAC on behalf of the SPIRE ICC 1.
The COMPASS (Catalogs of Objects and Measure Parameters for All Sky Surveys) Database Overview Gretchen Greene, Brian McLean, David Wolfe, and Charles.
From Pan-STARRS to PTF Wing-Huen Ip Institutes of Astronomy and Space Sciences National Central University Nov. 25, 2011 Sun-Moon Lake, Taiwan.
VST ATLAS: Requirements, Operations and Products Tom Shanks, Nigel Metcalfe, Jamie McMillan (Durham Univ.) +CASU.
Large high precision catalogues Large high precision catalogues of stellar data: of stellar data: current status and prospects N.V. Kharchenko 1, A.E.
The 6dF Galaxy Redshift Survey: Status Report and Data Release 2 Heath Jones ANU/AAO Will Saunders AAO Mike Read Edinburgh Matthew Colless AAO 100,000.
This presentation shall describe; The motivation to design a new system How the new system works and user interaction Current work/results Future work.
MGL/OATo 7th GSC2 Meeting, Barolo (I), 22 Oct 2001 The GSC2.2 Catalog: Global Properties Mario G. Lattanzi Osservatorio Astronomico di Torino.
DSS/Db Backups/HSM Overview David Wolfe. DSS-I 102 CD ROM set –POSS-I red XE plates –SERC-J blue S plates –Compressed ~10X –
IPHAS Early Data Release E. A. Gonzalez-Solares IPHAS Consortium AstroGrid National Astronomy Meeting, 2007.
Photometric Calibration Jorge F. García Yus GEMINI Observatory Barolo 2001.
Current Status of Users Meeting F URUSHO, Reiko (Astronomy Data Center, NAOJ) Topics: 1. Minor Bodies Search is Opened. 2. Number.
GSC2.2 Classification GSC II Annual Meeting October 2001.
MSW - July The Establishment of Astrometric Calibration Regions And the determination of positions for objects much fainter than existing reference.
Plate Scanning, Processing, and Compression Status Amy Rosenberg 7 th Annual GSC-II Meeting.
Canny Edge Detection Using an NVIDIA GPU and CUDA Alex Wade CAP6938 Final Project.
Pisa, 4 May 2009 Alessandro Spagna A new kinematic survey (from GSC-II and SDSS-DR7) to study the stellar populations of the Milky Way Alessandro Spagna.
Status of LAMOST ZHAO Yongheng National Astronomical Observatories of China.
26th October 2005 HST Calibration Workshop 1 The New GSC-II and it’s Use for HST Brian McLean Archive Sciences Branch.
Astrometry with Schmidt Plates Difficulty in attaining highly accurate positions -Bending of plates as they are placed in plate holder -Characteristics.
Auditing data management system Bruno Deprez Data Audit officer SPC, Oceanic Fisheries Program.
GSPC -II Program GOAL: extend GSPC-I photometry to B = V ˜ 20 add R band to calibrate red second-epoch surveys HOW: take B,V,R CCD exposures centered at.
STELLAR POSITIONS AT LWIR WAVELENGTHS NEW RESULTS USING MSX 6 Dec 00 Dr. Michael P. Egan Technical Advisor Space Infrared Technology CoE Air Force Research.
1 CAA 20 th Cross Calibration Meeting, MPS, Gottingen 16th Oct 2014 CAA Cross Cal Meeting Oct 2014 Pipeline Automation Chris Perry.
1 NHSC PACS NHSC/PACS Web Tutorials Running PACS photometer pipelines PACS-403 (for Hipe 13.0) Level 1 to Level 2.5 processing: The Unimap pipeline Prepared.
Enterprise Resource Planning
Jean Ballet, CEA Saclay GSFC, 31 May 2006 All-sky source search
H Stacked Images Reveal Large Numbers of PNe in the LMC
A “Field Guide” to WFPC2 HLA Image Anomalies
Imaging data obtained with the
Sensors and Logic Switches
COMPASS Database SPACE TELESCOPE SCIENCE INSTITUTE Gretchen Greene
FLAMES – an overview Features & points to be aware of, relevant to the observer. In addition to p2pp, there is FPOSS. Layout of FLAMES; link to GIRAFFE.
Delivering Business Value Faster
Presentation transcript:

GSC-II Plate Processing Pipeline Status C. Loomis, C. Sturch, F. Guglielmetti

Pipeline Status Update GSC2 export activities. POSSI IIIaE processing. Database recalibration-load-match. Future processing activities.

Pipeline Export Activities GSCII2.1.1 release nd epoch Schmidt plates 1.1 x 10 9 detections Comprehensive list and detailed information

Pipeline Export Activities -Recalibration pipeline tasks Catalog staging Photometric & astrometric calibrations ftp transfer of recalibrated headers to NT Apply Calibrations run on NT EXPORT

Recalibration (Cont…) Photosol

POSSI Pipeline Processing 99 % of POSSI red fields completed. Processing extends down to –30 degree zone. (Includes new 15  scans for plates south of –18 degrees that were not scanned for DSS1). Pipeline changes for POSSI survey. 1.Tasks requiring area inputs scaled to match the lower resolution of POSSI scans. 2.Detection threshold above sky raised from 3  to 4  to eliminate false detections from higher grain noise. 3.Cubic model for plate solution (telescope configuration).

Recalibration and database loading Short term goal: 2 colors all sky (roughly 3510 fields, excluding equatorial overlap fields) Surveys: POSSII IIIaF, POSSII IIIaJ, SES IIIaF, SES equatorial extension IIIaF, SES galactic plane IIIaF (4 min exposure), SERC IIIaJ, SERC galactic plane IIaD (5 min exposure) Photometric and astrometric recalibration for remaining fields. Combined database loading and matching to run sequentially upon release to database. Coupled with image and processing file loading into HSM (OOP, CIM, FPA, SKY, MCO…)

Recalibration and Database Loading (Cont…) Since mid-August recalibrated loaded and matched 663 plates. Total plates in COMPASS to Database plates by survey

Recalibration and database loading Improve efficiency by running concurrent processing and automation of data handling processes. Weak link is need for human intervention and decision about calibrations. Restore and process on VMS (30 to 60 min per plate) QA and Accept ? ftp,HSM ingest, database load, plate matching. 60 min per plate (depends on detections and number of overlap regions). Vms backup Oopindex Findcat, act,ty2,ppm,toc, gp2, Refraction, Photosol, plate_solution, Apply_calibrations Daemon process written in Perl and running c++ and fortran Under WINNT

Future activities -Processing: 1.Additional surveys such as POSSII IV-N, SES IV- N,POSSI IIIaO, Palomar quickV IIaD 2.Ingest of image and processing data into HSM for future reprocessing. 3. Global database recalibrations, iterating over the whole sky.