Astro-WISE and pre-calibrated data and external catalogs For the Astro-WISE consortium Gijs Verdoes Kleijn John McFarland.

Slides:



Advertisements
Similar presentations
ASTRO-WISE Science “A new approach to astronomical archiving & researching for the data- flooded era” Gijs Verdoes Kleijn OmegaCEN, Kapteyn Astronomical.
Advertisements

Chapter 26. Copyright 2003, Paradigm Publishing Inc. CHAPTER 26 BACKNEXTEND 26-2 LINKS TO OBJECTIVES Create a Form Template Create a Form Template Fill.
May 2013 Expanded Academic ASAP Gale’s Premier Database for All Academic Disciplines.
NOAO/Gemini Data workshop – Tucson,  Hosted by CADC in Victoria, Canada.  Released September 2004  Gemini North data from May 2000  Gemini.
Lofar Survey workshop 8/3/2007 Lofar -Surveys and AstroWise OmegaCEN NOVA – Kapteyn Institute -RuG 8 March 2007 Lofar Survey Workshop OmegaCEN NOVA – Kapteyn.
NWO Meeting 24 May 04 a Virtual Survey System AstroWise- OmegaCAM – Lofar – Euro-VO a Virtual Survey System AstroWise- OmegaCAM – Lofar – Euro-VO Edwin.
XLDB -Lyon Aug 2009 Astro-WISE -> TarGet OmegaCEN – TarGet Kapteyn Astronomical Institute University of Groningen 28 August 2009 UU OmegaCEN – TarGet Kapteyn.
PAWN V0.7 University of Maryland Institute for Advanced Computer Studies.
VO-lecture Valentijn 17/1/2007 Astro-Wise Wide Field imaging Facilitate: handling, calibration, quality control, pipelining, user tuned research, archiving,
Modern Information Retrieval Chapter 1 Introduction.
Collaborative Information Systems for Student Projects Chapter Extension 2.
Overseas Library Catalog – Advanced Search Overseas Library Catalog Advanced Search by Keywords: “mass media” & “Middle East”
Supplementary Training Modules on Good Manufacturing Practice
Data provenance in astronomy Bob Mann Wide-Field Astronomy Unit University of Edinburgh
QC | Slide 1 of 16 June 2006 Good Practices for Quality Control Laboratories P art 4: Inspecting the laboratory Supplementary Training Modules on Good.
1 LBT Web Services and Scheduler We present several LBT web tools to plan organize and schedule observations. Authors: Stefano Gallozzi Diego Paris Adriano.
Commissioning the NOAO Data Management System Howard H. Lanning, Rob Seaman, Chris Smith (National Optical Astronomy Observatory, Data Products Program)
GAUDI Ground-based Asteroseismology Uniform Database Interface E. Solano Bases de données en spectroscopie stellaire. Paris.
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
Astro-Wise workshop Nov 2005 Astro-Wise workshop Lorentz center Nov 2005 Munchen, Napoli, Paris, Bonn, Bochum, Nijmegen, Leiden, Groningen Lorentz.
Information Management Programme Management Managing Finances, Resources, and Information.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
Data Management Subsystem Jeff Valenti (STScI). DMS Context PRDS - Project Reference Database PPS - Proposal and Planning OSS - Operations Scripts FOS.
API, Interoperability, etc.  Geoffrey Fox  Kathy Benninger  Zongming Fei  Cas De’Angelo  Orran Krieger*
Planetary Science Archive PSA User Group Meeting #1 PSA UG #1  July 2 - 3, 2013  ESAC PSA Archiving Standards.
Bank Reconciliation Process
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks LOFAR Archive Information System Kor Begeman.
SDD/DFS R. Hook. VLT 2 nd Generation Instrumentation Pipelines, 18 Apr Richard Hook Sampo & ESO Reflex background, status and plans.
AstroWise((G)A)VO Meeting 6 May 2004 ASTRO-WISE- federation OmegaCEN AstroWise a Virtual Survey System OmegaCAM – Lofar – AstroGrid –((G)A) VO AstroWise.
SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr General Introduction Yves Jung.
DC2 Post-Mortem/DC3 Scoping February 5 - 6, 2008 DC3 Goals and Objectives Jeff Kantor DM System Manager Tim Axelrod DM System Scientist.
Data Curation Workflow
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Integration of Astro-WISE with Grid storage.
EURO-VO workshop 1 July 2005 A Virtual Survey SYSTEM Astro-Wise NOVA/Kapteyn – OA Capodimonte ESO – Terapix – US Munich/MPE National WFI datacenters NL-I-D-Fr/ESO.
Virtual Survey System sept 04 ASTRO-WISE- federation OmegaCEN AstroWise a Virtual Survey System OmegaCAM – Lofar – AstroGrid –((G)A) VO AstroWise a Virtual.
VO & Astro-Wise A.Belikov OmegaCEN
The Analysis of Binary Inspiral Signals in LIGO Data Jun-Qi Guo Sept.25, 2007 Department of Physics and Astronomy The University of Mississippi LIGO Scientific.
Prologue Major VO objective: “Data at your fingertips” (Lawrence) VO-user’s desire: “going backstage” –Data quality assessment by user (talk Gilmore)
The Global Pool The Global Pool AstroWise Workshop AstroWise Workshop list of active and potential calibrators.
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Pipeline Calibration Goals of CALCOS Association Table Input and Output Files High Level.
What the Data Products Program Offers Users Todd Boroson Dick Shaw Presentation to NOAO Users Committee October 23, 2003.
Photometric calibration in Astro-WISE For the Astro-WISE consortium: Gijs Verdoes Kleijn OmegaCEN/Kapteyn Groningen University (and Philippe Heraudeau,
DEVELOPMENT OF A SPECTROPHOTOMETRIC DATA REDUCTION PIPELINE ARIZONA SPACE GRANT CONSORTIUM NAU/NASA SPACE GRANT RYAN BUCKINGHAM NORTHERN ARIZONA UNIVERSITY.
HARPS Data Flow System Christophe Lovis Geneva Observatory HARPS-N PDR, 6-7 December 2007, Cambridge MA.
SDD/DFS A. Modigliani VLT 2 nd Generation Instrumentation Pipelines, 19 Apr ACCEPTANCE TESTS Andrea Modigliani.
COS PIPELINE CDR Jim Rose July 23, 2001OPUS Science Data Processing Space Telescope Science Institute 1 of 12 Science Data Processing
Ray Plante for the DES Collaboration BIRP Meeting August 12, 2004 Tucson Fermilab, U Illinois, U Chicago, LBNL, CTIO/NOAO DES Data Management Ray Plante.
AstroGrid NAM 2001 Andy Lawrence Cambridge NAM 2001 Andy Lawrence Cambridge Belfast Cambridge Edinburgh Jodrell Leicester MSSL.
Simplified Experiment Submit Proposal Results Excited Users Do Expt Data Analysis Feedback.
Data Processing Workshop NHSC, Pasadena, CA 02 – 04 February 2011 Stephan Ott VG #1 PACS Overview and new developments in.
Data Processing & Quality Control Group [ Faculty Meeting Sept Data Processing & Quality Control Quality Control Group head: Reinhard.
EPOXI HRI-IR C/ISON (2012 S1) Calibrated Spectra Second Review (Sitko) CATALOG: Contains ASCII text files on each science target (9P, 103P, Garradd, ISON,
WFC3 PIPELINE CDR Jim Rose October 16, 2001OPUS Science Data Processing Space Telescope Science Institute 1 of 13 Science Data Processing
The OmegaCAM photometric systems. OmegaCAM calibration plan (1) Basic OmegaCAM terminology : ● Key bands vs User bands ● Key band -> User band transformation.
Gijs Verdoes Kleijn Edwin Valentijn Marjolein Cuppen for the Astro-WISE consortium.
EMR-LINK Auto-complete troubleshooting guide. Auto-completion: 1.A feature enabled within Centricity when Ignis installs a bi-directional interface and.
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Monthly Status Review 15 September 1999.
If screen is locked press: Ctrl*Alt*Delete Login: Your Sharepoint Login & Password Go to: Sharepoint then Construction Administration Go to: Drawing Review.
Nh-x-mvic-2-plutocruise-v1.0 nh-x-mvic-3-plutocruise-v1.0 Matthew Knight (lead reviewer)
Astro-WISE. an Astronomical Wide-field Imaging System for Europe E v e r y o n e John P. McFarland OmegaCEN, Kapteyn Astronomical Institute Groningen,
26th October 2005 HST Calibration Workshop 1 The New GSC-II and it’s Use for HST Brian McLean Archive Sciences Branch.
REDCap vs Qualtrics - Similarities
The secondary standards programme for OmegaCAM at the VST Gijs Verdoes Kleijn, Ronald Vermeij & Edwin Valentijn For the Astro-Wise consortium OmegaCEN.
A step-by-Step Guide For labels or merges
SOAR Data Reduction Pipelines
A.Zanichelli, B.Garilli, M.Scodeggio, D.Rizzo
ACL SCREEN Start with Standard ACL Project Screen.
ASTRO-WISE workshop Nov 03
Commissioning Data – Who and Why?
Flowchart Symbols DFD Symbols Processes Input/Output Storage
Presentation transcript:

Astro-WISE and pre-calibrated data and external catalogs For the Astro-WISE consortium Gijs Verdoes Kleijn John McFarland

Why use Astro-WISE? L arge surveys, s mall projects New paradigm : full data lineage (L+s) – Inspect & verify back to raw data – automatic workflow Adaptable data reduction pipeline available (L+s) Share with your team (L+s) – Workload: data reduction, quality flagging Release data from end-product to raw data – share with team+flag=share with world Benefit from other’s existing work (L+s) – Calibration data – Science data, – Catalogs (cross-correlated, also SDSS, 2Mass, UKIDSS, etc) Computing power, storage space (L)

Where enter Astro-WISE system?

Getting in: ingestion HeaderTranslator: entry point of an instrument into Astro-WISE – makes header keywords uniform and provides information about instrument, detectors, and filters For pre-calibrated data: – Need to construct calibration objects AstrometricParameters PhotometricParameters WeightFrame

Using Astro-WISE 1.It’s about more than iraf/midas/idl/gipsy – Sharing, data-management, survey management – Not only fits files and headers 2.You need to make paradigm shift: – As clear as possible: documentation – As intuitive as possible 3.You need to know the things (without which this shift is not possible) – python – Object oriented+database approach to scientific analysis and data calibration and reduction 4.2 & 3 often require close user-developer interaction

Roadmap for ingesting an instrument John You get a HeaderTranslator template User edits: – Filter information – Chip information – Instrument information User checks HeaderTranslator on instrument data John verifies User does test ingestion (MyDB)

Show Header Translator – MPG-ESO/WFI – HST/ACS