Progress report on LSS Catalogue web site and database  (2) which database tables ? on which sky area ? who can access them ?  (3) what path did X-ray,

Slides:



Advertisements
Similar presentations
Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki
Advertisements

Organisation Of Data (1) Database Theory
VA. ACL USER’S GROUP Functions Intermediate to Advanced April 21, 2009 Kimberly M. Taylor, CPA, CISA Chesterfield County, VA.
Athabasca University Under Development for COMP 200 Gary Novokowsky
Supernova Remnants in the ChASeM33 X-ray Observations of M33 Knox Long, Bill Blair, Frank Winkler, Terry Gaetz, David Helfand, Jack Hughes, Kip Kuntz,
NOAO/Gemini Data workshop – Tucson,  Hosted by CADC in Victoria, Canada.  Released September 2004  Gemini North data from May 2000  Gemini.
XMM-Newton 1SCI-SA Populating the CCF Immediate needs are driven by the requirement to test the SAS over the summer before public release. (A release called.
Slews: Fast and Slow. Fast Slew Status  XMMSL1: catalogue of slew sources released May 2006 Revs , 5180 detections, 2692 Revs , 5180 detections,
VISTA pipelines summit pipeline: real time DQC verified raw product to Garching standard pipeline: instrumental signature removal, catalogue production,
X-ray Source-Galaxy Angular Cross-Correlation function in the COSMOS Field Takamitsu Miyaji, Richard E. Griffiths (Carnegie Mellon Univ.) COSMOS XMM-Newton.
The XMM slew survey: status and future Richard Saxton, Andrew Read, Michael Freyberg, Bruno Altieri.
UCL, Sept 16th 2008 Photometric redshifts in the SWIRE Survey - the need for infrared bands Michael Rowan-Robinson Imperial College London.
Attribute Data Input and Management
Comparing protein structure and sequence similarities Sumi Singh Sp 2015.
User guide Harris Broadcast May How to use Broadcast Go to: Click on broadcast.
The XMM-Newton hard band wide angle Survey Nicoletta Carangelo and Silvano Molendi (IASF-MI(CNR)) Epic Consortium Meeting Palazzo Steri, Palermo,
FALL 2012 DSCI5240 Graduate Presentation By Xxxxxxx.
Overview of Mini-Edit and other Tools Access DB Oracle DB You Need to Send Entries From Your Std To the Registry You Need to Get Back Updated Entries From.
Commissioning the NOAO Data Management System Howard H. Lanning, Rob Seaman, Chris Smith (National Optical Astronomy Observatory, Data Products Program)
Year End Processing Financial Management 1Freedom Systems - Year End Processing Webinar WELCOME TO THE YEAR END PROCESSING WEBINAR WE WILL BE WITH YOU.
SNLS : Spectroscopy of Supernovae with the VLT (status) Grégory Sainton LPNHE, CNRS/in2p3 University Paris VI & VII Paris, France On behalf of the SNLS.
CREATE THE DIFFERENCE Normalisation (special thanks to Janet Francis for this presentation)
IAGAP Access Database A Tutorial. Databases There are several databases available from the IAGAP Project. There are several databases available from the.
True OMR Second Darkest Mark Detection For Erasure Analysis.
The GAVO Cross-Matcher Application Hans-Martin Adorf, Gerard Lemson, Wolfgang Voges GAVO, Max-Planck-Institut für extraterrestrische Physik, Garching b.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
MS Word – Mail Merge Basic Steps Create Letter/Labels general information Create Excel File with variable Data Link Files through Mail Merge in Word Print.
WPM What’s behind the icon? Work Programme Management.
Luminosity and Mass functions in spectroscopically-selected groups at z~0.5 George Hau, Durham University Dave Wilman (MPE) Mike Balogh (Waterloo) Richard.
06/02/2006 M.Razzano - DC II Closeout Meeting Pulsars in DC2 preliminary results from an “optimized” analysis Gamma-ray Large Area Space Telescope Massimiliano.
© Logicalis Group Using DB2/400 effectively. Data integrity facilities Traditional iSeries database usage Applications are responsible for data integrity.
RADIO OBSERVATIONS IN VVDS FIELD : PAST - PRESENT - FUTURE P.Ciliegi(OABo), Marco Bondi (IRA) G. Zamorani(OABo), S. Bardelli (OABo) + VVDS-VLA collaboration.
Analysis of Complex Proteomic Datasets Using Scaffold Free Scaffold Viewer can be downloaded at:
(Spring 2015) Instructor: Craig Duckett Lecture 10: Tuesday, May 12, 2015 Mere Mortals Chap. 7 Summary, Team Work Time 1.
A look at XMM slew data Richard Saxton, Andrew Read, Michael Freyberg, Bruno Altieri, Puri Munuera.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
Swift/BAT Census of Black Holes Preliminary results in Markwardt et al ' energy coded color.
PIRSF Classification System PIRSF: Evolutionary relationships of proteins from super- to sub-families Homeomorphic Family: Homologous proteins sharing.
IPHAS EDR: Where are we? Catalogues up to January 2006 ingested Catalogue totals: Rows: 430M Gb: 400Gb Images are available.
The XMM SSC Bright Source sample * November 2006 Report to the SSC XID R. Della Ceca A. Caccianiga T. Maccacaro P. Severgnini F. Cocchia * on behalf of.
The Global Pool The Global Pool AstroWise Workshop AstroWise Workshop list of active and potential calibrators.
Initial Results from the Chandra Shallow X-ray Survey in the NDWFS in Boötes S. Murray, C. Jones, W. Forman, A. Kenter, A. Vikhlinin, P. Green, D. Fabricant,
Principles of Database Design, Conclusions MBAA 609 R. Nakatsu.
X-ray sky survey of bright, serendipitous sources with 2XMMi at the AIP Speaker: Alexander Kolodzig Origin: Humboldt-Uni Berlin, Germany Institute:AIP.
M. Pilar Esquej, XMM-Newton European Space Astronomy Centre Page 1 Status of the XMM-Newton Slew Survey P. Esquej, R. Saxton, B. Altieri, A. Read, M. Freyberg,
Data Verification and Validation
1 eCoRepair New Release Slide Expanded view of Circuit Details Change to Circuit Looping text4 View of closed faults up to 30 days old5 - 7 Minor.
Transportation Agenda 77. Transportation About Columns Each file in a library and item in a list has properties For example, a Word document can have.
Data Management Seminar, 8-11th July 2008, Hamburg WinDEM- Verification Checks Part I.
Data Management Seminar, 8-11th July 2008, Hamburg WinDEM - Merge Files.
The VVDS FINAL CATALOG Bianca Garilli, for the whole VVDS team.
Galaxy mass-to-light ratios at z> 1 from the Fundamental Plane: measuring the star formation epoch and mass evolution of galaxies van der Wel, Rix, Franx,
MICROSOFT ACCESS – CHAPTER 5 MICROSOFT ACCESS – CHAPTER 6 MICROSOFT ACCESS – CHAPTER 7 Sravanthi Lakkimsety Mar 14,2016.
Input, Output and Processing.. What data needs to be input into the system? Identify the sources – i.e. where does the data come from? What is the volume.
HADI Tutorial Void Inspection Contents 1.Basic Void Inspection Procedure 2.Smooth ROI 3.Background Processing (Flatten BG) 4.Thresholding (Void.
Validation of HLA Source Lists Feb. 4, 2008 Brad Whitmore 1.Overview 2.Plots 3.Summary.
CellExpress Tutorial A Comprehensive Microarray-Based Cancer Cell Line and Clinical Sample Gene Expression Analysis Online System :8080 NTU.
JWST FGS Guide Star Studies
Indexing and hashing.
The Hubble Legacy Archive (HLA) Slitless Spectroscopy Project
Post-Operation Status: Archiving, Calibration & Science
Institute of Cosmos Sciences - University of Barcelona
PAPER 7: The IR-Radio Correlation in High-Mass YSOs
Inside Module 10 Editing TurboIMAGE Datasets Page
GDSS – Digital Signature
CSc4730/6730 Scientific Visualization
SQL for Cleaning Data Farrokh Alemi, Ph.D.
Zoning Map Modernization with GIS
X-ray high resolution spectra in the VO: the case of XMM-Newton RGS
Presentation transcript:

Progress report on LSS Catalogue web site and database  (2) which database tables ? on which sky area ? who can access them ?  (3) what path did X-ray, SWIRE and CFHTLS data go through ?  (4) the ingestion of X-ray data  SWIRE and CFHTLS datasets  (5) access details and work done in the Milan database  (6) basic numerology  (7) where are we on the sky ?  (8) more numerology  (9) source density plots (required for identification checks)  identification  (10) the past (VVDS 4  )  (11) the present (XMDS)  (12) the future  (13) open issues LSS Brera Sep Lucio Chiappetti IASF MI

Where are we and what do we have ? LSS Brera Sep Lucio Chiappetti IASF MI X-rayoptical/IR/spectroother/external G fields + B fields VVDS field and also CFHTLS fields SWIRE radio clusters external other xmdsepic january03 may03 nov04 may05 virphot bad loiano vimos sacphot specfup d1 w1 swcatalogues swire virradio radio loto xlssc simbad ned usno galex?? who?

what path did data go through ? LSS Brera Sep Lucio Chiappetti IASF MI Saclay nov04 FITS CFHTLS D1 R presurvey may05 FITS CFHTLS W1 IPAC Milan ( repository ) data products swirexlss swirexd1 swirexpresurvey data products swirexmay05 swirexw1 Milan (DB) nov04 swire d1 may05 w1 SWIRE r1 SWIRE r band merging and ingestion 2 read all in scratch but keep only within 9” from XMDS 3 read all in scratch but keep only within 9” from XMDS and may05 non spurious within 10 arcmin offaxis non spurious any offaxis

The ingestion of X-ray data XMDS based on simultaneous detection (Baldi) in 5 bands data below probability threshold are not ingested 1322 detections in G fields nov04 based on independent detections (Xamin) in 5 bands outer part of FOV not considered by det. mask may05 also based on independent detections in 2 bands (B CD) only outermost part of FOV still not considered all sources supplied are ingested, need to filter on detection ML < 20 band merging at ingestion, within 10 arcsec inclusive P/E classification totalnon-spuriousnon- sp. & G fields original detections band B original detections band CD detections in merged file of which detected in both bands LSS Brera Sep Lucio Chiappetti IASF MI

SWIRE and CFHTLS datasets  full ASCII gz files supplied by IPAC based on correlations done there will not be ingested in database as such, available for retrieval  files with no X-ray (full SWIRE set) available only to Franco-Canado-SWIRians they are just hosted blindly in Milan, 'plain' LSS people cannot access (MoU1!)  unfortunately no direct X-ray to CFHTLS correlation supplied (but MoU2 ?)  Ingested blindly in short term temporary table (but need density plots ?) separately all SWIRE or all CFHTLS from a given date (Feb, Jul)  then extract all sources within 9” from X-ray position into a database table (access ?) only XMDS positions for Feb, also may05 (full) positions for Jul  SWIRE data affected by “tiling duplications”  Feb and Jul SWIRE data from different processing (no unique id's) Should we abandon Feb SWIRE release ? No Feb/may05 done, 50 XMDS kept  Feb D1 field re-observed by some of the 11 Jul (W1) fields D1 has ugriz+I, W1 has ugriz (10 fields) or ugiz (1 field) LSS Brera Sep Lucio Chiappetti IASF MI

Numerology LSS Brera Sep Lucio Chiappetti IASF MI X-ray src may05 given to IPAC of which only in band CD IPAC returned records swireXmay of which distinct sources IPAC records in swirexw1 11 fields of which distinct sources W1 ingested within 9” from xmds W1 ingested within 9” from may05 of which distinct non-spurious surviving close to xmds only swire ingested within 9” from xmds swire ingested within 9” from may05 of which distinct non-spurious surviving close to xmds only kept swire entries kept w1 entries X-ray src nov04 given to IPAC of which not in bands B nor CD ? IPAC returned records swireXlss of which distinct sources IPAC records in swirexd1 of which distinct sources IPAC records in swirexpresurvey of which distinct sources D1 ingested within 9” from xmds D1 ingested within 9” from may05 surviving close to xmds only swire ingested within 9” from xmds of which prob. not in July dataset (the rest has been removed) kept d1 entries X-rayFebruary 2005 (D1) datasetSWIREOptical X-rayJuly 2005 (W1) datasetSWIREOptical

where are we on the sky LSS Brera Sep Lucio Chiappetti IASF MI

More numerology LSS Brera Sep Lucio Chiappetti IASF MI counterpartsdistinct X-raycounterpartsdistinct X-ray XMDS with respect to may05 non spurious w.r.t. no counterpart within 9” within 9” no counterpart within 6" within 6” within 4” within 2” no counterpart within 9" within 9" no counterpart within 6" within 6" within 4" within 2" no counterpart within 9" within 9" no counterpart within 6" within 6" within 4" within 2" D1 W1 SWIRE no counterpart within 9” within 9” no counterpart within 6" within 6” within 4” within 2” no counterpart within 9" within 9" no counterpart within 6" within 6" within 4" within 2" no counterpart within 9" within 9" no counterpart within 6" within 6" within 4" within 2" D1 W1 SWIRE

“official” density functions wanted absolute area normalization ? completeness ? analytic fit ? LSS Brera Sep Lucio Chiappetti IASF MI

Identification : the past (the XMDS/VVDS 4  catalogue) (Chiappetti et al A&A 439, 413) Tables used : xmdsepic [may03]virphot+ [sacphot] virradio radio [loto][specfup] simbad ned Sample restricted to : S/N > 4  in one+ band and VVDS thumbnail Take all possible combinations (after coarse precorrelations) Primitive automatic cleanup on id's Manual cleanup checking distances (& visual inspection) Manual first ranking of identifications (visual inspection) Evaluation and application of astrometric correction (eposcorr) Manual final ranking of identifications (visual inspection) Verification a posteriori using chance probability <- density No spectroscopy data used LSS Brera Sep Lucio Chiappetti IASF MI valid ranks 364 sources

Identification : the present Tables used : xmdsepic may05 virphot+ vimos [sacphot] d1 w1 swire virradio radio [loto] [specfup] xlssc simbad ned usno No sample restriction (repeat also old; use XMDS as reference case) i.e. do XMDS any , inside and outside VVDS Take all possible combinations (after coarse precorrelations) Extensive automatic cleanup on id's, distances etc. Automatic pre-ranking of identifications (probabilities) Manual cleanup and re-ranking (& visual inspection when possible) Self-astrometric correction to be done for outer G fields (USNO based preliminary correction done for may05 not used forXMDS yet) TBD final ranking of identifications (visual inspection ?) LSS Brera Sep Lucio Chiappetti IASF MI TBD

Identification : the future First complete what is missing from the present ! Use all current tables with may05 (or newer) as “pivot” Define sample restrictions once forever Work similarly to the full XMDS sample, also exploiting the work already done for the XMDS=may05 X-ray sources USNO astrometric correction to be verified and improved Do we need any images for visual inspection ? LSS Brera Sep Lucio Chiappetti IASF MI

Open issues  which old datasets (X, opt, SWIRE) to drop ?  open d1 w1 swire tables ; how ?  official density functions wanted ; who ?  new IPAC release ?  number of bands, HRs, flux ratios as AGN signatures  usage of photometric z ? which data and models ? new Saclay release ? what changes ? B-CD distance ? new recipes ? S/N ? flux CF ?  usage of GALEX data ? LSS Brera Sep Lucio Chiappetti IASF MI

Oh what a mess 1322 X-ray sources get 9508 tentative counterparts after auto cleanup have a single counterpart 272 have just , 147, 42, have 3,4,5, and so on and so forth, two have 96, one has 540, and one has 912 ! LSS Brera Sep (aux) Lucio Chiappetti IASF MI source #755 “distinct” multiplicities (well known from XMDS/VVDS) auto cleanup on distance no cleanup 8 entries : 2 rank 2, 4 rank 3 w/ optical... radio source at 8” and NED NVSS also as rank 3 may05 1 virphot 5 sacphot 5 both 5 d1 2 3 w1 2 3 both swire virradio 4 radio 3 12 simbad 1 ned 4 48 usno 0 1 specfup 0 1 xlssc 0 1 loto no nulls w/ nulls 3 multiple components of VIRMOS1.4GHz J , also in NED,single in SIMBAD, also “Leiden” radio source is multiple, NED has also an NVSS : all far, 8” to 19”

space for title space for text LSS Brera Sep Lucio Chiappetti IASF MI