1. 26/03/2003 OmegaCAM: The 16k x 16k Survey Camera for the VST Observing and data reduction a Virtual Survey System Observing and data reduction a Virtual Survey System Edwin A. Valentijn

2. 26/03/2003 People involved The Netherlands Kapteyn Institute: J-W. Pel, K. Begeman, D.R. Boxhoorn, E. Valentijn, K. Kuijken Sterrewacht Leiden: R. Rengelink, E.R. Deul Germany Universitäts-Sternwarte München: R. Bender, L. Greggio, R. Häfner, U. Hopp, H. Kravkar, W. Mitsch, B. Muschielok, M. Neeser, R. Saglia Universitäts-Sternwarte Göttingen: R. Harke, H. Nicklas, W. Wellem Sternwarte der Universität Bonn: K. Reif Italy Astronomical Observatory of Capodimonte - Napoli: E. Cascone Osservatorio Astronomico di Padova: A. Baruffolo, E. Cappellaro, E. V. Held, H. Nazaryan, G. Piotto, H. Navarsadyan, L. Rizzi ESO D. Baade, A. Balestra, J-L. Beckers, C. Cavadore, C. Cumani, F. Christen, S. D'Odorico, S. Deiries, N. Devillard, C. Geimer, N. Haddad, G. Hess, J. Hess, O. Iwert, H. Kotzlowski, J-L Lizon, A. Longinotti, W. Nees, A. Renzini, J. Reyes Moreno, G. Sikkema, M. Tarenghi The Netherlands Kapteyn Institute: J-W. Pel, K. Begeman, D.R. Boxhoorn, E. Valentijn, K. Kuijken Sterrewacht Leiden: R. Rengelink, E.R. Deul Germany Universitäts-Sternwarte München: R. Bender, L. Greggio, R. Häfner, U. Hopp, H. Kravkar, W. Mitsch, B. Muschielok, M. Neeser, R. Saglia Universitäts-Sternwarte Göttingen: R. Harke, H. Nicklas, W. Wellem Sternwarte der Universität Bonn: K. Reif Italy Astronomical Observatory of Capodimonte - Napoli: E. Cascone Osservatorio Astronomico di Padova: A. Baruffolo, E. Cappellaro, E. V. Held, H. Nazaryan, G. Piotto, H. Navarsadyan, L. Rizzi ESO D. Baade, A. Balestra, J-L. Beckers, C. Cavadore, C. Cumani, F. Christen, S. D'Odorico, S. Deiries, N. Devillard, C. Geimer, N. Haddad, G. Hess, J. Hess, O. Iwert, H. Kotzlowski, J-L Lizon, A. Longinotti, W. Nees, A. Renzini, J. Reyes Moreno, G. Sikkema, M. Tarenghi

3. 26/03/2003 Paranal July 2004

4. 26/03/2003 VLT Survey Telescope-VST –Alt-AZ - Cassegrain –aperture 2.610 m –corrected FOV 1.47 degree –lens corrector: U - z –Atmospheric disp. correct.: B -z –f/5.5 –scale 14.266 arcsec/mm – CCD pixel size: 15 um – 0.214 arcsec/pixel –image quality: 80% EE –two-lens: 1.70 pixel –ADC: 1.77 - 2.18 pixel –Alt-AZ - Cassegrain –aperture 2.610 m –corrected FOV 1.47 degree –lens corrector: U - z –Atmospheric disp. correct.: B -z –f/5.5 –scale 14.266 arcsec/mm – CCD pixel size: 15 um – 0.214 arcsec/pixel –image quality: 80% EE –two-lens: 1.70 pixel –ADC: 1.77 - 2.18 pixel

5. 26/03/2003 VST factory - Napoli

6. 26/03/2003 Detectors Science array 1 x 1 degree, 32 CCDs –15 m pixels – 0.21 arcsec/pixel –Marconi (former EEV) 2k x 4k –16k x 16k pixels Auxiliary CCD’s – 4 CCDs –For guiding –Image analysis Science array 1 x 1 degree, 32 CCDs –15 m pixels – 0.21 arcsec/pixel –Marconi (former EEV) 2k x 4k –16k x 16k pixels Auxiliary CCD’s – 4 CCDs –For guiding –Image analysis

7. 26/03/2003 Focal plane - segmented

8. 26/03/2003 Filters Primary set: Sloan u’, g’, r’, i’, z’ high throughput interference Johnson B, V, Stromgren-v Segm H up to ~12000 km/s 658.8 665.5 672.2 678.9/ 10.7 nm –1100, 4200, 7300, 10400km/sec / 4900 km/sec Composite u’, g’, r’,i’ in four quadrants Segm Ly alpha z=2-3 372, 400, 450, 507nm / 8 nm Night sky leak CWL=851.8nm - 877.8nm /13nm Primary set: Sloan u’, g’, r’, i’, z’ high throughput interference Johnson B, V, Stromgren-v Segm H up to ~12000 km/s 658.8 665.5 672.2 678.9/ 10.7 nm –1100, 4200, 7300, 10400km/sec / 4900 km/sec Composite u’, g’, r’,i’ in four quadrants Segm Ly alpha z=2-3 372, 400, 450, 507nm / 8 nm Night sky leak CWL=851.8nm - 877.8nm /13nm

9. 26/03/2003 Sloan filter set

10. 26/03/2003 Throughput 10-sigma AB magni(point source, 3 arcsecaperture) Integration time u' g' r' i' z' Halpha 100 22.4 23.1 22.6 22.0 20.8 1.2E-15 1000 23.9 24.5 23.9 23.3 22.1 2.8E-16 10000 25.2 25.7 25.1 24.6 23.3 8.4E-17 10-sigma AB magni(point source, 3 arcsecaperture) Integration time u' g' r' i' z' Halpha 100 22.4 23.1 22.6 22.0 20.8 1.2E-15 1000 23.9 24.5 23.9 23.3 22.1 2.8E-16 10000 25.2 25.7 25.1 24.6 23.3 8.4E-17

11. 26/03/2003

12. Wide Field Imaging Science Provide targets for VLT ~60% of time through ESO’s OPC Individual programs –Supernovae, Lensing, Kuiper belt objects, Gamma ray, bursts, Microlensing, Brown dwarfs, High proper motion objects, Galactic halo objects, Quasars, AGNs Sky Surveys Long term archival research (10 yr mission) Science Cases –Finding exceptional single, rare objects –Statistics on large samples of objects Provide targets for VLT ~60% of time through ESO’s OPC Individual programs –Supernovae, Lensing, Kuiper belt objects, Gamma ray, bursts, Microlensing, Brown dwarfs, High proper motion objects, Galactic halo objects, Quasars, AGNs Sky Surveys Long term archival research (10 yr mission) Science Cases –Finding exceptional single, rare objects –Statistics on large samples of objects

13. 26/03/2003 Large Data Volume Wide-field imaging instruments, vast amounts of data –E.g.: VST = Southern sky (30 min exp, 300 nights/y) in 3 years. Large amount of data! 100 Tbyte of image data and Tbytes of source list data Wide-field imaging instruments, vast amounts of data –E.g.: VST = Southern sky (30 min exp, 300 nights/y) in 3 years. Large amount of data! 100 Tbyte of image data and Tbytes of source list data Science can only be archive-based Handling of the data is non-trivial –Pipeline data reduction –Calibration and re-calibration –Image comparisons and combinations –Working with source lists –Visualization Handling of the data is non-trivial –Pipeline data reduction –Calibration and re-calibration –Image comparisons and combinations –Working with source lists –Visualization ESO compliant ESO compliant } }

14. 26/03/2003 Concepts for solution Virtual Survey System Environment that provides systematic and controlled –Access to all raw and calibration data –Execution and modification reduction/calibration pipelines –Execution of source extraction algorithms –Archiving reduced data and source lists, or regenerates these dynamically –Can be federated to link different data centers Environment that provides systematic and controlled –Access to all raw and calibration data –Execution and modification reduction/calibration pipelines –Execution of source extraction algorithms –Archiving reduced data and source lists, or regenerates these dynamically –Can be federated to link different data centers Dynamical archive continuously grows, can be used for –small or large science projects –generating and checking calibration data –exchanging methods, scripts and configuration Dynamical archive continuously grows, can be used for –small or large science projects –generating and checking calibration data –exchanging methods, scripts and configuration Key functionality –Link back from source data to the original raw pixel data and calibration files Key functionality –Link back from source data to the original raw pixel data and calibration files

15. 26/03/2003 How to use this Deep multi-color fields –No need to take all data in one campaign –Combine data of particular quality, assess results –Select sources, visualize interesting ones, … 1-in-1,000,000 events spurious or not? Deep multi-color fields –No need to take all data in one campaign –Combine data of particular quality, assess results –Select sources, visualize interesting ones, … 1-in-1,000,000 events spurious or not? Large homogeneous surveys –E.g. weak lensing maps, cluster searches, star counts Large homogeneous surveys –E.g. weak lensing maps, cluster searches, star counts Variability (source list - or pixel based) –Proper motions (asteroids, nearby stars) –Flux variations Variability (source list - or pixel based) –Proper motions (asteroids, nearby stars) –Flux variations Monitor instrument (calibration files) Planning observations –View quality of existing data –Build on what already exists, add more filters, more exposure time, better seeing, … Planning observations –View quality of existing data –Build on what already exists, add more filters, more exposure time, better seeing, …

16. 26/03/2003 Keys -Solution Procedurizing –Data taking at telescope for both science and calibration data –Full integration with data reduction –Design –Data model (classes) defined for data reduction and calibration –View pipeline as an administrative problem Procedurizing –Data taking at telescope for both science and calibration data –Full integration with data reduction –Design –Data model (classes) defined for data reduction and calibration –View pipeline as an administrative problem

17. 26/03/2003 Observing Modes Dither Dither matching max. gap between arrays ~400 pixels –N pointings (N=5 is standard) –nearly cover all gaps in focal plane and maximizes sky coverage –Very complex context map –couple the photometry among individual CCDs. Dither matching max. gap between arrays ~400 pixels –N pointings (N=5 is standard) –nearly cover all gaps in focal plane and maximizes sky coverage –Very complex context map –couple the photometry among individual CCDs. –Dither with N = 5

18. 26/03/2003 Observing Modes Jitter Jitter matching the smallest gaps in CCDs ~5 pixels –optimizes for maximum homogeneity of the context map –observations for which the wide CCD gaps are not critical –all data from single sky pixel originates from single chip Jitter matching the smallest gaps in CCDs ~5 pixels –optimizes for maximum homogeneity of the context map –observations for which the wide CCD gaps are not critical –all data from single sky pixel originates from single chip

19. 26/03/2003 Observing Modes Stare and SSO Stare reobserving fixed pointing positions multiple times –main workhorse monitoring instrument and optical transients. Stare reobserving fixed pointing positions multiple times –main workhorse monitoring instrument and optical transients. SSO observing Solar System objects –non-siderial tracking and the auto guiding switched off. SSO observing Solar System objects –non-siderial tracking and the auto guiding switched off.

20. 26/03/2003 Strategies scheduling observing modes Standard –Single observations (one observing block) Deep –Long, multiple integrations –Selected atmospheric conditions –Several nights Frequent –Monitors same field –Timescales from minutes to months (overriding) Mosaïc –Maps areas of sky > 1 o Standard –Single observations (one observing block) Deep –Long, multiple integrations –Selected atmospheric conditions –Several nights Frequent –Monitors same field –Timescales from minutes to months (overriding) Mosaïc –Maps areas of sky > 1 o

21. 26/03/2003 Calibration procedures Sanity checks Quality control Calibration procedures Calibration procedures Image pipeline Source pipeline

22. 26/03/2003 Science Observations Photometric pipeline Bias pipeline Flatfield pipeline Image pipeline Source pipeline

23. 26/03/2003 Monitoring Photometric Calibration

24. 26/03/2003 Share the load AstroWise Survey System Processing –Hardware Beowulf processors – 32 (most cases) Multi Terabyte disks (10 – 100) –Data reduction Derive calibration Run image pipeline (1 Mpx/s) Processing –Hardware Beowulf processors – 32 (most cases) Multi Terabyte disks (10 – 100) –Data reduction Derive calibration Run image pipeline (1 Mpx/s) Archiving –Storage Images (100’s Tbyte), Calibration files (10 Tbyte) Source parameters (1-10 Tbyte) –Federate (network speed) 5 Mb/s (24 hours/day) full replication 200 Mb/s no replication, on-the-fly retrieval

25. 26/03/2003 Contents of federation Raw data –Observed images –Ancillary information Calibration results –Calibration files time stamped Reduced images –Single observation –Coadded images Software –Methods (pipelines) for processing calibration –Configuration files Source lists – catalogues –Extracted source information –Associated among different data objects Raw data –Observed images –Ancillary information Calibration results –Calibration files time stamped Reduced images –Single observation –Coadded images Software –Methods (pipelines) for processing calibration –Configuration files Source lists – catalogues –Extracted source information –Associated among different data objects

26. 26/03/2003 Concepts of federation Federation maintained by a single database- Oracle9i Full history tracking –of all input that went into result –providing on-the fly reprocessing Dynamical archive - Context as object attributes –Project: Calibration, Science, Survey, Personal –Owner: Pipeline, Developer, User –Strategy:Standard, Deep, Freq (monitoring), Mosaïc –Mode: Stare, Jitter, Dither, SSO –Time: Time stamping VO interface Software standards –Classes/data model/procedures –00 – inheritance/ persistency –Python scripts/ c-libraries USER Python Federation maintained by a single database- Oracle9i Full history tracking –of all input that went into result –providing on-the fly reprocessing Dynamical archive - Context as object attributes –Project: Calibration, Science, Survey, Personal –Owner: Pipeline, Developer, User –Strategy:Standard, Deep, Freq (monitoring), Mosaïc –Mode: Stare, Jitter, Dither, SSO –Time: Time stamping VO interface Software standards –Classes/data model/procedures –00 – inheritance/ persistency –Python scripts/ c-libraries USER Python

27. 26/03/2003

29. Schedule Hardware –3Tbyte + servers + 8 node cluster operational –Camera on telescope Q2 2004 –First run: May 2004 –Second run: July 2004 Software –Design – review Q2 2002- Done –WFI Prototype operational Q2 2003 –Basic operations – Q4 2003 –Evaluate and prepare for mass production 2004 –Qualify and populate 2005 –Deliver survey system – satellites Hardware –3Tbyte + servers + 8 node cluster operational –Camera on telescope Q2 2004 –First run: May 2004 –Second run: July 2004 Software –Design – review Q2 2002- Done –WFI Prototype operational Q2 2003 –Basic operations – Q4 2003 –Evaluate and prepare for mass production 2004 –Qualify and populate 2005 –Deliver survey system – satellites

30. 26/03/2003 Observing proposals Garanteed time NOVA 10% Call – 25 April--- see www.omegacam Super clusters- distant clusters Galactic structure –Weak shear, microlensing –Bulge 2dF, 100 Sq Degree, 10000 Sq Deg Deep field Lorentz center July 2003 Fall 2004 Garanteed time NOVA 10% Call – 25 April--- see www.omegacam Super clusters- distant clusters Galactic structure –Weak shear, microlensing –Bulge 2dF, 100 Sq Degree, 10000 Sq Deg Deep field Lorentz center July 2003 Fall 2004