ASTRO-WISE Science “A new approach to astronomical archiving & researching for the data- flooded era” Gijs Verdoes Kleijn OmegaCEN, Kapteyn Astronomical Institute, University of Groningen

The OmegaCEN team at Kapteyn Edwin Valentijn (lead) Kor Begeman Danny Boxhoorn Erik Deul (Leiden) Ewout Helmich Philippe Heraudeau John MacFarland Michiel Tempelaar Gijs Verdoes Kleijn Ronald Vermeij Willem Jan Vriend (Lofar) Kovac, Schneider, Sikkema (PhDs)

Archiving your vision Camera in front of eye: –resolution ~5/60 degree (my eye) –Field of view 90 2 degree –2x2 pixel sampling per resolution element –#pixels= –Dynamic range: ~ : take 2 16 –One image ~9Mbyte –One image/sec for 70years: 18Pbyte Storage cost (at 0.33euro/Gbyte)~ 6million euro –~ neurons Brain does something smart… ……intensive linking is a key ingredient.

If vision archive were implanted for re-analysis Applying improved analysis –(Raw data: improved calibrations as well) Analysis of previously deemed uninteresting parts of data Variability analysis if archives from different persons are combined –Denser coverage in space+time –3D construction of 2D view If fly’s UV eye archives are added: –Same object at different wavelengths Aim: exploit data for purposes or in ways not yet conceived flexible, internally linked, information system required

Key properties for intelligent information system Adaption: facilitate changes 1.Changes due to improved encoded methods 2.True physical changes of parameter values (e.g., change in instrument/atmospheric properties) 3.Improved insight in 2 or 3 Learning: “more and better” –take advantage of existing results (from you or others) –Fast (re-)reduction and (re-)analysis –From quick-look to high-quality results –Not only more but also better data over time: ‘accumulation of knowledge Anybody –Small Individual research projects –Large projects with many collaborators Everywhere: federation – federation of storage and computing capabilities Scalability –No limits due to design for storage, databse power/storage processing power,… A federated ‘brain’ interacting with many users

The A stro- W ise E nvironment a new archive/research tool for astronomical wide-field imaging Status environment is working First paper using Astro-Wise based results is out Expansions and improvements on-going

The A stro- W ise E nvironment User Python prompt (awe>) Web interfaces CPUs +algorithms Database metadata of images derived data from images (source lists)derived data from images (source lists) =‘spider spinning AWE web’ Contains ALL input/output Data server: images calibration & science From raw to reduced “Archiving” “Interpreting” “Processing”

Key properties for intelligent information system Adaption: facilitate changes 1.Changes due to improved encoded methodsChanges due to improved encoded methods 2.True physical changes of parameter values (e.g., change in instrument/atmospheric properties)True physical changes of parameter values (e.g., change in instrument/atmospheric properties) 3.Improved insight in 2 or 3Improved insight in 2 or 3 Learning –take advantage of existing results (from you or others)take advantage of existing results (from you or others) –Outdated not-yet existing dataOutdated not-yet existing data –Fast (re-)reduction and (re-)analysis –Not only more but also better data over time: ‘accumulation of knowledge‘ Anybody: –Each piece of information carries tag of ‘ownership’Each piece of information carries tag of ‘ownership’ –Small Individual research projectsSmall Individual research projects –Large projects with many collaboratorsLarge projects with many collaborators Anywhere: federation: –Own developed compute-grid and storage-grid –Nodes:active={Kapteyn, Bonn} almost={Munich, Paris,Naples}, in progress={Nijmegen, Leiden, Santiago}) Scalability: performance proportional to I/O speed and processing power. A federated ‘brain’ interacting with many users

Paradigm shift AWE“Classical” Information system Releases Dynamic archiveStatic archive ‘Continous’ releases (VO) Fixed releases End result ‘pulled’ via linking of all processing input/output Raw data ‘pushed’ through pipeline to end- product

Science projects with AWE PhDs Sikkema, Kovac, Schneider: galaxy surveys with WFI, WFC, MDM Test projects –Variable sources around CenA light curves (Δmag~0.03) for 2x10 4 objects around Centaurus A Valentijn –Asteroid detection Detections in WFI image from catalog of ~10 5 numbered asteroids Jeffrey Bout (student), myself Cen A 2dF Asteroids

Large public data projects with AWE using OmegaCAM at VST KIDS (PI: Kuijken): ESO Public Survey –Weak lensing; high-z QSOs; galaxy/cluster evolution; baryon oscillations; –5000 deg 2 u,g,r,i, ~500 nights KIDS North KIDS South

GTO science with AWE using OmegaCAM at VST OmegaWhite (PI: Groot): –discover Galactic Population of ultracompact binaries from periodic (<2hour) light curves –150 deg b=±5 o ; Sloan g’ 40sec exposures & additional ugriz’ coverage.

GTO science with AWE using OmegaCAM at VST OmegaTranS (Saglia; Snellen; Alcala) –Searching for planet transits (15-20 new transits expected in first year) –~1 order more powerful than OGLE-III

GTO science with AWE using OmegaCAM at VST VESUVIO (PIs: Valentijn, Capaccioli) –Galaxy/cluster evolution –Horologium Supercluster: 100 deg 2 medium deep (r'<25mag); ugriz –Hercules Supercluster: 12 deg 2 deep ugriz+Hα

Key ingredients to achieve the Astro-Wise environment –Strict global data acquisition and processing model –data model -> object model –storing all I/O in single (distributed) database –Database environment exploits OOP inheritance (Python) Complete linking (associations, references)

Conclusions New analysis environment for wide-field imaging in operation by OmegaCEN –Key ingredient: dynamic database Containing all I/O Fully linked data Versatile: could be used for other kinds of data –Collaboration with LOFAR on-going Large science projects with AWE when OmegaCAM starts operations (early 2007) To find out more –Visit : –download/get now 2 page overview article

Contribution to LOFAR project