1. XLDB -Lyon Aug 2009 Astro-WISE -> TarGet OmegaCEN – TarGet Kapteyn Astronomical Institute University of Groningen 28 August 2009 UU OmegaCEN – TarGet Kapteyn Astronomical Institute University of Groningen 28 August 2009 UU Edwin A. Valentijn

3. Astro-WISE information system – fully datacentric All data beyond pixel data is Metadata all pixel data data servers all Metadata database compute clusters / GRIDs all I/O to db all components scalable all components EU distributed All data beyond pixel data is Metadata all pixel data data servers all Metadata database compute clusters / GRIDs all I/O to db all components scalable all components EU distributed

4. OmegaCEN review 2008 VST - Virtual Survey Telescope

5. XLDB -Lyon Aug 2009 Centers – satellites Brasil

6. Astro-WISE paradigm “Classical” paradigm Forward chaining Target processing - Awe Backward chaining waterfall model TIER architecture User hunts upstream driven by input raw data Driven by query of user Process in pipeline workflow Process in bits and pieces on the fly Backward chaining Operators push data User pulls data Results in releases Provide information system Static archives – publish Dynamic archives –publish Internet Raw data - obsolete Raw data is sacred

7. TARGET diagram

8. Central role db All I/O via db: params, sources, events Objects persistent in db data server access by global filename=key Security, proprietary, privileges parallel processing Web services Synchronized real time National Nodes All I/O via db: params, sources, events Objects persistent in db data server access by global filename=key Security, proprietary, privileges parallel processing Web services Synchronized real time National Nodes XLDB -Lyon Aug 2009

9. the avalanche on-the fly re-processing for everything 5LS: 5 Lines Script Awe> prompt Python Trend analysis Awe > prompt All dependent bits are traced “tell_me_everything_tool Administration for parallel processing -compute GRID Connect to EGEE - Grid Global solutions astrometry/photometry Build–in workflow Fully user tunable – own provided script Context: projects/surveys, instruments, mydb Publish directly in EURO-VO on-the fly re-processing for everything 5LS: 5 Lines Script Awe> prompt Python Trend analysis Awe > prompt All dependent bits are traced “tell_me_everything_tool Administration for parallel processing -compute GRID Connect to EGEE - Grid Global solutions astrometry/photometry Build–in workflow Fully user tunable – own provided script Context: projects/surveys, instruments, mydb Publish directly in EURO-VO

10. Example 5LS # Find ScienceFrames for a ccd named ccd53 and filter Awe> q = (ReducedScienceFrame.chip.name == 'ccd53‘) and (ReducedScienceFrame.filter == ‘#841’) # From the query result, get the rms of the sky in image Awe> x = [k.imstat.stdev for k in q] # get the rms of the used Masterflat (dependency) Awe> y = [k.flat.imstat.stdev for k in q] # Make a plot Awe> pylab.scatter(x,y) # Find ScienceFrames for a ccd named ccd53 and filter Awe> q = (ReducedScienceFrame.chip.name == 'ccd53‘) and (ReducedScienceFrame.filter == ‘#841’) # From the query result, get the rms of the sky in image Awe> x = [k.imstat.stdev for k in q] # get the rms of the used Masterflat (dependency) Awe> y = [k.flat.imstat.stdev for k in q] # Make a plot Awe> pylab.scatter(x,y)

11. XLDB -Lyon Aug 2009 + WFCAM, MegaCAM, LBC, ISAAC, LOFAR

12. XLDB -Lyon Aug 2009 www.astro-wise.org

13. OmegaCEN review 2008 Web services Target processor

14. Quality view XLDB -Lyon Aug 2009

15. Extreme data lineage

16. TASK 24 TarGet XLDB -Lyon Aug 2009 2009-13 32M

17. UKIDSS workshop 2007 Astro-WISE – LOFAR LTA IBM- Blue Gene/P 5 Petabyte/yr - EoR - Survey IBM- Blue Gene/P 5 Petabyte/yr - EoR - Survey

18. XLDB -Lyon Aug 2009

19. Integration LifeLines 165.000 patients 30 yr METABASE MOLGENIS Healtcare Transaction Base (HTB/HL7) Molecular Care Physiology XLDB -Lyon Aug 2009

20. IJkdijk XLDB -Lyon Aug 2009