Commentary on: The Virtual Observatory G. Jogesh Babu Center for Astrostatistics

Slides:



Advertisements
Similar presentations
Microsoft Research Microsoft Research Jim Gray Distinguished Engineer Microsoft Research San Francisco SKYSERVER.
Advertisements

Trying to Use Databases for Science Jim Gray Microsoft Research
World Wide Telescope mining the Sky using Web Services Information At Your Fingertips for astronomers Jim Gray Microsoft Research Alex Szalay Johns Hopkins.
1 Online Science The World-Wide Telescope as a Prototype For the New Computational Science Jim Gray Microsoft Research
Online Science The World-Wide Telescope as a Prototype For the New Computational Science Jim Gray Microsoft Research
© Copyright 2008 All rights reserved 2 VO-India Project Started in 2002 as a collaboration between IUCAA and Persistent Systems Ltd. Part of International.
The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.
How to do Bayes-Optimal Classification with Massive Datasets: Large-scale Quasar Discovery Alexander Gray Georgia Institute of Technology College of Computing.
Big Data Kirk Borne George Mason University LSST All Hands Meeting August , 2012.
Future of Scientific Computing Marvin Theimer Software Architect Windows Server High Performance Computing Group Microsoft Corporation Marvin Theimer.
Codes for astrostatistics: StatCodes & VOStat Eric Feigelson Penn State.
14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.
National Center for Supercomputing Applications 259 th fastest computer in the world Michael Remijan NCSA –Research Programmer –Web-based Distributed Programming.
NWO Meeting 24 May 04 a Virtual Survey System AstroWise- OmegaCAM – Lofar – Euro-VO a Virtual Survey System AstroWise- OmegaCAM – Lofar – Euro-VO Edwin.
ELearning Applying Information Technology to Enhance Learning in Schools and Communities Presentation by Matthew Mitchell WSU Center to Bridge the Digital.
NSF DMS VOStat - HEAD 2004 Ashish Mahabal VOStat Arming Astronomers with Advanced Statistics Caltech: A. Mahabal, M. Graham,
The aims of SC4DEVO and SC4DEVO-1 Bob Mann Institute for Astronomy and National e-Science Centre, University of Edinburgh.
Data-Intensive Science (eScience) Ed Lazowska Bill & Melinda Gates Chair in Computer Science & Engineering University of Washington August 2011.
SC4DEVO-1 Programme. Monday afternoon Session 1: Introductions & Context SC4DEVO & SC4DEVO-1 (Bob Mann) Virtual Observatory (Roy Williams)
Dogan Seber, PhD San Diego Supercomputer Center University of California, San Diego I. DLESE Library II. DISCOVER OUR EARTH Earth Science Resources for.
1 Introducing the working groups for Simulation group Per Henrik Hogstad 2.
Big Data, Future of Computing, Parting Thoughts Slobodan Vucetic Associate Professor Department of Computer and Information Sciences Temple University.
WPI Center for Research in Exploratory Data and Information Analysis From Data to Knowledge: Exploring Industrial, Scientific, and Commercial Databases.
Data Mining: Introduction. Why Data Mining? l The Explosive Growth of Data: from terabytes to petabytes –Data collection and data availability  Automated.
ICS 499 Projects Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 12/7/20111Lipyeow Lim -- University of.
Supported by the National Science Foundation’s Information Technology Research Program under Cooperative Agreement AST with The Johns Hopkins University.
“Here comes the Grid” Mark Hayes Technical Director - Cambridge eScience Centre NIEeS Summer School 2003.
Hopkins Storage Systems Lab, Department of Computer Science A Workload-Driven Unit of Cache Replacement for Mid-Tier Database Caching Xiaodan Wang, Tanu.
E. Solano Centro de Astrobiología (INTA-CSIC) I.P. Observatorio Virtual Español El Observatorio Virtual: una infraestructura básica para la investigación.
National Center for Supercomputing Applications Observational Astronomy NCSA projects radio astronomy: CARMA & SKA optical astronomy: DES & LSST access:
A PPARC funded project AstroGrid: new technology for the virtual observatory SC2004 Pittsburgh, PA November 2004 Guy Rixon AstroGrid Technical Architect.
Alex Szalay, Jim Gray Analyzing Large Data Sets in Astronomy.
Distributed Computing Economics Jim Gray Microsoft Research Presentation To Microsoft Venture Capital Summit 28.
1 The Terabyte Analysis Machine Jim Annis, Gabriele Garzoglio, Jun 2001 Introduction The Cluster Environment The Distance Machine Framework Scales The.
Science Archive for Sky Surveys Data Providers and the VO - NeSC 2003 March Wide Field Astronomy Unit Institute for Astronomy.
Public Access to Large Astronomical Datasets Alex Szalay, Johns Hopkins Jim Gray, Microsoft Research.
Using the NSA Presentation to NOAO Users Committee October 5, 2005.
Home and Personal Communication Systems Lesson 3 – Internet Connections.
INTRODUCTION TO CLOUD COMPUTING ggg UNDERSTANDING CLOUD COMPUTING UNDERSTANDING CLOUD COMPUTING DEFINITION CLOUD COMPUTING.
EScience May 2007 From Photons to Petabytes: Astronomy in the Era of Large Scale Surveys and Virtual Observatories R. Chris Smith NOAO/CTIO, LSST.
“Big Data” and Data-Intensive Science (eScience) Ed Lazowska Bill & Melinda Gates Chair in Computer Science & Engineering University of Washington July.
A PPARC funded project Workflow and Job Control in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester.
The Virtual Observatory Europe and the VO: the Astrophysical Virtual Observatory and the EURO-VO Astrophysical Virtual Observatory and the EURO-VO Paolo.
The Swiss Grid Initiative Context and Initiation Work by CSCS Peter Kunszt, CSCS.
NVO Review -- San Diego Jan The VO compared to Other O‘s Jim Gray Microsoft T HE US N ATIONAL V IRTUAL O BSERVATORY.
Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.
March 1st, 2006Prospective PNG PNG: Databases - Virtual Observatory.
Sky Survey Database Design National e-Science Centre Edinburgh 8 April 2003.
Grid Based Chinese Virtual Observatory System Design Chenzhou CUI, Yongheng ZHAO National Astronomical Observatories, Chinese Academy of Sciences
Peter Lee Head, Computer Science Department Carnegie Mellon University.
The Virtual ObservatorySVO School, October 2009 E. Solano. The Virtual Observatory Enrique Solano Spanish VO Principal Investigator LAEX-CAB (INTA-CSIC)
AstroGrid NAM 2001 Andy Lawrence Cambridge NAM 2001 Andy Lawrence Cambridge Belfast Cambridge Edinburgh Jodrell Leicester MSSL.
EScience: Techniques and Technologies for 21st Century Discovery Ed Lazowska Bill & Melinda Gates Chair in Computer Science & Engineering Computer Science.
Data Mining Challenges and Opportunities in Astronomy The Punchline: Astronomy has become an immensely data- rich field (and growing) There is a need.
1 Online Science The World-Wide Telescope as a Prototype For the New Computational Science Jim Gray Microsoft Research
Emerging Technologies and Applications Josh Baron Associate Director for Instructional Technology Center for Improved Engineering and Science Education.
Virtual Observatory India VOStat Statistical Analysis for the Virtual Observatory By Deoyani and Mohasin.
1 eScience in Astronomy: Grid & VO GAVO III KickOff eScience in Astronomy: VO & GRID eScience: making the most advanced tools of IT available to scientists.
RI EGI-InSPIRE RI Astronomy and Astrophysics Dr. Giuliano Taffoni Dr. Claudio Vuerli.
Microsoft Research San Francisco (aka BARC: bay area research center) Jim Gray Researcher Microsoft Research Scalable servers Scalable servers Collaboration.
Data Grids, Digital Libraries and Persistent Archives: An Integrated Approach to Publishing, Sharing and Archiving Data. Written By: R. Moore, A. Rajasekar,
Virtual Repository Progress Lars Lindberg Christensen (ESA/ESO)
Data Mining Challenges and Opportunities in Astronomy
Fermi & ASDC Elisabetta Cavazzuti on behalf ASDC Team Fermi Meeting
Moving towards the Virtual Observatory Paolo Padovani, ST-ECF/ESO
Technology CURRICULUM at
Rick, the SkyServer is a website we built to make it easy for professional and armature astronomers to access the terabytes of data gathered by the Sloan.
Databases, Web Pages and Archives
Center for Astrostatistics
An Introduction to and Motivation for Visualization Research
Presentation transcript:

Commentary on: The Virtual Observatory G. Jogesh Babu Center for Astrostatistics

Moving Data is Slow! 14 minutes ,920, OC hours1000Gbps 1 day Mpbs 14 hours ,000155OC3 2 days2, ,00043T3 2 months2, ,2001.5T1 5 months Home DSL 6 years3,0861, Home phone Time/TB $/TB Sent $/Mbps Rent $/month Speed Mbps Context Source: TeraScale Sneakernet, Microsoft Research, Jim Gray et al. How long does it take to move a Terabyte? (how about a Petabyte?)

Federate the existing and forthcoming large digital sky surveys and archives, and provide the tools for their scientific exploitation A dynamical, interactive, web-based research environment for the new astronomy with massive data sets Technology-enabled, but science- driven Data analysis for VO

The VOStat Project Web-based statistics services for the VO (Babu, Feigelson,Djorgovski, Graham, Mahabal, Nichol, Wasserman, et al.) Data Client Powerful Statistical Computers VOStat Web service Tools for data exploration and analysis in the VO and other science environments

Streaming algorithms Object detection in multi-epoch/band data ( Data from different surveys of the same field ) Sequential smoothing of massive multivariate data. –Convex hull pealing methods Exploratory analysis on subsets of data through VOStat