10 June 2002Hanisch/NVO1 Building the Framework for the National Virtual Observatory: An Information Technology Research Initiative of the National Science Foundation Bob Hanisch and the NVO Collaboration

10 June 2002Hanisch/NVO2 The National Virtual Observatory National Academy of Sciences “Decadal Survey” recommended NVO as highest priority small (<$100M) project “Several small initiatives recommended by the committee span both ground and space. The first among them—the National Virtual Observatory (NVO)—is the committee’s top priority among the small initiatives. The NVO will provide a “virtual sky” based on the enormous data sets being created now and the even larger ones proposed for the future. It will enable a new mode of research for professional astronomers and will provide to the public an unparalleled opportunity for education and discovery.” —Astronomy and Astrophysics in the New Millennium, p. 14

10 June 2002Hanisch/NVO3 Project Team NSF ITR project, “Building the Framework for the National Virtual Observatory” is a collaboration of 17 funded and 3 unfunded organizations –Astronomy data centers –National observatories –Supercomputer centers –University departments –Computer science/information technology specialists PI and project director: Alex Szalay (JHU) CoPI: Roy Williams (Caltech/CACR)

10 June 2002Hanisch/NVO4 Proposal Team A. SzalayJHU R. WilliamsCaltech C. AlcockU. Penn.M. LivnyU. Wis. K. BorneADC/RaytheonC. LonsdaleIPAC T. CornwellNRAOT. McGlynnHEASARC/USRA D. DeYoungNOAOA. MooreCMU G. FabbianoSAOR. MooreSDSC/UCSD A. GoodmanHarvardJ. PierUSNO J. GrayMicrosoftR. PlanteNCSA/UIUC R. HanischSTScIT. PrinceCaltech G. HelouIPACE. SchreierSTScI S. KentFNALN. WhiteGSFC C. KesselmanUSC

10 June 2002Hanisch/NVO5 Proposal Team - Univ. Depts. A. SzalayJHU R. WilliamsCaltech C. AlcockU. Penn.M. LivnyU. Wis. K. BorneADC/RaytheonC. LonsdaleIPAC T. CornwellNRAOT. McGlynnHEASARC/USRA D. DeYoungNOAOA. MooreCMU G. FabbianoSAOR. MooreSDSC/UCSD A. GoodmanHarvardJ. PierUSNO J. GrayMicrosoftR. PlanteNCSA/UIUC R. HanischSTScIT. PrinceCaltech G. HelouIPACE. SchreierSTScI S. KentFNALN. WhiteGSFC C. KesselmanUSC

10 June 2002Hanisch/NVO6 Proposal Team – CS/IT A. SzalayJHU R. WilliamsCaltech C. AlcockU. Penn.M. LivnyU. Wis. K. BorneADC/RaytheonC. LonsdaleIPAC T. CornwellNRAOT. McGlynnHEASARC/USRA D. DeYoungNOAOA. MooreCMU G. FabbianoSAOR. MooreSDSC/UCSD A. GoodmanHarvardJ. PierUSNO J. GrayMicrosoftR. PlanteNCSA/UIUC R. HanischSTScIT. PrinceCaltech G. HelouIPACE. SchreierSTScI S. KentFNALN. WhiteGSFC C. KesselmanUSC

10 June 2002Hanisch/NVO7 Proposal Team – Nat’l Obs. A. SzalayJHU R. WilliamsCaltech C. AlcockU. Penn.M. LivnyU. Wis. K. BorneADC/RaytheonC. LonsdaleIPAC T. CornwellNRAOT. McGlynnHEASARC/USRA D. DeYoungNOAOA. MooreCMU G. FabbianoSAOR. MooreSDSC/UCSD A. GoodmanHarvardJ. PierUSNO J. GrayMicrosoftR. PlanteNCSA/UIUC R. HanischSTScIT. PrinceCaltech G. HelouIPACE. SchreierSTScI S. KentFNALN. WhiteGSFC C. KesselmanUSC

10 June 2002Hanisch/NVO8 Proposal Team - NASA A. SzalayJHU R. WilliamsCaltech C. AlcockU. Penn.M. LivnyU. Wis. K. BorneADC/RaytheonC. LonsdaleIPAC T. CornwellNRAOT. McGlynnHEASARC/USRA D. DeYoungNOAOA. MooreCMU G. FabbianoSAOR. MooreSDSC/UCSD A. GoodmanHarvardJ. PierUSNO J. GrayMicrosoftR. PlanteNCSA/UIUC R. HanischSTScIT. PrinceCaltech G. HelouIPACE. SchreierSTScI S. KentFNALN. WhiteGSFC C. KesselmanUSC

10 June 2002Hanisch/NVO9 Collaborators B. BerrimanIRSAJ. GoodIPAC R. BrissendenSAOI. GriffinSTScI R. BrunnerCaltechB. MadoreNED/IPAC C. CheungGSFCJ. MazzarellaNED/IPAC K. CookLLNLB. McLeanSTScI A. ConnollyU. Pitt.M. PostmanSTScI D. CurkendallJPLA. RotsSAO G. DjorgovskiCaltechS. StromNOAO I. FosterU. ChicagoA. ThakarJHU R. GalJHUD. TodyNOAO

10 June 2002Hanisch/NVO10 Project Management NSF CISE + AST External Review Committee PI/Project Director: Szalay Co-PI/Chief Architect: Williams Executive Committee Data Centers Project Scientist Project Manager System Architect E&O Coordinator Technical Working Group Science Working Group InfrastructureActivities Local/Distant Universe Digital Milky Way Rare/Exotic Objects AGN Census Extra-Solar Planets Science Prototypes Theoretical Astrophysics Portals/Workbenches Metadata Standards Grid Services/Testbed Data Models DataAccess/Resources Data Providers

10 June 2002Hanisch/NVO11 Team Organization Executive Committee –A. Szalay, R. Williams, R. Hanisch (PM), D. De Young (PS), R. Moore (SA), G. Helou, E. Schreier Education & Outreach –M. Voit, Coordinator First Working Groups established –Metadata (R. Plante/NCSA) –Systems (R. Moore/UCSD) –Science (D. De Young/NOAO) Project teams established for initial science demonstrations –GRB follow-up (T. McGlynn/HEASARC) –Brown dwarf search (B. Berriman/IPAC) –Cluster galaxy morphologies (R. Plante/NCSA)

10 June 2002Hanisch/NVO12 Education & Outreach Integral part of project Emphasis is on development of partnerships Kick-start with a workshop this summer at STScI (July 11-12) –Understand requirements on NVO services from perspective of formal education, informal education, commercial/corporate, and public outreach content developers

10 June 2002Hanisch/NVO13 Education/Outreach Partners Association of Science- Technology Centers International Planetarium Society National Air and Space MuseumSilicon Graphics (Digital Planetarium) Spitz (Electric Sky)Maryland Space Grant Consortium Gettysburg College (Project CLEA) UC Berkeley American Museum of Natural History

10 June 2002Hanisch/NVO14 Management Plan Wrote and delivered formal management plan to NSF in January 11 major work breakdown categories, with sub-elements to three levels All level-two technical WBS areas have designated lead who is responsible for tasks and schedule within that area Building the Framework for the National Virtual Observatory NSF Cooperative Agreement AST Management Plan December 2001 The challenge of building a framework to enable the National Virtual Observatory will be met with a management structure that supports distributed research and development. We take optimal advantage of the domain expertise already resident in the organizations supporting the existing archival systems, sky surveys, and source catalogs of the astronomy community and meld this diversity with state-of-the-art information technology. Our structure ensures strong communication and coordination among the distributed, multi- disciplinary, heterogeneous resources, with accountability to both the community and the funding agency. It ensures that astronomy needs drive technology development.

10 June 2002Hanisch/NVO15 Work Breakdown Structure

10 June 2002Hanisch/NVO16 Milestones Nov 2001 – Jan 2002: Established project structure April 15, 2002: VOTable V1.0 May 1, 2002: 50+ ConeSearch services registered May 8, 2002: Defined initial science demos June 10, 2002: Form International VO Alliance (IVOA) Nov 15, 2002: Internal testing of science demos January 2003: Initial science demonstrations (AAS) August 2003: Intermediate NVO science demos (IAU)

10 June 2002Hanisch/NVO17 Milestones (cont’d) Major technical initiatives (WBS) –Early standards: VOTable/ConeSearch (3.2.6, 3.3.2, 3.4.1, 3.4.4, 5.3.3) –Understanding Web Services (3.6) –Understanding Grid Services (3.1.7, 4.5, 5.3.4, 9.1) –Profile Registrations (3.4.2, 3.6.3) –Data Models (2.1, 2.2) –Resource Management (6.1.2) –System Architecture (4.1)

10 June 2002Hanisch/NVO18 Reporting and Communication Formal Quarterly and Annual Reports to NSF, being cc’d to NASA Informal monthly reports to project manager Biweekly project status telecons with level-two WBS leaders Weekly Executive Committee telecons Weekly or biweekly working group telecons (Metadata, Systems, Science) Archived exploders for all working groups and management discussions

10 June 2002Hanisch/NVO19 Development Approach First year: emphasize prototyping and experimentation, leading to real demos but not necessarily production-level software or system –Many IT tools now available; extensive evaluation through prototypes necessary to refine choices –Will set up framework for more formal software management (baseline, test, revision control) for a distributed development effort so that it is ready for second-year development NSF ITR project is not expected to define and “deliver” the entire NVO

10 June 2002Hanisch/NVO20 Critical Path Science demonstrations –Identified, scoped and scheduled Service registry issues –Needs international coordination (Garching) User interface issues –Need to retrofit existing portals EPO requirements –Impact on metadata standards

10 June 2002Hanisch/NVO21 Role of Science Prototypes Keep focus on user- and science needs Identify most common services Verify standardization efforts Encourage data providers to participate Demonstrate to community that NVO tools will –arrive soon –will be useful for everybody –can evolve incrementally First science demos planned for January 2003

10 June 2002Hanisch/NVO22 Initial Science Prototypes Brown-Dwarf search –Distributed query across several archives –Correlations with non-detections –Example of typical NVO search Gamma-Ray burst –Event follow up service –Exercise in standards compliance/interoperabilty Galaxy evolution in clusters –On-the-fly image analysis and pattern recognition –Exercise in grid computing

10 June 2002Hanisch/NVO23 NVO: How Will It Work? Define commonly used “atomic” services Build higher level toolboxes/portals on top We do not build “everything for everybody” Use the rule: –Define the standards and interfaces –Build the framework –Build the 10% of services that are used by 90% –Let the users build the rest from the components

10 June 2002Hanisch/NVO24 Atomic Services Metadata information about resources –Waveband –Sky coverage –Translation of names to universal dictionary (UCD) Simple search patterns on the resources –Cone Search –Image mosaic –Unit conversions Simple filtering, counting, histogramming On-the-fly recalibrations

10 June 2002Hanisch/NVO25 Higher Level Services Built on Atomic Services Perform more complex tasks Examples –Automated resource discovery –Cross-identifications –Photometric redshifts –Outlier detections –Visualization facilities Expectation: –Build custom portals in matter of days from existing building blocks (like today in IRAF or IDL)

10 June 2002Hanisch/NVO26 VOTable VOTable (R. Williams, F. Ochsenbein+ …) –Proper XML document –Supports streaming –Handles complex, self descriptive data sets –Developed jointly with European effort –VOTable V1.0 released on April 15, 2002 –Currently undergoing testing –Reference parser just finished –Design of API just started

10 June 2002Hanisch/NVO27 SkyQuery Distributed Query tool using a set of services Feasibility study, built in 6 weeks from scratch –Tanu Malik (JHU CS grad student) –Tamas Budavari (JHU astro postdoc) Won 2 nd prize in Microsoft.NET Contest Allows queries like: SELECT o.objId, o.ra, o.r, o.type, t.objId FROM SDSS:PhotoPrimary o, TWOMASS:PhotoPrimary t WHERE XMATCH(o,t)<3.5 AND AREA(181.3,-0.76,6.5) AND o.type=3

10 June 2002Hanisch/NVO28

10 June 2002Hanisch/NVO29 ConeSearch Search for catalog objects around a point Returns data in VOTable format Requires a registered profile Point of the exercise –A learning experience –Existing archives test and implement VOTable –Understand service description issues Automated test and verification In less than two weeks 7 groups, 50 services Cross-Identification service built on top

10 June 2002Hanisch/NVO30

10 June 2002Hanisch/NVO31 International Collaboration European initiatives underway –Astrophysical Virtual Observatory funded by European Commission (€3.3 million, three years) –AstroGrid, funded by UK e-science program (£5 million, three years) Other international efforts starting: –Canada (C$4M recently approved), India, Japan, Chile, Germany, Russia, Australia International VO roadmap being developed, to be announced at Garching VO conference International VO Alliance Regular telecons among NVO, AVO, and AstroGrid leadership Frequent technical contacts among partners

10 June 2002Hanisch/NVO32 International Standards Active collaboration among NVO, AVO, and AstroGrid on VOTable –V1.0 released on April 15 –Basis for testing metadata models, exchange protocols, encoding mechanisms Continued development of FITS standard –World Coordinate System definitions Framework definition (Paper I) Celestial coordinates (Paper II) Spectral dispersion relations (Paper III) Distortion functions (Paper IV) Time (TBD)

10 June 2002Hanisch/NVO33 Summary NSF ITR NVO project is one of four major and numerous other small VO-related initiatives now underway world-wide NVO is adopting, adapting, or developing necessary technology as derived from science requirements Our project management approach seems to be working based on the first six months experience NVO project is dealing with many of the management challenges that will face the ultimate VO organization