1. 2004-09-15NVO Summer School, Aspen Center for Physics1 Publishing and Resource Discovery with Registries Ray Plante Matthew Graham

2. 2004-09-15NVO Summer School, Aspen Center for Physics2 All about Registries Overview of the Registry Framework Publishing to the NVO Visits to Registries (publishing) VOResource: Resource Metadata in XML Visits to Registries (searching) Exercise: query registry in an application IVOA Standard Registry Interface

3. 2004-09-15NVO Summer School, Aspen Center for Physics3 The role of Resource Registries Used to discover and locate resourcesdata and servicesthat can be used in a VO application Resource: anything that is describable and identifiable. –Besides data and services: organizations, projects, software, … –Presently concerned with simple set of resource types Registry: a list of resource descriptions –Expressed as structured metadata to enable automated processing and searching

4. 2004-09-15NVO Summer School, Aspen Center for Physics4 An Overview of Data Discovery You can search the main NVO registry to find resources based on descriptive criteria NVO Registries are coarse-grained –You can find organizations, archives, catalogs –Wont find images, celestial objects, table records Registry framework contains multiple registries: –searchable registries –publishing registries

5. 2004-09-15NVO Summer School, Aspen Center for Physics5 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Framework

6. 2004-09-15NVO Summer School, Aspen Center for Physics6 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Framework harvest (pull)

7. 2004-09-15NVO Summer School, Aspen Center for Physics7 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Framework harvest (pull) replicate

8. 2004-09-15NVO Summer School, Aspen Center for Physics8 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Framework harvest (pull) replicate selective harvesting

9. 2004-09-15NVO Summer School, Aspen Center for Physics9 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Framework

10. 2004-09-15NVO Summer School, Aspen Center for Physics10 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Framework

11. 2004-09-15NVO Summer School, Aspen Center for Physics11 Caution: Construction Ahead Registries in the NVO are currently operating and functional –Supports DataScope but… Registries are at the leading edge of development –Standardized metadata –Standardized interfaces –Consistent behavior

12. 2004-09-15NVO Summer School, Aspen Center for Physics12 NVO Public Registries RegistryURLSearchable?Publishing? STScI/JHU NVO Registry http://nvo.stsci.edu/voregistry/Yes Caltech Carnivorehttp://mercury.cacr.caltech.edu:8080/carnivore/Yes NCSA Registration Portal http://nvo.ncsa.uiuc.edu/nvoregistration.htmlNoYes Private Publishing Registries HEASARC CDS Only support harvesting protocol

13. 2004-09-15NVO Summer School, Aspen Center for Physics13 Overview of Publishing Resources are published if one can use NVO facilities to find them. Multiple layers of publishing –Starts with registry description of resource –Data Access Services Incremental exposure for incremental effort Who are you? How you publish depends on what you want to publish. –An individual with a small data collection –An archive center –Someone with a cool service

14. 2004-09-15NVO Summer School, Aspen Center for Physics14 Small collections: VO-ready Repositories Repositories that allow users to deposit data to share with community –Guarantee long-term storage, availability Automatic support for VO publishing mechamisms –Entries into NVO Registry –Support for standard services: Cone Search, SIA, SSA, SkyNode Currently available Repositories –Images: NCSA Astronomy Digital Image Library http://adil.ncsa.uiuc.edu/ http://adil.ncsa.uiuc.edu/ –Spectra: Spectrum Service for the VO http://voservices.net/spectrum/ http://voservices.net/spectrum/ More public repositories are expected to emerge Check NVO website (http://us-vo.org/) for latesthttp://us-vo.org/

15. 2004-09-15NVO Summer School, Aspen Center for Physics15 Persistent Archives: Tools for Federation Registering your resources with a VO publishing registry –Enter description into registration form at one of the available NVO registries: STScI/JHU Registry: http://nvo.stsci.edu/voregistry/http://nvo.stsci.edu/voregistry/ NCSA Registration Portal: http://nvo.ncsa.uiuc.edu/nvoregistration.html http://nvo.ncsa.uiuc.edu/nvoregistration.html Caltech Carnivore: http://mercury.cacr.caltech.edu:8080/carnivore/ http://mercury.cacr.caltech.edu:8080/carnivore/ –If you have a large number of resources to register, you can run your own registry on your own site NCSA VORegistry-in-a-Box http://nvo.ncsa.uiuc.edu/VO/software/ http://nvo.ncsa.uiuc.edu/VO/software/ Caltech Carnivore: http://mercury.cacr.caltech.edu:8080/carnivore/ http://mercury.cacr.caltech.edu:8080/carnivore/

16. 2004-09-15NVO Summer School, Aspen Center for Physics16 Which registry should I register with? –Each registry is incomplete in different ways. –Behavior is slightly different –Different strategies for making it easier to register multiple resources Support for standard services is strongest –Cone Search, SIA –This week: use NCSA or STScI if you want to see integration into DataScope –SkyNode: go to STScI Next generation by Jan. 2005 –Better consistency, more complete support Your feedback is valuable! Caution: Construction Ahead

17. 2004-09-15NVO Summer School, Aspen Center for Physics17 What can/should you register? –Should: your Organization Declares yourself as a publisher with an ID –Should: your Collection –Can: your existing services Browser-based services: e.g. search page Traditional CGI services Web Services The next level… Implement and register one or more standard services –Cone Search –Simple Image Access –SkyNode* –Simple Spectral Access* *standard still in development Persistent Archives: Tools for Federation

18. 2004-09-15NVO Summer School, Aspen Center for Physics18 Cool Services: Integrating with the VO 1.Register your service at a registry Currently as a generic resource Improved support for non-standard services coming (immediate future: contact NVO project) 2.Integrate support for standard VO formats, schemas FITS and VOTable Standard Data Model schemas (emerging) VOResource, Space-time Coordinates, Spectra 3.Implement Standard Support Interface a standard in development for: Self-description, tracking health and usage

19. 2004-09-15NVO Summer School, Aspen Center for Physics19 A word about Identifiers… IVOA Identifier: a globally-unique URI identifying a resource Ex: ivo://adil.ncsa/targeted/SIA Required as part of a registered resource description As publisher, you control what it looks like Two components: –Authority ID: e.g. adil.ncsa Defines a namespace for identifiers Owned by a single publishing organization –Resource Key: e.g. targeted/SIA Name for the resource unique within the namespace Encourage re-use of local identifiers

20. 2004-09-15NVO Summer School, Aspen Center for Physics20 Visits to the Publishing Registries Publishing Registries –STScI/JHU Registry: http://nvo.stsci.edu/voregistry/ http://nvo.stsci.edu/voregistry/ –NCSA Registration Portal: http://nvo.ncsa.uiuc.edu/nvoregistration.html http://nvo.ncsa.uiuc.edu/nvoregistration.html –Caltech Carnivore: http://mercury.cacr.caltech.edu:8080/carnivore/ http://mercury.cacr.caltech.edu:8080/carnivore/ Recommend you stick with one registry –Authority IDs are currently stuck to a single registry Cant do something? Contact us!

21. 2004-09-15NVO Summer School, Aspen Center for Physics21 Resource Metadata: XML Schema Classes of Resources Organisation, DataCollection, Service, Registry –Specific classes inherit from generic Organized into separate schemas: –Core resource metadata: VOResource –Various extensions schemas containing specific types Capable of describing… –Data centers, research organizations, missions, observatories –Data collections, archives –VO standard services: Cone Search, Simple Image Access –Existing Browser/CGI-based services –Web Services

22. 2004-09-15NVO Summer School, Aspen Center for Physics22 Describing Resources with XML: VOResource Model: types of Resources –Generic Resource –Extensions: e.g. DataCollection, Service, ConeSearch, … VOResource: Family of XML schemas –Core schema: VOResource Common set of metadata applicable to all resources including Dublin Core Resource types: Resource, Service, Organisation –Extension schemas to describe specific kinds of resources Extended type inherits generic metadata adds metadata specific to the type of resource –Extensibility allows for evolution Developers only need to support types of interest to them Allows developers to experiment with non-standard extensions –Currently transitioning from v0.9 to v0.10 Lastest status of metadata standards: http://www.ivoa.net/twiki/bin/view/IVOA/ResourceMetadata

23. 2004-09-15NVO Summer School, Aspen Center for Physics23 Extension Schemas VODataService: describing data and services –Types: DataCollection, SkyService, TabularSkyService ConeSearch: describes location and behavior of a Cone Search service –Types: ConeSearch SIA: describes location and behavior of a Simple Image Access service –Types: SimpleImageAccess VORegistry: metadata for managing registries –Types: Registry, Authority

24. 2004-09-15NVO Summer School, Aspen Center for Physics24 Searching the Registry RegistryURLSearchable?Publishing? STScI/JHU NVO Registry http://nvo.stsci.edu/voregistry/Yes Caltech Carnivorehttp://mercury.cacr.caltech.edu:8080/carnivore/Yes NCSA Registration Portal http://nvo.ncsa.uiuc.edu/nvoregistration.htmlNoYes

25. 2004-09-15NVO Summer School, Aspen Center for Physics25 IVOA Standard Registry Interface IVOA Working Draft: 2 parts –Harvesting: sending descriptions from publishers to searchable registry –Searching Searching –methods keywordSearch(string words, boolean combineByOr) search(ADQLWhere constaints) –Returns a list of VOResource descriptions –Advanced searching with ADQL: Just the where parti.e. search constraints--of ADQL In place of column names, use XPath to VOResource element –Curation/Publisher like %NASA% –Query extensible to any VOResource XML extension –Maps readily to registry implementations based on RDBMS or XML-DB.

26. 2004-09-15NVO Summer School, Aspen Center for Physics26 Why might a client use the standard search interface? Uniform interface to all VO Registries –Not dependant on a single registry Direct relationship between information you are querying and information you get back. Extensible to any type of resource description Re-use of ADQL Custom interfaces to Registries: Registries can provide extended functionality –More advanced capabilities: e.g. XQuery –Simpler interfaces for specialized purposes Client toolkits can provide simplifying interfaces –Support for ADQL/s –XPath aliases

27. 2004-09-15NVO Summer School, Aspen Center for Physics27 What you might use this week STScI or Caltech Registry portals to search for resources Any of the registration portals to register resources nvoregistry as an example of querying the registry from an application