1. Digital Object Architecture and the Handle System Larry Lannom 20 June 2006 Corporation for National Research Initiatives http://www.cnri.reston.va.us/ http://www.handle.net/

2. Corporation for National Research Initiatives What is the Problem? Managing information in the Net over very long periods of time – e.g. centuries or more Dealing with very large amounts of information in the Net over time When information, its location(s) and even the underlying systems may change dramatically over time Respecting and protecting rights, interests and value

3. Corporation for National Research Initiatives A Meta-level Architecture Allows for arbitrary types of information systems Allows for dynamic formatting and data typing Can accommodate interoperability between multiple different information systems Allows metadata schema to be identified and typed

4. Corporation for National Research Initiatives Digital Object Architecture: Motivation T o reformulate the Internet architecture around the notion of uniquely identifiable data structures Enabling existing and new types of information to be reliably managed and accessed in the Internet environment over long periods of time Providing mechanisms to stimulate innovation, the creation of dynamic new forms of expression and to manifest older forms While supporting intellectual property protection, fine-grained access control, and enable well-formed business practices to emerge

5. Corporation for National Research Initiatives Digital Object Architecture: Components –Digital Objects (DOs) Structured data, independent of the platform on which it was created Consisting of elements of the form One of which is its unique, persistent identifier –Resolution of Unique Identifiers Maps an identifier into state information about the DO Handle System is a general purpose resolution system –Repositories from which DOs may be accessed And into which they may be deposited –Metadata Registries Repositories that contain general information about DOs Supports multiple metadata schemes Can map queries into unique DO specifications (via handles)

6. Corporation for National Research Initiatives What is a Digital Object Defined data structure, machine independent Consisting of a set of elements –Each of the form –One of which is the unique identifier Identifiers are known as Handles –Format is prefix/suffix –Prefix is unique to a naming authority –Suffix can be any string of bits assigned by that authority Data structure can be parsed; types can be resolved within the architecture Associated properties record and transaction record containing metadata and usage information

7. Corporation for National Research Initiatives Repository Notion Any Hardware & Software Configuration Logical External Interface RAP Repository Access Protocol

8. Corporation for National Research Initiatives Repositories & Digital Objects REPOSITORY Each Digital Object has its own unique & persistent ID Objects may be Replicated in Multiple Repositories Content Providers assign Ids No theoretical limits on number of DOs Per Repository RAP

9. Corporation for National Research Initiatives Provides basic identifier resolution system for Internet Logically centralized, but physically distributed and highly scalable Enables association of one or more typed values, e.g., IP address, public key, URL, with each id Optimized for speed and reliability Secure resolution with its own PKI as an option Open, well-defined protocol and data model Provides infrastructure for application domains, e.g., digital libraries & publishing, network mgmt, id mgmt... Handle System

10. Corporation for National Research Initiatives Handles Resolve to Typed Data URL2http://a-books.com/…. DLS9acme/repository HS_ADMIN100acme.admin/jsmith XYZ 1001110011110 12 Handle data Handle Data type Index 10.123/456URL1http://acme.com/….

11. Handle Resolution is a collection of handle services, each of which consists of one or more replicated sites, Site 1Site 2 Site 1 Site 2 Site 3 …... Site n Client The Handle System LHS GHR each of which may have one or more servers. 123.456/abcURL 4 http://www.acme.com/ http://www.ideal.com/8 URL #1#2#n#4#3 #1#2...

12. Handle Clients Global Handle Registry 1. Sends request to Global to resolve 0.NA/10.1000 (naming authority handle for 10.1000) Client Request to Client: Resolve hdl:10.1000/1

13. Handle Clients Client Request to Client: Resolve hdl:10.1000/1 Global Handle Registry 2. Global Responds with Service Information for 10.1000 Service Information Acme Local Handle Service xcccxv xc.. xc.. xc..... xcccxv xccx xcccxv xccx xcccxv xccx

14. Service Information - Acme Local Handle Service Primary Site 123.45.67.8 Port # Secondary Site B Server 1 Server 2 Server 3 Server 1 Server 2123.52.67.9 321.54.678.12 321.54.678.14 762.34.1.1 123.45.67.4 Public Key... 2641 K03RLQ... 2641 5&M#FG... F^*JLS... 3E$T%... A2S4D... N0L8H7...... IP Address Secondary Site A xcccxv xc.. xc.. xc..... xcccxv xccx xcccxv xccx xcccxv xccx Handle Clients

15. Service Information - Acme Local Handle Service Primary Site 123.45.67.8 Port # Secondary Site B Server 1 Server 2 Server 3 Server 1 Server 2123.52.67.9 321.54.678.12 321.54.678.14 762.34.1.1 123.45.67.4 Public Key... 2641 K03RLQ... 2641 5&M#FG... F^*JLS... 3E$T%... A2S4D... N0L8H7...... IP Address Secondary Site A xcccxv xc.. xc.. xc..... xcccxv xccx xcccxv xccx xcccxv xccx Handle Clients

16. Service Information - Acme Local Handle Service Primary Site 123.45.67.8 Port # Secondary Site B Server 1 Server 2 Server 3 Server 1 Server 2123.52.67.9 321.54.678.12 321.54.678.14 762.34.1.1 123.45.67.4 Public Key... 2641 K03RLQ... 2641 5&M#FG... F^*JLS... 3E$T%... A2S4D... N0L8H7...... IP Address Secondary Site A xcccxv xc.. xc.. xc..... xcccxv xccx xcccxv xccx xcccxv xccx Handle Clients

17. 3. Client queries Server 3 in Secondary Site A for 10.1000/1 Client Global Handle Registry #1#2 #1 #2 #3 Primary Site Secondary Site A Secondary Site B Request to Client: Resolve hdl:10.1000/1 Acme Local Handle Service

18. Handle Clients 4. Server responds with handle data Client Global Handle Registry #1#2 #1 #2 Primary Site Secondary Site A Secondary Site B Request to Client: Resolve hdl:10.1000/1 #3 Acme Local Handle Service

19. Handle Clients LHS GHR LHS Handle System Web Client Handle Administration Client Resolve Handle HTTP Redirect Proxy/ Web Server HTTP Get Handle Data http://hdl.handle.net/123.456/abc

20. Handle Clients LHS GHR LHS Handle System Client Handle Administration Client hdl:/123.456/abc Client Plug-In Resolve Handle Request Handle Data

21. Handle Clients LHS GHR LHS Handle System Web Handle Administration Client Web Server HTTP Handle Admin API Admin Forms

22. Handle Clients LHS GHR LHS Handle System Web Handle Administration Client Custom Client

23. Handle Clients LHS GHR LHS Handle System Handle Administration embedded in another process Web

24. Handle Clients LHS GHR LHS Handle System Handle Administration embedded in another process Handle Resolution embedded in another process

25. Corporation for National Research Initiatives Handle System Usage Library of Congress DTIC (Defense Technical Information Center) IDF (International DOI Foundation) –CrossRef (scholarly journal consortium) –CAL (Copyright Agency Ltd - Australia) –MEDRA (Multilingual European DOI Registration Agency) –Nielsen BookData (bibliographic data - ISBN) –R.R. Bowker (bibliographic data - ISBN) –Office of Publications of the European Community –German National Library of Science and Technology NTIS (National Technical Information Service) DSpace (MIT + HP) ADL (DoD Advanced Distributed Learning initiative) Assorted Digital Library Projects In development: Globus Alliance

26. Corporation for National Research Initiatives Assigned Prefixes (June 06) –DOI - 1772 –Other - 801 Handles –DOI - 22+ M –Other - Additional millions (total per prefix known only to prefix manager; LANL adding 600M but privately) Global –Core: three service sites (added locations being considered) –53 M resolutions Handle System Usage

27. Corporation for National Research Initiatives Specification –RFC 3650: Overview –RFC 3651: Namespace and Service Definition –RFC 3652: Protocol DoDI 1322 –Will mandate Handle System use as part of ADL-R ISO standards track for DOI HSAC - Handle System Advisory Committee –Approx 15 members representing big users –Goal: evolve to oversee the system Handle System Management and Standards

28. Corporation for National Research Initiatives Technological and Organizational Infrastructure –Register the existence and access conditions for Learning Objects relevant to the DoD Enterprise –Provide user interface to search the registry Integrates existing technologies –Handle System for identification and access –XML for object description and submission –LOM metadata –Repository for metadata object storage and access –Lucene search engine Running at CNRI in initial production phase ADL Registry (ADL-R)

29. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 Marines M1M1 M 10 M 20 Content Objects ADL-R Input CollectionsInput ProcessingRegistry Parse Authenticate Validate Return 1 N 1 metadata Search Engine Metadata Objects ADL-R

30. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 ADL-R Input CollectionsInput ProcessingRegistry Parse Authenticate Validate Return N 1 metadata Course 1 J-School 123/4........ Search Engine Metadata Objects ADL-R DTIC LOCIDFNSDL LHS UWisc LHS ADL-R GHR LHS Handle System NAVAIR has Handle Prefix 123 and names N 1 hdl:123/4 Marines M1M1 M 10 M 20 Content Objects

31. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 ADL-R Input CollectionsInput ProcessingRegistry Parse Authenticate Validate Return 2 Results Log Search Engine Metadata Objects ADL-R Marines M1M1 M 10 M 20 Content Objects

32. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 ADL-R Input CollectionsInput ProcessingRegistry Parse Authenticate Validate Return Search Engine Metadata Objects ADL-R Metadata Object matching Content Object N 1 hdl:abc/d xml DTIC LOCIDFNSDL LHS UWisc LHS ADL-R GHR LHS Handle System Input process creates Metadata Object for N 1 named hdl:abc/d... 3...and creates two handles: hdl:abc/d for the Metadata Object & hdl:123/4 for the Content Object. 4 Marines M1M1 M 10 M 20 Content Objects

33. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 Searching the Registry Search Engine Metadata Objects ADL-R Metadata Object matching Content Object N 1 hdl:abc/d xml Client Client does a search. Results point to Metadata Object abc/d. 1 DTIC LOCIDFNSDL LHS UWisc LHS ADL-R GHR LHS Handle System Marines M1M1 M 10 M 20 Content Objects

34. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 Searching the Registry Search Engine Metadata Objects ADL-R Metadata Object matching Content Object N 1 hdl:abc/d xml If desired, client gets Metadata Object abc/d to view full registry metadata. 2 DTIC LOCIDFNSDL LHS UWisc LHS ADL-R GHR LHS Handle System Client does a search. Results point to Metadata Object abc/d. 1 Marines M1M1 M 10 M 20 Content Objects Client

35. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 Searching the Registry Search Engine Metadata Objects ADL-R Metadata Object matching Content Object N 1 hdl:abc/d xml Client decides to get Content Object N 1 and resolves handle 123/4 to get its access location and other conditions. 3 DTIC LOCIDFNSDL LHS UWisc LHS ADL-R GHR LHS Handle System Client does a search. Results point to Metadata Object abc/d. 1 If desired, client gets Metadata Object abc/d to view full registry metadata. 2 Marines M1M1 M 10 M 20 Content Objects Client

36. ATSC A3 A2 A1A1 Content Objects NAVAIR N4N4 N3N3 N2N2 Content Objects N1N1 hdl:123/4 Searching the Registry Search Engine Metadata Objects ADL-R Metadata Object matching Content Object N 1 hdl:abc/d xml Client requests a copy of Content Object N 1 from NAVAIR. 4 Client decides to get Content Object N 1 and resolves handle 123/4 to get its access location and other conditions. 3 Client does a search. Results point to Metadata Object abc/d. 1 If desired, client gets Metadata Object abc/d to view full registry metadata. 2 DTIC LOCIDFNSDL LHS UWisc LHS ADL-R GHR LHS Handle System Marines M1M1 M 10 M 20 Content Objects Client

37. Corporation for National Research Initiatives CORDRA Community CORDRA Registry Content Repository Object Level Metadata Content Repository Content Repository Content Repository ADL-R

38. Content Repositories CORDRA Community CORDRA Registry CORDRA Registry Community Content Repositories CORDRA Community Content Repositories CORDRA Registry Intermediate Registry of Registries Federation Level Metadata CORDRA Registry CORDRA Community Federation Level Metadata Federation Level Metadata CORDRA Community CORDRA Registry Content Repositories CORDRA Community CORDRA Registry Content Repositories Intermediate Registry of Registries CORDRA Registry Community Federation Level Metadata Master Registry of Registries CORDRA Registry Community Content Repositories CORDRA Registry Federation Level Metadata CORDRA