1. Persistent Identifiers (PIDs) and Data Sharing
A Brief Overview
October 2016
Improving Data Sharing And Re-Use In And For Africa
Larry Lannom Corporation for National Research Initiatives

2. PIDs for Data– Why Bother?
Managing increasing amounts of primary and secondary data on the Net over long periods of time
Managing increasingly complex data relationships on the Net over long periods of time
When the attributes of that data such as location(s), responsible parties, and the underlying systems may change dramatically over time
Science builds on past work and increasingly relies on collaboration within virtual distributed communities
All of this absolutely requires reliable, long-term persistent references to bind together the distributed data, processes, and parties involved – referential integrity
This slide is 8 years or so old, pre-dating RDA. I was happy to see that there wasn’t any reason to change it.

3. PID Considerations – Big Picture
No lack of unique identifiers in the world – that part is easy
Unique identification is NOT a technical challenge (U.S. SS# )
Strength in numbers – at this point you would need a very good reason to start yet another PID scheme
Smaller independent schemes will be more fragile and vulnerable to a small group moving on in any fashion, i.e., less persistent
Reliable well-run systems will tend to grow (nobody gets fired for assigning DOIs?)
If there is some aspect of a current widely used scheme that doesn’t work for your case, talk to that community
What problem are you trying to solve?
Don’t start with deciding on a scheme, start with defining the requirements
Resolution Systems – basic decision point
Single authoritative resolution system (≠ single point of failure): DNS, Handle
No single authoritative system (but controlled minting): ISBN, SS#

4. Requirements: Identifier String
Not based on any changeable attributes of the entity
Location
Ownership
Any other attribute that may change w/o changing data itself
Opaque, preferably a ‘dumb number’
A well known pattern invites assumptions that may be misleading
Meaningful semantics invite IP wars, language problems
Unique
Avoid collisions, referential uncertainty
Nice to have
Human-readable
Cut-able, paste-able, embeddable
Fits common systems, e.g., URI specification
All of the above contribute to persistence

5. Requirements: Identifier Resolution System
Reliable
Redundant, no single points of failure
Fast enough to not appear broken
Scalable
Higher loads managed with more computers, not new software
Flexible
Adapt to changing computing environments
Useful to new applications
Trusted
Resolution/Administration must be trusted
Organization must be committed to the long term
Open Architecture
Leverage efforts of a community in building apps on your infrastructure
Transparent
But users needing details of the id/infrastructure NOT a good feature
Persistence, again

6. Using a Resolution System with Existing Identifiers
No lack of identifiers in the world
ISBN mapped to DOI
Example: /99990
The syntax specification, reading from left to right, is:
Handle System DOI name prefix = "10.”
ISBN (GS1) Bookland prefix = "978." or "979.”
ISBN Publisher prefix = variable length numeric string of 2 to 8 digits
Prefix/suffix divider = "/”
ISBN Title enumerator and checkdigit = variable length numeric string of 8 to 2 digits

7. Persistence is (primarily) an Organizational Issue
No technology runs itself
Organizations need to commit to persistence
Organizations need the resources to keep their commitments
Size helps
Business model needed (profits not required, but funds are)
Organizations need dedication to persistence
Conflicts of interest, e.g., if profit is the motive (not the case in any major system of which I am aware) then lack of profit will be a problem
Regional organizations will have difficulty growing
International organization is best, even with the accompanying political and cultural issues

8. Persistence is an Organizational Issue (but don’t make it harder than it needs to be)
Do not bake changeable attributes into the string, such that users and developers operate with mistaken assumptions
Ownership if ownership can change
Organizational names (count the orgs that are > 100 years old and still have the same name)
Assume resolution will change over time
Usage can and will change
Computing/networking environments that seem eternal will change
Indirection is a good thing
ID string will exist as a static set of bits in various formats while the computing and usage environments shift
Disconnect the string from those things that will change
New functions will evolve over time – don’t make it difficult to connect the ID to those new functions
Best example of the problem – broken URLs
An example of a solution – adding functionality to DOIs
Going from 1-to-1 to 1-to-Many
Adding linked data as a resolution option

10. PID Systems: Which One? Handle/DOI appears to be the leading candidate
DOI is a community using Handle System technology
DOI Registration Agencies (RAs) serve specific communities
The International DOI Federation (IDF) guarantees the back-end and the continued viability of each DOI even if the responsible RA goes out of business
Other Handle communities are looking to establish that same level of trust
GWDG, working with EPIC and Max Planck Institutes is going down this same path
International Foundation (DONA) has been established in Geneva to oversee the Handle System and related efforts
This was formerly the sole responsibility of CNRI
Six organizations have joined DONA, will manage the root servers and have the ability to mint prefixes
South Africa expected to join shortly

11. Many Other PID Systems URN (Uniform Resource Name)
RFC 1737: Functional Requirements for Uniform Resource Names (Sollins & Masinter, 1994)
RFC 2141: URN Syntax (Moats, 1997)
RFC 2168: Resolution of Uniform Resource Identifiers using the Domain Name System (Daniel & Mealling, 1997)
Many updates – new IETF group currently at work
PURL (Persistent URL)
OCLC 1995
Close ties to W3C
ARK (Archival Resource Key)
John Kunze, CDL
2001?
XRI (Extensible Resource Identifier)
OASIS
2005?
Recommend Persistent identifiers – an overview by Juha Hakala, 2010

12. Related RDA Efforts PID Interest Group Data Type Registries
Initial output at level of principles
How do I understand the data of others (MIME ++)
For PIDs – how do I interpret the return from PID resolution?
Follow-on Working Group addressing common type record
PID Info Types Working Group
In Maintenance/Adoption Mode
Focused on the types of the type/value pairs of PID resolution
Assumes a Data Type Registry
Data Citation Working Group
Fourteen Recommendations on Citation of Evolving Data

13. Progressions Pre-Internet The Web hits Organizational Efforts
Dedicated Systems – SSN, ISBN
Self-explanatory (affordance) – bib cite
The Web hits
DNS + file name – terrific for quick start, disaster for long-term info mgmt
Permalink, Cool URIs, etc. – just be careful – its not enough for the important stuff over time
Our notion of PID
One level of indirection, e.g., 1 DOI = 1 URL: good start, but is that all?
One to Many – id resolves to current state data (original Handle goal)
Multiple copies, Services such as LoD
Typing of id values, typing of ids (DOI profiles did not take, RDA Data Type Registry, now going to ISO)
Organizational Efforts
Research Orgs: CNRI, OCLC
Standards Bodies: IETF, W3C, RDA (small s standards), NISO, ISO
International Bodies: DONA, RDA again

14. What is Left?
Can we get to the same level of common infrastructure for digital object ids as we have for physical addresses (IP) on the Internet?
What would/should this include?
a common reliable resolution architecture (could be multiple, but all well known and obvious, possibly interchangeable)
an international governance body that was more dedicated to efficiency and openness than to profits
a platform on which multiple applications and businesses could be built – as unconstrained as possible
Enough accompanying information to allow the named entity to be understood and used
an abstraction/indirection layer for digital entities to take us to the next level – the ability to conjure a named entity into existence, given permissions, with no worries about where it is and how it is formed.

15. The DO Cloud End users, developers, and automated processes
ID: 123… A
ID: 987/… F
ID: 843… G
deal with persistently identified,consistently structured digital
objects
which are securely & redundantly managed & accessed via the Internet
Identifier Service
Repository
which is an overlay on
existing or future infor-mation storage systems.

16. “It's tough to make predictions, especially about the future.”
― Yogi Berra

18. PID Advantages Persistent Identity via Indirection
Static references into fluid systems over time
Data on networks moves
Ownership/responsibility change
Formats change
Embedded Ids
For data object in hand – current state data
Updates
New related entities
Networks of Persistent Links
Data / metadata links
Provenance chains
Inheritance across a broad set of entities

19. PID Disadvantages Extra level of effort / cost on creation
Analysis – what to identify / granularity
Coordination across organizations
Maintain resolution system
Persistence requires sustained effort
Organizational discipline
Technology necessary but not sufficient
Analyze cost/benefit ratio
Don’t start unless its worthwhile
Is your data worth it?

20. 20+ Years Later – Why Are We Still Talking About This?
Seems easy enough – assign an ID and, if there is a corresponding resolution system, keep it current
Assign it to what?
Resolve it to what?
Any required metadata and if so, where?
If it resolves how can I understand what comes back?
What if multiple pieces of data come back?
Who is in charge?
Who can I trust?
How long does it have to keep working?
Why will it keep working?
BUT