Repositories: Disruptive Technology or Disrupted Technology? Sandy Payette, Executive Director DORSDL Workshop at ECDL 2008 September 2008

What does this talk have to say about “applications” which is the theme of this track?

Social and Technical Forces  Waves of Repository-Enabled Applications Institutional Repo and Digital Library Apps IR: Scholars to deposit articles, etc. DL: Digital library search and access; collections Collaborative “Web 2.0” collaborative filtering annotate; discuss scholarly materials E-Science, E-Research, Data-Intensive Publications linked to data Data aggregation from distributed sources Fusion, simulation

Implications for Repositories more distributed more collaborativemore web-oriented more open more interoperable

We can build amazing private islands But should we? Repository Island

Infrastructure

Emergence of Infrastructure Source: Understanding Infrastructure: Lessons for New Scientific Infrastructure, Systems Heterogeneous components Central control Closed, stable Dedicated/improvised gateways Heterogeneous systems Distributed control Coordination Generic gateways Open, reconfigurable Networks

Repositories as components of networked information infrastructure

Which are repositories? Internet Archive Google Data DSpace Fedora E-Prints S3

Repositories: Disruptive or Disrupted? Where’s the multi-institutional perspective? Where’s the Web? What constraints do we impose by starting with the perspective of a single institution or “enterprise”? What impact will “Cloud” storage and computing have? What is the best path to real interoperability? When do you need complex vs. simple and good enough? How assert leadership in emerging value networks?

Right Direction … Exposure and Re-Use of Digital Content

Object Re-Use and Exchange (OAI-ORE) Data model for describing bounded aggregations in Web graph

Hubble optical observation Baltimore, MD Basic object information Strasbourg, France E-Science Aggregations And in digital scholarly communication, the single container concept is obsolete. text 2006 Astrophysics paper X-MM-Newton X-ray observation Vilspa, Spain Chandra X-ray observation Cambridge, MA A1795

Fedora objects can map to ORE Identified, bounded aggregations of related information units that form a logical whole. Components of a digital object may vary according to: –Semantic type: book, article, software, dataset, simulation, … –Media type: text, image, audio, video, mixed –Media format: PDF, HTML, JPEG, MP3, … –Network location –Relationships: internal, external HTML MP3 Identifier PDF ORE Resource Map Exposure ore:describes

Network of Digital Objects … aggregations related to aggregations PID 4 PID 1 PID 3 hasPart ore:describes DC HTML

Another right direction… Repositories Embedded in Infrastructure (Fedora as case study)

A History of Fedora Repository Project’s Technical Approaches to Interoperability 1998: Interfaces Generic interfaces to access and manage digital objects in a repository Extensible interfaces on digital objects (“Disseminators”) to provide uniform service access points for heterogeneous underlying content 2001: Web Services APIs to access and manage digital objects in a repository SOAP and REST over HTTP 2002: XML-based digital object serialization formats METS FOXML (Fedora Object XML) 2005: Semantic Technologies and RDF (RDF-based “Resource Index”)

Now, make it fit better with Web in : Atom Syndication Format New with Fedora 3.0 via ingest/export operations 2008: OAI-ORE Experiments completed Work in progress for Fedora 3.0/3.1 support : Adopt simple, common Web API(s) with wide appeal Atom Publishing Protocol SWORD Other? : Connect backend storage to “Cloud”

Fedora : new Web exposures New Web APIs Fedora APIs Registry Search RDF Query Ingest Export Manage Validate Access RDF IndexStoreRegistry File systemRDBMS (Registry) Disseminate New Web APIs: Atom Publishing SWORD Other TBD OAI-ORE (2008)Atom (2008)new formats Triplestore SPARQL/Mulgara Linked Data? New: Policy

Object serialization - Atom feed entry (1-n)

New Akubra Project … backend abstraction New Web APIs Fedora APIs Registry Search RDF Query Store Akubra Storage Abstraction Plug-in 1Plug-in 2Plug-in … File System Sun Honeycomb Fedora Repository Service registry db Cloud Storage: -Amazon S3 (now) -Google Data (next) - Other TBD Other stores TBD…

Repositories Not-Disrupted Expose repositories in a global networked environment; not just as local, closed systems Make it easier to access and re-use content that is stored in digital repositories, especially on the Web Make repositories conform to common protocols, formats, and standards that are being used in the Web; get repositories in the game of emerging value networks Provide transformations that enable content to be reusable in different contexts. Create repository connectors for common authoring applications to position repositories to capture content at beginning of its life, not at just end of it.

More Info: Questions and Discussion