The Fedora Project DLF Forum Albuquerque, NM November 17, 2003 Sandy Payette Cornell Information Science

The Fedora Project Fedora Digital Object Repository System Extensible digital object model Repository System exposed via Web service APIs Scalable, persistent storage for content and metadata Local and remote content Associate services with objects Content versioning Fedora Use cases Content Management (CMS) Digital Library architecture Digital Asset Management Institutional Repository Scholarly publishing Preservation Open source software

Fedora History Research (1997-present) : DARPA and NSF-funded research project at Cornell Reference implementation developed at Cornell First Application ( ) : University of Virginia digital library prototype Scale/stress testing for 10,000,000 objects Open Source Software (2002-present): Andrew W. Mellon Foundation granted Virginia and Cornell $1 million to develop a production-quality Fedora system Fedora 1.0 released in May 2003

Fedora Motivations Generic model to manage/access heterogeneous content Operations via the digital object abstraction; default disseminator Extensibility Add new functionality to objects via service associations Object Lifecycle and preservation Content versioning and event history Content repurposing Same content in different objects; dynamic transformations Easy integration with other applications and systems Web services with open APIs Clear separation of server from clients/web user interfaces Does not assume any one workflow or end-user application

Digital Object Model

Persistent ID (PID) Default Disseminator SystemMetadata Datastream (item) Digital object identifier Service view: methods for disseminating content Internal view: key metadata necessary to manage the object Content view: Set of data and metadata items Digital Object Model Architectural View Datastream (item) Extension

PID = uva-lib:100 Default Disseminator SystemMetadata Image (mrsid) Digital Object Model Simple Example DC (xml) Thumbnail (jpeg) Get Profile List Items Get Item List Methods Get DC Record Get Thumbnail Get Medium Get High Get Very High Image Disseminator

Some Common Use Cases [Simple Image]Simple Image Image Manip + DC graph [Scholarly Publication]Scholarly Publication Document Transformation

Content Versioning [Demo]Demo

Repository System software distribution

Fedora 1.2 Software Feature Set Open Fedora APIs Repository as web services (REST and SOAP bindings); WSDL interface defs Flexible Digital Object Model Content View: objects as bundle of items (content and metadata) Service View: objects as a set of service methods (“behaviors”) Extensible functionality by associating services with objects Repository System Core Services: Management, Access/Search, OAI-PMH Storage: XML object store; relational db object cache; relational db object registry Mediation - auto-dispatching to distributed web services for content transformation Auto-Indexing – system metadata and DC record of each object HTTP Basic Authentication and Access Control Built-in disseminator services: XSLT x-form, image manipulation, xml-to-PDF Content Versioning Automatic version control (saves version of content/metadata when modified) Enables date-time stamped API requests (see object as it looked at a point in time) Clients Fedora Administrator: GUI client to create/maintain objects Default Web browser interface: search; access objects via default disseminator Command line utilities (batch load, ingest, purge, others) Migration Utility – mass export/ingest

Fedora Repository Service Interfaces Management Service (API-M) Ingest - XML-encoded object submission Create - interactive object creation via API requests Maintain - interactive object modification via API requests Validate – application of integrity rules to objects Identify - generate unique object identifiers Security - authentication and access control Preserve - automatic content versioning and audit trail Export - XML-encoded object formats Access Service (API-A and API-A-LITE) Search - search repository for objects Object Reflection - what disseminations can the object provide? Object Dissemination - request a view of the object’s content OAI-PMH Provider Service OAI-DC records

Client and Web Service Interactions Fedora Repository System External Service Dispatch Client application Server application web browser Client application user Fedora Service APIs user Content Transform Service API Content Transform Service API

Fedora Mapping to OAIS Fedora Repository System METS 1.2/FO FOXML METS 1.3 DIDL Ingest Formats (SIPs) Export Formats (DIPs) METS 1.2/FO FOXML METS 1.3 DIDL FOXML = Fedora Object XML DIDL = Digital Item Description Language (MPEG21) Archival Format (AIP) R1.3 R2.0

Fedora Software Distribution Package Open Source (Mozilla Public License) 100% Java (Sun Java J2SDK1.4) Supporting Technologies Apache Tomcat 4.1 and Apache Axis (SOAP) Xerces for XML parsing and validation Saxon 6.5 for XSLT transformation Schematron 1.5 for validation MySQL and Mckoi relational database Oracle 9i support Deployment Platforms Windows 2000, NT, XP Solaris Linux

Fedora in Use

Projects using Fedora University of Virginia: digital library (images, EAD, e-texts)EAD VTLS: basis for new commercial product (library system) Indiana University: EVIA Digital Archive (video) EVIA Digital Archive Northwestern: academic technologies (images, art, video, e-texts)imagesart Rutgers University: digital library (e-journals, numeric data) Tufts University: educational (VUE/concept maps); digital library Yale University: Electronic Records Archive New York University: Humanities Computing Group

Sampling of sites using/evaluating Fedora: JSTOR American Geophysical Union NSDL at Cornell Cornell Information Technologies British Library National Library of Portugal Society of Biblical Literature National Archives of Australia Office of Defense Resources, Thailand Monash University, Australia Oxford Digital Library

Fedora Downloads since May 2003 Total downloads: 1427 Average downloads per day: 9 # Countries: 32 Types of orgs: Universities: libraries, IT, departments Software and technology companies Defense/military Banks National libraries and archives Publishers Research labs Library automation vendors Scholarly societies

design solution FEDORA is proving to be a flexible application development platform. Developers may dedicate more time toward building audience specific DL and educational applications. Content tools and digital resources are more easily shared among DL applications. Tufts Slide courtesy of David Kahle

design challenge Create a visual tool to assist students and faculty in organizing and creating pathways through local files, digital library resources and WWW content. Tufts Slide courtesy of David Kahle

content maps container node file node relationship Faculty may sketch out their course content, relationships and pathways through this content using a simple set of moveable objects or nodes. web resource notes Tufts Slide courtesy of David Kahle

OKI & FEDORA Leveraging OKI technical standards will facilitate the sharing, distribution and integration of this new educational tool in educational systems beyond Tufts. Tufts Slide courtesy of David Kahle

Images SimpleZoomLayers Core getThumbnail getCoverpage Basic getThumbnail get Medium getHigh getVeryHigh Hi-Res getRegion getViewer Layered getRegion getViewer Northwestern Slide courtesy of Bill Parod [images] [art]imagesart

Northwestern Slide courtesy of Bill Parod

Image dissemination with Flash zoom viewer Northwestern Slide courtesy of Bill Parod

Text NewspaperBookVideo transcript Audio transcript Core getThumbnail getCoverPage Text getPreview getTreeView getChunk getChunks getStaticView getDynamicView getPrintable getMaster Northwestern Slide courtesy of Bill Parod

ImageMapA/VBookNewsEText Core Image Hi-Res Layered Geo Time Text Northwestern Behaviors by Type Slide courtesy of Bill Parod

UVa EAD Collections [Search] [Angelica]SearchAngelica

UVa Images [image]image

Future Software Releases Fedora Object XML (FOXML) Internal storage format; direct expression of Fedora object model Better support for relationships (“kinship” metadata) Better support for audit trail (event history) Format identifiers for dynamic service binding Shibboleth authentication Policy Enforcement XACML expression language Fedora policy enforcement module Web interface for easy content submission Batch object modification utility Administrative Reporting Object Event History (ABC/RDF disseminations) Better support for “collections” New ingest and export formats (METS1.3, DIDL) December 2003 – December 2004

Future Development Proposals Digital Library in a Box Full-featured DL application with “Fedora inside” Optimized for common set of content types Fedora Power Server Integrity Management Tools Service and link liveness checker Fault Tolerance Mirroring and Replication Peer-to-peer interoperability features Repository clustering Load balancing Object Creation Tools Workflow applications based on content models Web interface for document/content submission

Release 1.2 on December 10, 2003 Questions