The Fedora Project Update as of January 2004 Ithaca, NY January 29, 2004 Sandy Payette Cornell Information Science

The Fedora Project Fedora F lexible E xtensible D igital O bject R epository A rchitecture Fedora Use Cases Digital Library Architecture Digital Asset Management Institutional Repository Content Management System (CMS) Scholarly publishing Preservation support Open source software Not RedHat ! Mozilla Public License

Fedora History Research (1997-present) : DARPA and NSF-funded research project at Cornell Reference implementation developed at Cornell Payette, Lagoze, Dushay First Application ( ) : University of Virginia digital library prototype Scale/stress testing for 10,000,000 objects Open Source Software (2002-present): Andrew W. Mellon Foundation granted UVA and Cornell $1 million to develop a production-quality Fedora system Fedora 1.0 (May 2003) Fedora 1.1 (Aug 2003) Fedora 1.2 (Dec 2003)

Why Fedora? Data model Generic abstraction for heterogeneous digital resources Flexibility to create different “content models” No bifurcation of metadata and content Aggregate both locally stored content and by-reference content Distributed repositories Common data model Common APIs for access and management Federation Content repurposing Provide multiple views of content/metadata Dynamic transformations of content/metadata Add new views/transformations over time Web Services Fedora is exposed via web services Fedora can interact with other web services Fedora uses WSDL and XML Object Lifecycle and preservation Content versioning Event history Easy integration with other applications and systems Web services with open APIs Does not assume any particular workflow or end-user application

Fedora in Use

Fedora Downloads as of Dec 2003 Total downloads: 4960 Average downloads per day: 19 # Countries: 50; # orgs: 360 Types of orgs: Universities: libraries, IT, academic departments Software and technology companies Defense/military Banks National libraries and archives Publishers Research labs Library automation vendors Scholarly societies

Selected Projects Committed to Fedora University of Virginia: digital library (images, EAD, e-texts)EAD Cornell and UVA: Tibetan and Himalayan Digital Library Tibetan and Himalayan Digital Library VTLS (library systems): new commercial product (VITAL)VITAL Tufts University: education (VUE/concept maps); digital libraryVUE Northwestern: academic technologies (images, art, video, e-texts)imagesart Indiana University: EVIA Digital Archive (video) EVIA Digital Archive Rutgers University: digital library (e-journals, numeric data) digital library New York University: Humanities Computing Humanities Computing

A Sampling of Fedora Usage: Some Active Prototypes and evaluations (we are tracking) JSTOR-ArtStore-EArchives (Ithaka) Harris Corporation (R&D; government systems; archives) American Geophysical Union National Library of Portugal Monash University with National Library of Australia NSDL at Cornell Some Interesting Download Sites (from our logs) British Library Society of Biblical Literature National Archives of Australia Office of Defense Resources, Thailand Microsoft Sun Microsystems Apple Cornell Information Technologies (CIT)

Digital Object Model

Persistent ID (PID) Default Disseminator SystemMetadata Datastream (item) Digital object identifier Service Perspective: methods for disseminating “views” of content Internal: key metadata necessary to manage the object Item Perspective: Set of content or metadata items Digital Object Model Architectural View Datastream (item) Your Extension

Datastream (Managed) Fedora stores and manages the content bytestream Fedora stores a reference (URL) to the content Fedora stores a reference (URL) to the content, but will not mediate access to content. Fedora stores a name-spaced block of XML content within the Fedora digital object XML file. Digital Object Model Datastream (External)Datastream (Redirect)Datastream (XML) 4 Classifications for Datastreams

PID = uva-lib:100 Default Views SystemMetadata Image (mrsid) Digital Object Model Example “content model” DC (xml) Thumbnail (jpeg) Image Views Metadata Views Get Profile List Items/Get Item List Methods Get OAI_DC Get Thumbnail Get Medium Get High Get MARC Get DC Multiple Disseminations

Persistent ID (PID) Service Definition Metadata (WSDL) SystemMetadata Datastreams Behavior Definition Object Persistent ID (PID) SystemMetadata Datastreams Disseminators Data Object Behavior Mechanism Object Persistent ID (PID) Service Binding Metadata (WSDL) SystemMetadata Datastreams External Service Digital Object Model Service Relationships

Repository System Architecture and Software

Fedora Server Design: 3 Layers 1. Interface Web Service for Access/Search Web Service for Management OAI Provider 2. Application Logic Implements all functionality in terms of the Fedora digital object model. 3. Storage RDBMS Object “cache” for performance Digital object registry XML object serializations Authoritative object with versioning All management operations on XML

Fedora Server Architecture

Fedora Repository System Client and Web Service Interactions Fedora Repository System Content Transform Service Content Transform Service user Web Service Dispatch Web Service Service BackendFrontend client application client application web browser user

Fedora 1.2 – Server Feature Set Management Web Service Identify - generate unique object identifiers (but will accept your identifier) Ingest - object submission in XML format (e.g., METS) Create - interactive object creation via API calls Maintain - interactive object modification and deletion via API calls Export - provides a copy of an object encoded in XML format (e.g., METS) Purge - permanently remove objects from repository Access and Search Web Service Search - locate objects via the default repository index Reflect - describe the disseminations an object can provide Disseminate - deliver a view of an object’s content OAI-PMH Provider Service Request - OAI-DC records Internal Features Modules – system configurable and ability to replace modules Performance – relational db object cache Storage – XML object wrappers; datastreams in native formats Replicate – XML object store to relational cache Validate - application of integrity rules to objects Secure - basic HTTP authentication and simple access control Preserve - automatic content versioning and audit trail

Fedora Clients Fedora Administrator Java Swing client Create/maintain objects Search repository Wizards for behavior objects Web Browser (via Fedora URL syntax) REST-based search REST-based access to objects Command Line Utilities Batch loading Ingest, purge, more Migration Utility General-purpose mass export/ingest Supports upgrading to new versions of Fedora

Fedora Software Package Open Source (Mozilla Public License) 100% Java (Sun Java J2SDK1.4) Supporting Technologies Apache Tomcat 4.1 and Apache Axis (SOAP) Xerces for XML parsing and validation Saxon 6.5 for XSLT transformation Schematron 1.5 for validation MySQL and Mckoi relational database Oracle 9i support Deployment Platforms Windows 2000, NT, XP Solaris Linux Mac OSX (upcoming)

Fedora Demos

UVA EAD Collections [Search] [Angelica]SearchAngelica

UVA Images [image]image

content maps container node file node relationship Faculty may sketch out their course content, relationships and pathways through this content using a simple set of moveable objects or nodes. web resource notes Tufts Slide courtesy of David Kahle

OKI & FEDORA Leveraging OKI technical standards will facilitate the sharing, distribution and integration of this new educational tool in educational systems beyond Tufts. Tufts Slide courtesy of David Kahle

ImageMapA/VBookNewsEText Core Image Hi-Res Layered Geo Time Text Northwestern Content Models Chart courtesy of Bill Parod Genre of digital resource Types of Behaviors

Northwestern Image courtesy of Bill Parod [images] [art]imagesart Dissemination: Merge two datastreams Image with Metadata

Dissemination: Repurpose datastream image with Flash zoom viewer Northwestern Image courtesy of Bill Parod [images] [art]imagesart

Fedora Administrator [Demo Object]Demo Object (Demo runs locally. Not available via public URL.)

Fedora Future

Fedora 1.3  2.0 (Jan-Dec 2004) Fedora Object XML (FOXML) New internal storage format Relationships metadata Better support for event history Format identifiers for dynamic service binding and OAI formats Performance Scale testing (benchmark ~10 million objects) Concurrent usage stress Performance tuning as needed (ingest, dissemination) Advanced Access Control Authentication (plug in modules for common schemes; Shibboleth) XACML policy expression language Fedora policy enforcement module Web forms for easy content submission Batch object modification utility Administrative Reporting New ingest and export formats (FOXML, METS1.3, DIDL) Various enhancements and special requests

Next Development Proposal Fedora R2R - Distributed, Federated Repositories Shared name resolution service Any repository can fulfill a dissemination request within a federation Fedora Proxy Service for distributed virtual repository Federated or distributed searching (Z30.50, OAI, other approaches)? Shared web services (for behaviors) Repositories as Service Registries (like UDDI) Fedora Power Server High Performance (>10 million objects) Storage expansion schemes Mirroring and Replication Repository clustering Load balancing Preservation feature set Quality of Service (QoS) and Fault Tolerance ? Object Creation Tools Simple workflow utilities based on content models Object “workbenches” Web interface for document/content submission

Questions