Choosing Technology That Can Evolve With User Needs VALA 2006 Melbourne, Australia February 2006 Sandy Payette Co-Director, Fedora Project Researcher, Cornell Information Science A service-oriented approach to e-research, e-scholarship, and advanced scholarly publication

Outline Connecting with Users What are the motivating contexts for libraries? –e-research and e-scholarship –Advanced digital libraries –Scholarly publication How do we position for the future? –Goals for the “new order” –Enabling Technologies Research and development at Cornell –Fedora Service Framework –National Science Digital Library –Pathways Moving Forward and Conclusions

Connecting with Users How are user needs evolving? Do we understand expectations of younger generation? Are we “hip?” Can we see current trends”? –Behavior –Technology Can we choose technology appropriately?

Upcoming Generation of Scholars Age 10 - play –Yahoo (music) –Google (bios, animals) –Neo pets (community) –Powerpoint (expression) Age 20 – social and study –Blogging –IM –Google –BitTorrent –Craigslist

How much should we worry now about choosing technology for evolving user needs? A Lot! Recent questions by member of audience: –“Do scholars really want new stuff or are we trying to hard to architect things that we think they want?” Let’s examine what’s already going on and position for the future…

Scholarly and Scientific Communities Documents  Integrated Information Networks

Early signs of change… Grid computing in sciences –Share computing resources –Share services and distributed virtual file systems –Examples Storage Resource Broker (SRB) Open Grid Services Infrastructure National Virtual Observatory ( Humanities computing –Hyperlinked historical documentary editions –New Forms of Digital Scholarship Rossetti archive ( Valley of the Shadow ( Perseus (

New Contexts - user and technical User Contexts Technical Contexts

First… User contexts

Key areas for connecting with users E-research E-scholarship Advanced digital library New models of scholarly communication

Technical contexts: understanding key trends…

Relevant Technology Trends Service-oriented architecture Web 2.0 Semantic Web SOA Web 2.0 RDF OWL-S OWL

Service-oriented architectures (SOA) Characteristics of services –Modular, atomic –Well-defined interfaces –Loosely coupled –Like building blocks –Standards for invoking operations (e.g., SOAP/REST, XML) Benefits –Flexibility –Enable creation of higher-level services –Enable customized end-user applications –Re-use services in different contexts –Evolution: create new services as needed –Orchestrate services to fulfill a process monolithic application

From S. Wilson, K. Blinco and D. Rehak Service Oriented Frameworks JISC/DEST Service Framework

Simple Example: Web Service using SOAP My Application SOAP/HTTP Google Web Service Request (XML) Response (XML) doSpellingSuggestion(payet) payette

Looking ahead…

Web 2.0

Implications of Web 2.0 Key themes –Services (not packaged apps) –Architecture of participation –Remix/transform data sources –Harness collective intelligence Emergent Behavior –Upcoming generations of scholars will have a completely different paradigm and expectations regarding technology –Collaborative classification (e.g., flickr) –Power of collective intelligence (amazon) –Alternative trust models (reputation – ebay; open-source)

Semantic Web Resource Description Framework (RDF) –data model for resources and relationships between them Ontologies –OWL to describe information resources –OWL-S to describe web services Rich, extensible description –no “fixed schema” –Relationships and graph-based models Knowledge inference –Equivalence –Transitivity A  B; B  C; A  C

Users and technology in action…

1.Creation and publication of new forms of “information units” 2.Services to better enable the processes of research and scholarship 3.Knowledge environments that captures semantic and factual relationships among information units 4.Promote information re-use and contextualization 5.Facilitate collaborative activity and capture information that is created as a byproduct of it Goals for enabling users in the “New Order”

Support the new “information unit” Documents Text Data Simulations Images Video Computations Automated Analyses Data

Key Projects at Cornell University Fedora Service Framework NSF Pathways (Cornell/LANL) National Science Digital Library (NSDL)

The Fedora Project Fedora –Flexible –Extensible –Digital –Object –Repository –Architecture History –Cornell Research (1997-) DARPA and NSF-funded research and reference implementation Distributed, Interoperable Repositories (experiments with CNRI) –Open Source Project (2002-present) Andrew W. Mellon Foundation funded Joint development by Cornell University and University of Virginia SOA RDF

Fedora Digital Objects Flexible object model can support –Documents, articles, journals –Electronic Scholarly Texts –Digital Images –Complex multimedia publications –Datasets –Metadata –Learning objects –More… Create “networks” of objects –Define object relationships and other properties via RDF –Collection/member; part/whole; etc.

Network of Digital Objects in a Fedora Repository

Fedora Service Framework ( )

eSciDoc (Max Planck Society and Fiz Karlsruhe)

Pathways Project SOA OWL-S A new system for scholarly communication OWL

Pathways – motivating context Decompose and distribute traditional steps in scholarly publishing value chain 1 –Registration – claim precedence for a scholarly finding. –Certification - establish validity of scholarly claim –Awareness - discover and access claims and findings –Archiving - preserves the scholarly record over time –Rewarding - based on metrics derived from that system Add new services to the mix –Workflow –Collaborative functions (e.g., annotation, re-use) –Data mining and analysis –Preservation monitoring and migration 1. Roosendaal and Geurts 1997

Pathways Vision: Interoperable Information Model Most things can be represented as a graph of nodes and arcs. Cornell University and Los Alamos Nat’l Lab

Service pathways (decomposed and distributed) Cornell University and Los Alamos Nat’l Lab

Pathways Challenges Phase 1 Current situation –Heterogeneous repository systems –Heterogeneous object models (or no object model) –Multiple protocols and service APIs –Services lacking formal interface definitions Can these ever play nicely together? Need common abstractions… –Ontology-based Information model –Ontology-based Service model

Core-1 Ontology: “Article” Example Cornell University and Los Alamos Nat’l Lab

Building Block: Repository Integration Cornell University and Los Alamos Nat’l Lab

NSDL – Core Integration SOA Advanced Digital Libraries: Beyond Search and Access Web 2.0 RDF

Information Flow in Traditional Library Knowledge In-BandOut-of-Band

Information Flow in the Digital Library In-BandOut-of-Band Knowledge Annotations Quality Ratings Relationships Reviews

NSDL Data Repository – How? Data as the asset –Structured core data model –Digital objects –Relationships –Augmented with unstructured and semi-structured Expose knowledge base via core service API NDR Technologies –Fedora repository – 2 million digital objects –Kowari RDF triplestore – 160 million triples –Services – both SOAP and REST

Conclusions: Practical steps, ongoing challenges…

Choosing technology to evolve with user needs… now Think in terms of flexible service frameworks Define fundamental services for libraries Repositories as web services Support for complex digital objects –Local and remote content –Mixed genre  documents, data, images, everything… –Dynamic views –XML expressions (esp. for ingest/export and migration) Model common entities with ontology-based metadata –Hope for interoperability via semantics –Relationships among objects are key

Ongoing Challenges Low barrier to entry –Simple protocols (e.g., like OAI) –Light-weight (REST vs. SOAP?) –Simple tools to create overlays Service matching (object-to-service) –Ontologies to expose objects with formats and semantics –OWL-S for semantic service description –Matching-making algorithms Security and Trust –Authentication and trust among repositories and services –Interoperability of authorization policy Preservation –Distributed and dynamic digital objects a challenging reality

Thank You! Questions and Comments…