1. Barriers to Interoperability Technical and Not So Technical William E. Moen School of Library and Information Sciences Texas Center for Digital Knowledge University of North Texas Denton, TX 72603 5 th Annual GILS Conference, April 7-10, 2003 Lisle, IL

2. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 2 Interoperability: Volume or value “…information production is growing at about 50% a year… Yet the amount of time people spend consuming is growing only 1.7% a year… As digital production and storage grow ever cheaper…a critical task ahead will be to stop volume from simply overwhelming value.” John Seely Brown and Paul Duguid, The Social Life of Information, 2002

3. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 3 Defining Interoperability System-oriented definition The ability of two or more systems or components to exchange information and use the exchanged information without special effort on either system User-oriented definition User’s ability to successfully search and retrieve, or otherwise interact with information in a meaningful way and have confidence in the results The condition achieved when two or more technical systems can exchange information directly in a way that is satisfactory to users of the systems (AAP)

4. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 4 Assessing Interoperability Binary Interoperable Not interoperable Continuum More or less interoperable Acceptable levels of interoperability

5. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 5 Fundamental assumptions Your users will interact with resources other than those available locally Your system will interact with other systems’ content Other systems will interact with your content Systems and content were often meant for local or proprietary use In the networked environment diverse systems and varied content must interact to serve users

6. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 6 The foundation: Communication protocols Interoperability is about communication – albeit between machines/software Two systems interchange bits/bytes Protocols (computer-to-computer communications) govern the exchange of bits/bytes Rules governing the structure, order, and content of the bits/bytes/messages between machines

7. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 7 Protocols and record formats/markup Protocols enable communication between systems Record formats/syntax/markup address resources HTML = HyperText Markup Language Markup language for simple web documents Display oriented MARC = Machine Readable Cataloging Communications format for exchange of bibliographic information XML = Extensible Markup Language Markup language for web environment Content oriented

8. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 8 Factors affecting interoperability Multiple protocols Z39.50, HTTP, SOAP, etc. Multiple data formats, syntax, metadata schemes MARC 21, UNIMARC, XML, / ISBD/AACR2-based, Dublin Core Multiple vocabularies, ontologies, disciplines LCSH, MESH, AAT Multiple languages, Multiple character sets Multiple and disparate systems Information retrieval systems, search functionality, etc. Indexing, word normalization, and word extraction policies

9. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 9 Assumptions Increasing interoperability factors Increased complexity/costs to achieve interoperability How to identify barriers to interoperability: In general? In specific applications?

10. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 10 Interoperability Testing Realizing the Vision of Networked Access to Library Resources: An Applied Research and Demonstration Project to Establish and Operate a Z39.50 Interoperability Testbed A U.S. Federal Institute of Museum and Library Services National Leadership Grant Research and demonstration project to: Establish a rigorous and trusted Z39.50 testbed Develop methodology and metrics for assessment From demonstration  operational testbed Goal: Improve Z39.50 semantic interoperability among libraries for information access and resource sharing FOR MORE INFORMATION, VISIT THE PROJECT WEBSITE… http://www.unt.edu/zinterop/

11. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 11 Components of the Testbed Test dataset 400,000 MARC 21 records from OCLC Z39.50 reference implementations Z-client, Z-server, information retrieval system Test scenarios & searches Searches with known result records from dataset Benchmarks Results of test searches against reference implementations

12. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 12 Interoperability findings Successes: Z39.50 profiles improve interoperability Z-clients and Z-servers can be configured to improve Common indexing policies support better interoperability New problems: Word extraction (what constitutes a word) Data normalization (special characters, diacritics) Indexing (leading articles, “first in field” searches) What is good enough interoperability?

13. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 13 ZLOT project The Library of Texas A statewide virtual library to extend access to information Z Implementation Component of the Library of Texas Developing a standards-based approach for the virtual library Identifying requirements for resource discovery across library catalogs and online databases Demonstrating feasibility and utility of resource discovery tools Collection Boundaries Organizational Boundaries Format Boundaries FOR MORE INFORMATION, VISIT THE PROJECT WEBSITE… http://www.unt.edu/zlot/

14. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 14 One approach for framing interoperability Information communities / communities of practice Networked information retrieval occurs within and across information communities Information communities Focal community Extended community Extra community Costs to achieve interoperability vary depending… Within focal community Within extended community Between disparate communities

15. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 15 Focal Community Community agreements exist (e.g., standards and rules) Interoperability factors reduced…more easily achieved Libraries as Focal Community  Relatively homogeneous  Standards-based MARC records  Content and structure prescribed by AACR  Commonly available access points  Use of controlled vocabularies  Z39.50 widely implemented  But…

16. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 16 Information exchange within/between… Focal Community (e.g., Libraries) Focal Community (e.g., Archives) Focal Community (e.g., Museum) Extended Community (e.g., Cultural Heritage) Focal Community (e.g., Geospatial ) Extra Community Focal Community (e.g., Natural History Museums) Extended Community Focal Community (e.g., Geospatial )

17. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 17 GILS as information community Conceptualizing GILS as information community Way to discuss barriers to interoperability: Types of data and resources? Common resource description scheme Commonly used database semantics? Common record syntax (e.g., XML) Commonly understood access points/search concepts? Commonly used controlled vocabulary? Common protocols for information retrieval and exchange?

18. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 18 Services context for interoperability Information communities provide setting Information communities offer services Value-added services to end users Infrastructure services for providers Mapping a service environment … Needs Services Service interfaces Content … to identify interoperability barriers/challenges

19. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 19 Characterizing the networked library Virtual library components An environment for provision of Services Resources Not a digital library Not all resources available are in digital/electronic form An evolving product that responds to Users’ needs Available resources Current and emerging technologies

20. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 20 Service-centric networked library User Groups Resources Technology Management SERVICES

21. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 21 Networked library components User groups Local, onsite Distributed, remote Services Local Collaborative Remote Resources Information Local Distributed Human Technology Local applications Distributed applications Standards Management Local Collaborative

22. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 22 User-driven services LIBRARY SERVICES User Group User Group User Group User Group Library Resources Library Technology

23. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 23 Building services for the networked library What are the requirements for the services? What are alternatives for deploying the services? What technologies are available to support the services? What standards are available? What are the tradeoffs of standards and non- standards approaches?

24. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 24 Services for the networked library Infrastructure/internal Collection identification, selection, and acquisition/access Collection building and digitization Collection organization and preparation Inter-organization access Trust and authentication Billing and payment Preservation and archiving Patron Resource discovery service Resource access service Reference service Instruction service Patron account service

25. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 25 Virtual library services environment

26. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 26 Architecture PROVIDER LAYER SERVICES LAYER USER LAYER Communication Protocols

27. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 27 Provider layer Digital Content LibrarianLibrarySupplier Provider Type Dependent on VL Service Resource Discovery Service Virtual Reference Service Resource Sharing Service Document Delivery Service USER LAYER 1 3 2

28. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 28 Services layer Standard Information Interchange Among Services FinancialAuthorization Resource Sharing Patron Account Library Information Education Resource Discovery ReferencePersonalization

29. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 29 Resource discovery: Services view Browse Select Collection Evaluate Search Common User Interface Personalize Save Acquire

30. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 30 Resource discovery: Architectural view CONTENT TexShareTRAILVCOthers USER LAYER LibraryHomeOfficeMobile SERVICES LAYER Resource Discovery Z39.50 HTTP OTHER Directory Authentication SOAP

31. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 31 Finding / exposing Finding I can find only what’s been exposed Exposing How content is exposed determines the how and if I can find it Common agreements on exposing and finding Protocols Syntaxes Semantics Database Language Goal is to connect users to information Strategy is interoperability on multiple levels (protocol, syntax, semantics, organization, etc.)

32. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 32 Service interfaces framework

33. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 33 Resource discovery: Operational view Content OAI Repository Standards-based Search & Retrieval Protocols Proprietary or Non-standard Search & Retrieval Protocols Content Gateway Common Interface

34. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 34 Infrastructure standards -- NISO ANSI/NISO Z39.50: Information Retrieval (Z39.50): Application Service Definition and Protocol Specification ISO 10160 & ISO 10161: Interlibrary Loan Application Service Definition & Protocol Specification ANSI/NISO Z39.83: Circulation Interchange Protocol (NCIP) ANSI/NISO Z39.84: Syntax for the Digital Object Identifier - 2000 ANSI/NISO Z39.85: Dublin Core Metadata Element Set - 2001 ANSI/NISO Z39.88: The OpenURL Framework for Context-Sensitive Services (2002) ANSI/NISO Z39.89: The U.S. National Z39.50 Profile for Library Applications (2002) Networked Reference Services (standards development underway) MetaSearch Initiative (may lead to new standards activities)

35. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 35 Standards supporting services Each standard enables different services Networked information retrieval (Z39.50) Automated interlibrary loan (ILL) Circulation applications (NCIP) Reference linking (OpenURL) Resource description (Dublin Core) But they can work together to support robust and interoperable information access and resource sharing services

36. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 36 Web services Building Blocks for Constructing Applications Service A Resource Discovery Service B Authorization

37. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 37 Web services: Finding a service Service A Resource Discovery Service B Authorization Service Registry Service Requestor Service Provider

38. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 38 Web services: Enabling technologies Service Registry Service Requestor Service Provider XML-Based Information Exchange SOAP UDDI - WSDL WSDL UDDI: Universal Description, Discovery and Integration WSDL: Web Service Description Framework SOAP: Simple Object Access Protocol XML: Extensible Markup Language

39. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 39 Web services Resource Discovery Service Service B Authorization Service Registry UDDI - WSDL Publish UDDI - WSDL Find WSDL - SOAP Invoke Service

40. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 40 ZING – Z 39.50 I nternational N ext G eneration Make intellectual/semantic content of Z39.50 more broadly available Make Z39.50 more attractive by lowering barriers to implementation Use of XML – to represent and encode data Use of HTTP – for transport Use of SOAP – for interaction between client and server based on Remote Procedure Call (RPC) Several initiatives: ZOOM, ez39.50, ZeeRex, SRW/U FOR MORE INFORMATION, VISIT THE PROJECT WEBSITE… http://www.loc.gov/z3950/agency/zing/zing.html

41. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 41 Search/Retrieve Web Service – Overview Builds on Z39.50 and web technologies Retains Z39.50 concepts: Results sets Abstract access points Abstract record schemas Explain & Diagnostics Web technologies: XML, SOAP/RPC, HTTP Combines several Z39.50 features into two “operation types” Search/Retrieve Explain Provides a standards-based method for representing and issuing queries, and retrieving records

42. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 42 Interoperability and the user Technical interoperability as foundation Value-adding services respond to user tasks Usability of the service Reliable Predictable Ease of use User interface …

43. Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 43 Final thoughts Interoperability is essential Interoperability is multi-faceted Interoperability is ultimately based on agreements Formal standards Community agreements Agreements result from Social processes Willingness to collaborate Balance of local needs in the networked environment It’s the “not so technical” we need to work on