© DSpace User Group Meeting April 21, 2006 — Bergen, Norway William Reilly, Larry Stone — MIT Libraries 2006 v _0945 Technical Introduction To and Initial Use Of the Lightweight Network Interface (LNI) (—to DSpace!)

DSUG Bergen 2006 Lightweight Network Interface (LNI) 2 Today's Talk 1.Context: CWSpace project 2.Web Services "Lightweight Network Interface" 3.WebDAV & DSpace "Technical Introduction To…" 4.MIT OpenCourseWare Submit (example) "Initial Use Of…" 5.Appeal for Use Cases, Future Dev!

DSUG Bergen 2006 Lightweight Network Interface (LNI) 3 Context: CWSpace project

DSUG Bergen 2006 Lightweight Network Interface (LNI) 4 Project Overview Vision: Increase value in Institute publication: OCW –Also: Increase value in Open Source platform: DSpace * CLE = Collaborative Learning Environment Goal: Archive OCW to MIT’s DSpace –Also: Make content available to CLEs * CWSpace: MIT iCampus project, nearing completion Method: Interoperability of systems –Use Content Packaging standard –Use Web Services standard

DSUG Bergen 2006 Lightweight Network Interface (LNI) 5 Perceived Need: Interoperability Obvious observation: Higher Ed. Technologies increasingly seeking integration with Library & Repository holdings Not so obvious(?): Libraries seeing their charter to include collection, preservation, dissemination of Teaching & Learning materials (?)

DSUG Bergen 2006 Lightweight Network Interface (LNI) 6 Interop: What’s Available Normalized data model (out of chaos) –"Sections" fit well to publication organization Normalized content files (.PDF) –Reduced ability to disaggregate, re-purpose 1 of 4

DSUG Bergen 2006 Lightweight Network Interface (LNI) 7 Interop: What’s Achievable 2 of 4

DSUG Bergen 2006 Lightweight Network Interface (LNI) 8 Interop: Challenges 3 of 4

DSUG Bergen 2006 Lightweight Network Interface (LNI) 9 Interop: Possible Resolution? 4 of 4

DSUG Bergen 2006 Lightweight Network Interface (LNI) 10 Web Services: "Lightweight Network Interface"

DSUG Bergen 2006 Lightweight Network Interface (LNI) 11 DSpace dev: LNI & PKGers

DSUG Bergen 2006 Lightweight Network Interface (LNI) 12 DSpace dev: LNI & PKGers

DSUG Bergen 2006 Lightweight Network Interface (LNI) 13 What was the Problem…? Content Providers wanted… Networked Access: Programmatically access the DSpace object model, but not running on the DSpace server Basic (Lightweight Network Interface (LNI)) [ Access, Deposit, Limited Admin. Modifications ] Submit Retrieve Item w. Metadata Build own U/I Collection Mgt, Map Items Advanced (possible future LNI (or other WS!) dev.) [ Replace, Withdraw, Delete, 3rd Party, Workflow ] Specialist metadata Automated metadata Quality control checks Format migration … DSpace Content Provider

DSUG Bergen 2006 Lightweight Network Interface (LNI) 14 What Is the LNI? The "Lightweight Network Interface" is: A remote API to control DSpace It's a separate module at the "application layer" — peer to the Web U/I, can run in same Tomcat It's essentially WebDAV: Extensions to HTTP –A SOAP (with WSDL) equivalent is provided It's a mapping: — –a "complete and comprehensive" view of the DSpace public object API… (the "business logic" layer) –…as mapped to WebDAV semantics

DSUG Bergen 2006 Lightweight Network Interface (LNI) 15 How Do I Use It? LNI is not for direct, interactive access * –e.g. HTTP browser Instead, the LNI is accessed via an application you develop Independence: Language, platform, machine –Code: Perl, C, Java, other WebDAV libraries –OSes: "Web Services" all about connecting different platforms Further "independence": choice of WebDAV or SOAP equivalent Server side? Minimal — install LNI & let it run. –Use existing DSpace authentication, e-People

DSUG Bergen 2006 Lightweight Network Interface (LNI) 16 WebDAV & DSpace: "Technical Introduction To" a.k.a. Micro-Intro-Tutorial-Review…

DSUG Bergen 2006 Lightweight Network Interface (LNI) 17 WebDAV & LNI WebDAV = "Distributed Authoring and Versioning" –using a Web server, rendered "DAV-able" WebDAV — 1999 spec. from the IETF (Internet Engineering Task Force) RFC 2518 – LNI implementation of WebDAV: –Compliance Level 1 (no Versioning or Locking) –To date: No DELETE nor MKCOL; other limitations… –…room for future development! ;^) LNI: DSpace model mapped to WebDAV model …

DSUG Bergen 2006 Lightweight Network Interface (LNI) 18 DSpace: Model DSpace digital archive Differs from file system –Submit URI not known –Workspace –Workflow (optional) –Metadata –Bundles –Handle to URL resolution (& vice versa)

DSUG Bergen 2006 Lightweight Network Interface (LNI) 19 WebDAV: Model, Basics Each Resource named by a URL Properties are "name value" pairs Properties are extensible Property values can be found, and modified DAV = "Distributed Authoring and Versioning" URL

DSUG Bergen 2006 Lightweight Network Interface (LNI) 20 WebDAV: Model, Fuller Resources can be Collections or leaf nodes Collections can nest Everything can be discovered descending from the DAV server's root '/' resource

DSUG Bergen 2006 Lightweight Network Interface (LNI) 21 DSpace Mapped to WebDAV

DSUG Bergen 2006 Lightweight Network Interface (LNI) 22 DSpace Translated to WebDAV When you have a Handle (e.g. from other services like SRW/U, OAI- PMH), then use LNI "lookup" functionality to convert it to LNI WebDAV URL. To get a Handle (given a LNI URL), do PROPFIND on that URL and read the "Handle" property value. LNI URL Handle

DSUG Bergen 2006 Lightweight Network Interface (LNI) 23 DSpace Implemented as WebDAV 1.Properties –WebDAV –Custom for DSpace 2.DSpace LNI URLs 3.WebDAV Methods

DSUG Bergen 2006 Lightweight Network Interface (LNI) 24 Properties: WebDAV DAV:creationdate DAV:displayname DAV:getcontentlength DAV:getcontenttype DAV:getlastmodified DAV:resourcetype … Akin to file system properties, with add'l. metadata

DSUG Bergen 2006 Lightweight Network Interface (LNI) 25 Properties: Custom DSpace (1 of 3) ALL DSpace Objects: –dspace:type "BITSTREAM", "ITEM", "COMMUNITY" etc. –DAV:resourcetype "COLLECTION" etc. –dspace:displayname –dspace:current-user-privilege-set

DSUG Bergen 2006 Lightweight Network Interface (LNI) 26 Properties: Custom DSpace (2 of 3) Community DSpace Objects –dspace:logo –dspace:short_description –dspace:side_bar_text –dspace:handle –…

DSUG Bergen 2006 Lightweight Network Interface (LNI) 27 Properties: Custom DSpace (3 of 3) Bitstream DSpace Objects –DAV:getcontentlength = getSize() –dspace:format = getFormat().getID() –dspace:sequence_id = getSequenceID() –… Many more DSpace Objects (some 10) –With many more Properties (some 87)

DSUG Bergen 2006 Lightweight Network Interface (LNI) 28 URLs: DSpace LNI (1 of 3) Site – Community – Collection –

DSUG Bergen 2006 Lightweight Network Interface (LNI) 29 URLs: DSpace LNI (2 of 3) Item – dso_1721.1$5543 – Bitstream – bitstream_13 – bitstream_13.pdf

DSUG Bergen 2006 Lightweight Network Interface (LNI) 30 URLs: DSpace LNI (3 of 3) Nota Bene The LNI URLs for DSpace objects are: only for use with the LNI. –They are therefore not to be used: as persistent URLs (e.g. bookmark, store for later use) as interactive URLs (e.g. browser) opaque — you must use the LNI to resolve to Handles Therefore please to not : Roll your own LNI URLs to send Infer Handles & Etc. from LNI URLs received

DSUG Bergen 2006 Lightweight Network Interface (LNI) 31 WebDAV Methods in LNI PROPFIND - very powerful request with multiple functions: –list names of properties –return property values –recurse through 1 or more levels of a "collection" resource. PROPPATCH - changes value of or deletes properties on one resource. GET - retrieves contents of a resource PUT - replace or add a new resource to a collection COPY - map a resource ("Item") to a different Collection. As noted, equivalent SOAP methods are provided Not Implemented: Locking, Versioning.

DSUG Bergen 2006 Lightweight Network Interface (LNI) 32 Initial Use: OpenCourseWare "Submit"

DSUG Bergen 2006 Lightweight Network Interface (LNI) 33 Current DSpace (pre-LNI) Content Provider, currently Batch import (& export) End user Web U/I dspace-admin Web U/I SRW, OAI-PMH DSpace Batch Importer OCW Comm. "Biology (7)" "Physics (8)" DSpace Object API Web U/I Web, but Non Auto- mated Non Std. Non Web Workflow Adm Item

DSUG Bergen 2006 Lightweight Network Interface (LNI) 34 DSpace with LNI Content Provider using LNI: LNI SOAP end-point URL LNI WebDAV URLs dspace-admin (partial) SRW, OAI-PMH –(Not replicated in LNI) Still available: –Batch import (& export) –End user Web U/I DSpace DSpaceLNI.class OCW Comm. "Biology (7)" DSpace Object API DSpace LNI WebDAV - Web. - Stds. - Batch- able. "Physics (8)" Item

DSUG Bergen 2006 Lightweight Network Interface (LNI) 35 Submission: Overview OpenCourseWare Client Needs 3 Values : 1.LNI SOAP end-point URL ace.mit.edu/dspace- lni/lni/DSpaceLNI 2.Handle for OCW Community = hdl:1721.1/125 3.Dept. Name (MIT "Course" Number) for Course to be submitted = "Physics (8)" DSpace DSpaceLNI.class OCW Comm /125 "Biology (7)" DSpace Object API DSpace LNI WebDAV Lang. - Indep. Platform - Indep. "Physics (8)" Item

DSUG Bergen 2006 Lightweight Network Interface (LNI) 36 Submission: 1. Lookup Request: OCW Comm. HANDLE LNI lookup() = lni/dav/lookup/handle/1721.1/1 25 Response: OCW Comm. LNI URL $125 DSpace DSpaceLNI.class OCW Comm /125 dso_1721.1$125 "Biology (7)" dso_1721.1$197 "Physics (8)" dso_1721.1$253 DSpace Object API DSpace LNI WebDAV Lookup

DSUG Bergen 2006 Lightweight Network Interface (LNI) 37 Submission: 2. Collections Request: PROPFIND on LNI URL for children DSpace Collections' "displaynames": Response: MULTISTATUS Biology (7) Physics (8)... DSpace DSpaceLNI.class OCW Comm /125 dso_1721.1$125 "Biology (7)" dso_1721.1$197 "Physics (8)" dso_1721.1$253 DSpace Object API DSpace LNI WebDAV Iterate…

DSUG Bergen 2006 Lightweight Network Interface (LNI) 38 Submission: 3. Submit (PUT) Request: PUT to DSpace Collection LNI URL, including specification of Packager Type (IMS-CP): PUT /dspace/dav/dso_1721.1$253?packa ge=OCW-IMSCP....package contents in body... Response: LNI URL for DSpace Item in "Location" Header: HTTP/ OK Location: _1721.1$ other headers.... DSpace DSpaceLNI.class OCW Comm /125 dso_1721.1$125 "Physics (8)" dso_1721.1$253 DSpace Object API DSpace LNI WebDAV Ingest Item dso_1721.1$836

DSUG Bergen 2006 Lightweight Network Interface (LNI) 39 Submission: 4. Item Handle Request: PROPFIND on DSpace Item LNI URL for the DSpace Handle: Response: Properties Requested === PROPERTIES Successfully returned: dspace:handle = " hdl:1721.1/836 " DSpace DSpaceLNI.class OCW Comm /125 dso_1721.1$125 "Physics (8)" dso_1721.1$253 DSpace Object API DSpace LNI WebDAV Reverse Lookup Item dso_1721.1$ /836 OCW Content Mgt. System Stores DSpace Handle SQL Server Table: hdl:1721.1/836

DSUG Bergen 2006 Lightweight Network Interface (LNI) 40 Submission: Summary In this 4-step OpenCourseWare submission example, we've seen: SOAP endpoint URL DSpace Handles DSpace LNI URLs LNI Lookup() function WebDAV Properties WebDAV Methods PROPFIND (LNI request) MULTISTATUS (LNI response) LNI "Reverse Lookup" WebDAV Headers (Location; Success/Fail) DSpace DSpaceLNI.class OCW Comm /125 "Physics (8)" /253 DSpace Object API DSpace LNI WebDAV Item /836

DSUG Bergen 2006 Lightweight Network Interface (LNI) 41 Subsequent Use: [ your use case here ]

DSUG Bergen 2006 Lightweight Network Interface (LNI) 42 LNI with Other Protocols Search –SRU/W { + LNI } Dublin Core metadata –OAI-PMH { + LNI } –METS Packager (DC only) { using LNI }

DSUG Bergen 2006 Lightweight Network Interface (LNI) 43 LNI Security Considerations Security –SSL (see wiki page) –Stateless –E-Person userid & pw with each invocation Authorization –E-Person – –Area for further development!

DSUG Bergen 2006 Lightweight Network Interface (LNI) 44 Code Status Downloads /!\ This code should not be considered stable. It is subject to change at any time, and may not work with the current development or released version of DSpace. Use it for exploration and evaluation only. There are not yet any final plans for releasing it; much depends on the course of the AddOnMechanism project. Patch to DSpace 1.4alpha source (28-Mar-06) New files to add to DSpace 1.4 alpha (28-Mar-06) Pre-built client library JAR and WSDL -- not needed when building from source.

DSUG Bergen 2006 Lightweight Network Interface (LNI) 45 Future Development Other DSpace objects –Bitstream Format Registry –Metadata Registry … Other WebDAV methods –PUT (for Replacement) –DELETE –MKCOL … Other Use Cases, Needs –Basic, Advanced …

© DSpace User Group Meeting April 21, 2006 — Bergen, Norway William Reilly, Larry Stone — MIT Libraries 2006 Lightweight Network Interface (LNI) Questions, & Etc.

DSUG Bergen 2006 Lightweight Network Interface (LNI) 47 Abstract The "Lightweight Network Interface" (LNI) is a work in progress to provide (yet) another way to gain networked access to the DSpace application-level API. Initially developed to support the requirement that "Web Services" be used on the CWSpace project (archiving MIT's OpenCourseWare (OCW) to DSpace), the ensuing vibrant (!) discussion regarding possible technical approaches (SOAP, RESTful, WebDAV, XML over HTTP) led us to define some high level goals for how this new interface ought to be shaped (e.g. platform-neutral; based on mature standards; readily extensible; work with (not replicate) existing remote APIs (SRW, OAI-PMH); comprehensive view of DSpace model; etc.). Dubbed the "Lightweight" network interface, the intent has been to largely adopt the robust and proven protocols (and "verbs") of WebDAV and HTTP, and to establish a proposed mapping of WebDAV's Resource-centric view onto the DSpace object model. (Note that a set of SOAP methods has also been developed on top of the WebDAV functionality, such that in fact either approach is supported.) The details of the resulting proposed API were posted on a lengthy page to the DSpace Wiki; a "smoketest" Java client to the LNI was posted to the CWSpace Wiki (along with detailed instructions on how to work with SSL and the LNI). This presentation introduces the technology, the API, the rationale for the approach, and a discussion of the mapping to the DSpace data model as well as example uses of the LNI to DSpace (e.g. submit; disseminate; copy to another collection; list collections; etc.). Closely related DSpace development activity of the Plugin Manager, Packager Plugins, and Crosswalk Plugins will be addressed briefly as well, as these are used in conjunction with the LNI on the CWSpace project for submission (and dissemination) of IMS Content Packages of OCW content (courseware websites).