Supporting Cyberscholarship in American Social History: The DLF Aquifer Story Katherine Kott With content contributed by Susan Harum and Chick Markley CNI Task Force Meeting December 11, 2007

Focus on American Social History Online Mellon funded project within DLF Aquifer initiative Outcome to make digital collections in American culture and life easier for scholars to find and use –Aggregate content to enable multiple collections to be used as one –Deliver content through multiple channels

Who’s who American Social History Online relies on Aquifer working groups, collaborative infrastructure in place Core team focused on development and assessment, coordination with working groups –Susan Harum (UIUC) business analyst/assessment expert –Kat Hagedorn (Michigan) data analyst –Chick Markley (Freewheeling contractor) systems architect/developer –Tom Habing (UIUC) developer

DLF Aquifer participant libraries Bibliotheca Alexandrina California Digital Library Cornell University Emory University Indiana University Johns Hopkins University Library of Congress New York University Stanford University University of Illinois, Urbana-Champaign University of Michigan University of Minnesota University of Southern California University of Tennessee University of Virginia Yale University

Overview of accomplishments Architecture Portal with Zotero—initial SEO analysis MODS –Guidelines –Levels of adoption –MARC to MODS transform Harvesting workflow Agile development process

Website with Zotero in Firefox RESERVED

How we got here Initially unsure we would build a website –SEO required for commercial search service scenario –Zotero implementation requires website –Website seemed like best way to present services (SRU) for Sakai and federated search scenarios –Website could be used as cohesive way to present the project –In June 2007, team set goal to have website ready for DLF Fall Forum in November 2007

Website development Started with baseline functionality on Michigan MODS portal Added and cross checked common business functions developed by SWG (integrated with DLF Services Framework) Architecture chosen to support –Website functionality –Sakai –Federated search

TWG architectural framework

ToJSON java MODS mysql JSON mysql to SOLR java Rails ruby SOLR/Lucene java Harvester DLXS eXist java to JSON java browsers to eXist java services bots Architectural implementation

A plug for Ruby on Rails from our systems architect It's the framework DRY (don't repeat yourself) Tightly coupled testing harness Exuberant and dedicated community

Enter, Agile development Small development team, distributed environment Short timeline—no time for “traditional” FRD process Desire within digital library community for lightweight process model Emphasis on user driven development Not to be confused with final assessment activities for American Social History Online

Model Timeframe: August 29 – November 2, 2007 (8 weeks) Business Analyst identified 8 participants –1 faculty member –3 graduate students –4 user services librarians Services Working Group and Business Analyst drafted questions

Method Initial conference call on August 27th, 2007 –all participants –business analyst –systems architect –developer Questions regarding –search and find –identify –obtain –manage and use –user workflow

Initial Brainstorming call Pros –developers present to hear what participants say –developers able to prioritize next tasks immediately after the call Cons –Level of participant feedback was lower than anticipated –Too much developer involvement?

Participant feedback and development action Need to see size and scale of collection to determine how much time to spend searching and browsing –Implementation of collection registry and “browse collections” feature. Need ability to limit a search (once you have results) in many ways –Implementation of additional ways to limit after initial search –Implementation of date range limiting Importance of ability to browse –Implementation of subject clouds for browsing – still working on faceting for additional browsing Importance of visual aids, especially for undergraduate students –Implementation of Simile timeline view for results –Thumbnail previews for item selection

Integration with collection registry at UIUC

Adaptively re-used technology CDL date normalization utility UIUC thumbgrabber with asset actions Simile timeline from MIT Collectus tool from UVA Just say no to NIH syndrome

Implementation of “asset actions” using Collectus from UVa

Simile Timeline Integration

Participant wish lists for future development Limiting Search Results –users want to “drill down” in the search continue to narrow using different limiters, e.g. rights for use Social Networking and Tagging –Implementation of StumbleIt for full records –Monitoring Zotero server development –Penn tags Sorting and Narrowing –Plan to implement faceted browsing based on NINES Collex flare development (NIH)

Tell me something I don’t know

Lessons learned Agile Assessment effective in prioritizing tasks Multiple phone interviews and updates resulted in buy in for participants Brainstorming session for distributed group not as effective as personal interviews Shorter more frequent 1:1 interviews yielded the best results Graduate students were most engaged

What’s next Add collections Workflow, including asset actions Support for static repositories Bug fixing Tagging? Faceted browse Map rights statement to CC license? Evaluate ORE model (March meeting) Develop “local implementations” Indiana –Federated UIUC More SEO Assessment planning Sustainability planning

Actively seeking collections Advantage to contributor –SEO drives traffic to your collection(s) –Benefit from date normalization Seeking collections –Pertaining to American culture and life –All formats –MODS—can be mapped from MARC –Static repository setup for contributors without OAI data providers

The DLF Aquifer engine From DLF for development to ??? for sustaining –Consistent attention –Administrative support The working groups –Organizational links –Community links Resources –From collaborators (directly and through investment in DLF) –Grant funding –Fees???

Sustaining Aquifer Part it out –Community standards –Software and services –Collections Take it to the next level? Affiliate with another (service provider) organization Move to participant library host

Resources for ongoing support “If enough people who are viewed as important want it, we will find a way to pay.” Paul Courant Assessment during development period is critical Revenue streams –Subscription –Libraries pay for services Increased visibility for collections Metadata enhancement

Emerging organizations sustaining inititiatives LOCKSS/CLOCKSS Fedora Commons DSpace Foundation Educopia Institute

Questions? Comments? Website: