GSLIS Research Showcase, April 9, 2010 Moving from science experiment to library service GSLIS Research Showcase, April 9, 2010 John Unsworth, Dean Graduate School of Library and Information Science

Funded for two years (2007-2009) by the Andrew W. Mellon Foundation Focus: apply text-mining tools to digital libraries in the humanities; facilitate “reading at library scale.” Funded for two years (2007-2009) by the Andrew W. Mellon Foundation Involved faculty and staff at Illinois (GSLIS and NCSA), Northwestern, Nebraska, Maryland, Alberta, McMaster Content (150M words of literary text) contributed by Virginia, Indiana, UNC, ProQuest, Cengage Coverage: literature of many genres, in English, from 1600- 1920s 2010 GSLIS Research Showcase

In 2009, project was complete, meaning: All texts had been normalized to TEI-A markup and modern spelling (with old spellings preserved), part-of-speech tagged, provided with enhanced item-level metadata, and ingested into a database. A web-based user interface had been produced, allowing users to define collections, select analytic routines and parameters, and examine or export results as tables or visualizations. 2010 GSLIS Research Showcase

However, one of the deliverables promised in the original grant proposal was: Beta installations of MONK alongside several large collections provided by libraries or publishers, hosted on their servers Tim Cole at the UIUC Library had been keeping up with the project as it progressed, and after wrapping up the work on interface and content, I worked with him and Mike Grady (CITES) to produce this deliverable. 2010 GSLIS Research Showcase

However, one of the deliverables promised in the original grant proposal was: Beta installations of MONK alongside several large collections provided by libraries or publishers, hosted on their servers Tim Cole at the UIUC Library had been keeping up with the project as it progressed, and after wrapping up the work on interface and content, I worked with him and Mike Grady (CITES) to produce this deliverable. 2010 GSLIS Research Showcase

Authentication was required because the texts from ProQuest and Cengage (about 100M of our 150M words) were licensed to CIC universities…. All the CIC libraries licensed EEBO and ECCO, but only about half licensed ProQuest’s 19th-century fiction collection ProQuest agreed to allow all CIC institutions access to these texts in MONK, so all we needed was a mechanism for authentication. CIC CIOs had recently agreed to deploy a federated identity management system, called InCommon, and they were willing to provide some funding for a proof-of-concept integration of MONK with InCommon, allowing MONK to be presented as a library service. 2010 GSLIS Research Showcase

2010 GSLIS Research Showcase

“In an email last week, I hinted that we had some collective work to do to complete the Shibboleth access protocol for MONK text analysis functionality.  For Shibboleth to work, authenticated user information has to be pre-loaded.  That’s relatively easy when we have a couple of hundred known users as is the case for CICme, but is a project of different order when we’re trying to authenticate 400,000 users.  Somehow, someway, the University of Illinois Library—which manages the MONK servers— needs to know who on each of the CIC campuses “deserves” to be granted access to MONK. “  2010 GSLIS Research Showcase

-- email from Mark Sandler, CIC “That list will look a lot like your library circulation or e- license authenticated user lists, but in this case you’d be transferring the data to another agency and that will make some of your campus colleagues—registrars and H.R. folks—nervous.  They’ll want to know who is getting the data and why, what the privacy policies are for the University of Illinois, what “attributes” (name, email address, campus status, etc) are being released, plans to refresh the data, etc. “ -- email from Mark Sandler, CIC 2010 GSLIS Research Showcase

At this point, we’re still working on getting access opened up to all CIC users. The problems have much more to do with policy, law, and institutional process than they have to do with technical challenges, although there are certain technicalities in MONK that raise policy problems—for example, the need to have identity information persist, so that work can be conducted in multiple sessions. Moral of the story? 2010 GSLIS Research Showcase