By Abe Lederman President and CTO June 26, 2011 Understanding Differences Between Federated Search and Discovery Services By Abe Lederman President and CTO June 26, 2011 Better Title:

About Deep Web Technologies... Founded by Abe Lederman in 2002 A co-founder of Verity 20+ years in information business 24 person company based in Santa Fe, New Mexico Over $5M in R&D Pioneered federated search Developed high profile applications Santa Fe

What Is Federated Search? Federated Search allows users to submit a real-time search in parallel to multiple information sources and retrieve aggregated, ranked and de-duplicated results.

One Search, Many Sources OPACs Blogs Subscription Sources eBooks Wikis Enter Your Search… Begin Search Internal Databases Public Web Sources Journals

Federated Search Has Gotten A Bad Rap It is too slow Connectors break Brings back too few results from each source Brings back too many results Unable to rank results well (meta-data differences, lack of info)

Federated Search vs. Discovery Services Index No Index One unified index Sources No limit Must have agreements Content Bias None Possibly Information Currency Real-time return Depends on index update Preparation Connector Development Index Development Speed 2 – 30 seconds 1 – 2 seconds Control Select sources No ability to select Alerts Yes Some

Landscape is Not So Clear Summon (ProQuest) Discovery Service EDS (EBSCO) Discovery Service + Federated Search WorldCat Local (OCLC) Primo (Ex Libris) Encore Synergy (Innovative Interfaces) Limited Discovery Service + Federated Search Explorit (Deep Web Technologies) Federated Search

Content Neutrality Do you comply with the International Coalition of Library Consortium (ICOLC) Statement, Principle 3? “We encourage publishers to allow their content to be made available through numerous vendors appropriate for their subject matter. We also encourage online providers and aggregators to allow their metadata to be included in emerging discovery layer services on a non-exclusive basis.” --Carl Grant, "Gladiators" to perform sleight-of-hand at Charleston Conference.” Commentary from Carl Grant. October 30, 2010.

Content Neutrality -Points by Carl Grant Is there unbiased ranking of content? Does competition have access to content? Can you control the ranking of results? Does the library have control over their content? Redo slide – Carl Grant’s remarks Table may not be that useful Focus on Ebsco and Proquest – focus on selling content Base it on Carl Grant;’s article (gladiators) and give him credit Ebsco and Summon’s will sometimes provide it for free so they can sell content

Lack of Transparency of Discovery Service Vendors What is being indexed? Which Journals/Databases What period is covered Currency of information Indexing full-text vs. meta-data only

Ability to Integrate How easy/hard/expensive/time consuming is it to add my catalog and other internal/special sources to index? How well do you authenticate/ integrate with link resolvers?

When Should You Choose Federated Search? Access to up-to-date information is important. You want control of your sources. You want to search internal/non-mainstream sources Your research is specialized (ex. medical/legal) You have a wide range of subscribed content (ex. EBSCO and ProQuest)

Getting Federated Search Right Display results incrementally Proactively monitor information sources Optimized connectors bring back greater number of high quality results Do relevance ranking well Present best results through filters and clusters

More Reading… Federated Search Primer Quality, Not Quantity Whitepaper Blog articles Discovery Services: Over-Hyped and Under-Performed Preparing for ALA Panel and Federated Search Neutrality Discovering the need for discovery solutions that also support meta/federated searching (Carl Grant)

This Powerpoint is located here: Thank you! Questions? Please email me: Abe Lederman abe@deepwebtech.com This Powerpoint is located here: www.deepwebtech.com/ala2011.ppt