Federated Searching In a Nutshell 21 st Century Literacies 18 November 2005

Federated Searching Rex Krajewski Reference Services Librarian Simmons College web.simmons.edu/~krajewsk/library/federatedsearching.html

What is Federated Searching? Process of searching multiple sources simultaneously

More Specifically Conducted using federated search engines

Federated Searching AKA parallel search meta search broadcast search one-search cross searching cross-database searching distributed searching single search

Well-known Models Dialog allows user to search many databases simultaneously (think: Dialindex One Search Categories) Dialindex One Search Categories Metasearch Engines—like Dogpile, Clusty, Mamma, and Metacrawler— allow users to search multiple search engines’ top results with a single searchDogpile ClustyMammaMetacrawler

Federated Searching The term “federated searching” came from the Open Archives Initiative’s Protocol for Metadata Harvesting (OIA-PMH) Single server harvests metadata for records from the holdings databases of “federated” databases. The resulting centralized data is searchable.

What’s in a name? NISO says “metasearching” Vendors prefer “federated search engines” because metasearching use by meta-search engines like Dogpile, Mamma, and Ask Jeeves.

Federated vs. Meta Meta search engines aggregate material that has already been searched and is freely available to anyone on the Web. Federated search engines run searches at the time they are queried and can include proprietary material and other Internet content not spidered by search engines

Why Federated Searching? Libraries offer hundreds of databases to search Allows libraries and users alike to manage the hundreds of database search tools.

Digital Reference: Why? 97% of surveyed adult internet users expect to find the information they need on government, health, commerce, and news on the internet.* *Counting on the Internet: Most expect to find key information online, most find the information they seek,many now turn to the Internet first, Pew Internet& American Life Project, 29 September 2002, Date accessed: 15 November 2005.

Digital Reference: Why? “Nearly three-quarters (73%) of college students say they use the Internet more than the library, while only 9% said they use the library more than the Internet for information searching.”* *The Internet Goes to College: How Students are Living in the Future with Today's Technology, Pew Internet& American Life Project, 15 September 2002, Date accessed: 03 November 2005.

Digital Reference: Why? “71% of students report using the Internet at their primary source for their last major project, and they also report accessing online study aids like Sparknotes or CliffNotes.”* * The Internet and Education, Pew Internet& American Life Project, 01 September, 2001, Date accessed: 21 October, 2005

Why Federated Searching In the age of Google, users expect the world of knowledge available quickly and easily at their fingertips…they expect the same kind of one-stop searching to be available in the library

How It Plays Out Searcher enters a single search into the federated search engine women and sports

What does the user see? A single research entry point A familiar interface A consistent search syntax

How It Plays Out Federated search engine translates query into syntax of multiple resource wom#n and sport* women and sports (woman or women) and sport! women and sports

How do they do it? Federated search engines translate single search query into syntax of multiple databases: Z39.50 XML Gateways HTTP Protocol “Others”

How It Plays Out Individual resources execute search based on the query supplied by the federated search engine wom#n and sport* (woman or women) and sport! women and sports

What does the user see? A friendly message indicating a search is in process

Federated Search Engines Search multiple databases: E-Journals Abstracting and indexing databases E-Books Web Online catalog(s) Any other searchable online source

How It Plays Out Federated search engine aggregates results Results +

What does the user see? Combined results of the search

How It Plays Out Searcher receives a single list of all the results from all the resources searched by the federated search engine Results

What does the user see? A single, combined list of results:

Value added in results Federated search engines deliver results from multiple databases in a single list: Standardized format De-duped Connect to fulltext using link resolvers Relevancy ranked

What are they selling? The technology behind federated searching is straight-forward enough to prompt one industry insider to describe it as a “commodity.”

The Major Players The major players attempt to distinguish themselves by adding value to the basic technology: Maintaining links Updating translators to remain compatible with search interfaces Results delivery: de-duping, ranking, sorting, fulltext linking, etc.

Ex Libris MetaLib Customer List

WebFeat’s Prism m Customer List

Fretwell-Downing Zportal Customer List

Endeavor ENCompass Customer List

More Major Players Sirsi Rooms TDNet TES MuseGlobal

What’s not to love? One-stop searching No danger of missing a possible source of information Users do not have to figure out where to start…just search them all Those expensive databases won’t be missed by searchers who could use them

What’s not to love? The whole process of research—even for scholarly, technical, and professional information—has been Googlized!

There’s a catch, right? While the search may be quick and broad, it is neither precise nor deep

Not for Power Searching The searching syntax among databases vary: Truncation, Boolean searching, phrase searching, and proximity searching may be lost Use of limits is limited Searchable fields may be eliminated— controlled vocabularies lose their punch Even keyword searching tough. MS = Microsoft or multiple sclerosis

Clusty Advanced Search

Dogpile Advanced Search

DogpileDogpile Advanced Search vs. Google Advanced SearchGoogle

May not be all they claim True de-duping is virtually impossible Too many variables for reliable relevancy ranking Sorting—a single basket for apples and oranges?

Still Has Much to Improve Access and verification—especially “off-site” users Not all federated search engines can search all sources—not everyone is using the Z39.50 or XML protocol Expensive and labor intensive

Sample searching BPL’s Big Dig Duke’s Metasearch BC’s MetaQuest

BPL’s Big Dig Powered by WebFeat

BC’s Metaquest Powered by ExLibris MetaLib v 2

Duke’s Metasearch Powered by ExLibris MetaLib v 3

And, What About…? Library OPACS/ILS Integration Google Scholar Amazon A9