1. Federated Searching In a Nutshell 21 st Century Literacies 18 November 2005

2. Federated Searching Rex Krajewski Reference Services Librarian Simmons College web.simmons.edu/~krajewsk/library/federatedsearching.html

3. What is Federated Searching? Process of searching multiple sources simultaneously

4. More Specifically Conducted using federated search engines

5. Federated Searching AKA parallel search meta search broadcast search one-search cross searching cross-database searching distributed searching single search

6. Well-known Models Dialog allows user to search many databases simultaneously (think: Dialindex One Search Categories) Dialindex One Search Categories Metasearch Engines—like Dogpile, Clusty, Mamma, and Metacrawler— allow users to search multiple search engines’ top results with a single searchDogpile ClustyMammaMetacrawler

7. Federated Searching The term “federated searching” came from the Open Archives Initiative’s Protocol for Metadata Harvesting (OIA-PMH) Single server harvests metadata for records from the holdings databases of “federated” databases. The resulting centralized data is searchable.

8. What’s in a name? NISO says “metasearching” Vendors prefer “federated search engines” because metasearching use by meta-search engines like Dogpile, Mamma, and Ask Jeeves.

9. Federated vs. Meta Meta search engines aggregate material that has already been searched and is freely available to anyone on the Web. Federated search engines run searches at the time they are queried and can include proprietary material and other Internet content not spidered by search engines

10. Why Federated Searching? Libraries offer hundreds of databases to search Allows libraries and users alike to manage the hundreds of database search tools.

11. Digital Reference: Why? 97% of surveyed adult internet users expect to find the information they need on government, health, commerce, and news on the internet.* *Counting on the Internet: Most expect to find key information online, most find the information they seek,many now turn to the Internet first, Pew Internet& American Life Project, 29 September 2002, Date accessed: 15 November 2005.

12. Digital Reference: Why? “Nearly three-quarters (73%) of college students say they use the Internet more than the library, while only 9% said they use the library more than the Internet for information searching.”* *The Internet Goes to College: How Students are Living in the Future with Today's Technology, Pew Internet& American Life Project, 15 September 2002, Date accessed: 03 November 2005.

13. Digital Reference: Why? “71% of students report using the Internet at their primary source for their last major project, and they also report accessing online study aids like Sparknotes or CliffNotes.”* * The Internet and Education, Pew Internet& American Life Project, 01 September, 2001, Date accessed: 21 October, 2005

14. Why Federated Searching In the age of Google, users expect the world of knowledge available quickly and easily at their fingertips…they expect the same kind of one-stop searching to be available in the library

15. How It Plays Out Searcher enters a single search into the federated search engine women and sports

16. What does the user see? A single research entry point A familiar interface A consistent search syntax

17. How It Plays Out Federated search engine translates query into syntax of multiple resource wom#n and sport* women and sports (woman or women) and sport! women and sports

18. How do they do it? Federated search engines translate single search query into syntax of multiple databases: Z39.50 XML Gateways HTTP Protocol “Others”

19. How It Plays Out Individual resources execute search based on the query supplied by the federated search engine wom#n and sport* (woman or women) and sport! women and sports

20. What does the user see? A friendly message indicating a search is in process

21. Federated Search Engines Search multiple databases: E-Journals Abstracting and indexing databases E-Books Web Online catalog(s) Any other searchable online source

22. How It Plays Out Federated search engine aggregates results Results +

23. What does the user see? Combined results of the search

24. How It Plays Out Searcher receives a single list of all the results from all the resources searched by the federated search engine Results

25. What does the user see? A single, combined list of results:

26. Value added in results Federated search engines deliver results from multiple databases in a single list: Standardized format De-duped Connect to fulltext using link resolvers Relevancy ranked

27. What are they selling? The technology behind federated searching is straight-forward enough to prompt one industry insider to describe it as a “commodity.”

28. The Major Players The major players attempt to distinguish themselves by adding value to the basic technology: Maintaining links Updating translators to remain compatible with search interfaces Results delivery: de-duping, ranking, sorting, fulltext linking, etc.

29. Ex Libris MetaLib http://www.exlibris-usa.com/metalib.htm Customer List

30. WebFeat’s Prism http://www.webfeat.org/products/prism.ht m Customer List

31. Fretwell-Downing Zportal http://www.fdusa.com/products/zportal.html Customer List

32. Endeavor ENCompass http://encompass.endinfosys.com/ Customer List

33. More Major Players Sirsi Rooms TDNet TES MuseGlobal

34. What’s not to love? One-stop searching No danger of missing a possible source of information Users do not have to figure out where to start…just search them all Those expensive databases won’t be missed by searchers who could use them

35. What’s not to love? The whole process of research—even for scholarly, technical, and professional information—has been Googlized!

36. There’s a catch, right? While the search may be quick and broad, it is neither precise nor deep

37. Not for Power Searching The searching syntax among databases vary: Truncation, Boolean searching, phrase searching, and proximity searching may be lost Use of limits is limited Searchable fields may be eliminated— controlled vocabularies lose their punch Even keyword searching tough. MS = Microsoft or multiple sclerosis

38. Clusty Advanced Search

39. Dogpile Advanced Search

40. DogpileDogpile Advanced Search vs. Google Advanced SearchGoogle

41. May not be all they claim True de-duping is virtually impossible Too many variables for reliable relevancy ranking Sorting—a single basket for apples and oranges?

42. Still Has Much to Improve Access and verification—especially “off-site” users Not all federated search engines can search all sources—not everyone is using the Z39.50 or XML protocol Expensive and labor intensive

43. Sample searching BPL’s Big Dig Duke’s Metasearch BC’s MetaQuest

44. BPL’s Big Dig Powered by WebFeat http://www.bpl.org/electronic/index.htm

45. BC’s Metaquest Powered by ExLibris MetaLib v 2 http://metaquest.bc.edu

46. Duke’s Metasearch Powered by ExLibris MetaLib v 3 http://metasearch.library.duke.edu

47. And, What About…? Library OPACS/ILS Integration Google Scholar Amazon A9