A.Frank 1 Internet Resources Discovery (IRD) Whither Search Engine (SE)?! Some Practical Recommendations

2 A.Frank Contents When not to use SEs? Use patterns of SEs Rules for choosing SEs Practical recommendations General/Specialty subject SEs

3 A.Frank Googlism

4 A.Frank When not to use SEs? You know it all. You prefer asking friends (or paid experts ). You know the Web site for it (and didn’t forget the exact URL or have auto-completion or bookmark or can access through another known site). You already found a specific/relevant digital library or database (maybe in Invisible Web). Tired of paid inclusions, SE spamming, and sponsored commercial results. Tired of chasing down useless URLs.

5 A.Frank Use patterns of SEs (2004) iProspect SE User Attitudes Survey Results, March 2004 ( –  57% of Web users use the same SE when they are looking for information.  Most searchers (92%, up from 71% in June 2002) are loyal to their favorite SE, and stick with it (by modifying their query) even if they don't initially find what they're looking for (in the first 3 pages).  Just 30% of Web users have a few specific SEs they use regularly.  Only 13% use a different SE depending on what they are looking for at that time.  Google has a "loyalty rate" of 66%, Yahoo! is next at 55%, followed by MSN at 54% and AOL at 49%.

6 A.Frank Web Index D irectory WEB Which kind to use? All Which kind to use? All S earch E ngine General SpecialtyGeneralSpecialty Meta-S earch E ngine

7 A.Frank When to use an Index? Need to search for a narrow piece of information. Have a specific objective/site in mind. Want to find/rank many related Web sites. Want to factor quantity in (index has crawler based results). Need to check/fix spelling (based on Web statistics).

8 A.Frank When to use a Directory? Clear about the exact topic of your query. Need general information on a rather broad topic/category. Want to amass knowledge on a fairly wide subject. Would like to browse (and then search) a certain area. Want to factor quality in (directory has human- powered results), not quantity. Need information that is usually carefully evaluated and even annotated.

9 A.Frank When to use a Meta-SE? When single Basic-SE fails to provide good results. One-stop shopping - prefer to search multiple SEs/sites at once to get blended ranked results (so as to save effort/time). When the query is simple (complex fields/options don't usually work). Searching for multi-faceted topics. Want to get clustered results to focus search on the relevant keywords. Looking for current events/news.

10 A.Frank When to use a Specialty-SE? When general-SE fails to provide good results. When your target is very topic/technology specific. Want to find more than just Web pages/sites. Need more results from the Invisible Web. Want your search terms to more likely have the meanings you intended them to have.

11 A.Frank So how to choose your SE(s)?

12 A.Frank General rules for choosing SEs Use "major" SEs that are both well-known and well-used (and that hopefully won’t be downgraded or disappear soon ). Prefer SEs that employ both a huge index and a comprehensive directory (gives better results; can also switch between). Stick to SEs of established companies that treat search as their main business/expertise.

13 A.Frank Technical rules for choosing SEs Insist on a thin (user-friendly) interface but also support an advanced search page. Fully index a broad range of file types. Provide multimedia (images, audio/MP3, video) search tags (also news, products). Suggest spelling correction based on Web statistics. Have a featured toolbar (easier to invoke from the desktop). Enable “Cursor Search” of word(s) on a Web page by right-clicking the mouse.

14 A.Frank Theory vs. Practice?!

15 A.Frank Popularity of Google vs. Yahoo! Search for “google” in – Google: 48,700,000 results. – Yahoo!: 57,500,000 results. Search for “yahoo!” in – Google: 152,000,000 results. – Yahoo!: 110,000,000 results.  Maybe “Know thy enemy”?!

16 A.Frank Common sense rules for choosing SEs "לעולם אל תשים את כל הביצים בסל אחד": בחר ביותר מאחד! "טובים השניים מהאחד": בחר בשניים עיקריים! "... עד שיבוא (הכתוב) השלישי ויכריע ביניהם": כדאי אבל שיהיה גם שלישי (מסוג שונה) כגיבוי! "בתי הקברות מלאים בכאלו שחשבו שאין להם תחליף": אז כדאי שיהיו גם תחליפים לכל אחד ואחד!

17 A.Frank Top 5 Search Engines/Sites (Jan 2004)

18 A.Frank Share of Searches Sites (Dec 2004)

19 A.Frank Share of Search Providers (Dec 2004)

20 A.Frank Practical recommendations Two major SEs (usually use both): 1.Google (GG) –Alternative: Personalized Google (labs.google.com/personalized)labs.google.com/personalized 2.Yahoo! search (YH) –Alternative : Teoma/AskJeeves (AJ) One Meta-SE (as a backup): 3.DogPile –Alternative: Vivisimo\Clusty Note: Choices are not Hebrew oriented.

21 A.Frank המלצות פרקטיות (לעברית) שני מנועי חיפוש עיקריים (לרוב שימוש בשניהם): 1.גוגל ישראל –חלופה: מורפיקס (לצורך המורפולוגיה) 2.וואלה! חיפוש –חלופה: נענע/תפוז (אבל כולם פורטלים בעיקר) מנוע חיפוש-על אחד (כגיבוי): 3.סטארט –חלופה:2Find (מנוע חיפוש משלנו ) הערה: התמיכה שלהם בעברית עדיין לא מושלמת.

22 A.Frank So what was the message ?

23 A.Frank So how does it look?

24 A.Frank Why use Google? (1) Biggest, most comprehensive coverage:  ~8 billion Web pages (but ~1 billion of it isn’t full-text searchable!)  ~10 billion documents, if you count images and newsgroup postings. Fastest around. Most relevant results (voted 3 times most outstanding SE by Search Engine Watch readers). Provides good directory results (PageRanks results of DMOZ Open Directory).

25 A.Frank Why use Google? (2) Has thinnest interface around. But provides rich set of advanced search features/tools(/hacks). Finds similar/related pages. Supports Web pages translation. Cached (HTML) copy of pages (great for quick view of DOCs/PDFs and for 404s ). Google alert – use of push technology.

26 A.Frank Why use Yahoo! search? (1) Has brand new Yahoo! search – gives highly relevant Web results (at Google level ). Still supports an expert’s humanly-compiled directory (dir.yahoo.com).dir.yahoo.com Has (also) a thin interface ( search.yahoo.com ) while providing a rich set of advanced search features/shortcuts. search.yahoo.com

27 A.Frank Why use Yahoo! search? (2) For legacy reasons (oldest of all directories). Puts particular emphasis on personalization and customization ( my.yahoo.com). my.yahoo.com Had enough of Googlism ( ). It devoured/uses (know-how from) Overture (Inktomi, AltaVista and AllTheWeb, etc)

28 A.Frank Yahoo acquisitions!? Yahoo google

29 A.Frank 4 Things Yahoo can and Google can’t! Find websites linking to a page –linkdomain: Mix syntax –Link:amdocs.com site:gov Long queries (>32 terms) –Especially important when using OR Search for XML/RSS

30 A.Frank New alternative - Why use MSN? Fresh from the Oven -- Launched Nov 11, Vast index of information: ~5 billion documents. The most up-to-date information – MSNbot is active all the time. Direct answers -- from Microsoft Encarta®, encyclopedia. Direct actions -- to MSN channels.

31 A.Frank Why use Teoma? It’s an expert/guide (term in Gallic ). Provides “subject specific” ranking of pages. One search – three responses: 1.Results: Lists relevant Web pages 2.Refine: Suggestions to narrow your search. 3.Resources: Recommends link collections from experts and enthusiasts.

32 A.Frank Why use AskJeeves (that uses Teoma)? Provides a Natural Language interface (uses NLP). Suggests related searches. Hides technical details (of Teoma). Just purchased Interactive Search Holdings (MyWay, MySearch, My Web Search, iWon, and Excite).

33 A.Frank Why use DogPile? Customizable list of indices, directories and SSEs. Indices searched include Google, Yahoo, and AskJeeves/Teoma. Directories searched include About, LookSmart and Open Directory. View results by their relevance or by search engine. Designed to identify the (non-)commercial intent of a user's search - proposes to refine your results. Winner of “Best Meta Search Engine” award from Search Engine Watch for 2003.

34 A.Frank Why use Vivisimo\Clusty? Indices searched include MSN, GigaBlast, Lycos, and WiseNut. Directories searched include LookSmart and Open Directory. Provides automatic clustering in hierarchical folders. The results are grouped into successively narrower subcategories, allowing to drill down through a topic without additional searching. Won second place for “Best Meta Search Engine” in the 2003 Search Engine Watch awards and winner in 2002.

35 A.Frank Choice of SEs is a delicate balance

36 A.Frank General Subject SEs  lii.org lii.org Librarians’ carefully selected index to the internet  Evaluated and annotated subject directory  infomine.ucr.edu infomine.ucr.edu Selected scholarly internet resource collections  Academic collection of "sites" on many subjects  Search Engine for Specialty Search Engines

37 A.Frank Specialty Subject SEs  Directory for Meta Specialty Searches  A collection of special SEs  Educational subject directory  Multi-subject guides to specialized SEs  lib.nmsu.edu/instruction/specialtysearch.htmlib.nmsu.edu/instruction/specialtysearch.htm Lists of Specialty Search Engines

38 A.Frank Ready Reference Desks  Comprehensive reference desk  Ready reference collection  lii.org/search/file/reference lii.org/search/file/reference Ready reference and quick facts  academicinfo.net/reffind.html academicinfo.net/reffind.html Educational reference desk  Help people find what they are looking for

39 A.Frank If more time... we could SEEk more

40 A.Frank Bibliography/Credits   searchengineshowdown.com/ searchengineshowdown.com/  ocate/adviceengine.html ocate/adviceengine.html  infopeople.org/search infopeople.org/search     (Hebrew)

41 A.Frank Bottom line  Seek/Search the best way for you!