© Tefko Saracevic, Rutgers University1 The Invisible Web - finding things that are hard to find - Tefko Saracevic, PhD Rutgers University ( contains also a list of sites relevant to the topic and this presentation) Tefko Saracevic, PhD Rutgers University ( contains also a list of sites relevant to the topic and this presentation)

© Tefko Saracevic, Rutgers University2 What is “Invisible Web?” Materials that general search engines cannot or WILL not include in their collection of web pages (indexes) You cannot find through general search engines Contains a vast amount of information –much of it authoritative, qualitative –much of it specialized Materials that general search engines cannot or WILL not include in their collection of web pages (indexes) You cannot find through general search engines Contains a vast amount of information –much of it authoritative, qualitative –much of it specialized

© Tefko Saracevic, Rutgers University3 Why search engines miss? Size: Web is huge, cannot cover all Economics: associated costs are high –also pay per crawl & rank Technical: still limited capabilities Spam: eliminating bad also looses good Restrictions: some site do not let in Deep structure: some sites complex Size: Web is huge, cannot cover all Economics: associated costs are high –also pay per crawl & rank Technical: still limited capabilities Spam: eliminating bad also looses good Restrictions: some site do not let in Deep structure: some sites complex

© Tefko Saracevic, Rutgers University4 Web size - who knows? Web Characterization Project - OCLCWeb Characterization Project –provides statistics about the web – 1998: 2.8, 2002: 9.04 mill web sites (IP address) In 2002: 35% public, 29% private, 36% provisional sites –Public sites (2002): 55% US, 7% German, 6% Japanese, 3% each French, Spanish, 2% each Italian, Dutch, Chinese,1% each Korean, Russian, Polish, Portuguese –Adult sites (2002): 3.3% –IP address volatility - all sites (disappearance pattern): 13% of sites in 2002 were also in 1998; 51% in 2001 Web Characterization Project - OCLCWeb Characterization Project –provides statistics about the web – 1998: 2.8, 2002: 9.04 mill web sites (IP address) In 2002: 35% public, 29% private, 36% provisional sites –Public sites (2002): 55% US, 7% German, 6% Japanese, 3% each French, Spanish, 2% each Italian, Dutch, Chinese,1% each Korean, Russian, Polish, Portuguese –Adult sites (2002): 3.3% –IP address volatility - all sites (disappearance pattern): 13% of sites in 2002 were also in 1998; 51% in 2001

© Tefko Saracevic, Rutgers University5 How search engines work? Crawlers, spiders: go out to find –new & changed sites; periodic, not for each query Databases, caches: –gather content; could be submitted, bought Indexing: creating appropriate entries –various, mostly proprietary algorithms Retrieval engine: searching on basis of query Interface: gathers query, displays results – could be ordered by pay Crawlers, spiders: go out to find –new & changed sites; periodic, not for each query Databases, caches: –gather content; could be submitted, bought Indexing: creating appropriate entries –various, mostly proprietary algorithms Retrieval engine: searching on basis of query Interface: gathers query, displays results – could be ordered by pay

© Tefko Saracevic, Rutgers University6 Search engines differ Substantial differences among search engines on each aspect Information about search engines:  Search Engine Watch Search Engine Watch  ratings, news, statistics, charts  Search Engine Showdown Search Engine Showdown  run by a librarian, news links, ratings  Extreme Searcher Extreme Searcher  update of a popular book Substantial differences among search engines on each aspect Information about search engines:  Search Engine Watch Search Engine Watch  ratings, news, statistics, charts  Search Engine Showdown Search Engine Showdown  run by a librarian, news links, ratings  Extreme Searcher Extreme Searcher  update of a popular book

© Tefko Saracevic, Rutgers University7 Search engine coverage No engine covers more than 16% of WWW Hard to discern & compare coverage Many national search engines - own coverage Many topical search engines – own coverage Many comprehensive sources independent of search engines No engine covers more than 16% of WWW Hard to discern & compare coverage Many national search engines - own coverage Many topical search engines – own coverage Many comprehensive sources independent of search engines

© Tefko Saracevic, Rutgers University8 Specialized sources Meta search engines Specialized engines & catalogs Domain (subject) engines & catalogs Reference sources Libraries as web sources Virtual libraries Subject databases Societies, organizations Meta search engines Specialized engines & catalogs Domain (subject) engines & catalogs Reference sources Libraries as web sources Virtual libraries Subject databases Societies, organizations

© Tefko Saracevic, Rutgers University9 Meta search engines Search engines that cover search engines  Search Engine Colossus Search Engine Colossus  international meta engine  Dogpile Dogpile  results from a number of search engines  Surfwax -gives statistics and text sources Surfwax  Search Engine Guide Search Engine Guide  categorized by topic; other engine information Search engines that cover search engines  Search Engine Colossus Search Engine Colossus  international meta engine  Dogpile Dogpile  results from a number of search engines  Surfwax -gives statistics and text sources Surfwax  Search Engine Guide Search Engine Guide  categorized by topic; other engine information

© Tefko Saracevic, Rutgers University10 meta engines … (cont.)  Vivisimo Vivisimo  clusters results; innovative  Complete Planet Complete Planet  over 100,000 databases & s engines  Webbrain Webbrain  results in tree structure – fun to use  Vivisimo Vivisimo  clusters results; innovative  Complete Planet Complete Planet  over 100,000 databases & s engines  Webbrain Webbrain  results in tree structure – fun to use

© Tefko Saracevic, Rutgers University11 Domain engines & catalogs Cover general & specific areas  Open Directory Project – large edited catalog of the web – global, run by volunteers Open Directory Project  BUBL LINK -selected Internet resources covering all academic subject areas – UK BUBL LINK  Profusion – search in categories Profusion Cover general & specific areas  Open Directory Project – large edited catalog of the web – global, run by volunteers Open Directory Project  BUBL LINK -selected Internet resources covering all academic subject areas – UK BUBL LINK  Profusion – search in categories Profusion

© Tefko Saracevic, Rutgers University12 domain engines … Exist in many domains & subjects – rich!  Psychcrawler Amer Psychological Association Psychcrawler  web index for psychology  Entrez PubMed – Nat Library of Medicine Entrez PubMed  CiteSeer - NEC Research Center CiteSeer  scientific literature, citations index - free  Think Quest – an international organization Think Quest  education resources, programs Exist in many domains & subjects – rich!  Psychcrawler Amer Psychological Association Psychcrawler  web index for psychology  Entrez PubMed – Nat Library of Medicine Entrez PubMed  CiteSeer - NEC Research Center CiteSeer  scientific literature, citations index - free  Think Quest – an international organization Think Quest  education resources, programs

© Tefko Saracevic, Rutgers University13 domain engines …  KIRKE - Katalog der Internetressourcen für die Klassische Philologie aus Erlangen KIRKE  a variety of resources  Perseus Digital Library Tufts University Perseus Digital Library  covers antiquity to renaissance  Sch of Slavonic & East European Studies, University College London Sch of Slavonic & East European Studies  includes country resources, e.g. Croatia  U Mich Document Center U Mich Document Center  official documents from all over the world  KIRKE - Katalog der Internetressourcen für die Klassische Philologie aus Erlangen KIRKE  a variety of resources  Perseus Digital Library Tufts University Perseus Digital Library  covers antiquity to renaissance  Sch of Slavonic & East European Studies, University College London Sch of Slavonic & East European Studies  includes country resources, e.g. Croatia  U Mich Document Center U Mich Document Center  official documents from all over the world

© Tefko Saracevic, Rutgers University14 Reference services Reference services - several models –Q&A, directories, answers etc.  Ask Jeeves! Ask Jeeves!  most popular, commercial  Information Please Information Please  almanac type questions Reference services - several models –Q&A, directories, answers etc.  Ask Jeeves! Ask Jeeves!  most popular, commercial  Information Please Information Please  almanac type questions

© Tefko Saracevic, Rutgers University15 reference … Digital reference - new service area for libraries  QuestionPoint L of Congress & OCLC QuestionPoint  project for a global reference network  Virtual Reference Desk – L of Congress Virtual Reference Desk  compilation of web reference sites  LiveRef - maintained at Iowa State U LiveRef  a registry of real time digital reference services Digital reference - new service area for libraries  QuestionPoint L of Congress & OCLC QuestionPoint  project for a global reference network  Virtual Reference Desk – L of Congress Virtual Reference Desk  compilation of web reference sites  LiveRef - maintained at Iowa State U LiveRef  a registry of real time digital reference services

© Tefko Saracevic, Rutgers University16 Libraries as web sources Academic libraries providing open collections & services; models vary  Rutgers libraries - big long term effort Rutgers libraries  University of California, Berkeley University of California, Berkeley  a most elaborate effort together with Sun Corporation  Bibliothèque Nationale de France Bibliothèque Nationale de France  includes virtual exhibitions, among others Academic libraries providing open collections & services; models vary  Rutgers libraries - big long term effort Rutgers libraries  University of California, Berkeley University of California, Berkeley  a most elaborate effort together with Sun Corporation  Bibliothèque Nationale de France Bibliothèque Nationale de France  includes virtual exhibitions, among others

© Tefko Saracevic, Rutgers University17 Virtual libraries on the Web Libraries emerging only on the Web  Virtual Library –Virtual Library  Switzerland, US, UK & other countries – ‘oldest virtual library on the Web’  Internet Public Library Michigan Internet Public Library  also a long term effort  Librarians Index of the Internet Librarians Index of the Internet  very popular and comprehensive Libraries emerging only on the Web  Virtual Library –Virtual Library  Switzerland, US, UK & other countries – ‘oldest virtual library on the Web’  Internet Public Library Michigan Internet Public Library  also a long term effort  Librarians Index of the Internet Librarians Index of the Internet  very popular and comprehensive

© Tefko Saracevic, Rutgers University18 virtual libraries …  Academic Info Digital Library Academic Info Digital Library  many links to digital collections & resources in various subjects  Gabriel Gabriel  Gateway to European National Libraries  Museum of online museums Museum of online museums  a delight  Academic Info Digital Library Academic Info Digital Library  many links to digital collections & resources in various subjects  Gabriel Gabriel  Gateway to European National Libraries  Museum of online museums Museum of online museums  a delight

© Tefko Saracevic, Rutgers University19 Subjects databases Many subject specific sites –rich & often unique coverage & services – different approaches & requirements Examples in health related domains:  WebMDHealth – news, medical information WebMDHealth  Rxlist - The Internet Drug Index Rxlist  Mayo Clinic HealthOasis – health advice Mayo Clinic HealthOasis Many subject specific sites –rich & often unique coverage & services – different approaches & requirements Examples in health related domains:  WebMDHealth – news, medical information WebMDHealth  Rxlist - The Internet Drug Index Rxlist  Mayo Clinic HealthOasis – health advice Mayo Clinic HealthOasis

© Tefko Saracevic, Rutgers University20 Societies, organizations Great many rich sources for searching –differences in requirements, depth, richness Examples from variety of organizations:  Assoc. for Computing Machinery Assoc. for Computing Machinery  Digital Library; subscription or registration  US State Department US State Department  about the U.S & other countries  Genealogy – Church of Later Day Saints Genealogy  most comprehensive historical list of records Great many rich sources for searching –differences in requirements, depth, richness Examples from variety of organizations:  Assoc. for Computing Machinery Assoc. for Computing Machinery  Digital Library; subscription or registration  US State Department US State Department  about the U.S & other countries  Genealogy – Church of Later Day Saints Genealogy  most comprehensive historical list of records

© Tefko Saracevic, Rutgers University21 Language barriers on the Web English still the major language – but declining, now slightly over 50% Multilingual retrieval search engines  Euroseek Euroseek  searches in a number of languages  All the Web All the Web  results in 45 languages English still the major language – but declining, now slightly over 50% Multilingual retrieval search engines  Euroseek Euroseek  searches in a number of languages  All the Web All the Web  results in 45 languages

© Tefko Saracevic, Rutgers University22 Language barriers: translations A number of translation sites –machine aided – i.e. plug in terms, phrases, sentences in one & review in the other language, but effectiveness???  Free Translations Free Translations  from to English, & 8 other languages  Babel Fish Babel Fish  from to English and 9 languages, translates URLs  Travlang Travlang  great for travelers, but annoying commercials A number of translation sites –machine aided – i.e. plug in terms, phrases, sentences in one & review in the other language, but effectiveness???  Free Translations Free Translations  from to English, & 8 other languages  Babel Fish Babel Fish  from to English and 9 languages, translates URLs  Travlang Travlang  great for travelers, but annoying commercials

© Tefko Saracevic, Rutgers University23 Web news; keeping up What is going on on the Web? Some major sources of news and evaluations:  Free Pint – newsletter, articles, links Free Pint  Internet Resources Newsletter – UK based Internet Resources Newsletter  ResearchBuzz – daily updates; many aspects ResearchBuzz  About.com Web Search – tools, Web Search Forum About.com Web Search  Resource Shelf – newsletter with archive Resource Shelf What is going on on the Web? Some major sources of news and evaluations:  Free Pint – newsletter, articles, links Free Pint  Internet Resources Newsletter – UK based Internet Resources Newsletter  ResearchBuzz – daily updates; many aspects ResearchBuzz  About.com Web Search – tools, Web Search Forum About.com Web Search  Resource Shelf – newsletter with archive Resource Shelf

© Tefko Saracevic, Rutgers University24 keeping up … Information Today –trade & professional monthly newspaper & web site –industry news –searcher columns –general analyses of trends Information Today –trade & professional monthly newspaper & web site –industry news –searcher columns –general analyses of trends

© Tefko Saracevic, Rutgers University25 Evaluations, ratings Many sources evaluate web sites:  The Scout Report – The Scout Report  librarians’ BIBLE! Annotations. Comprehensive.  Medical Library Assoc. – ten most useful sites; Medical Library Assoc.  MLA user guide for health inf., recommendations MLA user guide  Web 100 – commercial, user ratings, news Web 100  Evaluating web pages UC Berkeley Evaluating web pages –tutorial and guide Many sources evaluate web sites:  The Scout Report – The Scout Report  librarians’ BIBLE! Annotations. Comprehensive.  Medical Library Assoc. – ten most useful sites; Medical Library Assoc.  MLA user guide for health inf., recommendations MLA user guide  Web 100 – commercial, user ratings, news Web 100  Evaluating web pages UC Berkeley Evaluating web pages –tutorial and guide

© Tefko Saracevic, Rutgers University26 Archiving the web Internet Archive – a large undertakingInternet Archive –includes web archive & lots more publicly available & free –10 billion web pages archived from 1996 to a few months ago –Wayback Machine – search to look at old versions of web pages But there is more. e.g.: –Million Book ProjectMillion Book Project –International Children’s Digital LibraryInternational Children’s Digital Library Internet Archive – a large undertakingInternet Archive –includes web archive & lots more publicly available & free –10 billion web pages archived from 1996 to a few months ago –Wayback Machine – search to look at old versions of web pages But there is more. e.g.: –Million Book ProjectMillion Book Project –International Children’s Digital LibraryInternational Children’s Digital Library

© Tefko Saracevic, Rutgers University27 Needed for Web searching Knowledge & competencies on –variety of web sources & their organization –search engines –web search strategies –search dynamics, feedback Keeping up & up & up –constant updates, changes, innovations –many domain/subject specific Knowledge & competencies on –variety of web sources & their organization –search engines –web search strategies –search dynamics, feedback Keeping up & up & up –constant updates, changes, innovations –many domain/subject specific

© Tefko Saracevic, Rutgers University28 Needed for Web searching by professionals Knowledge of SOURCES in area of interest search engines not enough not too helpful in finding these other sources; structure hard to discern Evaluation of sources –a key professional skill! standard criteria & Web criteria: authority; accuracy; currency (timeliness); objectivity; coverage, persistence, usability Knowledge of SOURCES in area of interest search engines not enough not too helpful in finding these other sources; structure hard to discern Evaluation of sources –a key professional skill! standard criteria & Web criteria: authority; accuracy; currency (timeliness); objectivity; coverage, persistence, usability

© Tefko Saracevic, Rutgers University29 Needed competencies … Knowledge of users & use Knowledge of searching Use of technology Adaptability, flexibility Integration with other resources Teaching others Constant learning & update –keeping up, keeping up, keeping up Knowledge of users & use Knowledge of searching Use of technology Adaptability, flexibility Integration with other resources Teaching others Constant learning & update –keeping up, keeping up, keeping up

© Tefko Saracevic, Rutgers University30 information WWW But now really: How to do it?

© Tefko Saracevic, Rutgers University31

© Tefko Saracevic, Rutgers University32

© Tefko Saracevic, Rutgers University33 P.S. a few weird sites… SelectSmart.com – all kinds of quizzes for you James Dean official web site Deaducated –Dead Librarians’ Society Livejournal –blogs & authoring tools SelectSmart.com – all kinds of quizzes for you James Dean official web site Deaducated –Dead Librarians’ Society Livejournal –blogs & authoring tools

© Tefko Saracevic, Rutgers University34 Sources About.com Web Search Academic Info Digital Library All the Web Ask Jeeves! Assoc. for Computing Machinery Babelfish Bibliothèque Nationale de France BUBL LINK CDNET Search.com CiteSeer CompletePlanet Deaducated Dogpile Entrez PubMed Extreme Searcher Free Pint About.com Web Search Academic Info Digital Library All the Web Ask Jeeves! Assoc. for Computing Machinery Babelfish Bibliothèque Nationale de France BUBL LINK CDNET Search.com CiteSeer CompletePlanet Deaducated Dogpile Entrez PubMed Extreme Searcher Free Pint

© Tefko Saracevic, Rutgers University35 sources … Free Translations Gabriel Genealogy Information Please International Children’s Digital Library Internet Archive Internet Public Library, Michigan Internet Resources Newsletter. James Dean KIRKE Librarians Index to the Internet Live Journal LiveRef Mayo Clinic Free Translations Gabriel Genealogy Information Please International Children’s Digital Library Internet Archive Internet Public Library, Michigan Internet Resources Newsletter. James Dean KIRKE Librarians Index to the Internet Live Journal LiveRef Mayo Clinic

© Tefko Saracevic, Rutgers University36 sources … Medical Library Assoc. ten top sites Medical Library Assoc. user guide for health inf. Medscape Million Book Project Museum of online museums. OCLC Web Characterization Project Open Directory Project Perseus Digital Library Profusion Psychcrawler QuestionPoint ResearchBuzz. Resource Shelf Rutgers Libraries RxList Medical Library Assoc. ten top sites Medical Library Assoc. user guide for health inf. Medscape Million Book Project Museum of online museums. OCLC Web Characterization Project Open Directory Project Perseus Digital Library Profusion Psychcrawler QuestionPoint ResearchBuzz. Resource Shelf Rutgers Libraries RxList

© Tefko Saracevic, Rutgers University37 sources … Sch of East Eur & Slavonic Studies Search Engine Colossus Search Engine Guide Search Engine Showdown Search Engine Watch Select Smart.com Surfwax The Scout Report. Think Quest Travlang U California Berkeley U Mich Documents Center US State department Virtual Library Virtual Reference Desk Vivisimo Web Webbrain WebMD Sch of East Eur & Slavonic Studies Search Engine Colossus Search Engine Guide Search Engine Showdown Search Engine Watch Select Smart.com Surfwax The Scout Report. Think Quest Travlang U California Berkeley U Mich Documents Center US State department Virtual Library Virtual Reference Desk Vivisimo Web Webbrain WebMD