1. Mastering the Internet, XHTML, and JavaScript Chapter 7 Searching the Internet

2. 2Chapter 7 - Searching the InternetOutline Goals and Objectives Goals and Objectives Chapter Headlines Chapter Headlines Introduction Introduction Directories Directories Open Directory Project Open Directory Project Search Engines Search Engines Metasearch Engines Metasearch Engines Search techniques Search techniques Intelligent Agents Intelligent Agents Invisible Web Invisible Web Summary Summary

3. 3Chapter 7 - Searching the Internet Goals and Objectives Goals Goals Understand and master searching the internet to find relevant information fast, know what information to searach for, and how to search for it Objectives Objectives Subject Directories Subject Directories Open Directory project Open Directory project Search and metasearch engines Search and metasearch engines Search techniques Search techniques Intelligent agents Intelligent agents The visible web The visible web The invisible web The invisible web Search techniques for the invisible web Search techniques for the invisible web

4. 4Chapter 7 - Searching the Internet Chapter Headlines 7.1 Introduction 7.1 Introduction Use directory or search engines to search internet Use directory or search engines to search internet 7.2 Directories 7.2 Directories Search a subject tree manually or use its search engine Search a subject tree manually or use its search engine 7.3 Open Directory Project 7.3 Open Directory Project User rank search results according to their expertise User rank search results according to their expertise 7.4 Search Engines 7.4 Search Engines How do search engines do their amazing job How do search engines do their amazing job 7.5 Metasearch Engines 7.5 Metasearch Engines Use multiple search engines at once Use multiple search engines at once 7.6 Search Techniques 7.6 Search Techniques Search internet more effectively Search internet more effectively 7.7 Intelligent Agents 7.7 Intelligent Agents Search internet more effectively Search internet more effectively 7.8 Invisible Web 7.8 Invisible Web Search internet more effectively Search internet more effectively

5. 5Chapter 7 - Searching the InternetIntroduction Search Engines are a primary searching tool to find the information that we want from the internet Search Engines are a primary searching tool to find the information that we want from the internet Internet search can be time-consuming Internet search can be time-consuming To formulate a strategic search the user must know what to search for and how to search for it To formulate a strategic search the user must know what to search for and how to search for it Search results are generally web pages Search results are generally web pages Two main tools for searching the internet are: Two main tools for searching the internet are: Directory – it is a subject guide organized by major topics and subtopics Directory – it is a subject guide organized by major topics and subtopics Search Engines – it is a software that searches the internet Search Engines – it is a software that searches the internet Each of the tool uses database Each of the tool uses database Directory database is compiled by humans Directory database is compiled by humans Engine’s database is generated automatically by software Engine’s database is generated automatically by software

6. 6Chapter 7 - Searching the InternetDirectories Directories are human-powered search engines Directories are human-powered search engines A directory organizes information in a hierarchical tree by subjects A directory organizes information in a hierarchical tree by subjects Subjects at the top of the tree are very general and subjects at the bottom of the tree are specialized Subjects at the top of the tree are very general and subjects at the bottom of the tree are specialized There are two ways to search a directory There are two ways to search a directory Manual – browsing the directory subjects hierarchically Manual – browsing the directory subjects hierarchically Searching engine – typing the search words in a search field Searching engine – typing the search words in a search field Example – In Yahoo Directory at http://www.yahoo.com the user can browse the subjects by clicking on them or he can use the search field at the top of the directory to type in the search words, phrases, etc. Example – In Yahoo Directory at http://www.yahoo.com the user can browse the subjects by clicking on them or he can use the search field at the top of the directory to type in the search words, phrases, etc.http://www.yahoo.com

7. 7Chapter 7 - Searching the InternetDirectories

8. 8 Open Directory Project Search results from the internet are ranked Search results from the internet are ranked Directories use human editors to rank web pages Directories use human editors to rank web pages As the number of pages for a topic increase it becomes more time-consuming and cost-bearing to rank them As the number of pages for a topic increase it becomes more time-consuming and cost-bearing to rank them Open Directory Project hands over the ranking system to the users Open Directory Project hands over the ranking system to the users Users become editors and evaluate web sites in their area of expertise Users become editors and evaluate web sites in their area of expertise As a result of this project the services have a lot more content As a result of this project the services have a lot more content Visit http://dmoz.org for more information about Open Directory Project Visit http://dmoz.org for more information about Open Directory Projecthttp://dmoz.org

9. 9Chapter 7 - Searching the Internet Search Engines Examples of some of the important search engines are: 1. Google: http://www.google.comhttp://www.google.com 2. Yahoo: http://www.yahoo.comhttp://www.yahoo.com 3. MSN Search: http://search.msn.comhttp://search.msn.com 4. AOL Search: http://search.aol.comhttp://search.aol.com 5. Ask Jeeves: http://www.askjeeves.comhttp://www.askjeeves.com 6. AltaVista: http://www.altavista.comhttp://www.altavista.com The search engines provide comprehensive coverage and great relevancy To use the search engine type search string in a search field and then click search button Tips on using a search engine are available at http://www.searchenginewatch.com http://www.searchenginewatch.com

10. 10Chapter 7 - Searching the Internet Search Engines Search Engines are crawler based i.e. their listings are created automatically A search engine has three important parts 1. Spider: it is robot computer program to find web pages by following the links already in their databases 2. Indexer: it identifies the web page content and stores them in database files of the search engine 3. Searcher : it sifts through the engine’s index to find matches to a search string and it also ranks the matches Every result of a search is known as a hit Title/Frequency method is used to rank the results Search engines results may not always be relevant Different search engines produce different results

11. 11Chapter 7 - Searching the Internet Metasearch Engines Metasearch engines perform a multi-engine search i.e. they search other search engines Metasearch engines perform a multi-engine search i.e. they search other search engines Metasearch engine expands the internet search by using multiple search engines Metasearch engine expands the internet search by using multiple search engines Metasearch engine skips searching an engine which is down Metasearch engine skips searching an engine which is down Metasearch engines do not have their own database but use databases maintained by other engines Metasearch engines do not have their own database but use databases maintained by other engines Examples are: Examples are: http://www.dogpile.com http://www.dogpile.com http://www.dogpile.com http://www.metacrawler.com http://www.metacrawler.com http://www.metacrawler.com http://www.profusion.com http://www.profusion.com http://www.profusion.com

12. 12Chapter 7 - Searching the Internet Search Techniques Basic search involves typing a search string in a search field, and it usually gives satisfactory results Searching guidelines: Search engines lists best results first Change search string to get better results Search string is treated as key words and not an exact phrase Common Advanced Searching techniques are: Words and exact phrase Boolean search – uses boolean operators like AND, OR, NOT Title search – using web page title Site search – to limit search to a particular host name URL search, Link search Wildcard (fuzzy) search – uses the * symbol Features search – use special features of search engines

13. 13Chapter 7 - Searching the Internet Search Techniques

14. 14Chapter 7 - Searching the Internet Intelligent Agents New searching paradigms are needed to cope with the explosive growth of online information and databases Three retrieval paradigms exist: Statistical – correlations of word counts in documents Semantic – natural language processing and artificial intelligence Contextual – uses thesaurus and encoded relationships An intelligent agent is a program that gather information or performs services based on human input It uses the above three paradigms with other algorithms Example of intelligent agent is the Spider part of search engine

15. 15Chapter 7 - Searching the Internet Intelligent Agents Advantages of intelligent agents are: More intelligent search Create and update their own knowledge database Perform tasks quicker Communicate and co-operate with other agents Available all the time Agents are customizable Agents continuously scan internet for information

16. 16Chapter 7 - Searching the Internet Invisible Web A search engine cannot find every content due to format problems A search engine cannot find every content due to format problems Visible web is the result of a search that we can see Visible web is the result of a search that we can see Invisible web is the hidden web content Invisible web is the hidden web content Invisible web is estimated to be bigger than visible web Invisible web is estimated to be bigger than visible web A user can search the invisible web by using the following techniques: A user can search the invisible web by using the following techniques: Directories (Invisible Web Catalogue) Directories (Invisible Web Catalogue) Databases Databases Search engines – Google, and AllTheWeb Search engines – Google, and AllTheWeb

17. 17Chapter 7 - Searching the InternetSummary Internet is an endless repository of information Internet is an endless repository of information Search engines are used to search the internet Search engines are used to search the internet Directories maintain hierarchical trees by subjects Directories maintain hierarchical trees by subjects Open Directory Project is based on volunteer work by people and leads to building a superior method of organizing web content Open Directory Project is based on volunteer work by people and leads to building a superior method of organizing web content Google is the most popular search engine among others Google is the most popular search engine among others Metasearch engines is convenient for searching multiple search engines Metasearch engines is convenient for searching multiple search engines Basic search is enough most of the time Basic search is enough most of the time Intelligent agents allow intelligent searching Intelligent agents allow intelligent searching Invisible web may contain useful information Invisible web may contain useful information