1. ITIS 1210 Introduction to Web-Based Information Systems Chapter 27 How Internet Searching Works

2. Introduction  Major advantage of Internet  HUGE amounts of information available  Major DISadvantage:  It’s DISORGANIZED  There is no central ownership so there’s  No common filing system  No “Dewey Decimal System”  No common glossary

3. Introduction  I think the book confuses these two terms but …  Two popular solutions  Indexes  Search Engines  Indexes  Provide a lookup mechanism  Associate lookup terms with information  Example: telephone book

5. Introduction  On the Web, indexes tend to be hierarchical  Click on a click  Takes you to a list of sub-categories  May be several levels deep  At some point documents associated with that category become available  Yahoo! For example  Binary search!

6. Introduction  Yahoo! Yahoo!

7. Introduction  Search engines are massive databases  Don’t present information in a hierarchical form  Instead  You provide search term  Database returns relevant results  Confusion results from fact that database are organized around indexes

8. Introduction  Google tends to be the default search engine  Big mistake, other ones out there, some better  Have three basic characteristics  Spider – gathers information from Web  Database – Stores that information  Search tool – Lets users search the database

9. Introduction  Spiders work constantly  Retrieving information from Web sites  Presenting it to search engine for analysis  Engines characterize the data  Remember, HTML is basically just text files  By keyword in document  Some use all, some only a few  By entire content of document  By size, title, heading, metadata, etc.

10. Introduction  Results from this analysis stored in database  Typically keyword & URL  Other information might be stored as well  First few sentences on Web page  Last date updated  Search tool provides access to database for users

11. Introduction  User provides keyword (or other search term)  Tool returns list of data in database that fits that term  Different search engines return results in different ways  Weigh results  Sponsored links

12. Introduction  No search engine indexes the entire Internet  All have strengths and weaknesses  Limiting your self to a single search engine is  Intellectually lazy  Academically short-sighted  Ultimately self-defeating  Would you only shop at Wal-Mart?

13. How Internet Search Engines Work  Spiders are just programs  They follow a link to a Web page  Really, the Web page comes to them  Different spiders follow different rules  Written by different programmers for different companies  Follow every link  Ignore links to graphics files or newsgroups

14. How Internet Search Engines Work  Content analyzed and stored in database  Indexes are built and updated  Users visit search engine site  Provide search term  Describes the information you want  Might ask to match keyword(s)  Might be to limit by  Date  Type of file  Size

15. How Internet Search Engines Work  Search engine software  Searches database for information matching keywords  Results formatted as a Web page in HTML  Sent back to user

16. How Meta-Search Software Works  Problem  Web is too large for any one search engine to consistently provide superior results  Best approach is a combination of results from multiple search engines  Meta-search software lets you search multiple search engines for results  Better than searching each one individually

17. How Meta-Search Software Works  Some software uses multiple “agents”  Specialized software that does one thing very well – like searching  Each agent contacts a single search engine  The agents know how each engine functions  How to ask the right question

18. How Meta-Search Software Works  Individual search engines report their results to the agent  Agents report results back to meta-search engine  This process may be repeated multiple times by each agent

19. How Meta-Search Software Works  Meta-search engine deletes duplicate results  May rank based on how many search engines report a specific site  One indication of quality – lots of references  You see a result you like, click on the link, and are taken to that Web page

20. How Meta-Search Software Works  No two meta-search engines are alike  Some only refer to popular search engines  Some include lesser-known search engines, newsgroups, or database  Also differ on how results are presented  Some indicate which search engine returned results, some don’t

21. Links  http://www.dmoz.org/Computers/Internet/S earching/Metasearch// http://www.dmoz.org/Computers/Internet/S earching/Metasearch// http://www.dmoz.org/Computers/Internet/S earching/Metasearch//

22. Links  http://www.lib.berkeley.edu/TeachingLib/G uides/Internet/MetaSearch.html http://www.lib.berkeley.edu/TeachingLib/G uides/Internet/MetaSearch.html http://www.lib.berkeley.edu/TeachingLib/G uides/Internet/MetaSearch.html  http://news.zdnet.com/2100-9588_22- 5647280.html http://news.zdnet.com/2100-9588_22- 5647280.html http://news.zdnet.com/2100-9588_22- 5647280.html  http://www.dmoz.org/Computers/Internet/S earching/Metasearch// http://www.dmoz.org/Computers/Internet/S earching/Metasearch// http://www.dmoz.org/Computers/Internet/S earching/Metasearch//