PEERSPECTIVE.MPI-SWS.ORG ALAN MISLOVE KRISHNA P. GUMMADI PETER DRUSCHEL BY RAGHURAM KRISHNAMACHARI Exploiting Social Networks for Internet Search

Motivation WWW, Search engines, social networking Hyperlinks – author, human, index, rank Social Networks  No study to examine information exchange  Explicit links between users, not content  Can these links be used by search engines? In this paper  Compare mechanisms for publishing and location  Experiment: Social network based Web search  Challenges in leveraging social networks in the future

The Web verses Social Networks Publishing  Users place documents on Web server  Author places hyperlinks on Web page that refer to related pages  Links placed to increase rank and promote indexing Locating  Web search engines employing sophisticated technologies  Google: Uses hyperlink structure and query/page relevance  Limitations:  New pages: discovering/indexing, hyper-linking, link(s) discovery  # of links determines relevance -> reflects interests/biases of the Web  Ignored: Unlinked/private pages, pages with insufficient relevance

The Web verses Social Networks Ex: User shares Web content with friends; content is invisible to others; content is now linked between users Publishing  Content is posted by the user and is recommended by others  Links among users: Directed (distinct) & Undirected (mutual) Locating  Traversing the social network, keyword search, top 10 lists  Timely, relevant & reliable (non-)textual info can be found  Content is rated by consumers, not producers  Content is rated almost immediately; doesn’t rely on discovery

Integrating Web search and social networks Problem  No unified search tool, no unified finding tool as well  Social network-based search not used in Web and vice versa Questions  Leverage social network links to improve search results  Explore benefits of social network-based Web search Solution  Conduct an experiment to validate these

PeerSpective: The experiment Web content of 10 students/researchers are shared A HTTP proxy indexes all visited URLs by an user When a Google search (query) is performed  Local proxy forwards query to Google and to all peer proxies  All proxies execute the query on local index & return results  Results are collated and presented alongside the Google results

PeerSpective: Measurements & Experiences In a month long experimental deployment (10 users)  439, 384 HTTP requests  198, 492 distinct URLs (45%)  113, 800 HTML and PDF requests (25.9%) User base is small, with highly specialized interests The results may not represent a large, diverse group Technology  Local text search engine – Lucene  Local peer-peer overlay engine - FreePastry

Limits of hyperlink-based search Web search engines index only well linked content Limit: URLs visited by users / not indexed by Google Reasons why a page might not be indexed  The page could be too new (blogs, news)  The page could be in deep web and not well connected  The page could be in dark web (private pages)

PeerSpective verses Google For each HTTP request  Does Google’s index contain this URL  Has some peer in PeerSpective viewed this URL Static HTML content (No GET/POST)  6,679 requests (<6%) for 3,987 URLs (2%) Google Index  Covers 62.5% of the requests, 68.1% of the distinct URLs  1/3rd of all URL requests cannot be retrieved by Google PeerSpective Index  Covers 30.4% of requested URLs  Achieves half of Google’s coverage with a much smaller size  13.3% of the URLs were in PeerSpective but not in Google’s index  19.5% improvement by PeerSpective compared to Google search What are the documents that interests our users, but not Google ?

Benefits of social network-based search Search engines have to rank pages  Users rarely go beyond first 20 search results 1,730 Google searches were observere d  First page results: Google – 9.45, PeerSpective – 5.17  1,079 (62.3%) resulted in clicks on result(s)  307 (17.7%) were followed by a refined query  Users gave up 344 (19.8%) of the time  933 (86.5%) of clicked results were returned only by Google  83 (7.7%) of clicked results were returned only by PeerSpective  63 (5.7%) of clicked results were returned by both  9% improvement in search result clicks over Google alone

How PeerSpective outperforms Google Disambiguation  Search terms have multiple meanings depending on the context Ranking  Search engine: Top rank, Social Network: Nearby pages Serendipity  Making unexpected or fortunate discoveries

Opportunities and Challenges Online social networking enables new forms of information exchange  Users can very easily and conveniently publish information  Makes it possible to locate and access “WOM” information  Organizes information according to tastes and preferences of smaller groups of individuals Opportunities and Challenges  Privacy – willingness of individuals to share information  Membership and clustering of social networks  Content rating and ranking (page rank, views)  System architecture (centralized or distributed)

Thank You