1. School of Information University of Michigan Search in networks Lada Adamic (U. Michigan) NetSci Workshop May 16 th, 2006

2. Outline Search in structured networks small world experiments geographical models hierarchical models studies: HP labs email network (simulated) Club Nexus online community (simulated) Phone interview company (survey) LiveJournal (simulated) Search in unstructured networks power law networks Erdos-Renyi networks P2P networks (Gnutella example)

3. Search in structured networks

4. NE MA Milgram’s experiment (1960’s): Given a target individual and a particular property, pass the message to a person you correspond with who is “closest” to the target. Small world experiments then

5. Milgram’s small world experiment Target person worked in Boston as a stockbroker. 296 senders from Boston and Omaha. 20% of senders reached target. Typical strategy – if far from target choose someone geographically closer, if close to target geographically, choose someone professionally closer average chain length = 6.5 “Six degrees of separation”

6. Small world experiments now email experiment Dodds, Muhamad, Watts, Science 301, (2003) 18 targets 13 different countries 24,163 message chains 384 reached their targets average path length 4.0 image by Stephen G. Eick http://www.bell-labs.com/user/eick/index.html (unrelated to small world experiment…)

7. Small world experiment at Columbia Successful chains disproportionately used weak ties (Granovetter) professional ties (34% vs. 13%) ties originating at work/college target's work (65% vs. 40%)... and disproportionately avoided hubs (8% vs. 1%) (+ no evidence of funnels) family/friendship ties (60% vs. 83%) Strategy: Geography -> Work

8. Why study small world phenomena? Curiosity: Why is the world small? How are people able to route messages? Social Networking as a Business: Friendster, Orkut, MySpace,FaceBook LinkedIn, Spoke, VisiblePath

9. Six degrees of separation - to be expected Pool and Kochen (1978) - average person has 500-1500 acquaintances Ignoring clustering, other redundancy … ~ 10 3 first neighbors, 10 6 second neighbors, 10 9 third neighbors But networks are clustered: my friends’ friends tend to be my friends Watts & Strogatz (1998) - a few random links in an otherwise clustered graph give an average shortest path close to that of a random graph

10. Is this the whole picture? Why are small worlds navigable?

11. How to choose among hundreds of acquaintances? Strategy: Simple greedy algorithm - each participant chooses correspondent who is closest to target with respect to the given property Models geography Kleinberg (2000) hierarchical groups Watts, Dodds, Newman (2001), Kleinberg(2001) high degree nodes Adamic, Puniyani, Lukose, Huberman (2001), Newman(2003) How are people are able to find short paths?

12. Reverse small world experiment Killworth & Bernard (1978): Given hypothetical targets (name, occupation, location, hobbies, religion…) participants choose an acquaintance for each target Acquaintance chosen based on (most often) occupation, geography only 7% because they “know a lot of people” Simple greedy algorithm: most similar acquaintance two-step strategy rare

13. nodes are placed on a lattice and connect to nearest neighbors additional links placed with p uv ~ Spatial search “The geographic movement of the [message] from Nebraska to Massachusetts is striking. There is a progressive closing in on the target area as each new person is added to the chain” S.Milgram ‘The small world problem’, Psychology Today 1,61,1967 Kleinberg, ‘The Small World Phenomenon, An Algorithmic Perspective’ Proc. 32nd ACM Symposium on Theory of Computing, 2000. (Nature 2000)

14. When r=0, links are randomly distributed, ASP ~ log(n), n size of grid When r=0, any decentralized algorithm is at least a 0 n 2/3 no locality When r<2, expected time at least  r n (2-r)/3

15. Overly localized links on a lattice When r>2 expected search time ~ N (r-2)/(r-1)

16. Links balanced between long and short range When r=2, expected time of a DA is at most C (log N) 2

17. Hierarchical social network models

18. Kleinberg, ‘Small-World Phenomena and the Dynamics of Information’ NIPS 14, 2001 Hierarchical network models: Individuals classified into a hierarchy, h ij = height of the least common ancestor. Theorem: If  = 1 and outdegree is polylogarithmic, can s ~ O(log n) Group structure models: Individuals belong to nested groups q = size of smallest group that v,w belong to f(q) ~ q -  Theorem: If  = 1 and outdegree is polylogarithmic, can s ~ O(log n) h b=3 e.g. state-county-city-neighborhood industry-corporation-division-group ~

19. Sketch of proof T S R  |R|<|R’|< |R| k = c log 2 n calculate probability that s fails to have a link in R’ R’

20. Identity and search in social networks Watts, Dodds, Newman (Science,2001) individuals belong to hierarchically nested groups multiple independent hierarchies h=1,2,..,H coexist corresponding to occupation, geography, hobbies, religion… p ij ~ exp(-  x)

22. Identity and search in social networks Watts, Dodds, Newman (2001) Message chains fail at each node with probability p Network is ‘searchable’ if a fraction r of messages reach the target N=102400 N=409600 N=204800

23. Small World Model, Watts et al. Fits Milgram’s data well Model parameters: N = 10 8 z = 300 g = 100 b = 10  = 1, H = 2 L model = 6.7 L data = 6.5 http://www.aladdin.cs.cmu.edu/workshops/wsa/papers/dodds-2004-04-10search.pdf more slides on this:

24. Mary Bob Jane Who could introduce me to Richard Gere? High degree search Adamic et al. Phys. Rev. E, 64 46135 (2001)Phys. Rev. E, 64 46135 (2001)

25. Small world experiments so far Classic small world experiment: Given a target individual, forward to one of your acquaintances Observe chains but not the rest of the social network Reverse small world experiment (Killworth & Bernard) Given a hypothetical individual, which of your acquaintances would you choose Observe individual’s social network and possible choices, but not resulting chains or complete social network

26. Use a well defined network: HP Labs email correspondence over 3.5 months Edges are between individuals who sent at least 6 email messages each way 450 users median degree = 10, mean degree = 13 average shortest path = 3 Node properties specified: degree geographical location position in organizational hierarchy Can greedy strategies work? Testing search models on social networks advantage: have access to entire communication network and to individual’s attributes

27. Power-law degree distribution of all senders of email passing through HP labs Strategy 1: High degree search number of recipients sender has sent email to proportion of senders

28. Filtered network (at least 6 messages sent each way) Degree distribution no longer power-law, but Poisson It would take 40 steps on average (median of 16) to reach a target!

29. Strategy 2: Geography

30. 1U 2L3L 3U 2U 4U 1L 87 % of the 4000 links are between individuals on the same floor Communication across corporate geography

31. Cubicle distance vs. probability of being linked optimum for search

32. Strategy 3: Organizational hierarchy

33. Email correspondence superimposed on the organizational hierarchy

34. Example of search path distance 1 distance 2 hierarchical distance = 5 search path distance = 4 distance 1

35. Probability of linking vs. distance in hierarchy in the ‘searchable’ regime: 0 <  < 2 (Watts, Dodds, Newman 2001)

36. Results distancehierarchygeographygeodesicorgrandom median473628 mean5.7 (4.7)123.16.157.4 hierarchy geography

37. Expt 2 Searching a social networking website

39. Profiles: status (UG or G) year major or department residence gender Personality (choose 3 exactly): youfunny, kind, weird, … friendshiphonesty/trust, common interests, commitment, … romance- “ - freetimesocializing, getting outside, reading, … supportunconditional accepters, comic-relief givers, eternal optimists Interests (choose as many as apply) booksmystery & thriller, science fiction, romance, … movieswestern, biography, horror, … musicfolk, jazz, techno, … social activitiesballroom dancing, barbecuing, bar-hopping, … land sportssoccer, tennis, golf, … water sportssailing, kayaking, swimming, … other sportsski diving, weightlifting, billiards, …

40. Differences between data sets complete image of communication network affinity not reflected partial information of social network only friends listed HP labs email networkOnline community

41. Degree Distribution for Nexus Net 2469 users, average degree 8.2

42. Problem: how to construct hierarchies? Probability of linking by separation in years

43. Hierarchies not useful for other attributes: Geography Other attributes: major, sports, freetime activities, movie preferences…

44. Strategy using user profiles prob. two undergrads are friends (consider simultaneously) both undergraduate, both graduate, or one of each same or different year both male, both female, or one of each same or different residences same or different major/department Results random133390 high degree 39137 profile 21 53 strategymedianmean With an attrition rate of 25%, 5% of the messages get through at an average of 4.8 steps, => hence network is barely searchable

45. The accuracy of small world chains in social networks Peter D. Killworth, Christopher McCarty, H. Russell Bernard, Mark House Social Networks, 2006 First parallel study of individuals choices vs. actual shortest paths Network 105 members of an interviewing bureau 10,920 shortest path connections who knows whom who a person would select as the next link in a chain to a particular person x recent hire old timer worked a while

46. Accuracy of small world chains Shortest paths use the network of who-knows whom to calculate actual shortest paths compare to paths formed by individuals’ choices 21.7% fail through reaching missing data 23.7% reach cycles : i chooses j, j chooses i 54.6% reach the target, with chains that are 40% longer on average than the shortest path

47. Next choice accuracy and a Markov model 48% of the time, a person chooses a contact who is closer to the target over half of the choices are wrong! Markov model: terminate chain with probability  (attrition) choose someone closer to the target with probability p, otherwise choose someone at same distance

48. LiveJournal LiveJournal provides an API to crawl the friendship network + profiles friendly to researchers great research opportunity basic statistics Users How many users, and how many of those are active? Total accounts: 9980558... active in some way: 1979716... that have ever updated: 6755023... updating in last 30 days: 1300312... updating in last 7 days: 751301... updating in past 24 hours: 216581

49. Predominantly female & young demographic Male: 1370813 (32.4%) Female: 2856360 (67.6%) Unspecified: 1575389 1318483 1487505 15211445 16343922 17400947 18414601 19405472 20371789 21303076 22239255 23194379 24152569 25127121 2698900 2773392 2859188 2948666 Age distribution

50. Geographic Routing in Social Networks David Liben-Nowell, Jasmine Novak, Ravi Kumar, Prabhakar Raghavan, and Andrew Tomkins (PNAS 2005) data used Feb. 2004 500,000 LiveJournal users with US locations giant component (77.6%) of the network clustering coefficient: 0.2

51. Degree distributions The broad degree distributions we’ve learned to know and love but more probably lognormal than power law broader in degree than outdegree distribution

52. Results of a simple greedy geographical algorithm Choose source s and target t randomly Try to reach target’s city – not target itself At each step, the message is forwarded from the current message holder u to the friend v of u geographically closest to t stop if d(v,t) > d(u,t) 13% of the chains are completed stop if d(v,t) > d(u,t) pick a neighbor at random in the same city if possible, else stop 80% of the chains are completed

53. the geographic basis of friendship  = d(u,v) the distance between pairs of people The probability that two people are friends given their distance is equal to P(  ) =  + f(  ),  is a constant independent of geography  is 5.0 x 10 -6 for LiveJournal users who are very far apart

54. the geographic basis of friendship The average user will have ~ 2.5 non-geographic friends The other friends (5.5 on average) are distributed according to an approximate 1/distance relationship But 1/d was proved not to be navigable by Kleinberg, so what gives?

55. Navigability in networks of variable geographical density Kleinberg assumed a uniformly populated 2D lattice But population is far from uniform population networks and rank-based friendship probability of knowing a person depends not on absolute distance but on relative distance (i.e. how many people live closer) Pr[u ->v] ~ 1/rank u (v)

56. Structured search Conclusions Individuals associate on different levels into groups. Individuals tend to know others who are ‘close by’ Group structure facilitates decentralized search using social ties. Hierarchy search faster than geographical search Simple strategies are not perfect – but short (rather than shortest) chains can be found

57. Weighted shortest paths Routes shortest route from Chicago to Boston vertex: intersection edge weights: road distances alternative weights: expected time traveled, gas consumed… usually sum the weights from each segment start finish freeway, 65 mph 40 miles/65 mph ~ 37 minutes freeway, 70 mph 30 miles/70 mph ~ 26 minutes surface road 25 mph, 50 miles 2 hours

58. Reliable paths through social networks The probability of transmitting a message or infectious agent could be related to the strength of the tie e.g. rather than summing the weights, we might multiply the probabilities of getting through p = 0.5 p = 0.001 p = 1 Probability of getting an idea through to the head of labs via CEO (0.001*1 = 0.001), via direct manager (0.5*0.5 = 0.25) p = 0.05

59. Motivation Power-law (PL) networks, social and P2P Analysis of scaling of search strategies in PL networks Simulation artificial power-law topologies, real Gnutella networks Comparison with existing P2P search strategies Reflector, Morpheus Directed Search Freenet Search in random networks 2

60. Mary Bob Jane Who could introduce me to Richard Gere? How do we search?

61. AT&T Call Graph Aiello et al. STOC ‘00 # of telephone numbers from which calls were made # of telephone numbers called

62. Gnutella network power-law link distribution summer 2000, data provided by Clip2

63. Preferential attachment model Nodes join at different times The more connections a node has, the more likely it is to acquire new connections Growth process produces power-law network host cache ping

64. file sharing w/o a central index queries broadcast to every node within radius ttl  as network grows, encounter a bandwidth barrier (dial up modems cannot keep up with query traffic, fragmenting the network) Gnutella and the bandwidth barrier Clip 2 report Gnutella: To the Bandwidth Barrier and Beyond http://www.clip2.com/gnutella.html#q17

65. 1 6 54 63 67 2 94 number of nodes found power-law graph

66. 93 number of nodes found 1 3 7 1115 19 Poisson graph

67. Search with knowledge of 2 nd neighbors

68. Outline of search strategy pass query onto only one neighbor at each step requires that nodes sign query - avoid passing message onto a node twice requires knowledge of one’s neighbors degree - pass to the highest degree node requires knowledge of one’s neighbors neighbors - route to 2 nd degree neighbors OPTIONS

69. Generating functions M.E.J. Newman, S.H. Strogatz, and D.J. Watts ‘Random graphs with arbitrary degree distributions and their applications’, PRE, cond-mat/0007235 Generating functions for degree distributions Useful for computing moments of degree distribution, component sizes, and average path lengths

70. Fun with generating functions normalization condition: probabilities sum to 1 derivatives: the generating function contains all the information of the degree distribution

71. Fun with generating functions (cont’d) Expected degree of a randomly chosen vertex Higher moments of degree distribution

72. Example: Poisson distribution Let p = z/N be the probability of an edge existing between two vertices (z is the average degree) for large N just the regular Poisson distribution

73. Introducing cutoffs a node cannot have more connections than there are other nodes This is important for exponents close to 2 Probability that none of the nodes in a 1,000 node graph has 1000 or more neighbors: without a cutoff, for  = 2 have > 50% chance of observing a node with more neighbors than there are nodes for  = 2.1, have a 25% chance

74. # of sites linking to the site proportion of sites w/ so many links 1000 Selecting from a variety of cutoffs 1.2.Newman et al. 3. otherwise Generating Function Aiello et al. 1 million websites (~ 1997)

75. Aiello’s ‘conservative’ vs. Havlin’s ‘natural’ cutoff cutoff where expected number of nodes of degree k is 1 k n(k) k 1 1 cutoff so that expected number of nodes of degree > k is 1

76. The imposed cutoff can have a dramatic effect on the properties of the graph degrees drawn at random, for  = 2, and N = 1000

77. is the probability that a randomly chosen vertex has degree k is a generating function is the expected degree of a randomly chosen vertex is the distribution of remaining outgoing edges following and edge assuming neighbors don’t share edges is the expected number of second degree neighbors 2 2 2 2 2 2 1 1 1 Generating functions for degree distributions Random graphs with arbitrary degree distributions and their applications by Newman, Strogatz & Watts

78. search with knowledge of first neighbors Generating function with cutoff Average degree of vertex constant in N for 2<  <3, and k max ~N a, decreases with N Average number of neighbors following an edge ~

79. search with knowledge of first neighbors (cont’d) In the limit t->2, Let’s for the moment ignore the fact that as we do a random walk, we encounter neighbors that we’ve seen before s = number of steps = ~~ ~

80. Search time with different cutoffs If k max = N, If k max = N 1/(  -1), ~ ~ ~ ~ ~ ~ grow from 1,000 to 1,000,000 nodes, search time increases by a factor of ~2 grow 1000x search time increases 3x

81. search with knowledge of first neighbors (cont’d) If k max = N , So the best we can do is for exponents close to 2 2nd neighbor random walk, ignoring overlap: ~

82. Following the degree sequence a ~ s = # of steps taken 2 nd neighbors, ignoring overlap: Go to highest degree node, then next highest, … etc.

83. 0102030405060708090100 1  = 2.00  = 2.25  = 2.50  = 2.75  = 3.00  = 3.25  = 3.50  = 3.75 degree of node degree of neighbor - 1 degree of node 2 20 10 5 Ratio of the degree of a node to the expected degree of its highest degree neighbor for 10,000 node power-law graphs of varying exponents

84. Actor collaboration graph (imdb database)  ~ 2.0-2.2 Exponents  close to 2 required to search effectively World Wide Web,  ~ 2.0-2.3, high degree nodes: directories, search engines Social networks, AT&T call graph  ~ 2.1 Gnutella 10 0 10 1 10 2 10 3 10 4 10 0 10 1 10 2 10 3 10 4 10 5 number of costars number of actors/actresses actors,  = 2 actresses,  = 2.1

85. 1 5 50 18 17 8 10 9 6 Following the degree sequence

86. Complications Should not visit same node more than once Many neighbors of current node being visited were also neighbors of previously visited nodes, and there is a bias toward high degree nodes being ‘seen’ over and over again

87. 0100200300400500600 0 5 10 15 20 25 30 step degree of node not visited visited neighbors visited Status and degree of node visited

88. 1 10 -2 0.1 1 step proportion of nodes found at step random walk 10 2 3 4 5 6 -3 10 -4 020406080100 0.2 0.4 0.6 0.8 1 step cumulative nodes found at step random walk degree sequence seeking high degree nodes speeds up the search process about 50% of a 10,000 node graph is explored in the first 12 steps Progress of exploration in a 10,000 node graph knowing 2 nd degree neighbors 12 degree sequence

89. size of graph covertime for half the nodes random walk  = 0.37 fit degree sequence  = 0.24 fit Scaling of search time with size of graph

90. Comparison with a Poisson graph degree of current node step Poisson power-law 10 1 2 4 6 0 1 2 3 4 5 number of nodes in graph cover time for 1/2 of graph constant av. deg. = 3.4  = 1.0 fit expected degree and expected degree following a link are equal scaling is linear

91. 020406080100 0 0.2 0.4 0.6 0.8 1 step cumulative nodes found at step high degree seeking 1st neighbors high degree seeking 2nd neighbors 50% of the files in a 700 node network can be found in < 8 steps Gnutella network

92. Maintain a list of files in their neighborhood Check query against list. Periodically contact neighbors to maintain list Append ID to each query processed Required modifications to nodes Tradeoff storage/cpu (available) bandwidth (limited) for

93. localized indexing traffic routed to high degree nodes Partial implementation: Theory vs. reality: overloading high degree nodes but no worse than original scenario where all nodes handle all traffic assume high degree -> high bandwidth so can carry the traffic load fewer nodes used for routing, system is more susceptible to malicious attack

94. Clip2 Distributed Search Solutions http://dss.clip2.com © Clip2.com, Inc. Broadband user running Reflector Broadband user running Gnutella Dial-up user running Gnutella

95. LimeWire, BearShare: drop connections to unresponsive hosts drives slower hosts to have fewer connections & move to edge of network Kazaa, BearShare defender, Morpheus SuperNodes from Clip2: Morpheus out of the Underworld http://www.openp2p.com/pub/a/p2p/2001/07/02/morpheus.html Connection-preferencing rules Supernodes

96. Freenet Queries are passed to one peer at a time. Queries routed to high degree nodes. Has a power-law topology Theodore Hong, ‘Performance’ chapter in O’Reilley’s “Peer-to-Peer, Harnessing the Power of Disruptive Technologies” Scales as N 0.275 with the size of the network, N.

97. Theodore Hong, power - law link distribution of a simulated Freenet network

98. Theodore Hong, scaling of mean search time on a simulated Freenet network

99. Node specialization key to Freenet’s speed Each node forwards query to node with “closest” hash key Node passing back a match remembers the address the data came from Results in nodes developing a bias towards a part of the keyspace 112 659 ?356? 340 388 396 135 214 356 340 388 396 135 214 Queries are naturally routed to high degree nodes Use keys for orientation

100. Conclusions Search is faster and scales in power-law networks Networks intended to be searched, such as Gnutella, have a favorable P-L topology High degree strategy has partially been implemented in existing p2p clients, such as BearShare, Kazaa & Morpheus

101. Current research on search search in weighted networks expertise search P2P architectures with ‘friendship’ overlays weak ties vs. strong ties and online communication

102. A PL link distribution shortens the average shortest path Poisson:  = z 1 PL:  > z 1

104. A B What about the shortest path discovered along the way? B.J. Kim et al. ‘Path finding strategies in scale-free networks’, PRE (65) 027103. each node passes message to highest degree neighbor it hasn’t passed the message to previously ‘cut off’ loops

105. A high degree seeking strategy finds shortest paths whose average scales logarithmically with the size of the graph

106. Scaling of the path length found using a random strategy on a PL graph high-degree strategy on a Poisson graph

107. But… Search costs are prohibitive, might as well do a BFS