Virtual Communities and Gossiping in Social-Based P2P Systems

Slides:



Advertisements
Similar presentations
Virginia Career Network LinkedIn. What is LinkedIn? Overview Creating a profile Growing your contacts Sourcing capability.
Advertisements

Neighbour selection strategies in BitTorrent- like Peer-to-Peer systems L.G. Alex Sung, Herman Li March 30, 2005 for CS856 Web Data Management University.
CodeTorrent: Content Distribution using Network Coding in VANET Uichin Lee, JoonSang Park, Joseph Yeh, Giovanni Pau, Mario Gerla Computer Science Dept,
Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.
The BitTorrent Protocol. What is BitTorrent?  Efficient content distribution system using file swarming. Does not perform all the functions of a typical.
A P2P REcommender system based on Gossip Overlays (PREGO) ‏ R.Baraglia, P.Dazzi M.Mordacchini, L.Ricci A P2P REcommender system based on Gossip Overlays.
Addressing the P2P Bootstrap Problem for Small Overlay Networks David Wolinsky, Pierre St. Juste, P. Oscar Boykin, and Renato Figueiredo ACIS P2P Group.
Project in Computer Security Integrating TOR’s attacks into the I2P darknet Chen Avnery Amihay Vinter.
CompSci 356: Computer Network Architectures Lecture 21: Content Distribution Chapter 9.4 Xiaowei Yang
June 3, Correlating Topology and Path Characteristics of Overlay Networks and the Internet GP2PC’06, in conjunction with IEEE CCGrid2006 A. Iosup,
CMPT 401 Summer 2007 Dr. Alexandra Fedorova Lecture XV: Real P2P Systems.
Network Coding for Large Scale Content Distribution Christos Gkantsidis Georgia Institute of Technology Pablo Rodriguez Microsoft Research IEEE INFOCOM.
Improving ISP Locality in BitTorrent Traffic via Biased Neighbor Selection Ruchir Bindal, Pei Cao, William Chan Stanford University Jan Medved, George.
Spotlighting Decentralized P2P File Sharing Archie Kuo and Ethan Le Department of Computer Science San Jose State University.
CoolStreaming/DONet: A Data- driven Overlay Network for Peer- to-Peer Live Media Streaming INFOCOM 2005 Xinyan Zhang, Jiangchuan Liu, Bo Li, and Tak- Shing.
EMule behind the scenes. Overview Extends the eDonkey protocol File sharing network Several hundreds of eMule servers Millions of eMule clients Each server.
Exploiting Content Localities for Efficient Search in P2P Systems Lei Guo 1 Song Jiang 2 Li Xiao 3 and Xiaodong Zhang 1 1 College of William and Mary,
Part 1: Overview of Web Systems Part 2: Peer-to-Peer Systems Internet Computing Workshop Tom Chothia.
1March -05 Jiangchuan Liu with Xinyan Zhang, Bo Li, and T.S.P.Yum Infocom 2005 CoolStreaming/DONet: A Data-Driven Overlay Network for Peer-to-Peer Live.
On-Demand Media Streaming Over the Internet Mohamed M. Hefeeda, Bharat K. Bhargava Presented by Sam Distributed Computing Systems, FTDCS Proceedings.
1CS 6401 Peer-to-Peer Networks Outline Overview Gnutella Structured Overlays BitTorrent.
Middleware for P2P architecture Jikai Yin, Shuai Zhang, Ziwen Zhang.
1 Tribler: P2P Television Disruptive changes in the media landscape name:Johan Pouwelse Assistant Professor Gmail. com date:May 16nd 2007 event:TBM.
1 Proceeding the Second Exercises on Computer and Systems Engineering Professor OKAMURA Laboratory. Othman Othman M.M.
1 Napster & Gnutella An Overview. 2 About Napster Distributed application allowing users to search and exchange MP3 files. Written by Shawn Fanning in.
A measurement study of vehicular internet access using in situ Wi-Fi networks Vladimir Bychkovsky, Bret Hull, Allen Miu, Hari Balakrishnan, and Samuel.
Leveraging Social Networks for Increased BitTorrent Robustness Wojciech Galuba, Karl Aberer EPFL, Switzerland Zoran Despotovic, Wolfgang Kellerer Docomo.
Privacy-Preserving P2P Data Sharing with OneSwarm -Piggy.
COCONET: Co-Operative Cache driven Overlay NETwork for p2p VoD streaming Abhishek Bhattacharya, Zhenyu Yang & Deng Pan.
Network Layer (3). Node lookup in p2p networks Section in the textbook. In a p2p network, each node may provide some kind of service for other.
By Shobana Padmanabhan Sep 12, 2007 CSE 473 Class #4: P2P Section 2.6 of textbook (some pictures here are from the book)
University of Bologna, Italy How to cheat BitTorrent and why nobody does Simon Patarin and David Hales University of Bologna ECCS 2006,
BitTorrent Internet Technologies and Applications.

1 Telematica di Base Applicazioni P2P. 2 The Peer-to-Peer System Architecture  peer-to-peer is a network architecture where computer resources and services.
1 BitTorrent System Efrat Oune Bar-Ilan What is BitTorrent? BitTorrent is a peer-to-peer file distribution system (built for intensive daily use.
How do you spread the news using NNTP (Network News Transfer Protocol) - Avinash Gosavi
ITIS 1210 Introduction to Web-Based Information Systems Chapter 38 How Music Sharing and File Sharing Work.
A P2P file distribution system ——BitTorrent Pegasus Team CMPE 208.
1 BitHoc: BitTorrent for wireless ad hoc networks Jointly with: Chadi Barakat Jayeoung Choi Anwar Al Hamra Thierry Turletti EPI PLANETE 28/02/2008 MAESTRO/PLANETE.
2: Application Layer1 Chapter 2: Application layer r 2.1 Principles of network applications r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail  SMTP,
Do incentives build robustness in BitTorrent? Michael Piatek, Tomas Isdal, Thomas Anderson, Arvind Krishnamurthy, Arun Venkataramani.
Professor OKAMURA Laboratory. Othman Othman M.M. 1.
Bit Torrent A good or a bad?. Common methods of transferring files in the internet: Client-Server Model Peer-to-Peer Network.
From Torrent to Magnet Link Presenter: Guanlong Zhou Fan Feng a new way of sharing on P2P network TRIBLER: A social-based Peer-to-Peer system.
MULTI-TORRENT: A PERFORMANCE STUDY Yan Yang, Alix L.H. Chow, Leana Golubchik Internet Multimedia Lab University of Southern California.
Othman Othman M.M., Koji Okamura Kyushu University 1.
Freelib: A Self-sustainable Digital Library for Education Community Ashraf Amrou, Kurt Maly, Mohammad Zubair Computer Science Dept., Old Dominion University.
P2P Streaming Protocol (PPSP) Requirements draft-zong-ppsp-reqs-03.
Kaleidoscope – Adding Colors to Kademlia Gil Einziger, Roy Friedman, Eyal Kibbar Computer Science, Technion 1.
1 Peer-to-Peer Technologies Seminar by: Kunal Goswami (05IT6006) School of Information Technology Guided by: Prof. C.R.Mandal, School of Information Technology.
B IT T ORRENT T ECHNOLOGY Anthony Pervetich. H ISTORY Bram Cohen Designed the BitTorrent protocol in April 2001 Released July 2, 2001 Concept Late 90’s.
Flashback: A Peer-to-Peer Web Server for Flash Crowds Presented by Tom Batkiewicz CS 587x Fall ‘07.
SocialTube: P2P-assisted Video Sharing in Online Social Networks
6° of Darkness or Using Webs of Trust to Solve the Problem of Global Indexes.
1 NETWORKING 2012 Parallel and Distributed Systems Group, Delft University of Technology, the Netherlands May 22, 2012 Reducing the History in Decentralized.
1 HOTP2P 2011 Parallel and Distributed Systems Group, Delft University of Technology, the Netherlands May 20, 2011 Betweenness Centrality Approximations.
Bit Torrent Nirav A. Vasa. Topics What is BitTorrent? Related Terms How BitTorrent works Steps involved in the working Advantages and Disadvantages.
2-Hop TorrentSmell A distributed tracking algorithm name:Raynor Vliegendhart date:July 10, 2009 event:Tribler Dev Meeting.
INTERNET TECHNOLOGIES Week 10 Peer to Peer Paradigm 1.
Public and private BitTorrent communities: A measurement study M. Meulpolder, L. D’Acunto, M. Capotă, M. Wojciechowski, J.A. Pouwelse, D.H.J. Epema, H.J.
An example of peer-to-peer application
Copyright notice © 2008 Raul Jimenez - -
(Parallel and) Distributed Systems Group
Economics and Computation Week 7: The economics of P2P file sharing
The BitTorrent Protocol
Balancing Throughput, Robustness, and In-Order Delivery in P2P VoD
Overview Multimedia: The Role of WINS in the Network Infrastructure
#02 Peer to Peer Networking
Presentation transcript:

Virtual Communities and Gossiping in Social-Based P2P Systems Dick Epema Parallel and Distributed Systems Delft University of Technology Delft, the Netherlands Gossiping Workshop Leiden, 21 december 2006

The I-Share Research Project (1): P2P-TV Distributing TV is the killer P2P application in the internet in the next decade recorded: millions of PVRs form one huge repository (how to find things) live: low-cost entry for content distributors (how to stream things) P2P-TV forms a foundation for sharing with your friends (creating virtual communities) content (you can have what I have) interest profiles (you may like what I like) In the international arena, P2P-TV is increasingly seen as a viable and innovation-driving alternative to (server-client) IP-TV

The I-Share Research Project (2): Tribler P2P-TV client is an inspiring and concrete vehicle for multidisciplinary research Tests in a lab environment are not enough for this research: real users with real networks and real content are needed Hence the design and implementation of With P2P-TV/Tribler, we can meet a multitude of generic research challenges: Efficient internet protocols Efficient video streaming Understandable content navigation User profiling and recommending Protection of privacy Protection of rights … …

Outline Introduction (done) Virtual communities Tribler Gossiping in Tribler: Content recommendation: Buddycast Swarm discovery: Little Bird Maintaining a social-based P2P network: NN as yet Research Questions

Virtual communities (1): internet evolution Until about 7 years ago, the internet had a core of powerful servers 100s of millions of PCs (the dark matter of the internet) talking to those servers Currently, the internet is a powerful ISP-connected network with millions of powerful servers and billions of users connected though PCs/ADSL to each other (and those servers) Those users want to form Virtual Communities: fans of Madonna (or Mahler) Italy-loving amateur cooks fans of Feyenoord and myriads of others

Virtual communities (2): issues What types of VCs are there? differences with real communities number of participants/interactions How to create and manage VCs: membership management (become a member, prove membership, credentials) currently, virtually all VCs are centrally managed How to behave as a member: be a good citizen incentives to cooperate How to store and disseminate information: on membership information/content maintained by the VC Gossiping may help here!!!

Tribler (1): main features Is based on the Bittorrent P2P file-sharing system Looks at the peers as really representing actual users rather than as anonymous computers Adds social-based functionality De-anonymizes peers: peers have a quasi-unique public permanent identifier, which can be used to challenge a peer for its identity Can show the physical location of peers Uses gossiping for content recommendation, swarm discovery, and maintaining social networks Has been released on 17 march 2006

Tribler (2): data distribution model Borrowed from Bittorrent: Swarm – the group of peers (VC) downloading the same file Seeder – a peer who has the complete file and gives it away for free Leecher – a peer whose download is in progress Files are divided into chunks Chunks are exchanged between peers according to a tit-for-tat strategy

Gossiping 1 – BuddyCast: the basic idea Buddycast is an epidemic protocol for peer and content discovery and recommendation Peers maintain lists of buddies and of random peers Buddycast switches between sending a buddycast message to a buddy (exploitation) and a random peer (exploration) Exploitation finding similar peers and discover their files social network (your buddies) Exploration discover new peers other (random) peers

Gossiping 1 – BuddyCast: messages Message contents 50 my preferences (torrents) 10 taste buddies + 10 preferences per taste buddy 10 random peers Megacache: peers retain context (to replace search by epidemic information dissemination) Buddycast: every peer sends one buddycast message every 15 seconds pick a buddy or a random peer with some probability as the destination both communicating peers merge their buddy lists based on the information exchanged

Gossiping 1 – Buddycast: performance Mortality in VCs: How many buddies recorded in a buddycast message are still online when the message is received? measurement period: 520 hours number of messages: 5049 buddycast messages number of number of peers still alive per buddycast message

Gossiping 2 – swarm discovery: in Bittorrent There is a separate swarm for every file that is being downloaded: all peers downloading that file These swarms are centrally managed: a peer indicates its interest in a file to a tracker peers periodically contact a tracker to obtain the IP numbers of other peers downloading the same file a peer selects the best other peers as bartering partners swarm tracker bartering

Gossiping 2 – swarm discovery: in Tribler In Tribler we define a single overlay swarm that contains all peers The overlay swarm is used for decentralized peer and content discovery A peer, on install, contacts a bootstrappeer: to become members of the overlay swarm to get a set of initial contacts bootstrappeer overlay swarm swarms

Gossiping 2 – swarm discovery: Little Bird Peers maintain a swarm database in which they cache information on the swarms of which they have been a member (over the last 10 days) Two message types: GetPeers: request for peers in the swarm (contains swarm id and known peers in the swarm; check before you tell) PeerList: reply with a list of peers in the swarm (represented with a Bloom filter) Phase 1: Bootstrapping (find initial peers): direct GetPeers at peers with the same interests as derived from buddycast exchanges Phase 2: Find additional peers in the swarm Peer selection for GetPeers based on contributions of peers in the past (connectivity, activity) work by Jelle Roozenburg

Gossiping 2 – Little Bird: Swarm Coverage fraction coverage Swarm database effective number of hours online Evaluation with emulations

Gossiping 3 – social P2P networks: overview Known mechanisms: GMail MSN Messenger … PermIDs: spreading storing searching Mapping PermIDs onto IP addresses work by Steven Koolen

Gossiping 3 – social P2P networks: statistics friendster.com friends-of-a-friend probability friends probability Average number of friends: 243 friends-of-a-f: 9147 number of friends/friends-of-a-friend

Gossiping 3 – social P2P networks: message types Two message types (SET and GET) to exchange PermID-IP address information Only exchanges two hops away (friends and friends-of-friends) Results in a distance of 4

Gossiping 3 – social networks: IP dynamics (1) percentage of peers with number of IP addresses number of different IP address 1% of the peers has been seen with more than 4 IP addresses Conclusion: IP addresses of peers are not very dynamic

Gossiping 3 – social networks: IP dynamics (2) time between IP changes (s) in Tribler peers sorted by number of changes Conclusion: inter-IP-change time on the order of 3-300 hours

Gossiping 3 – social networks: peers online?? fraction of the time online in Tribler peers sorted by fraction online Conclusion: Unavailability of peers is high Peers are unconnectable because of NAT and firewalls (+/- 41% in a BitTorrent community, not shown)

Cooperative downloads: basic idea Problem: most users have asymmetric upload/download links because of the tit-for-tat mechanism of Bittorrent, this restricts the download speed Solution: let your friends help you for free bartering equal upload download friend for free = 1/2 1024 Kbps 256 Kbps peer contributions from friends bartering work by Pawel Garbacki and Alex Iosup

Collaborative downloads: another view Collaboration established between collector and helpers Collector aims at obtaining a complete copy of the file Helpers download distinct chunks and send them to the collector, not requesting any other chunk in return

Future Gossiping Research in I-Share/Tribler Thorough analysis of Buddycast, Little Bird, and NN: what is the connectivity among peers? how fast is new information propagated? what parameters should be used for deciding on: peer selection for gossiping frequency of gossiping which and how much information to gossip There are more opportunities for gossiping Let gossiping research be driven be real, specific applications Design real systems, deploy them in a real environment, and then analyze them

Contributors TU Delft-EEMCS-ICT Inald Lagendijk Marcel Reinders Jacco Taal Jun Wang Maarten Clements TU Delft-EEMCS-PDS Johan Pouwelse Henk Sips Pawel Garbacki Alexandru Iosup Jan David Mol Jie Yang Maarten ten Brinke Freek Zindel Jelle Roozenburg Steven Koolen TU-Delft-ID Jenneke Fokker Huib de Ridder Piet Westendorp More information: www.cs.vu.nl/ishare www.tribler.org dev.tribler.org www.ewi.pds.tudelft.nl (publication database) VU Maarten van Steen Arno Bakker