An Analysis of the P2P Traffic Characteristics on File Transfers Between Prefectures and Between Autonomous Systems in the Winny Network Nov. 1, Takashi Koizumi, Masahiro Yoshida, Satoshi Ohzahata, Konosuke Kawashima Tokyo University of Agriculture and Technology, The University of Tokyo, The University of Electro-Communications, APCC2010 1) 3)3) 2)2) 3)3) 2)2)

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results Analysis between prefectures Analysis between ASs Conclusion Nov. 1, 20102APCC2010

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results Analysis between prefectures Analysis between ASs Conclusion Nov. 1, 20103APCC2010

Introduction P2P file sharing network generates huge traffic. it has an enormous impact to backbone network. however, the technology of P2P is useful. VOD, Video streaming, etc... Analyzing P2P traffic is important. Problems: many ISPs do not make public report on their traffic data. difficulty of identifying file transfers from where did peer download files ? Nov. 1, 20104APCC2010 P2P network

Introduction Analysis methodology identify the file transfers between peers in Winny network. Analyze the characteristics of the traffic matrix. focus on between ASs and between Japanese Prefectures. Winny network one of the most popular P2P file sharing applications in Japan. it generates 20% of traffic in one ISP in Reveal the factor of increasing traffic from P2P file-sharing. Nov. 1, 20105APCC2010

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 20106APCC2010

File distribution protocol in Winny Nov. 1, Feature of Winny network - unstructured P2P network. - layered structure. - diffuse metadata of files (KEY) - 2 types of query. Feature of Winny network - unstructured P2P network. - layered structure. - diffuse metadata of files (KEY) - 2 types of query. APCC2010 KEY information file name file location(IP address, service port) file hash value lifetime of KEY (TTL) etc... (1)File uploading peer diffuses KEYs to other peers. (2)File searching peer generates search queries to find KEYs. (3)Download the file from the location of file uploading peer included in the KEY. File uploading peer File searching peer

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 20108APCC2010

Analyze time series of file KEY. focus on TTL and the file location in the KEY < TTL < file location is same with the KEY having peer. Peer A Definition of the file uploading peers Nov. 1, Crawler search query query hit APCC2010 Peer A is file uploading peer : File : File KEY 1500 < TTL < 2000 file location is Peer A. TTL should be smaller than 2000 seconds to exclude spam KEYs. TTL of the KEY in file uploading peer is set to be greater than 1500 seconds.

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, APCC2010

Definition of the file transfer between peers process of the file transfer. changes of a KEY having peer to a file uploading peer. Nov. 1, 2010APCC Peer APeer B Peer A is the file uploading peer Peer B is the KEY having peer Diffusion of KEY Peer B becomes a file uploading peer Peer B file transfer to Peer B from Peer A : File : File KEY Download demand Peer A with continuous crawling

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, APCC2010

Measurement environment Winny Crawling System collects KEYs. crawl more than 100,000 peers within 10 minutes. Measurement period Aug. 4, :00 ~(12 hours) Nov. 1, 2010APCC crawlers DB server Management server

Measurement environment Types of measured file. 15 popular files (selected in 1 hour crawling). books, movies and compressed files. average file size is 245MB Nov. 1, 2010APCC crawlers DB server Management server

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, APCC2010

Measurement results Analysis of the file transfer characteristics focus on between prefectures and between ASs. Origin-Destination file transfer matrix. measurement summary about 220,000 peers, 280 ASs. about 30,000 instances of the file transfers between 33,000 peers Nov. 1, results are for 47 prefectures and top 50 ASs with number of file transfers about 15 popular files. APCC2010

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, APCC2010

File transfer traffic matrix between prefectures src prefectures on Y-axis, dst prefectures on X-axis large portion of file transfers are generated in few prefectures. Nov. 1, APCC2010 Destination Prefectures Source Prefectures Tokyo(dst)Osaka(dst) Tokyo(src) Osaka(src) % highlow

File transfer traffic matrix between prefectures the sum of elements is 100% for each row. similar traffic distribution in every prefectures. Nov. 1, Destination Prefectures APCC2010 Source Prefectures within-prefectures highlow src and dst is same prefecture

Comparison of Winny and telephone traffic upper: Winny below: telephone traffic (in 2007) traffic within-prefectures Winny: 2% telephone traffic: 70% similar prefectures are selected as destinations. Tokyo, Osaka, etc. Nov. 1, 2010APCC TokyoOsaka

Comparison of Winny and Internet upper: Winny below: Internet (in 2008)* traffic within-prefectures. Winny: 2% Internet: 3% few locality of traffic. distribution patterns are similar. Nov. 1, * Kenjiro Cho, Kensuke Fukuda, Hiroshi Esaki, Akira Kato, Jun Murai, Where does all the traffic go? – observing trends in Japanese residential traffic –, International Workshop on Protocols for Future, Large-Scale and Diverse Network Transports, APCC2010

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, APCC2010

File transfer traffic matrix between ASs src ASs on Y-axis, dst ASs on X-axis large portion of file transfers is generated by few ASs. 20 ASs generated 85% of total number of file transfers. Nov. 1, Source ASs Destination ASs APCC2010 % highlow

File transfer traffic matrix between ASs the sum of elements is 100% for each row. traffic distribution is similar in each AS. very poor locality of AS network in Winny. Nov. 1, Source ASs Destination ASs APCC2010 within-AS highlow

Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, APCC2010

Conclusion Analyzed traffic characteristics in Winny network. many file transfers are generated by few prefectures or ASs. since there is no mechanism to consider IP network, many file transfers are not within-prefecture or within-ASs. Future work long term measurement. analysis and measurement of other P2P networks (BitTorrent, Share, etc.). Nov. 1, APCC2010

THANK YOU VERY MUCH FOR YOUR KIND ATTENTION. Nov. 1, 2010APCC201027

Percentage of file transfer in prefectures Nov. 1, top 10 prefectures accounted for 70% of the number of transfers. there were few file transfers within prefectures. APCC2010 Prefecture

Analysis of file transfer behavior of peers 3 cases of file transfers. Nov. 1, 2010APCC From within the same prefecture (files within-prefecture) From a different prefecture (files did not exist in destination prefecture) From a different prefecture (although files did exist in destination prefecture) Prefecture A Prefecture B Prefecture A Prefecture BPrefecture A

Analysis of file transfer behavior of peers Nov. 1, % of file transfers are from different prefectures although files did exist in same prefecture. it leads to increase the traffic between prefecture. APCC2010 the sum of inbound file transfer is 100% in each prefectures.

Percentage of file transfer in ASs Nov. 1, within-AS file transfers: 9%. 85 % of file transfers are generated by top 20 AS. result shows only top 50 ASs with number of file transfer. APCC2010 AS

Percentage of file transfer in ASs Nov. 1, sorted with times of file transfer result shows only top 50 ASs with number of file transfer. APCC2010 AS

Analysis of file transfer behavior of peers Nov. 1, the sum of inbound file transfer is 100% in each ASs. even if the file exists in same AS, most file transfers are from different ASs. it leads to increase of the AS traffic. APCC2010

Discussion Problem Small local region ASs communicate with large ASs. network latency and increasing traffic Considering the strategies both caching and peering between ASs. Choosing the optimal location of cache servers Building local region IXes. Nov. 1, 2010APCC201034

Discussion Problem in Winny network. peer selecting algorithm difficult for unstructured P2P networks. no central management servers. caching mechanism contribute file distribution in a Winny network. cannot effectively reduce the traffic. optimal caching and neighbor selection is important. geographical cost and IP network topology network characteristics of peers and files Nov. 1, 2010APCC201035