1. Comprehensive View of a Live Network Coding P2P System Presented by: Kenny, W. K. Cheung C. M. Fung

2. Outline of presentation 1.Introduction to the paper 2.What is Network Coding? 3.Prototype implementation details 4.Paper Result 5.Topology Construction 6.Connectivity 7.Security 8.Conclusion

3. Paper Objectives Present the first implementation of a P2P content distribution system that uses Network Coding Show that Network Coding is practical in a P2P setting (little CPU processing, I/O activity)

4. Paper Achievement Present the performance of a live P2P filecasting system from a novel set of angles. –E.g. quantity content provider’s saving overtime, the dynamics of topology, the number of unreachable nodes at any point the time, the overall system efficiency Present workers’ field experiences with implementing and using Network Coding. Show that Network Coding is feasible. Evaluated various P2P topology construction algorithms. Study the influence of unreachable nodes in the system’s efficiency. Study the performance of a novel set of security functions to secure Network Coding system. –Secure Random Checksums

5. Network Coding A method of attaining maximum information flow in a network. Core notion: –Mixing of data at intermediate network nodes. –Receiver deduces from data packets the messages that were originally intended for it.

6. For each of the k clients, server encode the packets follow the relation M i are packets to be sent originally G k, X k are sent each encoded message is a linear combination of the file Linear problem: X = GM When enough encoded messages are received, client can find M by using Gaussian elimination A high level explanation on Network Coding

7. A high level explanation on Network Coding example (2 packets) Server N1 N2 a 1 x 1 +b 1 x 2 =m 1 a 2 x 1 +b 2 x 2 =m 2 a 1 x 1 +b 1 x 2 =m 1 a 2 x 1 +b 2 x 2 =m 2 2 packets: x 1, x 2 Server sent: (a 1,b 1 ) and m 1 to N1 (a 2,b 2 ) and m 2 to N2

8. Network Coding example (attaining maximum information flow) Two sources Two destination nodes Each edge can carry only a single value. No routing scheme can transmit both A and B simultaneously to both destinations.

9. Network Coding in this paper Use of Network Coding for P2P networks Benefits: 1.The capacity of the seed server is fully utilized by constantly serving innovative information 2.Innovative information in a node propagates in optimal time to those nodes needing it, regardless the number of hops between the source and the sink

10. Prototype Implementation Implemented in C# Three type of participant: –One or more peers –A registrar –A logger

11. Prototype Implementation Peers: –Sources and sinks for content data –Server: a special peer that contains seeded content –Seeds: peers that finish the download, but remain in the system –Each peer maintains 4-8 connections to other peers. ( one for each direction) –Encodes, decodes, validates, and persists content data.

12. Prototype Implementation Registrar: enables peer discovery Logger: an aggregation point for peer and registrar trace messages

13. Evaluating Swarming Systems Evaluating swarming systems under the right context is critical Key issues: –Swarm efficiency with scale –Partial observation are not enough –Impact of arrival patterns

14. Data Summary Server upload capacity: 2.5Mbps, also served as registrar and logger.

15. Individual Peers Trial participants were diverse in terms of geographical location, access capacity and access type (e.g. corporate links, DSL/cable home users, wireless links)

16. Individual Peers Summary of participating users (Trial-1) Observation: Most users had download speed between 550 Kbps and 1.6Mbps non-NATed node get a much higher throughput than the NATed one.

17. Individual Peers Number of upload and download neighbors for each peer in Trial 1 Neighbors: 6 to 8 for downloads 4 to 6 for uploads

18. Individual Peers Observation: Most peers had multiple sessions (some up to 14) Number of upload and download neighbors for each peer in Trial 1.

19. System Rates Overall system throughput = the aggregate download rate.

20. System Rates The download efficiency of a user is the ratio of its average download rate over the maximum, ideally =1 Observation: The efficiency is similar for all groups, implying that nodes used the available resources efficiently even during the early stages of the trial.

21. Content Provider Savings The Content Provider Savings are proportional to the ratio of the aggregate download rate over the upload rate contributed by the server;

22. fair share ratio: divide server’s upload rate by the rate that the server would have to contribute if all nodes were uploading to their maximum capacity Content Provider Savings

23. Peer’s Performance Study the performance of a typical peer 2.2Mbps downlink, 300Kbps uplink capacity

25. Time on each activity (for 5 random peers and average)

26. Resource Consumption One typical peer Pentium IV @2GHz and 512MB RAM CPU usage and disk usage plot using interactive applications –e.g. word editing and WWW browsing –no observable decrease in responsiveness

27. Resource consumption on a typical machine during Trial-4. Left: CPU Activity. Right: Disk Activity.

28. Server Efficiency Network coding ensures optimal resource utilization since every block generated by the server can be used (with high probability) in place of any other block to reconstruct the original. encoding entire file impractical due to high encoding and decoding costs.

29. Server Efficiency Solution: Group coding –divide the blocks of the file into groups and we perform coding only inside the groups. New problem arise: –some segments may become unpopular (like the optimal scheduling of the block propagation) Writer’s argument: –segment scheduling is a much easier problem than block scheduling and penalty of group coding is small

30. Server Efficiency server efficiency: –the number of useful (unique) blocks that are served by the server in time sufficient to serve one full copy of the file (under optimal scheduling). network coding (entire file) : 1.0 ideal block propagation scheme : 1.0

31. Server Efficiency 5 runs per instance For group coding, scheduling decisions across different segments is random

32. Server Efficiency Group coding v.s. Local rarest heuristic –Group coding performance do not varies on scale –Local rarest performance depends on number of neighbors (estimation of the state of the network)

33. Download Progress current peer-to-peer systems have slow performance in: –Beginning (do not have anything to offer (or whatever they have is already popular in the system) –toward the end (cannot find the last missing blocks) –U-shape in performance

34. Download Progress Network coding don’t have this problem each encoded block is unique and useful to any node

35. Topology Construction Trial 1,2 : –Each new node picks a random subset of nodes out of those in the system (registered at the tracker) and connect to them. –The set of uploading nodes could be different than the set of downloading nodes. –The maximum number of download connections > The maximum number of upload connections.

36. Topology Construction Second eigenvalue : A standard metric measures global connectivity properties. Topologies with good connectivity have second eigenvalue bounded away from 1.

37. Topology Construction Trial 3, 4: –The topology construction algorithm periodically dropped one of its peer at random –Consideration: dropping frequency (drop one peer after five block are uploaded), selection policies for dropping (drop slow peers) –Motivation: to force rewiring of connections, hence, construct overlay topologies with better properties.

38. Topology Construction Problem of Trial 3: A large number of Too Busy (cluster) (Dropped peers try to contact other peers in other parts of the network, get continuously rejected since other nodes have their connection budget used) Trial 4: to avoid Too Busy problem, allowed a given peer’s peer-set to grow temporarily by some threshold (e.g. max-peer-set +2), creating an “elastic” maximum.

39. Connectivity Un-reachable peers: –Peers behind NATs and firewalls, cannot receive inbound connections. (Exclude peers behind NATs and firewalls configured to allow incoming connections.) –Can’t exchange content with each other, and, hence, can’t take advantage of the network capacity that exists between them.

40. Connectivity Calculate the optimal throughput of the system assuming all nodes are reachable: Calculate the optimal throughput of the system taking into consideration the set of unreachable peers: The optimal throughput at time t is computed as the sum of the peak upload rate of all active peers at time t.

41. Connectivity The effect of unreachable peers on the overall performance for two trials (left: Trial-1, right: Trial-4). The top curves are computed assuming all nodes are reachable; the lower curves take into consideration the set of unreachable peers over time.

42. SECURITY network coding each intermediate node produces “new” blocks Server can’t digitally sign each block Secure Random Checksums (SRCs)

43. For each of the clients, server generates: vector r = [r 1... r m ] in Z q –often q = 16 digits –m : number of symbols per block (=bytes/block) –random elements SRC of block i : Specific SRC and the seed of the random element vector is sent to the client through secure channel

44. Secure Random Checksums (SRCs) SRC of block i :encoded block: The above equalities should hold if the encode packet received is valid. So,the client calculates the both sides of the above equations and check if it holds to verify the arriving packet.

45. Performance of SRCs Fast : SRCs are linear operations On 3.0 GHz Pentium 4 with 1GB of RAM 2 GB file: 1 sec/SRC –Done before download commerce –~ time for once disk read of file Client SRC verification : ~1.6 Gbps,

46. Conclusion network coding overhead is relatively small, both in terms of CPU processing and I/O activity. verification of encoded blocks using SRCs is very efficient. high utilization of the system resources and large savings for the content provider even during flash-crowd events smooth file download progress Replace fancy block selection algorithms

47. Our point of view Network coding in p2p is very applicable from the experimental results. With further researches, it can be even more powerful: peer-matching algorithms provide much further improvement space On-the-fly decoding or parallelisms reduce overhead on decoding

48. Q & A

49. Thank you !