1. 1 Scalable Peer-to-Peer Networked Virtual Environment Master Thesis Oral Examination Dept. of CSIE, Tamkang Univ. Advisor: Dr. Chen Jui-Fa Shun-Yun Hu 2005/01/07

2. 2 Outline Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

3. 3 What is Networked Virtual Environment (NVE)? Virtual Reality + Internet 3D environment with people (avatar), objects, terrain, agents Military simulations (’80)  Massively Multiplayer Online Games (mid-‘90) Trends: larger scale, more realistic simulation

4. 4

5. 5 NVE: A Shared Space

6. 6 Issues for Creating NVE Consistency (events/states) Responsiveness multiplayer Security Scalability Persistency massively multiplayer Reliability (Fault-tolerance)

7. 7 The Scalability Problem Many nodes on a 2D plane ( > 1,000) Message exchange with those within Area of Interest (AOI) How does each node receive the relevant messages? Area of Interest

8. 8 A simple solution (point-to-point) N * (N-1) connections ≈ O(N 2 )  Not scalable! Source: [Funkhouser95]

9. 9 A better solution (client-server) Message filtering at server to reduce traffic N connections = O(N)  server is bottleneck Source: [Funkhouser95]

10. 10 Current solution (server-cluster) Still limited by servers. Expansive to deploy & maintain. Source: [Funkhouser95]

11. 11 Scalability Analysis Scalability constrains Computing resource(CPU) Network resource(Bandwidth) Non-scalable system vs. Scalable system x: number of entities y: resource consumption at the limiting system component Resource limit

12. 12 What Next? Strategies Increase resource  More servers Decrease consumption  Message filtering ArchitecturesScale Point-to-point (LAN)tens10^1 Client-serverhundreds10^2 Server-clusterthousands10^3 ?millions 10^6 … Peer-to-Peer

13. 13 What is Peer-to-Peer (P2P)? [Stoica et al. 2003] Distributed systems without any centralized control or hierarchical organization Runs software with equivalent functionality Examples File-sharing: Napster, Gnutella, eDonkey Distributed computing: SETI@Home (UC Berkeley) VoIP:Skype

14. 14 Peer-to-Peer Overlay A P2P overlay network source: [Keller & Simon 2003]

15. 15 Promise & Challenge of P2P Promises Growing resource, decentralized  Scalable Commodity hardware  Affordable Challenges Topology maintenance  dynamic join/leave Efficient content retrieval  no global knowledge

16. 16 Issues for Creating P2P NVE Consistency (events/states) Responsiveness multiplayer Security Scalability Persistency massively multiplayer Reliability (Fault-tolerance) Consistency (topology)  P2P NVE

17. 17 Related Works (1): SimMUD [Knutsson et al. 2004] (Univ. of Pennsylvania ) Pastry + Scribe Regions Coordinators (super-nodes) Fixed-size region Relay overhead

18. 18 Related Works (2) [Kawahara et al. 2004] (Univ. of Tokyo) Fully-distributed Nearest-neighbors List exchange High transmission Overlay partition

19. 19 Related Works (3): Solipsis [Keller & Simon 2003] (France Telecomm R&D) Links with AOI neighbor Mutual cooperation Inside convex hull Potentially slow discovery Inconsistent topology

20. 20 Outline Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

21. 21 Design Goals Observation: for virtual environment applications, the contents we want are messages from AOI neighbors Content discovery is a neighbor discovery problem Solve the Neighbor Discovery Problem in a fully- distributed, message-efficient manner. Specific goals: Scalable  Limit & minimize message traffics Responsive  Direct connection with AOI neighbors

22. 22 Voronoi Diagram 2D Plane partitioned into regions by sites, each region contains all the points closest to its site Can be used to find k-nearest neighbor easily Neighbors Site Region

23. 23 Design Concepts Identify enclosing and boundary neighbors Each node constructs a Voronoi of its neighbors Enclosing neighbors are minimally maintained Mutual collaboration in neighbor discovery CircleArea of Interest (AOI) Whiteself Yellowenclosing neighbor (E.N.) L. Blueboundary neighbor (B.N.) PinkE.N. & B.N. GreenAOI neighbor D. Blueunknown neighbor Use Voronoi to solve the neighbor discovery problem

24. 24 Procedure (JOIN) 1)Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2)Acceptor sends back its own neighbor list joining node connects with other nodes on the list Acceptor’s region Joining node

25. 25 Procedure (MOVE) 1)Positions sent to all neighbors, mark messages to B.N. B.N. checks for overlaps between mover’s AOI and its E.N. 2)Connect to new nodes upon notification by B.N. Disconnect any non-overlapped neighbor Boundary neighbors New neighbors Non-overlapped neighbors

26. 26 Procedure (LEAVE) 1)Simply disconnect 2)Others then update their Voronoi new B.N. is discovered via existing B.N. Leaving node (also a B.N.) New boundary neighbor

27. 27 Dynamic AOI Crowding within AOI can overload a particular node It’s better if AOI-radius can be adjusted in real time

28. 28 Adjustment Conditions AOI-radius decrease Number of connections > maximum allowable connections AOI-radius increase Maximum connections not exceeded Current AOI-radius < preferred AOI-radius Delay counter To avoid fluctuations

29. 29 Demonstration Simulation video General movements (20 nodes, 800x600 world) Local vs. global view Dynamic AOI adjustment

31. 31 Outline Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

32. 32 Simulation Method C++ implementation of Voronoi-based algorithm World size: 1000 x 1000, AOI: 150 Trials from 10 – 250 nodes Connection limit per node: 10 1000 time-steps (~ 100 simulated seconds, assuming 10 updates/seconds) Behavior model Random movement:random direction Constant velocity: 5 units/step Movement duration: random (1 – 25 steps)

33. 33 Consistency Metrics Topology Consistency [Kawahara, 2004] Number of observed AOI neighbors Number of actual AOI neighbors Drift Distance [Diot, 1999] Distance between observed position and actual position (average over all nodes)

34. 34 Basic Model Topology Consistency

35. 35 Basic Model Scalability (1)

36. 36 Basic Model Scalability (2)

37. 37 Dynamic AOI Model

38. 38 Dynamic AOI Scalability (1)

39. 39 Dynamic AOI Scalability (2)

40. 40 Dynamic AOI Scalability (3)

41. 41 Dynamic AOI Topology Consistency (1)

42. 42 Dynamic AOI Topology Consistency (2)

43. 43 Dynamic AOI Reliability (1)

44. 44 Dynamic AOI Reliability (2)

45. 45 Outline Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

46. 46 Analysis of Design Consistency (Topology) Topology is fully connected & consistent  enclosing neighbors Responsiveness Lowest latency  direct connection, no relay Scalability Resource-growing & decentralized resource consumption Reliability Self-organizing for small number of node failures

47. 47 P2P NVE Comparisons SimMUDNeighbor-list exchange SolipsisVON Consistency (topology) SupernodeNeighbor list- exchange (partitioning) Neighbor notify&query (undiscovery) Neighbor notify (consistent) Responsive- ness High overhead Medium overhead Low overhead ScalabilityRelied on supernode Fully- distributed ReliabilityLong up- time N/A Self- organizing

48. 48 Problems of Voronoi Approach Message traffic Circular round-up of nodes Redundant message sending (inherent to fully-distributed design) Incomplete neighbor discovery Can happen with inconsistent / incorrect neighbor list Fast moving node

49. 49 Outline Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

50. 50 Conclusion NVE scalability is achievable with P2P architecture and is a neighbor discovery problem A promising solution: Voronoi-based P2P Overlay Leverage knowledge of each peer to maintain topology Properties Scalable: fully-distributed, dynamic AOI Efficient: low irrelevant messages, zero relay Robust: consistent and self-organizing topology

51. 51 Potential Applications Online games Relieve server from position updates in current MMOGs Military Enable large-scale, affordable military training simulation 3D Web Provide multi-user interactivity to static 3D world Scientific simulations Distribute spatial simulation requiring frequent synchronization

52. 52 Future Works Short-term Reliability measurements  latency, packet loss, node fail Distributed event/state consistency Recovery from overlay partition Long-term Persistency issue(P2P-based database) Security issue (protection from malicious nodes) 3D content distribution (3D streaming on P2P) Massive, persistent 3D environment sharable by all!

53. 53 Acknowledgements Dr. Jui-Fa Chen ( 陳瑞發老師 ) Dr. Wei-Chuan Lin ( 林偉川老師 ) Members of the Alpha Lab, TKU CS Guan-Ming Liao( 廖冠名 ) Dr. Chin-Kun Hu( 胡進錕老師 ) LSCP, Institute of Physics, Academia Sinica Joaquin Keller(France Telecomm R&D, Solipsis) Bart Whitebook(butterfly.net) Jon Watte(there.com) Dr. Wen-Bing Horng( 洪文斌老師 ) Dr. Jiung-yao Huang( 黃俊堯老師 )

54. 54 Inconsistency caused by dAOI

55. 55 Reliability (0-500 steps)

56. 56 Reliability (501-1000 steps)