FLoD: A Framework for Peer-to-Peer 3D Streaming IEEE INFOCOM 2008 Shun-Yun Hu*, Ting-Hao Huang, Shao-Chen Chang*, Wei-Lun Sung*, Jehn-Ruey Jiang*, and Bing-Yu Chen National Central University*, National Taiwan University Taiwan, R.O.C.

Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

4

5 Adaptive Computing and Networking Lab, CSIE, NCU

6

NCU ACNLab

Massively Multiplayer Online Games MMOGs are growing quickly  Multi-billion dollar industry  10 million subscribers for World of Warcraft  600,000 concurrent users

NASA World Wind

Google Earth

Zoom in…

NCU ACNLab To HsinChu..

and NTHU..

It is going to be 3D

NTHU 3D Student Center…

To the ground

3-Dimensional Virtual Tourism Google Earth Virtual Earth NASA World Wind X3D Earth NCU ACNLab

DARPA SIMNET: screenshot From Bruce Sterling's "War is Virtual Hell,“ (1993).

CALVIN: a distributed collaborative virtual environments for architectural layout designs (1996)

20 NVE A networked virtual environment (NVE) is a computer-generated virtual world where multiple geographically dispersed users can assume virtual representatives (or avatars) to concurrently interact with each other in real time through networked devices. Also called a distributed virtual environment (DVE)

21 NVE Examples of NVEs include  early DARPA SIMNET and DIS systems  early distributed collaborative virtual environments  currently booming Massively Multiplayer Online Games (MMOGs)  future multi-user 3DVTs  future multi-user Web 3D browser

Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion NCU ACNLab

Motivation Two future trends for NVEs  More and more users  More worlds with larger and more dynamic contents

NCU ACNLab Motivation Q: How to support millions of concurrent users? A: Utilizing peer-to-peer schemes to relief the load of servers

NCU ACNLab Motivation Q: How to support large and dynamic worlds? A: 3D streaming is needed

Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion NCU ACNLab

27 Model for NVEs Many nodes on a 2D plane An avatar needs to know only those within Area of Interest (AOI)‏ ★ : self ▲ : neighbors Area of Interest (AOI)

How does each node receive the relevant messages? Completely connected point-to-point Client/Server Client/Server cluster Partially connected point-to-point (peer-to-peer) NCU ACNLab

29 A simple solution (point-to-point)‏ But…too many irrelevant messages N * (N-1) connections ≈ O(N 2 )  Not scalable! Source: [Funkhouser95]

30 A better solution (client-server)‏ Message filtering at server to reduce traffic N connections = O(N)  server is bottleneck Source: [Funkhouser95]

31 Current solution (server-cluster)‏ Still limited by servers. Expensive to deploy & maintain. Source: [Funkhouser95]

The Problem Client-server: resources limited by provisioning Resource limit [Funkhouser95]

The Solution Peer-to-Peer: resources grow with demand Resource limit [Keller & Simon 2003]

34 P2P NVE Neighbor Discovery We need to solve the neighbor discovery problem in a fully-distributed, message- efficient manner with specific goals:  Consistent  Good neighborship consistency  Scalable  Limit & minimize message traffics  Responsive  Direct connection with AOI neighbors

35 Neighborship Consistency (1) Definition # of current AOI neighbors observed # of current AOI neighbors

36 :is observed neighbor Neighborship Consistency (2) An example Neighborship Consistency = 4 / 5 = 80% :is actual neighbor

Voronoi-based Overlay Network : VON Use Voronoi diagram to solve the neighbor discovery problem  Each node constructs a Voronoi diagram of its neighbors  Identify enclosing and boundary neighbors  Mutual collaboration in neighbor discovery

38 Voronoi Diagram 2D Plane partitioned into regions by sites, each region contains all the points closest to its site Site Region

Voronoi-based Overlay Network : VON ● node i and the big circle is its AOI ■ enclosing neighbors ▲ boundary neighbors ★ both enclosing and boundary neighbors ▼ normal AOI neighbors ◆ irrelevant nodes

40 Procedure (JOIN) 1)Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2)Acceptor sends back its own neighbor list Joining node connects with other nodes on the list Acceptor’s region Joining node

41 Procedure (MOVE) 1)Positions sent to all neighbors, mark messages to B.N. B.N. checks for overlaps between mover’s AOI and its E.N. 2)Connect to new nodes upon notification by B.N. Boundary neighbors New neighbors

42 Procedure (LEAVE) 1)Simply disconnect 2)Others then update their Voronoi diagram new B.N. is discovered via existing B.N. Leaving node (also a B.N.) New boundary neighbor

Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

What is 3D streaming? Continuous and real-time delivery of 3D contents over network connections to allow user interactions without a full download. Contents are fragmented, transmitted, reconstructed, and then displayed.

3D streaming vs. media streaming Video / audio media streaming is very matured User access patterns are different for 3D content  Highly interactive  Latency-sensitive  Behaviour-dependent  Non-sequential

4 types of 3D streaming Object streaming Scene streaming Visualization streaming Image-based streaming

47/ Object streaming Hoppe 1996 Progressive Meshes

48/ Scene streaming Many objects Object selections & transmissions Teler &Lischinski 2001

P2P-based 3D Streaming Models & assumptions  Many 3D objects(position, orientation)  User navigations withAOI visibility  Objects are fragmented (base & refinement pieces)  Data are initially stored at server

50/ NCU ACNLab Observation Limited & predictable area of interest (AOI)‏ Overlapped visibility = shared content

FLoD: Flowing Level-of-Details Assume P2P-NVE overlay, such as VON Download 3D content from AOI neighbors Basic design  Each object has a unique ID and associated progressive mesh data  Scene description records object ID, orientation, and scale  World is partitioned into cells

IEEE INFOCOM 2008

Steps of FLoD 1. Peers exchange incremental formation of cached 3D data periodically 2. A peer sends requests for downloading scene descriptions 3. A peer sends requests for downloading object data 4. A peer asks the server if none of the peers responds

Object Prioritization Visual importance

Peer selection Multi-level request

Prototype experiment Progressive modeling and rendering of the scene (by NTU) P2P neighbor discovery and 3D streaming (by NCU)

Partition Cell-based construction Use an actual game scene 100x game scene (514KB -> 51.8MB)

Fragmentation

Simulation setup Environment  1000x1000 world, 100ms / step, 3000 steps  client: 1 Mbps / 256 Kbps, server: 10 Mbps (both)‏ Objects  Random object placement (500 objects)‏ User behavior  Random way-point  clustering movement (1.5 * ln(n) hotspots)‏

Metrics Server's perspective  Requests can be redirected (bandwidth usage) User's perspective  Visual quality (fill ratio)  Interactivity (base latency)

Server bandwidth usage

Client bandwidth usage

Fill ratio

Base latency

Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

NVE neighbor discovery can be solved in a P2P manner with  Good neighborship consistency  Low message overhead NVE 3D streaming can be solved in a P2P manner with  Minimal server resource usage  Low control message overhead  Short Latency

Conclusion Future work  Multi-user X3D browser  Multi-user X3D browser with P2P 3D streaming  TOPOS (Greek; common place): Authenticable P2P personal 3D space publication and navigation system  H E AVEN (a Hybrid Architecture for massively multi-user Virtual ENvironments): client assisted MMOGs

Adaptive Computing and Networking Lab. Q & A Thank you!

69 Related Work (1): DHT-based: SimMUD [Knutsson et al. 2004] (UPenn) Pastry (DHT mapping) + Scribe (Multicast) Fixed-Sized Regions Coordinators

70 PASTRY (P2P overlay) SCRIBE (Multicast support) MMOG GAME Related Work (1): DHT-based: SimMUD

71 Related Work (1): DHT-based: SimMUD

72 Related Work (2): Neighbor-list Exchange [Kawahara et al. 2004] (Univ. of Tokyo) Fully-distributed Nearest-neighbors List exchange High transmission Overlay partition

73 Related Work (3): Mutual Notification: Solipsis [Keller & Simon 2003] (France Telecomm R&D) Links with AOI neighbor Mutual cooperation Inside convex hull Potentially slow discovery Inconsistent topology

74/ Visualization streaming Large volume Time-varying Dedicated servers Olbrich & Pralle 1999

75/ Image-based streaming Server- rendered Thin clients Less responsive Cohen-Or et. al. 2002