1. MV-4474 Virtual Environment Network & Software Architectures Michael Zyda zyda@movesinstitute.org

2. Networked Virtual Environments - Design & Implementation Sandeep Singhal & Michael Zyda "Networked Virtual Environments - Design and Implementation," ACM Press Books, SIGGRAPH Series, July 1999, ISBN 0-201-32557-8.

3. MV-4474 Virtual Environment Network & Software Architectures MV-4474 Virtual Environment Network and Software Architectures (3-2) This course covers the design and implementation of network and software architectures for real-time, interactive 3D virtual environments (VEs). Network architecture topics include a taxonomy for networked virtual environments, distributed interactive simulation protocols (DIS and HLA), virtual reality modeling language (VRML), agent-based network protocols (Java/Telescript), proposed solutions for large-scale networked virtual environments (area of interest managers and object brokers), multicast backbone tools and developments, and virtual reality transfer protocol proposals. Software architecture topics include representative software architectures for VEs (NPSNET, DIVE, MASSIVE,...), commercial toolkits for VE development (WorldToolKit, Division's dvs, Performer,...), lag in multiprocessor virtual environments, and the HCI implications on VE network and software architectures. PREREQUISITE: MV-4473 or the consent of the instructor.

4. Course Logistics The lectures in this course follow the Singhal/Zyda text from chapter 1 through chapter 9. Each student is expected to develop a programming project showing their knowledge of networked virtual environments. That project must be approved by the instructor. An in-class presentation on that project is due near the end of the quarter. Students are required to turn in Powerpoint versions of their presentation slides. A Powerpoint template provided by the instructor must be used.An in-class presentation on that project is due near the end of the quarter. Students are required to turn in Powerpoint versions of their presentation slides. A Powerpoint template provided by the instructor must be used. The lectures in this course follow the Singhal/Zyda text from chapter 1 through chapter 9. Each student is expected to develop a programming project showing their knowledge of networked virtual environments. That project must be approved by the instructor. An in-class presentation on that project is due near the end of the quarter. Students are required to turn in Powerpoint versions of their presentation slides. A Powerpoint template provided by the instructor must be used.An in-class presentation on that project is due near the end of the quarter. Students are required to turn in Powerpoint versions of their presentation slides. A Powerpoint template provided by the instructor must be used.

5. Chapter One - Promises and Challenges of Networked Virtual Environments What Is a Networked Virtual Environment? 1 Graphics Engines and Displays 4Graphics Engines and Displays 4 Control and Communication Devices 5Control and Communication Devices 5 Processing Systems 6Processing Systems 6 Data Network 7Data Network 7 What Is a Networked Virtual Environment? 1 Graphics Engines and Displays 4Graphics Engines and Displays 4 Control and Communication Devices 5Control and Communication Devices 5 Processing Systems 6Processing Systems 6 Data Network 7Data Network 7 Challenges in Net-VE Design and Development 9 Network Bandwidth 9Network Bandwidth 9 Heterogeneity 10Heterogeneity 10 Distributed Interaction 11Distributed Interaction 11 Real-Time System Design and Resource Management 12Real-Time System Design and Resource Management 12

6. Chapter One - Promises and Challenges of Networked Virtual Environments Failure Management 13Failure Management 13 Scalability 15Scalability 15 Deployment and Configuration 16Deployment and Configuration 16 Conclusion 16 References 17 Failure Management 13Failure Management 13 Scalability 15Scalability 15 Deployment and Configuration 16Deployment and Configuration 16 Conclusion 16 References 17

7. Chapter Two - The Origin of Networked Virtual Environments Department of Defense Networked Virtual Environments 20 SIMNET 20SIMNET 20 Distributed Interactive Simulation 26Distributed Interactive Simulation 26 Department of Defense Networked Virtual Environments 20 SIMNET 20SIMNET 20 Distributed Interactive Simulation 26Distributed Interactive Simulation 26 Networked Games and Demos 34 SGI Flight and Dogfight 34SGI Flight and Dogfight 34 Doom 35Doom 35 Other Games 36Other Games 36

8. Chapter Two - The Origin of Networked Virtual Environments Academic Networked Virtual Environments 37 NPSNET 37NPSNET 37 PARADISE 42PARADISE 42 DIVE 44DIVE 44 BrickNet 46BrickNet 46 MR Toolkit Peer Package 48MR Toolkit Peer Package 48 Others 49Others 49 Academic Networked Virtual Environments 37 NPSNET 37NPSNET 37 PARADISE 42PARADISE 42 DIVE 44DIVE 44 BrickNet 46BrickNet 46 MR Toolkit Peer Package 48MR Toolkit Peer Package 48 Others 49Others 49 Conclusion 49 References 50

9. Chapter Three - A Networking Primer Fundamentals of Data Transfer 56 Network Latency 56Network Latency 56 Network Bandwidth 58Network Bandwidth 58 Network Reliability 58Network Reliability 58 Network Protocol 60Network Protocol 60 Fundamentals of Data Transfer 56 Network Latency 56Network Latency 56 Network Bandwidth 58Network Bandwidth 58 Network Reliability 58Network Reliability 58 Network Protocol 60Network Protocol 60 The BSD Sockets Architecture 61 Sockets and Ports 62Sockets and Ports 62 The Internet Protocol 65The Internet Protocol 65

10. Chapter Three - A Networking Primer Introducing the Internet Protocols for Net-VEs 65 Transmission Control Protocol 67Transmission Control Protocol 67 User Datagram Protocol 67 IP Broadcasting Using UDP 69User Datagram Protocol 67 IP Broadcasting Using UDP 69 IP Multicasting 70IP Multicasting 70 Introducing the Internet Protocols for Net-VEs 65 Transmission Control Protocol 67Transmission Control Protocol 67 User Datagram Protocol 67 IP Broadcasting Using UDP 69User Datagram Protocol 67 IP Broadcasting Using UDP 69 IP Multicasting 70IP Multicasting 70 Selecting a Net-VE Protocol 75 Using TCP/IP 77Using TCP/IP 77 Using UDP/IP 77Using UDP/IP 77 Using IP Broadcasting 80Using IP Broadcasting 80 Using IP Multicasting 82Using IP Multicasting 82 Conclusion 82 References 83

11. Chapter Four - Communication Architectures Two Players on a LAN 87 Multiplayer Client-Server Systems 90 Multiplayer Client-Server, with Multiple-Server Architectures 94 Peer-to-Peer Architectures 95 Two Players on a LAN 87 Multiplayer Client-Server Systems 90 Multiplayer Client-Server, with Multiple-Server Architectures 94 Peer-to-Peer Architectures 95 Conclusion 98 References 98

12. Chapter Five - Managing Dynamic Shared State The Consistency-Throughput Tradeoff 101 Proof of the Tradeoff 104Proof of the Tradeoff 104 Design Implications of the Tradeoff 105Design Implications of the Tradeoff 105 The Consistency-Throughput Tradeoff 101 Proof of the Tradeoff 104Proof of the Tradeoff 104 Design Implications of the Tradeoff 105Design Implications of the Tradeoff 105 Maintaining Shared State Inside Centralized Repositories 107 A File Repository 108A File Repository 108 A Repository in Server Memory 110A Repository in Server Memory 110 Virtual Repositories 112Virtual Repositories 112 Advantages and Drawbacks to Centralized RepositoriesAdvantages and Drawbacks to Centralized Repositories

13. Chapter Five - Managing Dynamic Shared State Reducing Coupling through Frequent State Regeneration 117 Explicit Entity Ownership 118Explicit Entity Ownership 118 Systems Using Frequent State Regeneration 122Systems Using Frequent State Regeneration 122 Reducing the Broadcast Scope 124Reducing the Broadcast Scope 124 Reducing Coupling through Frequent State Regeneration 117 Explicit Entity Ownership 118Explicit Entity Ownership 118 Systems Using Frequent State Regeneration 122Systems Using Frequent State Regeneration 122 Reducing the Broadcast Scope 124Reducing the Broadcast Scope 124 Advantages and Drawbacks of Frequent State Regeneration 125Advantages and Drawbacks of Frequent State Regeneration 125

14. Chapter Five - Managing Dynamic Shared State Dead Reckoning of Shared State 127 Prediction and Convergence 128Prediction and Convergence 128 Prediction Using Derivative Polynomials 129Prediction Using Derivative Polynomials 129 Object-Specialized Prediction 134Object-Specialized Prediction 134 Convergence Algorithms 137Convergence Algorithms 137 Dead Reckoning of Shared State 127 Prediction and Convergence 128Prediction and Convergence 128 Prediction Using Derivative Polynomials 129Prediction Using Derivative Polynomials 129 Object-Specialized Prediction 134Object-Specialized Prediction 134 Convergence Algorithms 137Convergence Algorithms 137 Addressing Consistency- Throughput through Nonregular Transmissions 140Addressing Consistency- Throughput through Nonregular Transmissions 140 Advantages and Limitations of Dead Reckoning 142Advantages and Limitations of Dead Reckoning 142 Conclusion 143 References 144

15. Chapter Six - Systems Design One Thread, Multiple Threads 147 With One Thread... 147With One Thread... 147 With Multiple Threads... 152With Multiple Threads... 152 One Thread, Multiple Threads 147 With One Thread... 147With One Thread... 147 With Multiple Threads... 152With Multiple Threads... 152 Important Subsystems 156 Real-Time Rendering: Polygon Culling and Level-of-Detail Processing 157Real-Time Rendering: Polygon Culling and Level-of-Detail Processing 157 Real-Time Collision Detection and Response 165Real-Time Collision Detection and Response 165 Computational Resource Management 174Computational Resource Management 174

16. Chapter Six - Systems Design Conclusion 175 References and Further Reading 176 Conclusion 175 References and Further Reading 176 The reading collection at the back of this chapter is key to really being able to build a net-VE system.

17. Chapter Seven - Resource Management for Scalability and Performance An Information-Centric View of Resources 183 Optimizing the Communications Protocol 185 Packet Compression 186Packet Compression 186 Packet Aggregation 191Packet Aggregation 191 An Information-Centric View of Resources 183 Optimizing the Communications Protocol 185 Packet Compression 186Packet Compression 186 Packet Aggregation 191Packet Aggregation 191 Controlling the Visibility of Data 195 Area-of-Interest Filtering Subscriptions 197Area-of-Interest Filtering Subscriptions 197 Multicasting 204Multicasting 204 Hybrid Multicast Aggregation 210Hybrid Multicast Aggregation 210

18. Chapter Seven - Resource Management for Scalability and Performance Taking Advantage of Perceptual Limitations 213 Exploiting Level-of-Detail Perception 215Exploiting Level-of-Detail Perception 215 Exploiting Temporal Perception 221Exploiting Temporal Perception 221 Taking Advantage of Perceptual Limitations 213 Exploiting Level-of-Detail Perception 215Exploiting Level-of-Detail Perception 215 Exploiting Temporal Perception 221Exploiting Temporal Perception 221 Enhancing the System Architecture 236 Server Clusters 237Server Clusters 237 Peer-Server Systems 243Peer-Server Systems 243 Conclusion 244 References 246

19. Chapter Eight - Internet Networked Virtual Environments VRML-Based Virtual Environments 251 Some Existing Networked VRML Worlds 253Some Existing Networked VRML Worlds 253 virtual reality transfer protocol 262 VRML-Based Virtual Environments 251 Some Existing Networked VRML Worlds 253Some Existing Networked VRML Worlds 253 virtual reality transfer protocol 262 Internet Gaming 263 Overview of Online Gaming Services 264Overview of Online Gaming Services 264 Internet Gaming Software Architectures 266Internet Gaming Software Architectures 266 Future Internet Gaming Directions 269Future Internet Gaming Directions 269 Conclusion 271 References 271

20. Chapter Nine - Perspective and Predictions Better Library Support 275 Bamboo 276Bamboo 276 High Level Architecture 279High Level Architecture 279 Java-Based Toolkits 283Java-Based Toolkits 283 Toolkits Being Developed Everywhere... 284Toolkits Being Developed Everywhere... 284 Better Library Support 275 Bamboo 276Bamboo 276 High Level Architecture 279High Level Architecture 279 Java-Based Toolkits 283Java-Based Toolkits 283 Toolkits Being Developed Everywhere... 284Toolkits Being Developed Everywhere... 284 Toward a Better Internet 285 Research Frontiers 287 Past, Present, and Future 289 References 290

21. Appendix - Network Communications in C, C++, and Java Using TCP/IP from C and C++ 293 Managing Concurrent Connections in C and C++ 297 Using TCP/IP from Java 300 Managing Concurrent Connections in Java 303 Using UDP/IP from C and C++ 305 Using TCP/IP from C and C++ 293 Managing Concurrent Connections in C and C++ 297 Using TCP/IP from Java 300 Managing Concurrent Connections in Java 303 Using UDP/IP from C and C++ 305 Using UDP/IP from Java 308 Broadcasting from C and C++ 310 Broadcasting from Java 310 Multicasting from C and C++ 311 Multicasting from Java 312 References 313