MV-4474 Virtual MV-4474 Virtual Environment Network Environment Network

& Software & Software ArchitecturesArchitectures

Michael ZydaMichael Zyda

zyda@movesinstitute.zyda@movesinstitute.orgorg

MV-4474 Virtual MV-4474 Virtual Environment Network Environment Network

& Software & Software ArchitecturesArchitectures

Michael ZydaMichael Zyda

zyda@movesinstitute.zyda@movesinstitute.orgorg

Networked Virtual Environments - Design & Implementation

Sandeep Singhal & Sandeep Singhal & Michael Zyda Michael Zyda "Networked Virtual "Networked Virtual Environments - Design Environments - Design and Implementation," and Implementation," ACM Press Books, ACM Press Books, SIGGRAPH Series, July SIGGRAPH Series, July 1999, ISBN 0-201-1999, ISBN 0-201-32557-8.32557-8.

Sandeep Singhal & Sandeep Singhal & Michael Zyda Michael Zyda "Networked Virtual "Networked Virtual Environments - Design Environments - Design and Implementation," and Implementation," ACM Press Books, ACM Press Books, SIGGRAPH Series, July SIGGRAPH Series, July 1999, ISBN 0-201-1999, ISBN 0-201-32557-8.32557-8.

MV-4474 Virtual Environment Network & Software Architectures

MV-4474 Virtual Environment Network and Software Architectures (3-MV-4474 Virtual Environment Network and Software Architectures (3-2) This course covers the design and implementation of network and 2) This course covers the design and implementation of network and software architectures for real-time, interactive 3D virtual software architectures for real-time, interactive 3D virtual environments (VEs). Network architecture topics include a taxonomy environments (VEs). Network architecture topics include a taxonomy for networked virtual environments, distributed interactive simulation for networked virtual environments, distributed interactive simulation protocols (DIS and HLA), virtual reality modeling language (VRML), protocols (DIS and HLA), virtual reality modeling language (VRML), agent-based network protocols (Java/Telescript), proposed solutions agent-based network protocols (Java/Telescript), proposed solutions for large-scale networked virtual environments (area of interest for large-scale networked virtual environments (area of interest managers and object brokers), multicast backbone tools and managers and object brokers), multicast backbone tools and developments, and virtual reality transfer protocol proposals. Software developments, and virtual reality transfer protocol proposals. Software architecture topics include representative software architectures for architecture topics include representative software architectures for VEs (NPSNET, DIVE, MASSIVE, ...), commercial toolkits for VE VEs (NPSNET, DIVE, MASSIVE, ...), commercial toolkits for VE development (WorldToolKit, Division's dvs, Performer, ...), lag in development (WorldToolKit, Division's dvs, Performer, ...), lag in multiprocessor virtual environments, and the HCI implications on VE multiprocessor virtual environments, and the HCI implications on VE network and software architectures. PREREQUISITE: MV-4473 or the network and software architectures. PREREQUISITE: MV-4473 or the consent of the instructor.consent of the instructor.

MV-4474 Virtual Environment Network and Software Architectures (3-MV-4474 Virtual Environment Network and Software Architectures (3-2) This course covers the design and implementation of network and 2) This course covers the design and implementation of network and software architectures for real-time, interactive 3D virtual software architectures for real-time, interactive 3D virtual environments (VEs). Network architecture topics include a taxonomy environments (VEs). Network architecture topics include a taxonomy for networked virtual environments, distributed interactive simulation for networked virtual environments, distributed interactive simulation protocols (DIS and HLA), virtual reality modeling language (VRML), protocols (DIS and HLA), virtual reality modeling language (VRML), agent-based network protocols (Java/Telescript), proposed solutions agent-based network protocols (Java/Telescript), proposed solutions for large-scale networked virtual environments (area of interest for large-scale networked virtual environments (area of interest managers and object brokers), multicast backbone tools and managers and object brokers), multicast backbone tools and developments, and virtual reality transfer protocol proposals. Software developments, and virtual reality transfer protocol proposals. Software architecture topics include representative software architectures for architecture topics include representative software architectures for VEs (NPSNET, DIVE, MASSIVE, ...), commercial toolkits for VE VEs (NPSNET, DIVE, MASSIVE, ...), commercial toolkits for VE development (WorldToolKit, Division's dvs, Performer, ...), lag in development (WorldToolKit, Division's dvs, Performer, ...), lag in multiprocessor virtual environments, and the HCI implications on VE multiprocessor virtual environments, and the HCI implications on VE network and software architectures. PREREQUISITE: MV-4473 or the network and software architectures. PREREQUISITE: MV-4473 or the consent of the instructor.consent of the instructor.

Course LogisticsThe lectures in this course follow the Singhal/Zyda text The lectures in this course follow the Singhal/Zyda text from chapter 1 through chapter 9. from chapter 1 through chapter 9.

Each student is expected to develop a programming Each student is expected to develop a programming project showing their knowledge of networked virtual project showing their knowledge of networked virtual environments. That project must be approved by the environments. That project must be approved by the instructor. instructor.

• An in-class presentation on that project is due near the end An in-class presentation on that project is due near the end of the quarter. Students are required to turn in Powerpoint of the quarter. Students are required to turn in Powerpoint versions of their presentation slides. A Powerpoint template versions of their presentation slides. A Powerpoint template provided by the instructor must be used.provided by the instructor must be used.

The lectures in this course follow the Singhal/Zyda text The lectures in this course follow the Singhal/Zyda text from chapter 1 through chapter 9. from chapter 1 through chapter 9.

Each student is expected to develop a programming Each student is expected to develop a programming project showing their knowledge of networked virtual project showing their knowledge of networked virtual environments. That project must be approved by the environments. That project must be approved by the instructor. instructor.

• An in-class presentation on that project is due near the end An in-class presentation on that project is due near the end of the quarter. Students are required to turn in Powerpoint of the quarter. Students are required to turn in Powerpoint versions of their presentation slides. A Powerpoint template versions of their presentation slides. A Powerpoint template provided by the instructor must be used.provided by the instructor must be used.

Chapter One - Promises and Challenges of Networked Virtual Environments

What Is a Networked What Is a Networked Virtual Environment? 1Virtual Environment? 1

• Graphics Engines and Graphics Engines and Displays 4Displays 4

• Control and Control and Communication Devices 5Communication Devices 5

• Processing Systems 6Processing Systems 6

• Data Network 7Data Network 7

What Is a Networked What Is a Networked Virtual Environment? 1Virtual Environment? 1

• Graphics Engines and Graphics Engines and Displays 4Displays 4

• Control and Control and Communication Devices 5Communication Devices 5

• Processing Systems 6Processing Systems 6

• Data Network 7Data Network 7

Challenges in Net-VE Challenges in Net-VE Design and Development 9Design and Development 9

• Network Bandwidth 9Network Bandwidth 9

• Heterogeneity 10Heterogeneity 10

• Distributed Interaction 11Distributed Interaction 11

• Real-Time System Design Real-Time System Design and Resource and Resource Management 12Management 12

Challenges in Net-VE Challenges in Net-VE Design and Development 9Design and Development 9

• Network Bandwidth 9Network Bandwidth 9

• Heterogeneity 10Heterogeneity 10

• Distributed Interaction 11Distributed Interaction 11

• Real-Time System Design Real-Time System Design and Resource and Resource Management 12Management 12

Chapter One - Promises and Challenges of Networked Virtual Environments• Failure Management 13Failure Management 13

• Scalability 15Scalability 15

• Deployment and Deployment and Configuration 16Configuration 16

Conclusion 16Conclusion 16

References 17References 17

• Failure Management 13Failure Management 13

• Scalability 15Scalability 15

• Deployment and Deployment and Configuration 16Configuration 16

Conclusion 16Conclusion 16

References 17References 17

Chapter Two - The Origin of Networked Virtual Environments

Department of Defense Department of Defense Networked Virtual Networked Virtual Environments 20Environments 20

• SIMNET 20SIMNET 20

• Distributed Interactive Distributed Interactive Simulation 26Simulation 26

Department of Defense Department of Defense Networked Virtual Networked Virtual Environments 20Environments 20

• SIMNET 20SIMNET 20

• Distributed Interactive Distributed Interactive Simulation 26Simulation 26

Networked Games and Networked Games and Demos 34Demos 34

• SGI Flight and Dogfight 34SGI Flight and Dogfight 34

• Doom 35Doom 35

• Other Games 36Other Games 36

Networked Games and Networked Games and Demos 34Demos 34

• SGI Flight and Dogfight 34SGI Flight and Dogfight 34

• Doom 35Doom 35

• Other Games 36Other Games 36

Chapter Two - The Origin of Networked Virtual Environments

Academic Networked Academic Networked Virtual Environments 37Virtual Environments 37

• NPSNET 37NPSNET 37

• PARADISE 42 PARADISE 42

• DIVE 44DIVE 44

• BrickNet 46BrickNet 46

• MR Toolkit Peer Package MR Toolkit Peer Package 4848

• Others 49Others 49

Academic Networked Academic Networked Virtual Environments 37Virtual Environments 37

• NPSNET 37NPSNET 37

• PARADISE 42 PARADISE 42

• DIVE 44DIVE 44

• BrickNet 46BrickNet 46

• MR Toolkit Peer Package MR Toolkit Peer Package 4848

• Others 49Others 49

Conclusion 49Conclusion 49

References 50References 50

Conclusion 49Conclusion 49

References 50References 50

Chapter Three - A Networking Primer

Fundamentals of Data Fundamentals of Data Transfer 56Transfer 56

• Network Latency 56Network Latency 56

• Network Bandwidth 58Network Bandwidth 58

• Network Reliability 58Network Reliability 58

• Network Protocol 60Network Protocol 60

Fundamentals of Data Fundamentals of Data Transfer 56Transfer 56

• Network Latency 56Network Latency 56

• Network Bandwidth 58Network Bandwidth 58

• Network Reliability 58Network Reliability 58

• Network Protocol 60Network Protocol 60

The BSD Sockets The BSD Sockets Architecture 61Architecture 61

• Sockets and Ports 62Sockets and Ports 62

• The Internet Protocol 65The Internet Protocol 65

The BSD Sockets The BSD Sockets Architecture 61Architecture 61

• Sockets and Ports 62Sockets and Ports 62

• The Internet Protocol 65The Internet Protocol 65

Chapter Three - A Networking Primer

Introducing the Internet Introducing the Internet Protocols for Net-VEs 65Protocols for Net-VEs 65

• Transmission Control Transmission Control Protocol 67Protocol 67

• User Datagram Protocol 67 User Datagram Protocol 67 IP Broadcasting Using UDP IP Broadcasting Using UDP 6969

• IP Multicasting 70IP Multicasting 70

Introducing the Internet Introducing the Internet Protocols for Net-VEs 65Protocols for Net-VEs 65

• Transmission Control Transmission Control Protocol 67Protocol 67

• User Datagram Protocol 67 User Datagram Protocol 67 IP Broadcasting Using UDP IP Broadcasting Using UDP 6969

• IP Multicasting 70IP Multicasting 70

Selecting a Net-VE Selecting a Net-VE Protocol 75Protocol 75

• Using TCP/IP 77Using TCP/IP 77

• Using UDP/IP 77Using UDP/IP 77

• Using IP Broadcasting 80Using IP Broadcasting 80

• Using IP Multicasting 82Using IP Multicasting 82

Conclusion 82Conclusion 82

References 83References 83

Selecting a Net-VE Selecting a Net-VE Protocol 75Protocol 75

• Using TCP/IP 77Using TCP/IP 77

• Using UDP/IP 77Using UDP/IP 77

• Using IP Broadcasting 80Using IP Broadcasting 80

• Using IP Multicasting 82Using IP Multicasting 82

Conclusion 82Conclusion 82

References 83References 83

Chapter Four - Communication Architectures

Two Players on a LAN 87Two Players on a LAN 87

Multiplayer Client-Server Multiplayer Client-Server Systems 90Systems 90

Multiplayer Client-Server, Multiplayer Client-Server, with Multiple-Server with Multiple-Server Architectures 94Architectures 94

Peer-to-Peer Architectures Peer-to-Peer Architectures 9595

Two Players on a LAN 87Two Players on a LAN 87

Multiplayer Client-Server Multiplayer Client-Server Systems 90Systems 90

Multiplayer Client-Server, Multiplayer Client-Server, with Multiple-Server with Multiple-Server Architectures 94Architectures 94

Peer-to-Peer Architectures Peer-to-Peer Architectures 9595

Conclusion 98Conclusion 98

References 98References 98

Conclusion 98Conclusion 98

References 98References 98

Chapter Five - Managing Dynamic Shared State

The Consistency-The Consistency-Throughput Tradeoff 101Throughput Tradeoff 101

• Proof of the Tradeoff 104Proof of the Tradeoff 104

• Design Implications of the Design Implications of the Tradeoff 105Tradeoff 105

The Consistency-The Consistency-Throughput Tradeoff 101Throughput Tradeoff 101

• Proof of the Tradeoff 104Proof of the Tradeoff 104

• Design Implications of the Design Implications of the Tradeoff 105Tradeoff 105

Maintaining Shared State Maintaining Shared State Inside Centralized Inside Centralized Repositories 107Repositories 107

• A File Repository 108A File Repository 108

• A Repository in Server A Repository in Server Memory 110Memory 110

• Virtual Repositories 112Virtual Repositories 112

• Advantages and Advantages and Drawbacks to Centralized Drawbacks to Centralized RepositoriesRepositories

Maintaining Shared State Maintaining Shared State Inside Centralized Inside Centralized Repositories 107Repositories 107

• A File Repository 108A File Repository 108

• A Repository in Server A Repository in Server Memory 110Memory 110

• Virtual Repositories 112Virtual Repositories 112

• Advantages and Advantages and Drawbacks to Centralized Drawbacks to Centralized RepositoriesRepositories

Chapter Five - Managing Dynamic Shared State

Reducing Coupling through Reducing Coupling through Frequent State Frequent State Regeneration 117Regeneration 117

• Explicit Entity Ownership Explicit Entity Ownership 118118

• Systems Using Frequent Systems Using Frequent State Regeneration 122State Regeneration 122

• Reducing the Broadcast Reducing the Broadcast Scope 124Scope 124

Reducing Coupling through Reducing Coupling through Frequent State Frequent State Regeneration 117Regeneration 117

• Explicit Entity Ownership Explicit Entity Ownership 118118

• Systems Using Frequent Systems Using Frequent State Regeneration 122State Regeneration 122

• Reducing the Broadcast Reducing the Broadcast Scope 124Scope 124

• Advantages and Advantages and Drawbacks of Frequent Drawbacks of Frequent State Regeneration 125State Regeneration 125

• Advantages and Advantages and Drawbacks of Frequent Drawbacks of Frequent State Regeneration 125State Regeneration 125

Chapter Five - Managing Dynamic Shared State

Dead Reckoning of Shared Dead Reckoning of Shared State 127State 127

• Prediction and Prediction and Convergence 128Convergence 128

• Prediction Using Derivative Prediction Using Derivative Polynomials 129Polynomials 129

• Object-Specialized Object-Specialized Prediction 134Prediction 134

• Convergence Algorithms Convergence Algorithms 137137

Dead Reckoning of Shared Dead Reckoning of Shared State 127State 127

• Prediction and Prediction and Convergence 128Convergence 128

• Prediction Using Derivative Prediction Using Derivative Polynomials 129Polynomials 129

• Object-Specialized Object-Specialized Prediction 134Prediction 134

• Convergence Algorithms Convergence Algorithms 137137

• Addressing Consistency-Addressing Consistency-Throughput through Throughput through Nonregular Transmissions Nonregular Transmissions 140140

• Advantages and Advantages and Limitations of Dead Limitations of Dead Reckoning 142Reckoning 142

Conclusion 143Conclusion 143

References 144References 144

• Addressing Consistency-Addressing Consistency-Throughput through Throughput through Nonregular Transmissions Nonregular Transmissions 140140

• Advantages and Advantages and Limitations of Dead Limitations of Dead Reckoning 142Reckoning 142

Conclusion 143Conclusion 143

References 144References 144

Chapter Six - Systems DesignOne Thread, Multiple One Thread, Multiple Threads 147Threads 147

• With One Thread . . . 147With One Thread . . . 147

• With Multiple Threads . . . With Multiple Threads . . . 152152

One Thread, Multiple One Thread, Multiple Threads 147Threads 147

• With One Thread . . . 147With One Thread . . . 147

• With Multiple Threads . . . With Multiple Threads . . . 152152

Important Subsystems 156Important Subsystems 156

• Real-Time Rendering: Real-Time Rendering: Polygon Culling and Level-Polygon Culling and Level-of-Detail Processing 157of-Detail Processing 157

• Real-Time Collision Real-Time Collision Detection and Response Detection and Response 165165

• Computational Resource Computational Resource Management 174Management 174

Important Subsystems 156Important Subsystems 156

• Real-Time Rendering: Real-Time Rendering: Polygon Culling and Level-Polygon Culling and Level-of-Detail Processing 157of-Detail Processing 157

• Real-Time Collision Real-Time Collision Detection and Response Detection and Response 165165

• Computational Resource Computational Resource Management 174Management 174

Chapter Six - Systems DesignConclusion 175Conclusion 175

References and Further References and Further Reading 176Reading 176

Conclusion 175Conclusion 175

References and Further References and Further Reading 176Reading 176

The reading collection at The reading collection at the back of this chapter is the back of this chapter is key to really being able to key to really being able to build a net-VE system.build a net-VE system.

The reading collection at The reading collection at the back of this chapter is the back of this chapter is key to really being able to key to really being able to build a net-VE system.build a net-VE system.

Chapter Seven - Resource Management for Scalability and Performance

An Information-Centric An Information-Centric View of Resources 183View of Resources 183

Optimizing the Optimizing the Communications Protocol Communications Protocol 185185

• Packet Compression 186Packet Compression 186

• Packet Aggregation 191Packet Aggregation 191

An Information-Centric An Information-Centric View of Resources 183View of Resources 183

Optimizing the Optimizing the Communications Protocol Communications Protocol 185185

• Packet Compression 186Packet Compression 186

• Packet Aggregation 191Packet Aggregation 191

Controlling the Visibility of Controlling the Visibility of Data 195Data 195

• Area-of-Interest Filtering Area-of-Interest Filtering Subscriptions 197Subscriptions 197

• Multicasting 204Multicasting 204

• Hybrid Multicast Hybrid Multicast Aggregation 210Aggregation 210

Controlling the Visibility of Controlling the Visibility of Data 195Data 195

• Area-of-Interest Filtering Area-of-Interest Filtering Subscriptions 197Subscriptions 197

• Multicasting 204Multicasting 204

• Hybrid Multicast Hybrid Multicast Aggregation 210Aggregation 210

Chapter Seven - Resource Management for Scalability and Performance

Taking Advantage of Taking Advantage of Perceptual Limitations 213Perceptual Limitations 213

• Exploiting Level-of-Detail Exploiting Level-of-Detail Perception 215Perception 215

• Exploiting Temporal Exploiting Temporal Perception 221Perception 221

Taking Advantage of Taking Advantage of Perceptual Limitations 213Perceptual Limitations 213

• Exploiting Level-of-Detail Exploiting Level-of-Detail Perception 215Perception 215

• Exploiting Temporal Exploiting Temporal Perception 221Perception 221

Enhancing the System Enhancing the System Architecture 236Architecture 236

• Server Clusters 237Server Clusters 237

• Peer-Server Systems 243Peer-Server Systems 243

Conclusion 244Conclusion 244

References 246References 246

Enhancing the System Enhancing the System Architecture 236Architecture 236

• Server Clusters 237Server Clusters 237

• Peer-Server Systems 243Peer-Server Systems 243

Conclusion 244Conclusion 244

References 246References 246

Chapter Eight - Internet Networked Virtual Environments

VRML-Based Virtual VRML-Based Virtual Environments 251Environments 251

• Some Existing Networked Some Existing Networked VRML Worlds 253VRML Worlds 253

virtual reality transfer virtual reality transfer protocol 262protocol 262

VRML-Based Virtual VRML-Based Virtual Environments 251Environments 251

• Some Existing Networked Some Existing Networked VRML Worlds 253VRML Worlds 253

virtual reality transfer virtual reality transfer protocol 262protocol 262

Internet Gaming 263Internet Gaming 263

• Overview of Online Overview of Online Gaming Services 264Gaming Services 264

• Internet Gaming Software Internet Gaming Software Architectures 266Architectures 266

• Future Internet Gaming Future Internet Gaming Directions 269Directions 269

Conclusion 271Conclusion 271

References 271References 271

Internet Gaming 263Internet Gaming 263

• Overview of Online Overview of Online Gaming Services 264Gaming Services 264

• Internet Gaming Software Internet Gaming Software Architectures 266Architectures 266

• Future Internet Gaming Future Internet Gaming Directions 269Directions 269

Conclusion 271Conclusion 271

References 271References 271

Chapter Nine - Perspective and Predictions

Better Library Support 275Better Library Support 275

• Bamboo 276Bamboo 276

• High Level Architecture High Level Architecture 279279

• Java-Based Toolkits 283Java-Based Toolkits 283

• Toolkits Being Developed Toolkits Being Developed Everywhere . . . 284Everywhere . . . 284

Better Library Support 275Better Library Support 275

• Bamboo 276Bamboo 276

• High Level Architecture High Level Architecture 279279

• Java-Based Toolkits 283Java-Based Toolkits 283

• Toolkits Being Developed Toolkits Being Developed Everywhere . . . 284Everywhere . . . 284

Toward a Better Internet Toward a Better Internet 285285

Research Frontiers 287Research Frontiers 287

Past, Present, and Future Past, Present, and Future 289289

References 290References 290

Toward a Better Internet Toward a Better Internet 285285

Research Frontiers 287Research Frontiers 287

Past, Present, and Future Past, Present, and Future 289289

References 290References 290

Appendix - Network Communications in C, C++, and Java

Using TCP/IP from C and Using TCP/IP from C and C++ 293C++ 293

Managing Concurrent Managing Concurrent Connections in C and C++ Connections in C and C++ 297297

Using TCP/IP from Java 300Using TCP/IP from Java 300

Managing Concurrent Managing Concurrent Connections in Java 303Connections in Java 303

Using UDP/IP from C and Using UDP/IP from C and C++ 305C++ 305

Using TCP/IP from C and Using TCP/IP from C and C++ 293C++ 293

Managing Concurrent Managing Concurrent Connections in C and C++ Connections in C and C++ 297297

Using TCP/IP from Java 300Using TCP/IP from Java 300

Managing Concurrent Managing Concurrent Connections in Java 303Connections in Java 303

Using UDP/IP from C and Using UDP/IP from C and C++ 305C++ 305

Using UDP/IP from Java Using UDP/IP from Java 308308

Broadcasting from C and Broadcasting from C and C++ 310C++ 310

Broadcasting from Java Broadcasting from Java 310310

Multicasting from C and C+Multicasting from C and C++ 311+ 311

Multicasting from Java 312Multicasting from Java 312

References 313References 313

Using UDP/IP from Java Using UDP/IP from Java 308308

Broadcasting from C and Broadcasting from C and C++ 310C++ 310

Broadcasting from Java Broadcasting from Java 310310

Multicasting from C and C+Multicasting from C and C++ 311+ 311

Multicasting from Java 312Multicasting from Java 312

References 313References 313